Sensory Integration Notes
From KubovyLab
Tasks
<tasks>
[1] Retooling as needed to allow visual files to express time relative to moment of impact. E.g. sound always begins playing when time=0. For example, if the time in an animation file runs from -2 to 2 seconds the sound is played halfway through, when time=0 (sfitch/mschutz) [1] Ability to enable/disable lines connecting dots (sfitch) [2] More elegant way of handling/recovering/notifying when a given file isn't found. Currently it hangs/crashes and/or doesn't notify us about such issues. [2] Ability to independently "freeze" x or y coordinates for a given point (mschutz) [2] Alternate response mechanism where subject holds down space bar until the note stops sounding (sfitch) [2] Adjust the playback speed via a parameter so we can play around with slow motion questions. (mschutz) [2] Ability to "mask" specific regions of space to see how much of actual impact must be viewed in order to show effect (sfitch) [2] Enforce a minimum time between trials. Right now audio alone trials begin immediately after pressing "OK" for previous trial. [3] Ability to display text--e.g. "Short", "Long", "Loud", etc--or picture, rather than animation, along with sound [3] Reflection about point of contact - e.g. let the mallet head move "through the bar." (mschutz) [3] Rotate the perspective? This should be mathematically simple but would allow us to do things like invert the video image so that the gesture is hitting something up above, or off to the side. This would address some potential confounds in the experiment and would be very handy.(sfitch) [3] Fill entire screen with neutral background--e.g. gray--to prevent distractions to subject (sfitch) [3] At some point, it would be useful to be able to do forced choice comparisons. E.g. playing two animation/sound pairs side by side/one after the other and asking the subject to choose which is longer. That way we can control the comparisons and move towards a point of subjective equality. This is a more sophisticated design for later on down the road, but I just wanted to bring it up now in case it impacts design/planning/etc. (mschutz) [3] Make default location of response box below animation screen. On my larger monitor, this currently happens automatically however on some monitors--e.g. the ones we use to run subjects--it defaults to covering part of the animation. (sfitch) [3] Replication of traditional SI work using light flashes and sine wave tones. Currently, we could do this by specifying dot coordinates and using a sine wave file, but it would be much simpler and more flexible if we could specify specific parameters abstractly without needing to spell things out explicitly. For example, define an array of light flash durations and an array of tone durations which would be used to generate specific AV comnbinations. While this would be very useful for down the road handy, it is currently a low priority. [3] Control over level of visibility of animation. "Lower visibility" values would result in very faint lines whereas "high visibility" are easily seen (sfitch) [3] Animation window gets "hidden" behind other windows, even when focused on that window. Similarly, controls for demo are always "hiding" other windows, even when not currently working in that program. [3] FYI, tasks specified with a priority level of 4 do not appear in this list! [3] Add version number to deployment disk image. Note, this should be an experimental version number, rather than a technical--e.g. code--version number (sfitch)
[x] Smooth playback of video files with experiment (sfitch) [x] Fix bug where every session generates a new data file header. (sfitch) [x] A way to specify priorities in this "wish list"! (sfitch) [x] Add configuration/setup screen.(sfitch) [x] Add persistence to configuration screen.(sfitch) [x] Create trial definition file (sfitch) [x] Integrate playing of sound file. (sfitch) [x] Record RA id number when starting up experiment (sfitch) [x] Allow negative subject IDs (sfitch) [x] Specify number of data points to use in animation, giving primacy to mallet head and counting backwards up the arm so that the shoulder is the lowest priority. Specifying 1 point would result in animating only the mallet head, 2 points the mallet head and hand, etc. (sfitch) [x] Refrain from writing data to file unless subject completes entire experiment. This is very important! It would be nice to record responses as they occur in another temporary file which can be examined seperately, but the main thing is to avoid recording data from sessions in which only a few trials are performed. (sfitch) [x] A way to indicate when the warmup block has been completed. Currently, the first block in the metaBlock is presented first as a "warmup" block--for which responses will later be discarded/ignored--, and then the blocks are cycled through in a random order. We tell subjects that they will receive a "warmup" period which they can use to get a sense of the range of short/long stimuli presented for the rest of the experiment. It is important they know when this period ends because at that point they will have seen the complete range of long and short possibilities and can begin establishing some type of internal scale for responding. (mschutz) [x] Fix dialog to remain on the screen below the animation window, rather than popping up each time. (sfitch) [x] Change point rendering to use GLUT disks instead of GL_POINT for better hardware compatibility. (sfitch) [x] Stamp version number into data filename. (mschutz) [x] Don't show agreement slider in audio-alone or video-alone blocks (mschutz) [x] Automatically time/date stamp each trial in data file. (mschutz) [x] Specify playback points: e.g. play only from .3 to 1.2 seconds of animation file. Eventually, we will begin whittling down the animations in an effort to find the specific parts that are most important in creating the visual influence. (sfitch) [x] Demo mode which allows for specification of upcoming trial. Useful down the line for giving talks/ demonstrations and also for exploring/ playing around with new stimuli (sfitch) [x] Demo mode fills in file name with visual short values when long visual is selected from the drop down box. Auditory drop down box works fine. Minor issue (mschutz)
</tasks>
Vision
Vision for the SI program and future programs (in terms of scalability and flexibility) would be the following:
- Design a generic experiment framework, which handles all the usual tasks of getting subject information, tracking session numbers, setting up data files, etc. Ideally, this will function independently of the experiment module.
- Design an experimental module specific to each task. This module is embedded with the generic framework discussed above and doesn't worry about opening data files, generating stimuli, etc. It merely takes a description of the stimuli using parameters (e.g. a row from the definition file), executes it, records the user response and then passes this info up the chain to the generic experiment module which saves it in the larger data file.
- Design a stimuli definition generation module which could be specific to several different tasks. This doesn't handle anything with regards to the actual stimuli, but generates the parameter description of the stimuli (e.g. a row in the definition file) and passes it up the chain to the experimental module. At it's most simple, the module would merely open a text file of the kind you are currently for using for the dot lattice experiment and feed it along to the experiment module in order to generate the stimuli. More complex versions (like I am envisioning for the SI experiment) would generate this on the fly based on other parameters. For example, rather than needing every stimuli description written a head of time, it will generate them from base parameters. For the SI experiment, this means it would take all visual types and pair them exhaustively with all audio types. This will make it easy to control the presentation order, since we can randomize the aspects we want (e.g. trials within each block) while controlling the aspects that should not be randomized (block order within an experiment).
- All of this will be driven by a final parameter file specific to each experiment. In this file, we can define all the relevant stimuli parameters which will be used in component 3 to generate the precise stimuli description which will be fed to module 2.
This will give us the maximum flexibility down the line, since each module can be changed independently. Ideally, every experiment will use the same generic framework (component 1), similar types of experiments will all share an experimental module (component 2), but have either different ways of generating the stimuli (module 3) or different parameters on the stimuli (module 4).
Other
- Review of sound facilities
- http://mail.python.org/pipermail/edu-sig/2001-December/001898.html
