BMI Whitepaper

:: dir :: teams :: bmi :: BMI Whitepaper

Section
Description
People

Area (1) Extraction of neural and force dynamic codes related to patterns of motor or sensory activity required for executing simple to complex motor or sensory activity (e.g. reaching, grasping, manipulating, running, walking, kicking, digging, hearing, seeing, tactile). Accessing sensory activity directly could result in the ability to monitor or transmit communications by the brain (visual, auditory, other). This will require the exploitation of new interfaces and algorithms for providing useful non-linear transformation, pattern extraction techniques, and the ability to test these in appropriate models or systems.
Jack Culpepper, Bob Keller
Area (2) Determination of necessary force and sensory feedback (positional, postural, visual, acoustic, other) from a peripheral device or interface that will provide critical inputs required for closed loop control of a working device (robotic appendage or other peripheral control device or system). Such feedback could be both from peripheral systems or directly into appropriate brain regions.
?
Area (3) New methods, processes, and instrumentation for accessing neural codes non-invasively at appropriate spatiotemporal resolution to provide closed loop control of a peripheral device. This could include both fundamental interactions of neural cells, tissue, and brain with energy profiles that could provide non-invasive access to codes (magnetics, light, other).
?
Area (4) New materials and device design and fabrication that embody compliance and elastic principles and capture force dynamics that integrate with neural control commands. These include the use of dynamic materials and designs into working prototypes.
?
Area (5) Demonstrations of plasticity from the neural system and from an integrated working device or system that result in real time control under relevant conditions of force perturbation and cluttered sensory environments from which tasks must be performed (e.g. recognizing and picking up a target and manipulating it).
Jack Culpepper, Bob Keller
Area (6) Biomimetic implementation of controllers (with robotics or other devices and systems) that integrate neural sensory or motor control integrated with force dynamics and sensory feedback from a working device or system.
Mark Cutkosky (?)

Area (1)

I read a recent paper (from IEEE Engineering in Medicine and Biology) about an algorithm that reduces the complexity of localizing multiple neural sources. If source localization is possible, the locations of active neural sources for a particular brain state could be used as inputs to an artificial neural network classifier. I haven't heard of anyone that has tried that. Here is the paper that describes a method for source localization:
Zhukov, Weinstein, Johnson
The problem of determining the locations of the neural sources from their projections on the scalp is called the "inverse problem". Here is the experiment that I propose:
We will construct a nine channel wireless EEG recorder. There are two-channel systems that are commercially available, which are very easy to put on and take off. We'll pick a symmetric map out of the major points on the 10-20 electrode placement system, and use those points as our locations from which to record on the scalp. We'll sample the voltage potentials (at an as-yet undetermined frequency -- the frequency should probably be dynamically adjusted with the performance of the classifier), and feed them into a computer. The computer will perform a variety of tasks in real-time. First, it will use Independent Component Analysis to separate line noise and other artifacts from the signal. This will result in eight separate sources of EEG data, and one channel of noise. These sources will have a different spatial orientation with respect to the skull than their unprocessed counterparts, but they are still valid for our purpose. We'll disgard the noise, and feed eight channels into a c implementation of Zhukov's source localization algorithm. Each time we sample a new packet of EEG data, we'll compute the power spectral density of the ICA signals, and feed the result (8 values, the position of the sources (8 values), and their direction in three-space (3 values) into two artificial neural network classifier: a regular backpropagation network, and a temporal backpropagation network. The two classifiers will be trained, independently, to differentiate between n states, where we will begin the experiement with n = 2, and hopefully progress to a much larger number of states as we learn more. The networks will be trained on pre-recorded data using a validation set: when a network's performance on the validation set does not improve for m epochs, training for that network is stopped. Once a network is trained, we'll freeze the weights and embed an evaluation of the network into the signal processing chain:
EEG ==> ICA ==> Power Spectral Density
Source Location
Source Direction ==> Classifier ==> forward
stop

We'll train hundreds of networks this way, and experiment using "committees" of trained networks. By this, I mean feed the 24 tuple of values from the signal processing stream into x trained classified instances. Each time a packet comes in, there are x "gueses" output by the classifiers. Of the x outputs, choose the y best performing classifiers, where the performance of a classifier is kept track of by counting the number of times it guesses "right" according to information that comes later in time. The y best performing classifiers are then treated as a democracy, and the most popular vote is output to the wheelchair controllers. Here is the final signal processing chain:
EEG ==> ICA ==> Power Spectral Density
Source Location
Source Direction ==> Classifier 0
Classifier 1
Classifier 2
...
Classifier x ==> Committee ==> forward
stop

Area (5)

Assuming (1) is feasible and pans out, we could cover (5) by connecting an EEG recognition system to a motorized wheelchair, and demonstrating control of the chair in a cluttered sensory environment.
-Jack

Section	Description	People
Area (1)	Extraction of neural and force dynamic codes related to patterns of motor or sensory activity required for executing simple to complex motor or sensory activity (e.g. reaching, grasping, manipulating, running, walking, kicking, digging, hearing, seeing, tactile). Accessing sensory activity directly could result in the ability to monitor or transmit communications by the brain (visual, auditory, other). This will require the exploitation of new interfaces and algorithms for providing useful non-linear transformation, pattern extraction techniques, and the ability to test these in appropriate models or systems.	Jack Culpepper, Bob Keller
Area (2)	Determination of necessary force and sensory feedback (positional, postural, visual, acoustic, other) from a peripheral device or interface that will provide critical inputs required for closed loop control of a working device (robotic appendage or other peripheral control device or system). Such feedback could be both from peripheral systems or directly into appropriate brain regions.	?
Area (3)	New methods, processes, and instrumentation for accessing neural codes non-invasively at appropriate spatiotemporal resolution to provide closed loop control of a peripheral device. This could include both fundamental interactions of neural cells, tissue, and brain with energy profiles that could provide non-invasive access to codes (magnetics, light, other).	?
Area (4)	New materials and device design and fabrication that embody compliance and elastic principles and capture force dynamics that integrate with neural control commands. These include the use of dynamic materials and designs into working prototypes.	?
Area (5)	Demonstrations of plasticity from the neural system and from an integrated working device or system that result in real time control under relevant conditions of force perturbation and cluttered sensory environments from which tasks must be performed (e.g. recognizing and picking up a target and manipulating it).	Jack Culpepper, Bob Keller
Area (6)	Biomimetic implementation of controllers (with robotics or other devices and systems) that integrate neural sensory or motor control integrated with force dynamics and sensory feedback from a working device or system.	Mark Cutkosky (?)