Towards Emotionally Aware Computers

A "mind reading" technology has been demonstrated at the Association for Computing Machinery (ACM) Symposium on User Interface Software and Technology (UIST 2007, Oct. 7 to 10). Using functional near-infrared spectroscopy Tufts University researchers have successfully crafted machine learning algorithms that deduce users' "stress levels," while performing tasks with varying levels of mental workload (from bored to overwhelmed) and adjust the man-machine interface to match.

Functional near-infrared spectroscopy, currently in clinical trials to detect tumors through the skin, has been re-purposed. The infrared sensors are mounted on a headband with eight laser diodes sending near-infrared light through the forehead to a depth of two to three centimeters (about an inch), where it scatters inside the brain's frontal lobe. Light passes through deoxygenated hemoglobin in the blood but is blocked by oxygenated hemoglobin. Since oxygenation levels indicate brain activity, the amount of scattered light detected by the two infrared sensors on the headband detect stress levels on a scale ranging from bored (low stress) to overwhelmed (high stress).

A perceptron neural network with 16 inputs, one hidden layer, and up to five outputs is employed to associate the sensed blood oxygenation levels with the stress levels of specific users. In the tests, different patterns of Rubik's cubes are presented to the users, giving them just nine seconds to identify how many colors are present on each cube. By starting with just two colors and working up from there, the researchers have been able to teach the neural network to recognize stress levels.