Machines Like Us

Tufts University researchers are developing techniques that could allow computers to respond to users’ thoughts of frustration -- too much work -- or boredom -- too little work. Applying non-invasive and easily portable imaging technology in new ways, they hope to gain real-time insight into the brain’s more subtle emotional cues and help provide a more efficient way to get work done.

“New evaluation techniques that monitor user experiences while working with computers are increasingly necessary,” said Robert Jacob, computer science professor and researcher. “One moment a user may be bored, and the next moment, the same user may be overwhelmed. Measuring mental workload, frustration and distraction is typically limited to qualitatively observing computer users or to administering surveys after completion of a task, potentially missing valuable insight into the users’ changing experiences.”

Sergio Fantini, biomedical engineering professor, in conjunction with Jacob’s human-computer interaction (HCI) group, is studying functional near-infrared spectroscopy (fNIRS) technology that uses light to monitor brain blood flow as a proxy for workload stress a user may experience when performing an increasingly difficult task. A $445,000 grant from the National Science Foundation will allow the interdisciplinary team to incorporate real-time biomedical data with machine learning to produce a more in-tune computer user experience.

Lighting Up the Brain

“fNIRS is an emerging non-invasive, lightweight imaging tool which can measure blood oxygenation levels in the brain,” said Fantini, also an associate dean for graduate education at Tufts’ School of Engineering.

The fNIRS device, which looks like a futuristic headband, uses laser diodes to send near-infrared light through the forehead at a relatively shallow depth -- only two to three centimeters -- to interact with the brain’s frontal lobe. Light usually passes through the body’s tissues, except when it encounters oxygenated or deoxygenated hemoglobin in the blood. Light waves are absorbed by the active, blood-filled areas of the brain and any remaining light is diffusely reflected to the fNIRS detectors.

“fNIRS, like MRI, uses the idea that blood flow changes to compensate for the increased metabolic demands of the area of the brain that’s being used,” said Erin Solovey, a graduate researcher at the School of Engineering.

“We don’t know how specific we can be about identifying users’ different emotional states,” said Fantini. “However, the particular area of the brain where the blood flow change occurs should provide indications of the brain metabolic changes and by extension workload, which could be a proxy for emotions like frustration.”

Tufts University