Using electric signals from human brains, new software can perform computerized image editing

Soon, computers could sense that users have a problem and come to the rescue. This is one of the possible implications of new research at University of Copenhagen and University of Helsinki.

“We can make a computer edit images entirely based on thoughts generated by human subjects. The computer has absolutely no prior information about which features it is supposed to edit or how. Nobody has ever done this before,” says Associate Professor Tuukka Ruotsalo, Department of Computer Science, University of Copenhagen.

The results are presented in a paper accepted for publication at the CVPR 2022 (Computer Vision and Pattern Recognition).

Brain activity as the sole input

In the underlying study, 30 participants were equipped with hoods containing EEG electrodes that map electrical brain signals. All participants were given the same 200 facial images to look at. Also, they were given a series of tasks such as looking for female faces, looking for older people, looking for blond hair, etc.

The participants did not perform any actions and looked briefly at the images—0.5 second for each image. Based on their brain activity, the machine first mapped the given preference and then edited the images accordingly. So if the task was to look for older people, the computer would modify the portraits of the younger persons, making them look older. And if the task was to look for a given hair color, all images would get that color.

“Notably, the computer had no knowledge of face recognition and would have no idea about gender, hair color, or any other relevant features. Still, it only edited the feature in question, leaving other facial features unchanged,” comments Ph.D. Student Keith Davis, University of Helsinki.

Some may argue that plenty of software capable of manipulating facial features already exists. That would be missing the point, Keith Davis explains:

“All the existing software has been previously trained with labeled input. So, if you want an app which can make people look older, you feed it thousands of portraits and tell the computer which ones are young, and which are old. Here, the brain activity of the subjects was the only input. This is an entirely new paradigm in artificial intelligence—using the human brain directly as the source of input.”

Possible applications in medicine

One possible application could be in medicine: “Doctors already use artificial intelligence in interpretation of scanning images. However, mistakes do happen. After all, the doctors are only assisted by the images but will take the decisions themselves. Maybe certain features in the images are more often misinterpreted than others. Such patterns might be discovered through an application of our research,” says Tuukka Ruotsalo.

Another application could be assistance to certain groups of disabled people, for instance allowing a paralyzed person to operate his or her computer.

“That would be fantastic,” says Tuukka Ruotsalo. “However, that is not the focus of our research. We have a broad scope, looking to improve machine learning in general. The range of possible applications will be wide. For instance, 10 or 20 years from now we may not need to use a mouse or type commands to operate our computer. Maybe we can just use mind control.”

Calls for policy regulation

However, the coin does have a flip side, according to Tuukka Ruotsalo: “Collecting individual brain signals does involve ethical issues. Whoever acquires this knowledge could potentially obtain deep insight into a persons’ preferences. We already see some trends. People buy ‘smart’ watches and similar devices able to record heart rate etc., but are we sure that data are not generated which give private corporations knowledge which we wouldn’t want to share?”

“I see this as an important aspect of academic work. Our research shows what is possible, but we shouldn’t do things just because they can be done. This is an area which in my view needs to be regulated by guidelines and public policies. If these are not adapted, private companies will just go ahead.”