Progress in the field of Brain-Computer Interaction: scientists take BCI research out of the lab and into the real world

The field of brain-computer interaction (BCI) has been heralded as the next frontier for both medical and personal technologies. BCI ultimately aims to enable people who are completely paralyzed to communicate with the outside world by using their thoughts. However, until now, most research on BCIs for direct communication and control has taken place in laboratories, and this has posed an obstacle to progress in the field. This is because data collection in the laboratory setting is limited to a small number of people, and findings in a lab may not translate to real-life scenarios. What is more, the complex medical equipment that is used in this research area makes systems inaccessible to non-expert users, relatives, or caregivers.

Scientists at the Max Planck Institute for Intelligent Systems and the University of Vienna have now addressed these challenges with MYND, an open-source software that combines consumer-grade recording hardware with an easy-to-use app. They present their research in the paper entitled “MYND: Unsupervised Evaluation of Novel BCI Control Strategies on Consumer Hardware”. The paper was recently presented at the ACM Symposium on User Interface Software and Technology (UIST), a leading forum for innovations in human-computer interfaces. It is now available in the Association for Computing Machinery’s (ACM) Digital Library.

In BCI research, machine learning models use recorded neural activity to deduct specific mental states. However, most systems are only tested in a laboratory and not in real-life environments. By making it possible to run studies outside the lab with affordable hardware, MYND can thus serve as a tool that effectively complements initial basic research by asking questions about actual everyday use. 

“With MYND, we hope to provide a framework that allows researchers to evaluate conceptual BCI systems in realistic conditions. After all, brain-computer interfaces are meant to function wherever and whenever the user needs them,” said Matthias Hohmann, the paper’s lead author, at the ACM Symposium on UIST. Hohmann recently completed his Ph.D. in the Empirical Inference Department at the Max Planck Institute for Intelligent Systems in Tübingen. The paper was co-authored by fellow MPI-IS researchers Lisa Konieczny, Michelle Hackl, Brian Wirth, Talha Zaman, and Raffi Enficiaud. Moritz Grosse-Wentrup, and a former MPI-IS group leader who is now professor of Neuroinformatics at the University of Vienna and Bernhard Schölkopf, Director of at thethe institute’s Empirical Inference Department at MPI-IS in Tübingen, supervised the project.

“Particularly in the midst of the global COVID-19 pandemic, we’ve seen that restricted access to laboratories can severely impact data collection and research efforts in empirical sciences. MYND addresses this challenge effectively. What’s more, given the results of our study, we are confident that our approach could be a viable basis for research on accessible BCI usage in daily life,” Hohmann continued.

Over the course of the MYND study, 30 participants were asked to use a consumer-grade electroencephalogram, a device that can be placed on a head and that records neural activity of the brain., while Ssubjects were asked to engage in specific mental activities: In one short task, they were asked to alternate between thinking of a happy memory and a mental calculation task. In the other, they were asked to alternate between remembering a song and a mental calculation task. Participants did both tasks several times over the course of one week while recording neural activity. After the study concluded, machine learning models were able to deduct the mental task from the recorded neural activity with a significant accuracy of 68.5 percent, and 64 percent, respectively. 

The scientists noted that deduction accuracy – i.e., how different neural activity patterns were between mental tasks – decreased as the study progressed. This may have been because participants were asked to complete the same exercise each day. This, the researchers suggest, could potentially be remedied if participants used MYND to track self-devised brain control strategies on a schedule of their choosing – something that would not be possible in a traditional laboratory setting. The open-source framework thus opens up the possibility to enhance BCI lab research with aspects of human-computer interaction. 

The open-source software is complemented with  code is available for further development and external contributionsresearch on github, along with documentation and guidance for the most likely adaptations, such as the addition of new recording hardware or experimental scenarios. The source code is available for further development and external contributions on github.

Resources

UIST Paper (ACM Digital Library, Open Access):

https://dl.acm.org/doi/10.1145/3379337.3415844

UIST Talk recorded (Youtube)

https://www.youtube.com/watch?v=MEBZ2lwOTrk

UIST Teaser (Youtube):

https://www.youtube.com/watch?v=KxnbNAP2DDk

Software Publication:

http://doi.org/10.5281/zenodo.3802683