Measuring emotion is a hot topic in science. All over the world scientists are studying different techniques and methods to measure emotion, often using with expensive equipment. All these efforts are focused on individuals in a controlled environment, with the most advanced technological equipment.
Our journey in measuring emotions started in 2014, when Eaglescience joined the COMMIT/ SWELL project. The goal was to be able to develop user-centric sensing and reasoning techniques that help to improve physical well-being in a private context and to improve well-working in a work environment. During this challenging project, we aimed for measuring and visualising emotions in real time, in order to enrich and heighten the experience of a person. Arnon Grunberg added to this project by writing a book for especially this purpose.
A portable observation lab, The Grunberg Lab, was set up, so Arnon could write his novella, “Het Bestand”. While writing this book, a satirical story in which the online world is being criticised, Arnon’s brain activity and subjective emotions were measured. The measurements were subsequently correlated to the text he wrote. For instance, if Arnon was writing about anger or disgust, would he experience these emotions himself? Next step in this project was to measure the emotions of almost 300 readers under controlled circumstances. Emotions were individually measured using brain activity (EEG), heart rate (ECG), and galvanic skin response (EDA), while reading Arnon’s novella. After analysis, it turned out that neurophysiological reactions could be matched with reported emotions.
“Within this system all collected data were analysed by algorithms that indicated individual emotions on a scale of valence and arousal.”
In 2015, the Brainwave project took the crucial step in the development of a commercial system based on off- the-shelf consumer electronics, used in a simulated real world setting. Our aim was a proof of concept consumer product, portable and comfortable, quick and easy to set up and connectable to WiFi or Bluetooth. The platform consisted of three components: tablet, sensors and server.
We developed a mobile application (running on a tablet), collecting and transferring data, from multiple sensors measuring brain activity (EEG), heart rate (ECG), Galvanic skin response (EDA) and facial expressions (webcam), to a system. Within this system all collected data were analysed by text mining algorithms that indicated individual emotions on a scale of valence and arousal. This resulted into the Brainwave eReader, which enhanced the reading experience for e-book/tablet readers by measuring emotions of the reader and sharing them with other readers. For the Brainwave eReader new features, like images and sounds, adjusting the storyline or engaging dialogue between readers, were created.
This year, we brought the results of the COMMIT/ SWELL project and Brainwave project one step closer together in the project Art Designed through Emotions (ADE). Our aim was to create a unique experience by measuring, visualising and presenting the group emotions of 20 people. In order to measure and visualise group emotions, we had to scale up our original platform intended for singe users. Different hardware had to be considered to allow the platform to be used with 20 people at the same time.
The time between measurement and output had to be reduced to real-time. In this project, we required algorithms to calculate the emotions of a small crowd. In short, we improved the platform, algorithms, and software to be able to cope with groups of users simultaneously. Each person was connected to one node.
The collected individual data of these nodes were then send to a group analyser node, which distributed the workload over several processing nodes, making optimal use of modern multi-core processors. We presented our unique group emotions measuring experience during Polaris: the opening act of the Amsterdam Dance Event 2016 and during the Frankfurter Buchmesse 2016, were Arnon Grunberg read his book “Het Bestand”.
The data we collected, will be examined by TNO for scientific purposes. This knowledge contributes to the goal of the COMMIT/ SWELL project. Following, we will be examining the possibilities for applications in a more commercial and creative context.