Cecilia Mascolo thinks of mobile phones rather differently to most of us. To her the mobile, as the most definitive, ubiquitous personal device that we carry, can give unique insights into our state of mind.
Mascolo has been part of a team at University of Cambridge’s Computer Laboratory exploring mobile phones as ‘sensors’ that can monitor how the user’s emotions change according to their location, surroundings, relationships or the time. EmotionSense integrates information gathered through different features of the phone – location through GPS, movement through the accelerometer, proximity to bluetooth devices as well as excerpts of conversations – to create an impression of how someone is feeling.
At the core is an audio sample library – the ‘Emotional Prosody Speech and Transcripts Library’ – which represents 14 categories of emotions. Excerpts from conversations are compared to this library and then overlaid with data on location and so on, illustrating trigger points for stress or mood at home or work, in crowds or alone and at different times of day.
Dr Mascolo, working with fellow computer scientists and psychologists, is keen to emphasise that EmotionSense does not monitor phone calls, but excerpts of real-world conversations that are deleted as soon as the analysis is completed. This is not a tool for spying, but a very specific development for psychological research, said Mascolo.
“This is very significant because mobile phones are carried by people continuously and they forget who forget [about being surveyed], so new psychological studies can happen over long time scales and with large samples – something they are not yet able to do.”
It’s still early days for this technology, which Mascolo stresses is an academic prototype rather than anything being developed commercially, or, as yet, anything that could make decisive psychological conclusions. But as a proof of concept it has succeeded so far, presented at the Ubiquitous Computing conference in Copenhagen today. Phase one used a small research group of 18 volunteers, using Nokia 6210 Navigator phones running EmotionSense software, over a 10-day period.
But initial results showed that 70% of the EmotionSense results tallied with what the volunteers had reported in a more traditional self-reporting survey. Grouping its analysis into either sadness, fear, anger, neutral or happy, EmotionSense found the home unsurprisingly triggered happy responses in 45% of results while being at work was responsible for 45% of ‘sad’ recordings. Evenings prompted more intense emotions and volunteers were less expressive when in larger crowds.
Phase two of development will focus on making the programme more energy efficient and exploring how additional features of the phone could be used to expand the tool. “The point is where is this technology going, how can we make it safe, secure and unobtrusive,” said Mascolo.
So what are the all important applications for this technology? Researchers are next moving the focus towards well being – what are the triggers for stress, and depression, for example? And all from your mobile phone.