Neurobehavioral classifier module

A set of algorithms implemented as an extension module to analyse the emotional statements of the users with two main aims:

  • Determine an emotional score between an ego and an alter, giving this info to feed the trust model.
  • Analyse the content shared creating metrics to give feedback on how the users response to a specific stimulus.

Module with the following functionalities:

  • Interface to communicate with Neurobehavioral classifier module
  • Metrics of accelerometer sensor for each event in chat activity
  • Sentiment analysis of each image sent or received through Helios chat
  • Sentiment analysis of Helios chat texts
  • Storage system for metrics of the module

HELIOS Neuro-behavioral classifier module is one of the HELIOS Module APIs as highlighted in the picture below:

HELIOS Neuro-behavioral classifier module

Cloning repository

git clone https://github.com/helios-h2020/h.extension-NeuroBehaviouralClassifier.git

Neurobehaviour module

Current version of Neurobehaviour module works with TestClient app (app + modules) with the purpose of integrate callings to Neurobehaviour module when the user interacts with Helios chat.


HELIOS app integration


Please follow the next steps to integrate Neurobehaviour module into your app.

Integration of the module in the app

Open your app in Android Studio.

File > New > Import Module

Go to neurobehaviouralclassifier folder. Select neurobehaviour folder like source directory. Click Finish to import.

New module should appears in first line of settings.gradle file:

include ':app', ':storage', ':messaging', ':profile', ':security', ':context', ':neurobehaviour'


Open build.gradle file of app module. In dependencies section add:

implementation project(":neurobehaviour")


HELIOS Neuro-behavioral classifier module

In order to test all features of Neurobehavioral module, you will need to modify some files in TestClient app folder. Please follow these steps.

Neurobehaviour module inputs

Helios app should activate Neurobehavioral module with this events and its correspondent calling:

We will use the same calling for all type of message (text / audio / image)

  • App starts -> startAccel function to start accelerometer measurement
  • App onDestroy method -> stopAccel function
  • Settings -> Nick name changed -> createCsv method to create files for analysis results
  • User sends a message -> sendingMsg function to start sentimental analysis of message
  • Updating Ego - Alter Trust value -> calling to egoAlterTrust function

Neurobehaviour module outputs

Ego - Alter sentimental analysis

Neurobehaviour module > Trust Manager

Neurobehaviour module extracts metrics from user behaviour:

  • Arousal: positive / negative
  • Valence: positive / negative
  • Attention: low / medium / high

Calling to egoAlterTrust function

When Trust module make the call, Neurobehaviour module sends this metrics to calculate the new Trust value.

Sensors analysis

After user has sent or read a message, Neurobehaviour module sends sensor metrics (acceleration average) using SensorValueListener interface (Context module)

DataHandler.java

//Using Context module interface to send acceleration value import eu.h2020.helios_social.core.sensor.ext.DeviceSensor;

obj.msgId = msgId; obj.accelAverage = averageAccel; deviceSensor.receiveValue(obj);

Using the module

How we can call module functions from other modules through methods of the listener:

Module structure - Java classes

Module view in Android Studio

HELIOS Neuro-behavioral classifier module

HELIOS Neuro-behavioral classifier module

Testing the module

Unit tests carried out using JUnit4 framework testing library

Instrumented unit tests for Neurobehaviour module in:

Android testing path

Info about Android instrumented unit tests:

https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests

Results of conducted tests

Results of unit tests for NeurobehaviourListener class:

Results for NeurobehaviourListener class

Results of unit tests for Acceleration class:

Results for Acceleration class

Running the module

Logcat tags to debug the module

  • v/listen > show info about listener callbacks
  • v/thread > acceleration measurement values (real time linear acceleration and average)
  • v/cv > functions for Computer Vision and results of image sentimental analysis
  • v/text > results of text sentimental analysis
  • v/audio > logs of sentimental analysis of audio files
  • v/storage > it shows info about module storage system operations
  • v/lab > functions for sessions of Neurobehavioural module validation