The software

The EEGsynth, especially the modules (see below), are programmed mainly in Python, which is the open-source programming language of choice for programming by many scientists, professionals and hobbyists. We use a minimum of compiled C-code. We are currently in the process of documenting the EEGsynth so that anyone should be able to understand it enough to use it, and that those who have a bit of experience with programming and hardware will be able to make their own. Here we explain how we designed the EEGsynth. If you want to install your own EEGsynth, and go into more detail, you can follow the more technical and specific documentaion and installation instructions on our GitHub page.

Modules

For many reasons, not the least to allow easy distributed collaboration, we developed a code architecture where the functions of the EEGsynth are split up into parallel modules, directly inspired by the modular synthesizer. Similarly as in a modular synthesizer, simple modules are connected, or “patched” together, to create complex and flexible behavior. The idea is that each module runs in parallel, performing a particular function. As an example, imagine module A is responsible for analyzing ECG (voltages from the heart) to determine the heart rate, while Module B sends out a signal to a synthesizer at every n milliseconds, i.e. functioning as a simple sequencer. By connecting (patching) the input of module B with the output of module A, the EEGsynth can be made to control a synthesizer at the speed of the heart rate.

Patching

So, how have we implemented the patching of modules? Indeed, there are many ways of doing this. Importantly, there it has to be a way by which modules can be connected in a flexible way, allowing many-to-many connections. Take a google-image look at analog modular synthesizers, and you know what I mean. We have opted to use the open-source Redis database which stores attribute-value pairs. Attribute-value pairs are nothing more than an attribute name with a value assigned to it, such as [“Name”, “John”] or [“Height”, 1.82]. A module can put anything it wants into the database, such as [“Heartrate”, 92]. Another module can ask the database to bring back the value belonging to [“Heartrate”]. Of course, the modules will need to know what to ask for, and where to (out)put their own values for other modules (if any) to use.

Figure 1. Visual depiction of communication between modules via either the FieldTrip buffer for raw data (orange) or via the Redis database (blue) for output and input parameters.

 

Initialization files

What is important to understand, is that there has to be a place to do all this book-keeping of the patches. It is not enough to know what modules are patched with what other modules, as modules can have more than one input and more than one output. So, in other words, we need a place where we specify which input attributes are connected to which output attributes. Furthermore, because modules are fully independent and don’t know anything about the other modules we need several places to do this – one for each module. Each modules therefor has it’s own .ini file (initialization file).

The .ini file is a basic text file with simple human-understandable formatting (according to Python’s ConfigParser class) where we define the attribute names for input and output. In addition, we will need to specify default values for the input attributes, in case the other module it depends on hasn’t supplied any yet. Finally, several modules need settings that have to be changes for different setups, e.g. which USB port to send MIDI codes to, or to receive control signals from. The .ini files can be edited with any text editor, or via a web interface the EEGsynth supplies.

Overview of the modules

Detailed information about each module can be found in the README.md included in each \module directory. You can also read them on the GitHub site.

Analysis

  • Brain Analyzes power in frequency bands in the raw data buffer
  • Muscle Calculates RMS from EMG recordings in the raw data buffer
  • Accelerometer Extracts accelerometer data (X,Y,Z) from onboard sensor of the OpenBCI  in the raw data buffer
  • EyeBlink Detects eye blinks in the raw data buffer
  • HeartRate Extracts heart rate in the raw data buffer

Data acquisition

  • Openbci2ft Records raw data from the OpenBCI amplifier and places it in the buffer. 
  • Jaga2ft Records raw data from Jaga amplifier and places it in the buffer
  • For more supported acquisition devices look here

Communication between modules

  • Redis The database for communicating data between modules
  • Buffer FieldTrip buffer for communicating raw data

Utilities for optimizing data flow, patching and prototyping

  • Playback Play back pre-recorded raw data
  • PlotSignal Plot raw data and frequency decomposition in real-time
  • Playbackctrl Playback pre-analyzed data (output from modules)
  • PostProcessor Allows computations, algorithms and combinations on the data (output from modules)
  • PreProcessor Preprocess raw data, and places it in new raw data buffer

External interfaces (open-source)

External interfaces (consumer)

Software synthesizer modules

  • Pulsegenerator Send pulses, i.e. for gates, or MIDI events synchronized with heart beats
  • Sequencer A basis sequencer to send out sequences of notes
  • Synthesizer A basic synthesizer to send our waveforms
  • Quantizer Quantize output chromatically or according to musical scales

COGITO project

  • Cogito Streaming EEG data (from the Gtec EEG) to audio for radio transmission, using our interstellar protocol