Abstract

submitted to ECCOM2017

Title:
Topos.sonification: a set of tools for interactive exploration of motion capture data

Author:
Dr. Luiz Naveda

School of music State University of Minas Gerais

Background:
Part of the development of research in music and movement have been promoted by the

dissemination of technologies that capture movement data. Experiments that make use of motion

capture systems are very specific and require a careful planning, experimental design and intense

processing of numerical data. However, researchers often ignore or even minimize qualitative

aspects involved in the development and realization of the experiments: During the process of

definition of hypothesis, definition of analytical procedures of even evaluation of movements,

researchers and subjects are engaged in routines of evaluation and perception of movement

representations. These routines include exploratory activities using visualization and annotation of

moving figures, inspection of graphs and videos and literature review. However, there are many

limitations in the human capacity of exploring relationships in massive databases. How the process

of exploration of movement features and music-movement relationships could be improved?

Mocap systems provide a systematic way of recording the position of markers attached to the

body of musicians and dancers. The position of each marker in the three-dimensional space can
transformed into movement descriptors that include displacement, velocity, and acceleration, which

can be further analyzed in a variety of ways. Not enough, the representation of the full human body

requires dozens of markers distributed on the body of the subject. Before and after the realization of

the experiment, researchers must carefully analyze the hypotheses governing the design. The

unsupported task of browsing data sources, graphs and moving pictures tend to become a tedious

and complex activity. Part of the diagnostic of hypotheses and possible analytical pertain to the

ability to grasp creative and qualitative insights from the available information.

Aim of the study:

The aim of this paper is to describe a set of computer tools that help to inspect and browse

relationships across movement descriptors, movement sources and music.

Methods:
The set of tools, built in open-source Pure Data language, provides realtime connections with

motion capture data representations and realtime motion capture devices via OSC communication.

The tools include functions that enable the evaluation of movement features by means of

sonification (e.g. modulating sound noise according to movement profiles), graphical and numerical

representation, among other functionalities. Since the tools are build using a realtime data

processing platform, they can also be used to creative projects that need to manipulate and process

realtime or offline data sources to control interactive systems.

Conclusion:
The tools provide a comprehensive form of dealing with the complexity of the representation o

movement data by allowing open experimentation, visualization, sonification and content browsing,

which is often not available in traditional analytical packages. The set of tools continues the
development of the Topos library for Pure Data, which provides other computational structures

that involve the processing of motion capture data. The set of tools and examples will be available

online.