Introduction | Background Work | Expressive Processing | Conclusions and Future Work | Bibliography

Expressive Processing

“(…) I think it could affect choreographer’s experience of movement in the same way electric light first altered the way visual artists saw the world (…)”

From Merce Cunningham Discussing on motion capture software “Life Forms” custom developed for his experimental choreography work. In Merce Cunningham - Modernizing the Modern Dance, Roger Copeland, 2004 Routleedge.

Project Objectives

This work intends to explore the usage of motion, gesture and expressiveness capture as means to provide video footage and control data for post-production, resulting in a video art piece supported on DVD, instead of the typical usage of this technology for performance in front of a live audience or public interactive installations. It is the author’s believe that this work process will lead to a very engaging result since he will depart from raw material, which is already strongly co-related.
The final result of this work will go beyond the audiovisual registration of a performance or a public display of an installation, conveying all of this in a final Video Dance Art Peace.

Creative Process

Conceptually, Expressive Processing departs from the idea of continuing the same line of work carried out by the author over the last years. It approaches the use of technology in the context of dance performance as an interactive performative element and as a tool to unveil hidden expressions and to correlate gesture, movement, sound and image.
At the performance stage a dancer faces her own representation as a shadow that mutates to an animal.
Compared to the dancer’s body this bidimensional shadow is a very raw representation in terms of movement and articulation. However, at some point a sense of displacement is achieved between the dancer and her shadow.
The dancer’s gestures mostly aim to produce an affect in her shadow as if she was embodied by her representation.
The same way that in “Expression’s Limits” physical constraints were imposed to the dancer’s gestures, by enclosing her inside a small box, and “Human Processing” limits the dancer’s gesture by removing contact with any steady surface, “Expressive Processing” constrains the dancers movement by providing an immersive environment in which the dancers body is displaced into a 2d virtual character with limited movement and articulation abilities.
A second iteration in this project concerns the use of motion capture technology in order to have a logic registry of the dancer’s gesture. This data is then Processed to algorithmically create an audiovisual piece that revels and emphasizes the Expressiveness attained by the dancer under the constraints imposed by the system. This expressiveness is present at the dancer’s body articulation, dynamics, subtle face and body posing and at an overall spontaneous experimental sense of movement.

Project Methodologies and Production Processes

Real-Time technology and computer vision is used to provide data (video and Logic-MIDI), which are synchronously bounded together since the moment of registration and provide rich raw material to be manipulated and post-produced.
Based on this process it is possible, in post-production (trough sound and image processing), to emphasize several aspects of the expressive motion, which convey the inner self of the performer and the way He/She reacts to the mutated reflection of His/Her body.
This method brings Micro-Expressions to Life. These Micro-Expressions somehow represent a measure of the performer’s human cognitive feedback.

Project Time-Line:

Process Design and definition of two production stages, technical requirements, physical and logic diagram.
Briefing with other artists (Dancer and Musician)
Experimentation with lightning and motion capture with the software EyesWeb

Implementation and enhancement from the previous version (Bestiario) of the virtual character in the Software Virtools. The Shadow is now thigh and more stable.
Implementation of an infrared lightning and video capture systems that allows independence from collateral lightning on stage (video projection and spotlights)
Operational setup of the installation at the main auditorium of Porto School of Arts. Equipment: Full video recording setup for 7 channels; Video retro-projection (5X4 meters screen); Lightning; System hardware (2 computers – Capture and Generation – communicating through MIDI interfaces); 4 technical operators.
Video registration of experimental performance (7 cameras, 60 minutes of video footage for each camera)

Basic digitization, editing of raw footage
Experiments with video compression to meet Jitter software requirements.
Development of basic Jitter algorithmic video editing patch.

Definition of matrix screen divisions
Definition of Scaling, cropping and positioning methods
Definition of templates for the final algorithmic edition
Routing of input MIDI Data (obtained from video detection of raw footage) to provide control of the Templates sequence.
Frame rate adjusting

Conversion of Video to MIDI with Eyesweb ans Sybelious Software, in order to provide basic MIDI data to the Music Composer (in sync with gesture captured on video)
Music Generation and editing
Video Generation and editing
Integration of music and video

System overview

The overall system is designed in such a way that it will induce a certain behavior in the performer during the interaction process. This behavior will develop throughout different phases in which the performer will raise awareness of His/Her reflection, at first with a monstrous and disturbing mutated body that unexpectedly becomes entertaining. Progressively the performer will empathize with this hybrid image and will complete the body displacement process.

The system is divided in two Stages:

[Stage 1: Capture]

[Stage 2: Post-Production]

Interfacing and mapping

| A Project by Helena Figueiredo: hfortuna[at]final.upf.es; hfigueiredo[at]porto.ucp.pt |