splines in space

One day when I was across the hall visiting Sonya Sekula, I noticed thatshe was painting left-handed. I said, “Sonya, aren’t you right-handed?” She said,“Yes, but I might lose the use of my right hand, and so I’m practicing using my left.”I laughed and said, “What if you lose the use of both hands?” She was busy paintingand didn’t bother to reply. Next day when I visited her, she was sitting on the floor,painting with difficulty, for she was holding the brush between two toes of her leftfoot.^[Cage]

how adaptive is our practice, do we make provision for eventualities that may affect the manner in which we work, are we mindful of the future? is it more suitable to transfer our existing methodology to new techniques or should we re-explore the entire working process. whilst painting with the feet may be a change of manipulator, the tools and creative process remain (generally) the same. as 'new' techniques are designed to replicate 'unavailable' techniques most foot (and mouth) panting is no different from works created by hand ^[mpfa]. even paintings created by mechanical manipulators are analogous to 'hand crafted' works due to human creation of the generative algorithms ^[Verostko].

with mechanical / technological systems the problem of 'similarity' can also be found in their predictable function, more than unique instructions or algorithms, 'glitches' of varying severity can skew the process and production in unexpected, and innovative ways ^[Scott]. it is these post-conceptual interventions that both replicate the chance discovery (that can be developed into a new technique) and help to retain 'aura' in the age of mechanical reproduction ^[Benjamin].

however, the glitch is not the answer, just a single solution. there must also be a shift from aesthetic conformity when evaluating 'new' forms. even if there is 'nothing new under the sun' ^{[Ecclesiastes 1:9-11]} concepts of intertextuality ^[litencyc] and intertwingularity make the notion of radical shifts redundant.

Intertwingularity is not generally acknowledged, people keep pretendingthey can make things deeply hierarchical, categorizable and sequential when they can't.Everything is deeply intertwingled. ^[Nelson]

if the skill is in perceiving subtle conceptual (and practical) shifts and revealing (some) of the interplay within the larger 'everything' then a generalist stance is the most obvious, and suitable approach. as a mode of specialisation, generalism is one way to re-unite the arts and sciences.

After a certain high level of technical skill is achieved, science andart tend to coalesce in esthetics, plasticity, and form. The greatest scientists arealways artists as well ^[Einstein]

dance needs a new generation of artist-scholars who are capable of developing new research (practical & theoretical) that is valid across multiple research domains. there are too few dance & science collaborations in which the dance practitioners have imparted significant knowledge of dance (to science), developed new constructs (in either field) or expanded existing knowledge. indeed, sometimes even the creative aspects of these sci-art collaborations is questionable. i particularly have deep misgivings about the motion^e project ^[ASU] in terms of cost, creativity and theory. A reading of the projects publications reveals elementary flaws in their understanding of dance praxis and minimal critical engagement with dance theory. i also question why a live version of How long ... ^[Brown] failed to make it to the uk (a video was shown instead).

yet (current) sci-art praxis should not have to bear any fruit, sometimes the act of engagement itself is important. nor should overly concern ourselves with theoretical flaws if the creative product is engaging (i.e. Constant Speed ^[Baldwin]), but we must identify any such flaws in our scholarly publications. dance theory is still seen as an overly subjective disapline that extensively borrows from other fields in order to validate itself. we can only defeat this notation by demonstrating the value of movement praxis by engaging with and effecting change in multiple disciplines. there is some great sci-art research that has clearly achieved this, but in general such collaborations need to acknowledge that dance more than a vehicle for demonstrating or validating concepts.

It is not the wind that moves, it is not the flag that moves; it is yourmind that moves. ^[Hui-neng]

we must be adaptive in our perspectives both practical and conceptual. the false oppositions of science vs. art, theory vs. practice, embodied vs. virtual must be thrown away. all practices are intertwingled yet can still remain 'pure'. adapting to new approaches of perceiving, producing, performing and participating in dance can only strengthen dance praxis. exploring the multiple intersects of dance and wider contexts will further reveal the integral role movement plays in our lives.

zach lieberman has posted a beta version of Rotosketch a tool developed with Scott DeLahaunta and Susan Rethorst as part of the Dance Theater Workshop’s Digital Fellows Program ^[dtw].

Rotosketch is quite simply an intuitive tool for sketching, doodling and notating on top of video,such that the marks that are made are linked in time with the video. This allows theuser to draw strokes along the the axis of time, as well as the normal x and y axes,and for those strokes to augment, analyze, interpret, or even obliterate a video sequence.

via: networked_performance

vvvv is a toolkit for real time video synthesis and connecting physicaldevices. [...] main features include runtime graphical programming, the effortlesshandling of a multitude of objects, support for a lot of exotic hardware, live integrationof directx shaders and support for the clustering of standard PCs (for creating seamlessmulti-projection or multi screen systems).

vvvv - a multipurpose toolkit

In Zen they say: If something is boring after two minutes, try it forfour. If still boring, try it for eight, sixteen, thirty-two, and so on. Eventuallyone discovers that it’s not boring at all but very interesting. ^[Cage]

there is a need (for me) to move beyond the experiential and (further) into the technical and functional. there are limits to our cognitive and experiential exploration that can be expanded and stimulated by algorithmic processing. what has become problematic are the contextual (and theoretical) issues of culture.

for much of computer science (CS) the socio-cultural impact of technological advances is relatively unimportant; a moral issue secondary to pure research. yet the affect technologies will have on dance praxis is largely underestimated. for me the key issue is developing technologies that will develop dance practice in new dimensions (physical, cognitive, virtual etc.). April Nunes got to the core of my 'real world' thinking;

I’d be really interested in investigating the effects of the programmeon the thought processes of performers especially in the context of improvisation.To me, it seems that data from this programme could serve as another layer in thecyclical mind-body feedback loop: The performer repeats an image in their mind untilthe body responds -- the performer perceives the body’s form/movement and envisionsthe image seen in the computer programme – the form/movement of the body respondsto this new image -- and so the cycle continues. ^[Nunes]

this kind of computer-augmented dance praxis is much more subtle that most practitioners and theorists would imagine, and relies (somewhat) on allowing digital dance culture to speak 'for itself' (and possibly) outside our preconceived aesthetic ideals. this is not synthetic stimuli but as valid as any other method we might use to re-visualise and re-imagine our dance practice.

what becomes problematic is how to explore digital culture to develop tools, this is more the realm of individual practice or socio-anthropological study than computer science. i'm not saying that is how it should be, but how 'established' patterns of research are playing out. taking a challenging research stance is not purely the preserve of the arts;

art and science are important for the training they give each practitionerin an ethics grounded on joining the stance of the rebel with the stance of respect.When one has learned to do this, one can participate meaningfully in a community foundedon a shared ethics. In this community each individual is truly free, but each is boundto the community by the respect we feel for those others who are willing to put everythingat risk in every working day, in the knowledge that this is the only way to createsomething worth preserving. ^[Smolin]

practitioners of all disciplines run the risk of getting lost in a sea of specialisation that becomes a maelstrom of the mundane. rebels are dismissed out of hand even if they do participate positively with the greater community. specialisation should lead us on a journey that interacts with other fields of study.

thus i find myself assuming opposing stances, both embracing and rejecting cultural contexts in order to 'do new'. so for the moement code is 'king' ...

Gough, M. (2005). Towards Computer Generated Choreography: Epikinetic Composition. In Proceedings of the Hothaus seminar series. Birmingham: Vivid.

Towards Computer Generated Choreography: Epikinetic Composition.

Whilst Dance and technology practitioners embrace computer software for the production of dance works, there is a general desire to retain human authorship in the conceptual and creative process. Artificially creative technologies developed for algorithmic art and music have yet to make an impact on the dance making process other than to provide visual or aural accompaniment. We believe that not only can the choreographic process be simulated and automated, but that such a simulation would present a viable alternative to more traditional methods of dance composition. After examining existing methods of algorithmic choreography we present an alternative method (epikinetics) that facilitates both algorithmic choreography and autonomous dancing avatars.

Algorithmic options

In 1995 Lansdown called for 'more use [to be] made of the algorithmic approach to choreography' in order to find new 'narratives' and methods of composition [1]. Whilst methods of algorithmic composition have been developed for music [2] and the visual arts [3], issues such as modelling unique, expressive performance [4] have stalled the development of similar systems for choreography. Yet dance is essentially a motive response to stimuli, consisting of ‘movement invention' [5] (concept - ideas / stimulation; movement realisation) and the human body. These elements can be simulated using a combination of neurological and biomechanical models. For the purposes of modelling we consider choreography (performance and creation) to be a three part process:

• Choreography: the organisation of the (composition) discrete structures into the larger 'whole'
• Composition: the discrete movements and strucures
• Improvisation: the 'navigation' between choreographic structures in the absence of instruction and the generation of 'novel' movement invention as it is realised (instantaneous choreography).

Existing methods of algorithmic choreography model one or two processes rather than all three and are achieved by several methods including:

• Choreographic software: for choreographic stimulation
• Performance technologies: enabling real-time choreography
• Human implementation: algorithmic / generative dance
• Simulation and Synthesis: autonomous virtual dancers

Choreographic software often takes the form of a multimedia 'sketch book' format such the Interactive Choreographic Sketchbook [6] or Limelight [7]. This group of applications allow the choreographer to enter images, music, dance notation, animation, motion capture data and other stimuli into a timeline based ‘stage’ and develop the choreography through rapid manipulation and pre-visualisation. Although such programs use a variety of algorithms to 'creatively' filter and process stimuli, they are incapable of autonomous composition and improvisation due to the relatively ‘static’ format of the data entered. The challenges of animating dance notation [8] and rapid capture, re-targeting and warping of motion capture data [9] severely limit the possibility of sweeping conceptual shifts though generative or emergent means.

A possible alternative to the sketch book concept is be a notation based 'pen and pad' model with animated 3D preview and Tanzkurven editing [10] that would allow increased interpretive and transformative freedom through the use of non-deterministic, context-sensitive genetic algorithms. This structure would allow movement scores to be generated from a notational lexis with minimal human intervention. A non-deterministic, syntactical, notation based approach could also be applied to performance technologies to facilitating real-time autonomous composition during a performance. The presence of algorithmic composition and improvisation in rule based performance technologies such as E-merge [11] and ChoreoGraph [12] would compliment the real-time choreography these ‘emergent’ systems already provide. Without this capability the advantage of performance technologies over generative dance (accumulation, chance procedures) is minimal due to a reliance on human composition and improvisation. Choreographic narratives remarkably similar to those generated by performance technologies can be found in traditional methods of composition and human implementations of algorithmic and cognitive models.

Davide Terlingo' has experimented with ‘fractal’ choreography with 'Nonlinear Generators' [13], a practical application of Genetic Algorithms for composition. Terlingo extends a set of 10 static positions and connecting movements through deterministic, context-free algorithmic composition to generate novel movement sequences. However, the dependence on human creativity results in a movement narrative influenced by the performer rather than the algorithmic procedure. Similarly, Hagendoorn's application of motor control, perception and cognition to dance improvisation [14] has also resulted in techniques and narratives similar to existing methods. The emphasis on organisation in Hargondorn and Terlingo's techniques (at the expense of movement invention) limits the possibilities for a computer based algorithmic implementation as they provide no mechanism for generating movement.

One approach to remove human movement bias (and allow new narratives) is through the use of virtual dancers. Virtual dancers can be used to pre visualise choreography generated by keyfame animation and motion capture data. Although these techniques generate ‘fixed’ movement material various 'noise' or 'motion texture' functions (layering pseudo random movement onto existing motion data) can be used to synthesise improvisation. Several methods of linking discreet segments of motion data (re-targeting, warping, chaotic selection, motion matching) can also be used to simulate compositional autonomy but do not facilitate a fully autonomous system.

To achieve compositional and improvisational autonomy for virtual dancers an algorithmic method for movement simulation is required. Nakata's 'automatic choreography' [15] uses an algorithmic approach to generate 'life like' movement without the need for motion capture data. However, his implementation of a somatically coordinated structure (limited of degrees of freedom to enable motive control) is unsuitable for improvised movement generation. Regardless of this limitation, algorithmically generated movement offers a method by which autonomous algorithmic choreography can be achieved.

Epikinetics

epi - "on, at, close upon (in space or time)"
kinetic - "moving, putting in motion"

Epikinetic motion simulation is related to Epigenetic Robotics (motion as genotype) but deals exclusively with the generation of raw, unstable movement rather than controlled motion. This raw movement is then processed by a series of algorithmic modules that shape the movement and present it as animation or notation. Generating unstable, rather than stable movement provides a high level of motive agility and responsiveness. Similar techniques can be found in advanced Aeronautics where adaptive motion control algorithms are used with unstable platforms (such as the F-117 'Nighthawk') to maintain level flight and keep the aircraft 'in the air'. It is through such adaptive algorithms, with differing methods of intervention that autonomous algorithmic choreography can creative distinct narrative forms.

An epikinetic system models the Somatic Nervous System (SNS) including effectors (muscle) receptors (nerves), and control systems such as the Premotor Cortex, Basal Ganglia and Cerebellum). Our system (chorea) consists of six principal modules:

• vitus (epikinetic improvisation)
• koan (choreography and composition)
• magnesium (contact improvisation)
• zanshin (event perception / memory)
• hierarchical skeleton / dynamic environment
• stimuli

vitus

The vitus module is the core of chorea functiality. An inversion of Nakata's coordinated whole it replicates the 'moment of movement' in multiple locations as an 'uncoordinated interrelated whole'. An epikinetic algorithm generates movement data by processing each joint and bone of the Hierarchical Skeleton concurrently, but individually in realtime. These uncoordinated motor images [16] are then passed onto the hierarchical skeleton and dynamic environment where they are bound by the rule of human physics and biomechanics (the related whole).

This method of producing motor images without first conceiving the motion to be generated provides a high level of improvisational fidelity and might be considered a simulation of neurological movement disorders such as chorea. The development of our movement disorder influenced method is distinct from recent choreographic research into Ataxia, the inability to coordinate movement [17].

Hierarchical skeleton / dynamic environment

The hierarchical skeleton / dynamic environment is used to process the motor images from the vitus module. Unlike existing approaches the movement simulation the skeleton does not 'drive' the motion but responds to the motor images. Rather than simulating the mechanical properties of motion we simulate the impulse, or ‘moment of movement’. This shift in perspective is reflected in the way the skeleton deals with bio mechanically impossible motor images, rather than stopping the movement at the human limitation it will arrange the surrounding limbs to accommodate the desired motion. Such adaptive reconfiguring can be found in numerous dance styles including salsa and contact improvisation.

The motor images are processed by a variety of existing methods for animation and movement simulation. Whilst some of these methods bear little relation to dance practice, other solutions have direct parallels:

• Forward Kinematics : Proximal movement
• Inverse Kinematics : Distal movement
• Forward and Inverse Dynamics : Release Technique

By using a range of motion synthesis techniques it is possible to reveal the full range of movement styles available to the human body. Processed motor images are exported as motion data in the form of animation (rendered motion capture data) or notation to enable human performances of the choreography.

zanshin

All events occurring in the performance are stored in the zanshin (awareness) module for the duration of the performance. This allows koan and magnesium modules to ‘perceive’ events occurring within the performance and react with new motive responses. Data entered into the stimuli module is also passes through the zanshin module, only knowledge of current performance retained, the absence of a long term memory allows greater improvisational freedom.

Koan

Algorithmic composition and choreography is generated by the koan module which has the ability to override motor images from both vitus and magnesium. Koan utilizes a range of formal composition techniques (cannon, mirroring, unison etc) and is responsible for generating motive responses to internal and external stimuli such as music and limb arrangement. Koan defines its own concept(s) for the dance work, ideas for the realisation of concepts, and motor images for processing by the hierarchical skeleton.

magnesium

Points of contact that occur during the performance are monitored by magnesium. This module provides an artificially intelligent method for contact improvisation along with more general touching and lifting tasks. Magnesium generates motor images and performs adaptive motion control through a recursive process (based on the principles of contact improvisation) whilst maintaining compositional freedom.

stimuli

The stimuli section may be used to specifically influence the algorithmic choreography of koan. Data entered is flagged with a different level of importance as it passes through to zanshin to become a motive response. Apart from stimuli such as music and images the inclusion of motion capture data is facilitated, this allows chorea to 'see' the dancer in motion before creating them a specific choreography.

Algorithmic receptivity

To achieve computer generated choreography (autonomous rather than human assisted) the entire process, from conception, ideas (choreography / composition) to the moment of movement (improvisation) need to utilise algorithmic methods. The ability to create movement without the need for existing movement data (chorea reuses motor images) is an essential requirement for any choreographic software designed to automate and or stimulate the choreographic process.

Epikenetic systems deal exclusively with motive responses and are unencumbered by a cognitive or emotive understanding of the stimuli. As with work generated by chance procedures the resultant movement is disassociated from the stimuli, thus ‘information - contemporary receptivity - movement’ [18] becomes ‘information - algorithmic receptivity - movement’. Although any emotional content has been removed the choreography remains embedded in physical reality, allowing it to be mounted on ‘real’ dancers and via motion capture the computer can 'coach' the dancers’ performance. The loss of human ownership in both the choreographic and interpretive processes should allow the deployment of new narrative forms, and physical techniques. These developments may eventually become a part of general dance practice rather than isolated in specific performance works.

Towards the future

Systems such as chorea will not replace dancers and choreographers, autonomous choreography virtual dancers can and will exist alongside their physical counterparts, illuminating new narratives and forms beyond our conceptually biased imagination. A loss of authorship to the computer should only strengthen artistic resolve and refresh creative energies. The simulated Sensorimotor system and artificial body intelligence of chorea reveals the capabilities of the human body rather than the limitations to which we confine ourselves. If we are develop new narratives and forms we must let our bodies do the talking, through Epikinetics this dialog is technologically mediated.

[...] ways to connect [movement] can be algorithmically redefined infinitely.Since we're no longer restricted to the prescribed classical methods of connection,we're open to an extraordinary leap in connection, which is just a matter of definingconnective space. ... Where I'd start is with the score. What's been missing so faris an intelligent kind of notation, one that would let us generate dances from a vastnumber of varied inputs. Not traditional notation, but a new kind mediated by thecomputer. William Forsythe [19]

With thanks to Bernard Easterford and Anna Jattkowski-Hudson

[1] Lansdown, J. (1995) Computer-Generated Choreography Revisited. in Proceedings of 4D Dynamics Conference. A. Robertson. (pp 89-99). Leicester: De Montfort University.

[2] Cope, D. (2001). Virtual music: computer synthesis of musical style. Cambridge, Mass.: MIT Press.

[3] Roman V. (1990) Epigenetic Painting: Software as Genotype. Leonardo 23 (1), pp. 17-23.

[4] Camurri, A., Mazzarino, B., and Volpe, G. (2004) Expressive interfaces. Cognition, Technology & Work 6 (1), pp. 15-22. Heidelberg: Springer-Verlag

[5] Rainer, Y. (1968) A Quasi Survey of Some 'Minimalist' Tendencies in the Quantitatively Minimal Dance Activity midst the Plethora, or An Analysis of Trio A. In Buttcock, G. (ed.) Minimal Art: A Critical Anthology (pp 263-73). New York: E. P. Dutton.

[6] deLahunta, S. (2004) Interactive Choreographic Sketchbook. Retrieved October 2004 from http://www.sdela.dds.nl/sfd/icsketch.html

[7] Griesbeck, C. (1996) Limelight. Retrieved October 2004 from http://user.uni-frankfurt.de/~griesbec/LIMEE.HTML

[8] Neagle, R.J., Ng, K.C., and Ruddle, R.A., (2004) Developing a Virtual Ballet Dancer to Visualise Choreography. In Proceedings of the AISB 2004 on Language, Speech & Gesture for Expressive Characters (pp 86-97). Brighton: AISB

[9] Zordan, V. B., and Horst, N. C., (2004) Mapping optical motion capture data to skeletal motion using a physical model. In Proceedings of the 2003 ACM SIGGRAPH / Eurographics Symposium on Computer animation (pp 245 - 250). Aire-la-Ville: Eurographics Association

[10] Nakata, T. (2003) Digital Tanzkurven. Retrieved October 2004 from http://staff.aist.go.jp/toru-nakata/tanz/tanzkurv.html

[11] Willcock, I. (2004) eMerge performance scripting. Retrieved October 2004 from http://www.unsafesound.com/Docs/eMergelanguage.doc

[12] deLahunta, S. (2002) Duplex / ChoreoGraph: in conversation with Barriedale Operahouse. Retrieved October 2004 from http://www.sdela.dds.nl/sfd/frankfin.html

[13] Terlingo, D. (2003) Nonlinear Generators. Retrieved October 2004 from http://www.choreograph.net/forum/showthread.php?threadid=121

[14] Hagendoorn, I.G. (2003). Cognitive Dance Improvisation. How Study of the Motor System Can Inspire Dance (and vice versa), Leonardo 36 (3), pp. 221-227.

[15] Nakata, T. (2002) Generation of whole-body expressive movement based on somatical theories. In Proceedings of the second international workshop on Epigemetic Robotics pp.105-114 [16] Gallagher, S. (2001) From Action to Interaction: An Interview with Marc Jeannerod. Institut des Sciences Cognitives web site Retrieved October 2004 from

[17] deLahunta, S. (2004) Choreography & Cognition: Experiments. Retrieved October 2004 from http://www.choreocog.net/exper.html

[18] Franko, M. (1995). Dancing modernism/performing politics. Bloomington: Indiana University Press

[19] Kaiser, P (1998) Dance Geometry: A conversation with William Forythe. http://www.openendedgroup.com/ideas/pdf/forsythe.pdf