splines in space

the folks at Hyperchoreography might want to take a good look a the Korsakow-System [via futurefeeder], a set of software for creating interactive films or interactive database narratives. Korsakow-projects are

• interactive - the viewer has influence on the film.
• rule-based - the author desides on the rules scenes relate to each other, [they do] not create a fixed order.
• generative - the order of the scenes is calculated while the viewer looks at a Korsakow-project.
• [only viewable] on a computer. They can be delivered via internet-streaming, DVD-Rom or CD-Rom.

there are several projects online including 13terStock and [7sons], you can also download the software and use it for yourself. whilst much closer to the nonlinear concept and examples than the hyperchoreography examples is it not 'the' solution for a 'true' hyperchoreographic solution. it does however show a path that could and should be expanded on from a dance perspective... well impressed.

after visting an exhibition that included Bibliomania (a project by Simon Morris) i construted my own list;

• Adshead-Lansdale, J. (1999). Dancing texts: intertextuality in interpretation. London: Dance Books
• Alberro, A. (1999). Conceptual art: a critical anthology. Cambridge, Mass.: MIT Press
• Asimov, I. (1982). The complete robot. London: Granada
• Beckett, S. (1984). Collected shorter plays. 1st ed. New York: Grove Press
• Hesse, H. (1998). Siddhartha. (H. Rosner [Trans]), London: Picador
• Hesse, H. (1987). The glass bead game. (R. Winston [Trans]), London: Picador
• Joyce, J. (2002). Finnegans wake. London: Faber
• Levi, P. (1996). The periodic table. (R. Rosenthal [Trans]), Everyman's library 218. New York: Alfred A. Knopf
• Melville, H. (2003). Moby-Dick, or, The whale. Penguin classics. New York: Penguin
• Wells, H.G. (1933). The scientific romances of H.G. Wells. London: V. Gollancz
• Yamamoto, T. (199u). Hagakure: the book of the samurai. (W.S. Wilson [Trans]), Tokyo: Kodansha International
• Zalewski, M. (2000). Feminism after postmodernism: theorising through practice. New York: Routledge

under willows shade
dragonflies dance ...
crushed grass.

a week of borking other peoples systems, trying to discover how easy/hard it is to implement some key aspects of the Æma construct. well it's not as simple as i would have wished. the big picture is getting bigger and bigger, and that worries me. there seems is so much work to do, what do i start with first ...

trying to finsh papers and code and ... ...

Cognitive Daily asks 'How does colour tell us about motion?' and 'What’s going on when we see objects in motion?' it strikes me (even more) that an entirely perceptual solution would be much simpler that the physical approach i'm taking. however, doing so would result in a solution totally to divergent to physical practice. But it would open up new avenues of visual (dance) aesthetic and enrich digital dance culture.

i'm not a fan of red wine, but after our Stormhoek tasting Sheila was keen to try the Stormhoek Shiraz. well i found it rather enjoyable, if a little strong (14% abv - seemed more); so it took a few days to finish the bottle. interestingly, the wine improved over the days it had been open, it nice to have a wine you can take you time over.

On university campuses, the newest technologists have to become renaissance geeks. They have to understand computing, but they also typically need deep knowledge of some other field, from biology to business, Wall Street to Hollywood. And they tend to focus less on the tools of technology than on how technology is used in the search for scientific breakthroughs, the development of new products and services, or the way work is done. [...] "There isn't the buzz and excitement about computer science that there should be," [Bill] Gates said. "We're on the threshold of extraordinary advances in computing that will affect not only the sciences but also how we work and our culture. We need to get the brightest people working on those opportunities."

change is good ... something fresh to clear the cobwebs. however unexpected the move from Article19 has been positive (for me at least). Marc has been kind enough to blogroll me again so if you are visiting via DA ... NCE greetings.

so to commemorate the old and celebrate the new i thought i would get another bottle of Stormhoek Sauvignon Blanc to compliment my freebie. However it was not to be and i ended up with the Piont Grigio.

However, when i got home a flicked through the booklet that came from Hugh i realised that i had an 'old' bottle of Piont Grigio and not the relaunched range.

The Sauvignon Blanc was stunning and went down well, very well it's a shame that Sainsburys had sold out. and no one else was selling it (they did have the Chiraz). The Piont Grigio was much less exciting .. not a great smell or taste. i can only hope that the modification they have made to the new line bring it closer to the quality of the Sauvignon Blanc.

well i fully intend to work my way through the Stormhoek line, with friends and family so thanks Hugh (and orbital). i was kinda bugged that A19 took my blog away from with with zero notice esp as i had signed up to the wine offer on the basis of that blog . i felt like i should almost send the bottle back, but i'm glad i didn't. The url on the label (http://www.article19.co.uk/mathew/index.php) will always remind me of the blog that really got my work going. so regardless of how it ended ... thanks article19, it was fun.

So what can you expect from this new blog, well more fresh thinking but with a little more freedom than before. that means more divergent projects, stay tuned (!).

returning to UEA i have lots of work to do. i see a dark winter ahead writing code so i'm setting myself some divergent goals. there is also the matter of a few lectures that i need to prepare a deliver ... time is going to be short.

i should get back to work ...

a few notes on hyperchoreography whilst i write a paper on the subject.

hyperchoreography is NOT:

• dance 'hypertext'
• interactive
• 'open'
• non-linear
• collaborative

hyperchoreography is:

• linear
• reactive
• hypermedia
• intertextual
• generative

i do not consider hyperchoreography (in it's present state) is a 'new' mode of choreographic practice, but a relocation of existing constructs. the use of hypermedia does not make it novel or sufficiently different to warrant a new category.

however, the notion of hyperchoreography is valid, and properly contextualised and developed along the 'true' concept of hypertext will present a radically new way of exploring choreographic practice.

Wow! i just found out (via boingboing) that John Adams and Peter Sellars have been working on an opera called Dr. Atomic. After reading up on the project (@ NewMusicBox) and listening to some midi samples i can't wait to see this. The world premiere is in San Francisco on October 1st 2005. Whilst i doubt i'll get to the USA that soon i hope to catch it at the De Nederlandse Opera (probably 2006). i'm sure the cd (when released) will be high on my play list, can't wait.

Neat, my bottle of Stormhoek has arrived. i've really been looking forward to this so i'm going to get another bottle (or two) from the supermarket, cook up some great food, and company ... then make an evening of it. i'm trusting Hugh and the other freebie wine bloggers that the wine is going to be good great but i'll let you know how it goes.
if you want to to be part of the 'wine blogging marketing disruption' you can still sign up (UK and Eire only)

Gough, M. (2005). Splines in space; quantifying esoteric dance. Proceedings of Immeasurable? The Dance in Dance Science, Laban 23rd July 2005. (preprint)

Splines in space; quantifying esoteric dance

Abstract

Existing movement notations for digital dancers (avatars) are specified according to visual effect (geometry). This external approach to the description of movement has poor motive fidelity that prevents ‘expressive’ interpretation of movement scores. The limited realism of exoteric notations reinforces the belief that expressive movement cannot be defined quantitatively.

Adopting a generalist perspective we have examined the interface between science and the arts to establish quantitatively qualitative (Qq) principles of expressive movement. The Qq principles were then used to create Æma (Affective – Esoteric, Motive Attribution), a computer based movement notation (Because it is not the purpose of this paper is to detail Æma the Qq principles are expressed implicitly rather than explicitly).

We propose to use combination of Functional Magnetic Resonance Imaging (fMRI), Electromyography (EMG), Motion Capture (Mocap) and experiential observation to generate synthetic motor images (Æma transcriptions). In order to evaluate Æma we will transcribe and simulate movement derived from Steve Paxton’s ‘Material for the spine’. Our intention is to demonstrate that even ‘internal’ dance practices can be replicated through quantitative notation.

Identify

Human movement is the emergent product of a Complex Adaptive System. Whether the body is operating as a ‘coordinated interrelated’ whole (Bernstein 1935), or an ‘uncoordinated interrelated’ whole (Gough 2005) the resultant movement will exhibit weighted distribution. This ‘heavy tail’ distribution is due to skeletomuscular function and reveals the unsuitability of Brownian motion as a metaphor for improvised movement. The Brownian model is best replaced with the Lévy flight whose scale invariant properties and heavy tail distribution are more representative of human motive probability.

We consider movement to be active (still) or motive (moving) rejecting the concept of motion as a product of two static positions. In order to establish a position we must make a transition, and in order to make a transition we must have formed a position, the two are interdependent. Furthermore, we argue that both transitions and positions are context dependant; repeated actions are self similar, but not exact replications.

The process of movement cognition and production can be simplified into three separate stages: Impulse, Intention and action. Movement impulse is an unconscious motive concept or stimulus. Communication of external movement impulses is impossible, what is commonly referred to as movement impulse is the initial vector of a movement action. Motive intention is a subconscious idea (representation) of how to realise the motive impulse; 'The concept and idea are different. The former implies a general direction while the latter is the component. Ideas implement the concept.' (LeWitt, 1993). Motive action is the motive effect of movement intention (motive affect). A conscious, adaptive process motive action differs from the intention for a variety of environmental, physiological or psychological reasons.

All human movement is without embedded meaning, other than that imposed by the spectator or performer. Because the range of potential movement is so large (limited only by the constraints of the human body), movement techniques reduce the number of possible solutions to provide a usable classification system.

There are no fixed points in the motive space, the ‘centre’ may located and relocated at the will of the performer. 

Measure

Computer science ontology for notating and animating human movement bears little relation to the experiential and scientific knowledge found in dance practice. By drawing parallels between the synthetic and somatic approaches we were able to re-define the computer science approach to measuring human movement.   

Human movement can be expressed as a series of points in space without a fixed, single frame of reference. Whilst ‘points in space’ are useful for navigating the kinespheric space they do not address the transitions between points. Laban’s vector symbols offer an alternative model of movement conception by replacing ‘points in space’ with discrete movement vectors (Longstaff, 2001).

Although movement vectors are an obvious candidate for computer implementation, their linearity limits the range of motive transitions available.  Kandinsky's Tanzkurven (dance curves) span the conceptual gap between ‘points in space’ and vectors through the use of figurative and symbolic representations to show the dancer and their movement trace (Kandinsky, 1926).

We have combined points, vectors and tanzkurven to express effort, shape and space with a single construct: the spline (a curve defined piecewise by numerical functions). Through the use of ‘splines in space’ we can measure and transcribe discrete movement quantitatively whilst retaining it’s qualitative features. Affective–Esoteric, Motive Attribution (Æma) uses multiple splines in a time centric structure to specify the position, orientation and movement curve for discrete bones (Muscle contractions and motive solutions are detailed where required).

Sets of splines for a movement are categorised into two classes: clusters and constellations, and sub classes (open and globular). Gross or global movement are defined as clusters and must use the sacrum as the significant segment (node), fine or modular movements are defined as constellations and may use any segment as the node. Open sub classes are visually bound descriptions (e.g. right thumb left thumb) whilst globular subclasses are physically bound (e.g. right thumb, right palm, right wrist).

Physics (dynamics) and effort-shape profiles can be inferred from skeletomuscular mass and the movement transcription.

Æma transcriptions are maps, not traces: 'The map is open and connectable in all its dimensions; it is detachable, reversible, susceptible to constant modification [...] A map has multiple entryways, as opposed to the tracing, which always comes back to the same' (Deleuze and Guattari, 1987, pp.3-25). Æma transcriptions facilitates topological and topographic descriptions without adaptation; partial navigation and map insertion are also possible.

However, Æma only provides a structure for measuring movement; it does not define the bounds for movement segmentation.

Segment

Merleau-Ponty’s statement that ‘Movement is a displacement or change of position, even if it cannot be defined as such.’ (Merleau-Ponty, 1942/1962 p.273) attests to the indeterminacy of movement segmentation. Somatic–cognitive integration and a state of ‘constant motion’ create a level of observational uncertainty when defining movement boundaries. In experiential observation, uncertainty is due to the act of observation altering the ‘state’ of the movement. An external observer’s accuracy is equally limited as they can only perceive the features of a continuous movement stream rather than discrete impulses.

Whilst it is possible to automatically segment movement by statistical means, such decomposition of human motion is arbitrary. Whilst automatic phrase structure detection has been demonstrated (Dyaberi Sundaram, James, and Qian, 2004) their theoretical perspective is questionable. Movement segmentation systems trained to recognise motive features of specific domain are not performing gestalt recognition but probabilistic assessment of motive minima. With many movement domains (e.g. Ballet, T'ai Chi Ch'uan, Figure skating) being pluricentric, the difficulty of achieving accurate movement segmentation increases. Variation between the technical and applied forms adds additional layers of complexity.

Recent research on the mechanisms of movement cognition and production lends support to the concept of motor imagery (Calvo-Merino, Glaser, Grezes, Passingham and Haggard 2004). Motor images are neural encodings and representations of movement, they are integral to motor planning, action, perception and recognition. As a model of movement intention (Jeannerod, 1994), the motor image forms the basis of our segmentation approach.

We propse to use a combination of Functional Magnetic Resonance Imaging (fMRI) and Electromyography (EMG) to extrapolate and measure the durational and muscular features of motor images. Establishing the initial features of a motor model (timing, muscles, bones) helps to minimise the uncertainty of movement segmentation. Discrete measurements of movement geometry can be performed using a ground truth of cognitive-somatic intention. Validation of the movement geometry (splines) is achieved though the use of experiential observation, domain specific knowledge and motion capture.

Synthesize

Using movement derived from Steve Paxton’s ‘Material for the spine’ we will create a synthetic movement profile for a digital dancer. Unlike existing avatar notations Æma transcriptions are avatar specific and designed to be adapted rather than performed ‘as is’. The process of adaptation is emergent (each ‘performance’ will be unique) and requires the conversion of qualitative motive solutions into quantitative solutions with divergence parameters.

To construct the divergence parameters we measure the range and percentage of motive adaptation that occurs between the motive intention (affect) and the motive action (effect). The divergence parameters reflect the discrepancy between practical theory and application: ‘Arabesque will always remain primarily a prescription, an ideal. There is a good arabesque and a bad arabesque and a phenomenal arabesque, but arabesque is about passing through. It's more about time than it is about position.’ (Forsythe, 1998).

Because Æma transcriptions are interpreted by deterministic, context sensitive processing, biofidelic movement is dependant on the use of appropriate quantitative movement solutions. By drawing parallels between existing computational solutions and qualitative, somatic descriptions we can ensure appropriate usage;

• Inverse kinematics = distal movement
• Forward kinematics = proximal movement
• Inverse dynamics = release technique (time driven)
• Forward Dynamics = release technique (momentum driven)
• Kinetics = body centring

It is important to recognise that movement solutions are functions, not descriptions. Movement cannot be specified by function alone, thus motive solutions require the constraints of a pose or gesture for usefull application.

Exclude

We reject the notion that emotion should be measured and applied to the simulation and synthesis of movement. Most attempts to simulate emotion rely on subjective interpretation of visual cues rather than establishing a ground truth. It is our belief that emotion is too deeply encoded within the movement process to be extracted. We are not suggesting that emotion could not be measured or standardised, but that when creating discrete movement transcriptions such data is redundant. Detailed, affective moment transcriptions are the key to expressive simulation.

Visualise

Defining the ‘measurable’ of expressive movement is not enough, the dance in dance science must be visualised as well as quantified. We need to develop tools that go beyond avatar representations and let us detect ‘deep’ patterns, explore and reason about those patterns. The visualisation of quantitative movement data is integral to dance research, teaching, performance and medicine. Yet, effective and interactive tools for dance visualisation will only emerge through the cooperation of science and the arts; the qualitative and the quantitative are one and the same. 

Acknowledgements

This work has been facilitated through the kind assistance of Gill Clarke, Steve Paxton and Independent Dance.

References

Bernstein, N. (1935). The problem of the interrelation between coordination and Localization. Archives of Biological Sciences Vol. 38, pp1–34.

Calvo-Merino, B., Glaser, D.E., Grezes, J., Passingham, R.E., and Haggard, P. (2004). Action observation and acquired motor skills: An fMRI study with expert dancers. Cerebral Cortex (pre print)

Deleuze, G., and Guattari, F. (1987). A Thousand Plateaus: Capitalism and Schizophrenia. (Trans. Massumi, B.) Minneapolis: University of Minnesota Press

Dyaberi, V. M., Sundaram, H., James, J. and Qian, G. (2004) Phrase Structure Detection in Dance. In Proceedings of the 12th annual ACM international conference on Multimedia, October 10 -16 (pp. 332 -335). New York: ACM

Forsythe, W. (1998). Interview with William Forsythe [Interview by Kaiser, P.] (Retrieved June, 20, 2005 from http://www.kaiserworks.com/ideas/forsytheframe.htm)

Gough, M. (2005). Towards Computer Generated Choreography: Epikinetic Composition. In Proceedings of the Hothaus seminar series. Birmingham: Vivid. (pre print)

Jeannerod, M. (1994). The representing brain: neural correlates of motor intention and imagery. Behavioural and Brain Sciences, Vol 17, pp 187-202

Kandinsky, W. (1926). Tanzkurven. Zu den Taenzen (Tanzen) der Palucca. Das Kunstblatt. Vol. 10, No 3, p117

LeWitt, S. (1993). Sentences on Conceptual Art. In Wood, P. and Harrison, C (Eds). Art in Theory 1900-1990: An Anthology of Changing Ideas (pp. 837-839). London: Blackwell.

Longstaff, J. S. (2001). Translating ‘vector symbols’ from Laban’s (1926) Choreographie. In Proceedings of the twenty-second biennial conference of the International Council of Kinetography Laban (ICKL), 26 July - 2 August (pp. 70-86). Ohio State University, Columbus, Ohio. USA: ICKL

Merleau-Ponty, M. (1962). Phenomenology of Perception (Trans. Smith, C.). London: Routledge (Original publication: Phénoménologie de la perception. Paris: 1945)

welcome to 'splines in space' a weblog about dance and technology, contemporary dance and theorising through practice.

these are the musings of Matthew Gough, a dance artist and researcher based in the U.K. and currently a MPhil/PhD Student at the School of Computing Sciences, University of East Anglia, Norwich.

you can contact me at: mpg @ cmp.uea.co.uk

this blog was predated by a guest blog at Article19 which can found at http://www.article19.co.uk/mathew/index.php