Network jamming

For the past six years a team of dedicated music educators, software developers, interaction designers and sociologists have been researching Network Jamming, the means to allow people to play music and do video remixing together over the internet in real time. Based in Brisbane at the Queensland University of Technology is the Australasian Cooperative Research Centre for Interaction Design (ACID). Led by ACID project leader Dr Steve Dillon, the Network Jamming team has developed cutting-edge technology called Jam2Jam; a suite of software applications that are generative and use computer processes to facilitate musical changes based on stylistic algorithms.

 According to Dr Dillon, the Network Jamming project explores how collaborative creativity using networked digital technologies can enhance learning and community. The project focuses particularly on the use of generative systems to increase access, participation and enjoyment for non-musical or novice users. ‘The project explores ways to enhance learning and community by designing interactive creative activities based on collaborative and generative digital technologies,’ he explains. ‘The project has developed new generative media software [Jam2Jam] that can be controlled over the internet and uses these to increase access to real-time creative interaction for novice users including children and people with disabilities.’

As a musician having spent much of his life on the road as a singer-songwriter, Dr Dillon came to academic research after teaching music in schools. With a strong interest in music making for children that is meaningful and engaging, and which leads to happiness, social connection and a sense of cultural belonging, he completed a PhD on the subject documented in his book Music Meaning and Transformation and the subsequent series Meaningful Music Making for Life. As director of Save To Disc (Documenting Innovation in Sound Communities – an independent research network that examines the role of music in communities) he is well placed to apply Jam2Jam and generative music making ideas to software development.

The Jam2Jam systems are networked computing set-ups connected to digital social networks that enable coordination, sharing and communication involved in collaborative creative activities. Embedded within the interface design featuring musical styles is well-defined musical knowledge. This knowledge is ‘encoded’ within the improvisational algorithm and a teacher or music coach can guide students towards this information through reflective practice.

‘Jammers are encouraged to be open to the random inputs/answers that the software produces and the visual dimension of the tool, that contributes to our musical toolbox,’ says Dr Dillon. ‘Depending on what kind of controllers that are used/connected to the software, the emphasis can shift between individual/collaborate, visual/aural, intuitive/analytic and brain/body.’

Jam2Jam involves a series of simple controllers – real or virtual sliders, or a computer game like environment – such that in individual’s movement effects a change in the music. Gestures with the interface facilitate changes in the density or complexity of musical activity, volume and timbre. ‘What is unique about it is that it is fundamentally collaborative,’ Dr Dillon says. ‘A small group of players can play in a virtual ensemble easily and without much musical knowledge or experience by networking their Jam2Jam systems. This enables groups of players to interact in real time like an ensemble and the music responds to individual changes in gesture. We call this Networked Improvisation.’

Jam2Jam and generative media differs in several ways from other music creation software products currently on the market. It is accessible, collaborative, based on performance and algorithm based at the note level. ‘First, it makes high quality music making safe, simple and accessible to inexperienced users,’ Dr Dillon elaborates. ‘Second, it is collaborative as it draws upon the ensemble qualities of group performance.’

For Dr Dillon, the current means of using computers and software for music creation is generally isolating and not in real time. Participants generally work by themselves or with only one person at the controls as they record and produce music. It can, therefore, be argued that other software is not a ‘performance’ tool in the essence of performing with others in real time. ‘Third, it is about performance and not composition or production as is the case with most other programs,’ he adds. ‘Jam2Jam is based on an improvisation or a DJ/VJ metaphor for performance.’

Another difference is that, although Jam2Jam can incorporate looped samples, it operates at the note level- meaning an algorithm that represents the possible notes within a style forms the base music materials. ‘This allows the greatest and least activity within a style to be able to be controlled for each instrument,’ says Dr Dillon. ‘It also allows for generative transformations to be used to affect expressive change with ease of access.’

Further, the process of development has been quite a different one: ‘We have simultaneously developed styles of teaching and activity strategies alongside software development. We have hundreds of hours of video of thousands of children, artists, teachers and community workers who have influenced the design based on our theories of meaningful engagement.’

Dr Dillon believes that the essence of the Network Jamming research is that it enables children aged from four years, people with disabilities, and youth and adults with limited musical skills to experience ensemble performance and meaningful creative activities. As reflection is built into the design, players can gain further musical knowledge through increased play and involvement. The initial development of the software began with a survey of the musical tastes of a group of children in a culturally diverse community in Delaware, Ohio as part of the Delaware Children’s Music Festival in 2002. The first version of Jam2Jam was written by associate professor Andrew Brown in Java with subsequent research and development supported by ACID. Since inception the project has tested iterations of the software on several thousand children and adults at music festivals, schools and arts installations around the world. The project has been supported by funding from ACID and has research and development partnerships with universities in six countries, as well as a growing Australian presence.

‘The majority of this research has been about improving the generative algorithms, applying networked jamming activities to arts education learning environments, the addition of visual media performance capabilities, and the development of digital social networks of Network Jamming users,’ Dr Dillon says. ‘Andrew [Sorensen] and I pioneered a user-led design process where we examined meaningful engagement and documented children’s responses. We incorporated these results into the design and science of teaching using a process called the Software Development as Research (SoDaR) method.’

The current model of Jam2Jam is built on ACID researcher Andrew Sorensen’s Impromptu platform – a Lisp-based programming environment. Impromptu is a free platform that is used for ‘live coding’ that uses the Apple Mac OS-X operating system. According to Dr Dillon, Jam2Jam places an accessible interface over the generative algorithms programmed in Impromptu to allow users to create music in real time and collaboratively through simple gestures. In the case of Jam2Jam this is presented as an X-Y pad for shifting instruments around to affect selected generative transformations to both sound and visual content. While Jam2Jam is the principle software development, the team is also developing versions for the web, for disabled access using controllers, for the ‘one laptop per child’ (OLPC) project going out to 100,000 remote communities in Australia, and also a mobile version for the iPhone.

Network Jamming is produced at the ACID laboratory/offices in the Impromptu programming environment on Core 2 Duo 2 Apple Macintosh computers (iMacs and MacBooks). They also use Behringer USB controllers and Korg Nano Series Controllers for installations and Nano bands. Four OLPC machines are used for development of the OLPC version, and an iPod Touch for testing iPhone versions. Field trials are then developed and delivered in various venues, schools and community centres throughout Europe, North America and Australasia.

According to Dr Dillon, the research is focused mainly in schools, as it requires an outcome that refers to curriculum knowledge and is measurable. The focus of the activity is always about how they engage users in meaningful collaborative creative activity. The clients range from those with severe disabilities, culturally diverse and Indigenous Australian communities, disengaged youth, mental health patients and children primarily aged between four and 16. They have observed and documented across all of those communities that the program provides ease of access, engagement and expressive activity, it encourages communication in wordless ways and provides a sense of pride in culture because the program allows user materials to be the basis of a jam.

‘Indigenous art works and musical style selected by young people provided a vehicle for expression with disengaged Indigenous youth,’ he says. ‘Young people unable to speak because of disability displayed great engagement and pleasure at the expressive and collaborative aspects of the system. In one example, a class in a primary school focusing on learning about bullying, composed and performed raps with Jam2Jam and VJ visual represen

One such testing site is the Humfryskolan (Humphrey Bogart) independent comprehensive school in Malmo, Sweden. Assistant secondary headmaster and co-owner, Per Sköld, started the school in 2008 with a strong profile on film, music and media. Having worked at a state school, Sköld got tired of how slow and conservative things were so set up his own school with two other partners. The school has 87 students across grades six to nine and has two rehearsal rooms each equipped with an electric bass, two electric guitars, digital drum sets, PA system and keyboards. They have Mac computers for students to use when they work with film editing, music recordings and art. Past students’ short films have won awards at the Creative festival in Berlin, and have been shown at the Stockholm film festival. Additionally, students have produced a TV show for the past three seasons called Genant. The show deals with sex education and is used as educational material for a lot of schools in Sweden. It has been broadcast in Malmo, Växjö and Berlin.

‘We can be more creative, progressive and take decisions quickly with all learning stimulated through aesthetic expressions,’ says Mr Sköld. ‘I have worked with Jam2Jam mostly with sixth graders (12-13 years old) and gone from a very teacher-centred approach, which didn’t work well with my students, to using Jam2Jam as a tool for exploring creativity within computer music.’

Mr Sköld believes that music is a great tool for teaching cooperation and democracy, fundamental principles of the school, along with tolerance, and understanding, and that attaining these attributes in its student population is more important than the music itself. ‘Jam2Jam allows anyone, even with no knowledge of playing an instrument, to jam with experienced musicians through Jam2Jam or with live-instruments,’ he says. ‘You can record your jam and reflect on your work afterwards. It’s also fun and looks easy, but can be quite challenging if you really want to get into it.’

Sydney-based MLC School Burwood is another test site for Jam2Jam. The school is a strong supporter of contemporary Australian art and music, and commissions many new works each year, particularly compositions written specifically for young voices. MLC runs a kindergarten to grade 12 music programme centred on singing and improvisation, and holds regular concerts at iconic venues such as the Sydney Opera House and the Sydney Town Hall, with every single student in the school performing at these events. In addition to this, the MLC Music department has won various awards for its projects over the past two decades and, this year, has been nominated as a finalist in the Classical Music Awards and the Australian Broadcasting Corporation’s Flame Awards. For the school’s composer in residence James Humberstone, (one of three on staff) Jam2Jam is a strong fit with MLC’s strong commitment to music and technology

‘MLC is an Apple Distinguished School, we have three in-house developers working on our own online learning delivery systems as well as an international cooperative learning network called Skoolaborate where students from different schools and countries complete units of work together via our website, via web cam meetings, and even in a special Second Life environment which we have built,’ says Mr Humberstone. ‘Therefore you can see anything new and innovative using technology with real teaching value is attractive to us at MLC.’

Having worked with Dr Dillon and Mr Brown on assorted projects, Mr Humberstone is a fan of their innovative research. MLC trialled the first version of Jam2Jam about four or five years ago and, according to Mr Humberstone, it was a hit with their elective students. He also enjoys the social element attained on Jam2Jam that isn’t usually present when kids are doing music on computers, the fact that no previous musical learning or literacy is required and the possibilities for further improvement in future releases. ‘At the moment Jam2Jam is kind of proof of concept and brilliant fun, but just scratching the surface of where this could go,’ he observes. ‘I expect future versions will be more customisable (import your own content) and will allow more detailed improvisation than the current parameters allow.’

The ACID Network Jamming project completes its funded research in April 2010. Following this, research will continue through the partnerships with Australian and International universities and transfer of developed prototypes will be made available to community interests such as libraries and galleries. Negotiations with commercial interests will take place to deliver the software development to industry with commercial and venture capital partnerships encouraged.

‘By April 2010 we will have robust commercial ready software and a series of working prototypes and models to demonstrate the further application of generative media systems,’ concludes Dr Dillon. ‘We at ACID are open to venture capital investment in these projects and commercialisation discussions.’

www.acid.net.au

Published in PAA May-June 2010