Sonic Art & Sound Design- P21 pdf

QC Preflight Point 1 st 33 p100 Job no : 76098 Title : The Fundamentals Of Sonic Art Client : AVA Scn : # 150 Size : 200(w)230(h)mm Co : M3 C0 (All To Spot)(Coagl) Dept : DTP D/O : 16.02.07 (Job no:000000 D/O : 00.00.07 Co: CM0) 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 100 1 st p100 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 100 1 st p100 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 100 PROCESS AND PRACTICE Introduction Interactivity is a feature common to many works of sound art. Interactive works are not uncommon in other art forms and much of what follows might be equally applicable to, for example, visual media. However, the proportion of sound work that depends upon the interaction between the artist and the system used is so substantial that interactivity has become almost a default mode for much of sonic art practice. This is not least due to the nature of the software often used to generate and transform sounds. Much of this has the potential for high degrees of ‘customisation’ and is therefore useful in the pursuit of innovative sounds and ideas. In turn, this implies the greater possibility of more fully interactive operation – the temptation is almost irresistible! Types of interactive works Interactivity may exist at a wide range of levels, from the more-or-less unnoticeable to a full-on format in which the input of the user or participant is central to the unfolding of the work. Most commonly, interactivity involves the use of computers and software that is custom-written but, once again, this is by no means inevitably so: sophisticated interactions are quite possible without the intervention of complex, software-dependent technologies. The images opposite show two very different installations, taken from an undergraduate degree show.They are both interactive to some extent – one using very simple audio technology and the other employing advanced and highly sophisticated custom software. The first image of Nathaniel Mann’s piece, Foley Works – The Sound of Snow Underfoot (2006), shows an installation work based upon the film soundtrack process known as ‘Foley’.This involves the creation of background sounds (such as, amongst many other things, footsteps in snow). In Mann’s work, buried microphones pick up the sounds made by the user and simply relay them back via headphones. The user hears these sounds and modifies their actions accordingly. Although essentially technically unsophisticated, the user is enabled to engage with the work on several levels and we see that what at first appears to be quite a simple work, invites users to involve themselves in thoughts and actions covering a wide scope. The second example (by Dani Joss) uses a video camera to detect the movements of the user.This information is then used to control the generation and playback of sound into the exhibition space. By implication, the user then responds to what he/she hears and, as in the first example, modifies his/her actions. 1 st p100 Interactivity 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 100 3 QC Preflight Point 1 st 33 p101 Job no : 76098 Title : The Fundamentals Of Sonic Art Client : AVA Scn : # 150 Size : 200(w)230(h)mm Co : M3 C0 (All To Spot)(Coagl) Dept : DTP D/O : 16.02.07 (Job no:000000 D/O : 00.00.07 Co: CM0) 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 101 1 st p101 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 101 1 st p101 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 101 101100 INTERACTIVITY 1 st p101 Left: ‘Foley Works – The Sound of Snow Underfoot’ An interactive sound installation by Nathaniel Mann in which visitors are invited to recreate the sound of footsteps in snow using the techniques adopted by Foley artists when making film soundtracks. At a deeper level, the work recalls the occasion during the First World War when British and German soldiers left their trenches to celebrate Christmas together in the snow. Image courtesy of Maria Militsi. Left: ‘Stasis/Kinesis’ by Dani Joss Joss describes this work thus: ‘An installation that can track and analyse the motion of a viewer standing on a clearly marked “sweet spot”, and use this information to influence and modulate musical composition. It features a custom sound diffusion system and video projection. As its name suggests, the central theme of this work is the stillness/motion dualism.This has been contrasted with repetition/variation and excitation/equilibrium, all abstract concepts in their own right. The installation is an attempt to move away from the culture of objects and symbolism and to revisit the aesthetic principles of abstract expressionism and the musical thought of the likes of M. Feldman through computer technology and a viewpoint inspired from the early experimentalists.’ Image courtesy of Dani Joss. 76098_CTP_072-107.qxd 3/24/07 6:12 AM Page 101 p102 QC Preflight Point 3 rd 33 Job no : 76098 Title : The Fundamentals Of Sonic Art Client : AVA Scn : # 150 Size : 200(w)230(h)mm Co : M3 C0 (All To Spot)(Coagl) Dept : DTP D/O : 16.02.07 (Job no:76098C2 D/O : 22.03.07 Co: CM3) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 102 p102 3 rd (Job no:76098C2 D/O : 22.03.07 Co: CM3) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 102 p102 3 rd (Job no:76098C2 D/O : 22.03.07 Co: CM3) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 102 PROCESS AND PRACTICE Concepts and definitions So what do we actually mean by Minteractivity? Chris Crawford has an excellent definition: Interaction: An iterative process of listening, thinking and speaking between two or more actors. 13 Clearly, Crawford uses the term ‘actors’ in a far broader sense than normal since, by implication, he includes non-human (indeed non-biological) systems in his definition. He breaks the process down into three components: ‘input’ (listening), ‘output’ (speaking) and ‘processing’ (thinking). Arguably, we may wish to add another component to this list – feedback – although, in a sense, this is already implicit in his definition. Critically, however, he points out that the process is iterative. If we move the mouse attached to our computer, we see the cursor move on the screen. We note its position and move the mouse in accordance with what we see, observe again and continue this process until the cursor comes to rest over the word that we seek. That’s a very simple level of interaction involving an iterative process and a dialogue between our computer and us. Importantly, it relies very heavily upon the points of contact between us: the mouse and the on-screen display. This is the interface, one of the most significant parts of any interactive system. p102 M INTERACTIVITY 3 rd (Job no:76098C2 D/O : 22.03.07 Co: CM3) ‘THE PERCEPTION OF SOUND DOES NOT JUST INVOLVE THE ACT OF HEARING, BUT IS IN FACT THE PROCESS OF LISTENING. THE LISTENING SYSTEM INCLUDES TWO EARS TOGETHER WITH THE MUSCLES FOR ORIENTING THEM TO A SOURCE OF SOUND.’ JANEK SCHAEFER, ‘AUDIO & IMAGE’ Interactive systems respond to those who encounter them. The forms of interactions may be many and varied but, most typically, the system will respond in some way to the presence or gesture (in the broadest sense of the word) of the onlooker. The viewer becomes a participant and to a certain extent influences the outcome of the work. Some theoreticians argue that all art is interactive to some degree but, in the sense that we use the term here, interaction takes place to a far greater extent than could possibly be the case if the work were to be, for example, a painting or a static sculpture. Interactive systems are often (but not inevitably) technically sophisticated and may use complicated technologies such as motion tracking – using video cameras and advanced software – or custom programmes, often created using systems such as MAX/MSP. 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 102 3 QC Preflight Point 1 st 33 p103 Job no : 76098 Title : The Fundamentals Of Sonic Art Client : AVA Scn : # 150 Size : 200(w)230(h)mm Co : M3 C0 (All To Spot)(Coagl) Dept : DTP D/O : 16.02.07 (Job no:000000 D/O : 00.00.07 Co: CM0) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 103 1 st p103 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 103 1 st p103 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 103 103102 Interactive systems in practice The interface represents and shapes our knowledge of the whole system and is arguably the single most important aspect of interactivity. It has to take information from the user, convey it to the system and return the system’s response back to the user. Clearly, the success of these processes is key to the success of the whole work. In order to ensure that success, one of the most important considerations is how one set of information is mapped to another. Mapping, however, is by no means the whole picture. Allied to it is the need to consider carefully what we can and cannot constructively use to provide input to our system. One fairly obvious idea is to use the electrical activity of the brain to control our system and this has been used in biofeedback trainers that detect certain brain rhythms associated with relaxation and emit a sound to reinforce them. Similarly, systems exist for people with special needs that allow them to ‘think’ an on-screen cursor to a particular position but these rely on one single piece of information – the presence or absence of a particular waveform in the brain. We cannot then modulate our own brain waves to, say, increase the volume or lower the pitch of a sound. Nonetheless, in 1965, Alvin Lucier’s work Music for Solo Performer used his brain waves as a basic sound source, which, when amplified, caused percussion instruments placed nearby to resonate. Arguably, Lucier would have heard this and his brain activity would have been influenced as a result.The interactivity of this ‘system’ is, however, quite limited – it was not as if Lucier could mentally opt to ‘play’ one or other instrument – and so we should make a distinction between works that are truly ‘interactive’ as opposed to those that are really simply ‘reactive’. How can we establish what will work and what will not? A primary requirement is a source of good, clear information. If we use a video camera to track movement, we need a good, well-lit image of an object, possibly of a distinctive colour, with clearly defined edges that contrast well with the background. With this quality of data, our system has an excellent chance of identifying the object, following it and being able to apply controls to the system in response to movements before the camera. The same principles apply to any control input: the information must be as clear as possible. So, good examples might be sound/silence, dark/light, moving/static and so on. More difficult examples might be volume of sound, intensity of light and speed of movement. In other words, a binary input (off/on) is simpler and more reliable than one that varies but, clearly, it has less potential for subtlety of interaction. INTERACTIVITY 13. Quoted in O’Sullivan, D. & Igoe,T. (2004) Physical Computing . Boston MA: Thomson Course Technology PTR. 1 st p103 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 103 p104 QC Preflight Point 3 rd 33 Job no : 76098 Title : The Fundamentals Of Sonic Art Client : AVA Scn : # 150 Size : 200(w)230(h)mm Co : M3 C0 (All To Spot)(Coagl) Dept : DTP D/O : 16.02.07 (Job no:76098C2 D/O : 22.03.07 Co: CM3) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 104 p104 3 rd (Job no:76098C2 D/O : 22.03.07 Co: CM3) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 104 p104 3 rd (Job no:76098C2 D/O : 22.03.07 Co: CM3) 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 104 PROCESS AND PRACTICE p104 Interfaces and physical computing 14 Once we have an idea of what form our input may take, we can consider what physical interfaces are required. Experience suggests that the design of any interface has a substantial influence on what form the dialogue with the system will take. For example, a musical keyboard will allow the user to interact with the system. However, it comes with significant cultural baggage. Using such a keyboard immediately references the Western European musical scale and so tends to impel the user’s interactions in a particular direction that relates to the intervals between notes, the structure of chords, key signatures and the like. In actuality, the keyboard is merely a collection of switches (so the note middle C could be mapped, not to a musical note but to ‘blue’ or ‘up’ or ‘loud’ – anything in fact) but the physical presentation pushes us into a quite specific mindset that we may or may not want. We need to carefully consider what form of input we want to offer the user.Take, for example, the idea of gesture as a controlling input. We could use the mouse for this or perhaps we could be a little less traditional and use a graphics tablet or a games controller.These all have the advantage that they plug into a USB port 3 rd (Job no:76098C2 D/O : 22.03.07 Co: CM3) Above: EyeCon This system uses inexpensive webcams to provide video input. The software can then track the movement of objects by their edges, colours or other criteria defined by the user.This information can then be used as a controlling input to other systems. Left: Stamp microcontroller Essentially a small computer on a single chip, the Stamp and others like it can accept inputs from almost any source(s) and translate them into data that can be accepted by computers. Alternatively, once programmed, they are able to control some systems entirely by themselves. Image courtesy of Parallax Inc., <www.parallax.com>. 76098_CTP_072-107.qxd 3/24/07 6:13 AM Page 104 . works of sound art. Interactive works are not uncommon in other art forms and much of what follows might be equally applicable to, for example, visual media. However, the proportion of sound work. The Sound of Snow Underfoot’ An interactive sound installation by Nathaniel Mann in which visitors are invited to recreate the sound of footsteps in snow using the techniques adopted by Foley artists. Mann’s piece, Foley Works – The Sound of Snow Underfoot (2006), shows an installation work based upon the film soundtrack process known as ‘Foley’.This involves the creation of background sounds (such as, amongst