Feedback on a musical creation in synergy with visual artists: reflections, strategies, achievements, lessons learned.
- Introduction: Hearing while watching
- The experience of analogy: by chance of coincidences
- The constructive experience: a joint adventure
- The exhibition in music, and after the exhibition...
- What about the video?
AUTHOR: Charles-Edouard Platel, composer of electroacoustic music and member of th Helium artistic association.
ABSTRACT:
A first approach to associate music and visuals is to produce sound forms from visual forms and colours and vice versa, by soliciting synesthesias between vision and hearing. However, despite an undeniable conceptual or ludic interest, these processes remain essentially formal.
Associating music and visual art is not just a matter of associating sounds with images or vice versa.
The idea presented here would be to work separately on sound forms and colours and visual forms and colours, identifiable for their own sake, and then to assemble them in a joint composition. For example, visual artists and musicians could work together to create a conscious approach to compose together their personal concepts. As a result, the visitor/listener may find himself in a more sensitive human experience of the association between musical and visual art.
For example, the Art tab of the catalogue presents musical works produced with artists.
1. Introduction : Hearing while watching
Hearing music while watching a work of art can influence the way we see it. Symmetrically, the presence of artworks can also influence the way music is heard. By combining music and artwork, the interest of the spectator / listener, his emotion, can be magnified. But interference from one to the other can also occur, causing disinterest, or worse, irritation. Thus the coupling can become disastrous.
The question does not arise in the case of opera, dance, theatre, cinema and musical comedy, which, by definition, are multimedia arts, associating sound and visuals in a complementary way, i. e. in a way that is essential to achieve a whole.
But it can concern the musician or composer:"What architectural setting, what plastic work, what images, what spectacle would make my music feel more fully? ». And for the painter, sculptor, plastic artist, in symmetrical terms:"what music could accompany the public closer and more intensely in contemplating my work?" . Nevertheless, daring to diffuse music in exhibition spaces seems a priori an incongruous practice: a soft atmosphere is required, out of respect for the individual contemplation of each visitor.
For an electroacoustic concert and especially for acousmatic music, to offer to see in addition seems a paradoxical proposition: an acousmatic listening is conceived by definition with closed eyes or in the darkness, concentrated only on sound. However, providing a visual focus during listening may facilitate this concentration.
For the visitor of an exhibition, acousmatic music adds only sound for its own sake: its origin is invisible and undetectable. Thus, unlike music caused by musicians and their instruments, the contemplation of the artistic work is not competed by a second spectacle, real or imagined.
I had to try. I experimented by composing music from works by artists, with their participation in the adventure.
I first tried the analogical experiment: sound forms resembling visual forms and vice versa. But this approach, although technically and even intellectually fascinating, quickly showed its limitations. So I sought instead to build a process of resonance with artists through our mutual sensitivities to each other's work.
2. The experience of analogy: by chance of coincidences
In writing chapter 5, "Morphology of sound objects", of my essay "Imaginary music", I suggested that words we use to characterize what we see could also be used to characterize what we hear. To summarize this, we can describe the visible by referring to four main types of shapes: continuous, raster, random, fractal. This chapter provides a series of visual and sound examples in analogy with these.
These shapes can be combined, in a simple way, to create complex, more intricate or original shapes. We can identify such figures in both visual space and sound space/time and use a common vocabulary to describe them. It is then attractive to extend the analogy to develop interesting sound forms from an interesting image or vice versa.
2.1 From the picture to the sound
Our traditional musical visual is the classical score: signs quantified according to a two-dimensional screening: tonal pitch and metronome duration. The score, written by the composer for the performers of his work, guides them in order to make a potentially sublime music heard. But, with possible exceptions, a score is not written for the beauty of its paper pages..
However, this may give rise to ideas. In particular to the composer, mathematician and architect Iannis Xenakis, designer of the UPIC machine. From shapes drawn on a graphic table, a computer generates a musical composition that it can play. The abscissa axis represents time and the ordinate axis represents acoustic frequencies. (See also here a fun web application).
From there, rather than drawing on a machine, why not exploit an existing image, such as the picture of a work of art?
For example, the "Metasynth" (U&I Software) software , offers a function to associate to each pixel of a digital image a note of music generated by a synthesizer. The process is simple: paste the image in question into the Metasynth window, then explore this image from left to right. The pitch scale corresponds to the scale of the played notes, on a raster that corresponds to the scale and the desired mode, and the scanning speed in width can be adjusted according to the rhythm of the pixel by pixel notes and the desired duration for the sound sequence. In addition, the color of each pixel determines the intensity and stereo position of the output sound, with black being the absence of sound.
Like a child with a new toy, I tried the process, hoping to bring out some spontaneous inventions Yes, by chance, some surprising and interesting sequences came out of it, but not really related to my aesthetic feeling of the original image. Nevertheless, some passages could be selected for their own qualities, potentially for other projects.
To draw a more convincing image / music correlation from the process, the image can be digitally worked out to make significant features emerge. First, identify a vertical, horizontal or oblique scanning direction. With experience, it is possible to identify which kinds of images are more likely to produce a drawing that would generate a plausible sound analogy. But it is not common to find images that lend themselves to it: the artist does not paint according to this deviated use.
Using this technique and retouching its results, I composed melodic lines that can be heard in Transmutations Acousmatiques (artist: Béatrice Carré) and in a few passages of Sables (artist: Jacques Suzat). By observing their paintings, the eye travels through remarkable lines which, thus underlined by their transposition into sound lines, gain in strength and persistence. The artists were sensitive to it, exploiting these soundtracks during their exhibitions and it was often the occasion for exciting exchanges with visitors telling their feelings of this surprising experience.
2.2 From the sound to the picture
Symmetrically, electroacoustic sound can generate its visual representations: waveform, spectrum and sonogram are diagrams familiar to the sound technician. The musical analyst, more interested in the composer's approach, will complete with representations created with the help of graphic tools, such as the Acousmographe.
Waveform and acoustic spectrum are images representing the acoustic content of a frozen instant of sound flow. In succession of images in real time, they compose a video animation that follows the music. Unless you are a good technician, the shape of a vibration or the graphics of its spectral bars are not very evocative of what you hear. Rather, the visual changes and their rhythm impress the spectator of moving images. Visual dynamics, along with sound dynamics, can enhance the listener's sensation. Sometimes too much is a matter of judgment..
The sonogram shows a global picture of the sound spectrum evolving over time. He draws up a map of the course of the work at a glance from start to finish. We could compare it to some kind of score. But while the score is prescriptive a priori, the sonagram is descriptive a posteriori. Its range of use is not limited to composed music, as it is first made to visualize sound recorded in the field. When it is used to analyse the natural environment for scientific and ecological purposes, the contemplation of the result can even suggest a beautiful aesthetic emotion, as for example, projected splendidly during Bernie Krause's exhibition « The big orchestra of the animals» (Cartier Foundation, 2016)) .
Also worth mentioning are the playlist playback applications, often embellished with a display drawn and coloured by the music being played. Similarly, the lighting of shows or places for dancing can be controlled by algorithms that follow the music, possibly controlled live by a lightshow artist or DJ bringing his own touch. The idea is to reinforce the sound impact by a luminous impact, the archetype of the genre still remains the fireworks.
2.3 Limits of analogic approach
The automatic generation of visual forms derived from sound forms and vice versa seems exciting. But beyond the first technical enchantments, does it really add value? A priori this mechanistic approach, intrinsically formal, presupposes that the affect carried by the image spreads implicitly with the transposition of the image towards the sound or vice versa. I'm not convinced of that. For example, the conversion of a melancholic image into an analogue sound sequence does not necessarily produce melancholic music.
This approach finally takes us away from the plastic arts workshops. In the studio I use waveform, spectrum and sonogram to work on the composition, but this remains in the studio. Through the Max/MSP/Jitter modular development environment, it is possible to customize very original sound visualization tools, but it seemed more judicious to me not to dwell on technical exploits and rather to direct my research throught the human angle of looking and listening.
3. The constructive experience: a joint adventure
3.1 Concept
The point of view changes in height: the association between image and music is no longer established by tangible correlations at the level of artifacts (the canvas, engraving, sculpture, photography, sound diffusion), but at the subjective level of the brain of the human being who sees and hears them.
This one recognizes shapes, textures, transparencies, colors. But more generally, he knows how to identify and memorize visual or sound entities. He is interested in them or not, memorizes them, recognizes them in an abstract way, conceptualising them in what I call, for the hearing,"sound beings". Linked to his memory, they take on meaning for each individual, become identifiable and subsequently re-identifiable.
This idea is presented in Chapter 8 "Sound Beings" in "Imaginary Music", then develops to speak of composition as a theatre where sound beings appear, disappear, return, co-exist while interacting, or vanish into the void of silence. At the time when I was writing these lines I didn't think I was working with artists yet, but when the opportunity arose, this thought was available at the right time to organize my approach and my interactions with them.
3.2 Collaborative process
With the interested artists we organize our works in the form of projects, a project being determined by an objective at a given date: an artistic exhibition during which music is played continuously, associated with one or two spatial electroacoustic concerts and in visibility with the plastic works. The soundtrack of the exhibition can be a stereo version of the concert or a specific arrangement for looping at moderate sound level.
The start of the project is naturally to ensure that our artistic and musical tastes match, more precisely that my style or aspects of my style suit the artist. To do this, he or she can listen to excerpts of works from my catalogue. In return he records his appreciations and feelings, in short, what pleases him or her more particularly. At this stage we already have an idea of how to continue (or not) and we are considering how to organize ourselves.
Then I start collecting sounds: samples from my library, field recordings, electroacoustic synthesis or concrete music experiments. For example, in resonance with my perception of the artist's work, I use the Ondolon sound sculpture tool to improvise and record musical sequences.
By assembling these sound objects, I construct the first sketches of composition, where we can begin to identify with "sound beings" that I think in harmony with the meaning of the artistic works that inspire us.
The successive drafts are the subject of several working meetings to agree on. Sometimes the artist can seize the music that has not yet been finalized to draw inspiration from it in a plastic work that will be exhibited. The whole process takes several months, in cooperation with one or two artists, sometimes more.
3.3 Achievements and feedbacks
Depending on the projects, the schedule of work meetings ranged from one to a dozen meetings. But what is the purpose of our conversations and what language are we talking about in these meetings?
At first I thought that the sound material of concrete music or synthesizer would be a difficulty for unfamiliar ears. In reality, the discussions addressed this aspect very little. Undoubtedly the artists have not feared adventure, feeling in any case committed to an unusual creation.
Our comments concern the construction of the musical composition, either by listening together several times to already composed parts, or directly in front of the screen of the arrangement software. Finally, the visual artist finds almost familiar patterns: the architecture of the different parts of the work, neighbourhoods and transitions between different objects, contrasts, superimpositions, empty spaces, recurring motifs, etc., including the perception of analogies between sound and image. We share a language that we master equally.
In short, the result of our joint work is that the visitor/listener will have a unique experience whose intensity develops both in space and time.
Often a unifying theme is found between artists and musician to give a title to the exhibition:« Coincidence » (with Rui Prazeres), « Vibration » (with Isabelle Garbil Fauve-Piot and Gomès), « Heritage » (with K-Rol and Catherine Seznec), « Migration » (with Srecko Boban), « Where do they go? » with Hélène Pouzoullic and Cécile Dachary), « The City » (with Cécile Dachary), « The Forest » (with Isabelle Garbil Fauve-Piot, Hervé Martin , Véronique Arnaud and Jean-Claude Bugny), « Sands» (with Jacques Suzat), "Water" (with Bernard Sustrac)….
The title provides a summary formulation to initiate cooperation. In a few cases we stuck to this simple statement to create each one on the theme, trusting each other, the appointment for final adjustments being done just before the installation. But in the majority of the cases we have cooperated throughout the construction of the musical work, starting from the emotional feeling of the plastic work, already ready or still under development.
4. The exhibition in music, and after the exhibition...
The playing of the looped music begins as soon as the exhibition opens to visitors. It will not stop, except at particular moments, such as the opening, during the guests' hustle and bustle and official speeches. The sound level of the music is precisely set according to the acoustics of the room, to allow conversations between a few people without raising the voice.
At first we feared that the repetition, of the same music for whole days would become tiresome. In reality we have found that the musical accompaniment has an effect on the walk of visitors in the exhibition. They spend more time on each work or travel through the exhibition space several times, and then more naturally come to talk with the artist if he or she is present. Stopping the music breaks that charm.
If possible we program one or two spatialized electroacoustic concerts in multiphony as an event associated with the exhibition. It is this version which constitutes the work in reference in my catalogue and possibly in stereo version on CD album. The life of these musics continues after the initial exhibition: both in the program of electroacoustic concerts or covered by the artists in their subsequent exhibitions.
5. What about the video?
Combining sound and visuals is technically easy to master via digital media: sound and still or video images are processed via the computer using software tools that are very similar.
In this onfiguration, The Time for a Move, The Sea, and Between four eyes are videos putting in music slideshows created by the photographer Michel Di Maggio. Conversely, Badinage for four hands and Vestiges are personal photo montages synchronized to pre-existing musical compositions. Lastly, the three videos of Coïncidence and Barley & poppies accompany each of these meditative musics with a sequence shot through slow geometric, colorimetric and texture transformations on a still image.
The fruit of two years of collaboration with Isabelle Garbil Fauve-Piot, Almost Nothing - Almost All is a musical and visual composition inspired by the contemplation of the monumental sculpture "Vibration". For this work created together, video animation and music were created in the same composition flow, according to a creation process significantly different from that of previous video works.
Continuing in this direction, since hearing is receptive to acoustic vibrations and seeing to light vibrations, I have sought to project a composition of images and sounds as tangible sensations for the spectator/listener in the sound/visual space-time.. MAYDAY this the Earth, and The WIND illustrate this approach. In the way that musique concrète treats original sounds, pictures and films are transformed in order to create optical sensations beyond the content of the initial document. Then images and sounds, darkness and silences, can be composed together, one not being the accompaniment or illustration of the other. Gathered in a common orchestra, they offer the spectator/listener a multi-sensorial and multi-dimensional experience
Charles-E. Platel,
(Oct 2018, revised September 2023)
Rui Prazeres: "Coincidence": Installation à la maison Elsa Triolet-Aragon (Photo J.Marcault)
