Feedback on musical creation in synergy with visual artists: reflections, strategies, achievements, lessons learned.
- Introduction: Hearing while watching
- The experience of analogy
- The constructive experience
- The exhibition in music, and after the exhibition...
- What about the video?
AUTHOR: Charles-Edouard Platel, composer of electroacoustic music and member of th Helium artistic association. By clicking on the Art tab of the catalogue, the works that appear are the result of projects combining artistic and musical practices.
ABSTRACT: The natural approach to combining sound and visual can consist in linking musical forms to visual forms and colours and vice versa, by soliciting synesthesias between vision and hearing. After having used processes in this logic, but formal in nature, the idea was rather to build between visual artists and musicians a conscious approach to compose together their personal concepts. As a result, the visitor/listener testifies to a more eloquent experience of the association between musical and visual art.
Hearing music while watching a work of art can influence the way we see it. Symmetrically, the presence of plastic works can also influence the way music is heard. By combining music and artwork, the interest of the spectator / listener, his emotion, can be magnified. But interference from one to the other can also occur, causing disinterest, or worse, irritation. The coupling is delicate to succeed.
The question does not arise in the case of opera, dance, theatre, cinema and musical comedy, which, by definition, are multimedia arts, associating sound and visuals in a complementary way, i. e. in a way that is essential to achieve a whole.
But it can concern the musician or composer:"What architectural setting, what plastic work, what images, what spectacle would make my music feel more fully? ». And for the painter, sculptor, plastic artist, in symmetrical terms:"what music could accompany the public closer and more intensely in contemplating my work?" . Nevertheless, daring to diffuse music in exhibition spaces seems a priori an incongruous practice: a soft atmosphere is required, out of respect for the individual contemplation of each visitor.
For an electroacoustic concert and especially for acousmatic music, the proposal is paradoxical: an acousmatic listening is conceived by definition with closed eyes or in the darkness, concentrated only on sound. However, providing a visual focus during listening naturally influences this concentration of the listener.
On the other hand, acousmatic music offers the visitor to an exhibition a sound reduced to sound for itself, without showing the natural, human or instrumental origin of its production. Thus the contemplation of the artistic work is not disturbed by the sensitivity to another visual spectacle, real or imagined.
I had to try. I experimented by composing music from works by artists, with their participation in the adventure.
In writing chapter 5, "Morphology of sound objects", of my essay "Imaginary music", I suggested that the words we use to characterize what we see could also be used to characterize what we hear. To summarize: we can describe the visible by referring to four main types of shapes: continuous, raster, random, fractal. This chapter provides a series of visual and sound examples in analogy with these.
These shapes can be combined, in a simple way, up to more intricate or original complex figures. We can spot such figures in both visual space and sound space/time. For the eyes as well as for the listening, the attention retains the original rather than the banal, to focus on the interesting and possibly the beautiful.
Thus, by exploiting such analogies, we have an approach to develop interesting sound forms from an interesting image or vice versa.
2.1 From the picture to the sound
Our traditional musical visual is the classical score: signs quantified according to a two-dimensional screening: tonal pitch and metronome duration. The score, written by the composer for the performers of his work, guides them in order to make a potentially sublime music heard. But, with possible exceptions, a score is not written for the beauty of its paper pages..
However, this may give rise to ideas. In particular to the composer, mathematician and architect Iannis Xenakis, designer of the UPIC machine. From shapes drawn on a graphic table, a computer generates a musical composition that it can play. The abscissa axis represents time and the ordinate axis represents acoustic frequencies. (See also here a fun web application).
For example, the "Metasynth" (U&I Software) software , allows to associate to each pixel of a digital image a note of music generated by a synthesizer. The process is simple: paste the image in question into the Metasynth window, then explore this image from left to right. The pitch scale corresponds to the scale of the played notes, on a raster that corresponds to the scale and the desired mode, and the scanning speed in width can be adjusted according to the rhythm of the pixel by pixel notes and the desired duration for the sound sequence. In addition, the color of each pixel determines the intensity and stereo position of the output sound, with black being the absence of sound.
Like a child with a new toy, I tried the process, hoping for some unexpected discoveries. Yes, by chance, some surprising and interesting sequences came out of it, but not really related to my aesthetic feeling of the original image. Nevertheless, some passages could be selected for their own qualities, potentially for other projects.
To draw a more convincing image / music correlation from the process, the image can be worked out to make significant features emerge. First, identify a vertical, horizontal or oblique scanning direction. With experience, it is possible to identify which kinds of images are more likely to produce a drawing that would generate a plausible sound analogy. But it is not common to find images that lend themselves to it: the artist does not paint according to this deviated use.
Using this technique and retouching its results, I composed melodic lines that can be heard in Transmutations Acousmatiques (artist: Béatrice Carré) and in a few passages of Sables (artist: Jacques Suzat). By observing their paintings, the eye travels through remarkable lines which, thus underlined by their transposition into sound lines, gain in strength and persistence. The artists were sensitive to it, exploiting these soundtracks during their exhibitions and it was often the occasion for exciting exchanges with visitors telling their feelings of this surprising experience.
2.2 From the sound to the picture
Symmetrically, electroacoustic sound can generate its visual representations: waveform, spectrum and sonogram are diagrams familiar to the sound technician. The musical analyst, more interested in the composer's approach, will complete with representations created with the help of graphic tools, such as the Acousmographe.
Waveform and acoustic spectrum are images representing the acoustic content of a frozen instant of sound flow. In succession of images in real time, they compose a video animation that follows the music. Unless you are a good technician, the shape of a vibration or the graphics of its spectral bars are not very evocative of what you hear. Rather, the visual changes and their rhythm impress the spectator of moving images. Visual dynamics, along with sound dynamics, can enhance the listener's sensation. Sometimes too much is a matter of judgment..
The sonogram shows a global picture of the sound spectrum evolving over time. He draws up a map of the course of the work at a glance from start to finish. We could compare it to some kind of score. But while the score is prescriptive a priori, the sonagram is descriptive a posteriori. Its range of use is not limited to composed music, as it is first made to visualize sound recorded in the field. When it is used to analyse the natural environment for scientific and ecological purposes, the contemplation of the result can even suggest a beautiful aesthetic emotion, as for example, projected splendidly during Bernie Krause's exhibition « The big orchestra of the animals» (Cartier Foundation, 2016)) .
Also worth mentioning are the playlist playlist playback applications, often embellished with a display drawn and coloured by the music being played. Similarly, the lighting of shows or places for dancing can be controlled by algorithms that follow the music, possibly controlled live by a lightshow artist or DJ bringing his own touch. The idea is to reinforce the sound impact by a luminous impact, the archetype of the genre still remains the fireworks.
2.3 Limits of analogic approach
The automatic generation of visual forms derived from sound forms and vice versa seems exciting. But beyond the first technical enchantments, does it really add value? A priori this mechanistic approach, intrinsically formal, presupposes that the affect carried by the image spreads implicitly with the transposition of the image towards the sound or vice versa. I'm not convinced of that. For example, the conversion of a melancholic image into an analogue sound sequence does not necessarily produce melancholic music.
This approach finally takes us away from the plastic arts workshops. In the studio I use waveform, spectrum and sonogram to work on the composition, but this remains in the studio. Through the Max/MSP/Jitter modular development environment, it is possible to customize very original sound visualization tools, but it seemed more judicious to me not to dwell on technical exploits and rather to direct my research towards the human aspect of looking and listening.
The point of view changes in height: the association between image and music is no longer established by tangible correlations at the level of artifacts (the canvas, engraving, sculpture, photography, sound diffusion), but at the subjective level of the brain of the human being who sees and hears them.
This one recognizes shapes, textures, transparencies, colors. But more generally, he knows how to identify and memorize visual or sound entities. He is interested in them or not, memorizes them, recognizes them in an abstract way, conceptualising them in what I call, for the hearing,"sound beings". Linked to his memory, they take on meaning for each individual, become identifiable and subsequently re-identifiable.
This idea is presented in Chapter 8 "Sound Beings" in "Imaginary Music", then develops to speak of composition as a theatre where sound beings appear, disappear, return, co-exist while interacting, or vanish into the void of silence. At the time when I was writing these lines I didn't think I was working with artists yet, but when the opportunity arose, this thought was available at the right time to organize my approach and my interactions with them.
With the interested artists we organize our works in the form of projects, a project being determined by an objective at a given date: an artistic exhibition during which music is played continuously, associated with one or two spatial electroacoustic concerts and in visibility with the plastic works. The soundtrack of the exhibition can be a stereo version of the concert or a specific arrangement for looping at moderate sound level.
The launch of the project is delicate: naturally ensure that our artistic and musical tastes match, and then, more precisely, that my style or aspects of my style suit the artist. To do this, I send him some excerpts from my catalogue or ask him to visit my site which contains them. In return he records his appreciations and feelings, in short, what pleases him more particularly. At this stage we already have an idea of how to continue (or not) and we are considering how to organize ourselves.
Then I start collecting sounds: samples from my library, field recordings, electroacoustic synthesis or concrete music experiments. For example, in resonance with my perception of the artist's work, I use the Ondolon sound sculpture tool to improvise and record musical sequences.
By assembling these sound objects, I construct the first sketches of composition, where we can begin to identify with "sound beings" that I think in harmony with the meaning of the artistic works that inspire us.
The successive drafts are the subject of several working meetings to agree on. Sometimes the artist can seize the music that has not yet been finalized to draw inspiration from it in a plastic work that will be exhibited. The whole process takes several months, in cooperation with one or two artists, sometimes more.
Depending on the projects, the schedule of work meetings ranged from one to a dozen meetings. But what is the purpose of our conversations and what language are we talking about in these meetings?
At first I thought that the sound material of concrete music or synthesizer would be a difficulty for unfamiliar ears. In reality, the discussions addressed this aspect very little. Undoubtedly the artists have not feared adventure, feeling in any case committed to an unusual creation.
Our comments concern the construction of the musical composition, either by listening together several times to already composed parts, or directly in front of the screen of the arrangement software. Finally, the visual artist finds almost familiar patterns: the architecture of the different parts of the work, neighbourhoods and transitions between different objects, contrasts, superimpositions, empty spaces, recurring motifs, etc., including the perception of analogies between sound and image. We share a language that we master equally.
In short, the result of our joint work is that the visitor/listener will have a unique experience whose intensity develops both in space and time.
Often a unifying theme is found between artists and musician to give a title to the exhibition:« Coincidence » (with Rui Prazeres), « Vibration » (with Isabelle Garbil Fauve-Piot and Gomès), « Heritage » (with K-Rol and Catherine Seznec), « Migration » (with Srecko Boban), « Where do they go? » with Hélène Pouzoullic and Cécile Dachary), « The City » (with Cécile Dachary), « The Forest » (with Isabelle Garbil Fauve-Piot, Hervé Martin , Véronique Arnaud and Jean-Claude Bugny), « Sands» (with Jacques Suzat)….
The title provides a summary formulation to initiate cooperation. In a few cases we stuck to this simple statement to create each one on the theme, trusting each other, the appointment for final adjustments being done just before the installation. But in the majority of the cases we have cooperated throughout the construction of the musical work, starting from the emotional feeling of the plastic work, already ready or still under development.
The playing of the looped music begins as soon as the exhibition opens to visitors. It will not stop, except at particular moments, such as the opening, during the guests' hustle and bustle and official speeches. The sound level of the music is precisely set according to the acoustics of the room, to allow conversations between a few people without raising the voice.
At first we feared that the repetition, of the same music for whole days would become tiresome. In reality we have found that the musical accompaniment has an effect on the walk of visitors in the exhibition. They spend more time on each work or travel through the exhibition space several times, and then more naturally come to talk with the artist if he or she is present. Stopping the music breaks that charm.
If possible we program one or two spatialized electroacoustic concerts in multiphony as an event associated with the exhibition. It is this version which constitutes the work in reference in my catalogue and possibly in stereo version on CD album. The life of these musics continues after the initial exhibition: both in the program of electroacoustic concerts or covered by the artists in their subsequent exhibitions.
Combining sound and visuals is technically easy to master via digital media: sound and still or video images are processed via the computer using software tools that are very similar.
The fruit of 2 years of collaboration with Isabelle Fauve-Piot, Almost Nothing - Almost All is a musical and visual composition inspired by the contemplation of the monumental sculpture "Vibration". For this work created together, video animation and music were created in the same composition flow, according to a creation process significantly different from that of previous video works.
Charles E. Platel,
(Oct 2018, revised Oct 2019)
Rui Prazeres: "Coincidence": Installation à la maison Elsa Triolet-Aragon (Photo J.Marcault)