Overview
Video Introduction (5:30)
The Jig-Sound project will be a new type of album: an online
jukebox of interactive Heaps of
music. A jig-sound is sort of like a jig-saw only with
sound. Heap is
a term I've chosen to describe a combinatorium of sounds (usually
loops) to be experienced via interactive sequencing and/or layering
the sounds by
the audience.
I am fascinated with the idea of Heap as musical form and with
creating software for an online audience to experience and play heaps. Normally
when we hear music, we
hear
a piece
from beginning to end. We don't tinker with it except in our head and possibly
we harmonize with it and drum the dashboard, fridge etc. The Jig-Sound project
is an attempt to both build some software and create some
music that
helps us develop the notion of the Heap as musical piece/form for an
online audience. As musical piece that we experience by exploring
the Heap's compositional/jig possibilities. Also, the software that frames
the Heap will allow the audience to 'play along' (via keyboard and mouse)
with
their own and others' compositions built from the Heap.
The challenge of creating heaps that possess strong
and diverse compositional possibilities is something I'm trying
to work on concerning the music. Concerning
the software, it has to be unusually good in the synchronization
it maintains of dynamically layered and sequenced rhythmic sounds,1 and
it has to be fun, provoking, and easy to use by the audience. I invite
you to check out both the music
I'm making
and play with the software to get a feel for the nature of it as
musical experience and meditative, compositional activity.
The Audio Software
Jig-Sound will provide people with a new way of experiencing
music online and on things like CDs and DVDs. There's a 'game mode'
where you are timed on how fast you can put a song back together again,
and your
score is
uploaded to the Internet. But there's also a 'play mode' where the
goal is simply to sequence and layer loops of music in ways that
you find interesting2—a bit like playing with sonic building blocks.
And you can share your jigs/constructs with others either via uploading
them or emailing them to friends.3 You can also save your jigs locally
for later recall.
Jig-Sound is meant for a general audience from seven-year-olds
to adults. It isn't only for musicians. It's a way to experience music,
not primarily
a way to compose music. Jig-Sound takes a kind of
cubistic approach to the experience of music. Cubism offers multiple
simultaneous
perspectives.
Jig-Sound gives you insight into the loop-based structure
of music and encourages you to think 'what if' compositionally. 'What
if I just
move this part around and change the order here?' Jig-Sound also
is a kind of instrument itself. You'll be able to play sounds using
the mouse and
keyboard, not simply put them together like building blocks and then
play the structure you make.
Jig-Sound's audio programming is unusually good for a
work of art. It synchronizes simultaneous sounds and makes consecutive
sounds sound
continuous. There are no audio gaps and the synchronization is
tight. Usually you only see this sort of programming in
commercially available
audio production tools. Consequently, the artistic form of the
rhythmic heap has been little explored in works of digital art,
and the exploration
Jig-Sound enables of rhythmic musical forms is part of
its unique artistic and technical achievement.
There are many works of digital art in which interactive music is
developed in all sorts of ways. But if the software does not permit
synchronized layering
and sequencing of heaps, then the territory it's exploring—and
the types of music it can treat—are distinctly different from Jig-Sound.
If you think of the operations you can perform on rhythmic music in
audience-interactive music software you've encountered, you'll realize
that either rhythm is not supported and the operations become rich,
or rhythm
is
maintained
and the interactive operations are usually either layering or sequencing
but not both. Jig-Sound supports both.
You can also copy sounds or structures you've
created and link them together. The constructive tools of Jig-Sound are
quite powerful but very simple to use.
Also, the object-linking paradigm of its visual interface is
a departure from the typical grid present in just about all
programs where one
may construct rhythmic music. The Jig-Sound paradigm
is more like a flow chart than a temporal, channeled two-dimensional
grid. Jig-Sound encourages
a view of a piece of music as a lattice of loops linked in
temporal sequence
and in layers
of intertwined rhythms. It also encourages you to see a piece
of music as a network of sounds through which the music flows
like blood
through the body or like data through the Internet.
The Music
The Shockwave pieces at top left of this page give you a sense of the
music that will form the Jig-Sound album. It's vocal music by me.
The percussion is finger-snapping. And other tracks will be of songs
by The Laughing Boot Quintet, a band
I was the drummer and audio producer of.
Software art often risks disappearing into the technology. The Jig-Sound project is sophisticated technologically, but the music for the album
is not particularly electronic. The Jig-Sound technology does not require
high-tech music.
You can listen to another couple of heaps I've made, which will also
be part of the album, in Nio and Oppen
Do Down. Like my other heaps, these never were linear songs. They're
specially composed heaps for being layered and sequenced.
How to Play
The best way to introduce you to the project is for you to visit vispo.com/jig and
play around a bit. What follows in this section is how to play in the interface-in-progress.
You
see a bunch of sound icons like A. Each of these icons represents a sound.
Click an icon to listen to its sound. This Heap of 17 sounds are ones I sang,
recorded and mixed in Sonar, and smithied in Sound Forge. These sounds never
were part of a linear song. They were specially made as a Heap. I am
writing the software for this also. In Director.
When you mouse-over a sound icon, the cursor changes into a hand 
�or a pencil 
. When it's a hand, you
can click and drag the icon around the window. When it's a pencil, click
and drag to link sound icons. You can link sound icons in two different ways.
You
can link icons with a green line when you click on the top or bottom of an
icon and drag. Two icons linked by a green line play simultaneously.
When you click an icon A that has any green links, that icon's sound plays
and so do the sounds of icons green-linked to A. You can link as many icons
together as you like with green-links. However, Director only has 8 stereo
channels (each of the 8 is stereo); you can only play at most 5 sounds simultaneously
in a jig-sound. Two of the remaining channels will be for keyboard playing
of sounds (not written yet) and the final channel is reserved for stuff relating
to sound synchronization. Basically one of the channels is always playing
silence with a cuepoint in it so I can synchronize the channels.
You
can also link icons with a blue line. You create a blue line when you click
on the left or right side of an icon and drag. Two icons linked by a blue
line play sequentially. In the graphic to the right, icon A plays
first. Then icon B plays. Sound flows out of the right side of icons, through
blue lines, and into the left side of blue-linked icons.
- Icons whose sound is currently playing are orange. The most recently
clicked icon (if it isn't playing) is blue-tinted. This is called the selected
icon.
- When the 'loop' button is depressed, which it is by default, then when
a Jig-Sound reaches the end (if it has an end), it will loop back to the
selected icon.
- To delete a line you've created, click in the area
where it joins an icon.
- To select a bunch of sound icons, click the black background and drag
a rectangle around the icons you want to select. You can also Shift+click
an icon to add it to the group of selected icons.
- To drag a bunch of selected icons, click and drag in the blue rectangle
you've created (but don't click one of the selected icons).
- To deselect a bunch of icons, click on the black background or on an
icon. You can deselect a single icon from the group of selected icons by
Alt+clicking the icon.
- To copy a bunch of selected icons, click the 'copy' button.4
- To delete a bunch of selected icons, click the 'del' button. Only icons
of which there are more than one copy will be deleted.
- The 'stop' button stops the audio. Click an icon to resume.
- The volume control adjusts audio volume.
- To swap the location and links of two sound icons, press and hold the
Ctrl key (Command key on Macs). Then drag and drop one of the icons onto
the other.
- The icon that has � as its symbol is the 'silent sound'. This icon behaves
like a regular sound icon but is silent. It is useful when you want to
blue-link to an icon A but the left blue connector of A already has a line
attached to it. In this case, you just green-link A to the silent sound
(or green-link the silent sound to any icon connected to A by a green link)
and then blue-link to the silent sound.
- To move between play mode and game mode, click the Jig-Sound logo at
top left.
Game Mode: Clock that Jig
What we just looked at is part of 'play mode'. There will also be a 'game
mode'. I'm still working on 'game mode'. I'm not sure how fun it is at this
point. I think it needs work (so did another game I made, early on: Arteroids).
But the basic idea of 'game mode' is to present something that is more like
a
jig-saw
than
what
you experienced
in
'play
mode' but, unlike a jig-saw, you are timed on how fast you put the puzzle
together, and scores will be uploaded to the Net (with the player's permission).
To
try out the basic idea of 'game mode', click the Jig-Sound logo
at top left of vispo.com/bc/a.
After you click the Jig-Sound logo,
you should see 22 sound icons that look a bit different than the previous
ones. These ones just have two connectors, not four, as illustrated by
the graphic in this paragraph. In 'game mode', you can't draw green lines.
In
other words, you can't layer sounds. You can only sequence them. Neither
can you copy icons. I took the copy button out because it doesn't help
you solve the puzzle.
The idea in this demo is to put Where Did Our Love Go? by Diana Ross
and the Supremes back together again. I won't be using other peoples' music
in the final version, but this gives you the idea of 'game mode'. The puzzle
is completed when the 22 sound icons are linked together and all the lines
between the sound icons are red, as shown below.
If two sounds are not consecutive in the original piece of music, then lines
between the corresponding sound icons are blue. Whereas if two linked icons
were consecutive in the original music, the line between them is red. This
is how you know whether the 'puzzle piece' fits or not.
This puzzle is kind of hard without the cheat I've provided you. This should
probably be level 3. Level 1 should probably contain about 6 sounds. The
six sounds would be sufficiently longer than these so that the full song
plays in the six sounds. Then level 2 would consist of 11 sounds. Level 3,
as above, would consist of 22 sounds. Level 4 would consist of 44 sounds.
Cut a good loop in two and you get two good loops (down to a certain level
of time).
A Song in Play Mode (the Green Heaps of Home)
Here is another 'play mode'
Heap: The Green Hills of Home. The
music in this piece is by The Laughing Boot Quintet. I was the band's drummer
and recording technician.
The sound icons are labelled in the order in which they occur in the original
song; in other words, you can hear the original song by blue-linking all
the icons together alphabetically.
The
sound icons in this Heap are color coded to indicate which sets of icons
can be layered amongst each other with
some success. The particular group I'm using here (o-v) are pretty good
layered together, in most instances. The music in the final project will
consist
of� Heaps by me and a few from music of The Laughing Boot Quintet.
Jig-Sound Visuals: Dbcinema
Dbcinema,
a piece I've written in Director, will supply the visuals for the Jig-Sound project. Dbcinema fetches
images from Google and Yahoo image searches.5 In the Jig-Sound project,
the current Heap will feed dbcinema language that is relevant to the
Heap so that images related to the music will be displayed in the background
of Jig-Sound. You can see that dbcinema is already quite advanced
in its speed and range of features.
However, it's currently more like dbslideshow, nonetheless. The way
I will take it to dbcinema is via processing of the images. In other
words, rather than simply displaying one image after another, as it does
now, it will use the images as content for full-screen collage and
abstract visual generation algorithms that can also be in rhythm with the
music.
To get a sense of the generative abstract visuals I have in mind, have
a look at some of the pieces at complexification.net .
This is a site by Jared Tarbell. And imagine collaged images being
revealed rather than what Tarbell reveals, which is simply colour.
Tarbell's work is done in the language Processing. Mine will be done
in Imaging Lingo (in Director).
Usually when generative computer visuals are successful, they have
a strong abstract
dimension. But usually they are weak representationally. dbcinema can
be strong both in its representational dimension (it already is) and
also in its abstract image processing dimension (still to be written).
I'm hoping this will be an interesting
alternative to the music video, and to the sort of totally
abstract visual accompaniment you see in Winamp or Windows Media
Player etc. Typically the latter simply present abstract patterns
in response to
audio amplitude. Music videos, as a genre, somehow never quite
blossomed into art, or at least not the commercial variety you see
on Much Music or MTV, usually.
Clearly there is lots of unexplored territory concerning the visual
representation of music.
The audience will be able to use dbcinema as you can now, or they
can choose to leave it alone and let the heap feed dbcinema language.
The audience will also be able to select among the algorithms that
will power the collage/generative visuals.
Some of those algorithms will work in rhythm to the music.
Luca Barbeni (Italy), in his curation of "Unlimited Cinema," said of
dbcinema:
Artist, programmer and theorist Jim Andrews is one of the
pioneers of online audio-visual experimentation. With dbcinema,
launched in 2005 and still in progress, he is part of a line of research
that,
starting from Duchamp�s ready-made works, nowadays finds an interesting
realization in works of art that use Google as a support and tool.
Our favourite search engine is becoming the most popular media for
retrieving images and sounds.
Part of the attraction of dbcinema, to me, is the way it presents
a kind of collective impression of the concepts you type into it. The
images range from professional photos to personal and idiosyncratic
gestures to online menu graphics. The aesthetic of dbcinema is more
from software art than art school. Also, it isn't simply about a look
as an interactive process of visual generation. The audience interacts
with dbcinema and dbcinema interacts with the collective humungous
image databases of the Internet.
The language a particular heap feeds dbcinema will be selectable by
the audience. For instance, if I were to use "Where Did Our Love Go?"
by the Supremes (which I'm not going to do--the album will consist
of music by me and the Laughing Boot Quintet), the audience might find
the below:
They
could then select among the search terms or let Jig-Sound feed
this language to dbcinema at its own pace. Or the audience could type
in
their own choice of language to generate the visuals. The audience
will also be able to click on images to open the web page from which
the image was drawn. So the experience is not only one of interactive
composition of the music, but selection or generation of the language
used to generate the visuals, and possibly also browsing of web pages
from which the images were drawn.
Still to be Done
As a form of music, there's some exploration to be done. Jig-Sounds can
deal nicely with any song in 'game mode' where layering is not allowed.
But 'play mode', since it involves layering, not only sequencing, is a more
creative challenge to make audio for.
I have made/sung several Heaps in the past, two of which you can interact
with in Nio and Oppen
Do Down. I want to make four new Heaps of original music for this
project. I also want to use songs by the Laughing Boot Quintet in this project.
I will be
working with the other members of the band on those songs and also using
recordings.
In 'play mode', you will be able to save jigs you compose
locally or upload them to the Net for others to download and listen
to or edit. Saved jigs
are just text files. Configuration information. Not sound files. So they
are quite portable; they're relatively small text files rather than 4
Mb MP3 files6. Also, people will be able
to email their friends a composition they're working
on which
the
friends
can
listen
to
and/or edit.
This functionality
is a mixture of native Director coding plus PHP programming, which is
server-side programming. Martin Kloss does the PHP in my work and provides
me with
a Lingo API to the PHP code. A piece in progress called War
Pigs was Martin
and my first go at combining Lingo and PHP. In 'play mode' of War Pigs,
you can download compositions by others and upload your own. And game
mode in
War Pigs uploads scores to the server. So we have most of the Lingo+PHP
worked out already. The email component needs to be done from scratch,
though.
'Play mode' is more creative than game mode. You're not competing. The sounds
you can use are not just cut-up songs but sounds composed for Heaps, usually.
Jig-Sound will be one integrated application. You'll be able to access
multiple heaps, but that access will be within one application.
There will be a menu that lets you select whether you want 'game mode' or
'play mode' and you will have a choice of Heaps in either mode. Once you
choose a Heap, it will be downloaded from the server using Deepdown,
a downloader I've created in Director. You'll have access to sounds as they
stream in. Currently you have to wait for the whole thing to download but
that will change.
