globalCompositeOperation in Net Art

Ted, Jim and globalCompositeOperation

Ted Warnell and I have been corresponding together about net art since 1996 or 97. We've both been creating net art using the HTML 5 canvas for about the last 6 years; we show each other what we're doing and talk about canvas-related JavaScript via email. He lives in Alberta and I live in British Columbia.

Ted's canvas-based stills and animations can be seen at warnell.com/mona. My canvas-based work includes Aleph Null versions 1.0 and 2.0 at, respectively, vispo.com/aleph and vispo.com/alephTouch.

One of the things we've talked about several times is globalCompositeOperation—which has got to be a candidate for longest-name-for-a-JavaScript-system-variable. The string value you give this variable determines "the type of compositing operation to apply when drawing new shapes" (mozilla.org). Or, as w3schools.com puts it:

"The globalCompositeOperation property sets or returns how a source (new) image is drawn onto a destination (existing) image.

Source image = drawings you are about to place onto the canvas.

Destination image = drawings that are already placed onto the canvas."

The reason we've talked about this variable and its effects is because globalCompositeOperation turns out to be important to all sorts of things in creating animations and stills that you wouldn't necessarily guess it had anything to do with. It's one of those things that keeps on popping up too much to be coincidental. The moral of the story seems to be that globalCompositeOperation is an important, fundamental tool in creating animations or stills with the canvas.

In this article, we'd like to show you what we've found it useful for. We'll show you the art works and how we used globalCompositeOperation in them to do what we did with it.

Ted's uses of globalCompositeOperation tend to be in the creation of effects. Mine have been for masking, fading to transparency, and saving a canvas to png or jpg.

Digital Compositing

"Compositing" is an interesting word. It's got "compose" and "composite" in it. "Compositing" is composing by combining images into composit images.

Keep in mind that each pixel of a digital image has four channels or components. The first three are color components. A pixel has a 'red' value, a 'green' value, and a 'blue' value. These are integers between 0 and 255. These combine to create a single color. The fourth channel or component is called the alpha channel. That's a number between 0 and 1. It determines the opacity of the pixel. If a pixel's alpha channel has a value of 1, the pixel is fully opaque. If it has a value of 0, the pixel is totally transparent. It can have intermediary values that give the pixel an intermediary opacity.

The default value of globalCompositeOperation is "source-over". When that's the value, when you paste a source image into a destination canvas, you get what you'd expect: the source is placed overtop of the destination.

There are 26 possible values for globalCompositeOperation which are described at mozilla.org. The first 8 of the options, shown below, are for compositing via the alpha channel. The remaining 18 are blend modes. You may be familiar with blend modes in Photoshop; they determine how the colors of two layers combine and include values such as "multiply", "screen", "darken", "lighten" and so on. Blend modes operate on the color channels of the two layers.

But the first 8 values shown below operate on the alpha channels of the two images. They don't change the colors. They determine what shows up in the result, not what color it is. The first 8 values in the below diagram can be thought of as a kind of Venn diagram of image compositing. There's the blue square (destination) and the red circle (source). There are 3 sections to that diagram:

  • A: the top left part of the blue square that doesn't intersect with the red circle;
  • B: the section where the square and circle intersect;
  • C: and the bottom right section of the red circle that doesn't intersect with the blue square.

Section A can be blue or be invisible; section B can be blue, red, or invisible; section C can be red or invisible. That makes for 12 possibilities, but some of those 12 possibilities, such as when everything is invisible, are of no use. When the useless possibilities are eliminated, we're left with the first 8 shown below. These possibilities form the basic sort of Venn logic of image compositing. You see this diagram not only with regard to JavaScript but in image compositing regarding other languages.

The first 8 values for globalCompositeOperation operate on the alpha channels of the source (red) and destination (blue)

 What is "compositing"? We read the following definition at Wikipedia:

Compositing is the combining of visual elements from separate sources into single images, often to create the illusion that all those elements are parts of the same scene. Live-action shooting for compositing is variously called "chroma key", "blue screen", "green screen" and other names. Today, most, though not all, compositing is achieved through digital image manipulation. Pre-digital compositing techniques, however, go back as far as the trick films of Georges Méliès in the late 19th century; and some are still in use. All compositing involves the replacement of selected parts of an image with other material, usually, but not always, from another image. In the digital method of compositing, software commands designate a narrowly defined color as the part of an image to be replaced. Then the software replaces every pixel within the designated color range with a pixel from another image, aligned to appear as part of the original. For example, one could record a television weather presenter positioned in front of a plain blue or green background, while compositing software replaces only the designated blue or green color with weather maps.

Whether the compositing is operating on the alpha or the color channels, compositing is about combining images via their color and/or alpha channels.

As we see at rekim.com, different browsers treat some of the values of globalCompositeOperation differently, which can make for dev headaches and gnashing of teeth but, for the most part, globalCompositeOperation works OK cross-browser and cross-platform.

Jim Andrews: Masking (source-atop)

Masking is when you fill a shape, such as a letter, with an image. The shape is said to mask the image; the mask hides part of the image. Masking was crucial to an earlier piece of software I wrote called dbCinema, a graphic synthesizer I wrote in Lingo, the language of Adobe Director. The main idea was of brushes/shapes that sampled from images and used the samples as a kind of 'paint'. My more recent piece Aleph Null 2.0, written in JavaScript, can do some masking, such as the sort of thing you see in SimiLily—and I'll be developing more of that sort of thing in Aleph Null.

Let's look at a simple example. You see it below. You can also see a copy of it at vispo.com/alephTouch/test/masking7.html, where it's easier to view the source code. There's a 300x350 canvas with a red border. We draw an 'H' on the canvas. We fill it with any color--red in this case. Then we set globalCompositeOperation = 'source-atop'. Then we draw a bitmap of a Kandinsky painting into the canvas, but the only part of the Kandinsky that we see fills the 'H'. Because when you set globalCompositeOperation = 'source-atop' and you then draw into an image, it only draws on pixels that were already on the canvas.

W3schools.com states it this way:

"source-atop displays the source image on top of the destination image. The part of the source image that is outside the destination image is not shown."

In other words, first you draw on the canvas to create the "destination" image (the 'H'). Then you set globalCompositeOperation = 'source-atop'. Then you draw the "source" image on the canvas (the Kandinsky).

Masking with globalCompositeOperation = ‘source-atop’

The most relevant code in the above example is shown below:

function drawIt(oldValue) { 
context.font = 'bold 400px Arial'; 
context.fillStyle='red'; 
context.fillText('H', 0,320); 
// The above three lines set the text font to bold, 
// 400px, Arial, red, and draw a red 'H' at (0,320). 
// This is the destination. 
// (0,320) is the bottom left of the 'H'. 
context.globalCompositeOperation = 'source-atop'; 
context.drawImage(newImg, -100,-100); 
// newImg is the rectangular Kandinsky image. 
context.globalCompositeOperation=oldValue; 
// Sets globalCompositeOperation back to what it was. 
}

In our example, the destination 'H' is fully opaque. However, if the destination is only partially opaque, so too will the result be partially opaque. The opacity of the mask determines the opacity of the result. You can see an example of that at vispo.com/alephTouch/test/masking.html. The mask, or destination, is an ellipse that grows transparent toward its edge. The source image, once again, is a fully opaque Kandinsky-like image.

You can see some of Aleph Null's masking ability if you click the Bowie Brush, shown below. It fills random polygons with images of the late, great David Bowie.

The Bowie Brush in Aleph Null fills random polygons with images of David Bowie

Ted Warnell: Albers by Numbers, February, 2017

Overview: Poem by Nari works are dynamically generated, autoactive and alinear, visual and code poetry from the cyberstream. Poem by Nari is Ted Warnell and friends. Following are four Poem by Nari works that demonstrate use of some of the HTML5 canvas globalCompositeOperation(s) documented in this article.

These works are tested and found to be working as intended on a PC running the following browsers: Google Chrome, Firefox, Firefox Developer Edition, Opera, Internet Explorer, Safari, and on an Android tablet. Additional browser specific notes are included below.

Albers by Numbers  warnell.com/mona/abnum.htm

Experimental. Albers by Numbers is one from a series of homages to German-American artist Josef Albers. Poem by Nari series is loosely based on the Albers series "Homage to the Square".

This work is accomplished in part by a complex interaction of stylesheet mixBlendMode(s) between the foreground and background canvases. All available mixBlendMode(s) are employed via a dedicated random selection function, x_BMX.

Interesting to me is how the work evolves from a single mass of randomly generated numeric digits to the Albers square-in-square motif. This emergence happens over a period of time, approximately one minute, and in a sense parallels emergence of the Albers series, which happened for Albers over a lifetime.

Note to IE and Safari users: works but not as intended.

Ted Warnell: Acid Rain Cloud 3, February 2017

Acid Rain Cloud 3
warnell.com/mona/ar3cloud.htm

Experimental. Another work from a series exploring a) acid, b) rain, c) clouds, d) all of the above.

globalCompositeOperation(s) "source-over" and "xor" are used here in combination with randomized color and get & putImageData functions. The result is a continually shifting vision of what d) all of the above, above, might look like.

Interesting to me here is that ever changing "barcode" effect in the lower half of the work - possibly the "rain" in this? Over time, that rain will turn from a strong black and white downpour to a gentle gray mist. This is globalCompositeOperation "xor" at work.

Note to Safari users: works but not as intended.

Ted Warnell: An Alinear Rembrandt, April 2017

An Alinear Rembrandt   warnell.com/mona/aa_rem.htm
Pinwheels  warnell.com/mona/pinwheel.htm

More experimentation. This work is for Mary & Ryan Maki, Canada

Full screen, variable canvas rotations, and globalCompositeOperation(s) "source-over" and "xor" with randomized color. "source-over" is default and is responsible for the vivid, solid colors in this work, while "xor" provides the muted, soft-edge color blends.

Pinwheels... I'm going to be a grandpa again.

Note to Safari users: does not work with Safari browser.

Fade to Transparency (destination-out)

The fader slider in Aleph Null

Aleph Null 2.0 has a fader slider. The greater the value of the fader slider, the quicker the screen fades to the current background color. This is implemented by periodically drawing a nearly-transparent fill of the background color over the whole canvas. The greater the value of the fader slider, the more frequent the drawing of that fill over the whole canvas.

That works well when there is just one canvas, when there is no notion of layers of canvases. Once you introduce layers, you have to be able to fade a layer to transparency, not to a background color, so that you can see what's on lower layers. I'm attempting to implement layers at the moment in Aleph Null. So I have to be able to fade a canvas to transparency.

So, then, how do you fade a canvas to transparency?

As Blindman67 explains at cpume.com, "...you can avoid the colour channels and fade only the alpha channel by using the global composite operation "destination-out" This will fade out the rendering by reducing the pixels' alpha." Each pixel has four channels: the red, the blue, the green, and the alpha channels; the alpha channel determines opacity. The code is like this:

ctx.globalAlpha = 0.01;  // fade rate 
ctx.globalCompositeOperation = "destination-out" 
ctx.fillRect(0,0,w,h) 
ctx.globalCompositeOperation = "source-over" 
ctx.globalAlpha = 1; // reset alpha

You do the above every frame, or every second frame, or every third frame, etc, depending on how quickly you want it to fade to transparency. Another parameter with which you control the speed of the fade is ctx.globalAlpha, which is always a number between 0 and 1. The higher it is, the closer to fully opaque the result will be on a canvas draw operation.

Blindman67 develops an interesting example of a fade to transparency in vispo.com/alephTouch/test/fade1.htm. You can see that it must be fading to transparency because the background color is dynamic, is constantly changing.

Note that the ctx.fillStyle color isn't really important because we're fading the alpha, not the color channels. ctx.fillStyle isn't even specified in the above code. When globalCompositeOperation = 'destination-out', the color values of the destination pixels remain unchanged. What changes is the alpha value of the destination pixels. The alpha values of the source pixels get subtracted from the alpha values of the destination pixels.

The performance of fading this way should be very good, versus mucking with the color channels, because you're changing less information; you're only changing the alpha channel of each pixel, not the three color channels.

I massaged the Blindman67 example into something simpler at vispo.com/alephTouch/test/fade2.htm. There's a fade function:

function fade() { 
gCtx1.globalAlpha = 0.15; // fade rate 
gCtx1.globalCompositeOperation = "destination-out"; 
gCtx1.fillRect(0,0,gCanW,gCanH); 
gCtx1.globalCompositeOperation = "source-over"; 
gCtx1.globalAlpha = 1; // reset alpha 
}

But compare the fade function with the code above it from Blindman67. It's the very same idea.

Above, we see an example much like what I wrote at vispo.com/alephTouch/test/fade2.htm

Finally, on this topic, I'm currently wondering about the best way to implement layers concerning canvases. Clearly compositing possibilities create a situation where, at least in some situations, you don't need multiple visible canvases; you can composit with offscreen canvases and only use one visible canvas. Whether this is better in general, and what the performance issues are, is currently unclear to me. There also exists at least one platform, namely concretejs, that supports canvas layers.

Save Canvas to Image File (destination-over)

globalCompositeOperation = 'destination-over' allows you to slip an image into the background of another image. The source image is written underneath the destination image.

It turns out that's precisely what is needed to fix some bad browser behavior when you save a canvas to an image file, as we'll see.

If you want to save a canvas to an image file, the simplest way to do it, at least on Chrome and Firefox, is to right-click (PC) or Control+click (Mac). You are presented with a menu that allows you to "Save As..." or, on some browser, "Copy Image". The problem is that some browsers insert a background into this image that probably isn't the same color as the background on the canvas.

On the PC, Chrome inserts a black background. Other browsers may insert other colors, or the right color, or no color at all. One solution to this problem is to create a button that runs some JavaScript that inserts the right background color. This is a job for globalCompositeOperation = 'destination-over' because it allows you to create a background with the source image.

The "save" button in Aleph Null

You can see the solution I've created at vispo.com/alephTouch/an.html, shown above. The controls contain a "save" button which, when clicked, copies a png-type image into a new tab, if permitted to do so. You may have to permit it by clicking on a red circle near the URL at the top of the browser. Once the image is in the new tab, right-click (PC) or Ctrl+click (Mac) and select "Save As...".

The code is basically this sort of thing:

var canvas=document.getElementById('canvas'); 
var context=canvas.getContext('2d'); 
// We assume the canvas already has the destination image on it. 
var oldGlobalComposite=context.globalCompositeOperation; 
context.globalCompositeOperation='destination-over'; 
context.fillStyle=backgroundColor; 
// backgroundColor is a string representing the desired background color. 
context.fillRect(0,0,canvas.width,canvas.height); 
context.globalCompositeOperation=oldGlobalComposite; 
var data=canvas.toDataURL('image/png'); 
window.open(data);

The toDataURL command can also create the image as a jpg or webp.

In your animations with the HTML 5 canvas, will globalCompositeOperation be of any use? The answer is that if you are combining images at all, doing any compositing at all, globalCompositeOperation is probably relevant to your task and may make it much easier.


2 Responses to “globalCompositeOperation in Net Art”

  • CJ: June 19, 2017 at 9:29 am

    Very interesting Jim, thanks for sharing this. I’ve played around with both methods of layering – multiple layers within the one canvas, and multiple canvases, and I’ve not noticed any huge performance difference, even at high speeds. However I tended to prefer the single canvas method for purposes of the ability to perform a single ‘wipe’ if necessary, rather than multiple clears on all canvases. globalCompositeOperation is full of possibilities as you’ve suggested. I did a series of Muybridge influenced pieces where the user can choose the mode ( e.g. http://remixworx.com/cj/2015/after-muybridge/horse/ , click the cog and the modes appear at the bottom left). Masking has some great potential – this was one where I use masking and multiple canvases – http://remixworx.com/cj/2015/quick-dirty/ . I think at the time, conceptually, for me, masking the text on each canvas seemed a lot simpler than masking multiple layers within a single canvas.

  • Jim Andrews: June 19, 2017 at 10:05 am

    Wow, looks like you’ve been wrassling this stuff pretty hard too, C. Impressive work.

    Good to hear you haven’t noticed any definitive difference between multiple canvases and one canvas. Except, yes, if one is having to constantly fade/clear multiple canvases.

    Have you tried any canvas platforms? I haven’t. There’s at least one out there that supports layers: http://www.concretejs.comCJ: