InterActiveImagetest

This sketch displays an image on canvas and warps it into a bulging, rippling field centered on your mouse cursor. When you click to activate your microphone, the intensity of the warp responds to sound levels and time-based oscillations, creating an interactive audio-visual sculpture.

🧪 Try This!

Experiment with the code by making these changes:

  1. Make the warp pinch instead of bulge — Swapping the audioMorphFactor range makes silence bulge and sound pinch the image inward - the opposite of the default
  2. Slow down the time oscillation — Lowering the 0.05 multiplier makes the continuous rippling pulse much more slowly, creating a meditative breathing effect
  3. Disable mouse-controlled radius — Set baseRadius to a fixed number instead of mapping mouseX - the warp zone stays the same size everywhere on screen
  4. Remove audio reactivity entirely — Setting audioMorphFactor to 0 removes all microphone influence, leaving only the time-based pulsing
  5. Make the pixels array update more often — Adding extra calls to loadPixels() forces p5.js to refresh from the canvas more frequently (for debugging, usually unnecessary)
Prefer the full editor? Open it there →

📖 About This Sketch

This sketch turns a static pencil-sketch image into a living, morphing surface that warps and bulges around your mouse position. The warp effect responds to both your microphone's audio level and a subtle time-based pulse, creating a hypnotic rippling effect that reacts to sound and voice. It combines several advanced p5.js techniques: direct pixel manipulation using the pixels array, audio analysis with p5.sound, polar-to-Cartesian coordinate conversion for smooth geometric distortion, and window-responsive canvas resizing.

The code is organized into five functions: preload() loads the image before anything else runs, setup() initializes the canvas and audio input, mousePressed() starts the microphone on user interaction (required by browser security), draw() performs the main warping effect each frame by iterating through every pixel and computing displacement, and windowResized() keeps the canvas properly sized when the browser changes. By studying it you will learn how to manipulate pixel data directly, respond to audio input in real time, and use polar coordinates to create smooth geometric deformations.

⚙️ How It Works

  1. When the sketch loads, preload() reads the image file, setup() creates a full-window canvas with pixel density set to 1 for accuracy, initializes the microphone object, and creates an amplitude analyzer that watches the mic's audio level.
  2. When you click the canvas, mousePressed() asks the browser to start recording your microphone (this requires user interaction due to security policies).
  3. Every frame, draw() loads both the canvas pixels and the image pixels into memory for efficient direct manipulation.
  4. The sketch calculates three reactive values: the amplitude (volume level) from the microphone, a time-based oscillation using sin(frameCount), and a morph strength that blends audio and time reactivity together.
  5. For every pixel on the canvas, the code computes its distance from the mouse cursor and checks if it falls within the morphing radius (controlled by your mouse X-position).
  6. Inside the radius, pixels are displaced outward or inward using polar coordinates: distance is modified by a sine-wave displacement, the angle is preserved, and new Cartesian coordinates are calculated to sample the original image at a warped location.
  7. Outside the radius, pixels copy directly from the original image, and the final pixel data is written back to the canvas with updatePixels().

🎓 Concepts You'll Learn

Direct pixel manipulationAudio reactivityPolar coordinate systemDisplacement mappingReal-time image warpingp5.sound amplitude analysis

📝 Code Breakdown

preload()

preload() is a special p5.js function that runs before setup(). Use it to load images, sounds, and data files so they are ready when your sketch needs them.

function preload() {
  // Load the image file
  // Make sure the filename matches exactly 'paul-on-tractor-pencil-pixels.jpg'
  img = loadImage('paul-on-tractor-pencil-pixels.jpg');
}
Line-by-line explanation (1 lines)
img = loadImage('paul-on-tractor-pencil-pixels.jpg');
Loads the image file from disk into the img variable. preload() ensures the image is fully loaded before setup() or draw() run, preventing errors when trying to access image data.

setup()

setup() runs once when the sketch starts. It is the perfect place to create your canvas, load audio systems, initialize variables, and prepare everything draw() will need. The canvas size directly affects the draw loop's workload - a larger canvas means more pixels to process every frame.

function setup() {
  // Create a canvas that fills the window
  createCanvas(windowWidth, windowHeight);
  // Important for pixel manipulation: ensure pixelDensity is 1
  // This makes sure 1 p5 pixel corresponds to 1 screen pixel,
  // preventing issues on high-DPI (Retina) displays.
  pixelDensity(1);
  // Optional: Set noSmooth() for a sharper, pixel-art look, especially for this image
  noSmooth();

  // Initialize audio input
  mic = new p5.AudioIn();
  // Initialize amplitude analysis
  amplitude = new p5.Amplitude();
  amplitude.setInput(mic); // Set the microphone as the input for amplitude analysis
}
Line-by-line explanation (6 lines)

🔧 Subcomponents:

calculation Canvas initialization createCanvas(windowWidth, windowHeight);

Creates a canvas that matches the full window size, making the sketch responsive

initialization Audio system setup mic = new p5.AudioIn(); amplitude = new p5.Amplitude(); amplitude.setInput(mic);

Initializes the microphone object and creates an amplitude analyzer that listens to it

createCanvas(windowWidth, windowHeight);
Creates a canvas that fills the entire browser window, making the sketch fullscreen and responsive
pixelDensity(1);
Sets the pixel density to 1, ensuring each p5.js pixel maps to exactly 1 screen pixel. This is critical for pixel manipulation - without it, high-DPI displays (like Retina) would cause off-by-one errors
noSmooth();
Disables anti-aliasing, giving the image and warping effect a sharper, more pixelated appearance that suits this pencil-sketch aesthetic
mic = new p5.AudioIn();
Creates a new microphone input object that will capture audio from your device
amplitude = new p5.Amplitude();
Creates a new amplitude analyzer object that can measure the volume level of an audio source
amplitude.setInput(mic);
Tells the amplitude analyzer to listen to the microphone, so it can measure how loud you are speaking

mousePressed()

mousePressed() is called automatically by p5.js every time the user clicks. Many browsers require user interaction to start audio recording for privacy reasons - this is why the microphone doesn't start on its own. The guard clause (the if statement) prevents unnecessary repeated calls to mic.start().

🔬 What happens if you remove the condition `!mic.started` so it becomes just `if (mic) {`? Try it and explain why this could be a problem.

if (mic && !mic.started) {
    mic.start();
    console.log("Microphone started. You can now speak to interact with the sketch.");
  }
function mousePressed() {
  // Start the microphone if it's not already started
  if (mic && !mic.started) {
    mic.start();
    console.log("Microphone started. You can now speak to interact with the sketch.");
  }
}
Line-by-line explanation (3 lines)

🔧 Subcomponents:

conditional Microphone guard if (mic && !mic.started) {

Ensures the microphone only starts once and hasn't already been initialized

if (mic && !mic.started) {
Checks two things: (1) mic exists (is not null), and (2) mic.started is false (hasn't already been started). The ! means 'NOT' - so this runs only if the mic has NOT been started yet
mic.start();
Tells the microphone to begin recording audio. This must happen in response to user interaction (like a click) due to browser security rules
console.log("Microphone started. You can now speak to interact with the sketch.");
Prints a message to the browser console confirming the microphone is active - helpful for debugging

draw()

draw() is the heart of the sketch and runs 60 times per second. The double-loop structure (iterating x then y) is the most common way to process every pixel. The nested conditional checks first whether a pixel is in the warp radius, then whether it's in the image bounds - this prevents errors from out-of-bounds array access. The polar-to-Cartesian conversion is the key technique that creates the smooth, organic-looking warp instead of a jagged distortion.

🔬 These three lines calculate the distance from each pixel to the warp center. What happens if you change sqrt(dx * dx + dy * dy) to just abs(dx) + abs(dy)? (This calculates 'Manhattan distance' instead of Euclidean.) Try it and describe how the warp shape changes.

      let dx = x - morphCenterX;
      let dy = y - morphCenterY;
      let dist = sqrt(dx * dx + dy * dy);

🔬 This sine wave creates a smooth bulge that peaks in the middle of the radius. What happens if you change sin(normalizedDist * PI) to cos(normalizedDist * PI)? Or to just normalizedDist (linear instead of curved)? Try each and describe the visual difference in how the displacement fades.

        let displacement = sin(normalizedDist * PI) * morphFactor * morphRadius;
function draw() {
  // Clear the canvas with a background color
  background(220);

  // Load the pixels of the canvas for direct manipulation
  // This is more efficient than using get() and set() repeatedly.
  loadPixels();

  // Load the pixels of the image
  // This ensures we have access to the original image data.
  img.loadPixels();

  // --- AUDIO AND TIME REACTIVITY ---

  // Get the current overall amplitude (volume) level from the microphone
  // The level ranges from 0.0 (silent) to 1.0 (loudest)
  let level = amplitude.getLevel();

  // Create a time-based oscillation using sin() function and frameCount
  // frameCount counts the number of frames since the sketch started
  // 0.05 controls the speed of the oscillation
  let timeFactor = sin(frameCount * 0.05);

  // Define the center of the morphing effect
  // It follows the mouse pointer for interactivity.
  let morphCenterX = mouseX;
  let morphCenterY = mouseY;

  // Define the radius of the morphing effect
  // The mouse's X-position controls the base radius, allowing you to make the effect larger or smaller.
  let baseRadius = map(mouseX, 0, width, 50, 300); // Base radius from 50 to 300 pixels
  let morphRadius = baseRadius;

  // Define the strength of the morphing effect (bulge/pinch)
  // Audio level controls the main intensity of the bulge/pinch effect.
  // timeFactor adds a subtle, continuous oscillation on top of the audio reactivity.
  let audioMorphFactor = map(level, 0, 1, -0.7, 0.7); // Stronger audio influence for noticeable effect
  let timeMorphFactor = map(timeFactor, -1, 1, -0.1, 0.1); // Subtle time oscillation for continuous movement
  let morphFactor = audioMorphFactor + timeMorphFactor;

  // --- END REACTIVITY ---

  // Loop through every pixel on the canvas
  for (let y = 0; y < height; y++) {
    for (let x = 0; x < width; x++) {
      // Calculate the index for the current pixel in the canvas's pixels array
      // Each pixel has 4 values (R, G, B, A)
      let canvasIndex = (x + y * width) * 4;

      // Calculate the distance and angle of the current pixel from the morph center
      let dx = x - morphCenterX;
      let dy = y - morphCenterY;
      let dist = sqrt(dx * dx + dy * dy);

      // Check if the current pixel is within the morphing radius
      if (dist < morphRadius) {
        // Normalize the distance (0 to 1 within the radius)
        let normalizedDist = dist / morphRadius;

        // Calculate a displacement value using a sine wave
        // This creates a smooth bulge/pinch effect that is strongest at the center
        // and fades out towards the edge of the morphRadius.
        let displacement = sin(normalizedDist * PI) * morphFactor * morphRadius;

        // Calculate the new distance from the morph center after displacement
        let newDist = dist + displacement;

        // Calculate the angle of the pixel relative to the morph center
        let angle = atan2(dy, dx);

        // Convert the new polar coordinates back to Cartesian coordinates
        let newX = morphCenterX + newDist * cos(angle);
        let newY = morphCenterY + newDist * sin(angle);

        // Constrain the new coordinates to be within the bounds of the original image
        // This prevents trying to access pixels outside the image, which would cause errors.
        newX = constrain(newX, 0, img.width - 1);
        newY = constrain(newY, 0, img.height - 1);

        // Calculate the index for the pixel at the new coordinates in the image's pixels array
        // We use floor() because pixel coordinates must be integers.
        let imgIndex = (floor(newX) + floor(newY) * img.width) * 4;

        // Copy the color from the original image at the new coordinates to the current canvas pixel
        pixels[canvasIndex] = img.pixels[imgIndex];       // Red
        pixels[canvasIndex + 1] = img.pixels[imgIndex + 1]; // Green
        pixels[canvasIndex + 2] = img.pixels[imgIndex + 2]; // Blue
        pixels[canvasIndex + 3] = img.pixels[imgIndex + 3]; // Alpha
      } else {
        // If the pixel is outside the morphing radius, just copy the original image pixel
        // Check if the canvas pixel (x, y) is within the bounds of the image
        if (x < img.width && y < img.height) {
          let imgIndex = (x + y * img.width) * 4;
          pixels[canvasIndex] = img.pixels[imgIndex];
          pixels[canvasIndex + 1] = img.pixels[imgIndex + 1];
          pixels[canvasIndex + 2] = img.pixels[imgIndex + 2];
          pixels[canvasIndex + 3] = img.pixels[imgIndex + 3];
        } else {
          // If the canvas pixel is outside the image bounds (e.g., if canvas is larger than image),
          // fill it with the background color.
          pixels[canvasIndex] = 220;
          pixels[canvasIndex + 1] = 220;
          pixels[canvasIndex + 2] = 220;
          pixels[canvasIndex + 3] = 255;
        }
      }
    }
  }

  // Apply all the pixel changes to the canvas
  updatePixels();
}
Line-by-line explanation (34 lines)

🔧 Subcomponents:

calculation Audio and time reactivity let level = amplitude.getLevel(); let timeFactor = sin(frameCount * 0.05); let audioMorphFactor = map(level, 0, 1, -0.7, 0.7); let timeMorphFactor = map(timeFactor, -1, 1, -0.1, 0.1); let morphFactor = audioMorphFactor + timeMorphFactor;

Calculates how strongly the warp effect should bulge based on microphone audio and time-based pulsing

for-loop Nested pixel iteration for (let y = 0; y < height; y++) { for (let x = 0; x < width; x++) {

Iterates through every pixel on the canvas in row-major order (top to bottom, left to right)

conditional Warp radius check if (dist < morphRadius) {

Determines whether the current pixel is inside the warp radius and should be displaced

calculation Coordinate transformation let angle = atan2(dy, dx); let newX = morphCenterX + newDist * cos(angle); let newY = morphCenterY + newDist * sin(angle);

Converts polar coordinates (distance and angle) to Cartesian (x, y) to find the warped pixel location

background(220);
Clears the canvas with a light gray color, erasing the previous frame so you see only the current warp state
loadPixels();
Loads the canvas's pixel data into the pixels array, making it ready for direct manipulation. This must be called before reading or writing pixels
img.loadPixels();
Loads the original image's pixel data into img.pixels, so you can read color values from it
let level = amplitude.getLevel();
Reads the current volume level from the microphone (0.0 is silent, 1.0 is loudest). This drives the main warp intensity
let timeFactor = sin(frameCount * 0.05);
Creates a smooth oscillation that ranges from -1 to 1 over time. frameCount increases by 1 each frame, and 0.05 controls the speed
let morphCenterX = mouseX;
The center of the warping effect follows your mouse's X position horizontally
let morphCenterY = mouseY;
The center of the warping effect follows your mouse's Y position vertically
let baseRadius = map(mouseX, 0, width, 50, 300);
The radius of the warp effect grows and shrinks based on mouse X - when the mouse is at the left edge, radius is 50 pixels; at the right edge, 300 pixels
let audioMorphFactor = map(level, 0, 1, -0.7, 0.7);
Converts the audio level (0 to 1) into a displacement strength (-0.7 to 0.7). Silence gives -0.7 (pinch), loud noise gives 0.7 (bulge)
let timeMorphFactor = map(timeFactor, -1, 1, -0.1, 0.1);
Adds a subtle continuous ripple to the warp by converting the time oscillation (-1 to 1) into a displacement (-0.1 to 0.1)
let morphFactor = audioMorphFactor + timeMorphFactor;
Combines audio and time reactivity - the total warp strength is the sum of both effects
for (let y = 0; y < height; y++) {
Outer loop iterates through every row of pixels from top (0) to bottom (height)
for (let x = 0; x < width; x++) {
Inner loop iterates through every column in the current row, processing each pixel left to right
let canvasIndex = (x + y * width) * 4;
Calculates the array index of the current pixel. Pixels are stored in a flat array with 4 values per pixel (RGBA), so multiply by 4
let dx = x - morphCenterX;
Calculates the horizontal distance from the current pixel to the warp center (negative if pixel is left of center)
let dy = y - morphCenterY;
Calculates the vertical distance from the current pixel to the warp center (negative if pixel is above center)
let dist = sqrt(dx * dx + dy * dy);
Uses the Pythagorean theorem to calculate the straight-line distance from the pixel to the warp center
if (dist < morphRadius) {
Checks if the pixel is inside the circular warp radius. If true, the pixel will be warped; if false, it copies directly from the image
let normalizedDist = dist / morphRadius;
Scales the distance to a 0-to-1 range, where 0 is the warp center and 1 is the edge of the radius. This is used to fade the effect from center to edge
let displacement = sin(normalizedDist * PI) * morphFactor * morphRadius;
Creates a smooth bulge/pinch using a sine curve. At center (normalizedDist=0), sin(0)=0 so no displacement. At edge (normalizedDist=1), sin(PI)=0 so it fades out. Peak displacement is at the middle of the radius
let newDist = dist + displacement;
Adds the displacement to the distance, pushing pixels outward (if morphFactor is positive) or inward (if negative)
let angle = atan2(dy, dx);
Calculates the angle (in radians) from the warp center to the current pixel. atan2 is preferred because it handles all four quadrants correctly
let newX = morphCenterX + newDist * cos(angle);
Converts polar coordinates (newDist, angle) back to Cartesian: multiply the new distance by cosine of the angle to get the X offset from center
let newY = morphCenterY + newDist * sin(angle);
Completes the polar-to-Cartesian conversion: multiply the new distance by sine of the angle to get the Y offset from center
newX = constrain(newX, 0, img.width - 1);
Clamps newX to stay within the image bounds (0 to image width minus 1). Prevents accessing pixels that don't exist
newY = constrain(newY, 0, img.height - 1);
Clamps newY to stay within the image bounds (0 to image height minus 1)
let imgIndex = (floor(newX) + floor(newY) * img.width) * 4;
Calculates the pixel index in the image's pixels array at the warped coordinates. floor() converts decimal coordinates to integers
pixels[canvasIndex] = img.pixels[imgIndex];
Copies the red channel (first of 4 values) from the warped location in the original image to the canvas pixel
pixels[canvasIndex + 1] = img.pixels[imgIndex + 1];
Copies the green channel (second value) from the image to the canvas
pixels[canvasIndex + 2] = img.pixels[imgIndex + 2];
Copies the blue channel (third value) from the image to the canvas
pixels[canvasIndex + 3] = img.pixels[imgIndex + 3];
Copies the alpha (transparency) channel (fourth value) from the image to the canvas
if (x < img.width && y < img.height) {
Checks if the canvas pixel at (x, y) is within the image's bounds. If canvas is larger than image, pixels outside need special handling
let imgIndex = (x + y * img.width) * 4;
For pixels outside the warp radius but inside the image, calculate the index in the image's pixels array using unwarped coordinates
updatePixels();
Applies all the pixel changes made to the pixels array back to the canvas, making them visible on screen

windowResized()

windowResized() is called automatically by p5.js whenever the browser window changes size. Without it, your canvas would stay at its original size instead of filling the screen. The pixelDensity() call is necessary after resizing to maintain pixel-perfect accuracy on high-resolution displays.

function windowResized() {
  resizeCanvas(windowWidth, windowHeight);
  // Re-apply pixelDensity(1) after resizing
  pixelDensity(1);
}
Line-by-line explanation (2 lines)
resizeCanvas(windowWidth, windowHeight);
Automatically resizes the canvas to match the new browser window dimensions when the user resizes their window
pixelDensity(1);
Re-applies the pixelDensity setting after resizing, ensuring high-DPI displays continue to map correctly

📦 Key Variables

img p5.Image

Stores the loaded image data that will be warped and displayed. Created in preload() and read in draw()

let img; // declared globally, assigned in preload()
mic p5.AudioIn

Represents the microphone input device. Started when the user clicks, then continuously captures audio

let mic; // declared globally, initialized in setup()
amplitude p5.Amplitude

Analyzes the microphone audio to extract the overall volume level (0.0 to 1.0) each frame

let amplitude; // declared globally, initialized in setup()
level number

Holds the current amplitude reading from the microphone (0.0 = silent, 1.0 = loudest) in each frame

let level = amplitude.getLevel();
timeFactor number

A time-based oscillation value that ranges from -1 to 1, creating continuous pulsing independent of audio

let timeFactor = sin(frameCount * 0.05);
morphCenterX number

The X-coordinate of the warp effect's center, which follows the mouse cursor

let morphCenterX = mouseX;
morphCenterY number

The Y-coordinate of the warp effect's center, which follows the mouse cursor

let morphCenterY = mouseY;
baseRadius number

The radius of the circular warp zone in pixels, controlled by mouse X position (50 to 300)

let baseRadius = map(mouseX, 0, width, 50, 300);
morphFactor number

The combined strength of the warp effect (-0.7 to 0.7), blending audio reactivity and time oscillation

let morphFactor = audioMorphFactor + timeMorphFactor;
dist number

The Euclidean distance from the current pixel to the warp center, used to determine if it's in the warp zone

let dist = sqrt(dx * dx + dy * dy);

🔧 Potential Improvements (5)

Here are some ways this code could be enhanced:

PERFORMANCE draw() - nested pixel loop

The double for-loop processes every pixel on the canvas every frame. On a large display (e.g., 4K), this could be over 33 million pixels per frame, causing slow performance on lower-end devices.

💡 Consider rendering to a lower-resolution offscreen buffer (using createGraphics) and scaling it up, or limit the warp effect to a smaller region instead of the entire canvas. Alternatively, use GPU-accelerated shaders if available.

BUG draw() - boundary handling

If the image is smaller than the canvas, pixels outside the image bounds are filled with gray (220), creating a gray border. This might not match the aesthetic if the image should be centered or if the canvas should show something else outside.

💡 Either ensure the image matches the canvas size, or add logic to center the image on the canvas and handle oversized canvases differently (e.g., by displaying only the image region or extending the image edges).

STYLE draw() - variable organization

Multiple variables are calculated before the pixel loop (morphCenterX, morphCenterY, baseRadius, audioMorphFactor, etc.) but only used inside. The draw function has grown complex and could be easier to understand if the warp-radius-check logic were extracted into a helper function.

💡 Create a helper function like `sampleWarpedPixel(x, y)` that takes a canvas pixel location and returns the warped RGBA values. This would make draw() much shorter and the warp logic reusable and testable.

FEATURE setup()

The microphone starts silent; users may not realize they need to click to activate it. There is a console.log message, but it only appears in the browser console - most users won't see it.

💡 Add an on-screen visual indicator (text or a button UI element) that says 'Click to enable microphone' or changes to 'Mic active' after clicking. This would improve user experience significantly.

BUG preload() and setup()

If the image file 'paul-on-tractor-pencil-pixels.jpg' does not exist, preload() will fail silently or throw an error that may be hard to debug. There is no fallback or error checking.

💡 Add error handling: check if img loaded successfully, or provide a fallback image. Alternatively, generate a simple placeholder shape if the image fails to load, so the sketch still runs.

🔄 Code Flow

Code flow showing preload, setup, mousepressed, draw, windowresized

💡 Click on function names in the diagram to jump to their code

graph TD start[Start] --> setup[setup] setup --> canvas-setup[Canvas Initialization] setup --> audio-setup[Audio System Setup] audio-setup --> mic-check[Microphone Guard] mic-check --> draw[draw loop] draw --> pixel-loop[Nested Pixel Iteration] pixel-loop --> warp-conditional[Warp Radius Check] warp-conditional --> polar-to-cartesian[Coordinate Transformation] polar-to-cartesian --> pixel-loop draw --> windowresized[windowResized] click setup href "#fn-setup" click draw href "#fn-draw" click canvas-setup href "#sub-canvas-setup" click audio-setup href "#sub-audio-setup" click mic-check href "#sub-mic-check" click pixel-loop href "#sub-pixel-loop" click warp-conditional href "#sub-warp-conditional" click polar-to-cartesian href "#sub-polar-to-cartesian"

❓ Frequently Asked Questions

What visual effects does the InterActiveImagetest sketch produce?

The sketch transforms a pencil-style image into a dynamic field of pixels that bulge and ripple in response to mouse movement and audio input.

How can users interact with the InterActiveImagetest sketch?

Users can click to activate their microphone, then move their mouse and speak to sculpt the image in real-time through their voice.

What creative coding concepts are demonstrated in the InterActiveImagetest sketch?

This sketch showcases real-time pixel manipulation and audio-reactive visuals, utilizing the p5.js library for sound analysis and interactive graphics.

Preview

InterActiveImagetest - p5.js creative coding sketch preview
Sketch Preview
Code flow diagram showing the structure of InterActiveImagetest - Code flow showing preload, setup, mousepressed, draw, windowresized
Code Flow Diagram