Voice Canvas - AI Art Director - XeLseD

This sketch turns spoken words into abstract generative art: you click a microphone button, describe a scene like 'ocean waves' or 'fire dance', and the sketch sends your words to OpenAI to choose a visual style, color, speed, and density, then animates that description on screen and speaks a description back to you.

Open in Editor Remix View in Gallery

🧪 Try This!

Experiment with the code by making these changes:

Supersize the particles — Bumping up the fixed diameter turns tiny dots into bold overlapping blobs.
Thicken the wavy lines — Increasing strokeWeight makes every line in the lines visual mode much bolder.
Start in circles mode — Changing the default visualType makes the sketch open with concentric circles instead of waves, before any voice command runs.

Prefer the full editor? Open it there →

📖 About This Sketch

Voice Canvas lets you talk to a canvas: press the microphone button, describe something like 'slow green circles' or 'fast vibrant waves', and the sketch reshapes itself into flowing waves, drifting particles, concentric circles, or wiggling lines that match your words, then reads its own creation back to you out loud. It is built on the browser's Web Speech API for speech-to-text, a fetch() call to OpenAI's chat completions endpoint to translate free-text descriptions into structured drawing parameters, and OpenAI's text-to-speech endpoint played through p5.sound for the spoken reply. Underneath the AI plumbing, the actual drawing is done with classic p5.js techniques: beginShape()/vertex() for waves and lines, sin() and noise() for organic motion, and a simple switch-case to pick which generative art function runs each frame.

The code is organized around one shared visualParams object (visualType, color, speed, count) that every drawing function reads from, so changing that object instantly changes what draw() renders. setup() wires up the microphone button and configures speech recognition event handlers; interpretSpeech() sends the recognized words to OpenAI and validates the JSON it gets back; speakBack() asks OpenAI to synthesize speech and plays it; and four small helper functions (drawWaves, drawParticles, drawCircles, drawLines) each turn visualParams into a different animated shape. Studying this sketch teaches how to combine external APIs, asynchronous JavaScript (async/await and fetch), and everyday p5.js drawing loops into one interactive experience.

⚙️ How It Works

When the page loads, preload() decodes an obfuscated API key and setup() creates a full-window canvas, hooks the HTML microphone button to startListening(), and configures a webkitSpeechRecognition object with handlers for when listening starts, produces a result, errors out, or ends
Every frame, draw() clears the background to black and then calls a switch statement that runs one of drawWaves(), drawParticles(), drawCircles(), or drawLines() depending on the current visualParams.visualType
When you click the mic button, startListening() starts the browser's speech recognizer; once you stop talking, the onresult handler collects the final transcript and passes it to interpretSpeech()
interpretSpeech() sends your words inside a carefully worded prompt to OpenAI's chat completions API, asking for a JSON object with visualType, color, speed, and count, then validates and clamps whatever comes back before overwriting the global visualParams
Because draw() reads visualParams every frame, the visuals change immediately and continuously animate using sin() (for waves and lines) or noise() (for particles) driven by frameCount and the AI-chosen speed
Finally interpretSpeech() calls speakBack(), which asks OpenAI's text-to-speech endpoint to generate an audio description of the new art and plays it back using p5.sound's loadSound()

🎓 Concepts You'll Learn

Web Speech API (speech recognition)Async/await and fetch() for API callsJSON parsing and validationText-to-speech with p5.soundswitch-case for state-driven renderingsin() wave animationPerlin noise motionXOR cipher obfuscation

📝 Code Breakdown

`getApiKey()`

This function reverses a lightweight two-step obfuscation (Base64 + XOR) that was used to hide the OpenAI API key from a casual glance at the source code. It is NOT real security - anyone can open the browser console and call getApiKey() directly to read the key, which is why shipping API keys in client-side code is considered unsafe.

function getApiKey() {
  return atob(encoded).split('').map(c => String.fromCharCode(c.charCodeAt(0) ^ key)).join('');
}

Line-by-line explanation (4 lines)

🔧 Subcomponents:

calculation Decode Chain atob(encoded).split('').map(c => String.fromCharCode(c.charCodeAt(0) ^ key)).join('');

Base64-decodes the stored string, then XORs every character with a fixed key to reveal the original API key text

atob(encoded): atob() decodes a Base64-encoded string back into regular text - this reverses the first layer of obfuscation
.split(''): Turns the decoded string into an array of single characters so each one can be transformed individually
.map(c => String.fromCharCode(c.charCodeAt(0) ^ key)): For every character, gets its numeric character code, flips bits using XOR (^) against the fixed 'key' value, then converts the result back into a character - this reverses a simple XOR cipher
.join(''): Glues the transformed characters back together into a single string - the final decoded API key

`preload()`

preload() is a special p5.js function that runs once before setup(), designed for loading assets that other code depends on. Here it's repurposed to make sure the API key is ready before any speech recognition or drawing happens.

function preload() {
  // Decode the API key before setup
  openaiApiKey = getApiKey();
  console.log("OpenAI API Key decoded.");
}

Line-by-line explanation (2 lines)

openaiApiKey = getApiKey();: Runs the XOR decoding function once, before setup(), and stores the plain-text API key in the global openaiApiKey variable
console.log("OpenAI API Key decoded.");: Logs a confirmation message to the browser console so you can verify the decoding step ran

`setup()`

setup() runs once when the sketch starts. Here it does double duty: preparing the p5.js canvas AND wiring up a browser-native Web Speech API object with several event handlers, which is a common pattern when combining p5.js with other web APIs.

🔬 This loop only keeps results marked isFinal. What happens if recognition.interimResults is set to true above and you also append the non-final results here - would the transcript update word by word as you speak?

      for (let i = event.resultIndex; i < event.results.length; ++i) {
        if (event.results[i].isFinal) {
          finalTranscript += event.results[i][0].transcript;
        }
      }

function setup() {
  createCanvas(windowWidth, windowHeight);
  background(0); // Dark background
  noStroke(); // Default to no stroke for most visuals

  // Create and position microphone button
  micButton = select('#micButton');
  micButton.mousePressed(startListening);

  // Create and position transcript div
  transcriptDiv = select('#transcriptDiv');
  transcriptDiv.html(transcriptText);

  // Initialize Web Speech API
  if ('webkitSpeechRecognition' in window) {
    recognition = new webkitSpeechRecognition();
    recognition.continuous = false; // Listen for a single phrase
    recognition.interimResults = false; // Only get final results
    recognition.lang = 'en-US'; // Set language

    recognition.onstart = function() {
      micButton.html('Listening...');
      micButton.attribute('disabled', ''); // Disable button while listening
      transcriptText = "Listening...";
      transcriptDiv.html(transcriptText);
      console.log("Speech recognition started.");
    };

    recognition.onresult = function(event) {
      let finalTranscript = '';
      for (let i = event.resultIndex; i < event.results.length; ++i) {
        if (event.results[i].isFinal) {
          finalTranscript += event.results[i][0].transcript;
        }
      }
      transcriptText = "You said: " + finalTranscript;
      transcriptDiv.html(transcriptText);
      console.log("Speech recognized: " + finalTranscript);
      interpretSpeech(finalTranscript);
    };

    recognition.onerror = function(event) {
      console.error("Speech recognition error: ", event.error);
      transcriptText = "Error: " + event.error;
      transcriptDiv.html(transcriptText);
      micButton.html('Start Listening');
      micButton.removeAttribute('disabled');
      if (event.error === 'no-speech') {
        transcriptText = "No speech detected. Please try again.";
      } else if (event.error === 'not-allowed') {
        transcriptText = "Microphone access denied. Please allow access in your browser settings.";
      }
      transcriptDiv.html(transcriptText);
    };

    recognition.onend = function() {
      micButton.html('Start Listening');
      micButton.removeAttribute('disabled');
      console.log("Speech recognition ended.");
      if (!speaking && transcriptText.startsWith("You said:")) {
        transcriptText = "Processing speech...";
        transcriptDiv.html(transcriptText);
      }
    };
  } else {
    transcriptText = "Web Speech API (webkitSpeechRecognition) not supported in this browser. Please use Chrome.";
    transcriptDiv.html(transcriptText);
    micButton.attribute('disabled', '');
    console.warn(transcriptText);
  }
}

Line-by-line explanation (9 lines)

🔧 Subcomponents:

calculation Microphone Button Wiring micButton.mousePressed(startListening);

Connects the HTML button click to the startListening() function that begins speech recognition

event-handler onstart Handler recognition.onstart = function() { ... }

Updates the button text and transcript display the moment the browser starts listening

event-handler onresult Handler recognition.onresult = function(event) { ... }

Collects the final recognized words from the speech event and sends them to interpretSpeech()

for-loop Collect Final Results for (let i = event.resultIndex; i < event.results.length; ++i) {

Loops through recognition results and appends only the ones marked as final (isFinal) to build the complete transcript

conditional onerror Handler if (event.error === 'no-speech') { ... } else if (event.error === 'not-allowed') { ... }

Shows a friendlier message depending on whether no speech was detected or microphone permission was denied

conditional Browser Support Check if ('webkitSpeechRecognition' in window) { ... } else { ... }

Only sets up speech recognition if the browser supports it, otherwise disables the mic button and shows a warning

createCanvas(windowWidth, windowHeight);: Creates a canvas that fills the entire browser window
micButton = select('#micButton');: Grabs the HTML button element with id 'micButton' so p5.js can attach behavior to it
micButton.mousePressed(startListening);: Tells p5.js to call startListening() whenever the mic button is clicked
recognition = new webkitSpeechRecognition();: Creates a new speech recognition object using the browser's built-in Web Speech API
recognition.continuous = false;: Tells the recognizer to stop listening after one phrase instead of listening forever
recognition.lang = 'en-US';: Sets the language the recognizer expects to hear
recognition.onresult = function(event) { ... };: Defines what happens once speech has been converted to text - here it builds the final transcript and calls interpretSpeech()
recognition.onerror = function(event) { ... };: Handles cases like no speech detected or microphone permission denied, updating the on-screen message accordingly
recognition.onend = function() { ... };: Runs when listening stops for any reason, re-enabling the mic button so the user can try again

`draw()`

draw() is the animation loop that runs continuously (about 60 times per second). Using a switch-case on a state variable like visualParams.visualType is a common pattern for letting one draw() function render completely different visuals depending on the app's current mode.

🔬 This case runs drawWaves() whenever visualType is 'waves'. What happens if you swap in drawCircles() here so saying anything wave-related actually draws circles instead?

    case 'waves':
      drawWaves();
      break;

function draw() {
  background(0); // Clear background each frame

  // Render art based on current parameters
  switch (visualParams.visualType) {
    case 'waves':
      drawWaves();
      break;
    case 'particles':
      drawParticles();
      break;
    case 'circles':
      drawCircles();
      break;
    case 'lines':
      drawLines();
      break;
    default:
      drawWaves(); // Fallback
      break;
  }
}

Line-by-line explanation (4 lines)

🔧 Subcomponents:

switch-case Visual Type Switch switch (visualParams.visualType) { ... }

Chooses which generative art function to run this frame based on the AI-selected visualType string

background(0); // Clear background each frame: Repaints the whole canvas black every frame, erasing the previous frame so shapes don't leave permanent trails
switch (visualParams.visualType) {: Checks the current value of visualParams.visualType (a string like 'waves' or 'circles') and jumps to the matching case
case 'waves':: If visualType is 'waves', runs drawWaves() and then break exits the switch
default:: If visualType doesn't match any known case (e.g. an unexpected value from OpenAI), falls back to drawing waves so the sketch never shows a blank screen

`startListening()`

This function is the bridge between the HTML button click and the Web Speech API. It's intentionally tiny - all the interesting behavior lives in the event handlers (onstart, onresult, onerror, onend) configured back in setup().

function startListening() {
  if (recognition) {
    recognition.start();
  }
}

Line-by-line explanation (2 lines)

🔧 Subcomponents:

conditional Recognition Availability Check if (recognition) {

Only tries to start listening if the speech recognition object was successfully created in setup()

if (recognition) {: Checks that the recognition object exists (it won't if the browser doesn't support webkitSpeechRecognition), preventing an error if you click the button on an unsupported browser
recognition.start();: Tells the browser to begin listening to the microphone; this triggers the onstart handler defined in setup()

`interpretSpeech()`

This function shows the full round-trip of talking to a text AI: build a prompt, send it with fetch(), await the response, parse the JSON it returns, and - critically - validate every field before trusting it in your own program. Never assume an AI's output is perfectly formed.

🔬 This code refuses to trust OpenAI's answer blindly. What happens if you add a fifth allowed value like 'spirals' to the array on the first line, even though there's no drawSpirals() function - which case in draw()'s switch would end up running?

    visualParams.visualType = ['waves', 'particles', 'circles', 'lines'].includes(parsedParams.visualType) ? parsedParams.visualType : 'waves';
    visualParams.color = /^#([0-9A-Fa-f]{3}){1,2}$/.test(parsedParams.color) ? parsedParams.color : defaultColors[floor(random(defaultColors.length))];

async function interpretSpeech(speechText) {
  transcriptText = "Asking OpenAI...";
  transcriptDiv.html(transcriptText);

  const prompt = `You are an art generator assistant... Now, generate parameters for: "${speechText}"`;

  try {
    const response = await fetch('https://api.openai.com/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${openaiApiKey}`
      },
      body: JSON.stringify({
        model: 'gpt-3.5-turbo',
        messages: [{
          role: 'user',
          content: prompt
        }],
        response_format: { type: "json_object" },
        temperature: 0.7,
        max_tokens: 150
      })
    });

    if (!response.ok) {
      const errorData = await response.json();
      throw new Error(`OpenAI API error: ${response.status} - ${errorData.error.message || 'Unknown error'}`);
    }

    const data = await response.json();
    const openAIResponseContent = data.choices[0].message.content;

    let parsedParams;
    try {
      parsedParams = JSON.parse(openAIResponseContent);
    } catch (e) {
      console.error("Failed to parse JSON from OpenAI:", e);
      transcriptText = "OpenAI returned invalid JSON. Trying again...";
      transcriptDiv.html(transcriptText);
      return;
    }

    visualParams.visualType = ['waves', 'particles', 'circles', 'lines'].includes(parsedParams.visualType) ? parsedParams.visualType : 'waves';
    visualParams.color = /^#([0-9A-Fa-f]{3}){1,2}$/.test(parsedParams.color) ? parsedParams.color : defaultColors[floor(random(defaultColors.length))];
    visualParams.speed = constrain(parsedParams.speed, 0.1, 5);
    visualParams.count = constrain(floor(parsedParams.count), 10, 200);

    transcriptText = `OpenAI says: Visual Type: ${visualParams.visualType}, Color: ${visualParams.color}, Speed: ${visualParams.speed}, Count: ${visualParams.count}`;
    transcriptDiv.html(transcriptText);

    speakBack(visualParams);

  } catch (error) {
    console.error("Error communicating with OpenAI:", error);
    transcriptText = "Error: Could not get art parameters from OpenAI. Please check your API key and try again. " + error.message;
    transcriptDiv.html(transcriptText);
    micButton.removeAttribute('disabled');
  }
}

Line-by-line explanation (9 lines)

🔧 Subcomponents:

conditional Outer Try/Catch try { ... } catch (error) { ... }

Wraps the entire network request so any failure (bad key, network error, API error) is caught and shown to the user instead of crashing the sketch

conditional Response OK Check if (!response.ok) { ... }

Detects when OpenAI returns an HTTP error status and throws a descriptive error instead of trying to use invalid data

conditional JSON Parse Try/Catch try { parsedParams = JSON.parse(openAIResponseContent); } catch (e) { ... }

Safely attempts to parse OpenAI's text response as JSON, bailing out gracefully if the AI didn't return valid JSON

calculation Parameter Validation

visualParams.visualType = ['waves', 'particles', 'circles', 'lines'].includes(parsedParams.visualType) ? parsedParams.visualType : 'waves';

Checks each value from the AI against safe limits before trusting it, protecting the sketch from bad or malicious data

const prompt = `You are an art generator assistant...`;: Builds a detailed instruction string that tells OpenAI exactly what JSON shape to return and gives it several examples, then inserts the user's spoken words at the end
const response = await fetch('https://api.openai.com/v1/chat/completions', { ... });: Sends an HTTP POST request to OpenAI's chat API and pauses this function (thanks to await) until a response arrives, without freezing the rest of the sketch
response_format: { type: "json_object" },: Tells OpenAI to guarantee its reply is valid JSON, which makes the next parsing step more reliable
if (!response.ok) { ... }: Checks the HTTP status code; if it's not in the success range, reads the error details and throws an exception that the outer catch block will handle
parsedParams = JSON.parse(openAIResponseContent);: Converts the text OpenAI sent back into a real JavaScript object so its properties (visualType, color, speed, count) can be read
visualParams.visualType = ['waves', 'particles', 'circles', 'lines'].includes(parsedParams.visualType) ? parsedParams.visualType : 'waves';: Only accepts visualType if it's one of the four known strings, otherwise silently falls back to 'waves' - this prevents the switch-case in draw() from ever seeing an unexpected value
visualParams.color = /^#([0-9A-Fa-f]{3}){1,2}$/.test(parsedParams.color) ? parsedParams.color : defaultColors[floor(random(defaultColors.length))];: Uses a regular expression to confirm the color is a real hex code like #RRGGBB; if not, picks a random color from the defaultColors fallback list
visualParams.speed = constrain(parsedParams.speed, 0.1, 5);: Clamps the speed value so it always stays between 0.1 and 5, even if OpenAI returns something out of range
speakBack(visualParams);: Once the new parameters are locked in, triggers the text-to-speech function to describe the new artwork out loud

`speakBack()`

This function demonstrates loading and playing dynamically generated audio in p5.js: converting a fetch() response into a Blob, turning that Blob into a playable URL with URL.createObjectURL(), and using p5.sound's loadSound()/onended() to control playback and clean up memory afterward.

🔬 This guard stops overlapping speech. What happens if you remove it entirely and speak two commands quickly in a row - do the audio clips talk over each other?

  if (speaking) {
    console.log("Speech synthesis is already in progress.");
    return;
  }

async function speakBack(params) {
  if (speaking) {
    console.log("Speech synthesis is already in progress.");
    return;
  }

  const textToSpeak = `Generating art with visual type ${params.visualType}, color ${params.color}, speed ${params.speed}, and count ${params.count}.`;
  console.log("Speaking back art description using OpenAI TTS...");

  speaking = true;

  try {
    const response = await fetch('https://api.openai.com/v1/audio/speech', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${openaiApiKey}`
      },
      body: JSON.stringify({
        model: 'tts-1',
        input: textToSpeak,
        voice: 'nova',
        response_format: 'mp3',
        speed: 1.0
      })
    });

    if (!response.ok) {
      const errorData = await response.json();
      throw new Error(`OpenAI TTS API error: ${response.status} - ${errorData.error.message || 'Unknown error'}`);
    }

    const audioBlob = await response.blob();
    const audioUrl = URL.createObjectURL(audioBlob);

    const sound = loadSound(audioUrl, () => {
      sound.play();
      console.log("OpenAI TTS audio started playing.");
    }, (error) => {
      console.error("p5.sound loadSound error:", error);
      speaking = false;
      URL.revokeObjectURL(audioUrl);
    });

    sound.onended(() => {
      speaking = false;
      console.log("OpenAI TTS audio finished playing.");
      URL.revokeObjectURL(audioUrl);
    });

  } catch (error) {
    console.error("Error communicating with OpenAI TTS:", error);
    transcriptText = "Error: Could not speak back art parameters from OpenAI. " + error.message;
    transcriptDiv.html(transcriptText);
    speaking = false;
  }
}

Line-by-line explanation (8 lines)

🔧 Subcomponents:

conditional Overlap Guard if (speaking) { ... return; }

Prevents starting a new spoken response while a previous one is still playing

conditional TTS Try/Catch try { ... } catch (error) { ... }

Wraps the text-to-speech network request so playback failures don't crash the sketch and reset the speaking flag properly

if (speaking) { console.log("Speech synthesis is already in progress."); return; }: Uses the global 'speaking' boolean as a lock, so if audio is already playing, this function exits immediately instead of overlapping two spoken responses
const textToSpeak = `Generating art with visual type ${params.visualType}, color ${params.color}, speed ${params.speed}, and count ${params.count}.`;: Builds a plain sentence describing the new art parameters, which will be converted to audio
speaking = true;: Locks the flag before starting the request so no second call can start speaking at the same time
const response = await fetch('https://api.openai.com/v1/audio/speech', { ... });: Sends the text to OpenAI's text-to-speech endpoint and waits for an audio response
const audioBlob = await response.blob();: Reads the response body as raw binary audio data (a Blob) rather than as text or JSON
const audioUrl = URL.createObjectURL(audioBlob);: Creates a temporary browser URL that points to the in-memory audio data, so it can be loaded like any other audio file
const sound = loadSound(audioUrl, () => { sound.play(); ... });: Uses p5.sound's loadSound() to load the temporary audio URL, then plays it once loading succeeds
sound.onended(() => { speaking = false; ... URL.revokeObjectURL(audioUrl); });: When playback finishes, unlocks the speaking flag and frees the temporary URL from memory to avoid a memory leak

`drawWaves()`

This function turns a single mathematical curve (sin()) into a filled, animated shape using beginShape()/vertex()/endShape(CLOSE) - a core p5.js technique for drawing custom polygons point by point instead of using built-in shapes.

🔬 This loop draws each point of the wave using sin(). What happens if you change the 0.01 multiplier on frameCount to 0.1 - does the wave animate much faster?

  for (let i = 0; i < visualParams.count; i++) {
    let x = i * waveWidth;
    let y = height / 2 + sin(frameCount * 0.01 * visualParams.speed + i * 0.1) * waveHeight;
    vertex(x, y);
  }

function drawWaves() {
  fill(visualParams.color);
  let waveHeight = height / 4;
  let waveWidth = width / (visualParams.count - 1);

  beginShape();
  vertex(0, height); // Start from bottom-left
  for (let i = 0; i < visualParams.count; i++) {
    let x = i * waveWidth;
    let y = height / 2 + sin(frameCount * 0.01 * visualParams.speed + i * 0.1) * waveHeight;
    vertex(x, y);
  }
  vertex(width, height); // Go to bottom-right
  endShape(CLOSE);
}

Line-by-line explanation (5 lines)

🔧 Subcomponents:

for-loop Wave Points Loop for (let i = 0; i < visualParams.count; i++) {

Adds one vertex per point across the width of the canvas, each offset vertically by a sine wave to create the wave shape

let waveHeight = height / 4;: Determines how tall the wave's peaks and troughs are, based on a quarter of the canvas height
let waveWidth = width / (visualParams.count - 1);: Spaces the wave's points evenly across the full width of the canvas depending on how many points (count) there are
beginShape(); vertex(0, height);: Starts a custom shape and adds its first point at the bottom-left corner, so the final shape can be filled as a solid mountain-like silhouette
let y = height / 2 + sin(frameCount * 0.01 * visualParams.speed + i * 0.1) * waveHeight;: Calculates each point's height using a sine wave: frameCount makes it animate over time, visualParams.speed scales that animation speed, and i * 0.1 offsets each point so the wave ripples across the screen instead of moving as one flat line
vertex(width, height); endShape(CLOSE);: Closes the shape at the bottom-right corner and connects back to the start, filling in the area under the wave

`drawParticles()`

This function demonstrates noise() as an alternative to random() for organic motion - by feeding it slowly changing time values, each particle's position drifts smoothly across the canvas frame by frame instead of teleporting randomly.

🔬 Perlin noise makes particles drift smoothly instead of jumping like random() would. What happens if you replace both noise() calls with random(1) - do the particles start flickering randomly every frame instead of drifting?

    let x = (noise(i * 0.01, frameCount * 0.005 * visualParams.speed) * width);
    let y = (noise(i * 0.02, frameCount * 0.005 * visualParams.speed) * height);
    circle(x, y, 10);

function drawParticles() {
  fill(visualParams.color);
  for (let i = 0; i < visualParams.count; i++) {
    let x = (noise(i * 0.01, frameCount * 0.005 * visualParams.speed) * width);
    let y = (noise(i * 0.02, frameCount * 0.005 * visualParams.speed) * height);
    circle(x, y, 10);
  }
}

Line-by-line explanation (3 lines)

🔧 Subcomponents:

for-loop Particle Drawing Loop for (let i = 0; i < visualParams.count; i++) {

Draws one particle per iteration, positioning it with Perlin noise so it drifts smoothly instead of jumping randomly

let x = (noise(i * 0.01, frameCount * 0.005 * visualParams.speed) * width);: Uses Perlin noise (a smooth pseudo-random function) with two inputs - a per-particle seed (i * 0.01) and a slowly changing time value - then scales the 0-1 noise result to fit the canvas width, giving each particle a smoothly drifting x position
let y = (noise(i * 0.02, frameCount * 0.005 * visualParams.speed) * height);: Same idea for the y position, but with a different seed multiplier (0.02) so the x and y movement don't look identical
circle(x, y, 10);: Draws a small filled circle at the calculated position with a fixed diameter of 10 pixels

`drawCircles()`

This is one of the simplest generative patterns in the sketch: a for-loop drawing the same shape at increasing sizes from a fixed point. It's a good example of how count from visualParams directly controls visual density.

🔬 This loop always centers circles on the canvas middle. What happens if you replace width / 2 and height / 2 with mouseX and mouseY so the rings follow your cursor?

  for (let i = 0; i < visualParams.count; i++) {
    let radius = (i + 1) * circleSpacing;
    let x = width / 2;
    let y = height / 2;
    circle(x, y, radius * 2);
  }

function drawCircles() {
  fill(visualParams.color);
  let maxRadius = min(width, height) / 3;
  let circleSpacing = maxRadius / visualParams.count;

  for (let i = 0; i < visualParams.count; i++) {
    let radius = (i + 1) * circleSpacing;
    let x = width / 2;
    let y = height / 2;
    circle(x, y, radius * 2);
  }
}

Line-by-line explanation (4 lines)

🔧 Subcomponents:

for-loop Concentric Rings Loop for (let i = 0; i < visualParams.count; i++) {

Draws progressively larger circles from the same center point, creating a ripple/target pattern

let maxRadius = min(width, height) / 3;: Caps how big the largest circle can get, based on whichever is smaller - the canvas width or height - divided by 3
let circleSpacing = maxRadius / visualParams.count;: Divides the maximum radius evenly across however many circles (count) are being drawn, so they fit neatly inside maxRadius
let radius = (i + 1) * circleSpacing;: Each loop iteration's circle is one 'ring' bigger than the last, since i increases by 1 each time
circle(x, y, radius * 2);: Draws a circle centered on the canvas; circle()'s third argument is diameter, so the radius is doubled

`drawLines()`

drawLines() combines a nested for-loop with beginShape()/vertex()/endShape() to build multiple independent wavy strokes, showing how the same sine-wave technique used in drawWaves() can be reused for an open, unfilled line pattern.

🔬 The trailing *20 controls how far each line wiggles up and down. What happens if you change it to *100?

    for (let x = 0; x <= width; x += 10) {
      let offset = sin(frameCount * 0.02 * visualParams.speed + x * 0.01) * 20;
      vertex(x, y + offset);
    }

function drawLines() {
  stroke(visualParams.color);
  strokeWeight(2);
  noFill();

  let lineCount = visualParams.count;
  let lineSpacing = height / (lineCount + 1);

  for (let i = 0; i < lineCount; i++) {
    let y = (i + 1) * lineSpacing;
    beginShape();
    for (let x = 0; x <= width; x += 10) {
      let offset = sin(frameCount * 0.02 * visualParams.speed + x * 0.01) * 20;
      vertex(x, y + offset);
    }
    endShape();
  }
}

Line-by-line explanation (5 lines)

🔧 Subcomponents:

for-loop Line Rows Loop for (let i = 0; i < lineCount; i++) {

Draws one horizontal wavy line per iteration, evenly spaced down the canvas

for-loop Line Points Loop for (let x = 0; x <= width; x += 10) {

Builds each line out of small segments every 10 pixels, offsetting each point vertically with a sine wave to create a wiggle

let lineSpacing = height / (lineCount + 1);: Evenly spaces however many lines (lineCount) fit within the canvas height, with a small gap at top and bottom
let y = (i + 1) * lineSpacing;: Sets the baseline vertical position for this particular line before wiggling is applied
let offset = sin(frameCount * 0.02 * visualParams.speed + x * 0.01) * 20;: Calculates a vertical wiggle amount using sine, animated over time (frameCount) and varying across the line's width (x * 0.01), scaled to a maximum of 20 pixels
vertex(x, y + offset);: Adds a point to the current line's shape at its wiggled vertical position
endShape();: Finishes drawing this line as an open (non-closed) shape, since noFill() and no CLOSE argument mean only the stroke is visible

`windowResized()`

windowResized() is a built-in p5.js callback that fires automatically on browser resize. Pairing it with resizeCanvas() is the standard way to make a full-window sketch stay responsive.

function windowResized() {
  resizeCanvas(windowWidth, windowHeight);
  background(0); // Clear background after resize
}

Line-by-line explanation (2 lines)

resizeCanvas(windowWidth, windowHeight);: Automatically called by p5.js whenever the browser window changes size; this resizes the canvas to match the new window dimensions
background(0); // Clear background after resize: Repaints the canvas black immediately after resizing so there's no leftover blank or stretched pixels before the next draw() call

📦 Key Variables

encoded string

A Base64-encoded, XOR-obfuscated version of the OpenAI API key, stored as plain text in the source

const encoded = 'KTF3Kig1MHcIaDhqCms+OA0A...';

key number

The fixed XOR key (0x5A) used to scramble and unscramble the API key

const key = 0x5A;

openaiApiKey string

Holds the decoded, usable OpenAI API key after getApiKey() runs in preload()

let openaiApiKey;

micButton object

Reference to the HTML microphone button element, used to change its text and enabled state

let micButton;

transcriptDiv object

Reference to the HTML div that displays status messages and recognized speech to the user

let transcriptDiv;

transcriptText string

Stores the current status/message text shown in transcriptDiv, updated at every stage of listening and AI processing

let transcriptText = "Click 'Start Listening' and describe your art!";

recognition object

Holds the browser's webkitSpeechRecognition instance used to convert spoken audio into text

let recognition;

speaking boolean

A lock flag preventing speakBack() from starting a new spoken response while one is already playing

let speaking = false;

visualParams object

The single source of truth for what the canvas currently looks like - visualType, color, speed, and count - read by draw() every frame and rewritten whenever OpenAI returns new parameters

let visualParams = { visualType: 'waves', color: '#FFFFFF', speed: 1, count: 50 };

defaultColors array

A list of fallback hex colors used when OpenAI's returned color fails hex-code validation

const defaultColors = ['#FFFFFF', '#FF0000', '#00FF00', '#0000FF', '#FFFF00', '#FF00FF', '#00FFFF'];

🔧 Potential Improvements (4)

Here are some ways this code could be enhanced:

BUG getApiKey() / global scope

The OpenAI API key is only lightly obfuscated with a fixed-key XOR cipher and Base64, then shipped in client-side JavaScript - anyone can open dev tools, run getApiKey(), and steal the real key.

💡 Move all OpenAI calls to a backend server or serverless function that holds the real key privately, and have this sketch call your own endpoint instead of api.openai.com directly.

BUG setup() -> recognition.onend

The onend handler checks if (!speaking && transcriptText.startsWith('You said:')) to decide whether to show 'Processing speech...', but interpretSpeech() usually already changes transcriptText to 'Asking OpenAI...' before onend fires, so this fallback rarely runs and can leave stale UI text.

💡 Track state explicitly with a dedicated boolean like isProcessing instead of comparing substrings of a UI message.

FEATURE interpretSpeech()

If OpenAI returns malformed JSON, the function logs an error and returns early, but never re-enables the microphone button, leaving it disabled until the page is reloaded.

💡 Call micButton.removeAttribute('disabled') in the invalid-JSON catch branch so users can immediately try speaking again.

PERFORMANCE drawWaves() / drawLines()

waveWidth is computed as width / (visualParams.count - 1), which will throw a division-by-zero (Infinity) if OpenAI or a future edit ever sets count to 1.

💡 Guard against count <= 1, for example by using max(visualParams.count, 2) when computing waveWidth.

🔄 Code Flow

Code flow showing getapikey, preload, setup, draw, startlistening, interpretspeech, speakback, drawwaves, drawparticles, drawcircles, drawlines, windowresized

💡 Click on function names in the diagram to jump to their code

graph TD start[Start] --> setup[setup] setup --> preload[preload] setup --> micsetup[mic-setup] setup --> browsercheck[browser-support-check] browsercheck -->|Supported| startlistening[startlistening] browsercheck -->|Not Supported| warning[Show Warning] setup --> draw[draw loop] click setup href "#fn-setup" click preload href "#fn-preload" click micsetup href "#sub-mic-setup" click browsercheck href "#sub-browser-support-check" click startlistening href "#fn-startlistening" draw --> visualswitch[visual-switch] visualswitch --> drawwaves[drawwaves] visualswitch --> drawparticles[drawparticles] visualswitch --> drawcircles[drawcircles] visualswitch --> drawlines[drawlines] click draw href "#fn-draw" click visualswitch href "#sub-visual-switch" click drawwaves href "#fn-drawwaves" click drawparticles href "#fn-drawparticles" click drawcircles href "#fn-drawcircles" click drawlines href "#fn-drawlines" drawwaves --> wavepointsloop[wave-points-loop] wavepointsloop --> drawwaves click wavepointsloop href "#sub-wave-points-loop" drawparticles --> particleloop[particle-loop] particleloop --> drawparticles click particleloop href "#sub-particle-loop" drawcircles --> ringsloop[rings-loop] ringsloop --> drawcircles click ringsloop href "#sub-rings-loop" drawlines --> linesouterloop[lines-outer-loop] linesouterloop --> linesinnerloop[lines-inner-loop] linesinnerloop --> drawlines click linesouterloop href "#sub-lines-outer-loop" click linesinnerloop href "#sub-lines-inner-loop" startlistening --> onstart[onstart-handler] onstart --> draw click onstart href "#sub-onstart-handler" startlistening --> onresult[onresult-handler] onresult --> resultloop[result-loop] resultloop --> interpretspeech[interpretspeech] click onresult href "#sub-onresult-handler" click resultloop href "#sub-result-loop" click interpretspeech href "#fn-interpretspeech" interpretspeech --> outertrycatch[outer-try-catch] outertrycatch --> responsecheck[response-check] responsecheck --> jsonparsetry[json-parse-try] jsonparsetry --> sanitizeparams[sanitize-params] click outertrycatch href "#sub-outer-try-catch" click responsecheck href "#sub-response-check" click jsonparsetry href "#sub-json-parse-try" click sanitizeparams href "#sub-sanitize-params" interpretspeech --> speakback[speakback] speakback --> speakingguard[speaking-guard] speakingguard --> ttstrycatch[tts-try-catch] click speakback href "#fn-speakback" click speakingguard href "#sub-speaking-guard" click ttstrycatch href "#sub-tts-try-catch"

❓ Frequently Asked Questions

What kind of visuals does the Voice Canvas - AI Art Director sketch generate?

The sketch creates abstract visuals based on user descriptions, such as ocean waves or starry nights, using AI interpretation.

How do users interact with the Voice Canvas sketch?

Users can click the microphone button to speak their desired art description, which the AI then transforms into visuals.

What creative coding techniques are showcased in this p5.js sketch?

This sketch demonstrates the integration of the Web Speech API and OpenAI for real-time voice recognition and AI-generated art.

Preview

Code flow diagram showing the structure of Voice Canvas - AI Art Director - XeLseD - Code flow showing getapikey, preload, setup, draw, startlistening, interpretspeech, speakback, drawwaves, drawparticles, drawcircles, drawlines, windowresized — Code Flow Diagram

🧪 Try This!

📖 About This Sketch

⚙️ How It Works

🎓 Concepts You'll Learn

📝 Code Breakdown

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

🔧 Subcomponents:

📦 Key Variables

🔧 Potential Improvements (4)

🔄 Code Flow

❓ Frequently Asked Questions

Explore More

Voice Aurora - AI Sound Visualizer - xelsed.ai

king of ai chatbots v2

BOB ai

talking ben

p5js.ai

Gallery

User Guide

Preview