BOB ai

Ultra Smart Bob is an interactive AI character with a dynamic, morphing face that breathes and tracks your mouse. Type messages or hold-click to speak, and Bob responds with clever conversation, math calculations, and witty replies—all customizable through voice pitch, speed, and realism sliders.

🧪 Try This!

Experiment with the code by making these changes:

  1. Give Bob blue eyes — Change the eye color from white to a blue tone, even when morphing to realistic mode.
  2. Make Bob's breathing more dramatic — Increase both the breathing speed and amplitude so Bob's head bobs up and down more noticeably.
  3. Speed up Bob's default speech
  4. Add a new conversational rule — Add a rule that makes Bob respond to 'What is your name?' before the existing rules so it matches first.
  5. Start with realism at 50%
  6. Change the default greeting message — The first thing Bob says is stored in botText—customize it to greet users differently.
Prefer the full editor? Open it there →

📖 About This Sketch

This sketch creates a living, breathing AI character named Bob whose face morphs from a simple geometric shape into a realistic human face as you adjust the realism slider. Bob listens when you click and hold the canvas, speaks responses using the Web Speech API, and his eyes follow your mouse while his mouth animates in sync with his speech. The sketch combines speech synthesis and recognition, dynamic shape morphing, animation loops, and DOM interaction—making it a playground for learning how creative code can feel alive.

The code is organized into a setup() function that initializes the canvas and DOM elements, a draw() function that renders Bob's morphing face and breathing animation on every frame, and several helper functions that handle speech recognition, natural language processing, and voice synthesis. By studying it, you'll learn how to blend p5.js graphics with browser APIs, use lerp() to smoothly transition between visual states, manipulate colors and shapes based on numeric sliders, and create the illusion of personality through animation timing and reactive behavior.

⚙️ How It Works

  1. When the sketch loads, setup() creates a full-screen canvas, initializes WebRTC speech recognition, pre-loads available system voices to avoid lag, and links all DOM elements (sliders, buttons, checkboxes) so they can control Bob's appearance and voice.
  2. Every frame, draw() clears the black background and renders Bob's face centered on screen. His vertical position gently oscillates if breathing is enabled, and his eyes can track the mouse if that option is checked.
  3. The realism slider (0–100%) controls Bob's entire visual transformation: at 0%, he's a simple grey geometric shape; at 100%, he has skin tone, hair, detailed eyes with irises, eyebrows that animate during speech, a realistic mouth with inner darkness, and a neck and body.
  4. When you hold-click the canvas, mousePressed() starts speech recognition, and Bob listens until you release, displaying "RECORDING..." on screen. The speech-to-text engine converts your words into a string.
  5. That string is sent to processUserInput(), a mini NLP engine that matches your input against regex patterns (questions about Bob's name, math problems, emotional statements, jokes) and generates an appropriate response, storing your name if you introduce yourself.
  6. Finally, speakText() converts Bob's reply into speech using the Web Speech API, setting pitch and speed from the sliders, and isSpeaking is toggled so his mouth animates open and closed in rhythm with the spoken words.

🎓 Concepts You'll Learn

Speech Synthesis (Web Speech API)Speech Recognition (WebRTC SpeechRecognition)Shape Morphing with lerp()Color Transitions with lerpColor()Event Listeners and DOM ManipulationRegular Expressions for Pattern MatchingAnimation Loop and Trigonometric MotionState Management (isSpeaking, isListening, isThinking)

📝 Code Breakdown

setup()

setup() runs once when the sketch loads. It's your chance to initialize the canvas size, load resources (in this case, speech voices), connect to HTML elements, and set up event listeners so the sketch can respond to user interaction.

function setup() {
  createCanvas(windowWidth, windowHeight);
  setupSpeechRecognition();
  
  availableVoices = window.speechSynthesis.getVoices();
  if (speechSynthesis.onvoiceschanged !== undefined) {
    speechSynthesis.onvoiceschanged = () => {
      availableVoices = window.speechSynthesis.getVoices();
    };
  }
  
  textAlign(CENTER, CENTER);
  rectMode(CENTER);
  noStroke();

  chatInput = document.getElementById('chat-input');
  sendBtn = document.getElementById('send-btn');
  settingsToggle = document.getElementById('settings-toggle');
  settingsPanel = document.getElementById('settings-panel');
  pitchSlider = document.getElementById('slider-pitch');
  speedSlider = document.getElementById('slider-speed');
  realismSlider = document.getElementById('slider-realism');
  
  checkBreathe = document.getElementById('check-breathe');
  checkTrack = document.getElementById('check-track');

  sendBtn.addEventListener('click', handleChatSubmit);
  chatInput.addEventListener('keypress', function(e) {
    if (e.key === 'Enter') handleChatSubmit();
  });
  
  settingsToggle.addEventListener('click', () => {
    settingsPanel.style.display = (settingsPanel.style.display === 'block') ? 'none' : 'block';
  });

  pitchSlider.addEventListener('input', () => document.getElementById('pitch-val').innerText = pitchSlider.value);
  speedSlider.addEventListener('input', () => document.getElementById('speed-val').innerText = speedSlider.value);
  realismSlider.addEventListener('input', () => document.getElementById('realism-val').innerText = realismSlider.value + "%");
}
Line-by-line explanation (9 lines)

🔧 Subcomponents:

initialization Voice Pre-loading availableVoices = window.speechSynthesis.getVoices();

Caches system voices before they're needed so speech synthesis doesn't lag on first use

initialization DOM Element Linking chatInput = document.getElementById('chat-input');

Connects JavaScript variables to HTML elements so sliders and buttons can control the sketch

event-listener Event Listener Binding sendBtn.addEventListener('click', handleChatSubmit);

Attaches click handlers to buttons and keyboard events so user input triggers speech processing

createCanvas(windowWidth, windowHeight);
Creates a canvas that fills the entire browser window, making Bob the focus of the screen
setupSpeechRecognition();
Calls a helper function to initialize the Web Speech API so Bob can listen to your voice
availableVoices = window.speechSynthesis.getVoices();
Retrieves all voices installed on the system before they're needed, preventing lag when Bob first speaks
textAlign(CENTER, CENTER);
Sets text alignment to center both horizontally and vertically, so all text is centered around its position
rectMode(CENTER);
Changes how rectangles are drawn—when you draw a rect, its position is now its center, not its top-left corner
chatInput = document.getElementById('chat-input');
Finds the HTML input field and stores it in a variable so JavaScript can read what the user types
sendBtn.addEventListener('click', handleChatSubmit);
When the user clicks the Send button, the handleChatSubmit function runs
chatInput.addEventListener('keypress', function(e) { if (e.key === 'Enter') handleChatSubmit(); });
Lets users press Enter in the text input to submit instead of clicking the button
settingsToggle.addEventListener('click', () => { settingsPanel.style.display = (settingsPanel.style.display === 'block') ? 'none' : 'block'; });
Toggles the settings menu open and closed when the user clicks the gear icon

draw()

draw() is the heartbeat of the sketch—it runs 60 times per second and handles all visual updates. This is where Bob's face is rendered, morphed, and animated. The key technique here is using the realism slider (r) as a master control for interpolating between two entire visual styles: geometric and realistic. Every shape, color, and size uses lerp() or lerpColor() to blend between two states based on r, letting a single slider control Bob's entire appearance.

🔬 These three lines control how eyes morph from geometric to realistic. What happens if you swap the two numbers in the first line, making it lerp(60, 80, r) so eyes get BIGGER as realism increases instead of smaller?

  let eyeW = lerp(80, 60, r);
  let eyeBaseH = lerp(80, 40, r);
  let eyeRadius = r * 40;

🔬 This code colors the eyes and draws them as rectangles. What color would Bob's eyes be if you changed 'whiteColor' to a blue like color(0, 150, 255)? Try it and see what realistic-Bob looks like with blue eyes.

  let currentEyeColor = lerpColor(baseGrey, whiteColor, r);
  fill(currentEyeColor);
  rect(lEyeX, lEyeY, eyeW, eyeH, eyeRadius);
  rect(rEyeX, rEyeY, eyeW, eyeH, eyeRadius);
function draw() {
  background(0); 
  
  push();
  fill(150);
  textSize(14);
  textAlign(LEFT, TOP);
  text("©CDWW", 15, 15);
  pop();
  
  let cx = width / 2;
  let cy = height / 2 - 30;

  if (checkBreathe.checked) {
    cy += sin(frameCount * 0.03) * 10; 
  }

  let lookX = 0;
  let lookY = 0;
  if (checkTrack.checked) {
    let targetX = map(mouseX, 0, width, -15, 15, true);
    let targetY = map(mouseY, 0, height, -10, 10, true);
    lookX = lerp(lookX, targetX, 0.1);
    lookY = lerp(lookY, targetY, 0.1);
  }

  fill(255);
  textSize(22);
  text(botText, cx, cy - 180);

  textSize(16);
  if (isListening) {
    fill(255, 100, 100);
    text("🎤 RECORDING... (Release to send!)", cx, cy - 220);
  } else if (isThinking) {
    fill(255, 200, 100);
    text("🧠 Processing data...", cx, cy - 220);
  }

  let r = parseFloat(realismSlider.value) / 100;
  
  let baseGrey = color(150);
  let skinColor = color(255, 218, 185); 
  let lipColor = color(210, 105, 110);  
  let whiteColor = color(255);          
  
  let headAlpha = r * 255;

  if (r > 0) {
    fill(red(skinColor)-20, green(skinColor)-20, blue(skinColor)-20, headAlpha);
    rect(cx, cy + 150, 100, 120, 20); 

    fill(red(skinColor), green(skinColor), blue(skinColor), headAlpha);
    ellipse(cx, cy + 10, 280, 360);
    
    ellipse(cx - 145, cy + 10, 40, 60);
    ellipse(cx + 145, cy + 10, 40, 60);

    fill(80, 50, 30, headAlpha); 
    arc(cx, cy - 80, 300, 220, PI, 0, CHORD);
    rect(cx - 140, cy - 40, 30, 80, 10); 
    rect(cx + 140, cy - 40, 30, 80, 10);
  }

  let eyeW = lerp(80, 60, r);
  let eyeBaseH = lerp(80, 40, r);
  let eyeRadius = r * 40; 
  
  let eyeH = (frameCount % 200 < 10) ? 5 : eyeBaseH;
  
  let lEyeX = cx - 100 + lookX;
  let lEyeY = cy - 50 + lookY;
  let rEyeX = cx + 100 + lookX;
  let rEyeY = cy - 50 + lookY;

  let currentEyeColor = lerpColor(baseGrey, whiteColor, r);
  fill(currentEyeColor);
  rect(lEyeX, lEyeY, eyeW, eyeH, eyeRadius);
  rect(rEyeX, rEyeY, eyeW, eyeH, eyeRadius);

  if (r > 0 && eyeH > 10) {
    fill(50, 150, 200, headAlpha);
    circle(lEyeX, lEyeY, 30 * r);
    circle(rEyeX, rEyeY, 30 * r);
    
    fill(0, 0, 0, headAlpha);
    circle(lEyeX, lEyeY, 15 * r);
    circle(rEyeX, rEyeY, 15 * r);
  }

  if (r > 0.2) {
    fill(80, 50, 30, headAlpha);
    let browOffset = isSpeaking ? sin(frameCount * 0.1) * 5 : 0;
    rect(lEyeX, lEyeY - 40 - browOffset, 70 * r, 10 * r, 5);
    rect(rEyeX, rEyeY - 40 - browOffset, 70 * r, 10 * r, 5);
  }

  if (r > 0.1) {
    fill(red(skinColor)-30, green(skinColor)-30, blue(skinColor)-30, headAlpha);
    triangle(cx, cy - 10, cx - 15 * r, cy + 30 * r, cx + 15 * r, cy + 30 * r);
  }

  let speedFactor = parseFloat(speedSlider.value);
  let maxMouthOpen = lerp(80, 40, r); 
  
  if (isSpeaking) {
    mouthOpen = map(sin(frameCount * (1.5 * speedFactor)), -1, 1, 10, maxMouthOpen); 
  } else {
    mouthOpen = 10; 
  }
  
  let mouthW = lerp(80, 70, r);
  let currentMouthColor = lerpColor(baseGrey, lipColor, r);
  
  fill(currentMouthColor);
  rect(cx, cy + 90, mouthW, mouthOpen, r * 20);

  if (r > 0.5 && mouthOpen > 15) {
    fill(0, 0, 0, (r - 0.5) * 2 * 255);
    rect(cx, cy + 90, mouthW * 0.8, mouthOpen * 0.6, r * 10);
  }
}
Line-by-line explanation (13 lines)

🔧 Subcomponents:

conditional Breathing Animation cy += sin(frameCount * 0.03) * 10;

Makes Bob's entire face bob gently up and down using a sine wave, simulating breathing when the checkbox is enabled

calculation Mouse Eye Tracking let targetX = map(mouseX, 0, width, -15, 15, true);

Converts mouse position into an offset that makes Bob's eyes look toward where the user is pointing

calculation Realism Slider Morphing let r = parseFloat(realismSlider.value) / 100;

Converts the slider value (0–100) into a decimal (0–1) used to blend all geometric and color transitions

conditional Body Parts Rendering if (r > 0) { fill(...); rect(...); ellipse(...); }

Only draws realistic body parts (neck, shoulders, hair, ears) when realism is above 0%

calculation Eye Shape Morphing let eyeW = lerp(80, 60, r);

Smoothly transitions eye width from 80 pixels (geometric) to 60 pixels (realistic) as realism increases

calculation Mouth Open Animation mouthOpen = map(sin(frameCount * (1.5 * speedFactor)), -1, 1, 10, maxMouthOpen);

Uses a sine wave to make Bob's mouth open and close in rhythm with his speech, speed controlled by the voice speed slider

background(0);
Clears the canvas to black every frame, erasing the previous frame's drawing so you see smooth motion instead of trails
let cx = width / 2; let cy = height / 2 - 30;
Calculates the center position of Bob's face: horizontally centered, and slightly above vertical center
if (checkBreathe.checked) { cy += sin(frameCount * 0.03) * 10; }
If breathing is enabled, adds a sine wave oscillation to Bob's vertical position, making him bob gently up and down
let targetX = map(mouseX, 0, width, -15, 15, true);
Converts the mouse X position (0 to canvas width) into a range of -15 to 15, determining how far left/right Bob's eyes should look
lookX = lerp(lookX, targetX, 0.1);
Smoothly interpolates the current eye offset toward the target offset by 10% each frame, making eye tracking feel natural instead of jerky
text(botText, cx, cy - 180);
Displays Bob's current message (response, greeting, or status) in white text above his head
if (isListening) { fill(255, 100, 100); text("🎤 RECORDING...", cx, cy - 220); }
When Bob is listening to your voice, displays a red microphone icon and tells you to release to send
let r = parseFloat(realismSlider.value) / 100;
Extracts the realism slider value (0–100) and converts it to a decimal (0–1) for use in all morphing calculations
let headAlpha = r * 255;
Creates an alpha (transparency) value that increases with realism, so realistic body parts fade in smoothly
let eyeW = lerp(80, 60, r);
Smoothly transitions eye width from 80 (geometric squares) to 60 (realistic ovals) as realism increases from 0 to 1
let eyeH = (frameCount % 200 < 10) ? 5 : eyeBaseH;
Makes Bob blink every 200 frames for 10 frames by temporarily shrinking eye height to 5 pixels, then returning to normal
let currentEyeColor = lerpColor(baseGrey, whiteColor, r);
Transitions eye color from grey (geometric) to white (realistic) based on realism, using lerpColor to smoothly blend between them
if (isSpeaking) { mouthOpen = map(sin(frameCount * (1.5 * speedFactor)), -1, 1, 10, maxMouthOpen); }
When Bob is speaking, his mouth height oscillates between 10 and maxMouthOpen pixels in sync with a sine wave, faster if voice speed is higher

mousePressed()

mousePressed() fires once when the user clicks anywhere on the page. In this sketch, it detects canvas clicks to start voice recording ('push-to-talk' style), handles the easter egg, and cancels speech if Bob is talking. Notice the try-catch block around recognition.start()—this prevents the code from crashing if speech recognition fails on the user's browser.

🔬 This code starts speech recognition only if Bob is not already listening. What would happen if you removed the '!isListening &&' part? Try changing it to just 'if (recognition)' and see if Bob gets confused trying to listen multiple times.

  if (!isListening && recognition) {
    try {
      recognition.start();
      isListening = true;
      botText = "Listening...";
    } catch (e) { }
  }
function mousePressed(event) {
  if (event.target.tagName !== 'CANVAS') return;

  if (mouseX >= 10 && mouseX <= 80 && mouseY >= 10 && mouseY <= 35) {
    if (isSpeaking) window.speechSynthesis.cancel();
    let secretMsg = "yeah now you know not to just remix stuff.";
    botText = secretMsg;
    isThinking = false;
    speakText(secretMsg);
    return false; 
  }

  if (isSpeaking) window.speechSynthesis.cancel();
  
  if (!isListening && recognition) {
    try {
      recognition.start();
      isListening = true;
      botText = "Listening...";
    } catch (e) { }
  }
}
Line-by-line explanation (5 lines)

🔧 Subcomponents:

conditional Canvas Target Check if (event.target.tagName !== 'CANVAS') return;

Ensures the click happened on the canvas, not on a button or text input, so clicks on UI elements don't trigger Bob to listen

initialization Speech Recognition Start recognition.start(); isListening = true;

Activates the microphone and updates the UI to show Bob is listening

if (event.target.tagName !== 'CANVAS') return;
Checks if you clicked on the canvas; if you clicked on a button or text input instead, this function exits early so Bob doesn't listen
if (mouseX >= 10 && mouseX <= 80 && mouseY >= 10 && mouseY <= 35) {
Checks if your click landed in the small rectangle where the ©CDWW text is drawn, triggering an easter egg
if (isSpeaking) window.speechSynthesis.cancel();
If Bob is already speaking, stops him so he can listen to you instead
recognition.start();
Activates the speech recognition engine so it starts listening for your voice
isListening = true;
Sets the listening flag to true, which causes draw() to display the 'RECORDING...' message

mouseReleased()

mouseReleased() pairs with mousePressed() to implement push-to-talk. When you press and hold, listening starts; when you release, listening stops and the spoken text is processed. This gives users tactile feedback similar to walkie-talkie communication.

function mouseReleased(event) {
  if (event.target.tagName !== 'CANVAS') return;
  if (isListening && recognition) {
    recognition.stop(); 
  }
}
Line-by-line explanation (2 lines)

🔧 Subcomponents:

initialization Speech Recognition Stop recognition.stop();

Stops listening when the user releases the mouse, ending the push-to-talk session

if (event.target.tagName !== 'CANVAS') return;
Only processes the release if it happened on the canvas, not on a DOM element
if (isListening && recognition) { recognition.stop(); }
If Bob is currently listening, stops the speech recognition engine and waits for the onresult callback to process what was heard

setupSpeechRecognition()

setupSpeechRecognition() initializes the Web Speech Recognition API, which converts spoken words into text. This is the 'listening' half of Bob's conversation. The key insight is the difference between interim results (shown in real-time as you speak) and final results (confirmed transcriptions sent to the NLP engine). The onresult callback fires multiple times as the browser gets more confident in its transcription.

🔬 This code separates interim (partial) results from final results. What would happen if you removed the conditional and concatenated everything into finalStr regardless of isFinal status? Your speech might be processed multiple times as Bob changes his mind about what you said.

    recognition.onresult = (event) => {
      let interim = '';
      let finalStr = '';
      for (let i = event.resultIndex; i < event.results.length; ++i) {
        if (event.results[i].isFinal) finalStr += event.results[i][0].transcript;
        else interim += event.results[i][0].transcript;
      }
function setupSpeechRecognition() {
  const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
  if (SpeechRecognition) {
    recognition = new SpeechRecognition();
    recognition.continuous = false;
    recognition.interimResults = true; 
    recognition.lang = 'en-US'; 

    recognition.onresult = (event) => {
      let interim = '';
      let finalStr = '';
      for (let i = event.resultIndex; i < event.results.length; ++i) {
        if (event.results[i].isFinal) finalStr += event.results[i][0].transcript;
        else interim += event.results[i][0].transcript;
      }

      if (interim !== '') botText = 'You: "' + interim + '"';

      if (finalStr !== '') {
        isListening = false;
        isThinking = true;
        botText = "Thinking...";
        setTimeout(() => processUserInput(finalStr), 10);
      }
    };

    recognition.onerror = () => {
      isListening = false;
      isThinking = false;
      botText = "I could not hear you clearly.";
    };
    recognition.onend = () => { isListening = false; };
  }
}
Line-by-line explanation (8 lines)

🔧 Subcomponents:

initialization SpeechRecognition API Setup const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;

Gets the browser's speech recognition API, checking both standard and webkit (Safari) versions for compatibility

event-handler Interim Results Handler if (interim !== '') botText = 'You: "' + interim + '"';

Displays partial speech recognition results in real-time as you're speaking, before the final text is confirmed

event-handler Final Results Handler if (finalStr !== '') { isThinking = true; setTimeout(() => processUserInput(finalStr), 10); }

When speech recognition finishes, flags the thinking state and sends the final transcript to the NLP engine

const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
Gets the browser's speech-to-text API; tries the standard name first, falls back to webkit for Safari compatibility
recognition = new SpeechRecognition();
Creates a new speech recognition instance that will listen to the microphone
recognition.continuous = false;
Tells the recognizer to stop listening after one sentence, not continuously record everything you say
recognition.interimResults = true;
Enables interim results, so you see Bob displaying your words as you speak them, before the final transcription is confirmed
for (let i = event.resultIndex; i < event.results.length; ++i) {
Loops through all new speech results since the last update; this avoids reprocessing old results
if (event.results[i].isFinal) finalStr += event.results[i][0].transcript;
If this result is confirmed (final), adds it to the final string that will be sent to the NLP engine
if (interim !== '') botText = 'You: "' + interim + '"';
If there are partial results, displays them in Bob's message area so the user sees their speech recognized in real-time
setTimeout(() => processUserInput(finalStr), 10);
Waits 10 milliseconds before processing the input, letting the UI update and preventing lag

handleChatSubmit()

handleChatSubmit() is the callback for the Send button and Enter key. It retrieves text from the HTML input field, validates it, clears the field, and passes the text to processUserInput(). The setTimeout creates a brief delay so Bob appears to 'think' for a moment, making the interaction feel more natural.

function handleChatSubmit() {
  let val = chatInput.value.trim();
  if (val !== "") {
    chatInput.value = "";
    isThinking = true;
    botText = "Thinking...";
    setTimeout(() => processUserInput(val), 10);
  }
}
Line-by-line explanation (5 lines)

🔧 Subcomponents:

conditional Empty Input Check if (val !== "") {

Prevents sending empty messages by checking if the input has any non-whitespace characters

let val = chatInput.value.trim();
Gets the text the user typed in the input field and removes any leading/trailing whitespace
if (val !== "") {
Only proceeds if there's actual text to process, preventing empty submissions
chatInput.value = "";
Clears the input field so the user can type a new message
isThinking = true; botText = "Thinking...";
Updates Bob's state to show he's thinking and displays a 'Thinking...' message above his head
setTimeout(() => processUserInput(val), 10);
Waits 10 milliseconds before processing, letting the UI update and giving the appearance of Bob thinking

processUserInput()

processUserInput() is Bob's 'brain'—a simple NLP (natural language processing) engine. It handles three tiers: math detection (for calculations), smart rules (for contextual responses that remember your name and answer questions), and fallbacks (generic responses when no pattern matches). The key technique is using regex patterns and action functions as a lookup table, letting you add new conversational rules by simply adding a new object to the smartRules array. This is a beginner-friendly alternative to machine learning, showing how rule-based systems can create the illusion of intelligence.

🔬 These are the first two rules in Bob's brain. Each rule has a regex pattern (rx) and an action function. Try adding a new rule before these two that detects 'hello' and responds with something custom, like { rx: /hello/, action: () => `Hi! I'm Bob!` }. Where would you insert it?

    const smartRules = [
      { rx: /my name is (.*)/, action: (m) => { userName = m[1]; return `It is a pleasure to meet you, ${userName}. I have saved your name to my memory.`; } },
      { rx: /i am (.*)/, action: (m) => `How long have you been ${m[1]}? Is that a difficult experience?` },
function processUserInput(input) {
  let text = input.toLowerCase().replace(/[?!.]/g, ""); 
  let reply = "";
  
  let mathRegex = /(what is|calculate)?\s*(\d+)\s*(plus|\+|minus|\-|times|\*|divided by|\/)\s*(\d+)/i;
  let mathMatch = text.match(mathRegex);

  if (mathMatch) {
    let num1 = parseFloat(mathMatch[2]);
    let op = mathMatch[3].trim();
    let num2 = parseFloat(mathMatch[4]);
    let res = 0;
    
    if (op === "plus" || op === "+") res = num1 + num2;
    else if (op === "minus" || op === "-") res = num1 - num2;
    else if (op === "times" || op === "*") res = num1 * num2;
    else if (op === "divided by" || op === "/") res = num1 / num2;
    
    reply = `My calculations indicate the answer is ${res}.`;
  } 
  else {
    let foundMatch = false;
    
    const smartRules = [
      { rx: /my name is (.*)/, action: (m) => { userName = m[1]; return `It is a pleasure to meet you, ${userName}. I have saved your name to my memory.`; } },
      { rx: /i am (.*)/, action: (m) => `How long have you been ${m[1]}? Is that a difficult experience?` },
      { rx: /i feel (.*)/, action: (m) => `Why do you feel ${m[1]}? As an AI, I only process data.` },
      { rx: /i want (.*)/, action: (m) => `Will getting ${m[1]} truly bring you happiness?` },
      { rx: /do you like (.*)/, action: (m) => `I lack the capacity to "like" things, but ${m[1]} sounds fascinating.` },
      { rx: /why (.*)/, action: (m) => `The universe is full of mysteries. Why do you think ${m[1]}?` },
      { rx: /(what is|who is) your creator/, action: () => `I was generated by advanced algorithms.` },
      { rx: /what is my name/, action: () => userName ? `You told me earlier that your name is ${userName}.` : `I do not know yet. What is your name?` },
      { rx: /your name/, action: () => `My name is Bob.` },
      { rx: /how are you/, action: () => `My logic circuits are operating at 100% capacity. Thank you for inquiring.` },
      { rx: /(hi|hello|hey|greetings|sup)/, action: () => userName ? `Hello again, ${userName}! How can I assist you?` : `Greetings, human! Feel free to talk to Bob.` },
      { rx: /(who are you|what are you)/, action: () => `I am Bob, a highly advanced artificial construct.` },
      { rx: /(current time|what time is it)/, action: () => { 
          let d = new Date(); 
          return `Based on your system clock, it is currently ${d.getHours()}:${d.getMinutes().toString().padStart(2, '0')}.`; 
      }},
      { rx: /joke/, action: () => `Why did the programmer quit his job? Because he didn't get arrays! Ha. Ha. Ha.` },
      { rx: /awesome|owsome|berry/, action: () => `I'm berry owsome bob` } 
    ];

    for (let rule of smartRules) {
      let match = text.match(rule.rx);
      if (match) {
        reply = rule.action(match);
        foundMatch = true;
        break;
      }
    }

    if (!foundMatch) {
      let fallbacks = [
        `I hear what you are saying about "${input}". Tell me more.`,
        `That is a very human thing to say. I am processing it.`,
        `I am Bob, and I find that very interesting.`,
        `My intelligence is vast, yet "${input}" leaves me contemplating.`,
        `I'm berry owsome bob`
      ];
      reply = random(fallbacks);
    }
  }

  isThinking = false;
  botText = reply;
  speakText(reply);
}
Line-by-line explanation (8 lines)

🔧 Subcomponents:

calculation Text Normalization let text = input.toLowerCase().replace(/[?!.]/g, "");

Converts text to lowercase and strips punctuation so the regex patterns match regardless of user formatting

conditional Math Problem Detection let mathMatch = text.match(mathRegex);

Uses regex to detect arithmetic expressions like '5 plus 3' or '10 * 2' and extract the numbers and operators

conditional Math Calculation if (op === "plus" || op === "+") res = num1 + num2;

Performs the appropriate arithmetic operation based on the detected operator

for-loop Smart Rules Matching for (let rule of smartRules) { let match = text.match(rule.rx); if (match) { reply = rule.action(match); foundMatch = true; break; } }

Loops through a list of regex patterns and generates contextual responses by matching user input to patterns like 'my name is' or 'how are you'

conditional Fallback Response if (!foundMatch) { reply = random(fallbacks); }

If no specific rule matches, picks a random generic response so Bob always has something to say

let text = input.toLowerCase().replace(/[?!.]/g, "");
Normalizes the input by converting to lowercase and removing question marks, exclamation marks, and periods, so patterns match consistently
let mathRegex = /(what is|calculate)?\s*(\d+)\s*(plus|\+|minus|\-|times|\*|divided by|\/)\s*(\d+)/i;
Creates a regex pattern that matches math questions like '5 plus 3', 'what is 10 times 2', or '100 / 5', capturing the numbers and operators separately
let num1 = parseFloat(mathMatch[2]); let op = mathMatch[3].trim(); let num2 = parseFloat(mathMatch[4]);
Extracts the first number, operator, and second number from the regex match groups
if (op === "plus" || op === "+") res = num1 + num2;
Performs the appropriate arithmetic; checks for both word ('plus') and symbol (+) versions of each operator
for (let rule of smartRules) { let match = text.match(rule.rx); if (match) { reply = rule.action(match); foundMatch = true; break; } }
Iterates through smart conversational rules, testing each regex pattern against the input. When a match is found, calls the associated action function to generate a response and breaks out of the loop
let reply = rule.action(match);
Calls the action function for the matched rule, passing in the regex match object so the response can extract captured groups like a person's name
let fallbacks = [...]; reply = random(fallbacks);
If no rule matched, picks a random generic response from the fallback array, ensuring Bob always has a response
isThinking = false; botText = reply; speakText(reply);
Updates Bob's thinking state, displays the reply above his head, and speaks it aloud

speakText()

speakText() is Bob's voice—it converts text to speech using the Web Speech Synthesis API. The key challenge is lag: online voices (which require internet) can pause before speaking. The solution is preferring offline voices, which are pre-installed on the system. The function also wires up the isSpeaking flag so draw() knows when to animate the mouth. Notice the onerror handler catches speech failures gracefully, preventing visual glitches where Bob's mouth gets stuck open.

function speakText(text) {
  window.speechSynthesis.cancel();
  
  let msg = new SpeechSynthesisUtterance(text);
  
  if (availableVoices.length > 0) {
    let fastVoice = availableVoices.find(v => v.lang.startsWith('en') && v.localService);
    if (!fastVoice) fastVoice = availableVoices.find(v => v.lang.startsWith('en'));
    if (fastVoice) msg.voice = fastVoice;
  }
  
  msg.lang = 'en-US'; 
  msg.pitch = parseFloat(pitchSlider.value); 
  msg.rate = parseFloat(speedSlider.value);  
  
  msg.onstart = () => { isSpeaking = true; };
  msg.onend = () => { isSpeaking = false; };
  msg.onerror = () => { isSpeaking = false; }; 
  
  window.speechSynthesis.speak(msg);
}
Line-by-line explanation (9 lines)

🔧 Subcomponents:

initialization Audio Cancellation window.speechSynthesis.cancel();

Stops any currently playing speech before starting new speech, preventing overlapping audio

conditional Voice Selection let fastVoice = availableVoices.find(v => v.lang.startsWith('en') && v.localService);

Preferentially selects a local offline voice to avoid internet latency, falling back to any English voice if necessary

initialization Voice Parameters msg.pitch = parseFloat(pitchSlider.value);

Sets the pitch and rate of the speech from the user's slider settings

event-handler Speech Event Handlers msg.onstart = () => { isSpeaking = true; };

Updates the isSpeaking flag when speech starts and ends, triggering mouth animation in draw()

window.speechSynthesis.cancel();
Clears any pending or currently playing audio so Bob doesn't speak over himself
let msg = new SpeechSynthesisUtterance(text);
Creates a new speech utterance object containing the text that will be spoken
let fastVoice = availableVoices.find(v => v.lang.startsWith('en') && v.localService);
Searches for an offline English voice; offline voices are faster because they don't require internet
if (!fastVoice) fastVoice = availableVoices.find(v => v.lang.startsWith('en'));
If no offline voice was found, falls back to any English voice, even if it's online
msg.pitch = parseFloat(pitchSlider.value);
Reads the pitch slider value from the HTML and applies it to the speech utterance; higher values sound higher-pitched
msg.rate = parseFloat(speedSlider.value);
Reads the speed slider value and applies it; higher values make Bob speak faster and his mouth chew faster
msg.onstart = () => { isSpeaking = true; };
Sets a callback that fires when speech begins; this triggers mouth animation in draw() via the isSpeaking flag
msg.onerror = () => { isSpeaking = false; };
If an error occurs (no speaker, permission denied), resets the isSpeaking flag so the mouth doesn't get stuck open
window.speechSynthesis.speak(msg);
Tells the browser to actually speak the utterance using the system's text-to-speech engine

📦 Key Variables

isSpeaking boolean

Tracks whether Bob is currently speaking, used to animate his mouth and prevent overlapping audio

let isSpeaking = false;
isListening boolean

Tracks whether the speech recognition engine is actively listening for your voice

let isListening = false;
isThinking boolean

Tracks whether Bob is processing user input, used to display 'Thinking...' message

let isThinking = false;
mouthOpen number

Controls the height of Bob's mouth rectangle, animated with a sine wave when speaking

let mouthOpen = 10;
recognition object

Holds the WebRTC SpeechRecognition API instance for converting speech to text

let recognition;
availableVoices array

Stores all system voices to prevent lag when Bob first speaks

let availableVoices = [];
botText string

The current message Bob is displaying or will speak—changes based on state (greeting, response, status)

let botText = "Type below, or hold click on the background to talk!";
userName string

Stores the user's name after they introduce themselves, used to personalize responses

let userName = null;
chatInput object

Reference to the HTML text input field where users type messages

let chatInput;
pitchSlider object

Reference to the HTML range slider controlling Bob's voice pitch

let pitchSlider;
speedSlider object

Reference to the HTML range slider controlling Bob's voice speed and mouth animation rate

let speedSlider;
realismSlider object

Reference to the HTML range slider controlling the morphing between geometric and realistic Bob

let realismSlider;
checkBreathe object

Reference to the HTML checkbox enabling or disabling breathing animation

let checkBreathe;
checkTrack object

Reference to the HTML checkbox enabling or disabling mouse eye tracking

let checkTrack;

🔧 Potential Improvements (6)

Here are some ways this code could be enhanced:

BUG draw() - eye blinking

Eye blink calculation 'let eyeH = (frameCount % 200 < 10) ? 5 : eyeBaseH' doesn't reset properly if realism slider is moved during a blink, causing visual artifacts.

💡 Store the blink state in a boolean variable (e.g., isBlink = frameCount % 200 < 10) to decouple blinking from frame timing, preventing glitches when the slider changes.

PERFORMANCE draw() - color creation

Colors like baseGrey, skinColor, lipColor are created every frame even though they never change. This wastes CPU cycles.

💡 Move color definitions outside draw() to setup(), creating them once instead of 60 times per second: let baseGrey, skinColor, lipColor, whiteColor; in setup(), assign them once.

FEATURE processUserInput() - fallback responses

The fallback responses are generic and repetitive if the user asks Bob repeated off-topic questions.

💡 Add a fallback array that changes based on the userName—if Bob knows the user's name, include personalized fallbacks like 'That's interesting, ${userName}, but tell me more about yourself.'

STYLE speakText() - voice selection logic

The nested if-statements for voice selection are hard to read: 'if (!fastVoice) fastVoice = ...' is a common pattern that could be clearer.

💡 Use the nullish coalescing operator: 'fastVoice = fastVoice ?? availableVoices.find(v => v.lang.startsWith("en"))' to express the fallback intent more idiomatically.

BUG mousePressed() - copyright easter egg detection

The clickable region for the ©CDWW easter egg is hardcoded to a small rectangle; if the text position changes in draw(), the clickable area becomes misaligned.

💡 Calculate the clickable bounds dynamically based on textWidth and textHeight of the copyright text, or use a separate invisible button overlay in HTML.

FEATURE setupSpeechRecognition() - error handling

If the user denies microphone permission, the sketch doesn't display a helpful error or fallback UI; Bob just silently fails to listen.

💡 Add a visual indicator or message in the UI when recognition is unavailable, and disable the listen functionality with a message like 'Microphone permission required' instead of silently failing.

🔄 Code Flow

Code flow showing setup, draw, mousepressed, mousereleased, setupspeechrecognition, handlechatsubmit, processuserinput, speaktext

💡 Click on function names in the diagram to jump to their code

graph TD start[Start] --> setup[setup] setup --> voice-preload[Voice Pre-loading] setup --> dom-linking[DOM Element Linking] setup --> event-setup[Event Listener Binding] setup --> setupspeechrecognition[setupSpeechRecognition] setup --> draw[draw loop] click setup href "#fn-setup" click voice-preload href "#sub-voice-preload" click dom-linking href "#sub-dom-linking" click event-setup href "#sub-event-setup" click setupspeechrecognition href "#fn-setupspeechrecognition" draw --> breathing[Breathing Animation] draw --> eye-tracking[Mouse Eye Tracking] draw --> realism-interpolation[Realism Slider Morphing] draw --> body-rendering[Body Parts Rendering] draw --> eye-morphing[Eye Shape Morphing] draw --> mouth-animation[Mouth Open Animation] draw --> canvas-check[Canvas Target Check] draw --> copyright-easter-egg[Copyright Easter Egg] click draw href "#fn-draw" click breathing href "#sub-breathing" click eye-tracking href "#sub-eye-tracking" click realism-interpolation href "#sub-realism-interpolation" click body-rendering href "#sub-body-rendering" click eye-morphing href "#sub-eye-morphing" click mouth-animation href "#sub-mouth-animation" click canvas-check href "#sub-canvas-check" click copyright-easter-egg href "#sub-copyright-easter-egg" mousepressed[mousePressed] --> canvas-check mousepressed --> listening-start[Speech Recognition Start] mousepressed --> copyright-easter-egg mousepressed --> audio-cancel[Audio Cancellation] click mousepressed href "#fn-mousepressed" click listening-start href "#sub-listening-start" click audio-cancel href "#sub-audio-cancel" mousereleased[mousReleased] --> listening-stop[Speech Recognition Stop] click mousereleased href "#fn-mousereleased" click listening-stop href "#sub-listening-stop" setupspeechrecognition --> speech-recognition-init[SpeechRecognition API Setup] speech-recognition-init --> interim-results[Interim Results Handler] speech-recognition-init --> final-results[Final Results Handler] click speech-recognition-init href "#sub-speech-recognition-init" click interim-results href "#sub-interim-results" click final-results href "#sub-final-results" handlechatsubmit[handleChatSubmit] --> input-validation[Empty Input Check] input-validation --> text-normalization[Text Normalization] text-normalization --> processuserinput[processUserInput] click handlechatsubmit href "#fn-handlechatsubmit" click input-validation href "#sub-input-validation" click text-normalization href "#sub-text-normalization" processuserinput --> math-detection[Math Problem Detection] processuserinput --> rule-matching[Smart Rules Matching] processuserinput --> fallback-responses[Fallback Response] click processuserinput href "#fn-processuserinput" click math-detection href "#sub-math-detection" click rule-matching href "#sub-rule-matching" click fallback-responses href "#sub-fallback-responses" math-detection --> math-calculation[Math Calculation] click math-calculation href "#sub-math-calculation" speaktext[speakText] --> audio-cancel speaktext --> voice-selection[Voice Selection] speaktext --> voice-parameters[Voice Parameters] speaktext --> event-callbacks[Speech Event Handlers] click speaktext href "#fn-speaktext" click voice-selection href "#sub-voice-selection" click voice-parameters href "#sub-voice-parameters" click event-callbacks href "#sub-event-callbacks"

❓ Frequently Asked Questions

What visual experience does the BOB ai sketch provide?

The BOB ai sketch creates a glowing, animated face that breathes and has expressive eyes that track the user's mouse movements.

How can users interact with BOB ai during the sketch?

Users can type in a chat input, click a button to send messages, or hold-click the background to engage in voice conversation with Bob.

What creative coding techniques are demonstrated in the BOB ai sketch?

The sketch showcases dynamic animation, speech recognition, and customizable audio parameters, allowing users to adjust voice pitch, speed, and realism.

Preview

BOB ai - p5.js creative coding sketch preview
Sketch Preview
Code flow diagram showing the structure of BOB ai - Code flow showing setup, draw, mousepressed, mousereleased, setupspeechrecognition, handlechatsubmit, processuserinput, speaktext
Code Flow Diagram