ོ࿐˖✶⊹₊𖦹 W.12 CID LAB WORKSHOP WITH JXKE TAN OF SERIAL_CO 𖦹₊⊹✶ ࣪˖࿐ོ


DRAW A MEMORY

Inspired by the QUICK DRAW! game by Google that I used to play.

Quick, Draw! is an online guessing game developed and published by Google that challenges players to draw a picture of an object or idea and then uses a neural network artificial intelligence to guess what the drawings represent. The AI learns from each drawing, improving its ability to guess correctly in the future.


"I am thinking of referencing the 'quick draw' game as I feel it is related. If a software is able to correctly guess the shapes the participant draws, it can also produce a sound from the shape with almost perfect accuracy."

  • ✲ Shape Recognition
    ꕀ Use a Drawing Library - (p5.js or Touchdesigner) for a drawing interface
    ꕀIntegrate Machine Learning - (TensorFlow.js) to recognise shapes as they're drawn

  • ✲ Sound Mapping
    ꕀ Sound Association - (pre-defined sound or set of sounds associated with shapes)
    ꕀ Dynamic Feedback - (immediate auditory feedback that corresponds to the drawn shape to feel more intuitive)

  • ✲ Interactive Experience:
    ꕀ Engaging Interface - (create a theme for a more engaging experience where participants can draw their memories or emotions as shapes while software responds by playing relevant sounds)

  • ✲ Technical Implementation
    ꕀ Usng p5.js with ml5.js - (the ml5.js library makes it relatively easy to integrate machine learning into your p5.js sketches)

Quick Draw powered by AI



a collection of duck drawings by 113,764 people

QUICK DRAW
IN P5.JS

adding the ml5.js library in p5.js by

inserting this code into index.html
"src="https://unpkg.com/ml5@1/dist/ml5.min.js">"

However,there is an error saying:
"TypeError: Cannot read properties of undefined because the ml5.js model cannot be identified."

It was a minor issue, but it was the first step toward testing the model. I likely pasted the code incorrectly or in the wrong spot, but with Jake's help, it finally worked.





ERROR: Could not identify ml5.js model

After configuring the errors with Jake, there's finally no error.









DRAWING
CIRCLE, STAR, AND SQUARE

The code works by generating 10 guesses everytime someone draws on the canvas and they are ranked by confidence levels, with 0 being the highest and 9 being the lowest.





ꕀ Jake mentioned that my cable might be the problem to the poor connection of touchboard and my computer so I am planning to get a new adapter.

ꕀ There is error in the code but it is still working in terms of generating guesses based on the drawings.


  • Dumbell, bottlecap, bracelet, necklace and soccerball were pretty accurate since they are all circular
  • Trying a slightly challenging shape and the results weren't anywhere near
  • I also tried drawing two squares overlapping each other and it turned out to produce the most accurate result

  • Linking the Bare Conductive Touchboard to Touchdesigner through selecting the correct Port
  • By adding switches, every node on the Touchboard is able to project a different visual








Moving Forward to Week 13

Read up more articles and journals on topics related to memory, mental wellbeing and sound. Adding in as the 4th pillar in Dissertation under Literature Review.

Consult Andreas about Dissertation on Tuesday.

Configure errors and using Bare Conductive Touchboard with Touchdesigner like projection mapping, during CID Lab Workshop on Thursday.

Readings and Writings
Experimenting & Making
Reflection on Making in Dissertation