ོ࿐˖✶⊹₊𖦹 W.4 RESEARCH PROPOSAL OUTLINE 𖦹₊⊹✶ ࣪˖࿐ོ



Wk.4 - RPO draft 1

ꕀ Building a concept map

We were tasked to create a concept map which should show a network of interconnected concepts and knowledge to get a clearer direction on what to do, instead of a mindmap where we just brainstorm random ideas.

We went around reading each other’s concept map while allocating stickers on areas we were most interested in. From here, I can see that many of my classmates relate to the association with sound and memory and are interested in digital visualisations that respond to music or sounds in VJ or therapeutic settings.

Others were interested in the concept of associative memory: how sound can evoke and communicate memories better than words through multisensory audio-visual experience. There’s also another sticker for translating memory to visual AI readings which I will explore in the next experiment.

In-Class Exercise - Marking areas of interest using dot stickers



    • Writing a timeline to keep myself in check of my progress. After that, each of us was tasked to create a concept map
    for next week which should show a clearer direction, further developing from the mindmap we did earlier.
    • Wk.3-4 notes



ꕀ READINGS: SOUND & MEMORY
Found 2 readings that are helpful and very relevant to my topic. The first journal talks about how sounds can be powerful triggers to memory compared to traditional ways like photographs or texts.

On the other hand, the second journal talks a little deeper about how music is a language on its own, more powerful than words due to its various characteristics.



    • Week 4 - Readings Notes
ꕀ How does a memory sound like to AI?
  • I know there are AI models that generate music based on texts and prompts given. Hence, this idea popped into my mind. What if I describe a specific memory to AI, will the music it generates based on my description sound like my memory? Each try, AI will generate 2 songs, one short version and another long version.
    Participant A’s input ⟢
    "prefer the longer version, perfectly describes how I was happy and then transitioned to frantic panic when I realised it late, it captures the feeling of helpless and sadness during the entire car journey back home”
  • Participant B’s input ⟢
    "longer version is exactly like the memory even though the words are cut short it still gives the right emotions”


    EXP.2 TRANSLATING MEMORY TO AI MUSIC READINGS




ꕀ How does a memory sound like to AI?

✲ After trying out different AI models, I came to a conclusion that the AI Music Factory Model is still the best at recreating memories with instrumental music.

✲ Came across this app called ‘musical memories’ which was interesting but it doesn’t work…. Found another called ‘imaginary soundscape’ but the images generated are not accurate.

✲ Next I used firefly adobe and the realistic photographs worked better, unfortunately it’s missing the sound element.



Also found another method where you make your own noise to generate patterns














NOTE TO SELF

ꕀ FEEDBACKS
  • ⟢ Tell myself to read every day for 3 minutes
  • ⟢ Iteration, it's okay to fail
  • ⟢ Failures are important, it leads us to improvements
  • ⟢ Arduino (capacitive sensor?)

ꕀ MOVING FORWARD
Next experiment ideation:
  • ⟢ recall a specific memory
  • ⟢ visualise that memory
  • ⟢ use lines to represent the sounds they hear
  • ⟢ use colours to represent recalled emotions






INSPO.3 - OLAFUR ELIASSON

ꕀ 'Your curious journey'

✲ Uses ephemeral materials, such as light, wind, fog, and water, to conjure evanescent phenomenal shimmering rainbows, swirling mists, the split-second sculptural form of a spouting fountain, or make invisible elements of our surroundings like air or magnetic fields experienceable.

✲ Spatial design to create immersive experience.

✲ Every ‘room’ I walked into was a pleasant surprise. It felt oddly therapeutic the evaporating water scene and some were mindblowing, especially the mini fountain where I get to see water flow IN FRAMES through my eyes, and lastly the green sky was yet another therapeutic experience.






INSPO.4 - “THE SOUND OF SOUND - COLOURS IN DREAM”

ꕀ By Joo Choon Lin and Colin Justin Wan,
2024, Art Science Museum

✲ Happened to chance upon this interesting looking ‘the dream creator’ machine which uses AI to generate your dreams using texts.

✲ “I was playing with a cute round fluffy black cat in my house. It kept looking up at me with its big eyes.”

✲ However, the result was disappointing.

✲ Cat was not fluffy.. neither was it round...


Generative AI 'The Dream Creator Machine' - Enter descriptions to generate an image of your dream



First experiment features experimental ways of visualising sounds through vibrations applied on water and slime which creates patterns.
Second experiment features an AI generated bird which keeps evolving, exploring the themes of transformation and fluidity.
This was supposed to be a reflection of our dreams, similarly they are constantly changing.






Moving Forward to Week 5

ꕀ NEXT STEP
Formalise my thoughts and develop a workshop or individually, start from the same starting point, me explaining, what i’m trying to gain from it, let them know what I want to analyse.

  • ⟢ How to communicate to someone who don’t know about my work eg. words, videos, website, make a book? Zines?
  • ⟢ How to make it visually more accessible
  • ⟢ How to communicate better, how to link it back to my project?
  • ⟢ Classmate saying my project reminded her of drama therapy?