Summary
This week was extremely productive and we learnt quickly through professional workshops. There were many guest speakers involved and we learnt practical, theoretical and commercial practice of AR, VJ and Digital Narratives.
Workshops
During the AudioVisual workshop, we learnt the concept of “visual jockey” (or VJ) and we incorporated this idea into our project. There are different types of visual styles when creating a VJ performance, such as low poly mountains, geometric and real footage. These visual styles can be performed in different audiovisual combinations, and examples can be theatrical performance, live synchronisation or visual projects, cinematic performances and even LED lights.
It was really inspiring to see a live VJ demonstration during the workshop, and reinvigorated our interest and passion for this project. During the demonstration, the VJ taught us how to mix visual and audio sources through a software known as “Resolume Arena 6”
- First create a new composition – new sequence
- Size of canvas is important – how big is the projection? HDMI or full HD
- Fastest MOV – SV3 codec, or else it will lag
- Download codec from website
- Add effects live – drag into the clip
- Audio FFT – sound control
- Detect the MIDI first
- Mapping – edit application MIDI
- Click on application, then click on MIDI
Idea Refinement & Iterations
Since our initial pitch at the start of the week, we made many iterations and refinements to our concept. We started with an ambitious four-screen display installation but we quickly realised we weren’t able to complete a cohesive work, hence the current idea is refined to only include a one-screen interactive display.
After discussing with the visual team, we realised we already had many shots that we can incorporate into the composition. It was a matter of refining and bringing these shots into a cohesive piece that had a strong theme. We had many ideas floating around but it was important to make a clear structure and define deliverables.
Explaining the Set-Up of the Installation
Team Work & Tasks
As we refined the idea, we also mapped out the visual and audio flow of the deliverables. Each flow has two outcomes – a base layer and an effects layer, which will then be complied together in Resolume Arena 6. Our teams have set field tasks to capture footage and sound from various areas of Malaysia. We created a list of all the video and sound recordings that are able to work together, which we will then capture in Kuala Lumpur, Batu Caves, KL Bird Park and various other locations.
Diagram Describing Workflow
My Contribution
As team leader of the visual flow, I involved my team members in a discussion about what types of shots will represent Malaysia and Melbourne, and how we will incorporate these together. Soon, a structure began to form and we added themes together, discussing how our existing footage can fit into this story.
From dusk till dawn, the projection will continue this endless loop until an interaction is inputed from the user. As soon as a button is pressed on the MIDI board, it will trigger a sound and visual reaction, which will overlay onto the existing loop. Our tasks are split into visual and auditory alignments, but working as an agile team, we are not constricted to them and will contribute to providing both contents that will be remixed.
As I have a visual arts background, I utilised my skills to create interesting visual effects that can be digitalised and created into an overlay and VJ content. As part of the experiment, I set up my camera tripod and watercolour to create flowing brush strokes and droplets as an interesting transition for the visual remix.
Watercolour Droplets Experiment
On Friday, there was a sudden downpour of rain, and it was a perfect opportunity to capture footage of this iconic tropical weather phenomenon in Malaysia. I experimented with depth of field and macro compositions, compiling various shots that can be effectively layered on top of the background loop.
Macro Raindrops in Black and White