Assignment One. A recap.
Our project was titled Mencari, the Malay word for ‘searching’. Mencari is an exploration of concepts related to the disuse of a metropolitan environment due to obsessive use of smart devices and personal technology, framed as a ‘choose your own adventure’ style game. The project was a collaboration with two students from Multimedia University. Liyana Rizal was our animator and Ilham Zurith Zulhazri was our Photographer and editor. From RMIT, Yue Yamanaka-Mead was the coder and video editor while Thomas Harmann was the Sound Specialist.
Yue Yamanaka-Mead and Thomas Harman worked on the story line together, Yue acting as the lead writer while Thomas provided several different arcs and acted as an editor and proof reader. The storyline we eventually produced consisted of sixteen possible story lines with six possible endings, four of which were death and repeat, two of which were success and repeat. The story follows a protagonist as they search through the disused environment. The user is only exposed to the nature of the search gradually, beginning the game with the prompt “I must keep searching…”. Eventually the user discovers that the Protagonist is searching for proof of the value of life outside of technology. This is a representative proof, embodied in the success scenes with the discovery of the lotus flower. The lotus flower was chosen as it has a particular meaning in Malaysia and association as the beauty that grows in mud. Poetically speaking, this is important to our story as it represents the ideal forgotten by the characters that have abandoned reality in favour of VR and tech.
The final draft of the narration was completed by Ilham Zurith Zulhazri and spoken in Malay. The original draft of the narration was read in English by Yue Yamanaka-Mead. We found this version less effective in highlighting the dramatisation of the storyline, and Ilham’s naturally contemplative and matter-of-fact mannerisms in his Malay narration style complimented the storyline more.
The installation piece involved projecting our project and using a makey makey to create an interaction between two tools (a hammer and a machete) and the user and the story. The user would pick up and use the tools to make each decision. To create an even more immersive experience, we used blackout curtains and filled the space inside the blackout curtains with disused furniture and rubbish sourced from around Cyberjaya. This transition from gallery to our space allowed the user to feel a physical change which eased them into the mental and perspective change.
Liyana’s animations were stripped back, our group collectively deciding that it should be simple animation that allowed the user to focus on the sound and their own physical responses and sensations. Liyana took her inspiration from Wayang Kulit. Directly translated this means Shadow Skin (skin refers to the leather the puppets are made from), it is the title given to shadow plays, a tradition still observed in many parts of Malaysia and intrinsic to many cultures history all over the world.
In Shadow Play, a narrator tells a story while puppets act out the different scenes. This form of storytelling, with a heavy focus on sound and a creation of interesting visuals through the play between light and dark, was an excellent starting point for our visual planning and had a strong impact on how we decided to frame the story.
Another source of inspiration came from Playdead’s Arnt Jensen’s hugely popular flagship game Limbo. Though the gameplay and story style itself was totally different, the animations employed a similar proportional value (ie leg length, arm length & head size etc.) and used a similar idea of a silhouette using hair style to create the character. This concept is incidentally most famously employed by Matt Groening of The Simpsons.
Ilham Zurith Zulhazri took the photos that we would use for scene setting. These photos were taken around the relatively newly formed city of Cyber Jaya, Malaysia. This city was an attempt at creating a futuristic town with the aim of becoming a vibrant hub (think Bijlmermeer, Holland only with less less focus on humanism). This city, like the Bijlmermeer project, turned out to be far less successful than anticipated and as a result has many abandoned areas and run down environments. This was ideal for us as our work was aimed at exploring the idea of a city lost to technology through disuse. Ilham took a series of high definition photos set in locations that he felt best illustrated the disused environment, then proceed to edit them. The editing process he used involved ‘ageing’ the buildings through photoshop. Zulhazri first used contrast editing as well as minor colouration changes to create a darker mood. Following this he used a series of masking techniques to apply moss textures and various grain filters to specific parts of the environment to create the effect of total disuse. Luckily the environment lent itself to this technique and worked harmoniously to create a believable and genuine looking series of images. These images were intrinsic to setting the space in the mind of the user, as well as being very important for the team to reference when writing the audio and story lines.
Audio Visual Relationships
The game involves a heavy focus on audio elements to create and set the mood and environment for Mencari. Visually the project is relatively bare, employing simplistic animation at key points. This audio visual relationship was an intentional reversal of standard roles. The more common practice for the balance between Audio Visual relationships is for the visuals to take the lead, while the Audio work emphasizes visual cues and supports the environment portrayed by the visuals. We decided to focus on the Audio environment and use visuals to emphasize specific Audio cues and support the Audio environment.
The reasoning for utilising audio as the key narrative driver as opposed to visuals was to maintain the abstract nature and obscurity of some elements within the story. This was a particularly important factor for creating the narratives antagonist – ‘The Cleaner’. The origin, intent and appearance of The Cleaner is intentionally left vague and ambiguous. Whilst a visual cue could have been created for the antagonist, the retention of ambiguity left its appearance to the users interpretation and made the aural appearance of the mysterious creature more tension-driven and terrifying.
The sound design and composition was completed by Tom Harman. As mentioned above, the sound design in Mencari is the main narrative focal point within the story. With simplistic visual stimulus, it was important for the sound to signify story cues and events in an easily recognisable manner whilst maintaining the abstraction necessary to retain the suspenseful nature of our narrative. The heavy use of non-diegetic sound materials left the visual interpretation to the users imagination, and hopefully equated to a more immersive experience.
The two main aural elements that required the most creative focus in order to be the most effective were the underlying soundscape bed as well as the aforementioned sound design for the antagonist ‘The Cleaner. These elements assisted in creating the obscure, otherworldly nature of our narrative. The overlaying foley (i.e. footsteps, breathing etc.) had to be more literal and grounded in reality in order to appeal to the listeners sense of movement and progression through the storyline.
Being a key element to our storyline and not being highlighted visually, ‘The Cleaner’s’ sound design was particularly important. The base audio elements were created utilising a treadmill in the upstairs gym at The Citadines Hotel in Cyberjaya, which had a naturally ugly, viscous mechanical aural nature to it. Varying morphing contortions of the treadmill recording created the bulk of The Cleaners sound design. This, alongside exacerbated footsteps with heavily emphasised low-end, gave the antagonist a large, robotic nature to its presence. The addition of animalistic ‘snarly’ sound elements as The Cleaners approaches juxtaposed the mechanical nature and helped employ the sense of ambiguity and obscurity to the exact aesthetic makeup of the creature.
Interactivity within the Piece
The users interaction with the piece was achieved by using a device called a Makey-Makey. The Makey-Makey’s functionality is simple; it utilises two conductive materials attached to a micro-chip via alligator clips, and the connection of these two materials closes an electrical circuit and triggers an assigned keyboard command. The simplicity of the devices functionality makes for an easy to understand user interface for the participant and a fun interactive method.
The player had two ‘tools’ at their disposal that triggered different story paths and guided them through the narrative. The two tools were a hammer and a machete. The groups original intent was to create the tools out of makeshift, discarded consumer products (i.e. a Selfie Stick as the handle of the hammer etc.) which would have correlated with the narratives overarching theme of over-reliance on the consumerist lifestyle. However, due to time constraints, the choice was made to modify this original plan and utilise an actual hammer and machete.
The tools would strike 2 adjacent ‘pads’ made from Play Dough. Due to its high volume of water and salt, Play Dough is naturally very conductive and made for a perfect conductive conduit. From observing participants use of the tools in early demos of our project, it was noted that varying degrees of force would be used to hit each pad; from light taps to hard smashes. The forcefulness of the interaction did not matter in the strength of the circuit connection, so by using Play Dough, the player could hit the pads as hard as they want without the concern of damaging the pad or breaking the electrical circuitry.
For future presentations of Mencari, the group’s plan is to employ more interactive methods to expand on the experience and introduce more variety for traversing through the story. Expansions may include introducing more ‘tools’ at the users disposal utilising the Makey-Makey technology.
Whilst, as mentioned above, the Makey-Makey’s simplistic functionality makes for an easily employed user interaction, its basic on/off functionality means it is limited in its versatility. For this reason, the group will explore utilising varying technologies such as Arduino, which offers more creative freedom and differing interactive methods that could be created.
The presentation of ‘Mencari’ at Multimedia University in Kuala Lumpur demonstrated a proof of concept for larger ambitions that will be expanded upon for the presentation in Melbourne. Continuing the project with Liyana and Ilham from MMU, the groups intent is to expand on the interactive capabilities in the piece to make the experience for varied for the user. Another way to ensure a more fulfilling experience for the participant is to flesh out the storyline, which will introduce more choices to traverse the piece. Due to time constraints, roughly one third of the original narrative had to be dropped for the presentation at MMU, so the group will continue to develop this storyline. A possible option is to continue to expand on the narrative in order to employ more diverging plot points. This would require heightened contact with Ilham, our narrator, who would then be required to record his own dialogue in Kuala Lumpur.