Digital Prototype 1:
For my first prototype in our CFC Media Lab Prototyping course, I made a virtual reality digital prototype using Unity and the HTC Vive. I created an immersive environment, where the user can change perspectives between a human, a tree, and a mushroom. These perspective shifts visualize a significant change in a sense of scale within the environment. While the user could not move their avatar or camera around the environment, they could orbit their view 360 degrees (similar to a 360 degree film). From outside of virtual reality (VR), I controlled the user's perspective through a mouse click on the scene.
In this prototype, I was interested in trying out different species' perspectives from a first-person lens, as well as playing with different perceptions of scale. I was primarily focused on testing the strengths and limitations of virtual reality for my thesis project. While I am interested in exploring the empathy, immersion and embodiment that virtual reality can provide, I am also concerned that VR can be limiting in game mechanics, interactions, and can in fact create a feeling of isolation, which is the opposite of what i am trying to convey: a world of entanglement, relationships, and vital symbiosis.
After creating this prototype and user testing it on my colleagues, professors, and CFC professionals, I received a lot of feedback, most of which enjoyed the VR experience. While I still have mixed feelings towards implementing virtual reality in my work, this prototype exercise and its response has caused me to consider virtual reality more seriously, as I work towards my next prototype and, ultimately, my thesis project.
Creative Process Maps
Before beginning the course, we were asked to submit a diagram of our creative process. While I tailored my diagram (pictured above) to my thesis project, I think that I follow a similar map in my creative process outside of my thesis project.
Typical Creative Process:
Oftentimes when I am creating, I do research before I begin creating, whether that is written work or created work, both for contextual reasons as well as for inspiration. After researching and exploring previous works done on similar subject matters, I begin to decide what medium I am going to use, as well as lay out the groundwork for the style and techniques I am going to use. I have experience with writing, film, photography, drawing, painting, 2D and 3D animation, and now game design, so I have a wide range of media I can choose between for my projects, particularly because I like to be interdisciplinary in my practice and dabble in many things. While choosing the medium usually feels like it naturally fits with whatever my subject matter is, I realize that this choice is still an active part of my creative process. Once I decide my subject matter, medium, and style, I start creating, constantly tinkering the piece, asking for feedback, and iterating until I feel that it is completed. Sometimes, I reflect on the final piece and compare it to what I originally envisioned. I find this reflection to be an important aspect of my creative process, because I can see the paths I took that changed the idea, what areas I excelled in, what areas I need to work in, etc. Finally I either send it to the client, place it online in my portfolio and on social media, or keep it to myself as a practice in my creative process.
Thesis Creative Process:
For my thesis creative process, I plan on framing my process similar to my typical creative process, yet this process will be more structured, methodical, and robust. The main difference between these two processes is the implementation of methodologies. For instance, after a context review of written and created works, I plan on using a multimodal critical discourse analysis to study how we represent nature in written and visual texts, and what that can do to our understanding of the natural world and our ecology. After undergoing this analysis, I will reflect on my findings to structure a few research questions that will lead my thesis work. Next, I will conduct a media analysis to study what channel of media would be most beneficial for me to teach and visualize scientific information and data to the general public. After conducting this analysis, I will decide upon a case study to narrow my focus and the information I convey. I suspect at this point that I will be using Sequoia trees and trees in general as my case study. Once I select my case study, I will do a secondary data analysis to gather and decipher the information I want to visualize in my project. This will lead me to an adaptation phase, where I will rephrase the data and information into a narrative structure. I believe that storytelling is a fantastic way to convey information, as it keeps the user interested and engaged while also allows the learning process to be easy and fluid. After transforming the data into a narrative and making a flow diagram, I will begin my iterative design process of creating a game, user testing, analyzing, and refining my design. Once my final design is completed, I will user test it, discuss my findings, and begin to convey the information I have received into my thesis document, while also using this information to lead my future work.
In our next ideation phase, we used a practice defined as "5 why's". In this process, one participant asks another participant what they are making. After their response, they follow up with a why question, repeating it 5 times. This helped me to reach the deep meaning of why I want to do this work for my thesis. These are my (very) roughly transcribed responses to Kristy's questions:
Kylie: I am interested in making an interactive immersive environment where you change perspective between species
Kylie: I am interested in exploring symbiosis and visualizing scientific data through storytelling
Kylie: I am interested in this because scientific data is usually a lot of complex topics, and graphs and charts don’t necessarily work well for conveying that information. Like environmental science, where there are many parts connected to one another.
Kylie: Interested in combining data visualization with storytelling - there is a lot of interesting climate science that we don't know, or the media doesn't make accessible. Grounding storytelling in data will make it interesting without it being too self created or too lecturing. And documentary is complicated.
Kylie: Documentary is complicated because it can be harsh and people don't want to truly learn it or take the time to do so. Also interactivity and animation make things we don't see with our eyes more accessible through visualization techniques. Interactivity can also get people to let their guard down.
Kylie: There is not consensus for climate change in the US and it’s upsetting, this gap between people and nature. Dichotomy of culture and nature. And I think that traditional data visualization, charts, graphs, or someone telling you the information, is obviously not working. Perhaps with interactivity it is more entertaining and people can still learn about this topic. And a more engaging way to talk about a tough topic.
Class Suggestions for Prototype
In our next ideation phase for prototyping, we assisted our classmates by going around the room and providing suggestions for prototypes to create for the next day. Here is a list of the suggestions I received:
- Look at wolves in Yellowstone
- Choose your own adventure story / narrative
- Observation on a actual tree
- Ecosystem infographic
- Story Interaction, Profiles
- Storyboard / weave together species' narrative
- I think the cause / effect of environmental issues is something to play with
- Make profiles of species as character to see how you can make a narrative
- Multiplayer somewhere outside VR collab to create an inside VR environment
- Storyboard, Character bios for all characters in story
- Take stats put in brain use to direct story, interactions, .... animals, climate, time
- Choosing of diff situations where diff species are in stages of an ecological imbalance
For my first prototype, I mainly focused on VR and character examples as a visualization, but I am contemplating using some of these for my final prototype for this class, including multiplayer somewhere outside VR as a collaboration that creates an inside VR environment, VR, observation of an actual tree, and storyboarding and user flow diagrams to weave the narrative and interactions together.
For the mindmaps exercise, I created three separate mindmaps. The first map was an exploration of the prototype through the medium, and what would be included in that medium. The next map was an exploration of my thought process of prototypes through the three fields I want to combine in my research: data visualization, storytelling, and interactivity. My third mindmap was based on our discussion in class. This was an exploration into the user, objects, interactions, and place for my prototype and final thesis.
Creation Process at CFC and at home
During our working time at CFC, as well as working at home, I created a Unity file with assets from Unity, Quixel Megascans, SpeedTree, SteamVR, as well as my own mushroom.
My first step was to create a landscape. This was done relatively quickly with the built in Unity terrain tool, along with my own terrain textures that I edited from Quixel. Once I built this terrain, I implemented the coniferous trees from SpeedTree. In order for the trees to look huge in both the mushroom and tree view, I scaled them up immensely. This created lag issues. In order to remedy some of the lag, I decreased their LOD distances, so that their detail would decrease immediately outside of the camera's range. I then placed my mushroom model, the blacksmith character, and other terrain assets (such as clover, grass, and rocks) into the scene.
My next step was placing cameras in each of the perspectives. I learned that cameras can not decrease in scale, so in order to fix this issue I created a new scene for the mushroom, as well as a new scene for the trees. In these new scenes, I changed the scale of the assets to imitate a change in scale of perspective. Luckily, I later learned that this was probably the best practice, because changing between cameras in the same scene might not work in virtual reality. These cameras were locked into position, but could rotate 360 degrees, in order for VR to feel the best. While placing the camera on the mushroom was easy, I had a hard time figuring out how the camera would work for the person, and in particular, the tree. When placing the camera on the human, it was weird to look down and see his body rotate exactly with the camera, even if the VR user's body was in place, and they were twisting their neck to see behind. I decided to leave this feeling for now, in hopes to solve it in the next prototype. For the tree perspective, it was difficult to figure out how a tree's perspective would be, as well as where it would be located. I decided to place the perspective at the top of the tree. I quickly found that if I placed it on the tree, the user could rotate their view and collide with the model, therefore breaking the camera and model setup. Yet, when parenting the tree model to the camera, the bottom of the tree would move in conjunction with the camera. This created the appearance that the entire tree was rotating on the surface of the landscape. Due to my limited amount of time, I decided to scrap this approach, and instead place the camera near many of the branches, but not directly onto the model. While this feels a bit like floating, it also gives the user the ability to look in 360 degrees without rotating the tree across the surface. After solidifying these views, I added post-processing effects, increasing the blur, depth of field, and chromatic aberration on the tree and mushroom perspectives. I used these effects because, in my opinion, their eyesight is probably not the best, and the use of depth of field helps to exaggerate the sense of scale, whether small or large.
After creating 3 scenes which imitate 3 different perspectives in the same scene, I made a script to change between scenes on mouse click. This code was adjusted in 3 different ways, so that I could cycle between the human, tree, and mushroom perspective. I could not figure out how to map this onto a VR controller without a VR system at my disposal, so I decided to keep the mouse click function, which would work only on my computer, and as a quick fix/ work around I would click for the users during testing.
My last step was implementing the SteamVR control rig into my scene. This was relatively easy. I used the same transformation, rotation, and scale numbers as my unity camera, and copied and pasted them onto a control rig in my scene. This allowed me to toggle between the VR camera and the Unity camera easily, so that I could change the view based on what technology I was using for the prototype. I repeated this process three times, using it in each scene.
While I could not test the VR at home, my computer version worked properly with the Unity cameras. I could click through the different perspectives, and rotate the views with my mouse. My only issue at this point was lag, which was acceptable for my computer, but would not work whatsoever if placed in VR. Once I got to the CFC centre, I used Max's high quality computer for displaying purposes. This immediately fixed my lag problem. He also helped me get VR up and running in my scene, changing the active cameras to the SteamVR control rigs. At this point, I was ready for user testing.
On our final day of the CFC Intensive for prototyping, I displayed my digital game on the HTC Vive. This was also the first time for me to experience it for myself, because I do not have a VR headset at home, and, once implemented into SteamVR, you cannot test it without a VR headset. After a little bit of tinkering with the camera rig to fit my unity camera, the VR headset worked, and allowed the viewer to look around the scene from different perspectives. The click function was not mapped to an HTC Vive controller, so from outside of VR, I changed the perspective for the user.
I received a lot of great feedback from my user testing session. While there were differences in opinions, and some contradictions, there was also areas of solid consensus. This feedback has given me a lot consider as I move forward in my prototyping process. Here are a few bulletin points of feedback I received from my notes (thanks Max!) :
In Tree's Perspective
- User 1: Reminds me of CN Tower - glass pane- scary
-User 1: Blur in foreground is noticeable
- User 2: Blur makes no sense- why so blurry?
- User 2: Could be standing on a branch
- User 3: It's weird because i'm floating. I should be attached to tree.
- User 3: Very floaty.
- User 4: I don't like the tree. I feel like I am floating.
-User 5: No, this is not disorienting (Answering my question if they feel ill or dizzy)
- User 6: I don't feel big. I just feel high up, like the position of the camera has changed.
- User 6: Tree rotation is weird (Billboard effect)
- User 6: I feel nauseous.
In Human's Perspective
- User 1: I don't want to be this man
-User 1: Shadows are very fast (shadows of trees in wind)
-User 1: This view is most disorienting - I feel suspended
- User 2: I feel like I'm not on the ground totally.
- User 2: Quite windy
- User 3: Very tall. Man's shadow is creepy.
- User 3: Man is too tall.
- User 3: Trees are very huge
- User 4: Should have human in human view, for size comparison.
- User 4: Human feels small.
- User 6: You should fade to black between changes in perspective.
In Mushroom's Perspective
- User 1:Big trees, this view is so pleasant
- User 2: Likes mushroom cap
- User 2: Juxtaposition between size is very nice
- User 3: Makes sense. A+ mushroom.
- User 4: Should have mushroom in mushroom view, for size comparison.
- User 5: No, not disorienting.
- User 6: I feel nauseous.
As you can see from the comments above, there were a wide range of responses to the different perspectives. While there were differences in opinions about disorientation in the tree and mushroom view, as well as the scale of the human in the human view, the greatest consensus was the tree's perspective. Most users did not feel big, they just felt like they were high up and floating. Also, the blur on the tree's was not well received. This leads me to think, what does it mean to speculate on a different species' visualization and sight? Will people be less responsive if the "sight" visualized is not the same as human perception?
After user testing and meeting with Emma and Ana, we discussed the next steps I can reflect upon and strive for in my future prototype. We discussed how not only scale would change, but the perception of time. We discussed time in relation to day and night cycles, as well as seasons. We also discussed different ways of visualizing what species "see" or better yet sense. As trees are connected to other trees, is the perspective we are visualizing instead a forest? How do trees intake information? Furthermore, what can embodiment in VR provide? Perhaps your legs or your arms are the tree branches. What does VR and embodiment bring to this experience?
All of these points brought up today bring me rich ideas to reflect upon, as I navigate my ideas to my next prototype. Overall, the prototyping process has been extremely useful, as it has caused me to go ahead and dive into the making process, rather than wait until my research is complete. It has also given me great feedback into what areas to look at, and how I can structure my thesis project in new, compelling ways.