11 Proposal
Proposal: Transcendence Audiovisual Instrument and Experience
MODULE: MA Immersive Arts Research & Proposal
DATE: April 2022
Contents
- SUMMARY
- AIMS
- BACKGROUND/LITERATURE
- GOALS & OBJECTIVES
- SCOPE OF WORK
- RISKS
- TIMETABLE/MILESTONES
- BUDGET
- EVALUATION
- REFERENCES
- BIBLIOGRAPHY
1. Summary
This project sets out to design, create and deliver a live guided audiovisual interactive experience themed around sacred geometry and transcendence. Building on geometry as the mathematical foundation of the universe and thus being a feature of nature, the piece sets out to connect the audience with nature with the aim of eliciting the emotion of awe.
At the end of the production timeline will be a live show in a fulldome with spatial sound. This medium offers high levels of immersion and can be used to create the illusion of vast space around the audience through the use of both visuals and sounds.
The audience will have a small amount of interactive capabilities so as not to distract from the vastness of the experience, but enough to show them they are connected to it. The target audience size is between 1 to 6 people.
The levels of awe experienced by the audience will be measured through the use of surveys that compare the difference between how they feel before and after the experience.
A core digital audio and visual system will be designed and built which can be set to either a performance mode where an artist takes greater control of the system, or an installation mode where the system is primarily generative with some audience interaction using body position sensors.
The artist/performer will have a high level of control of the system as they will act as a guide for the experience, making improvisational decisions in response to audience observation in order to further meet the aims of the experience.
While I will be doing most of the development work independently I will take an agile approach to project management and development made up of frequent user (artist and audience) testing throughout production to gather feedback to tweak and integrate what is working well, followed by iteration. In this way the project will evolve over time in direct response to real users.
This project has arisen from my prior experience in music production, performance, interactive visual art, event production, as well as my study of immersive arts over the past year and a half where I’ve looked at immersion, storytelling and explored various technologies. It is also relevant to my interest in and history of practicing meditation and yoga.
The timeline for this project is 15 weeks from start to delivery of the show.
2. Aims
By drawing from the themes of sacred geometry, nature and vastness I aim to create an interactive immersive experience that elicits awe and offers the audience a transcendent experience that has lasting positive effects on them.
The reason I have chosen this topic is because my research highlights many resulting benefits that arise from experiencing awe such as improved feelings of connectedness to others and the world around them and greater wellbeing, to name a few (see previous blog posts for references).
This topic aligns with my personal aim of trying to do good in the world throughout my own work.
I seek to answer the question of how can sound and visuals be designed in ways based on the underlying mathematics of nature and the universe in order to connect an audience to the experience to the extent that they lose themselves in the experience, with increased feelings of “small self” and awe. The success of this will be measured in the audience by comparing how they feel before and after.
To realise these aims I will design and program an audiovisual ‘core’ system which is capable of generating evolving sound and visuals as well as being able to facilitate a performance.
3. Background/Literature
This project seeks to elicit awe in the audience for greater wellbeing. The significance and benefits of this emotion have only fairly recently begun to be researched, yet I there there is significant enough research to inform this project.
Keltner & Haidt (2003) suggest that there are two components of awe:
- perceived vastness
- a need for accommodation (“defined as an inability to assimilate an experience into current mental structure”)
“Awe experiences are self-transcendent. They shift our attention away from ourselves, make us feel like we are part of something greater than ourselves, and make us more generous toward others.” (Allen 2018).
“Awe creates the mental state of wonder, which leads to the search for knowledge, a recognition of one’s position within the broader social context, and prosocial inclination.” (Keltner 2020). “Research suggests that awe diminishes a person’s sense of self, shifting their focus away from their own concerns. Accordingly, perhaps the most studied psychological effect associated with awe is the “small self” the feeling of being small relative to one’s surroundings.” (Allen 2018). Additionally, there is “evidence that awe plays an important role in nature’s ability to decrease stress and increase well-being” (Allen 2018).
What elicits awe? “Art, music, natural wonders, panoramic views, and other things of beauty can evoke many feelings, but the response common to all of these may best be labelled ‘awe’” (Keltner & Haidt, 2003). One study found that “awe was elicited by information-rich stimuli, particularly panoramic nature views and novel art and music, and reliving awe experiences made participants want to be in such environments again.” (Shiota, et al. 2007). This study also found that after nature, art or music were the next highest ranked stimulus events for awe.
I will try to elicit awe in the audience in several ways. Firstly by creating the illusion of vastness; immersing them in ‘spectacular natural scenes’ as a study showed these lead to “feeling more awe and smallness.” (Allen 2018). By drawing from number in the forms of music and geometry the audience will be connected with nature because “mathematics is the language of nature.” (Emmer 2005).
I’ll now go into the potential of the fulldome medium for achieving awe. “The dome envelops the audience in a way that no flat screen, no matter how large, can achieve even in stereoscopic 3-D.” (Lambert & Phillips 2012). The “sensation of height, distance and scale is essential to the dome experience and perhaps its most unique feature.” (Lambert & Phillips 2012). Further to this, domes can be used for communal group events… much like that of a group visiting a temple, and I hope that this will result in a more connected (to each other) experience for the audience.
To enhance the connectedness of the audience to the experience, sensors will give subtle but noticeable feedback as the audience moves around the space by means of particles that respond and react to the them. The intent here is for the audience to ‘get lost’ in the experience so the amount of interactive feedback to them will be tuned to optimise this through user testing.
Immersion will be increased through the use of spatial sound- an array of speakers around and above the audience. To fully utilise this some sounds will animate through the 3D space as the piece evolves, creating vastness through auditory perception.
Many more links to relevant literatures for this project can be found in the bibliography.
4. Goals & Objectives
Goals
- Create a mythical story world to guide creative decisions.
- Design and build an audiovisual engine capable of;
- generative playing by means of algorithms
- sample playback (eg for nature sounds, images or videos)
- interactivity through controllers and sensors
- high level of control for the performer
- subtle but noticeable interactivity for the audience
- scene sequencing (to allow a structure or narrative to be assembled).
- Build or find a fulldome suitable for the performance.
- If this is problematic find a space that can have many surfaces projection mapped onto as an alternative medium
- Design the show and performance space according to the theme, aims and story.
- Grow an audience for the experience using marketing practices such as newsletter mailing lists and social media accounts.
- Livestream the performance (ideally in 180 or 360 so it can be viewed on headsets) to grow the audience further for future events and projects. Also to gather feedback from people watching the livestream… eg what do they get from the experience?
- Record the performance for portfolio purposes and so people can view it at a later date
- Aim for 180 or 360 stereoscopic capture of this so it can be viewed in a VR headset
- Spatial sound that reacts to the positions of the visuals via OSC
Objectives
- Design and create generative immersive content themed around sacred geometry, nature and vastness made up of several scenes that can be sequenced manually or automatically that is delivered to the audience during the show.
- Source or make high quality recordings of a range of nature soundscape field recordings by week 6 of the project (audio engine production).
- Gather or create visual content on the theme of sacred geometry and the platonic solids.
- Gather or create visual content on the theme of vastness eg cosmos, large nature features
- Build an interactive evolving audiovisual piece that can be played by an artist using controllers and/or sensors
- Create a particle system that reacts to the positions of the audience members using Kinect sensors
- Encourages active audience participation through exploration and play without prior onboarding- the audience should be able to walk in with no prior knowledge and learn by exploring.
- Design and build a robust, functional and adaptable audio and visual core system that can be used in this and future projects across various mediums and formats. For the show has two operating modes;
- firstly as a generative interactive installation, and
- secondly as a system that can be performed with by an artist who has rehearsed with the system
- Produce a show and performance on this system on 26th August 2022 where the audience experiences both the generative environment and a performance within one experience.
- During production, iteratively test the system on a real audience, gathering feedback and integrating this to improve the system.
- Grow an audience of followers, some of whom will attend the live performance and some of whom will attend the livestream.
5. Scope of work
By breaking the project down into simple steps and sequencing these in order of dependance on one another, I can build a timeline backwards from the show date.
I’ve drawn up a system schematic which can be seen here;
Work to undertake / task list
- Log time on each element using a time tracker. Keep reviewing schedule to check progress throughout project
- Define audio elements
- Define visual elements
- Define sensors and controllers
- Define algorithms for generative aspects
- Develop narrative concept, myth and theme
- Create website with newsletter capture
- Build audio core in Ableton Live
- Source nature sounds
- Build sound generating voices
- Configure sequencer presets to generate notes
- Build visual core in TouchDesigner
- 3D geometry creation- platonic solids
- Create cosmos imagery
- Source or create nature imagery
- Particle systems that interact with the audience body shapes and positions
- Experiment with best ways to blend the elements
- Create scenes and transitions between these
- Set up output options from this core e.g. for fulldome ‘dome master’ and also option for multiple projector outputs
- Source and configure sensors and controllers to affect audio and visual cores
- Build generative algorithms (for sound and visuals)
- Test system performance, see if more computers are needed (esp. with GPU for visual processing)
- Design show, include projection specifics
- Design experience scenes and sequence these (storyboard)
- Spec livestream and set up social accounts for streaming to
- Post regular progress updates to website, style and design around theme and story
- Advertise show and livestream
- Iteratively test and improve system
- Rehearse performance
- Book show venue
- Find volunteers to help during the show with recording
- Book recording equipment including cameras and tripods
- Design surveys for before and after show to measure awe and record other findings and gather audience feedback
- Set up venue for show
- Test system
- Run show and performance and record it
6. Risks
I conducted a SWOT analysis of the project to identify strengths, weaknesses, opportunities and threats and to produce some tasks and potential solutions relating to these. This can be seen here:
7. Timetable / Milestones
I have created a schedule for the main segments of work that need to happen over the course of the 15 weeks between project start and show/performance delivery. I believe this is achievable but in my experience development can vary in the fact that some aspects will be completed faster than expected, while others will take longer. While I will try to stick to this I will try to work as efficiently as possible, and hopefully will complete some segments early, allowing me to move forward with other segments sooner than planned, with the overall aim being to complete the bulk of the work earlier than planned, allowing more time for practising/rehearsals.
8. Budget
This is a breakdown of all the necessary elements along with costs:
9. Evaluation
To evaluate this project I will review the production schedule to evaluate if I met this comfortably or if there were time pressures leading to rushed or low quality work.
I’ll gather audience data before and after (immediately and at a later date) their experience using surveys, and evaluate the results. For this I will replicate techniques to measure the ‘small-self’ of the individual as outlined in Allen’s 2018 white paper.
I will assess the capabilities of the core that is developed in the categories of performance and generative ability. I will evaluate the potential for this core to be used for other projects by considering how adaptable it is.
I’ll gather feedback from the audience about the level of agency they felt during the experience and evaluate how this affected their experience of it.
10. References
Agile Manifesto https://agilemanifesto.org/principles.html
Keltner, Dacher, and Paul K. Piff. “Self-Transcendent Awe as a Moral Grounding of Wisdom.” Psychological inquiry 31.2 (2020): 160–163. Web.
Summer Allen 2018 “The Science of Awe”. A white paper prepared for the John Templeton Foundation by the Greater Good Science Center at UC Berkeley
Piff, P. K., Dietze, P., Feinberg, M., Stancato, D. M., & Keltner, D.(2015). Awe, the small self, and prosocial behavior.Journal ofPersonality and Social Psychology,108(6), 883–899. doi:10.1037/pspi0000018.
Keltner, D., & Haidt, J. (2003). Approaching awe, a moral, spiritual, and aesthetic emotion. Cognition and Emotion. https://doi.org/10.1080/02699930302297
“Without mathematical structures it is not possible to understand nature: mathematics is the language of nature.” (Emmer 2005).
11. Bibliography
Anderson, C. L., et al. (2018). “Awe in nature heals: Evidence from military veterans, at-risk youth, and college students.” Emotion 18(8): 1195-1202.
Efstathiou, K. and M. Efstathiou (2018). “Celestial Gearbox.” Mechanical Engineering 140(09): 31-35.
Freeman, G. and D. Maloney (2021). “Body, Avatar, and Me: The Presentation and Perception of Self in Social Virtual Reality.” Proc. ACM Hum.-Comput. Interact. 4(CSCW3): Article 239.
Gottlieb, S., et al. (2018). “Awe as a Scientific Emotion.” Cognitive science 42(6): 2081-2094.
Goulding, C. and A. Shankar (2004). “Age is just a number.” European Journal of Marketing 38(5/6): 641-658.
Hameed, A. and A. Perkis (2018). Spatial Storytelling: Finding Interdisciplinary Immersion, Cham, Springer International Publishing.
Kac, E. and J. Elkins (2005). Telepresence & Bio Art: Networking Humans, Rabbits, & Robots, University of Michigan Press.
Kang, S., et al. (2019). “Immersive Journalism and Telepresence.” Digital Journalism 7(2): 294-313.
Keltner, D. and P. K. Piff (2020). “Self-Transcendent Awe as a Moral Grounding of Wisdom.” Psychological inquiry 31(2): 160-163.
Lamond, I. R. and K. Spracklen (2014). Protests as Events: Politics, Activism and Leisure, Rowman & Littlefield International.
Latoschik, M. E., et al. (2019). “Not Alone Here?! Scalability and User Experience of Embodied Ambient Crowds in Distributed Social Virtual Reality.” IEEE Transactions on Visualization and Computer Graphics 25(5): 2134-2144.
Martineau, J. (2020). Quadrivium, Wooden Books.
Michell, J. and A. Brown (2009). How the world is made: The story of creation according to sacred geometry, Thames & Hudson.
Mystakidis, S. and S. Mystakidis (2021). “Combat Tanking in Education: The TANC Model for Playful Distance Learning in Social Virtual Reality.” International Journal of Gaming and Computer-Mediated Simulations (IJGCMS) 13(4): 1-20.
Pollan, M. (2019). How to change your mind: What the new science of psychedelics teaches us about consciousness, dying, addiction, depression, and transcendence, Penguin.
Schnall, S., et al. (2012). “The Immersive Virtual Environment of the digital fulldome: Considerations of relevant psychological processes.” International Journal of Human-Computer Studies 70(8): 561-575.
Stellar, J. E., et al. (2017). “Self-Transcendent Emotions and Their Social Functions: Compassion, Gratitude, and Awe Bind Us to Others Through Prosociality.” Emotion Review 9(3): 200-207.
Tanenbaum, T. J., et al. (2020). “How do I make this thing smile?”: An Inventory of Expressive Nonverbal Communication in Commercial Social Virtual Reality Platforms. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery: 1–13.
Toussaint, G. T. (2019). The Geometry of Musical Rhythm: What Makes a “Good” Rhythm Good?, Second Edition, CRC Press.
Womack, Y. (2013). Afrofuturism : the world of black sci-fi and fantasy culture. Chicago, Chicago Review Press.
Yaden, D. B., et al. (2019). “The development of the Awe Experience Scale (AWE-S): A multifactorial measure for a complex emotion.” The Journal of Positive Psychology 14(4): 474-488.
Lambert, N. and M. Phillips (2012). “Introduction: Fulldome.” Digital creativity (Exeter) 23(1): 1-4.
Chandrasekera, T., et al. (2015). “Virtual environments with soundscapes: a study on immersion and effects of spatial abilities.” Environment and Planning B: Planning and Design 42(6): 1003-1019.
Preston, J. L. and F. Shin (2017). “Spiritual experiences evoke awe through the small self in both religious and non-religious individuals.” Journal of experimental social psychology 70: 212-221.