Being With Picture

I don’t remember exactly when but at some point in my earlier years I started imagining myself entering the pictures in my picture books. Someone would be reading to me and I would zone into the picture and wish I could step in. I would look at every detail and wonder what else there was. Franfreluche, a clown storyteller from a children’s tv show, did exactly that and I would watch her enviously. I wanted those giant storybooks that I could step into.

This curiosity about “being with” picture still animates me. To be clear I didn’t want to have an interactive picture world. I wasn’t interested in a game. I was interested in the sense of space, the colours, the world the characters were in. I wanted to see them do their thing while fully sensing their environment. A modest wish. Theatre and other live performances do this to a degree. We share a space and it is partially determining the performance. A crucial dimension to the pleasure of liveness is that we feel entangled with the performers through a shared space. Still with theatre there is a conceptual distance imposed by the stage. The characters’ world is not wholly accessible. In a way, films and books do a better job of immersion because they specifically ask us to dim some senses in order to merge on others. With live performance, our senses are awake and the merge or resonance happens through a more physical entanglement.

What I seek then is something in between theatre and movie, or between theatre and book. I want the merge/resonance of a book with the feeling of physical entanglement with all my senses awake. I do want to lose myself in the story. But I want to do so as a fully engaged observer.

Enter VR. The promise of the merge. In all its incarnations, we talked about immersion, like sinking into water, being surrounded by our imagination. And yet…not quite. Something is missing. The worlds are empty somehow, happening “out there” even as they surround us. We have readily adapted the gaming paradigm to the current VR platforms but so far have largely stayed away from the book or the theatre. But I have recently encountered two examples that are steering towards a “being with” picture. They have both made me happy, like discovering a new land that just feels interesting and that I always knew existed.

Cave is a piece showcased at SIGGRAPH 2018, by NYU’s Future Reality Lab (FRL). In this piece an audience of 30 people is positioned in two seated groups of fifteen pointed towards a front centre “stage” area. Each audience member dons a headset with headphones (or specialized Bose personal speakers). As soon as I put mine on I could see the other audience members as a virtual audience, their heads and bodies moving. We are not the usual avatars. We are semi-transparent, with large skeletal masks on our heads. The audience heads are tracked to their physical movements so I can look at my neighbour and if they are looking at me I would know to start a conversation. There was enough time before the show that I could do this, and I could also observe the world in which we had all been plunged: a prehistoric cave 12,000 years ago in Northern Europe. A young girl is illuminating the cave wall as (presumably) her mother is drawing on the wall. A soft humming song plays. It feels dim. It doesn’t feel cold though I suppose it must have been. I feel like there is a warm glow of a fire. It feels good to know that I am experiencing this shared space with others and that I can see them looking at different details of the cave. A few minutes later, the characters in the scene fade, and the show starts. The girl comes down from a ladder into the cave. She lights a fire and explains that her people are in grave danger and that they need a new shaman. Her mother was the last great shaman and she doubts if she possesses the same powers. As the story proceeds she transitions from doubt to confidence and activates her abilities to imagine and animate cave paintings. Along the way there is a magical mammoth, a visit from her mother, and handfuls of fairy dust. There were no cuts.

I saw Cave twice and the first time I was analyzing it from the perspective of someone who had just discovered a place where anything could happen. I looked around, I looked at the other people, I observed people as the “play” started, I looked for the intricacies of the movements of the characters, the voice, the sound quality. I wanted to sense the world I was in. How much detail was there? How many times could I watch this and still be enthralled? Why was I even asking these questions? The container was interesting and it was separate but wholly entangled with the story unfolding in front of me. Like theatre. But also like film. And like comic book. Not like a game. fascinating. The second time I saw Cave I decided to focus on the story. Could I be with the story and feel like we had been transported to her world as ghost witnesses? It turns out that the answer to that is “sometimes but not always”, and I think this is because it borrowed a bit too much from theatre at times, and a bit too much from film at others. There were amazing moments like a hug that was so well animated you could almost feel the brush of the hands against cloth and the gentle envelop of an embrace. We had time to linger and appreciate. In general, the timing of the piece was of theatre, with a slower more deliberate delivery (in fact the dialogue followed a Shakespearean iambic pentameter). It could have been even slower given the capacity of the space to speak for itself. I would have liked more detail in the movement or more expressiveness. It felt a little too much like theatre in the seating configuration. I wish I had been cross-legged on the cave floor with others strewn around me also seated on the ground. Though I recognize the desire to match physical situation (seated on a chair) with the virtual representation. I felt taken out of the moment when the music became too orchestral, too Disney-like. In a film, the music would have been appropriate for the moment but in this context it overwhelmed the space which was already providing many emotional cues. In that moment, focusing on complex environmental sounds might have served the same purpose as big orchestral sound in film. Still there were moments where tropes from film were very well integrated including magical visual effects. I think I could watch it a third time and still find something interesting. This is the first time I have felt this way about a VR piece.

To me, Cave is about “being with” picture. And yet it’s not moving pictures, it’s dancing space. I can see so many ways forward to explore the story space that FRL has opened. Many questions come to mind. What exactly is essential to what they created? Co-located audience? How many people before it’s too few or too many? Could some of the audience be less physically present (networked in)? Could the audience be moving around?  The possibilities for the space are also interesting. The audience could disappear. There could be cuts into different spaces including internal conceptual spaces. Once a world is created, it could be re-used for a number of stories, details filled in, characters introduced. This then starts to feel more like a comic book world: episodic and vast. I can also see the possibility of this kind of theatre/film/comic book mix in Mixed Reality. The questions then centre around how site-specific the story becomes, how much “liveness” is introduced into the story either through AI or other performative means.

The other piece that has changed the way I look at VR and space is Dami And Falian, a VR graphic novel being created by Edward Madojemu in the Basically Good Media Lab at Emily Carr University of Art + Design. Edward is creating a completely hand-drawn virtual world (drawn in AnimVR and Quill) where the world progresses in slices of time like a graphic novel. There are speech bubbles and portals in space and time. Within a slice of time you can explore the world, walk around. In that sense it feels a bit like Myst. There remains many questions to answer with regards to the design of a VR graphic novel including when a speech bubble appears/disappears, the orientation of speech bubbles, the use of sounds, the integration of object interaction. Yet the feeling of entering a world which exists regardless of the actions being enacted within it, is strong and interesting much like Cave. Edward has chosen to also integrate networked and social elements to Dami And Falien. You can leave notes in the environment and explore with someone else. This work is nascent and it’s unclear where it will land in the mix of theatre, film, comic, and game. Edward is in Generation Z. I am curious to see where his vision takes him and us.

As Dr. Ken Perlin of the Future Reality Lab reminded me in conversation, our generation has been heavily influenced by film and it is hard to see how much of what we do is driven by those tropes and visual conventions. I see Cave as an integration of past cultural loves and Dami And Falian as an integration of niche media, towards the creation of truly immersive spaces.

 

Links:

Future Reality Lab (their blog has followed the development of Cave as seen through the eyes of the production team)
Cave XR
Dami and Falian
Basically Good Media Lab

0

Memory and Space

The Basically Good Media Lab (BGML) has acquired a Hololens. This has been a long time coming, partially because I was waiting for the technology to come down in price. I have a loose rule that if I can’t buy at least two of something comfortably, I don’t think it’s social enough to have it. Being together in reality (*R?) is one of the tenets of the BGML.

I broke the rule because AR is seeping into the mainstream with ARKit and it seemed like experiencing what is currently the state of the art in AR would be a good thing. I want to have a feel for AR in the same way that I have a feel for VR.

Last weekend, I brought the Hololens home and spent some time with it, about an hour and a half. When I went back to my iPhone, it felt like ancient technology. More than ever I am convinced that AR is the future. Besides playing around with putting 3D models and animations in my kitchen (totally magical!), I experimented with the games that came pre-installed in the Hololens. There was a game that had silverfish-like robots coming out of the walls, which required artful dodging and shooting. I loved how much it induced movement and how I was not afraid to move since this was my own kitchen and I knew exactly where I was at any moment (unlike VR…where there’s always some mystery even with motion capture).

But then I tried a game called Fragments. When it starts, a man appears in my kitchen. This is not a man I would necessarily invite into my kitchen. I’m creeped out and take off the headset. A friend who is in the kitchen with me is curious about me being creeped out and puts the headset on. He continues the game. He then gets creeped out and takes it off. I get curious and put the headset back on. I turn and to my horror I see a young boy kneeling with a gun to his head. I immediately take the headset off. Now here’s the interesting thing. I now have a memory of this boy with a gun to his head in my kitchen. It’s a spatial memory and I can’t seem to shake it. There is something about an embedded virtual spatial narrative into my known physical space that caused a memory to be viscerally recorded in a way that a narrative on a 2D screen (or even VR) is not.

This brings up a lot of issues including ethics of representation in VR and AR, and informed consent in some cases. Jordon Wolfson’s “Real Violence” shocked me in a similar way at the Whitney Biennale. But even that piece didn’t stick in my memory in the same way. It may be that over time, I would become used to the idea of virtual characters and stories in my personal spaces and they would no longer impact me in that visceral way. But for now, I am pondering the role of spatial memory in emotional impact. I am used to being emotionally affected and sometimes shocked by screened movies but the memory of those narratives lives somewhere a lot more abstracted than this little boy with a gun to his head, in my kitchen.

0

I Am Afraid

For the last 5 months or so, I’ve been working on a new networked social VR application called I Am Afraid. This application brings together ideas that have been on my mind for over a year, perhaps longer. Ideas around voice, poetry, sculpture, and performance. Many people asked where the idea for IAA came from and I am surprised that I can’t remember the moment, or a moment, when I decided I wanted to see words and play with sounds in VR. I do know there have been lots of inspirations along the way, including the work I did with Greg Judelman in flowergarden, voice work with my friend Matthew Spears, clowning, theatre, friendly poets (Andrew Klobucar, Glen Lowry), sound artists (Simon Overstall, Julie Andreyev, prOphecy Sun), etc.

The basic idea is to build sound compositions and sculptures using textual and abstract objects that have embedded recorded sounds. When you are in the environment, you can speak words and have them appear as textual objects, and utter sounds that appear as abstract objects. Both kinds of objects contain the sound of your voice and can be replayed in a variety of ways. By intersecting with the objects, the sounds can be played back. The textual objects can be played in parts and at any speed/direction, using granular synthesis. The abstract sounds can be looped. Paths can be recorded through the objects and looped. In this way layered soundscapes can be created. The objects can also be interacted with in different ways like shaking and moving which alters the sound quality. Other actions are also planned, fleshing out a longstanding idea around a sonification engine based on the physicality of interaction with words.

I am often asked why the application is called I Am Afraid. As I was starting work on the application in January, I could sense an escalation of fear in the world, in my surroundings. I have been exploring fear for the last 17 years through different paths including meditation and art. One of the features of fear is that when we feel it, when it grips us, we start talking to ourselves. This is a bit a trap because we get more and more removed from what is actually going on. One of the goals of IAA is to externalize the discursiveness and be playful with the words and sounds. It can be a way to lighten up and see things more clearly, shift the internal dialogue. And it’s fun.

I used the application during my TEDxECUAD talk last March, which is about fear and technology. I’ve also used it in a performance at AR in Action, a conference at NYU at the beginning of June. It’s a great environment for performance (solo or group), exploration, and composition. I’ll be working on it for some time to come, adding features and (hopefully soon) porting it to Augmented Reality.

0

Searching

I had another VR dream last night. I was in a house-like environment and decided to search for the definition of “dispomania”. It became clear that my platform had been infected by a virus as every search I did hid the results in the environment like an easter egg. I had to, sometime literally, shake down the place to see if the result was there. I eventually found it written on a stuffed puppet that I had to continuously move in circles to reveal the definition.

0

Is this a coffee cup?

This is a followup to my post about the definition of reality from November. Last night I had a dream about virtual reality and objects.

The Setup:

Three characters in a virtual world. I am one of them, an avatar. Two other characters, one is an AI and the other an avatar like me. A disembodied voice says there is a newcomer in the philosophy of VR and objects and he’s all the rage, young, naive, brilliant, yada yada yada. This newcomer is the other avatar (not the AI). He’s in a virtual kitchen set with the AI and he’s holding a coffee cup. I’m slightly away from where they are in a kind of blank space.

The Conversation:

“Is this a coffee cup?” says the brilliant new guy to me.
“Aah…Maybe.”
“Is this a coffee cup?” he says to the AI while moving the coffee cup towards the her.
“Uhhh..Yaa…” the AI replies in a disbelieving teenage duh voice. Takes the coffee cup, takes a sip of coffee, and hands it back to the brilliant new guy.
“Is this a coffee cup?” says brilliant new guy as he now moves the coffee cup towards me, breaking some kind of fourth wall as he does so (visual force field effect to make this more pronounced).
“Mm..Less so.” I say.

The dream ends.

0

Virtual Reality and Epistemology

As I’ve told many recently, I just finished re-reading Neil Postman’s “Amusing Ourselves To Death.” This was prompted by two things: the U.S. election and the resulting shock to truth, and by a serendipitous encounter with an interview of Andrew Postman on CBC.

The book impacted me much more on a second read for many different reasons. On a first read, I was still very much implicated in a culture of books. I was in University and still typing my essays, though this would change very soon when I switched into Computing Science as a major. I did not have a television and globalization would not start to become an anxious topic until at least eight years later when the possibility of knowing your thesis topic got scooped by someone halfway across the world became a reality and a going concern. So this was a different time. Pre-Web. Pre-Social Media. Postman anticipated all of it. I think he would have been delighted by Twitter as a revival of telegraphy. A full circle of the death of depth.

Postman wrote his book in reaction to the epistemological shift brought on by television. This wave continued with the introduction of the World Wide Web technology and the media that it favours. We now sit on what seems like another push towards fewer words and more images. Mark Zuckerberg himself anticipates that video and image will be the dominant mode of communication, saying that within five years Facebook will contain only videos and claiming that “it helps us digest much more information.” He is a child of his time, adapted to what information has become.

Virtual Reality, tellingly invested in spectacular fashion by Facebook and Google, now promises to surround us with image. To merge us with information as it is currently valued. We have doubled down. We want to be in it fully with no separation. Surely this is the coming of the promise of “I know Kung Fu” of the Matrix.

And this is where I see a paradox. Here we are at this moment where we want to merge with information and yet are at the height of self-deception. The desire may not be so much to see everything clearly, but rather to live more fully in the castles we build in the sky. Don’t get me wrong, I am absolutely enamoured with virtual reality. When I donned the Rift headset in 2013 I knew it would be the future. It is the ultimate lucid dream on demand. I see even more promise in augmented reality where the virtual and material will be merged on an everyday basis. What I’m concerned about is the trajectory of the medium of virtual reality. What conversations it will enable. I’m very curious about this moment in time where we have the emergence of a new medium at the same time as a loss of faith in our ability to identify anything as truth.

Here is the thing about humans: we are extremely uncomfortable with uncertainty. Apparently we would rather have predictable physical pain than uncertainty which makes sense when you think about it. Knowing what’s coming means we can brace. We are now living in very uncertain times, more so every year. Or so it seems. Postman’s read on the telegraph was that it made the world smaller but filled it with irrelevant non-actionable pieces of news. That trend continued to the point that we have a “now this” practice of going from one piece of non-actionable horror to another, and back to the weather. A system filled with that kind of noise, especially if it’s peppered with outright deception, will soon feel more uncertain than what is practically the case. Most people get their news from Facebook which has a bivalent culture oscillating between Leave It to Beaver and outrage, and is by now known to be a quite notorious echo chamber. It is telling, this mirror we’ve created. Ironically, we’re caught in this Funhouse precisely because we want to reduce uncertainty. Tell me lies, tell me sweet little lies.

So here we are in a moment where more than ever we need to see the world more clearly and when the Funhouse music stopped and we heard unfamiliar voices. There is a desire to not be caught unaware again. And yet, we will be. That is the nature of a dynamic world. We don’t need more information. We need a willingness to look more broadly and more closely even as we are afraid.

A new technology like virtual and augmented reality will become a medium for new kinds of conversations. What is it likely to look like? Here are the questions that Neil Postman posed about the written word and television in “Amusing Ourselves to Death”:

What are the implications for public discourse of a written, or typographic, metaphor? What is the character of its content? What does it demand of the public? What uses of the mind does it favour?

What is television? What kinds of conversations does it permit? What are the intellectual tendencies it encourages? What sort of culture does it produce?

 

Can we start to answer these questions for VR and AR? In what follows, I will speak more towards AR because I see this as the more widespread technology in the future.

What are the implications for public discourse?

One of the things that Postman didn’t mention, and perhaps couldn’t yet be perceived then, is the “feeling” of swimming in tidbits of information. The feel of the zeitgeist comes through from the chorus of voices as a tantalizing manifestation of community, something we all crave. That’s not a discourse but it is a ground for discourse. What AR brings back is localized information as physical spaces are tagged graphically and sonically. With an increase in serendipitous local information, we could see a surge of interest in the development of public spaces and policy, especially if there is transparency in how decisions are made and clear influential avenues of contribution. Potentially the public could encounter itself asynchronously in an extended discourse about local issues and could be given immediate ways of contributing. Oscar Boyson’s mini-doc on the Future of Cities points to the enthusiasm for local solutions. Paul Bloom in conversation with Sam Harris points to the greater rationality of people in local situations (as opposed to national), because the stakes appear higher. It is obviously a kind of human pathology that we fall into these false dichotomies, and Harris has written a lot about this in his book The Moral Landscape, but in the case of augmented reality we could perhaps take advantage of the locality of information to promote deeper and more genuine dialogue.

We could also see pollution of the space with advertising and divertissements of all kinds. Which is more likely, or in what proportion? I think both will happen. My intuition is that we will see a return of oral culture, so the visual pollution won’t be as prevalent as it currently is online. Still information pollution doesn’t take sides when it comes to media. Whatever happens, we will still need to be literate and critical.

There is the tricky question of who owns the virtual space and its contents. As we saw with Pokemon Go, a free-for-all is not appropriate. If I own or rent a space, do I also have control over its augmented space? Are some places completely restricted? What happens to civil disobedience in public and private spaces? And where is all this information stored? So far we have relied on big private companies like Facebook and Google to share and disseminate information. Do our local public bodies have the infrastructure to store and manage public discourse in augmented space? This will be key to a relevant, trustworthy, and free public discourse.

What is the character of its content? What kinds of conversations does it permit?

There is a good chance that much of the AR content will be oral, delivered via chatbots. There are a couple reasons for this. One is that we tend to care a lot about outside space and visual pollution will probably not be a popular option. Text in particular is very cumbersome in virtual space, aesthetically and functionally. The second is that the everyday is social, dominated by spoken conversations. It is our preferred mode of interaction and remains a touchstone of trust and relationship building. As we integrate information spaces with physical spaces, the social everyday will be a mode to contend with. For example, if I’m approaching a new proposed development site, a chatbot avatar may become available to converse with me about the proposal and show me variations on the designs. As I speak to the bot, it takes my input and adds it to a feedback pool that will be used to synthesize new responses and perhaps new designs. Other less involved ways of giving feedback will also exist such as gestural interfaces to choose one thing over an other as we walk by.

I am enthusiastic about the increased level of movement we will have as we interact with information. This alone will alter our ability to process information deeply. If I could go for a walk with a Lee Smolin chatbot, I would. And I would feel the information more deeply than by just reading his books, especially if there were dialogue and interactions that included drawings and animations. In this way, not only does the information feel personal, it is also embodied.

It’s easy to see how wrong this could go with an overpopulation of chatbots vying for our attention — an army of virtual salespeople. Chatbots could easily be biased, as dangerous as fake news. Without getting in too deep in the ethics of AI, it’s fair to say that McLuhan’s “the medium is the message” keeps haunting us. If a chatbot delivers a message convincingly with all the markers of a conscious mind with an intentional stance we recognize, there is room for manipulation.

There is also likely to be many channels of augmentation to choose from: entertainment, public art, education, history, civic engagement, friends and family, etc. I think the main idea is that localized information provides new means of curation and discernment.

What does it demand of the public?

Like any medium it demands attention and literacy. The attention economy will not abate anytime soon so we will still have to be vigilant with our gaze. In comparison to our current means of accessing and contributing to knowledge, augmented reality is likely to be less taxing on the body because it will not require hours of sitting at a laptop or glancing down at a small phone screen. It will have fewer barriers to in-person interaction and in fact may enhance in-person interaction by being able to share information layers more seamlessly.

It will require renewed oratory skills. The role of narrative and persuasion will be greater. In response, we will need to develop good listening skills and body literacy. I am reminded of Postman’s description of the long (8-10 hours) of debates between Lincoln and Douglas. It is hard to imagine people having the skill to listen attentively for that long. Body literacy has waned in recent years because of the disembodied way we access and contribute to knowledge: talking heads, snippets of text, quotes on images, etc. We are likely to see a renewed emphasis on the performative as augmented reality takes hold.

It will demand more knowledge of and involvement with local issues. This falls out of the visibility of the needs within the city as information is localized.

What uses of the mind does it favour? What are the intellectual tendencies it encourages?

My hope is that it favours collaborative problem solving by getting us more physically involved with information that has live counterparts and real perceivable consequences.

What sort of culture does it produce?

Augmented reality is likely to produce a more visible and engaged local culture, with sharing happening in bigger chunks across cities. It has the potential of creating a more playful and therefore creative culture. Engagement with consequence reduces fear by creating a more responsive and trustworthy environment. While I admit this is an idealistic stance that will require many political changes and investment in public infrastructure, there is a clear path to that future. Recent discussions around basic income point to a future where work may be redefined as just responsive action — “doing the needful” as they say in India. Augmented spaces could facilitate that.

The dystopian possibilities exist and have been starkly illustrated in this imagined future of hyperreality. I don’t think this is where we’re headed. We are up against a culture of fear and that is not a small thing. Besides the personal work it takes to individually gain some confidence in the face of uncertainty, we need to reinstate the importance of the local to build confidence in our ability to effect change. It’s a step by step approach to widening our influence and taking part in the conversations that will rebuild trust in our own perceptions.

2

ObjectACTs Residency : Day 2

On day two we spent some time discussing how we might create a performance that would include the perspective of multiple actors, including those non-human and non-personified.

Situation Rooms by Rimini Protokoll

Situation Rooms by Rimini Protokoll

The example of The Situation Rooms from Rimini Protokoll came up. In this theatre work, participants (~20) wearing headphones and carrying ipads are directed to perform specific actions on a set made of several different rooms. The participants are separated and their actions are synchronized to sometimes interact with one another. The topic of the story is arms dealing. A detailed description of the rooms can be found in the Ruhr Triennale catalogue.

 

Kim showed us some of the environments she created using Roller Coaster Tycoon editor.

Image made with RCT

Image made with RCT

She explained the modelling of the terrain as “scooping up dirt” which had a really nice resonance with the object clumps we had been discussing. I love the floating islands and wondered if we could somehow fit the concept of roller coaster in the project to get around the fact that we can’t export from the RCT editor.

We also tested the Structure Sensor to see if we could get workable scans of some of heart trinkets that Catherine brought to Vancouver. It turns out the objects were hard to scan because of their small size and material properties (too reflective and transparent). Still one of the scans ended up intriguing enough that we may use it as a prototype or stand-in.

Here is the first working scan we got of a small rock heart. If you are viewing this on an iPhone and you want to use Google cardboard, use this link.

0

ObjectACTs Residency: Day 1

Today was the first day of the ObjectACTs residency which will continue until the end of the week. We took the morning to introduce everyone and share our thoughts on objects and agency. I took some notes and they are somewhat disjointed but at the very least I thought I would share some themes that arose during the discussion and a few things that particularly caught my attention.

James Luna - We Become Them

James Luna – From “We Become Them”

Richard Hill talked about coming across Jimmie Durham’s work which became the subject of his PhD dissertation. He talked of the deeply contextual nature of objects and our mutual co-creation. Within this discussion emerged the work of James Luna, We Become Them, where he embodies masks of different indigenous cultures as they are projected on slides. This struck me as quite interesting in the context of performance and getting at the question that Ian Bogost poses “What does it feel like to be a thing?” It also reminded me the first ten seconds of the Charlie Chaplin Dictator speech. In that ten seconds, which I could watch over and over, he settles into his body and grounds the work of rising. The very essence of becoming, embodied.

Mimi Gellman talked about the design of The Exploding Archive, a traveling structure which contains and activates maps and teaching bundles. This work has not yet been fabricated but forms the basis of a discussion of how sacred or ritualistic objects can travel with their own contexts. She talked of the Archive as being empowered to carry these objects that she herself is not empowered to carry. She also talked of the power of an object being enacted by its parts being joined (a pipe, for example). Even though she did not discuss it as much this morning, the maps that she has collected for the Archive are varied and are themselves of guides or paths to enactment.

At some point the question “do objects talk back?” arose and Mimi recounted an experience of seeing a mask in a museum which related to her so directly that she did not know that it was in an acrylic case until she asked for a photo of it. I talked of my Amazon Echo which quite literally speaks to me and has become an agent, a kind of person in my life. Alexa is real until she bumps up against the implicit expectations of conversation (see the post on virtuality). Richard pointed out it becomes even more strange when you know that through legislation Amazon is considered a person in the USA.  Alexa is the distributed avatar of Amazon. He also spoke of Daniel Dennett’s concept of the Intentional Stance.

Catherine Richards - I can't let go of them

Catherine Richards – I can’t let go of them

Catherine Richards spoke of her work with heart transplant recipients who have a complicated relationship with their donated heart. She spoke of the trauma always present in that moment where a heart goes from one being to another and how the “intruder” heart is always evading an immune system on alert for what is “not me.” In her work, I can’t let go of them, heart trinkets given to a cardiologist by heart transplant patients are represented in stereoscopic layers. She spoke of the deep meaning that these objects have for the cardiologist who could remember each one (and there are dozens, perhaps hundreds).

I spoke about my curiosity about the representation of objects in virtual environments, as familiar or more abstract entities. Is there a way to design an environment where objects have a kind of life force, that is not fully knowable and is alluring? I also spoke of my recent fascination with Karen Barad‘s work “Meeting the Universe Halfway” where she speaks of Agential Realism which posits that objects come in and out of existence as a function of relations. Catherine spoke of her encounter with a physicist who emphasized that we “cannot look without touching.” This surely relates to virtual environments, though, as Richard points out, we are always venturing somewhere between the “factual and poetic register” when it comes to language. Quantum physics is a good example.

We spent the afternoon experiencing VR apps in the HTC Vive and the GearVR. Kim Parker was our able guide on the Vive. I’ll be posting more about experiments in VR during the week.

A Zotero list has been started to host the references brought up during the residency.

0

What is (virtual) reality

My quick definition:

Reality (as opposed to virtual reality) is what is able to be shared in its essential quality. Virtual reality is shareable but is missing some  of the essential quality.

So it’s a moving target — subjective and context specific. If I’m having a conversation with you in a multiverse, the conversation is real and affective but the avatar is not. A chair in a VR environment may be real or not. It depends on how my intentions towards it match its affordances. BUT it is always real to itself and can become more real to me as I adjust to its being.

more later.

2

Women in Virtual Reality

Quite a bit has been said already about women in virtual reality. I was just at the Weird Reality conference in Pittsburgh last month (fabulous!) and the conference organizers made an effort to be inclusive in gender and ethnic background. Major props to Golan Levin. It was so refreshing. I felt like I could relax a bit and enjoy the show without a nagging discomfort about who was not there. Several women at the conference took their moment on stage to point out not only the lack of gender diversity in VR, but also the prevailing aggressively male environments of the VR industry. I know what they are talking about because I have experienced it. It seems to come from the unfortunate combination of lack of diversity and a fiercely competitive environment fuelled by a tech bubble which threatens to implode at any moment. Still there are some shining lights and I will list some of the women that I have run across lately. Hopefully it can be a starting point the next time you are looking for a woman on your panel or an artwork for your exhibition.

First, a NYMag article highlighting the work of women in VR: http://thecut.io/2dxhKPW

Also Kaleidoscope recently teamed up with Oculus to launch DevLab, “a new initiative to support independent VR creators and explore the boundaries of virtual reality as an art form.” Their initial lineup features quite a few women (yay!). Here they are:

 

Lab'Surd From Judith Guez

Lab’Surd From Judith Guez

 

At the Stereoscopy and Illusion conference I recently attended in Paris I met a fellow calico cat, Judith Guez, doing fabulous work with illusion in VR. The work I saw was called Lab’Surd, an installation using the HTC Vive as the VR device. The setup is beautifully designed and creates a seamless transition between the physical and virtual. The VR experience is about six minutes long during which the environment becomes more and more surrealistic. It is beautiful, dreamy, engaging. One of the best VR experiences I have seen in a long time. I will write a more extensive description of this work in a forthcoming post.

 

The quirky work of Laura Chen also deserves a shout out. She has created some wonderful VR sketches that are thoughtful,  fun, and freeing experiences. I tried her VR communal pooping experience and was immediately charmed. Don’t poo-poo it until you try it, preferably with a friend or three. It’s Chapter One of a series that she is developing. She is currently working on a chewing experience and has rigged a google cardboard with a brilliant DIY chewing sensor. Other works of note are a series of masks or headsets for experiencing reality.

Claire Hentschker is doing some very interesting work using photogrammetry with filmic sources (including The Shining!). She reconstructs the film sets using only the film frames and then lets you navigate though them. She also did a very charming collaboration with a nine-year-old, bringing to life his drawings and characters.

The group EleVR is an absolute marvel and they have been experimenting with 360 video and webvr for quite a while, making them one of the most innovative (and fun!) group I’ve seen. M Eifler showed me some of the work she has been doing with Hololens and even though I had tried Hololens a few times before, when she showed me what she had just created as we were sitting at the bar, I was completely sold on the future of AR, even with a (currently) narrow field of view. Andrea Hawksley is a brilliant inventor, genius at WebVR and camera rigs. Vi Hart…well she is just a spark of creation in this universe. Her 4D visualizations with monkeys blow me away. Anyway, just check out their work. Invite them to wherever you are. You won’t regret it.

Ok I’ll end it here for now but will keep adding to the list, posting more brilliant women in VR as I think of them or meet them.

0