A computer science and electronics Professor, a Doctor for theoretical mathematics, a film producer, a digital designer and 6 programmers, want to change the way we enjoy theater. The Israeli start-up company ARShow, presents the first play in the world combining Augmented Reality with live actors on stage.
https://www.themarker.com/markerweek/MAGAZINE-1.4704779
Tuesday, First night of Hanukkah, Gesher theater in Jaffa. Those who sit excited back stage are surprisingly not the actors, but people you’d least expect to meet in a theater: A computer science professor, a theoretical mathematician, a film producer, a digital designer and 6 computer programmers. This group calls itself ARShow, and they are responsible for producing the first show in Israel combining theater with Augmented Reality (AR) and the audience who came to watch the general rehearsal, was equipped with AR goggles, which add a layer of animation to a few segments of the live show on stage.
“When they gave the storyboard (the play’s graphical plans) to the programmers and animators, they told us it was impossible, that we’ve gone completely mad – Augmented Reality is for Pokémon Go on cellular devices and not for Lilliputians and giants in a play based on Gulliver’s Travels” – says Sasha Kreindlin, CEO of ARShow “And this week the show is premiering, and in it, the actors fly in a hot air balloon and interact with virtual Lilliputians and giants on stage”.
ARShow is dealing with quite a challenge: No one in the world ever done this before, and there are no veterans in the field to consult with. “This is nothing like video or film, there aren’t screens that look the same from each seat in the theater, and we have to explain this every day to the director and the actors”, says Kreindlin. “These are 3D objects and the person sitting in row 20 on the right receives it in different angle from the person sitting in row number 5 on the left – and still, 800 people in the crowd need to get a whole and similar experience. You need to “travel on stage”, tell an actor to move a little to the right or to the left cause he is stepping on the tail of a virtual creature, to make sure it works the same from each possible angle. We are creating a brand new language and “writing” the book of rules for Augmented Reality in theater as we go.
ARShow is comparing the revolution Augmented Reality is generating in theater to what CGI (computer-generated imagery) did to the world of film. “Nowadays there isn’t a movie without CGI” says Isaac Barel, the company’s CTO. This technology gave the option to invigorate water, monsters or even realistic wild animals that can’t be directed, like the tigress from the movie “Life of Pi”. This technology gave directors the freedom to dream and execute movies that in the past they’d never even dare to consider”.
And will these dreams bring back the interest in theater?
Kreindlin: “The theater doesn’t have an audience problem but a concept problem – people who don’t go to the theater thinks nobody goes to the theater. Truth is, more people visit the theater than they do soccer games. In London, the amount of people going to the theater every year is twice as big than the amount of people going to the Premier League games. If anything, the cinema is losing its crowd of viewers – but nobody’s talking about that”.
Well great. Than what is it you’re trying to solve?
Chairman of the Board, Prof. Simon Litsyn: “The problem in theater is that there are shows to 8 year’ old kids and after that, the next stage is shows for grownups. There is a gap of almost 10 years on which there are almost no materials for teenagers. Majority of these teenagers are gamers and addicted to technology and a large part of their social experience and interaction with friends their age is through the cellular phones. So why not engage them in theater by adapting the shows for them?
In cinema, a few failing experiences were made to try and improve the experience. 3D for instance, which isn’t remembered as to big of a success. Why would Augmented Reality succeed?
In a 3d movie there is no interaction with the objects in the scene and people don’t feel that it adds anything to the artistic content. This addition is also meaningless emotionally; it is merely an elusive visual development. It’s not that we believe that the entire world of theater needs to switch to AR, but when the use in the technology is just, it drastically expends the theater’s artistic abilities.
Kreindlin: “Gulliver is the perfect example for a just use in AR. Without it, the play would seem very infantile, like a man playing with toys in a room.
History of Reality
The concept of AR in art exists for several years now, but nothing took off just yet, perhaps due to the fact that the technology is still far from the way it appeared in visions of science fiction movies. Therefore, whether or not this show will prove itself to be an actual breakthrough in this field is truly intriguing. Still, it seems worthwhile to go over on how we made it this far. Film art started its way documenting the theater, and at first showed only one point of view – the viewer’s, sitting right in the middle of the theater, watching directly at the event happening in front of him. When the world of film discovered its ability to provide the viewers with several points of view, and breakthrough the limitations of the theatrical source, film expressionism came to the world. The theater, on the other hand, learned to use the cinema as an inspiration, weather it was converting movies in to stage performances or using means which developed in the world of cinema, due to the technical possibilities, which changed viewers’ conception.
Whilst one medium is getting a bit too close to the other, that last one is forced to re-invent itself, hence the establishment of the immersive theater, on which the audience takes an active part in the show. The most identified play in this theatrical field is probably “Sleep No More”, based on the Shakespearian tragedy “Macbeth”. The show takes place inside a building – an imaginary hotels spreading over six floors. The storyline isn’t linear and the viewers, wearing identical white masks and becoming a part of the setting and the show, are requested to walk behind the actors in order to follow the different scenes, yet they are also free to wonder on self, to “snoop around” and touch the desk drawers and any other object they find in the hotel.
In 2014, The British ensemble “Coleb” presented a show called “The Fifth Column”, in which the audience uses a cellular app, adding features of virtual reality and gaming elements, to scan the streets of London and track down spies – an experience closer to what we know as the “Escape Room”. On a certain level, the theater is going through a process similar to what photography did to the art of painting – realistic painting is neglected and in its place abstract art is blooming.
Partial use of Augmented Reality has reached New-York last year, in a show called “Elements of Oz”, to which the viewers were requested to arrive with their smartphones and tablets, providing them with additional visual information while watching the stage – for instance, a tornado storm, or rain around the house of Dorothy, the hero of the “Wizard of Oz”. “This is an addition, a nice improvement for the show, but not an integral part of it”, clarifies Elena Litsyn, CSO of the company. “It isn’t a part of the storyline, and if you’ll remove the app, you can still watch the show and enjoy it. In a full AR show, like the one we do, everyone get the same experience and you can’t watch the show without the goggles.”
Bring the Programmers in to the Theater
Unlike many other start-up companies, ARShow is nothing like your popular image of the common Israeli start-up company. It wasn’t founded in a garage by a bunch of motivated young people fresh out of their military service. Simon Litsyn (60), is a professor for Electronic Engineering in Tel Aviv University, ever since migrating to Israel on 1991, and registered on his name over 60 patents. The Israeli high-tech field knows him as the former chief scientist of M-Systems and part of the group that invented the USB flash drive, sold to SanDisk on 2006 for 1.6 billion dollars. Litsyn is also a co-founder in one of the more intriguing startups in Israel today, StoreDot, developing a technology for charging cellphones in just 30 seconds. The nowadays company also develops screens that will enable a sharper and brighter color display.
His wife, Elena Litsyn, is a Theoretical Mathematician and a lecturer and researcher in Ben-Gurion University and in Weizmann Institution, in addition, she is a co-founder in another startup in the field of theater – GalaPromter, developing a translate and subtitles app, making the theater accessible for deaf people and foreign visitors that don’t speak the language of the show.
Kreindlin (36), is the film maker of the bunch – a fourth generation descendant of stage artists, and the son of Gesher theater’s CEO, Lena Kreindlin. After producing and directing movies, series, plays and concerts, he’s been specializing in multimedia solutions for stage performances.
Barel (40), is an animator and a UI/UX expert, in charge of the creative aspect in the company, cofounder of “Navigaya”, a startup which developed a web streaming platform. In 2015 he founded “Studio B54” in Miami Florida, assisting startups to develop digital products.
ARShow was founded on March 2017 and so far raised 700,000$ from the Norma Investments Fund of the Jewish-Russian multi-millionaire Roman Abramovich, Duotem Capital fund of Honk-Kong and the Kazakh investor Igor Yucht.
As far as the entrepreneurs are concerned, the aspiration is to develop a comprehensive technological solution – software and hardware available for rent – for theater production to be able to produce their own AR performances, while offering full content creating services, for theater that will be interested in purchasing their artistic capabilities as well.
In its presentation for the investors, the company estimates the costs of develop in 100,000$-300,000$ per show, a sum it will impose on its clients, in addition to a profit sharing model. ARShow will charge 500$-1000$ per event, along with 2$-4$ per ticket, a sum of about 3,000$-12,000$ per show. A model by which a show that runs 100 times will produce an income of between 300 thousand and 1.2 million dollars.
The Audience is a Part of the Experiment
In the actors launge on Gesher theater, where we held the interview, the atmosphere is far from calm – actors and staff are constantly coming and going, and the four entrepreneurs keep squinting at the screen in the middle of the room, showing what’s happening on stage. They don’t need AR goggles to behold fires and virtuals cities. “There, now is the scene of Gulliver peeing on the city to put-off the fire” points off one of them. “Ahhh, now I see, that’s what the director has changed”, says the other.
A new technology almost always comes with struggles, mistakes and improvisations. One of the first missteps ARShow dealt with was a very basic element – the script. “We thought we could take every children’s play and add the Augmented Reality to it” laughes Simon, “Boy were we wrong” the entrepreneurs went back to the production floor along with the screenplay writer Noga Ashkenazy, who wrote a play from beginnign to end, implementing Augmented Reality. The script must include contential justifications to encourag the children to act – as during the play the audience is requested to alternatly put on or take off the AR goggles.
Another challenge was to find the right goggles. They should be cheap enough to order thousands of units for massive use, child resistant, light and comfortable. After a long research, the company put together a model made out of components from four different companies. The goggles are made out of a soft sponge and can be dropped on the floor wothout causing any damage, and are equiped with two buttons, creating a base to engaging the audience in the show, when the script determins they’re used to shoot virtual wasps .
The front opening of the goggles holds a cellular device of Xiaomi, which became the screen and the brain of the kit. This is where a new challenge arrose. “15% of the people are getting nausia from VR goggles, because they are closed, and infront of you is a screen blocking your vision with no horizon”, says Dr. Litsyn. “The smartphone broadcasts the stage in real time, and in the video we’re combining on top of it, we make sure to maintain the stage line for you, so that even if the actors are flying in a hot air baloon between the clouds, you will still see that line. The show is also built so that only one half of it requires the audience to put on the goggles, on the rest of the time, the eyes get time to rest and also stay a part of the live audience, unlike the isolating experience of VR.
Phrases like “being the first”, and “being a part of the technological revolution” are repeatedly heard during the interview, and yet moments before the general rehearsal, the company’s team is still dealing with a veriaty of technical issues. Barel is called upstage many times and is abscent for long periods of time, when he comes back he explains that the directore, Yevgeny Arie, asked to change the timeline, the siquence of animation intwined in the play.
“Theater artists are having a hard time understanding the limitations technology is creating” says Kreindlin. “When you build a play, you change it as you go, but cutting a scene in the last moment is not so simple anymore. In this case, theater with AR is much like a movie – you pick a location, put up a set, shoot scenes and you’re done. You can’t go back and refilm a segment all over again.
How Do You Bridge the Gap?
Kreindlin: “sometimes it brings out the best of us. In every other company the technology is the base for creating the content. In our company the content – in this case the director’s dreams – brought with it the technology many of the times.
Don’t You Fear the Audience Might Not Be Patient to the Technological Challenges?
Kreindlin: “I’m not ashamed to say that the technology is still primery. If nobody takes it and do it it’ll never get anywhere, somebody got to be the first. The theaters are conservative, they will never take it forward if we don’t push it. Our audience needs to understand thy’re all a part of an experiment, that technical faults might occure, and that we might have to start scenes from the top.
A 35 Billion Dollars Reality
Investments in startups of AR and VR made it up to 2 billion dollar on 2017, in comperison to 1.1 billion dollars in 2016 and to 3 billion dollars in 2012-2015. According to very carefull estimations advertised by Goldman Sachs Investment Bank, by 2025 the AR and VR solution market will be rolling revenues of about 35 billion dollars, the field of shows will take about 12% of it, and the gaming field up to one third.
While Virtual Reality instruments are bringing the user into the digital world, Augmented Reality adds layers of information over the existing reality.
Pokémon Go, a game launched in 2016 on which players run around real neighborhoods and trying to locate virtual animated figures using their smartphone’s screen, was a wonderful demonstration to the business opportunity. The game gained 65 million users in the US alone, in no more than a week from the day it was launched – more users than Twitter, an added to the value of the Japanese company “Nintendo”, which owns stocks of the company that created the game, almost 10 billion dollars’ worth.
The smashing success of the game captured the imagination of everyone who are targeting young audiences. Restaurants, coffee shops and small retailers purchased virtual Pokémon figures to “hang around” their business place and attract the players to walk in.
One of the companies that is most identified with the genre is “Magic Leap”, founded by the former Israeli citizen Rony Abovitz, which holds a develop center in Israel. The company raised 1.4 billion dollars, according to the value of 4.5 billion dollars from strong companies in the Silicon Valley, such as Google, which considers it to be its wing of Augmented Reality, the Chinese company Alibaba and the Andreessen Horowitz Fund. Magic Leap has been spreading promises for several years now about a very ambitious product its developing and is gaining vast coverage for it – a technology that will allow to combine virtual objects in the actual space, by transmitting a digital light-field, captured by the eye, the same way the eye can notice analogical objects. That being said, outside the company’s walls, very few have had the privilege to witness the technology with their own eyes.
Apple, which is looking for a groundbreaking device to inherit the iPhone, presents plans according to which the technology to produce Augmented Reality goggles will be ready on 2019, and that the first product will see the market on 2020. Tim Cook, sees Augmented Reality as a less isolating technology than Virtual Reality, and with a serious potential to be as revolutionary as the smartphone. Speaking to analysts while publishing the last quarterly report, Cook dedicated to this subject almost the same amount of time he dedicated to the increase in sales. “We can already see the things that will change the way in which you work, play and learn”, He said. “We believe that Augmented Reality is about to change forever the way we use technology”.
