By Noela Hueso

Imagine this: It’s Chicago, 1963. What was previously known as the United States has been under the rule of the Axis Powers since the Allies lost World War II. Hitler has died and his successor, Heinrich Himmler, has constructed a plan for the internment of the formerly allied Japanese citizens in Chicago. Meanwhile, a rumor circulates that hacked state televisions show a place where the Allies had won the war. Citizens must choose between personal safety under the State or risk joining an underground resistance movement that is determined to fight the Reich.

Such is the scenario in A Most Favored Nation, an ambitious UCLA TFT Theater Season live-to-tape event inspired by the Amazon series The Man in the High Castle (and the Philip K. Dick novel upon which it is based). Its live-to-tape recording will be available for viewing beginning Saturday, Feb. 5 for the UCLA community at https://remap.ucla.edu/amfn.

Nation has been a bold undertaking since it was conceived in 2019 by TFT Theater Professor-in-Residence Jeff Burke as a live, immersive production slated for the 2019-20 Theater Season and supported by the Skoll Center for Social Impact Entertainment, Epic Games and Intel Studios. It was headed for TFT’s Freud Playhouse stage before it was delayed, like so many events, by the onset of the Covid-19 pandemic. Undeterred, Burke and his team continued to work on the project, making adjustments and modifications to conform to the limitations of students suddenly working remotely while only a small group of research staff and faculty remained on campus. (Since its inception, the project has included more than 30 production students from TFT, computer science and architecture; 22 student actors; eight faculty supervisors; and both research and production staff from the Center for Research in Engineering, Media and Performance (REMAP) and the Department of Theater.)

The goal from the beginning was to explore the possibilities of incorporating augmented reality and volumetric capture in live theater, a first at TFT and a concept that is still in its infancy in the theater world at large.

“There really aren’t many examples of AR theater productions that are using AR with characters at all,” says Nation co-director Mira Winick (BA ’20), “but there definitely is a push towards more [incorporation of] green screen and theater that doesn’t have to be in person.”

Intel’s 10,000 square-foot volumetric capture stage, which was touted as the largest of its kind in the world, would provide students the opportunity to have actors’ movements captured by multiple cameras in a 360-degree environment, and then place the resulting imagery within a virtual 3D environment built in the computer graphics application Maya. In Frank Spotnitz’s adaptation of Castle for Amazon, Burke found a device that lent itself perfectly to the objective: a story-within-a-story concept. “Grasshoppers” are film reels, created by the mysterious Man in the High Castle, that circulate surreptitiously and suggest that the Allies were victorious. (In the Dick novel, The Grasshopper Lies Heavy is a book that serves the same purpose).

“That, to me, was the fundamental condition of the work that we wanted to do was to have a way for the technology and the form of the piece to be connected to the story,” Burke says. “Thematically, a look at varied forms of fascism seemed timely and relevant; and on a practical level, getting permission to use this interesting original material as a jumping off point for the production was a key factor.”

The question that Nation asks the audience to consider, says co-director Jean Carlo Yunén Aróstegui (MFA ’20), is “‘What is it that moves you to put yourself at risk to enact change in the world around you?’”

Presented in two timelines — the bleak “in real life” (IRL), in which the Axis Powers have taken over and the more colorful but still challenged one in which the Allies have won, seen in augmented reality (AR) — each timeline has three divergent storylines that come together in the end; viewers must choose one to follow.

In one scenario, a Japanese family must make the decision whether to leave their missing son behind as they prepare to escape from being deported as an ascendant Himmler increasingly pressures; in another, Jimmy, a closeted gay man, gets access to a Grasshopper, which “shows him out and relatively happy in the other timeline, and he has to decide whether or not reject or accept that part of himself,” says Nation co-writer Jennifer Zhao (MFA ’21). The third storyline focuses on Sabine, a Black teenager, who is considered a non-citizen. “Already with the Resistance, she has been looking into the Grasshopper and has seen herself and this girl, Natsuki, as performers and really wants to have a piece of that life, so she tracks down the Natsuki in her world tries to convince her to join her cause.”

The idea of choice has always been inherent in the format of the show, says Winick, “and how the seemingly small choices can be really big. That’s part of the reason why you don’t get to experience the whole show — you have to make choices about what scenes you see and how that illustrates this idea that no choice is made within a vacuum, it all affects other things.”

In the pre-pandemic version of Nation, audience members, immersed in the experience as citizens of the State, would be on the Freud stage, experiencing the AR portion of the story through hijacked Volksfernsehen (“people’s televisions”), which all citizens are required to own to receive government propaganda (in reality, iPhones running the app that students created in game engine Unreal Engine). In the story, “the resistance learned to hack Volksfernsehen in a way so that instead of seeing propaganda, audience members see a window into the other (AR) world,” says Zhao, who wrote the script with Ryan Stevens (MFA ’20).

But when Intel Studios closed its doors in October 2020, a casualty of the pandemic, a crucial aspect of how Burke and his students wanted to present their production was eliminated and they needed to brainstorm another way to bring the story to life. Cameras, green screens and costumes were sent to student actors at home, and the team tested what it would be like to create images of them as 2D planes.

“We ended up using just one camera and shooting everything [in 2D] on a green screen,” says Winick, who helmed the AR portion of the show. “We captured the performances from the front and backs [of the actors] and then we put those into the 3D worlds that were built in Maya. When you’re seeing it on the phone, the characters auto-rotate a little bit to keep the illusion possible, and then once you pass a certain point, it flips the back perspective. Unless you’re in that in-between before it completely rotates, it’s incredibly convincing that the characters are 360.”

Planning for a live-to-tape experience also meant that sets were simplified, story beats evolved and that an environment needed to be created that would be accessible on any device, while still giving viewers agency in the production, as they would have if they were seeing it in person.

The team built out an interactive video platform in Cinema 8 in which a user interface on the screen allows viewers to “choose and toggle between views,” Winick says. “We really wanted it to still have this element of simultaneous action even though it isn’t the same as it would have been in person.”

Adds Yunén Aróstegui, who directed the IRL scenes: “The viewer has the option at the beginning to choose one scene and once you’ve chosen it, there are different camera perspectives that are audience proxies. One of the perspectives has only the IRL, while others have the overlay of alternative reality and real life. We tried to capture as best as we could on the live-to-tape through multiple perspectives that semi-mirror [live] audience movement.”

Understandably, it’s a relief to the team that the show is finally seeing the light of day.

“This project was supposed to take a year and then ended up being a two-plus year endeavor; we had a lot of people who left and people who came back,” Winick says. “The core group who were around from the beginning had to adapt to different iterations of the project to make it happen; when we were in shoots every night, they’d end up having to rebuild the app and recreate the scenes that they’d seen in dress rehearsals…they did it and it was amazing that it happened.”

Despite the delays and obstacles, the experience has been rewarding and Winick believes there will be more interactive productions in the future.

“It isn’t that outside the realm of possibility that students can put on productions like this,” she says. “Over time it will be more accessible, and we will start seeing it a lot more because it is compelling and powerful. Despite all the craziness of this project and the years we have been doing it, I would do another AR production in a heartbeat.”

“There’s something really powerful about it,” Zhao agrees. “It can achieve things that traditional cinema or theater cannot…this project forces you to move around and explore the space in a way that you just wouldn’t otherwise.”