On-set virtual production explained
On-set virtual production (OSVP) is an entertainment technology for television and film production in which LED panels are used as a backdrop for a set, on which video or computer-generated imagery can be displayed in real-time. The use of OSVP became widespread after its use in the first season of The Mandalorian (2019), which used Unreal Engine, developed by Epic Games.
History
Australian film director Clayton Jacobson first had the idea of improving the green screen that was then in use when filming a TV ad for detergent in 2003. Watching his son playing videogames and seeing the 3D technology used in them gave him the idea. Eventually, in 2016, Jacobson and his son made one of the prototypes for a virtual production stage in their shed, using a set of LED screens. However, he could not get anyone to take an interest in developing the technology further, so gave up on it. Other filmmakers had also caught on to the idea though, and in 2018 an Australian cinematographer, Greig Fraser, used the technology to film the Star Wars franchise spin-off series, The Mandalorian (released 2019). Instead of using the green screen during the filming stage, the team combined post-production with the production stage of the series. They installed huge LED walls linked to powerful computers that ran Unreal Engine gaming software (used for Fortnite, among others). They called this soundstage "the volume", a term already used to refer to a stage where visual effects techniques take place.[1]
Since its inventive use in The Mandalorian, which used ILM's StageCraft, the technology has become increasingly popular. Miles Perkins, industry manager of film and TV for Epic Games and maker of the Unreal Engine, estimated that there were around 300 stages by October 2022, increased from only three in 2019.[2] Most of these were built during or after the COVID-19 pandemic, when lockdowns meant that production studios had to find ways to produce films without traveling to other locations.[1]
In March 2023, the world's largest virtual production stage was opened at the Docklands Studios Melbourne, in the city of Melbourne, Australia.[1]
Terminology
On-set virtual production (OSVP) is also known as virtual production (VP), immersive virtual production (IVP), In-Camera Visual Effects (ICVFX),[3] or The Volume.
Technology
With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location.[4] OSVP can be viewed as an application of extended reality. OSVP contrasts with virtual studio technology, in which a green screen backdrop surrounds the set, and the virtual surroundings are composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like lens distortion, depth of field effects, bokeh and lens flare. This makes it a far more natural experience that more closely approximates location shooting, making the film-making process faster and more intuitive than can be achieved on a virtual set.
To render parallax depth cues correctly from the viewpoint of a moving camera, the system requires the use of match moving of the background imagery based on data from low-latency real-time motion capture technology to track the camera, such systems as Stype and Mosys which track camera position within the volume and equate this position within the Unreal scene - so that any movement is reflected within the scene as well .
Industry organizations including SMPTE, the Academy of Motion Picture Arts and Sciences, and the American Society of Cinematographers have started initiatives to support the development of OSVP.[5] [6] [7]
Examples
Stages that use OSVP include the various StageCraft stages, Pixomondo's Toronto-based LED stage, which has a long-term lease from CBS, ZeroSpace's Volume in New York City,[8] or Lux Machina various stages. In Japan, the LED wall and virtual production were used by Toei Company for its Super Sentai shows Avataro Sentai Donbrothers[9] and Ohsama Sentai King-Ohger, with the latter also being produced in collaboration with Sony PCL Inc.[10] Toronto's Dark Slope Studios is the home of Virtual Production shots for Heated Rivalry, Gen V and Netflix' BET.[11]
Productions using the technologies
Television series
Feature films
Short films
See also
Notes and References
- Web site: Purtill . James . Virtual production studios are replacing green screens as world's largest opens in Melbourne . ABC Science . ABC News. . 22 March 2023 . 24 March 2023.
- Web site: Giardina . Carolyn . 2022-10-19 . Too Much Volume? The Tech Behind 'Mandalorian' and 'House of the Dragon' Faces Growing Pains . 2023-02-04 . . en-US.
- Web site: In-Camera Visual Effects (ICVFX) . 2023-02-04 . www.nepgroup.com.
- Web site: 2021: The On-Set Virtual Production Initiative. 2022-02-22. www.smpte.org. en.
- Web site: 2022-01-18. Partnering to Develop Advisory Group of the On-Set Virtual Production (OSVP) Initiative.. 2022-02-22. systemscontractor. en.
- Web site: SMPTE Hollywood To Explore Lighting and Color Science For Virtual Production. 2022-02-22. SHOOTonline. 14 October 2021 . en.
- Web site: AMPAS Science & Technology Council Presents Virtual Production Primer - The American Society of Cinematographers. 2022-02-22. theasc.com.
- Web site: Abram . Cleo . Cleo Abram . We Tested Sci-Fi Movie Tech (MKBHD) . YouTube . 12 March 2023.
- https://cgworld.jp/regular/2301-293-donbro.html VFXアナトミー インカメラVFXによる特撮をTVシリーズで実現! 『暴太郎戦隊ドンブラザーズ』
- https://www.itmedia.co.jp/news/articles/2303/06/news112.html 新番組「王様戦隊キングオージャー」の背景CGがハイクオリティーと話題に グリーンバックを使わない最新撮影技術とは?
- Web site: Casemore . Jamie . Dark Slope unveils expanded 75,000-square-foot campus . 2026-01-10.
- Web site: Gartenberg . Chaim . February 20, 2020 . How The Mandalorian teamed up with Fortnite creator Epic Games to create its digital sets . live . https://web.archive.org/web/20200220203837/https://www.theverge.com/2020/2/20/21145671/mandalorian-sets-stagecraft-epic-games-ilm-fortnite-baby-yoda-digital . February 20, 2020 . February 20, 2020 . The Verge.
- Web site: Lane . Carly . June 9, 2021 . Exclusive: Here's When 'The Mandalorian' Season 3 Is Filming; New Details on 'Book of Boba Fett' Connection . live . https://web.archive.org/web/20210609204843/https://collider.com/the-mandalorian-season-3-the-book-of-boba-fett-production-filming/ . June 9, 2021 . June 9, 2021 . Collider.
- Web site: Giardina . Carolyn . 2022-10-19 . Too Much Volume? The Tech Behind 'Mandalorian' and 'House of the Dragon' Faces Growing Pains . 2023-02-04 . The Hollywood Reporter . en-US.
- Web site: July 29, 2022 . ILM STAGECRAFT HONORED WITH ENGINEERING, SCIENCE & TECHNOLOGY EMMY® AWARD . July 29, 2022 . Industrial Light & Magic.
- Web site: Peterson . Karen M. . April 29, 2022 . How 'Our Flag Means Death' Designers Mixed History With Broad Theatricality . July 16, 2022 . Variety.
- Web site: Bui . Hoai-Tran . June 17, 2020 . The Obi-Wan Kenobi Disney+ Series Will Use the Same Technology as 'The Mandalorian' . live . https://web.archive.org/web/20200618161937/https://www.slashfilm.com/obi-wan-kenobi-tv-series-technology-the-mandalorian/ . June 18, 2020 . June 18, 2020 . /Film.
- Web site: StageCraft Industrial Light & Magic . 2023-02-04 . www.ilm.com.
- Web site: Hogg . Trevor . Virtual Production Now and Going Forward . The Magazine of the Visual Effects Society . 2025-10-14.
- Web site: Dominguez . Noah . November 14, 2022 . Andor Actually Used StageCraft Technology Despite Early Reports . November 25, 2022 . ComicBook.com.
- Web site: Vegvari . Steve . Being Transported by Amazon Studios Virtual Productions at Stage 15 . 26 June 2024 . iPhone in Canada . 2024-11-07.
- Web site: HOUSE OF THE DRAGON MAY LOOK TOTALLY DIFFERENT IN SEASON 2 FOR ONE CONTROVERSIAL REASON . 25 October 2022 . Inverse. 19 October 2022 .
- Web site: Inside Dark Bay, the spinning LED volume at the heart of Netflix's upcoming '1899' . 20 November 2022 . TechCrunch. 10 February 2022 .
- Web site: Brooks . Dan . How ILM Brought Its Own Christmas Magic to The Santa Clauses . Industrial Light & Magic . 2025-09-24.
- Web site: Silo . Lux Machina . 2023-06-27.
- Web site: DNEG 360 Shows . DNEG . 2024-11-07.
- Web site: Failes . Ian . See Eyeline Studios' volumetric capture rig in action for 'One Piece' . 31 May 2024 . Befores and Afters . 2024-09-04.
- Web site: Failes . Ian . How Eyeline Studios' new volumetric capture tech was used on 'Yu Yu Hakusho' . 25 March 2024 . 2024-09-04.
- Web site: Giardina . Carolyn . June 28, 2022 . 'Percy Jackson and the Olympians' Takes ILM Virtual Production Route (Exclusive) . live . https://web.archive.org/web/20220628171224/https://www.hollywoodreporter.com/tv/tv-news/percy-jackson-and-the-olympians-ilm-virtual-production-1235171943/ . June 28, 2022 . June 28, 2022 . The Hollywood Reporter.
- Web site: MoTA = @cary_fukunaga . 2023-02-04 . www.instagram.com . en.
- Web site: Magnopus brings Amazon's Fallout series to life with virtual production powered by Unreal Engine . May 13, 2024. Epic Games . 30 June 2024.
- Web site: Adi . Tantimedh . May 14, 2024 . 'Doctor Who: "BOOM" Sees Steven Moffat Regenerate into Freelancer' . June 27, 2024 . Bleeding Cool.
- Web site: Fisher . Jacob . June 24, 2021 . 'The Acolyte' Aiming To Begin Filming In February In London . live . https://web.archive.org/web/20210630124408/https://discussingfilm.net/2021/06/24/the-acolyte-aiming-to-begin-filming-in-february-in-london-exclusive/ . June 30, 2021 . June 30, 2021 . DiscussingFilm.
- Web site: Hogg . Trevor . Virtual Production Now and Going Forward . VFX Voice . 2025-10-14.
- Web site: McPherson . Chris . May 5, 2024 . 'Star Wars: Skeleton Crew' Uses Stop-Motion by Phil Tippett and Old School Matte Paintings [Exclusive] ]. live . https://web.archive.org/web/20240506013930/https://collider.com/star-wars-skeleton-crew-matte-paintings/ . May 6, 2024 . June 27, 2024 . Collider.
- Web site: Basso . Por Iván . 2025-05-03 . La magia del virtual production en El Eternauta: "Desde Argentina se pueden hacer cosas muy grandes desde la tecnología" . 2025-05-05 . infobae . es-ES.
- Web site: Jones . Tamera . Weintraub . Steven . "I Haven't Done It Since 'True Blood'": Alexander Skarsgård Broke His Own Rules for Apple TV+'s Critically Acclaimed 'Murderbot' . Collider . 10 June 2025 . 6 August 2025.
- Web site: Failes . Ian . VFX Breakdowns A sneak peek at how Christopher Lloyd's Professor Orloff scenes were filmed for 'Wednesday' . Befores & Afters . 2025-08-27.
- Web site: Casemore . Jamie . Dark Slope unveils expanded 75,000-square-foot campus . 2026-01-10.
- Web site: Shield . Brian . Last Summer I spent time in Toronto on the new HBO series IT: Welcome to Derry as the Virtual Production Producer for some car process work. I can't wait to see how this turned out. I couldn't have done it without the amazing crew we had. Steve D. it was such a pleasure working with you and the whole VFX team on this. Looks killer! . LinkedIn . 2025-07-20.
- Web site: Casemore . Jamie . Dark Slope unveils expanded 75,000-square-foot campus . 2026-01-10.
- Web site: Casemore . Jamie . Dark Slope unveils expanded 75,000-square-foot campus . 2026-01-10.
- Web site: Autodesk, Lightstorm Entertainment and Weta Digital Drive Virtual Production Forward . Autodesk, Inc. . 2025-04-07.
- Web site: Oblivion . 2023-02-04 . Lux Machina . en-US.
- Web site: Desowitz . Bill . How the 'Interstellar' VFX Team Built Wormholes, Black Holes and Alien Planets . 14 November 2014 . Indiewire . 2025-07-17.
- Web site: Bardini . Julio . 2022-08-14 . Industrial Light & Magic's Digital StageCraft Technology: What We Know About the Volume . 2023-02-04 . Collider . en.
- Web site: Lux Machina Consulting on Instagram: "We had a great time working with Director of Photography Jess Hall on Ghost in the Shell to provide LED green screen replacement for some of the driving scenes in the movie. Contact us for help with any film or television projects! We specialize in capturing realistic environments in the camera as part of the visual effects workflow! Pictured: Pilou Asbaek and Scarlett Johansson in Ghost in the Shell from Paramount Pictures and Dreamworks Pictures." . 2023-02-04 . Instagram . en.
- News: Beyond Christopher Nolan: 'First Man' Redefines In-Camera VFX. Desowitz. Bill. October 15, 2018. IndieWire. November 2, 2018. en-US. https://web.archive.org/web/20181103042914/https://www.indiewire.com/2018/10/first-man-vfx-christopher-nolan-in-camera-1202012086/. November 3, 2018. live.
- Web site: Desowitz . Bill . 'Alita: Battle Angel': Here's How Robert Rodriguez Tried to Channel VFX Master James Cameron . 12 February 2019 . Indiewire . 2025-04-07.
- Web site: February 5, 2020 . How Cutting-Edge ILM Technology Brought 'The Mandalorian' to Life . June 26, 2021 . No Film School.
- Web site: The Irishman . 2023-02-04 . Lux Machina . en-US.
- Web site: Lux Machina Consulting on Instagram: "Congrats to @starwarsriseofskywalker and @theirishmanfilm for the many Oscar nominations this week, including for Visual Effects! We are so grateful to have been a part of both productions. Looking forward to the big night, Feb 5 live at 8e 5p! . . . . . . #virtualproduction #incameravfx #vfx #film #technology #videoproduction #led #projection #videoscreens #luxmc #luxmachina #consultants #engineers #programmers" . 2023-02-04 . Instagram . en.
- Web site: The One and Only Ivan . Technicolor Group . 2024-09-04.
- Web site: Mike Seymour . January 21, 2021 . The Midnight Sky's Virtual Production using StageCraft . June 26, 2021 . Fxguide.
- Web site: Filippo Magnifico . December 23, 2020 . In Vacanza su Marte: il film con Boldi e De Sica realizzato con una tecnologia simile a quella di The Mandalorian . May 30, 2022 . . 4 February 2023 . https://web.archive.org/web/20230204211204/https://blog.screenweek.it/2020/12/in-vacanza-su-marte-film-boldi-de-sica-mandalorian-762136.php// . dead .
- Web site: DNEG 360 Shows . DNEG . 2024-12-07.
- Web site: The Next Generation of Extended Reality Sounds Stages . 2023-02-04 . Lux Machina . en-US.
- Web site: Projects - Eyeline Studios . Eyeline Studios . 2024-09-04.
- Web site: Giardina . Carolyn . October 19, 2020 . 'The Batman' Using 'Mandalorian' Virtual Production Techniques . live . https://web.archive.org/web/20201019234512/https://www.hollywoodreporter.com/behind-screen/the-batman-using-mandalorian-virtual-production-techniques . October 19, 2020 . October 19, 2020 . The Hollywood Reporter.
- Web site: Lux Machina Consulting on Instagram: "Congratulations to @topgunmovie "Maverick" on its six @theacademy nominations, including Best Visual Effects and Best Picture! It was an honor for @luxmc_ to be part of such an iconic project by providing in-camera virtual backgrounds and working closely with "Maverick's" overall VFX supervisor Ryan Tudhope and Director of Photography @claudiomirandadop We are thrilled at how much the fans, the critics, and now the Academy of Motion Picture Arts and Sciences have responded to the film. The Oscars will be handed out Sunday, March 12, 2023. Get your popcorn ready! (@luxmc_ is part of @nep_group) #vfx #visualeffects #luxmc #awards #oscars #nominations #academy #topgun #NEP #NEPVS #topgunmaverick #sequel #ICFVX #film #photography #directorofphotography" . 2023-02-04 . Instagram . en.
- Web site: Carolyn Giardina . September 10, 2020 . Industrial Light & Magic Expands Virtual Production Services, Supports 'Thor 4' . June 26, 2021 . The Hollywood Reporter.
- Web site: Lux Machina Consulting on Instagram: "Wishing Eddie Alcazar the best of luck as he heads to #Sundance to premier his newest film "Divinity." A portion of the film was shot using the #LuxMC LED volume, and Junaid Baig, VP of Research & Development for #NEPVS, used AI to provide slow motion footage of already animated stop motion sequences on the film." . 2023-02-04 . Instagram . en.
- Web site: Villaverde . Noah . June 8, 2022 . 'Black Adam' Director Talks Using Lucasfilm's Volume & Other New Tech . July 19, 2022 . Heroic Hollywood.
- Web site: Devotion . DNEG . 2024-06-27.
- Web site: Kadner . Noah . 2023-04-24 . Virtual Production—When The Fabelmans Met The Mandalorian. 2023-04-26. Frame.io Insider . en-US.
- Web site: Virtual Production In Action . Virtual Production Malta . 2025-04-07.
- Web site: Erao . Math . May 17, 2021 . Ant-Man and the Wasp: Quantumania Director Shares Photo From The Mandalorian's Virtual Set . live . https://web.archive.org/web/20210517204323/https://www.cbr.com/ant-man-wasp-quantumania-mandalorian-volume-photo/ . May 17, 2021 . May 17, 2021 . Comic Book Resources.
- Web site: The Pope's Exorcist . Sensel Studio . 2026-03-03.
- Web site: Failes . Ian . How those twinning/doubling shots in 'The Flash' were done . 29 August 2023 . Befores and Afters . 2024-09-04.
- Web site: Barbie . Lux Machina . 2024-06-27.
- Web site: Our Work . DNEG . 2025-07-17.
- Web site: Inside ILM: The Making of The Creator . . 18 February 2024 .
- Web site: Retribution . Sensel Studio . 2026-03-03.
- Web site: Kadner . Noah . Steven Shulgach on LinkedIn: Adapting to new virtual production tech while implement traditional… . Linkedlin . 30 June 2024.
- Web site: Failes . Ian . Jan 10, 2024 . How ILM's StageCraft was used on 'The Marvels' . live . https://web.archive.org/web/20240110112709/https://beforesandafters.com/2024/01/10/how-ilms-stagecraft-was-used-on-the-marvels/ . 10 January 2024 . June 27, 2024 . befores & afters .
- Web site: Amazon Studios Debuts AWS-Powered Virtual Production Mega Stage . Amazon MGM Studios . 2024-09-04.
- Web site: Kadner . Noah . LED Wall Evokes Early-Cinema Effects for Poor Things . American Cinematographer . 30 June 2024.
- Web site: Behind the VFX of J.A. Bayona's Society of the Snow | Netflix . . 11 December 2023 . 30 June 2024.
- Web site: Aquaman and the Lost Kingdom: Volumetric Capture Technology by Eyeline Studios . 2 April 2024 . Art of VFX . 2024-09-04.
- Web site: Szalai . Georg . 'Dune' VFX House DNEG Teams With Dimension on LED Volume Stages in London, Rome . The Hollywood Reporter . 30 April 2024 . 2024-09-04.
- Web site: 'Godzilla vs. Kong' Sequel reportedly used 'The Mandalorian' LED video wall tech . 31 July 2023 . Medium . 2025-06-05.
- Web site: Christy Hall Shot Daddio Entirely Inside a Taxi Cab. 12 September 2023.
- Web site: Transforming the Origins of Optimus Prime: Christopher Batty on His Role in Transformers One . ProductionHUB . 2025-08-27.
- Web site: Couch . Kim Masters, Scott Feinberg, Aaron . Masters . Kim . Feinberg . Scott . Couch . Aaron . 2023-01-09 . Francis Ford Coppola's 'Megalopolis' in Peril Amid Ballooning Budget, Crew Exodus (Exclusive) . 2023-02-04 . The Hollywood Reporter . en-US.
- Web site: Joker: Folie à Deux . Industrial Light and Magic . 2024-11-07.
- Web site: Venom: The Last Dance . Industrial Light & Magic . 2024-12-07.
- Web site: Wells . Chris . How virtual production transformed the latest Robert Zemeckis film 'Here' . Red Shark News . 2025-07-17.
- Web site: McGowan . Chris . Welcoming the Cultural Shift to Virtual Production . VFX Voice . 2025-10-14.
- Web site: On-Set Visual Effects Services . The Eisley Company . 2025-07-17.
- Web site: Sharf . Zack . Barry Jenkins Says All-Virtual Filmmaking 'Is Not My Thing' After Directing 'Mufasa'; Disney Sent Him a Note Worrying That One Long Take Was Too 'Slow' . 5 December 2024 . Variety . 2024-12-07.
- Web site: Zoller Seitz . Matt . Moonlight in the Lion's Den: Why Barry Jenkins gave up his improvisational shooting style to spend four years making 'Mufasa' on a soundstage . 5 December 2024 . Vulture . 2024-12-07.
- Web site: Chapman . Rob . MPC, Snow White, Virtual Art Department FX Artist / TD . ArtStation . 2025-06-05.
- Web site: Priestley . Jenny . Providing the virtual production building blocks for A Minecraft Movie . 9 May 2025 . TVB Europe . 2025-07-20.
- Web site: Hooper . Simon . How they made the skyview restaurant disaster in Final Destination Bloodlines . Any Good Films? . 21 May 2025 . 2025-06-05.
- Web site: James Gunn Unveils Behind-the-Scenes Glimpses of 'Superman,' Soaring with Virtual Production at Trilith Studios . PeachPopFive . 2025-08-27.
- Web site: Eyeline Studios and Scanline VFX recently hosted a screening in Los Angeles to celebrate their work on Happy Gilmore 2. After the screening, Eyeline's Virtual Production Supervisor Tarik Thompson shared how volumetric capture technology and virtual production stages were used to help create the world of Happy Gilmore. He also highlighted the collaboration between Eyeline and Scanline, led by VFX Supervisor Dann Tarmy, to deliver a film that balances nostalgia for longtime fans with fresh energy for a new generation. Congrats to all our artists and technicians who worked on this film. From tournament mayhem to flying golf balls and Happy flashbacks, the teams at Eyeline and Scanline brought their A-game to every frame. #eyelinestudios #scanlinevfx #happygilmore #golf #sports #tournament #adamsandler #netflix #streaming #visualeffedts #virtualproduction #screening (photos by Alex Chun) . LinkedIn . 2025-08-27.
- Web site: Taylor . Drew . 2025-10-18 . How Anime Influenced Kogonada’s ‘A Big Bold Beautiful Journey’ . 2026-03-03 . TheWrap . en-US.
- Web site: Failes . Ian . 2025-11-19 . How the bus scenes in 'The Lost Bus' were filmed . 2026-03-03 . befores & afters . en-US.
- Web site: Failes . Ian . December 4, 2025 . The ‘TRON: Ares’ issue is here! . live . 2026-03-03 . befores & afters.
- Web site: Bittorf . David . The Making of Avatar 3 . ArtStation . 2026-03-03.
- Web site: Tour Amazon MGM Studios' virtual soundstage that's revolutionizing movies and TV shows through the power of AWS . 20 June 2024 . Amazon. 2024-09-04.
- Web site: Virtual Productions . Sensel Studio . 2026-03-03.
- Web site: Campbeltown '69 . Dimension Studio . 2024-11-07.
- Web site: WWII thriller filming on Melbourne's cutting-edge virtual production stage . FilmInk . 2025-09-06.
- Web site: Lux Machina Consulting on Instagram: "What does it take to make an award winning film? We're glad you asked! Watch the new behind the scenes video and find out how the creative artists were able to seamlessly blend live action video with stop-motion for @eddiealcazar 's #VimeoBestOfTheYear award-winning short film "The Vandal." In "The Making of The Vandal," you'll see how we used our #LUXMC LED stage and interactive lighting sources for the miniature photography. Watch "The Making of The Vandal" via the link in our bio." . 2023-02-04 . Instagram . en.