• Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.

Lifejumper

Member
Oct 25, 2017
25,246
Instead of building massive sets, Lucasfilm is utilizing new "Stagecraft" technology that uses a rear-projected LED screen to create a reactive environment that essentially works as an immersive real-time green screen.

mandalorian-stagecraft-photo1-768x551.jpg

As you can see, the screen surrounds the entire set and allows the crew to shoot from any angle. The technology allows the screens to change how the environment looks based on how a scene is being shot.

mandalorian-stagecraft-photo7-e1579883127699-768x462.jpg

In this shot from the sixth episode of the series, you can see the busy background of the docking bay on Ranzar Malk's ship. Parts are strewn all over the place, and sparks fly from people fixing their ships. It would take countless props and crew members to bring that set to life in the real world[

But here, it's created digitally, with the assistance of physical foreground pieces to help blend the real with the digital.

mandalorian-stagecraft-photo5-768x457.jpg

The Stagecraft technique is even used for some shots of the Razor Crest in space. Again, a completed practical ship isn't needed, and the screens allows the camera to move around the cockpit of the ship without having to worry about replacing a green screen in post-production with visual effects

More here:

In depth look at the new tech:



Game changer for smaller scale productions.

edit:

Article can be found here. Page 45.

 
Last edited:

Rogue Blue

Avenger
Oct 27, 2017
2,269
Holy crap that's cool.

Really neat technology being used here.

So this is pretty much the new green screen, but now with elements that the actors can actually interact with in real time as opposed to...green screen.
 
Oct 26, 2017
902
Netherlands
Eager to see the results with the improved version of the tech they're apparently gonna be using for season 2. It looked really great in season 1, though apparently brightness was a bit of an issue (which is part of the reason even the HDR version is pretty dark). Apparently the updated tech is able to output at much brighter levels.
 

BAD

Member
Oct 25, 2017
9,565
USA
Interesting they'd rather try to pre-produce the CGI backgrounds than go deep on tailoring the backgrounds on green screen footage. I wonder what the pros and cons are.
 
Oct 25, 2017
10,708
Eager to see the results with the improved version of the tech they're apparently gonna be using for season 2. It looked really great in season 1, though apparently brightness was a bit of an issue (which is part of the reason even the HDR version is pretty dark). Apparently the updated tech is able to output at much brighter levels.

Yeah, the foreground lighting especially had a 'look' to it, relying on more diffuse lighting situations since I don't think the ability to match direct sunlight brightness with their setup was quite there.
 

Hella

Member
Oct 27, 2017
23,391
Wowser, this tech is amazing. It works so well I always assumed it was an actual green screen, but I never got that 'Green Screen' feel in most/all(?) of the shots, which is something that hadn't ocurred to me until now. Mandalorian had a lot of good environmental shots too, always looked super high budget.

(What I call the the 'Green Screen' feel is where the actors look like they're in another dimension next to the sets or background. You can see it a lot in the Prequel Trilogy. There's probably a better term, I dunno.)
 

jelly

Banned
Oct 26, 2017
33,841
Wow, that's really clever. I wonder if it will ever get past the green screen effect.
 

BAD

Member
Oct 25, 2017
9,565
USA
Wowser, this tech is amazing. It works so well I always assumed it was an actual green screen, but I never got that 'Green Screen' feel in most/all(?) of the shots, which is something that hadn't ocurred to me until now. Mandalorian had a lot of good environmental shots too, always looked super high budget.

(What I call the the 'Green Screen' feel is where the actors look like they're in another dimension next to the sets or background. You can see it a lot in the Prequel Trilogy. There's probably a better term, I dunno.)
Yeah, same with Marvel movies. It's very obvious when they were lit or shot in front of blue or green screens.
 

Lmo2017

Member
Oct 25, 2017
2,103
To the east of Parts Unknown...
Amazing, this is the kind of tech Lucas needed for the Prequels. The challenge has always been to try and get all these types of elements to mesh properly and now you can just see it and adjust in real time, maddness.

From playing a film out of focus behind a car to this!
 

Stinkles

Banned
Oct 25, 2017
20,459
I know folks sort of think of this stuff as cheating or "we'll fix it in post" and thanks to how the prequels were made, it gets a bad rap for actors, but the amount of craft and skill and prep to make it not just work, but additive holistically is seriously impressive. We're making tons of practical sets and objects and stuff for our project, but seeing VAST soundstages completely dressed for blue screen kinda triggers the imagination. You see the people and props and your brain starts filling in the missing vistas.


And great actors and directors do that automatically to immerse themselves and us in the reality of it.
 

metsallica

One Winged Slayer
Member
Oct 27, 2017
11,648
So Slash Film just cropped the leaked imgur pictures? OK.

Does anyone know where I can buy this issue of American Cinematographer to read the full thing? Or is there a link online?
 

smurfx

Banned
Oct 25, 2017
10,578
do they redo shots with the background being green just in case they want to change some stuff later?
 

collige

Member
Oct 31, 2017
12,772
The resolution on that video footage must be insane for them to just be filming it straight up. Maybe the distance to the screen is helping though, it's hard to tell the scale.
 

Deleted member 2533

User requested account closure
Banned
Oct 25, 2017
8,325
Yeah, the foreground lighting especially had a 'look' to it, relying on more diffuse lighting situations since I don't think the ability to match direct sunlight brightness with their setup was quite there.

It's hard to do on a green-screen as well, but at least now, the lighting department and cinematographer can look through the lens and see a phyisical representation of any light sources, and attempt to match them up with stage lights. And if they can't get it to work, they can change the real-time video screen to get the virtual lighting closer to the physical lighting.
 

winjet81

Member
Oct 27, 2017
1,010
Watched the Dumbo remake the other day and one of the things that was striking about it was the terrible and obvious green screen sets that were used.

Tim Burton could have really used this technology.
 

kurahador

Member
Oct 28, 2017
17,499
Damn, really impressive tech. This will help make tv shows looks more expensive than they really are. Having a Star Wars property using this, hopefully the maker can expand it.
 

Cuburger

Member
Oct 28, 2017
10,975
Really cool. I didn't know it was being used for Mandalorian, but I saw the tech before. Maybe it was just the Unreal demo video in the OP.

It really is incredible and seems like a breakthrough in filmmaking potential. I think Favreau also used Unity to prerender sets for Lion King and Jungle Book and used virtual camera rigs to figure out shots. This is a pretty awesome blend of the practical and the digital though, and it really seems like it could be a huge help in making the Disney Plus shows look amazing while keeping the budgets manageable for TV.
 

Kinthey

Avenger
Oct 27, 2017
22,249
I believe the first time this used in a modern production was in 'Oblivion'





What's funny about it that it's really just the super high tech version of how they used to film car scenes

 

Cuburger

Member
Oct 28, 2017
10,975
The resolution on that video footage must be insane for them to just be filming it straight up. Maybe the distance to the screen is helping though, it's hard to tell the scale.
They could also replace the backgrounds or up-res them if they have to. I found an article where Favreau talked about the technique:
Said Favreau: "We got a tremendous percentage of shots that actually worked in-camera, just with the real-time renders in engine, that I didn't think Epic was going to be capable of. For certain types of shots, depending on the focal length and shooting with anamorphic lensing, there's a lot of times where it wasn't just for interactive – we could see in camera, the lighting, the interactive light, the layout, the background, the horizon. We didn't have to mash things together later. Even if we had to up-res or replace them, we had the basis point and all the interactive light."

It also seems like it makes it so the lighting of the LED screens helps a lot for reference, even if they have to replace it with better quality versions later.
 

hikarutilmitt

Member
Dec 16, 2017
11,393
Interesting they'd rather try to pre-produce the CGI backgrounds than go deep on tailoring the backgrounds on green screen footage. I wonder what the pros and cons are.
If you think about it, it's combining greens screen sets a lot of movies use now with the older style of having a built set with a painted backdrop. In this case, you get the advantage of the former (any background you want) with the presentation of the latter (a "real" backdrop) but without the drawbacks you would normally have with each of them: green screen seams, the work associated with not having them or the background looking like a painting and having no life.
do they redo shots with the background being green just in case they want to change some stuff later?
Theoretically, you wouldn't really need to do that because you can just go to the set and load your set and do it again. You're still filming anyway, you're still getting in reshoots for scenes, but now you can tweak it on-set.
The resolution on that video footage must be insane for them to just be filming it straight up. Maybe the distance to the screen is helping though, it's hard to tell the scale.
It's... not stupid high but it's also not so low you can tell. Part of this is that you're going to soft focus the background anyway, so a lot of imperfections will be hidden by editing anyway. The brightness issues mentioned earlier are already being worked out, so we're going to have some really spectacular things going forward, to say nothing of much cheaper production costs.
 

metsallica

One Winged Slayer
Member
Oct 27, 2017
11,648
I know I'm asking in the wrong place, but does anyone know if the article is in the Jan 2020 issue of American Cinematographer or the Feb 2020 issue? I want to buy the right one, though the Feb issue doesn't appear to be for sale yet. The TOC on the Jan issue lists nothing about Mando.
 
Oct 25, 2017
4,424
Silicon Valley
Yep. Been talking with my studio about this. Ever since Oblivion and special cases like Gravity (for the close ups on the helmets) it was something we've been very keen on looking into, as it would allow us to do a lot more with the smaller budgets we have.
 

DemonCarnotaur

Community Resettler
Member
Oct 25, 2017
4,223
NYC
One thing I curious about, doesnt this mean more time in preproduction, locking in shots for these plates/set extensions?

I just know a lot of Studios dont like to operate that way, so I'm quite curious on how this meshes with the usual workflow
 

collige

Member
Oct 31, 2017
12,772
They could also replace the backgrounds or up-res them if they have to. I found an article where Favreau talked about the technique:


It also seems like it makes it so the lighting of the LED screens helps a lot for reference, even if they have to replace it with better quality versions later.
What I don't get is how would they be able to replace the video stream in post when there's no color to key against anymore?

One thing I curious about, doesnt this mean more time in preproduction, locking in shots for these plates/set extensions?

I just know a lot of Studios dont like to operate that way, so I'm quite curious on how this meshes with the usual workflow
From the article, it seems like it's exactly what it sounds like.
"This really requires [a filmmaker] like Deb who plans. This is not for somebody who's winging it. You absolutely need to plan ahead because in essence what you're doing is you're taking a lot of the post-production, and you're putting it in pre-production. You need to know your story, you need to know what those limitations are, you need to know what those effect shots already are so that they can be put onto the screen. So, Deb can talk to it practically, but it really does take a plan."

The tech is probably a lot more handy when you're using CG sources.
 

Cuburger

Member
Oct 28, 2017
10,975
What I don't get is how would they be able to replace the video stream in post though?
Probably just use similar techniques as dealing with an actual green screen. They might have to rotoscope/manually cut out the actors from the background rather than just replacing the green screen, but the lighting on the actors should be accurate and they could probably use the virtual camera movement data to get the exact background shots in Unreal Engine without the actors.
 
Article + high res screens
OP
OP
Lifejumper

Lifejumper

Member
Oct 25, 2017
25,246

Hampig

Member
Oct 25, 2017
1,703
Far future this might make smaller productions look more expensive, but from what I've read right now it's actually a pretty pricey way to do things, most specifically because of the type of material needed for the giant screens.
 

collige

Member
Oct 31, 2017
12,772
Probably just use similar techniques as dealing with an actual green screen. They might have to rotoscope/manually cut out the actors from the background rather than just replacing the green screen, but the lighting on the actors should be accurate and they could probably use the virtual camera movement data to get the exact background shots in Unreal Engine without the actors.
Yeah, the bolded is what I was thinking. Seems like it would be faster to just do reshoots
 

Cuburger

Member
Oct 28, 2017
10,975
Yeah, the bolded is what I was thinking. Seems like it would be faster to just do reshoots
But from what I understand Favreau said that most of their shots are good enough that they don't need to replace them, it was just the occasional shot that might not have looked sharp enough or whatever. Even with green screen, keying and compositing still requires an artist to make sure the edges look good, that foreground objects aren't mistaken as part of the background, and that the lighting of the green screen isn't noticeably being reflected in the foreground objects. I'm sure that while it may create more work in some ways, it also makes it a lot less work in other ways.

It does seem like this technique would be perfect for reshoots in general though since there isn't the need to reconstruct a set.
 

Gifmaker

Banned
Oct 29, 2017
964
What I really like about this technique is that I have used pretty much the same kind of concept before, working on a Lego Star Wars webcomic:

8-1.jpg


Though the quality cannot be compared and I only used my TV as the background, the advantages of matching lighting, reflections and not having to key the background out in post were evident back then already. Though ironically, I ended up replacing most the backgrounds anyway in later shots, as the nature of miniature photography made the backgrounds too much out of focus most of the times:

Me in 2017 said:
The asteroid field background in the control room scene was inserted in post production. But during shooting, the build was photographed against a TV displaying an even more elaborate asteroid field image created specifically for this purpose. But due to camera angles and depth of field, the background was never clearly visible and no asteroids could be recognized, so it was replaced in post. The original background with a blurred, blue-shining asteroid can still be seen in panel 8, in which no replacement was done.

Yeah, the bolded is what I was thinking. Seems like it would be faster to just do reshoots
Depends on the kind of rework they have to do. Though automated rotoscoping tools are getting better and better, plus if you have the camera motion tracked via gyro and the clean background plate, you can achieve a pretty good difference matte, too.

If I were to design those setups, I would have the background additionally lit with infrared light and capture that with an infrared camera during the performance as to get a perfect inverted alpha matte as well.
 
Last edited:

collige

Member
Oct 31, 2017
12,772
But from what I understand Favreau said that most of their shots are good enough that they don't need to replace them, it was just the occasional shot that might not have looked sharp enough or whatever. Even with green screen, keying and compositing still requires an artist to make sure the edges look good, that foreground objects aren't mistaken as part of the background, and that the lighting of the green screen isn't noticeably being reflected in the foreground objects. I'm sure that while it may create more work in some ways, it also makes it a lot less work in other ways.

It does seem like this technique would be perfect for reshoots in general though since there isn't the need to reconstruct a set.
I guess I was underestimating how much manual labor is still necessary for normal green screens. It just seems like cleaning up green screen footage would be far less headache inducing than cleaning up footage that obstenibly looks like the final product. The above infrared idea sounds genius though.
 

RyougaSaotome

Member
Oct 25, 2017
4,661
In some ways, this is a fascinating blend of new and old tech, or at least the concepts behind old tech.

Super cool.
 
Mar 3, 2019
1,831
I bet this is used in combination with CG after the fact to create a combined look that works. I can imagine this being a godsend for actors though, mich better than sitting in a giant green room
 

Cyanity

Member
Oct 25, 2017
9,345
I like how they set up physical set objects in the foreground that extend into the digital background, especially in cluttered shots. Makes the whole thing feel more believable.