In the Vive Pro Eye? Maybe, for some reason it feels like they've given off the idea that it's just the same resolution. But we'll have to wait for more details.
-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
-
We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.
HTC announces Vive Pro Eye -- VR headset with eye tracking for foveated rendering
- Thread starter Deleted member 12790
- Start date
The Pimax 8k does not allow for a native 8k input though. It uses upscaling from 1440p per eye.
Is it possible to import stuff like the Steam controller or the Steam Link where you live or is that off limits for whatever reason?
Great to see progress on this. I wonder how costly it is to implement...hopefully it comes down the cost curve quickly.
You can increase the rendered resolution where the eye is looking, and decrease it where it's not. So you can have a higher res display, for example, without necessarily having to drive a high resolution across the screen, but where the eye is focusing. So it makes it easier to drive higher res displays.
You can increase the rendered resolution where the eye is looking, and decrease it where it's not. So you can have a higher res display, for example, without necessarily having to drive a high resolution across the screen, but where the eye is focusing. So it makes it easier to drive higher res displays.
It doesn't magically improve resolution as that is locked in by the display, but it allows for much higher resolution displays to require the same workload as current displays. With a perfect form of eye-tracking and foveated rendering, you can go straight from the current 1080x1200 per eye of Rift/Vive to 5000x5000 per eye with basically no difference in performance.
If you know where the player is looking you can choose to just render that portion of the screen at high resolution. Peripheral vision can be extremely low resolution and not be at all perceivable as low res.
Turing specifically allows shaders to be lower fidelity in the out of focus portion of the screen as well (reducing complexity beyond just lower res), further increasing performance.
OP
OP
it doesn't improve the resolution of existing screens, rather it solves the problem of needing absurdly powerful hardware to push high resolutions. It makes it so that super high resolution screens are now viable, because our current hardware can now push enough pixels for them to work.
Like how you need so, so much more power to go from 1080p to 4k? Well, with foveated rendering, you no longer need those sorts of insane power increases, just a higher resolutions screen. Existing hardware, with foveated rendering, can in theory push resolutions higher than any screen that exists, without needing exponentially more powerful hardware. This is specifically due to quirks about how our own eyes and brains work, and how in truth we really only see a very tiny spot in the center of our vision at "full resolution" and our brain fills in the rest. It's similar to how games use level of details for models or frustum culling to reduce the amount of stuff that needs to be rendered, so it can focus on providing a very detailed scene. Like this:
Except instead of only culling down to the totality of your screen, it culls down to a tiny spot in the center of your vision, cutting out the vast, vast majority of things that need to be (expensively) rendered otherwise.
Yeah we bought eye tracking glasses like three or four years ago for $30k. A year later SMI brags in an interview they're confident they can bring the price down to $100 for HMDs. Thanks you fucks, lol. That it's apparently already ready for consumer headsets is stupefying really. I guess if the tracking tech is not yet quick enough they can make the high resolution rendering area a lot bigger, meaning it's a bit less efficient but still (I say this while having no idea what the average eye saccade number of degrees is though). Amazing.Keep in mind, eye tracking kits already exist:
they're just in the price ranges of tens of thousands of dollars, and have been until super recently way too slow to actually accomodate foveated rendering (since the tracking has to be lightning quick). Seeing a consumer headset with eye tracking this early is honestly mind blowing.
Super simplified explanation: allows the device to render high detail where you're looking, and spend less effort on the rest.
The human eye only offers detail vision on a very small area of our field of vision the area of our retinal membrane that deals with this super important part of our vision is called the Fovea.
(Which is you can't read words on one side of a display and the other side, without your eyes moving)
With eye tracking you can render the part that you are looking at with maximum resolution, the best anti aliasing, the highest quality texture.
All of the rest of the image can be rendered at a level of resolution and level of detail we physically discern any difference of.
It means 8K screens could be inside the headset, but you are only rendering an area 5% of that in full resolution at any given time as the fovea is so small.
This potentially makes it cheaper to render for VR than it does to render on a traditional display, albeit the framerate requirement is more important
I meant from even introduction into the ecosystem. But you have to start getting this into the hands of devs to have it established once it's out. Partnering with Nvidia for that is bad news for ecosystem lock in. But good news for progress in the area though. Trade off. Exclusivity for speedier/effective dev.
OP
OP
It's super exciting because it's evidence that VR tech is driving graphics technology at a skyrocketing pace. As in, way, way faster than most people -- even Michael Abrash -- imagined. This is a tech that went from flat out not existing, to being tens of thousands of dollars, to now being consumer tech in like the span of 5 years, when optimistic expectations were 10-15 years.
Even if this winds up being like $3k, which I actually doubt it will be, it would still be an insane price drop for the tech, something that happened way, way quicker than other visual technologies.
Yeah, I mentioned mobile VR shortly after posting my previous comment. This seems like the perfect tech for all three platforms - PC, consoles and mobile.
OP
OP
No details yet, but foveated rendering is a holy grail graphics technology that, even if it's exclusive to Nvidia at first, surly AMD will have their own solution soon enough.
So the way the fovea of the eye works. Is we have a point of absolutely clarity, then everything around it gradually loses focus until we hit our peripherals. Effectively when we look at any screen, but VR especially there is unnecessary clarity that's taking up processing power by having to render more in higher fidelity on the screen than we actually need. So assuming we know where the person is looking, and based on what we know about the eye/vision. We can effectively render much less of the screen in high detail at any given time. Reducing the cost of rendering that scene dramatically.
Up to 80% I believe if we just go by normal usage and not more advanced methods on top.
Foveated rendering is integral to mobile VR's future in my opinion.
Is it going to take a lot of work for game engines to accommodate this? It's like you have two render targets, the one the player is looking at and all of the rest, and the game needs to switch between them nearly instantaneously in response to eye movements, otherwise you'll see pop-in or briefly blurry IQ every time you make a saccade
OP
OP
Another thing worth talking about RE: Foveated Rendering is that it'd be an enormous break through for those with limited mobility. I think about people who are paralyzed from the neck down, and a technology like this could change their life. This type of tech could allow for the development of entirely gaze-controlled UIs.
Question from an ignoramus: Would the response time of foveated rendering come into play? Time between moving your eye to a new position vs the apparatus + 3D engine responds to your movement. Or is that a non issue?
That's technically the result of eye tracking rather than foveated rendering, but it's a good point. There are loads of uses for eye-tracking, one important one being socialization.
So good, if PSVR2 can get this tech on board by 2021 or 2022 I'll be there day 0, but if it isn't and stays PC only or too expensive I'll be a sad panda. This tech would fit consoles perfectly, because you just can't throw ever more power at it like on a PC.
VR is gonna get so awesome in the next couple of years.
VR is gonna get so awesome in the next couple of years.
OP
OP
No, it's a huge issue, and the main reason foveated rendering has been a "holy grail tech." In silence, behind the scenes, over the last several years, the tracking tech for eyes has been getting exponentially better. It's improvements in ways people not paying attention to larger pictures wouldn't quite understand. For example, things like predictive technologies used to assist with things like head tracking in VR, is also applied to speed up eye tracking.
The announcement of the first consumer headset with eye tracking is a signal that the technology has progressed much, much faster than even the most optimistic expected.
OP
OP
Eva Hoerth asked for some examples from researchers about what this will all mean, and I think this example is awesome:
OP
OP
Eye tracking and foveated rendering have lots and lots of immediate benefits, even before screens catch up to them.
I imagine the resolution increase will be on the software side here limited by the Vive Pro panel (if the Vive Pro Eye doesn't feature a better panel)?
Remember this:
https://www.roadtovr.com/varjo-bionic-display-headset-is-a-breathtaking-preview-of-vr-future/
Such an increase in image quality is not going to happen here with whatever HTC is coming up with I'd assume!
Remember this:
https://www.roadtovr.com/varjo-bionic-display-headset-is-a-breathtaking-preview-of-vr-future/
Such an increase in image quality is not going to happen here with whatever HTC is coming up with I'd assume!
So what about Valve's nuckle controllers? Did HTC break with Valve?
I might just buy my own VR headset this year, but I don't really know what to buy yet. This seems like it's amazing tech, but the price will probably be bonkers. And I can't wait for the knuckle controllers Valve has been working on.
I might just buy my own VR headset this year, but I don't really know what to buy yet. This seems like it's amazing tech, but the price will probably be bonkers. And I can't wait for the knuckle controllers Valve has been working on.
The eye tracking should ideally be faster than a single frame so that, with a little help from prediction based on eye velocity, artifacts will be as noticeable to the eye as frames rolling over are with your current monitor (aka not visible at all).
So this is a virtualink port driven headset? No Wireless Adapter Support? Kind of a sidegrade in the short term compared to the 2018 Pro, unless other specs have been upgraded.
This will be something REALLY big. Imagine a VR headset with a high enough resolution to show 4K games. With foveated rendering you only need to render 90% of the pixels, that alone will give you incredible performance gains from playing games in VR. That also applies for traditional games as well. When you compare that to a traditional monitor/TV that has to render a lot more pixels to fill the screen, the performance savings will be amazing.
Last edited:
I mean sure exciting to see progress, but it was also something we knew was coming. Abrash's annual predictions put it at like 2021/2022 for it to be integrated into a consumer headset, which I interpreted as falling in the $400-600 price range.
Since this is part of the "Pro" line it's targeted for enterprise so I would be surprised to see it cheaper than $1000 for the HMD Only and based on the few articles I read there are no other major upgrades to the resolution or anything. It's the Vive Pro with built-in eye tracking.
As far as I know HTC has not announced the hz for the eye tracking system on the Pro Eye, but I guess it should be around 250hz or more.
SMI's 250Hz Eye Tracking and Foveated Rendering Are For Real, and the Cost May Surprise You
The IR lights are invisible to the human eye but are picked up by each of the two Omnivision 250Hz camera sensors inside the headset, which use SMI's proprietary eye tracking SDK to tell the computer exactly where the user is looking. All of this information is processed by the hardware and software in under 2ms, allowing for fluid and most importantly accurate eye tracking.
"Getting over the 240Hz mark was important," says Villwock, "it allows us to track the saccadic motion of the eye." Saccadic motion being the unnoticed and involuntary motion of your eye as it moves between planes of focus.
https://uploadvr.com/smi-hands-on-250hz-eye-tracking/
You will understand it 100% after checking out this video (1:30:39 mark).
That's great news. I'll wait for real-world reporting but foveated rendering, if it's the real deal, is huge. Would love much higher displays for the upgrade from my OG Vive to be worth it, though.
That's cool as hell! Hope Sony is working on that for the PS5's VR. Some new controllers too...
All the more reason I'm saving up for my lasik. Already thinking vr is the best having to play with glasses or with vr insert lenses is the worst. Hopefully in a near future we have this plus the claw controller.
You are aware of how much it costs? Because adding that to PSVR2 would make it far more expensive that the PS5.
I don't expect them to launch PSVR2 alongside the PS5, so I expect the cost to come down in the interim. But maybe I'll be wrong.
Completely depends on how long they take. Oculus are aiming to get perfect eye-tracking along with many other major features in a consumer headset for 2022. There is no way they would ever release a headset higher than $600, so if Oculus can push all of that for sub $600 by 2022, Sony can likely push something for $400 if they don't have every advancement under the sun like Oculus.
No, you don't understand; this tech is far from being cheap enough for ever regular PCVR. If you want your PSVR2 to be $1000, you are in the wrong end of the VR gaming. if you want to spend that much then give up on Consoles, such tech are two gens out.
Sony is NOT interested in selling PSVR2 for $400. We saw how small a market that is, Sony want above all else to get the price down as priority. They are about selling to the mass market, if you want cutting edge then go PC. Sony would rather getting the headset down to $200, if not $150.
Frankly the only thing surprising me is that it took this long. Although I guess display resolution needed to get to a point where it made sense / where GPUs could not keep up.
For the people in the know, chances of PSVR2 using for foveated rendering?
Edit: I guess slim. What exactly is costly about it? The sensors / cameras tracking the eyes?
Edit 2: I guess I should read the thread, hahah. Didn't know these sensors were so expensive.
I know you know this, but your posts make it sound like FR is going to magically deliver an 8k+ image to users eyes
It will enable pcs to render at that res but we still need screens capable of displaying the images
An 8k image through even the vive pro lenses ain't gonna look crisp
There are microdisplays out there that can do circa 5k right now, but they're hugely expensive and it still will lose a lot when magnified for VR
But this is still exciting news
Edit - fucking mobile piece of shit
If this article is anything to go by there's a good chance. Eye tracking and Foveated Rendering makes even more sense on console VR, as the available power is more limited relative to what you can have on a PC and the hardware is locked for years. As soon as other headsets start to add eye tracking the prices should go way down.
SMI's 250Hz Eye Tracking and Foveated Rendering Are For Real, and the Cost May Surprise You
Right now, SMI's integration into current VR hardware requires a bit of handcrafting, it has to be hacked into the headsets. Then there are economies of scale. At small scale these parts, the software that drives them, and the labor to integrate them manually are too expensive for some sort of consumer upgrade driving the price upwards of $12,000 or more. That takes it wildly out of the range of the average consumer, but what happens when we get to a production scale of 1 million units?
"The total cost of all the hardware at scale is in the single digits," says Villwock, "a few dollars."
https://uploadvr.com/smi-hands-on-250hz-eye-tracking/