Digital Foundry: Project xCloud Beta Tested: Can Streaming Deliver A Handheld Xbox One Experience?

chandoog

Member
Oct 27, 2017
12,709


Rich shares his experiences with the Project xCloud beta and goes the extra mile in getting the best possible xCloud vs Xbox One S visual comparisons, latency metrics - and in the process solves the whole phone/clip situation...

Subscribe for more Digital Foundry: http://bit.ly/DFSubscribe Join the DF Patreon to support the team more directly and get access to pristine quality video downloads: https://www.digitalfoundry.net Background Music: Summer of Synth by Chris Huelsbeck - https://chrishuelsbeck.bandcamp.com/t...
 

harz-marz

Member
Oct 28, 2017
1,090
I think it's cool that this tech exists and it may even be the future but in the present, it simply lags too badly. I have a 400mb connection at home and games such as Tekken 7 and Forza have been almost unplayable. Yes some game genres will work better than others I know.... But my own personal experiences have not been good with cloud streaming.

Which is a shame as I was willing to give it a chance!
 

Nooblet

Member
Oct 25, 2017
5,208
Ace Combat and Tomb Raider work well.
Halo 5 feels quite janky and unplayable due to its pace, especially when fighting against prometheans.
 
Last edited:

gofreak

Member
Oct 26, 2017
3,065
Surprise surprise - 60fps (Killer Instinct) on xCloud lines right up, latency-wise, with 60fps tests Digital Foundry did previously on Stadia and PSNow. All, uncannnily, in the 12Xms range for 60fps fighting games tested on each.

That is to say... the tech is as good as it ever was, but it seems there are no new silver bullets for latency this year.
 

Theorry

Member
Oct 27, 2017
23,455
Very good vid. Also intresting about how different people feel about lag and how its different on a personal base.
Best thing. It does feel like a xbox one in a handheld mode and that it works and has a future.
 

RingRang

Member
Oct 2, 2019
827
Surprise surprise - 60fps (Killer Instinct) on xCloud lines right up, latency-wise, with 60fps tests Digital Foundry did previously on Stadia and PSNow. All, uncannnily, in the 12Xms range for 60fps fighting games tested on each.

That is to say... the tech is as good as it ever was, but it seems there are no new silver bullets for latency this year.
I got a good laugh out of people recently championing xCloud while crapping on Stadia's poor showing at launch. The technology is what it is, and that is laggy. How much a person is willing to put up with is the question.
 

zedox

Member
Oct 28, 2017
4,126
Surprise surprise - 60fps (Killer Instinct) on xCloud lines right up, latency-wise, with 60fps tests Digital Foundry did previously on Stadia and PSNow. All, uncannnily, in the 12Xms range for 60fps fighting games tested on each.

That is to say... the tech is as good as it ever was, but it seems there are no new silver bullets for latency this year.
You forget that xcloud has bluetooth mandatory as of right now so it actually has even more latency to deal with. It would be nice to see it when there can be direct connection. It's clear that xcloud isn't what people on this forum will like...but for the majority of gamers out there (the ones who don't even know what game mode on a tv is), it will be fine..and for the people who don't have access to consoles.

I got a good laugh out of people recently championing xCloud while crapping on Stadia's poor showing at launch. The technology is what it is, and that is laggy. How much a person is willing to put up with is the question.
I think the crapping on Stadia was how bad they botched it (like not giving out codes to people and waiting and messing up wires and such)...I don't think it was about the tech. I'm just happy that the tech works and I'm using both Stadia and xCloud.
 

Charpunk

Member
Oct 25, 2017
3,430
I got a good laugh out of people championing xCloud while crapping on Stadia's poor showing at launch.
People here were never going to give Stadia a chance. They were shitting on since the day it was announced.

Having used both, I don't personally see any difference in latency in either service using my phone. Both have worked really well for me.
 

efr

Member
Jun 19, 2019
1,709
Messed with this a bunch over the weekend.
Madden, halo, wwe were all very playable on cable 100down. Gears relies to much on twitch gaming and felt bad
 

Fezan

Member
Oct 26, 2017
817
Wait so all 3 cloud streaming service have similar latency? Why was I reading many comments here that xcloud feels much better than Psnow
 

Theorry

Member
Oct 27, 2017
23,455
Interesting remark about a cloud handheld from MS itself. Maybe they indeed do that. Now you still attaching things like a controller etc
What if MS made a "switch kind of thing" and makes it so it native wise has the best latency.
 

FairyEmpire

Member
Oct 25, 2017
17,244
I tried Forza Horizon 4 on it and while it's obviously not on par with my native 4K experience at home, it's more than playable. I don't expect to be playing Call Of Duty online or a rhythm game on streaming anytime soon, but what I could play so far is definitely good enough for 90%+ of the games I own. I don't see it turning my Xbox One into a useless product anytime soon, but I can't wait to be reliably continue my games when I'm on the move. And the tech, while being far from finished, is good already. Looking forward to be able to properly use the service when it launches in Italy and when (in a couple years) where I live will also be covered with good 5G. 4G is just a tad too little for it probably.
 

gofreak

Member
Oct 26, 2017
3,065
You forget that xcloud has bluetooth mandatory as of right now so it actually has even more latency to deal with. It would be nice to see it when there can be direct connection. It's clear that xcloud isn't what people on this forum will like...but for the majority of gamers out there (the ones who don't even know what game mode on a tv is), it will be fine..and for the people who don't have access to consoles.
The network would be the bulk of the latency. Stadia was test with wireless controllers - true, there's a direct-connect situation going on there, but it's also true, that ethernet was seemingly used here, while wifi was tested with PSNow. So the conditions of the testing aren't 1:1 across all these different scenarios, in ways that may favour each service - maybe in the future we'll get a same environment test, with the same connection, the same wired vs wifi etc.

But it is just remarkable how each hit almost exactly the same spot, down to a difference of single milliseconds, with 60fps fighting games, across all their different tests.
 

zedox

Member
Oct 28, 2017
4,126
The network would be the bulk of the latency. Stadia was test with wireless controllers - true, there's a direct-connect situation going on there, but it's also true, that ethernet was seemingly used here, while wifi was tested with PSNow. So the conditions of the testing aren't 1:1 across all these different scenarios, in ways that may favour each service - maybe in the future we'll get a same environment test, with the same connection, the same wired vs wifi etc.

But it is just remarkable how each hit almost exactly the same spot, down to a difference of single milliseconds, with 60fps fighting games, across all their different tests.
But all of them are tested in different factors as you just stated, so it's not "remarkable" as they are all in different situations...
 

Tharp

Member
Oct 27, 2017
3,251
I think it's cool that this tech exists and it may even be the future but in the present, it simply lags too badly. I have a 400mb connection at home and games such as Tekken 7 and Forza have been almost unplayable. Yes some game genres will work better than others I know.... But my own personal experiences have not been good with cloud streaming.

Which is a shame as I was willing to give it a chance!
Won't part of that be due to server location? Like Stadia for me, the closest Google server I can ping to is 50ms, and I can definitely feel the latency when gaming. But people who are closer probably won't notice it as much since it'll be lower latency and closer to a native experience. So if that's the same thing you run into, it might be something that improves in the future with more server locations.
 

LCGeek

Member
Oct 28, 2017
1,442
You can have lag in networking while your processing or making packets, fixable but not addressed. This is hidden and not added to latency measurements
You can have lag on hops, unfixable not worth addressing unless host/s sucks
You can have lag on wireless access point, thanks for mentioning still have no clue what kernel or features you're running so latency metric is basic.
You're account for some lag but totally ignoring one that can exist at the source. That's not my opinion it's what experts have found.
Thanks for accounting for bluetooth lag.

SQM works so those who have it will not have the problems of those you don't. Consumers need to use SQM to deal with Bandwidth problems becaused it exist their own lans not the isps or the cloud, no streaming tech or isp tech can fix something on your end. You've done this in two different vids now not accounting for the fact SQM literally fixes a problem you mentioned.

Seems you still need a highly debloated connection, decent kernel and hops not to suck on this tech. I actually felt xcloud was better than stadia even though it's beta.

Surprise surprise - 60fps (Killer Instinct) on xCloud lines right up, latency-wise, with 60fps tests Digital Foundry did previously on Stadia and PSNow. All, uncannnily, in the 12Xms range for 60fps fighting games tested on each.

That is to say... the tech is as good as it ever was, but it seems there are no new silver bullets for latency this year.
Saddens me they could be doing a bang up job in some areas but won't.

Latency isn't a silver bullet and its not helped if you use network features built for bandwidth but aren't latency friendly to test or outright just play on streaming service. None of these platforms aim to reduce latency in various areas so the results should laregly be the same. Based on exactly what you mentioned from test there is added 40-80ms added but since testing was limited and hops weren't properly fleshed out narrowing it down would be insanely hard.

I'm being blunt I seriously wonder what a debloated connection on haste would be like vs standard crap consumers are gonna see in test like these.
 
Last edited:

RingRang

Member
Oct 2, 2019
827
You forget that xcloud has bluetooth mandatory as of right now so it actually has even more latency to deal with. It would be nice to see it when there can be direct connection. It's clear that xcloud isn't what people on this forum will like...but for the majority of gamers out there (the ones who don't even know what game mode on a tv is), it will be fine..and for the people who don't have access to consoles.
Who are these people that don't have access to consoles? I know Google and Microsoft seem to believe lots of them exist, but I'm just not seeing it. If someone can afford to buy a Stadia kit, or a screen + controller for xCloud, I feel pretty confident they can just buy a $200 console if they really want one.
 

gofreak

Member
Oct 26, 2017
3,065
But all of them are tested in different factors as you just stated, so it's not "remarkable" as they are all in different situations...
I think it makes it even more remarkable - uncanny - that each hits, nearly to the millisecond, the same latency in the same type of game - despite years between some of these tests, the different connections etc. Quite the funny coincidence.

But coincidences and variances aside, I think it's completely fair to say there's no major tech breakthrough apparent in these results. They're bang in line with what you'd expect based on previous tests of cloud services from DF.

Wait so all 3 cloud streaming service have similar latency? Why was I reading many comments here that xcloud feels much better than Psnow
Everyone's going to have a different experience, both objectively and subjectively. That's the 'beauty' of cloud services. You might have a better experience with Stadia, I might have a better experience with xCloud, he might have a better experience with PSNow. Some people - like Digital Foundry it seems - might have quite a similar experience with each of them. It's going to vary completely by local setup - ironically - and even by time. I've been streaming with PSNow and privately from AWS with Steam, and at different points in time, and depending on what else is going on with my home network, have had better and worse experiences with each at different times. It's just how it is.
 
Last edited:

Belvedere

Member
Oct 27, 2017
1,296
Wait so all 3 cloud streaming service have similar latency? Why was I reading many comments here that xcloud feels much better than Psnow
Like others have said, there's no magic latency solving evolution introduced here as it performs right along Now and Stadia.

Would liked to have seen some sort of closer comparison to all three services from DF.
 

sirap

Member
Oct 25, 2017
2,801
South East Asia
My dreams of negative latency were short lived :(

Jokes aside, I'm genuinely disappointed that we've reached the limit of what's possible atm.
 

stealthyfrog

Avenger
Jan 6, 2019
8,136
I think it's cool that this tech exists and it may even be the future but in the present, it simply lags too badly. I have a 400mb connection at home and games such as Tekken 7 and Forza have been almost unplayable. Yes some game genres will work better than others I know.... But my own personal experiences have not been good with cloud streaming.

Which is a shame as I was willing to give it a chance!
Stadia tech is more polished with less input lag but there are so many problems surrounding the service itself and its business model.
 

TheZynster

Member
Oct 26, 2017
6,655
I think it's cool that this tech exists and it may even be the future but in the present, it simply lags too badly. I have a 400mb connection at home and games such as Tekken 7 and Forza have been almost unplayable. Yes some game genres will work better than others I know.... But my own personal experiences have not been good with cloud streaming.

Which is a shame as I was willing to give it a chance!

i don't believe you

everyone in the xcloud beta thread said its perfect and near exact response that they get normally

/s
 

zedox

Member
Oct 28, 2017
4,126
Who are these people that don't have access to consoles? I know Google and Microsoft seem to believe lots of them exist, but I'm just not seeing it. If someone can afford to buy a Stadia kit, or a screen + controller for xCloud, I feel pretty confident they can just buy a $200 console if they really want one.
People in other countries as was expressed in the video...
 

Vagabond

Member
Oct 26, 2017
1,359
United States
They should have a setting that enforces or warns when you arent on a 5GHz or WiFi6 network. My experience on that was pretty good. Never will be better than native but a decent enough alternative w/ Halo 5.

But out and about when stuck with 2.4GHz it was pretty bad.
 

Xx 720

Member
Nov 3, 2017
2,580
Wonder if this is going to be aimed primarily at cell phones, that’s where Stadia failed imo - when ms launch this it needs to be available on every phone model
 

zedox

Member
Oct 28, 2017
4,126
They should have a setting that enforces or warns when you arent on a 5GHz or WiFi6 network. My experience on that was pretty good. Never will be better than native but a decent enough alternative w/ Halo 5.

But out and about when stuck with 2.4GHz it was pretty bad.
I think it was xcloud that told me that I was on a 2.4GHz connection... I'm pretty sure it wasn't Stadia as I was able to continue playing...I could be wrong, I've been playing on both recently.
 

Fezan

Member
Oct 26, 2017
817
Like others have said, there's no magic latency solving evolution introduced here as it performs right along Now and Stadia.

Would liked to have seen some sort of closer comparison to all three services from DF.
Yes some People definitely said this but I was mostly reading that this was much better than Psnow. I was expecting some improvements as Microsoft owns azure and can directly tune them to decrease some latency but seems like we have reached peak streaming quality for the moment. Only thing better now would be higher fps
 

jacktink

Member
Oct 25, 2017
663
I had fun playing Sea of Thieves when streaming from my console. I got into the cloud beta recently as well and need some time to sit down and try that out.

There indeed is a bit of lag with it, so I wouldn't personally play FPS or fighting games with it, but something like SOT I learned pretty quickly how much input delay there was and adapted to it. Its fun!
 

Kyuuji

Member
Nov 8, 2017
8,614
I find xcloud playable but noticable in the 'feeling off' department at times. Stadia I have yet to feel that. Worst I've had is an occasional 1-2s hiccup every 45m or so on a couple of sessions.
 

bsigg

Member
Oct 25, 2017
9,549
Wonder if this is going to be aimed primarily at cell phones, that’s where Stadia failed imo - when ms launch this it needs to be available on every phone model
Mobile is first and foremost the target of xCloud. Console and PC are still their focus for in the home on a TV/monitor.
 

Kabukimurder

Member
Oct 28, 2017
181
Game streaming will make us all HPW(High Ping Whiner)’s again. Finally.

Hope LPB’s will stick to their own servers. I’ve always said It’s basically cheating.
 

BeImonkey

Member
Dec 9, 2017
1,667
I just hope we eventually get some sort of better mobile gaming experience, because a phone tacked on top of a controller just seems ugly and not very portable.
 

Dreamwriter

Member
Oct 27, 2017
3,903
Who are these people that don't have access to consoles? I know Google and Microsoft seem to believe lots of them exist, but I'm just not seeing it. If someone can afford to buy a Stadia kit, or a screen + controller for xCloud, I feel pretty confident they can just buy a $200 console if they really want one.
This video goes into that. India is a country for example where most people have a phone, but not a game console.
 

LCGeek

Member
Oct 28, 2017
1,442
My dreams of negative latency were short lived :(

Jokes aside, I'm genuinely disappointed that we've reached the limit of what's possible atm.
We haven't in any fashion gotten near what's possible.

The streaming platforms are badly tuned and don't use various low latency tech that has been out for 5-10 years.
 

pswii60

Member
Oct 27, 2017
11,488
UK
I got a good laugh out of people recently championing xCloud while crapping on Stadia's poor showing at launch. The technology is what it is, and that is laggy. How much a person is willing to put up with is the question.
The issue surrounding Stadia vs xCloud was about content and execution rather than technology.
 

Dreamwriter

Member
Oct 27, 2017
3,903
Surprise surprise - 60fps (Killer Instinct) on xCloud lines right up, latency-wise, with 60fps tests Digital Foundry did previously on Stadia and PSNow. All, uncannnily, in the 12Xms range for 60fps fighting games tested on each.

That is to say... the tech is as good as it ever was, but it seems there are no new silver bullets for latency this year.
Where you are getting +12Xms range? That wasn't in this video, nor in their Stadia article, nor in their PSNow article. And the values are quite different from each other, they aren't uncannily similar in the slightest. Stadia had +44ms latency in Mortal Kombat 11, PsNow had +68ms with Ultra Street Fighter 4, xCloud with Killer Instinct had +54, +75, and +90ms of latency depending on the phone.
 

Eoin

Member
Oct 27, 2017
1,886
Wait so all 3 cloud streaming service have similar latency? Why was I reading many comments here that xcloud feels much better than Psnow
All 3 work in pretty much the same way and deliver comparable streaming services in comparable conditions. Those conditions are variable though, from location to location, network to network, and moment to moment. Topics about these services are going to be full of posts saying how one service works great for them but another was unplayable, then others responding it was the exact opposite for them.

I don't think there will ever be a mass market streaming service that has a critical technical advantage over the others.
 

gofreak

Member
Oct 26, 2017
3,065
Where you are getting +12Xms range? That wasn't in this video, nor in their Stadia article, nor in their PSNow article. And the values are quite different from each other. Stadia had +44ms latency in Mortal Kombat 11, PsNow had +68ms with Ultra Street Fighter 4, xCloud with Killer Instinct had 54, 75, and 90ms of latency depending on the phone.
I was looking at the absolute total latency on each service (minus display latency), for Killer Instinct on xCloud, Ultra Street Fighter 4 on PSNow and Mortal Kombat 11 on Stadia. 121ms, 126ms, 122ms respectively. Looking at additional latency in isolation, we might indeed say DF found Stadia to be a bit better than xCloud and PSNow. It would be a good reason to ideally test the same game across all three services - though that still might not perfectly isolate the latency of the rest of the stack in each case, as I've a feeling devs have the opportunity to reduce the base input latency in their Stadia builds that they might not have deploying a game on xCloud or Now, for the moment anyway. (i.e. base input latencies in the stadia version of a game and the xbox version of a game and the ps version of a game may all vary for whatever reason, and we can't compare Stadia against it's 'local' latency alone).
 
Last edited:

Fezan

Member
Oct 26, 2017
817
Also one more thing I remembered watching this video is that all games for xcloud are running on Xbox one s which is worst performing current gen console. Some games will even feel worse.This is same issue with Psnow. Stadia can perform much better in this regards as most games will be targeting 60 fps
 

bcatwilly

Member
Oct 27, 2017
1,575
I have been more impressed than I expected to be with xCloud at the beta stage especially. My first attempts were on an admittedly ancient phone in the Galaxy Note 5 which pretty much was unplayable due to stuttering and such. I have since bought a Samsung Galaxy S6 tablet that works amazing with it now, and the scaling of the 720p stream on the tablet's 1440p works better than expected here too. I am not saying that I never notice any input lag at all, but in general I would say that I can play Xbox games on here without really thinking too much that I am playing on xCloud. My boys are far more sensitive to lag, and even they agreed that it felt much better than God of War on PSNow did when I tried it recently.

The bottom line is that there are no silver bullets yet for the overall tech and the various networking setups and otherwise, but Xbox Game Pass is what is really going to make this a compelling option for alternative play options. I was able to try out Mark of the Ninja: Remastered on xCloud and get achievements through the first level or two, and I ended up buying the game digitally during the Black Friday sale to be able to play locally on my Xbox One X when I want to with my progress saved of course.
 

zedox

Member
Oct 28, 2017
4,126
Also one more thing I remembered watching this video is that all games for xcloud are running on Xbox one s which is worst performing current gen console. Some games will even feel worse.This is same issue with Psnow. Stadia can perform much better in this regards as most games will be targeting 60 fps
Majority of gamers aren't going to care...I say that because Xbox One S sells the most.
 

amardilo

Member
Oct 30, 2017
132
UK
I have high hopes for xCloud.

The older I get the less time at home I have to game. But being able to play console games on my phone during my lunch break sounds like a great idea to me. My hope is that my current digital game library and Game Pass sub will allow me to play lots of games without having to buy into another ecosystem (just hope MS don't charge another fee for it and it's just included in Game Pass Ultimate).

I can understand the latency being an issue but I hope it will be fine for story based single player games like RPGs and action games. The video seemed promising on the latency front.

Also hope that with the next gen those games have a cloud option too.