• Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
I just learned about (and immediately tried) this, and it's pretty amazing. And a genius idea, even though the process seems rather obvious in retrospect (like many great ideas). I hope I didn't miss a thread about this!

Some background information:
  • Due to various factors, input lag increases when emulating a system have been an issue for a while -- generally more for older platforms (ie. 16 bit and earlier, maybe 32) that had short processing loops/pipelines
  • Over the past few years, many improvements were made to popular emulators (or emulation platforms such as retroarch) to reduce or even eliminate that increase in input lag (ie. hard GPU sync, reducing buffering, late-latching inputs, etc)
  • At this point, you can get close to (or even, in some cases, match) the original latency for some older systems
  • This generally takes more performance since many options to reduce latency trade efficiency for minimizing delay -- this is usually not an issue on PC, but might be on other platforms

A new innovation in Retroarch takes this (trading performance for decreased input latency) one step further: below real hardware.

Basically, most games, even on older hardware, have some built-in amount of input delay. On many NES games it's 1 frame (16 ms), on a lot of SNES games it's 2 frames (33 ms) and so on.
What the new options in Retroarch allow is this:
  1. Poll the input.
  2. Run one frame (frame 1) without output (video/audio).
  3. Create a save state.
  4. Emulate N-2 (configurable parameter) additional frames without output.
  5. Render a final frame (frame N) with output -- this is what you see.
  6. Restore the state saved in 3.
This process repeats for every "original" frame.
In effect, what this does from your perspective is move your input backwards in time, since you are reacting to output which happened N frames later. It's so neat.

(Note: there is an alternative mode which runs another instance of the emulator core "normally" to generate audio, since the audio output of some emulation cores gets confused/choppy using the method above)

Obviously, this has a high performance cost (i.e. it increases the CPU emulation overhead by a factor of N compared to normal, plus the additional save state overhead) but for fast cores and older systems it works perfectly.

Here's the thread discussing the feature on the libretro forums, with a download link.

Let me tag a few people who I remember from emulator latency discussions back in the old world Tain Dark1x
 

Deleted member 2620

User requested account closure
Banned
Oct 25, 2017
4,491
This... sounds really neat. Maybe I'm misunderstanding, but isn't this kind of similar in concept to what GGPO does? As in, at worst you'd get "rollback" effects? I should check it out.

I was kind of dismissive of Retroarch a few years back (I don't think I liked the results of running it without any form of v-sync, didn't see much reason not to use MAME/UME/MESS), but have warmed up to it a lot since.
 

Deleted member 135

User requested account closure
Banned
Oct 25, 2017
11,682
If the input lag was native to the consoles wouldn't the games be designed with it in mind? Seems like this could create unintended issues like removing the sprite line limit in NES games can cause weirdness.
 

OnionPowder

Banned
Oct 25, 2017
9,323
Orlando, FL
This is amazing. Something I would love to test out.

Time to start dumping my collection and move away from CRT at some point in the future I guess.
 
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
This... sounds really neat. Maybe I'm misunderstanding, but isn't this kind of similar in concept to what GGPO does? As in, at worst you'd get "rollback" effects? I should check it out.
It absolutely is. However, you won't get any visible rollback effects as long as you don't run ahead for more frames than the original game's internal lag.

You can easily try this for yourself, the current build allows running ahead up to 5 or 6 frames, which no game on those older platforms really approaches in terms of lag (as far as I know).

Note that to effectively test this you need a fast core, e.g. for SNES snes9x is preferable to bsnes for this purpose (and really, outside of corner cases it's hard to distinguish the two these days).
 
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
If the input lag was native to the consoles wouldn't the games be designed with it in mind? Seems like this could create unintended issues like removing the sprite line limit in NES games can cause weirdness.
From the perspective of emulation accuracy, it is in some ways less "dangerous" than e.g. removing the sprite limit: on the "reference time stream" that actually determines what happens nothing changes. The game doesn't know that you are looking into the future ;)

Of course, that's not to say that there can never be any problems with it, but as long as you keep below the internal delay people have had excellent experiences with it so far.
 

Nitpicker_Red

Member
Nov 3, 2017
1,282
If the input lag was native to the consoles wouldn't the games be designed with it in mind? Seems like this could create unintended issues like removing the sprite line limit in NES games can cause weirdness.
The lag is between the physical input and when the button is detected. not between two things in the software. So it shouldn't make the software run differently, but it could mess a smidge with your muscle memory if you were used to pressing the button a frame in advance to compensate.
 

1-D_FE

Member
Oct 27, 2017
8,246
It absolutely is. However, you won't get any visible rollback effects as long as you don't run ahead for more frames than the original game's internal lag.

You can easily try this for yourself, the current build allows running ahead up to 5 or 6 frames, which no game on those older platforms really approaches in terms of lag (as far as I know).

Note that to effectively test this you need a fast core, e.g. for SNES snes9x is preferable to bsnes for this purpose (and really, outside of corner cases it's hard to distinguish the two these days).

Is there really no sound difference between them? Because I can't tell if I'm just suffering from bias and placebo effect, but I feel like bsnes has better audio than SNES.

Amazing regardless. Retroarch already was only like .2 frames behind actual SNES hardware (when configured properly), but this means you could play SNES on a TV with 32ms lag and still have it play as responsive as native hardware did on CRT. That's awesome!
 
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
Oh, one more thing I wanted to mention: a potential argument I see against this is that it's not "accurate" or "authentic". This is, of course, true. However:
  • You probably played those games on a CRT with basically 0 display hardware based input lag. Conversely, your current TV might have ~20 ms if it's cream-of-the-crop in terms of latency, or more otherwise.
    So e.g. compensating 2 frames of latency with this method will make the gameplay feel as responsive as it would be on a 0-latency CRT on a 33 ms latency LCD display.
  • Or if you have a 3 ms latency G-sync monitor, you can still use this method to eliminate other emulator sources of latency, likely with better compatibility or less side-effects than previous methods.
  • Or you might just not care and enjoy the extremely responsive feeling of less-than-original latency ;)
Edit:
Amazing regardless. Retroarch already was only like .2 frames behind actual SNES hardware (when configured properly), but this means you could play SNES on a TV with 32ms lag and still have it play as responsive as native hardware did on CRT. That's awesome!
Heh!
 

Eila

Member
Oct 27, 2017
2,939
Too bad I don't have a low latency screen to really enjoy something like that, but every bit helps, I guess.
My laptop screen isn't so bad, much better than my TV, in any case.
 

exodus

Member
Oct 25, 2017
9,938
This is amazing.

How is the frame skipping situation these days? I've never been able to successfully play in the past without either frame skipping or juddering of some kind. Does G-Sync help alleviate this? I recall BSNES had issues in the past even working properly with G-Sync due to how it rendered to the screen.

My experience with emulators is quite out of date.
 

Talus

Banned
Dec 9, 2017
1,386
I really want to like and use Retroarch, but I can never seem to get things set up properly. There's almost too many options and it's daunting to get it all set up, and the cores I used were all terribly laggy, leading me to assume that I had things set up wrong. I freaking love the idea of it though.

Can anyone point to any decent tutorials out there?
 

Wowfunhappy

Member
Oct 27, 2017
3,102
How is the frame skipping situation these days? I've never been able to successfully play in the past without either frame skipping or juddering of some kind. Does G-Sync help alleviate this? I recall BSNES had issues in the past even working properly with G-Sync due to how it rendered to the screen.

This is one of the big advantages of Retroarch—it already has systems to eliminate this problem on non-variable refresh screens.
 

Bomblord

Self-requested ban
Banned
Jan 11, 2018
6,390
I gotta say I'm not quite understanding what you are saying. It almost sounds like you are saying the game is actually reacting to what I input before I actually hit the button.
 
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
Too bad I don't have a low latency screen to really enjoy something like that, but every bit helps, I guess.
My laptop screen isn't so bad, much better than my TV, in any case.
As per above, this is just as (or maybe even more) useful if you have a display with some input lag, since it will get the experience closer to what was originally intended by (at least partially) compensating for that source of lag.

This is amazing.

How is the frame skipping situation these days? I've never been able to successfully play in the past without either frame skipping or juddering of some kind. Does G-Sync help alleviate this? I recall BSNES had issues in the past even working properly with G-Sync due to how it rendered to the screen.

My experience with emulators is quite out of date.
Honestly, the whole presentation framework and the amount of options to customize it is on another level in Retroarch compared to most stand-alone emulators.
(And for good reason, it pools the individual efforts of tons of people and makes them easily available to lots of emulator cores -- that was the idea)

You can configure the output API used, the type of synchronization (in detail), how many frames to render ahead (in the traditional GPU sense, not this feature!), when input is polled, and so on.
And now of course this additional magic ;)
 

Ultima_5

Member
Oct 25, 2017
7,670
I really want to like and use Retroarch, but I can never seem to get things set up properly. There's almost too many options and it's daunting to get it all set up, and the cores I used were all terribly laggy, leading me to assume that I had things set up wrong. I freaking love the idea of it though.

Can anyone point to any decent tutorials out there?
This. Ive tried once or twice to get it going, but it's such a mess of options and none of them mean anything to me.
 

Skyfireblaze

Member
Oct 25, 2017
11,257
Okay this is the first time I read one of your posts and have a hard time grasping it Durante! :P Say I want to try it out now, do I just have to download the latest beta or nightly build?
 
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
I gotta say I'm not quite understanding what you are saying. It almost sounds like you are saying the game is actually reacting to what I input before I actually hit the button.
In a way it is like that.

Of course, that's not how causality works, so instead, the emulator makes you "see into the future". You see what will happen, press the button, and then that button press influences the game at the point in time where it "really" is, which is one or several frames earlier than what you see.

This. Ive tried once or twice to get it going, but it's such a mess of options and none of them mean anything to me.
I can see that, it's the danger of any really powerful tool.
But retroarch now is honestly far easier to use out of the box than it was a few years back. The UI is slick, controls are plug and play and auto-configured, you can download and install emulation cores straight out of the menu, and so on. There are still lots of options but you need only touch those that you are interested in, and you generally see the result immediately.
 

Dark1x

Digital Foundry
Verified
Oct 26, 2017
3,530
Wow, that's brilliant. It makes perfect sense but I wouldn't have thought of it, that's for certain.

The next step here is coupling Retroarch with a proper 15khz CRT monitor using the correct resolution. That should reduce latency to its absolute lowest point.
 

Yoshi

Banned
Oct 27, 2017
2,055
Germany
Conversely, your current TV might have ~20 ms if it's cream-of-the-crop in terms of latency, or more otherwise.
Talk about your TV, my 1998 tube has super low latency. Do the emulators only do speculative code execution or do actual rollbacks happen, i.e. does a frame appear where Mario dies and the next frame corrects this, because you actually jumped in time according to the low latency emulator?
 

Zutrax

Member
Oct 31, 2017
4,187
In a way it is like that.

Of course, that's not how causality works, so instead, the emulator makes you "see into the future". You see what will happen, press the button, and then that button press influences the game at the point in time where it "really" is, which is one or several frames earlier than what you see.

I'm also trying to wrap my head around this. So basically in laymans terms if I'm understanding this right, it saves a state of every single possible outcome based on all button presses/combinations, and once the button press is read it then rolls back instantly to the state that's associated with that press?
 

Wowfunhappy

Member
Oct 27, 2017
3,102
I saw this method on reddit about a week ago, and it took me several days to wrap my head around what is actually happening. Here's how I'm currently thinking about it. If my understanding is flawed, people more knowledgable than I am can feel free to correct me:

*******

Under normal circumstances, you press A, and the game's code begins reacting to that input on the next frame. Super Mario World's code does not actually begin Mario's jump animation until one frame after it receives the input.

With this method, you press A, and in the time it would normally take to generate a single frame, the emulator quickly runs through two frames, but only actually outputs the second frame to your screen.

You could keep increasing the number of "skipped" frames infinitely as long as your computer is fast enough to generate them in the time it would normally take to generate a single frame. However, if you rendered 3+ frames ahead in Super Mario World, for instance, you would press A, and Mario would "teleport" to a higher position on the screen, because you would have skipped the beginning part of the animation.
 
Last edited:
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
I'm also trying to wrap my head around this. So basically in laymans terms if I'm understanding this right, it saves a state of every single possible outcome based on all button presses/combinations, and once the button press is read it then rolls back instantly to the state that's associated with that press?
No, that would take far too much computing effort, and is also not necessary. Let me try to illustrate it.

Code:
 Real    Emulated time (frames)       Run ahead frame 1          Run ahead frame 2         ...
 Time     (where the game is)                                      (what you see)
  ||
  ||    |                     |
  ||    |      ...            |
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N          | -> |  "Preview" for N+1  | -> |  "Preview" for N+2  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N+1        | -> |  "Preview" for N+2  | -> |  "Preview" for N+3  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N+2        | -> |  "Preview" for N+3  | -> |  "Preview" for N+4  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N+3        | -> |  "Preview" for N+4  | -> |  "Preview" for N+5  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |      ...            |
  ||    |                     |
  ||
  \/
For every "real" frame, the emulator renders ahead a few frames and shows you the result of that, rather than the "real" current frame.
 

Mzo

Member
Nov 30, 2017
1,162
I'm also trying to wrap my head around this. So basically in laymans terms if I'm understanding this right, it saves a state of every single possible outcome based on all button presses/combinations, and once the button press is read it then rolls back instantly to the state that's associated with that press?
I don't think so, that's crazy.

It's more like you're running towards a pit and press jump when Mario is at the edge. The latency would take a frame or 2 to process and you would normally die from input lag. With this, you're seeing the future (Mario at the edge) while the game is actually in the past (a step or so away from the edge.) You press the button based on what you see now and it gets pressed at the earlier, actual time the game is running on so you jump and don't die.
 
OP
OP
Durante

Durante

Dark Souls Man
Member
Oct 24, 2017
5,074
Do the emulators only do speculative code execution or do actual rollbacks happen, i.e. does a frame appear where Mario dies and the next frame corrects this, because you actually jumped in time according to the low latency emulator?
This will not happen as long as you are not running ahead more than the internal input lag of the game (as long as you are not, the game won't have reacted to your input yet anyway in the time frame we are previewing, so there will be no visible rollback).
 

Jazzem

Member
Feb 2, 2018
2,678
Oh man that's incredible =O

I have a love/hate relationship with Retroarch and its unwieldy yet powerful interface, certainly going to have to tinker with it more and try this out.
 

oni-link

tag reference no one gets
Member
Oct 25, 2017
16,000
UK
Maybe I'm way off here but does this mean I can update my Retro Pi and kiss goodbye to input latency?
 

Burai

Member
Oct 27, 2017
2,080
Can't wait to try this. The 8bitdo controllers add a fair bit of latency over Bluetooth so it would nice to dial some of that out.
 

Wowfunhappy

Member
Oct 27, 2017
3,102
I have a love/hate relationship with Retroarch and its unwieldy yet powerful interface, certainly going to have to tinker with it more and try this out.

I know this doesn't actually help usability, but my solution (of a sort) has been to just hide almost every config option from the UI. I don't need to touch those options anyway, and now my copy of Retroarch has the single cleanest interface of any emulator I've ever used. :D

And then if I read about something cool like this, I can go and enable it in the config file.
 
Last edited:

Aeana

Member
Oct 25, 2017
6,918
I'd love to try this out with Mother 3, as that is the one emulated game I have personally felt suffers massively from emulator input lag. If this can solve that issue, I'd be ecstatic.
 

Aeana

Member
Oct 25, 2017
6,918
Is there really no sound difference between them? Because I can't tell if I'm just suffering from bias and placebo effect, but I feel like bsnes has better audio than SNES.

Amazing regardless. Retroarch already was only like .2 frames behind actual SNES hardware (when configured properly), but this means you could play SNES on a TV with 32ms lag and still have it play as responsive as native hardware did on CRT. That's awesome!
BSNES/Higan and snes9x literally have the same sound core and have for a couple of years.
 

Bomblord

Self-requested ban
Banned
Jan 11, 2018
6,390
No, that would take far too much computing effort, and is also not necessary. Let me try to illustrate it.

Code:
 Real    Emulated time (frames)       Run ahead frame 1          Run ahead frame 2         ...
 Time     (where the game is)                                      (what you see)
  ||
  ||    |                     |
  ||    |      ...            |
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N          | -> |  "Preview" for N+1  | -> |  "Preview" for N+2  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N+1        | -> |  "Preview" for N+2  | -> |  "Preview" for N+3  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N+2        | -> |  "Preview" for N+3  | -> |  "Preview" for N+4  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |  "Actual" Frame     |    +---------------------+    +---------------------+
  ||    |  at time N+3        | -> |  "Preview" for N+4  | -> |  "Preview" for N+5  |
  ||    |                     |    +---------------------+    +---------------------+
  ||    +---------------------+
  ||    |      ...            |
  ||    |                     |
  ||
  \/
For every "real" frame, the emulator renders ahead a few frames and shows you the result of that, rather than the "real" current frame.

I think I understand now so the emulator renders other possibilities and then shows the one that is corresponding to the button press instead of starting the rendering at the button press?
 

Mechanized

Member
Oct 27, 2017
3,442
This is kinda insane.

We're going back in time to input before the input is registered so there is less latency and this actually takes less time than the normal way wat.
 
Oct 25, 2017
3,721
I think I understand now so the emulator renders other possibilities and then shows the one that is corresponding to the button press instead of starting the rendering at the button press?
Running games in real hardware has a certain amount of lag between the inputs... Say one frame, or two frames.
This essentially 'skips' those one or two frames, so that you would see your input on the correct frame, reducing that latency.
 

oni-link

tag reference no one gets
Member
Oct 25, 2017
16,000
UK

scitek

Member
Oct 27, 2017
10,048
It sounds similar to how AVRs allow you to delay audio by a certain number of milliseconds in order to sync it up with video.
 

Barrel Cannon

The Fallen
Oct 25, 2017
9,286
I think I'll need a video to understand this better. I can't visualize what the OP means due to my technologial incompetence :(