How can they patent a process that has been implemented in publicly available open source emulators for at least half a decade?
-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
AI Neural Networks being used to generate HQ textures for older games (You can do it yourself!)
- Thread starter vestan
- Start date
Please don't shoot the messenger. Eurogamer has an article on it. Shout at them.
grats btw
I'm not shooting the messenger at all, I'm just again surprised at just how entirely dysfunctional the patent process seems to be.
(Even the details are the same -- e.g. taking a hash of the texture and using that to optionally reload and replace it after an artist provided a better one is pretty much exactly what I implemented in DSfix 7 or so years ago, and I don't think it was entirely novel even then)
(Even the details are the same -- e.g. taking a hash of the texture and using that to optionally reload and replace it after an artist provided a better one is pretty much exactly what I implemented in DSfix 7 or so years ago, and I don't think it was entirely novel even then)
I'd love to see what Ultimate Mortal Kombat 3 could look like using these tools, especially since it's got digitised actors.
I think there's a way to get texture dumpingloading working in Project 64, I'll add MK Trilogy to my list of stuff to try once I finish with my current batch of Gamecube games.
F-Zero GX HD Texture pack v1.0 is now available for download:
https://mega.nz/#!VJUCCSqa!O85Cd5dK2yK-TmNB8qSaWUUa1fT1QH_V5qbiazErHRU (585 MB, 6367 textures in DDS format)
Original textures were dumped from the NA version of the game (GFZE01). The zip is a resource pack which can be loaded If you're using a somewhat recent release of Dolphin that has support for it.
On Windows you can drop the zip into
https://mega.nz/#!VJUCCSqa!O85Cd5dK2yK-TmNB8qSaWUUa1fT1QH_V5qbiazErHRU (585 MB, 6367 textures in DDS format)
Original textures were dumped from the NA version of the game (GFZE01). The zip is a resource pack which can be loaded If you're using a somewhat recent release of Dolphin that has support for it.
On Windows you can drop the zip into
%UserProfile%\Dolphin Emulator\ResourcePacks. Go into ResourcePack manager and install from there. Otherwise, you can copy the GFZE01 folder inside the zip to %UserProfile%\Dolphin Emulator\Load\Textures
Big thanks to both of you for your tremendous efforts, these have turned out amazingly!
Viewtiful Joe said:
F-Zero GX said:
Album here in case of 'bandwidth exceeded' error
The choice to combine ESRGAN with Hypatias' excellent UI mod has given Viewtiful Joe a pretty stunning unofficial remaster, while I was surprised how noticeably pleasing the GX work was even with the high speed. Again, thank you both so much :D
Last edited:
Castlevania SotN sprite upscale tests
http://screenshotcomparison.com/comparison/129147
A mix of several methods. Some observations, Reduced Colors model seems to be the best with pixelart of low resolution, GMIC Anisotropic Smoothing applied before upscaling can change the outcome positively (colorbanding etc, reduce harshness of ESRGAN), applied after the upscaling process it can remove noise that was introduced into the image. GMIC smoothig creates a halo in uniform color areas, maybe could be avoided by adjusting the settings, not sure. Otherwise would have to be manually edited out (visible around the Dracula sprite in the blue areas)
Reduced Colors can be more precise than ESRGAN, it's just blurrier, combining two or more upscaled results and masking out the good and bad parts can be neccessary for more truthfulness to the original image.
ReducedColors also gave the most smooth edges but not the most detailed textures, combining the edges and textures from another upscale method can also increase quality.
Very difficult to keep quality and "ground truth" of low res pixelart with ESRGAN, the Nvidia upscaler maybe will be better for this (maybe even a one click upscale with good results?). Didn't try any Waifu2x here, might be even better or worse, dunno (also didn't try different small step upscale or antialiasing methods, too much to try out)
http://screenshotcomparison.com/comparison/129147
A mix of several methods. Some observations, Reduced Colors model seems to be the best with pixelart of low resolution, GMIC Anisotropic Smoothing applied before upscaling can change the outcome positively (colorbanding etc, reduce harshness of ESRGAN), applied after the upscaling process it can remove noise that was introduced into the image. GMIC smoothig creates a halo in uniform color areas, maybe could be avoided by adjusting the settings, not sure. Otherwise would have to be manually edited out (visible around the Dracula sprite in the blue areas)
Reduced Colors can be more precise than ESRGAN, it's just blurrier, combining two or more upscaled results and masking out the good and bad parts can be neccessary for more truthfulness to the original image.
ReducedColors also gave the most smooth edges but not the most detailed textures, combining the edges and textures from another upscale method can also increase quality.
Very difficult to keep quality and "ground truth" of low res pixelart with ESRGAN, the Nvidia upscaler maybe will be better for this (maybe even a one click upscale with good results?). Didn't try any Waifu2x here, might be even better or worse, dunno (also didn't try different small step upscale or antialiasing methods, too much to try out)
That could just mean it's vulnerable to attack by third parties at the USPTO if they try to enforce it against others, meaning they probably won't.
(Although more likely they'll have argued their method is different enough to be novel, the devil being in the detail)
Last edited:
Tony Hawk's Pro Skater 3 pack is out!
Download:
https://mega.nz/#!Wd8CUSYC!4kXspst7w3DJbcNqvns1XdJQKqZL0Z7u39NfcMoEDmg
Screenshots:
http://screenshotcomparison.com/comparison/129203
Download:
https://mega.nz/#!Wd8CUSYC!4kXspst7w3DJbcNqvns1XdJQKqZL0Z7u39NfcMoEDmg
Screenshots:
http://screenshotcomparison.com/comparison/129203
So a user named Shiryu over at the RE modding forums actually went ahead and used AI Gigapixel + manual touchups to improve REmaster HD's BGs. The results are pretty damn impressive:
More pictures + download link here:
https://reupscaleproject.blogspot.com/p/screenshots.html
The aqua ring renders actually looks pretty decent now, compared to the muddy/noisy vanilla upscale that Capcom did:
More pictures + download link here:
https://reupscaleproject.blogspot.com/p/screenshots.html
The aqua ring renders actually looks pretty decent now, compared to the muddy/noisy vanilla upscale that Capcom did:
So a user named Shiryu over at the RE modding forums actually went ahead and used AI Gigapixel + manual touchups to improve REmaster HD's BGs. The results are pretty damn impressive:
More pictures + download link here:
https://reupscaleproject.blogspot.com/p/screenshots.html
The aqua ring renders actually looks pretty decent now, compared to the muddy/noisy vanilla upscale that Capcom did:
Looks amazing!
The GMIC Anisotropic Smoothing Filter can reduce blotchy, low res compression artefacting quite well, here is a Twilight Princess Midna texture comparion:
http://screenshotcomparison.com/comparison/129244
The filter was applied before upscaling the textures but the setting/strength of the filter was reduced a bit so it wouldn't kill too much detail (which this is a bad comparison for as there is very little detail, only here and there). The models used were Reduced Color - Esrgan 04 interpolation, I also used that for the alpha textures, which was a mistake looking at the eyes, there is now a transparent ring and I also didn't up/downscale the alpha textures beforehand so they are still quite jaggy.
http://screenshotcomparison.com/comparison/129244
The filter was applied before upscaling the textures but the setting/strength of the filter was reduced a bit so it wouldn't kill too much detail (which this is a bad comparison for as there is very little detail, only here and there). The models used were Reduced Color - Esrgan 04 interpolation, I also used that for the alpha textures, which was a mistake looking at the eyes, there is now a transparent ring and I also didn't up/downscale the alpha textures beforehand so they are still quite jaggy.
Would this stuff be good at making old blurry ufo photos more clear?
something like this
something like this
Last edited:
Would this stuff be good at making old blurry ufo photos more clear?
something like this
So a user named Shiryu over at the RE modding forums actually went ahead and used AI Gigapixel + manual touchups to improve REmaster HD's BGs. The results are pretty damn impressive:
More pictures + download link here:
https://reupscaleproject.blogspot.com/p/screenshots.html
The aqua ring renders actually looks pretty decent now, compared to the muddy/noisy vanilla upscale that Capcom did:
As much as I love the upscaling tech on other screens in this thread, on these REmake screens the original looks more detailed. Just look at the loss of detail on the wallpaper designs in the upscale. Even in that first screenshot the grime on the wall near the ladder is smoothed way out and almost removed.
The "original" screens are the HD upscale, the "detail" is a crappy edge enhancement & sharpening fitler. He shouldn't have named them the original, the original Gamecube backgrounds didn't look like that.
Interesting. I guess the best way for them to do this REmake upscale would be to upscale the Gamecube versions.
Looks great. This is the kind of high res textures I've always wanted for old 3d games. Not the the noisy crap modders usually do.
I think they did?
"Some textures use the Gamecube original source (~90%) while others are upscaled from the HD Remastered texture (~10%)."
I'm really interested in looking at Riven, unfortunately using SFTGAN (which is what I'm guessing vestan used) I don't get anywhere near the same quality. What am I doing wrong? Tested many values in ESRGAN network interpolation and that yielded nothing close to the level of clarity vestan achieved.
Original Image:
My result:
vestans result:
The bottom one here really is spectacular; something about this process seems incredible well-suited to mid 90s - early 00s pre-rendered CG. I hope the process for setting up the software becomes a bit simpler sometime soon, I'd love to run some of the art from the Zork games through it.
It doesn't give you bad results it just doesn't do all that much to the overall appearance and just takes forever or you run out of RAM, you might as well scale your image the regular way once it is large enough so the algorithm doesn't have anything to latch onto anymore.
As far as I understand it, the entire process needs to be in a specific range of a low res pixelated image as a source image for the upscaling to have a visible effect. If your image is already too big or too "defined" in it's pixel matrix, then after you've upscaled that image you ask yourself if anything actually improved or even has changed?
That goes so far that you can sometimes trick this system into generating more fine detail by heavily scaling down the source image before upscaling it again to it's original size. So you didn't really upscale that image, you just generated more fine detail out it. I'll try to post an example of this tomorrow but you can often see it right in the source image when you open it up in an image editor / image viewer. If the contrast of fine detail already has several pixels of a smooth anti aliased halo area around it then the upscaling process will just scale that up as is with very little change. The source image has to be inside a certain range of size and pixel structure to give good upscaling results, if you have an image that looks exactly the same after upscaling it but is now just 4 times bigger, try scaling the source image down (to 25% of it's original size) and upscale that again back up to it's original size. You can lose authenticity but you can gain artificial detail, depends on the source image of course.
If people are worried about detail, I think layering the images and messing around with opacity could help bring back some "texture". Or even selectively applying it to parts of images.
There's clearly room for refinement with this kind of tool.
You could slightly sharpen them i suppose, but just not go overboard like Capcom did.
Here is a comparison of just downscaling the textures 25% and rescaling them up to their original size with ESRGAN, so no real upscaling if you will as the images are the same size after doing this, the pixels are just rearranged (I didn't care much for denoising and color or even consistency here):
Final Fantasy 12 - Fran Rescale Comparison
I assume those textures to be the original but I am not 100% sure as I didn't rip them and maybe someone messed with them. The thing with them was that just upscaling them as is, didn't do anything. No increase in details as their size and pixelstructure was not usable for the algo (although I didn't test several models, so maybe it is possible?).
As previously mentioned, you lose authenticity and detail with this method but you gain artificially created detail too. In theory you could upscale from that result and get a bigger difference out of it.
Final Fantasy 12 - Fran Rescale Comparison
I assume those textures to be the original but I am not 100% sure as I didn't rip them and maybe someone messed with them. The thing with them was that just upscaling them as is, didn't do anything. No increase in details as their size and pixelstructure was not usable for the algo (although I didn't test several models, so maybe it is possible?).
As previously mentioned, you lose authenticity and detail with this method but you gain artificially created detail too. In theory you could upscale from that result and get a bigger difference out of it.
Just FYI: The validity of a patent can be challenged even after that patent has been granted.
For my FF IX mod, I end up adding a lot of pre and post processing (de-dithering, sharpening, denoising) that took a long time to process.
So I'm retraining a ESGRAN network with the original background (no process) as input and the current version of my mod as output, and now, I will have a network that can get rid of dithering artefacts and have more details than the original process :
https://twitter.com/Ze_PilOt/status/1092709418276438016
I'm not sure that network will be usuable with any other kind of texture, but as many people use manga109 for stuff that are not drawed, it might work (probably for PSX textures)
So I'm retraining a ESGRAN network with the original background (no process) as input and the current version of my mod as output, and now, I will have a network that can get rid of dithering artefacts and have more details than the original process :
https://twitter.com/Ze_PilOt/status/1092709418276438016
I'm not sure that network will be usuable with any other kind of texture, but as many people use manga109 for stuff that are not drawed, it might work (probably for PSX textures)
No matter the result, I would like to try that model when finished (if you want to share of course)
Thank you for your continued amazing work :) Exciting to see further opportunities and experimentation continue with this already amazing tech
While it may seem like it at times, AI is not magic.
Can someone make this look good?
I wouldn't say that enough : training your dataset accordingly to what you are upscaling is the most important thing.
Here is direct comparaison of pre-existing techniques/dataset VS custom training. Upscaled from PSX, no pre or post process.
For some reasons, Gigapixels is offsetting the image slightly. That could be a huge issue when doing games upscales.
http://www.framecompare.com/image-compare/screenshotcomparison/199JNNNU
Noise management/training is one of the important part when dealing with compressed/altered image (in our case : dithering to a reduced color palette). Here is another exemple of the different solutions dealing with a noisy image :
http://www.framecompare.com/image-compare/screenshotcomparison/Y777NNNX
Here is direct comparaison of pre-existing techniques/dataset VS custom training. Upscaled from PSX, no pre or post process.
For some reasons, Gigapixels is offsetting the image slightly. That could be a huge issue when doing games upscales.
http://www.framecompare.com/image-compare/screenshotcomparison/199JNNNU
Noise management/training is one of the important part when dealing with compressed/altered image (in our case : dithering to a reduced color palette). Here is another exemple of the different solutions dealing with a noisy image :
http://www.framecompare.com/image-compare/screenshotcomparison/Y777NNNX
Last edited:
I always assumed that the noise reduction was as seperate process from the upscaling part in software like Waifu2x/Gigapixel etc. because they offer 3-4 steps of noise reduction intensity, did they train that into the models? (is your model training finished?)
Yes, Waifu is using different models / parameters depending of what you are asking.
For the training, it just reached epoch 30 and I still see some improvements over epoch 25, so no, it's not finished :)
Tried out Manga109 on a Resident Evil 3 background just to see what I would get, and it definitely looks better than Waifu2x to me. Is there anyone working on a pack to replace the Waifu2x mod out there for the Gamecube version?
Screenshot comparison with Waifu2x
Screenshot comparison with Waifu2x
Looks like the ESRGAN mod for Hexen is out:
Nexus Mods Link
Some of you may recall the earlier examples in the thread with the upscaled weapons:
Nexus Mods Link
Some of you may recall the earlier examples in the thread with the upscaled weapons:
Last edited: