-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
-
We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.
AI Neural Networks being used to generate HQ textures for older games (You can do it yourself!)
- Thread starter vestan
- Start date
This is perfect, thanks. Not about to do artifact removal 1 by 1 on 2,000 images.
Which model specifically is for jpeg artifact removal?
Last edited:
I'm pretty late to the party, but here are some Clock Tower (psx) backgrounds upscaled with the manga model:
Honestly, I don't know as I haven't tried it out yet. I just suggested it because it's one that I've been keeping an eye on for the last few months, and its the only one I know of that can do JPEG artifact removal.
These latest ones do look really impressive. I don't think any professional develoeprs would do a remaster ENTIRELY on just throwing an AI at all these assets and calling it a day, but I imagine it could fill in for a lot of the legwork artists would normally have to do in re-touching or remaking 2D assets like this. I imagine they'd selectively apply AI to assets and then still have human hands touch them up where needed afterwards.
Still seems to be the least effective on actual 2D pixel art games.
Still seems to be the least effective on actual 2D pixel art games.
I don't know but I assume that this is a given, it's too powerful a tool to not have integrated in an image editor like PS.
If you want the right experience for 2D Sprites.
Cheapest method for a selection of games would be Softmodded Wii with VC at 240p + Wii Component Cables + OSSC (Open Source Scan Converter) on an OLED TV. It comes very close to a CRT. I tested this with a DLP Projector at 12ft, and man, I was blown away.
Actually I'm playing my roms on a softmodded Wii on an actual CRT.
They demo'd a couple of visual-based neural network things recently, one being a style transfer feature as part of a separate program (taking the style of an illustration and applying it to a photograph, more accurately than most implementations I've seen) and the other an improved content-aware fill.
But yeah, super resolution scaling seems like something more generally useful/applicable. Not without drawbacks though as we've seen, which does matter when it comes to reproducibility in an image/video editor.
Heart of Darkness (psx), interpolation between Manga and ESRGAN models, also adjusted contrast slightly to compensate for its loss during upscaling.
Would love to see some N64 games get this treatment, which are known to be blurry as all hell. That Ocarina one is insane.
I don't have an N64 emulator installed but I took this texture from here : Texture Website and cut out the Whomp texture (I don't even know if it is the exact same from the game tbh)
Screenshot comparison:
Super Mario 64 Whomp Texture Comparison
Top left is that original texture, next to it the first upscaled image via Manga-ESRGAN 02 Inter.
The "On Mouse Out" Image is the original top left image upscaled via GIMP (no-halo-setting) to match the size of the final result.
The "On Mouse Over" Image is the first Manga-ESRGAN 02 Inter. upscaled image upscaled again with Manga-ESRGAN 02 Inter. and then I made a 100% ESRGAN (also from the first upscaled image) and overlayed that onto that with 50% in GIMP to get tiny little bit more sharper detail (didn't really work as well as I thought tho).
Wow the N64 texture is microscopic! Given what it had to work with, the ESRGAN result turned out great. That bandage texture!
Heart of Darkness (psx), interpolation between Manga and ESRGAN models, also adjusted contrast slightly to compensate for its loss during upscaling.
Why don't use the PC version? Shouldn't be better? BTW I just played a bit of the game the other day (PC), almost 20 years has passed since the first time I played it and this game is still incredible... it really deserve a remake but it's probably too hard to make it.
Actually the psx version has far superior color depth (like 256 colors on pc vs 16 million or something on psx) and hence the obvious choice for upscaling.
This is absurd and almost kinda scary how well the AI works.
Wow, looks great. I really want to see SM64 with some new textures applied.
I'm planning on doing a full HD re-texture mod for Metroid Prime using this, but my one concern at this point is alpha textures. If anyone knows a workaround that allows ESRGAN to work with transparent PNGs, please PM me. The results are going to be awesome either way, but I'd like to upscale water textures and effects too if possible.
That's a waste of effort considering that the HD project also has improved model detail, UV mapping fixes, and other tweaks that go beyond just new textures.
Oh, right. Speaking of which, couldn't you also train a NN to upscale/detail-increase 3D models? 🤔
The only workaround is to save them as non-transparent images (replace the transparent background with a flat primary color that's different from the colors the texture is using. Same as chroma key/green screen), and then manually make it transparent again after processing. I've heard people have found it helpful if they keep a duplicate of the alpha layer from the original and then simply do a separate upscale of that (since the alpha would just be a simple monochrome silhouette you could probably settle for a simpler upscaler like waifu2x to handle that)
Makes sense. Sadly, the green screen method probably wouldn't be feasible to automate since you'd have no way of knowning which shade to use as the key unless you wrote a fancy color detection script. My plan right now is to just write a simple script that checks for an alpha channel in the dumped textures and omits them from the build.
I'm sure someone is working on this, but in practical terms I can only imagine it being useful for models made in the Dreamcast gen or later, and then there's the possibility of the new models messing with existing textures.
A user on Qhimm that goes by Satsuki is currently playtesting their mod using these techniques combined with a higher poly models mod in FF7 and posted some in-game screenshots:
More here in their thread.
More here in their thread.
Good lord, those FFVII screenshots are good. The only thing I don't like about it are, well, the character models. They just seem...weird. Always have to me.
HD backgrounds though, oof. Nice.
HD backgrounds though, oof. Nice.
Wish these modders would do high-poly deformed models instead of this weird mishmash of proportions that don't match.A user on Qhimm that goes by Satsuki is currently playtesting their mod using these techniques combined with a higher poly models mod in FF7 and posted some in-game screenshots:
More here in their thread.
I *think* they just scaled the bones along their axis, but they've got the same amount of bones as the regular models.
Im not sure but I suspect there wasn't much touching of the rigs and it has its pros and cons.
A pro is that a good deal of animations surprisingly end up looking visably clearer due to the models having longer limbs. For instance something like characters scratching their heads look much better communicated with these models where before it looked like a jumble of polys vibrating into eachother.
An obvious con is how the arms sort of stick out now in the default idle pose because it was intended for the huge Lego brick hands.
Ahhhh that makes more sense. The weird arm positions are what made me suspect they'd done something, and I hadn't even though of the big lego block hands.
Current models were trained mostly on natural images and/or hand drawn art and don't work all that great with pixel art.
Alright.
Has anyone ran the tech on The Legend of Dragoon? The backgrounds of that game looked good.
Here are few more MI2 upscales. I was especially happy with the result created by the map upscales, as they can now be used as a wallpaper. I was able to upscale the maps second time using Photoshop's "Preserve Details 2.0" upscale, which makes them perfect for wallpapers (I didn't include those 2nd stage photoshop upscales here - these are straight out of manga model upscale).
More scenes from the game. While the AI upscaler certainly trips over characters quite a bit, I find backgrounds to be almost uniformly gorgeous. There's a certain artisinal quality to them that was ironically completely missing from the Special Edition. I really wish someone would process all the backgrounds and inject them back into the game - if that's even possible, due to their new and much larger resolution.
More scenes from the game. While the AI upscaler certainly trips over characters quite a bit, I find backgrounds to be almost uniformly gorgeous. There's a certain artisinal quality to them that was ironically completely missing from the Special Edition. I really wish someone would process all the backgrounds and inject them back into the game - if that's even possible, due to their new and much larger resolution.
Did you try this out eventually? I can't imagine how the scumm editor would support importing images of higher resolution than the originals - without scaling them back to the original 320x200 size that is.
This is frustrating to watch because he's just playing Half-Life like normal and not really SHOWING what's different. He stops and looks at a couple of surfaces, but usually for a brief second (like, literally, an actual full second) otherwise this is just normal gameplay footage from someone who's played Half-Life more than twice
KingdomAkrillic (the blogger in the OP) trained another model using random pieces of art.
Have yet to test this on anything but it's intriguing.
Have yet to test this on anything but it's intriguing.
No, I didn't have the time yet. I found another interesting program though, which allows to create your own scumm game https://github.com/AlbanBedel/scummc. It is open source, so I assume one can modify the resolution and rebuild one of the older games with high resolution images with it. But that would probably need a lot of effort and I know I won't have time for it.
So uh, for everyone saying devs should use this for remasters, well, all evidence points to the team that ported Catherine to PC having used Waifu2x to upscale the 2d assets.
Comparison of new in-game assets vs Waifu2x rescales of the original resolution assets:
https://twitter.com/PastelChum/status/1083432382039031808 (Linked because NSFW)
Comparison of new in-game assets vs Waifu2x rescales of the original resolution assets:
https://twitter.com/PastelChum/status/1083432382039031808 (Linked because NSFW)
If you want to use them as wallpapers, you'd be better off starting with the original background paintings and upscaling those: https://mixnmojo.com/media/galleries/Monkey-Island-2-LeChucks-Revenge-Concept-Art
Would be the most appropriate use for it, though judging by the Twitter comment they didn't include a copy of the original license text? Tsh, tsh.
So the guy who work on the upscale give a "beta" version of his mod to a streamer and he currently stream it on twitch (In French):
This is looking good.
Last edited:
No, because why would devs make low-res textures and then up-res them? It's usually the opposite.
But what this can do for old games is pretty cool.
But what this can do for old games is pretty cool.