AI Neural Networks being used to generate HQ textures for older games (You can do it yourself!)

[Miniature Kaiju]

Here's the main cockpit texture:

You're probably right that there's a way to clean everything up after the fact in Photoshop, but I'm not an artist and I'm really trying to make everything as automatable as possible. I think the main issue is that the source resolution is generally too low for ESRGAN to work properly with and at this point you might as well just make new textures from scratch. I might try throwing waifu2x at it, but for the time being I think this particular game might be a dud. I'll upload my results for posterity in case anyone else wants to mess around with them.

Click to shrink...

Upscale them naively through PS using Nearest Neighbor to, say, 128px x 128px and then the rest of the way through ESRGAN, maybe?

 

[Laser Man]

Here's the main cockpit texture:

You're probably right that there's a way to clean everything up after the fact in Photoshop, but I'm not an artist and I'm really trying to make everything as automatable as possible. I think the main issue is that the source resolution is generally too low for ESRGAN to work properly with and at this point you might as well just make new textures from scratch. I might try throwing waifu2x at it, but for the time being I think this particular game might be a dud. I'll upload my results for posterity in case anyone else wants to mess around with them.

Click to shrink...

Upscale them naively through PS using Nearest Neighbor to, say, 128px x 128px and then the rest of the way through ESRGAN, maybe?

Click to shrink...

This is a good idea, if you can then use Photoshops detail 2.0 upscaling, might result in a better image than what ESRGAN gets you.

A shot at the sky texture:
http://screenshotcomparison.com/comparison/130786

The cockpit texture (bad result):
http://screenshotcomparison.com/comparison/130787

The problem with the cockpit texture is that it mixes hard colors that have no transition/midtones (the cockpit lights and the border) with another image that has that (fox reflection).

I know that the best situation would be a one step upscale and that's that, but as long as that is not available, using filters is the next best thing to improve your results and that is still a world apart from actually creating new textures from scratch. If you find a good setting to a filter you can batch process images and check a few textures to see if it worked for most of them.

But I can see that even using filters can be a daunting task for someone that is not familiar with image processing in that specific area and wants to avoid it at all cost (but you can really step up your texture modding game if you look into it a bit :)
I would at least look into denoising images after upscaling them as I have not seen an image that's been upscaled by ESRGAN that didn't suffer from unwanted artefacts and noise.

 

[collige]

Upscale them naively through PS using Nearest Neighbor to, say, 128px x 128px and then the rest of the way through ESRGAN, maybe?

Click to shrink...

This is a good idea, trying it now.

This is a good idea, if you can then use Photoshops detail 2.0 upscaling, might result in a better image than what ESRGAN gets you.

A shot at the sky texture:
http://screenshotcomparison.com/comparison/130786

The cockpit texture (bad result):
http://screenshotcomparison.com/comparison/130787

The problem with the cockpit texture is that it mixes hard colors that have no transition/midtones (the cockpit lights and the border) with another image that has that (fox reflection).

I know that the best situation would be a one step upscale and that's that, but as long as that is not available, using filters is the next best thing to improve your results and that is still a world apart from actually creating new textures from scratch. If you find a good setting to a filter you can batch process images and check a few textures to see if it worked for most of them.

Click to shrink...

Those skybox upscales look amazing!

But I can see that even using filters can be a daunting task for someone that is not familiar with image processing in that specific area and wants to avoid it at all cost (but you can really step up your texture modding game if you look into it a bit :)
I would at least look into denoising images after upscaling them as I have not seen an image that's been upscaled by ESRGAN that didn't suffer from unwanted artefacts and noise.

Click to shrink...

I know how to use PS, but I don't own a copy of it and I'm not inclined to start paying Adobe a boat load of money when GIMP does pretty much everything I've needed to do so far. It would also be way simpler to have everything work off of scripts given the quantity of images we're working with. For example, Star Fox 64 have over 15,000 dumped textures, though most of the appear to be procedurally generated somehow which makes it a pain in the ass to track down individual textures and clean them up. This is also my first serious attempt at modding lol

 

[GodzFire]

Just signed up because of literally amazing this thread and technology has been. It truly is a breakthrough game-changer and I've been doing everything I can to spread the word about it everywhere because this isn't just for games, the potential for restoration and enhancement for photos is mind blowing to think of. I wish this would get a LOT more attention than to where it's confined to right now.

That said, I'm wondering if someone would be kind enough to do some for a game that's near and dear to my heart: Twisted Metal 2:

Logo:

Character Bios:
https://imgur.com/a/s0Hg7Rk



And these few random wallpapers:

 

[Laser Man]

This is a good idea, trying it now.


Those skybox upscales look amazing!


I know how to use PS, but I don't own a copy of it and I'm not inclined to start paying Adobe a boat load of money when GIMP does pretty much everything I've needed to do so far. It would also be way simpler to have everything work off of scripts given the quantity of images we're working with. For example, Star Fox 64 have over 15,000 dumped textures, though most of the appear to be procedurally generated somehow which makes it a pain in the ass to track down individual textures and clean them up. This is also my first serious attempt at modding lol

Click to shrink...

The look of the skybox textures is mostly due to the resynthesizer sharpening plug in for GIMP. A simple ESRGAN upscale was not detailed enough imo, so you could consider it not true to the original. BTW I use PS at work everyday but I do have the same view on adobe as you, I'm not going to pay a monthly subscription (workplace pays for the usage there) and I use GIMP/Krita etc at home. A lot of people are using Image Magic for batch processing images but GIMP could also be automated if (I guess that depends on how much experience you have with script-Fu or Phyton, I'm personally not good enough to write scripts without hours and hours of research, one thing that Photoshop made really easy for non programming people)

 

[collige]

The look of the skybox textures is mostly due to the resynthesizer sharpening plug in for GIMP. A simple ESRGAN upscale was not detailed enough imo, so you could consider it not true to the original. BTW I use PS at work everyday but I do have the same view on adobe as you, I'm not going to pay a monthly subscription (workplace pays for the usage there) and I use GIMP/Krita etc at home. A lot of people are using Image Magic for batch processing images but GIMP could also be automated if (I guess that depends on how much experience you have with script-Fu or Phyton, I'm personally not good enough to write scripts without hours and hours of research, one thing that Photoshop made really easy for non programming people)

Click to shrink...

I'm a developer by day so I guess we're in opposite positions. I already do some basic manipulation with a script, but neither my tools (which use libvips) nor Imagemagick support the kind of fancy upscaling or denoising filters that are built into newer versions of Photoshop, sadly.

 

[natanqb]

I am very curious to see how Dino Crisis 2 would look with these upgrades.

 

[collige]

Just signed up because of literally amazing this thread and technology has been. It truly is a breakthrough game-changer and I've been doing everything I can to spread the word about it everywhere because this isn't just for games, the potential for restoration and enhancement for photos is mind blowing to think of. I wish this would get a LOT more attention than to where it's confined to right now.

That said, I'm wondering if someone would be kind enough to do some for a game that's near and dear to my heart: Twisted Metal 2:

Character Bios:
https://imgur.com/a/s0Hg7Rk



And these few random wallpapers:

Click to shrink...

As it stands, the only full games that can be upscaled are those where the textures are easily accessible and not saved in a proprietary format, or emulated games played on a emulator that can dump textures in real time. As far as I know, that's only possible with Dolphin and Project64 so far (and Project64's support sucks, which is why I'm only doing Wii VC n64 games). This algorithm also requires a shit ton of VRAM that scales with image dimensions. In my case, I have a 1080ti with 11GB and I still can't upscale anything bigger than 1000x1000px.

 

[Shenmue-is-life]

Got this error message:

Don't know what I did wrong. :(

 

[collige]

Got this error message:

Don't know what I did wrong. :(

Click to shrink...

Is this my forked version of ESRGAN? If so, it's modified to work with the node script rather than directly with Python. If you wanna just run it from the command line, you should download the regular version or pass the path to the model in as a command line argument.

 

[Shenmue-is-life]

Is this my forked version of ESRGAN? If so, it's modified to work with the node script rather than directly with Python. If you wanna just run it from the command line, you should download the regular version or pass the path to the model in as a command line argument.

Click to shrink...

I have no clue about how these things work. I only followed the kingdomakrillic's tutorial from OP's text but the procedure was still very confusing. I feel I will probably give up.

 

[collige]

I have no clue about how these things work. I only followed the kingdomakrillic's tutorial from OP's text but the procedure was still very confusing. I feel I will probably give up.

Click to shrink...

Oh, I'm sorry, I was mistaken. That error is way easier to fix, you need to make sure you're actually passing the name of the model to the script. Put in "python test.py models/RRDB_PSNR_x4.pth" exactly.

 

[th1nk]

I am very curious to see how Dino Crisis 2 would look with these upgrades.

Click to shrink...

Great idea! I would like to see this, too! Played that game a lot when it came out.

 

[Shenmue-is-life]

Oh, I'm sorry, I was mistaken. That error is way easier to fix, you need to make sure you're actually passing the name of the model to the script. Put in "python test.py models/RRDB_PSNR_x4.pth" exactly.

Click to shrink...

Thanks, it works. I didn't figure out you had to put the line straight in the cmd.exe.

Indeed, the algorithm is very impressive!

 

[slowbeatle]

Hello all, I did this for the fmv on parasite eve. It gave some nicely sharp results!

 

[Patrick Jr.]

Good stuff!

 

[Sign My Guestbook!]

Hello all, I did this for the fmv on parasite eve. It gave some nicely sharp results!

Click to shrink...

I just realized I'd love to see someone do a real-time game in the style of Squaresoft CG from this period. We can do it now.

 

[GodzFire]

As it stands, the only full games that can be upscaled are those where the textures are easily accessible and not saved in a proprietary format, or emulated games played on a emulator that can dump textures in real time. As far as I know, that's only possible with Dolphin and Project64 so far (and Project64's support sucks, which is why I'm only doing Wii VC n64 games). This algorithm also requires a shit ton of VRAM that scales with image dimensions. In my case, I have a 1080ti with 11GB and I still can't upscale anything bigger than 1000x1000px.

Click to shrink...

I'm still hoping someone could do the stuff I posted though, it's not many. I wish I could.

 

[collige]

Quick proof of concept for Yoshi's Story. I'm hating the game so far, but damn if this doesn't look awesome.

http://screenshotcomparison.com/comparison/130897

 

[BreakAtmo]

Someone upscaled Claire's original RE2 intro.

 

[scitek]

Someone upscaled Claire's original RE2 intro.

Click to shrink...

It looks so clear!

 

[omaroni]

Hello all, I did this for the fmv on parasite eve. It gave some nicely sharp results!

Click to shrink...

is it possible to upload the source file somewhere? Youtube compression destroys every little bit of details.

 

[collige]

Quick test of Majora's Mask. I'm really happy with the results, but I'm not sure if it's worth doing the entire game when the 3DS remake exists. Would people have any interest in this?

http://screenshotcomparison.com/comparison/130955

 

[Jazzem]

Someone upscaled Claire's original RE2 intro.

Click to shrink...

"You're a cop, right?" to a man in uniform driving a police car will never cease to amuse

Great work as always on these trailers!

Quick test of Majora's Mask. I'm really happy with the results, but I'm not sure if it's worth doing the entire game when the 3DS remake exists. Would people have any interest in this?

http://screenshotcomparison.com/comparison/130955

Click to shrink...

Lovely results!

Though for what it's worth, an extensive HD texture pack that partly uses the 3DS textures as inspiration is out there, if that helps decide one way or another.



Still think an ESRGAN upgrade would be neat for authenticity but I can understand if it's felt that time would be better used on another game

 

[collige]

"You're a cop, right?" to a man in uniform driving a police car will never cease to amuse

Great work as always on these trailers!




Lovely results!

Though for what it's worth, an extensive HD texture pack that makes uses of 3DS assets among others is out there, if that helps decide one way or another.



Still think an ESRGAN upgrade would be neat for authenticity but I can understand if it's felt that time would be better used on another game

Click to shrink...

I think it might be cool to do to just to cover any textures that aren't being touched manually by that modder, but outside of that it kinda feels like a waste of effort.

 

[slowbeatle]

is it possible to upload the source file somewhere? Youtube compression destroys every little bit of details.

Click to shrink...

i deleted the source file, quality seems to be on par when i compared. Issues seen here are present in the original fmv. You can compare with other 4k uploads if you like.

 

[Deleted member 22713]

kingdomakrillic (the blogger in the OP who also trained the original Manga109 model) created a new model for removing colour banding.

The trick is to take scaling out of the equation. The HR and LR images should be identical, except that the LR images are degraded in some way, and the HR images should be upscaled with nearest neighbor sampling. Because scaling doesn't matter here, I chose to set "scale" in train_ESRGAN.json to 2 (setting it to 1 produced errors). This meant training a model from scratch, so I set "pretrain_model_G" to null.
Should note that if you change the scaling factor, you also need to change "upscale=4" in test.py when you're testing your model.
The model is here. I don't recommend actually using it, as I don't think I have the right combination of settings and dataset. It's just an attempt to show off the versatility of ESRGAN. https://www.mediafire.com/file/436f2cbzio6sgr4/DebandDemonstration.pth/file
And just for fun, a bad colorization model. https://www.mediafire.com/file/k09ozdl0dchkllp/ColorizeDemonstration.pth/file

Click to shrink...

---

On another note, I trained my own model as a fun experiment; basically curating a dataset of MegaZone 23 screenshots from two distinct sources - BD for HR and DVD for LR - each with their own totally different colours, and the DVD transfers flaws of slight sharpening halos and very slight chromatic abberation.

The only changes made were cropping each pairs screenshots 2 by 2 so they aligned with one another (different transfers with different framing), resizing the DVD shots to be 4:3 (DVD source had non-square pixels) and again to be 25% of the BD's resolution.

Ultimately a dud but if anyone feels like playing with it, here you go:
https://drive.google.com/file/d/1IUyKBfYLKcU2GU677x2udOnUwaYm78CN/view?usp=sharing

Enjoy your gross yellows, neon oversaturated blues and lots of orange. :)

 

[Laser Man]

kingdomakrillic (the blogger in the OP who also trained the original Manga109 model) created a new model for removing colour banding.



---

On another note, I trained my own model as a fun experiment; basically curating a dataset of MegaZone 23 screenshots from two distinct sources - BD for HR and DVD for LR - each with their own totally different colours, and the DVD transfers flaws of slight sharpening halos and very slight chromatic abberation.

The only changes made were cropping each pairs screenshots 2 by 2 so they aligned with one another (different transfers with different framing), resizing the DVD shots to be 4:3 (DVD source had non-square pixels) and again to be 25% of the BD's resolution.

Ultimately a dud but if anyone feels like playing with it, here you go:
https://drive.google.com/file/d/1IUyKBfYLKcU2GU677x2udOnUwaYm78CN/view?usp=sharing

Enjoy your gross yellows, neon oversaturated blues and lots of orange. :)

Click to shrink...

Edit: Nevermind (I need to learn to read better, needed to set upscale to 2, works now)

Here is a comparison using the DKC2 title screen:

http://www.framecompare.com/image-compare/screenshotcomparison/YWWPNNNX#

Only did this test but here the 2 times deband upscale looks quite good in terms of reducing the banding, except for the white in the eyes where it's still noticeable. Upscaling that result using myManga250000 doesn't really work well because the previous upscale ruins the input for the next upscale, myManga250000 sticks too close to the pixels and doesn't smooth over jaggy lines.

However, downscaling the deband upscale to its original size using no interpolation and then scaling that image with the lionking60000 model, is not that bad I think. I didn't denoise the result so it looks rather blotchy still.

The last image is a pre-processed (using GMIC) lionking60000 and Reduced Color mix, this was without the deband model and there are some interesting differences when comparing all results with the Kremlings in the background in mind. It's not easy to preserve their features (eyes, mouth). The botched/still pixelated result actually preserves them the best. There probably is a mix of settings that can get there without the pixelation.

 

[lazygecko]

Here's the Skyrim Whiterun dragon carving texture ingame. Did a bit of extra work on top of it (mostly color/hue stuff) but for the most part it's the result of ESRGAN. Making a whole texture pack partly leveraging ESRGAN for some of the more detailed things.

 

[Flappy Pannus]

Here's the Skyrim Whiterun dragon carving texture ingame. Did a bit of extra work on top of it (mostly color/hue stuff) but for the most part it's the result of ESRGAN. Making a whole texture pack partly leveraging ESRGAN for some of the more detailed things.

Click to shrink...

That looks great. I love the emulator upscales but I'm really curious to see how it does on more detailed source textures and this is a great example.

I'd love for someone to take on the Turians in ME2. The ALOT pack helps a ton but still hasn't touched Turians (outside of Garrus) and it really stands out.

 

[germuso]

Could anybody do It with silent Hill 1? Gameplay textures

 

[Tizoc]

Is there any done to The Longest Journey? I'm trying to get this game to look smoother but neither dgvoodoo nor NVidia Inspector were able to do much :X

 

[Jarmel]

I'm interested in seeing how this would perform on cartoons. Would SFTGAN be better for that?

 

[Deleted member 22713]

I'm interested in seeing how this would perform on cartoons. Would SFTGAN be better for that?

Click to shrink...

I tried ESRGAN on the pilot promo for Miraculous earlier this week.



This is sourced from a 360p YouTube re-upload; much of the detail had already been heavily damaged by compression and further denoising probably destroyed a lot of it (it came out too artifacted otherwise) - you'll shouldn't have any real problems with a good DVD quality source. :)

 

[Sinder]

Would love this to get done with Silent Hill 2 PC

 

[Nerrel]

Lovely results!

Though for what it's worth, an extensive HD texture pack that makes uses of 3DS assets among others is out there, if that helps decide one way or another.

Click to shrink...

Thanks for the mention, but there actually aren't any 3DS textures in the pack, I only based some of them off what the 3D version did. The example below is the 3DS texture vs the hand drawn version, you can see why I'm not using the 3D dump when considering the resolution difference:

The remake usually has textures that are 2x the size of the original, which is still not very high res (often 128x128 instead of N64's 64x64). And the original has a lot of textures that are 16x16 or even 8x8, which makes AI upscaling an unappealing choice. I think you need to have a certain amount of detail to begin with to get good results from it, and I don't think any of the N64 upscales I've seen look very good... it's still just kind of blotches rather than detail.

You'd get better results from the 3DS remakes, maybe to the point where hand drawn textures aren't necessary to do, but I still think the resolution is low enough that you'd get much better results with an actual drawn texture pack. I'd be interested in seeing what this texture would look like AI upscaled (native res here if anyone wants to try it):

 

[Detail]

I have no idea how to do this and sadly I don't have the time to learn but I would genuinely love if someone did this for Kingpin - Life Of Crime.

 

[Jazzem]

Thanks for the mention, but there actually aren't any 3DS textures in the pack, I only based some of them off what the 3D version did. The example below is the 3DS texture vs the hand drawn version, you can see why I'm not using the 3D dump when considering the resolution difference:

The remake usually has textures that are 2x the size of the original, which is still not very high res (often 128x128 instead of N64's 64x64). And the original has a lot of textures that are 16x16 or even 8x8, which makes AI upscaling an unappealing choice. I think you need to have a certain amount of detail to begin with to get good results from it, and I don't think any of the N64 upscales I've seen look very good... it's still just kind of blotches rather than detail.

You'd get better results from the 3DS remakes, maybe to the point where hand drawn textures aren't necessary to do, but I still think the resolution is low enough that you'd get much better results with an actual drawn texture pack. I'd be interested in seeing what this texture would look like AI upscaled (native res here if anyone wants to try it):

Click to shrink...

Thanks for posting in the thread! My bad, I misremembered the video. You did tremendous work :) I'm usually quite fussy on custom texture packs (especially for N64 era) but yours is easily one of if not the best I've seen

 

[Arukado]

I'd be interested in seeing what this texture would look like AI upscaled

Click to shrink...

Just a quick test with the manga109 model:

 

[gabdeg]

Could anybody do It with silent Hill 1? Gameplay textures

Click to shrink...

I tried ripping SH1 textures but the few possibilities you have with PS1 emulation weren't able to properly dump them. I think it has something to do with how the game renders.

Would love this to get done with Silent Hill 2 PC

Click to shrink...

I did some experimentation with TexMod and while it worked, you were severely limited in texture pack size. So even just for the first scenes with relatively few textures the thing was reaching it's limit. There would need to be a better way to get these textures in game.
Result were pretty good though:
http://screenshotcomparison.com/comparison/128826/
http://screenshotcomparison.com/comparison/128854
http://screenshotcomparison.com/comparison/128858/

 

[Sinder]

I tried ripping SH1 textures but the few possibilities you have with PS1 emulation weren't able to properly dump them. I think it has something to do with how the game renders.

I did some experimentation with TexMod and while it worked, you were severely limited in texture pack size. So even just for the first scenes with relatively few textures the thing was reaching it's limit. There would need to be a better way to get these textures in game.
Result were pretty good though:
http://screenshotcomparison.com/comparison/128826/
http://screenshotcomparison.com/comparison/128854
http://screenshotcomparison.com/comparison/128858/

Click to shrink...

That's still a noticeable improvement. I want!

 

[Deleted member 22713]

You'd get better results from the 3DS remakes, maybe to the point where hand drawn textures aren't necessary to do, but I still think the resolution is low enough that you'd get much better results with an actual drawn texture pack. I'd be interested in seeing what this texture would look like AI upscaled (native res here if anyone wants to try it):

Click to shrink...

Tried this with my own training of the Manga109 dataset (still training). It's kind of ended up looking a bit like the normal ESRGAN model. :P

 

[TyrantGuardian]

Looks like the ESRGAN mod for Hexen is out:



Nexus Mods Link

Some of you may recall the earlier examples in the thread with the upscaled weapons:

Click to shrink...

While this is super cool, I don't think this game is well served by sharper textures unfortunately. They repeat way too closely and just look over-detailed where the original washed out feel actually helped with that. Still, glad someone did it!

 

[Nerrel]

Just a quick test with the manga109 model:

Click to shrink...

That would be enough to make the 3DS textures look great in Citra, if it supported custom textures. At the very least, it would be a really solid basis to draw over. It still seems too low res to get the kind of crazy results more modern games have gotten in this thread, though.

 

[hateradio]

I've used waiffu2x before for upscaling images.

I should probably use this one, too.

Hello all, I did this for the fmv on parasite eve. It gave some nicely sharp results!

Click to shrink...

That looks cool.

Why did SE kill that series.

 

[Jarmel]

So I downloaded everything but I'm not getting a result when running the test Python file. Does it need to be in a certain directory along with the Python files?

 

[Laser Man]

So I downloaded everything but I'm not getting a result when running the test Python file. Does it need to be in a certain directory along with the Python files?

Click to shrink...

Leave everything as is once downloaded.
Source images go into the "LR" folder that rests inside the "ESRGAN-master" folder. Results show up in the "results" folder (shocker).

In your cmd or terminal etc you first change the directory so you are inside the "ESRGAN-master" folder with "cd 'path of said folder'", you can drag and drop the folder inside the window if you don't want to type it or copy and paste the path. Then you start the test.py file and chose a model with "python test.py models/RRDB_ESRGAN_x4.pth" (or whatever model you want to use).

 

[Tarot Deck]

How come I just found about this thread now.

I loving everything.

 

[hateradio]

So far I've been impressed with ESRGAN with two images.

It seems to work great with smaller, blurrier images. The results are pretty great compared to waifu2x, which would just result in a blurry image.

I also used SFTGAN a few times. It has some good results on images that are a medium size, because ESRGAN tends to have a pixelated look on boxes/rectangles that are diagonal or like 70 degrees.

However, SFTGAN requires that I upscale the images myself 4x before processing them with it, so it's an extra step that I'd rather not do.

Its results are only slightly better than waifu2x with the UpRes10 model, which is really good for most everything. Then the UpRGB and Y (for a few things).


I have to say that I'm liking all of these different options though.


Some points:

Waifu2x (windows users can use this https://github.com/lltcggie/waifu2x-caffe/releases )

• So far is best when dealing with images that contain JPEG artifacts and banding as it has different levels of cleaning up the image
• Works much better than Photoshop's Surface Blur to get rid of these artifacts
  • In fact, I throw in album art of covers that I can't find in high res
  • It usually sharpens and cleans up lines and letters much better than anything else
• UpRes10 model sometimes doesn't even need a level of blurring to give you the best results for 2D drawn images
• Sometimes leaves images looking a bit too blurry or with a "water color" look (which doesn't look perfect)

ESRGAN

• Works extremely well to sharpen and "draw" textures on small images
• Results are not 100% exactly as the original, but a new image that the AI perceives
• Works great on blurry images
• Results can be too sharp on normal resolution pics, but the different interpolation options help a bit
• Works decently on images that have a lot of JPEG artifacts and banding
  • It usually tries to "draw" the artifacts and banding into part of the image, which doesn't quite work all the time

SFTGAN

• Works well for most images
  • It doesn't leave much of a "water color" effect like Waifu2x
• Also works poorly on images that have a lot of JPEG artifacts and banding
• Doesn't apply a harsh sharpen like ESGAN
• However, requires images to be upsampled 4x nearest neighbor before working
• Seems to work better for some textures compared to ESGAN if you look at the Tumblr page

 

[BreakAtmo]

So far I've been impressed with ESRGAN with two images.

It seems to work great with smaller, blurrier images. The results are pretty great compared to waifu2x, which would just result in a blurry image.

I also used SFTGAN a few times. It has some good results on images that are a medium size, because ESRGAN tends to have a pixelated look on boxes/rectangles that are diagonal or like 70 degrees.

However, SFTGAN requires that I upscale the images myself 4x before processing them with it, so it's an extra step that I'd rather not do.

Its results are only slightly better than waifu2x with the UpRes10 model, which is really good for most everything. Then the UpRGB and Y (for a few things).


I have to say that I'm liking all of these different options though.


Some points:

Waifu2x (windows users can use this https://github.com/lltcggie/waifu2x-caffe/releases )

• So far is best when dealing with images that contain JPEG artifacts and banding as it has different levels of cleaning up the image
• Works much better than Photoshop's Surface Blur to get rid of these artifacts
  • In fact, I throw in album art of covers that I can't find in high res
  • It usually sharpens and cleans up lines and letters much better than anything else
• UpRes10 model sometimes doesn't even need a level of blurring to give you the best results for 2D drawn images
• Sometimes leaves images looking a bit too blurry or with a "water color" look (which doesn't look perfect)

ESRGAN

• Works extremely well to sharpen and "draw" textures on small images
• Results are not 100% exactly as the original, but a new image that the AI perceives
• Works great on blurry images
• Results can be too sharp on normal resolution pics, but the different interpolation options help a bit
• Works decently on images that have a lot of JPEG artifacts and banding
  • It usually tries to "draw" the artifacts and banding into part of the image, which doesn't quite work all the time

SFTGAN

• Works well for most images
  • It doesn't leave much of a "water color" effect like Waifu2x
• Also works poorly on images that have a lot of JPEG artifacts and banding
• Doesn't apply a harsh sharpen like ESGAN
• However, requires images to be upsampled 4x nearest neighbor before working
• Seems to work better for some textures compared to ESGAN if you look at the Tumblr page

Click to shrink...

Is there much chance we might get an AI upscaler that lets you put in a whole video for it to improve?