Huh? Textures in Dishonored 2 are extremely high quality. Dishonored's textures are low quality sure but its got this water paint look going on that would be ruined if it received the upscale treatment. I think it'd look worse as a whole IMO.
Gigapixel seems to have the upper hand here from what I can tell, I don't know about what options it has available but maybe the noise reduction is a bit too strong as there is some shadow detail missing (or rather some midtone detail here and there) but it recreates the original image more truthfully as if it has more options and puts more varied details where neccessary (I've only tried recreating the first image tho, it's not a huge difference depending on the model and how you prepare the source image but there is a difference)Pre rendered backgrounds from Ocarina of Time, all done in Gigapixel:
The options are very limited atm, it just lets you select the level of noise and blur reduction between none, moderate and strong. First image is with 'none' selected, second with 'strong':
I know Why would you do that? was working or a script to upscale the alpha channel separately, but I just sorted out the textures with an alpha channel and used waifu2x-caffe for them instead for the Metroid Prime pack.
Sure, did a quick pastebin of both :
No problem. I'd love to hear if it prove usefull and/or if you find problem with it or things to improve.
Function Test-ContainsAlpha($sourceImage)
{
$result = & "magick" identify -format '%[channels]' $sourceImage | Out-String
return ( $result -like '*rgba*' )
}
Function Expand-ImageAlpha($sourceImage, $rgbImage, $alphaImage)
{
& "magick" convert $sourceImage -alpha extract $alphaImage
& "magick" convert $sourceImage -alpha off $rgbImage
return $True
}
Function Merge-ImageAlpha($rgbImage, $alphaImage, $newImage)
{
& "magick" convert $rgbImage $alphaImage -compose CopyOpacity -composite PNG32:$newImage
return $True
}param
(
[string] $inDir
)
$alphaUtilsPath = Join-Path $PSScriptRoot "ImageAlphaUtils.psm1"
$filesToProcess = Get-ChildItem -Path $inDir -Include *.png -Recurse
$filesToProcess | Start-RSJob -ScriptBlock {
Import-Module $Using:alphaUtilsPath
if ($_.FullName -like "*_ALPHA.png" -or $_.FullName -like "*_RGB.png")
{
"Skipping (already processed): $_"
}
elseif (-not (Test-ContainsAlpha $_.FullName))
{
"Skipping (no alpha): $_"
}
else
{
$alphaName = $_.FullName.Replace('.png', '_ALPHA.png')
$rgbName = $_.FullName.Replace('.png', '_RGB.png')
if (Expand-ImageAlpha $_.FullName $rgbName $alphaName)
{
Remove-Item $_
"Processed: $_"
}
else
{
"Failed: $_"
}
}
} | Wait-RSJob | Receive-RSJobparam
(
[string] $inDir
)
$alphaUtilsPath = Join-Path $PSScriptRoot "ImageAlphaUtils.psm1"
$alphaFilesToProcess = Get-ChildItem -Path $inDir -Include *_ALPHA.png -Recurse
$alphaFilesToProcess | Start-RSJob -ScriptBlock {
Import-Module $Using:alphaUtilsPath
$rgbFileName = $_.FullName.Replace('_ALPHA.png', '_RGB.png')
$newFileName = $_.FullName.Replace('_ALPHA.png', '.png')
if (Merge-ImageAlpha $rgbFileName $_.FullName $newFileName)
{
Remove-Item $_ -Force
Remove-Item $rgbFileName -Force
"Processed: $_"
}
else
{
"Failed: $_"
}
} | Wait-RSJob | Receive-RSJobimport sys
import os
import glob
import cv2
import numpy as np
import torch
import architecture as arch
import traceback
model_path = sys.argv[1] # models/RRDB_ESRGAN_x4.pth OR models/RRDB_PSNR_x4.pth
device = torch.device('cuda') # if you want to run on CPU, change 'cuda' -> cpu
# device = torch.device('cpu')
import_folder = 'C:/FilesToProcess/In'
export_folder = 'C:/FilesToProcess/Out'
fail_log_path = os.path.join(export_folder, '_failureLog.txt')
consecutive_failures_to_abort = 5
model = arch.RRDB_Net(3, 3, 64, 23, gc=32, upscale=4, norm_type=None, act_type='leakyrelu', \
mode='CNA', res_scale=1, upsample_mode='upconv')
model.load_state_dict(torch.load(model_path), strict=True)
model.eval()
for k, v in model.named_parameters():
v.requires_grad = False
model = model.to(device)
print('Model path {:s}. \nTesting...'.format(model_path))
def process(torch, model, import_folder, export_folder):
# read image
img = cv2.imread(path, cv2.IMREAD_COLOR)
img = img * 1.0 / 255
img = torch.from_numpy(np.transpose(img[:, :, [2, 1, 0]], (2, 0, 1))).float()
img_LR = img.unsqueeze(0)
img_LR = img_LR.to(device)
output = model(img_LR).data.squeeze().float().cpu().clamp_(0, 1).numpy()
output = np.transpose(output[[2, 1, 0], :, :], (1, 2, 0))
output = (output * 255.0).round()
export_path = export_folder + path[len(import_folder):]
export_path_dir = os.path.dirname(os.path.abspath(export_path))
if not os.path.exists(export_path_dir):
os.makedirs(export_path_dir)
cv2.imwrite(export_path, output)
idx = 0
consecutive_failures = 0
fail_log = None
for path in glob.glob(import_folder + '/**/*.*', recursive=True):
idx += 1
print(idx, path)
try:
process(torch, model, import_folder, export_folder)
consecutive_failures = 0
except Exception as e:
print('Failed to process ({0})'.format(type(e).__name__))
fail_log = open(fail_log_path, 'a')
fail_log.write('***Failed image: ' + path)
fail_log.write('Error: ' + str(e))
fail_log.write(traceback.format_exc())
consecutive_failures += 1
if (consecutive_failures >= consecutive_failures_to_abort):
raise (RuntimeError("Maximum consecutive failures reached, aborting processing"))
finally:
if (fail_log):
fail_log.close()
fail_log = None
If you have integrated graphics, you could try using that as your primary GPU. That should free up the 1080ti for ESRGAN.
The first time in ever these backgrounds actually look decent.Here's Ocarina of Time actually running with some upscaled backgrounds.
Do you have any examples of the recombined final image using this method?
Interesting, I can only guess it would also be pretty easy to do for the other automated solution people have posted, but that seems like something easy enough to automate in my extract.py, this part already check if the alpha is not just only white ( so not transparency at all ) :
white = (255, 255, 255, 255)
# [...] rest of the code
# Get the number of colors in the alpha channel
colors = bg.getcolors()
# If there is only one color, and it's white ( opaque) then do nothing
if ( colors[0][1] == white and len(colors) == 1 ):
print('no alpha channel')
else: # else save the alpha channel as a new image with the same name + "_alpha"
Thanks. At the moment, it looks like there might be too many caveats for using this strategy for batch processing especially since waifu2x-caffe seems to be pretty good at handling alpha images to begin with. Ideally, I think it might be even better to modify the ESRGAN script to not throw away the alpha channel, but I have no idea what the data manipulations in the test.py script are doing. Naively, I think simply changing `cv2.IMREAD_COLOR` to `cv2.IMREAD_UNCHANGED` on line 31 of test.py might work, but I'm not at my PC to test it right now.
I've just been throwing the full color images into it directly without doing any of the alpha channel splitting stuff and it's been pretty good (see the VJ pic above). I really like the idea of doing the split, but my concern ATM is that the independently upscaled alpha channel stuff might result in textures being cutoff in unpredictable ways which may or may not be better than just using a naive method.
Every image with an alpha channel went through Waifu2x while those without were ESRGAN. I just checked in Photoshop, and yeah, it's an alpha mask. Here's the original sprite if you wanna mess around with it.
I just ran my sorting script through all the textures for THPS3 and annoyingly, it seems like pretty much all the textures were dumped with 0.5 transparency so I can't easily tell which ones actually use the alpha channel. I'm thinking that the best solution would be to:
I've got steps 1-6 down, which gives me this image (pre ESRGAN): https://abload.de/img/tex1_61x56_e52cf6d5ef78jf4.png It looks like your "antialiasing" idea works as well.
So it looks like it works. Left is waifu2x, right is ESRGAN used for both the main image and the alpha channel. I tried doing some brightness manipulation on the alpha channel like you recommended, but it didn't seem to make a difference. It also looks like ESRGAN adds way more noise than waifu2x ,which makes sense given their natures. ESRGAN also freaks out around the edges, which sucks and I'm not sure if that's fixable.
So it looks like it works. Left is waifu2x, right is ESRGAN used for both the main image and the alpha channel. I tried doing some brightness manipulation on the alpha channel like you recommended, but it didn't seem to make a difference. It also looks like ESRGAN adds way more noise than waifu2x ,which makes sense given their natures. ESRGAN also freaks out around the edges, which sucks and I'm not sure if that's fixable.
I'm thinking I might just keep using waifu2x for Viewtiful Joe specifically since it fits the artstyle better. I'm gonna try throwing all of THPS3 at it overnight and see if it's usable.
I was about to suggest that.
