Enhanced Deep Residual Networks for Single Image Super-Resolution (EDSR)#

Overview#

EDSR is a model that uses both deeper and wider architecture (32 ResBlocks and 256 channels) to improve performance. It uses both global and local skip connections, and up-scaling is done at the end of the network. It doesn't use batch normalization layers (input and output have similar distributions, normalizing intermediate features may not be desirable) instead it uses constant scaling layers to ensure stable training. An L1 loss function (absolute error) is used instead of L2 (MSE), the authors showed better performance empirically and it requires less computation.

The default parameters are for the base model (~5mb vs ~100mb) that includes just 16 ResBlocks and 64 channels.

It was introduced in the paper Enhanced Deep Residual Networks for Single Image Super-Resolution by Lim et al. (2017) and first released in this repository.

EdsrConfig#

This is the configuration class to store the configuration of a :class:~super_image.EdsrModel. It is used to instantiate the model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the EDSR base architecture. Configuration objects inherit from :class:~super_image.PretrainedConfig and can be used to control the model outputs. Read the documentation from :class:~super_image.PretrainedConfig for more information.

Examples:

from super_image import EdsrModel, EdsrConfig
# Initializing a configuration
config = EdsrConfig(
    scale=4,                                # train a model to upscale 4x
)
# Initializing a model from the configuration
model = EdsrModel(config)
# Accessing the model configuration
configuration = model.config

`init(self, scale=None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), no_upsampling=False, res_scale=1, data_parallel=False, **kwargs)` `special` #

Parameters:

Name	Type	Description	Default
`scale`	`int`	Scale for the model to train an upscaler/super-res model.	`None`
`n_resblocks`	`int`	Number of residual blocks.	`16`
`n_feats`	`int`	Number of filters.	`64`
`n_colors`	`int`	Number of color channels.	`3`
`rgb_range`	`int`	Range of RGB as a multiplier to the MeanShift.	`255`
`res_scale`	`int`	The res scale multiplier.	`1`
`rgb_mean`	`tuple`	The RGB mean of the train dataset. You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.	`(0.4488, 0.4371, 0.404)`
`rgb_std`	`tuple`	The RGB standard deviation of the train dataset. You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.	`(1.0, 1.0, 1.0)`
`no_upsampling`	`bool`	Option to turn off upsampling.	`False`
`data_parallel`	`bool`	Option to use multiple GPUs for training.	`False`

Source code in super_image\models\edsr\configuration_edsr.py

def __init__(self, scale: int = None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255,
             rgb_mean=DIV2K_RGB_MEAN, rgb_std=DIV2K_RGB_STD, no_upsampling=False,
             res_scale=1, data_parallel=False, **kwargs):
    """
    Args:
        scale (int): Scale for the model to train an upscaler/super-res model.
        n_resblocks (int): Number of residual blocks.
        n_feats (int): Number of filters.
        n_colors (int):
            Number of color channels.
        rgb_range (int):
            Range of RGB as a multiplier to the MeanShift.
        res_scale (int):
            The res scale multiplier.
        rgb_mean (tuple):
            The RGB mean of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        rgb_std (tuple):
            The RGB standard deviation of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        no_upsampling (bool):
            Option to turn off upsampling.
        data_parallel (bool):
            Option to use multiple GPUs for training.
    """
    super().__init__(**kwargs)
    self.scale = scale
    self.n_resblocks = n_resblocks
    self.n_feats = n_feats
    self.n_colors = n_colors
    self.rgb_range = rgb_range
    self.res_scale = res_scale
    self.rgb_mean = rgb_mean
    self.rgb_std = rgb_std
    self.no_upsampling = no_upsampling
    self.data_parallel = data_parallel

EdsrModel#

`config_class` #

This is the configuration class to store the configuration of a :class:~super_image.EdsrModel. It is used to instantiate the model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the EDSR base architecture. Configuration objects inherit from :class:~super_image.PretrainedConfig and can be used to control the model outputs. Read the documentation from :class:~super_image.PretrainedConfig for more information.

Examples:

from super_image import EdsrModel, EdsrConfig
# Initializing a configuration
config = EdsrConfig(
    scale=4,                                # train a model to upscale 4x
)
# Initializing a model from the configuration
model = EdsrModel(config)
# Accessing the model configuration
configuration = model.config

`init(self, scale=None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), no_upsampling=False, res_scale=1, data_parallel=False, **kwargs)` `special` #

Parameters:

Name	Type	Description	Default
`scale`	`int`	Scale for the model to train an upscaler/super-res model.	`None`
`n_resblocks`	`int`	Number of residual blocks.	`16`
`n_feats`	`int`	Number of filters.	`64`
`n_colors`	`int`	Number of color channels.	`3`
`rgb_range`	`int`	Range of RGB as a multiplier to the MeanShift.	`255`
`res_scale`	`int`	The res scale multiplier.	`1`
`rgb_mean`	`tuple`	The RGB mean of the train dataset. You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.	`(0.4488, 0.4371, 0.404)`
`rgb_std`	`tuple`	The RGB standard deviation of the train dataset. You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.	`(1.0, 1.0, 1.0)`
`no_upsampling`	`bool`	Option to turn off upsampling.	`False`
`data_parallel`	`bool`	Option to use multiple GPUs for training.	`False`

Source code in super_image\models\edsr\modeling_edsr.py

def __init__(self, scale: int = None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255,
             rgb_mean=DIV2K_RGB_MEAN, rgb_std=DIV2K_RGB_STD, no_upsampling=False,
             res_scale=1, data_parallel=False, **kwargs):
    """
    Args:
        scale (int): Scale for the model to train an upscaler/super-res model.
        n_resblocks (int): Number of residual blocks.
        n_feats (int): Number of filters.
        n_colors (int):
            Number of color channels.
        rgb_range (int):
            Range of RGB as a multiplier to the MeanShift.
        res_scale (int):
            The res scale multiplier.
        rgb_mean (tuple):
            The RGB mean of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        rgb_std (tuple):
            The RGB standard deviation of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        no_upsampling (bool):
            Option to turn off upsampling.
        data_parallel (bool):
            Option to use multiple GPUs for training.
    """
    super().__init__(**kwargs)
    self.scale = scale
    self.n_resblocks = n_resblocks
    self.n_feats = n_feats
    self.n_colors = n_colors
    self.rgb_range = rgb_range
    self.res_scale = res_scale
    self.rgb_mean = rgb_mean
    self.rgb_std = rgb_std
    self.no_upsampling = no_upsampling
    self.data_parallel = data_parallel

`forward(self, x)` #

Defines the computation performed at every call.

Should be overridden by all subclasses.

.. note:: Although the recipe for forward pass needs to be defined within this function, one should call the :class:Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Source code in super_image\models\edsr\modeling_edsr.py

def forward(self, x):
    x = self.head(x)

    res = self.body(x)
    res += x

    if self.args.no_upsampling:
        x = res
    else:
        x = self.tail(res)

    return x

`load_state_dict(self, state_dict, strict=True)` #

Copies parameters and buffers from :attr:state_dict into this module and its descendants. If :attr:strict is True, then the keys of :attr:state_dict must exactly match the keys returned by this module's :meth:~torch.nn.Module.state_dict function.

Parameters:

Name	Type	Description	Default
`state_dict`	`dict`	a dict containing parameters and persistent buffers.	required
`strict`	`bool`	whether to strictly enforce that the keys in :attr:`state_dict` match the keys returned by this module's :meth:`~torch.nn.Module.state_dict` function. Default: `True`	`True`

Returns:

Type	Description
``NamedTuple`` with ``missing_keys`` and ``unexpected_keys`` fields	missing_keys is a list of str containing the missing keys unexpected_keys is a list of str containing the unexpected keys

Source code in super_image\models\edsr\modeling_edsr.py

def load_state_dict(self, state_dict, strict=True):
    own_state = self.state_dict()
    for name, param in state_dict.items():
        if name in own_state:
            if isinstance(param, nn.Parameter):
                param = param.data
            try:
                own_state[name].copy_(param)
            except Exception:
                if name.find('tail') == -1:
                    raise RuntimeError(f'While copying the parameter named {name}, '
                                       f'whose dimensions in the model are {own_state[name].size()} and '
                                       f'whose dimensions in the checkpoint are {param.size()}.')
        elif strict:
            if name.find('tail') == -1:
                raise KeyError(f'unexpected key "{name}" in state_dict')

Enhanced Deep Residual Networks for Single Image Super-Resolution (EDSR)#

Overview#

EdsrConfig#

__init__(self, scale=None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), no_upsampling=False, res_scale=1, data_parallel=False, **kwargs) special #

EdsrModel#

config_class #

__init__(self, scale=None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), no_upsampling=False, res_scale=1, data_parallel=False, **kwargs) special #

forward(self, x) #

load_state_dict(self, state_dict, strict=True) #

`init(self, scale=None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), no_upsampling=False, res_scale=1, data_parallel=False, **kwargs)` `special` #

`config_class` #

`init(self, scale=None, n_resblocks=16, n_feats=64, n_colors=3, rgb_range=255, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), no_upsampling=False, res_scale=1, data_parallel=False, **kwargs)` `special` #

`forward(self, x)` #

`load_state_dict(self, state_dict, strict=True)` #