Skip to content

Cascading Residual Network (CARN)#

Overview#

The CARN model proposes an architecture that implements a cascading mechanism upon a residual network for accurate and lightweight image super-resolution.

This model also applies the balanced attention (BAM) method invented by Wang et al. (2021) to further improve the results.

It was introduced in the paper Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network by Ahn et al. (2018) and first released in this repository.

CarnConfig#

This is the configuration class to store the configuration of a :class:~super_image.CarnModel. It is used to instantiate the model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the CARN architecture. Configuration objects inherit from :class:~super_image.PretrainedConfig and can be used to control the model outputs. Read the documentation from :class:~super_image.PretrainedConfig for more information.

Examples:

from super_image import CarnModel, CarnConfig
# Initializing a configuration
config = CarnConfig(
    scale=4,                                # train a model to upscale 4x
)
# Initializing a model from the configuration
model = CarnModel(config)
# Accessing the model configuration
configuration = model.config

__init__(self, scale=None, bam=False, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), data_parallel=False, **kwargs) special #

Parameters:

Name Type Description Default
scale int

Scale for the model to train an upscaler/super-res model.

 None
bam bool

Train using balanced attention.

 False
rgb_mean tuple

The RGB mean of the train dataset. You can use ~super_image.utils.metrics.calculate_mean_std to calculate it.

 (0.4488, 0.4371, 0.404)
rgb_std tuple

The RGB standard deviation of the train dataset. You can use ~super_image.utils.metrics.calculate_mean_std to calculate it.

 (1.0, 1.0, 1.0)
data_parallel bool

Option to use multiple GPUs for training.

 False
Source code in super_image\models\carn\configuration_carn.py 
def __init__(self, scale: int = None, bam=False, rgb_mean=DIV2K_RGB_MEAN, rgb_std=DIV2K_RGB_STD,
             data_parallel=False, **kwargs):
    """
    Args:
        scale (int): Scale for the model to train an upscaler/super-res model.
        bam (bool): Train using balanced attention.
        rgb_mean (tuple):
            The RGB mean of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        rgb_std (tuple):
            The RGB standard deviation of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        data_parallel (bool):
            Option to use multiple GPUs for training.
    """
    super().__init__(**kwargs)
    self.scale = scale
    self.bam = bam
    self.rgb_mean = rgb_mean
    self.rgb_std = rgb_std
    self.data_parallel = data_parallel

CarnModel#

config_class #

This is the configuration class to store the configuration of a :class:~super_image.CarnModel. It is used to instantiate the model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the CARN architecture. Configuration objects inherit from :class:~super_image.PretrainedConfig and can be used to control the model outputs. Read the documentation from :class:~super_image.PretrainedConfig for more information.

Examples:

from super_image import CarnModel, CarnConfig
# Initializing a configuration
config = CarnConfig(
    scale=4,                                # train a model to upscale 4x
)
# Initializing a model from the configuration
model = CarnModel(config)
# Accessing the model configuration
configuration = model.config

__init__(self, scale=None, bam=False, rgb_mean=(0.4488, 0.4371, 0.404), rgb_std=(1.0, 1.0, 1.0), data_parallel=False, **kwargs) special #

Parameters:

Name Type Description Default
scale int

Scale for the model to train an upscaler/super-res model.

 None
bam bool

Train using balanced attention.

 False
rgb_mean tuple

The RGB mean of the train dataset. You can use ~super_image.utils.metrics.calculate_mean_std to calculate it.

 (0.4488, 0.4371, 0.404)
rgb_std tuple

The RGB standard deviation of the train dataset. You can use ~super_image.utils.metrics.calculate_mean_std to calculate it.

 (1.0, 1.0, 1.0)
data_parallel bool

Option to use multiple GPUs for training.

 False
Source code in super_image\models\carn\modeling_carn.py 
def __init__(self, scale: int = None, bam=False, rgb_mean=DIV2K_RGB_MEAN, rgb_std=DIV2K_RGB_STD,
             data_parallel=False, **kwargs):
    """
    Args:
        scale (int): Scale for the model to train an upscaler/super-res model.
        bam (bool): Train using balanced attention.
        rgb_mean (tuple):
            The RGB mean of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        rgb_std (tuple):
            The RGB standard deviation of the train dataset.
            You can use `~super_image.utils.metrics.calculate_mean_std` to calculate it.
        data_parallel (bool):
            Option to use multiple GPUs for training.
    """
    super().__init__(**kwargs)
    self.scale = scale
    self.bam = bam
    self.rgb_mean = rgb_mean
    self.rgb_std = rgb_std
    self.data_parallel = data_parallel

forward(self, x) #

Defines the computation performed at every call.

Should be overridden by all subclasses.

.. note:: Although the recipe for forward pass needs to be defined within this function, one should call the :class:Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Source code in super_image\models\carn\modeling_carn.py 
def forward(self, x):
    x = self.sub_mean(x)
    x = self.entry(x)
    c0 = o0 = x

    b1 = self.b1(o0)
    c1 = torch.cat([c0, b1], dim=1)
    o1 = self.c1(c1)

    b2 = self.b2(o1)
    c2 = torch.cat([c1, b2], dim=1)
    o2 = self.c2(c2)

    b3 = self.b3(o2)
    c3 = torch.cat([c2, b3], dim=1)
    o3 = self.c3(c3)

    out = self.upsample(o3, scale=self.scale)

    out = self.exit(out)
    out = self.add_mean(out)

    return out

load_state_dict(self, state_dict, strict=True) #

Copies parameters and buffers from :attr:state_dict into this module and its descendants. If :attr:strict is True, then the keys of :attr:state_dict must exactly match the keys returned by this module's :meth:~torch.nn.Module.state_dict function.

Parameters:

Name Type Description Default
state_dict dict

a dict containing parameters and persistent buffers.

 required
strict bool

whether to strictly enforce that the keys in :attr:state_dict match the keys returned by this module's :meth:~torch.nn.Module.state_dict function. Default: True

 True

Returns:

Type Description
``NamedTuple`` with ``missing_keys`` and ``unexpected_keys`` fields
  • missing_keys is a list of str containing the missing keys
    • unexpected_keys is a list of str containing the unexpected keys
Source code in super_image\models\carn\modeling_carn.py 
def load_state_dict(self, state_dict, strict=True):
    own_state = self.state_dict()
    for name, param in state_dict.items():
        if name in own_state:
            if isinstance(param, nn.Parameter):
                param = param.data
            try:
                own_state[name].copy_(param)
            except Exception:
                if name.find('tail') == -1:
                    raise RuntimeError(f'While copying the parameter named {name}, '
                                       f'whose dimensions in the model are {own_state[name].size()} and '
                                       f'whose dimensions in the checkpoint are {param.size()}.')
        elif strict:
            if name.find('tail') == -1:
                raise KeyError(f'unexpected key "{name}" in state_dict')