deepali.networks.resnet

Convolutional network with residual shortcut connections.

For further reference implementations of 2D and 3D ResNet, see for example:

• pytorch/vision

• kenshohara/3D-ResNets-PyTorch

Module Contents

Classes

ResidualUnit	Sequence of convolutional layers with a residual skip connection.
ResNetConfig	Configuration of residual network architecture.
ResNet	Residual network.

class ResNetConfig

Configuration of residual network architecture.

classmethod from_depth(model_depth: int, spatial_dims: int, in_channels: int = 1, **kwargs) → ResNetConfig

Get default ResNet configuration for given depth.

class ResNet(spatial_dims: int, in_channels: int = 1, num_channels: deepali.core.typing.ScalarOrTuple[int] = (64, 128, 256, 512), num_blocks: deepali.core.typing.ScalarOrTuple[int] = (3, 4, 6, 3), num_layers: int = 2, num_classes: Optional[int] = None, kernel_size: int = 3, stride: deepali.core.typing.ScalarOrTuple[int] = (1, 2, 2, 2), expansion: float = 1, padding_mode: Union[deepali.core.enum.PaddingMode, str] = 'zeros', recursive: deepali.core.typing.ScalarOrTuple[bool] = False, pre_conv: bool = False, post_deconv: bool = False, bias: Union[bool, str] = False, norm: deepali.networks.layers.NormArg = 'batch', acti: deepali.networks.layers.ActivationArg = 'relu', order: str = 'cna', skip: str = 'identity | conv1', residual_pre_conv: str = 'conv1', residual_post_conv: str = 'conv1', conv_layer: ModuleFactory = conv_layer, deconv_layer: ModuleFactory = Upsample, resnet_block: ModuleFactory = ResidualUnit, input_layer: Optional[ModuleFactory] = input_layer, output_layer: Optional[ModuleFactory] = None)

Residual network.

Note that unlike torchvision.models.ResNet, the __init__ function of this class does not initialize the parameters of the model, other than the standard initialization for each module type. In order to apply the initialization of the torchvision ResNet, call functions init_conv_modules(), init_norm_layers(), and zero_init_residuals() (in this order!) after constructing the ResNet model.

Initialize layers.

Parameters

• spatial_dims – Number of spatial tensor dimensions.

• in_channels – Number of input channels.

• num_channels – Number of feature channels at each level.

• num_blocks – Number of residual blocks at each level.

• num_layers – Number of convolutional layers in each residual block.

• num_classes – Number of output class probabilities of output_layer.

• kernel_size – Size of convolutional filters in residual blocks.

• stride – Stride of initial convolution at each level. Subsequent convolutions have stride 1.

• expansion – Expansion factor of num_channels. If specified, the number of input and output feature maps for each residual block are equal to expansion * num_channels, and the bottleneck convolutional layers after an initial convolution with kernel size 1 operate on feature maps with num_channels each, which are subsequently expanded again by the specified expansion level by another convolution with kernel size 1.

• padding_mode – Padding mode for convolutional layers with kernel size greater than 1.

• recursive – Whether residual blocks at each level are applied recursively. If True, all residual blocks at a given level share their convolutional modules. Other modules such as normalization layers are not shared. When recursive=False, a new residual block without any shared modules is created each time. When recursive=True and the number of feature channels or spatial size of a given residual block does not match the number of output channels or spatial size of the preceeding block, respectively, a pre-convolutional layer which adjusts the number of channels and/or spatial size is inserted between these residual blocks even when pre_conv=False.

• pre_conv – Always insert a separate convolutional layer between levels. When recursive=True, this is also the case when pre_conv=False if the output and input tensor shapes of final and first residual block in subsequent levels do not match.

• post_deconv – Whether to place upsampling layers after the sequence of residual blocks for levels with a stride that is less than 1. By default, upsampling is performed as part of the pre_conv. If post_deconv=False, a pre-upsampling layer is always inserted when a level has an initial stride of less than 1 regardless of the pre_conv setting.

• bias – Whether to use bias terms of convolutional layers. Can be either a boolean, or a string indicating the function used to initialize these bias terms (cf. ConvLayer).

• norm – Type of normalization to use in convolutional layers. Use no normalization if None.

• acti – Type of non-linear activation function to use in each convolutional layer.

• order – Order of convolution (C), normalization (N), and non-linear activation (A) in each convolutional layer. If this string does not contain the character n|N, no normalization is performed regardless of the setting of norm (cf. ConvLayer).

• skip – Type(s) of shortcut connections (cf. ResidualUnit).

• residual_pre_conv – Type of pre-convolution when residual unit is a bottleneck block. The kernel size is set to 1 if “conv1”, and kernel_size otherwise.

• residual_post_conv – Type of post-convolution when residual unit is a bottleneck block.

• conv_layer – Type or callable used to create convolutional layers (cf. pre_conv).

• deconv_layer – Type or callable used to create upsampling layers.

• resnet_block – Type or callable used to create residual blocks.

• input_layer – Type or callable used to create initial layer which receives the input tensor.

• output_layer – Type or callable used to create an output layer (head). If None and num_classes is specified, a default classification_head is added.

init_conv_modules() → ResNet

Initialize parameters of convolutions.

init_norm_layers() → ResNet

Initialize normalization layer weights and biases.

zero_init_residuals() → ResNet

Zero-initialize the last normalization layer in each residual branch.