Import fastai deep learning library including pretrained vision models.

from fastai2.basics import *
from fastai2.callback.all import *
from fastai2.vision.all import *
from torch import torch

Import the fa_convnav.navigator module

from fa_convnav.navigator import *

Create a fastai datablock and dataloader using the Oxford PetsII dataset (included with fastai install), and apply some simple image transforms in the process.

pets = DataBlock(blocks=(ImageBlock, CategoryBlock), 
                 get_items=get_image_files, 
                 splitter=RandomSplitter(),
                 get_y=RegexLabeller(pat = r'/([^/]+)_\d+.jpg$'),
                 item_tfms=Resize(460),
                 batch_tfms=[*aug_transforms(size=224, max_rotate=30, min_scale=0.75), Normalize.from_stats(*imagenet_stats)])

dls = pets.dataloaders(untar_data(URLs.PETS)/"images",  bs=128)

Download the pretrained model we want to use

model = resnet18

Create a fastai Learner object from the dataloader, the chosen model, an optimiser. We will use the error rate as our metric.

learn = cnn_learner(
    dls, 
    model, 
    opt_func=partial(Adam, lr=slice(3e-3), wd=0.01, eps=1e-8), 
    metrics=error_rate, 
    config=cnn_config(ps=0.33)).to_fp16()

The model is ready to be trained but, now we have a Learner, it can also be viewed in detail using fa_convnav. Create a ConvNav instance.

cn = ConvNav(learn, learn.summary())

Creating an instance automatically builds a datraframe representation of the model (a CNDF dataframe). This step can take a few moments, especially for larger models such as densenets or xresnets which have hundreds of modules. If you have more than one Learner in your project, instantiate a separate ConvNav instance for each one.

Once the dataframe is built we can view, search and select from it. For example, print some summary information:

print(cn.model_info)

Resnet: Resnet18
Input shape: [128 x 3 x 224 x 224] (bs, ch, h, w)
Output features: [128 x 37] (bs, classes)
Currently frozen to parameter group 3 out of 3

Examine summary information about the body and head of the model.

cn.divs

Resnet: Resnet18
Input shape: [128 x 3 x 224 x 224] (bs, ch, h, w)
Divisions:  body (0), head (1)

View the whole CNDF dataframe. For brevity, only the first ten modules are displayed here, but set top = False (or remove) and all 79 rows of the dataframe will be shown. Run the notebook and try it!

cn.view(top=True)

Resnet: Resnet18
Input shape: [128 x 3 x 224 x 224] (bs, ch, h, w)
Output features: [128 x 37] (bs, classes)
Currently frozen to parameter group 3 out of 3

...69 more layers

We can examine the layers of the head only.

cn.head

Resnet: Resnet18
Input shape: [128 x 512 x 7 x 7] (bs, filt, h, w)
Output features: [128 x 37] (bs, classes)

Select a single block of modules.

block = cn.search('0.4.0')

1 layers found matching searchterm(s): 0.4.0

Look at just those layers with dimension change between input and outputs.

layers = cn.dim_transitions

Resnet18
Layer dimension changes

When selections are made, the selected modules are shown in a dataframe (above) but the corresponding module objects are returned to the user in a list (below). Module objects can be used to apply Pytorch hooks and fastai callbacks to further investigate model function.

layers

[Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False),
 Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False),
 Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False),
 Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False),
 Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)]

Or select a number of blocks evenly spaced over the model. This, and many of the fa_convnav selection methods, are trivial on a small model like vgg or resnet18 but come into their own when used with large, complex models such as densenets and xresnets where there may be many dozens of blocks and hundreds of layers.

spread = cn.spread('block', 4)

resnet18
Spread of block where n = 4

Selected blocks objects are also returned for further model investigation.

for b in spread:
  print(f'{b}\n')

BasicBlock(
  (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)

BasicBlock(
  (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (downsample): Sequential(
    (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
    (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
)

BasicBlock(
  (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (downsample): Sequential(
    (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
    (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
)

BasicBlock(
  (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu): ReLU(inplace=True)
  (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)

Finally CNDF dataframes can be saved and retrieved from persistenmt storage (see documentation). This allows you to work with a CNDF dataframe without the need to create it again from scratch every time.

	Module_name	Model	Division	Container_child (num)	Container_block (num)	Layers	Torch_class	Output_dimensions	Parameters	Currently
Index
1	0	resnet18	Sequential	8	8	53	torch.nn.modules.container.Sequential	[128 x 512 x 7 x 7]	11176512	Frozen
67	1	resnet18	Sequential	9	0	10	torch.nn.modules.container.Sequential	[128 x 37]	546304

	Module_name	Model	Division	Container_child	Container_block	Layer_description	Torch_class	Output_dimensions	Parameters	Trainable	Currently
Index
0		Sequential					torch.nn.modules.container.Sequential
1	0		Sequential				torch.nn.modules.container.Sequential
2	0.0			Conv2d		Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 64 x 112 x 11]	9,408	False	Frozen
3	0.1			BatchNorm2d		BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)	torch.nn.modules.batchnorm.BatchNorm2d	[128 x 64 x 112 x 11]	128	True
4	0.2			ReLU		ReLU(inplace=True)	torch.nn.modules.activation.ReLU	[128 x 64 x 112 x 11]	0	False
5	0.3			MaxPool2d		MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)	torch.nn.modules.pooling.MaxPool2d	[128 x 64 x 56 x 56]	0	False
6	0.4			Sequential			torch.nn.modules.container.Sequential
7	0.4.0				BasicBlock		torchvision.models.resnet.BasicBlock
8	0.4.0.conv1					Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 64 x 56 x 56]	36,864	False	Frozen
9	0.4.0.bn1					BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)	torch.nn.modules.batchnorm.BatchNorm2d	[128 x 64 x 56 x 56]	128	True

	Module_name	Division	Container_child	Layer_description	Torch_class	Output_dimensions	Parameters	Trainable
Index
67	1	Sequential			torch.nn.modules.container.Sequential
68	1.0		AdaptiveConcatPool2d		fastai2.layers.AdaptiveConcatPool2d
69	1.0.ap			AdaptiveAvgPool2d(output_size=1)	torch.nn.modules.pooling.AdaptiveAvgPool2d	[128 x 512 x 1 x 1]	0	False
70	1.0.mp			AdaptiveMaxPool2d(output_size=1)	torch.nn.modules.pooling.AdaptiveMaxPool2d	[128 x 512 x 1 x 1]	0	False
71	1.1		Flatten	Flatten(full=False)	fastai2.layers.Flatten	[128 x 1024]	0	False
72	1.2		BatchNorm1d	BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)	torch.nn.modules.batchnorm.BatchNorm1d	[128 x 1024]	2,048	True
73	1.3		Dropout	Dropout(p=0.165, inplace=False)	torch.nn.modules.dropout.Dropout	[128 x 1024]	0	False
74	1.4		Linear	Linear(in_features=1024, out_features=512, bias=False)	torch.nn.modules.linear.Linear	[128 x 512]	524,288	True
75	1.5		ReLU	ReLU(inplace=True)	torch.nn.modules.activation.ReLU	[128 x 512]	0	False
76	1.6		BatchNorm1d	BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)	torch.nn.modules.batchnorm.BatchNorm1d	[128 x 512]	1,024	True
77	1.7		Dropout	Dropout(p=0.33, inplace=False)	torch.nn.modules.dropout.Dropout	[128 x 512]	0	False
78	1.8		Linear	Linear(in_features=512, out_features=37, bias=False)	torch.nn.modules.linear.Linear	[128 x 37]	18,944	True

	Module_name	Model	Division	Container_child	Container_block	Layer_description	Torch_class	Output_dimensions	Parameters	Trainable	Currently
Index
7	0.4.0				BasicBlock		torchvision.models.resnet.BasicBlock

	Module_name	Division	Container_child	Container_block	Layer_description	Torch_class	Output_dimensions	Parameters	Trainable	Currently
Index
2	0.0	0	Conv2d		Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 64 x 112 x 11]	9,408	False	Frozen
8	0.4.0.conv1	0	4	0	Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 64 x 56 x 56]	36,864	False	Frozen
21	0.5.0.conv1	0	5	0	Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 128 x 28 x 28]	73,728	False	Frozen
37	0.6.0.conv1	0	6	0	Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 256 x 14 x 14]	294,912	False	Frozen
53	0.7.0.conv1	0	7	0	Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)	torch.nn.modules.conv.Conv2d	[128 x 512 x 7 x 7]	1,179,648	False	Frozen

Examples00