https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf
13.3 Semantic segmentation: Per-pixel classification
13.3.1 The U-Net architecture
The U-Net architecture is a design for a neural network that can produce pixelwise output and that was invented for segmentation. As you can see from the highlight
in figure 13.6, a diagram of the U-Net architecture looks a bit like the letter U, which
explains the origins of the name. We also immediately see that it is quite a bit more
complicated than the mostly sequential structure of the classifiers we are familiar with.
Those earlier network designs had problems converging, however, most likely due
to the loss of spatial information during downsampling. Once information reaches a
large number of very downscaled images, the exact location of object boundaries gets harder to encode and therefore reconstruct. To address this, the U-Net authors added
the skip connections we see at the center of the figure. We first touched on skip connections in chapter 8, although they are employed differently here than in the ResNet architecture. In U-Net, skip connections short-circuit inputs along the downsampling path into the corresponding layers in the upsampling path.
