Deep Learning Chapter 9 Convolutional Networks

Brief outline below (more of personal guide actually): Read from link.

  1. Convolution Operation Description
    1. Cross Correlation
  2. Why Convolution?
    1. Sparse Interaction
    2. Parameter Sharing
    3. Equivariant Representation
  3. Conv Nets Operation
    1. Convolution
    2. Detector (Nonlinear Function)
    3. Pooling – adding strong prior that the function the layer learns must be invariant to small translations.
  4. Convolution may imply an infinitely strong prior that weights is shared among neighbors and that far edges have 0 weights. This prior makes sense if the feature is equivariant to translation.
  5. Variants of Convolution
    1. 1 kernel = 1 kind of feature. Usually use many kinds of kernel.
    2. downsampling (stride)
    3. border – zero padding
      1. valid convolution
      2. same convolution
      3. full convolution
    4. locally connected layers / unshared convolution
    5. tiled convolution
  6. Structured Output
    1. classification
    2. Tensor
  7. Data Types – can process inputs of varying spatial extents (contains varying amount of observation of the same kind of thing, not optionally contain varying amounts of observation)
  8. Efficient convolution algorithms – If the kernel is “separable”, a much more efficient approach can be used.
  9. We can use the following to train our convolutional network
    1. Random
    2. Greedy layer wise pre-training
    3. Unsupervised learning
  10. Neuroscience basic for conv nets
    1. Gabor Functions
  11. History – In a way, conv nets paved the way to the general acceptance of neural networks.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s