1. Inference in Code

• Dense to create a layer.

2. Data in Tensorflow

• row x column = 2D array / 2D Matrix
  
• 2 columns for each feature.
  
• 3 units makes 1 x 3 matrix.
• Tensor is a data type that represents an output matrix from the layer.
  • Can be converted to array with a1.numpy().

3. Building a Neural Network

• Sequential(Dense(), Dense(), ...) to group together the layers.
• model.compile() defines a loss function and specifies a compile optimization.
• model.fit(): runs gradient descent and fits the weights to the data.
• model.predict(): to inference the values according to the model.

4. Forward Prop in a Single Layer

• 1D array for x because only 1 training set data is used here.
• The size of w vector is the same as the number of columns of w.

5. General Implementation of Forward Propagation

• W are in 2 x 1 matrices added together to form 2 x 3 matrices:
  • Mathematical Intution: (1 x 2)(inputs) x (2 x 3)(W) -> (1 x 3)(Output matrix)
  • So the number of rows of W matrix must be the same as the number of columns of activation values.
• W.shape[1] is the same as the num of the columns.