[Deep Learning] vanila RNN에서 파라미터의 개수 구하기

출처 : https://stackoverflow.com/questions/50134334/number-of-parameters-for-keras-simplernn

from keras.models import Sequential
from keras.layers import SimpleRNN

model = Sequential()
model.add(SimpleRNN(4, input_shape=(2,3)))
# model.add(SimpleRNN(4, input_length=2, input_dim=3))와 동일함.
model.summary()

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn_2 (SimpleRNN)     (None, 4)                 32        
=================================================================
Total params: 32
Trainable params: 32
Non-trainable params: 0
_________________________________________________________________

Total params = recurrent_weights + input_weights + biases

= (num_units*num_units)+(num_features*num_units) + (1*num_units)

= (num_features + num_units)* num_units + num_units

결과적으로,

( unit 개수 * unit 개수 ) + ( input_dim(feature) 수 + unit 개수 ) + ( 1 * unit 개수)

를 참조하여 위의 total params 를 구하면

(4 4) + (3 4) + (1 * 4) = 32 개 임을 알 수 있다.