แ„‚ ๐Ÿ”ถ [18 ์ผ์ฐจ] : EXPLORATION 6. ์ธ๊ณต์ง€๋Šฅ์€ ๊ฐ์ •์„ ์ด์•ผ๊ธฐํ•  ์ˆ˜ ์žˆ์„๊นŒ?

๋ฐฑ๊ฑดยท2022๋…„ 1์›” 20์ผ
0

์ธ๊ณต์ง€๋Šฅ

  • ์ธ๊ณต์ง€๋Šฅ์€ ํ›ˆ๋ จ๋œ ๋ฐ์ดํ„ฐ๋ฅผ ๋ฐ”ํƒ•์œผ๋กœ ๊ฐ€์žฅ ์ตœ์ ์˜ ๋ฐ˜์‘์„ ์ฐพ๋Š” ๊ฒƒ.
  • ํ†ต๊ณ„์ ์ธ ๋ฐฉ๋ฒ•์œผ๋กœ ์˜ค์ฐจ๋ฅผ ํ•ด์†Œ
  • ์กฐ๊ฑด์— ๋”ฐ๋ฅธ ๊ฒฐ๊ณผ๊ฐ’์„ ์ถœ๋ ฅํ•˜๋Š” ๊ฒƒ์—์„œ
  • ์ถœ๋ ฅ๋œ ๋‹ค์ˆ˜์˜ ๊ฒฐ๊ณผ๊ฐ’ ์ค‘์—์„œ ๊ฐ€์ค‘์น˜๊ฐ€ ๋ฐ˜์˜๋œ ์ตœ์ ์˜ ๋ฐ˜์‘์„ ์ถœ๋ ฅํ•˜๋Š” ๊ฒƒ.

ํ…์ŠคํŠธ ๋ฐ์ดํ„ฐ ํ•™์Šต์˜ ๋ฌธ์ œ์ 

  • ํ…์ŠคํŠธ๋ฅผ ์ˆซ์žํ–‰๋ ฌ๋กœ ํ‘œํ˜„ํ•  ์ˆ˜ ์žˆ๋Š”๊ฐ€.
  • ํ…์ŠคํŠธ์˜ ์ž…๋ ฅ ๋ฐ์ดํ„ฐ ์ˆœ์„œ๋ฅผ ์ธ๊ณต์ง€๋Šฅ์— ์–ด๋–ป๊ฒŒ ๋ฐ˜์˜ํ•˜๋Š”๊ฐ€.
    โ†’ ์‚ฌ์ „์„ ์ด์šฉํ•˜์—ฌ ๋‹จ์–ด์˜ ์˜๋ฏธ๋ฅผ ๋‚˜ํƒ€๋‚ด๋Š” ๋ฒกํ„ฐ๋ฅผ ์ง์ง“๋Š”๋‹ค๋ฉด?
    โ†’ ๋”ฅ๋Ÿฌ๋‹์„ ํ†ตํ•ด ๋ฒกํ„ฐ๋ฅผ ์ƒ์„ฑ

๋ฐ์ดํ„ฐ ์–ป๊ธฐ

์—ฌ๋Ÿฌ txt ํŒŒ์ผ์„ ๊ฐ€์ ธ์™€์„œ

ํ•˜๋‚˜์˜ ๋ฐ์ดํ„ฐ๋กœ ๋ฌถ์–ด์„œ

๋ฐ์ดํ„ฐ ํ™•์ธ

์˜ˆ์ œ ๋ฌธ์žฅ

>i feel hungry  
i eat lunch  
now i feel happy

๋ฐ์ดํ„ฐ๋ฅผ ์ˆซ์ž๋กœ

๋„ˆ๋ฌด ๊ธด๋ฌธ์žฅ์€ ์—†์• ๊ณ 

๋‹จ์–ด๋ณ„๋กœ ์ชผ๊ฐœ๊ณ 

# ์ฒ˜๋ฆฌํ•ด์•ผ ํ•  ๋ฌธ์žฅ์„ ํŒŒ์ด์ฌ ๋ฆฌ์ŠคํŠธ์— ์˜ฎ๊ฒจ ๋‹ด์•˜์Šต๋‹ˆ๋‹ค.
sentences=['i feel hungry', 'i eat lunch', 'now i feel happy']

๋”•์…”๋„ˆ๋ฆฌ๋กœ ํ‘œํ˜„ํ•ด์„œ

# ํŒŒ์ด์ฌ split() ๋ฉ”์†Œ๋“œ๋ฅผ ์ด์šฉํ•ด ๋‹จ์–ด ๋‹จ์œ„๋กœ ๋ฌธ์žฅ์„ ์ชผ๊ฐœ ๋ด…๋‹ˆ๋‹ค.
word_list = 'i feel hungry'.split()
print(word_list)

[ํ…์ŠคํŠธ: ์ธ๋ฑ์Šค]๊ตฌ์กฐ๋ฅผ ๋งŒ๋“ค๊ณ 

index_to_word={}  # ๋นˆ ๋”•์…”๋„ˆ๋ฆฌ๋ฅผ ๋งŒ๋“ค์–ด์„œ

# ๋‹จ์–ด๋“ค์„ ํ•˜๋‚˜์”ฉ ์ฑ„์›Œ ๋ด…๋‹ˆ๋‹ค. ์ฑ„์šฐ๋Š” ์ˆœ์„œ๋Š” ์ผ๋‹จ ์ž„์˜๋กœ ํ•˜์˜€์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์‚ฌ์‹ค ์ˆœ์„œ๋Š” ์ค‘์š”ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. 
# <BOS>, <PAD>, <UNK>๋Š” ๊ด€๋ก€์ ์œผ๋กœ ๋”•์…”๋„ˆ๋ฆฌ ๋งจ ์•ž์— ๋„ฃ์–ด์ค๋‹ˆ๋‹ค. 
index_to_word[0]='<PAD>'  # ํŒจ๋”ฉ์šฉ ๋‹จ์–ด
index_to_word[1]='<BOS>'  # ๋ฌธ์žฅ์˜ ์‹œ์ž‘์ง€์ 
index_to_word[2]='<UNK>'  # ์‚ฌ์ „์— ์—†๋Š”(Unknown) ๋‹จ์–ด
index_to_word[3]='i'
index_to_word[4]='feel'
index_to_word[5]='hungry'
index_to_word[6]='eat'
index_to_word[7]='lunch'
index_to_word[8]='now'
index_to_word[9]='happy'

print(index_to_word)

๋ฐ์ดํ„ฐ๋ฅผ ์ˆซ์ž๋กœ ํ‘œํ˜„ํ•˜๊ณ 

# ํ…์ŠคํŠธ๋ฐ์ดํ„ฐ๋ฅผ ์ˆซ์ž๋กœ ๋ฐ”๊พธ๊ธฐ
word_to_index={word:index for index, word in index_to_word.items()}
print(word_to_index)

๊ฐ€์ง„ ๋ฐ์ดํ„ฐ๋ฅผ ์ˆซ์ž๋กœ ๋ณ€ํ™˜ํ•˜๊ณ 

# ๋ฌธ์žฅ 1๊ฐœ๋ฅผ ํ™œ์šฉํ•  ๋”•์…”๋„ˆ๋ฆฌ์™€ ํ•จ๊ป˜ ์ฃผ๋ฉด, ๋‹จ์–ด ์ธ๋ฑ์Šค ๋ฆฌ์ŠคํŠธ๋กœ ๋ณ€ํ™˜ํ•ด ์ฃผ๋Š” ํ•จ์ˆ˜๋ฅผ ๋งŒ๋“ค์–ด ๋ด…์‹œ๋‹ค.
# ๋‹จ, ๋ชจ๋“  ๋ฌธ์žฅ์€ <BOS>๋กœ ์‹œ์ž‘ํ•˜๋Š” ๊ฒƒ์œผ๋กœ ํ•ฉ๋‹ˆ๋‹ค. 
def get_encoded_sentence(sentence, word_to_index):
    return [word_to_index['<BOS>']]+[word_to_index[word] 
                                     if word in word_to_index 
                                     else word_to_index['<UNK>'] 
                                     for word in sentence.split()]

print(get_encoded_sentence('i eat lunch', word_to_index))


def get_encoded_sentences(sentences, word_to_index):
    return [get_encoded_sentence(sentence, word_to_index) 
            for sentence in sentences]

# sentences=['i feel hungry', 'i eat lunch', 'now i feel happy'] ๊ฐ€ 
# ์•„๋ž˜์™€ ๊ฐ™์ด ๋ณ€ํ™˜๋ฉ๋‹ˆ๋‹ค. 

encoded_sentences = get_encoded_sentences(sentences, word_to_index)
print(encoded_sentences)

cf.cf. ๋””์ฝ”๋”ฉํ•˜๊ธฐ (๋ฌธ์ž๋ฅผ ๋‹ค์‹œ ์ˆซ์ž๋กœ)

def get_decoded_sentence(encoded_sentence, index_to_word):
    return ' '.join(index_to_word[index] 
                    if index in index_to_word 
                    else '<UNK>' 
                    for index in encoded_sentence[1:])  
#[1:]๋ฅผ ํ†ตํ•ด <BOS>๋ฅผ ์ œ์™ธ



print(get_decoded_sentence([1, 3, 4, 5], index_to_word))

def get_decoded_sentences(encoded_sentences, index_to_word):
    return [get_decoded_sentence(encoded_sentence, index_to_word) 
            for encoded_sentence in encoded_sentences]

# encoded_sentences=[[1, 3, 4, 5], [1, 3, 6, 7], [1, 8, 3, 4, 9]] ๊ฐ€ 
# ์•„๋ž˜์™€ ๊ฐ™์ด ๋ณ€ํ™˜๋ฉ๋‹ˆ๋‹ค.

print(get_decoded_sentences(encoded_sentences, index_to_word))

Embedding ๋ ˆ์ด์–ด

Embedding ๋ ˆ์ด์–ด

  • ๋‹จ์–ด์˜ ์˜๋ฏธ๋ฅผ ๋‚˜ํƒ€๋‚ด๋Š” ๋ฒกํ„ฐ๋ฅผ ํ›ˆ๋ จ ๊ฐ€๋Šฅํ•œ ํŒŒ๋ผ๋ฏธํ„ฐ๋กœ ๋†“๊ณ 
  • ๋”ฅ๋Ÿฌ๋‹์„ ํ†ตํ•ด ์ตœ์ ํ™”
  • ๋ฌธ์žฅ ๋ฒกํ„ฐ๊ฐ€ ์ผ์ •ํ•˜์ง€ ์•Š์œผ๋ฉด ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒ

๋ฌธ์žฅ ํŒจ๋”ฉ

raw_inputs = tf.keras.preprocessing.sequence.pad_sequences(raw_inputs,
                                                       value=word_to_index['<PAD>'],
                                                       padding='post',
                                                       maxlen=5)
print(raw_inputs)

์‹œํ€€์Šค๋ฅผ ๋‹ค๋ฃจ๋Š” RNN

  • RNN์€ ์‹œํ€€์Šค(Sequence) ํ˜•ํƒœ์˜ ๋ฐ์ดํ„ฐ๋ฅผ ์ฒ˜๋ฆฌํ•˜๊ธฐ์— ์ตœ์ ์ธ ๋ชจ๋ธ
  • ๋‹ค์Œ๋ง์ด ๋ฌด์–ผ์ง€
  • ์‹œ๊ฐ„์˜ ํ๋ฆ„์— ๋”ฐ๋ผ ์ƒˆ๋กญ๊ฒŒ ๋“ค์–ด์˜ค๋Š” ์ž…๋ ฅ์— ๋”ฐ๋ผ ๋ณ€ํ•˜๋Š” ํ˜„์žฌ์˜ ์ƒํƒœ๋ฅผ ๋ฌ˜์‚ฌํ•˜๋Š”
  • State Machine



๊น€์„ฑํ›ˆ ๊ต์ˆ˜์˜ ๋ชจ๋‘์˜ ๋”ฅ๋Ÿฌ๋‹ ๊ฐ•์ขŒ 12๊ฐ•.RNN

์™œ CNN์„ ์“ฐ์ง€ ์•Š๊ณ  RNN์„ ์“ธ๊นŒ?

  • ํ…์ŠคํŠธ๋ฅผ ์ฒ˜๋ฆฌํ•˜๊ธฐ ์œ„ํ•ด RNN์ด ์•„๋‹ˆ๋ผ 1-D Convolution Neural Network(1-D CNN)๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜๋„
  • ์ด๋ฏธ์ง€ ๋ถ„๋ฅ˜๊ธฐ๋ฅผ ๊ตฌํ˜„ํ•˜๋ฉด์„œ 2-D CNN์„ ์‚ฌ์šฉ
  • ์ด๋ฏธ์ง€ ๋ถ„๋ฅ˜๊ธฐ ๋ชจ๋ธ์—๋Š” ์ด๋ฏธ์ง€ ์ „์ฒด๊ฐ€ ํ•œ๊บผ๋ฒˆ์— ์ž…๋ ฅ์œผ๋กœ ์‚ฌ์šฉ
  • 1-D CNN์€ ๋ฌธ์žฅ ์ „์ฒด๋ฅผ ํ•œ๊บผ๋ฒˆ์— ํ•œ ๋ฐฉํ–ฅ์œผ๋กœ ๊ธธ์ด 7์งœ๋ฆฌ ํ•„ํ„ฐ๋กœ ์Šค์บ๋‹ ํ•˜๋ฉด์„œ 7๋‹จ์–ด ์ด๋‚ด์—์„œ ๋ฐœ๊ฒฌ๋˜๋Š” ํŠน์ง•์„ ์ถ”์ถœํ•˜์—ฌ ๊ทธ๊ฒƒ์œผ๋กœ ๋ฌธ์žฅ์„ ๋ถ„๋ฅ˜ํ•˜๋Š” ๋ฐฉ์‹์œผ๋กœ ์‚ฌ์šฉ
  • CNN ๊ณ„์—ด์€ RNN ๊ณ„์—ด๋ณด๋‹ค ๋ณ‘๋ ฌ์ฒ˜๋ฆฌ๊ฐ€ ํšจ์œจ์ ์ด๊ธฐ ๋•Œ๋ฌธ์— ํ•™์Šต ์†๋„๋„ ํ›จ์”ฌ ๋น ๋ฅด๊ฒŒ ์ง„ํ–‰๋œ๋‹ค๋Š” ์žฅ์ 
vocab_size = 10  # ์–ดํœ˜ ์‚ฌ์ „์˜ ํฌ๊ธฐ์ž…๋‹ˆ๋‹ค(10๊ฐœ์˜ ๋‹จ์–ด)
word_vector_dim = 4   # ๋‹จ์–ด ํ•˜๋‚˜๋ฅผ ํ‘œํ˜„ํ•˜๋Š” ์ž„๋ฒ ๋”ฉ ๋ฒกํ„ฐ์˜ ์ฐจ์› ์ˆ˜์ž…๋‹ˆ๋‹ค. 

model = tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(vocab_size, word_vector_dim, input_shape=(None,)))
model.add(tf.keras.layers.Conv1D(16, 7, activation='relu'))
model.add(tf.keras.layers.MaxPooling1D(5))
model.add(tf.keras.layers.Conv1D(16, 7, activation='relu'))
model.add(tf.keras.layers.GlobalMaxPooling1D())
model.add(tf.keras.layers.Dense(8, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))  # ์ตœ์ข… ์ถœ๋ ฅ์€ ๊ธ์ •/๋ถ€์ •์„ ๋‚˜ํƒ€๋‚ด๋Š” 1dim ์ž…๋‹ˆ๋‹ค.

model.summary()

Model: "sequential_2"


Layer (type) Output Shape Param #
\=================================================================
embedding_4 (Embedding) (None, None, 4) 40


conv1d (Conv1D) (None, None, 16) 464


max_pooling1d (MaxPooling1D) (None, None, 16) 0


conv1d_1 (Conv1D) (None, None, 16) 1808


global_max_pooling1d (Global (None, 16) 0


dense_4 (Dense) (None, 8) 136


dense_5 (Dense) (None, 1) 9
\=================================================================
Total params: 2,457
Trainable params: 2,457
Non-trainable params: 0


  • ๊ฐ„๋‹จํžˆ๋Š” GlobalMaxPooling1D() ๋ ˆ์ด์–ด ํ•˜๋‚˜๋งŒ ์‚ฌ์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•
  • ์ „์ฒด ๋ฌธ์žฅ ์ค‘์—์„œ ๋‹จ ํ•˜๋‚˜์˜ ๊ฐ€์žฅ ์ค‘์š”ํ•œ ๋‹จ์–ด๋งŒ ํ”ผ์ฒ˜๋กœ ์ถ”์ถœ
  • ์ถ”์ถœํ•œ ๊ฒƒ์œผ๋กœ ๋ฌธ์žฅ์˜ ๊ธ์ •/๋ถ€์ •์„ ํ‰๊ฐ€ํ•˜๋Š” ๋ฐฉ์‹
vocab_size = 10  # ์–ดํœ˜ ์‚ฌ์ „์˜ ํฌ๊ธฐ์ž…๋‹ˆ๋‹ค(10๊ฐœ์˜ ๋‹จ์–ด)
word_vector_dim = 4   # ๋‹จ์–ด ํ•˜๋‚˜๋ฅผ ํ‘œํ˜„ํ•˜๋Š” ์ž„๋ฒ ๋”ฉ ๋ฒกํ„ฐ์˜ ์ฐจ์› ์ˆ˜์ž…๋‹ˆ๋‹ค. 

model = tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(vocab_size, word_vector_dim, input_shape=(None,)))
model.add(tf.keras.layers.GlobalMaxPooling1D())
model.add(tf.keras.layers.Dense(8, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))  # ์ตœ์ข… ์ถœ๋ ฅ์€ ๊ธ์ •/๋ถ€์ •์„ ๋‚˜ํƒ€๋‚ด๋Š” 1dim ์ž…๋‹ˆ๋‹ค.

model.summary()

Model: "sequential_3"


Layer (type) Output Shape Param #
\=================================================================
embedding_5 (Embedding) (None, None, 4) 40


global_max_pooling1d_1 (Glob (None, 4) 0


dense_6 (Dense) (None, 8) 40


dense_7 (Dense) (None, 1) 9
\=================================================================
Total params: 89
Trainable params: 89
Non-trainable params: 0


๊ฐ ๋ชจ๋ธ์˜ ํŒŒ๋ผ๋ฏธํ„ฐ์— ๋Œ€ํ•œ ์ดํ•ด ํ•„์š”

profile
๋งˆ์ผ€ํŒ…์„ ์œ„ํ•œ ์ธ๊ณต์ง€๋Šฅ ์„ค๊ณ„์™€ ์Šคํƒ€ํŠธ์—… Log

0๊ฐœ์˜ ๋Œ“๊ธ€