SQL - Python with MySQL

์†”๋น„ยท2024๋…„ 1์›” 23์ผ
0

Python - mysql.connector.Connect | ์„ค์น˜, ์—ฐ๊ฒฐ


๐Ÿงท ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์„ค์น˜
pip install mysql-connector-python
import mysql.connector
๐Ÿงท MySQL ์ ‘์† ์ฝ”๋“œ

  • Local Database ์—ฐ๊ฒฐ
local = mysql.connector.Connect(
    host = 'hostname',
    user = 'username',
    password = 'password'
    # ํŠน์ • Database ์— ์ ‘์†ํ•˜๊ธฐ ์œ„ํ•œ ์ฝ”๋“œ
    , database = 'dbname'
)
local.close()
  • AWS RDS (database-1) ์—ฐ๊ฒฐ
remote = mysql.connector.Connect(
    host = 'hostname',
    port = <port> ,
    user = 'username',
    password = 'password'
    # ํŠน์ • Database ์— ์ ‘์†ํ•˜๊ธฐ ์œ„ํ•œ ์ฝ”๋“œ
    , database = 'dbname'
)
remote.close()

๐Ÿ“Œ ์—ฐ๊ฒฐ ์‹œ ํŠน์ • database์— ์ ‘์†ํ•˜๊ณ ์ž ํ•œ๋‹ค๋ฉด,
database = ์˜ต์…˜์„ ์‚ฌ์šฉํ•˜๋ฉด ๋œ๋‹ค.
๐Ÿ“Œ ์‚ฌ์šฉ ํ›„์—๋Š” ๊ผญ close()๋กœ ๋‹ซ์•„์ฃผ๋„๋ก ํ•œ๋‹ค.


  • Local Database ์—ฐ๊ฒฐ
local = mysql.connector.Connect(
    host = 'localhost',
    user = 'root',
    password = 'pw'
    # ํŠน์ • Database ์— ์ ‘์†ํ•˜๊ธฐ ์œ„ํ•œ ์ฝ”๋“œ
    , database = 'zerobase'
)
local.close()
  • AWS RDS (database-1) ์—ฐ๊ฒฐ
remote = mysql.connector.Connect(
    host = 'database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com',
    port = 3306,
    user = 'admin',
    password = 'pw'
    # ํŠน์ • Database ์— ์ ‘์†ํ•˜๊ธฐ ์œ„ํ•œ ์ฝ”๋“œ
    , database = 'zerobase'
)
remote.close()



execute | Query ์‹คํ–‰


๐Ÿงท

  • ์ ‘์†์ฝ”๋“œ ๋ณ€์ˆ˜์— cursor()๋ฅผ ์—ฐ๊ฒฐํ•ด์ฃผ๊ณ ,
    execute()์˜ ๊ด„ํ˜ธ์•ˆ์— ์ฟผ๋ฆฌ๋ฌธ ์‚ฝ์ž…

  • zerobase db์— sql_file ์ƒ์„ฑ
remote = mysql.connector.Connect(
    host = 'database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com',
    port = 3306,
    user = 'admin',
    password = '๋น„๋ฒˆ์ž…๋ ฅ',
    database = 'zerobase'
)

cur = remote.cursor()
cur.execute('CREATE TABLE sql_file (id int, filename varchar(16))')

remote.close()
# ์ƒ์„ฑ์ „
mysql> use zerobase
Database changed
mysql> show tables;
+--------------------+
| Tables_in_zerobase |
+--------------------+
| celeb              |
| crime_status       |
| police_station     |
| snl_show           |
| test1              |
| test2              |
+--------------------+
6 rows in set (0.15 sec)

#์ƒ์„ฑํ›„
mysql> show tables;
+--------------------+
| Tables_in_zerobase |
+--------------------+
| celeb              |
| crime_status       |
| police_station     |
| snl_show           |
| sql_file           |
| test1              |
| test2              |
+--------------------+
7 rows in set (0.15 sec)

mysql> desc sql_file;
+----------+-------------+------+-----+---------+-------+
| Field    | Type        | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+-------+
| id       | int         | YES  |     | NULL    |       |
| filename | varchar(16) | YES  |     | NULL    |       |
+----------+-------------+------+-----+---------+-------+
2 rows in set (0.15 sec)
  • sql_file ํ…Œ์ด๋ธ” ์‚ญ์ œ
remote = mysql.connector.Connect(
    host = 'database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com',
    port = 3306,
    user = 'admin',
    password = '๋น„๋ฒˆ์ž…๋ ฅ',
    database = 'zerobase'
)

cur = remote.cursor()
cur.execute('DROP TABLE sql_file')

remote.close()
mysql> show tables;
+--------------------+
| Tables_in_zerobase |
+--------------------+
| celeb              |
| crime_status       |
| police_station     |
| snl_show           |
| test1              |
| test2              |
+--------------------+
6 rows in set (0.15 sec)



execute | SQL File


๐Ÿงท

  • execute()์•ˆ์— ์ฝ์–ด์˜จ sqlํŒŒ์ผ๋ณ€์ˆ˜ ๋„ฃ๊ธฐ

1. ๋‹จ์ผ์ฟผ๋ฆฌ ์˜ˆ์ œ

  • TEST03.sql์— ํ…Œ์ด๋ธ”์ƒ์„ฑ ์ฟผ๋ฆฌ ์ž‘์„ฑ ํ›„ ์ €์žฅ

  • ์‹คํ–‰์ฝ”๋“œ์ž‘์„ฑ

remote = mysql.connector.connect(
    host = "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)

cur = remote.cursor()
sql = open('TEST03.sql').read()
cur.execute(sql)

remote.close()
  • ํ…Œ์ด๋ธ”์ƒ์„ฑ ํ™•์ธ
mysql> show tables;
+--------------------+
| Tables_in_zerobase |
+--------------------+
| celeb              |
| crime_status       |
| police_station     |
| snl_show           |
| sql_file           |
| test1              |
| test2              |
+--------------------+
7 rows in set (0.15 sec)

mysql> desc sql_file;
+----------+-------------+------+-----+---------+-------+
| Field    | Type        | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+-------+
| id       | int         | YES  |     | NULL    |       |
| filename | varchar(16) | YES  |     | NULL    |       |
+----------+-------------+------+-----+---------+-------+
2 rows in set (0.16 sec)

2. ๋‹ค์ค‘์ฟผ๋ฆฌ์˜ˆ์ œ

  • TEST04.sql์— insert๋ฌธ ์ž‘์„ฑ ํ›„ ์ €์žฅ
  • ์‹คํ–‰์ฝ”๋“œ์ž‘์„ฑ
remote = mysql.connector.connect(
    host = "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)

cur = remote.cursor()
sql = open('TEST04.sql').read()
for i in cur.execute(sql,multi=True):
    if i.with_rows:
        print(i.fetchall())
    else :
        print(i.statement)

remote.commit()
remote.close()

โ›” error | Commands out of sync; you can't run this command now

remote = mysql.connector.connect(
    host = "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)
cur = remote.cursor()
sql = open('TEST04.sql').read()
cur.execute(sql, multi=True)
remote.commit()
remote.close()

for๋ฌธ์€ ๋ฐ์ดํ„ฐ๋ฅผ ์ฝ˜์†”์ฐฝ์— ์ฐ๊ธฐ์œ„ํ•ด ์ฝ”๋”ฉํ•œ์ค„์•Œ๊ณ 
์œ„์™€๊ฐ™์ด ์ฝ”๋“œ๋ฅผ ์งœ์„œ ์‹คํ–‰ํ•ด๋ณด์•˜๋Š”๋ฐ, error๊ฐ€ ๋–ด๋‹ค.
์ด ์˜ค๋ฅ˜๋Š” ์ด์ „ ์ฟผ๋ฆฌ์˜ ๊ฒฐ๊ณผ ์ง‘ํ•ฉ์„ ๋ชจ๋‘ ์†Œ๋ชจํ•˜์ง€ ์•Š์€ ์ƒํƒœ์—์„œ ๋‹ค์Œ ์ฟผ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•˜๋ ค๊ณ  ํ•  ๋•Œ ๋ฐœ์ƒํ•œ๋‹ค๊ณ  ํ•œ๋‹ค.
๊ฒฐ๊ณผ์ ์œผ๋กœ for๋ฌธ๊นŒ์ง€ ๋ชจ๋‘ ์‹คํ–‰์„ ํ•ด์•ผ๋งŒ ์ •์ƒ์ ์œผ๋กœ ๋‹ค์ค‘์ฟผ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ๋‹ค.

  • ๋ฐ์ดํ„ฐ ํ™•์ธ
mysql> select * from sql_file;
+------+------------+
| id   | filename   |
+------+------------+
|    1 | test01.sql |
|    2 | test02.sql |
|    3 | test03.sql |
|    4 | test04.sql |
+------+------------+
4 rows in set (0.15 sec)



fetch all


๐Ÿงท select ์ฟผ๋ฆฌ๋ฌธ์ฒ˜๋Ÿผ ๊ฒฐ๊ณผ ๊ฐ’์ด row๋ฅผ ํฌํ•จํ•˜๊ณ  ์žˆ์œผ๋ฉด Fetch All ํ•ด์„œ ์ถœ๋ ฅ
(์ฝ์–ด์˜ฌ ๋ฐ์ดํ„ฐ ์–‘์ด ๋งŽ์€ ๊ฒฝ์šฐ buffered=True)

  • sql_file ์ฝ์–ด์˜ค๊ธฐ
remote = mysql.connector.connect(
    host = "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)

cur = remote.cursor(buffered=True)
cur.execute("select * from sql_file")

result = cur.fetchall()
for result_iteraotr in result :
    print(result_iteraotr)

remote.close()
  • pandas๋กœ ๋ถˆ๋Ÿฌ์˜ค๊ธฐ

import pandas as pd

df = pd.DataFrame(result)
df



CSV ์— ์žˆ๋Š” ๋ฐ์ดํ„ฐ๋ฅผ Python ์œผ๋กœ INSERT


  • csv ํŒŒ์ผ ๋ถˆ๋Ÿฌ์˜จ ํ›„ ๋ฐ์ดํ„ฐ ํ™•์ธ

  • AWS mysql ์—ฐ๊ฒฐ

import mysql.connector

conn = mysql.connector.connect(
    host = "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)
  • cursor ์ƒ์„ฑ
cursor = conn.cursor(buffered=True)
  • insert๋ฌธ ์ž‘์„ฑ
sql = "INSERT INTO police_station VALUE (%s,%s)"
  • ๋ฐ์ดํ„ฐ ์ž…๋ ฅ
for i, row in df.iterrows() :
    cursor.execute(sql, tuple(row))
    print(tuple(row))
    conn.commit()

๐Ÿงท tuple ()
์œ„ df value๊ฐ’์— ํ•ด๋‹นํ•˜๋Š” ์—ด ๋ฐ์ดํ„ฐ๊ฐ€ ํŠœํ”Œ๋กœ ๋ฐ˜ํ™˜
๐Ÿงท (%s,%s)
cursor.execute(sql, tuple(row)) ๋ฌธ์—์„œ %s,%s ์— tuple(row)๊ฐ’์ด ๋“ค์–ด๊ฐ

  • ๋ฐ์ดํ„ฐ ํ™•์ธ

  • ๋ฐ์ดํ„ฐํ”„๋ ˆ์ž„์œผ๋กœ ํ™•์ธ



์—ฐ์Šต


crime_status ํ…Œ์ด๋ธ”์— 2020_crime.csv ๋ฐ์ดํ„ฐ๋ฅผ ์ž…๋ ฅํ•˜๋Š” ์ฝ”๋“œ๋ฅผ ์ž‘์„ฑํ•ด๋ณด์ž

  • 2020_crime.csv ๋ฐ์ดํ„ฐ ๋ถˆ๋Ÿฌ์˜ค๊ธฐ

  • AWS RDS(database-1) zerobase ์— ์ ‘์†

import mysql.connector

conn = mysql.connector.connect(
    host =  "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)
  • cursor์—ฐ๊ฒฐ
cursor = conn.cursor(buffered=True)
  • Insert ๋ฌธ ์ž‘์„ฑ
sql = "INSERT INTO crime_status VALUES ('2020',%s,%s,%s,%s)"
  • ๋ฐ์ดํ„ฐ์ž…๋ ฅ
  • ์ž…๋ ฅ๋œ ๋ฐ์ดํ„ฐ ์ฝ๊ธฐ
  • ๋ฐ์ดํ„ฐํ”„๋ ˆ์ž„ํ™”



์‹ค์Šต


  1. AWS RDS(database-1) zerobase ์— ์ ‘์†ํ•ฉ๋‹ˆ๋‹ค
import mysql.connector

conn = mysql.connector.connect(
    host =  "database-1.cpc8kqyoiq1s.ap-southeast-2.rds.amazonaws.com",
    port = 3306,
    user = 'admin',
    password = 'pw',
    database = 'zerobase'
)
  1. cctv Table ์„ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
sql = "CREATE TABLE cctv (๊ธฐ๊ด€๋ช… varchar(8), ์†Œ๊ณ„ int, 2013๋…„๋„์ด์ „ int, 2014๋…„ int, 2015๋…„ int, 2016๋…„ int)"
cursor = conn.cursor(buffered=True)
cursor.execute(sql)
  1. cctv CSV ๋ฐ์ดํ„ฐ๋ฅผ Pandas ๋กœ ์ฝ์–ด์˜ต๋‹ˆ๋‹ค

  2. ๋ฐ์ดํ„ฐ๋ฅผ cctv ํ…Œ์ด๋ธ”์— INSERT ํ•ฉ๋‹ˆ๋‹ค.

  1. cctv ํ…Œ์ด๋ธ”์˜ ๋ฐ์ดํ„ฐ ์กฐํšŒํ•˜์—ฌ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค

  1. ์กฐํšŒ๋œ ๋ฐ์ดํ„ฐ๋ฅผ Pandas ๋กœ ๋ณ€ํ™˜ํ•˜์—ฌ ์ถœ๋ ฅํ•ฉ๋‹ˆ๋‹ค

Daily Study Note

0๊ฐœ์˜ ๋Œ“๊ธ€