Naver API

Binny·2023년 5월 16일
0
post-thumbnail

Naver API

1. 네이버 API 사용 등록

2. 네이버 API 사용하기

  • urllib : http 프로토콜에 따라서 서버의 요청 / 응답을 처리하기 위한 모듈
  • urllib.request : 클라이언트의 요청을 처리하는 모듈
  • urllib.parse : url 주소에 대한 분석

1) 검색 : 블로그 (blog)

# 네이버 검색 API 예제 - 블로그 검색
import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("파이썬")
url = "https://openapi.naver.com/v1/search/blog?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)
  • response
  • response.getcode()
  • response.code
  • response.status
# 글자로 읽을 경우, decode 필요 : utf-8 설정

print(response_body.decode("utf-8"))

2) 검색 : 책(book)

import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("파이썬")
url = "https://openapi.naver.com/v1/search/book?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)

3) 검색 : 영화 (movie)

import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("파이썬")
url = "https://openapi.naver.com/v1/search/movie?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)

4) 검색 : 카페 (cafearticle)

import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("파이썬")
url = "https://openapi.naver.com/v1/search/cafearticle?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)

5) 검색 : 쇼핑 (shop)

import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("파이썬")
url = "https://openapi.naver.com/v1/search/shop?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)

6) 검색 : 백과사전 (encyc)

import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("파이썬")
url = "https://openapi.naver.com/v1/search/encyc?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)

3. 상품 검색

  • "몰스킨"
  • 만들어갈 함수 순서대로 
import os
import sys
import urllib.request

client_id = "oR8ebMed9NE4GjnTolJm"
client_secret = "CcUJOipS8Y"

encText = urllib.parse.quote("몰스킨")
url = "https://openapi.naver.com/v1/search/shop?query=" + encText # JSON 결과
# url = "https://openapi.naver.com/v1/search/blog.xml?query=" + encText # XML 결과
request = urllib.request.Request(url)
request.add_header("X-Naver-Client-Id",client_id)
request.add_header("X-Naver-Client-Secret",client_secret)
response = urllib.request.urlopen(request)
rescode = response.getcode()
if(rescode==200):
    response_body = response.read()
    print(response_body.decode('utf-8'))
else:
    print("Error Code:" + rescode)

1) gen_search_url()

def gen_search_url(api_node, search_text, start_num, disp_num):
    base = "https://openapi.naver.com/v1/search"
    node = "/" + api_node + ".json"
    param_query = "?query=" + urllib.parse.quote(search_text) 
    param_start = "&start=" + str(start_num)
    param_disp = "&display=" + str(disp_num)
    
    return base + node + param_query + param_start + param_disp
  • gen_search_url("shop", "TEST", 10, 3)
'https://openapi.naver.com/v1/search/shop.json?query=TEST&start=10&display=3'

2) get_result_onpage()

  • get data on one page
import json
import datetime

def get_result_onpage(url):
    request = urllib.request.Request(url)
    request.add_header("X-Naver-Client-Id",client_id)
    request.add_header("X-Naver-Client-Secret",client_secret)
    response = urllib.request.urlopen(request)
    print("[%s] Url Request Success" %datetime.datetime.now())
    return json.loads(response.read().decode("utf-8"))
  • datetime.datetime.now()
datetime.datetime(2023, 5, 20, 23, 8, 28, 756779)
  • url = gen_search_url("shop", "몰스킨", 1, 5)
    one_result = get_result_onpage(url)
[2023-05-20 23:10:01.574621] Url Request Success

  • one_result

  • json 파일 형태 : 딕셔너리

  • 딕셔너리 키 : [items] 키 값, 인덱스 0 첫번째 값, [items]의 벨류 값이 리스트, 리스트 안 [title] 키 값

  • one_result["items"][0]["title"]

'<b>몰스킨</b> 노트 가죽 하드커버 감성 고급 업무용 이쁜 심플'
  • one_result["items"][0]["link"]
'https://search.shopping.naver.com/gate.nhn?id=82526953942'
  • one_result["items"][0]["lprice"]
'28800'

3) get_fields()

  • convert pandas data frame
import pandas as pd

def get_fields(json_data):
    title = [each["title"] for each in json_data["items"]]
    link = [each["link"] for each in json_data["items"]]
    lprice = [each["lprice"] for each in json_data["items"]]
    mall_Name = [each["mallName"] for each in json_data["items"]]
    
    result_pd = pd.DataFrame({
        "title" : title ,
        "link" : link ,
        "lprice" : lprice ,
        "mall" : mall_Name ,
    }, columns = ["title","lprice" , "link","mall"])  # 컬럼 순서
    return result_pd
  • get_fields(one_result)

4) delete_tag()

  • < b > 제거
def delete_tag(input_str):
    input_str = input_str.replace("<b>","")
    input_str = input_str.replace("</b>","")
    return input_str
 
 
 
import pandas as pd

def get_fields(json_data):
    title = [delete_tag(each["title"]) for each in json_data["items"]]
    link = [each["link"] for each in json_data["items"]]
    lprice = [each["lprice"] for each in json_data["items"]]
    mall_Name = [each["mallName"] for each in json_data["items"]]
    
    result_pd = pd.DataFrame({
        "title" : title ,
        "link" : link ,
        "lprice" : lprice ,
        "mall" : mall_Name ,
    }, columns = ["title","lprice" , "link","mall"])  # 컬럼 순서
    return result_pd
  • get_fields(one_result)
  • url = gen_search_url("shop", "몰스킨",1, 5)
    json_result = get_result_onpage(url)
    pd_result = get_fields(json_result)
[2023-05-20 23:42:13.059721] Url Request Success
  • pd_result

5) actMain

  • All data gathering
for n in range(1, 1000, 100):
    print(n)
    
---------------------------------------------    


result_mol = []

for n in range(1, 1000, 100):
    url = gen_search_url("shop", "몰스킨",n , 100)
    json_result = get_result_onpage(url)
    pd_result = get_fields(json_result)
    
    result_mol.append(pd_result)

result_mol = pd.concat(result_mol)

  • result_mol.reset_index(drop=True, inplace=True)
    result_mol.info()

  • 가격 lprice object => float 형태 바꾸기
    result_mol["lprice"] = result_mol["lprice"].astype(float)
    result_mol.info()

6) to Excel()

  • Export to Excel
!pip install xlsxwriter
  • 엑셀로 내보내기 작업
writer = pd.ExcelWriter("../data/06_molskin_diary_in_naver_shop.xlsx", engine = "xlsxwriter")
result_mol.to_excel(writer, sheet_name = "sheet1")

workbook = writer.book
worksheet = writer.sheets["sheet1"]
worksheet.set_column("A:A" , 4)  # 4 : 간격
worksheet.set_column("B:B" , 60)
worksheet.set_column("C:C" , 10)
worksheet.set_column("D:D" , 10)
worksheet.set_column("E:E" , 50)
worksheet.set_column("F:F" , 10)

worksheet.conditional_format("C2:C1001" , {"type" : "3_color_scale"})
writer.save()

4. 위 데이터 시각화

import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
from matplotlib import rc
rc("font", family="Malgun Gothic")
plt.figure(figsize=(15,6))
sns.countplot(
    x=result_mol["mall"],   # x축
    data=result_mol,    # 데이터 내용
    palette="RdYlGn",    # 컬러 표현
    order=result_mol["mall"].value_counts().index  # 개수 세주기
)

plt.xticks(rotation=90)  # x축 이름들 겹치지 않게 각도 조절
plt.show()

profile
Binny
물음표 말고 느낌표 !
이전 포스트

유가분석

다음 포스트

Naver API

0개의 댓글

관련 채용 정보