[pyspark] Pandas DataFrame을 Spark DataFrame으로 변환(spark.createDataFrame)

Pandas DataFrame을 Spark DataFrame으로 변환(spark.createDataFrame)

import pandas as pd
## Create Pandas Frame
pd_df = df = pd.DataFrame({'id': ['a', 'b', 'c', 'd'],
            'col_1': [1, 2, 3, 4],
            'col_2': [1, 1, 2, 2]},
            columns = ['id', 'col_1', 'col_2'])
## Convert into Spark DataFrame
spark_df = spark.createDataFrame(pd_df)
## Write Frame out as Table
spark_df.write.mode("overwrite").saveAsTable("db.table_name")