📍Job
Hive에서 json을 사용 시 발생하는 에러
Py4JJavaError: An error occurred while calling o2704.showString.
: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.openx.data.jsonserde.JsonSerDe
at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:74)
at org.apache.spark.sql.hive.execution.HiveTableScanExec.addColumnMetadataToConf(HiveTableScanExec.scala:123)
at org.apache.spark.sql.hive.execution.HiveTableScanExec.hadoopConf$lzycompute(HiveTableScanExec.scala:101)
at org.apache.spark.sql.hive.execution.HiveTableScanExec.hadoopConf(HiveTableScanExec.scala:98)
at org.apache.spark.sql.hive.execution.HiveTableScanExec.org$apache$spark$sql$hive$execution$HiveTableScanExec$$hadoopReader$lzycompute(HiveTableScanExec.scala:110)
at
...- https://s3.us-west-2.amazonaws.com/crawler-public/json/serde/json-serde.jar 에서 jar 파일을 다운받아 S3에 업로드
- Glue Studio Notebook 첫번째 셀로 다음과 같은 코드를 추가
%extra_jars <업로드한 Jar 파일 S3 path(ex. s3://example-bucket/jars/son-serde.jar)>
어쩌다보니 데이터쟁이