jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.Hugging Face Spaces에 감성분석 모델을 올리려고 작업을 하던 중 토큰화 파트에서 문제가 발생했다. KoNLPy에서는 JAVA_HOME 설정이 필요한데, Spaces에서 이것이 제대로 작동하지 않았다.
모델을 학습할 때 토큰화를 Okt와 Komoran으로 수행했었는데, Spaces에서 예측을 수행할 때 문제가 발생한 것...
결국 JAVA_HOME을 설정하는 방법을 찾든, 다른 토크나이저로 바꾸든 선택을 해야 했음.
from transformers import BertTokenizer
# BERT 모델의 토크나이저 로드
tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased')
# 텍스트 토큰화 (형태소 분석과 유사)
def tokenize(text):
    tokens = tokenizer.tokenize(text)
    return tokens
# 예제
text = "저는 자연어 처리를 좋아합니다."
print(tokenize(text))from transformers import BertTokenizer
# BERT 모델의 토크나이저 로드
tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased')
def tokenize(text):
    tokens = tokenizer.tokenize(text)
    # ## 제거 후 재결합
    clean_tokens = []
    for token in tokens:
        if token.startswith("##"):
            clean_tokens[-1] += token[2:]
        else:
            clean_tokens.append(token)
    return clean_tokens
# 예제
text = "저는 자연어 처리를 좋아합니다."
print(tokenize(text))
'#'를 없애고 처리하는 코드
- 다만 형태소분석에 초점이 맞춰져 있지 않기 때문에 사전에 학습했던 모델과는 차이가 크다.
- 앞서 언급했던 것처럼 예측 성능이 크게 떨어졌다.
import os
import subprocess
# Java 설치 확인 및 설치
try:
    subprocess.run(["java", "-version"], check=True)
except FileNotFoundError:
    print("Java is not installed. Installing Java...")
    subprocess.run(["apt-get", "update"], check=True)
    subprocess.run(["apt-get", "install", "-y", "default-jdk"], check=True)  # 또는 'openjdk-17-jdk'
# JAVA_HOME 환경 변수 설정
java_home = "/usr/lib/jvm/java-17-openjdk-amd64"
if os.path.exists(java_home):
    os.environ['JAVA_HOME'] = java_home
else:
    raise EnvironmentError("JAVA_HOME could not be set because the path does not exist.")
print(f"JAVA_HOME is set to {java_home}")
#이후에 from konlpy.tag import Okt, Komoran와 형태소 분석 파트를 추가하면 된다 
#komoran = Komoran()
#okt = Okt()
# 토큰화를 위한 형태소 분석
# def tokenize(data):
힣힣 이대로 하니까 됐습니다 ^^...
1) setup.sh 파일 작성
setup.sh 파일을 프로젝트의 루트 디렉토리에 추가하고, 다음 내용을 포함하세요:
#!/bin/bash
# Java 설치
apt-get update && apt-get install -y openjdk-11-jdk
# JAVA_HOME 환경 변수 설정
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH
# JAVA_HOME 환경 변수를 ~/.bashrc에 추가하여 지속적으로 유지
echo "export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64" >> ~/.bashrc
echo "export PATH=\$JAVA_HOME/bin:\$PATH" >> ~/.bashrc
# 적용된 환경 변수를 확인
java -version
2) requirements.txt 수정
konlpy
jpype1
Dockerfile의 모습이다.
FROM python:3.10
# Java 설치
RUN apt-get update && apt-get install -y openjdk-11-jdk
# JAVA_HOME 환경 변수 설정
ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
ENV PATH $JAVA_HOME/bin:$PATH
# 필요한 Python 패키지 설치
COPY requirements.txt .
RUN pip install -r requirements.txt
# 앱 코드 복사
COPY . /home/user/app
WORKDIR /home/user/app
# 애플리케이션 실행
CMD ["python", "app.py"]
app.py의 앞부분에 추가
import subprocess
import jpype
import os
# Java 설치 (최초 실행 시 필요)
subprocess.run(["apt-get", "update"])
subprocess.run(["apt-get", "install", "-y", "openjdk-11-jdk"])
# 환경 변수 설정
os.environ['JAVA_HOME'] = '/usr/lib/jvm/java-11-openjdk-amd64'
os.environ['PATH'] = os.environ['JAVA_HOME'] + '/bin:' + os.environ['PATH']
# JVM 시작
jvm_path = "/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so"
jpype.startJVM(jvm_path)
# 나머지 코드
from mor import tokenize
결국 지피티도 빡쳣는지 ... 이런 것 같다고 결론을 냈ㄷ.
해결 방법:
- Java 설치가 완료되었는지 확인: Java 설치는 정상적으로 완료된 것 같습니다. 그러나 kr.lucypark.okt 패키지가 제대로 로드되지 않았습니다.
- KoNLPy와 Java 연결 확인:
kr.lucypark.okt는 KoNLPy의 Okt 분석기가 사용하는 Java 패키지입니다. 이 패키지가 올바르게 설치되고 로드될 수 있도록 확인해야 합니다.- 대체 분석기 고려:
JVM이 필요하지 않은 분석기(예: Mecab)로 전환을 고려할 수 있습니다.
이 문제는 Okt 분석기의 Java 패키지와 관련된 것이므로, 다른 분석기로 전환하거나 Java 환경을 다시 점검해 볼 필요가 있습니다.
File "/home/user/app/mor.py", line 3, in <module>
    komoran = Komoran()
  File "/usr/local/lib/python3.10/site-packages/konlpy/tag/_komoran.py", line 56, in __init__
    jvm.init_jvm(jvmpath, max_heap_size)
  File "/usr/local/lib/python3.10/site-packages/konlpy/jvm.py", line 55, in init_jvm
    jvmpath = jvmpath or jpype.getDefaultJVMPath()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 74, in getDefaultJVMPath
    return finder.get_jvm_path()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 212, in get_jvm_path
    raise JVMNotFoundException("No JVM shared library file ({0}) "
jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.이 오류는 KoNLPy에서 Komoran 형태소 분석기를 초기화할 때 발생하는 문제로, JVM(Java Virtual Machine)이 필요하지만 시스템에서 JVM을 찾지 못했기 때문에 발생합니다. 구체적으로, libjvm.so라는 공유 라이브러리 파일을 찾을 수 없다는 오류입니다.
Traceback (most recent call last):
  File "/home/user/app/app.py", line 6, in <module>
    from mor import tokenize
  File "/home/user/app/mor.py", line 3, in <module>
    komoran = Komoran()
  File "/usr/local/lib/python3.10/site-packages/konlpy/tag/_komoran.py", line 56, in __init__
    jvm.init_jvm(jvmpath, max_heap_size)
  File "/usr/local/lib/python3.10/site-packages/konlpy/jvm.py", line 55, in init_jvm
    jvmpath = jvmpath or jpype.getDefaultJVMPath()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 74, in getDefaultJVMPath
    return finder.get_jvm_path()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 212, in get_jvm_path
    raise JVMNotFoundException("No JVM shared library file ({0}) "
jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.
Traceback (most recent call last):
  File "/home/user/app/app.py", line 6, in <module>
    from mor import tokenize
  File "/home/user/app/mor.py", line 3, in <module>
    komoran = Komoran()
  File "/usr/local/lib/python3.10/site-packages/konlpy/tag/_komoran.py", line 56, in __init__
    jvm.init_jvm(jvmpath, max_heap_size)
  File "/usr/local/lib/python3.10/site-packages/konlpy/jvm.py", line 55, in init_jvm
    jvmpath = jvmpath or jpype.getDefaultJVMPath()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 74, in getDefaultJVMPath
    return finder.get_jvm_path()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 212, in get_jvm_path
    raise JVMNotFoundException("No JVM shared library file ({0}) "
jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.
이 오류는 Hugging Face Spaces에서 JVM(Java Virtual Machine)을 찾지 못했기 때문에 발생합니다. 이를 해결하기 위해서는 Java가 올바르게 설치되고 JAVA_HOME 환경 변수가 제대로 설정되어 있어야 합니다.
Traceback (most recent call last):
  File "/home/user/app/app.py", line 9, in <module>
    jpype.startJVM(jvm_path)
  File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
    _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
Traceback (most recent call last):
  File "/home/user/app/app.py", line 9, in <module>
    jpype.startJVM(jvm_path)
  File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
    _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
이 오류는 libjvm.so 파일을 찾을 수 없다는 것을 의미하며, 경로가 잘못되었거나 Java 설치가 제대로 이루어지지 않았을 가능성이 있습니다.
Traceback (most recent call last):
  File "/home/user/app/app.py", line 9, in <module>
    jpype.startJVM(jvm_path)
  File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
    _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
Traceback (most recent call last):
  File "/home/user/app/app.py", line 9, in <module>
    jpype.startJVM(jvm_path)
  File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
    _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
이 오류는 여전히 libjvm.so 파일을 찾지 못했다는 것을 의미합니다. 현재 지정한 경로가 잘못되었거나, Java가 해당 경로에 설치되지 않았을 가능성이 큽니다.
 Traceback (most recent call last):
  File "/home/user/app/app.py", line 7, in <module>
    from mor import tokenize
  File "/home/user/app/mor.py", line 3, in <module>
    okt = Okt()
  File "/usr/local/lib/python3.10/site-packages/konlpy/tag/_okt.py", line 51, in __init__
    jvm.init_jvm(jvmpath, max_heap_size)
  File "/usr/local/lib/python3.10/site-packages/konlpy/jvm.py", line 55, in init_jvm
    jvmpath = jvmpath or jpype.getDefaultJVMPath()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 74, in getDefaultJVMPath
    return finder.get_jvm_path()
  File "/usr/local/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 212, in get_jvm_path
    raise JVMNotFoundException("No JVM shared library file ({0}) "
jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.
  Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8788 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.8 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [176 kB]
Fetched 9232 kB in 1s (6903 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package openjdk-11-jdk
Traceback (most recent call last):
  File "/home/user/app/app.py", line 7, in <module>
    from mor import tokenize
  File "/home/user/app/mor.py", line 16, in <module>
    jpype.startJVM(jvm_path)
  File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
    _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8788 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.8 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [176 kB]
Fetched 9232 kB in 1s (8020 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package openjdk-11-jdk
Traceback (most recent call last):
  File "/home/user/app/app.py", line 7, in <module>
    from mor import tokenize
  File "/home/user/app/mor.py", line 16, in <module>
    jpype.startJVM(jvm_path)
  File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
    _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
이 오류는 Debian 환경에서 openjdk-11-jdk 패키지를 찾지 못해서 발생한 것입니다. 이 문제를 해결하기 위해 다음을 시도할 수 있습니다:
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8788 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.8 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [176 kB]
Fetched 9232 kB in 1s (7336 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package openjdk-8-jdk
Traceback (most recent call last):
File "/home/user/app/app.py", line 7, in <module>
  from mor import tokenize
File "/home/user/app/mor.py", line 17, in <module>
  jpype.startJVM(jvm_path)
File "/usr/local/lib/python3.10/site-packages/jpype/_core.py", line 247, in startJVM
  _jpype.startup(jvmpath, jvmargs + extra_jvm_args,
OSError: [Errno 0] JVM DLL not found: /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
- openjdk-11-jdk와 openjdk-8-jdk 패키지 모두 설치할 수 없음: Debian bookworm 레포지토리에서 이 패키지들이 없거나 사용할 수 없다는 것을 의미합니다.
- JVM 설치를 위한 다른 방법 필요: Debian 환경에서 지원되는 다른 Java 버전을 설치하거나, openjdk 패키지가 포함된 Docker 이미지로 전환하는 방법을 고려해야 합니다.
Adding debian:Security_Communication_ECC_RootCA1.pem
Adding debian:Security_Communication_RootCA2.pem
Adding debian:Security_Communication_RootCA3.pem
Adding debian:Security_Communication_Root_CA.pem
Adding debian:Starfield_Class_2_CA.pem
Adding debian:Starfield_Root_Certificate_Authority_-_G2.pem
Adding debian:Starfield_Services_Root_Certificate_Authority_-_G2.pem
Adding debian:SwissSign_Gold_CA_-_G2.pem
Adding debian:SwissSign_Silver_CA_-_G2.pem
Adding debian:T-TeleSec_GlobalRoot_Class_2.pem
Adding debian:T-TeleSec_GlobalRoot_Class_3.pem
Adding debian:TUBITAK_Kamu_SM_SSL_Kok_Sertifikasi_-_Surum_1.pem
Adding debian:TWCA_Global_Root_CA.pem
Adding debian:TWCA_Root_Certification_Authority.pem
Adding debian:TeliaSonera_Root_CA_v1.pem
Adding debian:Telia_Root_CA_v2.pem
Adding debian:TrustCor_ECA-1.pem
Adding debian:TrustCor_RootCert_CA-1.pem
Adding debian:TrustCor_RootCert_CA-2.pem
Adding debian:Trustwave_Global_Certification_Authority.pem
Adding debian:Trustwave_Global_ECC_P256_Certification_Authority.pem
Adding debian:Trustwave_Global_ECC_P384_Certification_Authority.pem
Adding debian:TunTrust_Root_CA.pem
Adding debian:UCA_Extended_Validation_Root.pem
Adding debian:UCA_Global_G2_Root.pem
Adding debian:USERTrust_ECC_Certification_Authority.pem
Adding debian:USERTrust_RSA_Certification_Authority.pem
Adding debian:XRamp_Global_CA_Root.pem
Adding debian:certSIGN_ROOT_CA.pem
Adding debian:certSIGN_Root_CA_G2.pem
Adding debian:e-Szigno_Root_CA_2017.pem
Adding debian:ePKI_Root_Certification_Authority.pem
Adding debian:emSign_ECC_Root_CA_-_C3.pem
Adding debian:emSign_ECC_Root_CA_-_G3.pem
Adding debian:emSign_Root_CA_-_C1.pem
Adding debian:emSign_Root_CA_-_G1.pem
Adding debian:vTrus_ECC_Root_CA.pem
Adding debian:vTrus_Root_CA.pem
done.
Setting up openjdk-17-jdk-headless:amd64 (17.0.12+7-2~deb12u1) ...
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jar to provide /usr/bin/jar (jar) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jarsigner to provide /usr/bin/jarsigner (jarsigner) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/javac to provide /usr/bin/javac (javac) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/javadoc to provide /usr/bin/javadoc (javadoc) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/javap to provide /usr/bin/javap (javap) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jdb to provide /usr/bin/jdb (jdb) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jdeprscan to provide /usr/bin/jdeprscan (jdeprscan) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jdeps to provide /usr/bin/jdeps (jdeps) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jfr to provide /usr/bin/jfr (jfr) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jimage to provide /usr/bin/jimage (jimage) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jinfo to provide /usr/bin/jinfo (jinfo) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jlink to provide /usr/bin/jlink (jlink) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jmap to provide /usr/bin/jmap (jmap) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jmod to provide /usr/bin/jmod (jmod) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jps to provide /usr/bin/jps (jps) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jrunscript to provide /usr/bin/jrunscript (jrunscript) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jshell to provide /usr/bin/jshell (jshell) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jstack to provide /usr/bin/jstack (jstack) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jstat to provide /usr/bin/jstat (jstat) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jstatd to provide /usr/bin/jstatd (jstatd) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/serialver to provide /usr/bin/serialver (serialver) in auto mode
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jhsdb to provide /usr/bin/jhsdb (jhsdb) in auto mode
Setting up openjdk-17-jre:amd64 (17.0.12+7-2~deb12u1) ...
Setting up openjdk-17-jdk:amd64 (17.0.12+7-2~deb12u1) ...
update-alternatives: using /usr/lib/jvm/java-17-openjdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode
Traceback (most recent call last):
 File "/home/user/app/app.py", line 7, in <module>
   from mor import tokenize
 File "/home/user/app/mor.py", line 19, in <module>
   okt = Okt()
 File "/usr/local/lib/python3.10/site-packages/konlpy/tag/_okt.py", line 54, in __init__
   OktInterfaceJavaClass = oktJavaPackage.OktInterface
AttributeError: Java package 'kr.lucypark.okt' is not valid이 오류는 Okt 분석기에서 Java 패키지를 로드하는 과정에서 발생한 문제입니다. kr.lucypark.okt Java 패키지가 제대로 설치되지 않았거나, JVM이 해당 패키지를 인식하지 못하고 있습니다.