[AI] GPU out-of-memory문제 시 메모리 해제!

• 메모리 확인

nvidia-smi

Tue Feb  4 14:12:32 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.120                Driver Version: 550.120        CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        Off |   00000000:0A:00.0 Off |                  Off |
|  0%   56C    P8             51W /  450W |   24160MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      1724      G   /usr/lib/xorg/Xorg                            105MiB |
|    0   N/A  N/A      1841      G   /usr/bin/gnome-shell                           17MiB |
|    0   N/A  N/A    118292      C   ...ee/anaconda3/envs/user/bin/python      24020MiB |
+-----------------------------------------------------------------------------------------+

PID 확인

 nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv
pid, process_name, used_gpu_memory [MiB]

pid, process_name, used_gpu_memory [MiB]
123456, /home/user/anaconda3/envs/aileen/bin/python, 24020 MiB

Kill Process(PID)

kill -9 123456