1. How does python find modules packages?
In Python, several modules are combined to form a project. A module package can be added using pip, and depending on the purpose, web development, AI/ML development, game development, etc. are possible. You can create more diverse projects using the local package module created by individual developers. So how does python find modules and packages using import ?
[Import Search Order]
Python finds modules/packages in the following order in three main areas.
sys.modules => built-in modules => sys.path
1) sys.module
This is the area that Python searches first to find a module or package, which is in the form of a dictionary.
>>> import sys >>> print(sys.modlues) {
'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': <module 'importlib._bootstrap' (frozen)>, '_imp': <module '_imp' (built-in)>, '_thread': <module '_thread' (built-in)>, '_warnings': <module '_warnings' (built-in)>, '_weakref': <module '_weakref' (built-in)>, 'zipimport': <module 'zipimport' (built-in)>, '_frozen_importlib_external': <module 'importlib._bootstrap_external' (frozen)>, '_io': <module 'io' (built-in)>, 'marshal': <module 'marshal' (built-in)>, 'posix': <module 'posix' (built-in)>, 'encodings': <module 'encodings' from .... '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/rlcompleter.py'>, 'mod1': <module 'mod1' from '/Users/YB/Google 드라이브/TIL/Documents/Python/mod1.py'>}If python is executed and imported more than once, the module or package is used by checking sys.modules immediately without searching for the module again.
Only modules or packages that have been used more than once or are already built-in exist in sys.module, so there are no new imports.
2) built-in modules
This is a library that is officially provided by Python. Of course, they are installed as soon as you install them, and they are easy to find. If you check the output of sys.modules print() above, you can easily check which are built-in modules.
3) sys.path
This is the last part that Python checks when looking for a module or package. Packages newly installed with pip are also found here, and when you want to use a newly created package or module, register the path here to find it. (The setting method is slightly different depending on the OS of the system. .) The variable is configured in the form of a list. (It can be assigned and deleted easily.)
>>> import sys >>> print(sys.path) [
'', '/Users/YB/Google 드라이브/TIL/Documents/Python', '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload', '/Users/YB/Library/Python/3.7/lib/python/site-packages', '/usr/local/lib/python3.7/site-packages'](If you look at the first index string in the list, it is confirmed that it is the registered path.)
If a module or package cannot be found in the above three areas, a ModuleNotFoundError is occured.
How to search sys module
If so, how do you find the sys module that is the source of the module/package search?
'sys': <module 'sys' (built-in)>Since the sys module is already built-in, it is found in the part where the built-in modules are.
2. Absolute path and Relative path(절대경로와 상대경로)
When using Python, if you know how to use the built-in module and the pip module well, you won't have much problem with the path.
(Because the path is clearly defined, built-in is built-in as it is installed, and pip is stored in site-package in each python installation path. The path is stored in sys-path) The problem is when using your own modules or packages.
my_app
├── main.py
├── pkg1
│ ├── __init__.py
│ ├── module1.py
│ └── module2.py
├── pkg2
│ ├── __init__.py
│ ├── module3.py
│ ├── module4.py
│ ├── module5.py
│ └── pkg4
│ ├── __init__.py
│ └── module6.py
└── pkg3
├── __init__.py
└── module7.pySuppose we have a project like the one above.
1) Absolute path (절대경로)
The absolute path means a clearly specified path without omission or abbreviation from the beginning to the end of modules and packages. It is always the same no matter where you use it, and it has the advantage of not getting confused when using it.
2) Relative path (상대경로)
# pkg2/module5.py
# relative path를 적용하기 전
from pkg1.module2 import func3
from pkg2 import module3
from pkg2.pkg4.module6 import func10The relative path is defined based on the path to which it belongs, not the top-level path. It is mainly used to refer to other local packages within a local package.
# pkg2/module5.py
# relative path를 적용한 후
from ..module2 import func3
from . import module3
from .pkg4.module6 import func10# pkg4/module6.py from ..module5 import func_12In both examples above, <from .. import> is newly applied based on the current path (pkg2, pkg4).
It has the advantage of being concise compared to the absolute path, but it can be confusing as the project grows and the complexity increases.
If the location of the file changes, the path must be reset accordingly.
Therefore, no matter how complex the project is, the use of absolute path is recommended for code consistency and to prevent mistakes.
3. Imports in Main Module
If the above example is imported and executed with a relative path, the following error occurs.
Traceback (most recent call last):
File "c:\Users\alsdn\Documents\Study_dir\test_cal_dir\main.py", line 5, in <module>
from .calculator.add_and_multiply import add_and_multiply
ImportError: attempted relative import with no known parent packageThe causes of error
-
When the Python interpreter performs relative import, the location of the module search base is determined by the name attribute.
-
When executing directly from the terminal, name == 'main', so the Python interpreter does not understand the location of the module called main.
To fix the Error: As a result of referring to the official Python documentation, the import path must be an absolute path for the module intended to be used as the main module.
Reference : https://docs.python.org/3/tutorial/modules.html#intra-package-references
# absoulte path
from calculator.add_and_multiply import add_and_multiply
if __name__ == '__main__':
print(add_and_multiply(1,2))
########################################################
#실행결과
54. using relative and absolute paths in add_and_multiply.py
* Relative path
from .multiplication import multiply #상대경로로 지정
def add_and_multiply(a,b):
return multiply(a,b) + (a+b)
#################################################
#main.py 실행결과
5*Absolute path
from calculator.multiplication import multiply
def add_and_multiply(a,b):
return multiply(a,b) + (a+b)
###############################################
#main.py 실행결과
5If it is not a module to be used for the main purpose, it is possible to select and use either a relative path or an absolute path.
5. Role of init.py
- As stated earlier, it is a file that makes the directory containing the file recognized as a package.
- According to the official Python documentation, this file prevents directory names from obscuring module names that appear in the module search path.
- It's an empty file, but you can run the package's initialization code.
- By providing a list named all, you can respond to commands that import multiple packages at the same time.
For example, if you add all = ["add_and_multiply", "multiply"] to init.py in the calculator directory you have created, you can import add_and_multiply and multiply at the same time in the code of from calculator import .
Resources
https://smartkuma.tistory.com/entry/Python-sys-module-path-%EC%A0%95%EB%A6%AC
