Modules, Packages, How Import Works

Modules & Packages

Creating modules

Simply create a file and implement functions or classes that you wish to reuse

## my_module.py
my_module_var = 7

def my_module_func():
    return "Hello!"

class MyModuleClass:
    pass

Importing the module

## main.py
import my_module

print(my_module.my_module_var)

my_module.my_module_func()

my_module_class = my_module.MyModuleClass()

It is critical to write the name of the module in front of the imported function, or the interpreter won't know which file to look up to process the function and throw an error --> "Name Space" structure

Alternative ways to import modules

from my_module import my_module_func, my_module_var

print(my_module_var)
my_module_func()

If the user is clear about the function to be used from the module, using "from...import" syntax allows omitting writing the name of the module when calling functions.

from my_module import *

print(my_module_var)
my_module_func()

Don't overuse this asterisk feature (imports all compoentns of the module), however, since there may be conflicts with funtion and variable names.

Import As
To avoid conflicts between imported functions or to shorten the name of certain modules, "import as" keyword is used.

from my_module  import my_func as f1
from my_module2 import my_func as f2
from my_module3 import function_with_name_too_long as f3

f1()
f2()
f3()

Or,

import my_module as m1

m1.my_module_func()

Packages

A package is a directory of several python files.

import pkg.mod1
from pkg.mod2 import func2

pkg.mod1.func2()
func2()

Package Initialization
There may be times when the user needs to provide initialization when importing packages. If __init__.py is included in the package, the codes in this file are automatically processed when the package is imported. This file enables to do the following:

(1) Shortening the length of import path

import pkg.mod1

pkg.mod1.func2()

can be reduced to

# __init__.py
from .mod1 import func2

# main.py
from pkg import func2

func2()

(2) Restrict certain variables, functions, and classes from being imported from the package

All elements that can be imported from the package are defined by the __all variable. By redefining the __all in __init__, the user gets to exert restriction.

# __init__.py
from .mod1 import func2
from .mod2 import func3

__all__ = ['func2', 'func3']

# main.py
from pkg import *

func2()
func3()
func4() ## <== Error. func4 not defined in __all__ 

Using Packages made by other people
The package needs to be installed using PIP (package manager).
For example,

pip install Django 

How Import Statement Finds Modules and Packages

Import search sequence:

1. sys.modules
sys.modules is a simple dictionary that contains already ipmported modules and packages.

>>> import sys
>>> print(sys.modules)
{'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': <module '_frozen_importlib' (frozen)>, '_imp': <module '_imp' (built-in)>, '_warnings': <module '_warnings' (built-in)>, '_frozen_importlib_external': <module '_frozen_importlib_external' (frozen)>, '_io': <module 'io' (built-in)>, 'marshal': <module 'marshal' (built-in)>, 'nt': <module 'nt' (built-in)>, '_thread': <module '_thread' (built-in)>, '_weakref': <module '_weakref' (built-in)>, 'winreg': <module 'winreg' (built-in)>, 'time': <module 'time' (built-in)>, 'zipimport': <module 'zipimport' (frozen)>, '_codecs': <module '_codecs' (built-in)>, 'codecs': <module 'codecs' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\codecs.py'>, 'encodings.aliases': <module 'encodings.aliases' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\encodings\\aliases.py'>, 'encodings': <module 'encodings' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\encodings\\__init__.py'>, 'encodings.utf_8': <module 'encodings.utf_8' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\encodings\\utf_8.py'>, '_signal': <module '_signal' (built-in)>, '__main__': <module '__main__' from 'c:\\Users\\junepyo\\Desktop\\Python_JumpToPython\\parameter.py'>, 'encodings.latin_1': <module 'encodings.latin_1' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\encodings\\latin_1.py'>, '_abc': <module '_abc' (built-in)>, 'abc': <module 'abc' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\abc.py'>, 'io': <module 'io' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\io.py'>, '_stat': <module '_stat' (built-in)>, 'stat': <module 'stat' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\stat.py'>, '_collections_abc': <module '_collections_abc' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\_collections_abc.py'>, 'genericpath': <module 'genericpath' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\genericpath.py'>, 'ntpath': <module 'ntpath' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\ntpath.py'>, 'os.path': <module 'ntpath' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\ntpath.py'>, 'os': <module 'os' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\os.py'>, '_sitebuiltins': <module '_sitebuiltins' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\_sitebuiltins.py'>, 'site': <module 'site' from 'C:\\Users\\junepyo\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\site.py'>}

2. built-in modules
Official libraries provided by Python that are installed along with Python

3. sys.path
sys.path is a list containing string elements. Each string element represents a path like below:

['',
 '/Users/song-eun-u/anaconda3/bin',
 '/Users/song-eun-u/anaconda3/lib/python36.zip',
 '/Users/song-eun-u/anaconda3/lib/python3.6',
 '/Users/song-eun-u/anaconda3/lib/python3.6/lib-dynload',
 '/Users/song-eun-u/anaconda3/lib/python3.6/site-packages',
 '/Users/song-eun-u/anaconda3/lib/python3.6/site-packages/aeosa',
 '/Users/song-eun-u/anaconda3/lib/python3.6/site-packages/IPython/extensions',
 '/Users/song-eun-u/.ipython']

Python traverses the list, looking if each path contains the package. Because sys itself is a module included in Python, the user can import sys module.

import sys

print(sys.path)
print(sys.modules)

If no package is found even in sys.path, ModuleNotFounderror is thrown.

Then how does Python find sys module?
Even though sys is a module that needs to be imported, Python can easily find the location of the sys module. This is because sys is a built-in module.

When sys.modules is printed, module 'sys' is indicated as a "built-in."

'sys': <module 'sys' (built-in)>

Difference between sys.modules and sys.path
As described above, sys.modules returns a dictionary mapping the names of modules to modules which have already been loaded (imported). On the other hand, sys.path is a list containing string elements, each of which contains a path to a locally created or an external module or package.

Absolute Path & Relative Path

Because built-in modules are built inside Python and installed packages are automatically saved in site-packages directory, which is included in sys.path, there is no problem in locating file paths.

Local packages developed by the user, however, need to have their paths well defined and declared.

Example project

└── my_app
    ├── main.py
    ├── package1
    │   ├── module1.py
    │   └── module2.py
    └── package2
        ├── __init__.py
        ├── module3.py
        ├── module4.py
        └── subpackage1
            └── module5.py

Absolute path:

from package1 import module1
from package1.module2 import function1
from package2 import class1
from package2.subpackage1.module5 import function2

The directory of current project is stored by default in sys.path. Absolute path therefore begins writing out path from current directory. Similarly, main.py is already in my_app Project, "my_app" is ommitted from the path.

Relative path:
Relative path is different from absolute path in that the path is declared in relation to the current file location that is importing, and not from a parent directory that is located at the top of the project. A dot represents abbreviated form of current position in current directory.

# package2/module3.py

from . import class1
from .subpackage1.module5 import function2

".." indicates moving up one level to an upper directory.

# subpackage1/module5.py
from ..module4 import class4

In general, stick to using absolute paths. It can be noted from the example below that using relative path to call imported functioons results in the following error.

ImportError: attempted relative import with no known parent package

On the other hand, absolute path works fine.