ShapeFile DB Indexing (with GeoDjango)

임우열 (WooYeol Lim)·2022년 11월 17일

GIS

목록 보기

1/2

ShapeFile (Vector Data)은 흔히 볼 수 있는 GIS Data 형태로 점, 선, 다각형으로 이루어져 있으며 확대 및 축소를 진행하여도 선명한 데이터를 확인 할 수 있습니다. Raster Data는 픽셀 다위의 데이터를 저장하는 행렬의 데이터 구조를 가지며 픽셀에 담기는 데이터에 따라 선명도가 나뉩니다.

ShapeFile 형태의 GIS Data를 DB에 저장하고 관리하고 싶다면 Postgresql의 확장인 PostGIS를 추천합니다.

인덱싱을 진행할 데이터는 전 세계 대륙 데이터와 해안선 데이터 두 가지입니다.

링크를 통해 데이터를 다운로드 받을 수 있습니다.

Django는 GeoDjango라는 컨셉을 가지고 있는데 GIS 데이터를 다룰 수 있는 ORM과 함수 등을 가지고 있습니다.

다음의 GeoDjango를 통해 Shapefile을 DB에 인덱싱 해보도록 하겠습니다.

GeoDjango Introduce

GeoDjango intends to be a world-class geographic web framework. Its goal is to make it as easy as possible to build GIS web applications and harness the power of spatially enabled data.

OGC geometries 와 raster data를 위한 Django Model 필드
Spatial Data에 대한 Django ORM 제공
GIS geometry와 raster 작업 및 다양한 형식의 데이터 조작을 위한 Loose-coupled, high-level 파이썬 인터페이스 조작
Admin Page에서 Geometry 필드 편집

Install

Python & Django

Django Version	Python Versions
2.2	3.5, 3.6, 3.7, 3.8 (added in 2.2.8), 3.9 (added in 2.2.17)
3.0	3.6, 3.7, 3.8, 3.9 (added in 3.0.11)
3.1	3.6, 3.7, 3.8, 3.9 (added in 3.1.3)
3.2	3.6, 3.7, 3.8, 3.9, 3.10 (added in 3.2.9)
4.0	3.8, 3.9, 3.10

Spatial Database

PostGIS is recommended, because it is the most mature and feature-rich open source spatial database.

Database	Library Requirements	Supported Versions	Notes
PostgreSQL	GEOS,GDAL,PROJ,PostGIS	10+	Requires PostGIS
MySQL	GEOS,GDAL	5.7+	Limited Functionally
Oracle	GEOS,GDAL	19+	XE not supported
SQLite	GEOS,GDAL,PROJ,SpatialLite	3.9.0+	Requires SpatialLite 4.3+

$ brew install postgresql
$ brew install postgis
$ brew install gdal
$ brew install libgeoip

Setting Up

Create a New Project

$ django-admin startproject geodjango

$ cd geodjango
$ python manage.py startapp world

Configure settings.py

Edit the database connection settings to math your setup

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': 'geodjango',
        'USER': 'geo',
    },
}

Modify the [INSTALLED_APPS](https://docs.djangoproject.com/en/4.0/ref/settings/#std-setting-INSTALLED_APPS)setting to include[django.contrib.admin](https://docs.djangoproject.com/en/4.0/ref/contrib/admin/#module-django.contrib.admin), [django.contrib.gis](https://docs.djangoproject.com/en/4.0/ref/contrib/gis/#module-django.contrib.gis), and {Own App}

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.gis',
    '{Own App}',
]

Model API

Spatial Field Types

GeometryField
PointField
```
pnt = Point(5, 23)
pnt = Point([5, 23])
```

LineStringField

ls = LineString((0, 0), (1, 1))
ls = LineString(Point(0, 0), Point(1, 1))
ls = LineString( ((0, 0), (1, 1)) )
ls = LineString( [Point(0, 0), Point(1, 1)] )

PolygonField

ext_coords = ((0, 0), (0, 1), (1, 1), (1, 0), (0, 0))
int_coords = ((0.4, 0.4), (0.4, 0.6), (0.6, 0.6), (0.6, 0.4), (0.4, 0.4))
poly = Polygon(ext_coords, int_coords)
poly = Polygon(LinearRing(ext_coords), LinearRing(int_coords))

MultiPointField

mp = MultiPoint(Point(0, 0), Point(1, 1))
mp = MultiPoint( (Point(0, 0), Point(1, 1)) )

MultiLineStringField

ls1 = LineString((0, 0), (1, 1))
ls2 = LineString((2, 2), (3, 3))
mls = MultiLineString(ls1, ls2)
mls = MultiLineString([ls1, ls2])

MultiPolygonField

p1 = Polygon( ((0, 0), (0, 1), (1, 1), (0, 0)) )
p2 = Polygon( ((1, 1), (1, 2), (2, 2), (1, 1)) )
mp = MultiPolygon(p1, p2)
mp = MultiPolygon([p1, p2])

GeometryCollectionField

poly = Polygon( ((0, 0), (0, 1), (1, 1), (0, 0)) )
gc = GeometryCollection(Point(0, 0), MultiPoint(Point(0, 0), Point(1, 1)), poly)
gc = GeometryCollection((Point(0, 0), MultiPoint(Point(0, 0), Point(1, 1)), poly))

RasterField

Spatial Field Options
- srid
  - Sets the SRID(Spatial Reference System Identity) of the geometry field to the given value.
  - Default : 4326 (= WGS84)
- spatial_index
  - Defalut : True
Geometry Field Options
- dim (차원)
  - Dimension
  - This option may be used for customizing the coordinate dimension of the geometry field.
  - Default : 2
- geography (지리)
  - If set to True, this option will create a database column of type geography, rather than geometry.(Geography support is limited to PostGIS and will force the SRID to be 4326.)

GDAL

GeoDjango provides a high-level Python interface for some of the capabilities of OGR, including the reading and coordinate transformation of vector spatial data and minimal support for GDAL’s features with respect to raster (image) data.

GeoDjango는 벡터 공간 데이터의 읽기 및 좌표 변환(OGR)과 Raster(이미지) 데이터와 관련해서 (GDAL) 최소한의 기능을 지원

기능 종류

데이터 생성

GDALRaster

>>> rst = GDALRaster({  # Creates an in-memory raster
...     'srid': 4326,
...     'width': 4,
...     'height': 4,
...     'datatype': 1,
...     'bands': [{
...         'data': (2, 3),
...         'offset': (1, 1),
...         'size': (2, 2),
...         'shape': (2, 1),
...         'nodata_value': 5,
...     }]
... })

데이터 변환

GDALRaster.transform - Rater를 다른 srs(spatial reference system)으로 변환

>>> rst = GDALRaster({
...     "width": 6, "height": 6, "srid": 3086,
...     "origin": [500000, 400000],
...     "scale": [100, -100],
...     "bands": [{"data": range(36), "nodata_value": 99}]
... })
>>> target_srs = SpatialReference(4326)
>>> target = rst.transform(target_srs)
>>> target.origin
[-82.98492744885776, 27.601924753080144]

GDALRaster.geotransform - geotransform

>>> rst = GDALRaster({'width': 10, 'height': 20, 'srid': 4326})
>>> rst.geotransform
[0.0, 1.0, 0.0, 0.0, 0.0, -1.0]

CoordTransform - 좌표계 변환

>>> ct = CoordTransform(SpatialReference('WGS84'), SpatialReference('NAD83'))
>>> for feat in layer:
...     geom = feat.geom # getting clone of feature geometry
...     geom.transform(ct) # transforming

데이터 정보 조회

GDALRaster

info - 기존 GDAL의 gdalinfo와 같은 기능

metadata

>>> rst = GDALRaster({'width': 10, 'height': 20, 'srid': 4326})
>>> rst.metadata
{}
>>> rst.metadata = {'DEFAULT': {'OWNER': 'Django', 'VERSION': '1.0'}}
>>> rst.metadata
{'DEFAULT': {'OWNER': 'Django', 'VERSION': '1.0'}}
>>> rst.metadata = {'DEFAULT': {'OWNER': None, 'VERSION': '2.0'}}
>>> rst.metadata
{'DEFAULT': {'VERSION': '2.0'}}

name
width
height
scale
skew
extend
bands
driver
srs
srid
origin
vsi_buffer
is_vsi_based

데이터 처리

Importing Spatial Data

Ogrinspect

$ python manage.py ogrinspect [options] <data_source> <model_name> [options]

$ python manage.py ogrinspect world_boundary/data/World_Continents/World_Continents.shp WorldContinents --srid=4326 --mapping --multi

The -srid=4326 option sets the SRID for the geographic field.
The -mapping option tells ogrinspect to also generate a mapping dictionary for use with [LayerMapping](https://docs.djangoproject.com/en/4.1/ref/contrib/gis/layermapping/#django.contrib.gis.utils.LayerMapping).
The -multi option is specified so that the geographic field is a[MultiPolygonField](https://docs.djangoproject.com/en/4.1/ref/contrib/gis/model-api/#django.contrib.gis.db.models.MultiPolygonField) instead of just a [PolygonField](https://docs.djangoproject.com/en/4.1/ref/contrib/gis/model-api/#django.contrib.gis.db.models.PolygonField).

# This is an auto-generated Django model module created by ogrinspect.
from django.contrib.gis.db import models

class WorldContinents(models.Model):
    fid = models.IntegerField()
    continent = models.CharField(max_length=13)
    sqmi = models.FloatField()
    sqkm = models.FloatField()
    shape__are = models.FloatField()
    shape__len = models.FloatField()
    geom = models.MultiPolygonField(srid=4326)

# Auto-generated `LayerMapping` dictionary for WorldContinents model
worldcontinents_mapping = {
    'fid': 'FID',
    'continent': 'CONTINENT',
    'sqmi': 'SQMI',
    'sqkm': 'SQKM',
    'shape__are': 'Shape__Are',
    'shape__len': 'Shape__Len',
    'geom': 'MULTIPOLYGON',
}

Layer Mapping

from pathlib import Path
from django.contrib.gis.utils import LayerMapping
from .models import WorldBorder, WorldContinents

world_mapping = {
    'fips' : 'FIPS',
    'iso2' : 'ISO2',
    'iso3' : 'ISO3',
    'un' : 'UN',
    'name' : 'NAME',
    'area' : 'AREA',
    'pop2005' : 'POP2005',
    'region' : 'REGION',
    'subregion' : 'SUBREGION',
    'lon' : 'LON',
    'lat' : 'LAT',
    'mpoly' : 'MULTIPOLYGON',
}

world_shp = Path(__file__).resolve().parent / 'data' / 'TM_WORLD_BORDERS-0.3.shp'

def run(verbose=True):
    lm = LayerMapping(WorldBorder, world_shp, world_mapping, transform=False)
    lm.save(strict=True, verbose=verbose)

worldcontinents_mapping = {
    'fid': 'FID',
    'continent': 'CONTINENT',
    'sqmi': 'SQMI',
    'sqkm': 'SQKM',
    'shape_are': 'Shape__Are',
    'shape_len': 'Shape__Len',
    'geom': 'MULTIPOLYGON',
}

worldcontinents_shp = Path(__file__).resolve().parent / 'data' / 'World_Continents' /'World_Continents.shp'

def run_continents(verbose=True):
    lm = LayerMapping(WorldContinents, worldcontinents_shp, worldcontinents_mapping, transform=False)
    lm.save(strict=True, verbose=verbose)

python manage.py shell

>>> from world_boundary import load
>>> load.run()
>>> load.run_continents()

Geographic Admin

ModelAdmin

from django.contrib.gis import admin
from .models import {My Model}

admin.site.register({My Model}, admin.ModelAdmin)

S1B_EW_RAW__0SDH_20200301T011115_20200301T011135_020491_026D3F_5878.SAFE

S1B_EW_RAW__0SDH_20200301T011115_20200301T011135_020491_026D3F_5878.SAFE

GISModelAdmin

from django.contrib.gis import admin
from .models import {My Model}

admin.site.register({My Model}, admin.GISModelAdmin)

S1B_EW_RAW__0SDH_20200301T011115_20200301T011135_020491_026D3F_5878.SAFE

S1B_EW_RAW__0SDH_20200301T011115_20200301T011135_020491_026D3F_5878.SAFE

GeoModelAdmin

from django.contrib.gis import admin
from .models import {My Model}

admin.site.register({My Model}, admin.GeoModelAdmin)

GeoDjango를 사용하면 Django Model 생성부터 인덱싱 작업까지 손 쉽게 진행 할 수 있음을 알 수 있었습니다.

Reference

GeoDjango | Django documentation | Django

임우열 (WooYeol Lim)

Hungry Engineer

다음 포스트