Mercurial > dive4elements > river

annotate backend/contrib/shpimporter/uesg.py @ 9657:a79881a892c9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Importer (s/u-info) extensions: depth-evolution: corrected directory name Bezug_aktueller_GlW, detecting and logging of wrong units, then cancelling, various checks of the plausibility of the meta data year values and cancelling in case of errors, detecting and logging missing change values, skipping those lines
author mschaefer
date Mon, 23 Mar 2020 15:26:50 +0100
parents 249b8766a178
children
rev   line source
5004
f1c01fecf194 Fix imports of ogr, this removes a warning in gdal 1.9 but keeps
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
1 try:
f1c01fecf194 Fix imports of ogr, this removes a warning in gdal 1.9 but keeps
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
2 from osgeo import ogr
5077
c5187ab9f571 Fix Syntax Errror
Andre Heinecke <aheinecke@intevation.de>
parents: 5004
diff changeset
3 except ImportError:
5004
f1c01fecf194 Fix imports of ogr, this removes a warning in gdal 1.9 but keeps
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
4 import ogr
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
5
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
6 from importer import Importer
5294
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
7 import os.path
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
8 import utils
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
9
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
10 TABLE_NAME="floodmaps"
5276
170f514a4f29 SCHEMA CHANGE: lookup table for floodmaps and source-field added, Importer adapted to new directory names
Tom Gottfried <tom@intevation.de>
parents: 5077
diff changeset
11 PATH="Hydrologie/UeSG"
3654
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2861
diff changeset
12 NAME="UESG"
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
13
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
14
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
15 class UESG(Importer):
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
16
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
17 def getPath(self, base):
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
18 return "%s/%s" % (base, PATH)
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
19
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
20
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
21 def getTablename(self):
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
22 return TABLE_NAME
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
23
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
24
3654
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2861
diff changeset
25 def getName(self):
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2861
diff changeset
26 return NAME
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2861
diff changeset
27
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2861
diff changeset
28
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
29 def isGeometryValid(self, geomType):
5294
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
30 return geomType in [ogr.wkbMultiPolygon,
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
31 ogr.wkbPolygon]
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
32
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
33 def getKind(self, path):
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
34 kind = 0
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
35 if path.find("Berechnung") > 0:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
36 kind = kind + 100
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
37
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
38 if path.find("Aktuell") > 0:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
39 kind = kind + 10
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
40 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
41 kind = kind + 20
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
42
5276
170f514a4f29 SCHEMA CHANGE: lookup table for floodmaps and source-field added, Importer adapted to new directory names
Tom Gottfried <tom@intevation.de>
parents: 5077
diff changeset
43 if path.find("Bundesl") > 0:
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
44 kind = kind + 2
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
45 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
46 kind = kind + 1
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
47 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
48 kind = kind + 200
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
49
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
50 return kind
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
51
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
52
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
53 def createNewFeature(self, featureDef, feat, **args):
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
54 kind = self.getKind(args['path'])
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
55 newFeat = ogr.Feature(featureDef)
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
56 newFeat.SetGeometry(feat.GetGeometryRef())
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
57
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
58 if self.IsFieldSet(feat, "river_id"):
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
59 riverId = feat.GetField(feat)
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
60 else:
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
61 riverId = self.river_id
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
62
2861
b0132e1b9719 Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2853
diff changeset
63 if self.IsFieldSet(feat, "diff"):
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
64 diff = feat.GetFieldAsDouble("diff")
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
65 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
66 diff = 0
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
67
2861
b0132e1b9719 Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2853
diff changeset
68 if self.IsFieldSet(feat, "count"):
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
69 count = feat.GetFieldAsInteger("count")
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
70 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
71 count = 0
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
72
2861
b0132e1b9719 Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2853
diff changeset
73 if self.IsFieldSet(feat, "area"):
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
74 area = feat.GetFieldAsDouble("area")
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
75 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
76 area = 0
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
77
2861
b0132e1b9719 Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2853
diff changeset
78 if self.IsFieldSet(feat, "perimeter"):
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
79 perimeter = feat.GetFieldAsDouble("perimeter")
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
80 else:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
81 perimeter = 0
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
82
5854
249b8766a178 Schema change: new column 'waterbody' for floodmaps and importer
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5838
diff changeset
83 if self.IsFieldSet(feat, "GEWAESSER"):
249b8766a178 Schema change: new column 'waterbody' for floodmaps and importer
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5838
diff changeset
84 newFeat.SetField("waterbody", feat.GetField("GEWAESSER"))
249b8766a178 Schema change: new column 'waterbody' for floodmaps and importer
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5838
diff changeset
85
5294
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
86 if kind >= 200:
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
87 newFeat.SetField("source",
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
88 os.path.basename(os.path.dirname(args['path'])))
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
89
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
90
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
91 groupId = 2
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
92
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
93 newFeat.SetField("river_id", riverId)
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
94 newFeat.SetField("diff", diff)
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
95 newFeat.SetField("count", count)
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
96 newFeat.SetField("area", area)
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
97 newFeat.SetField("perimeter", perimeter)
2853
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
98 newFeat.SetField("kind", kind)
bd9e76e0b55d Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
99 newFeat.SetField("name", args['name'])
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
100
5294
3f35d9db48c1 Force UESG to multipolygon and import source field for messungen
Andre Heinecke <aheinecke@intevation.de>
parents: 5276
diff changeset
101 return utils.convertToMultiPolygon(newFeat)
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
102
http://dive4elements.wald.intevation.org