Mercurial > dive4elements > river

annotate backend/contrib/shpimporter/hws.py @ 7475:2e4d02207239

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Datacage recommendations caching: Use LFU strategy to rotate parameter recommendations out earlier. This should be a better config in multiuser user setups because the more frequently used terminal state recs will survive longer.
author Sascha L. Teichmann <teichmann@intevation.de>
date Wed, 30 Oct 2013 17:35:12 +0100
parents 5aa05a7a34b7
children aef987124822
rev   line source
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
1 # -*- coding: utf-8 -*-
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
2 import os
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
3
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
4 try:
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
5 from osgeo import ogr
5077
c5187ab9f571 Fix Syntax Errror
Andre Heinecke <aheinecke@intevation.de>
parents: 5006
diff changeset
6 except ImportError:
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
7 import ogr
2862
998f1a7dcfde New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
8
998f1a7dcfde New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
9 from importer import Importer
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
10 import utils
2862
998f1a7dcfde New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
11
5006
769593a84606 Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents: 5003
diff changeset
12 import logging
5215
c8e79cff622a Importer use different name for HWS log output
Andre Heinecke <aheinecke@intevation.de>
parents: 5178
diff changeset
13 logger = logging.getLogger("HWS")
5006
769593a84606 Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents: 5003
diff changeset
14
2862
998f1a7dcfde New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
15 PATH="Hydrologie/HW-Schutzanlagen"
3654
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2862
diff changeset
16 NAME="HWS"
2862
998f1a7dcfde New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
17
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
18 # Keep in sync with hws_kinds table:
5178
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
19 # strings need to be lowercase
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
20 HWS_KIND = {
5002
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
21 "durchlass" : 1,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
22 "damm" : 2,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
23 "deich" : 2,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
24 "hochufer" : 2,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
25 "graben" : 3,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
26 "rohr1" : 1,
5218
028ff568b196 added HWS_KINDS to be accepted by importer
Tom Gottfried <tom@intevation.de>
parents: 5215
diff changeset
27 "rohr 1" : 1,
028ff568b196 added HWS_KINDS to be accepted by importer
Tom Gottfried <tom@intevation.de>
parents: 5215
diff changeset
28 "rohr 2" : 1,
5143
d25eebfc5cee corrected HWS_KIND field-map
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5142
diff changeset
29 "hauptdeich" : 2,
d25eebfc5cee corrected HWS_KIND field-map
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5142
diff changeset
30 "sommerdeich" : 2
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
31 }
2862
998f1a7dcfde New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
32
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
33 # Keep in sync with fed_states table:
5178
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
34 # strings need to be lowercase
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
35 FED_STATES = {
5002
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
36 "bayern" : 1,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
37 "hessen" : 2,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
38 "niedersachsen" : 3,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
39 "nordrhein-westfalen" : 4,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
40 "nordrhein westfalen" : 4,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
41 "rheinland-pfalz" : 5,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
42 "rheinland pfalz" : 5,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
43 "saarland" : 6,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
44 "schleswig-holstein" : 7,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
45 "schleswig holstein" : 7,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
46 "brandenburg" : 8,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
47 "mecklenburg-vorpommern" : 9,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
48 "mecklenburg vorpommern" : 9,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
49 "thüringen" : 10,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
50 "baden-württemberg" : 11,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
51 "baden württemberg" : 11,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
52 "sachsen-anhalt" : 12,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
53 "sachsen anhalt" : 12,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
54 "sachsen" : 13,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
55 "berlin" : 14,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
56 "bremen" : 15,
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
57 "hamburg" : 16,
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
58 }
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
59
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
60 class HWSPoints(Importer):
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
61 fieldmap = {
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
62 "name$" : "name",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
63 "quelle$" : "source",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
64 "anmerkung$" : "description",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
65 "stand$" : "status_date",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
66 "verband$" : "agency",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
67 "Deich_{0,1}KM$" : "dike_km",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
68 "Bereich$" : "range",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
69 "H[oeö]{0,2}he_{0,1}SOLL$" : "z_target",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
70 "(WSP_){0,1}BfG_{0,1}100$" : "rated_level",
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
71 "H[oeö]{0,2}he_{0,1}IST$" : "z",
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
72 }
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
73
5178
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
74 printedforpath=[]
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
75
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
76 def getPath(self, base):
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
77 return "%s/%s" % (base, PATH)
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
78
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
79 def getTablename(self):
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
80 return "hws_points"
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
81
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
82 def getName(self):
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
83 return "HWS_POINTS"
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
84
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
85 def isGeometryValid(self, geomType):
4934
11b459a3eb5c Importer: Import hws_points based on geom type
Andre Heinecke <aheinecke@intevation.de>
parents: 4933
diff changeset
86 return geomType == ogr.wkbPoint or geomType == ogr.wkbPoint25D
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
87
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
88 def isShapeRelevant(self, name, path):
4934
11b459a3eb5c Importer: Import hws_points based on geom type
Andre Heinecke <aheinecke@intevation.de>
parents: 4933
diff changeset
89 shp = ogr.Open(path)
4955
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
90 return self.isGeometryValid(shp.GetLayerByName(name).GetGeomType())
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
91
5178
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
92 def getFedStateIDfromPath(self, path):
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
93 """
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
94 Tries to get extract a bundesland from the path
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
95 """
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
96 for state in sorted(FED_STATES.keys(), key = len, reverse = True):
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
97 if state in path.lower():
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
98 if not path in self.printedforpath:
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
99 logger.info("Extracted federal state from path: %s" % state)
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
100 self.printedforpath.append(path)
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
101 return FED_STATES[state]
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
102
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
103 def createNewFeature(self, featureDef, feat, **args):
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
104 newFeat = ogr.Feature(featureDef)
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
105 geometry = feat.GetGeometryRef()
4881
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
106 geometry.SetCoordinateDimension(2)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
107
4884
b457532dae63 Importer: Move copy fields into importer class and track imported/unimported fields
Andre Heinecke <aheinecke@intevation.de>
parents: 4881
diff changeset
108 self.copyFields(feat, newFeat, self.fieldmap)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
109
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
110 newFeat.SetGeometry(geometry)
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
111
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
112 artname = self.searchField("art$")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
113 if self.IsFieldSet(feat, artname):
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
114 self.handled(artname)
5002
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
115 kind_id = HWS_KIND.get(feat.GetField(artname).lower())
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
116 if not kind_id:
5006
769593a84606 Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents: 5003
diff changeset
117 logger.warn("Unknown Art: %s" % \
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
118 feat.GetField(artname))
5382
f66a6c1a73e7 Explicitly force the kind to be 2 when it is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5218
diff changeset
119 newFeat.SetField("kind_id", 2)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
120 else:
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
121 newFeat.SetField("kind_id", kind_id)
5382
f66a6c1a73e7 Explicitly force the kind to be 2 when it is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5218
diff changeset
122 else:
f66a6c1a73e7 Explicitly force the kind to be 2 when it is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5218
diff changeset
123 newFeat.SetField("kind_id", 2)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
124
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
125 fname = self.searchField("Bundesland$")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
126 if self.IsFieldSet(feat, fname):
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
127 self.handled(fname)
5002
baa32f1df47d Importer HWS: case insensitive matching for HWS_KIND and FED_STATES
Andre Heinecke <aheinecke@intevation.de>
parents: 4955
diff changeset
128 fed_id = FED_STATES.get(feat.GetField(fname).lower())
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
129
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
130 if not fed_id:
5006
769593a84606 Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents: 5003
diff changeset
131 logger.warn("Unknown Bundesland: %s" % \
5178
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
132 feat.GetField(fname))
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
133 else:
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
134 newFeat.SetField("fed_state_id", fed_id)
5178
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
135 else:
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
136 # Try to get the bundesland from path
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
137 fed_id = self.getFedStateIDfromPath(args['path'])
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
138 if fed_id:
13f3484d4618 Extract Bundesland from path as a fallback if bundesland field is not set.
Andre Heinecke <aheinecke@intevation.de>
parents: 5143
diff changeset
139 newFeat.SetField("fed_state_id", fed_id)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
140
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
141 fname = self.searchField("(ufer$)|(flussseite$)")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
142 if self.IsFieldSet(feat, fname):
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
143 self.handled(fname)
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
144 shoreString = feat.GetField(fname)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
145 if "links" in shoreString.lower():
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
146 newFeat.SetField("shore_side", True)
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
147 elif "rechts" in shoreString.lower():
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
148 newFeat.SetField("shore_side", False)
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
149
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
150
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
151 fname = self.searchField("river_{0,1}id$")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
152 if self.IsFieldSet(feat, fname):
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
153 self.handled(fname)
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
154 if feat.GetField(fname) != self.river_id:
5006
769593a84606 Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents: 5003
diff changeset
155 logger.warn("River_id mismatch between shapefile and"
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
156 " importer parameter.")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
157 newFeat.SetField("river_id", feat.GetField(fname))
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
158 else:
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
159 newFeat.SetField("river_id", self.river_id)
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
160
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
161 fname = self.searchField("name$")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
162 if not self.IsFieldSet(feat, fname):
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
163 newFeat.SetField("name", args['name'])
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
164
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
165 fname = self.searchField("offiziell$")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
166 if self.IsFieldSet(feat, fname):
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
167 self.handled(fname)
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
168 offiziell = feat.GetField(fname)
4881
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
169 if offiziell == "1" or offiziell == 1:
5003
563ff8893d47 Importer: Fix fieldname of "official" in the new feature and set it
Andre Heinecke <aheinecke@intevation.de>
parents: 5002
diff changeset
170 newFeat.SetField("official", True)
563ff8893d47 Importer: Fix fieldname of "official" in the new feature and set it
Andre Heinecke <aheinecke@intevation.de>
parents: 5002
diff changeset
171 # Set the official value based on the file name as a fallback
563ff8893d47 Importer: Fix fieldname of "official" in the new feature and set it
Andre Heinecke <aheinecke@intevation.de>
parents: 5002
diff changeset
172 elif args.get("name", "").lower() == "rohre_und_sperren" or \
563ff8893d47 Importer: Fix fieldname of "official" in the new feature and set it
Andre Heinecke <aheinecke@intevation.de>
parents: 5002
diff changeset
173 args.get("name", "").lower() == "rohre-und-sperren":
563ff8893d47 Importer: Fix fieldname of "official" in the new feature and set it
Andre Heinecke <aheinecke@intevation.de>
parents: 5002
diff changeset
174 newFeat.SetField("official", True)
5392
95050a944d81 Importer default official = False for hws
Andre Heinecke <aheinecke@intevation.de>
parents: 5382
diff changeset
175 else:
95050a944d81 Importer default official = False for hws
Andre Heinecke <aheinecke@intevation.de>
parents: 5382
diff changeset
176 newFeat.SetField("official", False)
4881
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
177
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
178 if self.IsFieldSet(newFeat, "z") and \
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
179 self.IsFieldSet(newFeat, "rated_level"):
4935
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
180 fname = self.searchField("freibord(_m){0,1}$")
c0a58558b817 Importer: - Handle regular expressions for attribute names
Andre Heinecke <aheinecke@intevation.de>
parents: 4934
diff changeset
181 self.handled(fname)
4881
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
182 z = newFeat.GetFieldAsDouble("z")
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
183 rl = newFeat.GetFieldAsDouble("rated_level")
9b03cf6b85c9 (importer) calculate freeboard and expect sql
Andre Heinecke <aheinecke@intevation.de>
parents: 4875
diff changeset
184 newFeat.SetField("freeboard", z - rl)
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
185
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
186 return newFeat
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
187
4955
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
188 class HWSLines(HWSPoints):
4875
37fa93f65971 (importer) Add classes for hws_points and hws_lines
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
189
4955
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
190 # TODO: GEOM_target, GEOM_rated_level, dike_km_from, dike_km_to
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
191 fieldmap = {
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
192 "name$" : "name",
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
193 "quelle$" : "source",
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
194 "anmerkung$" : "description",
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
195 "stand$" : "status_date",
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
196 "verband$" : "agency",
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
197 "Bereich$" : "range",
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
198 }
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
199
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
200 def getPath(self, base):
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
201 return "%s/%s" % (base, PATH)
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
202
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
203 def getTablename(self):
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
204 return "hws_lines"
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
205
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
206 def getName(self):
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
207 return "HWS_LINES"
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
208
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
209 def isGeometryValid(self, geomType):
5137
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
210 return geomType in [ogr.wkbLineString,
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
211 ogr.wkbLineString25D,
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
212 ogr.wkbMultiLineString25D,
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
213 ogr.wkbMultiLineString]
4955
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
214
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
215 def isShapeRelevant(self, name, path):
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
216 shp = ogr.Open(path)
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
217 return self.isGeometryValid(shp.GetLayerByName(name).GetGeomType())
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
218
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
219 def createNewFeature(self, featureDef, feat, **args):
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
220 newFeat = HWSPoints.createNewFeature(self, featureDef, feat, **args)
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
221 geometry = feat.GetGeometryRef()
5137
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
222 if geometry.GetCoordinateDimension() == 2:
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
223 geometry.SetCoordinateDimension(3)
5139
76db0e48dbc0 Importer: When a hws linestring was two dimensional import it with z=9999
Andre Heinecke <aheinecke@intevation.de>
parents: 5137
diff changeset
224 for i in range(0, geometry.GetPointCount()):
76db0e48dbc0 Importer: When a hws linestring was two dimensional import it with z=9999
Andre Heinecke <aheinecke@intevation.de>
parents: 5137
diff changeset
225 x,y,z = geometry.GetPoint(i)
76db0e48dbc0 Importer: When a hws linestring was two dimensional import it with z=9999
Andre Heinecke <aheinecke@intevation.de>
parents: 5137
diff changeset
226 z = 9999
76db0e48dbc0 Importer: When a hws linestring was two dimensional import it with z=9999
Andre Heinecke <aheinecke@intevation.de>
parents: 5137
diff changeset
227 geometry.SetPoint(i, x, y, z)
4955
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
228 newFeat.SetGeometry(geometry)
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
229
5137
04eb62eae722 River Axes and HWS_lines are now multiline
Andre Heinecke <aheinecke@intevation.de>
parents: 5122
diff changeset
230 return utils.convertToMultiLine(newFeat)
4955
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
231
7323847fa7df Importer: Add HWS lines stub. Needs some example data to finish
Andre Heinecke <aheinecke@intevation.de>
parents: 4935
diff changeset
232
http://dive4elements.wald.intevation.org