Mercurial > dive4elements > river
annotate flys-backend/contrib/shpimporter/shpimporter.py @ 5051:2847371be7fe
ImportRiver: Store fixations also if only "waterlevels" were parsed.
author | Felix Wolfsteller <felix.wolfsteller@intevation.de> |
---|---|
date | Wed, 20 Feb 2013 12:15:26 +0100 |
parents | 63f82bdd45f8 |
children | 7323847fa7df |
rev | line source |
---|---|
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
1 try: |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
2 from osgeo import ogr |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
3 except ImportErrror: |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
4 import ogr |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
5 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
6 import utils, optparse |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
7 |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
8 from uesg import UESG |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
9 from axis import Axis |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
10 from km import KM |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
11 from lines import Line |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
12 from fixpoints import Fixpoint |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
13 from buildings import Building |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
14 from crosssectiontracks import CrosssectionTrack |
2861
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
15 from floodplains import Floodplain |
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
16 from boundaries import HydrBoundary, HydrBoundaryPoly |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
17 from hws import HWSLines, HWSPoints |
2862
998f1a7dcfde
New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2861
diff
changeset
|
18 from gauges import GaugeLocation |
998f1a7dcfde
New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2861
diff
changeset
|
19 from catchments import Catchment |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
20 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
21 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
22 VERBOSE_DEBUG=2 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
23 VERBOSE_INFO=1 |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
24 |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
25 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
26 def DEBUG(msg): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
27 config = getConfig() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
28 if config.verbose >= VERBOSE_DEBUG: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
29 print "DEBUG: %s" % msg |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
30 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
31 def INFO(msg): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
32 config = getConfig() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
33 if config.verbose >= VERBOSE_INFO: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
34 print "INFO: %s" % msg |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
35 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
36 def ERROR(msg): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
37 config = getConfig() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
38 print "ERROR: %s" % msg |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
39 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
40 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
41 def getImporters(config, dbconn): |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
42 return [ |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
43 Axis(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
44 KM(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
45 CrosssectionTrack(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
46 Line(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
47 Fixpoint(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
48 Building(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
49 Floodplain(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
50 HydrBoundary(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
51 HydrBoundaryPoly(config, dbconn), |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
52 HWSLines(config, dbconn), |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
53 HWSPoints(config, dbconn), |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
54 GaugeLocation(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
55 Catchment(config, dbconn), |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
56 UESG(config, dbconn) |
2861
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
57 ] |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
58 |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
59 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
60 def getConfig(): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
61 parser = optparse.OptionParser() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
62 parser.add_option("--directory", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
63 parser.add_option("--target_srs", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
64 parser.add_option("--host", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
65 parser.add_option("--user", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
66 parser.add_option("--password", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
67 parser.add_option("--river_id", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
68 parser.add_option("--verbose", type="int", default=1) |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
69 parser.add_option("--dry_run", type="int", default=0) |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
70 parser.add_option("--ogr_connection", type="string") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
71 parser.add_option("--skip_axis", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
72 parser.add_option("--skip_hydr_boundaries", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
73 parser.add_option("--skip_buildings", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
74 parser.add_option("--skip_crosssections", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
75 parser.add_option("--skip_lines", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
76 parser.add_option("--skip_fixpoints", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
77 parser.add_option("--skip_floodplains", type="int") |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
78 parser.add_option("--skip_hws_lines", type="int") |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
79 parser.add_option("--skip_hws_points", type="int") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
80 parser.add_option("--skip_gauge_locations", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
81 parser.add_option("--skip_catchments", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
82 parser.add_option("--skip_kms", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
83 parser.add_option("--skip_uesgs", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
84 (config, args) = parser.parse_args() |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
85 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
86 if config.directory == None: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
87 ERROR("No river directory specified!") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
88 raise Exception("Invalid config") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
89 if not config.ogr_connection: |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
90 if not config.host: |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
91 ERROR("No database host specified!") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
92 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
93 if not config.user: |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
94 ERROR("No databaser user specified!") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
95 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
96 if not config.password: |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
97 ERROR("No password specified!") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
98 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
99 if config.river_id == None: |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
100 ERROR("No river id specified!") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
101 raise Exception("Invalid config") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
102 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
103 return config |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
104 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
105 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
106 def skip_importer(config, importer): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
107 if config.skip_axis == 1 and isinstance(importer, Axis): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
108 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
109 elif config.skip_hydr_boundaries == 1 and isinstance(importer, HydrBoundary): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
110 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
111 elif config.skip_hydr_boundaries == 1 and isinstance(importer, HydrBoundaryPoly): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
112 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
113 elif config.skip_buildings == 1 and isinstance(importer, Building): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
114 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
115 elif config.skip_crosssections == 1 and isinstance(importer, CrosssectionTrack): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
116 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
117 elif config.skip_lines == 1 and isinstance(importer, Line): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
118 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
119 elif config.skip_fixpoints == 1 and isinstance(importer, Fixpoint): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
120 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
121 elif config.skip_floodplains == 1 and isinstance(importer, Floodplain): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
122 return True |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
123 elif config.skip_hws_points == 1 and isinstance(importer, HWSPoints): |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
124 return True |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
125 elif config.skip_hws_lines == 1 and isinstance(importer, HWSLines): |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
126 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
127 elif config.skip_gauge_locations == 1 and isinstance(importer, GaugeLocation): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
128 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
129 elif config.skip_catchments == 1 and isinstance(importer, Catchment): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
130 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
131 elif config.skip_kms == 1 and isinstance(importer, KM): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
132 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
133 elif config.skip_uesgs == 1 and isinstance(importer, UESG): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
134 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
135 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
136 return False |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
137 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
138 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
139 def main(): |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
140 config=None |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
141 try: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
142 config = getConfig() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
143 except: |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
144 return -1 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
145 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
146 if config == None: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
147 ERROR("Unable to read config from command line!") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
148 return |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
149 |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
150 if config.dry_run > 0: |
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
151 INFO("You enable 'dry_run'. No database transaction will take place!") |
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
152 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
153 if config.ogr_connection: |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
154 connstr = config.ogr_connection |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
155 else: |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
156 connstr = 'OCI:%s/%s@%s' % (config.user, config.password, config.host) |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
157 |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
158 dbconn = ogr.Open(connstr) |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
159 |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
160 if dbconn == None: |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
161 ERROR("Could not connect to database %s" % connstr) |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
162 return -1 |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
163 |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
164 importers = getImporters(config, dbconn) |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
165 types = {} |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
166 |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
167 for importer in importers: |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
168 if skip_importer(config, importer): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
169 INFO("Skip import of '%s'" % importer.getName()) |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
170 continue |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
171 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
172 INFO("Start import of '%s'" % importer.getName()) |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
173 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
174 shapes = utils.findShapefiles(importer.getPath(config.directory)) |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
175 DEBUG("Found %i Shapefiles" % len(shapes)) |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
176 |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
177 for shpTuple in shapes: |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
178 geomType = importer.walkOverShapes(shpTuple) |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
179 try: |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
180 if geomType is not None: |
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
181 num = types[geomType] |
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
182 types[geomType] = num+1 |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
183 except: |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
184 types[geomType] = 1 |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
185 |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
186 for key in types: |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
187 DEBUG("%i x geometry type %s" % (types[key], key)) |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
188 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
189 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
190 if __name__ == '__main__': |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
191 main() |