Mercurial > dive4elements > river
annotate flys-backend/contrib/shpimporter/shpimporter.py @ 5785:d38004f0c52f
Datacage: Always clone XML template for builders. If the first is not cloned they get interlinked somehow. Feels like a bug in Java's DOM implementation.
author | Sascha L. Teichmann <teichmann@intevation.de> |
---|---|
date | Mon, 22 Apr 2013 17:01:17 +0200 |
parents | ed149d5d7fb7 |
children |
rev | line source |
---|---|
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
1 try: |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
2 from osgeo import ogr |
4977
ee908e4946cd
corrected Typo
Tom Gottfried <tom.gottfried@intevation.de>
parents:
4975
diff
changeset
|
3 except ImportError: |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
4 import ogr |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
5 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
6 import utils, optparse |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
7 import sys |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
8 import os |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
9 import logging |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
10 |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
11 from uesg import UESG |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
12 from axis import Axis |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
13 from km import KM |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
14 from fixpoints import Fixpoint |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
15 from buildings import Building |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
16 from crosssectiontracks import CrosssectionTrack |
2861
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
17 from floodplains import Floodplain |
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
18 from boundaries import HydrBoundary, HydrBoundaryPoly |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
19 from hws import HWSLines, HWSPoints |
2862
998f1a7dcfde
New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2861
diff
changeset
|
20 from gauges import GaugeLocation |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
21 from jetties import Jetties |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
22 from dgm import insertRiverDgm |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
23 from floodmarks import Floodmark |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
24 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
25 logger = logging.getLogger("shpimporter") |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
26 |
5219
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
27 os.environ["NLS_LANG"] = ".AL32UTF8" |
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
28 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
29 def initialize_logging(level): |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
30 """Initializes the logging system""" |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
31 root = logging.getLogger() |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
32 root.setLevel(level) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
33 hdlr = logging.StreamHandler() |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
34 fmt = logging.Formatter("%(levelname)s %(name)s: %(message)s") |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
35 hdlr.setFormatter(fmt) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
36 root.addHandler(hdlr) |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
37 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
38 def getImporters(river_id, dbconn, dry_run): |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
39 return [ |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
40 Axis(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
41 KM(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
42 CrosssectionTrack(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
43 Fixpoint(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
44 Building(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
45 Floodplain(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
46 HydrBoundary(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
47 HydrBoundaryPoly(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
48 HWSLines(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
49 HWSPoints(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
50 GaugeLocation(river_id, dbconn, dry_run), |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
51 Jetties(river_id, dbconn, dry_run), |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
52 Floodmark(river_id, dbconn, dry_run), |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
53 UESG(river_id, dbconn, dry_run) |
2861
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
54 ] |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
55 |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
56 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
57 def getConfig(): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
58 parser = optparse.OptionParser() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
59 parser.add_option("--directory", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
60 parser.add_option("--target_srs", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
61 parser.add_option("--host", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
62 parser.add_option("--user", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
63 parser.add_option("--password", type="string") |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
64 parser.add_option("--river_name", type="string") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
65 parser.add_option("--verbose", type="int", default=1) |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
66 parser.add_option("--dry_run", type="int", default=0) |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
67 parser.add_option("--ogr_connection", type="string") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
68 parser.add_option("--skip_axis", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
69 parser.add_option("--skip_hydr_boundaries", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
70 parser.add_option("--skip_buildings", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
71 parser.add_option("--skip_crosssections", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
72 parser.add_option("--skip_fixpoints", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
73 parser.add_option("--skip_floodplains", type="int") |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
74 parser.add_option("--skip_hws_lines", type="int") |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
75 parser.add_option("--skip_hws_points", type="int") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
76 parser.add_option("--skip_gauge_locations", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
77 parser.add_option("--skip_kms", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
78 parser.add_option("--skip_uesgs", type="int") |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
79 parser.add_option("--skip_dgm", type="int") |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
80 parser.add_option("--skip_jetties", type="int") |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
81 parser.add_option("--skip_floodmarks", type="int") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
82 (config, args) = parser.parse_args() |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
83 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
84 if config.verbose > 1: |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
85 initialize_logging(logging.DEBUG) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
86 elif config.verbose == 1: |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
87 initialize_logging(logging.INFO) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
88 else: |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
89 initialize_logging(logging.WARN) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
90 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
91 if config.directory == None: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
92 logger.error("No river directory specified!") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
93 raise Exception("Invalid config") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
94 if not config.ogr_connection: |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
95 if not config.host: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
96 logger.error("No database host specified!") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
97 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
98 if not config.user: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
99 logger.error("No databaser user specified!") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
100 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
101 if not config.password: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
102 logger.error("No password specified!") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
103 raise Exception("Invalid config") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
104 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
105 return config |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
106 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
107 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
108 def skip_importer(config, importer): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
109 if config.skip_axis == 1 and isinstance(importer, Axis): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
110 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
111 elif config.skip_hydr_boundaries == 1 and isinstance(importer, HydrBoundary): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
112 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
113 elif config.skip_hydr_boundaries == 1 and isinstance(importer, HydrBoundaryPoly): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
114 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
115 elif config.skip_buildings == 1 and isinstance(importer, Building): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
116 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
117 elif config.skip_crosssections == 1 and isinstance(importer, CrosssectionTrack): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
118 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
119 elif config.skip_fixpoints == 1 and isinstance(importer, Fixpoint): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
120 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
121 elif config.skip_floodplains == 1 and isinstance(importer, Floodplain): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
122 return True |
4963
1469066cc7d9
Importer: Fix the order of the config check
Andre Heinecke <aheinecke@intevation.de>
parents:
4955
diff
changeset
|
123 elif config.skip_hws_lines == 1 and isinstance(importer, HWSLines): |
1469066cc7d9
Importer: Fix the order of the config check
Andre Heinecke <aheinecke@intevation.de>
parents:
4955
diff
changeset
|
124 return True |
5011
514948fdae54
Do not skip hws_lines when skip hws_points is true
Andre Heinecke <aheinecke@intevation.de>
parents:
5006
diff
changeset
|
125 elif config.skip_hws_points == 1 and isinstance(importer, HWSPoints) and \ |
514948fdae54
Do not skip hws_lines when skip hws_points is true
Andre Heinecke <aheinecke@intevation.de>
parents:
5006
diff
changeset
|
126 not isinstance(importer, HWSLines): |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
127 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
128 elif config.skip_gauge_locations == 1 and isinstance(importer, GaugeLocation): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
129 return True |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
130 elif config.skip_jetties == 1 and isinstance(importer, Jetties): |
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
131 return True |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
132 elif config.skip_kms == 1 and isinstance(importer, KM): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
133 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
134 elif config.skip_uesgs == 1 and isinstance(importer, UESG): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
135 return True |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
136 elif config.skip_floodmarks == 1 and isinstance(importer, Floodmark): |
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
137 return True |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
138 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
139 return False |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
140 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
141 def main(): |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
142 config=None |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
143 try: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
144 config = getConfig() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
145 except: |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
146 return -1 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
147 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
148 if config == None: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
149 logger.error("Unable to read config from command line!") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
150 return |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
151 |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
152 if config.dry_run > 0: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
153 logger.info("You enable 'dry_run'. No database transaction will take place!") |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
154 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
155 if config.ogr_connection: |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
156 connstr = config.ogr_connection |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
157 else: |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
158 connstr = 'OCI:%s/%s@%s' % (config.user, config.password, config.host) |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
159 |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
160 oracle = False # Marker if oracle is used. |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
161 if 'OCI:' in connstr: |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
162 oracle = True |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
163 try: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
164 import cx_Oracle as dbapi |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
165 raw_connstr=connstr.replace("OCI:", "") |
5077
c5187ab9f571
Fix Syntax Errror
Andre Heinecke <aheinecke@intevation.de>
parents:
5036
diff
changeset
|
166 except ImportError: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
167 logger.error("Module cx_Oracle not found in: %s\n" |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
168 "Neccessary to connect to a Oracle Database.\n" |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
169 "Please refer to the installation " |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
170 "documentation." % sys.path) |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
171 return -1 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
172 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
173 else: # Currently only support for oracle and postgres |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
174 try: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
175 import psycopg2 as dbapi |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
176 raw_connstr=connstr.replace("PG:", "") |
4977
ee908e4946cd
corrected Typo
Tom Gottfried <tom.gottfried@intevation.de>
parents:
4975
diff
changeset
|
177 except ImportError: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
178 logger.error("Module psycopg2 not found in: %s\n" |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
179 "Neccessary to connect to a Posgresql Database.\n" |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
180 "Please refer to the installation " |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
181 "documentation." % sys.path) |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
182 return -1 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
183 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
184 dbconn_raw = dbapi.connect(raw_connstr) |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
185 dbconn = ogr.Open(connstr) |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
186 |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
187 if dbconn == None: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
188 logger.error("Could not connect to database %s" % connstr) |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
189 return -1 |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
190 |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
191 types = {} |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
192 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
193 directories = [] |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
194 if not config.river_name: |
5005
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
195 for file in [os.path.join(config.directory, d) for d in \ |
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
196 os.listdir(config.directory)]: |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
197 if os.path.isdir(file): |
5005
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
198 directories.append(file) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
199 else: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
200 directories.append(config.directory) |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
201 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
202 for directory in directories: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
203 if not config.river_name: |
5005
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
204 river_name = utils.getUTF8Path( |
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
205 os.path.basename(os.path.normpath(directory))) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
206 else: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
207 river_name = config.river_name |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
208 river_id = utils.getRiverId(dbconn_raw, river_name, oracle) |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
209 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
210 if not river_id: |
5221
aa043641d9c8
Fix syntax error when river id is not found (pylint could have told me this)
Andre Heinecke <aheinecke@intevation.de>
parents:
5219
diff
changeset
|
211 logger.info(u"Could not find river in database. Skipping: %s" |
aa043641d9c8
Fix syntax error when river id is not found (pylint could have told me this)
Andre Heinecke <aheinecke@intevation.de>
parents:
5219
diff
changeset
|
212 % unicode(utils.getUTF8(river_name), "UTF-8")) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
213 continue |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
214 else: |
5219
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
215 logger.info(u"Importing River: %s" % unicode( |
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
216 utils.getUTF8(river_name), "UTF-8")) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
217 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
218 for importer in getImporters(river_id, dbconn, config.dry_run): |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
219 if skip_importer(config, importer): |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
220 logger.info("Skip import of '%s'" % importer.getName()) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
221 continue |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
222 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
223 logger.info("Start import of '%s'" % importer.getName()) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
224 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
225 shapes = utils.findShapefiles(importer.getPath(config.directory)) |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
226 logger.debug("Found %i Shapefiles" % len(shapes)) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
227 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
228 for shpTuple in shapes: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
229 geomType = importer.walkOverShapes(shpTuple) |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
230 try: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
231 if geomType is not None: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
232 num = types[geomType] |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
233 types[geomType] = num+1 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
234 except: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
235 types[geomType] = 1 |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
236 |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
237 for key in types: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
238 logger.debug("%i x geometry type %s" % (types[key], key)) |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
239 |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
240 if not config.skip_dgm: |
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
241 dgmfilename = os.path.join( |
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
242 config.directory, "..", "DGMs.csv") |
4989
f4d61ac84b76
If we bother to check if the file is readable we might also
Andre Heinecke <aheinecke@intevation.de>
parents:
4978
diff
changeset
|
243 if not os.access(dgmfilename, os.R_OK) or not \ |
f4d61ac84b76
If we bother to check if the file is readable we might also
Andre Heinecke <aheinecke@intevation.de>
parents:
4978
diff
changeset
|
244 os.path.isfile(dgmfilename): |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
245 logger.info("Could not find or access DGM file: %s \n" |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
246 "Skipping DGM import." % dgmfilename) |
4978
b5ac7588cf4d
Use os.access instead of try/except open
Andre Heinecke <aheinecke@intevation.de>
parents:
4977
diff
changeset
|
247 else: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
248 logger.info("Inserting DGM meta information in 'dem' table.") |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
249 insertRiverDgm(dbconn_raw, dgmfilename, river_name, |
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
250 config.dry_run, oracle) |
5012
a79f6a085ab6
Add message if DGM import is skipped
Andre Heinecke <aheinecke@intevation.de>
parents:
5011
diff
changeset
|
251 else: |
a79f6a085ab6
Add message if DGM import is skipped
Andre Heinecke <aheinecke@intevation.de>
parents:
5011
diff
changeset
|
252 logger.info("Skip import of DGM.") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
253 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
254 if __name__ == '__main__': |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
255 main() |