Mercurial > dive4elements > river
annotate backend/contrib/shpimporter/shpimporter.py @ 6278:cd6947eaa5ed
Revert halfhearted logfile move. (r6127)
Either we change this for river mapfile and for user mapfile
or we don't change it.
Also if you change paths in the configuration make sure that the
make release script is modified accordingly!
author | Andre Heinecke <aheinecke@intevation.de> |
---|---|
date | Tue, 11 Jun 2013 17:19:59 +0200 |
parents | 5aa05a7a34b7 |
children | e68e414dceb2 |
rev | line source |
---|---|
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
1 try: |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
2 from osgeo import ogr |
4977
ee908e4946cd
corrected Typo
Tom Gottfried <tom.gottfried@intevation.de>
parents:
4975
diff
changeset
|
3 except ImportError: |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
4 import ogr |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
5 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
6 import utils, optparse |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
7 import sys |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
8 import os |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
9 import logging |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
10 |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
11 from uesg import UESG |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
12 from axis import Axis |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
13 from km import KM |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
14 from fixpoints import Fixpoint |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
15 from buildings import Building |
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
16 from crosssectiontracks import CrosssectionTrack |
2861
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
17 from floodplains import Floodplain |
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
18 from boundaries import HydrBoundary, HydrBoundaryPoly |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
19 from hws import HWSLines, HWSPoints |
2862
998f1a7dcfde
New shape importers and schema modifications.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2861
diff
changeset
|
20 from gauges import GaugeLocation |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
21 from jetties import Jetties |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
22 from dgm import insertRiverDgm |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
23 from floodmarks import Floodmark |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
24 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
25 logger = logging.getLogger("shpimporter") |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
26 |
5219
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
27 os.environ["NLS_LANG"] = ".AL32UTF8" |
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
28 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
29 def initialize_logging(level): |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
30 """Initializes the logging system""" |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
31 root = logging.getLogger() |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
32 root.setLevel(level) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
33 hdlr = logging.StreamHandler() |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
34 fmt = logging.Formatter("%(levelname)s %(name)s: %(message)s") |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
35 hdlr.setFormatter(fmt) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
36 root.addHandler(hdlr) |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
37 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
38 def getImporters(river_id, dbconn, dry_run): |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
39 return [ |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
40 Axis(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
41 KM(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
42 CrosssectionTrack(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
43 Fixpoint(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
44 Building(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
45 Floodplain(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
46 HydrBoundary(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
47 HydrBoundaryPoly(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
48 HWSLines(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
49 HWSPoints(river_id, dbconn, dry_run), |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
50 GaugeLocation(river_id, dbconn, dry_run), |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
51 Jetties(river_id, dbconn, dry_run), |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
52 Floodmark(river_id, dbconn, dry_run), |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
53 UESG(river_id, dbconn, dry_run) |
2861
b0132e1b9719
Added further shape importers and added the option to reproject shapes during the import process.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2853
diff
changeset
|
54 ] |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
55 |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
56 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
57 def getConfig(): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
58 parser = optparse.OptionParser() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
59 parser.add_option("--directory", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
60 parser.add_option("--target_srs", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
61 parser.add_option("--host", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
62 parser.add_option("--user", type="string") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
63 parser.add_option("--password", type="string") |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
64 parser.add_option("--river_name", type="string") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
65 parser.add_option("--verbose", type="int", default=1) |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
66 parser.add_option("--dry_run", type="int", default=0) |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
67 parser.add_option("--ogr_connection", type="string") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
68 parser.add_option("--skip_axis", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
69 parser.add_option("--skip_hydr_boundaries", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
70 parser.add_option("--skip_buildings", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
71 parser.add_option("--skip_crosssections", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
72 parser.add_option("--skip_fixpoints", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
73 parser.add_option("--skip_floodplains", type="int") |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
74 parser.add_option("--skip_hws_lines", type="int") |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
75 parser.add_option("--skip_hws_points", type="int") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
76 parser.add_option("--skip_gauge_locations", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
77 parser.add_option("--skip_kms", type="int") |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
78 parser.add_option("--skip_uesgs", type="int") |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
79 parser.add_option("--skip_dgm", type="int") |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
80 parser.add_option("--skip_jetties", type="int") |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
81 parser.add_option("--skip_floodmarks", type="int") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
82 (config, args) = parser.parse_args() |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
83 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
84 if config.verbose > 1: |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
85 initialize_logging(logging.DEBUG) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
86 elif config.verbose == 1: |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
87 initialize_logging(logging.INFO) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
88 else: |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
89 initialize_logging(logging.WARN) |
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
90 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
91 if config.directory == None: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
92 logger.error("No river directory specified!") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
93 raise Exception("Invalid config") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
94 if not config.ogr_connection: |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
95 if not config.host: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
96 logger.error("No database host specified!") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
97 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
98 if not config.user: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
99 logger.error("No databaser user specified!") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
100 raise Exception("Invalid config") |
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
101 if not config.password: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
102 logger.error("No password specified!") |
4871
890eaa0a5162
Add ogr_connection parameter to allow directly setting the ogr
Andre Heinecke <aheinecke@intevation.de>
parents:
3655
diff
changeset
|
103 raise Exception("Invalid config") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
104 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
105 return config |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
106 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
107 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
108 def skip_importer(config, importer): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
109 if config.skip_axis == 1 and isinstance(importer, Axis): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
110 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
111 elif config.skip_hydr_boundaries == 1 and isinstance(importer, HydrBoundary): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
112 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
113 elif config.skip_hydr_boundaries == 1 and isinstance(importer, HydrBoundaryPoly): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
114 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
115 elif config.skip_buildings == 1 and isinstance(importer, Building): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
116 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
117 elif config.skip_crosssections == 1 and isinstance(importer, CrosssectionTrack): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
118 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
119 elif config.skip_fixpoints == 1 and isinstance(importer, Fixpoint): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
120 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
121 elif config.skip_floodplains == 1 and isinstance(importer, Floodplain): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
122 return True |
4963
1469066cc7d9
Importer: Fix the order of the config check
Andre Heinecke <aheinecke@intevation.de>
parents:
4955
diff
changeset
|
123 elif config.skip_hws_lines == 1 and isinstance(importer, HWSLines): |
1469066cc7d9
Importer: Fix the order of the config check
Andre Heinecke <aheinecke@intevation.de>
parents:
4955
diff
changeset
|
124 return True |
5011
514948fdae54
Do not skip hws_lines when skip hws_points is true
Andre Heinecke <aheinecke@intevation.de>
parents:
5006
diff
changeset
|
125 elif config.skip_hws_points == 1 and isinstance(importer, HWSPoints) and \ |
514948fdae54
Do not skip hws_lines when skip hws_points is true
Andre Heinecke <aheinecke@intevation.de>
parents:
5006
diff
changeset
|
126 not isinstance(importer, HWSLines): |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
127 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
128 elif config.skip_gauge_locations == 1 and isinstance(importer, GaugeLocation): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
129 return True |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
130 elif config.skip_jetties == 1 and isinstance(importer, Jetties): |
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5221
diff
changeset
|
131 return True |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
132 elif config.skip_kms == 1 and isinstance(importer, KM): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
133 return True |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
134 elif config.skip_uesgs == 1 and isinstance(importer, UESG): |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
135 return True |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
136 elif config.skip_floodmarks == 1 and isinstance(importer, Floodmark): |
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
137 return True |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
138 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
139 return False |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
140 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
141 def main(): |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
142 config=None |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
143 try: |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
144 config = getConfig() |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
145 except: |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
146 return -1 |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
147 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
148 if config == None: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
149 logger.error("Unable to read config from command line!") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
150 return |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
151 |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
152 if config.dry_run > 0: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
153 logger.info("You enable 'dry_run'. No database transaction will take place!") |
3655
8654e4480fc6
Shape importer: added command line option dry_run to supress database transactions.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
3654
diff
changeset
|
154 |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
155 if config.ogr_connection: |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
156 connstr = config.ogr_connection |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
157 else: |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
158 connstr = 'OCI:%s/%s@%s' % (config.user, config.password, config.host) |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
159 |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
160 oracle = False # Marker if oracle is used. |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
161 if 'OCI:' in connstr: |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
162 oracle = True |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
163 try: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
164 import cx_Oracle as dbapi |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
165 raw_connstr=connstr.replace("OCI:", "") |
5077
c5187ab9f571
Fix Syntax Errror
Andre Heinecke <aheinecke@intevation.de>
parents:
5036
diff
changeset
|
166 except ImportError: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
167 logger.error("Module cx_Oracle not found in: %s\n" |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
168 "Neccessary to connect to a Oracle Database.\n" |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
169 "Please refer to the installation " |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
170 "documentation." % sys.path) |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
171 return -1 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
172 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
173 else: # Currently only support for oracle and postgres |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
174 try: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
175 import psycopg2 as dbapi |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
176 raw_connstr=connstr.replace("PG:", "") |
4977
ee908e4946cd
corrected Typo
Tom Gottfried <tom.gottfried@intevation.de>
parents:
4975
diff
changeset
|
177 except ImportError: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
178 logger.error("Module psycopg2 not found in: %s\n" |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
179 "Neccessary to connect to a Posgresql Database.\n" |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
180 "Please refer to the installation " |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
181 "documentation." % sys.path) |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
182 return -1 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
183 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
184 dbconn_raw = dbapi.connect(raw_connstr) |
4879
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
185 dbconn = ogr.Open(connstr) |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
186 |
63f82bdd45f8
(importer) Enable hws_* importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4872
diff
changeset
|
187 if dbconn == None: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
188 logger.error("Could not connect to database %s" % connstr) |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
189 return -1 |
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
190 |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
191 types = {} |
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
192 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
193 directories = [] |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
194 if not config.river_name: |
5005
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
195 for file in [os.path.join(config.directory, d) for d in \ |
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
196 os.listdir(config.directory)]: |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
197 if os.path.isdir(file): |
5005
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
198 directories.append(file) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
199 else: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
200 directories.append(config.directory) |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
201 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
202 for directory in directories: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
203 if not config.river_name: |
5005
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
204 river_name = utils.getUTF8Path( |
eb2d6609387c
Importer: Fix directory paths and river name encoding
Andre Heinecke <aheinecke@intevation.de>
parents:
4998
diff
changeset
|
205 os.path.basename(os.path.normpath(directory))) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
206 else: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
207 river_name = config.river_name |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
208 river_id = utils.getRiverId(dbconn_raw, river_name, oracle) |
2853
bd9e76e0b55d
Improved the python shapefile importer.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2798
diff
changeset
|
209 |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
210 if not river_id: |
5221
aa043641d9c8
Fix syntax error when river id is not found (pylint could have told me this)
Andre Heinecke <aheinecke@intevation.de>
parents:
5219
diff
changeset
|
211 logger.info(u"Could not find river in database. Skipping: %s" |
aa043641d9c8
Fix syntax error when river id is not found (pylint could have told me this)
Andre Heinecke <aheinecke@intevation.de>
parents:
5219
diff
changeset
|
212 % unicode(utils.getUTF8(river_name), "UTF-8")) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
213 continue |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
214 else: |
5219
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
215 logger.info(u"Importing River: %s" % unicode( |
40dc001594e4
Allow unicode river names
Andre Heinecke <aheinecke@intevation.de>
parents:
5077
diff
changeset
|
216 utils.getUTF8(river_name), "UTF-8")) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
217 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
218 for importer in getImporters(river_id, dbconn, config.dry_run): |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
219 if skip_importer(config, importer): |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
220 logger.info("Skip import of '%s'" % importer.getName()) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
221 continue |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
222 |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
223 logger.info("Start import of '%s'" % importer.getName()) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
224 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
225 shapes = utils.findShapefiles(importer.getPath(config.directory)) |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
226 logger.debug("Found %i Shapefiles" % len(shapes)) |
4970
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
227 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
228 for shpTuple in shapes: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
229 geomType = importer.walkOverShapes(shpTuple) |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
230 try: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
231 if geomType is not None: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
232 num = types[geomType] |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
233 types[geomType] = num+1 |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
234 except: |
174fbaa3d344
Add handling of River Names and remove target_src parameter
Andre Heinecke <aheinecke@intevation.de>
parents:
4963
diff
changeset
|
235 types[geomType] = 1 |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
236 |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
237 for key in types: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
238 logger.debug("%i x geometry type %s" % (types[key], key)) |
2798
5a654f2e35bc
Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
239 |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
240 if not config.skip_dgm: |
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
241 dgmfilename = os.path.join( |
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
242 config.directory, "..", "DGMs.csv") |
4989
f4d61ac84b76
If we bother to check if the file is readable we might also
Andre Heinecke <aheinecke@intevation.de>
parents:
4978
diff
changeset
|
243 if not os.access(dgmfilename, os.R_OK) or not \ |
f4d61ac84b76
If we bother to check if the file is readable we might also
Andre Heinecke <aheinecke@intevation.de>
parents:
4978
diff
changeset
|
244 os.path.isfile(dgmfilename): |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
245 logger.info("Could not find or access DGM file: %s \n" |
4975
9eea3cd22ee7
Add dgm file reader although an awk script should have been more appropiate
Andre Heinecke <aheinecke@intevation.de>
parents:
4970
diff
changeset
|
246 "Skipping DGM import." % dgmfilename) |
4978
b5ac7588cf4d
Use os.access instead of try/except open
Andre Heinecke <aheinecke@intevation.de>
parents:
4977
diff
changeset
|
247 else: |
5006
769593a84606
Importer: Behold, Logging!
Andre Heinecke <aheinecke@intevation.de>
parents:
5005
diff
changeset
|
248 logger.info("Inserting DGM meta information in 'dem' table.") |
5036
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
249 insertRiverDgm(dbconn_raw, dgmfilename, river_name, |
d6860fca89e4
Pull information about the backend into dgm.
Andre Heinecke <aheinecke@intevation.de>
parents:
5028
diff
changeset
|
250 config.dry_run, oracle) |
5012
a79f6a085ab6
Add message if DGM import is skipped
Andre Heinecke <aheinecke@intevation.de>
parents:
5011
diff
changeset
|
251 else: |
a79f6a085ab6
Add message if DGM import is skipped
Andre Heinecke <aheinecke@intevation.de>
parents:
5011
diff
changeset
|
252 logger.info("Skip import of DGM.") |
3654
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
253 |
59ca5dab2782
Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2862
diff
changeset
|
254 if __name__ == '__main__': |
4872
a563e9f58f93
Improve error handling and unify dbconn for all importers
Andre Heinecke <aheinecke@intevation.de>
parents:
4871
diff
changeset
|
255 main() |