Mercurial > dive4elements > river

annotate flys-backend/contrib/shpimporter/utils.py @ 4882:e9880b224c2f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
(Importer) Add comment about OFT types
author Andre Heinecke <aheinecke@intevation.de>
date Fri, 25 Jan 2013 15:38:34 +0100
parents b1d7e600b43b
children b457532dae63
rev   line source
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
1 import os
4874
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
2 import sys
3654
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
3 from shpimporter import DEBUG, INFO, ERROR
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
4
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
5 SHP='.shp'
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
6
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
7 def findShapefiles(path):
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
8 shapes = []
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
9
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
10 for root, dirs, files in os.walk(path):
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
11 if len(files) == 0:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
12 continue
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
13
3654
59ca5dab2782 Shape importer: use python's OptionParse to read user specific configuration from command line.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2798
diff changeset
14 DEBUG("Processing directory '%s' with %i files " % (root, len(files)))
2798
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
15
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
16 for f in files:
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
17 idx = f.find(SHP)
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
18 if (idx+len(SHP)) == len(f):
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
19 shapes.append((f.replace(SHP, ''), root + "/" + f))
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
20
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
21 return shapes
5a654f2e35bc Added a python tool to import shapefiles into database.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
22
4874
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
23 def getUTF8Path(path):
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
24 """
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
25 Tries to convert path to utf-8 by first checking the filesystemencoding
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
26 and trying the default windows encoding afterwards.
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
27 Returns a valid UTF-8 encoded unicode object or throws a UnicodeDecodeError
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
28 """
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
29 try:
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
30 return unicode.encode(unicode(path, sys.getfilesystemencoding()), "UTF-8")
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
31 except UnicodeDecodeError:
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
32 # Probably Windows iso-8859-x names so lets try again
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
33 return unicode.encode(unicode(path, "iso-8859-15"), "UTF-8")
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
34
4882
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
35
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
36 # 0 OFTInteger, Simple 32bit integer
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
37 # 1 OFTIntegerList, List of 32bit integers
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
38 # 2 OFTReal, Double Precision floating point
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
39 # 3 OFTRealList, List of doubles
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
40 # 4 OFTString, String of ASCII chars
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
41 # 5 OFTStringList, Array of strings
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
42 # 6 OFTWideString, deprecated
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
43 # 7 OFTWideStringList, deprecated
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
44 # 8 OFTBinary, Raw Binary data
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
45 # 9 OFTDate, Date
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
46 # 10 OFTTime, Time
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
47 # 11 OFTDateTime, Date and Time
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
48
e9880b224c2f (Importer) Add comment about OFT types
Andre Heinecke <aheinecke@intevation.de>
parents: 4874
diff changeset
49
4874
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
50 def copyFields(src, target, mapping):
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
51 """
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
52 Checks the mapping dictonary for key value pairs to
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
53 copy from the source to the destination feature.
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
54
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
55 The Key is the attribute of the source feature to be copied
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
56 into the target attribute named by the dict's value.
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
57 """
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
58 for key, value in mapping.items():
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
59 if src.GetFieldIndex(key) == -1:
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
60 continue
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
61 if src.IsFieldSet(src.GetFieldIndex(key)):
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
62 if src.GetFieldType(key) == 2:
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
63 target.SetField(value, src.GetFieldAsDouble(key))
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
64 else:
b1d7e600b43b (importer) Add utility function to convert paths to utf-8
Andre Heinecke <aheinecke@intevation.de>
parents: 3654
diff changeset
65 target.SetField(value, src.GetField(key))
http://dive4elements.wald.intevation.org