roundup-cc: collect_issues.py annotate

annotate collect_issues.py @ 36:59e1659a0a0b tip

Update 'TODO.creole' * Old TODOs moved to 'doc/old_TODO.creole'

author	Magnus Schieder <mschieder@intevation.de>
date	Mon, 26 Nov 2018 16:52:45 +0100
parents	b07588ac28b6
children

rev	line source
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	1 #!/usr/bin/env python3
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	2 """ Fetch issues from a roundup-tracker and save them in a databse.
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	3
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	4 author: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	5 author: Bernhard Reiter <bernhard@intevation.de>
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	6 author: Sean Engelhardt <sean.engelhardt@intevation.de>
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	7 author: Magnus Schieder <magnus.schieder@intevation.de>
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	8
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	9 (c) 2010, 2015, 2018 by Intevation GmbH
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	10
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	11 This is Free Software unter the terms of the
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	12 GNU GENERAL PUBLIC LICENSE Version 3 or later.
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	13 See http://www.gnu.org/licenses/gpl-3.0.txt for details
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	14
17 adca5b3780d2 Add collecting no-prio issues. Restructure. Bernhard Reiter <bernhard@intevation.de> parents: 15 diff changeset	15 For usage see examples/.
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	16 """
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	17
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	18 import http.cookiejar
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	19 import urllib.parse
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	20 import urllib.request
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	21 import csv
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	22 import io
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	23 import sqlite3 as db
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	24 import os
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	25 import sys
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	26
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	27
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	28 # Getting keywords and their ids.
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	29 CHECK_KEYWORD_VALUES = "keyword?@action=export_csv&@columns=id,name"
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	30 # Getting states and their ids.
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	31 CHECK_STATUS_VALUES = "status?@action=export_csv&@columns=id,name"
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	32 # Getting priorities and their ids.
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	33 CHECK_PRIO_VALUES = "priority?@action=export_csv&@columns=id,name"
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	34
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	35 # Getting the priority of each issue with the filter status, keyword ,priority
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	36 SEARCH_ROUNDUP_PRIO = "issue?@action=export_csv&@columns=priority&@filter=status,keyword,priority&@pagesize=500&@startwith=0&status={status_values}&keyword={keyword_values}&priority={priority_values}"
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	37 # Getting the status of each issue with the filter keyword, priority, status
31 9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	38 SEARCH_ROUNDUP_STATUS = "issue?@action=export_csv&@columns=status&@filter=priority,keyword,status&@pagesize=500&@startwith=0&priority={priority_values}&keyword={keyword_values}&status={status_values}"
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	39
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	40
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	41 def connect_to_server(params, baseurl):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	42 enc_data = urllib.parse.urlencode(params).encode()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	43 cj = http.cookiejar.CookieJar()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	44 opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	45 req = urllib.request.Request(url=baseurl, data=enc_data)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	46 opener.open(req)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	47 return opener
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	48
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	49
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	50 def get_csv_from_server(opener, roundup_url, sub_url):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	51 csv_req = urllib.request.Request(url=roundup_url+sub_url)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	52 f = opener.open(csv_req)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	53 csv_reader = csv.DictReader(io.TextIOWrapper(f))
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	54 return csv_reader
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	55
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	56
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	57 def check_create_database(database_file, sql_create_db):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	58 if not os.path.isfile(database_file):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	59 con = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	60 cur = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	61 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	62 con = db.connect(database_file)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	63 cur = con.cursor()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	64 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	65 cur.execute(sql_create_db)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	66 con.commit()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	67 os.chmod(database_file, 0o644)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	68 except:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	69 con.rollback()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	70 raise
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	71 finally:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	72 if cur:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	73 cur.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	74 if con:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	75 con.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	76
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	77 def get_ids(opener, baseurl, parameter, url, include_no_prio=False):
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	78 """Returns the IDs of the respective search parameters as string and list.
80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	79 """
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	80
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	81 if parameter == [""]:
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	82 return ("", [])
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	83
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	84 parameter_csv = get_csv_from_server(opener, baseurl, url)
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	85 parameter_dict = {}
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	86 for x in parameter_csv:
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	87 parameter_dict[x["name"]] = x["id"]
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	88
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	89 if include_no_prio:
e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	90 parameter_dict["None"] = "-1"
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	91
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	92 parameter_ids = []
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	93 for x in parameter:
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	94 if x not in parameter_dict:
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	95 print('The parameter "%s" does not exist in the tracker.' % x)
cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	96 sys.exit(0)
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	97
27 cdab667c6abb Delete Code Duplication and Clean Up. Magnus Schieder <mschieder@intevation.de> parents: 26 diff changeset	98 parameter_ids.append(parameter_dict[x])
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	99
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	100 return (",".join(parameter_ids), parameter_ids)
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	101
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	102
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	103 def issues_to_quantities(issue_csv, columns_ids):
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	104 """Count issues per priority.
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	105
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	106 Returns: a list of ints, containing how often a prio/status occurred
80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	107 in the order in which they are specified in the config.
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	108 """
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	109
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	110 order_dict = {}
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	111 z = 0
e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	112 for x in columns_ids:
e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	113 order_dict[x] = z
e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	114 z += 1
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	115
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	116 quantities = [0] * z
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	117
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	118 for issue in issue_csv:
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	119 priority = issue[issue_csv.fieldnames[0]]
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	120
2 3e9f4a6803d1 replaced funtion with string.isdigit. sean parents: 1 diff changeset	121 if priority.isdigit() == True :
33 946b9f458fcc Priorities with 'None' are not sorted out unintentionally Magnus Schieder <mschieder@intevation.de> parents: 32 diff changeset	122 quantities[order_dict[priority]] += 1
946b9f458fcc Priorities with 'None' are not sorted out unintentionally Magnus Schieder <mschieder@intevation.de> parents: 32 diff changeset	123 else: # no priority set
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	124 quantities[-1] += 1
33 946b9f458fcc Priorities with 'None' are not sorted out unintentionally Magnus Schieder <mschieder@intevation.de> parents: 32 diff changeset	125 #print("quantities : " + str(quantities))
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	126
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	127 return quantities
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	128
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	129
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	130 def save_issues_to_db(quantities, database_file, sql_create_db, sql_insert_in_db):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	131 check_create_database(database_file, sql_create_db)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	132
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	133 cur = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	134 con = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	135
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	136 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	137 con = db.connect(database_file)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	138 cur = con.cursor()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	139 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	140 cur.execute(sql_insert_in_db, quantities)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	141 con.commit()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	142 except:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	143 con.rollback()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	144 raise
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	145 finally:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	146 if cur:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	147 cur.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	148 if con:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	149 con.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	150
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	151
31 9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	152 def save_stats_in_db(search, login_parmeters, baseurl, db_file, columns, sql_create_db, sql_insert_in_db, keywords, search_parameters, include_no_prio):
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	153 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	154
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	155 opener = connect_to_server(login_parmeters, baseurl)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	156
31 9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	157 keywords_ids_url, _ = get_ids(opener, baseurl, keywords,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	158 CHECK_KEYWORD_VALUES)
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	159
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	160 if search == "prio":
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	161 # search_parameters are states.
31 9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	162 status_ids_url, _ = get_ids(opener, baseurl,search_parameters ,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	163 CHECK_STATUS_VALUES, include_no_prio)
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	164 prio_ids_url, columns_ids = get_ids(opener, baseurl, columns,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	165 CHECK_PRIO_VALUES, include_no_prio)
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	166 formated_search_url = SEARCH_ROUNDUP_PRIO.format(
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	167 status_values=status_ids_url,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	168 keyword_values=keywords_ids_url,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	169 priority_values=prio_ids_url)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	170
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	171 elif search == "status":
31 9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	172 # search_parameters are priorities.
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	173 prio_ids_url, _ = get_ids(opener, baseurl, search_parameters,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	174 CHECK_PRIO_VALUES, include_no_prio)
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	175 status_ids_url, columns_ids = get_ids(opener, baseurl, columns,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	176 CHECK_STATUS_VALUES)
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	177 formated_search_url = SEARCH_ROUNDUP_STATUS.format(
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	178 priority_values=prio_ids_url,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	179 keyword_values=keywords_ids_url,
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	180 status_values=status_ids_url)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	181
31 9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	182 current_issues_csv = get_csv_from_server(opener, baseurl,
9aca070c86bd Add the filter 'priority' for the search by status. Magnus Schieder <mschieder@intevation.de> parents: 30 diff changeset	183 formated_search_url)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	184
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	185 opener.close()
32 80bbd06fe8ec Add Comments Magnus Schieder <mschieder@intevation.de> parents: 31 diff changeset	186 #print(baseurl + formated_search_url)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	187
28 e2864dabdb8c fixes a logical error in the filtering of columns. Magnus Schieder <mschieder@intevation.de> parents: 27 diff changeset	188 quantities = issues_to_quantities(current_issues_csv, columns_ids)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	189
9 e95f7bee8643 Really safe data. Bernhard Reiter <bernhard@intevation.de> parents: 2 diff changeset	190 save_issues_to_db(quantities, db_file, sql_create_db, sql_insert_in_db)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	191
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	192 except urllib.error.URLError as e:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	193 print("No Valid Connection to server : " + baseurl + "\nerror: " + str(e))

Mercurial > roundup-cc

annotate collect_issues.py @ 36:59e1659a0a0b tip