roundup-cc: collect_issues.py annotate

annotate collect_issues.py @ 25:7161ce4e7ab1

The web-based display is dynamically generated. * All graphs are passed in one object. All graphs are passed in one object. The graphs are generated iteratiev and assigned a color. * roundup_cc.py can count how many issues are in which state.

author	Magnus Schieder <mschieder@intevation.de>
date	Mon, 12 Nov 2018 18:03:26 +0100
parents	3bb3d9a9f1b7
children	761ee2351f58

rev	line source
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	1 #!/usr/bin/env python3
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	2 """ Fetch issues from a roundup-tracker and save them in a databse.
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	3
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	4 author: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	5 author: Bernhard Reiter <bernhard@intevation.de>
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	6 author: Sean Engelhardt <sean.engelhardt@intevation.de>
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	7
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	8 (c) 2010, 2015, 2018 by Intevation GmbH
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	9
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	10 This is Free Software unter the terms of the
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	11 GNU GENERAL PUBLIC LICENSE Version 3 or later.
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	12 See http://www.gnu.org/licenses/gpl-3.0.txt for details
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	13
17 adca5b3780d2 Add collecting no-prio issues. Restructure. Bernhard Reiter <bernhard@intevation.de> parents: 15 diff changeset	14 For usage see examples/.
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	15 """
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	16
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	17 import http.cookiejar
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	18 import urllib.parse
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	19 import urllib.request
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	20 import csv
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	21 import io
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	22 import sqlite3 as db
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	23 import os
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	24
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	25
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	26 # Getting all priority in their order.
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	27 CHECK_ROUNDUP_ORDER_PRIO = "priority?@action=export_csv&@columns=id,order"
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	28 # Getting all statuses in their order.
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	29 CHECK_ROUNDUP_ORDER_STATUS = "status?@action=export_csv&@columns=id,order"
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	30 # Getting keywords and their ids.
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	31 CHECK_KEYWORD_ORDER = "keyword?@action=export_csv&@columns=id,name"
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	32 # Getting states and their ids.
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	33 CHECK_ROUNDUP_SEARCH_VALUES = "status?@action=export_csv&@columns=id,name&@filter=open&open=1"
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	34 # Getting the priority of each issue with the filter status and keywords
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	35 SEARCH_ROUNDUP_PRIO = "issue?@action=export_csv&@columns=priority&@filter=status,keyword&@pagesize=500&@startwith=0&status={search_values}&keyword={keyword_values}"
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	36 # Getting the status of each issue with the filter keywords
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	37 SEARCH_ROUNDUP_STATUS = "issue?@action=export_csv&@columns=status&@filter=keyword&@pagesize=500&@startwith=0&keyword={keyword_values}"
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	38
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	39
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	40 def connect_to_server(params, baseurl):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	41 enc_data = urllib.parse.urlencode(params).encode()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	42 cj = http.cookiejar.CookieJar()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	43 opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	44 req = urllib.request.Request(url=baseurl, data=enc_data)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	45 opener.open(req)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	46 return opener
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	47
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	48
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	49 def get_csv_from_server(opener, roundup_url, sub_url):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	50 csv_req = urllib.request.Request(url=roundup_url+sub_url)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	51 f = opener.open(csv_req)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	52 csv_reader = csv.DictReader(io.TextIOWrapper(f))
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	53 return csv_reader
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	54
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	55
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	56 def check_create_database(database_file, sql_create_db):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	57 if not os.path.isfile(database_file):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	58 con = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	59 cur = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	60 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	61 con = db.connect(database_file)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	62 cur = con.cursor()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	63 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	64 cur.execute(sql_create_db)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	65 con.commit()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	66 os.chmod(database_file, 0o644)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	67 except:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	68 con.rollback()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	69 raise
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	70 finally:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	71 if cur:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	72 cur.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	73 if con:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	74 con.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	75
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	76 def get_keyword_ids(opener, baseurl, keywords):
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	77 if keywords == [""]:
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	78 return ""
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	79
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	80 keywords_csv = get_csv_from_server(opener, baseurl, CHECK_KEYWORD_ORDER)
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	81 keywords_dict = {}
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	82 for x in keywords_csv:
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	83 keywords_dict[x["name"]] = x["id"]
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	84
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	85 keywords_ids = []
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	86 for x in keywords:
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	87 keywords_ids.append(keywords_dict[x])
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	88
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	89 return ",".join(keywords_ids)
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	90
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	91 def get_status_ids(opener, baseurl, status):
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	92
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	93 if status == [""]:
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	94 return ""
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	95
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	96 status_csv = get_csv_from_server(opener, baseurl, CHECK_ROUNDUP_SEARCH_VALUES)
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	97
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	98 status_dict = {}
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	99 for x in status_csv:
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	100 status_dict[x["name"]] = x["id"]
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	101
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	102 staus_ids = ["-1"]
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	103 for x in status:
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	104 staus_ids.append(status_dict[x])
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	105
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	106 return ",".join(staus_ids)
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	107
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	108
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	109 def issues_to_quantities(issue_csv, columns, orders_csv):
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	110 """Count issues per priority.
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	111
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	112 Returns: a list of ints, containing how often a prio occurred [:-1]
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	113 in order of the priorities, with the last being the "None" prio
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	114 """
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	115
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	116 quantities = [0] * (len(columns) +1)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	117 order_dict = {}
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	118
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	119 #convert the csv-dict reader to real dict
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	120 for row in orders_csv:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	121 order_dict[row["id"]] = int(float(row["order"])) # int(float()) because the order-value is indeed "1.0, 2.0" etc
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	122
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	123 for issue in issue_csv:
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	124 priority = issue[issue_csv.fieldnames[0]]
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	125
2 3e9f4a6803d1 replaced funtion with string.isdigit. sean parents: 1 diff changeset	126 if priority.isdigit() == True :
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	127 quantities[order_dict[priority] -1 ] += 1
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	128 else: # no priority set
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	129 quantities[-1] += 1
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	130
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	131 # print("quantities : " + str(quantities))
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	132
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	133 return quantities
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	134
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	135
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	136 def save_issues_to_db(quantities, database_file, sql_create_db, sql_insert_in_db):
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	137 check_create_database(database_file, sql_create_db)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	138
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	139 cur = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	140 con = None
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	141
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	142 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	143 con = db.connect(database_file)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	144 cur = con.cursor()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	145 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	146 cur.execute(sql_insert_in_db, quantities)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	147 con.commit()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	148 except:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	149 con.rollback()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	150 raise
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	151 finally:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	152 if cur:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	153 cur.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	154 if con:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	155 con.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	156
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	157
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	158 def save_stats_in_db(search, login_parmeters, baseurl, db_file, columns, sql_create_db, sql_insert_in_db, keywords, status, include_no_prio=False):
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	159 try:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	160
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	161 opener = connect_to_server(login_parmeters, baseurl)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	162
20 3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	163 keywords_ids_url = get_keyword_ids(opener, baseurl, keywords)
3bb3d9a9f1b7 Filter by keywords and states. Magnus Schieder <mschieder@intevation.de> parents: 17 diff changeset	164
25 7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	165 if search == "prio":
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	166 order_csv = get_csv_from_server(opener, baseurl, CHECK_ROUNDUP_ORDER_PRIO)
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	167 status_ids_url = get_status_ids(opener, baseurl, status)
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	168 formated_search_url = SEARCH_ROUNDUP_PRIO.format(search_values=status_ids_url, keyword_values=keywords_ids_url)
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	169 elif search == "status":
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	170 order_csv = get_csv_from_server(opener, baseurl, CHECK_ROUNDUP_ORDER_STATUS)
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	171 formated_search_url = SEARCH_ROUNDUP_STATUS.format(keyword_values=keywords_ids_url)
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	172 else:
7161ce4e7ab1 The web-based display is dynamically generated. Magnus Schieder <mschieder@intevation.de> parents: 20 diff changeset	173 print("Incorrect search parameter. (prio, status)")
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	174
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	175 current_issues_csv = get_csv_from_server(opener, baseurl, formated_search_url)
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	176
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	177 opener.close()
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	178
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	179 quantities = issues_to_quantities(current_issues_csv, columns, order_csv)
15 d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	180 if not include_no_prio:
d0c439d1e833 Prepare for tracker entries without priority. Bernhard Reiter <bernhard@intevation.de> parents: 9 diff changeset	181 quantities = quantities[:-1]
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	182
9 e95f7bee8643 Really safe data. Bernhard Reiter <bernhard@intevation.de> parents: 2 diff changeset	183 save_issues_to_db(quantities, db_file, sql_create_db, sql_insert_in_db)
1 2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	184
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	185 except urllib.error.URLError as e:
2df45f6ecd81 new appereance (solid and dotted lines), resonsive layout, new legend, sean parents: diff changeset	186 print("No Valid Connection to server : " + baseurl + "\nerror: " + str(e))

Mercurial > roundup-cc

annotate collect_issues.py @ 25:7161ce4e7ab1