Hi Lucas, On Mon, Apr 13, 2020 at 10:37:57PM +0200, Lucas Nussbaum wrote: > I'm sorry if I haven't paid enough attention. But what is the difference > with the 'ci' importer? > > https://salsa.debian.org/qa/udd/-/blob/master/rimporters/ci.rb
I think the problem is that UDD is not documented and I simply did not know about the ci table. :-( However, when looking at it there is a difference: udd=# select status, arch, count(*) from ci group by status, arch order by status, arch; status | arch | count ---------+-------+------- fail | amd64 | 925 neutral | amd64 | 1593 pass | amd64 | 10805 tmpfail | amd64 | 14 (4 Zeilen) udd=# select status, architecture, count(*) from autopkgtest group by status, architecture order by status, architecture; status | architecture | count ---------+--------------+------- fail | amd64 | 1561 fail | arm64 | 1471 fail | ppc64el | 711 neutral | amd64 | 2879 neutral | arm64 | 1322 neutral | ppc64el | 298 pass | amd64 | 21373 pass | arm64 | 11114 pass | ppc64el | 2458 tmpfail | amd64 | 11 tmpfail | arm64 | 85 tmpfail | ppc64el | 7 (12 Zeilen) I guess we should merge both and make sure that all data is imported. I would never have written a new importer if I would have been aware of the existing one - but I do not speak Ruby to fix the existing one. Kind regards Andreas. > On 11/04/20 at 07:12 +0200, Andreas Tille wrote: > > Hi Paul, > > > > thanks for the clarification. This commit > > > > > > https://salsa.debian.org/qa/udd/-/commit/6a874a89365671dd37a14a9bca25290dc55a1fc9 > > > > imports the current data. I will tests this a bit more and than activate > > in a cron job as importer. > > > > Thanks a lot for your contribution > > > > Andreas. > > > > On Fri, Apr 10, 2020 at 09:05:31PM +0200, Paul Gevers wrote: > > > Hi Andreas, > > > > > > On 09-04-2020 22:53, Andreas Tille wrote: > > > > valid_keys = ( 'run_id', > > > > # 'created_at', # Paul Gevers: should be ignored > > > > # 'updated_at', # Paul Gevers: should be ignored > > > > 'suite', > > > > 'arch', > > > > 'package', # ----> should be renamed to > > > > 'source' > > > > 'version', > > > > 'trigger', # usually package.*version > > > I expected you to mostly see "" or "migration-reference/0" here, with > > > some hand crafted text from random DD's. > > > > > > > 'status', > > > > 'requestor', # 'britney', 'debci' or e-mail > > > > > > Debian login to be precise, not e-mail. > > > > > > > 'pin_packages', # [] > > > > > > Since a couple of months this json and other pages only show "pure" > > > suite runs, to pin_packages is always empty. pin_packages contains which > > > packages are taken from another suite than the base suite. > > > > > > > # 'worker', # Paul Gevers: should be > > > > ignored (is 'null' anyway) > > > > > > Oh, bug somewhere I guess. > > > > > > > 'date', > > > > 'duration_seconds', > > > > 'last_pass_date', > > > > 'last_pass_version', > > > > 'message', # see below > > > > 'previous_status', > > > > # 'duration_human', # Paul Gevers: duration_seconds > > > > and duration_human feel double and the former is leaner for in a > > > > database > > > > # 'blame', # Paul Gevers: should be ignored > > > > ) > > > > > > > > # message can be > > > > # $ grep '"message"' packages*.json | sed 's/^.*\.json: *//' | sort | > > > > uniq > > > > # "message": "All tests passed" > > > > -> "status": "pass" > > > > # "message": "Could not run tests due to a temporary testbed failure" > > > > -> "status": "tmpfail" > > > > # "message": "elbrus" > > > > -> "status": "tmpfail" > > > > # "message": "Erroneous package" > > > > -> "status": "fail" > > > > # "message": null > > > > -> "status": "fail" > > > > # "message": "No tests in this package or all skipped" > > > > -> "status": "neutral" > > > > # "message": "Tests failed", > > > > -> "status": "fail" > > > > # "message": "Tests failed, and at least one test skipped" > > > > -> "status": "fail" > > > > # "message": "Tests passed, but at least one test skipped" > > > > -> "status": "pass" > > > > # "message": "Unexpected autopkgtest exit code 20" > > > > -> "status": "tmpfail" > > > > > > > > I agree that leaving out worker which is really always null makes sense > > > > but I tend to leave message since leaving this out looks like loosing > > > > information. I tried to find a relation to status but it seems the same > > > > status can result in different messages. I think just a field in > > > > addition > > > > will not blow up UDD way more than it recently is - may be I consider > > > > a normalised form, but usually UDD is not very normalised at all. > > > > > > In general this is the final output from autopkgtest. But, as you see my > > > name there, I had to clean up several times and to be able to find those > > > back, I added my nick to the message. The list thus may change over time. > > > > > > Paul > > > > > > > > > > > > > -- > > http://fam-tille.de > > > > > -- http://fam-tille.de