Hey I wanted keep everyone in the loop about thunderbolt and about the idea to use manifests:
12:52 <@zyga> spineau: let's talk about manifests 12:52 <@spineau> zyga: to handle job upgrade? 12:53 <@zyga> spineau: in my earlier drafts I was proposing to add a manifest entry unit. each item would have an ID, a name (as listed in the manifest), a type (bool, int, we can add more as needed) and optional unit type 12:53 <@zyga> spineau: no, for thunderbolt 12:53 <@zyga> spineau: so for TB, we could say that there's a unit like this: 12:53 <@zyga> unit: manifest entry 12:53 <@zyga> id: number-of-thunderbolt-ports 12:53 <@zyga> type: int 12:54 <@zyga> _name: Number of thunderbolt ports 12:54 <@zyga> _description: This unit describes the number of external thunderbolt ports as visible from the POV of the user. 12:54 <@zyga> (I would now rename the id to number-of-external-thunderbolt-ports) 12:54 <@zyga> that's it 12:55 <@zyga> plainbox would see that, would load a manifest file and expose it as a special resource like thing 12:55 <@zyga> spineau: in jobs we could say that a job requires: number-of-thunderbolt-ports > 0 12:55 <@zyga> (now I would use underscore instead of dashes) 12:56 <@zyga> spineau: and this could be a special requirement on manifes entry or a normal requirement, not sure yet 12:56 <@zyga> spineau: the actual manifest file could be rfc822 or json 12:56 <@zyga> spineau: we could pre-supply one on command line 12:56 <@zyga> spineau: we could get one from hexr or somewhere like that 12:56 <@spineau> zyga: to me this could be done dynamically, before the test selection to allow the tester to manually specify the #ports 12:56 <@zyga> spineau: or we could ask the user before all testing is started and cache it 12:56 <@spineau> same idea same second 12:57 <@zyga> spineau: we don't want to ask the user over and over as this is static data 12:57 <@zyga> spineau: that's why I want to treat it differently than resource jobs that run via dependencies 12:57 <@zyga> spineau: one more thing 12:57 <@zyga> spineau: I think we could use a "info" job/unit 12:57 <@zyga> spineau: or some other setup support 12:58 <@zyga> spineau: I see an unit that says "dear tester, this machine has a thunderbolt port, please connect it like this (picture)" 12:58 <@zyga> spineau: and those would run before all testing as well 12:58 <@zyga> spineau: or if we go for intra-testing, those would be just jobs we can put anywhere we want 12:58 <@zyga> spineau: those would not be a test of any kind 12:58 <@zyga> spineau: last idea 12:58 <@zyga> spineau: a support hardware unit 12:59 <@zyga> spineau: so a test that needs to do daisy chain can indicate it needs this hardware 12:59 <@zyga> spineau: and can be skipped if the operator doesn't have the hardware 12:59 <@zyga> spineau: this would be exclusive to stuff that's not being tested, external hardware like a usb disk 12:59 <@zyga> spineau: and we could collect those and tell the user before testing 12:59 <@zyga> spineau: exactly what they will need 12:59 <@zyga> spineau: adding those units is very simple 13:00 <@zyga> spineau: adding support in checkbox touch is interesting 13:00 <@zyga> spineau: what do you think? 13:00 <@spineau> zyga: I like the idea of computing hw req based on the testplan content 13:00 <@zyga> spineau: (blank cds, monitors, cables, whatever is needed) 13:00 <@zyga> spineau: this could help a lot in making the process better 13:01 <@spineau> zyga: I'm a bit more skeptical about the manifest unit and how to pass it to a gui for example and how to decide when we need one without first running a cert plan of the device 13:02 <@zyga> spineau: we always need it 13:02 <@zyga> spineau: we may not use it 13:02 <@zyga> spineau: but it has to be filled in 13:03 <@zyga> spineau: it would not be passed to the gui, the system would load one from a database (in real testing in taiwan) 13:03 <@zyga> spineau: or we'd make one once for each of our development systems 13:03 <@zyga> spineau: all we need is something that can be used as a key (a CID perhaps) 13:03 <@spineau> zyga: but for cdts offline testing that won't scale 13:04 <@zyga> spineau: why not? 13:04 <@spineau> zyga: unless someone knows that this system MAY fail resources detection 13:04 <@zyga> spineau: for cdts the tester will just create the manifest first time 13:04 <@zyga> spineau: no noe 13:04 <@spineau> zyga: so someone has to run and review a first testrun 13:04 <@zyga> spineau: no 13:05 <@zyga> spineau: manifests replace resources detection for things that we deem appropriate 13:05 <@zyga> spineau: there is no detection 13:05 <@zyga> spineau: lack of a manifest won't mean that hardware is gone 13:05 <@zyga> spineau: it will prompt the user to create the manifest interactively 13:05 <@zyga> spineau: and cache it 13:06 <@zyga> spineau: we can create a small CLI tool for that 13:06 <@zyga> spineau: so that we won't have to touch checkbox-gui 13:06 <@zyga> spineau: but it should be a part of normal flow on checkbox-touch and all non-frozen codebases 13:06 <@zyga> spineau: for certification in the labs we can have a different manifest delivery mechanism (some local database from hexr) 13:07 <@spineau> zyga: one thing we could do is a tool/step/screen to list what we automatically see and the tester can decide to add more items (and then cache the result) 13:07 <@zyga> spineau: no, I disagree, we fundamentally need to stop detecting flaky things 13:07 <@zyga> spineau: users will just ignore that and bad data will stay 13:07 <@zyga> spineau: and this has to be correct 13:07 <@spineau> zyga: but several resources jobs are reliable 13:07 <@zyga> spineau: so those stay as resources 13:07 <@zyga> spineau: this is for stuff like media keys 13:08 <@zyga> spineau: physical ports (do you really have that hdmi0 or is it just dead unsoldered crap on the m/b) 13:08 <@zyga> spineau: things that we can detect reliably should stay as such 13:08 <@zyga> spineau: this is for real-life observable elements of the device 13:08 <@zyga> spineau: that we cannot probe 13:09 <@zyga> spineau: leds, keyboard arrangment, connectors, form factor, etc) 13:09 <@spineau> zyga: IIRC we should also add 80211 ac as we found one system that didn't advertise itself as compatible 13:09 <@zyga> spineau: firmware will be broken and will always lie or not tell in general, if our testing depends on it we should not rely on a flaky component 13:09 <@zyga> spineau: yep 13:09 <@zyga> spineau: and 13:10 <@zyga> spineau: a subset of the manifest can be used on the certification website 13:10 <@zyga> spineau: as icons/badges etc 13:10 <@zyga> spineau: e.g. thunderbolt icon 13:10 <@zyga> spineau: usb 3.0 icon 13:10 <@zyga> spineau: whatever we want to use there 13:10 <@zyga> spineau: the point is that this is something that is unchangable 13:11 <@zyga> spineau: we need to be careful with servers and other machines that have lots of add-on cards 13:11 <@zyga> spineau: but for phones, tables and laptops 13:11 <@zyga> spineau: this is perfect 13:11 <@zyga> spineau: screen size, amount of memory, available connection systems (modem type) 13:11 <@zyga> spineau: all of that stuff 13:11 <@zyga> spineau: actually 13:11 <@zyga> spineau: this also nicely blends with one core addition I wanted to do 13:11 <@zyga> spineau: static resources 13:11 <@zyga> spineau: so that we instantiate templates staitically 13:12 <@zyga> spineau: and verify that 13:12 <@spineau> zyga: let's keep this idea opened, and restart the discussion tomorrow as I have to go 13:12 <@spineau> sorry 13:12 <@zyga> spineau: like the selftest provider that statically tells which python libraries we have 13:12 <@zyga> spineau: ok :) 13:12 <@zyga> spineau: thanks! 13:12 <@zyga> spineau: I'll send this to the ML 13:12 <@spineau> zyga: thanks to you 13:12 <@spineau> ok 13:12 <@spineau> bye Best regards ZK -- Mailing list: https://launchpad.net/~checkbox-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~checkbox-dev More help : https://help.launchpad.net/ListHelp
