Re: [linux] thoughts on secure over-the-air (OTA) updates?
I am going state the obvious. First, and most importantly, every protocol that talks to the terminals MUST be versioned or there will be chaos. Secondly, your server end has to be able to handle multiple protocol versions because you may not know which device has been upgraded successfully or not. Large binary blob ie complete SW system image are easier to QC since there is only that blob which has to be tested but with unreliable OTA delivery, there will be risks that some terminal will never be upgraded. Never, ever. Very fine grained deployments are more likely to be delivered but can be a nightmare to support and debug since some device may receive A before B while others may receive B before A. Fine grain delivery like continuous deployment frameworks I would only consider for reliable networking scenario ie well connected devices over reliable network. Package level deployment is Ok as long as you can solve dependency issue. Though I am not sure I really want to do OS upgrade small piece at a time. If you have the backend system, complete OS upgrades works well eg. iOS devices are upgraded in the hundreds of millions over the course of a week. Anyways, interesting problem which crosses the technical domain into the organizational domain ie. What the organization is willing to accept. Spencer > On Feb 17, 2020, at 10:40, Robert P. J. Day wrote: > > > for an upcoming project, one of the longer-term goals is to > establish a secure OTA update system for potentially hundreds or > thousands of remote (internet-connected) devices, which might need to > be updated en masse identically, or updated individually. > > i'm currently collecting examples of available solutions > (https://mender.io/, https://sbabic.github.io/swupdate/swupdate.html, > and so on), and the current custodians of the system have already had > animated discussions as to what they think would work, and what > properties it should have. > > on the one hand, there is the notion of a package-based system, > where one can download and update individual packages (rpm, apt, ipk, > etc...) as necessary. the ostensible downside to this is that, over > time, it's entirely possible that different remote systems will have > different updates applied and slowly get out of sync with one another. > > is that a big deal? well, if you're a linux person, no, since we're > used to updating packages as we see fit. however, embedded developers > who are used to installing a single, monolithic executable would argue > that the downside is that there is no single identifier for the > software currently running. if one asks, "what version of the OS are > you running?", there is no single identifier that means anything -- > you would need to examine the entire manifest of installed software. > > on the other hand, if the only supported OTA update is to replace > the entirety of the installed OS, you could theoretically version > every single possibility, but that gets cumbersome. > > i could go on, but has anyone here worked with such a system and be > willing to describe their solution? i'm just collecting possibilities > to be further debated. > > rday > > -- > > > Robert P. J. Day Ottawa, Ontario, CANADA > http://crashcourse.ca > > Twitter: http://twitter.com/rpjday > LinkedIn: http://ca.linkedin.com/in/rpjday > > > To unsubscribe send a blank message to linux+unsubscr...@linux-ottawa.org > To get help send a blank message to linux+h...@linux-ottawa.org > To visit the archives: https://lists.linux-ottawa.org > To unsubscribe send a blank message to linux+unsubscr...@linux-ottawa.org To get help send a blank message to linux+h...@linux-ottawa.org To visit the archives: https://lists.linux-ottawa.org
Re: [linux] thoughts on secure over-the-air (OTA) updates?
Robert: You should be thinking more of CI/CD. The main page at mender.io says"We have created an infrastructure that allows us to do automated unit, acceptance and integration tests on each code change we make to the product. We run tests on both virtual and physical devices." When some software change has passed its integration tests it gets deployed automatically or semi-automatically. You need to ensure that all devices get updated to match the configuration which was tested. The complexity of an rpm package management system is working against you. I would invoke the KISS principle, doing everything the simplest possible way. Also think of the term "idempotent": a service which can be called one or many times giving the same result. Yes, you could replace the entirety of the installed OS. You would tag the releases, and your git keeps track of what is in each tag. How well does this scale? Maybe well enough. But I have no skin in this game. You decide. cheers -- Rick On 2/17/20 4:39 AM, Robert P. J. Day wrote: for an upcoming project, one of the longer-term goals is to establish a secure OTA update system for potentially hundreds or thousands of remote (internet-connected) devices, which might need to be updated en masse identically, or updated individually. i'm currently collecting examples of available solutions (https://mender.io/, https://sbabic.github.io/swupdate/swupdate.html, and so on), and the current custodians of the system have already had animated discussions as to what they think would work, and what properties it should have. on the one hand, there is the notion of a package-based system, where one can download and update individual packages (rpm, apt, ipk, etc...) as necessary. the ostensible downside to this is that, over time, it's entirely possible that different remote systems will have different updates applied and slowly get out of sync with one another. is that a big deal? well, if you're a linux person, no, since we're used to updating packages as we see fit. however, embedded developers who are used to installing a single, monolithic executable would argue that the downside is that there is no single identifier for the software currently running. if one asks, "what version of the OS are you running?", there is no single identifier that means anything -- you would need to examine the entire manifest of installed software. on the other hand, if the only supported OTA update is to replace the entirety of the installed OS, you could theoretically version every single possibility, but that gets cumbersome. i could go on, but has anyone here worked with such a system and be willing to describe their solution? i'm just collecting possibilities to be further debated. rday
[linux] thoughts on secure over-the-air (OTA) updates?
for an upcoming project, one of the longer-term goals is to establish a secure OTA update system for potentially hundreds or thousands of remote (internet-connected) devices, which might need to be updated en masse identically, or updated individually. i'm currently collecting examples of available solutions (https://mender.io/, https://sbabic.github.io/swupdate/swupdate.html, and so on), and the current custodians of the system have already had animated discussions as to what they think would work, and what properties it should have. on the one hand, there is the notion of a package-based system, where one can download and update individual packages (rpm, apt, ipk, etc...) as necessary. the ostensible downside to this is that, over time, it's entirely possible that different remote systems will have different updates applied and slowly get out of sync with one another. is that a big deal? well, if you're a linux person, no, since we're used to updating packages as we see fit. however, embedded developers who are used to installing a single, monolithic executable would argue that the downside is that there is no single identifier for the software currently running. if one asks, "what version of the OS are you running?", there is no single identifier that means anything -- you would need to examine the entire manifest of installed software. on the other hand, if the only supported OTA update is to replace the entirety of the installed OS, you could theoretically version every single possibility, but that gets cumbersome. i could go on, but has anyone here worked with such a system and be willing to describe their solution? i'm just collecting possibilities to be further debated. rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday To unsubscribe send a blank message to linux+unsubscr...@linux-ottawa.org To get help send a blank message to linux+h...@linux-ottawa.org To visit the archives: https://lists.linux-ottawa.org
[linux] recommendations for online resources related to embedded security?
i am currently going over the newest openembedded/yocto project manuals, and the section "Making Images More Secure" in the dev manual: https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#making-images-more-secure refers to some really, really, *really* old online material, so i want to update those links. anyone want to recommend more current links that reflect specifically security issues WRT to embedded systems? rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday To unsubscribe send a blank message to linux+unsubscr...@linux-ottawa.org To get help send a blank message to linux+h...@linux-ottawa.org To visit the archives: https://lists.linux-ottawa.org
