Re: [riot-users] RIOT OS for data logger system with focus on maintenance
Hi Daniel, see inline some answers and ideas for your use case. Kind regards, Marian On Fri, 30 Oct 2020 17:37:04 +0100 Daniel Beiter wrote: > Dear RIOT OS community, > > I'm Daniel a PhD in Hydrology at GFZ Potsdam and currently working on a > data logger system for use in extensive field campaigns. As far as I > have understood, RIOT OS seems to be very suitable for the tasks I have > in mind. Since I am new to using an OS on microcontrollers (rather than > running simple single-thread applications), I would very much like to > discuss with a RIOT expert (or several) what is possible and what is an > illusion of my perception. Here are some points to give you a bit of an > idea what I mean: > > * Adding pre-compiled sensor driver packages to a running application > Let's say we have 20 stations running but with varying numbers and > types of sensors (e.g. sensors monitoring trees only in forests, > discharge sensors only at rivers, etc.). Is it possible to have a > common application running on all stations and adding/removing > pre-compiled sensor driver packages to change sensor setup? It would > allow easier on-site maintenance, where fast adding/removing sensors > would not lead to changing source code and compiling a new > application for one particular station (maintenance of course always > happens in winter, at night, with strong winds, heavy rain, in a > remote forest, hard to work on source code :) ). It would certainly be possible to use a common firmware that targets the "full-fledged" node, but flash it on nodes with fewer sensors plugged in. Our goal is that drivers perform during initialization a self check (usually read out manufacturer and device ID and compare it against the expected values) - if no such sensor is present, the initialization should return with an error code. So my proposal is to just let the firmware detect upon start up which sensors are present. Sadly, you will likely not be able to use auto initialization with that approach, as you really want to check the return value of the initialization to verify if the sensor is available and functional. (Well, you could use SAUL with auto-initialization, as only successfully initialized sensors should be registered in the SAUL registry.) Note however, that it is pretty much impossible to detect hardware configuration at runtime due to the plethora of different and incompatible communication standards and the lack of any "plug and pray" protocols. So my suggestion would only work if for each sensor the configuration is known at compile time and each sensor can either be present in this very configuration, or being unconnected. But I fear that the test coverage for this scenario is not too well, so be prepared to file a few bug reports regarding the self checks. (I expect some drivers to still initialize even without hardware present, and some other drivers to infinitely loop while trying to communicate with non-existing hardware.) > * Having a web interface running, accessible via Ethernet or BLE (for > on-site maintenance) and remote (e.g. via LTE CAT M1/NB1 for > administration) to allow configuring sensor setup (number and type) > as well as general configuration (measurement interval, time of data > upload, etc.) > A simple graphical web interface would ease the maintenance for the > technician as he would "only" need to change configurations rather > than changing source code and compiling (see point above). Since I > am working currently with OpenMediaVault, I really appreciate their > web interface concept, with the possibility to revert all currently > made changes. That would certainly be possible to implement on top of RIOT. I'm unaware of anyone running an HTTP server on RIOT, but that shouldn't be too difficult to achieve. However, maybe a CoAP server would be a better option? By reusing the core concepts of HTTP based REST APIs (same message-response model, URIs, content formats (lightweight MIME Types), ...) CoAP can be trivially reached from HTTP by using cross protocol proxies that transparently translate between HTTP and CoAP. And you might be able to use the same CoAP interfaces for both human 2 machine maintenance scenarios as well regular m2m communication. Btw: lwm2m could also be something you might want to check out as an option for device management. > * All these configurations would need to be stored locally, I imagine > as JSON on an EEPROM with some kind of file system. Is this how it > is done in general? Usually the overhead of file systems is considered to be too huge for EEPROM devices. RIOT has the EEPROM registry module that works well as a low-overhead key value storage - which could be viewed as a primitive non-hierarchical file system. That should actually be quite a good match for storing configuration data. If you want to go with an actual file system, you're likely better off
Re: [riot-users] RIOT OS for data logger system with focus on maintenance
* Having a web interface running, accessible via Ethernet or BLE (for on-site maintenance) and remote (e.g. via LTE CAT M1/NB1 for administration) to allow configuring sensor setup (number and type) as well as general configuration (measurement interval, time of data upload, etc.) A simple graphical web interface would ease the maintenance for the technician as he would "only" need to change configurations rather than changing source code and compiling (see point above). Since I am working currently with OpenMediaVault, I really appreciate their web interface concept, with the possibility to revert all currently made changes. Okay, that point is a bit tricky. At least tricky, if you want to have this web interface directly on the Microcontroller. I would use CoAP for this use-case. It makes the most sense. But there is one issue: Normal browser, like Mozilla, Chrome etc. don't support CoAP by default. There is Copper(https://github.com/mkovatsc/Copper), but as you can see it is discontinued. You can use an old browser, but I guess your point was to use a modern browser, right? There are other options as well, of course. LibCoAP is well supported. So, you can use it in a programming language of your choice. Every major language has some kind of CoAP lib. So, you could provide an CoAP interface you can interact with. Writing a CLI for it, shouldn't be a big deal. From the remote point of view: Just host a central management frontend which interacts with the devices. Collecting configs, collecting data etc. You probably also have to use edge device/border router/gateway in your setup. So, you could also put an instance of this web interface there as well. So that you have a web interface on-site as well. But having a CLI backup is probably also a good idea in case the web interface crashes. You can also have it more decentralized and let the edge device collect the data in some way. There is room for different architecturial decisions. *All these configurations would need to be stored locally, I imagine as JSON on an EEPROM with some kind of file system. Is this how it is done in general? Not sure, if JSONs is the best idea here, since you have to parse the JSON on the MCU. Using a filesystem in RIOT is quite simple. So, you can store it in files. When you call the API, you change the value in memory and on the file system. When the device wakes up it loads the configuration from the filesystem. I would also like to hear the input from other people, since I didn't had dynamic configuration yet. *All configurations should also be synced with the remote server (e.g. running ODM2) Just provide an API for this at your devices. So that you can change the configuration remotly. Of course, this should be done in a secured way. Maybe it makes sense to use Web of Things (https://www.w3.org/WoT/) in your use-case. Especially considering you want to collect all these data, change the configuration etc. There is the WoT Scripting API which makes it quite simple to interact with the device and write a web frontend for it, once the WoT TD is present. Node-WoT(https://github.com/eclipse/thingweb.node-wot) is quite an advanced implementation of it. I wrote a PoC implementation for RIOT (https://github.com/w3c/wot-testing/tree/master/events/2020.09.Online/prototypes/RIOT-OS). If that sounds interesting to you, just ping me. I could imagine that you write some generic plugin/extension for ODM2 and support WoT TD on it. So that you can provide an interoperability layer. Eventually you could also replace your device with commercial available ones. They just have to support the WoT TD. best regards Philipp ___ users mailing list users@riot-os.org https://lists.riot-os.org/mailman/listinfo/users
Re: [riot-users] RIOT OS for data logger system with focus on maintenance
On Fri, 30 Oct 2020 17:37:04 +0100 Daniel Beiter wrote: > * Adding pre-compiled sensor driver packages to a running > application Let's say we have 20 stations running but with varying > numbers and types of sensors (e.g. sensors monitoring trees only in > forests, discharge sensors only at rivers, etc.). Is it possible to > have a common application running on all stations and adding/removing > pre-compiled sensor driver packages to change sensor setup? It > would allow easier on-site maintenance, where fast adding/removing > sensors would not lead to changing source code and compiling a new > application for one particular station (maintenance of course > always happens in winter, at night, with strong winds, heavy rain, in > a remote forest, hard to work on source code :) ). An easier solution would be to just compile in all your sensor drivers into one image. Usually drivers will have a way to detect if a sensor is connected (and if not, it should be added). When you access sensors via SAUL you give it a sensor type and if a matching sensor present and initialized correctly, it will be used to get you the data. You can of course do OTA updates of the whole firmware using SUIT. > * Having a web interface running, accessible via Ethernet or BLE > (for on-site maintenance) and remote (e.g. via LTE CAT M1/NB1 for > administration) to allow configuring sensor setup (number and > type) as well as general configuration (measurement interval, time of > data upload, etc.) > A simple graphical web interface would ease the maintenance for > the technician as he would "only" need to change configurations rather > than changing source code and compiling (see point above). Since I > am working currently with OpenMediaVault, I really appreciate > their web interface concept, with the possibility to revert all > currently made changes. Consider using an App that runs on a phone and talks to the sensor using Bluetooth Low Energy (BLE). If you want to use IP, we have a CoAP server in RIOT (although it should be possible to run a HTTP server using LwIP) and there are CoAP <-> HTTP bridges. Again consider running most of the logic on the client. > * All these configurations would need to be stored locally, I > imagine as JSON on an EEPROM with some kind of file system. Is this > how it is done in general? Unless you want to store the data on an SD card for easy reading/editing on a PC, you'd probably much rather use a binary protocol like ProtoBuf or CBOR. And since wear-leveling flash file systems like littlefs2 and spiffs exist, you can also just use Flash memory instead of an EEPROM. > * All configurations should also be synced with the remote server > (e.g. running ODM2) Mind you that the nodes won't do TCP. You can run a CoAP server and have the nodes connect to that to upload data / check for new configurations. What you then use in the backend is up to you. Best, Benjamin ___ users mailing list users@riot-os.org https://lists.riot-os.org/mailman/listinfo/users
Re: [riot-users] RIOT OS for data logger system with focus on maintenance
I find it interesting enough to keep this discussion on the open list. El 30 de oct. de 2020 13:37 -0300, RIOT OS users list , escribió: > > I'm Daniel a PhD in Hydrology at GFZ Potsdam and currently working on a data > logger system for use in extensive field campaigns. As far as I have > understood, RIOT OS seems to be very suitable for the tasks I have in mind. > Since I am new to using an OS on microcontrollers (rather than running simple > single-thread applications), I would very much like to discuss with a RIOT > expert (or several) what is possible and what is an illusion of my > perception. Here are some points to give you a bit of an idea what I mean: > > > • Adding pre-compiled sensor driver packages to a running application > Let's say we have 20 stations running but with varying numbers and types of > sensors (e.g. sensors monitoring trees only in forests, discharge sensors > only at rivers, etc.). Is it possible to have a common application running on > all stations and adding/removing pre-compiled sensor driver packages to > change sensor setup? It would allow easier on-site maintenance, where fast > adding/removing sensors would not lead to changing source code and compiling > a new application for one particular station (maintenance of course always > happens in winter, at night, with strong winds, heavy rain, in a remote > forest, hard to work on source code :) ). > • Having a web interface running, accessible via Ethernet or BLE (for on-site > maintenance) and remote (e.g. via LTE CAT M1/NB1 for administration) to allow > configuring sensor setup (number and type) as well as general configuration > (measurement interval, time of data upload, etc.) > A simple graphical web interface would ease the maintenance for the > technician as he would "only" need to change configurations rather than > changing source code and compiling (see point above). Since I am working > currently with OpenMediaVault, I really appreciate their web interface > concept, with the possibility to revert all currently made changes. > • All these configurations would need to be stored locally, I imagine as JSON > on an EEPROM with some kind of file system. Is this how it is done in general? > • All configurations should also be synced with the remote server (e.g. > running ODM2) > > > I hope these points make some sense even if the answer is: No, what you want > is not possible. And if not, I am happy to learn more. As mentioned before, I > would appreciate a discussion about this use case and the potential use of > RIOT OS with experienced people who find it interesting. In case spamming the > user list is not desired you can of course contact me via my email address > daniel.bei...@gmx.at ___ users mailing list users@riot-os.org https://lists.riot-os.org/mailman/listinfo/users
[riot-users] Release 2020.10
|And hopefully this time with newlines: Dear RIOTers, We are happy to announce the 25th official release of RIOT: --- * RIOT 2020.10 * --- | |The 2020.10 release includes: - Support for PicoLIBC as standard C library implementation - A new radio abstraction layer for ieee802.15.4 radios - Improvement to the linking of modules - An improved algorithm for generating local unique identifiers - Pagewise addressing support for MTD devices - 23 additional modules supported by Kconfig - Initial rework of the clock modelling on stm32 devices - 5 new boards, 2 new device drivers, 7 packages updated 482 pull requests, composed of 1355 commits, have been merged since the last release, and 84 issues have been solved. 64 people contributed with code in 94 days. 2426 files have been touched with 133358 (+) insertions and 756906 deletions (-). ||You can download the RIOT release from Github by cloning the repository and checkout the release tag [1] or by downloading the tarball [2], and look up the release notes for further details [3]. A big thank you to everybody who contributed! Best regards, Koen Zandberg [1]:https://github.com/RIOT-OS/RIOT/tree/2020.10 [2]:https://github.com/RIOT-OS/RIOT/archive/2020.10.tar.gz [3]:https://github.com/RIOT-OS/RIOT/releases/tag/2020.10| || OpenPGP_0x69568F2BF8114A27.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature ___ users mailing list users@riot-os.org https://lists.riot-os.org/mailman/listinfo/users
[riot-users] Release 2020.10
|Dear RIOTers, We are happy to announce the 25th official release of RIOT: --- * RIOT [2020.10] * --- The 2020.10 release includes: - Support for PicoLIBC as standard C library implementation - A new radio abstraction layer for ieee802.15.4 radios - Improvement to the linking of modules - An improved algorithm for generating local unique identifiers - Pagewise addressing support for MTD devices - 23 additional modules supported by Kconfig - Initial rework of the clock modelling on stm32 devices - 5 new boards, 2 new device drivers, 7 packages updated ||482 pull requests, composed of 1355 commits, have been merged since the last release, and 84 issues have been solved. 64 people contributed with code in 94 days. 2426 files have been touched with 133358 (+) insertions and 756906 deletions (-). You can download the RIOT release from Github by cloning the repository and checkout the release tag [1] or by downloading the tarball [2], and look up the release notes for further details [3]. A big thank you to everybody who contributed! Best regards, Koen Zandberg [1]:https://github.com/RIOT-OS/RIOT/tree/2020.10 [2]:https://github.com/RIOT-OS/RIOT/archive/2020.10.tar.gz [3]:https://github.com/RIOT-OS/RIOT/releases/tag/2020.10| || OpenPGP_0x69568F2BF8114A27.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature ___ users mailing list users@riot-os.org https://lists.riot-os.org/mailman/listinfo/users
[riot-users] RIOT OS for data logger system with focus on maintenance
Dear RIOT OS community, I'm Daniel a PhD in Hydrology at GFZ Potsdam and currently working on a data logger system for use in extensive field campaigns. As far as I have understood, RIOT OS seems to be very suitable for the tasks I have in mind. Since I am new to using an OS on microcontrollers (rather than running simple single-thread applications), I would very much like to discuss with a RIOT expert (or several) what is possible and what is an illusion of my perception. Here are some points to give you a bit of an idea what I mean: * Adding pre-compiled sensor driver packages to a running application Let's say we have 20 stations running but with varying numbers and types of sensors (e.g. sensors monitoring trees only in forests, discharge sensors only at rivers, etc.). Is it possible to have a common application running on all stations and adding/removing pre-compiled sensor driver packages to change sensor setup? It would allow easier on-site maintenance, where fast adding/removing sensors would not lead to changing source code and compiling a new application for one particular station (maintenance of course always happens in winter, at night, with strong winds, heavy rain, in a remote forest, hard to work on source code :) ). * Having a web interface running, accessible via Ethernet or BLE (for on-site maintenance) and remote (e.g. via LTE CAT M1/NB1 for administration) to allow configuring sensor setup (number and type) as well as general configuration (measurement interval, time of data upload, etc.) A simple graphical web interface would ease the maintenance for the technician as he would "only" need to change configurations rather than changing source code and compiling (see point above). Since I am working currently with OpenMediaVault, I really appreciate their web interface concept, with the possibility to revert all currently made changes. * All these configurations would need to be stored locally, I imagine as JSON on an EEPROM with some kind of file system. Is this how it is done in general? * All configurations should also be synced with the remote server (e.g. running ODM2) I hope these points make some sense even if the answer is: No, what you want is not possible. And if not, I am happy to learn more. As mentioned before, I would appreciate a discussion about this use case and the potential use of RIOT OS with experienced people who find it interesting. In case spamming the user list is not desired you can of course contact me via my email address daniel.bei...@gmx.at Enjoy your weekend and stay safe Daniel ___ users mailing list users@riot-os.org https://lists.riot-os.org/mailman/listinfo/users
