Hi Antal! I've had some discussions with my friends and as a result I've changed my mind: I've published the generator as well as some docs to get you started. https://github.com/modm-io/modm-devices <https://github.com/modm-io/modm-devices>
However, as a compromise for point 1 I've made it clear what my maintenance commitments are. As for point 2 it's irrelevant to the publication of the generator. Point 3, meh, not my problem. Point 4, meh get over it. Have fun parsing, don't frustrate yourself! Niklas > On 8. Feb 2018, at 01:52, Niklas Hauser <niklas.hau...@rwth-aachen.de> wrote: > > Hi Antal! > >> I'm asking because I looked into STM32 programming with Rust recently, and >> platform libraries are starting to show up, but nothing as high level as >> XPCC. I looked into the SVD files, but I don't think that alternate function >> pins are encoded in them for example (the thing that enables the type-safe >> connect() methods). > > Have you seen @japaric's post on that topic? > http://blog.japaric.io/brave-new-io <http://blog.japaric.io/brave-new-io> > > Unfortunately, SVD files don't contain information beyond the memory map and > even worse, the SVD files are often incorrect in some minor way. > Btw, so are the official CMSIS header files that ST ships. > https://github.com/modm-io/cmsis-header-stm32/commit/1ec448711db342deb4d4bf20b670166701a38f26#r27042841 > > <https://github.com/modm-io/cmsis-header-stm32/commit/1ec448711db342deb4d4bf20b670166701a38f26#r27042841> > >> Could someone give me a small description of how the device file generation >> for STM32 works (both the one currently used in XPCC, and/or the one in >> MODM)? What I mean is, what information comes from what sources, and how is >> it put together. > > In short: The data comes out of CubeMX, which contains a bunch of XML files. > These are transformed, cross-referenced and filtered to generate these device > files. > https://github.com/modm-io/modm-devices/tree/develop/devices/stm32 > <https://github.com/modm-io/modm-devices/tree/develop/devices/stm32> > > I've held a talk about how we use these device files in modm at EmBO++17: > https://www.slideshare.net/emBO_Conference/datadriven-hal-generation > <https://www.slideshare.net/emBO_Conference/datadriven-hal-generation> > > These device files are more complete and more correct than the ones in xpcc > and have several important bug fixes (like correctly supporting STM32F1 group > remap). > I rewrote the generator from scratch to fix these issue, but unfortunately > I've made the decision not to publish it. Let me justify this. > > > 1. The internal data in ST's CubeMX is good, but since it's internal data, > the format changes and data sometimes gets removed, added, changed. > There is still a lot of manual cleanup that needs to happen. I've even sent > ST patches of their own internal data, some of which have been applied, > others not. > Unfortunately, the CubeMX data does not contain some of the most important > data either. This I have to manually extract from dozens of reference > manuals/datasheets, compare and add manually. This includes: some of STM32F1 > GPIO remap, ALL groups of memory sizes (yes!), ALL peripheral types and all > device groups. > > This is an insane amount of work, and I'm not willing to put in the hours for > this anymore than is absolutely necessary. > I fear that if I were to publish this generator and it were to become popular > (which the Rust community surrounding @japaric absolutely has the ability to > do), I'd be overwhelmed with support requests for all sorts of additional > data, regardless of how labor intensive it is to procure. > > I've spent the last 4 years looking at this data, if it's not in the above > device tree data already, it is _not_ trivial to add. An example is the clock > tree data, which exists in the raw data, but it's lacking critical data, > which again I'd have to manually add. Ultimately the CubeMX data is > incomplete and sometimes just plain wrong. > > > 2. There is a real risk of creating a pseudo-standard of assumptions from my > work, since there is no other STM32 data available elsewhere at the moment. > I've been following the Linux Device Tree community for a while now, > especially the discussion about the Device Tree Specification: > https://github.com/devicetree-org/devicetree-specification > <https://github.com/devicetree-org/devicetree-specification> > > To caricature the discussion so far: A few (truly very smart) engineers are > extrapolating their personal, anecdotal experiences of how *they* used data > for a few devices into a grand theory of how this data should be formatted > and used for *every* other device out there. And they are just simply wrong > about that. > You are wrong about what you think you know. You experiences mean nothing, > you cannot possibly know all the hardware details of thousands of devices. > ST has ~1200 devices, Atmel ~600 AVRs, and their configuration data is not as > regular as you'd like to think. > Here is an (out-of-data) and incomplete overview of ~1000 STM32 devices: > http://salkinium.com/stm32.html <http://salkinium.com/stm32.html> > > A better way would be to formulate your personal experiences as executable > assumption checkers, and run them over the raw data. > This is actually what I did in modm in the GPIO driver, because the > assumptions I made in xpcc regarding GPIO AF were wrong. > In fact they were _so_ wrong that our assumption check failed on the majority > of STM32 devices and we had to rewrite the API from scratch. > https://image.slidesharecdn.com/niklashauser-170228153235/95/datadriven-hal-generation-47-1024.jpg > > <https://image.slidesharecdn.com/niklashauser-170228153235/95/datadriven-hal-generation-47-1024.jpg> > > Now imagine a fuckup like that at device tree level, nobody is rewriting > anything. I don't want to be yelled at by Linus. > > > 3. Not a single vendor actually wants to give you this data, it makes them so > very deeply comparable to each other. Vendors are already unhappy with the > rather detailed Digikey search options. The CubeMX data was a very lucky > find, and I have not found data even remotely as extensive as this for other > vendors. > So this is an ST only solution (the AVR data is even more incomplete), and so > I refer back to point 2. > > > 4. I recently quit my job at ARM working on mbed OS and uVisor because they > didn't allow any out-of-the-box thinking on embedded software designs, like > we do in xpcc/modm. I mean, they don't even generate their linker-/startup > scripts, which is absolutely TRIVIAL to do with even the most basic data. > It's insane how much unnecessary manual labor goes into porting/maintaining > mbed OS. > My takeaway was that with the entire IoT craze, nobody in this industry takes > the time to fix the foundational issues, like HAL design, tool support, > library support (don't get me started on Newlib). Most engineers I met can't > even point out any flaws in these, they are just so used to it. > I'm still downbeat and angry about that and I need time to gain some distance > from that experience. > > I've been contributing to xpcc/modm now since 2011, that's a long time. > I'm not a student anymore, I need choose more carefully what I want to spend > time on now. > So I'm taking a step back to reorient myself a little and build up motivation > again. > There are also interesting and more lucrative opportunities in other > industries. > > > For now I recommend you use the device files as they are. You're not going to > be able to extract any more data from CubeMX without unreasonable effort. > I'll keep them updated with every quarterly release of modm. I'll reevaluate > my position if more projects start using that data and people are willing to > take up some of the maintenance effort like what has happened in xpcc. That > really is a joy to maintain, I feel like people care about this stuff :-) > Make sure to look at modm if you want to generate code/data/documentation > from this data, since it was designed for exactly that purpose. > > Or use the DeviceFile class directly: > https://github.com/modm-io/modm-devices/blob/develop/tools/device/modm/device_file.py > > <https://github.com/modm-io/modm-devices/blob/develop/tools/device/modm/device_file.py> > Usage: https://github.com/modm-io/modm/blob/develop/repo.lb#L29-L39 > <https://github.com/modm-io/modm/blob/develop/repo.lb#L29-L39> > > > Sorry for being in a bad mood today, > (sad) Niklas >
_______________________________________________ xpcc-dev mailing list xpcc-dev@lists.rwth-aachen.de https://mailman.rwth-aachen.de/mailman/listinfo/xpcc-dev
