Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 17 Jan 2017, at 13:09, Peter Maydell  wrote:
> 
> losing the license info and attribution

inside the SVD files there is no license info or attribution.

the files themselves were created by the silicon vendors, but are distributed 
by ARM, inside the CMSIS Packs.

the original CMSIS license stated:

---
1.1 CMSIS DELIVERABLES
ARM hereby grants to you, subject to the terms and conditions of this Licence, 
a non-exclusive, non-transferable licence, to use and copy the CMSIS 
Deliverables for the purpose of:

(i) subject to clause 1.5(i), developing, having developed, manufacturing, 
having manufactured, offering to sell, selling, supplying or otherwise 
distributing products that comply with the CMSIS Deliverables; ...

1.5 CONDITIONS ON REDISTRIBUTION.

(i) If you distribute (directly or through your customers and authorised 
distributors) the products you have created pursuant to Clause 1.1 (i) you 
agree: (a) not to use ARM’s name, logo or trademarks to market any or all of 
the products created under Clause 1.1 (i); (b) to preserve any copyright 
notices included in the CMSIS Deliverables; and (c) to ensure your customers 
and authorised distributors comply with this Clause 1.5(i).
---

starting with CMSIS 5, ARM switched to Apache 2, exactly to relax licensing. as 
far as I can tell, there are no "patent, trademark, and attribution notices 
from the Source form of the Work" related to the SVD files.

if ARM wants these files used largely, but FSF refuses them, I think FSF has a 
problem.


regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 17 January 2017 at 10:42, Dr. David Alan Gilbert  wrote:
> Being able to read a machine description from a file at run time
> sounds reasonable, as do tools to manipulate those descriptions.
> I don't see why such a description file would be any different
> from a firmware file or config file, especially if you were to think
> of QEMU in this case more like a simulation environment
> which I think is what Liviu is trying to use it as.

Yes, I agree that's fine and I'd be happy with that. That
means reading the XML file at runtime that the user provides...

> I agree actually including/distributing those description files is a 
> completely
> different issue if they're derived from someone elses files;

...not preprocessing the XML into JSON, losing the license
info and attribution and shipping the JSON with QEMU. It's
those aspects that I think drop into the 'grey area'.

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Dr. David Alan Gilbert]

* Peter Maydell (peter.mayd...@linaro.org) wrote:
> On 9 January 2017 at 17:47, Liviu Ionescu  wrote:
> > I can't dispute this, but does't the GPLv2 requirements apply to
> > the qemu executable only? my JSON files are not compiled into the
> > binary, but are data files, read by the executable when required.
> > and anyway, the original license covers the original XML files,
> > mine are not only JSON, but are also slightly different.
> 
> I'm not a lawyer, and this isn't legal advice, but merely
> my own opinion:
>  (1) if you're reading in the data files at runtime then there's
> a spectrum between "just reading data provided by the user which
> didn't ship with QEMU, and QEMU works OK without this particular
> data" (clearly OK) and "reading data that shipped with QEMU
> and where QEMU won't work at all without it" (the data files
> are clearly part of the program as a whole and the GPL
> applies to them even if they don't happen to be binaries);
> where you stand legally probably depends on where on the
> spectrum you are.
> 
>  (2) your JSON files are a derived work from the original
> XML, so the license of those XML files (whatever it is) still
> applies to them. Making modifications or a mechanical
> transformation of the original files is not sufficient to
> allow you to discard the license...
> 
> With my project maintainer hat on, I prefer to avoid
> grey areas, because it avoids potential trouble later on.

Being able to read a machine description from a file at run time
sounds reasonable, as do tools to manipulate those descriptions.
I don't see why such a description file would be any different
from a firmware file or config file, especially if you were to think
of QEMU in this case more like a simulation environment
which I think is what Liviu is trying to use it as.

I agree actually including/distributing those description files is a completely
different issue if they're derived from someone elses files;
but that wouldn't stop you creating some simple examples (e.g.
the existing related machine types) from scratch.

Dave

> thanks
> -- PMM
> 
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 16 Jan 2017, at 21:23, Peter Maydell  wrote:
> 
> doing that is somewhere between "grey area" and
> "GPL violation".

too bad for GPL. :-(

the fact is that I'll continue to improve support for Cortex-M in my branch of 
QEMU, and the SVD definitions are part of it.

if you are not comfortable with it, no problem, it will remain a separate 
branch; by all means I'm not trying to force you to accept merging it into the 
mainstream.

if you don't mind, I'll continue to post messages regarding the progress of GNU 
ARM Eclipse QEMU to inform the other developers on the new solutions, to 
possibly save some duplicate efforts, and, hopefully, get some feedback.


regards,

Liviu



 

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 16 January 2017 at 18:59, Liviu Ionescu  wrote:
>
>> On 16 Jan 2017, at 20:17, Peter Maydell  wrote:
>>
>> "reading data that shipped with QEMU
>> and where QEMU won't work at all without it" (the data files
>> are clearly part of the program as a whole and the GPL
>> applies to them even if they don't happen to be binaries);
>
> I'm not a lawyer either, but isn't this case similar to packing
> binary modules with the Linux kernel?

Yes, and doing that is somewhere between "grey area" and
"GPL violation". Views of the kernel authors on that
topic if you want to dig into the detail:
http://www.tldp.org/HOWTO/Module-HOWTO/copyright.html
http://yarchive.net/comp/linux/gpl_modules.html

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 16 Jan 2017, at 20:17, Peter Maydell  wrote:
> 
> "reading data that shipped with QEMU
> and where QEMU won't work at all without it" (the data files
> are clearly part of the program as a whole and the GPL
> applies to them even if they don't happen to be binaries);

I'm not a lawyer either, but isn't this case similar to packing binary modules 
with the Linux kernel? 

Regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 9 January 2017 at 17:47, Liviu Ionescu  wrote:
> I can't dispute this, but does't the GPLv2 requirements apply to
> the qemu executable only? my JSON files are not compiled into the
> binary, but are data files, read by the executable when required.
> and anyway, the original license covers the original XML files,
> mine are not only JSON, but are also slightly different.

I'm not a lawyer, and this isn't legal advice, but merely
my own opinion:
 (1) if you're reading in the data files at runtime then there's
a spectrum between "just reading data provided by the user which
didn't ship with QEMU, and QEMU works OK without this particular
data" (clearly OK) and "reading data that shipped with QEMU
and where QEMU won't work at all without it" (the data files
are clearly part of the program as a whole and the GPL
applies to them even if they don't happen to be binaries);
where you stand legally probably depends on where on the
spectrum you are.

 (2) your JSON files are a derived work from the original
XML, so the license of those XML files (whatever it is) still
applies to them. Making modifications or a mechanical
transformation of the original files is not sufficient to
allow you to discard the license...

With my project maintainer hat on, I prefer to avoid
grey areas, because it avoids potential trouble later on.

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 9 Jan 2017, at 19:31, Peter Maydell  wrote:
> 
> FSF's view is that Apache 2.0 is not compatible with GPLv2:

I can't dispute this, but does't the GPLv2 requirements apply to the qemu 
executable only? my JSON files are not compiled into the binary, but are data 
files, read by the executable when required. and anyway, the original license 
covers the original XML files, mine are not only JSON, but are also slightly 
different.


regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 9 January 2017 at 16:51, Liviu Ionescu  wrote:
>> On 9 Jan 2017, at 18:16, Peter Maydell  wrote:
>> The obvious issue that comes to mind
>> is licensing, though -- what licenses are the SVD files under,
>> and would those be compatible with QEMU's GPLv2 ?
>
> that's a common question with cmsis files. I could not find
> any specific license applying to svd files, so, since these
> files are provided as part of the arm cmsis packs, I would
> consider the generic license, which, starting with the latest
> cmsis release, was changed to Apache.

The FSF's view is that Apache 2.0 is not compatible with GPLv2:
https://www.gnu.org/licenses/license-list.html#apache2

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 9 Jan 2017, at 18:16, Peter Maydell  wrote:
> 
> On 28 December 2016 at 18:49, Liviu Ionescu  wrote:
>> The latest release of GNU ARM Eclipse QEMU (2.8.0-20161227) introduced
>> a new technology for implementing peripherals, based on standard
>> CMSIS SVD definitions
>> (http://www.keil.com/pack/doc/CMSIS/SVD/html/index.html).
>> 
>> The SVD files are large XML files produced by the silicon vendors,
>> and generally are considered the final hardware reference for
>> the Cortex-M devices, so they are expected to provide the most
>> accurate peripheral emulation.
> 
> This certainly seems like a good idea in principle, if there's
> enough information in the SVD files to reliably create a
> complete board model.

the general answer is that this information should be there.

in practice, it mostly is. however, the SVD specs are quite lax and each vendor 
understands them slightly differently.

to compensate for these differences, my workflow added an extra step of 
patching the original content. the patch files are minimal compared to the 
original files, and the effort to create them is perfectly acceptable.

> The obvious issue that comes to mind
> is licensing, though -- what licenses are the SVD files under,
> and would those be compatible with QEMU's GPLv2 ?

that's a common question with cmsis files. I could not find any specific 
license applying to svd files, so, since these files are provided as part of 
the arm cmsis packs, I would consider the generic license, which, starting with 
the latest cmsis release, was changed to Apache.

anyway, in my opinion, after experimenting both with classical (switch based) 
implementations and table based (peripheral/register/bitfield) implementations, 
if I would have to deal with a device which does not have the vendor SVD files, 
I would probably prefer to write the JSON file by hand and allow the automated 
tools to handle the rest.

actually I'll have to do this when I'll rewrite the Cortex-M system 
peripherals, not included in the vendor provided SVDs.


regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 28 December 2016 at 18:49, Liviu Ionescu  wrote:
> The latest release of GNU ARM Eclipse QEMU (2.8.0-20161227) introduced
> a new technology for implementing peripherals, based on standard
> CMSIS SVD definitions
> (http://www.keil.com/pack/doc/CMSIS/SVD/html/index.html).
>
> The SVD files are large XML files produced by the silicon vendors,
> and generally are considered the final hardware reference for
> the Cortex-M devices, so they are expected to provide the most
> accurate peripheral emulation.

This certainly seems like a good idea in principle, if there's
enough information in the SVD files to reliably create a
complete board model. The obvious issue that comes to mind
is licensing, though -- what licenses are the SVD files under,
and would those be compatible with QEMU's GPLv2 ?

thanks
-- PMM

[Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

CMSIS SVD
-

The latest release of GNU ARM Eclipse QEMU (2.8.0-20161227) introduced a new 
technology for implementing peripherals, based on standard CMSIS SVD 
definitions (http://www.keil.com/pack/doc/CMSIS/SVD/html/index.html).

The SVD files are large XML files produced by the silicon vendors, and 
generally are considered the final hardware reference for the Cortex-M devices, 
so they are expected to provide the most accurate peripheral emulation.

For convenience, the original SVD files are converted to JSON files, which are 
generally easier to parse. 

There is one file for each sub-family; the files currently used by GNU ARM 
Eclipse QEMU are grouped in the devices folders:


https://github.com/gnuarmeclipse/qemu/tree/gnuarmeclipse-dev/gnuarmeclipse/devices


Please note that some devices may include multiple instances of similar 
peripherals, for example multiple timers, with each instance slightly different 
from the others. The SVD files include all these differences, so strictly 
following the content of the SVD files is mandatory; extracting a definition 
common to all instances may seem attractive, but it is not realistic, since it 
may not be accurate for all instances.


Peripheral/register/bitfield


The basic objects used to implement peripherals are the 'registers'; a 
peripheral is actually an array of registers, each with its value.

For convenience, registers can be viewed as collections of bitfields; bitfield 
objects do not have their own values, reading a bitfield refers to the parent 
register, which is masked, shifted and finally returned.

Accessing bitfields is quite straightforward:

const char *enabling_bit_name = "/machine/mcu/stm32/RCC/AHB1ENR/GPIOAEN";
state->enabling_bit = OBJECT(cm_device_by_name(enabling_bit_name));
// ...
if (register_bitfield_is_non_zero(state->enabling_bit)) {
  // ...
}


Generated code
--

In addition to using vendor SVD files, GNU ARM Eclipse QEMU goes one step 
further, by generating most of the peripheral code, to the point that new 
peripherals can be added simply be adding the generated files to the project.

Examples of the files currently used are in sub subfolders of the 
devices/support folder:


https://github.com/gnuarmeclipse/qemu/tree/gnuarmeclipse-dev/gnuarmeclipse/devices/support


As per the current STM32 implementation, to avoid redundancy, each peripheral 
file includes definitions for all families; adding a new device implies 
generating the code for the new device sub-family, and copy/paste-ing the 
required code in the peripheral implementation.

An example of such a peripheral implementation is the SYSFCG:


https://github.com/gnuarmeclipse/qemu/blob/gnuarmeclipse-dev/hw/cortexm/stm32/syscfg.c



Supported devices
-

GNU ARM Eclipse QEMU 2.8 supports the following boards:

  MapleLeafLab Arduino-style STM32 microcontroller board (r5)
  NUCLEO-F103RBST Nucleo Development Board for STM32 F1 series
  NUCLEO-F411REST Nucleo Development Board for STM32 F4 series
  NetduinoGo   Netduino GoBus Development Board with STM32F4
  NetduinoPlus2Netduino Development Board with STM32F4
  OLIMEXINO-STM32  Olimex Maple (Arduino-like) Development Board
  STM32-E407   Olimex Development Board for STM32F407ZGT6
  STM32-H103   Olimex Header Board for STM32F103RBT6
  STM32-P103   Olimex Prototype Board for STM32F103RBT6
  STM32-P107   Olimex Prototype Board for STM32F107VCT6
  STM32F0-DiscoveryST Discovery kit for STM32F051 lines
  STM32F4-DiscoveryST Discovery kit for STM32F407/417 lines
  STM32F429I-Discovery ST Discovery kit for STM32F429/439 lines

Supported MCUs:
  STM32F051R8
  STM32F103RB
  STM32F107VC
  STM32F405RG
  STM32F407VG
  STM32F407ZG
  STM32F411RE
  STM32F429ZI

Functionally, the boards include animated LEDs and active push buttons (reset 
and user).


Test projects (blinky) for all supported boards are available from

https://github.com/gnuarmeclipse/eclipse-qemu-test-projects


Conclusion
--

Although implementing peripherals remains a challenge, using the SVD 
definitions, plus the tools to generate code, significantly improve and 
simplify the process.

Unfortunately, SVD files are available only for Cortex-M devices, but I think 
that, when not available, creating the JSON files and using the automated code 
generator is still easier than manually implementing the peripherals.


Personally I plan to use this technology to re-define the system Cortex-M 
devices, which, right now, are improperly implemented.


Regards,

Liviu