I assume you have suffered through the precursor to this writeup, 'a
mini-tutorial on the flow of status information in LinuxCNC'.
--
Before I start, let me point out the following facts in the example I used to
describe the flow of status information:
- we started with a HAL pin
- we ended up with a Python attribute.
Now both are 'late binding' - most of you are used to defining HAL pins in
components, and how pins are glued together at _startup time_. Also many of you
are familiar with Python programming; that isnt compiled either - reedit,
reexecute, done.
Now remember the steps from motion 'probe-input' to Python 'probe_input'. We
went through 2 data structures - emcmot_status_t and EMC_MOT_STAT to convey
that single bit. Unfortunately, these data structures are compiled with the C
and C++ compilers respectively - that's called 'early binding' and it
translates into the following limitation:
The type and layout of information bits which can be conveyed through an
NML message is fixed at compile time.
So no matter how flexible HAL or Python is - there you go: either them LinuxCNC
gurus have forethought your use case for an extra pin value in a NML message,
or you're out on the limb. Can't be done without without several nontrivial
changes here and there, and it's pretty hard to get right first time, nevermind
having it supported in mainline code. Thats why nobody does it, and it is also
the reason why emcmot_status_t and EMC_MOT_STAT have become the kitchen sink
of status information: just in case somebody might need it, keep it there.
Result: several incomprehensible whales of datastructures.
Now assume a non-LinuxCNC application of HAL/RTAPI for a moment - just for the
sake of argument, assume the prototypical 'garden sprinkler' application. Its
is safe to assume that the user will find HAL to be super-flexible, but be less
than enthusiastic about the set of NML messages provided to talk to it, and
report status from it - assuming the sprinkler controller is some headless
board, maybe a Beaglebone. Trajectory status doesnt make a whole lot of sense
when reporting whether a faucet is open or closed.
Standing back a bit, it's like we've just shot ourselves in the foot big time:
we have super-flexible languages for GUI's, we have HAL and its stunning
arsenal of components, and we have a veritable straightjacket of a limitation
when communicating between the two. No wonder folks hit upon the idea they
could circumvent the NML restrictions by including HAL access in UI's, and
thats where we are with PyVCP, and GladeVCP: we have a very restricted
mechanism to communicate which is in principle network capable, and a much more
generic mechanism which is not network capable, and we have _both of them in
parallel_, inheriting the union of restrictions derived from both.
I'm sorry, this cannot be it. So we arrive at requirement #1:
Any future NML replacement should be late-binding capable.
That means that message contents become subject to integrator definition and
extension, as opposed to guru discretion. Note that I'm not suggesting
everybody _has_ to do this; to the contrary, common message types will surely
be held in some library, and reused. It is all about enabling it, not forcing
it.
----
Let's look for a moment what type of information is reported from HAL
components to task and other using code: it turns out
It is all scalar values: floats, signed ints, unsigned ints, booleans.
(there are some minor exceptions like the gcode filename which are not relevant
here).
That maps perfectly on the HAL primitive datatypes. However, an essential
feature of NML is to arrange single scalar values into a compound structure,
like EMC_MOTION_STAT or EMC_IO_STAT, and to report those structures 'in one
go'. HAL as it stands doesnt support that.
So we arrive at requirement #2:
Any future NML replacement must support grouping of scalars into compound
messages.
That brings the question: how and where is that grouping done? In essence you
need a way of expressing 'I want message X to contain values A, B and C, which
are of type int, float and bool'. That's a typical language feature: C does
it, C++ does it, Python does it, Google Protobufs does it, unfortunately
HAL/halcmd does not.
So the options are: a) use an existing language to do the grouping b) extend
HAL to support grouping.
a) has been the NML approach, with the consequences outlined in the previous
article. Whatever the language: it is a second language, which must be learned
and understood, and brings its own set of restrictions.
b) is new and requires some thought.
If route b) is chosen, it would be wise to get away with the very minimum bloat
of HAL per se, and do the rest elsewhere - in using code, like components. So
lets think though this a bit.
Object grouping in HAL
----------------------
In HAL right now we have the following objects: pins, signals, params, threads,
modules, thread functions.
Assume we add to the above two new objects: a 'group', and a 'member' object -
both of them having a HAL name, a type, and several attributes, just like the
rest of them.
HAL groups would have the following attributes:
- a name and a uniqe integer ID
- some, say 2, integer attributes, which are not interpreted by HAL per se but
are for 'using code only'.
- an ordered list of members.
A HAL member would have the following attributes:
- any member is a part of a single given group.
- a member refers to a HAL signal, which must exist at the time when the member
is added to a group.
- again, say 2, attributes; an integer and a float value; again not interpreted
by HAL per se but by using code only.
To support groups and members, the HAL C library needs a bit of extension -
adding, deleting or iterating over groups; adding and deleting members.
Also halcmd needs to understand the new objects, and be able to display them.
Smells like a spec, so let's do
A simple usage example: defining a group in halcmd:
---------------------------------------------------
We use 'monitoring a battery-backed power supply' as example scenario.
Assume a meaningful status report for such a power supply would contain:
current voltage, current load, is the mains fuse blown/OK, are we're running on
battery, or on mains.
Now let's translate that into a HAL group:
----- extended halcmd ----
# declare the signals - these must exist before using them in a member
newsig volt float
newsig amps float
newsig mains bit
newsig fuse-ok bit
# here we would link any pins reporting actual values from the power supply
# to the signals declared at the top
net volt meter.voltage
# etc etc
# declare named group 'power-suply'
# groups have a unique ID (23 below), plus two optional userargs (10 and 1000).
# HAL doesnt care about userargs, except they be integer
newg power-supply 23 10 1000
# add some members to this group:
# note members can have an optional int and a float userargs parameter
# HAL ascribes no meaning to them
newm power-supply volt 1 0.1
newm power-supply amps
newm power-supply mains 1
newm power-supply fuse-ok 1
----------------------------
Great, so what does this gain us? Nothing except we now have a named HAL object
'power-supply' which has several member signals. It's a group definition,
nothing else - HAL per se will not do anything with it. We added some int and
float params to group and members. Not much more.
That's why the support code in HAL and halcmd for dealing with groups and
members is fairly minimal. This turns us to
How is a group used?
--------------------
There are two users of the HAL group code:
- halcmd - to define a group and members, as outlined above.
- one or several HAL components: to actually do something useful with a group:
the HAL library would provide a few new functions:
- iterate of the list of groups, retrieve a group name and its parameters
- likewise, iterate over members of a group, and retrieve their names and
parameters.
Now let's assume we would design a 'status reporter' for our power supply. What
that would do is inspect a HAL group periodically, and send a status update
message to some interested entity, maybe a GUI.
That reporter could well be a userland HAL component. The outline would look
like so:
- normal HAL initialisation. Nothing new here.
- ask HAL for group 'power-supply', retrieving group parameters.
- walk the list of group members, retrieving each member parameter.
- do something useful periodically, based on the above, running until shutdown.
at this point, the report component can ascribe some meaning to group and
member parameters. For instance, report could:
- interpret group parameter 1 (1000) as a periodic update timer - meaning a
status update message should be sent unconditionally every second.
- interpret group parameter 2 (10) as a an internal change detection timer -
meaning members are checked as follows:
-- default member parameter - zero - means no periodic check of this member
signal
-- member parameter '1' means - include in periodic check
---- in the case of the float 'volt' value, the float param '0.1' is used as an
epsilon value to trigger the 'this value has changed' condition - float
equality comparisons are notoriously unreliable and an epsilon parameter deals
with that issue.
Note that the behavior of the report component - timers, what to check for etc
- is completely driven by the HAL group and member attributes, so there is
actually very few code which is specific to the 'power supply' theme, in fact
its completely generic.
The behavior would be as follows:
- every second, a status update is sent no matter what.
- if any of: mains goes away, the fuse is blown, or the voltage changes by more
than 0.1 volts a status update is sent after the next 10msec internal poll
interval.
Also, note a single component can report for an arbitrary number of groups 'in
one go'.
The only thing left here is 'what happens when a change is detected'?
Well, a HAL status update message is sent, by a mechanism which I already
discussed - using zeroMQ, and - for a start - the PUBLISH/SUBSCRIBE mechanism
to any interested readers - for instance HAL widgets in a GUI.
I leave the details how such an update message is sent with ZeroMQ and
protobufs for a separate installment of the 'How to break LinuxCNC effectively'
miniseries; but be it enough here that even at that level, the group mechanism
is very useful:
- the member list drives the encoding of values in the protobuf serialisation.
The message format need not know about the power supply fields, types and so on
- see requirement #1.
- the group name can be used as a ZeroMQ channel name - interested consumers
subscribe to the 'power-supply' ZMQ channel, and get updates - done.
In summary:
HAL groups and members support the definition of messages, with a late binding
approach, and with a fairly minimal extension to HAL per se, and no
interpretation or using code within the HAL library whatsoever. Any using code
will be components which are group/member aware.
Great. Now how's that going to help with dumping NML and replacing EMC_STAT
with this grandiose scheme?
-------------------------------------------------------------------------------------------------------
Well, really not much different like the above group definition, just longer.
All the needed types are there.
Nested structures can be dealt with by HAL naming - for instance
"status.traj.probe_tripped" conveys the 'object nesting', even if its just a
HAL name in a flat namespace.
The only thing which is not very well covered is arrays, for instance the array
of axes, or array of analog/digital I/O pins used in the interpreter. That has
it's reason in the fact that HAL doesnt support an array notation. I dont see
that as a reason to add one - that doesnt bring enough upside IMO. The issue
can be dealt by rolling the index into the HAL name - for instance,
'status.digital-in-04' and thats' not different from current practice.
So the above lets us define the equivalent of EMC_STAT with HAL groups.
So how's the migration of motion and iocontrol going to happen?
---------------------------------------------------------------
Unfortunately, motion and iocontrol dont have a clue about groups and members
right now, so how's that going to work?
The answer to this is: they dont need to know zilch about groups or members,
that 'need to know' is confined to the reporting mechanism.
What is needed, however, is:
--> each scalar which is to be reported through the group mechanism needs to
be exported as a HAL pin so it can be linked to a member signal <----
Note that many members of emcmot_status_t are already exported as pin, so
there's nothing new here, and also the breakage potential is limited: the old
scheme can remain in place, and run in fact in parallel, while the new needed
pins are added and status reporting is defined. Eventually we can put EMC_STAT,
and emcmot_status_t to rest on the old code burial grounds.
---
Please let me know if where above makes any sense, or not at all, where it is
unclear, and point me to out outright errors and omissions.
- Michael
ps: for the very curious: I did a proof-of-concept of groups and members,
hal_lib and halcmd support, as well as a generic reporter with ZMQ messaging
and protobufs here:
http://git.mah.priv.at/gitweb?p=emc2-dev.git;a=tree;f=src/hal/halreport;hb=4259166865c681017b6009f041ac4d1ed5e16439
This is not anything mergeable, and way too bloated - it supports a group
notification mechanism from kernel modules which isnt needed, and lots of other
fluff which will got away. But it gives a rough effort indication: v0.001 of
the generic HAL reporter component is less than 1000 lines; the hal_lib.c
group/member support much less than that.
A very basic Python subscriber to the reporter's channel is here:
http://git.mah.priv.at/gitweb?p=emc2-dev.git;a=blob;f=src/hal/halreport/sub.py;h=85ffeae0e4a576755aaab186d7378b29cdeca75d;hb=4259166865c681017b6009f041ac4d1ed5e16439
------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
Emc-developers mailing list
Emc-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/emc-developers
