Mr. Calianu,
> > No no no... He gets the word "real" and "time" is left
> in the FIFO.
> > They are FIFOs - First In First Out, so the real in
> realtime would come out
> > first. No reversing necessary.
> >
>
> Ehh.. never mind. I kind of braind farted on that one. Sorry :(
It happens, no big deal. You made such a point of it in your
original message that I wasn't sure if you knew how FIFOs worked. Thus my
rather, err..., basic message.
> Ok, this is where my knowledge was lacking. I didn't realize that in
> the kernel the rtf_put() would actually partially fail. My
> assumption was
> that it would go ahead and clobber anything that's already in
> the fifo.
And I learned, after reading another posters message and re-browsing
the source, that it will not partially fail. If there is not enough space
in the FIFO for the current record, it will simply fail to write with the
error code indicated in the man page.
> I only wish flow control was on option for me. Unfortunately
> it really
> isn't and I have to accept the theoretical possibility of losing data
> no matter how hard I try to avoid that fact.
The theoretical possibility always exists - someone could pull the
plug on the computer, a meteor could smash into your lab, a bug in the Linux
kernel could crash the system. It is just a question of probability. By
working with your system and the FIFO size, you can simply make it highly
unlikely to fail.
> Again, I didn't know that the kernel rtf_put() would
> partially fail and
> only write in the first 4 bytes of the word 'realtime', I
> assumed it would
> write the whole word and clobber what was alrady in the fifo.
As I am sure you konw by now, and as I wrote in an earlier e-mail, I
was mistaken. The write will simply fail with an -ENOSPC error code. I
hope this hasn't caused too much trouble.
> Yes, the above thought experiment is correct.. :)
Except that it was wrong due to the issue outlined above. The RT
process will fail on the write when you try to write more data than the FIFO
will hold.
> However, someone else told me that if you do a read() for N
> bytes in user
> land on an RTF you will actually get that contiguous chunk of
> N bytes (if
> it's available at the time) without being interrupted. Is this true?
I do not know about the "without being interrupted" part, but I do
know that you will get the bytes if the FIFO has them available. If you
need eight bytes and simply wait for eight bytes, you will be OK.
> I understand fifos. You don't need to get condescending.
Hmmm... Wasn't trying to get condescending - it was just that you
wrote the backwards stuff enough times in your original message to give me
the impression that you did not understand them. Sorry to have given any
offense.
> > 1) Make sure your user space process can consume much more
> data than your
> > realtime process can generate.
>
> You can't be sure of that, really, though.
No, but you can be pretty damned certain. If my user space program
gets 90+% of the CPU and I am writing a couple of kilobytes per second to
the disk and I have a two hundred kilobyte FIFO, I can say with, oh...
99.9999% confidence that I will never have a problem.
> The problem is that if you
> really don't want to throw away data, you can't stand around
> waiting for
> userland to read your data.
But if you have to store the data, you have a problem here.
> Because you have no guaratnees
> that userland
> will get to your data before you run out of memory (assuming
> you do things
> like dynamically allocate more and more memory as your queue
> fills). :/
There is really no way to dynamically allocate FIFO sizes. You pick
a size and go with it.
> Basically what I am saying is that flow control is not an option in my
> application...
I would never advocate flow control. That would suggest that your
realtime system is not realtime.
> because ultimately the real-world source of
> my data cannot
> stop and wait for it to be consumed.. it is just producing data at a
> steady rate with no regard for things such as task schedulers, machine
> load, CPU speed, etc...
That sounds like a realtime system to me!
> Flow control is only as good as the ultimate
> producer of the data, which in this case has no regard for
> flow control.
Where did this flow control issue come from in the first place?
> In my application it's the human heart, which really can't stop
> beating because your machine isn't ready for the data!!
Nope! Of course, most of the data on the human heart is pretty low
bandwidth, so I don't expect you will have a problem anyway.
> Again, can't guarantee that either.
Sigh... Nope, can't guarantee anything... I can, however, say with
relative certainty that you will be able to write a few kilobytes of data
per second to disk with nearly 100% reliability. If you use a big FIFO for
this, I am going to say that it will be nearly as reliable due to this as it
is due to not being struck by a meteor.
> Yes, that is true... however that only cures the symptoms for a time..
> doesn't cure the 'disease' of data loss.
Huh?!? If your RT system is producing more data than your userspace
can consume, you have a problem. Period, end of story. If not, buffering
gives the userspace program time to catch up. How much buffering is
required is left as an exercise to a person that knows something about the
problem.
> This is not an issue.
Yes is is.
> Whenever you use a library that takes a pointer
> to a struct.. or compile a module that works with kernel
> structs, you may
> run into exactly the same issue. But you don't. I really
> think this is a
> non-issue.
As long as all the libraries are current, etc. it is not an issue.
As long as everybody is using the same compiler it is not an issue. As long
as you never change architectures or stuff the data into a network, it is
not an issue. Sure, if it is gcc for the kernel stuff and gcc for the
userspace stuff you almost certainly won't have a problem. When you decide
to put a GUI on it with Python, however, things will go down hill quickly.
If what you want is quick and dirty, then write structs to the pipes. If
you want long-term bullet-proof code, I wouldn't do it that way.
> There's no difference between copying around structs in
> memory and passing
> references to them around to libraries that you didn't compile.
Agreed. And have you noticed that when people update these
libraries they occasionally tell you that you have to recompile your
userspace code? Happens all the time.
> The
> latter, noone ever warns you about.. but somehow there is concern here
> about the former.
Not true - library updates often require a recompile.
> I think the real reason people don't like to copy
> structs into transient, volatile, and otherwise untrustworthy
> things like
> fifos is that if you get a hiccup in the producer/consumer
> relationship
> you end up with bad structs which for some reason always make
> developers
> uncomfortable, as it's not trivial to detect/test the integrity of a
> struct.
Or that they have been bitten by this before. I can remember months
of pain on a system that transferred data from a PC to a Sun system and
between a .DLL written in C and PowerBuilder on the PC. Every time one guy
made a change to his code, everybody had to change things. It was a miracle
that everything was ever in synch.
> Once and for all I would like to settle this 'structs should
> not be copied
> around' issue by contacting some compiler designers and having them
> clarify matters.
OK, I'll restate my position:
Structs should not be copied around unless you know that you are
using the same (or structure alignment compatible) compiler at both ends of
the data stream. Is that better? :-)
> Well, I would agree with you completely here, except that
> really I don't
> buy that fear programmers have about struct member order being changed
> around randomly by compilers.
Nooooo, not *ordering*, but *packing* is the problem. Some
compilers and (though not with GCC apparently) with some optimization or
command line switches within a compiler will change structure packing. Some
x86 compilers like to pack structure elements on paragraph boundaries (16
bytes), others on word boundaries (4 bytes) while others pack for the
smallest size on byte boundaries. This can cause no end of problems.
** IF YOU MUST WRITE STRUCTS INTO FIFOS *** Be sure to use
sizeof(struct blah) to figure out how many bytes to write (and read at the
other end). This will not be the intuitive size.
The other problem you run into is when you assume that a data
element will be at a particular offset from the start of the struct in a
different language. Reading data stuffed into a FIFO with Tcl or Python for
a GUI is an excellent example. Things just aren't often where your
intuition would tell you they are.
Hope this helps. Sorry for the appearance of condescension, I was
just going back to basics as you seemed confused.
Regards,
Steve Cohen
--------------------------
Stephen D. Cohen
Xybion Sensor Positioning Systems
11528 53rd Street North
Clearwater, FL 33760
Voice: (727) 299-0150
Fax: (727) 299-0804
[EMAIL PROTECTED]
www.xybion.com
-- [rtl] ---
To unsubscribe:
echo "unsubscribe rtl" | mail [EMAIL PROTECTED] OR
echo "unsubscribe rtl <Your_email>" | mail [EMAIL PROTECTED]
--
For more information on Real-Time Linux see:
http://www.rtlinux.org/
