Re: Which delimiter to use for TCP functions?

2004-02-08 Thread David Ferster
Eric,

Another way to approach this (and one that I use without trouble) is 
to send the data as ASCII text. Since your data are all numeric, you 
can then use a non-numeric character as your delimiter. I send 
strings like

[abcde 0.02 1233.4 12 3.14]

where abcde is a fixed length code that tells what kind of data are 
to follow. The receiver then just formats these strings back into 
numbers. Since you are not sending binary data, there is never, ever 
a possibility that a part of a flattened string will mimic the stop 
and start codons of your messages (in this case [ and ]).

The downside is that if you are sending large streams of 
high-precision data, the number of bytes required for each message 
increases, and therefore the speed of transmission potentially slows. 
So whether you can use this will depend on how large your arrays are 
and how fast/busy your connection is.

David



 Hi all, maybe someone has run into this problem before:
 I have a program that uses the following code:
 Several arrays (numeric) are flattened to strings, then the strings are
 concatenated (with a delimiter between each string) before it is sent
 to a TCP write function. On the other end, I parse the TCP read string
 using the match pattern with the delimiter being the same as the write
 side (currently a tab constant). Pretty basic, and works fine most of
 the time. However, it seems that sometimes a tab shows up as part of the
 flattened string, and confuses the match pattern into thinking this is
 the delimiter. This causes all kinds of havoc with the program.
 Is this a possible situation?
 Does anyone know of a good delimiter that is not one of the flattened
 string characters (if it exists)?
 Thanks in advance,=20
 Eric Scott


Christian,
you replied:
  How to do it?
  You define an escape character (maybe really Esc=0x1B or even better a
  char that is unlikely to show up in your data). Than replace any
  occurance
  of this char in your original data with a sequence of escape0x1 and
  afterwards all occurances of delimiter with escape0x2. Send this
  data
  and perform the inverse operation before using your delimiter.
  Both transcodings can be transparent.
  I do not have VIs at hand, but exactly this is what is done when
  transfering our medical measurements between different 'nodes' (may
  those
  be instances of operational units, devices or even machines).
 
  A side note:
  You should test a set of your raw data and do a histogram of all chars
  therein. Use the two chars that do show up least often as escape and
  as
  delimiter.
 Since the array is binary data, potentially any bit pattern can appear in
 it.  You are just making the delimiter longer to decrease the odds of a
 false positive. Theoretically, I could always create artificial sample data
  that would fail your approach.
No. This is a misunderstanding. This is a failsafe aproach if done
properly!
I replace any occurence of the delimiter and of an Esc char with a set of
two chars, where the first is that Esc char. This adds some extra chars
to the string; the number of extra chars depends on the number of Esc and
delimiter char in the raw string.
After trancoding every occurence of Esc together with its successor is a
representation of one of the original special chars as are Esc and
delimiter.
 Actually (after sleeping over this), I think the programmatically simplest
 solution would be to make an array of strings, then flatten it again.
 To decode: you unflatten the string to an array of strings, then
 unflatten
 each string back to the numeric arrays.
 It is so much easier to work with arrays compared to searching through a
 string.
You're right, but with a lot of short strings (and a more slow connection)
the overhead can be very important. 4 extra bytes for every String and at
least 8 extra bytes for the array and its size.
Greetings from Germany!
--
Uwe Frenz
~
Dr. Uwe Frenz
Entwicklung
getemed Medizin- und Informationtechnik AG
Oderstr. 59
D-14513 Teltow
Tel.  +49 3328 39 42 0
Fax   +49 3328 39 42 99
[EMAIL PROTECTED]
WWW.Getemed.de


--
David Ferster
Actimetrics, Inc.
1024 Austin St., Evanston, IL 60202
http://www.actimetrics.com
847/922-2643 Phone
847/589-8103 FAX



Re: Which delimiter to use for TCP functions?

2004-01-30 Thread Uwe Frenz
Eric,

you asked on Thu, 29 Jan 2004 15:16:56 -0800:
 Hi all, maybe someone has run into this problem before:
 I have a program that uses the following code:
 Several arrays (numeric) are flattened to strings, then the strings are
 concatenated (with a delimiter between each string) before it is sent 
 to a TCP write function. On the other end, I parse the TCP read string
 using the match pattern with the delimiter being the same as the write
 side (currently a tab constant). Pretty basic, and works fine most of
 the time. However, it seems that sometimes a tab shows up as part of the
 flattened string, and confuses the match pattern into thinking this is
 the delimiter. This causes all kinds of havoc with the program.
 Is this a possible situation?
 Does anyone know of a good delimiter that is not one of the flattened
 string characters (if it exists)?
 Thanks in advance,=20
 Eric Scott


Christian Altenbach  [EMAIL PROTECTED] answered
 Eric,
 
 Your problem has really nothing to do with TCP per se.
 Instead of a delimiter, prepend each flattened substring with its flattened
 string lenght before concatenating them.
 When you unflatten, just clip the first 4 chars of the string, unflatten to
 I32, then clip a string with that lenght, clip another I32, and so on.
 If the strings are shorter than 256, you could use an U8 for length.

There is another possibel aproach that does a transparent exchange of your
delimiter chars in the original data with some extra code.
This may be faster ore more code effective than Christians aproach, but as
always. It depends...

How to do it?
You define an escape character (maybe really Esc=0x1B or even better a
char that is unlikely to show up in your data). Than replace any occurance
of this char in your original data with a sequence of escape0x1 and
afterwards all occurances of delimiter with escape0x2. Send this data
and perform the inverse operation before using your delimiter.
Both transcodings can be transparent. 
I do not have VIs at hand, but exactly this is what is done when
transfering our medical measurements between different 'nodes' (may those
be instances of operational units, devices or even machines).

A side note:
You should test a set of your raw data and do a histogram of all chars
therein. Use the two chars that do show up least often as escape and as
delimiter.

Greetings from Germany!
-- 
Uwe Frenz


~
Dr. Uwe Frenz
Entwicklung
getemed Medizin- und Informationtechnik AG
Oderstr. 59
D-14513 Teltow

Tel.  +49 3328 39 42 0
Fax   +49 3328 39 42 99
[EMAIL PROTECTED]
WWW.Getemed.de