Hi again,
> WAV is an example of a file format where *everyone* added their own
> custom headers/chunks, without any planning. As a result, no program
> can read all existing WAV files; WAV is considered an example how
> *not* to do a file format.
And still we have to use it.....
> Would you consider a very simple, easy to parse header like this:
>
> typedef struct {
> short header_len; // This field is always present, and always first
> short data_len; // This field is always present, and always second
> [ header contents, including header version, type, etc. goes here ]
> char data[];
> } NET_RX_STRUCT;
Perhaps. At the moment I do not see any advantage. You want to
add the flexibility of a variable data_len. The cost is that there
are bytes ahead of the data so it is less straightforward to
read directly into the final buffer. One could copy the few bytes
back, but there is an increaced time delay of one buffer. This
is not a big deal.
> This is still easy to parse, since all a user needs to do is something like
>
> struct NET_RX_STRUCT *rx_packet;
> char *my_data;
> short i, my_data_len;
>
> rx_packet = ReceiveData();
> my_data = ((char *) rx_packet) + rx_packet->header_len;
> for(i = 0; i < rx_packet->data_len; i++)
> DoSomethingWithMyData(my_data[i]);
Yes, but but these modern ways of writing scares off all my
friends who can use old-fashioned C but not C++.
First of all ReceiveData() has to be written, separate
buffers of size NET_RX_STRUCT have to be allocated and
managed etc. I do not currently have such code and I suspect
it involves needless copy operations. I am looking
for bandwidths of 2 MHz and above (for VHF noise blanking
to remove static rain) so needless copy - probably
up and down to main memory is something I want to avoid.
In Linrad It is like this now:
NET_RX_STRUCT *msg;
rxin_char=(void*)(&timf1_char[timf1p_pa]);
timf1p_pa=(timf1p_pa+ad_read_bytes)&timf1_bytemask;
for(j=0; j<ad_read_bytes; j+=NET_MULTICAST_PAYLOAD)
{
msg=(void*)&rxin_char[j];
seltest16:;
if(kill_all_flag) goto rawnet_rxin_error_exit;
testfds=fds;
if( select(FD_SETSIZE, &testfds, (fd_set *)0, (fd_set *)0, &timeout) > 0
{
if(FD_ISSET(netfd.rec_rx,&testfds))
{
recvfrom(netfd.rec_rx,msg,sizeof(NET_RX_STRUCT),0,
(struct sockaddr *)
&rx_addr,&addrlen);
// รถ fill zeroes and move data if block_no not correct here.
}
}
}
else
{
lir_sleep(100000);
goto seltest16;
}
}
This might look like more code, but there is nothing hidden here.
timf1_char is the circular buffer which the fft routines use as input.
Data is transferred directly into this buffer which has a size of
2 to power N. Some extra space is reserved for the header so
writes do not become illegal when the pointer is at its last position.
If it were not for the requirement that the thread has to
exit nicely even if the network is broken the same code
would look like this:
NET_RX_STRUCT *msg;
rxin_char=(void*)(&timf1_char[timf1p_pa]);
timf1p_pa=(timf1p_pa+ad_read_bytes)&timf1_bytemask;
for(j=0; j<ad_read_bytes; j+=NET_MULTICAST_PAYLOAD)
{
msg=(void*)&rxin_char[j];
recvfrom(netfd.rec_rx,msg,sizeof(NET_RX_STRUCT),0,
(struct sockaddr *) &rx_addr,&addrlen);
}
This will be fine in a single-threaded program where it would
be ok to exit with ctrl C if the program hangs on the recvfrom
call because the network is broken.
> > > That, too, makes it harder for dedicated hardware receivers; ideally
> >> these would not need _any_ communication from the slave to the
> >> master. As I see it, encoding this information in the header of each
> >> package is a low-overhead way to reduce ambiguity, too.
> >The problem is that there are so many possibillities. I do not
> >want to invent a complicated scheme for describing the myriad
> >of things I can imagine now only to discover in a few years that
> >something entirely different has evolved.
>
> I would suggest keeping it extremely simple. There is not very much
> information that varies between sampled systems:
>
> - sample size
> - sample rate
> - number of channels (could even be fixed to 'always I/Q')
Ooooh! We talk about the wideband output from an SDR. It is far
more complicated. Are noise pulses subtracted? Are some points
blanked out (how many percent ?) Are some carriers (spurs) removed
Is ALC splatter from some SSB transmission processed etc. etc.
There are really many possible wideband processes that serve
the purpose of removing interference of various kinds. Removing
the second harmonic at 1836 kHz of an AM transmitter at 918 kHz
is one recent example. One would like to do it while processing
over 1 MHz bandwidth in order to have access to the fundamental
but already in 90 kHz bandwidth one should be able to do a lot
by use of the knowledge that the modulation sidebands (splatter)
belong to a second harmonic.
> and, for radio systems,
>
> - center frequency
AND phase. Is Q before or after I? (direction of the
frequency scale)
> This is enough for basic decoding of any stream, no ? Even the center
> frequency can be seen as superfluous (since it's only for display and
> not strictly needed for decoding).
The primary usage of the Linrad network was for the second operator
in a contest station. It is an obvious advantage that the display
is always correct - particularly if several bands are monitored
simultaneously.
> I cannot imagine what systems would evolve over the coming five years
> that couldn't fit in this framework.
Linrad can also send data in the frequency domain and there is
quite a lot of info that a slave will need. Admittedly those
formats are likely to be used by Linrad only but they carry
many more complications.
> At 17:47 +0100 04-01-2007, Leif Asbrink wrote (in another mail):
> >Would you agree on milliseconds since midnight? From JDB I
> >learned that a double with seconds since Unix epoch
> >would be a bad idea since conversion may be difficult on
> >non-PC platforms. (It is the internal time format within
> >Linrad however)
>
> I would use the same interface that gettimeofday() uses: a long with
> seconds since the Epoch (Jan 1 1970), and a long with microseconds.
>
> >The formats I intend to use within Linrad will use IA32 little endian
> >(as well as IA32 float) I have no intention to make Linrad portable
> >to other platforms and I am pretty sure I will not change my mind
> >on this point for the next 5 years or more. Probably never.
>
> OK, that's fine, so please document this somewhere so those of us on
> non-IA32 can deal with it.
>
> As an example: I'm currently soldering the prototype of an
> audio-to-Ethernet converter as part of a portable hard disk recorder.
> This design uses the CS5381 ADC, one of the best professional audio
> converters on the market with a dynamic range approaching 120dB. This
> is an open-hardware system[1], and with a few modifications I could
> see it being usable for Linrad. A lot of the limitations (time jitter
> on the system clock etc) that are present on a PC platform simply do
> not appear for such a dedicated device. How would you like me to
> interface such a system to Linrad ? Should it be able to act as a
> Linrad master ?
Probably it would be better to connect it to a socket on a
dedicated ethernet port on one computer, the one which has
the controls for the radio hardware connected to this soundcard.
The master wants 100% reliable data because I assume you do not
want to put the master system clock on the audio-to-Ethernet
converter. Nevertheless it would be good if it could be used
as a master anyway - but then without a reliable time stamp.
Are you aware of any standard format for streaming unprocessed
audio data?
What data format were you contemplating before this discussion
started?
73
Leif
#############################################################
This message is sent to you because you are subscribed to
the mailing list <linrad@antennspecialisten.se>.
To unsubscribe, E-mail to: <[EMAIL PROTECTED]>
To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]>
To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]>
Send administrative queries to <[EMAIL PROTECTED]>
