Re: [Discuss-gnuradio] Stream to PDU - Guaranteed to not lump tags?

[Dave NotTelling]

That does help!  Thanks Michael :)

On Wed, Jan 10, 2018 at 9:10 PM, Michael Dickens 
wrote:

> Hi Dave - The tagged stream -> PDU block will generate exactly 1 PDU per
> call to work. In your example, it is possible that all 3 of the PDUs will
> end up being in the stream, but, because of the way "tagged_stream_block"
> works, work() will be called with "ninput_items[0]" being just the value
> entered for the tag representing any given PDU length, no more or less.
>
> This stated, it is possible to end up with multiple same-named tag offset
> values in the stream on the same item; I've seen this recently. There are
> also sometimes other tags within range on the stream. All of the tags that
> aren't removed as the provided length key will be added to the PDU as
> meta-data. This might be the source of your "multiple offset values"
> comment.
>
> Hope this helps! - MLD
>
> On Wed, Jan 10, 2018, at 8:32 PM, Dave NotTelling wrote:
>
>  Thanks for the response!  Apologies for the unclear question.  Here's
> another shot:
>
> Can one call to the work function yield multiple offsets from the
> `get_tags_in_range(0, nitems_read(0), nitems_read(0) + noutput_items)`
> call?  With multiple offsets then that means there are multiple sets of
> PDUs (assuming that the stream was created from pdu_to_tagged_stream).
> Let's say for fun that there are 3 PDUs like the following (CAR, CDR):
>
>
>- ( {'pdu_len': 10, 'offset': 0, 'frame' : 1}, (0x00 0x00 0x00 0x00
>0x00 0x00 0x00 0x00 0x00 0x00) )
>- ( {'pdu_len': 10, 'offset': 10, 'frame' : 2}, (0x00 0x00 0x00 0x00
>0x00 0x00 0x00 0x00 0x00 0x00) )
>- ( {'pdu_len': 10, 'offset': 11, 'frame' : 3}, (0x00 0x00 0x00 0x00
>0x00 0x00 0x00 0x00 0x00 0x00) )
>
> What I'm curious about is whether or not a single call to work() will end
> up with tags for more than one of the PDUs above.  For instance, if the
> buffer for the block is large enough, would 20 samples get lumped in to a
> single call to work() resulting in say offset 0 and offset 10 both showing
> up in 'get_tags_in_range(0, nitems_read(0), nitems_read(0) +
> noutput_items)'?  This of course assumes that pdu_to_tagged_stream and
> tagged_stream_to_pdu both are set to use 'pdu_len' as the length tag.
>
>  I seem to remember having that very issue once.  I got multiple
> offset values in a work() function in the past.  Pretty sure I did at least
> O.o  Really, just curious if there is a guarantee somewhere deeper in the
> block code that ensures that can't happen.
>
>  As I write this I realize that this is a pretty easy thing to test
> out.  If it's still unclear, then I'll just do that and post results :)
>
>
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Stream to PDU - Guaranteed to not lump tags?

[Michael Dickens]

Hi Dave - The tagged stream -> PDU block will generate exactly 1 PDU per
call to work. In your example, it is possible that all 3 of the PDUs
will end up being in the stream, but, because of the way
"tagged_stream_block" works, work() will be called with
"ninput_items[0]" being just the value entered for the tag representing
any given PDU length, no more or less.
This stated, it is  possible to end up with multiple same-named tag
offset values in the stream on the same item; I've seen this recently.
There are also sometimes other tags  within range on the stream. All of
the tags that aren't removed as the provided length key will be added to
the PDU as meta-data. This might be the source of your "multiple offset
values" comment.
Hope this helps! - MLD

On Wed, Jan 10, 2018, at 8:32 PM, Dave NotTelling wrote:
>  Thanks for the response!  Apologies for the unclear question.
>  Here's another shot:> 
>> Can one call to the work function yield multiple offsets from the
>> `get_tags_in_range(0, nitems_read(0), nitems_read(0) +
>> noutput_items)` call?  With multiple offsets then that means there
>> are multiple sets of PDUs (assuming that the stream was created from
>> pdu_to_tagged_stream).  Let's say for fun that there are 3 PDUs like
>> the following (CAR, CDR):
>>  * ( {'pdu_len': 10, 'offset': 0, 'frame' : 1}, (0x00 0x00 0x00 0x00
>>0x00 0x00 0x00 0x00 0x00 0x00) )
>>  * ( {'pdu_len': 10, 'offset': 10, 'frame' : 2}, (0x00 0x00 0x00 0x00
>>0x00 0x00 0x00 0x00 0x00 0x00) )
>>  * ( {'pdu_len': 10, 'offset': 11, 'frame' : 3}, (0x00 0x00 0x00 0x00
>>0x00 0x00 0x00 0x00 0x00 0x00) )>> What I'm curious about is whether or 
>> not a single call to work() will
>> end up with tags for more than one of the PDUs above.  For instance,
>> if the buffer for the block is large enough, would 20 samples get
>> lumped in to a single call to work() resulting in say offset 0 and
>> offset 10 both showing up in 'get_tags_in_range(0, nitems_read(0),
>> nitems_read(0) + noutput_items)'?  This of course assumes that
>> pdu_to_tagged_stream and tagged_stream_to_pdu both are set to use
>> 'pdu_len' as the length tag.>  I seem to remember having that very issue 
>> once.  I got multiple
>  offset values in a work() function in the past.  Pretty sure I
>  did at least O.o  Really, just curious if there is a guarantee
>  somewhere deeper in the block code that ensures that can't
>  happen.> 
>  As I write this I realize that this is a pretty easy thing to
>  test out.  If it's still unclear, then I'll just do that and post
>  results :)
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Stream to PDU - Guaranteed to not lump tags?

[Dave NotTelling]

Michael,

 Thanks for the response!  Apologies for the unclear question.  Here's
another shot:

Can one call to the work function yield multiple offsets from the
`get_tags_in_range(0, nitems_read(0), nitems_read(0) + noutput_items)`
call?  With multiple offsets then that means there are multiple sets of
PDUs (assuming that the stream was created from pdu_to_tagged_stream).
Let's say for fun that there are 3 PDUs like the following (CAR, CDR):


   - ( {'pdu_len': 10, 'offset': 0, 'frame' : 1}, (0x00 0x00 0x00 0x00
   0x00 0x00 0x00 0x00 0x00 0x00) )
   - ( {'pdu_len': 10, 'offset': 10, 'frame' : 2}, (0x00 0x00 0x00 0x00
   0x00 0x00 0x00 0x00 0x00 0x00) )
   - ( {'pdu_len': 10, 'offset': 11, 'frame' : 3}, (0x00 0x00 0x00 0x00
   0x00 0x00 0x00 0x00 0x00 0x00) )

What I'm curious about is whether or not a single call to work() will end
up with tags for more than one of the PDUs above.  For instance, if the
buffer for the block is large enough, would 20 samples get lumped in to a
single call to work() resulting in say offset 0 and offset 10 both showing
up in 'get_tags_in_range(0, nitems_read(0), nitems_read(0) +
noutput_items)'?  This of course assumes that pdu_to_tagged_stream and
tagged_stream_to_pdu both are set to use 'pdu_len' as the length tag.


 I seem to remember having that very issue once.  I got multiple offset
values in a work() function in the past.  Pretty sure I did at least O.o
Really, just curious if there is a guarantee somewhere deeper in the block
code that ensures that can't happen.

 As I write this I realize that this is a pretty easy thing to test
out.  If it's still unclear, then I'll just do that and post results :)

Thanks!

On Wed, Jan 10, 2018 at 3:32 PM, Michael Dickens 
wrote:

> Hi Dave - The tagged stream -> PDU block will look at for the provided tag
> string at the current 0 offset & if found then that's what the PDU data
> size (in items) will be [this is actually handled in the
> "tagged_stream_block" parent class]. If there are interim tags (within the
> output PDU size in items), then they are just added as meta-data to the
> PDU. Each PDU is created independent of the other PDUs, and just 1 created
> per call to "work". Not sure if this is what you were looking to have
> answered; if not, please clarity. Cheers! - MLD
>
> On Wed, Jan 10, 2018, at 11:30 AM, Dave NotTelling wrote:
> > The wording of the title likely needs work, but the basic idea is this:
> >
> >  * Suppose that I have a ZMQ message source that has arbitrarily sized
> vectors of some consistent type
> >  * I convert that over to a tagged stream, do some operations on it,
> then convert it back to a PDU
> >  * Assume for a second that several of the offsets in the tagged stream
> are very small (only say 10 samples each) and back to back
> >  * Is there a guarantee that each of the 10 sample offsets will result
> in a single PDU?
> >
> >Looking at the source of tagged_stream_to_pdu it seems that there is no
> check that multiple tag offsets arrived in a single call to work.  Or is
> that something that cannot happen?  Having a contract that no more than one
> offset can arrive at a single call to work would be really nice, but I
> imagine that doing so could cause some pretty serious performance issues.
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Stream to PDU - Guaranteed to not lump tags?

[Michael Dickens]

Hi Dave - The tagged stream -> PDU block will look at for the provided tag 
string at the current 0 offset & if found then that's what the PDU data size 
(in items) will be [this is actually handled in the "tagged_stream_block" 
parent class]. If there are interim tags (within the output PDU size in items), 
then they are just added as meta-data to the PDU. Each PDU is created 
independent of the other PDUs, and just 1 created per call to "work". Not sure 
if this is what you were looking to have answered; if not, please clarity. 
Cheers! - MLD

On Wed, Jan 10, 2018, at 11:30 AM, Dave NotTelling wrote:
> The wording of the title likely needs work, but the basic idea is this:
> 
>  * Suppose that I have a ZMQ message source that has arbitrarily sized 
> vectors of some consistent type
>  * I convert that over to a tagged stream, do some operations on it, then 
> convert it back to a PDU
>  * Assume for a second that several of the offsets in the tagged stream are 
> very small (only say 10 samples each) and back to back
>  * Is there a guarantee that each of the 10 sample offsets will result in a 
> single PDU?
>
>Looking at the source of tagged_stream_to_pdu it seems that there is no check 
>that multiple tag offsets arrived in a single call to work.  Or is that 
>something that cannot happen?  Having a contract that no more than one offset 
>can arrive at a single call to work would be really nice, but I imagine that 
>doing so could cause some pretty serious performance issues.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio