Bastian, Ben, Seth, Nathan and Paul,
Thank you all for your useful feedback!
It's really exciting the fact that this project is a chance for such an
interesting discussion.
I will try to make some initial comments on your responses and I'll
start working on the updated proposal as soon as possible.
On 03/24/2017 06:01 PM, Ben Hilburn wrote:
Hi Kostis -
Great proposal! I'm really excited to see someone tackling the SigMF
item on the Ideas List.
I'll respond in-line to Bastian, and then add some of my own:
- You mention that sources and sinks will handle PDUs and Tagged
Streams. I wonder what would be one PDU. A SigMF "capture segment" or
some other chunk of samples. Also are you sure you want to use Tagged
Streams? Maybe you meant normal streams with tags? With Tagged
Streams,
you would have to fit the whole capture segment in the input buffer of
the block. That might not be possible.
I was actually referring to normal streams with tags. I should fix that
in my proposal.
Furthermore, I will examine the case of Tagged Streams in order to be
able to safely reject them.
- You mention that RapidJSON code is messy. Does it auto-generate
code?
That could even be an advantage, since SigMF might well go through
some
more iterations. Did you plan to also create wrappers for Python?
Messy was a word I've chosen to describe that the produced code is not
so easy to read and understand.
Python wrappers were not in my initial intentions, but as long as the
timeline permits it I would be interested on implementing them.
- The goal of SigMF seems to be to provide a very minimal spec, and I
think many users will have to rely on custom extension in namespaces.
Maybe you could describe a bit how source/sinks and the GUI could deal
with these extensions.
- You will provide the means to filter data. Did you only plan to
filter
specific capture segments or also arbitrary samples based on
annotations? Since you mentioned modulation, which is, AFAIK,
currently
not specified, did you plan to allow filtering on custom properties?
So, thinking about this a bit, if Kostis' implementation enables
filtering on fields in the `core` namespace, it shouldn't really be
any additional effort to support non-`core` namespaced fields. Since
any field would be in a capture / annotation segment, which thus would
be mapped to a sample index / number of samples, Kostis' software
doesn't have to care *what* that field is in order to "select it" in
the tool.
That's how I read the proposal, anyway. Kostis, it might be good to
clarify this in the proposal.
I think Ben's answer covered the concept I had in my mind. For sure I
have to rephrase/clarify this in the proposal.
- Regarding the GUI, it seems as if you plan to develop something from
scratch. Did you consider extending projects like "such samples" from
Tim or Inspectrum?
(Seth just responded while I was writing this message, but my comments
are similar.)
I agree with Seth and Bastian, here. Generally, I'm a fan of extending
existing functionality rather than creating something from scratch.
That said, using one of the existing things will also require development.
In the "such samples" case, it's actually just itself a flowgraph
(https://github.com/osh/gr-pyqt/blob/master/apps/such_samples2.grc).
In order to integrate SigMF into it (which I don't think is a bad
idea), new blocks / code will need to be written. Using "Inspectrum"
is actually more problematic, especially since it doesn't use GNU
Radio under-the-hood. You would effectively be doing work for a
different project on GNU Radio GSoC money, which I don't think makes
sense.
It might be worth looking at what blocks / code would be necessary to
allow the existing Qt GUI features (e.g., "such samples") to make use
of the functionality you will be writing in GNU Radio. Ideally, any
work you do there can be used in other flowgraphs (not just "such
samples"), but using "such samples" would be a great way to reduce
your own burden. If you determine that there is really no way to build
on things that already exist, then implementing a new tool (like the
one in your proposal, now) that is integrated with GRC (e.g., the
filter design tool) might indeed be the best way forward. It would be
good to explain the justification behind the final decision in the
proposal.
(It's worth noting that I would really like to see SigMF support in
Inspectrum, and I don't think SigMF support there means it wouldn't be
useful here, as well. Inspectrum does more than just visualize data,
and one of the key goals of SigMF is portability between tools &
workflows.)
It is true that in my mind I was thinking the proposed SigMF GUI app as
an integrated GRC tool, just like filter design. Nevertheless, I totally
understand the philosophy behind reusing/extending instead of
re-inventing the wheel. I should give a careful look on gr-pyqt and QT
sinks though.
- I wonder what would be the workflow when creating a SigMF recording.
An issue might be that you cannot immediately write the SigMF meta
file
during recording. (For example, the hash has to be calculated
after the
recording. Or the annotations have to be written after the capture
segments.). Did you plan to create the meta data in memory and
write the
file when the flow graph shuts down (which might cause data loss when
you experience a segfault) or do you write meta data to disk in an
intermediate format that you have to post-process manually to
create the
final SigMF files.
So the SigMF spec is designed specifically to enable writing the
metadata and dataset files in a streaming manner. The spec is careful
not to dictate how applications create SigMF recordings, so it's not a
problem if you write a bunch of stuff and then after you finish
streaming to the files you open them up and add some more stuff to
finish them off.
Bastian's point, though, is a good one. In the use-case where you are
streaming metadata and samples into files and then you close your
application, do you plan on needing to go back to add more information
to the metadata file? If so, how do you do that in an automatic way if
the flowgraph has stopped?
Maybe I could let the destructor of the block in combination with fseekm
handle the finalization of the metafile. The case of segfault is
actually a question that needs more attention.
- We are working hard to get the SigMF spec into a stable state as
soon as possible, but as you can see from the issue tracker (which is
where we are hosting our discussions) there are aspects of the spec
that are still being debated. Many of these would impact your
implementation. I think it would be good to explain your plan to (a)
participate in the SigMF spec discussions, especially with the insight
you gain from implementing support for it and (b) deal with pieces of
the spec that may be a moving target.
I have already started looking at the open issues. Do you think it would
be meaningful to move the discussion there, in a dedicated issue?
- Since this will create a dependency on RapidJSON (or whatever you
use), we'll want to be sure that *not* having that installed doesn't
prevent building anything not related to SigMF within GNU Radio. I
really dislike adding dependencies, but since we don't already have a
JSON library there isn't much of an option, here, so we just need to
be sure to minimize the burden to users, developers, and packagers.
According to the description in the main page, RapidJSON "is
self-contained and /header/-only". I dont know if MIT is a problem though.
- "to-SigMF Converters": An important tool will be something that
converts a file written with the current GNU Radio File Sink into a
SigMF recording. I would limit the scope of the converter work to this.
Cheers,
Ben
On 03/23/2017 07:37 PM, Kostis Triantafyllakis wrote:
> Hello,
>
> My name is Kostis Triantafyllakis, graduate student of Computer
Science
> at the University of Crete.
> I'm interested in the project of SigMF functionality
implementation for
> GNU Radio.
>
> I have started working on my proposal and I kindly ask for your
feedback.
> Please find the draft below:
>
> https://github.com/ctriant/GSoC17-Proposal/blob/master/README.md
<https://github.com/ctriant/GSoC17-Proposal/blob/master/README.md>
>
https://github.com/ctriant/GSoC17-Proposal/blob/master/gsoc17_proposal.pdf
<https://github.com/ctriant/GSoC17-Proposal/blob/master/gsoc17_proposal.pdf>
>
> Thank you in advance,
>
> Kostis
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio