Hi Kartik,
thanks for the feedback!
so, I took the time and tried to read up a bit on what Bokeh does, how
it's partitioned. I'd like to describe this here as shortly as possible,
mostly for my own understanding and to foster more discussion with
others that can't find the time, and would kindly ask you to confirm
this is your understanding, too:
* Bokeh is Python library that produces plots
* It's plotting frontend is HTML & Javascript
* This enables interactive plotting (ie. you can zoom in, pan etc
without updating the dataset visualized)
* The frontend Javascript is a library (BokehJS) used in a "standard"
piece of code that you feed two things:
o The ID of the HTML <div> tag to be filled with the graph
o The data and plot settings to be visualized as JSON
* Bokeh (the python library) comes with a Python-implemented server
that you're planning to use
Now the things I'm not sure about:
* said server has a REST API, i.e. is /polled/ from the client only?
Or can there be pushing data from server -> client? What is the
model you'd prefer? Your proposal says "stream to the clients", but
I can't find that in Bokeh (but then again: no experience on
interactive Web dev on my side at all, so this might be really easy
for you, if possible).
* I've no idea how to talk GR data -> bokeh-server. Maybe you could
elaborate on how you do that?
* the JSON is relatively verbose, and contains basically all
imaginable settings for a plot. However, I presume a data update
from the bokeh-server would only consist of the model data, which,
for a 5-point line graph would consist of (this is pretty-printed,
Bokeh omits all the whitespace/line breaks):
{
"type" : "ColumnDataSource",
"id" : "ce58de4d-0ef2-43cf-b15a-23d431781c1a",
"attributes" : {
"callback" : null,
"column_names" : [
"y",
"x"
],
"data" : {
"x" : [
1,
2,
3,
4,
5
],
"y" : [
6,
7,
2,
4,
5
]
}
}
},
* Hence, if performance of the python server is doubted, or
integration into GR is complicated, you could also just write a C++
block that takes numerical values from GNU Radio and serves them via
Websocket (how does one do that?), and point BokehJS at that
instead? (example from [0], found the embed_items declaration in [1])
* However, when running [2] and changing the data source, I see all
the plot JSON being transferred, the ColumnDataSource object just
being about half of the 12 kB response object (amongst dozens of
requests made to react to this change of model, but that might be an
effect of this being a feature-rich plot).
* Can you please explain the user work flow? I would have imagined
that the users design their "plot layout" either
o implicitly (by just using your sinks in their flow graph), so
that gr-bokehgui builds a HTML page containing the plots they
want on its own, or
o explicitly by designing an HTML page with <div> placeholders on
their own and tell gr-bokehgui "hey, use that; the plot <div>
IDs are [list of IDs]",
* but I'm not sure you'd support the second method, since you mention
the ability to add labels and such, which would be relatively
redundant with that.
Best regards,
Marcus
[0] https://demo.bokehplots.com/apps/movies
[1]
https://github.com/bokeh/bokeh/blob/0.12.4/bokehjs/src/coffee/embed.coffee#L161
[2] https://demo.bokehplots.com/apps/crossfilter
On 21.03.2017 12:07, Kartik Patel wrote:
> Dear Ben and Bastian,
>
> Thank you very much for the feedback.
>
> @Ben: Thank you for this error. I have not tried to dig deep in this
> error because it was just a prototype.
>
> @Bastian: You got the server architecture correct. I am planning to do
> exactly same.
>
> _Coming to queries about using Bokeh:_
> I agree that we are all-in to Bokeh. I also agree that it is good to
> have different modules for frontend and backend. But in that case we
> will be simply replicating Bokeh's backend (with much less features)
> and Bokeh's frontend (independently this time). I believe it will be
> like re-inventing the wheel.
>
> Also, the features like simultaneous change in parameters in all
> connected documents are necessary when working with the hardware or
> even during simulation. Implementing this feature and some more like
> these will not be easy. Also, considering that Bokeh library includes
> all these in itself, I think it's implementation will be more
> efficient than what I will make. (This last statement is an
> assumption. Not sure how valid.)
>
> So, I think now only issue is unreliability of the library. In other
> terms, if somehow in future Bokeh become obsolete, then this entire
> module will die. In order to become less dependent on this, we have to
> make a independent manual python based server and Plotly based
> frontend as suggested.
>
> It means we have a trade-off between the scope of the project and
> reliability of Bokeh. I am not sure how good it is to develop
> something which already exists because of the possibility that the
> library will be obsolete one day. We can discuss on this.
>
> Only reason, I am resistant to the idea of not using the Bokeh server
> is that the existing features and scope of the project will be
> reduced. Since, all the plots and widgets are necessary in order to
> make the module usable, limiting the plots will not be good enough.
> Although I am open to contributions on this module even after GSoC,
> this is not valid statement for the proposal in GSoC.
>
> I am fully aware with the probable issues of having the external
> library as core dependency. I can also imagine the module having
> independent frontend and backend. But is it worth rewriting the code
> that exist in Bokeh?
>
> If we still decide to go for independent backend and frontend, I will
> need one weekend to write full proposal for that. So, it would be good
> if we can conclude the discussion before this Friday.
>
> Thank you very much for the suggestions.
>
> Regards,
> Kartik Patel
>
>
>
> On Tue, Mar 21, 2017 5:18 AM, Ben Hilburn [email protected]
> <mailto:[email protected]> wrote:
>
> Hi Kartik -
>
> I apologize for the delayed response! Nice work getting this out
> last week, and great proposal. I gave your prototype a whirl - it
> generated some output (see attached screenshot), but then died
> with this error (I'm not worried about the error, just giving you
> a heads-up):
>
> Created new window in existing browser session.
> handler caught exception: Set changed size during iteration
> Traceback (most recent call last):
> File
>
> "/home/bhilburn/usr/lib64/python2.7/site-packages/gnuradio/gr/gateway.py",
> line 55, in eval
> try: self._callback()
> File
>
> "/home/bhilburn/usr/lib64/python2.7/site-packages/gnuradio/gr/gateway.py",
> line 160, in __gr_block_handle
> ) for i in self.__out_indexes],
> File
>
> "/home/bhilburn/usr/lib64/python2.7/site-packages/htmlgui/time_sink_f.py",
> line 62, in work
> self.ds.stream(new_data, rollover = 1000)
> File
> "/usr/lib/python2.7/site-packages/bokeh/models/sources.py",
> line 287, in stream
> self.data._stream(self.document, self, new_data, rollover,
> setter)
> File
> "/usr/lib/python2.7/site-packages/bokeh/core/property/containers.py",
> line 345, in _stream
> hint=ColumnsStreamedEvent(doc, source, new_data, rollover,
> setter))
> File
> "/usr/lib/python2.7/site-packages/bokeh/core/property/containers.py",
> line 106, in _notify_owners
> for (owner, prop) in self._owners:
> RuntimeError: Set changed size during iteration
> thread[thread-per-block[2]: <block time_sink_f (1)>]: SWIG
> director method error. Error detected when calling 'feval_ll.eval'
>
> Regarding the proposal, as Martin and Bastian said, this is
> looking really good. Expanding a bit on the point about being
> "all-in" on Bokeh, that's not necessarily a bad thing, but it
> would be good to understand exactly where the line is between
> Bokeh and GNU Radio. If someone came along after you to implement
> something in a different web framework, would they start in the
> same place as you are starting, now, or would they build on
> something you are creating? If there is a way to architect this
> such that can be built on with non-bokeh frameworks later, that
> would be preferable to locking everything down.
>
> I also am a fan of the `gr-bokehgui` suggestion that Martin gave.
>
> Another benefit of a web-based front-end is the ability to
> visualize data while remote from whatever computer actually has
> the hardware connected to it and interact with the flowgraph. It
> would be good to call this out in your "Features" list and add
> detail regarding this where appropriate.
>
> Cheers,
> Ben
>
> On Mon, Mar 20, 2017 at 12:02 PM, Bastian Bloessl
> <[email protected] <mailto:[email protected]>> wrote:
>
> Hi Kartik,
>
> great proposal and it seems that Bokeh really provides all the
> required features.
>
> As far as I understand, you would use an architecture similar
> to the one shown in the user guide
>
> http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#connecting-with-bokeh-client
>
> <http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#connecting-with-bokeh-client>
>
> Then the GNU Radio flow graph feeds data to the Bokeh server,
> which distributes and syncs the data in the client’s browsers.
> As Martin already mention, that would mean that we are really
> all-in Bokeh. So I guess the biggest question is if that’s a
> good idea. If we, at some point, find out that Bokeh has
> (performance) problems, there might not be a lot of code that
> can be reused. Do you know if Bokeh is tuned towards high data
> rates and frequent updates? Maybe it would be a good idea to
> also ask for feedback from the Bokeh guys and see what they
> think about this particular use-case.
>
> The alternate architecture would be to have a more generic GNU
> Radio block that starts a web server and streams the incoming
> samples through web sockets (using Flask, for example). This
> would make the HTML frontend independent from the plotting
> library. We could, for example, have a Bokeh time sink and a
> Plotly frequency sink. Apart from that, we would not have to
> rewrite everything if we want to change the plotting library.
> Did you consider this architecture?
>
> Best,
> Bastian
>
>
> > On 19 Mar 2017, at 21:18, Kartik Patel
> <[email protected] <mailto:[email protected]>>
> wrote:
> >
> > Dear Martin,
> >
> > Thank you for the comments.
> >
> > FG in GRC will be same as the current one. Instead of
> including QT blocks, you will have to include HTML blocks.
> Also, the Options blocks will have one option called Web GUI
> which will enable the web portal.
> >
> > I agree with your point about the name of gr-bokeh . Only
> thing I am unsure about is, it does not represent the purpose.
> So, we can make it gr-bokehgui or gr-bokweb if that is fine.
> >
> > Also, I will add acknowledgement details in my final PDF.
> Thank you for the reminder.
> >
> > Regards,
> > Kartik Patel
> >
> >
> >
> >
> >
> > On Sun, Mar 19, 2017 6:40 AM, Martin Braun
> [email protected] <mailto:[email protected]> wrote:
> > Also, please make sure you have acknowledged all the
> formalities, and
> >
> > provide details on how you would like to update the
> community on a
> >
> > regular basis.
> >
> >
> >
> > -- M
> >
> >
> >
> > On 03/18/2017 05:59 PM, Martin Braun wrote:
> >
> > > Kartik,
> >
> > >
> >
> > > this is a pretty good draft for a proposal! You have all
> the relevant
> >
> > > bits, and your previous contributions to GNU Radio are noted.
> >
> > >
> >
> > > One thing I'm not fully understanding is the workflow. Is
> it true that
> >
> > > the FG generation in GRC is the same as before? If so, can
> I run this
> >
> > > entire thing from within GRC (it would open a browser, but
> otherwise can
> >
> > > I execute it from GRC)?
> >
> > >
> >
> > > You've called your OOT both gr-webgui and gr-htmlgui, but
> that's a minor
> >
> > > issue. I'm wondering if gr-bokeh wouldn't be more apt; it
> seems like
> >
> > > your choice of framework is pretty central to the code
> (that's OK), but
> >
> > > what if someone else implements gr-webgui using fancy
> other framework XYZ?
> >
> > >
> >
> > > Cheers,
> >
> > > Martin
> >
> > >
> >
> > > On 03/14/2017 09:07 AM, Kartik Patel wrote:
> >
> > >> Hello,
> >
> > >>
> >
> > >> I am Kartik Patel, undergraduate student from Indian
> Institute of
> >
> > >> Technology Roorkee. I am interested in the project /A web
> based GUI for
> >
> > >> GNU Radio/.
> >
> > >>
> >
> > >> Here
> >
> > >>
>
> <https://github.com/kartikp1995/GSoC2017/blob/master/WebBasedGUIForGNURadio.md
>
> <https://github.com/kartikp1995/GSoC2017/blob/master/WebBasedGUIForGNURadio.md>>[1]
> >
> > >> is the first draft of my proposal for the project. I have
> tried to cover
> >
> > >> the background, the output as well as the flow of
> implementation. Kindly
> >
> > >> review it and provide your valuable suggestion. In
> particular,
> >
> > >> suggestions on the timeline, the defined scope of the
> project and
> >
> > >> implementation overview is highly appreciated. Also,
> please ask any
> >
> > >> query that is not exactly covered in the proposal. I can
> answer that and
> >
> > >> also add it in the proposal.
> >
> > >>
> >
> > >> Thank you for your help.
> >
> > >>
> >
> > >> Regards,
> >
> > >> Kartik Patel
> >
> > >>
> >
> > >> [1]
> >
> > >>
>
> https://github.com/kartikp1995/GSoC2017/blob/master/WebBasedGUIForGNURadio.md
>
> <https://github.com/kartikp1995/GSoC2017/blob/master/WebBasedGUIForGNURadio.md>
> >
> > >>
> >
> > >>
> >
> > >>
> >
> > >> _______________________________________________
> >
> > >> Discuss-gnuradio mailing list
> >
> > >> [email protected] <mailto:[email protected]>
> >
> > >> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
> >
> > >>
> >
> > >
> >
> > >
> >
> > > _______________________________________________
> >
> > > Discuss-gnuradio mailing list
> >
> > > [email protected] <mailto:[email protected]>
> >
> > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
> >
> > >
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Discuss-gnuradio mailing list
> >
> > [email protected] <mailto:[email protected]>
> >
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
> >
> >
> >
> > _______________________________________________
> > Discuss-gnuradio mailing list
> > [email protected] <mailto:[email protected]>
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
>
> --
> Dipl.-Inform. Bastian Bloessl
> GitHub/Twitter: @bastibl
> https://www.bastibl.net/
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> [email protected] <mailto:[email protected]>
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
>
>
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> [email protected]
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
_______________________________________________
Discuss-gnuradio mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
