I like to use the socketpair(2) syscall for this, if you don't really need a message broker.
--Albin On Wed, May 29, 2019, 21:26 Brad Hein <linuxb...@gmail.com> wrote: > Packaging the json string as a vector of u8 integers worked like a charm, > thank you for the suggestion! > > Python code: > https://gist.github.com/regulatre/38e7162d6f470ad0dbd77b171a69dc79 > C++ block code: > https://gist.github.com/regulatre/f264fb4751ca3bd8d6b98abf9792e95d > > Memory consumption by the python flowgraph is no longer showing signs of > memory leakage (at least in the 4 hours it's been running :) > > > On Fri, Mar 1, 2019 at 1:50 PM Müller, Marcus (CEL) <muel...@kit.edu> > wrote: > >> Hi Brad, >> >> so, yes, your observation is correct: PMT symbols are/were meant to be >> used as "identifiers", not as "data carriers"; the motivation behind >> the hash table you find in pmt.cc is that there's only one instance of >> any given PMT symbol, and thus, a simple address comparison suffices to >> tell whether two PMT symbols are the same. >> >> You'll notice that on x86 (and presumably modern ARM) string comparison >> is pretty fast, and that you'd need to do *a lot* comparisons to offset >> the cost of hashing a symbol once. Anyway, yes, that table grows >> unboundedly. >> >> Since your string isn't actually a "symbol", I'd recommend simply >> encoding it safely (that's probably already the case), and putting it >> in a uniform PMT vector of 8bit integers (u8vector). >> >> On a different note, there's actually "unintended" (as in: I don't >> intend GNU Radio to have an unbounded hash map, but it's at least what >> was originally intended) memory leaks with PMTs and tag_t on the >> Python/C++ boundary, and there's quite some broken concepts within PMT. >> >> Long or medium term, we'll be replacing PMT with something that is >> actually a common serialization format for usage in external software >> (i.e. not for usage within the same flow graph), and ideally with the >> unserialized universal container that comes with such a format. Stay >> tuned. However, not happening in 3.8 or anything before. >> >> Best regards, >> Marcus >> >> On Thu, 2019-02-28 at 14:48 -0500, Brad Hein wrote: >> > >> > In my gnuradio test application I’m attempting to pass a JSON formatted >> string from my C++ custom block, to my python flow graph containing the >> block. >> > >> > The String message are being received by my python flow graph, but >> there’s a memory leak. Over time, my flowgraph RSS (memory footprint) grows >> at about 32k/s up until the application runs out of memory. >> > >> > The leak didn’t start until after I retrofitted my flowgraph and custom >> block with polymorphic type (PMT) based message passing. I’m passing a 200 >> or so byte JSON string (as a symbol) from C++ block to python flow graph >> about 60 times per second. >> > >> > Sleuthing through the pmt.cc code [1] I see the String (symbol) objects >> are stored in a hash. I suspect what is happening is that since all of my >> JSON strings are unique, they’re getting added to the hash and causing the >> hash to grow unbounded over time. >> > >> > From what I can tell by reading the wiki [2], the approach I’m using is >> the best way to get a string from my custom block and into my python >> flowgraph. >> > >> > [1] >> https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/pmt/pmt.cc >> > [2] https://www.gnuradio.org/doc/doxygen/page_pmt.html >> > >> > >> > Sample c++ block code snippet from my work function: >> > >> > // Calculate metrics. Occasionally (60/second) we'll get back a JSON >> formatted string of metrics. >> > crossingData = calculateWaveCrossingMetrics(lastSampleValue,in[i]); >> > >> > if (crossingData.length() > 0) { >> > pmt::pmt_t messageString; >> > messageString = pmt::string_to_symbol(crossingData.c_str()); >> > message_port_pub(polymorphicMessageDestination, messageString); >> > >> > >> > Questions: >> > >> > 1. Is this the best way to get a string from my C++ block into my >> python application? >> > 2. Is there a way to pass my messages that doesn’t result in memory >> growing unbounded? >> > >> > >> > Thank you >> > Brad >> > >> > _______________________________________________ >> > Discuss-gnuradio mailing list >> > Discuss-gnuradio@gnu.org >> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> > _______________________________________________ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >
_______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
