Thanks for the clever insights, Marcus. I'll consider the PMT vector approach you mentioned, while at the same time knowing PMT is likely to undergo changes in the future as you mentioned, I'll also look into using TCP/IP to pass the needed information, perhaps ZeroMQ or a UDP socket even.
On Fri, Mar 1, 2019 at 1:50 PM Müller, Marcus (CEL) <muel...@kit.edu> wrote: > Hi Brad, > > so, yes, your observation is correct: PMT symbols are/were meant to be > used as "identifiers", not as "data carriers"; the motivation behind > the hash table you find in pmt.cc is that there's only one instance of > any given PMT symbol, and thus, a simple address comparison suffices to > tell whether two PMT symbols are the same. > > You'll notice that on x86 (and presumably modern ARM) string comparison > is pretty fast, and that you'd need to do *a lot* comparisons to offset > the cost of hashing a symbol once. Anyway, yes, that table grows > unboundedly. > > Since your string isn't actually a "symbol", I'd recommend simply > encoding it safely (that's probably already the case), and putting it > in a uniform PMT vector of 8bit integers (u8vector). > > On a different note, there's actually "unintended" (as in: I don't > intend GNU Radio to have an unbounded hash map, but it's at least what > was originally intended) memory leaks with PMTs and tag_t on the > Python/C++ boundary, and there's quite some broken concepts within PMT. > > Long or medium term, we'll be replacing PMT with something that is > actually a common serialization format for usage in external software > (i.e. not for usage within the same flow graph), and ideally with the > unserialized universal container that comes with such a format. Stay > tuned. However, not happening in 3.8 or anything before. > > Best regards, > Marcus > > On Thu, 2019-02-28 at 14:48 -0500, Brad Hein wrote: > > > > In my gnuradio test application I’m attempting to pass a JSON formatted > string from my C++ custom block, to my python flow graph containing the > block. > > > > The String message are being received by my python flow graph, but > there’s a memory leak. Over time, my flowgraph RSS (memory footprint) grows > at about 32k/s up until the application runs out of memory. > > > > The leak didn’t start until after I retrofitted my flowgraph and custom > block with polymorphic type (PMT) based message passing. I’m passing a 200 > or so byte JSON string (as a symbol) from C++ block to python flow graph > about 60 times per second. > > > > Sleuthing through the pmt.cc code [1] I see the String (symbol) objects > are stored in a hash. I suspect what is happening is that since all of my > JSON strings are unique, they’re getting added to the hash and causing the > hash to grow unbounded over time. > > > > From what I can tell by reading the wiki [2], the approach I’m using is > the best way to get a string from my custom block and into my python > flowgraph. > > > > [1] > https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/pmt/pmt.cc > > [2] https://www.gnuradio.org/doc/doxygen/page_pmt.html > > > > > > Sample c++ block code snippet from my work function: > > > > // Calculate metrics. Occasionally (60/second) we'll get back a JSON > formatted string of metrics. > > crossingData = calculateWaveCrossingMetrics(lastSampleValue,in[i]); > > > > if (crossingData.length() > 0) { > > pmt::pmt_t messageString; > > messageString = pmt::string_to_symbol(crossingData.c_str()); > > message_port_pub(polymorphicMessageDestination, messageString); > > > > > > Questions: > > > > 1. Is this the best way to get a string from my C++ block into my > python application? > > 2. Is there a way to pass my messages that doesn’t result in memory > growing unbounded? > > > > > > Thank you > > Brad > > > > _______________________________________________ > > Discuss-gnuradio mailing list > > Discuss-gnuradio@gnu.org > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >
_______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
