I used mallinfo <https://github.com/emscripten-core/emscripten/blob/5187ea263120ff57424a184c1bc73259e24dc9a9/tests/mallinfo.cpp> to figure out memory issue and looks like DAWG is taking more memory as I am storing whole generated DAWG in another vector by giving pointer reference.
On Wednesday, February 6, 2019 at 5:50:02 PM UTC+5:30, Floh wrote: > > This sounds like it's a DAWG-specific issue, not something specific to > emscripten. The memory allocation behaviour should be mostly the same > between an emscripten application and a natively compiled version of the > same code. E.g. if your emscripten heap grows to 64 MBytes, a native > version of your application should also use somewhere between 32 MBytes and > 64 MBytes of process memory (usually a bit more than emscripten's because > in native processes the overhead from operating system libraries is added > as well). > > Emscripten has a visual memory profiler which is linked in with the > --memoryprofiler commandline option, maybe this can also help to pinpoint > the issue (see: https://emscripten.org/docs/tools_reference/emcc.html). > But most of the time I do this sort of debugging in the native version of > the code (e.g. both Visual Studio and Xcode have very good memory debuggers > / profilers which track all allocations and associate them with the source > code location where the allocation happens). > > On Tuesday, 5 February 2019 13:01:47 UTC+1, [email protected] wrote: >> >> I am able to reduce initial memory allocation by setting TOTAL_MEMORY and >> TOTAL_STACK parameters. >> >> ./emcc test/dawg-builder-test.bc -o dawg-builder-test.html -s >> ALLOW_MEMORY_GROWTH=1 -s WASM=1 -s NO_EXIT_RUNTIME=1 -std=c++14 -s >> EXPORTED_FUNCTIONS="['_addToDawg', '_findInDawg']" -s >> EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' -s TOTAL_MEMORY=5242880 >> -s TOTAL_STACK=242880 >> >> >> When I am adding more data into DAWG graph then HEAP size getting >> increased and reaches up to 64MB. I found HEAP array too big and having >> lots of indexed with value as 0. What can be a possible issue? If I store >> data in string variable then it is occupying less space (< 5MB). My purpose >> is to further reduce space using DAWG but I am facing memory usage issue. >> >> On Monday, February 4, 2019 at 10:47:48 PM UTC+5:30, Floh wrote: >>> >>> The number in the screenshot is 16 MBytes, which seems quite reasonable >>> to me (this is the default starting size for the emscripten heap). >>> >>> Anyway... you can tweak the starting size with the "-s TOTAL_MEMORY=xxx" >>> linker option, see here: >>> >>> >>> https://github.com/emscripten-core/emscripten/blob/dcf6dff7a85d1dfd4c2abd24f2b6a7e27e878f1c/src/settings.js#L91-L94 >>> >>> If you're concerned about memory usage I would suggest turning off >>> memory growth, at least during development, that way you'll notice if you >>> have a memleak somewhere. >>> >>> -Floh. >>> >>> On Monday, 4 February 2019 08:07:00 UTC+1, [email protected] >>> wrote: >>>> >>>> Hi, >>>> >>>> Please find sample extension code with dawg implementation here >>>> <https://github.com/Nitin-Er/DawgTest>. In screenshot, You can find >>>> details of memory allocation. Follow below steps to check memory >>>> allocation. >>>> >>>> 1. Manually load extension in chrome browser. >>>> 2. Go to chrome://extensions/ tab >>>> 3. Find DAWG 2 extension block >>>> 4. Click on background page link, Chrome Devtools will get opened. >>>> 5. Go to memory tab. >>>> 6. Take a heap short. >>>> >>>> On Saturday, February 2, 2019 at 3:11:14 AM UTC+5:30, Alon Zakai wrote: >>>>> >>>>> To add detail to that, it would be good to clarify what "HEAP size is >>>>> too high" means - how are you measuring it? Too high compared to what? >>>>> And >>>>> same question about the zeros that are mentioned. >>>>> >>>>> On Fri, Feb 1, 2019 at 7:33 AM Shlomi Fish <[email protected]> >>>>> wrote: >>>>> >>>>>> On Fri, 1 Feb 2019 05:27:55 -0800 (PST) >>>>>> [email protected] wrote: >>>>>> >>>>>> > Hi, >>>>>> > >>>>>> > >>>>>> > I am working on implementation of DAWG algorithm in my project. I >>>>>> am using >>>>>> > emscripten to convert it into webassembly. >>>>>> > >>>>>> > >>>>>> > Library Link: here <http://code.google.com/p/dawgdic/> >>>>>> > Code Link: here < >>>>>> https://demophp.digi-corp.com/nitin/dawg-builder-test.cpp> >>>>>> > >>>>>> > >>>>>> > ./emcc -g test/dawg-builder-test.bc -o dawg.html -s >>>>>> NO_EXIT_RUNTIME=1 -s >>>>>> > ALLOW_MEMORY_GROWTH=1 -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", >>>>>> > "cwrap"]' -g4 -O2 >>>>>> > >>>>>> > >>>>>> > I need to store multiple string of hostnames like this >>>>>> > "lequipe.fr|ilgiornale.it|sportmediaset.mediaset.it|meteo.it| >>>>>> ilmessaggero.it|ilmattino.it|today.it|romatoday.it|lavanguardia.com| >>>>>> lalsace.fr|fussballtransfers.com|lejsl.com|bienpublic.com|dna.fr| >>>>>> leprogres.fr|ledauphine.com|estrepublicain.fr|vosgesmatin.fr| >>>>>> footmercato.net|fichajes.com|abc.es|buzzfeed.com|pcworld.es| >>>>>> buzzfeednews.com|macworld.co.uk|cio.co.uk >>>>>> |computerworlduk.com........" >>>>>> > >>>>>> > >>>>>> > When I load webassembly in javascript and load data then HEAP size >>>>>> become >>>>>> > too high..and I see lots of indexes with value 0. >>>>>> > >>>>>> >>>>>> Hi! >>>>>> >>>>>> Please see https://github.com/shlomif/how-to-share-code-online and >>>>>> https://github.com/shlomif/writing-the-perfect-question . >>>>>> >>>>>> >>>>>> >>>>>> > >>>>>> > Please help me. >>>>>> > >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ----------------------------------------------------------------- >>>>>> Shlomi Fish http://www.shlomifish.org/ >>>>>> https://github.com/sindresorhus/awesome - curated list of lists >>>>>> >>>>>> The only reason some jokes never die is because Chuck Norris is not >>>>>> interested >>>>>> in killing them. >>>>>> — http://www.shlomifish.org/humour/bits/facts/Chuck-Norris/ >>>>>> >>>>>> Please reply to list if it's a mailing list post - >>>>>> http://shlom.in/reply . >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
