This sounds like it's a DAWG-specific issue, not something specific to emscripten. The memory allocation behaviour should be mostly the same between an emscripten application and a natively compiled version of the same code. E.g. if your emscripten heap grows to 64 MBytes, a native version of your application should also use somewhere between 32 MBytes and 64 MBytes of process memory (usually a bit more than emscripten's because in native processes the overhead from operating system libraries is added as well).
Emscripten has a visual memory profiler which is linked in with the --memoryprofiler commandline option, maybe this can also help to pinpoint the issue (see: https://emscripten.org/docs/tools_reference/emcc.html). But most of the time I do this sort of debugging in the native version of the code (e.g. both Visual Studio and Xcode have very good memory debuggers / profilers which track all allocations and associate them with the source code location where the allocation happens). On Tuesday, 5 February 2019 13:01:47 UTC+1, [email protected] wrote: > > I am able to reduce initial memory allocation by setting TOTAL_MEMORY and > TOTAL_STACK parameters. > > ./emcc test/dawg-builder-test.bc -o dawg-builder-test.html -s > ALLOW_MEMORY_GROWTH=1 -s WASM=1 -s NO_EXIT_RUNTIME=1 -std=c++14 -s > EXPORTED_FUNCTIONS="['_addToDawg', '_findInDawg']" -s > EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' -s TOTAL_MEMORY=5242880 > -s TOTAL_STACK=242880 > > > When I am adding more data into DAWG graph then HEAP size getting > increased and reaches up to 64MB. I found HEAP array too big and having > lots of indexed with value as 0. What can be a possible issue? If I store > data in string variable then it is occupying less space (< 5MB). My purpose > is to further reduce space using DAWG but I am facing memory usage issue. > > On Monday, February 4, 2019 at 10:47:48 PM UTC+5:30, Floh wrote: >> >> The number in the screenshot is 16 MBytes, which seems quite reasonable >> to me (this is the default starting size for the emscripten heap). >> >> Anyway... you can tweak the starting size with the "-s TOTAL_MEMORY=xxx" >> linker option, see here: >> >> >> https://github.com/emscripten-core/emscripten/blob/dcf6dff7a85d1dfd4c2abd24f2b6a7e27e878f1c/src/settings.js#L91-L94 >> >> If you're concerned about memory usage I would suggest turning off memory >> growth, at least during development, that way you'll notice if you have a >> memleak somewhere. >> >> -Floh. >> >> On Monday, 4 February 2019 08:07:00 UTC+1, [email protected] wrote: >>> >>> Hi, >>> >>> Please find sample extension code with dawg implementation here >>> <https://github.com/Nitin-Er/DawgTest>. In screenshot, You can find >>> details of memory allocation. Follow below steps to check memory allocation. >>> >>> 1. Manually load extension in chrome browser. >>> 2. Go to chrome://extensions/ tab >>> 3. Find DAWG 2 extension block >>> 4. Click on background page link, Chrome Devtools will get opened. >>> 5. Go to memory tab. >>> 6. Take a heap short. >>> >>> On Saturday, February 2, 2019 at 3:11:14 AM UTC+5:30, Alon Zakai wrote: >>>> >>>> To add detail to that, it would be good to clarify what "HEAP size is >>>> too high" means - how are you measuring it? Too high compared to what? And >>>> same question about the zeros that are mentioned. >>>> >>>> On Fri, Feb 1, 2019 at 7:33 AM Shlomi Fish <[email protected]> >>>> wrote: >>>> >>>>> On Fri, 1 Feb 2019 05:27:55 -0800 (PST) >>>>> [email protected] wrote: >>>>> >>>>> > Hi, >>>>> > >>>>> > >>>>> > I am working on implementation of DAWG algorithm in my project. I am >>>>> using >>>>> > emscripten to convert it into webassembly. >>>>> > >>>>> > >>>>> > Library Link: here <http://code.google.com/p/dawgdic/> >>>>> > Code Link: here < >>>>> https://demophp.digi-corp.com/nitin/dawg-builder-test.cpp> >>>>> > >>>>> > >>>>> > ./emcc -g test/dawg-builder-test.bc -o dawg.html -s >>>>> NO_EXIT_RUNTIME=1 -s >>>>> > ALLOW_MEMORY_GROWTH=1 -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", >>>>> > "cwrap"]' -g4 -O2 >>>>> > >>>>> > >>>>> > I need to store multiple string of hostnames like this >>>>> > "lequipe.fr|ilgiornale.it|sportmediaset.mediaset.it|meteo.it| >>>>> ilmessaggero.it|ilmattino.it|today.it|romatoday.it|lavanguardia.com| >>>>> lalsace.fr|fussballtransfers.com|lejsl.com|bienpublic.com|dna.fr| >>>>> leprogres.fr|ledauphine.com|estrepublicain.fr|vosgesmatin.fr| >>>>> footmercato.net|fichajes.com|abc.es|buzzfeed.com|pcworld.es| >>>>> buzzfeednews.com|macworld.co.uk|cio.co.uk|computerworlduk.com........" >>>>> > >>>>> > >>>>> > When I load webassembly in javascript and load data then HEAP size >>>>> become >>>>> > too high..and I see lots of indexes with value 0. >>>>> > >>>>> >>>>> Hi! >>>>> >>>>> Please see https://github.com/shlomif/how-to-share-code-online and >>>>> https://github.com/shlomif/writing-the-perfect-question . >>>>> >>>>> >>>>> >>>>> > >>>>> > Please help me. >>>>> > >>>>> >>>>> >>>>> >>>>> -- >>>>> ----------------------------------------------------------------- >>>>> Shlomi Fish http://www.shlomifish.org/ >>>>> https://github.com/sindresorhus/awesome - curated list of lists >>>>> >>>>> The only reason some jokes never die is because Chuck Norris is not >>>>> interested >>>>> in killing them. >>>>> — http://www.shlomifish.org/humour/bits/facts/Chuck-Norris/ >>>>> >>>>> Please reply to list if it's a mailing list post - >>>>> http://shlom.in/reply . >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
