Regarding the overriding of the cache directory (which I don't recommend you do) the reason that didn't work is because I gave you the wrong variable name. It should be `EM_CACHE` not `EMCC_CACHE`. Sorry about that!
On Thu, Oct 31, 2024 at 11:42 AM Sam Clegg <[email protected]> wrote: > Interesting. It seems like `os.access(path, os.W_OK)` must be returning > `False` in our python code for `upstream/emscripten/cache`. > > Are you able to touch a new file in `upstream/emscripten/cache`? > > What does python show when you run: python3 -c "import os; > print(os.access('upstream/emscripten/cache', os.W_OK))" > > On Thu, Oct 31, 2024 at 7:56 AM John Dallman <[email protected]> > wrote: > >> > Can you try running `emcc` with `EMCC_DEBUG=1` as well as `EMCC_CACHE` >> set? >> >> Done. >> >> > Do you see the ` 'Using home-directory for emscripten cache due to >> read-only root` message? >> >> Yes, I do. I have: >> >> $ touch $EMCC_CACHE/test.jgd >> $ ls -l $EMCC_CACHE >> total 0 >> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Oct 31 14:42 test.jgd >> $ ls -ld $EMCC_CACHE >> drwxrwxr-x 2 kerman UK-Parasolid-GG 4096 Oct 31 14:42 >> /local/kernel_webasm/tools/emcache/ >> $ python3 --version >> Python 3.6.8 >> >> > Where is your emscripten config coming from? Are you using `source >> emsdk_env.sh`? >> >> Yes, I'm sourcing that script. I have not altered any config files. >> >> > Making `emsdk/upstream/emscripten/cache` group writable is the correct >> solution, there >> > should be no need to serate of alternate cache directories in this case >> I think. >> >> Checking that: >> >> $ whoami >> kerman >> $ pwd >> /local/kernel_webasm/tools/emsdk >> $ ls -ld upstream/emscripten/cache >> drwxrwxr-x 4 kerman UK-Parasolid-GG 4096 Oct 24 23:29 >> upstream/emscripten/cache >> $ ls -l upstream/emscripten/cache >> total 12 >> drwxr-xr-x 2 kerman UK-Parasolid-GG 4096 Oct 24 23:29 build >> -rwxr-xr-x 1 kerman UK-Parasolid-GG 0 Oct 24 23:29 cache.lock >> drwxr-xr-x 5 kerman UK-Parasolid-GG 4096 Oct 24 23:18 sysroot >> -rw-r--r-- 1 kerman UK-Parasolid-GG 1 Oct 24 23:18 >> sysroot_install.stamp >> >> Those files were extracted with those timestamps: Oct 24 is well before I >> installed this Emscripten, and I don't work that late at night. Is the >> presence of that cache.lock file the problem? >> >> Thanks, >> >> John >> >> >> On Wed, Oct 30, 2024 at 6:33 PM 'Sam Clegg' via emscripten-discuss < >> [email protected]> wrote: >> >>> >>> >>> On Wed, Oct 30, 2024 at 5:46 AM John Dallman <[email protected]> >>> wrote: >>> >>>> This doesn't seem to be working. I'm using two accounts: "kerman" is my >>>> system management account, which can sudo, and "jgd" is my personal >>>> account. Both are members of the same primary group. >>>> >>>> I installed 3.1.70, the latest as of today, into >>>> /local/kernel_webasm/tools, and made emsdk/upstream/emscripten/cache >>>> group-writable. I then did a compile as jgd and found >>>> ~jgd/.emscripten_cache was populated and used. I removed that and thought >>>> again. >>>> >>>> I tried making the emsdk directory group-writable, and tried again. >>>> ~jgd/.emscripten_cache was populated and used. I removed it again. >>>> >>>> I created a cache directory, /local/kernel_webasm/tools/emcache/, made >>>> sure it was group-writable, and in my jgd session, did: >>>> >>>> EMCC_CACHE=/local/kernel_webasm/tools/emcache >>>> export EMCC_CACHE >>>> >>>> I tried compiling again, and once again, ~jgd/.emscripten_cache was >>>> populated and used. >>>> >>> >>> Something strange must be going on here. Can you try running `emcc` >>> with `EMCC_DEBUG=1` as well as `EMCC_CACHE` set? Do you see the ` 'Using >>> home-directory for emscripten cache due to read-only root` message? >>> >>> The `CACHE = os.path.expanduser(os.path.join('~', '.emscripten_cache'))` >>> should not even execute when you have an explicit cache configured. >>> >>> Where is your emscripten config coming from? Are you using `source >>> emsdk_env.sh`? >>> >>> Making `emsdk/upstream/emscripten/cache` group writable is the correct >>> solution, there should be no need to serate of alternate cache directories >>> in this case I think. >>> >>> cheers, >>> sam >>> >>> P.S. I am about to land a change that completely removes the fallback to >>> `$HOME/.emscripten_cache`: >>> https://github.com/emscripten-core/emscripten/pull/22801 >>> >>> >>> >>>> I'm running Rocky Linux 8.10. I really do want to have many accounts >>>> capable of using the same Emscripten installation, because putting it on a >>>> network drive makes standardizing development tools very much easier. >>>> >>>> Thanks, >>>> >>>> John >>>> >>>> On Tue, Oct 29, 2024 at 9:28 PM 'Sam Clegg' via emscripten-discuss < >>>> [email protected]> wrote: >>>> >>>>> The emscripten cache actually defaults to living inside of the >>>>> emscripten directory. The line where this occurs is `CACHE = >>>>> path_from_root('cache')`. Then `os.path.expanduser(os.path.join('~', >>>>> '.emscripten_cache'))` location is only used when the emscripten directory >>>>> is read only. >>>>> >>>>> When using emsdk the cache location should always be >>>>> `emsdk/upstream/emscripten/cache`. emsdk also shipped with a >>>>> pre-populated cache so most system libraries are already built there. >>>>> >>>>> For those who want to use a different cache location you can use the >>>>> `EMCC_CACHE` environment variable or the `CACHE` key in your emscripten >>>>> config file (both of these override the default). However, it doesn't >>>>> sounds like you need to do either of those things and the default location >>>>> inside the emsdk tree should work for you. Side note: emcc will also >>>>> using the $HOME location if the in-tree location is not writable. You can >>>>> set `EMCC_FROZEN_CACHE` if you want to accept this read-only location >>>>> (obviously new libraries cannot be placed there in that case though). See >>>>> https://github.com/emscripten-core/emscripten/blob/31f9fb3c71b1aacf5edf65e35515d41b59c10391/tools/config.py#L82-L92 >>>>> . >>>>> >>>>> cheers, >>>>> sam >>>>> >>>>> On Mon, Oct 28, 2024 at 10:25 AM John Dallman <[email protected]> >>>>> wrote: >>>>> >>>>>> Greetings, all! >>>>>> >>>>>> I have a somewhat unusual first project with Emscripten. I need to >>>>>> get a domain-specific C-like language generating WebAssembly. This is not >>>>>> too bad, because the DSL compiles into C code, which I then feed to the C >>>>>> compiler for the relevant platform. At this level, WebAssembly is "just >>>>>> another platform" but as always, the details are more complicated. >>>>>> >>>>>> I need to provide declarations for the platform's C run-time library >>>>>> functions, types, constants, and so on to the DSL. This is a fairly >>>>>> routine >>>>>> task, given the platform's C headers, but there are a few things about >>>>>> the >>>>>> Emscripten headers that are puzzling me. >>>>>> >>>>>> To forestall the obvious question, no, I can't just use the provided >>>>>> headers. The DSL is C-like, but has some syntax differences. Unlike C++, >>>>>> many C programs are not valid programs in the DSL, which gives much more >>>>>> freedom in the language design. It's a separate development that split >>>>>> from >>>>>> normal C in the mid-eighties and is still very much worth using for its >>>>>> specialised role. Yes, new hires have to learn it, which takes about two >>>>>> days for someone who knows C or C++. Learning about the specialised >>>>>> application area it is used for takes much longer. >>>>>> >>>>>> I'm running Emscripten on Linux. I started by looking at Emscripten >>>>>> 3.1.41, since that's the version used by a couple of other product teams >>>>>> that work on my site. That is fairly simple when I run a few emcc >>>>>> compiles >>>>>> with -H to get a report of what files are referenced. The top-level >>>>>> headers >>>>>> come from emsdk/upstream/emscripten/cache/sysroot/include. A few >>>>>> Clang-associated ones come from emsdk/upstream/lib/clang/17/include. >>>>>> That's >>>>>> all fine with me. >>>>>> >>>>>> Then I decided to check the latest emsdk, and found that with 3.1.69, >>>>>> the headers come from a different place. It builds a cache of the headers >>>>>> and other files it uses under ~/.emscripten_cache. The problems with that >>>>>> are: >>>>>> >>>>>> Storage: It will take 36MB in the user directory of everyone who ever >>>>>> compiles with Emscripten. We keep all our user directories on a server >>>>>> disk, because that's enormously convenient in many ways. But we really >>>>>> don't want to burn space with duplicates of that cache. >>>>>> >>>>>> Version lock: We need to be able to have several Emscripten versions >>>>>> in use simultaneously, by the same account, without conflicts. Our reason >>>>>> for this is that we plan to release products on WebAssembly, and from >>>>>> time >>>>>> to time, update the version of Emscripten we use, to get access to new C >>>>>> and C++ standards, compiler bug fixes, and so on. But we will not update >>>>>> the tools used to build a product version that's been released and is >>>>>> under >>>>>> maintenance, because we'd have to re-do a lot of the QA that we do at a >>>>>> release. So the service accounts that run our builds can't have caches of >>>>>> version-specific headers in their user directories. >>>>>> >>>>>> I need a way to tell Emscripten to put that cache somewhere else. I >>>>>> did some grep'ing of the Emscripten scripts, and found this line, in both >>>>>> 3.1.41 and 3.1.69: >>>>>> >>>>>> upstream/emscripten/tools/config.py: CACHE = >>>>>> os.path.expanduser(os.path.join('~', '.emscripten_cache')) >>>>>> >>>>>> I have not yet attempted to read the Python code and learn how it all >>>>>> works, because that could take ages; I don't know Python well at all and >>>>>> am >>>>>> not keen on it. I'm a naturally low-level programmer, much happier with >>>>>> assembly code than object orientation >>>>>> >>>>>> Is there an environment variable I can use to relocate that cache? >>>>>> >>>>>> Thanks very much, >>>>>> >>>>>> John >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion visit >>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion visit >> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-hp%2B2qcpv9vBv-trao9Z8n3xEMbiBYSh892SFDt7PMFA%40mail.gmail.com.
