On Fri, Nov 1, 2024 at 3:18 AM John Dallman <[email protected]> wrote:
> > Are you able to touch a new file in `upstream/emscripten/cache`?
>
> Yes:
>
> $ echo $EMSDK
> /local/kernel_webasm/tools/emsdk
> $ echo $EMCC_CACHE
> <blank line>
> $ touch /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
> $ ls -l /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Nov 1 10:07
> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>
> > What does python show when you run: python3 -c "import os;
> print(os.access('upstream/emscripten/cache', os.W_OK))"
>
> $> cat ../../pytest.sh
> python3 -c "import os; print(os.access('upstream/emscripten/cache',
> os.W_OK))"
> $ source ../../pytest.sh
>
Did you run this from the emsdk directory?
Can you add `print(os.stat('upstream/emscripten/cache')` too?
>
False
>
> Is my Python too old? It's the one that came with Rocky Linux 8.10.
>
> $ python3 --version
> Python 3.6.8
>
> Thanks very much,
>
> John
>
> On Thu, Oct 31, 2024 at 6:43 PM 'Sam Clegg' via emscripten-discuss <
> [email protected]> wrote:
>
>> Interesting. It seems like `os.access(path, os.W_OK)` must be returning
>> `False` in our python code for `upstream/emscripten/cache`.
>>
>> Are you able to touch a new file in `upstream/emscripten/cache`?
>>
>> What does python show when you run: python3 -c "import os;
>> print(os.access('upstream/emscripten/cache', os.W_OK))"
>>
>> On Thu, Oct 31, 2024 at 7:56 AM John Dallman <[email protected]>
>> wrote:
>>
>>> > Can you try running `emcc` with `EMCC_DEBUG=1` as well as `EMCC_CACHE`
>>> set?
>>>
>>> Done.
>>>
>>> > Do you see the ` 'Using home-directory for emscripten cache due to
>>> read-only root` message?
>>>
>>> Yes, I do. I have:
>>>
>>> $ touch $EMCC_CACHE/test.jgd
>>> $ ls -l $EMCC_CACHE
>>> total 0
>>> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Oct 31 14:42 test.jgd
>>> $ ls -ld $EMCC_CACHE
>>> drwxrwxr-x 2 kerman UK-Parasolid-GG 4096 Oct 31 14:42
>>> /local/kernel_webasm/tools/emcache/
>>> $ python3 --version
>>> Python 3.6.8
>>>
>>> > Where is your emscripten config coming from? Are you using `source
>>> emsdk_env.sh`?
>>>
>>> Yes, I'm sourcing that script. I have not altered any config files.
>>>
>>> > Making `emsdk/upstream/emscripten/cache` group writable is the correct
>>> solution, there
>>> > should be no need to serate of alternate cache directories in this
>>> case I think.
>>>
>>> Checking that:
>>>
>>> $ whoami
>>> kerman
>>> $ pwd
>>> /local/kernel_webasm/tools/emsdk
>>> $ ls -ld upstream/emscripten/cache
>>> drwxrwxr-x 4 kerman UK-Parasolid-GG 4096 Oct 24 23:29
>>> upstream/emscripten/cache
>>> $ ls -l upstream/emscripten/cache
>>> total 12
>>> drwxr-xr-x 2 kerman UK-Parasolid-GG 4096 Oct 24 23:29 build
>>> -rwxr-xr-x 1 kerman UK-Parasolid-GG 0 Oct 24 23:29 cache.lock
>>> drwxr-xr-x 5 kerman UK-Parasolid-GG 4096 Oct 24 23:18 sysroot
>>> -rw-r--r-- 1 kerman UK-Parasolid-GG 1 Oct 24 23:18
>>> sysroot_install.stamp
>>>
>>> Those files were extracted with those timestamps: Oct 24 is well before
>>> I installed this Emscripten, and I don't work that late at night. Is the
>>> presence of that cache.lock file the problem?
>>>
>>> Thanks,
>>>
>>> John
>>>
>>>
>>> On Wed, Oct 30, 2024 at 6:33 PM 'Sam Clegg' via emscripten-discuss <
>>> [email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Oct 30, 2024 at 5:46 AM John Dallman <[email protected]>
>>>> wrote:
>>>>
>>>>> This doesn't seem to be working. I'm using two accounts: "kerman" is
>>>>> my system management account, which can sudo, and "jgd" is my personal
>>>>> account. Both are members of the same primary group.
>>>>>
>>>>> I installed 3.1.70, the latest as of today, into
>>>>> /local/kernel_webasm/tools, and made emsdk/upstream/emscripten/cache
>>>>> group-writable. I then did a compile as jgd and found
>>>>> ~jgd/.emscripten_cache was populated and used. I removed that and thought
>>>>> again.
>>>>>
>>>>> I tried making the emsdk directory group-writable, and tried again.
>>>>> ~jgd/.emscripten_cache was populated and used. I removed it again.
>>>>>
>>>>> I created a cache directory, /local/kernel_webasm/tools/emcache/, made
>>>>> sure it was group-writable, and in my jgd session, did:
>>>>>
>>>>> EMCC_CACHE=/local/kernel_webasm/tools/emcache
>>>>> export EMCC_CACHE
>>>>>
>>>>> I tried compiling again, and once again, ~jgd/.emscripten_cache was
>>>>> populated and used.
>>>>>
>>>>
>>>> Something strange must be going on here. Can you try running `emcc`
>>>> with `EMCC_DEBUG=1` as well as `EMCC_CACHE` set? Do you see the ` 'Using
>>>> home-directory for emscripten cache due to read-only root` message?
>>>>
>>>> The `CACHE = os.path.expanduser(os.path.join('~',
>>>> '.emscripten_cache'))` should not even execute when you have an explicit
>>>> cache configured.
>>>>
>>>> Where is your emscripten config coming from? Are you using `source
>>>> emsdk_env.sh`?
>>>>
>>>> Making `emsdk/upstream/emscripten/cache` group writable is the correct
>>>> solution, there should be no need to serate of alternate cache directories
>>>> in this case I think.
>>>>
>>>> cheers,
>>>> sam
>>>>
>>>> P.S. I am about to land a change that completely removes the fallback
>>>> to `$HOME/.emscripten_cache`:
>>>> https://github.com/emscripten-core/emscripten/pull/22801
>>>>
>>>>
>>>>
>>>>> I'm running Rocky Linux 8.10. I really do want to have many accounts
>>>>> capable of using the same Emscripten installation, because putting it on a
>>>>> network drive makes standardizing development tools very much easier.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> John
>>>>>
>>>>> On Tue, Oct 29, 2024 at 9:28 PM 'Sam Clegg' via emscripten-discuss <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> The emscripten cache actually defaults to living inside of the
>>>>>> emscripten directory. The line where this occurs is `CACHE =
>>>>>> path_from_root('cache')`. Then `os.path.expanduser(os.path.join('~',
>>>>>> '.emscripten_cache'))` location is only used when the emscripten
>>>>>> directory
>>>>>> is read only.
>>>>>>
>>>>>> When using emsdk the cache location should always be
>>>>>> `emsdk/upstream/emscripten/cache`. emsdk also shipped with a
>>>>>> pre-populated cache so most system libraries are already built there.
>>>>>>
>>>>>> For those who want to use a different cache location you can use the
>>>>>> `EMCC_CACHE` environment variable or the `CACHE` key in your emscripten
>>>>>> config file (both of these override the default). However, it doesn't
>>>>>> sounds like you need to do either of those things and the default
>>>>>> location
>>>>>> inside the emsdk tree should work for you. Side note: emcc will also
>>>>>> using the $HOME location if the in-tree location is not writable. You
>>>>>> can
>>>>>> set `EMCC_FROZEN_CACHE` if you want to accept this read-only location
>>>>>> (obviously new libraries cannot be placed there in that case though).
>>>>>> See
>>>>>> https://github.com/emscripten-core/emscripten/blob/31f9fb3c71b1aacf5edf65e35515d41b59c10391/tools/config.py#L82-L92
>>>>>> .
>>>>>>
>>>>>> cheers,
>>>>>> sam
>>>>>>
>>>>>> On Mon, Oct 28, 2024 at 10:25 AM John Dallman <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Greetings, all!
>>>>>>>
>>>>>>> I have a somewhat unusual first project with Emscripten. I need to
>>>>>>> get a domain-specific C-like language generating WebAssembly. This is
>>>>>>> not
>>>>>>> too bad, because the DSL compiles into C code, which I then feed to the
>>>>>>> C
>>>>>>> compiler for the relevant platform. At this level, WebAssembly is "just
>>>>>>> another platform" but as always, the details are more complicated.
>>>>>>>
>>>>>>> I need to provide declarations for the platform's C run-time library
>>>>>>> functions, types, constants, and so on to the DSL. This is a fairly
>>>>>>> routine
>>>>>>> task, given the platform's C headers, but there are a few things about
>>>>>>> the
>>>>>>> Emscripten headers that are puzzling me.
>>>>>>>
>>>>>>> To forestall the obvious question, no, I can't just use the provided
>>>>>>> headers. The DSL is C-like, but has some syntax differences. Unlike C++,
>>>>>>> many C programs are not valid programs in the DSL, which gives much more
>>>>>>> freedom in the language design. It's a separate development that split
>>>>>>> from
>>>>>>> normal C in the mid-eighties and is still very much worth using for its
>>>>>>> specialised role. Yes, new hires have to learn it, which takes about two
>>>>>>> days for someone who knows C or C++. Learning about the specialised
>>>>>>> application area it is used for takes much longer.
>>>>>>>
>>>>>>> I'm running Emscripten on Linux. I started by looking at Emscripten
>>>>>>> 3.1.41, since that's the version used by a couple of other product teams
>>>>>>> that work on my site. That is fairly simple when I run a few emcc
>>>>>>> compiles
>>>>>>> with -H to get a report of what files are referenced. The top-level
>>>>>>> headers
>>>>>>> come from emsdk/upstream/emscripten/cache/sysroot/include. A few
>>>>>>> Clang-associated ones come from emsdk/upstream/lib/clang/17/include.
>>>>>>> That's
>>>>>>> all fine with me.
>>>>>>>
>>>>>>> Then I decided to check the latest emsdk, and found that with
>>>>>>> 3.1.69, the headers come from a different place. It builds a cache of
>>>>>>> the
>>>>>>> headers and other files it uses under ~/.emscripten_cache. The problems
>>>>>>> with that are:
>>>>>>>
>>>>>>> Storage: It will take 36MB in the user directory of everyone who
>>>>>>> ever compiles with Emscripten. We keep all our user directories on a
>>>>>>> server
>>>>>>> disk, because that's enormously convenient in many ways. But we really
>>>>>>> don't want to burn space with duplicates of that cache.
>>>>>>>
>>>>>>> Version lock: We need to be able to have several Emscripten versions
>>>>>>> in use simultaneously, by the same account, without conflicts. Our
>>>>>>> reason
>>>>>>> for this is that we plan to release products on WebAssembly, and from
>>>>>>> time
>>>>>>> to time, update the version of Emscripten we use, to get access to new C
>>>>>>> and C++ standards, compiler bug fixes, and so on. But we will not update
>>>>>>> the tools used to build a product version that's been released and is
>>>>>>> under
>>>>>>> maintenance, because we'd have to re-do a lot of the QA that we do at a
>>>>>>> release. So the service accounts that run our builds can't have caches
>>>>>>> of
>>>>>>> version-specific headers in their user directories.
>>>>>>>
>>>>>>> I need a way to tell Emscripten to put that cache somewhere else. I
>>>>>>> did some grep'ing of the Emscripten scripts, and found this line, in
>>>>>>> both
>>>>>>> 3.1.41 and 3.1.69:
>>>>>>>
>>>>>>> upstream/emscripten/tools/config.py: CACHE =
>>>>>>> os.path.expanduser(os.path.join('~', '.emscripten_cache'))
>>>>>>>
>>>>>>> I have not yet attempted to read the Python code and learn how it
>>>>>>> all works, because that could take ages; I don't know Python well at all
>>>>>>> and am not keen on it. I'm a naturally low-level programmer, much
>>>>>>> happier
>>>>>>> with assembly code than object orientation
>>>>>>>
>>>>>>> Is there an environment variable I can use to relocate that cache?
>>>>>>>
>>>>>>> Thanks very much,
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "emscripten-discuss" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> To view this discussion visit
>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com
>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "emscripten-discuss" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion visit
>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "emscripten-discuss" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion visit
>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com
>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "emscripten-discuss" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion visit
>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "emscripten-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion visit
>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion visit
>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2837g9UFhYbz9o6r_eF5RYp5--Xog4UbPmuPQX_65j7Pw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2837g9UFhYbz9o6r_eF5RYp5--Xog4UbPmuPQX_65j7Pw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion visit
> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgmpJbfFdmeGLtsTrvAwKAgd8x%2B1jaNGJtuobp8PYVcp7Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgmpJbfFdmeGLtsTrvAwKAgd8x%2B1jaNGJtuobp8PYVcp7Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/emscripten-discuss/CAL_va288ZgNXryD78HOp-dTgC9CLJ%3DU5xNFhLOy_DY_HD-50hw%40mail.gmail.com.
