On Wed, Nov 6, 2024 at 6:39 AM John Dallman <[email protected]> wrote:
> Sorry to be slow replying, things have been busy.
>
> > Did you run this from the emsdk directory?
>
> No. I don't read Python well and didn't realise it was necessary. Silly of
> me.
>
> Can you add `print(os.stat('upstream/emscripten/cache')` too?
>
> Sure, and the missing ')'.
>
> $ echo $EMSDK
> /local/kernel_webasm/tools/emsdk
> $ pushd $EMSDK
> /local/kernel_webasm/tools/emsdk ~/regimes/tools_webasm/patssy/lx86
>
> $ cat ~/regimes/tools_webasm/pytest.sh
> python3 -c "import os; print(os.stat('upstream/emscripten/cache'))"
> python3 -c "import os; print(os.access('upstream/emscripten/cache',
> os.W_OK))"
>
> $ source ~/regimes/tools_webasm/pytest.sh
> os.stat_result(st_mode=16893, st_ino=101453189, st_dev=64774, st_nlink=4,
> st_uid=5790, st_gid=5200, st_size=4096, st_atime=1730862972,
> st_mtime=1730455667, st_ctime=1730455667)
> True
>
>
In that case I don't see how the code in cache.py could be detecting that
directory as not writable.
Can you try again after installing the latest emsdk version (3.1.71) which
completely removes the fallback to `$HOME/.emscripten_cache`?
> Thanks very much,
>
> John
>
> On Fri, Nov 1, 2024 at 5:49 PM 'Sam Clegg' via emscripten-discuss <
> [email protected]> wrote:
>
>>
>>
>> On Fri, Nov 1, 2024 at 3:18 AM John Dallman <[email protected]>
>> wrote:
>>
>>> > Are you able to touch a new file in `upstream/emscripten/cache`?
>>>
>>> Yes:
>>>
>>> $ echo $EMSDK
>>> /local/kernel_webasm/tools/emsdk
>>> $ echo $EMCC_CACHE
>>> <blank line>
>>> $ touch
>>> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>>> $ ls -l
>>> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>>> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Nov 1 10:07
>>> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>>>
>>> > What does python show when you run: python3 -c "import os;
>>> print(os.access('upstream/emscripten/cache', os.W_OK))"
>>>
>>> $> cat ../../pytest.sh
>>> python3 -c "import os; print(os.access('upstream/emscripten/cache',
>>> os.W_OK))"
>>> $ source ../../pytest.sh
>>>
>>
>> Did you run this from the emsdk directory?
>>
>> Can you add `print(os.stat('upstream/emscripten/cache')` too?
>>
>>
>>>
>> False
>>>
>>> Is my Python too old? It's the one that came with Rocky Linux 8.10.
>>>
>>> $ python3 --version
>>> Python 3.6.8
>>>
>>> Thanks very much,
>>>
>>> John
>>>
>>> On Thu, Oct 31, 2024 at 6:43 PM 'Sam Clegg' via emscripten-discuss <
>>> [email protected]> wrote:
>>>
>>>> Interesting. It seems like `os.access(path, os.W_OK)` must be
>>>> returning `False` in our python code for `upstream/emscripten/cache`.
>>>>
>>>> Are you able to touch a new file in `upstream/emscripten/cache`?
>>>>
>>>> What does python show when you run: python3 -c "import os;
>>>> print(os.access('upstream/emscripten/cache', os.W_OK))"
>>>>
>>>> On Thu, Oct 31, 2024 at 7:56 AM John Dallman <[email protected]>
>>>> wrote:
>>>>
>>>>> > Can you try running `emcc` with `EMCC_DEBUG=1` as well
>>>>> as `EMCC_CACHE` set?
>>>>>
>>>>> Done.
>>>>>
>>>>> > Do you see the ` 'Using home-directory for emscripten cache due to
>>>>> read-only root` message?
>>>>>
>>>>> Yes, I do. I have:
>>>>>
>>>>> $ touch $EMCC_CACHE/test.jgd
>>>>> $ ls -l $EMCC_CACHE
>>>>> total 0
>>>>> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Oct 31 14:42 test.jgd
>>>>> $ ls -ld $EMCC_CACHE
>>>>> drwxrwxr-x 2 kerman UK-Parasolid-GG 4096 Oct 31 14:42
>>>>> /local/kernel_webasm/tools/emcache/
>>>>> $ python3 --version
>>>>> Python 3.6.8
>>>>>
>>>>> > Where is your emscripten config coming from? Are you using `source
>>>>> emsdk_env.sh`?
>>>>>
>>>>> Yes, I'm sourcing that script. I have not altered any config files.
>>>>>
>>>>> > Making `emsdk/upstream/emscripten/cache` group writable is the
>>>>> correct solution, there
>>>>> > should be no need to serate of alternate cache directories in this
>>>>> case I think.
>>>>>
>>>>> Checking that:
>>>>>
>>>>> $ whoami
>>>>> kerman
>>>>> $ pwd
>>>>> /local/kernel_webasm/tools/emsdk
>>>>> $ ls -ld upstream/emscripten/cache
>>>>> drwxrwxr-x 4 kerman UK-Parasolid-GG 4096 Oct 24 23:29
>>>>> upstream/emscripten/cache
>>>>> $ ls -l upstream/emscripten/cache
>>>>> total 12
>>>>> drwxr-xr-x 2 kerman UK-Parasolid-GG 4096 Oct 24 23:29 build
>>>>> -rwxr-xr-x 1 kerman UK-Parasolid-GG 0 Oct 24 23:29 cache.lock
>>>>> drwxr-xr-x 5 kerman UK-Parasolid-GG 4096 Oct 24 23:18 sysroot
>>>>> -rw-r--r-- 1 kerman UK-Parasolid-GG 1 Oct 24 23:18
>>>>> sysroot_install.stamp
>>>>>
>>>>> Those files were extracted with those timestamps: Oct 24 is well
>>>>> before I installed this Emscripten, and I don't work that late at night.
>>>>> Is
>>>>> the presence of that cache.lock file the problem?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> John
>>>>>
>>>>>
>>>>> On Wed, Oct 30, 2024 at 6:33 PM 'Sam Clegg' via emscripten-discuss <
>>>>> [email protected]> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 30, 2024 at 5:46 AM John Dallman <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> This doesn't seem to be working. I'm using two accounts: "kerman" is
>>>>>>> my system management account, which can sudo, and "jgd" is my personal
>>>>>>> account. Both are members of the same primary group.
>>>>>>>
>>>>>>> I installed 3.1.70, the latest as of today, into
>>>>>>> /local/kernel_webasm/tools, and made emsdk/upstream/emscripten/cache
>>>>>>> group-writable. I then did a compile as jgd and found
>>>>>>> ~jgd/.emscripten_cache was populated and used. I removed that and
>>>>>>> thought
>>>>>>> again.
>>>>>>>
>>>>>>> I tried making the emsdk directory group-writable, and tried again.
>>>>>>> ~jgd/.emscripten_cache was populated and used. I removed it again.
>>>>>>>
>>>>>>> I created a cache directory, /local/kernel_webasm/tools/emcache/,
>>>>>>> made sure it was group-writable, and in my jgd session, did:
>>>>>>>
>>>>>>> EMCC_CACHE=/local/kernel_webasm/tools/emcache
>>>>>>> export EMCC_CACHE
>>>>>>>
>>>>>>> I tried compiling again, and once again, ~jgd/.emscripten_cache was
>>>>>>> populated and used.
>>>>>>>
>>>>>>
>>>>>> Something strange must be going on here. Can you try running `emcc`
>>>>>> with `EMCC_DEBUG=1` as well as `EMCC_CACHE` set? Do you see the `
>>>>>> 'Using
>>>>>> home-directory for emscripten cache due to read-only root` message?
>>>>>>
>>>>>> The `CACHE = os.path.expanduser(os.path.join('~',
>>>>>> '.emscripten_cache'))` should not even execute when you have an explicit
>>>>>> cache configured.
>>>>>>
>>>>>> Where is your emscripten config coming from? Are you using `source
>>>>>> emsdk_env.sh`?
>>>>>>
>>>>>> Making `emsdk/upstream/emscripten/cache` group writable is the
>>>>>> correct solution, there should be no need to serate of alternate cache
>>>>>> directories in this case I think.
>>>>>>
>>>>>> cheers,
>>>>>> sam
>>>>>>
>>>>>> P.S. I am about to land a change that completely removes the fallback
>>>>>> to `$HOME/.emscripten_cache`:
>>>>>> https://github.com/emscripten-core/emscripten/pull/22801
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I'm running Rocky Linux 8.10. I really do want to have many accounts
>>>>>>> capable of using the same Emscripten installation, because putting it
>>>>>>> on a
>>>>>>> network drive makes standardizing development tools very much easier.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> On Tue, Oct 29, 2024 at 9:28 PM 'Sam Clegg' via emscripten-discuss <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> The emscripten cache actually defaults to living inside of the
>>>>>>>> emscripten directory. The line where this occurs is `CACHE =
>>>>>>>> path_from_root('cache')`. Then `os.path.expanduser(os.path.join('~',
>>>>>>>> '.emscripten_cache'))` location is only used when the emscripten
>>>>>>>> directory
>>>>>>>> is read only.
>>>>>>>>
>>>>>>>> When using emsdk the cache location should always be
>>>>>>>> `emsdk/upstream/emscripten/cache`. emsdk also shipped with a
>>>>>>>> pre-populated cache so most system libraries are already built there.
>>>>>>>>
>>>>>>>> For those who want to use a different cache location you can use
>>>>>>>> the `EMCC_CACHE` environment variable or the `CACHE` key in your
>>>>>>>> emscripten
>>>>>>>> config file (both of these override the default). However, it doesn't
>>>>>>>> sounds like you need to do either of those things and the default
>>>>>>>> location
>>>>>>>> inside the emsdk tree should work for you. Side note: emcc will also
>>>>>>>> using the $HOME location if the in-tree location is not writable. You
>>>>>>>> can
>>>>>>>> set `EMCC_FROZEN_CACHE` if you want to accept this read-only location
>>>>>>>> (obviously new libraries cannot be placed there in that case though).
>>>>>>>> See
>>>>>>>> https://github.com/emscripten-core/emscripten/blob/31f9fb3c71b1aacf5edf65e35515d41b59c10391/tools/config.py#L82-L92
>>>>>>>> .
>>>>>>>>
>>>>>>>> cheers,
>>>>>>>> sam
>>>>>>>>
>>>>>>>> On Mon, Oct 28, 2024 at 10:25 AM John Dallman <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Greetings, all!
>>>>>>>>>
>>>>>>>>> I have a somewhat unusual first project with Emscripten. I need to
>>>>>>>>> get a domain-specific C-like language generating WebAssembly. This is
>>>>>>>>> not
>>>>>>>>> too bad, because the DSL compiles into C code, which I then feed to
>>>>>>>>> the C
>>>>>>>>> compiler for the relevant platform. At this level, WebAssembly is
>>>>>>>>> "just
>>>>>>>>> another platform" but as always, the details are more complicated.
>>>>>>>>>
>>>>>>>>> I need to provide declarations for the platform's C run-time
>>>>>>>>> library functions, types, constants, and so on to the DSL. This is a
>>>>>>>>> fairly
>>>>>>>>> routine task, given the platform's C headers, but there are a few
>>>>>>>>> things
>>>>>>>>> about the Emscripten headers that are puzzling me.
>>>>>>>>>
>>>>>>>>> To forestall the obvious question, no, I can't just use the
>>>>>>>>> provided headers. The DSL is C-like, but has some syntax differences.
>>>>>>>>> Unlike C++, many C programs are not valid programs in the DSL, which
>>>>>>>>> gives
>>>>>>>>> much more freedom in the language design. It's a separate development
>>>>>>>>> that
>>>>>>>>> split from normal C in the mid-eighties and is still very much worth
>>>>>>>>> using
>>>>>>>>> for its specialised role. Yes, new hires have to learn it, which takes
>>>>>>>>> about two days for someone who knows C or C++. Learning about the
>>>>>>>>> specialised application area it is used for takes much longer.
>>>>>>>>>
>>>>>>>>> I'm running Emscripten on Linux. I started by looking at
>>>>>>>>> Emscripten 3.1.41, since that's the version used by a couple of other
>>>>>>>>> product teams that work on my site. That is fairly simple when I run
>>>>>>>>> a few
>>>>>>>>> emcc compiles with -H to get a report of what files are referenced.
>>>>>>>>> The
>>>>>>>>> top-level headers come from
>>>>>>>>> emsdk/upstream/emscripten/cache/sysroot/include. A few
>>>>>>>>> Clang-associated
>>>>>>>>> ones come from emsdk/upstream/lib/clang/17/include. That's all fine
>>>>>>>>> with
>>>>>>>>> me.
>>>>>>>>>
>>>>>>>>> Then I decided to check the latest emsdk, and found that with
>>>>>>>>> 3.1.69, the headers come from a different place. It builds a cache of
>>>>>>>>> the
>>>>>>>>> headers and other files it uses under ~/.emscripten_cache. The
>>>>>>>>> problems
>>>>>>>>> with that are:
>>>>>>>>>
>>>>>>>>> Storage: It will take 36MB in the user directory of everyone who
>>>>>>>>> ever compiles with Emscripten. We keep all our user directories on a
>>>>>>>>> server
>>>>>>>>> disk, because that's enormously convenient in many ways. But we really
>>>>>>>>> don't want to burn space with duplicates of that cache.
>>>>>>>>>
>>>>>>>>> Version lock: We need to be able to have several Emscripten
>>>>>>>>> versions in use simultaneously, by the same account, without
>>>>>>>>> conflicts. Our
>>>>>>>>> reason for this is that we plan to release products on WebAssembly,
>>>>>>>>> and
>>>>>>>>> from time to time, update the version of Emscripten we use, to get
>>>>>>>>> access
>>>>>>>>> to new C and C++ standards, compiler bug fixes, and so on. But we
>>>>>>>>> will not
>>>>>>>>> update the tools used to build a product version that's been released
>>>>>>>>> and
>>>>>>>>> is under maintenance, because we'd have to re-do a lot of the QA that
>>>>>>>>> we do
>>>>>>>>> at a release. So the service accounts that run our builds can't have
>>>>>>>>> caches
>>>>>>>>> of version-specific headers in their user directories.
>>>>>>>>>
>>>>>>>>> I need a way to tell Emscripten to put that cache somewhere else.
>>>>>>>>> I did some grep'ing of the Emscripten scripts, and found this line,
>>>>>>>>> in both
>>>>>>>>> 3.1.41 and 3.1.69:
>>>>>>>>>
>>>>>>>>> upstream/emscripten/tools/config.py: CACHE =
>>>>>>>>> os.path.expanduser(os.path.join('~', '.emscripten_cache'))
>>>>>>>>>
>>>>>>>>> I have not yet attempted to read the Python code and learn how it
>>>>>>>>> all works, because that could take ages; I don't know Python well at
>>>>>>>>> all
>>>>>>>>> and am not keen on it. I'm a naturally low-level programmer, much
>>>>>>>>> happier
>>>>>>>>> with assembly code than object orientation
>>>>>>>>>
>>>>>>>>> Is there an environment variable I can use to relocate that cache?
>>>>>>>>>
>>>>>>>>> Thanks very much,
>>>>>>>>>
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "emscripten-discuss" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to [email protected].
>>>>>>>>> To view this discussion visit
>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com
>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "emscripten-discuss" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to [email protected].
>>>>>>>> To view this discussion visit
>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com
>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>> .
>>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "emscripten-discuss" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> To view this discussion visit
>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com
>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "emscripten-discuss" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion visit
>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "emscripten-discuss" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion visit
>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com
>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "emscripten-discuss" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion visit
>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2837g9UFhYbz9o6r_eF5RYp5--Xog4UbPmuPQX_65j7Pw%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2837g9UFhYbz9o6r_eF5RYp5--Xog4UbPmuPQX_65j7Pw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "emscripten-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion visit
>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgmpJbfFdmeGLtsTrvAwKAgd8x%2B1jaNGJtuobp8PYVcp7Q%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgmpJbfFdmeGLtsTrvAwKAgd8x%2B1jaNGJtuobp8PYVcp7Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion visit
>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va288ZgNXryD78HOp-dTgC9CLJ%3DU5xNFhLOy_DY_HD-50hw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va288ZgNXryD78HOp-dTgC9CLJ%3DU5xNFhLOy_DY_HD-50hw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion visit
> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqg%3DgisAnDkhFmLz3EVuTVh5pFhkffp_umeCDKk6-CQN_NQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqg%3DgisAnDkhFmLz3EVuTVh5pFhkffp_umeCDKk6-CQN_NQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/emscripten-discuss/CAL_va296bBv-OdD9281oVg-GcmQJPEGXuRzLhmR5rpdmhha-sw%40mail.gmail.com.
