bug#52139: jupyter trying to modify /gnu/store
Hi Alexander, > [W 13:52:18.502 LabApp] 500 PUT /lab/api/workspaces/lab?1639227138494 > (127.0.0.1): [Errno 30] Read-only file system: > '[user]/.guix-profile/etc/jupyter/lab' I’ve fixed this via https://github.com/guix-science/guix-science/commit/ca5d2b79cc730f6d52f93f4e7347102b620ed988 Cheers, Lars
bug#52139: jupyter trying to modify /gnu/store
Hi, > If that works for you, go for it. merged as c63b55d1283d9a4bfc5ecaf1cab01cd98a467b69. I’ll investigate why jupyterlab is trying to write to /gnu/store when trying to save settings. Cheers, Lars
bug#52139: jupyter trying to modify /gnu/store
Hi Lars,
Lars-Dominik Braun skribis:
>> But precisely: as Alexander wrote, when JUPYTER_CONFIG_DIR points to the
>> store, jupyterlab cannot drop a config file there. Or am I missing
>> something?
> sorry, my message was unclear here. The config file is written at
> build time.
Oh I see.
>> BTW, if JUPYTER_CONFIG_DIR is meant to contain a directory name, as
>> opposed to a colon-separated search path, we should make this change:
> Looking at the documentation[1] again this is correct, but I feel we
> should use JUPYTER_CONFIG_PATH instead, because it supports
> colon-delimited entries, see attached patch. However that does not get
> rid of error messages like these, when trying to use Settings→JupyterLab
> Theme for example:
>
> [W 08:10:14.476 LabApp] 500 PUT /lab/api/workspaces/lab?1639383014500
> (127.0.0.1): [Errno 30] Read-only file system:
> '/gnu/store/8q7wdpdddfqh46plbbsa3rwci5092n5y-profile/etc/jupyter/lab'
>
> So it seems that JUPYTER_CONFIG_PATH overrides the default
> JUPYTER_CONFIG_DIR, when the latter is not set. Or maybe guix-science’s
> jupyterlab is simply too old – not sure right now.
OK.
> +++ b/gnu/packages/python-xyz.scm
> @@ -8478,7 +8478,7 @@ (define-public python-jupyter-core
> ;; search paths.
> (native-search-paths
> (list (search-path-specification
> -(variable "JUPYTER_CONFIG_DIR")
> +(variable "JUPYTER_CONFIG_PATH")
> (files '("etc/jupyter")))
> (search-path-specification
> (variable "JUPYTER_PATH")
> @@ -12145,8 +12145,6 @@ (define-public python-nbconvert
> (when tests?
> ;; Some tests invoke the installed nbconvert binary.
> (add-installed-pythonpath inputs outputs)
> - ;; Tries to write to this path.
> - (unsetenv "JUPYTER_CONFIG_DIR")
> ;; Tests depend on templates installed to output.
> (setenv "JUPYTER_PATH"
> (string-append
> @@ -12254,6 +12252,8 @@ (define-public python-notebook
> ;; Some tests do not expect all files to be installed in the
> ;; same directory, but JUPYTER_PATH contains multiple entries.
> (unsetenv "JUPYTER_PATH")
> + ;; Interferes with tests that check paths.
> + (unsetenv "JUPYTER_CONFIG_PATH")
> ;; Some tests need HOME
> (setenv "HOME" "/tmp")
> (with-directory-excursion "/tmp"
If that works for you, go for it.
Thanks,
Ludo’.
bug#52139: jupyter trying to modify /gnu/store
Hi Ludo,
> But precisely: as Alexander wrote, when JUPYTER_CONFIG_DIR points to the
> store, jupyterlab cannot drop a config file there. Or am I missing
> something?
sorry, my message was unclear here. The config file is written at
build time.
> BTW, if JUPYTER_CONFIG_DIR is meant to contain a directory name, as
> opposed to a colon-separated search path, we should make this change:
Looking at the documentation[1] again this is correct, but I feel we
should use JUPYTER_CONFIG_PATH instead, because it supports
colon-delimited entries, see attached patch. However that does not get
rid of error messages like these, when trying to use Settings→JupyterLab
Theme for example:
[W 08:10:14.476 LabApp] 500 PUT /lab/api/workspaces/lab?1639383014500
(127.0.0.1): [Errno 30] Read-only file system:
'/gnu/store/8q7wdpdddfqh46plbbsa3rwci5092n5y-profile/etc/jupyter/lab'
So it seems that JUPYTER_CONFIG_PATH overrides the default
JUPYTER_CONFIG_DIR, when the latter is not set. Or maybe guix-science’s
jupyterlab is simply too old – not sure right now.
Cheers,
Lars
[1] https://jupyter.readthedocs.io/en/latest/use/jupyter-directories.html
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index db2ab8e5f0..450d17208f 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -8478,7 +8478,7 @@ (define-public python-jupyter-core
;; search paths.
(native-search-paths
(list (search-path-specification
-(variable "JUPYTER_CONFIG_DIR")
+(variable "JUPYTER_CONFIG_PATH")
(files '("etc/jupyter")))
(search-path-specification
(variable "JUPYTER_PATH")
@@ -12145,8 +12145,6 @@ (define-public python-nbconvert
(when tests?
;; Some tests invoke the installed nbconvert binary.
(add-installed-pythonpath inputs outputs)
- ;; Tries to write to this path.
- (unsetenv "JUPYTER_CONFIG_DIR")
;; Tests depend on templates installed to output.
(setenv "JUPYTER_PATH"
(string-append
@@ -12254,6 +12252,8 @@ (define-public python-notebook
;; Some tests do not expect all files to be installed in the
;; same directory, but JUPYTER_PATH contains multiple entries.
(unsetenv "JUPYTER_PATH")
+ ;; Interferes with tests that check paths.
+ (unsetenv "JUPYTER_CONFIG_PATH")
;; Some tests need HOME
(setenv "HOME" "/tmp")
(with-directory-excursion "/tmp"
bug#52139: jupyter trying to modify /gnu/store
Hi Lars,
Lars-Dominik Braun skribis:
>> The JUPYTER_CONFIG_DIR search path was added to ‘python-jupyter-core’ in
>> commit 1683e9d9677d54f695a54fc44ed309287989fc34. Lars, what are you
>> thoughts?
> Please *do not* remove this environment variable. It is required, because
> packages like jupyterlab drop a config file into this directory.
But precisely: as Alexander wrote, when JUPYTER_CONFIG_DIR points to the
store, jupyterlab cannot drop a config file there. Or am I missing
something?
BTW, if JUPYTER_CONFIG_DIR is meant to contain a directory name, as
opposed to a colon-separated search path, we should make this change:
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index b3a350aca4..57f66e18f1 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -8479,6 +8479,7 @@ (define-public python-jupyter-core
(native-search-paths
(list (search-path-specification
(variable "JUPYTER_CONFIG_DIR")
+(separator #f);single entry
(files '("etc/jupyter")))
(search-path-specification
(variable "JUPYTER_PATH")
WDYT?
Thanks,
Ludo’.
bug#52139: jupyter trying to modify /gnu/store
Nope: > stat: cannot statx > '/gnu/store/djb0y6kn86ijbvkx93q48q484krp1kbb-profile/etc/jupyter:/home/aster/.config/jupyter/migrated': > No such file or directory Cheers, Alex On Sat, Dec 11 2021, 10:36:45, Lars-Dominik Braun wrote: > Hi, > >> I can’t reproduce it with: […] > I cannot reproduce it either and this migration should not happen™, > because the phase 'disable-migration should disable it. (See > /gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py:145). > > Alexander: Does this file exists for you too? > > guix shell python-notebook -- sh -c "stat \$JUPYTER_CONFIG_DIR/migrated" > >> The JUPYTER_CONFIG_DIR search path was added to ‘python-jupyter-core’ in >> commit 1683e9d9677d54f695a54fc44ed309287989fc34. Lars, what are you >> thoughts? > Please *do not* remove this environment variable. It is required, because > packages like jupyterlab drop a config file into this directory. Without > it the jupyterlab extension will not be loaded by default and a simple > `guix shell python-jupyterlab -- jupyter lab` will not work. > > Cheers, > Lars signature.asc Description: PGP signature
bug#52139: jupyter trying to modify /gnu/store
Hi, > I can’t reproduce it with: […] I cannot reproduce it either and this migration should not happen™, because the phase 'disable-migration should disable it. (See /gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py:145). Alexander: Does this file exists for you too? guix shell python-notebook -- sh -c "stat \$JUPYTER_CONFIG_DIR/migrated" > The JUPYTER_CONFIG_DIR search path was added to ‘python-jupyter-core’ in > commit 1683e9d9677d54f695a54fc44ed309287989fc34. Lars, what are you > thoughts? Please *do not* remove this environment variable. It is required, because packages like jupyterlab drop a config file into this directory. Without it the jupyterlab extension will not be loaded by default and a simple `guix shell python-jupyterlab -- jupyter lab` will not work. Cheers, Lars
bug#52139: jupyter trying to modify /gnu/store
Hi Ludovic,
On Fri, Dec 10 2021, 14:27:49, Ludovic Courtès wrote:
> Hi Alexander,
>
> Alexander Asteroth skribis:
>
>> I found the cause:
>>
>> in ~/.guix-profile/etc/profile environment variables are set for jupyter:
>>
>>> export
>>> JUPYTER_PATH="${GUIX_PROFILE:-/gnu/store/ichrcarq2k4lhdmm2izc3f1xjbziz49v-profile}/share/jupyter${JUPYTER_PATH:+:}$JUPYTER_PATH"
>>> export
>>> JUPYTER_CONFIG_DIR="${GUIX_PROFILE:-/gnu/store/ichrcarq2k4lhdmm2izc3f1xjbziz49v-profile}/etc/jupyter${JUPYTER_CONFIG_DIR:+:}$JUPYTER_CONFIG_DIR"
>>
>> The jupyter config path causes the problem, since jupyter tries to write
>> into it (and it is located in a read only filesystem). Unsetting the
>> environment variable solves the issue.
>
> I can’t reproduce it with:
>
> $ guix describe
> Generacio 196 Dec 06 2021 11:23:08(nuna)
> guix 9478a25
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 9478a252509a93f3e7f486cbc549631185ba071d
> $ guix shell jupyter -CN --no-cwd -- jupyter notebook
>
> Perhaps that’s because I didn’t trigger actions that would cause Jupyter
> to attempt to write to $JUPYTER_CONFIG_DIR?
Yes. I happes for also only if I used jupyter once. At the second attempt
the error occured.
The problem is easily solved by setting the config dir to (e.g.)
~/.config/jupyter.
Cheers,
Alex
>
> The JUPYTER_CONFIG_DIR search path was added to ‘python-jupyter-core’ in
> commit 1683e9d9677d54f695a54fc44ed309287989fc34. Lars, what are you
> thoughts?
>
> Thanks,
> Ludo’.
signature.asc
Description: PGP signature
bug#52139: jupyter trying to modify /gnu/store
Hi Alexander,
Alexander Asteroth skribis:
> I found the cause:
>
> in ~/.guix-profile/etc/profile environment variables are set for jupyter:
>
>> export
>> JUPYTER_PATH="${GUIX_PROFILE:-/gnu/store/ichrcarq2k4lhdmm2izc3f1xjbziz49v-profile}/share/jupyter${JUPYTER_PATH:+:}$JUPYTER_PATH"
>> export
>> JUPYTER_CONFIG_DIR="${GUIX_PROFILE:-/gnu/store/ichrcarq2k4lhdmm2izc3f1xjbziz49v-profile}/etc/jupyter${JUPYTER_CONFIG_DIR:+:}$JUPYTER_CONFIG_DIR"
>
> The jupyter config path causes the problem, since jupyter tries to write
> into it (and it is located in a read only filesystem). Unsetting the
> environment variable solves the issue.
I can’t reproduce it with:
--8<---cut here---start->8---
$ guix describe
Generacio 196 Dec 06 2021 11:23:08(nuna)
guix 9478a25
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 9478a252509a93f3e7f486cbc549631185ba071d
$ guix shell jupyter -CN --no-cwd -- jupyter notebook
--8<---cut here---end--->8---
Perhaps that’s because I didn’t trigger actions that would cause Jupyter
to attempt to write to $JUPYTER_CONFIG_DIR?
The JUPYTER_CONFIG_DIR search path was added to ‘python-jupyter-core’ in
commit 1683e9d9677d54f695a54fc44ed309287989fc34. Lars, what are you
thoughts?
Thanks,
Ludo’.
bug#52139: jupyter trying to modify /gnu/store
I found the cause:
in ~/.guix-profile/etc/profile environment variables are set for jupyter:
> export
> JUPYTER_PATH="${GUIX_PROFILE:-/gnu/store/ichrcarq2k4lhdmm2izc3f1xjbziz49v-profile}/share/jupyter${JUPYTER_PATH:+:}$JUPYTER_PATH"
> export
> JUPYTER_CONFIG_DIR="${GUIX_PROFILE:-/gnu/store/ichrcarq2k4lhdmm2izc3f1xjbziz49v-profile}/etc/jupyter${JUPYTER_CONFIG_DIR:+:}$JUPYTER_CONFIG_DIR"
The jupyter config path causes the problem, since jupyter tries to write
into it (and it is located in a read only filesystem). Unsetting the
environment variable solves the issue.
Cheers,
Alex
On Sat, Nov 27 2021, 11:14:07, Alexander Asteroth
wrote:
> Dear all,
>
> yesterday I added jupyter to my profile and it worked perfectly
> fine. Overnight I shut down the computer and rebooted today.
> Now jupyter refuses to start with the messages below. It seems that
> jupyter-notebook tries to create directories (or modifiy them) in the
> location it
> is installed wich is mounted ro. I have no idea why this didn't happen
> yesterday but it seems that there is some misconfiguration somewhere...
>
> Cheers,
> Alex
>
> PS: same effect with jupyterlab from the guix-science channel.
>
> ---
> ~$ jupyter-notebook
> Traceback (most recent call last):
> File
> "/gnu/store/1v8dgchnis87a4p3g9i3j1ayv0dfr4ds-python-notebook-6.3.0/bin/.jupyter-notebook-real",
> line 11, in
> load_entry_point('notebook==6.3.0', 'console_scripts',
> 'jupyter-notebook')()
> File
> "/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
> line 254, in launch_instance
> return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
> File
> "/gnu/store/i037rnmqzwcc7nrgcswzb8dcckfrngzp-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
> line 663, in launch_instance
> app.initialize(argv)
> File
> "/gnu/store/f7y0fd230nc6l2qwq58g7q26rlkwjkf3-python-decorator-5.0.9/lib/python3.8/site-packages/decorator.py",
> line 232, in fun
> return caller(func, *(extras + args), **kw)
> File
> "/gnu/store/i037rnmqzwcc7nrgcswzb8dcckfrngzp-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
> line 87, in catch_config_error
> return method(app, *args, **kwargs)
> File
> "/gnu/store/1v8dgchnis87a4p3g9i3j1ayv0dfr4ds-python-notebook-6.3.0/lib/python3.8/site-packages/notebook/notebookapp.py",
> line 2122, in initialize
> super().initialize(argv)
> File
> "/gnu/store/f7y0fd230nc6l2qwq58g7q26rlkwjkf3-python-decorator-5.0.9/lib/python3.8/site-packages/decorator.py",
> line 232, in fun
> return caller(func, *(extras + args), **kw)
> File
> "/gnu/store/i037rnmqzwcc7nrgcswzb8dcckfrngzp-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
> line 87, in catch_config_error
> return method(app, *args, **kwargs)
> File
> "/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
> line 229, in initialize
> self.migrate_config()
> File
> "/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
> line 155, in migrate_config
> migrate()
> File
> "/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/migrate.py",
> line 244, in migrate
> ensure_dir_exists(env['jupyter_config'])
> File
> "/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/utils/__init__.py",
> line 11, in ensure_dir_exists
> os.makedirs(path, mode=mode)
> File
> "/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
> line 213, in makedirs
> makedirs(head, exist_ok=exist_ok)
> File
> "/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
> line 213, in makedirs
> makedirs(head, exist_ok=exist_ok)
> File
> "/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
> line 213, in makedirs
> makedirs(head, exist_ok=exist_ok)
> [Previous line repeated 7 more times]
> File
> "/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
> line 223, in makedirs
> mkdir(name, mode)
> OSError: [Errno 30] Read-only file system:
> '/home/[user]/.guix-profile/etc/jupyter:'
bug#52139: jupyter trying to modify /gnu/store
Dear all,
yesterday I added jupyter to my profile and it worked perfectly
fine. Overnight I shut down the computer and rebooted today.
Now jupyter refuses to start with the messages below. It seems that
jupyter-notebook tries to create directories (or modifiy them) in the location
it
is installed wich is mounted ro. I have no idea why this didn't happen
yesterday but it seems that there is some misconfiguration somewhere...
Cheers,
Alex
PS: same effect with jupyterlab from the guix-science channel.
---
~$ jupyter-notebook
Traceback (most recent call last):
File
"/gnu/store/1v8dgchnis87a4p3g9i3j1ayv0dfr4ds-python-notebook-6.3.0/bin/.jupyter-notebook-real",
line 11, in
load_entry_point('notebook==6.3.0', 'console_scripts', 'jupyter-notebook')()
File
"/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 254, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File
"/gnu/store/i037rnmqzwcc7nrgcswzb8dcckfrngzp-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 663, in launch_instance
app.initialize(argv)
File
"/gnu/store/f7y0fd230nc6l2qwq58g7q26rlkwjkf3-python-decorator-5.0.9/lib/python3.8/site-packages/decorator.py",
line 232, in fun
return caller(func, *(extras + args), **kw)
File
"/gnu/store/i037rnmqzwcc7nrgcswzb8dcckfrngzp-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File
"/gnu/store/1v8dgchnis87a4p3g9i3j1ayv0dfr4ds-python-notebook-6.3.0/lib/python3.8/site-packages/notebook/notebookapp.py",
line 2122, in initialize
super().initialize(argv)
File
"/gnu/store/f7y0fd230nc6l2qwq58g7q26rlkwjkf3-python-decorator-5.0.9/lib/python3.8/site-packages/decorator.py",
line 232, in fun
return caller(func, *(extras + args), **kw)
File
"/gnu/store/i037rnmqzwcc7nrgcswzb8dcckfrngzp-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File
"/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 229, in initialize
self.migrate_config()
File
"/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 155, in migrate_config
migrate()
File
"/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/migrate.py",
line 244, in migrate
ensure_dir_exists(env['jupyter_config'])
File
"/gnu/store/8ncan0ipzb240h23fwfspdhrzkzdw277-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/utils/__init__.py",
line 11, in ensure_dir_exists
os.makedirs(path, mode=mode)
File
"/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File
"/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File
"/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 7 more times]
File
"/gnu/store/sd2ic7bpv8fx3imy1j8xcjclx71sv6q0-python-3.8.2/lib/python3.8/os.py",
line 223, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system:
'/home/[user]/.guix-profile/etc/jupyter:'
