Re: org-babel not finding executables when using direnv [Was: Re: ob-sql is not finding psql when using direnv+guix]

2021-05-13 Thread Tim Cross


"Adolfo De Unanue"  writes:

> On Wed, May 12, 2021, at 08:49, Cook, Malcolm wrote:
>> > > >I am using Guix with direnv.
>> > > 
>> > > What is your shell?
>> > > 
>> > 
>> > My shell is bash, originally I was using zsh and I thought that was the
>> > problem, so I switched to bash and still not working.
>> > 
>> > > How/When do you "hook direnv into your shell" (https://direnv.net/)?
>> > > 
>> > 
>> > In the .profile file, at login.
>> > 
>> > > > In an specific folder I am installing and using psql and postgresql 
>> > > > using direnv+guix as follows:
>> > > >use guix --manifest=cdpp-manifest.scm
>> > > >
>> > > >export PGUSER=food_user
>> > > >export PGPASSWORD=some_password
>> > > >export PGDATABASE=food
>> > > >
>> > > >layout postgres
>> > > 
>> > > When are you doing this?
>> > > 
>> > > a) in the "specific folder"'s .envrc file?
>> > > b) in an org-mode shell block that you execute prior to the sql block?
>> > > c) ??
>> > 
>> > Option (a)
>> > 
>> > 
>> > > You seem to be following [Per\-project 
>> > > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
>> > > but with guix instead of nix. Nice.
>> > > 
>> > 
>> > Yep, great post. direnv + guix change the way I develop software, do data 
>> > science and write lectures and papers.
>> > 
>> > > If you enter the directory and then call emacs, earthing should just 
>> > > work, no changes neede.
>> > > 
>> > 
>> > It works for almost everything (sql-buffers, python buffers, etc), except 
>> > for org-babel sql blocks.
>> > 
>> > I am using EXWM, so emacs is always on.
>> > 
>> > 
>> > > If you want to tell emacs to adopt the environment established by a 
>> > > .direnv, you probably want [direnv integration for 
>> > > emacs\.](https://github.com/wbolster/emacs-direnv)
>> > > 
>> > 
>> > emacs-direnv was my first choice, but then I changed to envrc
>> > (https://github.com/purcell/envrc) . In both I got the error.
>> > 
>> 
>> I see.  Envrc looks superior.  Good to know.
>> 
>> You've covered all my bases.  Shooting in the dark,  I would 
>> confirm/check the following:
>> 
>
> I have news:
>
> It fails for python too. Using the same files as before and this block:
>
> #+begin_src python 
> import pandas as pd
> import matplotlib.pyplot as plt
> #+end_src
>
> It finds matplotlib, but fails to find pandas. I tried the same trick as 
> before
> (but with the python executable, no psql), added the line
>
> (org-babel-python-command . ".direnv/.guix-profile/bin/python3")
>
> but is not working. 
>
> But if I am in shell, or in a python buffer or in a inferior-python process
> Emacs is finding all the libraries and executables.
>
> So, I am assumming that the problem lies in how org-babel searchs the path ...

This is very unlikely. If there were issues in this area, we would be
seeing many more bug reports. More likely is that the PATH variable
doesn't contain what you think it does. First thing I would do is try
running the following source block

+being_src emacs-lisp
  (getenv "PATH")
+end_src

and verify the direnv bin directory is actually in your path.

Emacs will inherit the path from the shell running when emacs is
started. Changes made in your shell after Emacs has started will have no
effect on the PATH variable for emacs.

You mentioned at one point your running exwm. If you are not running
exwm in a login shell and your not sourcing your ~/.profile prior to
starting Emacs/exwm (assuming the direnv settings and path are setup
there), then the path variable is not going to have the direnv bin
directory. Either you need to ensure the appropriate direnv bin
directory is in your path before starting exwm or you need to add a
configuration step (e.g. possibly a hook function) which will add the
direnv bin directory using setenv.



-- 
Tim Cross



RE: org-babel not finding executables when using direnv [Was: Re: ob-sql is not finding psql when using direnv+guix]

2021-05-13 Thread Cook, Malcolm
> > > > >I am using Guix with direnv.
> > > > 
> > > > What is your shell?
> > > > 
> > > 
> > > My shell is bash, originally I was using zsh and I thought that was the 
> > > problem, so I switched to bash and still not working.
> > > 
> > > > How/When do you "hook direnv into your shell" (https://direnv.net/)?
> > > > 
> > > 
> > > In the .profile file, at login.
> > > 
> > > > > In an specific folder I am installing and using psql and postgresql 
> > > > > using direnv+guix as follows:
> > > > >use guix --manifest=cdpp-manifest.scm
> > > > >
> > > > >export PGUSER=food_user
> > > > >export PGPASSWORD=some_password
> > > > >export PGDATABASE=food
> > > > >
> > > > >layout postgres
> > > > 
> > > > When are you doing this?
> > > > 
> > > > a) in the "specific folder"'s .envrc file?
> > > > b) in an org-mode shell block that you execute prior to the sql block?
> > > > c) ??
> > > 
> > > Option (a)
> > > 
> > > 
> > > > You seem to be following [Per\-project 
> > > > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
> > > > but with guix instead of nix. Nice.
> > > > 
> > > 
> > > Yep, great post. direnv + guix change the way I develop software, do data 
> > > science and write lectures and papers.
> > > 
> > > > If you enter the directory and then call emacs, earthing should just 
> > > > work, no changes neede.
> > > > 
> > > 
> > > It works for almost everything (sql-buffers, python buffers, etc), except 
> > > for org-babel sql blocks.
> > > 
> > > I am using EXWM, so emacs is always on.
> > > 
> > > 
> > > > If you want to tell emacs to adopt the environment established by a 
> > > > .direnv, you probably want [direnv integration for 
> > > > emacs\.](https://github.com/wbolster/emacs-direnv)
> > > > 
> > > 
> > > emacs-direnv was my first choice, but then I changed to envrc 
> > > (https://github.com/purcell/envrc) . In both I got the error.
> > > 
> > 
> > I see. Envrc looks superior. Good to know.
> > 
> > You've covered all my bases. Shooting in the dark, I would 
> > confirm/check the following:
> > 
> 
> I have news:
> 
> It fails for python too. Using the same files as before and this block:
> 
> #+begin_src python 
> import pandas as pd
> import matplotlib.pyplot as plt
> #+end_src
> 
> It finds matplotlib, but fails to find pandas. I tried the same trick as 
> before (but with the python executable, no psql), added the line
> 
> (org-babel-python-command . ".direnv/.guix-profile/bin/python3")
> 
> but is not working. 
> 
> But if I am in shell, or in a python buffer or in a inferior-python process 
> Emacs is finding all the libraries and executables.
> 
> So, I am assumming that the problem lies in how org-babel searchs the path ...
> 
> Answers to your questions below:
> 
> 
> > How do you know that envrc is working otherwise?
> > 
> > When you switch to a direnv controlled buffer do you get any 
> > informative diagnostics in *Messages*?
> > 
> 
> Yes
> 
> > Do you get any more if you first `(setq envrc-debug t)`?
> > 
> Didn't try, Will do later
> 
> > Is this buffer in the same directory as the .direnvrc file?
> > 
> 
> Yes
> 
> > Presumably TRAMP is not in the mix in any way.
> >
> 
> No
> 
> > Variable sql-postgres-program is unchanged from default value of psql
> > 
> > What is result of
> > 
> > m-x shell-command
> > which psql
> > 
> 
> Is empty when I am not in a direnv controlled directory, the correct path 
> when I enter to those directories.
> 
> > and does it change as expected/required depending on the current active 
> > .direnvrc?
> > 
> > What happens if you eval (getenv 'psql) while in buffer governed by 
> > direnv, or alternatively
> > 
> > m-x getenv
> > psql

Ok - all this suggest that purcess/envrc processing is correctly resetting PATH 
in the emacs environment, as desired, but it is not being picked up in the 
environment which emacs consults when resolving the 'psql' to its full path.

A little sleuthing leads to `comint-exec-1 `function which sets exec-path and 
calls > start-file-process > start-process  which is documented to search 
exec-path and not PATH.

So, apparently exec-path is not being set/reset by emacs correctly (or at all).

Following up on this, it looks like it is trying, looking at these 2 code 
results in purcell/envrc https://github.com/purcell/envrc/search?q=exec-path

But, let's follow up on this, tell me, what does emacs reply when you, both 
inside & outside your direnv controlled directory, eval  `(describe-variable 
"exec-path")`, or alternatively:

c-h v exec-path

I think this will focus us on root cause

~Malcolm

> > 
> > If you `m-x shell`, does normal direnv work as expected in the created 
> > shell?
> > 
> 
> Yes, also works in vterm
> > > > >
> > > > >
> > > > >Where cdpp-manifest.scm contains the following:
> > > > >
> > > > >(specifications->manifest
> > > > >'("python"
> > > > >   "python-pandas"
> > > > >   "python-numpy"
> > > > >   "python-flask"
> > > > >   "python-graphene"
> > > > >