[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 I have separated this PR. I'm sorry for the confusion. https://github.com/apache/zeppelin/pull/2578 ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 Thank you @1ambda @zjffdu @felixcheung I think my code with various purposes is distracting you. And, unfortunately, my English ability is difficult to describe in a sophisticated way. Therefore, I will share PR. In this PR, we will only deal with the wrong library installation of conda. Many thanks for your comments. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/2574 @cloverhearts There is no problem about python lib paths. `%python.conda install` and `%python conda list` commands are the problem. They don't aware activated env by Zeppelin. So you don't need to write code for python path.  ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2574 @cloverhearts `There is only one python library path to be added.` Do you mean the py4j ? IIUC, PythonInterpreter will add py4j to PYTHONPATH, it doesn't relate with `PythonCondaInterpreter`. That's why I mention we just need to set `zeppelin.python`, Let me know if I miss anything. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/2574 @cloverhearts There is no problem about python lib paths. `%python.conda install` command is the problem. So you don't need to write code for python path.  ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @zjffdu @1ambda @felixcheung Thank you for your kind reply despite my lack of English. I hope this PR is merged. So, i will try to reflect your opinions as much as possible. Thank you. :) ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 In my opinion, allowing for fluctuations in the python environment is itself a problem. But zeppelin are supporting it. If it does not, then the semantics of conda activate and conda deactivate disappear. In fact, I also think that changing the python environment can be a problem. However, the current Zeppelin supports it, and a bug exists. This PR is a PR that solves the bug. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 I have a solution to remove conda related members from PythonInterpreter. also PythonCondaInterpreter will not have any members. However, it differs in focus from this PR. (The problem is that there are changes related to the start of the Python Interpreter.) This PR is intended to be used when the user changes the environment by installing new python while using conda. This also solves the Zeppelin Python Interpreter's infinite loop problem. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @1ambda Currently there is a possibility that it will be structurally. However, if you store it in PythonCondaInterpreter, the problem is greatly aggravated. 1. PythonInterpreter needs to monitor PythonCondaInterpreter at all times. Â Â Â Â (Python Interpreter should check Conda's environment activation when restarting.) 2. Other configurations other than PythonCondaInterpreter If an interpreter is added (as you might have guessed), the Python Interpreter is in a situation where you need to constantly query for all other Interpreter environments. This will affect the life cycle of the Python Interpreter. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @zjffdu This change will work exactly as your request. There is only one python library path to be added. 1. If the user does not install python via conda. (If the user installs only the library) Zeppelin use to `zeppelin.python`. (Used in the original environment.) 2. If you install python via conda. (separate python from zeppelin.python) if when the conda environment is active, add to library path and replace to installed python via conda from zeppelin.python. Zeppelin to work in the new python environment. 3. The user disables the environment. (conda deactiavte) It is restored to the original zeppelin.python environment. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2574 @cloverhearts Just to clarify my concern. Here's what I see when I use conda outside of zeppelin. Step 1. Create enviroment `env_1` using conda, and install pandas Step 2. Set `zeppelin.python` to the binPath of `env_1` and restart python interpreter Step 3. Now when I use python interpreter, I am using python of `env_1` and can use any libraries installed under `env_1` That means I just need to set `zeppelin.python` to switch to another conda env. That's why I don't understand why you need to set additional Python library path for the new environment ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/2574 @cloverhearts I have few questions. What happens when we want to implement `PythonPyenvInterpreter` which is similar to `PythonCondaInterpreter`? - In that case, we have to add new functions to PythonInterpreter like `setPyEnvLib()...`? ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 I think we are worried about `conda interpreter` and coupling for` conda env name`. However, the conda env name is more important to the python interpreter than the conda interpreter. Also, the python interpreter should not query the conda interpreter for its own state. I think this misunderstanding is due to the member name `conda env name`. If I have any additional problems, I will change this to `virtualEnvName`. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @zjffdu Thank you for your opinion, I present scenarios for environment changes using conda. Python version and module names are arbitrary, so do not worry. What is important is change in every environment. 1. Basically Zeppelin's python python: 2.71 installed library: (none) use to zeppelin ``` %python import sys print(sys.version) print(sys.path) ``` result ``` python version 2.7.1 %ZEPPELIN_HOME%/interpreter/python/lib:%PYTHON_HOME ``` ``` %python import myModule ``` result ``` no module myModule ``` 2. install conda and new python ``` %python.conda create --name python3 python=3.4 ``` ``` %python.conda activate python3 ``` ``` %python import sys print(sys.version) print(sys.path) ``` result ``` python version 3.4 %ZEPPELIN_HOME%/interpreter/python/lib:%PYTHON_HOME:%CONDA_ENV%/python3/lib ``` ``` %python import myModule ``` result ``` no module myModule ``` try again ``` %python.conda install myModule ``` ``` import myModule ``` result ``` imported. ``` 3. restore python environment ``` %python.conda deactivate ``` check for recovered to the original Zeppelin environment. ``` %python import sys print(sys.version) print(sys.path) ``` result ``` python version 2.7.1 %ZEPPELIN_HOME%/interpreter/python/lib:%PYTHON_HOME ``` ``` import myModule ``` result ``` no module myModule ``` All three must be supported by conda. 1. Support to installing and using the python library 2. Support to change the python version 3. Support to configuring non-python environments old zeppelin does works for this part(0.71). but, the Python Interpreter has changed significantly since 0.71. I guess missing the environment part for this. I modified it. python env set code (https://github.com/apache/zeppelin/blob/master/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java#L151) ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @zjffdu @felixcheung Thank you for your opinions :) I was left my comment. Could you check this one? ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 https://travis-ci.org/malayhm/zeppelin/jobs/272788659 https://travis-ci.org/1ambda/zeppelin/jobs/273195461 ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 ``` Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.456 sec <<< FAILURE! - in org.apache.zeppelin.interpreter.remote.RemoteInterpreterServerTest testStartStop(org.apache.zeppelin.interpreter.remote.RemoteInterpreterServerTest) Time elapsed: 10.037 sec <<< FAILURE! java.lang.AssertionError: expected: but was: at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServerTest.testStartStop(RemoteInterpreterServerTest.java:62) ``` This error occurs equally in other PRs and branches. It has been confirmed that it is not related to my changes. ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @1ambda I'll check it, thank you :) ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/2574 @cloverhearts I am getting this error.   ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user cloverhearts commented on the issue: https://github.com/apache/zeppelin/pull/2574 @1ambda Thank you, and i will more test :) ---
[GitHub] zeppelin issue #2574: [ZEPPELIN-2921 : 0.7x] does not work conda environment...
Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/2574 Thanks for the fix. Let me test and feedback soon. ---
