[GitHub] zeppelin issue #2227: [ZEPPELIN-2359] Support Spell as Display
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2227 @Leemoonsoo I wanted to introduce minimum changes in javascript and stay aligned with the idea of interpreters returning different display types. Introducing a new `spell` display has sounded logic and easy to me. But I agree that in this case, user has still to add a second, potentially confusing, magic keyword to define the targeted spell. The other way would be to consider that if interpreter returns a value that begins with `%`, we consider it as a spell... @1ambda WDYT? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2226: [ZEPPELIN-2235] Helium load.js should reside in subfol...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2226 This is solved with #2210 Bundle helium packages one by one --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2226: [ZEPPELIN-2235] Helium load.js should reside in...
Github user echarles closed the pull request at: https://github.com/apache/zeppelin/pull/2226 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2182: [ZEPPELIN-2248] Fix javax.ws.rs and cxf version mismat...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2182 How does this PR relates to #932 ? #932 simply replaces cxf with jersey2. Here I see we still have some cxf deps. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2182: [ZEPPELIN-2248] Fix javax.ws.rs and cxf version mismat...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2182 This issue is recurring since a long time and depends IMHO on each environment (maven version...) - The main parameter to fix this is the `javax.ws.rsapi.version`. This PR upgrades cxf from v2 to v3 and jersey from v1 to v2 which confuses me. I would rather see using only jersey2 as in #932. Also Moving from jersey-v1 to jersey-v2 needs an update of the licence file as explained in #932. Maybe the minimal fix would be leaving cxf as such and simply enforcing `javax.ws.rsapi.version` (I did that some time ago as temp fix). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2227: [ZEPPELIN-2359] Support Spell as Display
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2227
@Leemoonsoo @1ambda I see your point and agree with it.
However, we need to decide which InterpreterResult to return. For now it
can be { TABLE, IMG, SVG, NULL }. The Java code needs to decide which value to
return to the Javascript, and the Javascript based on that value needs to take
decisions (display as a known display-type, or render code via Spell).
For now, I don't see how to implement / recognize on Java side the
InterpreterResult to return.
Can you describe any option you have in mind?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 Jenkins build is now green. @jongyoul @bzz Any other inputs/questions on this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 It is working here with shiro-enabled (simple username password in shiro,ini, not the ldaprealm). I would need to test with ldaprealm... @sohaibiftikhar is it working for your with the simple username/password in shiro.ini ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2227: [ZEPPELIN-2359] Support Spell as Display
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2227
@1ambda thx a lot for the follow-up and your great work.
This branch now supports what we want:
```
println("""%json
{"array":[1,2,3],"boolean":true,"null":null,"number":123,"object":{"a":"b","c":"d","e":"f"},"string":"Hello
World"}""")
```
I would be great if you could test it and give me any feedback.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2227: [ZEPPELIN-2359] Support Spell as Display
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2227 @1ambda I have added the log. Regarding you second comment, I am not sure what you mean. Is there something I should change? Is it not working as expected? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2418: [ZEPPELIN-2645] Adding way to register RemoteInterpret...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2418 I'd like adding more points to the discussion: - Do we need to support kerberos-like auth in the communication protocol (you can achieve this with Thrift, not sure with Netty)? - Do we target a single implementation dropping the other or something that can be configured with the burden to maintain multiple implementations? - What about REST (for more easy and universal clients) or WebSocket (for push) as more alternatives? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2329: [ZEPPELIN-2040] ClusterManager to support launching in...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2329 Late question: This implementation uses the raw Yarn APIs. Did you consider using higher-level APIs such as Twill (http://twill.apache.org) or Slider (https://slider.incubator.apache.org) ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2418: [ZEPPELIN-2645] Adding way to register RemoteInterpret...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2418 In case of SNPEGO Web Auth (not yet supported by Zeppelin0,, you could consider transfert the Kerberos ticket from front-end to back-end and use it to , this the usecase I was thinking to. Well, multiple and pluggable communication between frontend and interpreters would be great and useful, but I am aware of the impact on code level. If we have to move to something else than thrift, what would be the added value of netty (just asking). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2418: [ZEPPELIN-2645] Adding way to register RemoteInterpret...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2418 @jongyoul sorry, I was not clear enough. With frontend, I was meaning the web layer (so jetty). Jetty could communicate via a variety of protocols to the interpreters. Now going further in that direction, having protocols like REST or WebSocket would allow any king of clients to access the interpeters in direct - Those clients could be the current javascript of Zeppelin webapp, or any other kind of client (could be bash or python script...) This is unique opportunity to extend the Zeppelin user base - Btw, having the AAA (Authentication, Authorization, Auditing) functions at Interpreter level is IMHO a good design, even at the cost of some extra development or refactoring. I am curious to know what the community thinks about this... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2418: [ZEPPELIN-2645] Adding way to register RemoteInterpret...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2418 @zjffdu I am thinking to automation (using the notes in an scheduled or batch process). Does it make sense to you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 @matyix I have a local spark-k8s setup and have this branch (without success so far, so debugging SparkK8RemoteInterpreterManagedProcess to tackle down the issue). A few questions: + I see that in case of 'k8s://...' master, the 'SparkK8RemoteInterpreterManagedProcess' is first search for a driver pod starting with prefix `zri-` - Does that mean I have to manually instanciate that pod? + If created one such pod, and for now, if the pod is found, the 'SparkK8RemoteInterpreterManagedProcess' considers at as "remote process" - Actually, you don't create a process via the classical zeppelin 'interpreter.sh' command. Right? + Then, I receive `java.net.ConnectException: Connection refused`. Would the classical `spark-submit` via `interpreter.sh` not be another option? Did you try it? Here, using that approach, the driver is created but exits directly. I guess https://github.com/apache-spark-on-k8s/spark/pull/402 would help. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 Just tried the `vanilla` (so without this #2637 PR) zeppelin on spark-k8s with the `in-cluster client mode` branch (https://github.com/apache-spark-on-k8s/spark/pull/456). It works fine out-of-the-box (with the ad-hoc spark interpreter parameters). @matyix Do you see any reason to further work on this PR? Maybe you want to address additional goals with this? ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2637
@matyix I have tested your last commits and was able to make it work in my
env (with both zeppelin `in-` and `out` k8s cluster).
Your implement a new (specific for spark-k8s) launch and remote executor.
In another local branch, I have tried to stick as much as possible to the
current zeppelin paradigm (thrift servers both sides of the interpreters
processes with CallbackInfo) and 2 parameters (host, port) for interpreter.sh -
I still have issue with the callback, so I finally think the approach you
propose is good and does the job.
My feedbacks:
+ The branch as-such need basic updates: I had to fix compilation issue
with the new classes (`SparkK8sInterpreterLauncher` and
`SparkK8sRemoteIntepreterManagedProcess`) and had to add
`${ZEPPELIN_SPARK_CONF}` in the `interpreter.sh` script.
+ To find the running driver pod, you actually poll on regular basis. The
ideal would be to be notified when the pod is ready (not sure if the k8s client
support this. We would closely map the current mechanism of the thrift
notification via the CallbackInfo, but here with a pure k8s mechanism. This
could be also extended to other interpreters we would want to see in k8s.
+ We need to set `spark.app.name` with must a value starting with `zri-` -
If you don't set this in the interpreter settings, the k8s client will not find
the driver pod - I wonder if we can make this more configurable, let's say
using metada or simply using the InterpreterContext with contains a
`properties` attributes with all the given props - the launcher could retrieve
this and search for a pod starting with a dynamic prefix rather than with this
hardcoded one.
+ The current vanilla zeppelin supports out-of-the-box the spark-k8s
`client-mode` (assuming you are using
https://github.com/apache/zeppelin/pull/2637). The condition to use the
`SparkK8sInterpreterLauncher` needs to check for `spark.submit.deployMode`
being `cluster` and continue to use the normal ManagedProcess for `client`.
+ On documentation level, certainly mention that the app name must start
with `zri-`. Also, relying on the kubespark docker image would be better to
ensure nothing special is added in the docker image.
WDYT?
Do you prefer me to submit a PR on your PR and will you make another push?
---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 PS1: I have pushed my fixes in zeppelin-k8s/tree/spark-interpreter-k8s-fixes branch (which has merge conflict with master due to latest commit 3b1a03f that touch the launchers and remote executors) PS2: I have opened on PR to document this on the spark-k8s docs repo apache-spark-on-k8s/userdocs/pull/21 ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 Tested latest commit on AWS K8S cluster and works great as well in `client` as `cluster` modes - Kudos @matyix ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 @matyix If my tests are correct, for now, we can not set `spark.app.name` and `spark.kubernetes.driver.pod.name` (if you set them, the driver will not be found by zeppelin). Upon that is not user-friendly, the side effect is that pod driver and executors names start with `null-...`. IMHO it is a matter of passing the correct parameters to spark-submit depending on the presence or not of those properties. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 I am playing in cluster mode setting/removing via the interpreter page the `spark.app.name` and `spark.kubernetes.driver.pod.name` and so far the only to make it work (= remote executor finding the spark driver) is really having the absence of those props in the interpreter page (maybe I am fooled with `spark.app.name`). Yes, checking if a specific `spark.kubernetes.driver.pod.name` prop and using it will be an expected enhancement, and may even solve the `spark.app.name` potential issue. It also seems to me that Zeppelin adds `spark.app.name` and `master` props if they are not present. I guess we should for now rely on that behavior and ensure in this PR that enough intuitive and documented material is available for the end-user. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 @matyix I made a few more tests, and now `spark.app.name` and `spark.kubernetes.driver.pod.name` can be set without problem... (to be further confirmed). ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 Thx @matyix. I will test and give feedback. Did you adding external dependencies (via the interpreter page). It works on my setup in `client` mode but fails (ClassNotFound) in `cluster` mode. I can debug but your tests on this can help. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 @matyix There is a long history in Zeppelin on `spark.dep` vs `external dependency in interpreter settings`. I am fan of the later (interpreter settings), so if the --packages flag can make it work, this would be wonderful. I don't see in Spark doc that --package add the jars on the executor classpath. The `spark.jars` property (Comma-separated list of local jars to include on the driver and executor classpaths) may be an alternative. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 @matyix sure, we can to address the dep in separate PR, especially if it is beneficial for all deploy modes. The downside of it is that we will have everybody happy with any change on deps management (if you search the mailing list, you will a lot of questions and discussions around this). An additional parameter is the definition of "local" dependencies that don't come from any maven repo on the Internet. I regularly have to define local jars on the local disk file system (or even resources files that are not jar) and with the current `yarn-client` behavior of Zeppelin (probably not for `yarn-cluster`, I didn't test), those jars/resources are available on the classpath (of the driver and executor). If we could inject this now while this branch is being discussed, that would be really a good thing. ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 @matyix I've given a try on you last commit and can not get the additional deps (in settings page) working. I don't see the `spark.jars` property in the generate command (by interpreter.sh): ``` /opt/spark/bin/spark-submit --class org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer --driver-class-path ":/opt/zeppelin/interpreter/spark/*:/opt/zeppelin/lib/interpreter/*::/opt/zeppelin/interpreter/spark/zeppelin-spark_2.11-0.8.0-SNAPSHOT.jar:/etc/hdfs-k8s/conf" --driver-java-options " -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///opt/zeppelin/conf/log4j.properties -Dzeppelin.log.file=/opt/zeppelin/logs/zeppelin-interpreter---zeppelin-k8s-hdfs-locality-zeppelin-7cd554b49d-dpq2k.log" --master k8s://https://kubernetes:443 --conf spark.cores.max='1' --conf spark.shuffle.service.enabled='false' --conf spark.yarn.dist.archives=/opt/spark/R/lib/sparkr.zip --conf spark.executor.instances='3' --conf spark.sql.catalogImplementation='in-memory' --conf spark.app.name='zeppelin-k8s-spark' --conf spark.executor.memory='1g' --conf spark.master='k8s://https://kubernetes:443' --conf spark.kubernetes.namespace='default' --conf spark.kubernetes.executor.docker.image='datalaye r/spark-k8s-executor:2.2.0-0.5.0' --conf spark.kubernetes.driver.docker.image='datalayer/spark-k8s-driver:2.2.0-0.5.0' --conf spark.kubernetes.initcontainer.docker.image='datalayer/spark-k8s-init:2.2.0-0.5.0' --conf spark.kubernetes.resourceStagingServer.uri='http://10.108.197.6:1' --conf ``` ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 I still have issues with deps in cluster mode. @matyix Can you confirm it is working on your env? I will retest with your latest push in the coming days. But yes, absolutely, it will be great to get this already merged as-is. Btw, spark-k8s merge is not complete (there are other pending PR to get it functional in the upstream apache spark repo). ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 Feedback/Question on latest commit: + For `cluster` mode, I sometimes receive a null pointer at first run, running a second time directly after is fine. + Are you enforcing the executor pod name? + Additional dep for cluster mode works with `%spark.dep`, but not via the Spark interpreter setting UI (see screnshot)  ---
[GitHub] zeppelin issue #2637: Add support to run Spark interpreter on a Kubernetes c...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2637 thx @matyix I had taken logs but don't have them anymore... but I can confirm that in both cases (deps via %spark.dep or via the ui) the command generated by the interpreter.sh is the same and does not contain the `--jar` option. If you have a cluster at hand, you should easily be able to confirm that it does not work via the ui. ---
[GitHub] zeppelin issue #2730: ZEPPELIN-3166. R plotting resolution and image width i...
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2730
`fig.retina` sounds like a good property to play with. However, fixing
`out.width` to an arbitrary value may give good result for the example you
give, but did you try with other plots?
Showing nice-looking plots generated by knitr depends on your screen
resolution and browser actual window width.
We had before (I hope it is still there) the ability to force the image
width passing a json after the magic keyword
```
%spark.r {"imageWidth": "400px"}
```
If you use that `imageWidth` property and push a bit more `fig.retina` to 3
or 4, could it be that most cases will be covered?
---
[GitHub] zeppelin issue #2730: ZEPPELIN-3166. R plotting resolution and image width i...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2730 Agree, but the default will always find case where it will not work. For this image size / resolution, there may be more failure than success I am afraid. To be tested with various plots, screen resolutions and paragraphs width to measure this. ---
[GitHub] zeppelin issue #2730: ZEPPELIN-3166. R plotting resolution and image width i...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2730 GTM ---
[GitHub] zeppelin issue #2404: fix bugs for: when interpreter setting restart of one ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2404 @wary I am facing the same issue (all paragraphs aborted on interpreter restart in isolate per user configuration). I see you have deleted your branch and that the proposed patch does not apply anymore in the zeppelin head. Did you implement another branch for this? If you give me directions, I will for sure try it as this is a blocker for me. ---
[GitHub] zeppelin issue #2404: fix bugs for: when interpreter setting restart of one ...
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2404
I have just worked on this issue and found a bug in the
`InterpreterSetting` class. The `getInterpreterSessionId` uses
`option.perNoteScoped` in its tests, but it should be `option.perNoteIsolated`.
With the below change, the paragraphs are not more aborted and each users
benefit from a separated scheduler (for now, the jobs of the different users
have to queue as they have a single scheduler).
```
private String getInterpreterSessionId(String user, String noteId) {
String key;
if (option.isExistingProcess()) {
key = Constants.EXISTING_PROCESS;
} else if (option.perNoteIsolated() && option.perUserIsolated()) {
key = user + ":" + noteId;
} else if (option.perUserIsolated()) {
key = user;
} else if (option.perNoteIsolated()) {
key = noteId;
} else {
key = SHARED_SESSION;
}
return key;
}
```
---
[GitHub] zeppelin issue #2404: fix bugs for: when interpreter setting restart of one ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2404 @jongyoul I understand the difference about restarting an interpreter from a note vs. restarting from the interpeter page. My point is about the session id which should be different in case of perUser isolation. ---
[GitHub] zeppelin issue #2404: fix bugs for: when interpreter setting restart of one ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2404 @zjffdu yes, but the fix implemented in #2769 only addresses scheduler isolation per interpreter groups which is a good thing, but not enough to cover the case we are talking in this PR (isolation per user). Actually I am running with both #2769 change + the snippet I have given in https://github.com/apache/zeppelin/pull/2404#issuecomment-372529822 to get a fully isolated scheduled interpreter groups per user. cc/ @prabhjyotsingh @wary ---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-804 Refactoring registra...
Github user echarles commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/835#issuecomment-221787153 @jongyoul your conclusion are correct. I would also say there is something happeing with the zeppelin.R.render.options` nowt taken into account (the `echo = FALSE` is there to ensure the commands you type are not printed in the result). Here, it seems the R result is returned with the given command. Maybe the interperter settings overrides the values defined in the interepreter code? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-804 Refactoring registra...
Github user echarles commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/835#issuecomment-221816775 @jongyoul Building with both -Pr -Pspark may lead to unexpected behavior, even without the changes you brought... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin pull request: Make path for .spark-dist folder ...
GitHub user echarles opened a pull request: https://github.com/apache/incubator-zeppelin/pull/930 Make path for .spark-dist folder always accessible ### What is this PR for? Current Path for .spark-dist is outside the reactor, which can breaks build. [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 30.337 s [INFO] Finished at: 2016-05-30T15:34:30+02:00 [INFO] Final Memory: 73M/469M [INFO] [ERROR] Failed to execute goal com.googlecode.maven-download-plugin:download-maven-plugin:1.2.1:wget (download-pyspark-files) on project zeppelin-spark-dependencies: IO Error: Destination '/opt/zeppelin-master/spark-dependencies/target/../../.spark-dist/spark-1.6.1.tgz' directory cannot be created -> [Help 1] ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN/ZEPPELIN-902 ### How should this be tested? Build with pyspark profile ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer pyspark-build Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-zeppelin/pull/930.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #930 commit cf678ad1ace62c84f0bef23883f1cb591e30e634 Author: Eric Charles Date: 2016-05-30T13:39:25Z Make path for .spark-dist folder always accessible --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin pull request: Remove unnedeed commons-exec decl...
GitHub user echarles opened a pull request: https://github.com/apache/incubator-zeppelin/pull/931 Remove unnedeed commons-exec declaration in scope test [ZEPPELIN-904] ### What is this PR for? zeppelin-server should simply rely on transitive commons-exec from zeppelin-interpreter. ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN/ZEPPELIN-533 ### How should this be tested? Simply build ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? N You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer commons-exec-compile-scope Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-zeppelin/pull/931.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #931 commit b791ba723a173e2266c39b272cbf52b59f6ba76e Author: Eric Charles Date: 2016-05-30T15:34:08Z remove unnedeed commons-exec declaration in scope test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin pull request: Replace CXF with Jersey2 [ZEPPELI...
GitHub user echarles opened a pull request: https://github.com/apache/incubator-zeppelin/pull/932 Replace CXF with Jersey2 [ZEPPELIN-903] ### What is this PR for? Use jersey2 instead of CXF to manage JSON REST requests and responses. Jersey2 can better manage the JSON bodies and mappings. ### What type of PR is it? [Improvement] ### Todos ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN/ZEPPELIN-903 ### How should this be tested? Build, run and check the ### Screenshots (if appropriate) N/A ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? N You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer jersey2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-zeppelin/pull/932.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #932 commit 9eef80cb71bd7f456145830eca59a635b4627b83 Author: Eric Charles Date: 2016-05-30T15:50:51Z Replace CXF with Jersey2 commit dbac7d9cafc9d8496455b382949106fd94b9fc65 Author: Eric Charles Date: 2016-05-30T15:55:37Z Fix code style commit d149a728eba1cbc2fd7ae2ee016cb71510286279 Author: Eric Charles Date: 2016-05-30T16:13:53Z Ensure dependency convergence --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin pull request: Make path for .spark-dist folder always acc...
Github user echarles commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/930
@bzz Just commented on ZEPPELIN-902 - My concern was on my local build, not
on Travis stability.
As mentionned onthe JIRA, the target `target` folder is not present after a
clean, because the download-pyspark-files is binded to the `validate` phase.
I guess using ${basedir} could do the job, but then we are not 100% safe
because building in the spark-dependencies folder would put the spark-dist at a
different location...
Maybe ${session.executionRootDirectory} ? (see
http://stackoverflow.com/questions/3084629/finding-the-root-directory-of-a-multi-module-maven-reactor-project).
Another approach would be to bind to the `compile` phase, but I guess it
will have side-effects.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Make path for .spark-dist folder always acc...
Github user echarles commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/930
Tried with the ${session.executionRootDirectory} and got [ERROR] around
Ant part ...
[GitHub] incubator-zeppelin issue #949: [ZEPPELIN-941] jsoup NoSuchMethodError in Zep...
Github user echarles commented on the issue: https://github.com/apache/incubator-zeppelin/pull/949 Thx for jumping on this. It was mentioned on the mailing list, but seems like not everyone encounters this (maybe different maven options, profiles...) Going to first will solve issue for everyone. I checked Travis: all green execept TEST_SELENIUM=true which returns : `Tests in error: ZeppelinIT.testSparkInterpreterDependencyLoading:214->AbstractZeppelinIT.createNewNote:133 û Timeout` - Not related to this PR. So LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin issue #949: [ZEPPELIN-941] jsoup NoSuchMethodError in Zep...
Github user echarles commented on the issue: https://github.com/apache/incubator-zeppelin/pull/949 All Green now. LGTM confirmed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/incubator-zeppelin/pull/932 Any feedback on moving to jersey 2? (btw CI is green) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin pull request #980: [ZEPPELIN-871] [WIP] spark 2.0 interpr...
GitHub user echarles opened a pull request: https://github.com/apache/incubator-zeppelin/pull/980 [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.11 ### What is this PR for? Spark interpreter for spark version 2.0.0 and scala 2.11 (implemented in Scala) ### What type of PR is it? [Feature] ### Todos * [ ] - Clean code * [ ] - Test SQL * [ ] - Test Python * [ ] - Test R ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN--871 ### How should this be tested? Build it with ``` mvn install -Pscala-2.11 -Dscala.binary.version=2.11 -Dscala.version=2.11.8 -Pspark-2.0 -Dspark.version=2.0.0-SNAPSHOT -Phadoop-2.6 -Dhadoop.version=2.7.2 -Pyarn -Dmaven.findbugs.enable=false -Drat.skip=true -Ppyspark -Psparkr -Dcheckstyle.skip=true -Dcobertura.skip=true -Pbuild-distr -pl '!alluxio,!cassandra,!elasticsearch,!file,!flink,!hbase,!hive,!ignite,!jdbc,!kylin,!lens,!phoenix,!postgresql,!tajo' -DskipTests ``` Run and test the spark paragraph. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? Y You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer spark-2.0 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-zeppelin/pull/980.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #980 commit f041ec3e4b524594a2a32692c5418e04f7335323 Author: Eric Charles Date: 2016-06-09T14:31:50Z Initial implemenation for spark 2.0 interpreter on scala 2.11 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on...
Github user echarles commented on the issue: https://github.com/apache/incubator-zeppelin/pull/980 This is WIP. I'd like to get feedback on the approach (scala implementation in the current spark module), taking into consideration: + Spark 2.0 will ship by defaulit on scala 2.11 (makes sense to me to develop the interpeter on scala 2.11) + SqlContext and HiveContext are deprected in favor of SparkSession. + HttpServer is removed and some classes methods have access restricted to spark only packages. + SpakIMain and SparkJLineCompletion don't exist anymore. Building on the current java classes with method invocation may be possible, but would make the code more than difficult to read and develop. This PR proposes separated scala classes for this very specific 2.0 API breaking changes. WDYT? Based on feedback, I will further validate the functionalities (for now, a simple spark 2.0 call works well on my local env). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on...
Github user echarles commented on the issue: https://github.com/apache/incubator-zeppelin/pull/980 Sure, we can wait on #747 merge. I see there is a `spark/src/main/scala-2.11/org/apache/zeppelin/spark/SparkInterpreter.java`: does this mean that there will be 2 separate implementations: one for 2.10 et and for 2.11? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.1...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/980 Dealing with mixed scala 2.10/2.11 and spark 1.x/2.x with a same implementation is always possible but drives to code plenty of method invocation (see e.g. https://github.com/lresende/incubator-zeppelin/pull/1/files#diff-dbda0c4083ad9c59ff05f0273b5e760fR216). At the end, you have an implementation which a succession of if (...) then invokeMethod... On the other hand, having multiple implementations drives to maintenance overhead and a risk of divergence. In this particular case, I was thinking that having 2 lines was something worth to discuss: 1.- Spark 1.x on scala 2.10 2.- Spark 2.x on scala 2.11 I also don't see why we should still support the DepInterpreter in future developments (normally, deps should be configured via the interperter settings). Also, the 2 lines approach would make easier other evolutions such as for example having a ZeppelinContext available for all interpreters. Also, we have kind-off already more than one spark impementation: The Livy one has its own implementation and features. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.1...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/980 Agree with your agurment. If we push a bit further the discussion, we can not have a single binary which fits all users needs. What if I want spark-2-scala-2.11 with flink-0.9-scala-2.12 ? You single binary will not give me any solution... I would rather see a `zeppelin-kernel` (nearly naked without interpeters) with the ability for the user to enable any interpreter via a shell command. Regarding DepIntepreter, I can understand what users says, but for some packages, it simply does not work (example: spark-csv running on yarn if I remember well) - Not sure if it can be easily fixed. Bottom line: I am no fan of the `invoke` coding style, but I will wait on the other PR and try to build on top of this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.1...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/980 @Leemoonsoo - I was just saying that a single binary will not fit all various users need (multiple combination of interpeters and versions). - #629 is merged. What remains to get ZEPPELIN-598 operational? @felixcheung Is there already a jira for the proposal you are thinkg to? I feel we have nearly everything open, but maybe we see an umbrella? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.1...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/980 @Leemoonsoo Oh, #908 is the one I was looking for --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.1...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/980 - If you ship in the same folder scala libs from different versions, it will at runtime - You don't have to rebuilt, but you have to arrange classpath is built with scala libs of the same version - Gui, cli... Makes sense. Is t is what @felixcheung was refereing to? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/994 why spark is not published. is it related to the assembly? Btw, spark 2 doesn't use assembly any more and favor dependencies in separate jars. See https://issues.apache.org/jira/browse/SPARK-11157 for more details. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/994 Didn't look at the code... just saw in the description (and @minahlee just confirmed it would be published). Thinking more about this PR: - Why the deploy and release plugins were not working? Using scrips instead of existing plugins is not ideal to me. - I understand distribution should not be published, but server should be. It contains the repo components that can be used elsewhere. Apart from the fact we could have a separate module for the repo, all components (excpect distribution) should be published. That's a rule of thumb and if we don't follow it, someone is gonna come sooner or later and ask for the missing bits. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/994 Good to hear the release plugin is working fine. The release manager is responsible for the tools he wants. Fine to me if it is a script or a plugin, but at the end the artifacts must find their way to maven central. For this PR, is it possible to ensure `zeppelin-server` will be also published? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #993: [ZEPPELIN-987] Enable user to secure interpreter settin...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/993 @AhyoungRyu If you can come to an implementation that fits @prabhjyotsingh good idea, it would be great if you could make it `generic`, with a configurable list of functions that reflect in to the menu. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/994 @minahlee thx for this precision and context. True, I just realized after sending the comment that repo is not part of zeppelin-server. However, I need zeppelin-server in my project (I kind off embed the web and websocket server) - Publishing all artifcats is also in my understanding a common practice. WDYT. Is there any issue you face to publish the server? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #998: Authenticated user aware notebook storage layer
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/998 LGTM I have tested and works well for me. Gone through the code and changes make sense. Will be useful to get it into 0.6 release (especially if #836 is also in 0.6, as methods are added in the NotebookServer, which will have impact on this PR). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #930: Make path for .spark-dist folder always accessible [ZEP...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/930 pong :) I don't see any solution for this. Does this issue bother anyone else? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #930: Make path for .spark-dist folder always accessible [ZEP...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/930 No, not ready. None of the envisaged solutions are practicable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #980: [ZEPPELIN-871] [WIP] spark 2.0 interpreter on scala 2.1...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/980 Closing this. Spark 2.0 is implemented with #1195 Optionally, a new PR can be opened to port the Spark interpreter from Java to Scala if interest. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1195: [ZEPPELIN-759] Spark 2.0 support
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1195 Tested this with spark 2.0 and scala 2.11. Spark 2.0 works well with scala, R and python. @minahlee comment makes sense - we should ship the latest version. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 @Leemoonsoo Just pushed the asked changes regarding licenses. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1259: [ZEPPELIN-1260] R interpreter doesn't work with Spark ...
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/1259
@minahlee Just saw you comment on ZEPPELIN-1260, but @Leemoonsoo has been
fast...
I have tested the changes, and now sparkr dataframes are working well (with
the `spark` variable which referts to the new Spark2 session).
The R tutorial note refers to sqlContext which is no more present
```
Error in dispatchFunc("createDataFrame(data, schema = NULL, samplingRatio =
1.0)", : object 'sqlContext' not found
```
Patch could also expose sqlContext.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1265: [ZEPPELIN-1210] Run interpreter per user
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1265 Quickly went through the changes, but not sure to get it. In which way is this related to the various intepreter modes we already have? In which way is this related to ZEPPELIN-1000 (Multiple simultaneous users on a single WEB server)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 Failing with org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.TApplicationException: Internal error processing createInterpreter at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:238) Is this a WKZCIF (Well Know Zeppelin CI Failure) ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1259: [ZEPPELIN-1260] R interpreter doesn't work with Spark ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1259 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 @bzz I have updated the PR descrption (first comment) with the need to update license and adding a note on how easy it is with jersey to map a full json to a pojo being passed as method parameter. I have also checked the transitive deps [1] and updated `zeppelin-distribution/src/bin-license/LICENSE` accordinly. Lets's if build is green. + com.fasterxml.jackson.core:jackson-core:jar:2.5.4:compile + com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.5.4:compile + com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.5.4:compile + com.fasterxml.jackson.core:jackson-databind:jar:2.5.4:compile + com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.5.4:co â Apache 2 + javax.annotation:javax.annotation-api:jar:1.2:compile â CDDL2 + javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile â CDDL2 + javax.validation:validation-api:jar:1.1.0.Final:compile â Apache2 + org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile + org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile + org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile + org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile + org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile + org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile â CDDL2 + org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile + org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile + org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile + org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile + org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile + org.glassfish.jersey.ext:jersey-entity-filtering:jar:2.22.2:compile + org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile + org.glassfish.jersey.media:jersey-media-json-jackson:jar:2.22.2:compile â CDDL + org.javassist:javassist:jar:3.18.1-GA:compile â Apache2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 @bzz I have moved `LICENSE-jersery-2` to `zeppelin-distribution/src/bin_license`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 Whatever `su` or `ssh` is used, I feel the main trick is the user provisioning on the host running the interpreter. Until now, the shiro authentication system had no impact on the user provisioning. This PR changes this. I guess we all agree and are aware that adding user `foo` to shiro.ini, and enabling impersonation, will require to run `adduser foo` manually. We should make this clear in the doc but also stress it in the UI (with a hover, or a clear text/link near the User Impersonate. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 To make ZEPPELIN-1337 Umbrella for multiple user support for zeppelin more readable, should we rename the following: - ZEPPELIN-1340: "Run Hadoop-based interpreter process on Kerberos as web front end user" - ZEPPELIN-1320: "Run zeppelin interpreter process as web front end user" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 ... and make ZEPPELIN-1320 a subtask of ZEPPELIN-1337 ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 Btw, for the hadoop case (or spark on yarn case), this PR may give an issue for `doAs`. Typically, you configure `hadoop.proxyuser.foo.hosts` and `hadoop.proxyuser.foo.group`, `foo` being the os/kerberos user under which you run your java code that calls doAs. If we run ssh/su as the front-end user, we will not fullfill what the hadoop/yarn cluster is expecting. We thus should have two checkboxes: - One for the OS/kerberos impersonation (this PR only adresses OS). - The other for Hadoop impersonation. If you select one, I would expect the other one to be disabled. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #1361: [ZEPPELIN-1354] [WIP] Inject Services
GitHub user echarles opened a pull request: https://github.com/apache/zeppelin/pull/1361 [ZEPPELIN-1354] [WIP] Inject Services ### What is this PR for? Services like authentication, authorization, security filters... should be injected rather than manually instanciated. This will allow to give central + finer control and abstracting the functionaly in interface and allow third parties to implement their own functions in with configurable injection points. For this, I propose to use Guice as injection framework. ### What type of PR is it? Improvement ### Todos * [ ] - Documentation * [ ] - Tests ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1354 ### How should this be tested? Just compile and run ### Screenshots (if appropriate) NA ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? Y You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer inject-services Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1361.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1361 commit 5002a1583681cff847c08977e614963a854d8f1b Author: Eric Charles Date: 2016-08-24T16:05:52Z add guice dependencies commit 0fd9096e44c92b715cf7efa9dc82fc672564542e Author: Eric Charles Date: 2016-08-25T08:16:51Z Merge branch 'master' into inject-services Conflicts: pom.xml zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java commit 1d2b80a9be1054312ccba3b5d4b4f3086ecd3f33 Author: Eric Charles Date: 2016-08-25T08:17:58Z Initial working implemenation of service injection for WebSecurity and SearchService --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 @bzz are we ok now with the license information? Travis is red for 3 on the 9 profiles, but this comes from a timeout trying to launch/access spark ```16/08/17 15:36:04 INFO ExecutorRunner: Launch command: "/usr/lib/jvm/java-7-oracle/bin/java" "-cp" "/home/travis/build/apache/zeppelin/spark-1.6.1-bin-hadoop2.3/conf/:/home/travis/build/apache/zeppelin/spark-1.6.1-bin-hadoop2.3/lib/spark-assembly-1.6.1-hadoop2.3.0.jar:/home/travis/build/apache/zeppelin/spark-1.6.1-bin-hadoop2.3/lib/datanucleus-core-3.2.10.jar:/home/travis/build/apache/zeppelin/spark-1.6.1-bin-hadoop2.3/lib/datanucleus-api-jdo-3.2.6.jar:/home/travis/build/apache/zeppelin/spark-1.6.1-bin-hadoop2.3/lib/datanucleus-rdbms-3.2.9.jar" "-Xms1024M" "-Xmx1024M" "-Dspark.driver.port=47348" "-XX:MaxPermSize=256m" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@172.17.14.104:47348" "--executor-id" "0" "--hostname" "172.17.14.104" "--cores" "2" "--app-id" "app-20160817153604-0003" "--worker-url" "spark://Worker@172.17.14.104:41888" 16/08/17 15:36:40 INFO Worker: Asked to kill executor app-20160817153604-0003/0 16/08/17 15:36:40 INFO ExecutorRunner: Runner thread for executor app-20160817153604-0003/0 interrupted 16/08/17 15:36:40 INFO ExecutorRunner: Killing process! 16/08/17 15:36:41 INFO Worker: Executor app-20160817153604-0003/0 finished with state KILLED exitStatus 143 16/08/17 15:36:41 INFO Worker: Cleaning up local directories for application app-20160817153604-0003 16/08/17 15:36:41 INFO ExternalShuffleBlockResolver: Application app-20160817153604-0003 removed, cleanupLocalDirs = true travis_time:end:17fafc60:start=1471448259153159015,finish=1471448259166402551,duration=13243536 ``` Anything else I can do to get this merged? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1357: ZEPPELIN-1364: Return only the ran paragraph details f...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1357 Didn't try the changes but definitively +1 for this to enhance performance (tried to imagine side effects but didn't find any atm) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 Merged with latest master. @bzz I don't know where I found CDDL 2.0 (sounded indeed strange when I was writing this some time ago). jax-rs is now CDDL 1.0, beanvalidation is now ASL2 and finally, excluded javax.annotation jar (classes are shipped in JRE). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1371: [ZEPPELIN-1372]Automatically Detect the data type in t...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1371 Upon numeric values, the automatic detection of dates... would be great. Detection at client (javascript) side could be misleading as you only get a subset of the data. On the other side, detection on server (scala) side may be more time consuming for hughe resultset. Still this could be a real added value as you could expose the types, units... via the API. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1371: [ZEPPELIN-1372]Automatically Detect the data type in t...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1371 @Peilin-Yang say you resultset is 50.000 lines, clients only have 1.000 lines (if I remember well). There is a risk your type guess is not correct. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #1385: [ZEPPELIN-1395] Local or Remote Interpreter by ...
GitHub user echarles opened a pull request: https://github.com/apache/zeppelin/pull/1385 [ZEPPELIN-1395] Local or Remote Interpreter by Configuration ### What is this PR for? The current remote interpreter (launched via shell) is scalable and open but suffers from 2 limitations: - It does not allow to easily debug in the IDE. - It does not allow to embedd Zeppelin (for small deployements) in a single jar. A configuration per interpreter "Remote Interpreter" (true/false, true by default) is proposed. An additional property `zeppeline.interpreter.remote' in zeppelin-site.xml is also proposed (a new interpreter will be created with that value for it `option.remote` attribute). ### What type of PR is it? [Improvement] ### Todos * [ ] - Value defined in zeppelin-site.xml is not honored when creating a new interepreter * [ ] - Unit Test * [ ] - Documentatoin ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1395 ### How should this be tested? Load Zeppelin source in your IDE and start ZeppelinServer in debug mode Put breakpoints in the interpreter sources, Set / Uset the "Remote Interpeter (see screeenshot). ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? Y You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer remote-interpreter-config Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1385.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1385 commit f70331d28eb84158e05493ab5e989b37781dbc2b Author: Eric Charles Date: 2016-08-30T03:48:52Z initial working implementation of local/remote interpreter configuration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #1390: [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] Note and ...
GitHub user echarles opened a pull request: https://github.com/apache/zeppelin/pull/1390 [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] Note and services per user ### What is this PR for? This implements "notes per user" and "Multiple-users on a single WEB server" - For implementation reasons (the static fields present on the zeppelin server, and the need for each users to bind/configure their own note to the interpereters managed by the InterpreterFactory), this PR implements both - In other words, it would have been too difficult to decouple both). ### What type of PR is it? [Feature] ### Todos * [ ] - More test * [ ] - Documentation * [ ] - Migration (notebook folder structure changes) * [ ] - Discuss the S3, Azure... repo cases * [ ] - Check unit tests * [ ] - Forsee a "shared" notebook workspace to mimick current behavior ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1339 * https://issues.apache.org/jira/browse/ZEPPELIN-1000 ### How should this be tested? Logon / Logout with different users and check that the notes are different. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? Y * Does this needs documentation? Y You can merge this pull request into a Git repository by running: $ git pull https://github.com/datalayer/zeppelin-datalayer multiuser Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1390.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1390 commit 60c9380c3545e294ae8e776275b628899738cfc1 Author: Eric Charles Date: 2016-08-31T15:43:09Z WIP Initial working implementation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1390: [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] Note and service...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1390 @khalidhuseynov Good to discuss with you about what a multiuser note management would be. For the file system repo, this PR will give you: ``` âââ anonynmous â  âââ 2A94M5J1Z â  â  âââ note.json â  âââ 2BQA35CJZ â  â  âââ note.json â  âââ r â  â âââ note.json â  âââ interpreter-setting.json âââ user1 â  âââ 2BWADFP17 â  â  âââ note.json â  âââ interpreter-setting.json âââ user2 âââ 2BU5DAHBJ â  âââ note.json âââ interpreter-setting.json ``` The note folders are moved one level-down, each user having its workspace. Each user also has its own `interpreter-setting.json` (was before in the `conf` folder) - Just like before, each interpreter contribute to the settings via classpath or file system. Interpreter settings and bindings assignment is made possible by keeping separate interpreterfactory, searchservice... per web user session. To ensure backwards compatibility, I propose (not yet implemented) a `zeppelin.notes.per-user` configuration being `false` by default. The other repository implements (azure, github...) need more love with the credential service (each user can have their own credentials). PS: I had a quick look at #1392 - Let's continue this discussion - Depending on the outcomes, we can decide `what`, `who` and `where` - About the `when`, the soone the better. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1394: rename r directory to 2BWJFTXKJ
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1394 @prabhjyotsingh yes, assumption is correct. At the time of the R interpreter dev, I was used to manually rename the note folder and edit the json to set the id the same value. This PR takes it back to the default layout (didn't test it, but checked that the id value in the json was also correct). LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1390: [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] [ZEPPELIN-1338] ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1390 @zjffdu Correct, this PR also implements ZEPPELIN-1338 User level interpreter setting (I have updated the title and description on this PR). @Leemoonsoo Will check and adapt if needed the behavior for multiple owners and will further digg into #1265 to assess any conflit/divergence. I'd like to second the comment made by @zjffdu in #1265 on the interpreter options complexity. Btw, at the time of the introduction of the current interpreter modes, I already raised my worries on the complexity of those and honestly I am not convinced that I understand them fully (at least at first sight without digging into the code) and I fear that #1265 will make the situation still more complex. Taking back @zjffdu statement: *Besides, I think we should think more deeply on the relationship between note, user and interpreter.*, and I would add also the relation with the SearchService, Helium... and any other services that Zeppelin uses (cc/ @jongyoul) - All these questions also binds to ZEPPELIN-1236 "Multi-user notebook with user controls support" created by @frosiere talking about shared dashboard... Having this discussion on the ma iling list independently of any PR would better define the efforst to be taken in each PR. @khalidhuseynov I have the bad habit to build with `-Dcheckstyle.skip=true` - I commit a fix in the next hour. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1385: [ZEPPELIN-1395] Local or Remote Interpreter by Configu...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1385 @astroshim Thx for the review. I have pushed a fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1385: [ZEPPELIN-1395] Local or Remote Interpreter by Configu...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1385 @jongyoul `ZeppelinConfiguration.create()` is now used and method signature are now like before. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1390: [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] [ZEPPELIN-1338] ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1390 I have introduced a parameter in the URL `?runAs=...` that allows to impersonate from an authenticated user another user. The needed R/W authorization are respected. An alternative would be to intercalate the user in the URL: http://localhost/#/userName/noteId Regarding multiple owners, I would simply drop this feature (just like a file, a note belongs to a unique user). Apart minor navigation glitches, it works pretty well here. I continue to follow #1265 and try to keep this PR orthogonal to what is discussed/implemented in there. I will add a note regarding ZEPPELIN-1144 Zeppelin home page should only list notebooks with read or write permission #1330 - If listing all notes with read access is mandatory, I would have to fallback to the single `notebook-authorization.json`, or making the NotebookAuthorization service independent from the Notebook object. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1330: [ZEPPELIN-1144]Zeppelin home page should only list not...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1330 If the current single `notebook-authorization.json` is managed by user (with multiple files, and no more with a single one), how would you handle it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1390: [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] [ZEPPELIN-1338] ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1390 @khalidhuseynov - have fixed the format issues - I didn't change Notebook#getNotebookAuthorization, it was already present. However, a single global NotebookAuthorization as you like will change this. Will push something that moves the NotebookAuthorization out of the Notebook. - The ZEPPELIN_AUTH_USER_KEY is used to remove the object from the user WEB session when user session expires... (to free memory and avoid OOM) however it is not yet used to put the objects... Will also push something. @corneadoug I will write asap a small document explaining the logic behing this PR (interpreter factory by user, runAs...). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1385: [ZEPPELIN-1395] Local or Remote Interpreter by Configu...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1385 @jongyoul ZeppelinConfiguration is now remove also on that method. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #930: Make path for .spark-dist folder always accessib...
Github user echarles closed the pull request at: https://github.com/apache/zeppelin/pull/930 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #930: Make path for .spark-dist folder always accessible [ZEP...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/930 Closing this PR, I can not reproduce the issue anymore. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/932 @bzz pushed my local merge. Anything else that should be done before you merge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1390: [WIP] [ZEPPELIN-1339] [ZEPPELIN-1000] [ZEPPELIN-1338] ...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1390 I have a prototype that supports the 3 modes. Give me a few days to polish and commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1764: ZEPPELIN-1816. Implement getProgress in SparkRInterpre...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1764 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2060 I think this change breaks the installation of spell modules that are not yet published in npmjs repository. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/2060 I am adding a `json` file in the `helium` folder with the artifact that points to a full path on my local drive. In that location I have the spell project. It was working 3 days ago. Today, I updated, and it does not work anymore unless I revert the HeliumBundleFactory class to its previous version. But if it works at your side, it must be my env that needs to be fixed. I will double check and report back. Thx again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
