[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 Could you create another PR to trigger the build ? ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user mebelousov commented on the issue: https://github.com/apache/zeppelin/pull/2442 Cache is clean now. I try to retrigger from both accounts. The result is the sameÑ ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 ping @tinkoff-dwh @mebelousov ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 @mebelousov Could you clean the travis cache and for a dummy force push to trigger the build ? ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user mebelousov commented on the issue: https://github.com/apache/zeppelin/pull/2442 @zjffdu I have retriggered several times. Travis fails with the same error on all jobs. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 @tinkoff-dwh @mebelousov Could you retrigger the build ? Travis seems fail ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 @tinkoff-dwh Could you rebase the PR ? ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user Sanikadze commented on the issue: https://github.com/apache/zeppelin/pull/2442 +1 ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user Aspekt112 commented on the issue: https://github.com/apache/zeppelin/pull/2442 +1 ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user anermakov commented on the issue: https://github.com/apache/zeppelin/pull/2442 +1 Must have ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user kapustor commented on the issue: https://github.com/apache/zeppelin/pull/2442 +1 Must-have feature for enterprise usage. Pre-code may be a resource-management code, or a security code, etc. - users must not have possibility to change it, only admins. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user mebelousov commented on the issue: https://github.com/apache/zeppelin/pull/2442 @zjffdu hooks are user defined code. We want to have opportunity to log the user sessions, to bind queries to notes. It can be possible only by force execution of special query before user queries, that is in precode. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 Sorry @mebelousov I don't get why the general hooks could not satisfy you. The hook can run for each paragraph. Could you explain a little bit more ? Maybe an example would be very helpful. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user mebelousov commented on the issue: https://github.com/apache/zeppelin/pull/2442 @zjffdu The hooks as pre-code were discussed [previously ](https://github.com/apache/zeppelin/pull/2096#issuecomment-284220634) We want precode execute before pre-hook for easy finding the note. Meanwhile the hooks doesn't realized in JDBC interpreter. The JDBC interpreter allow to split query and precode must run before each statement in paragraph. Therefore the single way for precode for all interpreters looks unrealizable. The rename to `zeppelin.interpreter.precode` is good idea. Do this? ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 @tinkoff-dwh I just realised that zeppelin already support hooks for interpreter. I think you don't need to do it specifically for jdbc interpreter, instead we could introduce one general property like `zeppelin.interpreter.pre_code` which could execute pre code before running the paragraph text. What do you think ? ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user mebelousov commented on the issue: https://github.com/apache/zeppelin/pull/2442 @zjffdu, you are right, sessionPrecode is renamed to statementPrecode. Since 0.8.0 I will recommend to run JDBC interpreter per note in isolated mode due to Interpreter Lifecycle Manager: the table results accumulate in ResourcePool and JVMs grow in size, but ILM will drop non-active JVMs. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 Mostly LGTM, one concern is about the name `session` which might confuse users. How about changing it to `statement` ? Besides, I am curious to know how do you use jdbc interpreter. Which mode do you use ? (Shared/Scoped/Isolated) . Because jdbc interpreter use connection pool, that's why I ask this question, just want to know how do you schedule the connection across users ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user mebelousov commented on the issue:
https://github.com/apache/zeppelin/pull/2442
@zjffdu @felixcheung
Every day our users run about 1000 queries in the Greenplum (open source
Postgres-based DB).
There is great need for binding the query to the user and/or to the note.
The precode `set application_name=#{noteId};` allows quickly find the note.
This precode must be set by Zeppelin administrator, business users will not
write any precode in each note for each jdbc-interpreter.
I believe the precode in JDBC interpreter is necessary feature for
entreprise users.
---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user dwhsys commented on the issue: https://github.com/apache/zeppelin/pull/2442 Can someone review please? ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 Ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue:
https://github.com/apache/zeppelin/pull/2442
it is JDBC interpretator, set variables in first paragraph will not work
because we have connection pool and the paragraphs will be executed in
different statemnets. What problem with shared notes. Фny user can execute and
everyone will have their `precode`. `set search_path='#{user}` #{user} will
be replaced with the username from the context
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 I am thinking that every interpreter could have one special interpreter for initializing interpreter setting and custom enviroment requirement for his note. And do the initialization at the first paragraph. e.g. ``` %spark.init set property_1=value_1 set property_2=value_2 ``` So that the note could be shared to other people, and other people can reproduce the note result easily. My point is that note should be self-contained thing that could be reproduced when it is shared to other people However it is still very immature idea. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 I am not sure whether this is a good idea to put custom initialization code in interpreter setting. Because I think one goal of notebook is to allow user to reproduce it easily without change note. So any non-trivial thing should be put to note itself. Otherwise user don't know how to reproduce the note result in another zeppelin instance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue:
https://github.com/apache/zeppelin/pull/2442
@zjffdu
yep, per statement.
for exp.
There is jdbc interpreter per user (postgres).
There are many users and each user has scheme to work.
1. set search_path='#{user}' (default.sessionPrecode)
2. run paragraph `select ...`
the user will work with the correct scheme
or set another variables and values will be available to code in the
paragraph (SET TIME ZONE, custom variables, etc...). User == analitic (usually)
and he doesn't have to write this in the code of paragraph
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2442 @tinkoff-dwh What do you mean `database session` ? From the code, `precode` is executed per statement. And what is the usage scenario for this ? Could you give an example ? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2442: [ZEPPELIN-2679] JDBC. precode for session
Github user tinkoff-dwh commented on the issue: https://github.com/apache/zeppelin/pull/2442 @felixcheung session == database session. added documentation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
