[GitHub] [incubator-kyuubi] hddong opened a new issue #891: Add audit log for sql statement.
hddong opened a new issue #891: URL: https://github.com/apache/incubator-kyuubi/issues/891 ### 1. Describe the feature Add a additional log file to audit sql statement. ### 2. Motivation Audit sql statement. ### 3. Describe the solution ### 4. Additional context -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #891: Add audit log for sql statement.
yaooqinn commented on issue #891: URL: https://github.com/apache/incubator-kyuubi/issues/891#issuecomment-892526320 Looks like a good feature to have -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka opened a new issue #893: Add query watchdog to stop disruptive queries
RamakrishnaChilaka opened a new issue #893: URL: https://github.com/apache/incubator-kyuubi/issues/893 ### 1. Describe the feature Add query watchdog to stop disruptive queries ### 2. Motivation Add query watchdog to stop disruptive queries. will stop these rogue queries to saturate the cluster. ### 3. Describe the solution ### 4. Additional context -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #893: Add query watchdog to stop disruptive queries
yaooqinn commented on issue #893: URL: https://github.com/apache/incubator-kyuubi/issues/893#issuecomment-893127460 Do we have a design doc for this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 opened a new issue #894: Enhance kyuubi-zookeeper module
timothy65535 opened a new issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894 ### 1. Describe the feature Kyuubi use `TestingServer` as the EmbeddedZookeeper service in codebase, and require many args to init the service. These `electionPort`, `quorumPort`, `serverId` parameters only work in cluster mode, these ports are not bound to the host. From hadoop and bookkeeper project, they use `ZooKeeperServer` as a built-in zk service, especially the bookeeper project. - https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java - https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java ### 2. Describe the solution Improve - Remove `electionPort`, `quorumPort`, `serverId` which are nerver be used, hadoop and bookkeer do the same - Replace `TestingServer` with `ZooKeeperServer` - Remove `curator-test` dependency -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #894: Enhance kyuubi-zookeeper module
yaooqinn commented on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893135770 Removing TestingServer sounds good to me > Remove electionPort, quorumPort, serverId which are nerver be used, hadoop and bookkeer do the same Can we make each KyuubiServer served as a quorum peer? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 opened a new issue #895: Enhance output uses the default spark conf
timothy65535 opened a new issue #895:
URL: https://github.com/apache/incubator-kyuubi/issues/895
### 1. Describe the bug
When start the server, If not specified `SPARK_CONF_DIR`, it should output
default.
```
KYUUBI_WORK_DIR_ROOT: /root/tmp/kyuubi/work
SPARK_HOME: /root/package/spark-3.1.2-bin-hadoop2.7
SPARK_CONF_DIR:
HADOOP_CONF_DIR: /root/package/hadoop-3.2.0/etc/hadoop
```
### 2. Environments
2.1 Versions
- Kyuubi
- [ ] 1.1.0
- [ ] 1.0.3
- [ ] 0.8.x and earlier
- Spark
- [ ] 3.1.x
- [ ] 3.0.x
- [ ] 2.4.x and earlier
2.2 Kyuubi Configurations
2.3 Spark Configurations
### 3. Steps to reproduce
3.1 Operations
3.2 Expected behavior
3.3 Actual behavior
### 4. Additional context
4.1 Log of the server
4.2 Log of the corresponding engine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on issue #894: Enhance kyuubi-zookeeper module
timothy65535 commented on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893138616 > Removing TestingServer sounds good to me > > > Remove electionPort, quorumPort, serverId which are nerver be used, hadoop and bookkeer do the same > > Can we make each KyuubiServer served as a quorum peer? Maybe we don't need to do this, because `TestingServer` is not a full feature supported. For zookeeper quorum peer, it is recommended to use a separate zookeeper cluster. For kyuubi quorum peer, it seems raft is a better choice(I not sure). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #894: Enhance kyuubi-zookeeper module
yaooqinn commented on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893139151 Ok, we can keep optimizing this module when we find better solutions someday. we can remove the testing server here first -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you closed issue #895: Enhance output uses the default spark conf
ulysses-you closed issue #895: URL: https://github.com/apache/incubator-kyuubi/issues/895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on issue #894: Enhance kyuubi-zookeeper module
timothy65535 commented on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893182251 Got @yaooqinn, do you have any ideas @pan3793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #894: Enhance kyuubi-zookeeper module
pan3793 commented on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893196646 Thanks for this proposal, replace `TestingServer` with `ZooKeeperServer` make sense to me. > For Kyuubi quorum peer, it seems raft is a better choice(I not sure). Do you mean implements a Raft component like KRaft in Kafka 3.0? Personally, I think Paxos or ZAB or Raft or Gossip is not really mattered here, maybe an abstract service discover layer with adapters to K8s `api-server`, `Zookeeper`, `Etcd` make more sense to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on issue #894: Enhance kyuubi-zookeeper module
timothy65535 commented on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893201807 Will remove the testing server first. > maybe an abstract service discover layer with adapters to K8s api-server, Zookeeper, Etcd make more sense to me. Let's think about it, will let you know if have any ideas. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 edited a comment on issue #894: Enhance kyuubi-zookeeper module
timothy65535 edited a comment on issue #894: URL: https://github.com/apache/incubator-kyuubi/issues/894#issuecomment-893201807 Do remove the testing server first. > maybe an abstract service discover layer with adapters to K8s api-server, Zookeeper, Etcd make more sense to me. Let's think about it, will let you know if have any ideas. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you opened a new issue #897: Support stop Spark engine through web ui
ulysses-you opened a new issue #897: URL: https://github.com/apache/incubator-kyuubi/issues/897 ### 1. Describe the feature Enhance Spark web ui to allow user stop the engine. ### 2. Motivation In **user** shared level, an engine could exist a long time then it's useful to provide an entrance to alow user stop the engine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
