[GitHub] [incubator-livy] jerryshao closed pull request #171: [LIVY-591] Changed ACLs enforcement to occur on proxy user, if available
jerryshao closed pull request #171: [LIVY-591] Changed ACLs enforcement to occur on proxy user, if available URL: https://github.com/apache/incubator-livy/pull/171 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #172: allow proxy user to view or modify his session
jerryshao closed pull request #172: allow proxy user to view or modify his session URL: https://github.com/apache/incubator-livy/pull/172 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #202: [LIVY-592][Server] Allow proxy user to view or modify his session
jerryshao closed pull request #202: [LIVY-592][Server] Allow proxy user to view or modify his session URL: https://github.com/apache/incubator-livy/pull/202 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] TisonKun commented on issue #203: [hotfix][log] log stack trace properly
TisonKun commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-523719419 Thanks for your review! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #203: [hotfix][log] log stack trace properly
jerryshao closed pull request #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #203: [hotfix][log] log stack trace properly
jerryshao commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-523718168 Thanks for the fix, merging to master branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] TisonKun commented on issue #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
TisonKun commented on issue #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200#issuecomment-523456656 I think another solution should be use an auto-retest trigger instead of close-reopen which would pollute notification quite a lot... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] lucasces commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
lucasces commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523426742 > @lucasces > Nice to know that!) Can you share your feedback on using it? > > In any case you can use your own charts or propose contribution to mine ones. I plan to propose them in a separate PR after this one will be merged. Sure, @jahstreet. I will open a PR right away. In a short version, the way *StatefulSets* work make possible for spark to resolve livy's pod ip by hostname and talk to it directly, bypassing the *Service*. Our experience with livy on kubernetes has trully exceded our expectations. I've used livy with hadoop cluster on k8s for more than a year and was not a pleasant experience. It worked fine for my case as we had few small jobs running, but could not handle more jobs easely. Our demand grew and the current setup is on EKS, running cluster-autoscaller and almost all of your proposed stack (livy, spark-history-server, jupyterhub, monitoring). It fits nicely with our workflow and now our test environment is much more reliable and cheap than our production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jahstreet commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
jahstreet commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523409197 @lucasces Nice to know that!) Can you share your feedback on using it? In any case you can use your own charts or propose contribution to mine ones. I plan to propose them in a separate PR after this one will be merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] lucasces commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
lucasces commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523407598 > I'm now trying to get this PR to work and I'm facing an issue where the started driver pod fails to connect back to the livy server to port 1. > > There's a relevant log line from the driver: > `RSCDriver:160 - Connecting to: livy.spark.svc:1` > > The livy.spark.svc is a valid DNS hostname pointing to the service which points to the livy, but it maps only port 80. If the driver would instead have the livy server pod ip it would work. I'm not sure how this is supposed to work. If you installed with @jahstreet charts, there is a [coment](https://github.com/jahstreet/spark-on-kubernetes-helm/blob/c7c7cd22c76e50fc9335882f4bf7daa400f3d141/charts/livy/values.yaml#L38) in livy's chart values.yaml sugesting that you should declare that ports manually. I'm using this version of livy on k8s for a few months now and figured that converting the *Deployment* into a *StatefulSet* is more effective. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jahstreet edited a comment on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
jahstreet edited a comment on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523406514 @garo , @lukatera I assume you are running interactive apps, right? If you set `service.additionalPorts` (https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/livy/values.yaml#L41) property on deploying chart everything should work. This property adds ports both to service and deployment specs. Please refer: - Livy service template: https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/livy/templates/service.yaml#L17 - Livy deployment template: https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/livy/templates/deployment.yaml# BTW: combined spark-cluster chart already contains all you need: https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/spark-cluster/values.yaml#L24 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jahstreet commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
jahstreet commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523406514 @garo , @lukatera I assume you are running interactive apps, right? If you set `service.additionalPorts` (https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/livy/values.yaml#L41) property on deploying chart everything should work. This property adds ports both to service and deployment specs. Please refer: - Livy service template: https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/livy/templates/service.yaml#L17 - Livy deployment template: https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/livy/templates/deployment.yaml# This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 ## What changes were proposed in this pull request? Spark beeline use old hive-jdbc-client doesn’t do null point ref check. So when new TRowSet, setColumes make sure column set not null. ## How was this patch tested? Connect to livy's thriftserver using spark beeline. And create/use/drop database will no longer have nullpointerexceptions after execution.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] lukatera commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
lukatera commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523393751 > I'm now trying to get this PR to work and I'm facing an issue where the started driver pod fails to connect back to the livy server to port 1. > > There's a relevant log line from the driver: > `RSCDriver:160 - Connecting to: livy.spark.svc:1` > > The livy.spark.svc is a valid DNS hostname pointing to the service which points to the livy, but it maps only port 80. If the driver would instead have the livy server pod ip it would work. I'm not sure how this is supposed to work. The livy service needs to be a headless service. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on issue #193: [LIVY-621]add dynamic service discovery for thrift server
mgaido91 commented on issue #193: [LIVY-621]add dynamic service discovery for thrift server URL: https://github.com/apache/incubator-livy/pull/193#issuecomment-523392761 the point here is: from the project perspective we should first achieve the same level of HA and robustness for the Livy server part. Then, we can also have it for the thriftserver. It'd be very weird that HA is available only for the thrift part and not for the REST API for instance. So I think that in order to have this, we should first wait for #189. Once that is over, we can port the same functionality to the thrift part, eventually leveraging the solution already present there. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift se
mgaido91 commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r316100953
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/SparkCatalogOperation.scala
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.commons.lang.StringUtils
+import org.apache.hive.service.cli._
+
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+import org.apache.livy.thriftserver.serde.ThriftResultSet
+import org.apache.livy.thriftserver.session.{CleanupCatalogResultJob,
FetchCatalogResultJob}
+
+/**
+ * SparkCatalogOperation is the base class for operations which need to fetch
catalog information
+ * from spark session.
+ */
+abstract class SparkCatalogOperation(
+sessionHandle: SessionHandle,
+opType: OperationType,
+sessionManager: LivyThriftSessionManager)
+ extends Operation(sessionHandle, opType) {
+
+ // The initialization need to be lazy in order not to block when the
instance is created
+ protected lazy val rscClient = {
+// This call is blocking, we are waiting for the session to be ready.
+sessionManager.getLivySession(sessionHandle).client.get
+ }
+
+ protected lazy val jobId = {
+this.opHandle.getHandleIdentifier.getPublicId.toString + "-" +
+ this.opHandle.getHandleIdentifier.getSecretId.toString
+ }
+
+ protected lazy val sessionId = {
+sessionHandle.getSessionId.toString
+ }
+
+ @throws[HiveSQLException]
+ override def close(): Unit = {
+val cleaned = rscClient.submit(new CleanupCatalogResultJob(sessionId,
jobId)).get()
+if (!cleaned) {
+ warn(s"Fail to cleanup fetch catalog job (session = ${sessionId}), " +
+"this message can be ignored if the job failed.")
+}
+setState(OperationState.CLOSED)
+ }
+
+ @throws[HiveSQLException]
+ override def cancel(stateAfterCancel: OperationState): Unit = {
+setState(OperationState.CANCELED)
+// Spark fetch schema is not a really spark job. It only run on driver and
cannot be cancelled
+throw new UnsupportedOperationException("SparkCatalogOperation.cancel()")
+ }
+
+ /**
+ * Convert wildchars and escape sequence from JDBC format to
datanucleous/regex
+ *
+ * This is ported from Spark Hive Thrift MetaOperation.
+ */
+ protected def convertIdentifierPattern(pattern: String, datanucleusFormat:
Boolean): String = {
+if (pattern == null) {
+ convertPattern("%", datanucleusFormat = true)
+} else {
+ convertPattern(pattern, datanucleusFormat)
+}
+ }
+
+ /**
+ * Convert wildchars and escape sequence of schema pattern from JDBC format
to datanucleous/regex
+ * The schema pattern treats empty string also as wildchar.
+ *
+ * This is ported from Spark Hive Thrift MetaOperation.
+ */
+ protected def convertSchemaPattern(pattern: String): String = {
+if (StringUtils.isEmpty(pattern)) {
+ convertPattern("%", datanucleusFormat = true)
+} else {
+ convertPattern(pattern, datanucleusFormat = true)
+}
+ }
+
+ private def convertPattern(pattern: String, datanucleusFormat: Boolean):
String = {
+val wStr = if (datanucleusFormat) "*" else ".*"
+pattern
+ .replaceAll("([^])%", "$1" + wStr)
+ .replaceAll("%", "%")
+ .replaceAll("^%", wStr)
+ .replaceAll("([^])_", "$1.")
+ .replaceAll("_", "_")
+ .replaceAll("^_", ".")
+ }
+
+ override def getNextRowSet(orientation: FetchOrientation, maxRowsL: Long):
ThriftResultSet = {
+validateFetchOrientation(orientation)
+assertState(Seq(OperationState.FINISHED))
+setHasResultSet(true)
+val maxRows = maxRowsL.toInt
+val results = rscClient.submit(new FetchCatalogResultJob(sessionId, jobId,
maxRows)).get()
+
+val rowSet = ThriftResultSet.apply(getResultSetSchema, protocolVersion)
Review comment:
why not `ThriftResultSet.apply(results)`?
This is an automated
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift se
mgaido91 commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r316096943
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/MetadataOperation.scala
##
@@ -21,6 +21,12 @@ import org.apache.hive.service.cli.{FetchOrientation,
HiveSQLException, Operatio
import org.apache.livy.thriftserver.serde.ThriftResultSet
+/**
+ * MetadataOperation is the base class for operations which do not perform
any call on Spark side
+ *
+ * @param sessionHandle
Review comment:
what does this mean? I mean, no description at all, the name of the
parameters can also be read from the method signature...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #194: [LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions, GetSchemas, GetTables, GetColumns in Livy thrift se
mgaido91 commented on a change in pull request #194:
[LIVY-622][LIVY-623][LIVY-624][LIVY-625][Thrift]Support GetFunctions,
GetSchemas, GetTables, GetColumns in Livy thrift server
URL: https://github.com/apache/incubator-livy/pull/194#discussion_r316096354
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/GetFunctionsOperation.scala
##
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.livy.thriftserver.operation
+
+import org.apache.hive.service.cli.{HiveSQLException, OperationState,
OperationType, SessionHandle}
+
+import org.apache.livy.Logging
+import org.apache.livy.thriftserver.session.GetFunctionsJob
+import org.apache.livy.thriftserver.types.{BasicDataType, Field, Schema}
+import org.apache.livy.thriftserver.LivyThriftSessionManager
+
+class GetFunctionsOperation(
+sessionHandle: SessionHandle,
+catalogName: String,
+schemaName: String,
+functionName: String,
+sessionManager: LivyThriftSessionManager)
+ extends SparkCatalogOperation(
+sessionHandle, OperationType.GET_FUNCTIONS, sessionManager) with Logging {
+
+ @throws(classOf[HiveSQLException])
+ override protected def runInternal(): Unit = {
+setState(OperationState.RUNNING)
+try {
+ rscClient.submit(new GetFunctionsJob(
+convertSchemaPattern(schemaName),
+convertFunctionName(functionName),
+sessionId,
+jobId
+ )).get()
+
+ setState(OperationState.FINISHED)
+} catch {
+ case e: Throwable =>
+error("Remote job meet an exception: ", e)
+setState(OperationState.ERROR)
+throw new HiveSQLException(e)
+}
+ }
+
+ @throws(classOf[HiveSQLException])
+ override def getResultSetSchema: Schema = {
+assertState(Seq(OperationState.FINISHED))
+GetFunctionsOperation.SCHEMA
+ }
+
+ private def convertFunctionName(name: String): String = {
Review comment:
sorry, I still don't understand why we need this method. May you explain me?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] garo commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
garo commented on issue #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes URL: https://github.com/apache/incubator-livy/pull/167#issuecomment-523389950 I'm now trying to get this PR to work and I'm facing an issue where the started driver pod fails to connect back to the livy server to port 1. There's a relevant log line from the driver: ```RSCDriver:160 - Connecting to: livy.spark.svc:1``` The livy.spark.svc is a valid DNS hostname pointing to the service which points to the livy, but it maps only port 80. If the driver would instead have the livy server pod ip it would work. I'm not sure how this is supposed to work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r316082378
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
##
@@ -204,6 +204,12 @@ public BitSet getNulls() {
return nulls != null ? BitSet.valueOf(nulls) : new BitSet();
}
+ /**
+ * Extract subset data to a new column buffer.
+ * @param start start row number
+ * @param end end row number, which is not included in the subset data
+ * @return
+ */
public ColumnBuffer extractSubset(int start, int end) {
ColumnBuffer subset = new ColumnBuffer(type);
subset.currentSize = end - start;
Review comment:
I am afraid that the `arraycopy` can go out of the boundaries (eg. a column
with 110 nulls). Can you please check what was done in
https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/thrift/ColumnBuffer.java
and do something similar here?
Moreover, may you please add tests for the case mentioned above? Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc commented on issue #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc commented on issue #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200#issuecomment-523345755 Hi @TisonKun Now livy's Travis is unstable and needs several retries to succeed. So I turn this PR on and off each time to re-execute Travis. The problem of unstable Travis needs to be solved later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] TisonKun commented on issue #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
TisonKun commented on issue #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200#issuecomment-523336154 Hi @jerryshao shall we make progress on this thread? @captainzmc may I ask why you intermittently close and reopen the issue? Do you want to retest the pull request? If so, maybe there is or we can implement a retest approach instead of close and reopen. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 ## What changes were proposed in this pull request? Spark beeline use old hive-jdbc-client doesn’t do null point ref check. So when new TRowSet, setColumes make sure column set not null. ## How was this patch tested? Connect to livy's thriftserver using spark beeline. And create/use/drop database will no longer have nullpointerexceptions after execution.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #206: Branch 0.6
jerryshao commented on issue #206: Branch 0.6 URL: https://github.com/apache/incubator-livy/pull/206#issuecomment-523334810 Hi @EthanDCX I guess you mistakenly opened this PR, I'm going to close it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #206: Branch 0.6
jerryshao closed pull request #206: Branch 0.6 URL: https://github.com/apache/incubator-livy/pull/206 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 ## What changes were proposed in this pull request? Spark beeline use old hive-jdbc-client doesn’t do null point ref check. So when new TRowSet, setColumes make sure column set not null. ## How was this patch tested? Connect to livy's thriftserver using spark beeline. And create/use/drop database will no longer have nullpointerexceptions after execution.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
