[GitHub] [incubator-livy] captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc opened a new pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 ## What changes were proposed in this pull request? Spark beeline use old hive-jdbc-client doesn’t do null point ref check. So when new TRowSet, setColumes make sure column set not null. ## How was this patch tested? Connect to livy's thriftserver using spark beeline. And create/use/drop database will no longer have nullpointerexceptions after execution.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc closed pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts… URL: https://github.com/apache/incubator-livy/pull/200 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315990550
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
##
@@ -204,6 +204,12 @@ public BitSet getNulls() {
return nulls != null ? BitSet.valueOf(nulls) : new BitSet();
}
+ /**
+ * Extract subset data to a new column buffer.
+ * @param start start row number
+ * @param end end row number, which is not included in the subset data
+ * @return
+ */
public ColumnBuffer extractSubset(int start, int end) {
ColumnBuffer subset = new ColumnBuffer(type);
subset.currentSize = end - start;
Review comment:
Yes, you're right. I should handle the nulls in the `extractSubset` method.
The tests should not pass. I changed null to empty string in the test case, as
I didn't realize it's a big issue...
I have fixed this and change back the tests. I also handle some illegal
start/end input of the method and added some [test
cases](https://github.com/apache/incubator-livy/pull/197/files#diff-275dca4a619dd9b0b7f3abbfdfeb00f3R233)
to cover this. Can you please review? Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315990550
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
##
@@ -204,6 +204,12 @@ public BitSet getNulls() {
return nulls != null ? BitSet.valueOf(nulls) : new BitSet();
}
+ /**
+ * Extract subset data to a new column buffer.
+ * @param start start row number
+ * @param end end row number, which is not included in the subset data
+ * @return
+ */
public ColumnBuffer extractSubset(int start, int end) {
ColumnBuffer subset = new ColumnBuffer(type);
subset.currentSize = end - start;
Review comment:
Yes, you're right. I didn't handle the nulls when extractSubset.
I have fixed this and handle some illegal start/end input. I have also added
some [test
cases](https://github.com/apache/incubator-livy/pull/197/files#diff-275dca4a619dd9b0b7f3abbfdfeb00f3R233)
to cover this. Can you please review? Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] EthanDCX opened a new pull request #206: Branch 0.6
EthanDCX opened a new pull request #206: Branch 0.6 URL: https://github.com/apache/incubator-livy/pull/206 how to get progress same as spark webUI This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] TisonKun commented on issue #203: [hotfix][log] log stack trace properly
TisonKun commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-523259917 @zjffdu thanks for your reply. Shall we move forward? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315695134
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
##
@@ -204,6 +204,12 @@ public BitSet getNulls() {
return nulls != null ? BitSet.valueOf(nulls) : new BitSet();
}
+ /**
+ * Extract subset data to a new column buffer.
+ * @param start start row number
+ * @param end end row number, which is not included in the subset data
+ * @return
Review comment:
useless to have this line...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315750553
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
##
@@ -204,6 +204,12 @@ public BitSet getNulls() {
return nulls != null ? BitSet.valueOf(nulls) : new BitSet();
}
+ /**
+ * Extract subset data to a new column buffer.
+ * @param start start row number
+ * @param end end row number, which is not included in the subset data
+ * @return
+ */
public ColumnBuffer extractSubset(int start, int end) {
ColumnBuffer subset = new ColumnBuffer(type);
subset.currentSize = end - start;
Review comment:
I am not sure about this method, expecially regarding the handling of nulls.
Nulls are not inserted in the vaues but the nulls bitset account for them. I am
very surprised that the tests pass with this. I expect worng results with
nulls. May you please check?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315692855
##
File path:
thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
##
@@ -204,6 +204,12 @@ public BitSet getNulls() {
return nulls != null ? BitSet.valueOf(nulls) : new BitSet();
}
+ /**
+ * Extract subset data to a new column buffer.
+ * @param start start row number
+ * @param end end row number, which is not included in the subset data
+ * @return
+ */
Review comment:
I forgot to delete this in another patch... Please review this method...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315691809
##
File path:
thriftserver/server/src/test/scala/org/apache/livy/thriftserver/ThriftServerSuites.scala
##
@@ -17,7 +17,11 @@
package org.apache.livy.thriftserver
-import java.sql.{Connection, Date, SQLException, Statement}
+import java.sql.Connection
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315691910
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/MetadataOperation.scala
##
@@ -42,8 +46,14 @@ abstract class MetadataOperation(sessionHandle:
SessionHandle, opType: Operation
assertState(Seq(OperationState.FINISHED))
validateFetchOrientation(orientation)
if (orientation.equals(FetchOrientation.FETCH_FIRST)) {
- rowSet.setRowOffset(0)
+ offset = 0
+}
+
+if (offset >= rowSet.numRows) {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
yiheng commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315691718
##
File path:
thriftserver/server/src/test/scala/org/apache/livy/thriftserver/ThriftServerSuites.scala
##
@@ -177,6 +181,70 @@ trait CommonThriftTests {
assert(columnsResultSet.getString(23) == "NO")
assert(!columnsResultSet.next())
}
+
+ def getCatalogTest(connection: Connection): Unit = {
+val metadata = connection.getMetaData
+val catalogResultSet = metadata.getCatalogs()
+// Spark doesn't support getCatalog. In current implementation, it's a
no-op and not return
Review comment:
fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] captainzmc commented on a change in pull request #200: [LIVY-637]Fix NullPointerException when create database using thrifts…
captainzmc commented on a change in pull request #200: [LIVY-637]Fix
NullPointerException when create database using thrifts…
URL: https://github.com/apache/incubator-livy/pull/200#discussion_r315686289
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/serde/ThriftResultSet.scala
##
@@ -112,6 +112,7 @@ class ColumnOrientedResultSet(
override def toTRowSet: TRowSet = {
val tRowSet = new TRowSet(rowOffset, new util.ArrayList[TRow])
+tRowSet.setColumns(new util.ArrayList[TColumn]())
Review comment:
OK, now I had added a comment here to explain this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] runzhiwang opened a new pull request #205: [LIVY-628][LIVY-629][LIVY-630][THRIFT][WIP]: Support renewDelegationToken, getDelegationToken, cancelDelegationToken
runzhiwang opened a new pull request #205: [LIVY-628][LIVY-629][LIVY-630][THRIFT][WIP]: Support renewDelegationToken, getDelegationToken, cancelDelegationToken URL: https://github.com/apache/incubator-livy/pull/205 ## What changes were proposed in this pull request? Support renewDelegationToken, getDelegationToken, cancelDelegationToken ## How was this patch tested? we will test it in the near future. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] Jassy1994 opened a new pull request #204: [LIVY-642] Fix a rare status happened in yarn cause SparkApp change into error s…
Jassy1994 opened a new pull request #204: [LIVY-642] Fix a rare status happened in yarn cause SparkApp change into error s… URL: https://github.com/apache/incubator-livy/pull/204 …tate ## What changes were proposed in this pull request? to support this yarn state with YarnApplicationState.RUNNING and FinalApplicationStatus.SUCCEEDED instead return SparkApp.State.FAILED This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315331803
##
File path:
thriftserver/server/src/test/scala/org/apache/livy/thriftserver/ThriftServerSuites.scala
##
@@ -17,7 +17,11 @@
package org.apache.livy.thriftserver
-import java.sql.{Connection, Date, SQLException, Statement}
+import java.sql.Connection
Review comment:
please get back to the previous format for imports
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315575612
##
File path:
thriftserver/server/src/main/scala/org/apache/livy/thriftserver/operation/MetadataOperation.scala
##
@@ -42,8 +46,14 @@ abstract class MetadataOperation(sessionHandle:
SessionHandle, opType: Operation
assertState(Seq(OperationState.FINISHED))
validateFetchOrientation(orientation)
if (orientation.equals(FetchOrientation.FETCH_FIRST)) {
- rowSet.setRowOffset(0)
+ offset = 0
+}
+
+if (offset >= rowSet.numRows) {
Review comment:
I see now the reason of the `extractSubset` you introduced in the other PR.
While in the other case it is not needed, IMHO, here it is the right solution.
May you please take that and use it here? Thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT] Add tests for metadata operations
mgaido91 commented on a change in pull request #197: [LIVY-574][TESTS][THRIFT]
Add tests for metadata operations
URL: https://github.com/apache/incubator-livy/pull/197#discussion_r315331573
##
File path:
thriftserver/server/src/test/scala/org/apache/livy/thriftserver/ThriftServerSuites.scala
##
@@ -177,6 +181,70 @@ trait CommonThriftTests {
assert(columnsResultSet.getString(23) == "NO")
assert(!columnsResultSet.next())
}
+
+ def getCatalogTest(connection: Connection): Unit = {
+val metadata = connection.getMetaData
+val catalogResultSet = metadata.getCatalogs()
+// Spark doesn't support getCatalog. In current implementation, it's a
no-op and not return
Review comment:
```suggestion
// Spark doesn't support getCatalog. In current implementation, it's a
no-op and does not return
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io commented on issue #203: [hotfix][log] log stack trace properly
codecov-io commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-522916828 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=h1) Report > Merging [#203](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/22c73e37fd2c4d93362e576f05ab2a11a7a22714?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #203 +/- ## + Coverage 68.53% 68.54% +0.01% Complexity 911 911 Files 100 100 Lines 5682 5682 Branches856 856 + Hits 3894 3895 +1 + Misses 1232 1228 -4 - Partials556 559 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `75.15% <0%> (ø)` | `20 <0> (ø)` | :arrow_down: | | [...c/main/scala/org/apache/livy/repl/ReplDriver.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9SZXBsRHJpdmVyLnNjYWxh) | `30.76% <0%> (-2.57%)` | `7% <0%> (ø)` | | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (ø)` | `2% <0%> (ø)` | :arrow_down: | | [...ain/java/org/apache/livy/rsc/driver/RSCDriver.java](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvUlNDRHJpdmVyLmphdmE=) | `77.96% <0%> (ø)` | `41% <0%> (ø)` | :arrow_down: | | [...scala/org/apache/livy/repl/SparkRInterpreter.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TcGFya1JJbnRlcnByZXRlci5zY2FsYQ==) | `60.69% <0%> (+0.57%)` | `14% <0%> (ø)` | :arrow_down: | | [...cala/org/apache/livy/repl/ProcessInterpreter.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9Qcm9jZXNzSW50ZXJwcmV0ZXIuc2NhbGE=) | `59.52% <0%> (+2.38%)` | `6% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=footer). Last update [22c73e3...05c5663](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] codecov-io commented on issue #203: [hotfix][log] log stack trace properly
codecov-io commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-522916825 # [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=h1) Report > Merging [#203](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-livy/commit/22c73e37fd2c4d93362e576f05ab2a11a7a22714?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #203 +/- ## + Coverage 68.53% 68.54% +0.01% Complexity 911 911 Files 100 100 Lines 5682 5682 Branches856 856 + Hits 3894 3895 +1 + Misses 1232 1228 -4 - Partials556 559 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `75.15% <0%> (ø)` | `20 <0> (ø)` | :arrow_down: | | [...c/main/scala/org/apache/livy/repl/ReplDriver.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9SZXBsRHJpdmVyLnNjYWxh) | `30.76% <0%> (-2.57%)` | `7% <0%> (ø)` | | | [.../scala/org/apache/livy/sessions/SessionState.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-Y29yZS9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvc2Vzc2lvbnMvU2Vzc2lvblN0YXRlLnNjYWxh) | `61.11% <0%> (ø)` | `2% <0%> (ø)` | :arrow_down: | | [...ain/java/org/apache/livy/rsc/driver/RSCDriver.java](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvUlNDRHJpdmVyLmphdmE=) | `77.96% <0%> (ø)` | `41% <0%> (ø)` | :arrow_down: | | [...scala/org/apache/livy/repl/SparkRInterpreter.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TcGFya1JJbnRlcnByZXRlci5zY2FsYQ==) | `60.69% <0%> (+0.57%)` | `14% <0%> (ø)` | :arrow_down: | | [...cala/org/apache/livy/repl/ProcessInterpreter.scala](https://codecov.io/gh/apache/incubator-livy/pull/203/diff?src=pr=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9Qcm9jZXNzSW50ZXJwcmV0ZXIuc2NhbGE=) | `59.52% <0%> (+2.38%)` | `6% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=footer). Last update [22c73e3...05c5663](https://codecov.io/gh/apache/incubator-livy/pull/203?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] zjffdu commented on issue #203: [hotfix][log] log stack trace properly
zjffdu commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-522908961 LGTM, I think it is fine without creating jira ticket for this trivial change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] TisonKun commented on issue #203: [hotfix][log] log stack trace properly
TisonKun commented on issue #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203#issuecomment-522903218 I can see a corresponding JIRA is necessary but given this is quite a nit change I am not sure whether we also create a dedicated JIRA. Let me know if it is expected. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] TisonKun opened a new pull request #203: [hotfix][log] log stack trace properly
TisonKun opened a new pull request #203: [hotfix][log] log stack trace properly URL: https://github.com/apache/incubator-livy/pull/203 ## What changes were proposed in this pull request? Log stack trace properly. With an automatic check pass this is the only instance. ## How was this patch tested? straightforward This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] shanyu opened a new pull request #187: LIVY-617: Livy session leak on Yarn when creating session duplicated names
shanyu opened a new pull request #187: LIVY-617: Livy session leak on Yarn when creating session duplicated names URL: https://github.com/apache/incubator-livy/pull/187 ## What changes were proposed in this pull request? When creating a session with duplicated name, instead of throw exception in SessionManager.register() method, we should stop the session. Otherwise the session driver process will keep running and end up creating a leaked Yarn application. https://issues.apache.org/jira/browse/LIVY-617 ## How was this patch tested? This is just a simple fix and verified with manual end to end test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #187: LIVY-617: Livy session leak on Yarn when creating session duplicated names
jerryshao closed pull request #187: LIVY-617: Livy session leak on Yarn when creating session duplicated names URL: https://github.com/apache/incubator-livy/pull/187 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-livy] Jassy1994 commented on a change in pull request #152: [LIVY-567] fix SparkApp of InteractiveSession cannot be recovered
Jassy1994 commented on a change in pull request #152: [LIVY-567] fix SparkApp
of InteractiveSession cannot be recovered
URL: https://github.com/apache/incubator-livy/pull/152#discussion_r315547571
##
File path:
server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala
##
@@ -392,8 +392,12 @@ class InteractiveSession(
app = mockApp.orElse {
val driverProcess = client.flatMap { c => Option(c.getDriverProcess) }
.map(new LineBufferedProcess(_,
livyConf.getInt(LivyConf.SPARK_LOGS_SIZE)))
- driverProcess.map { _ => SparkApp.create(appTag, appId, driverProcess,
livyConf, Some(this)) }
+if (livyConf.isRunningOnYarn()) {
Review comment:
Thanks for this comment, I will modify UT instead
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] yiheng commented on issue #172: allow proxy user to view or modify his session
yiheng commented on issue #172: allow proxy user to view or modify his session URL: https://github.com/apache/incubator-livy/pull/172#issuecomment-522876165 Just found this patch is duplicate with this one https://github.com/apache/incubator-livy/pull/171 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
