[jira] [Created] (HIVE-19888) Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState

2018-06-13 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created HIVE-19888: - Summary: Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState Key: HIVE-19888 URL: https://issues.apache.org/jira/browse/HIVE-19888

Re: Review Request 33422: HIVE-10434 - Cancel connection when remote Spark driver process has failed [Spark Branch]

2015-04-24 Thread Marcelo Vanzin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33422/#review81520 --- Ship it! Ship It! - Marcelo Vanzin On April 23, 2015, 6:54 p.m

Re: Review Request 33422: HIVE-10434 - Cancel connection when remote Spark driver process has failed [Spark Branch]

2015-04-23 Thread Marcelo Vanzin
--- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33422/ > --- > > (Updated April 23, 2015, 6:11 p.m.) > > > Review request for hive and Marc

Re: Review Request 33422: HIVE-10434 - Cancel connection when remote Spark driver process has failed [Spark Branch]

2015-04-23 Thread Marcelo Vanzin
/java/org/apache/hive/spark/client/rpc/RpcServer.java <https://reviews.apache.org/r/33422/#comment131664> To avoid races, I'd do: final ClientInfo cinfo = pendingClients.remove(clientId); if (cinfo == null) { /* nothing to do */ } - Marcelo Vanzin On April 22,

Re: Review Request 33422: HIVE-10434 - Cancel connection when remote Spark driver process has failed [Spark Branch]

2015-04-21 Thread Marcelo Vanzin
java <https://reviews.apache.org/r/33422/#comment131351> While the only current call site reflects the error message, this method seems more generic than that. Maybe pass the error message as a parameter to the method? - Marcelo Vanzin On April 22, 2015, 12:30 a.m

Review Request 32631: [HIVE-10143] Properly clean up client state when client times out.

2015-03-30 Thread Marcelo Vanzin
/hive/spark/client/rpc/TestRpc.java 8207514342bed544e1a01fc41c892825f330cf3c Diff: https://reviews.apache.org/r/32631/diff/ Testing --- Thanks, Marcelo Vanzin

[jira] [Created] (HIVE-10143) HS2 fails to clean up Spark client state on timeout

2015-03-30 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created HIVE-10143: - Summary: HS2 fails to clean up Spark client state on timeout Key: HIVE-10143 URL: https://issues.apache.org/jira/browse/HIVE-10143 Project: Hive Issue

[jira] [Commented] (HIVE-9410) ClassNotFoundException occurs during hive query case execution with UDF defined [Spark Branch]

2015-02-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305668#comment-14305668 ] Marcelo Vanzin commented on HIVE-9410: -- Ah, I see, didn't notice that. T

[jira] [Commented] (HIVE-9410) ClassNotFoundException occurs during hive query case execution with UDF defined [Spark Branch]

2015-02-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303659#comment-14303659 ] Marcelo Vanzin commented on HIVE-9410: -- Hi [~chengxiang li], I think this patch

[jira] [Updated] (HIVE-9487) Make Remote Spark Context secure [Spark Branch]

2015-01-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9487: - Attachment: HIVE-9487.2-spark.patch > Make Remote Spark Context secure [Spark Bra

[jira] [Commented] (HIVE-9487) Make Remote Spark Context secure [Spark Branch]

2015-01-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297457#comment-14297457 ] Marcelo Vanzin commented on HIVE-9487: -- I failed "git branch management

[jira] [Commented] (HIVE-9487) Make Remote Spark Context secure [Spark Branch]

2015-01-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297200#comment-14297200 ] Marcelo Vanzin commented on HIVE-9487: -- Hmm, weird. I definitely did not touch t

Re: Review Request 30385: Use SASL to establish the remote context connection.

2015-01-28 Thread Marcelo Vanzin
il. To reply, visit: https://reviews.apache.org/r/30385/#review70119 ------- On Jan. 28, 2015, 11:22 p.m., Marcelo Vanzin wrote: > > --- > This is an automatically generat

Review Request 30385: Use SASL to establish the remote context connection.

2015-01-28 Thread Marcelo Vanzin
d3e5d4f9fdf8ca9 Diff: https://reviews.apache.org/r/30385/diff/ Testing --- Unit tests. Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9487) Make Remote Spark Context secure [Spark Branch]

2015-01-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9487: - Status: Patch Available (was: Open) > Make Remote Spark Context secure [Spark Bra

[jira] [Updated] (HIVE-9487) Make Remote Spark Context secure [Spark Branch]

2015-01-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9487: - Attachment: HIVE-9487.1-spark.patch > Make Remote Spark Context secure [Spark Bra

[jira] [Commented] (HIVE-9493) Failed job may not throw exceptions [Spark Branch]

2015-01-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14295544#comment-14295544 ] Marcelo Vanzin commented on HIVE-9493: -- Looks good, thanks for fixing it. >

[jira] [Created] (HIVE-9487) Make Remote Spark Context secure [Spark Branch]

2015-01-27 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created HIVE-9487: Summary: Make Remote Spark Context secure [Spark Branch] Key: HIVE-9487 URL: https://issues.apache.org/jira/browse/HIVE-9487 Project: Hive Issue Type: Sub

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-20 Thread Marcelo Vanzin
sparkJobIds.add(sparkJobId) seems irrelevant > > to any changes to listeners, unless I missed anything. I don't understand > > why either of the two cases might happen as you suggested. > > Marcelo Vanzin wrote: > Threads: T1 updating the job handle, T

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-16 Thread Marcelo Vanzin
m pretty easy to understand to me. The current code avoids both of them. - Marcelo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/#review68513 -----------

[jira] [Updated] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]

2015-01-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9179: - Attachment: HIVE-9179.3-spark.patch > Add listeners on JobHandle so job status change can

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-16 Thread Marcelo Vanzin
/java/org/apache/hive/spark/client/TestJobHandle.java PRE-CREATION spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29954/diff/ Testing --- Thanks, Marcelo Vanzin

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-16 Thread Marcelo Vanzin
bIds.add() is in the synchronized block. However, we have > > code accessing the same variable (sparkJobIds) such as in > > RemoteSparkJobStatus class. Does that also needs protection? > > Marcelo Vanzin wrote: > No, we don't. The job id list itself is thread-safe. The s

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-16 Thread Marcelo Vanzin
fter the handle is in a final state. Stay tuned.) - Marcelo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/#review68492 ----------- On Jan. 16, 2015, 9:22 p.m., Marcelo Vanzin wrote: > >

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-16 Thread Marcelo Vanzin
/org/apache/hive/spark/client/TestJobHandle.java PRE-CREATION spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29954/diff/ Testing --- Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]

2015-01-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9179: - Attachment: HIVE-9179.2-spark.patch > Add listeners on JobHandle so job status change can

Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-16 Thread Marcelo Vanzin
-- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29954/#review68430 --- On Jan. 16, 2015, 1:05 a.m., Marcelo Vanzin wrote: > >

[jira] [Commented] (HIVE-9370) Enable Hive on Spark for BigBench and run Query 8, the test failed [Spark Branch]

2015-01-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14280595#comment-14280595 ] Marcelo Vanzin commented on HIVE-9370: -- For the curious: SPARK-1021. HIVE-9179 sh

Review Request 29954: HIVE-9179. Add listener API to JobHandle.

2015-01-15 Thread Marcelo Vanzin
/TestJobHandle.java PRE-CREATION spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29954/diff/ Testing --- Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]

2015-01-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9179: - Status: Patch Available (was: Open) > Add listeners on JobHandle so job status change can

[jira] [Updated] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]

2015-01-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9179: - Attachment: HIVE-9179.1-spark.patch > Add listeners on JobHandle so job status change can

Re: Review Request 29832: HIVE-9178. Add a synchronous RPC API to the remote Spark context.

2015-01-14 Thread Marcelo Vanzin
/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29832/diff/ Testing --- Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9178: - Attachment: HIVE-9178.2-spark.patch > Create a separate API for remote Spark Context RPC ot

Re: Review Request 29832: HIVE-9178. Add a synchronous RPC API to the remote Spark context.

2015-01-14 Thread Marcelo Vanzin
/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab Diff: https://reviews.apache.org/r/29832/diff/ Testing --- Thanks, Marcelo Vanzin

[jira] [Commented] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276514#comment-14276514 ] Marcelo Vanzin commented on HIVE-9178: -- [~chengxiang li] ah, good catch. This me

[jira] [Commented] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276297#comment-14276297 ] Marcelo Vanzin commented on HIVE-9178: -- I'll take a closer look at the c

[jira] [Commented] (HIVE-9364) Document non-Public API's which Spark SQL uses

2015-01-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276100#comment-14276100 ] Marcelo Vanzin commented on HIVE-9364: -- Here are three APIs that changed between

[jira] [Commented] (HIVE-9360) TestSparkClient throws Timeoutexception

2015-01-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275783#comment-14275783 ] Marcelo Vanzin commented on HIVE-9360: -- Yeah, I dislike timeouts in tests but in

[jira] [Commented] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275735#comment-14275735 ] Marcelo Vanzin commented on HIVE-9178: -- Should I worry about those test failure

Re: Review Request 29832: HIVE-9178. Add a synchronous RPC API to the remote Spark context.

2015-01-13 Thread Marcelo Vanzin
nerated e-mail. To reply, visit: https://reviews.apache.org/r/29832/#review67813 ----------- On Jan. 13, 2015, 12:31 a.m., Marcelo Vanzin wrote: > > --- > This is an automatically g

Review Request 29832: HIVE-9178. Add a synchronous RPC API to the remote Spark context.

2015-01-12 Thread Marcelo Vanzin
/hive/spark/client/SparkClient.java 5e767ef5eb47e493a332607204f4c522028d7d0e spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java f8b2202a465bb8abe3d2c34e49ade6387482738c Diff: https://reviews.apache.org/r/29832/diff/ Testing --- Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9178: - Status: Patch Available (was: Open) > Create a separate API for remote Spark Context RPC ot

[jira] [Updated] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9178: - Attachment: HIVE-9178.1-spark.patch > Create a separate API for remote Spark Context RPC ot

[jira] [Assigned] (HIVE-9178) Create a separate API for remote Spark Context RPC other than job submission [Spark Branch]

2015-01-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned HIVE-9178: Assignee: Marcelo Vanzin > Create a separate API for remote Spark Context RPC other t

[jira] [Assigned] (HIVE-9179) Add listeners on JobHandle so job status change can be notified to the client [Spark Branch]

2015-01-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned HIVE-9179: Assignee: Marcelo Vanzin > Add listeners on JobHandle so job status change can be notif

[jira] [Commented] (HIVE-8972) Implement more fine-grained remote client-level events [Spark Branch]

2014-12-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14250353#comment-14250353 ] Marcelo Vanzin commented on HIVE-8972: -- The patch looks ok to me. I though a

Re: Review Request 29145: HIVE-9094 TimeoutException when trying get executor count from RSC [Spark Branch]

2014-12-17 Thread Marcelo Vanzin
very generic, since it's only applied to a couple of jobs submitted to the client. But I don't have a good suggestion here. - Marcelo Vanzin On Dec. 17, 2014, 6:28 a.m., chengxiang li wrote: > > --- > This is an automa

[jira] [Commented] (HIVE-9094) TimeoutException when trying get executor count from RSC [Spark Branch]

2014-12-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14248560#comment-14248560 ] Marcelo Vanzin commented on HIVE-9094: -- 60s sounds reasonable. This initial tim

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245051#comment-14245051 ] Marcelo Vanzin commented on HIVE-9017: -- In Spark-speak, "executor" i

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245029#comment-14245029 ] Marcelo Vanzin commented on HIVE-9017: -- I filed https://issues.apache.org/jira/br

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244992#comment-14244992 ] Marcelo Vanzin commented on HIVE-9017: -- See https://github.com/apache/spark/co

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244990#comment-14244990 ] Marcelo Vanzin commented on HIVE-9017: -- You can't do that, because these

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244951#comment-14244951 ] Marcelo Vanzin commented on HIVE-9017: -- Correct. All files written by spark will

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244948#comment-14244948 ] Marcelo Vanzin commented on HIVE-9017: -- P.S.: that solution will probably not

[jira] [Commented] (HIVE-9017) Clean up temp files of RSC [Spark Branch]

2014-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244944#comment-14244944 ] Marcelo Vanzin commented on HIVE-9017: -- These files are created by Spark

[jira] [Commented] (HIVE-9085) Spark Client RPC should have larger default max message size [Spark Branch]

2014-12-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243535#comment-14243535 ] Marcelo Vanzin commented on HIVE-9085: -- When an exception is thrown in the write

[jira] [Commented] (HIVE-9085) Spark Client RPC should have larger default max message size [Spark Branch]

2014-12-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243516#comment-14243516 ] Marcelo Vanzin commented on HIVE-9085: -- LGTM (as discussed by e-mail). >

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-09 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.6-spark.patch > Replace akka for remote spark client RPC [Spark Bra

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-09 Thread Marcelo Vanzin
ally generated e-mail. To reply, visit: https://reviews.apache.org/r/28779/#review64411 ------- On Dec. 9, 2014, 6:49 p.m., Marcelo Vanzin wrote: > > --- > This is an au

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-09 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.5-spark.patch > Replace akka for remote spark client RPC [Spark Bra

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.4-spark.patch > Replace akka for remote spark client RPC [Spark Bra

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

[jira] [Commented] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238670#comment-14238670 ] Marcelo Vanzin commented on HIVE-9036: -- I have a live job in that state, shoul

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
odec.checkSize(KryoMessageCodec.java:117) > > at > > org.apache.hive.spark.client.rpc.KryoMessageCodec.decode(KryoMessageCodec.java:77) > > at > > io.netty.handler.codec.ByteToMessageCodec$1.decode(ByteToMessageCodec.java:42) > > at > > io.netty.handler.codec.B

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.3-spark.patch > Replace akka for remote spark client RPC [Spark Bra

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
e connection to be established; adding a header will add nothing to the process, just complexity. - Marcelo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28779/#review64279 -

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Status: Patch Available (was: Open) > Replace akka for remote spark client RPC [Spark Bra

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Status: Open (was: Patch Available) > Replace akka for remote spark client RPC [Spark Bra

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.2-spark.patch > Replace akka for remote spark client RPC [Spark Bra

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: (was: HIVE-9036.2-spark.patch) > Replace akka for remote spark client RPC [Sp

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
https://reviews.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.2-spark.patch > Replace akka for remote spark client RPC [Spark Bra

Re: Review Request 28779: [spark-client] Netty-based RPC implementation.

2014-12-08 Thread Marcelo Vanzin
.apache.org/r/28779/diff/ Testing --- spark-client unit tests, plus some qtests. Thanks, Marcelo Vanzin

[jira] [Commented] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236447#comment-14236447 ] Marcelo Vanzin commented on HIVE-9036: -- I'll look at why the patch isn&#

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Status: Patch Available (was: Open) Patch is rather large but shouldn't be too complicated

[jira] [Updated] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-9036: - Attachment: HIVE-9036.1-spark.patch > Replace akka for remote spark client RPC [Spark Bra

[jira] [Created] (HIVE-9036) Replace akka for remote spark client RPC [Spark Branch]

2014-12-05 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created HIVE-9036: Summary: Replace akka for remote spark client RPC [Spark Branch] Key: HIVE-9036 URL: https://issues.apache.org/jira/browse/HIVE-9036 Project: Hive Issue

[jira] [Resolved] (HIVE-8574) Enhance metrics gathering in Spark Client [Spark Branch]

2014-12-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved HIVE-8574. -- Resolution: Not a Problem I'll close this as "not a problem" for now. If we deci

[jira] [Commented] (HIVE-8991) Fix custom_input_output_format [Spark Branch]

2014-12-02 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231866#comment-14231866 ] Marcelo Vanzin commented on HIVE-8991: -- I didn't mean to stop you guys from

[jira] [Commented] (HIVE-8957) Remote spark context needs to clean up itself in case of connection timeout [Spark Branch]

2014-12-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230478#comment-14230478 ] Marcelo Vanzin commented on HIVE-8957: -- If you don't mind the bug

[jira] [Commented] (HIVE-8995) Find thread leak in RSC Tests

2014-12-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230273#comment-14230273 ] Marcelo Vanzin commented on HIVE-8995: -- You don't need to call that method

[jira] [Commented] (HIVE-8995) Find thread leak in RSC Tests

2014-12-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230254#comment-14230254 ] Marcelo Vanzin commented on HIVE-8995: -- The three threads are from akka; I wonde

[jira] [Commented] (HIVE-8991) Fix custom_input_output_format [Spark Branch]

2014-12-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230227#comment-14230227 ] Marcelo Vanzin commented on HIVE-8991: -- Hi [~lirui], the patch looks good i

[jira] [Commented] (HIVE-8574) Enhance metrics gathering in Spark Client [Spark Branch]

2014-11-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226668#comment-14226668 ] Marcelo Vanzin commented on HIVE-8574: -- Rounding up, each task metrics data struc

[jira] [Commented] (HIVE-8574) Enhance metrics gathering in Spark Client [Spark Branch]

2014-11-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226517#comment-14226517 ] Marcelo Vanzin commented on HIVE-8574: -- Actually, after a quick look at the

[jira] [Commented] (HIVE-8957) Remote spark context needs to clean up itself in case of connection timeout [Spark Branch]

2014-11-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226515#comment-14226515 ] Marcelo Vanzin commented on HIVE-8957: -- I think a fix here will be a little

[jira] [Commented] (HIVE-8956) Hive hangs while some error/exception happens beyond job execution [Spark Branch]

2014-11-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226504#comment-14226504 ] Marcelo Vanzin commented on HIVE-8956: -- I haven't looked at akka in that mu

[jira] [Commented] (HIVE-8574) Enhance metrics gathering in Spark Client [Spark Branch]

2014-11-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225276#comment-14225276 ] Marcelo Vanzin commented on HIVE-8574: -- Hey [~chengxiang li], I'd like

[jira] [Commented] (HIVE-8956) Hive hangs while some error/exception happens beyond job execution[Spark Branch]

2014-11-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225082#comment-14225082 ] Marcelo Vanzin commented on HIVE-8956: -- This is ok if it unblocks something right

[jira] [Commented] (HIVE-8836) Enable automatic tests with remote spark client.[Spark Branch]

2014-11-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225049#comment-14225049 ] Marcelo Vanzin commented on HIVE-8836: -- I talked briefly with Brock about this,

[jira] [Commented] (HIVE-8951) Spark remote context doesn't work with local-cluster [Spark Branch]

2014-11-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224022#comment-14224022 ] Marcelo Vanzin commented on HIVE-8951: -- Xuefu, increasing the timeout should be

[jira] [Commented] (HIVE-8951) Spark remote context doesn't work with local-cluster [Spark Branch]

2014-11-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223661#comment-14223661 ] Marcelo Vanzin commented on HIVE-8951: -- Not from just those logs. Is this ea

[jira] [Commented] (HIVE-8951) Spark remote context doesn't work with local-cluster [Spark Branch]

2014-11-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223507#comment-14223507 ] Marcelo Vanzin commented on HIVE-8951: -- That `BindException` should not be f

[jira] [Commented] (HIVE-8951) Spark remote context doesn't work with local-cluster [Spark Branch]

2014-11-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223388#comment-14223388 ] Marcelo Vanzin commented on HIVE-8951: -- `SparkClientImpl` has `stop()`, which sh

  1   2   >