subject:"\\\[GitHub\\\] spark issue #14961\\\: \\\[SPARK\\\-17379\\\] \\\[BUILD\\\] Upgrade netty\\\-all to 4.0.41 final ..."

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
Confirmed the issue was introduced by 
https://github.com/netty/netty/commit/d58dec8862e02fc2a98f8dcdb166db4b788be50a#diff-8d83d75ebf8a18cc48bf0a0b1183c188

Add `System.setProperty("io.netty.maxDirectMemory", "0");` to disable this 
feature then the tests pass.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
Hm, I can reproduce the same error using this command `build/sbt "project 
core" "test-only *Shuffle*"` locally. The first broken version is 4.0.37.Final.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
Oh, the allocator is set here: 
https://github.com/apache/spark/blob/master/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java#L95


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
@a-roberts could you binary search the first broken netty version? Since 
this cannot be reproduced locally, you have to push new commits.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
Still saw the following errors in the unit-test log:

```
16/09/13 07:41:18.817 shuffle-server-466-7 WARN TransportChannelHandler: 
Exception in connection from /127.0.0.1:36871
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 
byte(s) of direct memory (used: 3120562176, max: 3132096512)
at 
io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:528)
at 
io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:482)
at 
io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:688)
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:677)
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:237)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:213)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:141)
at 
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271)
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:177)
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:168)
at 
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:129)
at 
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
at 
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
at 
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:611)
at 
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552)
at 
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at 
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
```

Seems the error cannot be reported correctly in Jenkins.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-13 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14961
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65322/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #65322 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65322/consoleFull)**
 for PR 14961 at commit 
[`faefd9c`](https://github.com/apache/spark/commit/faefd9ca2975830b6afff0ed78ac5e90c69ec9f4).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-13 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14961
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #65322 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65322/consoleFull)**
 for PR 14961 at commit 
[`faefd9c`](https://github.com/apache/spark/commit/faefd9ca2975830b6afff0ed78ac5e90c69ec9f4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-13 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
Had a look to see how to do this


https://github.com/netty/netty/blob/a01519e4f86690323647b5db45d9ffcb184b1a84/buffer/src/main/java/io/netty/buffer/ByteBufUtil.java

so I'll add io.netty.allocator.type to be unpooled, will make this change 
in TransportConf again (removing the recycler experiment)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-13 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
Yep that makes more sense, UnpooledByteBufAllocator usage coming up


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-13 Thread srowen

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
OK, so same failure with this change. Hm. I don't think it's that something 
is just slow but that the error in 
https://github.com/apache/spark/pull/14961#issuecomment-245090209 causes netty 
to never start at all.

A next step would be to try modifying the code to use the Netty unpooled 
buffer? just to see if that avoids the issue? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-13 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
[info] - using external shuffle service *** FAILED *** (1 minute)
[info]   java.util.concurrent.TimeoutException: Can't find 2 executors 
before 6 milliseconds elapsed

60 seconds really is an eternity, I can't reproduce this on my local set 
up, I expect we've got deadlock going on after the upgrade and would require 
some proper debugging (again, if only I could reproduce it on my test systems 
with access to tools like gdb/healthcenter/servicing APIs we use here). My 
systems have between two and eight cores and I know this farm has a lot more 
available...could be that having more cores increases the chances of thread 
contention.

I had a look at other pull requests being tested and see it typically 
completes in 3 seconds on a good run
using external shuffle service (3 seconds, 822 milliseconds)
at 
https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3258/consoleText

using external shuffle service (4 seconds, 543 milliseconds)

https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3233/consoleText


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3256 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3256/consoleFull)**
 for PR 14961 at commit 
[`502ebf4`](https://github.com/apache/spark/commit/502ebf45f4fa9791cbf26ec5ea7e0167ecbc68a0).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3256 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3256/consoleFull)**
 for PR 14961 at commit 
[`502ebf4`](https://github.com/apache/spark/commit/502ebf45f4fa9791cbf26ec5ea7e0167ecbc68a0).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14961
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14961
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65264/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #65264 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65264/consoleFull)**
 for PR 14961 at commit 
[`502ebf4`](https://github.com/apache/spark/commit/502ebf45f4fa9791cbf26ec5ea7e0167ecbc68a0).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #65264 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65264/consoleFull)**
 for PR 14961 at commit 
[`502ebf4`](https://github.com/apache/spark/commit/502ebf45f4fa9791cbf26ec5ea7e0167ecbc68a0).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
No new test failures with my runs ranging from Hadoop 2.3 to Hadoop 2.7 
today so pushed the commit above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
Sean, yep, I've had trouble reproducing it too, kicked off a bunch of 
builds over the weekend including one using Hadoop-2.3 which was my initial 
theory (only difference between our testing environments apart from the options 
I mention below)

I'll add
```
  static {
System.setProperty("io.netty.recycler.maxCapacity", "0");
  }
```
in TransportConf then build and test locally before updating this.

FWIW I use these Java options for testing as our boxes have limited memory:

-Xss2048k **-Dspark.buffer.pageSize=1048576** -Xmx4g


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-10 Thread srowen

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
@a-roberts are you in a position to add this change to this PR as an 
experiment? I can try it on the side too. I can't seem to reproduce the failure 
locally, even when fully rebuilding the project with a newer netty.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-07 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
In addition, I think we should figure out why upgrading the netty version 
will fail. The issue about Recycler seems also in `4.0.29.Final`. Is it because 
netty starts to track the memoryprint since 
https://github.com/netty/netty/commit/62655c00a93a3c4c9e421fabbf5991c19dfb4161#diff-8d83d75ebf8a18cc48bf0a0b1183c188
 or there is a new issue introduced by the new netty version?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-07 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
> I suppose one hacky way to test the theory above is to push a commit here 
that sets this in NettyUtils:

Let's add it in `TransportConf` so that it's easy to find since it's the 
place of the network configs.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-07 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
I'm not familiar with netty's Recycler. But the default value of 
`io.netty.recycler.maxCapacity` is 262144. This seems too big for Spark anyway. 
I don't think we need to cache 260k objects.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-07 Thread srowen

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961

Hm, I can't get this test to fail with Netty 4.0.41 when I 'mvn install'
and run the test suite locally. I'm having a hard time seeing what could
alleviate the failure.

I suspect that this change might be relevant:
https://github.com/netty/netty/commit/1a9ea2d349c42f7c7ccb56daea29f083ee60870e#diff-23eafd00fcd66829f8cce343b26c236a

In any event, we ideally want to fix-forward and find a way to work with
the current version, unless we have reason to believe it's an unresolved bug
(and we don't have a pressing bug fix we need to ingest).

I suppose one hacky way to test the theory above is to push a commit here
that sets this in `NettyUtils`:

```
static {
System.setProperty("io.netty.recycler.maxCapacity", "0");
}
```

... or post a change that tries the `UnpooledByteBufAllocator` to see how
that behaves.

It may not be worth it if it's hard to get this working, but, if it's _not_
really a Netty bug but a behavior change, we'll eventually have to sort this
out.

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
I think we can binary search the first broken netty version. It would be 
easy to find out the real issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
> Is the lesson here to not bother with pooling and use the 
UnpooledByteBufAllocator?

Not sure. Pooling is for improving the performance because allocating 
direct buffers is pretty slow.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
Aha, possibly this:
https://groups.google.com/forum/#!topic/netty/3BoF7q34Z4I

Is the lesson here to not bother with pooling and use the 
UnpooledByteBufAllocator?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/14961
  
I saw the error in the log:
```
16/09/05 08:21:56.758 shuffle-server-593-8 WARN TransportChannelHandler: 
Exception in connection from /127.0.0.1:44788
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 
byte(s) of direct memory (used: 3120562176, max: 3132096512)
at 
io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:528)
at 
io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:482)
at 
io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:688)
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:677)
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:237)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:213)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:141)
at 
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271)
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:177)
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:168)
at 
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:129)
at 
io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
at 
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
at 
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:611)
at 
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552)
at 
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at 
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
```
Seems some configuration in netty has been changed and caused the failure?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
Hm, no I take it back, it's a consistent failure that doesn't show up in 
the main test builds (for any Hadoop version):

```
[info] - using external shuffle service *** FAILED *** (1 minute, 33 
seconds)
[info]   org.apache.spark.SparkException: Job aborted due to stage failure: 
Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 
0.0 (TID 11, localhost): java.io.IOException: 
org.apache.spark.storage.BlockFetchException: Failed to fetch block after 1 
fetch failures. Most recent failure cause:
[info]  at 
org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1260)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:174)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:65)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:65)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:89)
[info]  at 
org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
[info]  at 
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:72)
[info]  at 
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
[info]  at org.apache.spark.scheduler.Task.run(Task.scala:86)
[info]  at 
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
[info]  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[info]  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[info]  at java.lang.Thread.run(Thread.java:745)
[info] Caused by: org.apache.spark.storage.BlockFetchException: Failed to 
fetch block after 1 fetch failures. Most recent failure cause:
[info]  at 
org.apache.spark.storage.BlockManager.getRemoteBytes(BlockManager.scala:567)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1.apply$mcVI$sp(TorrentBroadcast.scala:136)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1.apply(TorrentBroadcast.scala:125)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$org$apache$spark$broadcast$TorrentBroadcast$$readBlocks$1.apply(TorrentBroadcast.scala:125)
[info]  at scala.collection.immutable.List.foreach(List.scala:381)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast.org$apache$spark$broadcast$TorrentBroadcast$$readBlocks(TorrentBroadcast.scala:125)
[info]  at 
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:186)
[info]  at 
org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1253)
[info]  ... 12 more
[info] Caused by: org.apache.spark.SparkException: Exception thrown in 
awaitResult: 
[info]  at 
org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:194)
[info]  at 
org.apache.spark.network.BlockTransferService.fetchBlockSync(BlockTransferService.scala:104)
[info]  at 
org.apache.spark.storage.BlockManager.getRemoteBytes(BlockManager.scala:556)
[info]  ... 19 more
[info] Caused by: java.io.IOException: Connection reset by peer
[info]  at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
[info]  at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
[info]  at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
[info]  at sun.nio.ch.IOUtil.read(IOUtil.java:192)
[info]  at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
[info]  at 
io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221)
[info]  at 
io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899)
[info]  at 
io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:275)
[info]  at 
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
[info]  at 
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:611)
[info]  at 
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552)
[info]  at 
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466)
[info]  at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438)
[info]  at 
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
[info]  at 
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
[info]  ... 1 more
[info] 
```

The rest are

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3249 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3249/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
Thanks, I did a ctrl-f for "** fail", you'd have a better idea of what the 
known flakies are in this farm though, my quick checking:

- using external shuffle service -> looks to be a timeout

These six mention a stopped Spark context (perhaps the previous test 
doesn't clean up properly and causes this):
- kryo objects are serialised consistently in different processes *** 
FAILED *** (1 minute) -> never seen
- cluster mode, FIFO scheduler *** FAILED *** (1 minute) -> never seen
- cluster mode, fair scheduler *** FAILED *** -> never seen
- verify that correct log urls get propagated from workers *** FAILED *** 
(1 minute) -> never seen
- verify that log urls reflect SPARK_PUBLIC_DNS (SPARK-6175) *** FAILED *** 
(1 minute) -> never seen
- task throws not serializable exception *** FAILED *** (1 minute) -> never 
seen

- simple groupByKey *** FAILED *** (1 minute) [info]   
java.lang.NullPointerException:
[info]   at org.apache.spark.SparkContext.(SparkContext.scala:552) > 
different causes now, never seen

- onTaskGettingResult() called when result fetched remotely *** FAILED *** 
(135 milliseconds) -> never seen

- set spark.sql.warehouse.dir *** FAILED *** (5 minutes, 0 seconds) -> 
never seen
- set hive.metastore.warehouse.dir *** FAILED *** (10 seconds, 552 
milliseconds) -> never seen


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
Hm, I see just one in the PR builder here, really. And it's different from 
run to run so this could well be spurious. Re-running tests one more time here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3249 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3249/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
In the description I mentioned that for testing I used "Existing unit tests 
against branch-1.6 and branch-2.0 using IBM Java 8 on Intel, Power and Z 
architectures", so clarifying that I only used Hadoop 2.6 and Hadoop 2.7, but I 
see in the pull request builder we use Hadoop 2.3 and a dozen failures happened


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
Are you saying thousands of tests fail with certain Hadoop versions and 
this version change? That's hard to believe. I'd be very surprised if this 
caused a test failure. However I do see this PR fail a Hive test, and that's 
failed twice. It could still be unrelated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread a-roberts

Github user a-roberts commented on the issue:

https://github.com/apache/spark/pull/14961
  
Thanks, so are we saying netty 4.0.29 can't be upgraded to 4.0.41 without 
breaking changes? That's not even a minor version change...

On branch 1.6 with the netty change for myself I see 8477 tests, two 
failures (flaky network events and DateTimeUtilsSuite to UTC timestamp, 
unrelated to this)

And against master I see 11,148 tests with unrelated failures again 
(furtherRequestsDelay, hive metastore warehouse dir, executor allocation 
manager basic functionality, replsuite clone and clean line object)

I'm using two maven commands to first build and then run
```
mvn -T 1C ${R_PROFILE} -Pyarn -Phadoop-${HADOOP_VERSION} -Phive 
-Phive-thriftserver -DskipTests -Dscala-$SCALA_VERSION clean package
```

```
mvn -Pyarn -Phadoop-${HADOOP_VERSION} -Phive -Phive-thriftserver 
-Dscala-$SCALA_VERSION -Dtest.exclude.tags=org.apache.spark.tags.DockerTest 
${TESTS_RUN_OPTIONS} -fn test
```

In this case the profiles used are for **Hadoop 2.6**, Scala 2.10 on 
branch-1.6 then **Hadoop 2.7** and Scala 2.11 for branch-2.0, no additional 
test run options. Therefore I think it's all about the Hadoop version we use 
because in the community job I see:
```
**-Phadoop-2.3** -Phive -Pyarn -Pmesos -Phive-thriftserver -Pkinesis-asl 
-Dtest.exclude.tags=org.apache.spark.tags.ExtendedHiveTest,org.apache.spark.tags.ExtendedYarnTest
 test
```

If this is the case then surely we actually should not upgrade the version 
of Netty until we either drop support for Hadoop 2.3 and below (and perhaps we 
see the problem in 2.4 too) or make the necessary changes in our Spark codebase 
to address issues seen in the above jobs using Hadoop 2.3.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
@jerryshao that's a good point though in theory a maintenance release 
contains no API or behavior changes (that aren't bugs). Let's perhaps not touch 
1.6 then to be conservative. Hadoop uses a different major release of Netty. I 
agree with caution though we also need to be able to get some bug fixes in 
where necessary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-05 Thread jerryshao

Github user jerryshao commented on the issue:

https://github.com/apache/spark/pull/14961
  
Also many other downstream and upstream applications may also use different 
version of Netty jar, it would be better to keep stable for these fundamental 
dependences.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-05 Thread jerryshao

Github user jerryshao commented on the issue:

https://github.com/apache/spark/pull/14961
  
Upgrading Netty version to branch 1.6 may cause API version incompatible 
issue for yarn shuffle service, please see 
[SPARK-16018](https://issues.apache.org/jira/browse/SPARK-16018) and 
[SPARK-15178](https://issues.apache.org/jira/browse/SPARK-15178). We also need 
to shade netty if we want to backport to 1.6.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3247 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3247/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3247 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3247/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3246 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3246/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #3246 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3246/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-05 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14961
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-05 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14961
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64938/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14961
  
**[Test build #64938 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64938/consoleFull)**
 for PR 14961 at commit 
[`38ca07b`](https://github.com/apache/spark/commit/38ca07b1f65d4d16d8ccf0a5cfbc570c2c7882e1).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-05 Thread srowen

Github user srowen commented on the issue:

https://github.com/apache/spark/pull/14961
  
Looks good for master to 1.6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...