[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225960#comment-14225960
]
zzc commented on SPARK-2468:
AH, Aaron Davidson, With patch #3465, I can run successful
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225681#comment-14225681
]
Aaron Davidson commented on SPARK-2468:
---
Hey guys, I finally got a chance to run a
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225773#comment-14225773
]
Lianhui Wang commented on SPARK-2468:
-
[~adav] that 's very great. With patch #3465, I
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225778#comment-14225778
]
Reynold Xin commented on SPARK-2468:
Glad that we are able to resolve this!
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225790#comment-14225790
]
zzc commented on SPARK-2468:
Great. I will test it later.
Netty-based block server / client
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212064#comment-14212064
]
zzc commented on SPARK-2468:
Hi, Aaron Davidson, I send a email to you about shuffle data
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14209477#comment-14209477
]
zzc commented on SPARK-2468:
Aaron Davidson, thank you for your reply. I will try it again.
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14209489#comment-14209489
]
zzc commented on SPARK-2468:
Aaron Davidson, I find that #3155 has merged into master, but
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14210074#comment-14210074
]
Aaron Davidson commented on SPARK-2468:
---
Here is my spark configuration (note 32
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207826#comment-14207826
]
zzc commented on SPARK-2468:
Hi, Aaron Davidson, I am sure that I ran my last test with the
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207839#comment-14207839
]
zzc commented on SPARK-2468:
Hi, Aaron Davidson, can you describe your test, including the
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14209407#comment-14209407
]
Aaron Davidson commented on SPARK-2468:
---
My test was significantly less strict in
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207774#comment-14207774
]
Aaron Davidson commented on SPARK-2468:
---
[~zzcclp] Ah, man, that's not good. Just to
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14204222#comment-14204222
]
zzc commented on SPARK-2468:
@Aaron Davidson, Thank you for the suggestion.
The configuration
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14203316#comment-14203316
]
Lianhui Wang commented on SPARK-2468:
-
ok, thanks.[~adav] i will try to do as you say.
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201765#comment-14201765
]
zzc commented on SPARK-2468:
@Lianhui Wang, How to view the associated logs with yarn still
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201766#comment-14201766
]
Aaron Davidson commented on SPARK-2468:
---
[~zzcclp] Yes, please do. What's the memory
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201772#comment-14201772
]
zzc commented on SPARK-2468:
aa...@databricks.com?
Netty-based block server / client module
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201774#comment-14201774
]
Aaron Davidson commented on SPARK-2468:
---
Yup, that would work.
Netty-based block
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201778#comment-14201778
]
Lianhui Wang commented on SPARK-2468:
-
[~zzcclp] in am's log, you can find this log:
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201783#comment-14201783
]
Aaron Davidson commented on SPARK-2468:
---
Thanks a lot for those diagnostics. Can you
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201812#comment-14201812
]
Aaron Davidson commented on SPARK-2468:
---
Looking at the netty code a bit more, it
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201832#comment-14201832
]
Aaron Davidson commented on SPARK-2468:
---
[~lianhuiwang] I have created
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201844#comment-14201844
]
zzc commented on SPARK-2468:
By the way, My test code:
val mapR = textFile.map(line = {
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201849#comment-14201849
]
Aaron Davidson commented on SPARK-2468:
---
[~zzcclp] Thank you for the writeup. Is it
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201865#comment-14201865
]
zzc commented on SPARK-2468:
Hi, Aaron Davidson, what do you mean that Is it really the case
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14202056#comment-14202056
]
Lianhui Wang commented on SPARK-2468:
-
[~adav] yes,with
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14202136#comment-14202136
]
zzc commented on SPARK-2468:
The performance of Netty is worse than NIO in my test. Why?@Aaron
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14202490#comment-14202490
]
Aaron Davidson commented on SPARK-2468:
---
[~lianhuiwang] Can you try again with
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200037#comment-14200037
]
zzc commented on SPARK-2468:
Hi, Aaron Davidson, I set
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200096#comment-14200096
]
Lianhui Wang commented on SPARK-2468:
-
[~adav] i use your branch and memory overhead
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200519#comment-14200519
]
Aaron Davidson commented on SPARK-2468:
---
[~zzcclp] Use of epoll mode is highly
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201378#comment-14201378
]
zzc commented on SPARK-2468:
@Aaron Davidson, Thank you for your recommendation. By the way,
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201432#comment-14201432
]
Aaron Davidson commented on SPARK-2468:
---
[~zzcclp] I believe it is close to merging.
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201482#comment-14201482
]
zzc commented on SPARK-2468:
Thank you.
Netty-based block server / client module
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201483#comment-14201483
]
Reynold Xin commented on SPARK-2468:
It's been merged.
Netty-based block server /
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201487#comment-14201487
]
zzc commented on SPARK-2468:
YES, I see it, and start to compile and test.
Netty-based block
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201684#comment-14201684
]
zzc commented on SPARK-2468:
@Aaron Davidson, error still occur by setting the spark config
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201754#comment-14201754
]
Lianhui Wang commented on SPARK-2468:
-
[~adav] when i set the spark config
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199911#comment-14199911
]
Aaron Davidson commented on SPARK-2468:
---
This could be due to the netty transfer
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199945#comment-14199945
]
zzc commented on SPARK-2468:
Thank you for your reply. I will focus on it.
Netty-based block
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195884#comment-14195884
]
Reynold Xin commented on SPARK-2468:
Are you running on YARN? It seems like YARN just
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195952#comment-14195952
]
zzc commented on SPARK-2468:
Yes, running on yarn client mode, but application is running, not
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196251#comment-14196251
]
zzc commented on SPARK-2468:
If I use less data, it can run successfully, such as 24G snappy
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189825#comment-14189825
]
zzc commented on SPARK-2468:
Hi, Reynold Xin, [SPARK-3453] Netty-based BlockTransferService,
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186910#comment-14186910
]
zzc commented on SPARK-2468:
Hi，Reynold Xin, What time does this issue can be solved?
I need
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187183#comment-14187183
]
Reynold Xin commented on SPARK-2468:
Scheduled to go in in 1.2.
Netty-based block
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187832#comment-14187832
]
zzc commented on SPARK-2468:
Hi，Reynold Xin, When version 1.2 released, approximate time?
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187837#comment-14187837
]
Reynold Xin commented on SPARK-2468:
Take a look here
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187881#comment-14187881
]
zzc commented on SPARK-2468:
thanks
Netty-based block server / client module
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098805#comment-14098805
]
Apache Spark commented on SPARK-2468:
-
User 'rxin' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094449#comment-14094449
]
Apache Spark commented on SPARK-2468:
-
User 'rxin' has created a pull request for this
52 matches
Mail list logo
