Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4380#issuecomment-172753680
I think it's a useful feature and widely used in hive. Why not finish this
feature and merge it to branch-1.6?
---
If your project is set up for it, yo
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4380#issuecomment-141643159
@scwf @yhuai It works in "local" mode, but not well in "yarn-client" mode.
```
15/09/19 18:03:48 ERROR thriftserver.SparkSQLDrive
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4380#issuecomment-141642640
```
case Token(destinationToken(),
Token("TOK_DIR", path :: formats) :: Nil) =>
var isLocal = false
fo
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4380#issuecomment-141636685
@scwf @yhuai I apply this patch to my branch-1.5 code. It works!
But I found a bug. When I use lower case "local", it will try to insert to
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4380#issuecomment-141625598
@yhuai Got it, thank you very much.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4380#issuecomment-140999722
@scwf @liancheng Is there any plan to merge this PR to branch-1.5 recently?
I think this feature is pretty useful.
---
If your project is set up for it, you can
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7421#issuecomment-140960637
@piaozhexiu OK, I got it, thank you very much!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7421#issuecomment-140709152
@liancheng @piaozhexiu Have you cherry-pick this PR to spark branch-1.5?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7125#issuecomment-119939554
@srowen Thanks for your help :smiley:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user litao-buptsse closed the pull request at:
https://github.com/apache/spark/pull/7055
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7055#issuecomment-119169525
@srowen OK I will close this PR and remain only #7125 for this issue
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7055#issuecomment-119163076
@srowen #7125 is for master branch and passed all the test case. You can
consider to merge it to master.
But for branch-1.4, not only contains the bugs
Github user litao-buptsse commented on a diff in the pull request:
https://github.com/apache/spark/pull/7055#discussion_r34025366
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -912,7 +943,10 @@ object Client extends Logging {
val
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7125#issuecomment-118572696
@srowen :ok:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/4142#issuecomment-118109886
@vanzin I think it will be nice to allow user to choose use hadoop
configuration on cluster-side or on client-side. In my company's hadoop
cluster, when
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7125#issuecomment-117222680
@srowen #7125 passes all tests. You can consider to merge it to master.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7055#issuecomment-117206030
@andrewor14 @srowen #7125 is for master and #7055 is for branch-1.4
---
If your project is set up for it, you can reply to this email and have your
reply appear
GitHub user litao-buptsse opened a pull request:
https://github.com/apache/spark/pull/7125
[SPARK-8657] [YARN] Fail to upload resource to viewfs
Fail to upload resource to viewfs in spark-1.4
JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657
You can merge this pull
Github user litao-buptsse closed the pull request at:
https://github.com/apache/spark/pull/7053
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7053#issuecomment-116045509
@srowen I think maybe the first fs param in addResource() is needed,
especially it's useful when doing unit test. I see the
ClientDistributedCacheManager
Github user litao-buptsse closed the pull request at:
https://github.com/apache/spark/pull/7042
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-116004053
I create a clean new PR #7055 (ajust scala code style) for this issue and I
will close this PR. @srowen you can merge this PR to branch-1.4 temporaryly.
In
GitHub user litao-buptsse opened a pull request:
https://github.com/apache/spark/pull/7055
[SPARK-8657] [YARN] Fail to upload conf archive to viewfs
Fail to upload conf archive to viewfs in spark-1.4
JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657
You can merge this
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115979188
OK. I will try to make a version same as master #7053 for branch-1.4
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115978366
Scala style tests failed. I will view my code again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7053#issuecomment-115977907
Some unit tests failed. I will review my PR code again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115942818
@srowen @vanzin I also create a PR to simplify and correct method
addResource() for master branch.
PR Link: https://github.com/apache/spark/pull/7053
---
If
GitHub user litao-buptsse opened a pull request:
https://github.com/apache/spark/pull/7053
[SPARK-8657] [YARN] Simplify method addResource() and get the correct fs
from destPath.
[SPARK-8657] [YARN] Simplify method addResource() and get the correct fs
from destPath.
JIRA
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115934397
ok, I will try to finish a another PR for master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115760662
@vanzin It was my mistake, I didn't check master code carefully. As you
said, my PR can be accepted to branch-1.4?
---
If your project is set up for it, yo
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115700878
@srowen Yes, I see the related code has be deleted by this big issue. But
the modify changes is still in master branch, not merge to branch-1.4. Will you
bring
GitHub user litao-buptsse reopened a pull request:
https://github.com/apache/spark/pull/7042
[YARN] SPARK-8657: Fail to upload conf archive to viewfs.
Fail to upload conf archive to viewfs in spark-1.4
JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657
You can merge
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115695804
@srowen I found this bug appear only in branch-1.4 code, but not in master
code. Should I reopen this issue?
---
If your project is set up for it, you can reply
Github user litao-buptsse closed the pull request at:
https://github.com/apache/spark/pull/7042
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115690926
@srowen you mean I should push this bug fix PR to master, not branch-1.4?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115687872
@srowen I sure want to push to branch-1.4, you mean my PR title format is
not correct? Should it be like "[SPARK-8657] [YARN] Fail to upload conf archiv
Github user litao-buptsse closed the pull request at:
https://github.com/apache/spark/pull/7041
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7041#issuecomment-115652558
@srowen I create a new PR to branch-1.4, PR link:
https://github.com/apache/spark/pull/7042
---
If your project is set up for it, you can reply to this email and
GitHub user litao-buptsse opened a pull request:
https://github.com/apache/spark/pull/7042
[YARN] SPARK-8657: Fail to upload conf archive to viewfs.
Fail to upload conf archive to viewfs in spark-1.4
JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657
You can merge this
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/7041#issuecomment-115652387
@srowen I checkout code from branch-1.4, should I create a merge request to
branch-1.4 not master?
---
If your project is set up for it, you can reply to this
GitHub user litao-buptsse opened a pull request:
https://github.com/apache/spark/pull/7041
[YARN] SPARK-8657: Fail to upload conf archive to viewfs in spark-1.4
Fail to upload conf archive to viewfs in spark-1.4
JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657
You
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/6028#issuecomment-111534747
I got it. I will check my classpath. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user litao-buptsse commented on the pull request:
https://github.com/apache/spark/pull/6028#issuecomment-111528289
Hi all:
I compiled spark new 1.4 version today. But when I run WordCount demo, it
throws NoSuchMethodError "java.lang.NoSuchMethod
43 matches
Mail list logo
