[GitHub] spark issue #17503: [SPARK-3159][MLlib] Check for reducible DecisionTree

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17503 Can you do some benchmark to show how much improvement this change will bring ? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19357: support histogram in filter cardinality estimatio...

2017-09-26 Thread ron8hu
GitHub user ron8hu opened a pull request: https://github.com/apache/spark/pull/19357 support histogram in filter cardinality estimation ## What changes were proposed in this pull request? Histogram is effective in dealing with skewed distribution. After we generate

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 In a quick look, both tests failures: ``` File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/readwriter.py", line 303, in parquet return

[GitHub] spark pull request #19336: [SPARK-21947][SS] Check and report error when mon...

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19336#discussion_r141215037 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -516,6 +516,14 @@ class

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82210/ Test PASSed. ---

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19355 **[Test build #82210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82210/testReport)** for PR 19355 at commit

[GitHub] spark issue #19348: [BUILD] Close stale PRs

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19348 Added. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19348: [BUILD] Close stale PRs

2017-09-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19348 Add: #15009 #18253 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19020: [SPARK-3181] [ML] Implement huber loss for LinearRegress...

2017-09-26 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19020 > We have two candidate name: epsilon or m I see; that seems fine then, though I worry that we use "epsilon" in MLlib (tests) for "a very small positive number." Can we document it more

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-26 Thread pgandhi999
Github user pgandhi999 commented on the issue: https://github.com/apache/spark/pull/19270 Ok, I will look into it. I am currently fixing ui bugs and unit tests, so will commit those changes first, then will look into the above issue. Thank you. ---

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-26 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19270 Ok so I'm still doing more testing but I've narrowed the above problem. The above error is occurring when using either local or standalone, the error doesn't appear when using yarn. I'll continue

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82211/testReport)** for PR 19327 at commit

[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19218 Hi, @fjh100456 . I left a few comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r141188386 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -728,4 +728,120 @@ class InsertSuite extends QueryTest with

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r141187890 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -728,4 +728,120 @@ class InsertSuite extends QueryTest with

[GitHub] spark issue #18098: [SPARK-16944][Mesos] Improve data locality when launchin...

2017-09-26 Thread gpang
Github user gpang commented on the issue: https://github.com/apache/spark/pull/18098 Hi @srowen @vanzin , would you be able to help me with this PR? Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r141187555 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -728,4 +728,120 @@ class InsertSuite extends QueryTest with

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r141186378 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -728,4 +728,120 @@ class InsertSuite extends QueryTest with

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-26 Thread szhem
Github user szhem commented on the issue: https://github.com/apache/spark/pull/19294 Hello guys, are there a change for this patch to be merged to master? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-09-26 Thread kalvinnchau
Github user kalvinnchau commented on the issue: https://github.com/apache/spark/pull/19272 @ArtRand thanks! I've been testing a local version of doing that, I'll pull that change in and test it as well. --- - To

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82206/ Test PASSed. ---

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18887 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18887 **[Test build #82206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82206/testReport)** for PR 18887 at commit

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19355 **[Test build #82210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82210/testReport)** for PR 19355 at commit

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19355 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19355 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19355 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82209/ Test FAILed. ---

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19355 **[Test build #82209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82209/testReport)** for PR 19355 at commit

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-09-26 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19272 Hey @kalvinnchau good catch on the first renewal time. I believe I addressed it. Have a look. Thanks again. --- - To

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82205/ Test PASSed. ---

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19355 **[Test build #82205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82205/testReport)** for PR 19355 at commit

[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...

2017-09-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19343 OK, closing this and the jira --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19343: [SPARK-22121][SQL] Correct database location for ...

2017-09-26 Thread squito
Github user squito closed the pull request at: https://github.com/apache/spark/pull/19343 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19327 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19327 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82208/ Test FAILed. ---

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82208/testReport)** for PR 19327 at commit

[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...

2017-09-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19343 Thanks! Maybe we can close it now and revisit it when we have a better way to resolve the file system specific issues? ---

[GitHub] spark issue #19356: Merge pull request #1 from apache/master

2017-09-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19356 Close this @yaozhang2016 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19349 Nice job on refactoring `PythonRunner`! I think we should just replace the arrow file format with stream format for pandas udf instead of having a new conf to enable it, as long as all the

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19353 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82204/ Test PASSed. ---

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19353 **[Test build #82204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82204/testReport)** for PR 19353 at commit

[GitHub] spark issue #19356: Merge pull request #1 from apache/master

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19356 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19356: Merge pull request #1 from apache/master

2017-09-26 Thread yaozhang2016
GitHub user yaozhang2016 opened a pull request: https://github.com/apache/spark/pull/19356 Merge pull request #1 from apache/master update from origin ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How

[GitHub] spark pull request #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream f...

2017-09-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19349#discussion_r141142944 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowStreamPythonUDFRunner.scala --- @@ -0,0 +1,197 @@ +/* + *

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19355 **[Test build #82209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82209/testReport)** for PR 19355 at commit

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82208/testReport)** for PR 19327 at commit

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82207/testReport)** for PR 19327 at commit

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19327 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19327 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82207/ Test FAILed. ---

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19186 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82200/ Test PASSed. ---

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19186 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19186 **[Test build #82200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82200/testReport)** for PR 19186 at commit

[GitHub] spark issue #19327: [WIP] Implement stream-stream outer joins.

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82207/testReport)** for PR 19327 at commit

[GitHub] spark pull request #19355: [SPARK-22130][Core] UTF8String.trim() scans " " t...

2017-09-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19355#discussion_r141131388 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -501,14 +501,13 @@ public UTF8String trim() { int e =

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18887 **[Test build #82206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82206/testReport)** for PR 18887 at commit

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141127050 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -424,208 +459,105 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141125410 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -424,208 +459,105 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141124934 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -685,26 +618,11 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141124127 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -722,75 +640,215 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141123714 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141123619 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141123050 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19250 @HyukjinKwon you might be interested in this one also --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141122539 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141121472 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -496,7 +517,7 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141121349 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/config.scala --- @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141120995 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -722,75 +640,215 @@ private[history] class

[GitHub] spark pull request #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to ad...

2017-09-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19353 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141120298 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/LevelDBTypeInfo.java --- @@ -0,0 +1,516 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19353 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141119638 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/LevelDBTypeInfo.java --- @@ -0,0 +1,516 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141119266 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/KVStore.java --- @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...

2017-09-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19343 I don't see much point in putting this in the docs ... it seems too fine-grained a detail to be useful there. I just don't see the users who encounter this exception from going to look at the spot

[GitHub] spark issue #19355: [SPARK-22130][Core] UTF8String.trim() scans " " twice

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19355 **[Test build #82205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82205/testReport)** for PR 19355 at commit

[GitHub] spark pull request #19355: [SPARK-22130][Core] UTF8String.trim() scans " " t...

2017-09-26 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/19355 [SPARK-22130][Core] UTF8String.trim() scans " " twice ## What changes were proposed in this pull request? This PR allows us to scan a string including only white space (e.g. `" "`)

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82199/ Test PASSed. ---

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19353 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19353 **[Test build #82199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82199/testReport)** for PR 19353 at commit

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19194 one more thought -- have you considered adding stage properties? I know that is more involved since its an API change, but I dunno how many times I've wanted something like that, eg. for labelling

[GitHub] spark pull request #19325: [SPARK-22106][PYSPARK][SQL] Disable 0-parameter p...

2017-09-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19325#discussion_r141112455 --- Diff: python/pyspark/sql/functions.py --- @@ -2183,14 +2187,28 @@ def pandas_udf(f=None, returnType=StringType()): :param f: python

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19352 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82194/ Test PASSed. ---

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18760 cc @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19352 **[Test build #82194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82194/testReport)** for PR 19352 at commit

[GitHub] spark pull request #18209: [SPARK-20992][Scheduler] Add support for Nomad as...

2017-09-26 Thread barnardb
Github user barnardb closed the pull request at: https://github.com/apache/spark/pull/18209 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19354: [SPARK-20992][Scheduler] Add links in documentation to N...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19354 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...

2017-09-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19343 @squito Thank you! Instead of changing the source codes, could we just update the document https://spark.apache.org/docs/2.2.0/sql-programming-guide.html#hive-tables ? This might be

[GitHub] spark pull request #19354: [SPARK-20992][Scheduler] Add links in documentati...

2017-09-26 Thread barnardb
GitHub user barnardb opened a pull request: https://github.com/apache/spark/pull/19354 [SPARK-20992][Scheduler] Add links in documentation to Nomad integration. ## What changes were proposed in this pull request? Adds links to the fork that provides integration with Nomad,

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82196/ Test FAILed. ---

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19330 **[Test build #82196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82196/testReport)** for PR 19330 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19349 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream f...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19349#discussion_r141105065 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowStreamPythonUDFRunner.scala --- @@ -0,0 +1,197 @@ +/* + * Licensed

[GitHub] spark pull request #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream f...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19349#discussion_r141104830 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowStreamPythonUDFRunner.scala --- @@ -0,0 +1,197 @@ +/* + * Licensed

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/18760 The hold mode is still valid, I resolved the conflict and add the logic into `AsyncEventQueue`, it can confirm by the test case added in this

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 umm.. I test it fine using Python 3.4 in my local. I'm not sure why did it test fail sometime... :( --- - To unsubscribe,

[GitHub] spark pull request #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream f...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19349#discussion_r141103372 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -925,6 +925,13 @@ object SQLConf { .intConf

<    1   2   3   4   5   >