[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95950/ Test PASSed. ---

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22357 **[Test build #95950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95950/testReport)** for PR 22357 at commit

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216788096 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216788079 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216788016 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -623,8 +623,9 @@ private[spark] class TaskSetManager( *

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22391: [SPARK-25371][SQL][BACKPORT-2.3] struct() should ...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22391#discussion_r216786657 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala --- @@ -147,4 +147,12 @@ class VectorAssemblerSuite

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95946/ Test FAILed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95946/testReport)** for PR 21596 at commit

[GitHub] spark issue #22386: [SPARK-25399][SS] Continuous processing state should not...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22386 **[Test build #95961 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95961/testReport)** for PR 22386 at commit

[GitHub] spark issue #22386: [SPARK-25399][SS] Continuous processing state should not...

2018-09-11 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/22386 LGTM. Just one super nit. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22386: [SPARK-25399] Continuous processing state should ...

2018-09-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22386#discussion_r216781599 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDD.scala --- @@ -74,9 +75,14 @@ class StateStoreRDD[T: ClassTag,

[GitHub] spark issue #22357: [SPARK-25363][SQL] Fix schema pruning in where clause by...

2018-09-11 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/22357 LGTM. Thank you all for participating the discussion. @cloud-fan and @gatorsmile, do you have any further comment? If not, I would like to merge it tomorrow into both master and rc branch

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22396 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22396 @gatorsmile @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22396 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22396 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in...

2018-09-11 Thread sujith71955
GitHub user sujith71955 opened a pull request: https://github.com/apache/spark/pull/22396 [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS path for loadtable command. What changes were proposed in this pull request Updated the Migration guide for the behavior changes

[GitHub] spark pull request #22357: [SPARK-25363][SQL] Fix schema pruning in where cl...

2018-09-11 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/22357#discussion_r216776055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -110,7 +110,17 @@ private[sql]

[GitHub] spark issue #22289: [SPARK-25200][YARN] Allow specifying HADOOP_CONF_DIR as ...

2018-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22289 > I failed to find documentation about the RM adding its own Hadoop config files to the AM/executors' classpath See `Client.getYarnAppClasspath` and `Client.getMRAppClasspath`. >

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216771752 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95960/testReport)** for PR 21596 at commit

[GitHub] spark pull request #22242: Branch 2.3

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22242 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95943/ Test FAILed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95943/testReport)** for PR 21596 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95942/ Test FAILed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22341 There was a trivial conflict in 2.3, I fixed it manually. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95942/testReport)** for PR 21596 at commit

[GitHub] spark issue #22386: [SPARK-25399] Continuous processing state should not aff...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22386 **[Test build #95959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95959/testReport)** for PR 22386 at commit

[GitHub] spark pull request #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-251...

2018-09-11 Thread henryr
Github user henryr closed the pull request at: https://github.com/apache/spark/pull/22211 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-09-11 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/22211 Merged to 2.1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95952/ Test FAILed. ---

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22394 **[Test build #95952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95952/testReport)** for PR 22394 at commit

[GitHub] spark pull request #22341: [SPARK-24889][Core] Update block info when unpers...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22341 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22341 Merging to master, 2.4 and 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95951/ Test FAILed. ---

[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22394 **[Test build #95951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95951/testReport)** for PR 22394 at commit

[GitHub] spark pull request #22386: [SPARK-25399] Continuous processing state should ...

2018-09-11 Thread mukulmurthy
Github user mukulmurthy commented on a diff in the pull request: https://github.com/apache/spark/pull/22386#discussion_r216750746 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDD.scala --- @@ -74,9 +74,14 @@ class StateStoreRDD[T:

[GitHub] spark issue #22386: [SPARK-25399] Continuous processing state should not aff...

2018-09-11 Thread mukulmurthy
Github user mukulmurthy commented on the issue: https://github.com/apache/spark/pull/22386 > ``` > If a continuous processing job runs and the same thread gets reused for a microbatch execution job in the same environment > ``` > Little confuse about this scenario, could

[GitHub] spark issue #21649: [SPARK-23648][R][SQL]Adds more types for hint in SparkR

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21649 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95956/ Test PASSed. ---

[GitHub] spark issue #21649: [SPARK-23648][R][SQL]Adds more types for hint in SparkR

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21649 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21649: [SPARK-23648][R][SQL]Adds more types for hint in SparkR

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21649 **[Test build #95956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95956/testReport)** for PR 21649 at commit

[GitHub] spark issue #18304: [SPARK-21098] Set lineseparator csv multiline and csv wr...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18304 Is this still valid, @cse68197 and @HyukjinKwon ? #18581 is superceded by #20727 . And #20727 seems to be merged. Are we waiting for the others? ---

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95958/testReport)** for PR 22337 at commit

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22364 **[Test build #95957 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95957/testReport)** for PR 22364 at commit

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22364 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22341: [SPARK-24889][Core] Update block info when unpers...

2018-09-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22341#discussion_r216741392 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -538,6 +538,10 @@ private class LiveRDD(val info: RDDInfo) extends LiveEntity

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95938/ Test FAILed. ---

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22337 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95940/ Test FAILed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22390: [SPARK-25402][SQL] Null handling in BooleanSimpli...

2018-09-11 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22390#discussion_r216739580 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -263,10 +263,12 @@ object

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22337 **[Test build #95938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95938/testReport)** for PR 22337 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95940/testReport)** for PR 21596 at commit

[GitHub] spark issue #22384: [SPARK-25398] Minor bugs from comparing unrelated types

2018-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22384 Looks fine. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22288: [SPARK-22148][Scheduler] Acquire new executors to avoid ...

2018-09-11 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22288 Ok I looked at jiras, and this looks it also covers SPARK-15815, right? you could add that to the summary too. You mention some future improvements: > Taking into account static

[GitHub] spark pull request #22213: [SPARK-25221][DEPLOY] Consistent trailing whitesp...

2018-09-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22213 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95944/ Test FAILed. ---

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22364 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-09-11 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22379#discussion_r216734651 --- Diff: R/pkg/R/functions.R --- @@ -3720,3 +3720,22 @@ setMethod("current_timestamp", jc <- callJStatic("org.apache.spark.sql.functions",

[GitHub] spark issue #22364: [SPARK-25379][SQL] Improve AttributeSet and ColumnPrunin...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22364 **[Test build #95944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95944/testReport)** for PR 22364 at commit

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22213 Merging to master / 2.4. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22348: [SPARK-25354][SQL] Reduce unneeded operation in nextKeyV...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22348 Thank you for experiment, @SongYadong . IMO, if the gain is not obvious, we had better keep the current proven one because new one can introduce some future regressions (or even bugs). ---

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95937/ Test PASSed. ---

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216732765 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22341 **[Test build #95937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95937/testReport)** for PR 22341 at commit

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22387 Since this is merged, could you close this PR, @wangyum ? Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95953/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #95953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95953/testReport)** for PR 22295 at commit

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22387 Merged to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22386: [SPARK-25399] Continuous processing state should ...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22386#discussion_r216730172 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDD.scala --- @@ -74,9 +74,14 @@ class StateStoreRDD[T:

[GitHub] spark issue #22390: [SPARK-25402][SQL] Null handling in BooleanSimplificatio...

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22390 **[Test build #95955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95955/testReport)** for PR 22390 at commit

[GitHub] spark issue #22374: [SPARK-25387][SQL] Fix for NPE caused by bad CSV input

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22374 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22374: [SPARK-25387][SQL] Fix for NPE caused by bad CSV input

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22374 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95939/ Test PASSed. ---

[GitHub] spark pull request #22386: [SPARK-25399] Continuous processing state should ...

2018-09-11 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/22386#discussion_r216728848 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -391,6 +393,7 @@ class

[GitHub] spark issue #22395: [SPARK-16323][SQ] Add IntegralDivide expression

2018-09-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22395 `[SQ]` -> `[SQL]` in the title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22374: [SPARK-25387][SQL] Fix for NPE caused by bad CSV input

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22374 **[Test build #95939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95939/testReport)** for PR 22374 at commit

[GitHub] spark issue #21649: [SPARK-23648][R][SQL]Adds more types for hint in SparkR

2018-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21649 **[Test build #95956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95956/testReport)** for PR 21649 at commit

[GitHub] spark issue #21649: [SPARK-23648][R][SQL]Adds more types for hint in SparkR

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21649 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21649: [SPARK-23648][R][SQL]Adds more types for hint in SparkR

2018-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21649 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216724373 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216725731 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216723175 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216726755 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -623,8 +623,9 @@ private[spark] class TaskSetManager( *

[GitHub] spark pull request #22288: [SPARK-22148][Scheduler] Acquire new executors to...

2018-09-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r216726323 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-09-11 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r216728326 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #22386: [SPARK-25399] Continuous processing state should not aff...

2018-09-11 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22386 ``` If a continuous processing job runs and the same thread gets reused for a microbatch execution job in the same environment ``` Little confuse about this scenario, could you

[GitHub] spark pull request #21649: [SPARK-23648][R][SQL]Adds more types for hint in ...

2018-09-11 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21649#discussion_r216727458 --- Diff: R/pkg/R/DataFrame.R --- @@ -3939,7 +3929,15 @@ setMethod("hint", signature(x = "SparkDataFrame", name = "character"),

<    1   2   3   4   5   6   7   >