[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19353 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19353 **[Test build #82204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82204/testReport)** for PR 19353 at commit

[GitHub] spark pull request #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream f...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19349#discussion_r141101690 --- Diff: python/pyspark/serializers.py --- @@ -251,6 +256,36 @@ def __repr__(self): return "ArrowPandasSerializer" +class

[GitHub] spark pull request #19338: [SPARK-22123][CORE] Add latest failure reason for...

2017-09-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19338#discussion_r141098993 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala --- @@ -61,6 +61,16 @@ private[scheduler] class TaskSetBlacklist(val conf:

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82192/ Test PASSed. ---

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19287 **[Test build #82192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82192/testReport)** for PR 19287 at commit

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19353 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82197/ Test FAILed. ---

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19353 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18760 is this still valid? Now `LiveListenerBus` have multiple queues and it's very unlikely the system queues will block. --- - To

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19353 **[Test build #82197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82197/testReport)** for PR 19353 at commit

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-09-26 Thread michal-databricks
Github user michal-databricks commented on the issue: https://github.com/apache/spark/pull/18704 I don't fully understand the big picture here, but I am assuming the goal is to have more efficient access to the data stored in compressible spark columnar cache. Either way this

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141095056 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -685,26 +618,11 @@ private[history] class

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 Thanks @mgaido91 , I will update the ML api, maybe also python and java API. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141093806 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -424,208 +459,105 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141093356 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -424,208 +459,105 @@ private[history] class

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82203/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82203/ Test PASSed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141088131 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -318,25 +357,21 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141090672 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -722,75 +640,215 @@ private[history] class

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82202/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82202/ Test FAILed. ---

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141086913 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82201/testReport)** for PR 19339 at commit

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19337 I know, but you should add the parameter to the ml API, otherwise the change is pointless. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82201/ Test PASSed. ---

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141090082 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -722,75 +640,215 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141089537 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -720,19 +633,67 @@ private[history] class

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141089707 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -722,75 +640,215 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141087495 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 @hvanhovell cc: @rednaxelafx Surprisingly, this PR improves memory accesses for `UTF8String ` by about 1.8x due to passing the concrete type to `Platform.getByte()/putByte()`. Without

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141086280 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141085863 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,42 +282,38 @@ private[history] class

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 For LDA, the implementation is in mllib, ml calls mllib. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141084103 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/LevelDBTypeInfo.java --- @@ -0,0 +1,516 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82203/testReport)** for PR 19339 at commit

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19337 As far as I know, mllib is not maintained anymore. You should add the `epsilon` parameter to the new ml API. --- - To

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141082457 --- Diff: python/pyspark/sql/readwriter.py --- @@ -336,6 +336,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141082210 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/LevelDBTypeInfo.java --- @@ -0,0 +1,516 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19339 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141081450 --- Diff: python/pyspark/sql/readwriter.py --- @@ -336,6 +336,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82202/testReport)** for PR 19339 at commit

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141080917 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/LevelDBTypeInfo.java --- @@ -0,0 +1,516 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141080522 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,29 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141080545 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,29 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141080129 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,29 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r141079536 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/KVStore.java --- @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82198/ Test PASSed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82201/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82198/testReport)** for PR 19339 at commit

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141075069 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,25 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19186 **[Test build #82200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82200/testReport)** for PR 19186 at commit

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141073210 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -496,7 +517,7 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141073484 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -496,7 +517,7 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141071110 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/config.scala --- @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19353 **[Test build #82199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82199/testReport)** for PR 19353 at commit

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r141068700 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -722,75 +640,215 @@ private[history] class

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82198/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141055958 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,25 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141059483 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,25 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 This is so weird. I run it fine using Python 3.5.2 but it seems to have some problem using Python 3.4. Let me try Python 3.4 in my local. ---

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19353 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to ad...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19353#discussion_r141065014 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -335,16 +335,15 @@ class

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19353 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to ad...

2017-09-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19353#discussion_r141064436 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -335,16 +335,15 @@ class CodegenContext

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 Hm to me .. I'd actually leave it for now. I am less sure if we should fix it now as we could sweep it out when we remove the deprecated ones later together and, for the current status, it

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19353 **[Test build #82197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82197/testReport)** for PR 19353 at commit

[GitHub] spark issue #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerC...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19353 @cloud-fan @viirya I addressed the post-merge comments you had in https://github.com/apache/spark/pull/19324 --- - To

[GitHub] spark pull request #19353: [SPARK-22103][FOLLOWUP] Rename addExtraCode to ad...

2017-09-26 Thread juliuszsompolski
GitHub user juliuszsompolski opened a pull request: https://github.com/apache/spark/pull/19353 [SPARK-22103][FOLLOWUP] Rename addExtraCode to addInnerClass ## What changes were proposed in this pull request? Address PR comments that appeared post-merge, to rename

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82195/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82195/ Test FAILed. ---

[GitHub] spark pull request #19168: [SPARK-21956][CORE] Fetch up to max bytes when bu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19168#discussion_r141060766 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -439,6 +443,8 @@ final class

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 ok, so maybe I create another JIRA for this issue? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141060165 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -328,6 +331,22 @@ class CodegenContext {

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141058787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -328,6 +331,22 @@ class CodegenContext {

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 Yea, let's do it separately. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19339 As it relies on a deprecated API, I think it is also good to replace pyspark json to use Dataset. But I think it is better in another PR. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 @HyukjinKwon I think your way works fine after fixing a variable name bug (`_jsqlContext` >> `_jssql_ctx`). Should we need to modify the json part to be consistent with the csv part? ---

[GitHub] spark pull request #19261: [SPARK-22040] Add current_date function with time...

2017-09-26 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/19261 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82195/testReport)** for PR 19339 at commit

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19330 **[Test build #82196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82196/testReport)** for PR 19330 at commit

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19352 **[Test build #82194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82194/testReport)** for PR 19352 at commit

[GitHub] spark pull request #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to av...

2017-09-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19352#discussion_r141053165 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -93,7 +93,7 @@ jackson-core-asl-1.9.13.jar jackson-databind-2.6.7.1.jar jackson-jaxrs-1.9.13.jar

[GitHub] spark pull request #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to av...

2017-09-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19352#discussion_r141052746 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -93,7 +93,7 @@ jackson-core-asl-1.9.13.jar jackson-databind-2.6.7.1.jar jackson-jaxrs-1.9.13.jar

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18760 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82188/ Test PASSed. ---

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19352 **[Test build #82193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82193/testReport)** for PR 19352 at commit

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19352 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82193/ Test FAILed. ---

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19352 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18760 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18760 **[Test build #82188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82188/testReport)** for PR 18760 at commit

[GitHub] spark issue #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to avoid Byt...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19352 **[Test build #82193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82193/testReport)** for PR 19352 at commit

[GitHub] spark pull request #19352: [SPARK-22128][CORE] Update paranamer to 2.8 to av...

2017-09-26 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/19352 [SPARK-22128][CORE] Update paranamer to 2.8 to avoid BytecodeReadingParanamer ArrayIndexOutOfBoundsException with Scala 2.12 + Java 8 lambda ## What changes were proposed in this pull request?

[GitHub] spark issue #19261: [SPARK-22040] Add current_date function with timezone id

2017-09-26 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/19261 OK I feel convinced that you feel convinced Spark SQL should not offer this as part of the public API. Thanks for being with me for so long and patient to explain the things. Thanks. ---

[GitHub] spark issue #19263: [SPARK-22050][CORE] Allow BlockUpdated events to be opti...

2017-09-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19263 I see, thanks for the explanation. @vanzin would you please help to review this PR, thanks! --- - To unsubscribe,

<    1   2   3   4   5   >