[GitHub] spark issue #19421: Spark 22192

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19421 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19421: Spark 22192

2017-10-03 Thread ahshahid
GitHub user ahshahid opened a pull request: https://github.com/apache/spark/pull/19421 Spark 22192 Fix for Bug 22192. Recursively handle the nested POJOs when converting an RDD of nested JavaBean object into a DataFrame. Added a bug test (Bug SPARK-22192 Nested POJO

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142499286 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -519,3 +519,18 @@ case class CoGroup(

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142499092 --- Diff: python/pyspark/sql/functions.py --- @@ -2120,6 +2120,7 @@ def wrapper(*args): else

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142498939 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,91 @@ +/* + * Licensed

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142498841 --- Diff: python/pyspark/worker.py --- @@ -74,17 +75,33 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142498880 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142497616 --- Diff: python/pyspark/sql/functions.py --- @@ -2120,6 +2120,7 @@ def wrapper(*args): else

[GitHub] spark issue #19417: [SPARK-22158][SQL][BRANCH-2.2] convertMetastore should n...

2017-10-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19417 Thank you so much! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19417: [SPARK-22158][SQL][BRANCH-2.2] convertMetastore s...

2017-10-03 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/19417 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82423/ Test PASSed. ---

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19395 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19395 **[Test build #82423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82423/testReport)** for PR 19395 at commit

[GitHub] spark issue #19327: [SPARK-22136][SS] Implement stream-stream outer joins.

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82428/testReport)** for PR 19327 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142495095 --- Diff: python/pyspark/sql/functions.py --- @@ -2129,8 +2130,12 @@ def _create_udf(f, returnType, vectorized): def _udf(f,

[GitHub] spark issue #19393: [SPARK-21644][SQL] LocalLimit.maxRows is defined incorre...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19393 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82424/ Test PASSed. ---

[GitHub] spark issue #19393: [SPARK-21644][SQL] LocalLimit.maxRows is defined incorre...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19393 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-10-03 Thread brad-kaiser
Github user brad-kaiser commented on the issue: https://github.com/apache/spark/pull/19041 Hi @vanzin, I have addressed all of your comments. If there's anything else I can do please let me know. Thanks for your help. ---

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142493967 --- Diff: python/pyspark/sql/functions.py --- @@ -2120,6 +2120,7 @@ def wrapper(*args): else

[GitHub] spark issue #19393: [SPARK-21644][SQL] LocalLimit.maxRows is defined incorre...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19393 **[Test build #82424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82424/testReport)** for PR 19393 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142493581 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd =

[GitHub] spark issue #19405: [SPARK-22178] [SQL] Refresh Persistent Views by REFRESH ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82425/ Test PASSed. ---

[GitHub] spark issue #19405: [SPARK-22178] [SQL] Refresh Persistent Views by REFRESH ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19405 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19405: [SPARK-22178] [SQL] Refresh Persistent Views by REFRESH ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19405 **[Test build #82425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82425/testReport)** for PR 19405 at commit

[GitHub] spark issue #19417: [SPARK-22158][SQL][BRANCH-2.2] convertMetastore should n...

2017-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19417 Thanks! Merged to 2.2 Could you close it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142486439 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17357 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82418/ Test PASSed. ---

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17357 **[Test build #82418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82418/testReport)** for PR 17357 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142482842 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142482577 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #82427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82427/testReport)** for PR 19041 at commit

[GitHub] spark issue #18933: [WIP][SPARK-21722][SQL][PYTHON] Enable timezone-aware ti...

2017-10-03 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18933 Hi @ueshin , I've been following SPARK-12297 PR https://github.com/apache/spark/pull/19250 that deals with some of the same issues as here. I think they are proposing a conf that the user

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-10-03 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r142478432 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -120,6 +124,10 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142478440 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142476424 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #19418: [SPARK-19984][SQL] Fix for ERROR codegen.CodeGene...

2017-10-03 Thread rekhajoshm
Github user rekhajoshm commented on a diff in the pull request: https://github.com/apache/spark/pull/19418#discussion_r142475030 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -697,7 +697,12 @@ class

[GitHub] spark pull request #19420: [SPARK-22191] [SQL] Add hive serde example with s...

2017-10-03 Thread crlalam
GitHub user crlalam opened a pull request: https://github.com/apache/spark/pull/19420 [SPARK-22191] [SQL] Add hive serde example with serde properties ## What changes were proposed in this pull request? Added an example for, specifying serde with serde properties for hive

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19394: [SPARK-22170][SQL] Reduce memory consumption in b...

2017-10-03 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19394#discussion_r142474766 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -280,13 +280,20 @@ abstract class SparkPlan extends

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82421/ Test PASSed. ---

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82421 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82421/testReport)** for PR 19083 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142474570 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark issue #19420: [SPARK-22191] [SQL] Add hive serde example with serde pr...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19420 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19406 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19406 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82419/ Test FAILed. ---

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19406 **[Test build #82419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82419/testReport)** for PR 19406 at commit

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18704 **[Test build #82426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82426/testReport)** for PR 18704 at commit

[GitHub] spark pull request #19413: [SPARK-20466][CORE] HadoopRDD#addLocalConfigurati...

2017-10-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19413#discussion_r142470965 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -157,20 +157,23 @@ class HadoopRDD[K, V]( if

[GitHub] spark pull request #19413: [SPARK-20466][CORE] HadoopRDD#addLocalConfigurati...

2017-10-03 Thread sahilTakiar
Github user sahilTakiar commented on a diff in the pull request: https://github.com/apache/spark/pull/19413#discussion_r142466295 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -157,20 +157,23 @@ class HadoopRDD[K, V]( if

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-10-03 Thread akopich
Github user akopich commented on the issue: https://github.com/apache/spark/pull/18924 @WeichenXu123, the PR seems to receive no attention for 10 days now... What should I do? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-10-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19250 FYI Imran is probably going to be out for a few weeks so I'll try to address the feedback here. It would be nice to have people take a look at this, though. ---

[GitHub] spark issue #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-10-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18468 This is followed by #18704 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector...

2017-10-03 Thread kiszk
Github user kiszk closed the pull request at: https://github.com/apache/spark/pull/18468 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-10-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17436 I will rebase this with the latest. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #15219: [WIP][SPARK-14098][SQL] Generate Java code to bui...

2017-10-03 Thread kiszk
Github user kiszk closed the pull request at: https://github.com/apache/spark/pull/15219 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2017-10-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15219 @jiangxb1987 Thank you for pinging me. Sure, since we are working for this feature in other PRs, I close this. --- - To

[GitHub] spark issue #19405: [SPARK-22178] [SQL] Refresh Persistent Views by REFRESH ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19405 **[Test build #82425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82425/testReport)** for PR 19405 at commit

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19406 cc @WeichenXu123 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule conflict between ...

2017-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19149 Actually, the root cause does not resolve by https://github.com/apache/spark/pull/19201. Will add a unit test case later. ---

[GitHub] spark issue #19393: [SPARK-21644][SQL] LocalLimit.maxRows is defined incorre...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19393 **[Test build #82424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82424/testReport)** for PR 19393 at commit

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19395 **[Test build #82423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82423/testReport)** for PR 19395 at commit

[GitHub] spark pull request #19405: [SPARK-22178] [SQL] Refresh Persistent Views by R...

2017-10-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19405#discussion_r142450897 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala --- @@ -31,14 +31,22 @@ import

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19395 **[Test build #82422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82422/testReport)** for PR 19395 at commit

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19395 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19395 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82422/ Test FAILed. ---

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18714 Yes. This is still needed. The target is 3.3 release --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19395 **[Test build #82422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82422/testReport)** for PR 19395 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142448316 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark issue #19395: [SPARK-22171] [SQL] Describe Table Extended Failed when ...

2017-10-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19395 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19419: [SPARK-22188] [CORE] Adding security headers for ...

2017-10-03 Thread krishna-pandey
Github user krishna-pandey commented on a diff in the pull request: https://github.com/apache/spark/pull/19419#discussion_r142446016 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -89,6 +92,9 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark issue #17359: [SPARK-20028][SQL] Add aggreagate expression nGrams

2017-10-03 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/17359 @gatorsmile Sure, I haven't read the context, but it's been nearly half a year since last update, is this PR still active @gczsjdy ? ---

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142446103 --- Diff: python/pyspark/sql/types.py --- @@ -1624,6 +1624,34 @@ def toArrowType(dt): return arrow_type +def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142445946 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark pull request #19419: [SPARK-22188] [CORE] Adding security headers for ...

2017-10-03 Thread krishna-pandey
Github user krishna-pandey commented on a diff in the pull request: https://github.com/apache/spark/pull/19419#discussion_r142445649 --- Diff: conf/spark-defaults.conf.template --- @@ -19,9 +19,16 @@ # This is useful for setting default environmental settings. #

[GitHub] spark pull request #19327: [SPARK-22136][SS] Implement stream-stream outer j...

2017-10-03 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/19327#discussion_r142442657 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingJoinSuite.scala --- @@ -425,6 +426,10 @@ class StreamingJoinSuite extends

[GitHub] spark pull request #19419: [SPARK-22188] [CORE] Adding security headers for ...

2017-10-03 Thread krishna-pandey
Github user krishna-pandey commented on a diff in the pull request: https://github.com/apache/spark/pull/19419#discussion_r142442356 --- Diff: conf/spark-defaults.conf.template --- @@ -19,9 +19,16 @@ # This is useful for setting default environmental settings. #

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19406 Your proposed way is more consistent with the paper and I also think it's more natural to start from 0. The purpose of this patch is to not introduce accuracy regression for other cases, and

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142440939 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142440350 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142439787 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142439736 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142439639 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19406 I think we should probably follow the paper, no? this should fix more cases. Yes, this case also failed for me. The answer 499 is OK too. ---

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19406 @srowen I tried your way on my laptop, it works fine. But an existing test case failed: (500 expected , 499 returned) ``` test("percentile_approx, supports constant folding for parameter

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142438609 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -519,3 +519,18 @@ case class CoGroup(

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142438748 --- Diff: python/pyspark/worker.py --- @@ -32,8 +32,9 @@ from pyspark.serializers import write_with_length, write_int, read_long, \

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142438464 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,91 @@ +/* + * Licensed

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142438142 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala --- @@ -44,14 +66,24 @@ case class

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142438108 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142437297 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -519,3 +519,18 @@ case class CoGroup(

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142436721 --- Diff: python/pyspark/worker.py --- @@ -74,17 +75,33 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-10-03 Thread rezasafi
Github user rezasafi commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r142435495 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +873,45 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19406 Your new test also passes when changing ... ``` val targetError = relativeError * count ... var i = 0 ``` I think this is more likely to be the correct fix as

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-10-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 I will rebase this next a few hours. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142432834 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col,

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18704 **[Test build #82420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82420/testReport)** for PR 18704 at commit

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18704 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82420/ Test FAILed. ---

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18704 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82421/testReport)** for PR 19083 at commit

<    1   2   3   4   5   >