[jira] [Resolved] (SPARK-15312) Detect Duplicate Key in Partition Spec and Table Properties

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15312. - Resolution: Fixed > Detect Duplicate Key in Partition Spec and Table Properties > ---

[jira] [Updated] (SPARK-15312) Detect Duplicate Key in Partition Spec and Table Properties

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15312: Assignee: Xiao Li > Detect Duplicate Key in Partition Spec and Table Properties > -

[jira] [Assigned] (SPARK-15471) ScalaReflection cleanup

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15471: Assignee: Wenchen Fan (was: Apache Spark) > ScalaReflection cleanup > ---

[jira] [Assigned] (SPARK-15471) ScalaReflection cleanup

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15471: Assignee: Apache Spark (was: Wenchen Fan) > ScalaReflection cleanup > ---

[jira] [Commented] (SPARK-15471) ScalaReflection cleanup

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295433#comment-15295433 ] Apache Spark commented on SPARK-15471: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-15471) ScalaReflection cleanup

2016-05-21 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-15471: --- Summary: ScalaReflection cleanup Key: SPARK-15471 URL: https://issues.apache.org/jira/browse/SPARK-15471 Project: Spark Issue Type: Improvement Compo

[jira] [Updated] (SPARK-10903) Simplify SQLContext method signatures and use a singleton

2016-05-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10903: Target Version/s: 2.0.0 > Simplify SQLContext method signatures and use a singleton > -

[jira] [Resolved] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15396. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 > [Spark] [SQL] [DOC] It

[jira] [Resolved] (SPARK-15415) Marking partitions for broadcast broken

2016-05-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15415. - Resolution: Fixed Assignee: Jurriaan Pruis Fix Version/s: 2.0.0 > Marking partiti

[jira] [Commented] (SPARK-15194) Add Python ML API for MultivariateGaussian

2016-05-21 Thread praveen dareddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295413#comment-15295413 ] praveen dareddy commented on SPARK-15194: - [~josephkb][~holdenk] I have sent PR

[jira] [Commented] (SPARK-15194) Add Python ML API for MultivariateGaussian

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295410#comment-15295410 ] Apache Spark commented on SPARK-15194: -- User 'praveendareddy21' has created a pull r

[jira] [Assigned] (SPARK-15194) Add Python ML API for MultivariateGaussian

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15194: Assignee: Apache Spark > Add Python ML API for MultivariateGaussian >

[jira] [Assigned] (SPARK-15194) Add Python ML API for MultivariateGaussian

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15194: Assignee: (was: Apache Spark) > Add Python ML API for MultivariateGaussian > -

[jira] [Commented] (SPARK-15446) catalyst using BigInteger.longValueExact that not supporting java 7 and compile error

2016-05-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295409#comment-15295409 ] Weichen Xu commented on SPARK-15446: OK. I got it, thanks. > catalyst using BigInteg

[jira] [Resolved] (SPARK-15206) Add testcases for Distinct Aggregation in Having clause

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15206. - Resolution: Fixed > Add testcases for Distinct Aggregation in Having clause > ---

[jira] [Updated] (SPARK-15206) Add testcases for Distinct Aggregation in Having clause

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15206: Assignee: Xin Wu > Add testcases for Distinct Aggregation in Having clause > --

[jira] [Commented] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295402#comment-15295402 ] Kazuaki Ishizaki commented on SPARK-15258: -- This PR is not for SPARK-15258. This

[jira] [Assigned] (SPARK-15470) Unify the Configuration Interface in SQLContext

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15470: Assignee: (was: Apache Spark) > Unify the Configuration Interface in SQLContext >

[jira] [Commented] (SPARK-15470) Unify the Configuration Interface in SQLContext

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295391#comment-15295391 ] Apache Spark commented on SPARK-15470: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-15470) Unify the Configuration Interface in SQLContext

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15470: Assignee: Apache Spark > Unify the Configuration Interface in SQLContext > ---

[jira] [Created] (SPARK-15470) Unify the Configuration Interface in SQLContext

2016-05-21 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15470: --- Summary: Unify the Configuration Interface in SQLContext Key: SPARK-15470 URL: https://issues.apache.org/jira/browse/SPARK-15470 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-15330) Reset Command

2016-05-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15330. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 > Reset Command > --

[jira] [Updated] (SPARK-15469) Datanode is not starting correctly: Retrying connect to server: masternode/10.18.0.50:8020. Already tried 5 time(s); maxRetries=45

2016-05-21 Thread Jon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon updated SPARK-15469: Description: when I execute jps command datanode appears, but it seems that is not starting correctly, because if

[jira] [Updated] (SPARK-15469) Datanode is not starting correctly: Retrying connect to server: masternode/10.18.0.50:8020. Already tried 5 time(s); maxRetries=45

2016-05-21 Thread Jon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon updated SPARK-15469: Description: when I execute jps command datanode appears, but it seems that is not starting correctly, because if

[jira] [Created] (SPARK-15469) Datanode is not starting correctly: Retrying connect to server: masternode/10.18.0.50:8020. Already tried 5 time(s); maxRetries=45

2016-05-21 Thread Jon (JIRA)
Jon created SPARK-15469: --- Summary: Datanode is not starting correctly: Retrying connect to server: masternode/10.18.0.50:8020. Already tried 5 time(s); maxRetries=45 Key: SPARK-15469 URL: https://issues.apache.org/jira/brow

[jira] [Assigned] (SPARK-15468) fix some typos while browsing the codes

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15468: Assignee: Apache Spark > fix some typos while browsing the codes > ---

[jira] [Commented] (SPARK-15468) fix some typos while browsing the codes

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295339#comment-15295339 ] Apache Spark commented on SPARK-15468: -- User 'bomeng' has created a pull request for

[jira] [Assigned] (SPARK-15468) fix some typos while browsing the codes

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15468: Assignee: (was: Apache Spark) > fix some typos while browsing the codes >

[jira] [Created] (SPARK-15468) fix some typos while browsing the codes

2016-05-21 Thread Bo Meng (JIRA)
Bo Meng created SPARK-15468: --- Summary: fix some typos while browsing the codes Key: SPARK-15468 URL: https://issues.apache.org/jira/browse/SPARK-15468 Project: Spark Issue Type: Bug Compo

[jira] [Commented] (SPARK-15140) ensure input object of encoder is not null

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295313#comment-15295313 ] Wenchen Fan commented on SPARK-15140: - I'm a little worried about supporting null inp

[jira] [Comment Edited] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295299#comment-15295299 ] Kazuaki Ishizaki edited comment on SPARK-15285 at 5/21/16 11:33 PM: ---

[jira] [Commented] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295301#comment-15295301 ] Yin Huai commented on SPARK-15280: -- https://github.com/apache/spark/pull/13066/files ext

[jira] [Updated] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15280: - Description: Summary: This is a proposal to move ORC serialization logic from OrcOutputWriter to a new c

[jira] [Comment Edited] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295299#comment-15295299 ] Kazuaki Ishizaki edited comment on SPARK-15285 at 5/21/16 11:12 PM: ---

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295299#comment-15295299 ] Kazuaki Ishizaki commented on SPARK-15285: -- I created a (PR)[https://github.com/

[jira] [Updated] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15280: - Description: Summary: This is a proposal to move ORC serialization logic from OrcOutputWriter to a new c

[jira] [Updated] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15280: - Assignee: Ergin Seyfe > Extract ORC serialization logic from OrcOutputWriter for reusability > -

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295296#comment-15295296 ] Apache Spark commented on SPARK-15285: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15285: Assignee: (was: Apache Spark) > Generated SpecificSafeProjection.apply method grows be

[jira] [Assigned] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15285: Assignee: Apache Spark > Generated SpecificSafeProjection.apply method grows beyond 64 KB

[jira] [Resolved] (SPARK-15280) Extract ORC serialization logic from OrcOutputWriter for reusability

2016-05-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-15280. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13066 [https://github.com/

[jira] [Created] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-21 Thread Don Drake (JIRA)
Don Drake created SPARK-15467: - Summary: Getting stack overflow when attempting to query a wide Dataset (>200 fields) Key: SPARK-15467 URL: https://issues.apache.org/jira/browse/SPARK-15467 Project: Spark

[jira] [Assigned] (SPARK-15466) Make `SparkSession` as the entry point to programming with RDD too

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15466: Assignee: Apache Spark > Make `SparkSession` as the entry point to programming with RDD to

[jira] [Commented] (SPARK-15466) Make `SparkSession` as the entry point to programming with RDD too

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295277#comment-15295277 ] Apache Spark commented on SPARK-15466: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-15466) Make `SparkSession` as the entry point to programming with RDD too

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15466: Assignee: (was: Apache Spark) > Make `SparkSession` as the entry point to programming

[jira] [Updated] (SPARK-15466) Make `SparkSession` as the entry point to programming with RDD too

2016-05-21 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15466: -- Description: `SparkSession` greatly reduces the number of concepts which Spark users must know

[jira] [Created] (SPARK-15466) Make `SparkSession` as the entry point to programming with RDD too

2016-05-21 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-15466: - Summary: Make `SparkSession` as the entry point to programming with RDD too Key: SPARK-15466 URL: https://issues.apache.org/jira/browse/SPARK-15466 Project: Spark

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295244#comment-15295244 ] Wenchen Fan commented on SPARK-15285: - [~kiszk] how is it going? I think this issue i

[jira] [Commented] (SPARK-15465) AnalysisException: cannot cast StructType to VectorUDT

2016-05-21 Thread Dmitry Zhukov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295243#comment-15295243 ] Dmitry Zhukov commented on SPARK-15465: --- The same code works absolutely perfect on

[jira] [Updated] (SPARK-15465) AnalysisException: cannot cast StructType to VectorUDT

2016-05-21 Thread Dmitry Zhukov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Zhukov updated SPARK-15465: -- Description: The following code throws exception when using latest Spark 2.0.0-SNAPSHOT: {code:

[jira] [Created] (SPARK-15465) AnalysisException: cannot cast StructType to VectorUDT

2016-05-21 Thread Dmitry Zhukov (JIRA)
Dmitry Zhukov created SPARK-15465: - Summary: AnalysisException: cannot cast StructType to VectorUDT Key: SPARK-15465 URL: https://issues.apache.org/jira/browse/SPARK-15465 Project: Spark Issu

[jira] [Resolved] (SPARK-15394) ML user guide typos and grammar audit

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15394. --- Resolution: Fixed Resolved by https://github.com/apache/spark/pull/13180 > ML user guide typos and g

[jira] [Updated] (SPARK-10216) Avoid creating empty files during overwrite into Hive table with group by query

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10216: -- Fix Version/s: (was: 2.0.0) > Avoid creating empty files during overwrite into Hive table with grou

[jira] [Updated] (SPARK-3000) Drop old blocks to disk in parallel when memory is not large enough for caching new blocks

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3000: - Target Version/s: (was: 2.0.0) > Drop old blocks to disk in parallel when memory is not large enough for

[jira] [Updated] (SPARK-15101) Audit: ml.clustering and ml.recommendation

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15101: -- Assignee: zhengruifeng > Audit: ml.clustering and ml.recommendation > -

[jira] [Updated] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15078: -- Assignee: Sameer Agarwal > Add all TPCDS 1.4 benchmark queries for SparkSQL > -

[jira] [Resolved] (SPARK-15452) Mark aggregator API as experimental

2016-05-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15452. - Resolution: Fixed Fix Version/s: 2.0.0 > Mark aggregator API as experimental > ---

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295225#comment-15295225 ] Sean Owen commented on SPARK-15457: --- You or anyone can add the following changes to a b

[jira] [Updated] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15078: Fix Version/s: 2.0.0 > Add all TPCDS 1.4 benchmark queries for SparkSQL > -

[jira] [Assigned] (SPARK-15415) Marking partitions for broadcast broken

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15415: Assignee: (was: Apache Spark) > Marking partitions for broadcast broken >

[jira] [Commented] (SPARK-15415) Marking partitions for broadcast broken

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295220#comment-15295220 ] Apache Spark commented on SPARK-15415: -- User 'jurriaan' has created a pull request f

[jira] [Assigned] (SPARK-15415) Marking partitions for broadcast broken

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15415: Assignee: Apache Spark > Marking partitions for broadcast broken > ---

[jira] [Commented] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295170#comment-15295170 ] Apache Spark commented on SPARK-15258: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15258: Assignee: (was: Apache Spark) > Nested/Chained case statements generate codegen over 6

[jira] [Assigned] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15258: Assignee: Apache Spark > Nested/Chained case statements generate codegen over 64k exceptio

[jira] [Commented] (SPARK-12071) Programming guide should explain NULL in JVM translate to NA in R

2016-05-21 Thread Krishna Kalyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295161#comment-15295161 ] Krishna Kalyan commented on SPARK-12071: [~holdenk] [~felixcheung] Can I take up

[jira] [Comment Edited] (SPARK-15329) When start spark with yarn: spark.SparkContext: Error initializing SparkContext.

2016-05-21 Thread Vijay Parmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294753#comment-15294753 ] Vijay Parmar edited comment on SPARK-15329 at 5/21/16 5:41 PM:

[jira] [Updated] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python testsuites

2016-05-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15464: --- Labels: test (was: ) > Replace SQLContext and SparkContext with SparkSession using builder pattern

[jira] [Updated] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python testsuites

2016-05-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15464: --- Summary: Replace SQLContext and SparkContext with SparkSession using builder pattern in python testsu

[jira] [Commented] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295148#comment-15295148 ] Apache Spark commented on SPARK-15464: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15464: Assignee: (was: Apache Spark) > Replace SQLContext and SparkContext with SparkSession

[jira] [Assigned] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15464: Assignee: Apache Spark > Replace SQLContext and SparkContext with SparkSession using build

[jira] [Created] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code

2016-05-21 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-15464: -- Summary: Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code Key: SPARK-15464 URL: https://issues.apache.org/jira/browse/SPARK-15464

[jira] [Updated] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-05-21 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15453: Summary: Improve join planning for bucketed / sorted tables (was: Sort Merge Join to use bucketing

[jira] [Resolved] (SPARK-15114) Column name generated by typed aggregate is super verbose

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15114. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13045 [https://githu

[jira] [Updated] (SPARK-15114) Column name generated by typed aggregate is super verbose

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15114: Assignee: Dilip Biswal > Column name generated by typed aggregate is super verbose > --

[jira] [Resolved] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15462. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13241 [https://githu

[jira] [Updated] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15462: Assignee: Dongjoon Hyun > Checking `resolved === false` is enough for testcases. >

[jira] [Commented] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295066#comment-15295066 ] Wenchen Fan commented on SPARK-15441: - I think we can't always transform a row with a

[jira] [Issue Comment Deleted] (SPARK-15461) modify python test script using default version 2.7

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15461: -- Comment: was deleted (was: Oh..it really still support python 2.6 but need install python module unitt

[jira] [Closed] (SPARK-15461) modify python test script using default version 2.7

2016-05-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu closed SPARK-15461. -- Resolution: Fixed install unittest2 then we can use python 2.6 run spark python tests. > modify python

[jira] [Commented] (SPARK-15461) modify python test script using default version 2.7

2016-05-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295056#comment-15295056 ] Weichen Xu commented on SPARK-15461: Oh..it really still support python 2.6 but need

[jira] [Commented] (SPARK-15461) modify python test script using default version 2.7

2016-05-21 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295057#comment-15295057 ] Weichen Xu commented on SPARK-15461: Oh..it really still support python 2.6 but need

[jira] [Commented] (SPARK-15461) modify python test script using default version 2.7

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295042#comment-15295042 ] Sean Owen commented on SPARK-15461: --- Hm, I thought Python 2.6 was still supposed to be

[jira] [Created] (SPARK-15463) Support for creating a dataframe from CSV in RDD[String]

2016-05-21 Thread PJ Fanning (JIRA)
PJ Fanning created SPARK-15463: -- Summary: Support for creating a dataframe from CSV in RDD[String] Key: SPARK-15463 URL: https://issues.apache.org/jira/browse/SPARK-15463 Project: Spark Issue Ty

[jira] [Updated] (SPARK-15463) Support for creating a dataframe from CSV in RDD[String]

2016-05-21 Thread PJ Fanning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-15463: --- Issue Type: Improvement (was: Bug) > Support for creating a dataframe from CSV in RDD[String] >

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295014#comment-15295014 ] Kazuaki Ishizaki commented on SPARK-15285: -- I see, I started doing this. > Gene

[jira] [Updated] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2016-05-21 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14174: - Description: The MiniBatchKMeans is a variant of the KMeans algorithm which uses mini-batches to

[jira] [Updated] (SPARK-15445) Build fails for java 1.7 after adding java.math.BigInteger support [SPARK-11827][SQL]

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15445: -- Assignee: Sandeep Singh > Build fails for java 1.7 after adding java.math.BigInteger support > [SPARK-

[jira] [Resolved] (SPARK-15445) Build fails for java 1.7 after adding java.math.BigInteger support [SPARK-11827][SQL]

2016-05-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15445. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13223 [https://github.co

[jira] [Updated] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15462: -- Description: In only `catalyst` module, there exists 8 evaluation test cases on unresolved exp

[jira] [Commented] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294818#comment-15294818 ] Apache Spark commented on SPARK-15462: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15462: Assignee: (was: Apache Spark) > Checking `resolved === false` is enough for testcases.

[jira] [Assigned] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15462: Assignee: Apache Spark > Checking `resolved === false` is enough for testcases. >

[jira] [Updated] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15462: -- Description: In only `catalyst` module, there exists 7 evaluation test cases on unresolved exp

[jira] [Updated] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15462: -- Description: In only `catalyst` module, there exists 7 evaluation test cases on unresolved exp

[jira] [Commented] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294817#comment-15294817 ] Dongjoon Hyun commented on SPARK-15462: --- Hi, [~cloud_fan]. This is the PR according

[jira] [Updated] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15462: -- Priority: Minor (was: Major) Issue Type: Test (was: Bug) > Checking `resolved === false

[jira] [Created] (SPARK-15462) Checking `resolved === false` is enough for testcases.

2016-05-21 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-15462: - Summary: Checking `resolved === false` is enough for testcases. Key: SPARK-15462 URL: https://issues.apache.org/jira/browse/SPARK-15462 Project: Spark Issu

[jira] [Updated] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15441: --- Assignee: Wenchen Fan > dataset outer join seems to return incorrect result > ---

  1   2   >