[jira] [Created] (SPARK-17307) Document what all access is needed on S3 bucket when trying to save a model

2016-08-29 Thread Aseem Bansal (JIRA)
Aseem Bansal created SPARK-17307: Summary: Document what all access is needed on S3 bucket when trying to save a model Key: SPARK-17307 URL: https://issues.apache.org/jira/browse/SPARK-17307 Project:

[jira] [Commented] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

2016-08-29 Thread Teng Yutong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447793#comment-15447793 ] Teng Yutong commented on SPARK-17290: - So this is an issueSorry for the duplication. Should I

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-08-29 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447791#comment-15447791 ] Sun Rui commented on SPARK-13525: - yes, if spark fails to launch R worker as a process, it should throw

[jira] [Commented] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

2016-08-29 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447775#comment-15447775 ] Liwei Lin commented on SPARK-17290: --- Oh [~hyukjin.kwon] you are so devoted :-D > Spark CSVInferSchema

[jira] [Commented] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

2016-08-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447758#comment-15447758 ] Hyukjin Kwon commented on SPARK-17290: -- BTW, there is a related PR here,

[jira] [Commented] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

2016-08-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447755#comment-15447755 ] Hyukjin Kwon commented on SPARK-17290: -- This should be a duplicate of SPARK-16462, SPARK-16460,

[jira] [Commented] (SPARK-13573) Open SparkR APIs (R package) to allow better 3rd party usage

2016-08-29 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447710#comment-15447710 ] Sun Rui commented on SPARK-13573: - [~chipsenkbeil], we have made public the method for creating Java

[jira] [Resolved] (SPARK-17301) Remove unused classTag field from AtomicType base class

2016-08-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17301. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Remove unused classTag

[jira] [Assigned] (SPARK-3162) Train DecisionTree locally when possible

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3162: --- Assignee: Apache Spark > Train DecisionTree locally when possible >

[jira] [Commented] (SPARK-3162) Train DecisionTree locally when possible

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447662#comment-15447662 ] Apache Spark commented on SPARK-3162: - User 'smurching' has created a pull request for this issue:

[jira] [Assigned] (SPARK-3162) Train DecisionTree locally when possible

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3162: --- Assignee: (was: Apache Spark) > Train DecisionTree locally when possible >

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products by default

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447642#comment-15447642 ] Srinath commented on SPARK-17298: - I've updated the description. Hopefully it is clearer. Note that

[jira] [Updated] (SPARK-17298) Require explicit CROSS join for cartesian products by default

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinath updated SPARK-17298: Description: Require the use of CROSS join syntax in SQL (and a new crossJoin DataFrame API) to specify

[jira] [Updated] (SPARK-17298) Require explicit CROSS join for cartesian products by default

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinath updated SPARK-17298: Description: Require the use of CROSS join syntax in SQL (and a new crossJoin DataFrame API) to specify

[jira] [Updated] (SPARK-17298) Require explicit CROSS join for cartesian products by default

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinath updated SPARK-17298: Description: Require the use of CROSS join syntax in SQL (and a new crossJoin DataFrame API) to specify

[jira] [Updated] (SPARK-17298) Require explicit CROSS join for cartesian products by default

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinath updated SPARK-17298: Summary: Require explicit CROSS join for cartesian products by default (was: Require explicit CROSS join

[jira] [Updated] (SPARK-17306) Memory leak in QuantileSummaries

2016-08-29 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Zhong updated SPARK-17306: --- Component/s: SQL > Memory leak in QuantileSummaries > > >

[jira] [Created] (SPARK-17306) Memory leak in QuantileSummaries

2016-08-29 Thread Sean Zhong (JIRA)
Sean Zhong created SPARK-17306: -- Summary: Memory leak in QuantileSummaries Key: SPARK-17306 URL: https://issues.apache.org/jira/browse/SPARK-17306 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447518#comment-15447518 ] Apache Spark commented on SPARK-17304: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17304: Assignee: Apache Spark (was: Josh Rosen) > TaskSetManager.abortIfCompletelyBlacklisted

[jira] [Assigned] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17304: Assignee: Josh Rosen (was: Apache Spark) > TaskSetManager.abortIfCompletelyBlacklisted

[jira] [Created] (SPARK-17305) Cannot save ML PipelineModel in pyspark, PipelineModel.params still return null values

2016-08-29 Thread Hechao Sun (JIRA)
Hechao Sun created SPARK-17305: -- Summary: Cannot save ML PipelineModel in pyspark, PipelineModel.params still return null values Key: SPARK-17305 URL: https://issues.apache.org/jira/browse/SPARK-17305

[jira] [Updated] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17304: --- Description: If you run {code} sc.parallelize(1 to 10, 10).map(identity).count() {code}

[jira] [Updated] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17304: --- Target Version/s: 2.1.0 > TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in

[jira] [Updated] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17304: --- Affects Version/s: 2.1.0 > TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in

[jira] [Updated] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17304: --- Assignee: Josh Rosen Issue Type: Bug (was: Improvement) >

[jira] [Created] (SPARK-17304) TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark

2016-08-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17304: -- Summary: TaskSetManager.abortIfCompletelyBlacklisted is a perf. hotspot in scheduler benchmark Key: SPARK-17304 URL: https://issues.apache.org/jira/browse/SPARK-17304

[jira] [Commented] (SPARK-17303) dev/run-tests fails if spark-warehouse directory exists

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447344#comment-15447344 ] Apache Spark commented on SPARK-17303: -- User 'frreiss' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17303) dev/run-tests fails if spark-warehouse directory exists

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17303: Assignee: (was: Apache Spark) > dev/run-tests fails if spark-warehouse directory

[jira] [Assigned] (SPARK-17303) dev/run-tests fails if spark-warehouse directory exists

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17303: Assignee: Apache Spark > dev/run-tests fails if spark-warehouse directory exists >

[jira] [Created] (SPARK-17303) dev/run-tests fails if spark-warehouse directory exists

2016-08-29 Thread Frederick Reiss (JIRA)
Frederick Reiss created SPARK-17303: --- Summary: dev/run-tests fails if spark-warehouse directory exists Key: SPARK-17303 URL: https://issues.apache.org/jira/browse/SPARK-17303 Project: Spark

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-29 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447285#comment-15447285 ] Gang Wu commented on SPARK-17243: - Yup you're right. I finally got some app_ids that were not in the

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-29 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447266#comment-15447266 ] Alex Bozarth commented on SPARK-17243: -- that's odd, how long did you wait before accessing the app

[jira] [Created] (SPARK-17302) Cannot set non-Spark SQL session variables in hive-site.xml, spark-defaults.conf, or using --conf

2016-08-29 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-17302: - Summary: Cannot set non-Spark SQL session variables in hive-site.xml, spark-defaults.conf, or using --conf Key: SPARK-17302 URL: https://issues.apache.org/jira/browse/SPARK-17302

[jira] [Assigned] (SPARK-17301) Remove unused classTag field from AtomicType base class

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17301: Assignee: Apache Spark (was: Josh Rosen) > Remove unused classTag field from AtomicType

[jira] [Commented] (SPARK-17301) Remove unused classTag field from AtomicType base class

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447191#comment-15447191 ] Apache Spark commented on SPARK-17301: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17301) Remove unused classTag field from AtomicType base class

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17301: Assignee: Josh Rosen (was: Apache Spark) > Remove unused classTag field from AtomicType

[jira] [Created] (SPARK-17301) Remove unused classTag field from AtomicType base class

2016-08-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17301: -- Summary: Remove unused classTag field from AtomicType base class Key: SPARK-17301 URL: https://issues.apache.org/jira/browse/SPARK-17301 Project: Spark Issue

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-29 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447172#comment-15447172 ] Gang Wu commented on SPARK-17243: - I imported the last change. I can get all application list from rest

[jira] [Created] (SPARK-17300) ClosedChannelException caused by missing block manager when speculative tasks are killed

2016-08-29 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-17300: - Summary: ClosedChannelException caused by missing block manager when speculative tasks are killed Key: SPARK-17300 URL: https://issues.apache.org/jira/browse/SPARK-17300

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-29 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447112#comment-15447112 ] Alex Bozarth commented on SPARK-17243: -- [~wgtmac] I'm not sure which version of the pr you tested,

[jira] [Resolved] (SPARK-14662) LinearRegressionModel uses only default parameters if yStd is 0

2016-08-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14662. --- Resolution: Fixed Assignee: Yanbo Liang Fix Version/s: 2.0.0 This

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-29 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447099#comment-15447099 ] Gang Wu commented on SPARK-17243: - I've test this PR. It indeed reduces the number of application

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447015#comment-15447015 ] Sean Owen commented on SPARK-17298: --- Agree, though spark.sql.crossJoin.enabled=false by default, so the

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446988#comment-15446988 ] Sameer Agarwal commented on SPARK-17298: Sean, if I understand correctly, here are the new

[jira] [Assigned] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17296: Assignee: Apache Spark > Spark SQL: cross join + two joins = BUG >

[jira] [Assigned] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17296: Assignee: (was: Apache Spark) > Spark SQL: cross join + two joins = BUG >

[jira] [Commented] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446979#comment-15446979 ] Apache Spark commented on SPARK-17296: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446946#comment-15446946 ] Sean Owen commented on SPARK-17298: --- This should be an error unless you set the property to allow

[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446930#comment-15446930 ] Sean Owen commented on SPARK-17299: --- That's probably the intent yeah. If that's how the other engines

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446920#comment-15446920 ] Srinath commented on SPARK-17298: - So if I do the following: create temporary view nt1 as select * from

[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-29 Thread Jeremy Beard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446902#comment-15446902 ] Jeremy Beard commented on SPARK-17299: -- What is the priority for compatibility with other SQL

[jira] [Resolved] (SPARK-16581) Making JVM backend calling functions public

2016-08-29 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-16581. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue

[jira] [Assigned] (SPARK-16581) Making JVM backend calling functions public

2016-08-29 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman reassigned SPARK-16581: - Assignee: Shivaram Venkataraman > Making JVM backend calling functions

[jira] [Updated] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17299: -- Priority: Minor (was: Major) Component/s: Documentation I'm almost certain its intent is to

[jira] [Created] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-29 Thread Jeremy Beard (JIRA)
Jeremy Beard created SPARK-17299: Summary: TRIM/LTRIM/RTRIM strips characters other than spaces Key: SPARK-17299 URL: https://issues.apache.org/jira/browse/SPARK-17299 Project: Spark Issue

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446740#comment-15446740 ] Sean Owen commented on SPARK-17298: --- This already results in an error. You mean that it will not result

[jira] [Assigned] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17298: Assignee: Apache Spark > Require explicit CROSS join for cartesian products >

[jira] [Assigned] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17298: Assignee: (was: Apache Spark) > Require explicit CROSS join for cartesian products >

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446719#comment-15446719 ] Apache Spark commented on SPARK-17298: -- User 'srinathshankar' has created a pull request for this

[jira] [Commented] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Srinath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446682#comment-15446682 ] Srinath commented on SPARK-17298: - You are correct that with this change, queries of the form {noformat}

[jira] [Updated] (SPARK-16578) Configurable hostname for RBackend

2016-08-29 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-16578: -- Issue Type: New Feature (was: Sub-task) Parent: (was: SPARK-15799)

[jira] [Resolved] (SPARK-17063) MSCK REPAIR TABLE is super slow with Hive metastore

2016-08-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17063. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

2016-08-29 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1544#comment-1544 ] Tomer Kaftan commented on SPARK-17110: -- Hi Miao, all that is needed using the fully default

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446615#comment-15446615 ] Apache Spark commented on SPARK-17289: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17289: Assignee: (was: Apache Spark) > Sort based partial aggregation breaks due to

[jira] [Assigned] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17289: Assignee: Apache Spark > Sort based partial aggregation breaks due to SPARK-12978 >

[jira] [Updated] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17298: -- Priority: Minor (was: Major) Hm, aren't you suggesting that cartesian joins be _allowed_ when

[jira] [Created] (SPARK-17298) Require explicit CROSS join for cartesian products

2016-08-29 Thread Srinath (JIRA)
Srinath created SPARK-17298: --- Summary: Require explicit CROSS join for cartesian products Key: SPARK-17298 URL: https://issues.apache.org/jira/browse/SPARK-17298 Project: Spark Issue Type: Story

[jira] [Updated] (SPARK-17297) window function generates unexpected results due to startTime being relative to UTC

2016-08-29 Thread Pete Baker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pete Baker updated SPARK-17297: --- Description: In Spark 2.0.0, the {{window(Column timeColumn, String windowDuration, String

[jira] [Updated] (SPARK-16240) model loading backward compatibility for ml.clustering.LDA

2016-08-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16240: -- Shepherd: Joseph K. Bradley > model loading backward compatibility for

[jira] [Commented] (SPARK-17297) window function generates unexpected results due to startTime being relative to UTC

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446391#comment-15446391 ] Sean Owen commented on SPARK-17297: --- I don't think there's an assumption about what a day is in here,

[jira] [Created] (SPARK-17297) window function generates unexpected results due to startTime being relative to UTC

2016-08-29 Thread Pete Baker (JIRA)
Pete Baker created SPARK-17297: -- Summary: window function generates unexpected results due to startTime being relative to UTC Key: SPARK-17297 URL: https://issues.apache.org/jira/browse/SPARK-17297

[jira] [Commented] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446371#comment-15446371 ] Herman van Hovell commented on SPARK-17296: --- I think you have found a bug in the parser. Your

[jira] [Commented] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Furcy Pin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446382#comment-15446382 ] Furcy Pin commented on SPARK-17296: --- Yes, this is not critical though, a workaround is to invert the

[jira] [Updated] (SPARK-17297) window function generates unexpected results due to startTime being relative to UTC

2016-08-29 Thread Pete Baker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pete Baker updated SPARK-17297: --- Description: In Spark 2.0.0, the {{window(Column timeColumn, String windowDuration, String

[jira] [Commented] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446367#comment-15446367 ] Sean Owen commented on SPARK-17296: --- Pardon if I'm missing something, but you are not joining T3 with

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446344#comment-15446344 ] Takeshi Yamamuro commented on SPARK-17289: -- okay. I'll add tests and open pr. > Sort based

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446341#comment-15446341 ] Herman van Hovell commented on SPARK-17289: --- Looks good. Can you open a PR? > Sort based

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446323#comment-15446323 ] Takeshi Yamamuro commented on SPARK-17289: -- This is probably because EnsureRequirements does not

[jira] [Commented] (SPARK-15453) Improve join planning for bucketed / sorted tables

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446283#comment-15446283 ] Apache Spark commented on SPARK-15453: -- User 'tejasapatil' has created a pull request for this

[jira] [Commented] (SPARK-16992) Pep8 code style

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446255#comment-15446255 ] Apache Spark commented on SPARK-16992: -- User 'Stibbons' has created a pull request for this issue:

[jira] [Created] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-08-29 Thread Furcy Pin (JIRA)
Furcy Pin created SPARK-17296: - Summary: Spark SQL: cross join + two joins = BUG Key: SPARK-17296 URL: https://issues.apache.org/jira/browse/SPARK-17296 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-17295) Create TestHiveSessionState use reflect logic based on the setting of CATALOG_IMPLEMENTATION

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17295: Assignee: Apache Spark > Create TestHiveSessionState use reflect logic based on the

[jira] [Commented] (SPARK-17295) Create TestHiveSessionState use reflect logic based on the setting of CATALOG_IMPLEMENTATION

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446152#comment-15446152 ] Apache Spark commented on SPARK-17295: -- User 'jiangxb1987' has created a pull request for this

[jira] [Assigned] (SPARK-17295) Create TestHiveSessionState use reflect logic based on the setting of CATALOG_IMPLEMENTATION

2016-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17295: Assignee: (was: Apache Spark) > Create TestHiveSessionState use reflect logic based

[jira] [Commented] (SPARK-17291) The shuffle data fetched based on netty were directly stored in off-memoryr?

2016-08-29 Thread song fengfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446142#comment-15446142 ] song fengfei commented on SPARK-17291: -- Thanks very much,this is the first time to create issue,and

[jira] [Created] (SPARK-17295) Create TestHiveSessionState use reflect logic based on the setting of CATALOG_IMPLEMENTATION

2016-08-29 Thread Jiang Xingbo (JIRA)
Jiang Xingbo created SPARK-17295: Summary: Create TestHiveSessionState use reflect logic based on the setting of CATALOG_IMPLEMENTATION Key: SPARK-17295 URL: https://issues.apache.org/jira/browse/SPARK-17295

[jira] [Commented] (SPARK-17291) The shuffle data fetched based on netty were directly stored in off-memoryr?

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446119#comment-15446119 ] Sean Owen commented on SPARK-17291: --- I replied on the other JIRA. Questions go to

[jira] [Commented] (SPARK-17291) The shuffle data fetched based on netty were directly stored in off-memoryr?

2016-08-29 Thread song fengfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446110#comment-15446110 ] song fengfei commented on SPARK-17291: -- yeah, they are same,but neither was resolved and instead,

[jira] [Resolved] (SPARK-17294) Caching invalidates data on mildly wide dataframes

2016-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17294. --- Resolution: Duplicate Duplicate #5, popular issue > Caching invalidates data on mildly wide

[jira] [Created] (SPARK-17294) Caching invalidates data on mildly wide dataframes

2016-08-29 Thread Kalle Jepsen (JIRA)
Kalle Jepsen created SPARK-17294: Summary: Caching invalidates data on mildly wide dataframes Key: SPARK-17294 URL: https://issues.apache.org/jira/browse/SPARK-17294 Project: Spark Issue

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446059#comment-15446059 ] Takeshi Yamamuro commented on SPARK-17289: -- yea, I'll check this. > Sort based partial

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446013#comment-15446013 ] Herman van Hovell commented on SPARK-17289: --- cc [~maropu] > Sort based partial aggregation

[jira] [Commented] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15446015#comment-15446015 ] Herman van Hovell commented on SPARK-17289: --- [~clockfly] Are you working on this one? > Sort

[jira] [Updated] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17289: -- Priority: Blocker (was: Major) > Sort based partial aggregation breaks due to

[jira] [Closed] (SPARK-17293) seperate view handling from CreateTableCommand

2016-08-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-17293. --- Resolution: Invalid sorry my mistake > seperate view handling from CreateTableCommand >

[jira] [Created] (SPARK-17293) seperate view handling from CreateTableCommand

2016-08-29 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-17293: --- Summary: seperate view handling from CreateTableCommand Key: SPARK-17293 URL: https://issues.apache.org/jira/browse/SPARK-17293 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17289) Sort based partial aggregation breaks due to SPARK-12978

2016-08-29 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Zhong updated SPARK-17289: --- Description: For the following query: {code} val df2 = (0 to 1000).map(x => (x % 2,

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-08-29 Thread Vincent (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15445873#comment-15445873 ] Vincent commented on SPARK-17219: - Cool. I will refine the patch. thanks [~srowen] :) >

  1   2   >