[jira] [Commented] (SPARK-19536) Improve capability to merge SQL data types

2017-02-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860816#comment-15860816 ] Hyukjin Kwon commented on SPARK-19536: -- Thank you for kindly adding a link. > Improve capability to

[jira] [Assigned] (SPARK-17498) StringIndexer.setHandleInvalid should have another option 'new'

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17498: Assignee: Apache Spark > StringIndexer.setHandleInvalid should have another option 'new'

[jira] [Commented] (SPARK-17498) StringIndexer.setHandleInvalid should have another option 'new'

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860812#comment-15860812 ] Apache Spark commented on SPARK-17498: -- User 'VinceShieh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17498) StringIndexer.setHandleInvalid should have another option 'new'

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17498: Assignee: (was: Apache Spark) > StringIndexer.setHandleInvalid should have another

[jira] [Commented] (SPARK-19536) Improve capability to merge SQL data types

2017-02-09 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860748#comment-15860748 ] koert kuipers commented on SPARK-19536: --- great, that only leaves maps and nested structs then i

[jira] [Assigned] (SPARK-19544) Improve error message when some column types are compatible and others are not in set/union operations

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19544: Assignee: Apache Spark > Improve error message when some column types are compatible and

[jira] [Commented] (SPARK-19544) Improve error message when some column types are compatible and others are not in set/union operations

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860719#comment-15860719 ] Apache Spark commented on SPARK-19544: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-19544) Improve error message when some column types are compatible and others are not in set/union operations

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19544: Assignee: (was: Apache Spark) > Improve error message when some column types are

[jira] [Created] (SPARK-19544) Improve error message when some column types are compatible and others are not in set/union operations

2017-02-09 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-19544: Summary: Improve error message when some column types are compatible and others are not in set/union operations Key: SPARK-19544 URL:

[jira] [Updated] (SPARK-19544) Improve error message when some column types are compatible and others are not in set/union operations

2017-02-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19544: - Description: Currently, {code} Seq((1,("a", 1))).toDF union Seq((1L,("a", "b"))).toDF

[jira] [Assigned] (SPARK-19543) from_json fails when the input row is empty

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19543: Assignee: (was: Apache Spark) > from_json fails when the input row is empty >

[jira] [Assigned] (SPARK-19543) from_json fails when the input row is empty

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19543: Assignee: Apache Spark > from_json fails when the input row is empty >

[jira] [Commented] (SPARK-19543) from_json fails when the input row is empty

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860671#comment-15860671 ] Apache Spark commented on SPARK-19543: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Created] (SPARK-19543) from_json fails when the input row is empty

2017-02-09 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-19543: --- Summary: from_json fails when the input row is empty Key: SPARK-19543 URL: https://issues.apache.org/jira/browse/SPARK-19543 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19536) Improve capability to merge SQL data types

2017-02-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860665#comment-15860665 ] Hyukjin Kwon commented on SPARK-19536: -- Oh, for {{ArrayType}}, I filed a JIRA, SPARK-19435 and

[jira] [Commented] (SPARK-19542) Delete the temp checkpoint if a query is stopped without errors

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860661#comment-15860661 ] Apache Spark commented on SPARK-19542: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19542) Delete the temp checkpoint if a query is stopped without errors

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19542: Assignee: Apache Spark (was: Shixiong Zhu) > Delete the temp checkpoint if a query is

[jira] [Assigned] (SPARK-19542) Delete the temp checkpoint if a query is stopped without errors

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19542: Assignee: Shixiong Zhu (was: Apache Spark) > Delete the temp checkpoint if a query is

[jira] [Created] (SPARK-19542) Delete the temp checkpoint if a query is stopped without errors

2017-02-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-19542: Summary: Delete the temp checkpoint if a query is stopped without errors Key: SPARK-19542 URL: https://issues.apache.org/jira/browse/SPARK-19542 Project: Spark

[jira] [Updated] (SPARK-19540) Add ability to clone SparkSession wherein cloned session has a reference to SharedState and an identical copy of the SessionState

2017-02-09 Thread Kunal Khamar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19540: - Issue Type: Improvement (was: Story) > Add ability to clone SparkSession wherein cloned session

[jira] [Assigned] (SPARK-19541) High Availability support for ThriftServer

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19541: Assignee: Apache Spark > High Availability support for ThriftServer >

[jira] [Commented] (SPARK-19541) High Availability support for ThriftServer

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860545#comment-15860545 ] Apache Spark commented on SPARK-19541: -- User 'lvdongr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19541) High Availability support for ThriftServer

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19541: Assignee: (was: Apache Spark) > High Availability support for ThriftServer >

[jira] [Updated] (SPARK-19539) CREATE TEMPORARY TABLE needs to avoid existing temp view

2017-02-09 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-19539: --- Summary: CREATE TEMPORARY TABLE needs to avoid existing temp view (was: CREATE TEMPORARY TABLE need to

[jira] [Commented] (SPARK-19539) CREATE TEMPORARY TABLE need to avoid existing temp view

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860512#comment-15860512 ] Apache Spark commented on SPARK-19539: -- User 'xwu0226' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19539) CREATE TEMPORARY TABLE need to avoid existing temp view

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19539: Assignee: Apache Spark > CREATE TEMPORARY TABLE need to avoid existing temp view >

[jira] [Assigned] (SPARK-19539) CREATE TEMPORARY TABLE need to avoid existing temp view

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19539: Assignee: (was: Apache Spark) > CREATE TEMPORARY TABLE need to avoid existing temp

[jira] [Created] (SPARK-19541) High Availability support for ThriftServer

2017-02-09 Thread LvDeShui (JIRA)
LvDeShui created SPARK-19541: Summary: High Availability support for ThriftServer Key: SPARK-19541 URL: https://issues.apache.org/jira/browse/SPARK-19541 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-19540) Add ability to clone SparkSession wherein cloned session has a reference to SharedState and an identical copy of the SessionState

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19540: Assignee: Apache Spark > Add ability to clone SparkSession wherein cloned session has a

[jira] [Assigned] (SPARK-19540) Add ability to clone SparkSession wherein cloned session has a reference to SharedState and an identical copy of the SessionState

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19540: Assignee: (was: Apache Spark) > Add ability to clone SparkSession wherein cloned

[jira] [Commented] (SPARK-19540) Add ability to clone SparkSession wherein cloned session has a reference to SharedState and an identical copy of the SessionState

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860508#comment-15860508 ] Apache Spark commented on SPARK-19540: -- User 'kunalkhamar' has created a pull request for this

[jira] [Created] (SPARK-19540) Add ability to clone SparkSession wherein cloned session has a reference to SharedState and an identical copy of the SessionState

2017-02-09 Thread Kunal Khamar (JIRA)
Kunal Khamar created SPARK-19540: Summary: Add ability to clone SparkSession wherein cloned session has a reference to SharedState and an identical copy of the SessionState Key: SPARK-19540 URL:

[jira] [Created] (SPARK-19539) CREATE TEMPORARY TABLE need to avoid existing temp view

2017-02-09 Thread Xin Wu (JIRA)
Xin Wu created SPARK-19539: -- Summary: CREATE TEMPORARY TABLE need to avoid existing temp view Key: SPARK-19539 URL: https://issues.apache.org/jira/browse/SPARK-19539 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-19538) DAGScheduler and TaskSetManager can have an inconsistent view of whether a stage is complete.

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19538: Assignee: Apache Spark (was: Kay Ousterhout) > DAGScheduler and TaskSetManager can have

[jira] [Commented] (SPARK-19538) DAGScheduler and TaskSetManager can have an inconsistent view of whether a stage is complete.

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860427#comment-15860427 ] Apache Spark commented on SPARK-19538: -- User 'kayousterhout' has created a pull request for this

[jira] [Assigned] (SPARK-19538) DAGScheduler and TaskSetManager can have an inconsistent view of whether a stage is complete.

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19538: Assignee: Kay Ousterhout (was: Apache Spark) > DAGScheduler and TaskSetManager can have

[jira] [Assigned] (SPARK-19537) Move the pendingPartitions variable from Stage to ShuffleMapStage

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19537: Assignee: Kay Ousterhout (was: Apache Spark) > Move the pendingPartitions variable from

[jira] [Commented] (SPARK-19537) Move the pendingPartitions variable from Stage to ShuffleMapStage

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860417#comment-15860417 ] Apache Spark commented on SPARK-19537: -- User 'kayousterhout' has created a pull request for this

[jira] [Assigned] (SPARK-19537) Move the pendingPartitions variable from Stage to ShuffleMapStage

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19537: Assignee: Apache Spark (was: Kay Ousterhout) > Move the pendingPartitions variable from

[jira] [Created] (SPARK-19538) DAGScheduler and TaskSetManager can have an inconsistent view of whether a stage is complete.

2017-02-09 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-19538: -- Summary: DAGScheduler and TaskSetManager can have an inconsistent view of whether a stage is complete. Key: SPARK-19538 URL: https://issues.apache.org/jira/browse/SPARK-19538

[jira] [Created] (SPARK-19537) Move the pendingPartitions variable from Stage to ShuffleMapStage

2017-02-09 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-19537: -- Summary: Move the pendingPartitions variable from Stage to ShuffleMapStage Key: SPARK-19537 URL: https://issues.apache.org/jira/browse/SPARK-19537 Project: Spark

[jira] [Commented] (SPARK-5864) support .jar as python package

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860373#comment-15860373 ] Apache Spark commented on SPARK-5864: - User 'kunalkhamar' has created a pull request for this issue:

[jira] [Created] (SPARK-19536) Improve capability to merge SQL data types

2017-02-09 Thread koert kuipers (JIRA)
koert kuipers created SPARK-19536: - Summary: Improve capability to merge SQL data types Key: SPARK-19536 URL: https://issues.apache.org/jira/browse/SPARK-19536 Project: Spark Issue Type:

[jira] [Closed] (SPARK-16713) Limit codegen method size to 8KB

2017-02-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-16713. - Resolution: Won't Fix According to the discussion on PR, I think we can close this as WON'T FIX

[jira] [Commented] (SPARK-19525) Enable Compression of Spark Streaming Checkpoints

2017-02-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860236#comment-15860236 ] Shixiong Zhu commented on SPARK-19525: -- [~rameshaaditya117] Sounds a good idea. I thought the

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2017-02-09 Thread Srikanth Daggumalli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860220#comment-15860220 ] Srikanth Daggumalli commented on SPARK-13510: - OOM is not an issue. But would like to know

[jira] [Commented] (SPARK-10802) Let ALS recommend for subset of data

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860110#comment-15860110 ] Joseph K. Bradley commented on SPARK-10802: --- Linking related issue for feature parity in

[jira] [Created] (SPARK-19535) ALSModel recommendAll analogs

2017-02-09 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-19535: - Summary: ALSModel recommendAll analogs Key: SPARK-19535 URL: https://issues.apache.org/jira/browse/SPARK-19535 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-13857) Feature parity for ALS ML with MLLIB

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860108#comment-15860108 ] Joseph K. Bradley commented on SPARK-13857: --- Hi all, catching up these many ALS discussions

[jira] [Resolved] (SPARK-19509) GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19509. --- Resolution: Fixed Assignee: StanZhai Fix Version/s: 2.1.1

[jira] [Commented] (SPARK-19512) codegen for compare structs fails

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860053#comment-15860053 ] Apache Spark commented on SPARK-19512: -- User 'bogdanrdc' has created a pull request for this issue:

[jira] [Commented] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860028#comment-15860028 ] Shixiong Zhu commented on SPARK-19523: -- You can create a HiveContext before creating

[jira] [Commented] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-09 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860025#comment-15860025 ] Egor Pahomov commented on SPARK-19523: -- Probably my bad. I have {code} rdd.sparkContext {code}

[jira] [Commented] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-09 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860023#comment-15860023 ] Egor Pahomov commented on SPARK-19524: -- I'm really confused. I expected "new" to be the files

[jira] [Resolved] (SPARK-19481) Fix flaky test: o.a.s.repl.ReplSuite should clone and clean line object in ClosureCleaner

2017-02-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-19481. Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved by pull

[jira] [Closed] (SPARK-10141) Number of tasks on executors still become negative after failures

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-10141. - Resolution: Done > Number of tasks on executors still become negative after failures >

[jira] [Assigned] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-17975: - Assignee: Tathagata Das > EMLDAOptimizer fails with ClassCastException on YARN

[jira] [Resolved] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-17975. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1

[jira] [Updated] (SPARK-14804) Graph vertexRDD/EdgeRDD checkpoint results ClassCastException:

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14804: -- Fix Version/s: (was: 3.0.0) 2.2.0 > Graph vertexRDD/EdgeRDD

[jira] [Resolved] (SPARK-16554) Spark should kill executors when they are blacklisted

2017-02-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-16554. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16650

[jira] [Commented] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859984#comment-15859984 ] Joseph K. Bradley commented on SPARK-17975: --- Will do, thanks! > EMLDAOptimizer fails with

[jira] [Commented] (SPARK-10141) Number of tasks on executors still become negative after failures

2017-02-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859982#comment-15859982 ] Joseph K. Bradley commented on SPARK-10141: --- I'll close this if no one has seen it in Spark 2.0

[jira] [Resolved] (SPARK-19025) Remove SQL builder for operators

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19025. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-19512) codegen for compare structs fails

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19512. --- Resolution: Fixed Assignee: Bogdan Raducanu Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-19514) Range is not interruptible

2017-02-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19514. - Resolution: Fixed Assignee: Ala Luszczak Fix Version/s: 2.2.0 > Range is not

[jira] [Commented] (SPARK-18005) optional binary Dataframe Column throws (UTF8) is not a group while loading a Dataframe

2017-02-09 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859913#comment-15859913 ] Eric Maynard commented on SPARK-18005: -- This bug appears in the 1.6.x branch as well. > optional

[jira] [Comment Edited] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

2017-02-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859795#comment-15859795 ] Christophe Préaud edited comment on SPARK-19477 at 2/9/17 5:01 PM: --- As

[jira] [Issue Comment Deleted] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

2017-02-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe Préaud updated SPARK-19477: -- Comment: was deleted (was: As a workaround, mapping the dataset with *identity* fix

[jira] [Comment Edited] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

2017-02-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859795#comment-15859795 ] Christophe Préaud edited comment on SPARK-19477 at 2/9/17 5:00 PM: --- As

[jira] [Comment Edited] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

2017-02-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859795#comment-15859795 ] Christophe Préaud edited comment on SPARK-19477 at 2/9/17 4:59 PM: --- As

[jira] [Commented] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

2017-02-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859795#comment-15859795 ] Christophe Préaud commented on SPARK-19477: --- As a workaround, mapping the dataset with

[jira] [Commented] (SPARK-19348) pyspark.ml.Pipeline gets corrupted under multi threaded use

2017-02-09 Thread Peter D Kirchner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859737#comment-15859737 ] Peter D Kirchner commented on SPARK-19348: -- Per the above, perhaps a fix could address both

[jira] [Deleted] (SPARK-19494) Remove java7 pom profile

2017-02-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin deleted SPARK-19494: > Remove java7 pom profile > > > Key: SPARK-19494 >

[jira] [Updated] (SPARK-19494) Remove java7 pom profile

2017-02-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19494: -- Priority: Trivial (was: Major) (I think this part is pretty trivial and covered in the base change

[jira] [Updated] (SPARK-19534) Convert Java tests to use lambdas, Java 8 features

2017-02-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19534: -- Issue Type: Sub-task (was: Task) Parent: SPARK-19493 > Convert Java tests to use lambdas,

[jira] [Created] (SPARK-19534) Convert Java tests to use lambdas, Java 8 features

2017-02-09 Thread Sean Owen (JIRA)
Sean Owen created SPARK-19534: - Summary: Convert Java tests to use lambdas, Java 8 features Key: SPARK-19534 URL: https://issues.apache.org/jira/browse/SPARK-19534 Project: Spark Issue Type:

[jira] [Created] (SPARK-19533) Convert Java examples to use lambdas, Java 8 features

2017-02-09 Thread Sean Owen (JIRA)
Sean Owen created SPARK-19533: - Summary: Convert Java examples to use lambdas, Java 8 features Key: SPARK-19533 URL: https://issues.apache.org/jira/browse/SPARK-19533 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19533) Convert Java examples to use lambdas, Java 8 features

2017-02-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19533: -- Issue Type: Sub-task (was: Task) Parent: SPARK-19493 > Convert Java examples to use lambdas,

[jira] [Created] (SPARK-19532) [Core]`DataStreamer for file` threads of DFSOutputStream leak if set `spark.speculation` to true

2017-02-09 Thread StanZhai (JIRA)
StanZhai created SPARK-19532: Summary: [Core]`DataStreamer for file` threads of DFSOutputStream leak if set `spark.speculation` to true Key: SPARK-19532 URL: https://issues.apache.org/jira/browse/SPARK-19532

[jira] [Updated] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19509: -- Priority: Major (was: Critical) > [SQL]GROUPING SETS throws NullPointerException when

[jira] [Updated] (SPARK-19509) GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19509: -- Summary: GROUPING SETS throws NullPointerException when use an empty column (was:

[jira] [Commented] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859541#comment-15859541 ] Apache Spark commented on SPARK-19509: -- User 'stanzhai' has created a pull request for this issue:

[jira] [Commented] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859528#comment-15859528 ] Apache Spark commented on SPARK-19509: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19509: Assignee: (was: Apache Spark) > [SQL]GROUPING SETS throws NullPointerException when

[jira] [Assigned] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19509: Assignee: Apache Spark > [SQL]GROUPING SETS throws NullPointerException when use an empty

[jira] [Commented] (SPARK-19514) Range is not interruptible

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859499#comment-15859499 ] Apache Spark commented on SPARK-19514: -- User 'ala' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19514) Range is not interruptible

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19514: Assignee: Apache Spark > Range is not interruptible > -- > >

[jira] [Assigned] (SPARK-19514) Range is not interruptible

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19514: Assignee: (was: Apache Spark) > Range is not interruptible >

[jira] [Resolved] (SPARK-17874) Additional SSL port on HistoryServer should be configurable

2017-02-09 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-17874. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.2.0 >

[jira] [Created] (SPARK-19531) History server doesn't refresh jobs for long-life apps like thriftserver

2017-02-09 Thread Oleg Danilov (JIRA)
Oleg Danilov created SPARK-19531: Summary: History server doesn't refresh jobs for long-life apps like thriftserver Key: SPARK-19531 URL: https://issues.apache.org/jira/browse/SPARK-19531 Project:

[jira] [Commented] (SPARK-19416) Dataset.schema is inconsistent with Dataset in handling columns with periods

2017-02-09 Thread Thomas Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859401#comment-15859401 ] Thomas Sebastian commented on SPARK-19416: -- [~josephkb] If I understand correctly the consistent

[jira] [Comment Edited] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-02-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859398#comment-15859398 ] Nattavut Sutyanyong edited comment on SPARK-18874 at 2/9/17 11:48 AM:

[jira] [Comment Edited] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-02-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859398#comment-15859398 ] Nattavut Sutyanyong edited comment on SPARK-18874 at 2/9/17 11:49 AM:

[jira] [Commented] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-02-09 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859398#comment-15859398 ] Nattavut Sutyanyong commented on SPARK-18874: - Thank you, [~hvanhovell][~dkbiswal], for

[jira] [Commented] (SPARK-19493) Remove Java 7 support

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859369#comment-15859369 ] Apache Spark commented on SPARK-19493: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19493) Remove Java 7 support

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19493: Assignee: Reynold Xin (was: Apache Spark) > Remove Java 7 support >

[jira] [Assigned] (SPARK-19493) Remove Java 7 support

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19493: Assignee: Apache Spark (was: Reynold Xin) > Remove Java 7 support >

[jira] [Commented] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859342#comment-15859342 ] Sean Owen commented on SPARK-19523: --- Not sure what this means -- there is one HiveContext per driver

[jira] [Commented] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859303#comment-15859303 ] Sean Owen commented on SPARK-19524: --- Well, that is its definition of 'new' -- what do you expect

[jira] [Assigned] (SPARK-19496) to_date with format has weird behavior

2017-02-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19496: Assignee: (was: Apache Spark) > to_date with format has weird behavior >

  1   2   >