[jira] [Updated] (SPARK-12291) Support UnsafeRow in BroadcastLeftSemiJoinHash

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12291: -- Component/s: SQL > Support UnsafeRow in BroadcastLeftSemiJoinHash >

[jira] [Updated] (SPARK-12284) Output UnsafeRow from window function

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12284: -- Component/s: SQL > Output UnsafeRow from window function > - > >

[jira] [Updated] (SPARK-12295) Manage the memory used by window function

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12295: -- Component/s: SQL > Manage the memory used by window function >

[jira] [Updated] (SPARK-12289) Support UnsafeRow in TakeOrderedAndProject/Limit

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12289: -- Component/s: SQL > Support UnsafeRow in TakeOrderedAndProject/Limit >

[jira] [Updated] (SPARK-12287) Support UnsafeRow in MapPartitions/MapGroups/CoGroup

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12287: -- Component/s: SQL > Support UnsafeRow in MapPartitions/MapGroups/CoGroup >

[jira] [Resolved] (SPARK-11707) StreamCorruptedException if authentication is enabled

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11707. --- Resolution: Cannot Reproduce > StreamCorruptedException if authentication is enabled >

[jira] [Commented] (SPARK-12260) Graceful Shutdown with In-Memory State

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054923#comment-15054923 ] Sean Owen commented on SPARK-12260: --- I'm asking what you can't do with updateStateByKey that you need

[jira] [Resolved] (SPARK-12238) s/Advanced sources/External Sources in docs.

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12238. --- Resolution: Won't Fix > s/Advanced sources/External Sources in docs. >

[jira] [Updated] (SPARK-12290) Change the default value in SparkPlan

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12290: -- Component/s: SQL > Change the default value in SparkPlan > - > >

[jira] [Updated] (SPARK-12292) Support UnsafeRow in Generate

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12292: -- Component/s: SQL > Support UnsafeRow in Generate > - > >

[jira] [Updated] (SPARK-12294) Support UnsafeRow in HiveTableScan

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12294: -- Component/s: SQL > Support UnsafeRow in HiveTableScan > -- > >

[jira] [Updated] (SPARK-12293) Support UnsafeRow in LocalTableScan

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12293: -- Component/s: SQL > Support UnsafeRow in LocalTableScan > --- > >

[jira] [Updated] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12288: -- Component/s: SQL > Support UnsafeRow in Coalesce/Except/Intersect >

[jira] [Resolved] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12306. --- Resolution: Invalid > Add an option to ignore BlockRDD partition data loss >

[jira] [Commented] (SPARK-12282) Document spark.jars

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054927#comment-15054927 ] Sean Owen commented on SPARK-12282: --- Yeah, I suspect it happens to work fine, since that's what --jars

[jira] [Resolved] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12305. --- Resolution: Invalid > Add Receiver scheduling info onto Spark Streaming web UI >

[jira] [Created] (SPARK-12310) Add write.json and write.parquet for SparkR

2015-12-13 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-12310: --- Summary: Add write.json and write.parquet for SparkR Key: SPARK-12310 URL: https://issues.apache.org/jira/browse/SPARK-12310 Project: Spark Issue Type:

[jira] [Created] (SPARK-12315) isnotnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-12315: Summary: isnotnull operator not pushed down for JDBC datasource. Key: SPARK-12315 URL: https://issues.apache.org/jira/browse/SPARK-12315 Project: Spark

[jira] [Commented] (SPARK-12314) isnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055344#comment-15055344 ] Hyukjin Kwon commented on SPARK-12314: -- I will work on this. > isnull operator not pushed down for

[jira] [Commented] (SPARK-12315) isnotnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055347#comment-15055347 ] Hyukjin Kwon commented on SPARK-12315: -- I will work on this. > isnotnull operator not pushed down

[jira] [Assigned] (SPARK-12314) isnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12314: Assignee: Apache Spark > isnull operator not pushed down for JDBC datasource. >

[jira] [Commented] (SPARK-12314) isnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055356#comment-15055356 ] Apache Spark commented on SPARK-12314: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-12314) isnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12314: Assignee: (was: Apache Spark) > isnull operator not pushed down for JDBC datasource.

[jira] [Commented] (SPARK-12057) Prevent failure on corrupt JSON records

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055414#comment-15055414 ] Apache Spark commented on SPARK-12057: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12310) Add write.json and write.parquet for SparkR

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12310: Assignee: Apache Spark > Add write.json and write.parquet for SparkR >

[jira] [Updated] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

2015-12-13 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Description: Spark 1.5 has added better Receiver scheduling support, via which users can deploy

[jira] [Updated] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

2015-12-13 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Description: Spark 1.5 has added better Receiver scheduling support, > Add Receiver scheduling info

[jira] [Created] (SPARK-12311) [CORE] Restore previous "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12311: Summary: [CORE] Restore previous "os.arch" property in test suites after forcing to set specific value to "os.arch" property Key: SPARK-12311 URL:

[jira] [Updated] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12311: - Summary: [CORE] Restore previous value of "os.arch" property in test suites after

[jira] [Assigned] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12309: Assignee: (was: Apache Spark) > Use sqlContext from MLlibTestSparkContext for

[jira] [Assigned] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12309: Assignee: Apache Spark > Use sqlContext from MLlibTestSparkContext for spark.ml test

[jira] [Updated] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12311: - Description: Although current BlockManagerSuite.scala and SizeEstimatorSuite.scala set

[jira] [Commented] (SPARK-12293) Support UnsafeRow in LocalTableScan

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055019#comment-15055019 ] Apache Spark commented on SPARK-12293: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12293) Support UnsafeRow in LocalTableScan

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12293: Assignee: Apache Spark > Support UnsafeRow in LocalTableScan >

[jira] [Assigned] (SPARK-12293) Support UnsafeRow in LocalTableScan

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12293: Assignee: (was: Apache Spark) > Support UnsafeRow in LocalTableScan >

[jira] [Assigned] (SPARK-12293) Support UnsafeRow in LocalTableScan

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12293: Assignee: Apache Spark > Support UnsafeRow in LocalTableScan >

[jira] [Created] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2015-12-13 Thread nabacg (JIRA)
nabacg created SPARK-12312: -- Summary: JDBC connection to Kerberos secured databases fails on remote executors Key: SPARK-12312 URL: https://issues.apache.org/jira/browse/SPARK-12312 Project: Spark

[jira] [Commented] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

2015-12-13 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054983#comment-15054983 ] Liwei Lin commented on SPARK-12305: --- Sorry for the inconvenience; I've added some detailed description,

[jira] [Updated] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

2015-12-13 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12306: -- Description: Currently in Spark Streaming, a Receiver stores the received data into some BlockManager

[jira] [Updated] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

2015-12-13 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12306: -- Description: Currently in Spark Streaming, a Receiver stores the received data into some BlockManager

[jira] [Commented] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

2015-12-13 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054995#comment-15054995 ] Liwei Lin commented on SPARK-12306: --- Sorry for the inconvenience; I've added some detailed description,

[jira] [Commented] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055073#comment-15055073 ] Sean Owen commented on SPARK-12311: --- LGTM, feel free to make a pull request. > [CORE] Restore previous

[jira] [Commented] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055110#comment-15055110 ] Xiao Li commented on SPARK-12288: - [~davies] It sounds like Except and Intersect can support UnsafeRow?

[jira] [Updated] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2015-12-13 Thread nabacg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nabacg updated SPARK-12312: --- Description: When loading DataFrames from JDBC datasource with Kerberos authentication, remote executors

[jira] [Comment Edited] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055110#comment-15055110 ] Xiao Li edited comment on SPARK-12288 at 12/13/15 8:27 PM: --- [~davies] It sounds

[jira] [Closed] (SPARK-10798) JsonMappingException with Spark Context Parallelize

2015-12-13 Thread Dev Lakhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dev Lakhani closed SPARK-10798. --- Resolution: Cannot Reproduce > JsonMappingException with Spark Context Parallelize >

[jira] [Commented] (SPARK-10798) JsonMappingException with Spark Context Parallelize

2015-12-13 Thread Dev Lakhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055130#comment-15055130 ] Dev Lakhani commented on SPARK-10798: - byte[] data= Kryo.serialize(List) This is just shorthand for

[jira] [Commented] (SPARK-11410) Add a DataFrame API that provides functionality similar to HiveQL's DISTRIBUTE BY

2015-12-13 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055182#comment-15055182 ] Maciej Bryński commented on SPARK-11410: Is it possible for Spark to use information about

[jira] [Comment Edited] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055110#comment-15055110 ] Xiao Li edited comment on SPARK-12288 at 12/13/15 9:49 PM: --- [~davies] It sounds

[jira] [Commented] (SPARK-11410) Add a DataFrame API that provides functionality similar to HiveQL's DISTRIBUTE BY

2015-12-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055198#comment-15055198 ] Yin Huai commented on SPARK-11410: -- oh, i see. This is the table partitioning mechanism. If you use

[jira] [Updated] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2015-12-13 Thread Gobinathan SP (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gobinathan SP updated SPARK-12313: -- Description: When enabled spark.sql.hive.metastorePartitionPruning, the getPartitionsByFilter

[jira] [Commented] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055207#comment-15055207 ] Apache Spark commented on SPARK-12062: -- User 'BryanCutler' has created a pull request for this

[jira] [Commented] (SPARK-11410) Add a DataFrame API that provides functionality similar to HiveQL's DISTRIBUTE BY

2015-12-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055188#comment-15055188 ] Yin Huai commented on SPARK-11410: -- Yes, we do. For example, if you cache the table after call

[jira] [Created] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2015-12-13 Thread Gobinathan SP (JIRA)
Gobinathan SP created SPARK-12313: - Summary: getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns Key: SPARK-12313 URL: https://issues.apache.org/jira/browse/SPARK-12313

[jira] [Updated] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2015-12-13 Thread Gobinathan SP (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gobinathan SP updated SPARK-12313: -- Description: When enabled spark.sql.hive.metastorePartitionPruning, the getPartitionsByFilter

[jira] [Updated] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

2015-12-13 Thread Gobinathan SP (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gobinathan SP updated SPARK-12313: -- Description: When enabled spark.sql.hive.metastorePartitionPruning, the getPartitionsByFilter

[jira] [Assigned] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12288: Assignee: Apache Spark > Support UnsafeRow in Coalesce/Except/Intersect >

[jira] [Issue Comment Deleted] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12288: Comment: was deleted (was: [~davies] It sounds like Except and Intersect can support UnsafeRow? We just

[jira] [Commented] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055256#comment-15055256 ] Apache Spark commented on SPARK-12288: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12288: Assignee: (was: Apache Spark) > Support UnsafeRow in Coalesce/Except/Intersect >

[jira] [Comment Edited] (SPARK-11410) Add a DataFrame API that provides functionality similar to HiveQL's DISTRIBUTE BY

2015-12-13 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055189#comment-15055189 ] Maciej Bryński edited comment on SPARK-11410 at 12/13/15 10:16 PM: --- And

[jira] [Commented] (SPARK-11410) Add a DataFrame API that provides functionality similar to HiveQL's DISTRIBUTE BY

2015-12-13 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055189#comment-15055189 ] Maciej Bryński commented on SPARK-11410: And what about such example. We can partition table by

[jira] [Commented] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

2015-12-13 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055205#comment-15055205 ] Bryan Cutler commented on SPARK-12062: -- I read the past conversations discussing this in the related

[jira] [Commented] (SPARK-12180) DataFrame.join() in PySpark gives misleading exception when column name exists on both side

2015-12-13 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055429#comment-15055429 ] Jeff Zhang commented on SPARK-12180: Could you paste your code ? It works fine for me to join 2

[jira] [Assigned] (SPARK-12315) isnotnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12315: Assignee: (was: Apache Spark) > isnotnull operator not pushed down for JDBC

[jira] [Commented] (SPARK-12315) isnotnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055358#comment-15055358 ] Apache Spark commented on SPARK-12315: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-12315) isnotnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12315: Assignee: Apache Spark > isnotnull operator not pushed down for JDBC datasource. >

[jira] [Created] (SPARK-12314) isnull operator not pushed down for JDBC datasource.

2015-12-13 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-12314: Summary: isnull operator not pushed down for JDBC datasource. Key: SPARK-12314 URL: https://issues.apache.org/jira/browse/SPARK-12314 Project: Spark Issue

[jira] [Created] (SPARK-12317) Value should be configurated with unit(e.g. kb/mb/gb) in SQL

2015-12-13 Thread Yadong Qi (JIRA)
Yadong Qi created SPARK-12317: - Summary: Value should be configurated with unit(e.g. kb/mb/gb) in SQL Key: SPARK-12317 URL: https://issues.apache.org/jira/browse/SPARK-12317 Project: Spark

[jira] [Updated] (SPARK-12317) Support configurate value with unit(e.g. kb/mb/gb) in SQL

2015-12-13 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-12317: -- Summary: Support configurate value with unit(e.g. kb/mb/gb) in SQL (was: Value should be configurated

[jira] [Created] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2015-12-13 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-12316: Summary: Stack overflow with endless call of `Delegation token thread` when application end. Key: SPARK-12316 URL: https://issues.apache.org/jira/browse/SPARK-12316

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2015-12-13 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-12316: - Description: When application end, AM will clean the staging dir. But if the driver trigger to

[jira] [Updated] (SPARK-12275) No plan for BroadcastHint in some condition

2015-12-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12275: - Assignee: yucai > No plan for BroadcastHint in some condition >

[jira] [Resolved] (SPARK-12281) Fixed potential exceptions when exiting a local cluster.

2015-12-13 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12281. -- Resolution: Fixed Assignee: Shixiong Zhu (was: Apache Spark) Fix Version/s:

[jira] [Commented] (SPARK-12176) SparkLauncher's setConf() does not support configs containing spaces

2015-12-13 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055508#comment-15055508 ] Saisai Shao commented on SPARK-12176: - It is OK in my local test against latest master branch, seems

[jira] [Resolved] (SPARK-12213) Query with only one distinct should not having on expand

2015-12-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-12213. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10228

[jira] [Updated] (SPARK-12275) No plan for BroadcastHint in some condition

2015-12-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12275: - Fix Version/s: 2.0.0 1.6.1 > No plan for BroadcastHint in some condition >

[jira] [Updated] (SPARK-12275) No plan for BroadcastHint in some condition

2015-12-13 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12275: - Target Version/s: 1.5.3 > No plan for BroadcastHint in some condition >

[jira] [Assigned] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12311: Assignee: (was: Apache Spark) > [CORE] Restore previous value of "os.arch" property

[jira] [Assigned] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12311: Assignee: Apache Spark > [CORE] Restore previous value of "os.arch" property in test

[jira] [Commented] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055578#comment-15055578 ] Apache Spark commented on SPARK-12311: -- User 'kiszk' has created a pull request for this issue: