[jira] [Commented] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074137#comment-15074137 ] Apache Spark commented on SPARK-12362: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Updated] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12554: -- Priority: Major (was: Critical) > Standalone app scheduler will hang when app.coreToAssign <

[jira] [Created] (SPARK-12559) Standalone cluster mode doesn't work with --packages

2015-12-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-12559: - Summary: Standalone cluster mode doesn't work with --packages Key: SPARK-12559 URL: https://issues.apache.org/jira/browse/SPARK-12559 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074189#comment-15074189 ] Andrew Or commented on SPARK-12554: --- I've downgraded the issue. I also think the behavior is by

[jira] [Commented] (SPARK-12538) bucketed table support

2015-12-29 Thread Nong Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074114#comment-15074114 ] Nong Li commented on SPARK-12538: - I posted the design here:

[jira] [Resolved] (SPARK-12530) Build break at Spark-Master-Maven-Snapshots from #1293

2015-12-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-12530. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10488

[jira] [Commented] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074001#comment-15074001 ] Sean Owen commented on SPARK-4819: -- OK, I wrestled with this a couple hours and think this will fail: the

[jira] [Resolved] (SPARK-12551) Not able to load CSV package

2015-12-29 Thread chintan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chintan resolved SPARK-12551. - Resolution: Fixed Fix Version/s: 1.5.2 Target Version/s: 1.5.2 for the above i finally

[jira] [Updated] (SPARK-12530) Build break at Spark-Master-Maven-Snapshots from #1293

2015-12-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12530: - Assignee: Kazuaki Ishizaki (was: Apache Spark) > Build break at Spark-Master-Maven-Snapshots from #1293

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

2015-12-29 Thread Luciano Resende (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074085#comment-15074085 ] Luciano Resende commented on SPARK-5159: Is this still an issue ? Most of the code on the initial

[jira] [Updated] (SPARK-12558) AnalysisException when multiple functions applied in GROUP BY clause

2015-12-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej BryƄski updated SPARK-12558: --- Description: Hi, I have following issue when trying to use functions in group by clause.

[jira] [Resolved] (SPARK-12551) Not able to load CSV package

2015-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12551. --- Resolution: Not A Problem Target Version/s: (was: 1.5.2) > Not able to load CSV package

[jira] [Reopened] (SPARK-12551) Not able to load CSV package

2015-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-12551: --- > Not able to load CSV package > > > Key: SPARK-12551 >

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2015-12-29 Thread Mario Briggs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074131#comment-15074131 ] Mario Briggs commented on SPARK-12177: -- with regards to review comment 'C' (2 comments above), the

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12511: - Affects Version/s: 1.6.0 > streaming driver with checkpointing unable to finalize leading to OOM

[jira] [Updated] (SPARK-12559) Standalone cluster mode doesn't work with --packages

2015-12-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12559: -- Description: >From the mailing list: {quote} Another problem I ran into that you also might is that

[jira] [Updated] (SPARK-12507) Expose closeFileAfterWrite and allowBatching configurations for Streaming

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12507: - Summary: Expose closeFileAfterWrite and allowBatching configurations for Streaming (was: Update

[jira] [Created] (SPARK-12566) GLM model family, link function support

2015-12-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-12566: - Summary: GLM model family, link function support Key: SPARK-12566 URL: https://issues.apache.org/jira/browse/SPARK-12566 Project: Spark Issue

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: Apache Spark > SqlTestUtils.stripSparkFilter needs to copy utf8strings >

[jira] [Commented] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074427#comment-15074427 ] Shixiong Zhu commented on SPARK-12511: -- I submitted a PR to py4j to fix this issue:

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: Apache Spark > SqlTestUtils.stripSparkFilter needs to copy utf8strings >

[jira] [Created] (SPARK-12568) Add BINARY to Encoders

2015-12-29 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-12568: Summary: Add BINARY to Encoders Key: SPARK-12568 URL: https://issues.apache.org/jira/browse/SPARK-12568 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-12568) Add BINARY to Encoders

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074499#comment-15074499 ] Apache Spark commented on SPARK-12568: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Resolved] (SPARK-12549) UDFs' input type specification should take Option[Seq[DataType]] rather than Seq[DataType]

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12549. - Resolution: Fixed Assignee: Reynold Xin (was: Apache Spark) Fix Version/s: 2.0.0

[jira] [Comment Edited] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074543#comment-15074543 ] Lijie Xu edited comment on SPARK-12554 at 12/30/15 2:11 AM: I agree that it

[jira] [Created] (SPARK-12572) Initial import of the Hive parser

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12572: --- Summary: Initial import of the Hive parser Key: SPARK-12572 URL: https://issues.apache.org/jira/browse/SPARK-12572 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: (was: Apache Spark) > SqlTestUtils.stripSparkFilter needs to copy

[jira] [Commented] (SPARK-11745) Enable more JSON parsing options for parsing non-standard JSON files

2015-12-29 Thread Cazen Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074445#comment-15074445 ] Cazen Lee commented on SPARK-11745: --- Good Day [~rxin] This is Cazen I'm sorry for asking question, but

[jira] [Assigned] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12511: Assignee: Shixiong Zhu (was: Apache Spark) > streaming driver with checkpointing unable

[jira] [Created] (SPARK-12570) DecisionTreeRegressor: provide variance of prediction: user guide update

2015-12-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-12570: - Summary: DecisionTreeRegressor: provide variance of prediction: user guide update Key: SPARK-12570 URL: https://issues.apache.org/jira/browse/SPARK-12570

[jira] [Created] (SPARK-12569) DecisionTreeRegressor: provide variance of prediction: Python API

2015-12-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-12569: - Summary: DecisionTreeRegressor: provide variance of prediction: Python API Key: SPARK-12569 URL: https://issues.apache.org/jira/browse/SPARK-12569 Project:

[jira] [Updated] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijie Xu updated SPARK-12554: - Description: In scheduleExecutorsOnWorker() in Master.scala, *val keepScheduling = coresToAssign >=

[jira] [Updated] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijie Xu updated SPARK-12554: - Description: In scheduleExecutorsOnWorker() in Master.scala, {{val keepScheduling = coresToAssign >=

[jira] [Resolved] (SPARK-12002) offsetRanges attribute missing in Kafka RDD when resuming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12002. -- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 1.6.0 > offsetRanges

[jira] [Created] (SPARK-12567) Add aes_{encrypt,decrypt} UDFs

2015-12-29 Thread Kai Jiang (JIRA)
Kai Jiang created SPARK-12567: - Summary: Add aes_{encrypt,decrypt} UDFs Key: SPARK-12567 URL: https://issues.apache.org/jira/browse/SPARK-12567 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074496#comment-15074496 ] Apache Spark commented on SPARK-12511: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-12558) AnalysisException when multiple functions applied in GROUP BY clause

2015-12-29 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074508#comment-15074508 ] Dilip Biswal commented on SPARK-12558: -- I would like to work on this one. > AnalysisException when

[jira] [Commented] (SPARK-12567) Add aes_{encrypt,decrypt} UDFs

2015-12-29 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074511#comment-15074511 ] Kai Jiang commented on SPARK-12567: --- already working on it. > Add aes_{encrypt,decrypt} UDFs >

[jira] [Assigned] (SPARK-12568) Add BINARY to Encoders

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12568: Assignee: Apache Spark (was: Michael Armbrust) > Add BINARY to Encoders >

[jira] [Commented] (SPARK-6724) Model import/export for FPGrowth

2015-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074512#comment-15074512 ] Joseph K. Bradley commented on SPARK-6724: -- Thanks for your patience. I'll take a look now! >

[jira] [Commented] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074543#comment-15074543 ] Lijie Xu commented on SPARK-12554: -- I agree that it is a design problem.

[jira] [Commented] (SPARK-12564) Improve missing column AnalysisException

2015-12-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074546#comment-15074546 ] Xiao Li commented on SPARK-12564: - Let me push one PR: https://github.com/apache/spark/pull/10518 >

[jira] [Updated] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12362: Priority: Critical (was: Major) > Create a full-fledged built-in SQL parser >

[jira] [Commented] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074574#comment-15074574 ] Reynold Xin commented on SPARK-12362: - [~hvanhovell] please create additional subtasks as part of

[jira] [Resolved] (SPARK-12572) Initial import of the Hive parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12572. - Resolution: Fixed Fix Version/s: 2.0.0 > Initial import of the Hive parser >

[jira] [Created] (SPARK-12573) Add acknowledge that the parser was initially from Hive

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12573: --- Summary: Add acknowledge that the parser was initially from Hive Key: SPARK-12573 URL: https://issues.apache.org/jira/browse/SPARK-12573 Project: Spark Issue

[jira] [Updated] (SPARK-12572) Initial import of the Hive parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12572: Description: Pull request: https://github.com/apache/spark/pull/10509 > Initial import of the Hive

[jira] [Created] (SPARK-12574) Move parser from hive module to catalyst (or sql-core) module

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12574: --- Summary: Move parser from hive module to catalyst (or sql-core) module Key: SPARK-12574 URL: https://issues.apache.org/jira/browse/SPARK-12574 Project: Spark

[jira] [Updated] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12362: Assignee: Herman van Hovell (was: Apache Spark) > Create a full-fledged built-in SQL parser >

[jira] [Comment Edited] (SPARK-10789) Cluster mode SparkSubmit classpath only includes Spark assembly

2015-12-29 Thread Jonathan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074225#comment-15074225 ] Jonathan Kelly edited comment on SPARK-10789 at 12/29/15 8:13 PM: -- Yes,

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

2015-12-29 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074268#comment-15074268 ] Maciej Szymkiewicz commented on SPARK-6459: --- [~marmbrus] Isn't this warning obsolete in 1.5+? >

[jira] [Created] (SPARK-12565) Document standalone mode jar distribution more clearly

2015-12-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-12565: - Summary: Document standalone mode jar distribution more clearly Key: SPARK-12565 URL: https://issues.apache.org/jira/browse/SPARK-12565 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12562) DataFrame.write.format("text") requires the column name to be called value

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074456#comment-15074456 ] Apache Spark commented on SPARK-12562: -- User 'xguo27' has created a pull request for this issue:

[jira] [Commented] (SPARK-10638) spark streaming stop gracefully keeps the spark context

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074467#comment-15074467 ] Shixiong Zhu commented on SPARK-10638: -- Do you have any stack trace or reproduce codes? Maybe there

[jira] [Resolved] (SPARK-12004) RDD checkpointing does not preserve partitioner

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12004. -- Resolution: Fixed Fix Version/s: 1.6.0 > RDD checkpointing does not preserve

[jira] [Created] (SPARK-12571) AWS credentials not available for read.parquet in SQLContext

2015-12-29 Thread Kostiantyn Kudriavtsev (JIRA)
Kostiantyn Kudriavtsev created SPARK-12571: -- Summary: AWS credentials not available for read.parquet in SQLContext Key: SPARK-12571 URL: https://issues.apache.org/jira/browse/SPARK-12571

[jira] [Updated] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12148: Target Version/s: 2.0.0 > SparkR: rename DataFrame to SparkDataFrame >

[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074570#comment-15074570 ] Reynold Xin commented on SPARK-12148: - +1 We should consider just renaming this to SparkDataFrame in

[jira] [Assigned] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12511: Assignee: Apache Spark (was: Shixiong Zhu) > streaming driver with checkpointing unable

[jira] [Resolved] (SPARK-12407) ClassCast Exception when restarting spark streaming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12407. -- Resolution: Duplicate Fix Version/s: SPARK-5206 The cause is Streaming checkpointing

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

2015-12-29 Thread Sen Fang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074462#comment-15074462 ] Sen Fang commented on SPARK-5159: - As of Spark 1.5.2, we have a similar issue that might be related to

[jira] [Comment Edited] (SPARK-12407) ClassCast Exception when restarting spark streaming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074461#comment-15074461 ] Shixiong Zhu edited comment on SPARK-12407 at 12/30/15 12:08 AM: - The

[jira] [Updated] (SPARK-12407) ClassCast Exception when restarting spark streaming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12407: - Fix Version/s: (was: SPARK-5206) > ClassCast Exception when restarting spark streaming from

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

2015-12-29 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074322#comment-15074322 ] Maciej Szymkiewicz commented on SPARK-6459: --- I've been trying to reproduce the problem on 1.5.2

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

2015-12-29 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074342#comment-15074342 ] Maciej Szymkiewicz commented on SPARK-6459: --- Thanks for clarification. > Warn when Column API

[jira] [Commented] (SPARK-12542) Support union/intersect/except in Hive SQL

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074368#comment-15074368 ] Davies Liu commented on SPARK-12542: [~viirya] [~xiaol] That's true, updated the title. Once

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Assignee: Xiao Li > Support union/intersect/except in Hive SQL >

[jira] [Commented] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074380#comment-15074380 ] Apache Spark commented on SPARK-4819: - User 'srowen' has created a pull request for this issue:

[jira] [Closed] (SPARK-12415) Do not use closure serializer to serialize task result

2015-12-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-12415. - Resolution: Won't Fix > Do not use closure serializer to serialize task result >

[jira] [Created] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-12560: Summary: SqlTestUtils.stripSparkFilter needs to copy utf8strings Key: SPARK-12560 URL: https://issues.apache.org/jira/browse/SPARK-12560 Project: Spark

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2015-12-29 Thread Nikita Tarasenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074262#comment-15074262 ] Nikita Tarasenko commented on SPARK-12177: -- Hi, Mario! Thank you for review! It very helpful for

[jira] [Commented] (SPARK-12286) Support UnsafeRow in all SparkPlan (if possible)

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074349#comment-15074349 ] Apache Spark commented on SPARK-12286: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12294) Support UnsafeRow in HiveTableScan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12294: Assignee: Apache Spark > Support UnsafeRow in HiveTableScan >

[jira] [Assigned] (SPARK-12294) Support UnsafeRow in HiveTableScan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12294: Assignee: (was: Apache Spark) > Support UnsafeRow in HiveTableScan >

[jira] [Assigned] (SPARK-12290) Change the default value in SparkPlan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12290: Assignee: Apache Spark > Change the default value in SparkPlan >

[jira] [Assigned] (SPARK-12290) Change the default value in SparkPlan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12290: Assignee: (was: Apache Spark) > Change the default value in SparkPlan >

[jira] [Updated] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4819: - Attachment: SPARK_4819_null_do_not_merge.patch Here was what I tried when replacing Optional with use of

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Summary: Support union/intersect/except in Hive SQL (was: Support intersect/except in SQL) >

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Description: (was: Also `UNION ALL`in Hive SQL parser) > Support union/intersect/except in Hive

[jira] [Commented] (SPARK-12415) Do not use closure serializer to serialize task result

2015-12-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074201#comment-15074201 ] Andrew Or commented on SPARK-12415: --- Closed as Won't Fix. We can't use Kryo to serialize task result

[jira] [Commented] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074288#comment-15074288 ] Apache Spark commented on SPARK-12560: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

2015-12-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074327#comment-15074327 ] Michael Armbrust commented on SPARK-6459: - We do special case that very common problem, but I bet

[jira] [Assigned] (SPARK-12286) Support UnsafeRow in all SparkPlan (if possible)

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12286: Assignee: Davies Liu (was: Apache Spark) > Support UnsafeRow in all SparkPlan (if

[jira] [Created] (SPARK-12563) "No suitable driver" when calling JdbcUtils.saveTable in isolation

2015-12-29 Thread Sonya Huang (JIRA)
Sonya Huang created SPARK-12563: --- Summary: "No suitable driver" when calling JdbcUtils.saveTable in isolation Key: SPARK-12563 URL: https://issues.apache.org/jira/browse/SPARK-12563 Project: Spark

[jira] [Commented] (SPARK-12290) Change the default value in SparkPlan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074346#comment-15074346 ] Apache Spark commented on SPARK-12290: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12286) Support UnsafeRow in all SparkPlan (if possible)

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12286: Assignee: Apache Spark (was: Davies Liu) > Support UnsafeRow in all SparkPlan (if

[jira] [Commented] (SPARK-12488) LDA describeTopics() Generates Invalid Term IDs

2015-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074199#comment-15074199 ] Joseph K. Bradley commented on SPARK-12488: --- Just saw this:

[jira] [Commented] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074198#comment-15074198 ] Andrew Or commented on SPARK-12554: --- If anything I would say it's the calculation of

[jira] [Updated] (SPARK-11199) Improve R context management story and add getOrCreate

2015-12-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11199: - Assignee: Hossein Falaki (was: Felix Cheung) > Improve R context management story and add getOrCreate >

[jira] [Resolved] (SPARK-11199) Improve R context management story and add getOrCreate

2015-12-29 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-11199. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9185

[jira] [Commented] (SPARK-10789) Cluster mode SparkSubmit classpath only includes Spark assembly

2015-12-29 Thread Jonathan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074225#comment-15074225 ] Jonathan Kelly commented on SPARK-10789: Yes, using this patch requires a rebuild. If you are

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2015-12-29 Thread Nikita Tarasenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074265#comment-15074265 ] Nikita Tarasenko commented on SPARK-12177: -- I'm not against to use only Direct approach and

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2015-12-29 Thread Nikita Tarasenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074266#comment-15074266 ] Nikita Tarasenko commented on SPARK-12177: -- Should we use here

[jira] [Assigned] (SPARK-12290) Change the default value in SparkPlan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12290: Assignee: (was: Apache Spark) > Change the default value in SparkPlan >

[jira] [Created] (SPARK-12564) Improve missing column AnalysisException

2015-12-29 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-12564: Summary: Improve missing column AnalysisException Key: SPARK-12564 URL: https://issues.apache.org/jira/browse/SPARK-12564 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-12290) Change the default value in SparkPlan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12290: Assignee: Apache Spark > Change the default value in SparkPlan >

[jira] [Assigned] (SPARK-12294) Support UnsafeRow in HiveTableScan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12294: Assignee: Apache Spark > Support UnsafeRow in HiveTableScan >

[jira] [Assigned] (SPARK-12294) Support UnsafeRow in HiveTableScan

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12294: Assignee: (was: Apache Spark) > Support UnsafeRow in HiveTableScan >

[jira] [Assigned] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4819: --- Assignee: Apache Spark (was: Sean Owen) > Remove Guava's "Optional" from public API >

[jira] [Assigned] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4819: --- Assignee: Sean Owen (was: Apache Spark) > Remove Guava's "Optional" from public API >

  1   2   3   >