[jira] [Assigned] (SPARK-19751) Create Data frame API fails with a self referencing bean

[ https://issues.apache.org/jira/browse/SPARK-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19751: Assignee: (was: Apache Spark) > Create Data frame API fails with a self referencing be

[jira] [Assigned] (SPARK-19751) Create Data frame API fails with a self referencing bean

[ https://issues.apache.org/jira/browse/SPARK-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19751: Assignee: Apache Spark > Create Data frame API fails with a self referencing bean > --

[jira] [Commented] (SPARK-19751) Create Data frame API fails with a self referencing bean

[ https://issues.apache.org/jira/browse/SPARK-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898967#comment-15898967 ] Apache Spark commented on SPARK-19751: -- User 'maropu' has created a pull request for

[jira] [Created] (SPARK-19848) Regex Support in StopWordsRemover

Mohd Suaib Danish created SPARK-19848: - Summary: Regex Support in StopWordsRemover Key: SPARK-19848 URL: https://issues.apache.org/jira/browse/SPARK-19848 Project: Spark Issue Type: Wish

[jira] [Updated] (SPARK-19848) Regex Support in StopWordsRemover

[ https://issues.apache.org/jira/browse/SPARK-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohd Suaib Danish updated SPARK-19848: -- Description: Can we have regex feature in StopWordsRemover in addition to the provided

[jira] [Commented] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899228#comment-15899228 ] Sean Owen commented on SPARK-19829: --- Why wouldn't log4j be a good solution here? its pu

[jira] [Updated] (SPARK-19810) Remove support for Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19810: -- Target Version/s: 2.3.0 > Remove support for Scala 2.10 > - > >

[jira] [Updated] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14220: -- Affects Version/s: 2.1.0 Target Version/s: 2.3.0 > Build and test Spark against Scala 2.12 > -

[jira] [Updated] (SPARK-19848) Regex Support in StopWordsRemover

[ https://issues.apache.org/jira/browse/SPARK-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19848: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Wish) If it's more of a question, as

[jira] [Comment Edited] (SPARK-19848) Regex Support in StopWordsRemover

[ https://issues.apache.org/jira/browse/SPARK-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899250#comment-15899250 ] Nick Pentreath edited comment on SPARK-19848 at 3/7/17 11:06 AM: --

[jira] [Commented] (SPARK-19848) Regex Support in StopWordsRemover

[ https://issues.apache.org/jira/browse/SPARK-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899250#comment-15899250 ] Nick Pentreath commented on SPARK-19848: Perhaps the ML pipeline components menti

[jira] [Commented] (SPARK-19836) Customizable remote repository url for hive versions unit test

[ https://issues.apache.org/jira/browse/SPARK-19836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899292#comment-15899292 ] Song Jun commented on SPARK-19836: -- I have do this similar https://github.com/apache/spa

[jira] [Resolved] (SPARK-19836) Customizable remote repository url for hive versions unit test

[ https://issues.apache.org/jira/browse/SPARK-19836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19836. --- Resolution: Duplicate > Customizable remote repository url for hive versions unit test >

[jira] [Assigned] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19831: Assignee: Apache Spark > Sending the heartbeat master from worker maybe blocked by other

[jira] [Assigned] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19831: Assignee: (was: Apache Spark) > Sending the heartbeat master from worker maybe block

[jira] [Commented] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899334#comment-15899334 ] Apache Spark commented on SPARK-19831: -- User 'hustfxj' has created a pull request fo

[jira] [Assigned] (SPARK-19478) JDBC Sink

[ https://issues.apache.org/jira/browse/SPARK-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19478: Assignee: (was: Apache Spark) > JDBC Sink > - > > Key: SPARK-1

[jira] [Assigned] (SPARK-19478) JDBC Sink

[ https://issues.apache.org/jira/browse/SPARK-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19478: Assignee: Apache Spark > JDBC Sink > - > > Key: SPARK-19478 >

[jira] [Commented] (SPARK-19478) JDBC Sink

[ https://issues.apache.org/jira/browse/SPARK-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899458#comment-15899458 ] Apache Spark commented on SPARK-19478: -- User 'GaalDornick' has created a pull reques

[jira] [Created] (SPARK-19849) Support ArrayType in to_json function/expression

Hyukjin Kwon created SPARK-19849: Summary: Support ArrayType in to_json function/expression Key: SPARK-19849 URL: https://issues.apache.org/jira/browse/SPARK-19849 Project: Spark Issue Type:

[jira] [Created] (SPARK-19850) Support aliased expressions in function parameters

Herman van Hovell created SPARK-19850: - Summary: Support aliased expressions in function parameters Key: SPARK-19850 URL: https://issues.apache.org/jira/browse/SPARK-19850 Project: Spark

[jira] [Updated] (SPARK-19850) Support aliased expressions in function parameters

[ https://issues.apache.org/jira/browse/SPARK-19850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19850: -- Issue Type: Improvement (was: Bug) > Support aliased expressions in function parameter

[jira] [Commented] (SPARK-14471) The alias created in SELECT could be used in GROUP BY and followed expressions

[ https://issues.apache.org/jira/browse/SPARK-14471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899584#comment-15899584 ] Apache Spark commented on SPARK-14471: -- User 'maropu' has created a pull request for

[jira] [Assigned] (SPARK-14471) The alias created in SELECT could be used in GROUP BY and followed expressions

[ https://issues.apache.org/jira/browse/SPARK-14471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14471: Assignee: (was: Apache Spark) > The alias created in SELECT could be used in GROUP BY

[jira] [Assigned] (SPARK-14471) The alias created in SELECT could be used in GROUP BY and followed expressions

[ https://issues.apache.org/jira/browse/SPARK-14471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14471: Assignee: Apache Spark > The alias created in SELECT could be used in GROUP BY and followe

[jira] [Commented] (SPARK-15463) Support for creating a dataframe from CSV in Dataset[String]

[ https://issues.apache.org/jira/browse/SPARK-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899665#comment-15899665 ] Jayesh lalwani commented on SPARK-15463: Does it make sense to have a to_csv and

[jira] [Assigned] (SPARK-19849) Support ArrayType in to_json function/expression

[ https://issues.apache.org/jira/browse/SPARK-19849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19849: Assignee: (was: Apache Spark) > Support ArrayType in to_json function/expression > ---

[jira] [Commented] (SPARK-19849) Support ArrayType in to_json function/expression

[ https://issues.apache.org/jira/browse/SPARK-19849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899705#comment-15899705 ] Apache Spark commented on SPARK-19849: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-19849) Support ArrayType in to_json function/expression

[ https://issues.apache.org/jira/browse/SPARK-19849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19849: Assignee: Apache Spark > Support ArrayType in to_json function/expression > --

[jira] [Resolved] (SPARK-19637) add to_json APIs to SQL

[ https://issues.apache.org/jira/browse/SPARK-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19637. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 > add to_json APIs to S

[jira] [Updated] (SPARK-19840) Disallow creating permanent functions with invalid class names

[ https://issues.apache.org/jira/browse/SPARK-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19840: -- Description: Currently, Spark raises exceptions on creating invalid **temporary** functions, b

[jira] [Updated] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19765: Labels: release_notes (was: ) > UNCACHE TABLE should also un-cache all cached plans that refer to this tab

[jira] [Updated] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19765: Description: DropTableCommand, TruncateTableCommand, AlterTableRenameCommand, UncacheTableCommand, RefreshT

[jira] [Resolved] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19765. - Resolution: Fixed Fix Version/s: 2.2.0 > UNCACHE TABLE should also un-cache all cached plans that

[jira] [Resolved] (SPARK-18549) Failed to Uncache a View that References a Dropped Table.

[ https://issues.apache.org/jira/browse/SPARK-18549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-18549. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.2.0 > Failed to Uncache a View t

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899964#comment-15899964 ] Ari Gesher commented on SPARK-19764: We narrowed this down to driver OOM that wasn't

[jira] [Resolved] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ari Gesher resolved SPARK-19764. Resolution: Not A Bug > Executors hang with supposedly running task that are really finished. > ---

[jira] [Created] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

Michael Styles created SPARK-19851: -- Summary: Add support for EVERY and ANY (SOME) aggregates Key: SPARK-19851 URL: https://issues.apache.org/jira/browse/SPARK-19851 Project: Spark Issue Typ

[jira] [Commented] (SPARK-19348) pyspark.ml.Pipeline gets corrupted under multi threaded use

[ https://issues.apache.org/jira/browse/SPARK-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899979#comment-15899979 ] Apache Spark commented on SPARK-19348: -- User 'BryanCutler' has created a pull reques

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899980#comment-15899980 ] Shixiong Zhu commented on SPARK-19764: -- [~agesher] Do you have the OOM stack trace?

[jira] [Commented] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899988#comment-15899988 ] Michael Styles commented on SPARK-19851: https://github.com/apache/spark/pull/171

[jira] [Resolved] (SPARK-17498) StringIndexer.setHandleInvalid should have another option 'new'

[ https://issues.apache.org/jira/browse/SPARK-17498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-17498. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16883 [h

[jira] [Updated] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Styles updated SPARK-19851: --- Description: Add support for EVERY and ANY (SOME) aggregates. - EVERY returns true if all in

[jira] [Created] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

Joseph K. Bradley created SPARK-19852: - Summary: StringIndexer.setHandleInvalid should have another option 'new': Python API and docs Key: SPARK-19852 URL: https://issues.apache.org/jira/browse/SPARK-19852

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1599#comment-1599 ] Ari Gesher commented on SPARK-19764: We were collecting more data than we had heap fo

[jira] [Resolved] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19516. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16856 [https://githu

[jira] [Resolved] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-19803. Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0 Thanks for fixin

[jira] [Updated] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-19803: --- Component/s: Tests > Flaky BlockManagerProactiveReplicationSuite tests >

[jira] [Updated] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-19803: --- Affects Version/s: (was: 2.3.0) 2.2.0 > Flaky BlockManagerProactiv

[jira] [Updated] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-19803: --- Labels: flaky-test (was: ) > Flaky BlockManagerProactiveReplicationSuite tests > ---

[jira] [Updated] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19851: - Component/s: (was: Spark Core) > Add support for EVERY and ANY (SOME) aggregates > --

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900127#comment-15900127 ] Shixiong Zhu commented on SPARK-19764: -- So you don't set an UncaughtExceptionHandler

[jira] [Commented] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900152#comment-15900152 ] Chris Rogers commented on SPARK-16207: -- The lack of documentation on this is immense

[jira] [Commented] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900162#comment-15900162 ] Sean Owen commented on SPARK-16207: --- [~rcrogers] where would you document this? we coul

[jira] [Resolved] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-19561. Resolution: Fixed Assignee: Jason White Fix Version/s: 2.2.0 2.1.

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900184#comment-15900184 ] Nick Afshartous commented on SPARK-19767: - Yes, I completed the steps in the Prer

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900207#comment-15900207 ] Sean Owen commented on SPARK-19767: --- Oh, are you not running from the {{docs/}} directo

[jira] [Assigned] (SPARK-19702) Increasse refuse_seconds timeout in the Mesos Spark Dispatcher

[ https://issues.apache.org/jira/browse/SPARK-19702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19702: - Assignee: Michael Gummelt Priority: Minor (was: Major) Fix Version/s: 2.2.0 R

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900216#comment-15900216 ] Nick Afshartous commented on SPARK-19767: - Missed that one, thanks. > API Doc

[jira] [Commented] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900220#comment-15900220 ] Chris Rogers commented on SPARK-16207: -- [~srowen] since there is no documentation ye

[jira] [Comment Edited] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900220#comment-15900220 ] Chris Rogers edited comment on SPARK-16207 at 3/7/17 9:52 PM: -

[jira] [Created] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

Chris Bowden created SPARK-19853: Summary: Uppercase Kafka topics fail when startingOffsets are SpecificOffsets Key: SPARK-19853 URL: https://issues.apache.org/jira/browse/SPARK-19853 Project: Spark

[jira] [Updated] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

[ https://issues.apache.org/jira/browse/SPARK-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Bowden updated SPARK-19853: - Description: When using the KafkaSource with Structured Streaming, consumer assignments are not

[jira] [Updated] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

[ https://issues.apache.org/jira/browse/SPARK-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19853: - Target Version/s: 2.2.0 > Uppercase Kafka topics fail when startingOffsets are SpecificOffsets >

[jira] [Updated] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ari Gesher updated SPARK-19764: --- We're driving everything from Python. It may be a bug that we're not getting the error to propagate up

[jira] [Updated] (SPARK-18138) More officially deprecate support for Python 2.6, Java 7, and Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-18138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18138: Labels: releasenotes (was: ) > More officially deprecate support for Python 2.6, Java 7, and Scala

[jira] [Created] (SPARK-19854) Refactor file partitioning strategy to make it easier to extend / unit test

Reynold Xin created SPARK-19854: --- Summary: Refactor file partitioning strategy to make it easier to extend / unit test Key: SPARK-19854 URL: https://issues.apache.org/jira/browse/SPARK-19854 Project: Sp

[jira] [Created] (SPARK-19855) Create an internal FilePartitionStrategy interface

Reynold Xin created SPARK-19855: --- Summary: Create an internal FilePartitionStrategy interface Key: SPARK-19855 URL: https://issues.apache.org/jira/browse/SPARK-19855 Project: Spark Issue Type:

[jira] [Created] (SPARK-19856) Turn partitioning related test cases in FileSourceStrategySuite into unit tests

Reynold Xin created SPARK-19856: --- Summary: Turn partitioning related test cases in FileSourceStrategySuite into unit tests Key: SPARK-19856 URL: https://issues.apache.org/jira/browse/SPARK-19856 Project

[jira] [Updated] (SPARK-19856) Turn partitioning related test cases in FileSourceStrategySuite from integration tests into unit tests

[ https://issues.apache.org/jira/browse/SPARK-19856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19856: Summary: Turn partitioning related test cases in FileSourceStrategySuite from integration tests int

[jira] [Assigned] (SPARK-19855) Create an internal FilePartitionStrategy interface

[ https://issues.apache.org/jira/browse/SPARK-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19855: Assignee: Apache Spark (was: Reynold Xin) > Create an internal FilePartitionStrategy inte

[jira] [Commented] (SPARK-19855) Create an internal FilePartitionStrategy interface

[ https://issues.apache.org/jira/browse/SPARK-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900315#comment-15900315 ] Apache Spark commented on SPARK-19855: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-19855) Create an internal FilePartitionStrategy interface

[ https://issues.apache.org/jira/browse/SPARK-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19855: Assignee: Reynold Xin (was: Apache Spark) > Create an internal FilePartitionStrategy inte

[jira] [Created] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

Marcelo Vanzin created SPARK-19857: -- Summary: CredentialUpdater calculates the wrong time for next update Key: SPARK-19857 URL: https://issues.apache.org/jira/browse/SPARK-19857 Project: Spark

[jira] [Assigned] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19857: Assignee: Apache Spark > CredentialUpdater calculates the wrong time for next update > ---

[jira] [Assigned] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19857: Assignee: (was: Apache Spark) > CredentialUpdater calculates the wrong time for next u

[jira] [Commented] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900363#comment-15900363 ] Apache Spark commented on SPARK-19857: -- User 'vanzin' has created a pull request for

[jira] [Created] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

Shixiong Zhu created SPARK-19858: Summary: Add output mode to flatMapGroupsWithState and disallow invalid cases Key: SPARK-19858 URL: https://issues.apache.org/jira/browse/SPARK-19858 Project: Spark

[jira] [Assigned] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19858: Assignee: Apache Spark (was: Shixiong Zhu) > Add output mode to flatMapGroupsWithState an

[jira] [Commented] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900377#comment-15900377 ] Apache Spark commented on SPARK-19858: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19858: Assignee: Shixiong Zhu (was: Apache Spark) > Add output mode to flatMapGroupsWithState an

[jira] [Created] (SPARK-19859) The new watermark should override the old one

Shixiong Zhu created SPARK-19859: Summary: The new watermark should override the old one Key: SPARK-19859 URL: https://issues.apache.org/jira/browse/SPARK-19859 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19859: Assignee: Apache Spark (was: Shixiong Zhu) > The new watermark should override the old on

[jira] [Commented] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900410#comment-15900410 ] Apache Spark commented on SPARK-19859: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19859: Assignee: Shixiong Zhu (was: Apache Spark) > The new watermark should override the old on

[jira] [Resolved] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19857. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

[ https://issues.apache.org/jira/browse/SPARK-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900463#comment-15900463 ] Vincent commented on SPARK-19852: - I can work on this issue, since it is related to SPARK

[jira] [Commented] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900524#comment-15900524 ] Apache Spark commented on SPARK-19561: -- User 'JasonMWhite' has created a pull reques

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900533#comment-15900533 ] Jim Kleckner commented on SPARK-16333: -- I ended up here when looking into why an upg

[jira] [Assigned] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-18055: Assignee: Michael Armbrust > Dataset.flatMap can't work with types from customized

[jira] [Commented] (SPARK-19810) Remove support for Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900548#comment-15900548 ] Min Shen commented on SPARK-19810: -- [~srowen], Want to get an idea regarding the timeli

[jira] [Updated] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18055: - Target Version/s: 2.2.0 > Dataset.flatMap can't work with types from customized jar > ---

[jira] [Assigned] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18055: Assignee: Apache Spark (was: Michael Armbrust) > Dataset.flatMap can't work with types fr

[jira] [Assigned] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18055: Assignee: Michael Armbrust (was: Apache Spark) > Dataset.flatMap can't work with types fr

[jira] [Commented] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900568#comment-15900568 ] Apache Spark commented on SPARK-18055: -- User 'marmbrus' has created a pull request f

[jira] [Created] (SPARK-19860) DataFrame join get conflict error if two frames has a same name column.

ount1=8175477), Row(fdate=u'20170222', in_amount1=11032303), Row(fdate=u'20170216', in_amount1=11986702), Row(fdate=u'20170209', in_amount1=9082380), Row(fdate=u'20170214', in_amount1=8142569), Row(fdate=u'20170307', in_amount1=11092829), Ro

[jira] [Commented] (SPARK-18359) Let user specify locale in CSV parsing

[ https://issues.apache.org/jira/browse/SPARK-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900608#comment-15900608 ] Takeshi Yamamuro commented on SPARK-18359: -- Since JDK9 use CLDR as locale by def

[jira] [Created] (SPARK-19861) watermark should not be a negative time.

Genmao Yu created SPARK-19861: - Summary: watermark should not be a negative time. Key: SPARK-19861 URL: https://issues.apache.org/jira/browse/SPARK-19861 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-19861) watermark should not be a negative time.

[ https://issues.apache.org/jira/browse/SPARK-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19861: Assignee: Apache Spark > watermark should not be a negative time. > --

[jira] [Assigned] (SPARK-19861) watermark should not be a negative time.

[ https://issues.apache.org/jira/browse/SPARK-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19861: Assignee: (was: Apache Spark) > watermark should not be a negative time. > ---

  1   2   >