[jira] [Commented] (SPARK-15790) Audit @Since annotations in ML

[ https://issues.apache.org/jira/browse/SPARK-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904652#comment-15904652 ] Ehsun Behravesh commented on SPARK-15790: - Does this JIRA still need someone to work on? >

[jira] [Commented] (SPARK-19894) Tasks entirely assigned to one executor on Yarn-cluster mode for default-rack

[ https://issues.apache.org/jira/browse/SPARK-19894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904621#comment-15904621 ] Yuechen Chen commented on SPARK-19894: -- https://github.com/apache/spark/pull/17238 > Tasks entirely

[jira] [Comment Edited] (SPARK-19885) The config ignoreCorruptFiles doesn't work for CSV

[ https://issues.apache.org/jira/browse/SPARK-19885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904606#comment-15904606 ] Wenchen Fan edited comment on SPARK-19885 at 3/10/17 7:22 AM: -- This is

[jira] [Commented] (SPARK-19885) The config ignoreCorruptFiles doesn't work for CSV

[ https://issues.apache.org/jira/browse/SPARK-19885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904606#comment-15904606 ] Wenchen Fan commented on SPARK-19885: - This is because we support different charset for CSV files,

[jira] [Assigned] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

[ https://issues.apache.org/jira/browse/SPARK-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19852: Assignee: (was: Apache Spark) > StringIndexer.setHandleInvalid should have another

[jira] [Commented] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

[ https://issues.apache.org/jira/browse/SPARK-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904590#comment-15904590 ] Apache Spark commented on SPARK-19852: -- User 'VinceShieh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

[ https://issues.apache.org/jira/browse/SPARK-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19852: Assignee: Apache Spark > StringIndexer.setHandleInvalid should have another option 'new':

[jira] [Resolved] (SPARK-19891) Await Batch Lock not signaled on stream execution exit

[ https://issues.apache.org/jira/browse/SPARK-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19891. -- Resolution: Fixed Assignee: Tyson Condie Fix Version/s: 2.2.0

[jira] [Assigned] (SPARK-19893) Cannot run intersect/except with map type

[ https://issues.apache.org/jira/browse/SPARK-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19893: Assignee: Wenchen Fan (was: Apache Spark) > Cannot run intersect/except with map type >

[jira] [Created] (SPARK-19894) Tasks entirely assigned to one executor on Yarn-cluster mode for default-rack

Yuechen Chen created SPARK-19894: Summary: Tasks entirely assigned to one executor on Yarn-cluster mode for default-rack Key: SPARK-19894 URL: https://issues.apache.org/jira/browse/SPARK-19894

[jira] [Assigned] (SPARK-19893) Cannot run intersect/except with map type

[ https://issues.apache.org/jira/browse/SPARK-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19893: Assignee: Apache Spark (was: Wenchen Fan) > Cannot run intersect/except with map type >

[jira] [Created] (SPARK-19893) Cannot run intersect/except with map type

Wenchen Fan created SPARK-19893: --- Summary: Cannot run intersect/except with map type Key: SPARK-19893 URL: https://issues.apache.org/jira/browse/SPARK-19893 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-19008) Avoid boxing/unboxing overhead of calling a lambda with primitive type from Dataset program

[ https://issues.apache.org/jira/browse/SPARK-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19008: --- Assignee: Kazuaki Ishizaki > Avoid boxing/unboxing overhead of calling a lambda with

[jira] [Commented] (SPARK-19893) Cannot run intersect/except with map type

[ https://issues.apache.org/jira/browse/SPARK-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904587#comment-15904587 ] Apache Spark commented on SPARK-19893: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19008) Avoid boxing/unboxing overhead of calling a lambda with primitive type from Dataset program

[ https://issues.apache.org/jira/browse/SPARK-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19008. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17172

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904497#comment-15904497 ] Saisai Shao commented on SPARK-19143: - Hi all, I wrote a rough design doc based on the comments

[jira] [Assigned] (SPARK-19320) Allow guaranteed amount of GPU to be used when launching jobs

[ https://issues.apache.org/jira/browse/SPARK-19320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19320: Assignee: (was: Apache Spark) > Allow guaranteed amount of GPU to be used when

[jira] [Commented] (SPARK-19320) Allow guaranteed amount of GPU to be used when launching jobs

[ https://issues.apache.org/jira/browse/SPARK-19320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904484#comment-15904484 ] Apache Spark commented on SPARK-19320: -- User 'yanji84' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19320) Allow guaranteed amount of GPU to be used when launching jobs

[ https://issues.apache.org/jira/browse/SPARK-19320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19320: Assignee: Apache Spark > Allow guaranteed amount of GPU to be used when launching jobs >

[jira] [Commented] (SPARK-19320) Allow guaranteed amount of GPU to be used when launching jobs

[ https://issues.apache.org/jira/browse/SPARK-19320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904479#comment-15904479 ] Ji Yan commented on SPARK-19320: i'm proposing to add a configuration parameter to guarantee a hard limit

[jira] [Commented] (SPARK-19892) Implement findAnalogies method for Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904471#comment-15904471 ] Apache Spark commented on SPARK-19892: -- User 'benradford' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19892) Implement findAnalogies method for Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19892: Assignee: Apache Spark > Implement findAnalogies method for Word2VecModel >

[jira] [Assigned] (SPARK-19892) Implement findAnalogies method for Word2VecModel

[ https://issues.apache.org/jira/browse/SPARK-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19892: Assignee: (was: Apache Spark) > Implement findAnalogies method for Word2VecModel >

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904460#comment-15904460 ] Kazuaki Ishizaki commented on SPARK-14083: -- I rebased this with master:

[jira] [Commented] (SPARK-6634) Allow replacing columns in Transformers

[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904445#comment-15904445 ] Tree Field commented on SPARK-6634: --- I want this feature too. because I often overwrite UnaryTransformer

[jira] [Commented] (SPARK-11569) StringIndexer transform fails when column contains nulls

[ https://issues.apache.org/jira/browse/SPARK-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904448#comment-15904448 ] Apache Spark commented on SPARK-11569: -- User 'crackcell' has created a pull request for this issue:

[jira] [Created] (SPARK-19892) Implement findAnalogies method for Word2VecModel

Benjamin Radford created SPARK-19892: Summary: Implement findAnalogies method for Word2VecModel Key: SPARK-19892 URL: https://issues.apache.org/jira/browse/SPARK-19892 Project: Spark

[jira] [Assigned] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18112: Assignee: Apache Spark (was: Xiao Li) > Spark2.x does not support read data from Hive

[jira] [Assigned] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18112: Assignee: Xiao Li (was: Apache Spark) > Spark2.x does not support read data from Hive

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904429#comment-15904429 ] Apache Spark commented on SPARK-18112: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Resolved] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17557. -- Resolution: Cannot Reproduce ^ I can't reproduce too. Let me resolve this. Please reopen this

[jira] [Comment Edited] (SPARK-16283) Implement percentile_approx SQL function

[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904397#comment-15904397 ] Zhenhua Wang edited comment on SPARK-16283 at 3/10/17 4:09 AM: --- [~erlu] I

[jira] [Commented] (SPARK-16283) Implement percentile_approx SQL function

[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904397#comment-15904397 ] Zhenhua Wang commented on SPARK-16283: -- [~erlu] I think it's been made clear from above discussions,

[jira] [Updated] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18112: Component/s: (was: Spark Submit) > Spark2.x does not support read data from Hive 2.x metastore >

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904362#comment-15904362 ] Xiao Li commented on SPARK-18112: - Let me resolve it for supporting Hive 2.1.0 metastore. > Spark2.x

[jira] [Updated] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18112: Priority: Critical (was: Major) > Spark2.x does not support read data from Hive 2.x metastore >

[jira] [Reopened] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-18112: - > Spark2.x does not support read data from Hive 2.x metastore >

[jira] [Assigned] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-18112: --- Assignee: Xiao Li > Spark2.x does not support read data from Hive 2.x metastore >

[jira] [Commented] (SPARK-19711) Bug in gapply function

[ https://issues.apache.org/jira/browse/SPARK-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904346#comment-15904346 ] Yeonseop Kim commented on SPARK-19711: -- It seems to be more nice Using "stringsAsFactor = FALSE",

[jira] [Commented] (SPARK-17322) 'ANY n' clause for SQL queries to increase the ease of use of WHERE clause predicates

[ https://issues.apache.org/jira/browse/SPARK-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904285#comment-15904285 ] Hyukjin Kwon commented on SPARK-17322: -- Let me leave a link that might be helpful -

[jira] [Commented] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904272#comment-15904272 ] Hyukjin Kwon commented on SPARK-16754: -- Today, I just tested this for my curiosity. It seems prints

[jira] [Resolved] (SPARK-16255) Spark2.0 doesn't support the following SQL statement:"insert into directory "/u_qa_user/hive_testdata/test1/t1" select * from d_test_tpc_2g_txt.auction" while Hive supp

[ https://issues.apache.org/jira/browse/SPARK-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16255. -- Resolution: Duplicate I think this is a duplicate of SPARK-4131. Please reopen this if I was

[jira] [Resolved] (SPARK-19886) reportDataLoss cause != null check is wrong for Structured Streaming KafkaSource

[ https://issues.apache.org/jira/browse/SPARK-19886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19886. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > reportDataLoss cause

[jira] [Assigned] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19507: Assignee: Apache Spark > pyspark.sql.types._verify_type() exceptions too broad to debug

[jira] [Assigned] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19507: Assignee: (was: Apache Spark) > pyspark.sql.types._verify_type() exceptions too broad

[jira] [Reopened] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-19507: -- I am reopening this per https://github.com/apache/spark/pull/17213#issuecomment-285530248 and

[jira] [Resolved] (SPARK-19871) Improve error message in verify_type to indicate which field the error is for

[ https://issues.apache.org/jira/browse/SPARK-19871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19871. -- Resolution: Duplicate > Improve error message in verify_type to indicate which field the error

[jira] [Assigned] (SPARK-19891) Await Batch Lock not signaled on stream execution exit

[ https://issues.apache.org/jira/browse/SPARK-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19891: Assignee: Apache Spark > Await Batch Lock not signaled on stream execution exit >

[jira] [Assigned] (SPARK-19891) Await Batch Lock not signaled on stream execution exit

[ https://issues.apache.org/jira/browse/SPARK-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19891: Assignee: (was: Apache Spark) > Await Batch Lock not signaled on stream execution

[jira] [Commented] (SPARK-19891) Await Batch Lock not signaled on stream execution exit

[ https://issues.apache.org/jira/browse/SPARK-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904116#comment-15904116 ] Apache Spark commented on SPARK-19891: -- User 'tcondie' has created a pull request for this issue:

[jira] [Created] (SPARK-19891) Await Batch Lock not signaled on stream execution exit

Tyson Condie created SPARK-19891: Summary: Await Batch Lock not signaled on stream execution exit Key: SPARK-19891 URL: https://issues.apache.org/jira/browse/SPARK-19891 Project: Spark Issue

[jira] [Updated] (SPARK-19890) Make MetastoreRelation statistics estimation more accurately

[ https://issues.apache.org/jira/browse/SPARK-19890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-19890: --- Description: Currently the MetastoreRelation statistics is retrieved on the analyze phase, and the

[jira] [Commented] (SPARK-19353) Support binary I/O in PipedRDD

[ https://issues.apache.org/jira/browse/SPARK-19353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904058#comment-15904058 ] Apache Spark commented on SPARK-19353: -- User 'superbobry' has created a pull request for this issue:

[jira] [Created] (SPARK-19890) Make MetastoreRelation statistics estimation more accurately

Zhan Zhang created SPARK-19890: -- Summary: Make MetastoreRelation statistics estimation more accurately Key: SPARK-19890 URL: https://issues.apache.org/jira/browse/SPARK-19890 Project: Spark

[jira] [Created] (SPARK-19889) Make TaskContext synchronized

Herman van Hovell created SPARK-19889: - Summary: Make TaskContext synchronized Key: SPARK-19889 URL: https://issues.apache.org/jira/browse/SPARK-19889 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19611) Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files

[ https://issues.apache.org/jira/browse/SPARK-19611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904033#comment-15904033 ] Apache Spark commented on SPARK-19611: -- User 'budde' has created a pull request for this issue:

[jira] [Created] (SPARK-19888) Seeing offsets not resetting even when reset policy is configured explicitly

Justin Miller created SPARK-19888: - Summary: Seeing offsets not resetting even when reset policy is configured explicitly Key: SPARK-19888 URL: https://issues.apache.org/jira/browse/SPARK-19888

[jira] [Assigned] (SPARK-19611) Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files

[ https://issues.apache.org/jira/browse/SPARK-19611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19611: --- Assignee: Adam Budde > Spark 2.1.0 breaks some Hive tables backed by case-sensitive data

[jira] [Resolved] (SPARK-19611) Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files

[ https://issues.apache.org/jira/browse/SPARK-19611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19611. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16944

[jira] [Comment Edited] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903778#comment-15903778 ] Andrew Milkowski edited comment on SPARK-19364 at 3/9/17 8:17 PM: --

[jira] [Commented] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903778#comment-15903778 ] Andrew Milkowski commented on SPARK-19364: -- thanks @Takeshi Yamamuro , will try to see if I can

[jira] [Resolved] (SPARK-12334) Support read from multiple input paths for orc file in DataFrameReader.orc

[ https://issues.apache.org/jira/browse/SPARK-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-12334. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 10307

[jira] [Assigned] (SPARK-12334) Support read from multiple input paths for orc file in DataFrameReader.orc

[ https://issues.apache.org/jira/browse/SPARK-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-12334: --- Assignee: Jeff Zhang > Support read from multiple input paths for orc file in DataFrameReader.orc >

[jira] [Updated] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19887: --- Summary: __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned

[jira] [Created] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ not interpreted as NULL partition value in partitioned persisted tables

Cheng Lian created SPARK-19887: -- Summary: __HIVE_DEFAULT_PARTITION__ not interpreted as NULL partition value in partitioned persisted tables Key: SPARK-19887 URL: https://issues.apache.org/jira/browse/SPARK-19887

[jira] [Commented] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ not interpreted as NULL partition value in partitioned persisted tables

[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903749#comment-15903749 ] Cheng Lian commented on SPARK-19887: cc [~cloud_fan] > __HIVE_DEFAULT_PARTITION__ not interpreted as

[jira] [Assigned] (SPARK-19886) reportDataLoss cause != null check is wrong for Structured Streaming KafkaSource

[ https://issues.apache.org/jira/browse/SPARK-19886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19886: Assignee: Apache Spark (was: Burak Yavuz) > reportDataLoss cause != null check is wrong

[jira] [Assigned] (SPARK-19886) reportDataLoss cause != null check is wrong for Structured Streaming KafkaSource

[ https://issues.apache.org/jira/browse/SPARK-19886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19886: Assignee: Burak Yavuz (was: Apache Spark) > reportDataLoss cause != null check is wrong

[jira] [Commented] (SPARK-19886) reportDataLoss cause != null check is wrong for Structured Streaming KafkaSource

[ https://issues.apache.org/jira/browse/SPARK-19886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903733#comment-15903733 ] Apache Spark commented on SPARK-19886: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Created] (SPARK-19886) reportDataLoss cause != null check is wrong for Structured Streaming KafkaSource

Burak Yavuz created SPARK-19886: --- Summary: reportDataLoss cause != null check is wrong for Structured Streaming KafkaSource Key: SPARK-19886 URL: https://issues.apache.org/jira/browse/SPARK-19886

[jira] [Resolved] (SPARK-19861) watermark should not be a negative time.

[ https://issues.apache.org/jira/browse/SPARK-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19861. -- Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903680#comment-15903680 ] Apache Spark commented on SPARK-19507: -- User 'dgingrich' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19715) Option to Strip Paths in FileSource

[ https://issues.apache.org/jira/browse/SPARK-19715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19715. -- Resolution: Fixed Assignee: Liwei Lin Fix Version/s: 2.2.0 > Option to Strip

[jira] [Commented] (SPARK-19882) Pivot with null as the pivot value throws NPE

[ https://issues.apache.org/jira/browse/SPARK-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903643#comment-15903643 ] Apache Spark commented on SPARK-19882: -- User 'aray' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19793) Use clock.getTimeMillis when mark task as finished in TaskSetManager.

[ https://issues.apache.org/jira/browse/SPARK-19793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19793. Resolution: Fixed Assignee: jin xing Fix Version/s: 2.2.0 > Use

[jira] [Updated] (SPARK-19757) Executor with task scheduled could be killed due to idleness

[ https://issues.apache.org/jira/browse/SPARK-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-19757: --- Reporter: jin xing (was: Jimmy Xiang) > Executor with task scheduled could be killed due to

[jira] [Resolved] (SPARK-19757) Executor with task scheduled could be killed due to idleness

[ https://issues.apache.org/jira/browse/SPARK-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19757. Resolution: Fixed Assignee: Jimmy Xiang Fix Version/s: 2.2.0 > Executor

[jira] [Created] (SPARK-19885) The config ignoreCorruptFiles doesn't work for CSV

Shixiong Zhu created SPARK-19885: Summary: The config ignoreCorruptFiles doesn't work for CSV Key: SPARK-19885 URL: https://issues.apache.org/jira/browse/SPARK-19885 Project: Spark Issue

[jira] [Commented] (SPARK-19875) Map->filter on many columns gets stuck in constraint inference optimization code

[ https://issues.apache.org/jira/browse/SPARK-19875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903584#comment-15903584 ] Kazuaki Ishizaki commented on SPARK-19875: -- I got the following stack trace. This stuck seems to

[jira] [Comment Edited] (SPARK-19067) mapGroupsWithState - arbitrary stateful operations with Structured Streaming (similar to DStream.mapWithState)

[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903532#comment-15903532 ] Amit Sela edited comment on SPARK-19067 at 3/9/17 6:15 PM: --- [~tdas] I just read

[jira] [Commented] (SPARK-19067) mapGroupsWithState - arbitrary stateful operations with Structured Streaming (similar to DStream.mapWithState)

[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903532#comment-15903532 ] Amit Sela commented on SPARK-19067: --- [~tdas] I just read the PR, and I'm very excited for Spark to

[jira] [Resolved] (SPARK-19884) Add the ability to get all registered functions from a SparkSession

[ https://issues.apache.org/jira/browse/SPARK-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yael Aharon resolved SPARK-19884. - Resolution: Not A Problem Thank you so much for your reply > Add the ability to get all

[jira] [Commented] (SPARK-19884) Add the ability to get all registered functions from a SparkSession

[ https://issues.apache.org/jira/browse/SPARK-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903257#comment-15903257 ] Herman van Hovell commented on SPARK-19884: --- You can use the catalog for that by calling

[jira] [Commented] (SPARK-17248) Add native Scala enum support to Dataset Encoders

[ https://issues.apache.org/jira/browse/SPARK-17248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903245#comment-15903245 ] Lee Dongjin commented on SPARK-17248: - [~pdxleif] // Although it may be an expired question, let me

[jira] [Created] (SPARK-19884) Add the ability to get all registered functions from a SparkSession

Yael Aharon created SPARK-19884: --- Summary: Add the ability to get all registered functions from a SparkSession Key: SPARK-19884 URL: https://issues.apache.org/jira/browse/SPARK-19884 Project: Spark

[jira] [Commented] (SPARK-15474) ORC data source fails to write and read back empty dataframe

[ https://issues.apache.org/jira/browse/SPARK-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902991#comment-15902991 ] Hyukjin Kwon commented on SPARK-15474: -- I see. Thank you for your advice. Let me maybe give a shot

[jira] [Issue Comment Deleted] (SPARK-19300) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19300: Comment: was deleted (was: [~zsxwing] I also met this issue, and you can look

[jira] [Commented] (SPARK-19300) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902988#comment-15902988 ] hustfxj commented on SPARK-19300: - [~zsxwing] I also met this issue, and you can look

[jira] [Commented] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902990#comment-15902990 ] Apache Spark commented on SPARK-19851: -- User 'ptkool' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19851: Assignee: Apache Spark > Add support for EVERY and ANY (SOME) aggregates >

[jira] [Assigned] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19851: Assignee: (was: Apache Spark) > Add support for EVERY and ANY (SOME) aggregates >

[jira] [Closed] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj closed SPARK-19829. --- Resolution: Invalid > The log about driver should support rolling like executor >

[jira] [Commented] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902982#comment-15902982 ] hustfxj commented on SPARK-19829: - ok. > The log about driver should support rolling like executor >

[jira] [Commented] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902980#comment-15902980 ] hustfxj commented on SPARK-19883: - [~srowen] Saw this issue again today. So spark may not solve this

[jira] [Resolved] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19883. --- Resolution: Duplicate This is at best quite incomplete, but you are saying it is a duplicate. >

[jira] [Assigned] (SPARK-19882) Pivot with null as the pivot value throws NPE

[ https://issues.apache.org/jira/browse/SPARK-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19882: Assignee: (was: Apache Spark) > Pivot with null as the pivot value throws NPE >

[jira] [Commented] (SPARK-19882) Pivot with null as the pivot value throws NPE

[ https://issues.apache.org/jira/browse/SPARK-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902977#comment-15902977 ] Apache Spark commented on SPARK-19882: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Closed] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-19883. - > Executor is waiting for lock > > > Key: SPARK-19883 >

[jira] [Assigned] (SPARK-19882) Pivot with null as the pivot value throws NPE

[ https://issues.apache.org/jira/browse/SPARK-19882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19882: Assignee: Apache Spark > Pivot with null as the pivot value throws NPE >

[jira] [Comment Edited] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902972#comment-15902972 ] hustfxj edited comment on SPARK-19883 at 3/9/17 12:25 PM: -- [~srowen] A stage is

  1   2   >