[jira] [Updated] (SPARK-15117) Generate code that get a value in each compressed column from CachedBatch when DataFrame.cache() is called

[ https://issues.apache.org/jira/browse/SPARK-15117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15117: Target Version/s: 2.5.0 (was: 2.4.0) > Generate code that get a value in each compressed column f

[jira] [Updated] (SPARK-22386) Data Source V2 improvements

[ https://issues.apache.org/jira/browse/SPARK-22386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-22386: Target Version/s: 2.5.0 (was: 2.4.0) > Data Source V2 improvements > ---

[jira] [Updated] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-22231: Target Version/s: 3.0.0 (was: 2.4.0, 3.0.0) > Support of map, filter, withColumn, dropColumn in n

[jira] [Resolved] (SPARK-22739) Additional Expression Support for Objects

[ https://issues.apache.org/jira/browse/SPARK-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22739. - Resolution: Not A Problem Target Version/s: (was: 2.4.0) > Additional Expression Sup

[jira] [Commented] (SPARK-22739) Additional Expression Support for Objects

[ https://issues.apache.org/jira/browse/SPARK-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623151#comment-16623151 ] Wenchen Fan commented on SPARK-22739: - I'm closing this ticket, since avro is now a

[jira] [Commented] (SPARK-19724) create a managed table with an existed default location should throw an exception

[ https://issues.apache.org/jira/browse/SPARK-19724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623139#comment-16623139 ] Apache Spark commented on SPARK-19724: -- User 'rxin' has created a pull request for

[jira] [Assigned] (SPARK-25271) Creating parquet table with all the column null throws exception

[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25271: Assignee: Apache Spark > Creating parquet table with all the column null throws exception

[jira] [Commented] (SPARK-25271) Creating parquet table with all the column null throws exception

[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623133#comment-16623133 ] Apache Spark commented on SPARK-25271: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-25271) Creating parquet table with all the column null throws exception

[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25271: Assignee: (was: Apache Spark) > Creating parquet table with all the column null throw

[jira] [Updated] (SPARK-21318) The exception message thrown by `lookupFunction` is ambiguous.

[ https://issues.apache.org/jira/browse/SPARK-21318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21318: Target Version/s: 2.5.0 (was: 2.4.0) > The exception message thrown by `lookupFunction` is ambigu

[jira] [Assigned] (SPARK-25384) Clarify fromJsonForceNullableSchema will be removed in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25384: --- Assignee: Reynold Xin > Clarify fromJsonForceNullableSchema will be removed in Spark 3.0 >

[jira] [Resolved] (SPARK-25384) Removing spark.sql.fromJsonForceNullableSchema

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25384. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22509 [https://gith

[jira] [Updated] (SPARK-25384) Clarify fromJsonForceNullableSchema will be removed in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25384: Description: (was: Disabling the spark.sql.fromJsonForceNullableSchema flag is error prone. We

[jira] [Updated] (SPARK-25384) Clarify fromJsonForceNullableSchema will be removed in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25384: Summary: Clarify fromJsonForceNullableSchema will be removed in Spark 3.0 (was: Removing spark.sq

[jira] [Resolved] (SPARK-25487) Refactor PrimitiveArrayBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25487. -- Resolution: Fixed Assignee: Chenxiao Mao Fix Version/s: 2.5.0 Issue re

[jira] [Resolved] (SPARK-15693) Write schema definition out for file-based data sources to avoid schema inference

[ https://issues.apache.org/jira/browse/SPARK-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15693. - Resolution: Won't Fix Target Version/s: (was: 2.4.0) > Write schema definition out f

[jira] [Updated] (SPARK-25204) rate source test is flaky

[ https://issues.apache.org/jira/browse/SPARK-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-25204: - Fix Version/s: (was: 3.0.0) 2.4.0 > rate source test is flaky > -

[jira] [Commented] (SPARK-15693) Write schema definition out for file-based data sources to avoid schema inference

[ https://issues.apache.org/jira/browse/SPARK-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623101#comment-16623101 ] Wenchen Fan commented on SPARK-15693: - I think we don't need to do this anymore. We'

[jira] [Updated] (SPARK-24999) Reduce unnecessary 'new' memory operations

[ https://issues.apache.org/jira/browse/SPARK-24999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-24999: - Fix Version/s: (was: 3.0.0) 2.4.0 > Reduce unnecessary 'new' memory opera

[jira] [Updated] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-22187: - Fix Version/s: (was: 3.0.0) 2.4.0 > Update unsaferow format for saved sta

[jira] [Updated] (SPARK-24441) Expose total estimated size of states in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-24441: - Fix Version/s: (was: 3.0.0) 2.4.0 > Expose total estimated size of states

[jira] [Assigned] (SPARK-19355) Use map output statistices to improve global limit's parallelism

[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19355: Assignee: Liang-Chi Hsieh (was: Apache Spark) > Use map output statistices to improve gl

[jira] [Assigned] (SPARK-19355) Use map output statistices to improve global limit's parallelism

[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19355: Assignee: Apache Spark (was: Liang-Chi Hsieh) > Use map output statistices to improve gl

[jira] [Commented] (SPARK-19355) Use map output statistices to improve global limit's parallelism

[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623099#comment-16623099 ] Apache Spark commented on SPARK-19355: -- User 'rxin' has created a pull request for

[jira] [Updated] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20184: Target Version/s: 2.5.0 (was: 2.4.0) > performance regression for complex/long sql when enable wh

[jira] [Updated] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16196: Target Version/s: 2.5.0 (was: 2.4.0) > Optimize in-memory scan performance using ColumnarBatches

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12978: Target Version/s: 2.5.0 (was: 3.0.0) > Skip unnecessary final group-by when input data already cl

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12978: Target Version/s: 3.0.0 (was: 2.4.0) > Skip unnecessary final group-by when input data already cl

[jira] [Commented] (SPARK-25380) Generated plans occupy over 50% of Spark driver memory

[ https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623087#comment-16623087 ] Jungtaek Lim commented on SPARK-25380: -- I thought about this as edge case which we

[jira] [Commented] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623085#comment-16623085 ] Apache Spark commented on SPARK-25499: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623084#comment-16623084 ] Apache Spark commented on SPARK-25499: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25499: Assignee: Apache Spark > Refactor BenchmarkBase and Benchmark > -

[jira] [Assigned] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25499: Assignee: (was: Apache Spark) > Refactor BenchmarkBase and Benchmark > --

[jira] [Resolved] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25494. - Resolution: Fixed Assignee: Kris Mok Fix Version/s: 2.5.0 > Upgrade Spark's use of Janin

[jira] [Created] (SPARK-25499) Refactor BenchmarkBase and Benchmark

Gengliang Wang created SPARK-25499: -- Summary: Refactor BenchmarkBase and Benchmark Key: SPARK-25499 URL: https://issues.apache.org/jira/browse/SPARK-25499 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623077#comment-16623077 ] Apache Spark commented on SPARK-25498: -- User 'maropu' has created a pull request fo

[jira] [Assigned] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25498: Assignee: (was: Apache Spark) > Fix SQLQueryTestSuite failures when the interpreter m

[jira] [Assigned] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25498: Assignee: Apache Spark > Fix SQLQueryTestSuite failures when the interpreter mode enabled

[jira] [Created] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

Takeshi Yamamuro created SPARK-25498: Summary: Fix SQLQueryTestSuite failures when the interpreter mode enabled Key: SPARK-25498 URL: https://issues.apache.org/jira/browse/SPARK-25498 Project: Spa

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623068#comment-16623068 ] Jungtaek Lim commented on SPARK-23682: -- With SPARK-24717 you don't even need to adj

[jira] [Commented] (SPARK-24717) Split out min retain version of state for memory in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623066#comment-16623066 ] Jungtaek Lim commented on SPARK-24717: -- Looks like version of this issue wasn't cha

[jira] [Updated] (SPARK-24717) Split out min retain version of state for memory in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-24717: - Fix Version/s: (was: 3.0.0) 2.4.0 > Split out min retain version of state

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623065#comment-16623065 ] Imran Rashid commented on SPARK-24523: -- Looks to me like {{SQLAppStatusListener}} i

[jira] [Commented] (SPARK-25422) flaky test: org.apache.spark.DistributedSuite.caching on disk, replicated (encryption = on) (with replication as stream)

[ https://issues.apache.org/jira/browse/SPARK-25422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623063#comment-16623063 ] Apache Spark commented on SPARK-25422: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25422) flaky test: org.apache.spark.DistributedSuite.caching on disk, replicated (encryption = on) (with replication as stream)

[ https://issues.apache.org/jira/browse/SPARK-25422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623064#comment-16623064 ] Apache Spark commented on SPARK-25422: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623052#comment-16623052 ] Sahil Aggarwal commented on SPARK-23682: Thanks [~kabhwan], will try that out. W

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623004#comment-16623004 ] Apache Spark commented on SPARK-25321: -- User 'WeichenXu123' has created a pull requ

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623003#comment-16623003 ] Apache Spark commented on SPARK-25321: -- User 'WeichenXu123' has created a pull requ

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622996#comment-16622996 ] Liang-Chi Hsieh commented on SPARK-25497: - Yes. Thanks for pinging me. I will lo

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622981#comment-16622981 ] Wenchen Fan commented on SPARK-25497: - cc [~viirya] do you have interest of fixing i

[jira] [Created] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

Wenchen Fan created SPARK-25497: --- Summary: limit operation within whole stage codegen should not consume all the inputs Key: SPARK-25497 URL: https://issues.apache.org/jira/browse/SPARK-25497 Project: S

[jira] [Assigned] (SPARK-25489) Refactor UDTSerializationBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25489: Assignee: Apache Spark > Refactor UDTSerializationBenchmark > ---

[jira] [Assigned] (SPARK-25489) Refactor UDTSerializationBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25489: Assignee: (was: Apache Spark) > Refactor UDTSerializationBenchmark >

[jira] [Commented] (SPARK-25384) Removing spark.sql.fromJsonForceNullableSchema

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622960#comment-16622960 ] Apache Spark commented on SPARK-25384: -- User 'rxin' has created a pull request for

[jira] [Commented] (SPARK-25384) Removing spark.sql.fromJsonForceNullableSchema

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622961#comment-16622961 ] Apache Spark commented on SPARK-25384: -- User 'rxin' has created a pull request for

[jira] [Commented] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622958#comment-16622958 ] Apache Spark commented on SPARK-23549: -- User 'rxin' has created a pull request for

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622956#comment-16622956 ] Wenchen Fan commented on SPARK-23715: - I'll write one and emphasize the behavior of

[jira] [Created] (SPARK-25496) Deprecate from_utc_timestamp and to_utc_timestamp

Reynold Xin created SPARK-25496: --- Summary: Deprecate from_utc_timestamp and to_utc_timestamp Key: SPARK-25496 URL: https://issues.apache.org/jira/browse/SPARK-25496 Project: Spark Issue Type: B

[jira] [Resolved] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23715. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) > from_utc_timestamp returns incorr

[jira] [Reopened] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23715: - Assignee: (was: Wenchen Fan) > from_utc_timestamp returns incorrect results for some UTC d

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622953#comment-16622953 ] Reynold Xin commented on SPARK-23715: - the current behavior is that it only takes a

[jira] [Commented] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622944#comment-16622944 ] Apache Spark commented on SPARK-14681: -- User 'WeichenXu123' has created a pull requ

[jira] [Commented] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622945#comment-16622945 ] Apache Spark commented on SPARK-14681: -- User 'WeichenXu123' has created a pull requ

[jira] [Assigned] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25321: Assignee: Apache Spark (was: Yanbo Liang) > ML, Graph 2.4 QA: API: New Scala APIs, docs

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622942#comment-16622942 ] Apache Spark commented on SPARK-25321: -- User 'WeichenXu123' has created a pull requ

[jira] [Assigned] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25321: Assignee: Yanbo Liang (was: Apache Spark) > ML, Graph 2.4 QA: API: New Scala APIs, docs

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622932#comment-16622932 ] Reynold Xin commented on SPARK-23715: - we can't fail queries in 2.x.   > from_utc_

[jira] [Assigned] (SPARK-25453) OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]

[ https://issues.apache.org/jira/browse/SPARK-25453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25453: Assignee: (was: Apache Spark) > OracleIntegrationSuite IllegalArgumentException: Time

[jira] [Assigned] (SPARK-25453) OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]

[ https://issues.apache.org/jira/browse/SPARK-25453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25453: Assignee: Apache Spark > OracleIntegrationSuite IllegalArgumentException: Timestamp forma

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622928#comment-16622928 ] Wenchen Fan commented on SPARK-23715: - I have no idea how to document the current be

[jira] [Resolved] (SPARK-24777) Add write benchmark for AVRO

[ https://issues.apache.org/jira/browse/SPARK-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24777. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.4.0 > Add write benchmark fo

[jira] [Commented] (SPARK-24777) Add write benchmark for AVRO

[ https://issues.apache.org/jira/browse/SPARK-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622918#comment-16622918 ] Apache Spark commented on SPARK-24777: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25465: Assignee: (was: Apache Spark) > Refactor Parquet test suites in project Hive > --

[jira] [Assigned] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25465: Assignee: Apache Spark > Refactor Parquet test suites in project Hive > -

[jira] [Commented] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622916#comment-16622916 ] Apache Spark commented on SPARK-25465: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622915#comment-16622915 ] Apache Spark commented on SPARK-25465: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25495: Assignee: Apache Spark (was: Shixiong Zhu) > FetchedData.reset doesn't reset _nextOffset

[jira] [Commented] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622911#comment-16622911 ] Apache Spark commented on SPARK-25495: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25495: Assignee: Shixiong Zhu (was: Apache Spark) > FetchedData.reset doesn't reset _nextOffset

[jira] [Commented] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622910#comment-16622910 ] Apache Spark commented on SPARK-25495: -- User 'zsxwing' has created a pull request f

[jira] [Created] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

Shixiong Zhu created SPARK-25495: Summary: FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll Key: SPARK-25495 URL: https://issues.apache.org/jira/browse/SPARK-25495 Project

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622877#comment-16622877 ] Apache Spark commented on SPARK-22036: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622861#comment-16622861 ] Jungtaek Lim commented on SPARK-23682: -- [~awked06] Your issue could be remedied by

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622860#comment-16622860 ] Umayr Hassan commented on SPARK-24523: -- Yet another informative stack: [^spark-stop

[jira] [Updated] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umayr Hassan updated SPARK-24523: - Attachment: spark-stop-jstack.log.3 > InterruptedException when closing SparkContext > -

[jira] [Assigned] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25494: Assignee: Apache Spark > Upgrade Spark's use of Janino to 3.0.10 > --

[jira] [Assigned] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25494: Assignee: (was: Apache Spark) > Upgrade Spark's use of Janino to 3.0.10 > ---

[jira] [Commented] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622852#comment-16622852 ] Apache Spark commented on SPARK-25494: -- User 'rednaxelafx' has created a pull reque

[jira] [Created] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

Kris Mok created SPARK-25494: Summary: Upgrade Spark's use of Janino to 3.0.10 Key: SPARK-25494 URL: https://issues.apache.org/jira/browse/SPARK-25494 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622841#comment-16622841 ] Apache Spark commented on SPARK-25469: -- User 'mn-mikke' has created a pull request

[jira] [Assigned] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25469: Assignee: (was: Apache Spark) > Eval methods of Concat, Reverse and ElementAt should

[jira] [Commented] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622842#comment-16622842 ] Apache Spark commented on SPARK-25469: -- User 'mn-mikke' has created a pull request

[jira] [Assigned] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25469: Assignee: Apache Spark > Eval methods of Concat, Reverse and ElementAt should use pattern

[jira] [Commented] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622830#comment-16622830 ] Apache Spark commented on SPARK-25472: -- User 'brkyvz' has created a pull request fo

[jira] [Comment Edited] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622826#comment-16622826 ] Jungtaek Lim edited comment on SPARK-10816 at 9/20/18 10:48 PM: --

[jira] [Commented] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622828#comment-16622828 ] Burak Yavuz commented on SPARK-25472: - Resolved by https://github.com/apache/spark/p

[jira] [Commented] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622827#comment-16622827 ] Apache Spark commented on SPARK-25472: -- User 'brkyvz' has created a pull request fo

[jira] [Resolved] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-25472. - Resolution: Fixed Fix Version/s: 2.5.0 > Structured Streaming query.stop() doesn't always

[jira] [Commented] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622826#comment-16622826 ] Jungtaek Lim commented on SPARK-10816: -- [~rxin] I completely agreed on we would be

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622821#comment-16622821 ] Apache Spark commented on SPARK-23715: -- User 'gatorsmile' has created a pull reques

  1   2   3   >