[jira] [Resolved] (SPARK-15693) Write schema definition out for file-based data sources to avoid schema inference

[ https://issues.apache.org/jira/browse/SPARK-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15693. - Resolution: Won't Fix Target Version/s: (was: 2.4.0) > Write schema definition out

[jira] [Updated] (SPARK-25204) rate source test is flaky

[ https://issues.apache.org/jira/browse/SPARK-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-25204: - Fix Version/s: (was: 3.0.0) 2.4.0 > rate source test is flaky >

[jira] [Commented] (SPARK-15693) Write schema definition out for file-based data sources to avoid schema inference

[ https://issues.apache.org/jira/browse/SPARK-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623101#comment-16623101 ] Wenchen Fan commented on SPARK-15693: - I think we don't need to do this anymore. We've already

[jira] [Updated] (SPARK-24999) Reduce unnecessary 'new' memory operations

[ https://issues.apache.org/jira/browse/SPARK-24999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-24999: - Fix Version/s: (was: 3.0.0) 2.4.0 > Reduce unnecessary 'new' memory

[jira] [Updated] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-22187: - Fix Version/s: (was: 3.0.0) 2.4.0 > Update unsaferow format for saved

[jira] [Updated] (SPARK-24441) Expose total estimated size of states in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-24441: - Fix Version/s: (was: 3.0.0) 2.4.0 > Expose total estimated size of

[jira] [Assigned] (SPARK-19355) Use map output statistices to improve global limit's parallelism

[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19355: Assignee: Liang-Chi Hsieh (was: Apache Spark) > Use map output statistices to improve

[jira] [Assigned] (SPARK-19355) Use map output statistices to improve global limit's parallelism

[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19355: Assignee: Apache Spark (was: Liang-Chi Hsieh) > Use map output statistices to improve

[jira] [Commented] (SPARK-19355) Use map output statistices to improve global limit's parallelism

[ https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623099#comment-16623099 ] Apache Spark commented on SPARK-19355: -- User 'rxin' has created a pull request for this issue:

[jira] [Updated] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20184: Target Version/s: 2.5.0 (was: 2.4.0) > performance regression for complex/long sql when enable

[jira] [Updated] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16196: Target Version/s: 2.5.0 (was: 2.4.0) > Optimize in-memory scan performance using ColumnarBatches

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12978: Target Version/s: 2.5.0 (was: 3.0.0) > Skip unnecessary final group-by when input data already

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12978: Target Version/s: 3.0.0 (was: 2.4.0) > Skip unnecessary final group-by when input data already

[jira] [Commented] (SPARK-25380) Generated plans occupy over 50% of Spark driver memory

[ https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623087#comment-16623087 ] Jungtaek Lim commented on SPARK-25380: -- I thought about this as edge case which we might be unsure

[jira] [Commented] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623085#comment-16623085 ] Apache Spark commented on SPARK-25499: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623084#comment-16623084 ] Apache Spark commented on SPARK-25499: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25499: Assignee: Apache Spark > Refactor BenchmarkBase and Benchmark >

[jira] [Assigned] (SPARK-25499) Refactor BenchmarkBase and Benchmark

[ https://issues.apache.org/jira/browse/SPARK-25499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25499: Assignee: (was: Apache Spark) > Refactor BenchmarkBase and Benchmark >

[jira] [Resolved] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25494. - Resolution: Fixed Assignee: Kris Mok Fix Version/s: 2.5.0 > Upgrade Spark's use of

[jira] [Created] (SPARK-25499) Refactor BenchmarkBase and Benchmark

Gengliang Wang created SPARK-25499: -- Summary: Refactor BenchmarkBase and Benchmark Key: SPARK-25499 URL: https://issues.apache.org/jira/browse/SPARK-25499 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623077#comment-16623077 ] Apache Spark commented on SPARK-25498: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25498: Assignee: (was: Apache Spark) > Fix SQLQueryTestSuite failures when the interpreter

[jira] [Assigned] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25498: Assignee: Apache Spark > Fix SQLQueryTestSuite failures when the interpreter mode

[jira] [Created] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

Takeshi Yamamuro created SPARK-25498: Summary: Fix SQLQueryTestSuite failures when the interpreter mode enabled Key: SPARK-25498 URL: https://issues.apache.org/jira/browse/SPARK-25498 Project:

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623068#comment-16623068 ] Jungtaek Lim commented on SPARK-23682: -- With SPARK-24717 you don't even need to adjust

[jira] [Commented] (SPARK-24717) Split out min retain version of state for memory in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623066#comment-16623066 ] Jungtaek Lim commented on SPARK-24717: -- Looks like version of this issue wasn't changed while

[jira] [Updated] (SPARK-24717) Split out min retain version of state for memory in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-24717: - Fix Version/s: (was: 3.0.0) 2.4.0 > Split out min retain version of

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623065#comment-16623065 ] Imran Rashid commented on SPARK-24523: -- Looks to me like {{SQLAppStatusListener}} is still busy,

[jira] [Commented] (SPARK-25422) flaky test: org.apache.spark.DistributedSuite.caching on disk, replicated (encryption = on) (with replication as stream)

[ https://issues.apache.org/jira/browse/SPARK-25422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623063#comment-16623063 ] Apache Spark commented on SPARK-25422: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-25422) flaky test: org.apache.spark.DistributedSuite.caching on disk, replicated (encryption = on) (with replication as stream)

[ https://issues.apache.org/jira/browse/SPARK-25422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623064#comment-16623064 ] Apache Spark commented on SPARK-25422: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623052#comment-16623052 ] Sahil Aggarwal commented on SPARK-23682: Thanks [~kabhwan], will try that out. Was able to limit

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623004#comment-16623004 ] Apache Spark commented on SPARK-25321: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623003#comment-16623003 ] Apache Spark commented on SPARK-25321: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622996#comment-16622996 ] Liang-Chi Hsieh commented on SPARK-25497: - Yes. Thanks for pinging me. I will look into this. >

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622981#comment-16622981 ] Wenchen Fan commented on SPARK-25497: - cc [~viirya] do you have interest of fixing it? > limit

[jira] [Created] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

Wenchen Fan created SPARK-25497: --- Summary: limit operation within whole stage codegen should not consume all the inputs Key: SPARK-25497 URL: https://issues.apache.org/jira/browse/SPARK-25497 Project:

[jira] [Assigned] (SPARK-25489) Refactor UDTSerializationBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25489: Assignee: Apache Spark > Refactor UDTSerializationBenchmark >

[jira] [Assigned] (SPARK-25489) Refactor UDTSerializationBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25489: Assignee: (was: Apache Spark) > Refactor UDTSerializationBenchmark >

[jira] [Commented] (SPARK-25384) Removing spark.sql.fromJsonForceNullableSchema

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622960#comment-16622960 ] Apache Spark commented on SPARK-25384: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-25384) Removing spark.sql.fromJsonForceNullableSchema

[ https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622961#comment-16622961 ] Apache Spark commented on SPARK-25384: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622958#comment-16622958 ] Apache Spark commented on SPARK-23549: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622956#comment-16622956 ] Wenchen Fan commented on SPARK-23715: - I'll write one and emphasize the behavior of casting string

[jira] [Created] (SPARK-25496) Deprecate from_utc_timestamp and to_utc_timestamp

Reynold Xin created SPARK-25496: --- Summary: Deprecate from_utc_timestamp and to_utc_timestamp Key: SPARK-25496 URL: https://issues.apache.org/jira/browse/SPARK-25496 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23715. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) > from_utc_timestamp returns

[jira] [Reopened] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23715: - Assignee: (was: Wenchen Fan) > from_utc_timestamp returns incorrect results for some UTC

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622953#comment-16622953 ] Reynold Xin commented on SPARK-23715: - the current behavior is that it only takes a timestamp type

[jira] [Commented] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622944#comment-16622944 ] Apache Spark commented on SPARK-14681: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622945#comment-16622945 ] Apache Spark commented on SPARK-14681: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25321: Assignee: Apache Spark (was: Yanbo Liang) > ML, Graph 2.4 QA: API: New Scala APIs, docs

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622942#comment-16622942 ] Apache Spark commented on SPARK-25321: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25321: Assignee: Yanbo Liang (was: Apache Spark) > ML, Graph 2.4 QA: API: New Scala APIs, docs

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622932#comment-16622932 ] Reynold Xin commented on SPARK-23715: - we can't fail queries in 2.x.   > from_utc_timestamp

[jira] [Assigned] (SPARK-25453) OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]

[ https://issues.apache.org/jira/browse/SPARK-25453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25453: Assignee: (was: Apache Spark) > OracleIntegrationSuite IllegalArgumentException:

[jira] [Assigned] (SPARK-25453) OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]

[ https://issues.apache.org/jira/browse/SPARK-25453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25453: Assignee: Apache Spark > OracleIntegrationSuite IllegalArgumentException: Timestamp

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622928#comment-16622928 ] Wenchen Fan commented on SPARK-23715: - I have no idea how to document the current behavior. Shall we

[jira] [Resolved] (SPARK-24777) Add write benchmark for AVRO

[ https://issues.apache.org/jira/browse/SPARK-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24777. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.4.0 > Add write benchmark

[jira] [Commented] (SPARK-24777) Add write benchmark for AVRO

[ https://issues.apache.org/jira/browse/SPARK-24777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622918#comment-16622918 ] Apache Spark commented on SPARK-24777: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25465: Assignee: (was: Apache Spark) > Refactor Parquet test suites in project Hive >

[jira] [Assigned] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25465: Assignee: Apache Spark > Refactor Parquet test suites in project Hive >

[jira] [Commented] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622916#comment-16622916 ] Apache Spark commented on SPARK-25465: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622915#comment-16622915 ] Apache Spark commented on SPARK-25465: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25495: Assignee: Apache Spark (was: Shixiong Zhu) > FetchedData.reset doesn't reset

[jira] [Commented] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622911#comment-16622911 ] Apache Spark commented on SPARK-25495: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25495: Assignee: Shixiong Zhu (was: Apache Spark) > FetchedData.reset doesn't reset

[jira] [Commented] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

[ https://issues.apache.org/jira/browse/SPARK-25495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622910#comment-16622910 ] Apache Spark commented on SPARK-25495: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-25495) FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll

Shixiong Zhu created SPARK-25495: Summary: FetchedData.reset doesn't reset _nextOffsetInFetchedData and _offsetAfterPoll Key: SPARK-25495 URL: https://issues.apache.org/jira/browse/SPARK-25495

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622877#comment-16622877 ] Apache Spark commented on SPARK-22036: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622861#comment-16622861 ] Jungtaek Lim commented on SPARK-23682: -- [~awked06] Your issue could be remedied by SPARK-24763 and

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622860#comment-16622860 ] Umayr Hassan commented on SPARK-24523: -- Yet another informative stack: [^spark-stop-jstack.log.3] 

[jira] [Updated] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umayr Hassan updated SPARK-24523: - Attachment: spark-stop-jstack.log.3 > InterruptedException when closing SparkContext >

[jira] [Assigned] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25494: Assignee: Apache Spark > Upgrade Spark's use of Janino to 3.0.10 >

[jira] [Assigned] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25494: Assignee: (was: Apache Spark) > Upgrade Spark's use of Janino to 3.0.10 >

[jira] [Commented] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

[ https://issues.apache.org/jira/browse/SPARK-25494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622852#comment-16622852 ] Apache Spark commented on SPARK-25494: -- User 'rednaxelafx' has created a pull request for this

[jira] [Created] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

Kris Mok created SPARK-25494: Summary: Upgrade Spark's use of Janino to 3.0.10 Key: SPARK-25494 URL: https://issues.apache.org/jira/browse/SPARK-25494 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622841#comment-16622841 ] Apache Spark commented on SPARK-25469: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25469: Assignee: (was: Apache Spark) > Eval methods of Concat, Reverse and ElementAt should

[jira] [Commented] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622842#comment-16622842 ] Apache Spark commented on SPARK-25469: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25469: Assignee: Apache Spark > Eval methods of Concat, Reverse and ElementAt should use

[jira] [Commented] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622830#comment-16622830 ] Apache Spark commented on SPARK-25472: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622826#comment-16622826 ] Jungtaek Lim edited comment on SPARK-10816 at 9/20/18 10:48 PM: [~rxin]

[jira] [Commented] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622828#comment-16622828 ] Burak Yavuz commented on SPARK-25472: - Resolved by https://github.com/apache/spark/pull/22478 >

[jira] [Commented] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622827#comment-16622827 ] Apache Spark commented on SPARK-25472: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Resolved] (SPARK-25472) Structured Streaming query.stop() doesn't always stop gracefully

[ https://issues.apache.org/jira/browse/SPARK-25472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-25472. - Resolution: Fixed Fix Version/s: 2.5.0 > Structured Streaming query.stop() doesn't

[jira] [Commented] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622826#comment-16622826 ] Jungtaek Lim commented on SPARK-10816: -- [~rxin] I completely agreed on we would be better to

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622821#comment-16622821 ] Apache Spark commented on SPARK-23715: -- User 'gatorsmile' has created a pull request for this

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622820#comment-16622820 ] Apache Spark commented on SPARK-23715: -- User 'gatorsmile' has created a pull request for this

[jira] [Updated] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umayr Hassan updated SPARK-24523: - Attachment: spark-stop-jstack.log.2 > InterruptedException when closing SparkContext >

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622818#comment-16622818 ] Umayr Hassan commented on SPARK-24523: -- Another stack trace: [^spark-stop-jstack.log.2] >

[jira] [Commented] (SPARK-25480) Dynamic partitioning + saveAsTable with multiple partition columns create empty directory

[ https://issues.apache.org/jira/browse/SPARK-25480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622814#comment-16622814 ] Daniel Mateus Pires commented on SPARK-25480: - * Didn't try accessing S3 without EMR * EMRFS

[jira] [Comment Edited] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622807#comment-16622807 ] Reynold Xin edited comment on SPARK-10816 at 9/20/18 10:30 PM: --- I will let

[jira] [Commented] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622807#comment-16622807 ] Reynold Xin commented on SPARK-10816: - I will let [~marmbrus] chime in ...  As the initial person

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622802#comment-16622802 ] Reynold Xin commented on SPARK-23715: - Great discussions. Since you don't mind, let's revert the

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622790#comment-16622790 ] Umayr Hassan commented on SPARK-24523: -- [~irashid] Attaching a stack trace. 

[jira] [Assigned] (SPARK-25118) Need a solution to persist Spark application console outputs when running in shell/yarn client mode

[ https://issues.apache.org/jira/browse/SPARK-25118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25118: Assignee: Apache Spark > Need a solution to persist Spark application console outputs

[jira] [Assigned] (SPARK-25118) Need a solution to persist Spark application console outputs when running in shell/yarn client mode

[ https://issues.apache.org/jira/browse/SPARK-25118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25118: Assignee: (was: Apache Spark) > Need a solution to persist Spark application console

[jira] [Commented] (SPARK-25118) Need a solution to persist Spark application console outputs when running in shell/yarn client mode

[ https://issues.apache.org/jira/browse/SPARK-25118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622785#comment-16622785 ] Apache Spark commented on SPARK-25118: -- User 'ankuriitg' has created a pull request for this issue:

[jira] [Updated] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umayr Hassan updated SPARK-24523: - Attachment: spark-stop-jstack.log.1 > InterruptedException when closing SparkContext >

[jira] [Resolved] (SPARK-25366) Document Zstd and brotli CompressionCodec requirements for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-25366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25366. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22358

[jira] [Updated] (SPARK-25366) Document Zstd and brotli CompressionCodec requirements for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-25366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25366: -- Component/s: Documentation Issue Type: Improvement (was: Bug) Summary: Document Zstd and

[jira] [Assigned] (SPARK-25366) Document Zstd and brotli CompressionCodec requirements for Parquet files

[ https://issues.apache.org/jira/browse/SPARK-25366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25366: - Assignee: liuxian > Document Zstd and brotli CompressionCodec requirements for Parquet files >

  1   2   >