[jira] [Resolved] (SPARK-25391) Make behaviors consistent when converting parquet hive table to parquet data source

[ https://issues.apache.org/jira/browse/SPARK-25391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxiao Mao resolved SPARK-25391. -- Resolution: Won't Do > Make behaviors consistent when converting parquet hive table to parquet

[jira] [Commented] (SPARK-25442) Support STS to run in K8S deployment with spark deployment mode as cluster

[ https://issues.apache.org/jira/browse/SPARK-25442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616601#comment-16616601 ] Suryanarayana Garlapati commented on SPARK-25442: - Following is the PR f

[jira] [Created] (SPARK-25442) Support STS to run in K8S deployment with spark deployment mode as cluster

Suryanarayana Garlapati created SPARK-25442: --- Summary: Support STS to run in K8S deployment with spark deployment mode as cluster Key: SPARK-25442 URL: https://issues.apache.org/jira/browse/SPARK-25442

[jira] [Updated] (SPARK-24479) Register StreamingQueryListener in Spark Conf

[ https://issues.apache.org/jira/browse/SPARK-24479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24479: Labels: (was: feature) > Register StreamingQueryListener in Spark Conf > --

[jira] [Comment Edited] (SPARK-25434) failed to locate the winutils binary in the hadoop binary path

[ https://issues.apache.org/jira/browse/SPARK-25434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616578#comment-16616578 ] Dongjoon Hyun edited comment on SPARK-25434 at 9/16/18 3:44 AM: --

[jira] [Resolved] (SPARK-25434) failed to locate the winutils binary in the hadoop binary path

[ https://issues.apache.org/jira/browse/SPARK-25434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25434. --- Resolution: Not A Problem > failed to locate the winutils binary in the hadoop binary path >

[jira] [Commented] (SPARK-25434) failed to locate the winutils binary in the hadoop binary path

[ https://issues.apache.org/jira/browse/SPARK-25434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616578#comment-16616578 ] Dongjoon Hyun commented on SPARK-25434: --- Welcome to the Apache Spark community, [~

[jira] [Resolved] (SPARK-25439) TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25439. - Resolution: Fixed Assignee: Nicolas Poggi Fix Version/s: 2.4.0 > TPCHQuerySuite customer

[jira] [Assigned] (SPARK-23748) Support select from temp tables

[ https://issues.apache.org/jira/browse/SPARK-23748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23748: --- Assignee: Saisai Shao > Support select from temp tables > --- > >

[jira] [Assigned] (SPARK-23503) continuous execution should sequence committed epochs

[ https://issues.apache.org/jira/browse/SPARK-23503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23503: --- Assignee: Efim Poberezkin > continuous execution should sequence committed epochs > ---

[jira] [Updated] (SPARK-22238) EnsureStatefulOpPartitioning shouldn't ask for the child RDD before planning is completed

[ https://issues.apache.org/jira/browse/SPARK-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22238: Fix Version/s: (was: 2.4.0) 2.3.2 > EnsureStatefulOpPartitioning shouldn't ask for

[jira] [Updated] (SPARK-22238) EnsureStatefulOpPartitioning shouldn't ask for the child RDD before planning is completed

[ https://issues.apache.org/jira/browse/SPARK-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22238: Fix Version/s: (was: 2.3.2) 2.3.0 > EnsureStatefulOpPartitioning shouldn't ask for

[jira] [Updated] (SPARK-22956) Union Stream Failover Cause `IllegalStateException`

[ https://issues.apache.org/jira/browse/SPARK-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22956: Fix Version/s: (was: 2.4.0) > Union Stream Failover Cause `IllegalStateException` > --

[jira] [Updated] (SPARK-22018) Catalyst Optimizer does not preserve top-level metadata while collapsing projects

[ https://issues.apache.org/jira/browse/SPARK-22018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22018: Fix Version/s: (was: 2.4.0) 2.3.0 > Catalyst Optimizer does not preserve top-level

[jira] [Updated] (SPARK-22017) watermark evaluation with multi-input stream operators is unspecified

[ https://issues.apache.org/jira/browse/SPARK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22017: Fix Version/s: (was: 2.3.2) 2.3.0 > watermark evaluation with multi-input stream op

[jira] [Updated] (SPARK-22017) watermark evaluation with multi-input stream operators is unspecified

[ https://issues.apache.org/jira/browse/SPARK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22017: Fix Version/s: (was: 2.4.0) 2.3.2 > watermark evaluation with multi-input stream op

[jira] [Commented] (SPARK-25425) Extra options must overwrite sessions options

[ https://issues.apache.org/jira/browse/SPARK-25425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616554#comment-16616554 ] Dongjoon Hyun commented on SPARK-25425: --- This is resolved via https://github.com/a

[jira] [Resolved] (SPARK-25438) Fix FilterPushdownBenchmark to use the same memory assumption

[ https://issues.apache.org/jira/browse/SPARK-25438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25438. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22427 [https://

[jira] [Commented] (SPARK-25431) Fix function examples and unify the format of the example results.

[ https://issues.apache.org/jira/browse/SPARK-25431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616552#comment-16616552 ] Dongjoon Hyun commented on SPARK-25431: --- I reopened this since it's reverted now.

[jira] [Updated] (SPARK-25431) Fix function examples and unify the format of the example results.

[ https://issues.apache.org/jira/browse/SPARK-25431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25431: -- Fix Version/s: (was: 2.4.0) > Fix function examples and unify the format of the example re

[jira] [Reopened] (SPARK-25431) Fix function examples and unify the format of the example results.

[ https://issues.apache.org/jira/browse/SPARK-25431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-25431: --- > Fix function examples and unify the format of the example results. > -

[jira] [Resolved] (SPARK-25425) Extra options must overwrite sessions options

[ https://issues.apache.org/jira/browse/SPARK-25425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25425. --- Resolution: Fixed Fix Version/s: 2.5.0 > Extra options must overwrite sessions option

[jira] [Assigned] (SPARK-25425) Extra options must overwrite sessions options

[ https://issues.apache.org/jira/browse/SPARK-25425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25425: - Assignee: Maxim Gekk > Extra options must overwrite sessions options >

[jira] [Commented] (SPARK-25434) failed to locate the winutils binary in the hadoop binary path

[ https://issues.apache.org/jira/browse/SPARK-25434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616545#comment-16616545 ] WEI PENG commented on SPARK-25434: -- Thank you, [~VeenitShah] , it works!! > failed to

[jira] [Resolved] (SPARK-25436) Bump master branch version to 2.5.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-25436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25436. - Resolution: Fixed Fix Version/s: 2.5.0 > Bump master branch version to 2.5.0-SNAPSHOT > -

[jira] [Resolved] (SPARK-25426) Remove the duplicate fallback logic in UnsafeProjection

[ https://issues.apache.org/jira/browse/SPARK-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25426. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.5.0 > Remove the duplicate

[jira] [Updated] (SPARK-25439) [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25439: -- Component/s: SQL > [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead o

[jira] [Updated] (SPARK-25439) TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25439: -- Summary: TPCHQuerySuite customer.c_nationkey should be bigint instead of string (was: [TESTS]

[jira] [Updated] (SPARK-25439) TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25439: -- Issue Type: Bug (was: Improvement) > TPCHQuerySuite customer.c_nationkey should be bigint ins

[jira] [Updated] (SPARK-25439) [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25439: -- Affects Version/s: 2.4.0 2.3.0 > [TESTS][SQL] TPCHQuerySuite customer.c

[jira] [Updated] (SPARK-25425) Extra options must overwrite sessions options

[ https://issues.apache.org/jira/browse/SPARK-25425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25425: -- Affects Version/s: 2.4.0 > Extra options must overwrite sessions options > ---

[jira] [Commented] (SPARK-25434) failed to locate the winutils binary in the hadoop binary path

[ https://issues.apache.org/jira/browse/SPARK-25434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616489#comment-16616489 ] Veenit Shah commented on SPARK-25434: - Are you on Windows? I faced the same issue. T

[jira] [Created] (SPARK-25441) calculate term frequency in CountVectorizer()

Xinyong Tian created SPARK-25441: Summary: calculate term frequency in CountVectorizer() Key: SPARK-25441 URL: https://issues.apache.org/jira/browse/SPARK-25441 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-25303) A DStream that is checkpointed should allow its parent(s) to be removed and not persisted

[ https://issues.apache.org/jira/browse/SPARK-25303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616381#comment-16616381 ] Nikunj Bansal commented on SPARK-25303: --- Patch available at PR [#22424|https://git

[jira] [Commented] (SPARK-25302) ReducedWindowedDStream not using checkpoints for reduced RDDs

[ https://issues.apache.org/jira/browse/SPARK-25302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616378#comment-16616378 ] Nikunj Bansal commented on SPARK-25302: --- Patch available at PR [#22423|https://git

[jira] [Comment Edited] (SPARK-25439) [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616370#comment-16616370 ] Nicolas Poggi edited comment on SPARK-25439 at 9/15/18 4:10 PM: --

[jira] [Commented] (SPARK-25439) [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616370#comment-16616370 ] Nicolas Poggi commented on SPARK-25439: --- Created the[ PR with the patch|[https://g

[jira] [Created] (SPARK-25440) Dump query execution info to a file

Maxim Gekk created SPARK-25440: -- Summary: Dump query execution info to a file Key: SPARK-25440 URL: https://issues.apache.org/jira/browse/SPARK-25440 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-25439) [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string

[ https://issues.apache.org/jira/browse/SPARK-25439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Poggi updated SPARK-25439: -- Description:   The [TPCHQuerySuite|https://github.com/apache/spark/blob/be454a7cef1cb5c76fb2

[jira] [Created] (SPARK-25439) [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string

Nicolas Poggi created SPARK-25439: - Summary: [TESTS][SQL] TPCHQuerySuite customer.c_nationkey should be bigint instead of string Key: SPARK-25439 URL: https://issues.apache.org/jira/browse/SPARK-25439

[jira] [Assigned] (SPARK-25438) Fix FilterPushdownBenchmark to use the same memory assumption

[ https://issues.apache.org/jira/browse/SPARK-25438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25438: - Assignee: Dongjoon Hyun > Fix FilterPushdownBenchmark to use the same memory assumption

[jira] [Updated] (SPARK-25438) Fix FilterPushdownBenchmark to use the same memory assumption

[ https://issues.apache.org/jira/browse/SPARK-25438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25438: -- Description: This issue aims to fix three things in `FilterPushdownBenchmark`. 1. Use the sam

[jira] [Commented] (SPARK-15041) adding mode strategy for ml.feature.Imputer for categorical features

[ https://issues.apache.org/jira/browse/SPARK-15041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616243#comment-16616243 ] Manu Zhang commented on SPARK-15041: Is there a plan to add such strategies as min/m

[jira] [Created] (SPARK-25438) Fix FilterPushdownBenchmark to use the same memory assumption

Dongjoon Hyun created SPARK-25438: - Summary: Fix FilterPushdownBenchmark to use the same memory assumption Key: SPARK-25438 URL: https://issues.apache.org/jira/browse/SPARK-25438 Project: Spark

[jira] [Assigned] (SPARK-25427) Add BloomFilter creation test cases

[ https://issues.apache.org/jira/browse/SPARK-25427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25427: - Assignee: Dongjoon Hyun > Add BloomFilter creation test cases > ---

[jira] [Updated] (SPARK-25425) Extra options must overwrite sessions options

[ https://issues.apache.org/jira/browse/SPARK-25425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25425: -- Affects Version/s: 2.3.0 > Extra options must overwrite sessions options > ---