[jira] [Assigned] (SPARK-27225) Implement join strategy hints

2019-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27225: --- Assignee: Maryann Xue > Implement join strategy hints > - > >

[jira] [Resolved] (SPARK-27225) Implement join strategy hints

2019-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27225. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24164

[jira] [Created] (SPARK-27447) Add collaborate filtering Explain API in SPARKML

2019-04-11 Thread guohao xiao (JIRA)
guohao xiao created SPARK-27447: --- Summary: Add collaborate filtering Explain API in SPARKML Key: SPARK-27447 URL: https://issues.apache.org/jira/browse/SPARK-27447 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-27270) Add Kafka dynamic JAAS authentication debug possibility

2019-04-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27270: -- Assignee: Gabor Somogyi > Add Kafka dynamic JAAS authentication debug possibility >

[jira] [Resolved] (SPARK-27270) Add Kafka dynamic JAAS authentication debug possibility

2019-04-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27270. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24204

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-11 Thread Robert Joseph Evans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815846#comment-16815846 ] Robert Joseph Evans commented on SPARK-27396: - [~kiszk],   The exact detail of some of the

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815812#comment-16815812 ] Kazuaki Ishizaki commented on SPARK-27396: -- Sorry for being late to comment since I am

[jira] [Resolved] (SPARK-27445) Update SQLQueryTestSuite to process files ending with `.sql`

2019-04-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27445. --- Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 3.0.0 This is

[jira] [Created] (SPARK-27446) RBackend always uses default values for spark confs

2019-04-11 Thread Bago Amirbekian (JIRA)
Bago Amirbekian created SPARK-27446: --- Summary: RBackend always uses default values for spark confs Key: SPARK-27446 URL: https://issues.apache.org/jira/browse/SPARK-27446 Project: Spark

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2019-04-11 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815763#comment-16815763 ] Victor Tso commented on SPARK-20144: This one was clearly decided against. I ended up writing my own

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2019-04-11 Thread David Greenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815755#comment-16815755 ] David Greenberg commented on SPARK-20144: - Hello, this issue is also a major one for me. Almost

[jira] [Resolved] (SPARK-27404) Fix build warnings for 3.0: postfixOps edition

2019-04-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27404. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24314

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815706#comment-16815706 ] Hyukjin Kwon commented on SPARK-27442: -- Why does Spark has to be able to read it? IIRC, this

[jira] [Commented] (SPARK-27440) Optimize uncorrelated predicate subquery

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815705#comment-16815705 ] Hyukjin Kwon commented on SPARK-27440: -- Please avoid to set Fix Version which is usually set when

[jira] [Updated] (SPARK-27440) Optimize uncorrelated predicate subquery

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27440: - Fix Version/s: (was: 3.0.0) > Optimize uncorrelated predicate subquery >

[jira] [Resolved] (SPARK-27433) Spark Structured Streaming left outer joins returns outer nulls for already matched rows

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27433. -- Resolution: Invalid > Spark Structured Streaming left outer joins returns outer nulls for

[jira] [Commented] (SPARK-27433) Spark Structured Streaming left outer joins returns outer nulls for already matched rows

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815703#comment-16815703 ] Hyukjin Kwon commented on SPARK-27433: -- Let me leave this resolved anyway. Questions should go to

[jira] [Commented] (SPARK-27432) Spark job stuck when no jobs/stages are pending

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815700#comment-16815700 ] Hyukjin Kwon commented on SPARK-27432: -- Can you describe how you reproduce this? Otherwise, no one

[jira] [Commented] (SPARK-27431) move HashedRelation to global UnifiedMemoryManager and enable offheap

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815699#comment-16815699 ] Hyukjin Kwon commented on SPARK-27431: -- Sounds like a question. If you're not sure, ask it to

[jira] [Resolved] (SPARK-27431) move HashedRelation to global UnifiedMemoryManager and enable offheap

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27431. -- Resolution: Invalid > move HashedRelation to global UnifiedMemoryManager and enable offheap >

[jira] [Commented] (SPARK-27429) [SQL] to_timestamp function with additional argument flag that will allow exception if value could not be cast

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815697#comment-16815697 ] Hyukjin Kwon commented on SPARK-27429: -- Usually we take an approach to permissively process them. I

[jira] [Commented] (SPARK-27428) Test "metrics StatsD sink with Timer " fails on BigEndian

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815696#comment-16815696 ] Hyukjin Kwon commented on SPARK-27428: -- I am sure there are multiple BigEndian issues. Can you link

[jira] [Resolved] (SPARK-27408) functions.coalesce working on csv but not on Mongospark

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27408. -- Resolution: Invalid Sounds like the issue is specific to mongo-spark > functions.coalesce

[jira] [Resolved] (SPARK-27385) use mapPartitionsWIthIndex in Dataframe

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27385. -- Resolution: Won't Fix > use mapPartitionsWIthIndex in Dataframe >

[jira] [Commented] (SPARK-27385) use mapPartitionsWIthIndex in Dataframe

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815693#comment-16815693 ] Hyukjin Kwon commented on SPARK-27385: -- I think we can just use

[jira] [Resolved] (SPARK-27353) PySpark Row __repr__ bug

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27353. -- Resolution: Cannot Reproduce > PySpark Row __repr__ bug > -- > >

[jira] [Resolved] (SPARK-27352) Apply for translation of the Chinese version, I hope to get authorization!

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27352. -- Resolution: Invalid Let me leave this resolved since spark-website doesn't need a JIRA

[jira] [Commented] (SPARK-27352) Apply for translation of the Chinese version, I hope to get authorization!

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815690#comment-16815690 ] Hyukjin Kwon commented on SPARK-27352: -- Maybe we can consider listing up in

[jira] [Resolved] (SPARK-27375) cache not working after discretizer.fit(df).transform(df)

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27375. -- Resolution: Cannot Reproduce Looks it's fixed in the upstream. It should be great if the JIRA

[jira] [Resolved] (SPARK-27345) Expose approx_percentile in python api

2019-04-11 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27345. -- Resolution: Won't Fix > Expose approx_percentile in python api >

[jira] [Assigned] (SPARK-27387) Replace sqlutils assertPandasEqual with Pandas assert_frame_equal in tests

2019-04-11 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-27387: Assignee: Bryan Cutler > Replace sqlutils assertPandasEqual with Pandas

[jira] [Created] (SPARK-27445) Update SQLQueryTestSuite to process files ending with `.sql`

2019-04-11 Thread Dilip Biswal (JIRA)
Dilip Biswal created SPARK-27445: Summary: Update SQLQueryTestSuite to process files ending with `.sql` Key: SPARK-27445 URL: https://issues.apache.org/jira/browse/SPARK-27445 Project: Spark

[jira] [Updated] (SPARK-27388) expression encoder for avro objects

2019-04-11 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Summary: expression encoder for avro objects (was: expression encoder for avro like

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-11 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: *What changes were proposed in this pull request?* The PR adds support

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-11 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: *What changes were proposed in this pull request?* This PR adds

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-11 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: *What changes were proposed in this pull request?* This PR adds

[jira] [Resolved] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

2019-04-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27403. --- Resolution: Fixed Assignee: Sujith Chacko Fix Version/s: 3.0.0 This is

[jira] [Comment Edited] (SPARK-27409) Micro-batch support for Kafka Source in Spark 2.3

2019-04-11 Thread Prabhjot Singh Bharaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815491#comment-16815491 ] Prabhjot Singh Bharaj edited comment on SPARK-27409 at 4/11/19 3:10 PM:

[jira] [Comment Edited] (SPARK-27409) Micro-batch support for Kafka Source in Spark 2.3

2019-04-11 Thread Prabhjot Singh Bharaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815491#comment-16815491 ] Prabhjot Singh Bharaj edited comment on SPARK-27409 at 4/11/19 3:10 PM:

[jira] [Commented] (SPARK-27409) Micro-batch support for Kafka Source in Spark 2.3

2019-04-11 Thread Prabhjot Singh Bharaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815491#comment-16815491 ] Prabhjot Singh Bharaj commented on SPARK-27409: --- {code:java} In [5]: df =

[jira] [Comment Edited] (SPARK-27364) User-facing APIs for GPU-aware scheduling

2019-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815465#comment-16815465 ] Thomas Graves edited comment on SPARK-27364 at 4/11/19 2:37 PM: So there

[jira] [Commented] (SPARK-27364) User-facing APIs for GPU-aware scheduling

2019-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815465#comment-16815465 ] Thomas Graves commented on SPARK-27364: --- So there is actually another one we need for standalone

[jira] [Created] (SPARK-27444) multi-select can be used in subquery

2019-04-11 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-27444: --- Summary: multi-select can be used in subquery Key: SPARK-27444 URL: https://issues.apache.org/jira/browse/SPARK-27444 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-27443) Support UDF input_file_name in file source V2

2019-04-11 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27443: -- Summary: Support UDF input_file_name in file source V2 Key: SPARK-27443 URL: https://issues.apache.org/jira/browse/SPARK-27443 Project: Spark Issue

[jira] [Updated] (SPARK-27435) Support schema pruning in Orc V2

2019-04-11 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-27435: --- Description: Currently, the optimization rule `SchemaPruning` only works for Parquet/Orc

[jira] [Updated] (SPARK-27435) Support schema pruning in Orc V2

2019-04-11 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-27435: --- Summary: Support schema pruning in Orc V2 (was: Support schema pruning in file source V2)

[jira] [Commented] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-04-11 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815435#comment-16815435 ] shahid commented on SPARK-27068: Hi [~vanzin], Currently when cleaning up jobs when the UI job limit

[jira] [Comment Edited] (SPARK-27364) User-facing APIs for GPU-aware scheduling

2019-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812488#comment-16812488 ] Thomas Graves edited comment on SPARK-27364 at 4/11/19 1:48 PM: There

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-11 Thread Robert Joseph Evans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815426#comment-16815426 ] Robert Joseph Evans commented on SPARK-27396: - Thanks [~tgraves] I updated the JIRA with you

[jira] [Updated] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-11 Thread Robert Joseph Evans (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated SPARK-27396: Shepherd: Thomas Graves > SPIP: Public APIs for extended Columnar Processing

[jira] [Commented] (SPARK-27434) memory leak in spark driver

2019-04-11 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815424#comment-16815424 ] shahid commented on SPARK-27434: Could you please provide steps for reproducing the issue? > memory

[jira] [Commented] (SPARK-27176) Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

2019-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815414#comment-16815414 ] Thomas Graves commented on SPARK-27176: --- looks like I see one:

[jira] [Commented] (SPARK-27176) Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

2019-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815408#comment-16815408 ] Thomas Graves commented on SPARK-27176: --- It looks like the hadoop-3.2 profile no longer works, do

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815403#comment-16815403 ] Thomas Graves commented on SPARK-27396: --- I can shephard it. > SPIP: Public APIs for extended

[jira] [Assigned] (SPARK-27436) Add spark.sql.optimizer.nonExcludedRules

2019-04-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-27436: - Assignee: Dongjoon Hyun > Add spark.sql.optimizer.nonExcludedRules >

[jira] [Created] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2019-04-11 Thread JIRA
Jan Vršovský created SPARK-27442: Summary: ParquetFileFormat fails to read column named with invalid characters Key: SPARK-27442 URL: https://issues.apache.org/jira/browse/SPARK-27442 Project: Spark

[jira] [Assigned] (SPARK-27435) Support schema pruning in file source V2

2019-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27435: --- Assignee: Gengliang Wang > Support schema pruning in file source V2 >

[jira] [Resolved] (SPARK-27435) Support schema pruning in file source V2

2019-04-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27435. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24338

[jira] [Updated] (SPARK-27405) Restrict the range of generated random timestamps

2019-04-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27405: -- Priority: Minor (was: Trivial) > Restrict the range of generated random timestamps >

[jira] [Resolved] (SPARK-27405) Restrict the range of generated random timestamps

2019-04-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27405. --- Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 3.0.0 This is

[jira] [Resolved] (SPARK-27407) File source V2: Invalidate cache data on overwrite/append

2019-04-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27407. --- Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 This is

[jira] [Updated] (SPARK-27441) Add read/write tests to Hive serde tables

2019-04-11 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27441: Description: The versions between Hive, Parquet and ORC after the built-in Hive upgrade to

[jira] [Created] (SPARK-27441) Add read/write tests to Hive serde tables

2019-04-11 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27441: --- Summary: Add read/write tests to Hive serde tables Key: SPARK-27441 URL: https://issues.apache.org/jira/browse/SPARK-27441 Project: Spark Issue Type:

[jira] [Created] (SPARK-27440) Optimize uncorrelated predicate subquery

2019-04-11 Thread Mingcong Han (JIRA)
Mingcong Han created SPARK-27440: Summary: Optimize uncorrelated predicate subquery Key: SPARK-27440 URL: https://issues.apache.org/jira/browse/SPARK-27440 Project: Spark Issue Type:

[jira] [Created] (SPARK-27439) createOrReplaceTempView cannot update old dataset

2019-04-11 Thread xjl (JIRA)
xjl created SPARK-27439: --- Summary: createOrReplaceTempView cannot update old dataset Key: SPARK-27439 URL: https://issues.apache.org/jira/browse/SPARK-27439 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27300) Create the new graph projects in Spark and set up build/test

2019-04-11 Thread Martin Junghanns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815241#comment-16815241 ] Martin Junghanns commented on SPARK-27300: -- I will work on this. [~mengxr] please assign me. >

[jira] [Commented] (SPARK-27433) Spark Structured Streaming left outer joins returns outer nulls for already matched rows

2019-04-11 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815213#comment-16815213 ] Jungtaek Lim commented on SPARK-27433: -- I guess you're hitting SPARK-26154. Could you please check

[jira] [Updated] (SPARK-26154) Stream-stream joins - left outer join gives inconsistent output

2019-04-11 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-26154: - Priority: Blocker (was: Critical) > Stream-stream joins - left outer join gives inconsistent

[jira] [Commented] (SPARK-26154) Stream-stream joins - left outer join gives inconsistent output

2019-04-11 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815211#comment-16815211 ] Jungtaek Lim commented on SPARK-26154: -- This issue is reported again: SPARK-27433. I'd say it

[jira] [Resolved] (SPARK-27437) check the parquet column format

2019-04-11 Thread xzh_dz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xzh_dz resolved SPARK-27437. Resolution: Won't Fix > check the parquet column format > --- > >