[jira] [Assigned] (SPARK-27036) Even Broadcast thread is timed out, BroadCast Job is not aborted.

[ https://issues.apache.org/jira/browse/SPARK-27036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27036: Assignee: (was: Apache Spark) > Even Broadcast thread is timed out, BroadCast Job is

[jira] [Assigned] (SPARK-27036) Even Broadcast thread is timed out, BroadCast Job is not aborted.

[ https://issues.apache.org/jira/browse/SPARK-27036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27036: Assignee: Apache Spark > Even Broadcast thread is timed out, BroadCast Job is not

[jira] [Created] (SPARK-27113) remove CHECK_FILES_EXIST_KEY option in file source

Wenchen Fan created SPARK-27113: --- Summary: remove CHECK_FILES_EXIST_KEY option in file source Key: SPARK-27113 URL: https://issues.apache.org/jira/browse/SPARK-27113 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-27056) Remove `start-shuffle-service.sh`

[ https://issues.apache.org/jira/browse/SPARK-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27056. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23975

[jira] [Assigned] (SPARK-27056) Remove `start-shuffle-service.sh`

[ https://issues.apache.org/jira/browse/SPARK-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27056: - Assignee: liuxian > Remove `start-shuffle-service.sh` > -- >

[jira] [Updated] (SPARK-27101) Drop the created database after the test in test_session

[ https://issues.apache.org/jira/browse/SPARK-27101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27101: - Summary: Drop the created database after the test in test_session (was: clean up testcase in

[jira] [Resolved] (SPARK-27101) Drop the created database after the test in test_session

[ https://issues.apache.org/jira/browse/SPARK-27101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27101. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24021

[jira] [Assigned] (SPARK-27101) Drop the created database after the test in test_session

[ https://issues.apache.org/jira/browse/SPARK-27101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27101: Assignee: Sandeep Katta > Drop the created database after the test in test_session >

[jira] [Assigned] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27112: Assignee: Apache Spark > Spark Scheduler encounters two independent Deadlocks when

[jira] [Assigned] (SPARK-27111) A continuous query may fail with InterruptedException when kafka consumer temporally 0 partitions temporally

[ https://issues.apache.org/jira/browse/SPARK-27111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27111: Assignee: Apache Spark (was: Shixiong Zhu) > A continuous query may fail with

[jira] [Assigned] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27112: Assignee: (was: Apache Spark) > Spark Scheduler encounters two independent Deadlocks

[jira] [Assigned] (SPARK-27111) A continuous query may fail with InterruptedException when kafka consumer temporally 0 partitions temporally

[ https://issues.apache.org/jira/browse/SPARK-27111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27111: Assignee: Shixiong Zhu (was: Apache Spark) > A continuous query may fail with

[jira] [Updated] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-27112: - Attachment: Screen Shot 2019-02-26 at 4.10.48 PM.png > Spark Scheduler encounters two

[jira] [Updated] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-27112: - Attachment: Screen Shot 2019-02-26 at 4.11.26 PM.png > Spark Scheduler encounters two

[jira] [Updated] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-27112: - Attachment: Screen Shot 2019-02-26 at 4.11.11 PM.png > Spark Scheduler encounters two

[jira] [Updated] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

[ https://issues.apache.org/jira/browse/SPARK-27112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-27112: - Attachment: Screen Shot 2019-02-26 at 4.10.26 PM.png > Spark Scheduler encounters two

[jira] [Created] (SPARK-27112) Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting

Parth Gandhi created SPARK-27112: Summary: Spark Scheduler encounters two independent Deadlocks when trying to kill executors either due to dynamic allocation or blacklisting Key: SPARK-27112 URL:

[jira] [Created] (SPARK-27111) A continuous query may fail with InterruptedException when kafka consumer temporally 0 partitions temporally

Shixiong Zhu created SPARK-27111: Summary: A continuous query may fail with InterruptedException when kafka consumer temporally 0 partitions temporally Key: SPARK-27111 URL:

[jira] [Assigned] (SPARK-27004) Code for https uri authentication in Spark Submit needs to be removed

[ https://issues.apache.org/jira/browse/SPARK-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27004: Assignee: (was: Apache Spark) > Code for https uri authentication in Spark Submit

[jira] [Assigned] (SPARK-27004) Code for https uri authentication in Spark Submit needs to be removed

[ https://issues.apache.org/jira/browse/SPARK-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27004: Assignee: Apache Spark > Code for https uri authentication in Spark Submit needs to be

[jira] [Resolved] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27044. Resolution: Duplicate > Maven dependency resolution does not support classifiers >

[jira] [Commented] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788369#comment-16788369 ] Imran Rashid commented on SPARK-27097: -- I'm kind of amazed Spark works at all on different

[jira] [Commented] (SPARK-27110) Moves some functions from AnalyzeColumnCommand to command/CommandUtils for reuse

[ https://issues.apache.org/jira/browse/SPARK-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788354#comment-16788354 ] Apache Spark commented on SPARK-27110: -- User 'maropu' has created a pull request for this issue:

[jira] [Updated] (SPARK-27110) Moves some functions from AnalyzeColumnCommand to command/CommandUtils for reuse

[ https://issues.apache.org/jira/browse/SPARK-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27110: - Description: To reuse some common logics for improving {{Analyze}} commands (See the

[jira] [Assigned] (SPARK-27110) Moves some functions from AnalyzeColumnCommand to command/CommandUtils for reuse

[ https://issues.apache.org/jira/browse/SPARK-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27110: Assignee: (was: Apache Spark) > Moves some functions from AnalyzeColumnCommand to

[jira] [Assigned] (SPARK-27110) Moves some functions from AnalyzeColumnCommand to command/CommandUtils for reuse

[ https://issues.apache.org/jira/browse/SPARK-27110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27110: Assignee: Apache Spark > Moves some functions from AnalyzeColumnCommand to

[jira] [Created] (SPARK-27110) Moves some functions from AnalyzeColumnCommand to command/CommandUtils for reuse

Takeshi Yamamuro created SPARK-27110: Summary: Moves some functions from AnalyzeColumnCommand to command/CommandUtils for reuse Key: SPARK-27110 URL: https://issues.apache.org/jira/browse/SPARK-27110

[jira] [Assigned] (SPARK-27109) Refactoring of TimestampFormatter and DateFormatter

[ https://issues.apache.org/jira/browse/SPARK-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27109: Assignee: (was: Apache Spark) > Refactoring of TimestampFormatter and DateFormatter

[jira] [Assigned] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27097: Assignee: Apache Spark (was: Kris Mok) > Avoid embedding platform-dependent offsets

[jira] [Assigned] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27097: Assignee: Kris Mok (was: Apache Spark) > Avoid embedding platform-dependent offsets

[jira] [Assigned] (SPARK-27109) Refactoring of TimestampFormatter and DateFormatter

[ https://issues.apache.org/jira/browse/SPARK-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27109: Assignee: Apache Spark > Refactoring of TimestampFormatter and DateFormatter >

[jira] [Created] (SPARK-27109) Refactoring of TimestampFormatter and DateFormatter

Maxim Gekk created SPARK-27109: -- Summary: Refactoring of TimestampFormatter and DateFormatter Key: SPARK-27109 URL: https://issues.apache.org/jira/browse/SPARK-27109 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-8547) xgboost exploration

[ https://issues.apache.org/jira/browse/SPARK-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8547. -- Resolution: Won't Fix I think the resolution is "xgboost4j-scala" > xgboost exploration >

[jira] [Resolved] (SPARK-9610) Class and instance weighting for ML

[ https://issues.apache.org/jira/browse/SPARK-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9610. -- Resolution: Done Except for GBTs, this is done, so I'm going to close the umbrella > Class and

[jira] [Resolved] (SPARK-9478) Add sample weights to Random Forest

[ https://issues.apache.org/jira/browse/SPARK-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9478. -- Resolution: Duplicate > Add sample weights to Random Forest > --- > >

[jira] [Resolved] (SPARK-14599) BaggedPoint should support weighted instances.

[ https://issues.apache.org/jira/browse/SPARK-14599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14599. --- Resolution: Duplicate Weighted points were added with SPARK-19591 > BaggedPoint should support

[jira] [Assigned] (SPARK-27108) Add parsed CreateTable plans to Catalyst

[ https://issues.apache.org/jira/browse/SPARK-27108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27108: Assignee: (was: Apache Spark) > Add parsed CreateTable plans to Catalyst >

[jira] [Assigned] (SPARK-27108) Add parsed CreateTable plans to Catalyst

[ https://issues.apache.org/jira/browse/SPARK-27108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27108: Assignee: Apache Spark > Add parsed CreateTable plans to Catalyst >

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788206#comment-16788206 ] Eyal Farago commented on SPARK-17556: - why was this abandoned? [~viirya]'s pull request seems

[jira] [Created] (SPARK-27108) Add parsed CreateTable plans to Catalyst

Ryan Blue created SPARK-27108: - Summary: Add parsed CreateTable plans to Catalyst Key: SPARK-27108 URL: https://issues.apache.org/jira/browse/SPARK-27108 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-27103) SparkSql reserved keywords don't list in alphabet order

[ https://issues.apache.org/jira/browse/SPARK-27103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27103. --- Resolution: Fixed Assignee: SongYadong Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-27093) Honor ParseMode in AvroFileFormat

[ https://issues.apache.org/jira/browse/SPARK-27093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788177#comment-16788177 ] Tim Cerexhe commented on SPARK-27093: - Thanks for that idea [~Gengliang.Wang]. It catches some of

[jira] [Assigned] (SPARK-27093) Honor ParseMode in AvroFileFormat

[ https://issues.apache.org/jira/browse/SPARK-27093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27093: Assignee: (was: Apache Spark) > Honor ParseMode in AvroFileFormat >

[jira] [Assigned] (SPARK-27093) Honor ParseMode in AvroFileFormat

[ https://issues.apache.org/jira/browse/SPARK-27093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27093: Assignee: Apache Spark > Honor ParseMode in AvroFileFormat >

[jira] [Assigned] (SPARK-27079) Fix typo & Remove useless imports

[ https://issues.apache.org/jira/browse/SPARK-27079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27079: - Assignee: EdisonWang > Fix typo & Remove useless imports > - >

[jira] [Resolved] (SPARK-27079) Fix typo & Remove useless imports

[ https://issues.apache.org/jira/browse/SPARK-27079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27079. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24000

[jira] [Updated] (SPARK-27079) Fix typo & Remove useless imports

[ https://issues.apache.org/jira/browse/SPARK-27079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27079: -- Priority: Trivial (was: Minor) Yeah [~EdisonWang] this isnt' meaningful as a JIRA. If it's truly

[jira] [Assigned] (SPARK-27090) Deplementing old LEGACY_DRIVER_IDENTIFIER ("")

[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27090: Assignee: (was: Apache Spark) > Deplementing old LEGACY_DRIVER_IDENTIFIER ("") >

[jira] [Assigned] (SPARK-27090) Deplementing old LEGACY_DRIVER_IDENTIFIER ("")

[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27090: Assignee: Apache Spark > Deplementing old LEGACY_DRIVER_IDENTIFIER ("") >

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788075#comment-16788075 ] Hien Luu commented on SPARK-27027: -- Hi [~hyukjin.kwon],  yes I did and here is how I started the

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788021#comment-16788021 ] Dhruve Ashar commented on SPARK-27107: -- [~dongjoon] can you review the PR for ORC to fix this

[jira] [Created] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

Dhruve Ashar created SPARK-27107: Summary: Spark SQL Job failing because of Kryo buffer overflow with ORC Key: SPARK-27107 URL: https://issues.apache.org/jira/browse/SPARK-27107 Project: Spark

[jira] [Commented] (SPARK-25928) NoSuchMethodError net.jpountz.lz4.LZ4BlockInputStream.(Ljava/io/InputStream;Z)V

[ https://issues.apache.org/jira/browse/SPARK-25928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787996#comment-16787996 ] Abhinay commented on SPARK-25928: - We are running into the same issue with EMR 5.20.0 with Oozie 5.0.0

[jira] [Assigned] (SPARK-27106) merge CaseInsensitiveStringMap and DataSourceOptions

[ https://issues.apache.org/jira/browse/SPARK-27106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27106: Assignee: Wenchen Fan (was: Apache Spark) > merge CaseInsensitiveStringMap and

[jira] [Commented] (SPARK-25863) java.lang.UnsupportedOperationException: empty.max at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.updateAndGetCompilationStats(CodeGenerator.scala

[ https://issues.apache.org/jira/browse/SPARK-25863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787989#comment-16787989 ] Val Feldsher commented on SPARK-25863: -- [~maropu] yes it works now, I still see the warning but it

[jira] [Assigned] (SPARK-27106) merge CaseInsensitiveStringMap and DataSourceOptions

[ https://issues.apache.org/jira/browse/SPARK-27106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27106: Assignee: Apache Spark (was: Wenchen Fan) > merge CaseInsensitiveStringMap and

[jira] [Created] (SPARK-27106) merge CaseInsensitiveStringMap and DataSourceOptions

Wenchen Fan created SPARK-27106: --- Summary: merge CaseInsensitiveStringMap and DataSourceOptions Key: SPARK-27106 URL: https://issues.apache.org/jira/browse/SPARK-27106 Project: Spark Issue

[jira] [Resolved] (SPARK-23012) Support for predicate pushdown and partition pruning when left joining large Hive tables

[ https://issues.apache.org/jira/browse/SPARK-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rick Kramer resolved SPARK-23012. - Resolution: Fixed > Support for predicate pushdown and partition pruning when left joining

[jira] [Updated] (SPARK-23012) Support for predicate pushdown and partition pruning when left joining large Hive tables

[ https://issues.apache.org/jira/browse/SPARK-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rick Kramer updated SPARK-23012: Fix Version/s: 2.4.0 > Support for predicate pushdown and partition pruning when left joining

[jira] [Commented] (SPARK-23012) Support for predicate pushdown and partition pruning when left joining large Hive tables

[ https://issues.apache.org/jira/browse/SPARK-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787968#comment-16787968 ] Rick Kramer commented on SPARK-23012: - [~Saurabh Santhosh] We worked around it by materializing the

[jira] [Commented] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787834#comment-16787834 ] Gabor Somogyi commented on SPARK-27044: --- Thanks [~herberts] rebuilt shaded jar was my main

[jira] [Commented] (SPARK-27090) Deplementing old LEGACY_DRIVER_IDENTIFIER ("")

[ https://issues.apache.org/jira/browse/SPARK-27090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787822#comment-16787822 ] Attila Zsolt Piros commented on SPARK-27090: I would have waited a little bit to find out

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787815#comment-16787815 ] Ajith S commented on SPARK-27060: - Just checked on master branch, seems the property doesn't work

[jira] [Assigned] (SPARK-24252) DataSourceV2: Add catalog support

[ https://issues.apache.org/jira/browse/SPARK-24252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24252: --- Assignee: Ryan Blue > DataSourceV2: Add catalog support >

[jira] [Comment Edited] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787797#comment-16787797 ] Ajith S edited comment on SPARK-27060 at 3/8/19 11:20 AM: -- [~sachin1729] can

[jira] [Resolved] (SPARK-24252) DataSourceV2: Add catalog support

[ https://issues.apache.org/jira/browse/SPARK-24252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24252. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23915

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787803#comment-16787803 ] Ajith S commented on SPARK-27060: - Please refer SPARK-26215 > DDL Commands are accepting Keywords like

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787797#comment-16787797 ] Ajith S commented on SPARK-27060: - [~sachin1729] can setting *spark.sql.parser.ansi.enabled=true* fix

[jira] [Commented] (SPARK-27098) Flaky missing file parts when writing to Ceph without error

[ https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787792#comment-16787792 ] Steve Loughran commented on SPARK-27098: It was my suggestion to file this. * If this was AWS

[jira] [Commented] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787788#comment-16787788 ] Mathias Herberts commented on SPARK-27044: -- This is exactly what Jungtaek said, the ability to

[jira] [Commented] (SPARK-24291) Data source table is not displaying records when files are uploaded to table location

[ https://issues.apache.org/jira/browse/SPARK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787786#comment-16787786 ] Sushanta Sen commented on SPARK-24291: -- Yes,after refresh it fetches the data.But why this is

[jira] [Commented] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787784#comment-16787784 ] Gabor Somogyi commented on SPARK-27044: --- I've already started but wanted to double check things

[jira] [Updated] (SPARK-27098) Flaky missing file parts when writing to Ceph without error

[ https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-27098: --- Description:

[jira] [Comment Edited] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787762#comment-16787762 ] Jungtaek Lim edited comment on SPARK-27044 at 3/8/19 10:39 AM: --- It's like

[jira] [Created] (SPARK-27105) Prevent exponential complexity in ORC `createFilter`

Ivan Vergiliev created SPARK-27105: -- Summary: Prevent exponential complexity in ORC `createFilter` Key: SPARK-27105 URL: https://issues.apache.org/jira/browse/SPARK-27105 Project: Spark

[jira] [Commented] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787762#comment-16787762 ] Jungtaek Lim commented on SPARK-27044: -- It's like "hive-exec" with default classifier vs "core"

[jira] [Commented] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787754#comment-16787754 ] Gabor Somogyi commented on SPARK-27044: --- [~herberts] Could you give an example of {quote}fetch

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787749#comment-16787749 ] Hyukjin Kwon commented on SPARK-27100: -- Hm, can you show reproducible steps including codes? I

[jira] [Resolved] (SPARK-27087) Inability to access to column alias in pyspark

[ https://issues.apache.org/jira/browse/SPARK-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27087. -- Resolution: Invalid > Inability to access to column alias in pyspark >

[jira] [Commented] (SPARK-27087) Inability to access to column alias in pyspark

[ https://issues.apache.org/jira/browse/SPARK-27087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787732#comment-16787732 ] Hyukjin Kwon commented on SPARK-27087: -- Scala side shows the same output as well: {code} scala>

[jira] [Updated] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27027: - Component/s: Spark Shell > from_avro function does not deserialize the Avro record of a struct

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787722#comment-16787722 ] Hyukjin Kwon commented on SPARK-27027: -- [~hluu], just to confirm, did you face this issue in Spark

[jira] [Updated] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27027: - Affects Version/s: 3.0.0 > from_avro function does not deserialize the Avro record of a struct

[jira] [Reopened] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-27027: -- > from_avro function does not deserialize the Avro record of a struct column > type correctly >

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787720#comment-16787720 ] Gabor Somogyi commented on SPARK-27027: --- After compile {quote}./bin/spark-shell --packages

[jira] [Commented] (SPARK-23012) Support for predicate pushdown and partition pruning when left joining large Hive tables

[ https://issues.apache.org/jira/browse/SPARK-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787716#comment-16787716 ] Saurabh Santhosh commented on SPARK-23012: -- [~yumwang] [~reks95] Hi, tested this in Spark

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787708#comment-16787708 ] Hyukjin Kwon commented on SPARK-27027: -- How did you run spark-shell? > from_avro function does not

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787701#comment-16787701 ] Gabor Somogyi commented on SPARK-27027: --- With spark-shell, in unit test the functionality works.

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787695#comment-16787695 ] Hyukjin Kwon commented on SPARK-27027: -- So, how did we reproduce in the current master? Please set

[jira] [Updated] (SPARK-27104) Spark Fair scheduler across applications in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-27104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Zhang updated SPARK-27104: -- Description: Spark in standalone mode currently only supports FIFO (first-in-first-out) scheduler 

[jira] [Updated] (SPARK-27104) Spark Fair scheduler across applications in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-27104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Zhang updated SPARK-27104: -- Description: Spark in standalone mode currently only supports FIFO (first-in-first-out) scheduler 

[jira] [Created] (SPARK-27104) Spark Fair scheduler across applications in standalone mode

Hua Zhang created SPARK-27104: - Summary: Spark Fair scheduler across applications in standalone mode Key: SPARK-27104 URL: https://issues.apache.org/jira/browse/SPARK-27104 Project: Spark Issue

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787662#comment-16787662 ] Gabor Somogyi commented on SPARK-27027: --- Please see the comments. The issue fully reproducible. >

[jira] [Comment Edited] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787662#comment-16787662 ] Gabor Somogyi edited comment on SPARK-27027 at 3/8/19 8:34 AM: --- Please see