[jira] [Commented] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219727#comment-17219727 ] Rohit Mishra commented on SPARK-33220: -- [~angerszhuuu], Can you please add the description? >

[jira] [Commented] (SPARK-32935) File source V2: support bucketing

[ https://issues.apache.org/jira/browse/SPARK-32935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198242#comment-17198242 ] Rohit Mishra commented on SPARK-32935: -- [~Gengliang.Wang], Can you please add a description? >

[jira] [Commented] (SPARK-32838) Connot overwite different partition with same table

[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194232#comment-17194232 ] Rohit Mishra commented on SPARK-32838: -- [~CHENXCHEN], Please refrain from marking priority as

[jira] [Updated] (SPARK-32838) Connot overwite different partition with same table

[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32838: - Priority: Major (was: Critical) > Connot overwite different partition with same table >

[jira] [Commented] (SPARK-32801) Make InferFiltersFromConstraints take in account EqualNullSafe

[ https://issues.apache.org/jira/browse/SPARK-32801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191020#comment-17191020 ] Rohit Mishra commented on SPARK-32801: -- [~tanelk], Can you please add a description? > Make

[jira] [Commented] (SPARK-32180) Getting Started - Installation

[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190247#comment-17190247 ] Rohit Mishra commented on SPARK-32180: -- Hi [~hyukjin.kwon], I have created a new pull request as

[jira] [Commented] (SPARK-32770) Add missing imports

[ https://issues.apache.org/jira/browse/SPARK-32770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189030#comment-17189030 ] Rohit Mishra commented on SPARK-32770: -- [~fokko], While you are working on the PR, can you please

[jira] [Commented] (SPARK-32180) Getting Started - Installation

[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188366#comment-17188366 ] Rohit Mishra commented on SPARK-32180: -- Hi [~hyukjin.kwon], I have updated the pull request with

[jira] [Commented] (SPARK-32762) Enhance the verification of sql-expression-schema.md

[ https://issues.apache.org/jira/browse/SPARK-32762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188364#comment-17188364 ] Rohit Mishra commented on SPARK-32762: -- [~LuciferYang], Can you please add a description. >

[jira] [Updated] (SPARK-32723) Security Vulnerability due to JQuery version in Spark Master/Worker UI

[ https://issues.apache.org/jira/browse/SPARK-32723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32723: - Target Version/s: (was: 3.1.0) > Security Vulnerability due to JQuery version in Spark

[jira] [Commented] (SPARK-32723) Security Vulnerability due to JQuery version in Spark Master/Worker UI

[ https://issues.apache.org/jira/browse/SPARK-32723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186403#comment-17186403 ] Rohit Mishra commented on SPARK-32723: -- [~ashish23aks], Please refrain from marking the target

[jira] [Commented] (SPARK-32692) Support INSERT OVERWRITE DIR cross cluster

[ https://issues.apache.org/jira/browse/SPARK-32692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183611#comment-17183611 ] Rohit Mishra commented on SPARK-32692: -- [~angerszhuuu], Can you please add a description? >

[jira] [Commented] (SPARK-32660) Show Avro related API in documentation

[ https://issues.apache.org/jira/browse/SPARK-32660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181404#comment-17181404 ] Rohit Mishra commented on SPARK-32660: -- [~Gengliang.Wang], Can you please add a description? >

[jira] [Commented] (SPARK-32668) HiveGenericUDTF initialize UDTF should use StructObjectInspector method

[ https://issues.apache.org/jira/browse/SPARK-32668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181403#comment-17181403 ] Rohit Mishra commented on SPARK-32668: -- [~ulysses], Can you please add a description? >

[jira] [Commented] (SPARK-32669) test expression nullability when checking result

[ https://issues.apache.org/jira/browse/SPARK-32669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181401#comment-17181401 ] Rohit Mishra commented on SPARK-32669: -- [~cloud_fan], Can you please add a description? > test

[jira] [Commented] (SPARK-32670) Group exception messages in Catalyst Analyzer in one file

[ https://issues.apache.org/jira/browse/SPARK-32670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181400#comment-17181400 ] Rohit Mishra commented on SPARK-32670: -- [~smilegator], Can you kindly add more to the description?

[jira] [Commented] (SPARK-32667) Scrip transformation no-serde mode when column less then output length , Use null fill

[ https://issues.apache.org/jira/browse/SPARK-32667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181393#comment-17181393 ] Rohit Mishra commented on SPARK-32667: -- [~angerszhuuu], Can you please add a description? > Scrip

[jira] [Commented] (SPARK-32180) Getting Started - Installation

[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179996#comment-17179996 ] Rohit Mishra commented on SPARK-32180: -- Hi [~hyukjin.kwon], if you have some time can you please

[jira] [Commented] (SPARK-32636) AsyncEventQueue: Exception scala.Some cannot be cast to java.lang.String

[ https://issues.apache.org/jira/browse/SPARK-32636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179354#comment-17179354 ] Rohit Mishra commented on SPARK-32636: -- [~abubakarj], Thanks for raising the bug but please refrain

[jira] [Updated] (SPARK-32636) AsyncEventQueue: Exception scala.Some cannot be cast to java.lang.String

[ https://issues.apache.org/jira/browse/SPARK-32636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32636: - Fix Version/s: (was: 3.0.1) > AsyncEventQueue: Exception scala.Some cannot be cast to

[jira] [Commented] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178148#comment-17178148 ] Rohit Mishra commented on SPARK-32613: -- [~dagrawal3409], Please avoid populating Fix version/s as

[jira] [Updated] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32613: - Fix Version/s: (was: 3.1.0) > DecommissionWorkerSuite has started failing sporadically

[jira] [Commented] (SPARK-32619) converting dataframe to dataset for the json schema

[ https://issues.apache.org/jira/browse/SPARK-32619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178147#comment-17178147 ] Rohit Mishra commented on SPARK-32619: -- [~Manjay7869], Can you please update- Environment detail &

[jira] [Comment Edited] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177246#comment-17177246 ] Rohit Mishra edited comment on SPARK-32609 at 8/13/20, 6:28 PM:

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32609: - Target Version/s: (was: 2.4.5, 2.4.7) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177246#comment-17177246 ] Rohit Mishra commented on SPARK-32609: -- [~mingjial], Please refrain from adding Priority as

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32609: - Priority: Major (was: Critical) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Updated] (SPARK-32601) Issue in converting an RDD of Arrow RecordBatches in v3.0.0

[ https://issues.apache.org/jira/browse/SPARK-32601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32601: - Fix Version/s: (was: 2.3.4) > Issue in converting an RDD of Arrow RecordBatches in v3.0.0 >

[jira] [Commented] (SPARK-32601) Issue in converting an RDD of Arrow RecordBatches in v3.0.0

[ https://issues.apache.org/jira/browse/SPARK-32601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176771#comment-17176771 ] Rohit Mishra commented on SPARK-32601: -- [~tahmad], Thanks for raising the bug. Please refrain from

[jira] [Commented] (SPARK-32585) Support scala enumeration in ScalaReflection

[ https://issues.apache.org/jira/browse/SPARK-32585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175272#comment-17175272 ] Rohit Mishra commented on SPARK-32585: -- [~ulysses], Can you please add a description? > Support

[jira] [Commented] (SPARK-32583) PySpark Structured Streaming Testing Support

[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175205#comment-17175205 ] Rohit Mishra commented on SPARK-32583: -- cc [~hyukjin.kwon].  > PySpark Structured Streaming

[jira] [Resolved] (SPARK-32583) PySpark Structured Streaming Testing Support

[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra resolved SPARK-32583. -- Resolution: Not A Problem > PySpark Structured Streaming Testing Support >

[jira] [Commented] (SPARK-32583) PySpark Structured Streaming Testing Support

[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175200#comment-17175200 ] Rohit Mishra commented on SPARK-32583: -- [~FelixKJose], Questions are usually answered using either

[jira] [Commented] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173708#comment-17173708 ] Rohit Mishra commented on SPARK-32570: -- Thanks, [~maropu]. If I may add one more point- Please

[jira] [Updated] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32570: - Priority: Major (was: Critical) > Thriftserver LDAP failed > > >

[jira] [Commented] (SPARK-32562) Pyspark drop duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-32562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173199#comment-17173199 ] Rohit Mishra commented on SPARK-32562: -- [~amotex], Please avoid adding fix version. You can always

[jira] [Updated] (SPARK-32562) Pyspark drop duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-32562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32562: - Fix Version/s: (was: 3.0.0) > Pyspark drop duplicate columns >

[jira] [Commented] (SPARK-32180) Getting Started - Installation

[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172938#comment-17172938 ] Rohit Mishra commented on SPARK-32180: -- [~hyukjin.kwon], I will start contributing to this module,

[jira] [Commented] (SPARK-32558) ORC target files that Spark_3.0 produces does not work with Hive_2.1.1 (work-around of using spark.sql.orc.impl=hive is also not working)

[ https://issues.apache.org/jira/browse/SPARK-32558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172635#comment-17172635 ] Rohit Mishra commented on SPARK-32558: -- [~ramks], Thanks for raising this bug but Please refrain

[jira] [Updated] (SPARK-32558) ORC target files that Spark_3.0 produces does not work with Hive_2.1.1 (work-around of using spark.sql.orc.impl=hive is also not working)

[ https://issues.apache.org/jira/browse/SPARK-32558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32558: - Priority: Major (was: Blocker) > ORC target files that Spark_3.0 produces does not work with

[jira] [Commented] (SPARK-32542) add a batch for optimizing logicalPlan

[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171492#comment-17171492 ] Rohit Mishra commented on SPARK-32542: -- [~wangshikai], Can you please add a description to this

[jira] [Updated] (SPARK-32527) How to disable port 8080 in Spark?

[ https://issues.apache.org/jira/browse/SPARK-32527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32527: - Priority: Minor (was: Critical) > How to disable port 8080 in Spark? >

[jira] [Resolved] (SPARK-32527) How to disable port 8080 in Spark?

[ https://issues.apache.org/jira/browse/SPARK-32527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra resolved SPARK-32527. -- Resolution: Not A Problem > How to disable port 8080 in Spark? >

[jira] [Commented] (SPARK-32527) How to disable port 8080 in Spark?

[ https://issues.apache.org/jira/browse/SPARK-32527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171069#comment-17171069 ] Rohit Mishra commented on SPARK-32527: -- [~shinudin], * Please use stack overflow for any question.

[jira] [Comment Edited] (SPARK-32527) How to disable port 8080 in Spark?

[ https://issues.apache.org/jira/browse/SPARK-32527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171069#comment-17171069 ] Rohit Mishra edited comment on SPARK-32527 at 8/4/20, 7:16 PM: ---

[jira] [Commented] (SPARK-32528) The analyze method should make sure the plan is analyzed

[ https://issues.apache.org/jira/browse/SPARK-32528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171062#comment-17171062 ] Rohit Mishra commented on SPARK-32528: -- [~cloud_fan], Can you please populate the description

[jira] [Commented] (SPARK-32511) Add dropFields method to Column class

[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169087#comment-17169087 ] Rohit Mishra commented on SPARK-32511: -- [~fqaiser94], Can you please add a description? > Add

[jira] [Commented] (SPARK-32509) Unused DPP Filter causes issue in canonicalization and prevents reuse exchange

[ https://issues.apache.org/jira/browse/SPARK-32509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168856#comment-17168856 ] Rohit Mishra commented on SPARK-32509: -- [~prakharjain09], Meanwhile you are working on the pull

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168115#comment-17168115 ] Rohit Mishra commented on SPARK-32500: -- [~abhishekd0907], It will be helpful if you can add

[jira] [Commented] (SPARK-32490) Upgrade netty-all to 4.1.51.Final

[ https://issues.apache.org/jira/browse/SPARK-32490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167691#comment-17167691 ] Rohit Mishra commented on SPARK-32490: -- [~LuciferYang], Can you please add a description? >

[jira] [Commented] (SPARK-32465) How do I get the SPARK shuffle monitoring indicator?

[ https://issues.apache.org/jira/browse/SPARK-32465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167471#comment-17167471 ] Rohit Mishra commented on SPARK-32465: -- [~MOBIN], Can you please ask questions using the mailing

[jira] [Commented] (SPARK-32403) SCRIP TRANSFORM Extract common method from process row to avoid repeated judgement

[ https://issues.apache.org/jira/browse/SPARK-32403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167461#comment-17167461 ] Rohit Mishra commented on SPARK-32403: -- [~angerszhuuu], Can you please add a description? > SCRIP

[jira] [Commented] (SPARK-32469) ApplyColumnarRulesAndInsertTransitions should be idempotent

[ https://issues.apache.org/jira/browse/SPARK-32469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167455#comment-17167455 ] Rohit Mishra commented on SPARK-32469: -- [~cloud_fan], Can you please add a description while you

[jira] [Commented] (SPARK-32444) Infer filters from DPP

[ https://issues.apache.org/jira/browse/SPARK-32444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167451#comment-17167451 ] Rohit Mishra commented on SPARK-32444: -- [~yumwang], Can you please add a description? > Infer

[jira] [Commented] (SPARK-32445) Make NullType.sql as VOID to support hive

[ https://issues.apache.org/jira/browse/SPARK-32445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167447#comment-17167447 ] Rohit Mishra commented on SPARK-32445: -- [~ulysses], Can you please add a description?  > Make

[jira] [Commented] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167440#comment-17167440 ] Rohit Mishra commented on SPARK-32485: -- [~mundaym], can you please add environment detail and steps