[jira] [Commented] (SPARK-34145) Combine scalar subqueries

[ https://issues.apache.org/jira/browse/SPARK-34145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696545#comment-17696545 ] Sujith Chacko commented on SPARK-34145: --- [~yumwang]  Is anyone working on this feature? > Combine

[jira] [Commented] (SPARK-27714) Support Join Reorder based on Genetic Algorithm when the # of joined tables > 12

[ https://issues.apache.org/jira/browse/SPARK-27714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696543#comment-17696543 ] Sujith Chacko commented on SPARK-27714: --- [~xinxianyin]  any work is in progress for this issue? I

[jira] [Comment Edited] (SPARK-34117) Disable LeftSemi/LeftAnti push down over Aggregate

[ https://issues.apache.org/jira/browse/SPARK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696541#comment-17696541 ] Sujith Chacko edited comment on SPARK-34117 at 3/5/23 8:57 AM: --- [~yumwang]

[jira] [Commented] (SPARK-34117) Disable LeftSemi/LeftAnti push down over Aggregate

[ https://issues.apache.org/jira/browse/SPARK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696541#comment-17696541 ] Sujith Chacko commented on SPARK-34117: --- Is any work going for this improvement, do we have any PR

[jira] [Commented] (SPARK-29250) Upgrade to Hadoop 3.3.1

[ https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553439#comment-17553439 ] Sujith Chacko commented on SPARK-29250: --- If we upgrade spark version from spark 3.0.x to 3.2.x, Is

[jira] [Commented] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031647#comment-17031647 ] Sujith Chacko commented on SPARK-24615: --- Great!!Thanks for the update. > SPIP: Accelerator-aware

[jira] [Commented] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031378#comment-17031378 ] Sujith Chacko commented on SPARK-24615: --- Will this feature be a part of Spark 3.0?  Any update on

[jira] [Updated] (SPARK-28930) Spark DESC FORMATTED TABLENAME information display issues

[ https://issues.apache.org/jira/browse/SPARK-28930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-28930: -- Description: Spark DESC FORMATTED TABLENAME information display issues.Showing incorrect 

[jira] [Commented] (SPARK-28930) Spark DESC FORMATTED TABLENAME information display issues

[ https://issues.apache.org/jira/browse/SPARK-28930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919563#comment-16919563 ] Sujith Chacko commented on SPARK-28930: --- @ [~jobitmathew] As i remember Issue 3 is already handled

[jira] [Commented] (SPARK-28707) Spark SQL select query result size issue

[ https://issues.apache.org/jira/browse/SPARK-28707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906873#comment-16906873 ] Sujith Chacko commented on SPARK-28707: --- [~angerszhuuu] [~dongjoon] thanks a lot for the

[jira] [Commented] (SPARK-28707) Spark SQL select query result size issue

[ https://issues.apache.org/jira/browse/SPARK-28707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906187#comment-16906187 ] Sujith Chacko commented on SPARK-28707: --- [~angerszhuuu]  In case of limit as you told data will be

[jira] [Commented] (SPARK-28707) Spark SQL select query result size issue

[ https://issues.apache.org/jira/browse/SPARK-28707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906080#comment-16906080 ] Sujith Chacko commented on SPARK-28707: --- In both case the result size shall be 1000,  In case of

[jira] [Commented] (SPARK-28707) Spark SQL select query result size issue

[ https://issues.apache.org/jira/browse/SPARK-28707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905868#comment-16905868 ] Sujith Chacko commented on SPARK-28707: --- cc [~dongjoon] > Spark SQL select query result size

[jira] [Commented] (SPARK-28707) Spark SQL select query result size issue

[ https://issues.apache.org/jira/browse/SPARK-28707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905867#comment-16905867 ] Sujith Chacko commented on SPARK-28707: --- [~jobitmathew] Thanks for reporting the issue, will check

[jira] [Commented] (SPARK-28084) LOAD DATA command resolving the partition column name considering case senstive manner

[ https://issues.apache.org/jira/browse/SPARK-28084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865758#comment-16865758 ] Sujith Chacko commented on SPARK-28084: --- insert command the partition column will be resolved

[jira] [Updated] (SPARK-28084) LOAD DATA command resolving the partition column name considering case senstive manner

[ https://issues.apache.org/jira/browse/SPARK-28084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-28084: -- Attachment: parition_casesensitive.PNG > LOAD DATA command resolving the partition column

[jira] [Created] (SPARK-28084) LOAD DATA command resolving the partition column name considering case senstive manner

Sujith Chacko created SPARK-28084: - Summary: LOAD DATA command resolving the partition column name considering case senstive manner Key: SPARK-28084 URL: https://issues.apache.org/jira/browse/SPARK-28084

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832671#comment-16832671 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 6:28 PM: --- CREATE

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832671#comment-16832671 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 6:27 PM: --- CREATE

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832718#comment-16832718 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 6:20 PM: --- HIVE does

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832718#comment-16832718 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 6:18 PM: --- HIVE does

[jira] [Commented] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832718#comment-16832718 ] Sujith Chacko commented on SPARK-27617: --- HIVE does not "externalise" table  if LOCATION specified,

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832671#comment-16832671 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 5:32 PM: --- CREATE

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832671#comment-16832671 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 5:15 PM: --- CREATE

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832671#comment-16832671 ] Sujith Chacko edited comment on SPARK-27617 at 5/3/19 5:06 PM: --- CREATE

[jira] [Commented] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832671#comment-16832671 ] Sujith Chacko commented on SPARK-27617: --- CREATE TABLE IF NOT EXISTS ext2 (name STRING) LOCATION

[jira] [Updated] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27617: -- Description: In Spark whenever user specifies location uri in create table without external

[jira] [Updated] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27617: -- Affects Version/s: 2.0.0 > Not able to specify LOCATION for internal table >

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831183#comment-16831183 ] Sujith Chacko edited comment on SPARK-27617 at 5/1/19 6:49 PM: --- cc

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831183#comment-16831183 ] Sujith Chacko edited comment on SPARK-27617 at 5/1/19 6:48 PM: --- cc

[jira] [Comment Edited] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831183#comment-16831183 ] Sujith Chacko edited comment on SPARK-27617 at 5/1/19 6:48 PM: --- cc

[jira] [Commented] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831183#comment-16831183 ] Sujith Chacko commented on SPARK-27617: --- cc [~dongjoon]    cc [~cloud_fan] I observed above

[jira] [Updated] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27617: -- Affects Version/s: 3.0.0 > Not able to specify LOCATION for internal table >

[jira] [Updated] (SPARK-27617) Not able to specify LOCATION for internal table

[ https://issues.apache.org/jira/browse/SPARK-27617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27617: -- Description: In spark whenever user specifies location uri in create table without external

[jira] [Created] (SPARK-27617) Not able to specify LOCATION for internal table

Sujith Chacko created SPARK-27617: - Summary: Not able to specify LOCATION for internal table Key: SPARK-27617 URL: https://issues.apache.org/jira/browse/SPARK-27617 Project: Spark Issue

[jira] [Updated] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27403: -- Affects Version/s: 2.3.0 2.3.1 2.3.2

[jira] [Comment Edited] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812095#comment-16812095 ] Sujith Chacko edited comment on SPARK-27403 at 4/8/19 4:28 AM: --- This has

[jira] [Commented] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812095#comment-16812095 ] Sujith Chacko commented on SPARK-27403: --- This has impact in the previous version also, will upate

[jira] [Comment Edited] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811940#comment-16811940 ] Sujith Chacko edited comment on SPARK-27403 at 4/7/19 6:33 PM: --- if user

[jira] [Updated] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27403: -- Description: system shall update the table stats automatically if user set 

[jira] [Updated] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27403: -- Description: system shall update the table stats automatically if user set 

[jira] [Commented] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811940#comment-16811940 ] Sujith Chacko commented on SPARK-27403: --- I will analyze further and raise a PR for handling the

[jira] [Updated] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27403: -- Description: system shall update the table stats automatiaclly if user set 

[jira] [Updated] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27403: -- Environment: (was: system shall able to update the table stats automatically if user sets 

[jira] [Updated] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

[ https://issues.apache.org/jira/browse/SPARK-27403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27403: -- Description: scala> spark.sql("insert into table1 select 'a',29") res2:

[jira] [Created] (SPARK-27403) Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue

Sujith Chacko created SPARK-27403: - Summary: Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue Key: SPARK-27403 URL:

[jira] [Comment Edited] (SPARK-2647) DAGScheduler plugs others when processing one JobSubmitted event

[ https://issues.apache.org/jira/browse/SPARK-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806554#comment-16806554 ] Sujith Chacko edited comment on SPARK-2647 at 4/1/19 9:32 AM: -- [~joshrosen]

[jira] [Comment Edited] (SPARK-2647) DAGScheduler plugs others when processing one JobSubmitted event

[ https://issues.apache.org/jira/browse/SPARK-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806554#comment-16806554 ] Sujith Chacko edited comment on SPARK-2647 at 4/1/19 9:30 AM: -- [~joshrosen]

[jira] [Commented] (SPARK-2647) DAGScheduler plugs others when processing one JobSubmitted event

[ https://issues.apache.org/jira/browse/SPARK-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806554#comment-16806554 ] Sujith Chacko commented on SPARK-2647: -- [~joshrosen] - May i know the Jira is where we already

[jira] [Comment Edited] (SPARK-2647) DAGScheduler plugs others when processing one JobSubmitted event

[ https://issues.apache.org/jira/browse/SPARK-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806554#comment-16806554 ] Sujith Chacko edited comment on SPARK-2647 at 4/1/19 9:30 AM: -- [~joshrosen]

[jira] [Updated] (SPARK-27261) Spark submit passing multiple configurations not documented clearly

[ https://issues.apache.org/jira/browse/SPARK-27261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-27261: -- Priority: Trivial (was: Major) > Spark submit passing multiple configurations not documented

[jira] [Commented] (SPARK-27261) Spark submit passing multiple configurations not documented clearly

[ https://issues.apache.org/jira/browse/SPARK-27261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799763#comment-16799763 ] Sujith Chacko commented on SPARK-27261: --- cc [~cloud_fan] [~dongjoon] > Spark submit passing

[jira] [Commented] (SPARK-27261) Spark submit passing multiple configurations not documented clearly

[ https://issues.apache.org/jira/browse/SPARK-27261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799762#comment-16799762 ] Sujith Chacko commented on SPARK-27261: --- Will raise a PR to document the above scenario regarding

[jira] [Created] (SPARK-27261) Spark submit passing multiple configurations not documented clearly

Sujith Chacko created SPARK-27261: - Summary: Spark submit passing multiple configurations not documented clearly Key: SPARK-27261 URL: https://issues.apache.org/jira/browse/SPARK-27261 Project: Spark

[jira] [Commented] (SPARK-26176) Verify column name when creating table via `STORED AS`

[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791060#comment-16791060 ] Sujith Chacko commented on SPARK-26176: --- Issue is still happening with spark 2.4 latest version. I

[jira] [Comment Edited] (SPARK-23450) jars option in spark submit is documented in misleading way

[ https://issues.apache.org/jira/browse/SPARK-23450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790558#comment-16790558 ] Sujith Chacko edited comment on SPARK-23450 at 3/12/19 1:43 PM: As per

[jira] [Commented] (SPARK-23450) jars option in spark submit is documented in misleading way

[ https://issues.apache.org/jira/browse/SPARK-23450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790558#comment-16790558 ] Sujith Chacko commented on SPARK-23450: --- As pr my understanding the jar will be distributed to the

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788778#comment-16788778 ] Sujith Chacko commented on SPARK-27060: --- cc [~maropu] > DDL Commands are accepting Keywords like

[jira] [Comment Edited] (SPARK-27036) Even Broadcast thread is timed out, BroadCast Job is not aborted.

[ https://issues.apache.org/jira/browse/SPARK-27036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782840#comment-16782840 ] Sujith Chacko edited comment on SPARK-27036 at 3/5/19 3:49 PM: --- It seems

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784589#comment-16784589 ] Sujith Chacko commented on SPARK-27060: --- Yes, Quite surprising. In hive they are validating all

[jira] [Comment Edited] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784503#comment-16784503 ] Sujith Chacko edited comment on SPARK-27060 at 3/5/19 3:11 PM: --- This looks

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784503#comment-16784503 ] Sujith Chacko commented on SPARK-27060: --- This looks like a compatibility issue with other engines.