[jira] [Updated] (SPARK-40350) Use spark config to configure the parameters of volcano podgroup

2022-09-05 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-40350:
-
Summary: Use spark config to configure the parameters of volcano podgroup  
(was: Improve the configuration of volcano scheduler)

> Use spark config to configure the parameters of volcano podgroup
> 
>
> Key: SPARK-40350
> URL: https://issues.apache.org/jira/browse/SPARK-40350
> Project: Spark
>  Issue Type: Improvement
>  Components: Kubernetes
>Affects Versions: 3.3.0
>Reporter: Sun BiaoBiao
>Priority: Major
>
> Now we use volcano as our scheduler， we need  specify  the following 
> configuration options:
>  
>  
> {code:java}
> spark.kubernetes.scheduler.name=volcano
> spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
> spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
> spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
>  {code}
>  
>  
> we should use configMap to mount the /path/to/podgroup-template.yaml file
>  
> If we use spark config to specify the parameters of the podgroup, it will be 
> much more convenient,we don't need configmap to mount static files
>  
> In our scenario, we need to dynamically specify the volcano queue, but it is 
> not convenient to create a static podgroup configuration file to mount 
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-40350) Improve the configuration of volcano scheduler

2022-09-05 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-40350:
-
Description: 
Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient,we don't need configmap to mount static files

 

In our scenario, we need to dynamically specify the volcano queue, but it is 
not convenient to create a static podgroup configuration file to mount 

 

 

 

  was:
Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient,we don't need configmap to mount static files

 

 

 


> Improve the configuration of volcano scheduler
> --
>
> Key: SPARK-40350
> URL: https://issues.apache.org/jira/browse/SPARK-40350
> Project: Spark
>  Issue Type: Improvement
>  Components: Kubernetes
>Affects Versions: 3.3.0
>Reporter: Sun BiaoBiao
>Priority: Major
>
> Now we use volcano as our scheduler， we need  specify  the following 
> configuration options:
>  
>  
> {code:java}
> spark.kubernetes.scheduler.name=volcano
> spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
> spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
> spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
>  {code}
>  
>  
> we should use configMap to mount the /path/to/podgroup-template.yaml file
>  
> If we use spark config to specify the parameters of the podgroup, it will be 
> much more convenient,we don't need configmap to mount static files
>  
> In our scenario, we need to dynamically specify the volcano queue, but it is 
> not convenient to create a static podgroup configuration file to mount 
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-40350) Improve the configuration of volcano scheduler

2022-09-05 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-40350:
-
Description: 
Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient, so that we don't need configmap to mount static files

 

 

 

  was:
Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient, so that we can mount static files without using configmap

 

 

 


> Improve the configuration of volcano scheduler
> --
>
> Key: SPARK-40350
> URL: https://issues.apache.org/jira/browse/SPARK-40350
> Project: Spark
>  Issue Type: Improvement
>  Components: Kubernetes
>Affects Versions: 3.3.0
>Reporter: Sun BiaoBiao
>Priority: Major
>
> Now we use volcano as our scheduler， we need  specify  the following 
> configuration options:
>  
>  
> {code:java}
> spark.kubernetes.scheduler.name=volcano
> spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
> spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
> spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
>  {code}
>  
>  
> we should use configMap to mount the /path/to/podgroup-template.yaml file
>  
> If we use spark config to specify the parameters of the podgroup, it will be 
> much more convenient, so that we don't need configmap to mount static files
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-40350) Improve the configuration of volcano scheduler

2022-09-05 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-40350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-40350:
-
Description: 
Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient,we don't need configmap to mount static files

 

 

 

  was:
Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient, so that we don't need configmap to mount static files

 

 

 


> Improve the configuration of volcano scheduler
> --
>
> Key: SPARK-40350
> URL: https://issues.apache.org/jira/browse/SPARK-40350
> Project: Spark
>  Issue Type: Improvement
>  Components: Kubernetes
>Affects Versions: 3.3.0
>Reporter: Sun BiaoBiao
>Priority: Major
>
> Now we use volcano as our scheduler， we need  specify  the following 
> configuration options:
>  
>  
> {code:java}
> spark.kubernetes.scheduler.name=volcano
> spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
> spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
> spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
>  {code}
>  
>  
> we should use configMap to mount the /path/to/podgroup-template.yaml file
>  
> If we use spark config to specify the parameters of the podgroup, it will be 
> much more convenient,we don't need configmap to mount static files
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-40350) Improve the configuration of volcano scheduler

2022-09-05 Thread Sun BiaoBiao (Jira) 
Sun BiaoBiao created SPARK-40350:


 Summary: Improve the configuration of volcano scheduler
 Key: SPARK-40350
 URL: https://issues.apache.org/jira/browse/SPARK-40350
 Project: Spark
  Issue Type: Improvement
  Components: Kubernetes
Affects Versions: 3.3.0
Reporter: Sun BiaoBiao


Now we use volcano as our scheduler， we need  specify  the following 
configuration options:

 

 
{code:java}
spark.kubernetes.scheduler.name=volcano
spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml

spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep

spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
 {code}
 

 

we should use configMap to mount the /path/to/podgroup-template.yaml file

 

If we use spark config to specify the parameters of the podgroup, it will be 
much more convenient, so that we can mount static files without using configmap

 

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 8:55 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y|n|n|y| | | | | | | | | | | | | | |
|char|y|n|y|y|y|y|y|y|y| | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n|n|n|y| | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|n|n|n|n|y| | | | | | | | | | | | | | |
|Concat|y|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y|y|y|y|n|n|n| | | | | | | | | | | | | | |
|right|y|n|y|y|y|y|n|n|n| | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y|n|n| | | | | | | | | | | | | | | |
|char|y|n|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n|n|n| | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y|n|n|n| | | | | | | | | | | | | | | |
|Concat|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 8:52 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y|n|n| | | | | | | | | | | | | | | |
|char|y|n|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n|n|n| | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y|n|n|n| | | | | | | | | | | | | | | |
|Concat|y|y|y|y|y|y|y|y| | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y|y|y|y|n|n| | | | | | | | | | | | | | | |
|right|y|n|y|y|y|y|n|n| | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n|n| | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y|y| | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y|y| | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y|n|n| | | | | | | | | | | | | | | | |
|Concat|y|y|y|y|y|y|y| | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 8:49 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n|n| | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y|y| | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y|y| | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y|n|n| | | | | | | | | | | | | | | | |
|Concat|y|y|y|y|y|y|y| | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y|y|y|y|n| | | | | | | | | | | | | | | | |
|right|y|n|y|y|y|y|n| | | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n| | | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y| | | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y| | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y|n| | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 8:45 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y|n| | | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y|y| | | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y|y| | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y|n| | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|right|y|n|y|y|y|y| | | | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 6:41 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|right|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 6:34 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|char|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|chr|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|char_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|character_length|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y|y|n| | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y|y|y| | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|right|y|n|y|y|y| | | | | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y| | | | | | | | | | | | | | | | | | | | |
|char|y|n|y| | | | | | | | | | | | | | | | | | | | |
|chr|y|n|y| | | | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y| | | | | | | | | | | | | | | | | | | | |
|char_length|y|y|y| | | | | | | | | | | | | | | | | | | | |
|character_length|y|y|y| | | | | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y| | | | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Comment Edited] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-08-06 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao edited comment on SPARK-39929 at 8/6/22 6:30 AM:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| y|n|y| | | | | | | | | | | | | | | | | | | | |
|char|y|n|y| | | | | | | | | | | | | | | | | | | | |
|chr|y|n|y| | | | | | | | | | | | | | | | | | | | |
|contains|n| |n| | | | | | | | | | | | | | | | | | | | |
|startswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|endswith|n| |n| | | | | | | | | | | | | | | | | | | | |
|base64|n| |n| | | | | | | | | | | | | | | | | | | | |
|bit_length|y|y|y| | | | | | | | | | | | | | | | | | | | |
|char_length|y|y|y| | | | | | | | | | | | | | | | | | | | |
|character_length|y|y|y| | | | | | | | | | | | | | | | | | | | |
|concat_ws|y|y|y| | | | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad|y|y|y| | | | | | | | | | | | | | | | | | | | |
|ltrim|y|y|y| | | | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad|y|y|y| | | | | | | | | | | | | | | | | | | | |
|rtrim|y|y|y| | | | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left|y|n|y| | | | | | | | | | | | | | | | | | | | |
|right|y|n|y| | | | | | | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |


was (Author: biaobiao.sun):
||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| | | | | | | | | | | | | | | | | | | | | | | |
|char| | | | | | | | | | | | | | | | | | | | | | | |
|chr| | | | | | | | | | | | | | | | | | | | | | | |
|contains| | | | | | | | | | | | | | | | | | | | | | | |
|startswith| | | | | | | | | | | | | | | | | | | | | | | |
|endswith| | | | | | | | | | | | | | | | | | | | | | | |
|base64| | | | | | | | | | | | | | | | | | | | | | | |
|bit_length| | | | | | | | | | | | | | | | | | | | | | | |
|char_length| | | | | | | | | | | | | | | | | | | | | | | |
|character_length| | | | | | | | | | | | | | | | | | | | | | | |
|concat_ws| | | | | | | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |

[jira] [Commented] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-07-31 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573529#comment-17573529
 ] 

Sun BiaoBiao commented on SPARK-39929:
--

||Function 
name||PostgreSQL||ClickHouse||H2||MySQL||Oracle||Redshift||Presto||Teradata||Snowflake||DB2||Vertica||Exasol||SqlServer||Yellowbrick||Impala||Mariadb||Druid||Pig||Singlestore||ElasticSearch||SQLite||Influxdata||Sybase||
|ascii| | | | | | | | | | | | | | | | | | | | | | | |
|char| | | | | | | | | | | | | | | | | | | | | | | |
|chr| | | | | | | | | | | | | | | | | | | | | | | |
|contains| | | | | | | | | | | | | | | | | | | | | | | |
|startswith| | | | | | | | | | | | | | | | | | | | | | | |
|endswith| | | | | | | | | | | | | | | | | | | | | | | |
|base64| | | | | | | | | | | | | | | | | | | | | | | |
|bit_length| | | | | | | | | | | | | | | | | | | | | | | |
|char_length| | | | | | | | | | | | | | | | | | | | | | | |
|character_length| | | | | | | | | | | | | | | | | | | | | | | |
|concat_ws| | | | | | | | | | | | | | | | | | | | | | | |
|decode| | | | | | | | | | | | | | | | | | | | | | | |
|elt| | | | | | | | | | | | | | | | | | | | | | | |
|encode| | | | | | | | | | | | | | | | | | | | | | | |
|find_in_set| | | | | | | | | | | | | | | | | | | | | | | |
|format_number| | | | | | | | | | | | | | | | | | | | | | | |
|format_string| | | | | | | | | | | | | | | | | | | | | | | |
|to_number| | | | | | | | | | | | | | | | | | | | | | | |
|try_to_number| | | | | | | | | | | | | | | | | | | | | | | |
|to_char| | | | | | | | | | | | | | | | | | | | | | | |
|get_json_object| | | | | | | | | | | | | | | | | | | | | | | |
|initcap| | | | | | | | | | | | | | | | | | | | | | | |
|instr| | | | | | | | | | | | | | | | | | | | | | | |
|lcase| | | | | | | | | | | | | | | | | | | | | | | |
|length| | | | | | | | | | | | | | | | | | | | | | | |
|levenshtein| | | | | | | | | | | | | | | | | | | | | | | |
|like| | | | | | | | | | | | | | | | | | | | | | | |
|ilike| | | | | | | | | | | | | | | | | | | | | | | |
|octet_length| | | | | | | | | | | | | | | | | | | | | | | |
|locate| | | | | | | | | | | | | | | | | | | | | | | |
|lpad| | | | | | | | | | | | | | | | | | | | | | | |
|ltrim| | | | | | | | | | | | | | | | | | | | | | | |
|json_tuple| | | | | | | | | | | | | | | | | | | | | | | |
|position| | | | | | | | | | | | | | | | | | | | | | | |
|printf| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_extract_all| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_replace| | | | | | | | | | | | | | | | | | | | | | | |
|repeat| | | | | | | | | | | | | | | | | | | | | | | |
|replace| | | | | | | | | | | | | | | | | | | | | | | |
|overlay| | | | | | | | | | | | | | | | | | | | | | | |
|rlike| | | | | | | | | | | | | | | | | | | | | | | |
|regexp_like| | | | | | | | | | | | | | | | | | | | | | | |
|regexp| | | | | | | | | | | | | | | | | | | | | | | |
|rpad| | | | | | | | | | | | | | | | | | | | | | | |
|rtrim| | | | | | | | | | | | | | | | | | | | | | | |
|sentences| | | | | | | | | | | | | | | | | | | | | | | |
|soundex| | | | | | | | | | | | | | | | | | | | | | | |
|space| | | | | | | | | | | | | | | | | | | | | | | |
|split| | | | | | | | | | | | | | | | | | | | | | | |
|split_part| | | | | | | | | | | | | | | | | | | | | | | |
|substr| | | | | | | | | | | | | | | | | | | | | | | |
|substring| | | | | | | | | | | | | | | | | | | | | | | |
|left| | | | | | | | | | | | | | | | | | | | | | | |
|right| | | | | | | | | | | | | | | | | | | | | | | |
|substring_index| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|translate| | | | | | | | | | | | | | | | | | | | | | | |
|btrim| | | | | | | | | | | | | | | | | | | | | | | |

> DS V2 supports push down string  functions(non ANSI)
> 
>
> Key: SPARK-39929
> URL: https://issues.apache.org/jira/browse/SPARK-39929
> Project: Spark
>  Issue Type: Sub-task
>  Components: SQL
>Affects Versions: 3.4.0
>Reporter: Sun BiaoBiao
>Priority: Major
>
> Currently, Spark have a lot string functions which is not defined in ANSI 
> standard.
> But these functions is commonly used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-39929) DS V2 supports push down string functions(non ANSI)

2022-07-31 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-39929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-39929:
-
Summary: DS V2 supports push down string  functions(non ANSI)  (was: DS V2 
supports push down  functions(non ANSI))

> DS V2 supports push down string  functions(non ANSI)
> 
>
> Key: SPARK-39929
> URL: https://issues.apache.org/jira/browse/SPARK-39929
> Project: Spark
>  Issue Type: Sub-task
>  Components: SQL
>Affects Versions: 3.4.0
>Reporter: Sun BiaoBiao
>Priority: Major
>
> Currently, Spark have a lot string functions which is not defined in ANSI 
> standard.
> But these functions is commonly used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-39929) DS V2 supports push down functions(non ANSI)

2022-07-31 Thread Sun BiaoBiao (Jira) 
Sun BiaoBiao created SPARK-39929:


 Summary: DS V2 supports push down  functions(non ANSI)
 Key: SPARK-39929
 URL: https://issues.apache.org/jira/browse/SPARK-39929
 Project: Spark
  Issue Type: Sub-task
  Components: SQL
Affects Versions: 3.4.0
Reporter: Sun BiaoBiao


Currently, Spark have a lot string functions which is not defined in ANSI 
standard.
But these functions is commonly used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-39908) Translate common used aggregate functions for pushdown (non ANSI)

2022-07-29 Thread Sun BiaoBiao (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-39908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573194#comment-17573194
 ] 

Sun BiaoBiao commented on SPARK-39908:
--

all   aggregate functions doesn't follow ANSI  list:

 

 
||name||comment||
|count_if|wil be repaced by count  at runtime, skip  |
|first_value| |
|any_value| |
|kurtosis| |
|last| |
|last_value| |
|max_by| |
|min_by| |
|percentile| |
|median| |
|collect_list| |
|array_agg| |
|collect_set| |
|bool_and|wil be repaced by   Min at runtime, skip|
|any|wil be repaced by   Max at runtime , skip|
|some|wil be repaced by Max  at runtime, skip|
|bool_or|wil be repaced by Max  at runtime, skip|
|every|wil be repaced by Min  at runtime, skip|
| |Col A2|

 

 

> Translate common used aggregate functions for pushdown (non ANSI)
> -
>
> Key: SPARK-39908
> URL: https://issues.apache.org/jira/browse/SPARK-39908
> Project: Spark
>  Issue Type: Sub-task
>  Components: SQL
>Affects Versions: 3.4.0
>Reporter: jiaan.geng
>Priority: Major
>
> Currently, Spark have a lot of common used aggregate functions doesn't follow 
> ANSI standard.
> Because these aggregate functions are common used, we should push down them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-36005) in spark3.1.2 version The canCast method of type of char/varchar needs to be consistent with StringType

2021-07-02 Thread Sun BiaoBiao (Jira) 
Sun BiaoBiao created SPARK-36005:


 Summary: in spark3.1.2 version The canCast method of type of 
char/varchar needs to be consistent with StringType
 Key: SPARK-36005
 URL: https://issues.apache.org/jira/browse/SPARK-36005
 Project: Spark
  Issue Type: Improvement
  Components: SQL
Affects Versions: 3.1.2
Reporter: Sun BiaoBiao
 Fix For: 3.1.3


In https://github.com/apache/spark/pull/32109 this pr, we introduced the 
char/varchar type,

As described in this issue:

To be safe, this PR doesn't add char/varchar type to the query 
engine(expression input check, internal row framework, codegen framework, 
etc.). We will replace char/varchar type by string type with metadata 
(Attribute.metadata or StructField.metadata) that includes the original type 
string before it goes into the query engine. That said, the existing code will 
not see char/varchar type but only string type.


so The canCast method of type of char/varchar needs to be consistent with 
StringType



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-35964) spark 3.1.1 version the method schema of class DataframeReader will lose the original type information of char/varchar

2021-07-01 Thread Sun BiaoBiao (Jira) 
Sun BiaoBiao created SPARK-35964:


 Summary: spark 3.1.1  version the  method  schema  of class  
DataframeReader will lose the original type information of char/varchar
 Key: SPARK-35964
 URL: https://issues.apache.org/jira/browse/SPARK-35964
 Project: Spark
  Issue Type: Improvement
  Components: SQL
Affects Versions: 3.1.2, 3.1.1
Reporter: Sun BiaoBiao


in  spark 3.1.1 version,  the method schema of class DataframeReader will lose 
the original type information of char/varchar

 

!image-2021-07-01-14-54-49-922.png!

 

 

!image-2021-07-01-14-55-03-117.png!

 

 

It is expected that it can be placed in the metadata of the field, and the 
original type information can be obtained in subsequent processing

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-35007) Spark 2.4.x version does not support numeric

2021-04-09 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-35007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-35007:
-
Summary: Spark 2.4.x version does not support numeric  (was: Spark 2.4 
version does not support numeric)

> Spark 2.4.x version does not support numeric
> 
>
> Key: SPARK-35007
> URL: https://issues.apache.org/jira/browse/SPARK-35007
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 2.4.7
>Reporter: Sun BiaoBiao
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> as said in this pr:  [https://github.com/apache/spark/pull/31891] , 
>  
> So seems the key difference between DEICMAL and NUMERIC are:
> {quote}implementation-defined decimal precision equal to or greater than the 
> value of the specified precision.
> {quote}
> Decimal can have _at least_ specified precision (which means can have more) 
> whereas numeric should have exactly specified precision.
> Spark's decimal satisfy both so I think NUMERIC as a synonym of DECIMAL makes 
> sense.
>  
> my  code need  needs to use NUMERIC type in spark2.4 version 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-35007) Spark 2.4.x version does not support numeric

2021-04-09 Thread Sun BiaoBiao (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-35007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sun BiaoBiao updated SPARK-35007:
-
Description: 
as said in this pr:  [https://github.com/apache/spark/pull/31891] , 

 

So seems the key difference between DEICMAL and NUMERIC are:
{quote}implementation-defined decimal precision equal to or greater than the 
value of the specified precision.
{quote}
Decimal can have _at least_ specified precision (which means can have more) 
whereas numeric should have exactly specified precision.

Spark's decimal satisfy both so I think NUMERIC as a synonym of DECIMAL makes 
sense.

 

my  code  needs to use NUMERIC type in spark2.4 version 

 

  was:
as said in this pr:  [https://github.com/apache/spark/pull/31891] , 

 

So seems the key difference between DEICMAL and NUMERIC are:
{quote}implementation-defined decimal precision equal to or greater than the 
value of the specified precision.
{quote}
Decimal can have _at least_ specified precision (which means can have more) 
whereas numeric should have exactly specified precision.

Spark's decimal satisfy both so I think NUMERIC as a synonym of DECIMAL makes 
sense.

 

my  code need  needs to use NUMERIC type in spark2.4 version 

 


> Spark 2.4.x version does not support numeric
> 
>
> Key: SPARK-35007
> URL: https://issues.apache.org/jira/browse/SPARK-35007
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 2.4.7
>Reporter: Sun BiaoBiao
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> as said in this pr:  [https://github.com/apache/spark/pull/31891] , 
>  
> So seems the key difference between DEICMAL and NUMERIC are:
> {quote}implementation-defined decimal precision equal to or greater than the 
> value of the specified precision.
> {quote}
> Decimal can have _at least_ specified precision (which means can have more) 
> whereas numeric should have exactly specified precision.
> Spark's decimal satisfy both so I think NUMERIC as a synonym of DECIMAL makes 
> sense.
>  
> my  code  needs to use NUMERIC type in spark2.4 version 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-35007) Spark 2.4 version does not support numeric

2021-04-09 Thread Sun BiaoBiao (Jira) 
Sun BiaoBiao created SPARK-35007:


 Summary: Spark 2.4 version does not support numeric
 Key: SPARK-35007
 URL: https://issues.apache.org/jira/browse/SPARK-35007
 Project: Spark
  Issue Type: Improvement
  Components: SQL
Affects Versions: 2.4.7
Reporter: Sun BiaoBiao


as said in this pr:  [https://github.com/apache/spark/pull/31891] , 

 

So seems the key difference between DEICMAL and NUMERIC are:
{quote}implementation-defined decimal precision equal to or greater than the 
value of the specified precision.
{quote}
Decimal can have _at least_ specified precision (which means can have more) 
whereas numeric should have exactly specified precision.

Spark's decimal satisfy both so I think NUMERIC as a synonym of DECIMAL makes 
sense.

 

my  code need  needs to use NUMERIC type in spark2.4 version 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org