[jira] [Updated] (SPARK-38608) [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Summary: [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any` (was: Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns) > [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` > and`DataFrame.any` > --- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Priority: Major > > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38608) [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Description: Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any` to include only boolean columns. (was: [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`) > [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` > and`DataFrame.any` > --- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Priority: Major > > Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any` to > include only boolean columns. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38608) [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Description: [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any` (was: Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns) > [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` > and`DataFrame.any` > --- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Priority: Major > > [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` > and`DataFrame.any` -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-35781) Support Spark on Apple Silicon on macOS natively on Java 17
[ https://issues.apache.org/jira/browse/SPARK-35781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-35781: Labels: release-notes (was: ) > Support Spark on Apple Silicon on macOS natively on Java 17 > --- > > Key: SPARK-35781 > URL: https://issues.apache.org/jira/browse/SPARK-35781 > Project: Spark > Issue Type: New Feature > Components: Build >Affects Versions: 3.3.0 >Reporter: DB Tsai >Assignee: Dongjoon Hyun >Priority: Major > Labels: release-notes > > This is an umbrella JIRA tracking the progress of supporting Apple Silicon on > macOS natively. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38610) UI for Pandas API on Spark
[ https://issues.apache.org/jira/browse/SPARK-38610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38610: - Priority: Critical (was: Major) > UI for Pandas API on Spark > -- > > Key: SPARK-38610 > URL: https://issues.apache.org/jira/browse/SPARK-38610 > Project: Spark > Issue Type: Improvement > Components: PySpark, Web UI >Affects Versions: 3.4.0 >Reporter: Hyukjin Kwon >Priority: Critical > > Currently Pandas API on Spark does not have its dedicated UI which mixes up > with SQL UI tab. It should be better to have a dedicated page -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38610) UI for Pandas API on Spark
[ https://issues.apache.org/jira/browse/SPARK-38610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38610: - Component/s: Web UI > UI for Pandas API on Spark > -- > > Key: SPARK-38610 > URL: https://issues.apache.org/jira/browse/SPARK-38610 > Project: Spark > Issue Type: Improvement > Components: PySpark, Web UI >Affects Versions: 3.4.0 >Reporter: Hyukjin Kwon >Priority: Major > > Currently Pandas API on Spark does not have its dedicated UI which mixes up > with SQL UI tab. It should be better to have a dedicated page -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-38610) UI for Pandas API on Spark
Hyukjin Kwon created SPARK-38610: Summary: UI for Pandas API on Spark Key: SPARK-38610 URL: https://issues.apache.org/jira/browse/SPARK-38610 Project: Spark Issue Type: Improvement Components: PySpark Affects Versions: 3.4.0 Reporter: Hyukjin Kwon Currently Pandas API on Spark does not have its dedicated UI which mixes up with SQL UI tab. It should be better to have a dedicated page -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-38607) Test result report for ANSI mode
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38607. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35916 [https://github.com/apache/spark/pull/35916] > Test result report for ANSI mode > > > Key: SPARK-38607 > URL: https://issues.apache.org/jira/browse/SPARK-38607 > Project: Spark > Issue Type: Improvement > Components: Project Infra >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Assignee: Hyukjin Kwon >Priority: Major > Fix For: 3.3.0 > > > We should add the test report results like > https://github.com/apache/spark/runs/5618763442 for ANSI mode too. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38607) Test result report for ANSI mode
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38607: Assignee: Hyukjin Kwon > Test result report for ANSI mode > > > Key: SPARK-38607 > URL: https://issues.apache.org/jira/browse/SPARK-38607 > Project: Spark > Issue Type: Improvement > Components: Project Infra >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Assignee: Hyukjin Kwon >Priority: Major > > We should add the test report results like > https://github.com/apache/spark/runs/5618763442 for ANSI mode too. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38488) Spark doc build not work on Mac OS M1
[
https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509585#comment-17509585
]
Apache Spark commented on SPARK-38488:
--
User 'Yikun' has created a pull request for this issue:
https://github.com/apache/spark/pull/35918
> Spark doc build not work on Mac OS M1
> -
>
> Key: SPARK-38488
> URL: https://issues.apache.org/jira/browse/SPARK-38488
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
>Affects Versions: 3.3.0, 3.4.0
>Reporter: Yikun Jiang
>Priority: Major
>
>
> {code:java}
> diff --git a/docs/.bundle/config b/docs/.bundle/config
> index b13821f801..68c1ee493a 100644
> --- a/docs/.bundle/config
> +++ b/docs/.bundle/config
> @@ -1,2 +1,3 @@
> ---
> BUNDLE_PATH: ".local_ruby_bundle"
> +BUNDLE_BUILD__FFI: "--enable-libffi-alloc"
> diff --git a/docs/Gemfile b/docs/Gemfile
> index f991622708..6c35201296 100644
> --- a/docs/Gemfile
> +++ b/docs/Gemfile
> @@ -17,6 +17,7 @@
> source "https://rubygems.org;
> +gem "ffi", "1.15.5"
> gem "jekyll", "4.2.1"
> gem "rouge", "3.26.0"
> gem "jekyll-redirect-from", "0.16.0"
> {code}
> After above patch redo `bundle install`, then it works, you could see this as
> ref if you meet the same issue.
> will take a deep look to solve this.
>
> related: https://github.com/ffi/ffi/issues/864
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38488) Spark doc build not work on Mac OS M1
[
https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38488:
Assignee: Apache Spark
> Spark doc build not work on Mac OS M1
> -
>
> Key: SPARK-38488
> URL: https://issues.apache.org/jira/browse/SPARK-38488
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
>Affects Versions: 3.3.0, 3.4.0
>Reporter: Yikun Jiang
>Assignee: Apache Spark
>Priority: Major
>
>
> {code:java}
> diff --git a/docs/.bundle/config b/docs/.bundle/config
> index b13821f801..68c1ee493a 100644
> --- a/docs/.bundle/config
> +++ b/docs/.bundle/config
> @@ -1,2 +1,3 @@
> ---
> BUNDLE_PATH: ".local_ruby_bundle"
> +BUNDLE_BUILD__FFI: "--enable-libffi-alloc"
> diff --git a/docs/Gemfile b/docs/Gemfile
> index f991622708..6c35201296 100644
> --- a/docs/Gemfile
> +++ b/docs/Gemfile
> @@ -17,6 +17,7 @@
> source "https://rubygems.org;
> +gem "ffi", "1.15.5"
> gem "jekyll", "4.2.1"
> gem "rouge", "3.26.0"
> gem "jekyll-redirect-from", "0.16.0"
> {code}
> After above patch redo `bundle install`, then it works, you could see this as
> ref if you meet the same issue.
> will take a deep look to solve this.
>
> related: https://github.com/ffi/ffi/issues/864
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38488) Spark doc build not work on Mac OS M1
[
https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38488:
Assignee: (was: Apache Spark)
> Spark doc build not work on Mac OS M1
> -
>
> Key: SPARK-38488
> URL: https://issues.apache.org/jira/browse/SPARK-38488
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
>Affects Versions: 3.3.0, 3.4.0
>Reporter: Yikun Jiang
>Priority: Major
>
>
> {code:java}
> diff --git a/docs/.bundle/config b/docs/.bundle/config
> index b13821f801..68c1ee493a 100644
> --- a/docs/.bundle/config
> +++ b/docs/.bundle/config
> @@ -1,2 +1,3 @@
> ---
> BUNDLE_PATH: ".local_ruby_bundle"
> +BUNDLE_BUILD__FFI: "--enable-libffi-alloc"
> diff --git a/docs/Gemfile b/docs/Gemfile
> index f991622708..6c35201296 100644
> --- a/docs/Gemfile
> +++ b/docs/Gemfile
> @@ -17,6 +17,7 @@
> source "https://rubygems.org;
> +gem "ffi", "1.15.5"
> gem "jekyll", "4.2.1"
> gem "rouge", "3.26.0"
> gem "jekyll-redirect-from", "0.16.0"
> {code}
> After above patch redo `bundle install`, then it works, you could see this as
> ref if you meet the same issue.
> will take a deep look to solve this.
>
> related: https://github.com/ffi/ffi/issues/864
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38488) Spark doc build not work on Mac OS M1
[
https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-38488:
Affects Version/s: 3.4.0
> Spark doc build not work on Mac OS M1
> -
>
> Key: SPARK-38488
> URL: https://issues.apache.org/jira/browse/SPARK-38488
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
>Affects Versions: 3.3.0, 3.4.0
>Reporter: Yikun Jiang
>Priority: Major
>
>
> {code:java}
> diff --git a/docs/.bundle/config b/docs/.bundle/config
> index b13821f801..68c1ee493a 100644
> --- a/docs/.bundle/config
> +++ b/docs/.bundle/config
> @@ -1,2 +1,3 @@
> ---
> BUNDLE_PATH: ".local_ruby_bundle"
> +BUNDLE_BUILD__FFI: "--enable-libffi-alloc"
> diff --git a/docs/Gemfile b/docs/Gemfile
> index f991622708..6c35201296 100644
> --- a/docs/Gemfile
> +++ b/docs/Gemfile
> @@ -17,6 +17,7 @@
> source "https://rubygems.org;
> +gem "ffi", "1.15.5"
> gem "jekyll", "4.2.1"
> gem "rouge", "3.26.0"
> gem "jekyll-redirect-from", "0.16.0"
> {code}
> After above patch redo `bundle install`, then it works, you could see this as
> ref if you meet the same issue.
> will take a deep look to solve this.
>
> related: https://github.com/ffi/ffi/issues/864
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509565#comment-17509565 ] Apache Spark commented on SPARK-38609: -- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/35917 > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of > KOALAS_USAGE_LOGGER > --- > > Key: SPARK-38609 > URL: https://issues.apache.org/jira/browse/SPARK-38609 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Minor > > There's one place missing about the renaming Koalas to pandas API on Spark > (POS). We should add an alias for this. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38609: Assignee: Apache Spark > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of > KOALAS_USAGE_LOGGER > --- > > Key: SPARK-38609 > URL: https://issues.apache.org/jira/browse/SPARK-38609 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Assignee: Apache Spark >Priority: Minor > > There's one place missing about the renaming Koalas to pandas API on Spark > (POS). We should add an alias for this. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38609: Assignee: (was: Apache Spark) > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of > KOALAS_USAGE_LOGGER > --- > > Key: SPARK-38609 > URL: https://issues.apache.org/jira/browse/SPARK-38609 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Minor > > There's one place missing about the renaming Koalas to pandas API on Spark > (POS). We should add an alias for this. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509566#comment-17509566 ] Apache Spark commented on SPARK-38609: -- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/35917 > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of > KOALAS_USAGE_LOGGER > --- > > Key: SPARK-38609 > URL: https://issues.apache.org/jira/browse/SPARK-38609 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Minor > > There's one place missing about the renaming Koalas to pandas API on Spark > (POS). We should add an alias for this. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38609: - Summary: Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER (was: Rename KOALAS_USAGE_LOGGER to POS_USAGE_LOGGER) > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of > KOALAS_USAGE_LOGGER > --- > > Key: SPARK-38609 > URL: https://issues.apache.org/jira/browse/SPARK-38609 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Minor > > There's one place missing about the renaming Koalas to pandas API on Spark > (POS). We should rename that enironment variable -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38609: - Description: There's one place missing about the renaming Koalas to pandas API on Spark (POS). We should add an alias for this. (was: There's one place missing about the renaming Koalas to pandas API on Spark (POS). We should rename that enironment variable) > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of > KOALAS_USAGE_LOGGER > --- > > Key: SPARK-38609 > URL: https://issues.apache.org/jira/browse/SPARK-38609 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Minor > > There's one place missing about the renaming Koalas to pandas API on Spark > (POS). We should add an alias for this. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions
[
https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38556:
Assignee: Apache Spark
> Disable Pandas usage logging for method calls inside @contextmanager functions
> --
>
> Key: SPARK-38556
> URL: https://issues.apache.org/jira/browse/SPARK-38556
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 3.2.1
>Reporter: Yihong He
>Assignee: Apache Spark
>Priority: Minor
>
> Currently, calls inside @contextmanager functions are treated as external for
> *with* statements.
> For example, the below code records config.set_option calls inside
> ps.option_context(...)
> {code:java}
> with ps.option_context("compute.ops_on_diff_frames", True):
> pass {code}
> We should disable usage logging for calls inside @contextmanager functions to
> improve accuracy of the usage data
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions
[
https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38556:
Assignee: (was: Apache Spark)
> Disable Pandas usage logging for method calls inside @contextmanager functions
> --
>
> Key: SPARK-38556
> URL: https://issues.apache.org/jira/browse/SPARK-38556
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 3.2.1
>Reporter: Yihong He
>Priority: Minor
>
> Currently, calls inside @contextmanager functions are treated as external for
> *with* statements.
> For example, the below code records config.set_option calls inside
> ps.option_context(...)
> {code:java}
> with ps.option_context("compute.ops_on_diff_frames", True):
> pass {code}
> We should disable usage logging for calls inside @contextmanager functions to
> improve accuracy of the usage data
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-38609) Rename KOALAS_USAGE_LOGGER to POS_USAGE_LOGGER
Hyukjin Kwon created SPARK-38609: Summary: Rename KOALAS_USAGE_LOGGER to POS_USAGE_LOGGER Key: SPARK-38609 URL: https://issues.apache.org/jira/browse/SPARK-38609 Project: Spark Issue Type: Improvement Components: PySpark Affects Versions: 3.3.0 Reporter: Hyukjin Kwon There's one place missing about the renaming Koalas to pandas API on Spark (POS). We should rename that enironment variable -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions
[
https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reopened SPARK-38556:
--
Reverted at
https://github.com/apache/spark/commit/dcc66e4b4933c3493be3c6f2bb0118cd1680b530
and and
https://github.com/apache/spark/commit/1a720d30f77a8b63fe484c60fc4f4953f9ff1af8.
Ref: https://github.com/apache/spark/pull/35861#discussion_r830716928
> Disable Pandas usage logging for method calls inside @contextmanager functions
> --
>
> Key: SPARK-38556
> URL: https://issues.apache.org/jira/browse/SPARK-38556
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 3.2.1
>Reporter: Yihong He
>Priority: Minor
>
> Currently, calls inside @contextmanager functions are treated as external for
> *with* statements.
> For example, the below code records config.set_option calls inside
> ps.option_context(...)
> {code:java}
> with ps.option_context("compute.ops_on_diff_frames", True):
> pass {code}
> We should disable usage logging for calls inside @contextmanager functions to
> improve accuracy of the usage data
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38608: Assignee: (was: Apache Spark) > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns > -- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Priority: Major > > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509550#comment-17509550 ] Apache Spark commented on SPARK-38608: -- User 'xinrong-databricks' has created a pull request for this issue: https://github.com/apache/spark/pull/35888 > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns > -- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Priority: Major > > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509549#comment-17509549 ] Apache Spark commented on SPARK-38608: -- User 'xinrong-databricks' has created a pull request for this issue: https://github.com/apache/spark/pull/35888 > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns > -- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Priority: Major > > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38608: Assignee: Apache Spark > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns > -- > > Key: SPARK-38608 > URL: https://issues.apache.org/jira/browse/SPARK-38608 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 3.4.0 >Reporter: Xinrong Meng >Assignee: Apache Spark >Priority: Major > > Implement `bool_only` parameter of `DataFrame.all` to include only boolean > columns -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns
Xinrong Meng created SPARK-38608: Summary: Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns Key: SPARK-38608 URL: https://issues.apache.org/jira/browse/SPARK-38608 Project: Spark Issue Type: Improvement Components: PySpark Affects Versions: 3.4.0 Reporter: Xinrong Meng Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38607) Test result report for ANSI mode
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38607: Assignee: (was: Apache Spark) > Test result report for ANSI mode > > > Key: SPARK-38607 > URL: https://issues.apache.org/jira/browse/SPARK-38607 > Project: Spark > Issue Type: Improvement > Components: Project Infra >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Major > > We should add the test report results like > https://github.com/apache/spark/runs/5618763442 for ANSI mode too. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38607) Test result report for ANSI mode
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509545#comment-17509545 ] Apache Spark commented on SPARK-38607: -- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/35916 > Test result report for ANSI mode > > > Key: SPARK-38607 > URL: https://issues.apache.org/jira/browse/SPARK-38607 > Project: Spark > Issue Type: Improvement > Components: Project Infra >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Priority: Major > > We should add the test report results like > https://github.com/apache/spark/runs/5618763442 for ANSI mode too. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38607) Test result report for ANSI mode
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38607: Assignee: Apache Spark > Test result report for ANSI mode > > > Key: SPARK-38607 > URL: https://issues.apache.org/jira/browse/SPARK-38607 > Project: Spark > Issue Type: Improvement > Components: Project Infra >Affects Versions: 3.3.0 >Reporter: Hyukjin Kwon >Assignee: Apache Spark >Priority: Major > > We should add the test report results like > https://github.com/apache/spark/runs/5618763442 for ANSI mode too. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-38607) Test result report for ANSI mode
Hyukjin Kwon created SPARK-38607: Summary: Test result report for ANSI mode Key: SPARK-38607 URL: https://issues.apache.org/jira/browse/SPARK-38607 Project: Spark Issue Type: Improvement Components: Project Infra Affects Versions: 3.3.0 Reporter: Hyukjin Kwon We should add the test report results like https://github.com/apache/spark/runs/5618763442 for ANSI mode too. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token
[
https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38456:
Assignee: Apache Spark
> Improve error messages of no viable alternative, extraneous input and missing
> token
>
>
> Key: SPARK-38456
> URL: https://issues.apache.org/jira/browse/SPARK-38456
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.3.0
>Reporter: Xinyi Yu
>Assignee: Apache Spark
>Priority: Major
>
> Please view the parent task description for the general idea:
> https://issues.apache.org/jira/browse/SPARK-38384
> h1. No viable alternative
> Query
> {code:java}
> select ( {code}
> Before
> {code:java}
> no viable alternative at input ‘(‘(line 1, pos 8){code}
> After
> {code:java}
> Syntax error at or near end of input(line 1, pos 8){code}
> Changes
> # Messages change. From ‘no viable alternative’ to ‘syntax error at or near’
> # Problematic snippet changes. From the rule’s starting token to the
> offending token, to the pure problematic token (in this case it’s EOF).
> # Substitute the EOF to user-readable ones, end of input.
> h1. Extraneous Input
> Query
> {code:java}
> CREATE TABLE my_tab(a: INT COMMENT 'test', b: STRING) USING parquet {code}
> Before
> {code:java}
> extraneous input ':' expecting {'APPLY', 'CALLED', 'CHANGES', 'CLONE',
> 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIONS', 'CREDENTIAL',
> 'CREDENTIALS', 'DEEP', 'DEFINER', 'DELTA', 'DETERMINISTIC', 'ENCRYPTION',
> 'EXPECT', 'FAIL', 'FILES', 'FORMAT_OPTIONS', 'HISTORY', 'INCREMENTAL',
> 'INPUT', 'INVOKER', 'LANGUAGE', 'LIVE', 'MATERIALIZED', 'MODIFIES',
> 'OPTIMIZE', 'PATTERN', 'READS', 'RESTORE', 'RETURN', 'RETURNS', 'SAMPLE',
> 'SCD TYPE 1', 'SCD TYPE 2', 'SECURITY', 'SEQUENCE', 'SHALLOW', 'SNAPSHOT',
> 'SPECIFIC', 'SQL', 'STORAGE', 'STREAMING', 'UPDATES', 'UP_TO_DATE',
> 'VIOLATION', 'ZORDER', 'ADD', 'AFTER', 'ALL', 'ALTER', 'ALWAYS', 'ANALYZE',
> 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION',
> 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE',
> 'CAST', 'CATALOG', 'CATALOGS', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER',
> 'CLUSTERED', 'CODE', 'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS',
> 'COMMENT', 'COMMIT', 'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE',
> 'CONSTRAINT', 'COST', 'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE',
> 'CURRENT_TIME', 'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DAY', 'DATA',
> 'DATABASE', 'DATABASES', 'DATEADD', 'DATE_ADD', 'DATEDIFF', 'DATE_DIFF',
> 'DBPROPERTIES', 'DEFAULT', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC',
> 'DESCRIBE', 'DFS', 'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE',
> 'DIV', 'DROP', 'ELSE', 'END', 'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE',
> 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE',
> 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT', 'FIRST', 'FN', 'FOLLOWING', 'FOR',
> 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS',
> 'GENERATED', 'GLOBAL', 'GRANT', 'GRANTS', 'GROUP', 'GROUPING', 'HAVING',
> 'HOUR', 'IDENTITY', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INCREMENT', 'INDEX',
> 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT',
> 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEY', 'KEYS', 'LAST', 'LATERAL',
> 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'ILIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD',
> 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED',
> 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL',
> 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR',
> 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY',
> 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS', 'PERCENTILE_CONT',
> 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING', 'PRIMARY',
> 'PRINCIPALS', 'PROPERTIES', 'PROVIDER', 'PROVIDERS', 'PURGE', 'QUALIFY',
> 'QUERY', 'RANGE', 'RECIPIENT', 'RECIPIENTS', 'RECORDREADER', 'RECORDWRITER',
> 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'REMOVE', 'RENAME', 'REPAIR',
> 'REPEATABLE', 'REPLACE', 'REPLICAS', 'RESET', 'RESPECT', 'RESTRICT',
> 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW',
> 'ROWS', 'SECOND', 'SCHEMA', 'SCHEMAS', 'SELECT', 'SEMI', 'SEPARATED',
> 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHARE',
> 'SHARES', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS',
> 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'SYNC', 'SYSTEM_TIME',
> 'SYSTEM_VERSION', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES',
> TEMPORARY, 'TERMINATED', 'THEN', 'TIME', 'TIMESTAMP', 'TIMESTAMPADD',
> 'TIMESTAMPDIFF', 'TO', 'TOUCH',
[jira] [Assigned] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token
[
https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38456:
Assignee: (was: Apache Spark)
> Improve error messages of no viable alternative, extraneous input and missing
> token
>
>
> Key: SPARK-38456
> URL: https://issues.apache.org/jira/browse/SPARK-38456
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.3.0
>Reporter: Xinyi Yu
>Priority: Major
>
> Please view the parent task description for the general idea:
> https://issues.apache.org/jira/browse/SPARK-38384
> h1. No viable alternative
> Query
> {code:java}
> select ( {code}
> Before
> {code:java}
> no viable alternative at input ‘(‘(line 1, pos 8){code}
> After
> {code:java}
> Syntax error at or near end of input(line 1, pos 8){code}
> Changes
> # Messages change. From ‘no viable alternative’ to ‘syntax error at or near’
> # Problematic snippet changes. From the rule’s starting token to the
> offending token, to the pure problematic token (in this case it’s EOF).
> # Substitute the EOF to user-readable ones, end of input.
> h1. Extraneous Input
> Query
> {code:java}
> CREATE TABLE my_tab(a: INT COMMENT 'test', b: STRING) USING parquet {code}
> Before
> {code:java}
> extraneous input ':' expecting {'APPLY', 'CALLED', 'CHANGES', 'CLONE',
> 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIONS', 'CREDENTIAL',
> 'CREDENTIALS', 'DEEP', 'DEFINER', 'DELTA', 'DETERMINISTIC', 'ENCRYPTION',
> 'EXPECT', 'FAIL', 'FILES', 'FORMAT_OPTIONS', 'HISTORY', 'INCREMENTAL',
> 'INPUT', 'INVOKER', 'LANGUAGE', 'LIVE', 'MATERIALIZED', 'MODIFIES',
> 'OPTIMIZE', 'PATTERN', 'READS', 'RESTORE', 'RETURN', 'RETURNS', 'SAMPLE',
> 'SCD TYPE 1', 'SCD TYPE 2', 'SECURITY', 'SEQUENCE', 'SHALLOW', 'SNAPSHOT',
> 'SPECIFIC', 'SQL', 'STORAGE', 'STREAMING', 'UPDATES', 'UP_TO_DATE',
> 'VIOLATION', 'ZORDER', 'ADD', 'AFTER', 'ALL', 'ALTER', 'ALWAYS', 'ANALYZE',
> 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION',
> 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE',
> 'CAST', 'CATALOG', 'CATALOGS', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER',
> 'CLUSTERED', 'CODE', 'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS',
> 'COMMENT', 'COMMIT', 'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE',
> 'CONSTRAINT', 'COST', 'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE',
> 'CURRENT_TIME', 'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DAY', 'DATA',
> 'DATABASE', 'DATABASES', 'DATEADD', 'DATE_ADD', 'DATEDIFF', 'DATE_DIFF',
> 'DBPROPERTIES', 'DEFAULT', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC',
> 'DESCRIBE', 'DFS', 'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE',
> 'DIV', 'DROP', 'ELSE', 'END', 'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE',
> 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE',
> 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT', 'FIRST', 'FN', 'FOLLOWING', 'FOR',
> 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS',
> 'GENERATED', 'GLOBAL', 'GRANT', 'GRANTS', 'GROUP', 'GROUPING', 'HAVING',
> 'HOUR', 'IDENTITY', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INCREMENT', 'INDEX',
> 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT',
> 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEY', 'KEYS', 'LAST', 'LATERAL',
> 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'ILIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD',
> 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED',
> 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL',
> 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR',
> 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY',
> 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS', 'PERCENTILE_CONT',
> 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING', 'PRIMARY',
> 'PRINCIPALS', 'PROPERTIES', 'PROVIDER', 'PROVIDERS', 'PURGE', 'QUALIFY',
> 'QUERY', 'RANGE', 'RECIPIENT', 'RECIPIENTS', 'RECORDREADER', 'RECORDWRITER',
> 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'REMOVE', 'RENAME', 'REPAIR',
> 'REPEATABLE', 'REPLACE', 'REPLICAS', 'RESET', 'RESPECT', 'RESTRICT',
> 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW',
> 'ROWS', 'SECOND', 'SCHEMA', 'SCHEMAS', 'SELECT', 'SEMI', 'SEPARATED',
> 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHARE',
> 'SHARES', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS',
> 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'SYNC', 'SYSTEM_TIME',
> 'SYSTEM_VERSION', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES',
> TEMPORARY, 'TERMINATED', 'THEN', 'TIME', 'TIMESTAMP', 'TIMESTAMPADD',
> 'TIMESTAMPDIFF', 'TO', 'TOUCH', 'TRAILING', 'TRANSACTION',
[jira] [Commented] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token
[
https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509534#comment-17509534
]
Apache Spark commented on SPARK-38456:
--
User 'anchovYu' has created a pull request for this issue:
https://github.com/apache/spark/pull/35915
> Improve error messages of no viable alternative, extraneous input and missing
> token
>
>
> Key: SPARK-38456
> URL: https://issues.apache.org/jira/browse/SPARK-38456
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.3.0
>Reporter: Xinyi Yu
>Priority: Major
>
> Please view the parent task description for the general idea:
> https://issues.apache.org/jira/browse/SPARK-38384
> h1. No viable alternative
> Query
> {code:java}
> select ( {code}
> Before
> {code:java}
> no viable alternative at input ‘(‘(line 1, pos 8){code}
> After
> {code:java}
> Syntax error at or near end of input(line 1, pos 8){code}
> Changes
> # Messages change. From ‘no viable alternative’ to ‘syntax error at or near’
> # Problematic snippet changes. From the rule’s starting token to the
> offending token, to the pure problematic token (in this case it’s EOF).
> # Substitute the EOF to user-readable ones, end of input.
> h1. Extraneous Input
> Query
> {code:java}
> CREATE TABLE my_tab(a: INT COMMENT 'test', b: STRING) USING parquet {code}
> Before
> {code:java}
> extraneous input ':' expecting {'APPLY', 'CALLED', 'CHANGES', 'CLONE',
> 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIONS', 'CREDENTIAL',
> 'CREDENTIALS', 'DEEP', 'DEFINER', 'DELTA', 'DETERMINISTIC', 'ENCRYPTION',
> 'EXPECT', 'FAIL', 'FILES', 'FORMAT_OPTIONS', 'HISTORY', 'INCREMENTAL',
> 'INPUT', 'INVOKER', 'LANGUAGE', 'LIVE', 'MATERIALIZED', 'MODIFIES',
> 'OPTIMIZE', 'PATTERN', 'READS', 'RESTORE', 'RETURN', 'RETURNS', 'SAMPLE',
> 'SCD TYPE 1', 'SCD TYPE 2', 'SECURITY', 'SEQUENCE', 'SHALLOW', 'SNAPSHOT',
> 'SPECIFIC', 'SQL', 'STORAGE', 'STREAMING', 'UPDATES', 'UP_TO_DATE',
> 'VIOLATION', 'ZORDER', 'ADD', 'AFTER', 'ALL', 'ALTER', 'ALWAYS', 'ANALYZE',
> 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION',
> 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE',
> 'CAST', 'CATALOG', 'CATALOGS', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER',
> 'CLUSTERED', 'CODE', 'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS',
> 'COMMENT', 'COMMIT', 'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE',
> 'CONSTRAINT', 'COST', 'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE',
> 'CURRENT_TIME', 'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DAY', 'DATA',
> 'DATABASE', 'DATABASES', 'DATEADD', 'DATE_ADD', 'DATEDIFF', 'DATE_DIFF',
> 'DBPROPERTIES', 'DEFAULT', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC',
> 'DESCRIBE', 'DFS', 'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE',
> 'DIV', 'DROP', 'ELSE', 'END', 'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE',
> 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE',
> 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT', 'FIRST', 'FN', 'FOLLOWING', 'FOR',
> 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS',
> 'GENERATED', 'GLOBAL', 'GRANT', 'GRANTS', 'GROUP', 'GROUPING', 'HAVING',
> 'HOUR', 'IDENTITY', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INCREMENT', 'INDEX',
> 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT',
> 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEY', 'KEYS', 'LAST', 'LATERAL',
> 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'ILIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD',
> 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED',
> 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL',
> 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR',
> 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY',
> 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS', 'PERCENTILE_CONT',
> 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING', 'PRIMARY',
> 'PRINCIPALS', 'PROPERTIES', 'PROVIDER', 'PROVIDERS', 'PURGE', 'QUALIFY',
> 'QUERY', 'RANGE', 'RECIPIENT', 'RECIPIENTS', 'RECORDREADER', 'RECORDWRITER',
> 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'REMOVE', 'RENAME', 'REPAIR',
> 'REPEATABLE', 'REPLACE', 'REPLICAS', 'RESET', 'RESPECT', 'RESTRICT',
> 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW',
> 'ROWS', 'SECOND', 'SCHEMA', 'SCHEMAS', 'SELECT', 'SEMI', 'SEPARATED',
> 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHARE',
> 'SHARES', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS',
> 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'SYNC', 'SYSTEM_TIME',
> 'SYSTEM_VERSION', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES',
> TEMPORARY, 'TERMINATED', 'THEN',
[jira] [Updated] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token
[
https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinyi Yu updated SPARK-38456:
-
Description:
Please view the parent task description for the general idea:
https://issues.apache.org/jira/browse/SPARK-38384
h1. No viable alternative
Query
{code:java}
select ( {code}
Before
{code:java}
no viable alternative at input ‘(‘(line 1, pos 8){code}
After
{code:java}
Syntax error at or near end of input(line 1, pos 8){code}
Changes
# Messages change. From ‘no viable alternative’ to ‘syntax error at or near’
# Problematic snippet changes. From the rule’s starting token to the offending
token, to the pure problematic token (in this case it’s EOF).
# Substitute the EOF to user-readable ones, end of input.
h1. Extraneous Input
Query
{code:java}
CREATE TABLE my_tab(a: INT COMMENT 'test', b: STRING) USING parquet {code}
Before
{code:java}
extraneous input ':' expecting {'APPLY', 'CALLED', 'CHANGES', 'CLONE',
'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIONS', 'CREDENTIAL',
'CREDENTIALS', 'DEEP', 'DEFINER', 'DELTA', 'DETERMINISTIC', 'ENCRYPTION',
'EXPECT', 'FAIL', 'FILES', 'FORMAT_OPTIONS', 'HISTORY', 'INCREMENTAL', 'INPUT',
'INVOKER', 'LANGUAGE', 'LIVE', 'MATERIALIZED', 'MODIFIES', 'OPTIMIZE',
'PATTERN', 'READS', 'RESTORE', 'RETURN', 'RETURNS', 'SAMPLE', 'SCD TYPE 1',
'SCD TYPE 2', 'SECURITY', 'SEQUENCE', 'SHALLOW', 'SNAPSHOT', 'SPECIFIC', 'SQL',
'STORAGE', 'STREAMING', 'UPDATES', 'UP_TO_DATE', 'VIOLATION', 'ZORDER', 'ADD',
'AFTER', 'ALL', 'ALTER', 'ALWAYS', 'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE',
'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET',
'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE', 'CAST', 'CATALOG', 'CATALOGS',
'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED', 'CODE', 'CODEGEN',
'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT', 'COMPACT',
'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST', 'CREATE',
'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME',
'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DAY', 'DATA', 'DATABASE', 'DATABASES',
'DATEADD', 'DATE_ADD', 'DATEDIFF', 'DATE_DIFF', 'DBPROPERTIES', 'DEFAULT',
'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS', 'DIRECTORIES',
'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DIV', 'DROP', 'ELSE', 'END', 'ESCAPE',
'ESCAPED', 'EXCEPT', 'EXCHANGE', 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED',
'EXTERNAL', 'EXTRACT', 'FALSE', 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT',
'FIRST', 'FN', 'FOLLOWING', 'FOR', 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM',
'FULL', 'FUNCTION', 'FUNCTIONS', 'GENERATED', 'GLOBAL', 'GRANT', 'GRANTS',
'GROUP', 'GROUPING', 'HAVING', 'HOUR', 'IDENTITY', 'IF', 'IGNORE', 'IMPORT',
'IN', 'INCREMENT', 'INDEX', 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT',
'INSERT', 'INTERSECT', 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEY',
'KEYS', 'LAST', 'LATERAL', 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'ILIKE', 'LIMIT',
'LINES', 'LIST', 'LOAD', 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL',
'MACRO', 'MAP', 'MATCHED', 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE',
'NAMESPACES', 'NATURAL', 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY',
'OPTION', 'OPTIONS', 'OR', 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER',
'OVERLAPS', 'OVERLAY', 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS',
'PERCENTILE_CONT', 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING',
'PRIMARY', 'PRINCIPALS', 'PROPERTIES', 'PROVIDER', 'PROVIDERS', 'PURGE',
'QUALIFY', 'QUERY', 'RANGE', 'RECIPIENT', 'RECIPIENTS', 'RECORDREADER',
'RECORDWRITER', 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'REMOVE',
'RENAME', 'REPAIR', 'REPEATABLE', 'REPLACE', 'REPLICAS', 'RESET', 'RESPECT',
'RESTRICT', 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP',
'ROW', 'ROWS', 'SECOND', 'SCHEMA', 'SCHEMAS', 'SELECT', 'SEMI', 'SEPARATED',
'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHARE',
'SHARES', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS',
'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'SYNC', 'SYSTEM_TIME',
'SYSTEM_VERSION', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES', TEMPORARY,
'TERMINATED', 'THEN', 'TIME', 'TIMESTAMP', 'TIMESTAMPADD', 'TIMESTAMPDIFF',
'TO', 'TOUCH', 'TRAILING', 'TRANSACTION', 'TRANSACTIONS', 'TRANSFORM', 'TRIM',
'TRUE', 'TRUNCATE', 'TRY_CAST', 'TYPE', 'UNARCHIVE', 'UNBOUNDED', 'UNCACHE',
'UNION', 'UNIQUE', 'UNKNOWN', 'UNLOCK', 'UNSET', 'UPDATE', 'USE', 'USER',
'USING', 'VALUES', 'VERSION', 'VIEW', 'VIEWS', 'WHEN', 'WHERE', 'WINDOW',
'WITH', 'WITHIN', 'YEAR', 'ZONE', IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1,
pos 21){code}
After
{code:java}
Syntax error at or near ':': extra input ':'(line 1, pos 21){code}
Changes
# Messages change. Add a “Syntax error at or near” message.
# Left all expecting.
# Words change. From ‘extraneous’ to ‘extra’.
h1. Missing token
Query
{code:java}
select count(a from b
[jira] [Updated] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token
[
https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinyi Yu updated SPARK-38456:
-
Description:
Please view the parent task description for the general idea:
https://issues.apache.org/jira/browse/SPARK-38384
h1. No viable alternative
Query
{code:java}
select ( {code}
Before
{code:java}
no viable alternative at input ‘(‘(line 1, pos 8){code}
After
{code:java}
Syntax error at or near end of input(line 1, pos 8){code}
Changes
# Messages change. From ‘no viable alternative’ to ‘syntax error at or near’
# Problematic snippet changes. From the rule’s starting token to the offending
token, to the pure problematic token (in this case it’s EOF).
# Substitute the EOF to user-readable ones, end of input.
h1. Extraneous Input
Query
{code:java}
CREATE TABLE my_tab(a: INT COMMENT 'test', b: STRING) USING parquet {code}
Before
{code:java}
extraneous input ':' expecting {'APPLY', 'CALLED', 'CHANGES', 'CLONE',
'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIONS', 'CREDENTIAL',
'CREDENTIALS', 'DEEP', 'DEFINER', 'DELTA', 'DETERMINISTIC', 'ENCRYPTION',
'EXPECT', 'FAIL', 'FILES', 'FORMAT_OPTIONS', 'HISTORY', 'INCREMENTAL', 'INPUT',
'INVOKER', 'LANGUAGE', 'LIVE', 'MATERIALIZED', 'MODIFIES', 'OPTIMIZE',
'PATTERN', 'READS', 'RESTORE', 'RETURN', 'RETURNS', 'SAMPLE', 'SCD TYPE 1',
'SCD TYPE 2', 'SECURITY', 'SEQUENCE', 'SHALLOW', 'SNAPSHOT', 'SPECIFIC', 'SQL',
'STORAGE', 'STREAMING', 'UPDATES', 'UP_TO_DATE', 'VIOLATION', 'ZORDER', 'ADD',
'AFTER', 'ALL', 'ALTER', 'ALWAYS', 'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE',
'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET',
'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE', 'CAST', 'CATALOG', 'CATALOGS',
'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED', 'CODE', 'CODEGEN',
'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT', 'COMPACT',
'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST', 'CREATE',
'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME',
'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DAY', 'DATA', 'DATABASE', 'DATABASES',
'DATEADD', 'DATE_ADD', 'DATEDIFF', 'DATE_DIFF', 'DBPROPERTIES', 'DEFAULT',
'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS', 'DIRECTORIES',
'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DIV', 'DROP', 'ELSE', 'END', 'ESCAPE',
'ESCAPED', 'EXCEPT', 'EXCHANGE', 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED',
'EXTERNAL', 'EXTRACT', 'FALSE', 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT',
'FIRST', 'FN', 'FOLLOWING', 'FOR', 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM',
'FULL', 'FUNCTION', 'FUNCTIONS', 'GENERATED', 'GLOBAL', 'GRANT', 'GRANTS',
'GROUP', 'GROUPING', 'HAVING', 'HOUR', 'IDENTITY', 'IF', 'IGNORE', 'IMPORT',
'IN', 'INCREMENT', 'INDEX', 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT',
'INSERT', 'INTERSECT', 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEY',
'KEYS', 'LAST', 'LATERAL', 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'ILIKE', 'LIMIT',
'LINES', 'LIST', 'LOAD', 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL',
'MACRO', 'MAP', 'MATCHED', 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE',
'NAMESPACES', 'NATURAL', 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY',
'OPTION', 'OPTIONS', 'OR', 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER',
'OVERLAPS', 'OVERLAY', 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS',
'PERCENTILE_CONT', 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING',
'PRIMARY', 'PRINCIPALS', 'PROPERTIES', 'PROVIDER', 'PROVIDERS', 'PURGE',
'QUALIFY', 'QUERY', 'RANGE', 'RECIPIENT', 'RECIPIENTS', 'RECORDREADER',
'RECORDWRITER', 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'REMOVE',
'RENAME', 'REPAIR', 'REPEATABLE', 'REPLACE', 'REPLICAS', 'RESET', 'RESPECT',
'RESTRICT', 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP',
'ROW', 'ROWS', 'SECOND', 'SCHEMA', 'SCHEMAS', 'SELECT', 'SEMI', 'SEPARATED',
'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHARE',
'SHARES', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS',
'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'SYNC', 'SYSTEM_TIME',
'SYSTEM_VERSION', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES', TEMPORARY,
'TERMINATED', 'THEN', 'TIME', 'TIMESTAMP', 'TIMESTAMPADD', 'TIMESTAMPDIFF',
'TO', 'TOUCH', 'TRAILING', 'TRANSACTION', 'TRANSACTIONS', 'TRANSFORM', 'TRIM',
'TRUE', 'TRUNCATE', 'TRY_CAST', 'TYPE', 'UNARCHIVE', 'UNBOUNDED', 'UNCACHE',
'UNION', 'UNIQUE', 'UNKNOWN', 'UNLOCK', 'UNSET', 'UPDATE', 'USE', 'USER',
'USING', 'VALUES', 'VERSION', 'VIEW', 'VIEWS', 'WHEN', 'WHERE', 'WINDOW',
'WITH', 'WITHIN', 'YEAR', 'ZONE', IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1,
pos 21){code}
After
{code:java}
Syntax error at or near ':': extra input ':'(line 1, pos 21){code}
Changes
# Messages change. Add a “Syntax error at or near” message.
# Left all expecting.
# Words change. From ‘extraneous’ to ‘extra’.
h1. Missing token
Query
{code:java}
select
[jira] [Assigned] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38606: Assignee: (was: Apache Spark) > Update document to make a good guide of multiple versions of the Spark > Shuffle Service > --- > > Key: SPARK-38606 > URL: https://issues.apache.org/jira/browse/SPARK-38606 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.2.0 >Reporter: tonydoen >Priority: Trivial > Fix For: 3.3.0 > > > We have tried to run multiple versions of the Spark Shuffle Service > according > [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] > but, it wont work. > Then we solved it by using colon when writing %s.classpath instead of commas. > > Related discussing is in > https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38606: Assignee: Apache Spark > Update document to make a good guide of multiple versions of the Spark > Shuffle Service > --- > > Key: SPARK-38606 > URL: https://issues.apache.org/jira/browse/SPARK-38606 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.2.0 >Reporter: tonydoen >Assignee: Apache Spark >Priority: Trivial > Fix For: 3.3.0 > > > We have tried to run multiple versions of the Spark Shuffle Service > according > [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] > but, it wont work. > Then we solved it by using colon when writing %s.classpath instead of commas. > > Related discussing is in > https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509437#comment-17509437 ] Apache Spark commented on SPARK-38606: -- User 'TonyDoen' has created a pull request for this issue: https://github.com/apache/spark/pull/35914 > Update document to make a good guide of multiple versions of the Spark > Shuffle Service > --- > > Key: SPARK-38606 > URL: https://issues.apache.org/jira/browse/SPARK-38606 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.2.0 >Reporter: tonydoen >Priority: Trivial > Fix For: 3.3.0 > > > We have tried to run multiple versions of the Spark Shuffle Service > according > [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] > but, it wont work. > Then we solved it by using colon when writing %s.classpath instead of commas. > > Related discussing is in > https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509432#comment-17509432 ] tonydoen edited comment on SPARK-38606 at 3/20/22, 11:01 AM: - Related Issue : [https://github.com/apache/spark/pull/35914] was (Author: JIRAUSER285351): [https://github.com/apache/spark/pull/35914] > Update document to make a good guide of multiple versions of the Spark > Shuffle Service > --- > > Key: SPARK-38606 > URL: https://issues.apache.org/jira/browse/SPARK-38606 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.2.0 >Reporter: tonydoen >Priority: Trivial > Fix For: 3.3.0 > > > We have tried to run multiple versions of the Spark Shuffle Service > according > [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] > but, it wont work. > Then we solved it by using colon when writing %s.classpath instead of commas. > > Related discussing is in > https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509432#comment-17509432 ] tonydoen commented on SPARK-38606: -- [https://github.com/apache/spark/pull/35914] > Update document to make a good guide of multiple versions of the Spark > Shuffle Service > --- > > Key: SPARK-38606 > URL: https://issues.apache.org/jira/browse/SPARK-38606 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.2.0 >Reporter: tonydoen >Priority: Trivial > Fix For: 3.3.0 > > > We have tried to run multiple versions of the Spark Shuffle Service > according > [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] > but, it wont work. > Then we solved it by using colon when writing %s.classpath instead of commas. > > Related discussing is in > https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tonydoen updated SPARK-38606: - Description: We have tried to run multiple versions of the Spark Shuffle Service according [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] but, it wont work. Then we solved it by using colon when writing %s.classpath instead of commas. Related discussing is in https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 was: We have tried to run multiple versions of the Spark Shuffle Service according [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] but, it wont work. Then we solved it by using colon when writing %s.classpath instead of commas. > Update document to make a good guide of multiple versions of the Spark > Shuffle Service > --- > > Key: SPARK-38606 > URL: https://issues.apache.org/jira/browse/SPARK-38606 > Project: Spark > Issue Type: Documentation > Components: Documentation >Affects Versions: 3.2.0 >Reporter: tonydoen >Priority: Trivial > Fix For: 3.3.0 > > > We have tried to run multiple versions of the Spark Shuffle Service > according > [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] > but, it wont work. > Then we solved it by using colon when writing %s.classpath instead of commas. > > Related discussing is in > https://issues.apache.org/jira/browse/YARN-4577?focusedCommentId=17493624=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17493624 > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service
tonydoen created SPARK-38606: Summary: Update document to make a good guide of multiple versions of the Spark Shuffle Service Key: SPARK-38606 URL: https://issues.apache.org/jira/browse/SPARK-38606 Project: Spark Issue Type: Documentation Components: Documentation Affects Versions: 3.2.0 Reporter: tonydoen Fix For: 3.3.0 We have tried to run multiple versions of the Spark Shuffle Service according [https://github.com/apache/spark/blob/master/docs/running-on-yarn.md] but, it wont work. Then we solved it by using colon when writing %s.classpath instead of commas. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-38605) Retrying on file manager operation in HDFSMetadataLog
[ https://issues.apache.org/jira/browse/SPARK-38605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509394#comment-17509394 ] Jungtaek Lim commented on SPARK-38605: -- I don't have a strong idea about this. It sounds great to be resilient to single failure, but we still need to ensure the behavior is still atomic among multiple trials. The number of trials and proper interval among trials would be something need to think through. (Even if we let them be configurable, reasonable default values are needed.) > Retrying on file manager operation in HDFSMetadataLog > - > > Key: SPARK-38605 > URL: https://issues.apache.org/jira/browse/SPARK-38605 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming >Affects Versions: 3.4.0 >Reporter: L. C. Hsieh >Priority: Major > > Currently HDFSMetadataLog uses CheckpointFileManager to do some file > operation like opening metadata file. It is very easy to be affected by > network blips and causes the streaming query failed. Although we can restart > the streaming query, but it takes more time to recover. > Such file operations should be resilient with such situation by retrying. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
