[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-----------------------------
Description:
[https://github.com/apache/spark/pull/40510] introduce `message StorageLevel`
to `base.proto`, but if we try to import `base.proto` in `catalog.proto` to
reuse `StorageLevel` in `message CacheTable` and run `build/sbt
"connect-common/compile" to compile, there will be following message in compile
log:
{code:java}
spark/connect/base.proto:23:1: File recursively imports itself:
spark/connect/base.proto -> spark/connect/commands.proto ->
spark/connect/relations.proto -> spark/connect/catalog.proto ->
spark/connect/base.proto
spark/connect/catalog.proto:22:1: Import "spark/connect/base.proto" was not
found or had errors.
spark/connect/catalog.proto:144:12: "spark.connect.DataType" seems to be
defined in "spark/connect/types.proto", which is not imported by
"spark/connect/catalog.proto". To use it here, please add the necessary import.
spark/connect/catalog.proto:161:12: "spark.connect.DataType" seems to be
defined in "spark/connect/types.proto", which is not imported by
"spark/connect/catalog.proto". To use it here, please add the necessary import.
spark/connect/relations.proto:25:1: Import "spark/connect/catalog.proto" was
not found or had errors.
spark/connect/relations.proto:84:5: "Catalog" is not defined.
spark/connect/commands.proto:22:1: Import "spark/connect/relations.proto" was
not found or had errors.
spark/connect/commands.proto:63:3: "Relation" is not defined.
spark/connect/commands.proto:81:3: "Relation" is not defined.
spark/connect/commands.proto:142:3: "Relation" is not defined.
spark/connect/base.proto:23:1: Import "spark/connect/commands.proto" was not
found or had errors.
spark/connect/base.proto:25:1: Import "spark/connect/relations.proto" was not
found or had errors.
.... {code}
So we should move `message StorageLevel` from `base.proto` to a separate file
to avoid this issue
was:
https://github.com/apache/spark/pull/40510 introduce `message StorageLevel` to
`base.proto`, but if we try to import `base.proto` in `catalog.proto` to reuse
`StorageLevel` in `message CacheTable` and run `build/sbt
"connect-common/compile" to compile, there will be following message in compile
log:
{code:java}
spark/connect/base.proto:23:1: File recursively imports itself:
spark/connect/base.proto -> spark/connect/commands.proto ->
spark/connect/relations.proto -> spark/connect/catalog.proto ->
spark/connect/base.proto
spark/connect/catalog.proto:22:1: Import "spark/connect/base.proto" was not
found or had errors.
spark/connect/catalog.proto:144:12: "spark.connect.DataType" seems to be
defined in "spark/connect/types.proto", which is not imported by
"spark/connect/catalog.proto". To use it here, please add the necessary import.
spark/connect/catalog.proto:161:12: "spark.connect.DataType" seems to be
defined in "spark/connect/types.proto", which is not imported by
"spark/connect/catalog.proto". To use it here, please add the necessary import.
spark/connect/relations.proto:25:1: Import "spark/connect/catalog.proto" was
not found or had errors.
spark/connect/relations.proto:84:5: "Catalog" is not defined.
spark/connect/commands.proto:22:1: Import "spark/connect/relations.proto" was
not found or had errors.
spark/connect/commands.proto:63:3: "Relation" is not defined.
spark/connect/commands.proto:81:3: "Relation" is not defined.
spark/connect/commands.proto:142:3: "Relation" is not defined.
spark/connect/base.proto:23:1: Import "spark/connect/commands.proto" was not
found or had errors.
spark/connect/base.proto:25:1: Import "spark/connect/relations.proto" was not
found or had errors.
.... {code}
So we should move `message StorageLevel` from a to a separate file to avoid
this issue
> Should move message StorageLevel from into a separate file
> ----------------------------------------------------------
>
> Key: SPARK-42901
> URL: https://issues.apache.org/jira/browse/SPARK-42901
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: 3.4.0
> Reporter: Yang Jie
> Priority: Major
>
> [https://github.com/apache/spark/pull/40510] introduce `message StorageLevel`
> to `base.proto`, but if we try to import `base.proto` in `catalog.proto` to
> reuse `StorageLevel` in `message CacheTable` and run `build/sbt
> "connect-common/compile" to compile, there will be following message in
> compile log:
>
> {code:java}
> spark/connect/base.proto:23:1: File recursively imports itself:
> spark/connect/base.proto -> spark/connect/commands.proto ->
> spark/connect/relations.proto -> spark/connect/catalog.proto ->
> spark/connect/base.proto
> spark/connect/catalog.proto:22:1: Import "spark/connect/base.proto" was not
> found or had errors.
> spark/connect/catalog.proto:144:12: "spark.connect.DataType" seems to be
> defined in "spark/connect/types.proto", which is not imported by
> "spark/connect/catalog.proto". To use it here, please add the necessary
> import.
> spark/connect/catalog.proto:161:12: "spark.connect.DataType" seems to be
> defined in "spark/connect/types.proto", which is not imported by
> "spark/connect/catalog.proto". To use it here, please add the necessary
> import.
> spark/connect/relations.proto:25:1: Import "spark/connect/catalog.proto" was
> not found or had errors.
> spark/connect/relations.proto:84:5: "Catalog" is not defined.
> spark/connect/commands.proto:22:1: Import "spark/connect/relations.proto" was
> not found or had errors.
> spark/connect/commands.proto:63:3: "Relation" is not defined.
> spark/connect/commands.proto:81:3: "Relation" is not defined.
> spark/connect/commands.proto:142:3: "Relation" is not defined.
> spark/connect/base.proto:23:1: Import "spark/connect/commands.proto" was not
> found or had errors.
> spark/connect/base.proto:25:1: Import "spark/connect/relations.proto" was not
> found or had errors.
> .... {code}
> So we should move `message StorageLevel` from `base.proto` to a separate file
> to avoid this issue
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
