Zouxxyy commented on code in PR #10900:
URL:
https://github.com/apache/incubator-gluten/pull/10900#discussion_r2447516812
##########
gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/IcebergWriteUtil.scala:
##########
@@ -27,8 +28,8 @@ import org.apache.iceberg.types.Types.{ListType, MapType}
object IcebergWriteUtil {
- def isDataWrite(write: Write): Boolean = {
- write.isInstanceOf[SparkWrite]
+ def supportsWrite(write: Write): Boolean = {
+ SparkReflectionUtil.isInstanceOfClassName(write,
"org.apache.iceberg.spark.source.SparkWrite")
Review Comment:
I think option 1 is more reasonable. It seems we need to add a method to the
component to check whether the environment supports it, and then we can remove
all the existing reflection-based judge logic.
The main purpose of this PR is to fix the issue where, after offload to
Iceberg but verification failed, can't back to other format. I'd prefer to
propel this PR first, and then proceed with the refactoring proposed in 1. What
do you all think? @PHILO-HE @zhztheplayer
As for 2, the reason don't use name-based matching is that
`org.apache.iceberg.spark.source.SparkWrite` is an abstract class and is
implemented using anonymous classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
