Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
xumingming closed issue #5044: [VL] spark.read.csv("/tmp/test.csv") throws
Exception
URL: https://github.com/apache/incubator-gluten/issues/5044
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
xumingming commented on issue #5044: URL: https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2024637533 @PHILO-HE Thanks for the information, closing it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
PHILO-HE commented on issue #5044: URL: https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2022823432 > @PHILO-HE Thanks for the information! I tried with parquet data(nation table in TPCH), the details are the following: > > ``` > == Fallback Summary == > (4) Project: Not supported to map spark function name to substrait function name: toprettystring(n_nationkey#23, Some(Asia/Shanghai)), class name: ToPrettyString. > (5) CollectLimit: Gluten does not touch it or does not support it > > == Physical Plan == > CollectLimit (5) > +- Project (4) >+- VeloxColumnarToRowExec (3) > +- ^ Scan parquet (1) > ``` > > Is the fallback for `Project` expected? Yes, because `toprettystring` is not supported in Gluten. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
xumingming commented on issue #5044: URL: https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2020579428 @PHILO-HE Thanks for the information! I tried with parquet data(nation table in TPCH), the details are the following: ``` == Fallback Summary == (4) Project: Not supported to map spark function name to substrait function name: toprettystring(n_nationkey#23, Some(Asia/Shanghai)), class name: ToPrettyString. (5) CollectLimit: Gluten does not touch it or does not support it == Physical Plan == CollectLimit (5) +- Project (4) +- VeloxColumnarToRowExec (3) +- ^ Scan parquet (1) ``` Is the fallback for `Project` expected? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
PHILO-HE commented on issue #5044: URL: https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2019393809 Hi @xumingming, the exception looks irrelevant. Currently, Gluten doesn't support CSV format. So reading CSV will fall back to vanilla Spark. You can check the fallback summary in `Gluten SQL / DataFrame` tab of history server UI. https://github.com/apache/incubator-gluten/assets/13806761/9b828b7d-7611-4c77-b4c1-f4e7e054ff76;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
xumingming commented on issue #5044:
URL:
https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2014638253
Full log:
```
scala> val path = "/tmp/person.csv"
path: String = /tmp/person.csv
scala> val person = spark.read.csv(path);
E20240322 16:52:40.086603 25592971 Exceptions.h:69] Line:
/Users/abei/Code/gluten/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:1811,
Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode:
INVALID_STATE
person: org.apache.spark.sql.DataFrame = [_c0: string, _c1: string ... 1
more field]
scala> person.registerTempTable("person");
warning: one deprecation (since 2.0.0); for details, enable `:setting
-deprecation' or `:replay -deprecation'
scala> spark.sql("select count(*) from person limit 10").show();
++
|count(1)|
++
| 2|
++
```
If the log is expected, I will close this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
xumingming commented on issue #5044: URL: https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2014061708 @zhztheplayer The result is correct, just a simple select. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [VL] spark.read.csv("/tmp/test.csv") throws Exception [incubator-gluten]
zhztheplayer commented on issue #5044: URL: https://github.com/apache/incubator-gluten/issues/5044#issuecomment-2011655932 Was the query result correct? Sometimes this log doesn't mean there is anything wrong. If it's the case we can find a way to suppress that log from Velox. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
