Re: Issue Upgrading to 3.2

2021-11-01 Thread Adam Binford
Sorry yeah good question. It happens on the call to spark.table("pixels") On Mon, Nov 1, 2021 at 12:36 PM Wenchen Fan wrote: > To confirm: Does the error happen during view creation, or when we read > the view later? > > On Mon, Nov 1, 2021 at 11:28 PM Adam Binford wrote: > >> I don't have a

Re: Issue Upgrading to 3.2

2021-11-01 Thread Wenchen Fan
To confirm: Does the error happen during view creation, or when we read the view later? On Mon, Nov 1, 2021 at 11:28 PM Adam Binford wrote: > I don't have a minimal reproduction right now but here's more relevant > code snippets. > > Stacktrace: > org.apache.spark.sql.AnalysisException:

Re: Issue Upgrading to 3.2

2021-11-01 Thread Adam Binford
I don't have a minimal reproduction right now but here's more relevant code snippets. Stacktrace: org.apache.spark.sql.AnalysisException: Undefined function: 'ST_PolygonFromEnvelope'. This function is neither a registered temporary function nor a permanent function registered in the database

Re: [DISCUSS] SPIP: Row-level operations in Data Source V2

2021-11-01 Thread Wenchen Fan
The general idea looks great. This is indeed a complicated API and we probably need more time to evaluate the API design. It's better to commit this work earlier so that we have more time to verify it before the 3.3 release. Maybe we can commit the group-based API first, then the delta-based one,

Re: Issue Upgrading to 3.2

2021-11-01 Thread Wenchen Fan
Hi Adam, Thanks for reporting this issue! Do you have the full stacktrace or a code snippet to reproduce the issue at Spark side? It looks like a bug, but it's not obvious to me how this bug can happen. Thanks, Wenchen On Sat, Oct 30, 2021 at 1:08 AM Adam Binford wrote: > Hi devs, > > I'm