Re: Spatial join performances

2021-08-05 Thread Jia Yu
Hi Pietro, As you see from our conversation, for the time being, you can disable Spark Adaptive Query processing by "spark.sql.adaptive.enabled=false". I believe this will fix this issue. Adam and I will dive deep in this issue and fix this bug. Thanks, Jia On Thu, Aug 5, 2021 at 3:10 PM Adam

Re: Spatial join performances

2021-08-05 Thread Adam Binford
I don't think that's the issue. The join detection is the same for both broadcast and non-broadcast, so the same match statement needs to run either way. I created an issue for what I found from the stack trace (don't have a copy of the stack trace to share easily): https://issues.apache.org/jira/b

Re: Inquiry for the ShapefileReader.readToGeometryRDD cannot successfully express the features in the sharefile

2021-08-05 Thread Jia Yu
Hi Miles, What is your shape file structure? Did you follow the structure as described here? https://sedona.apache.org/tutorial/rdd/#create-a-generic-spatialrdd In addition, Sedona shapefile reader has a small issue. If you put files other than dbf shx shp, it may not read data properly. See http

[jira] [Closed] (SEDONA-38) : java.lang.NoClassDefFoundError: org/opengis/referencing/FactoryException

2021-08-05 Thread Jia Yu (Jira)
[ https://issues.apache.org/jira/browse/SEDONA-38?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Yu closed SEDONA-38. Resolution: Not A Bug > : java.lang.NoClassDefFoundError: org/opengis/referencing/FactoryException > ---

[jira] [Created] (SEDONA-56) Broadcast join doesn't work with AQE enabled

2021-08-05 Thread Adam Binford (Jira)
Adam Binford created SEDONA-56: -- Summary: Broadcast join doesn't work with AQE enabled Key: SEDONA-56 URL: https://issues.apache.org/jira/browse/SEDONA-56 Project: Apache Sedona Issue Type: Bug