[GitHub] [sedona] dependabot[bot] opened a new pull request, #822: Bump spark-core_2.12 from 3.3.0 to 3.4.0

2023-04-21 Thread via GitHub
dependabot[bot] opened a new pull request, #822: URL: https://github.com/apache/sedona/pull/822 Bumps spark-core_2.12 from 3.3.0 to 3.4.0. [![Dependabot compatibility

RE: Sedona workers stuck in loop: DynamicIndexLookupJudgement: [xx, PID=xx] [Streaming shapes] Reached a milestone: xxxxxxx

2023-04-21 Thread Zhang, Hanxi (ISED/ISDE)
Hi Jia, Thank you very much for your suggestion. I tried to repartition both input datasets df1 and df2 and unfortunately it did not help; my cluster was always stuck at the last 3 tasks out of 100-200 when doing: SELECT * FROM df1, df2 WHERE ST_Contains(df1.geom, df2.geom) However,

Re: Sprint Goals for Apache projects next week

2023-04-21 Thread Jia Yu
Hi Joana, *I agree with the timeslot. Here is an intro of my tutorial:* Integrate Apache SIS with Apache Sedona – CRS Transformation, GeoTiff Reader and more Apache Sedona is a high-performance cluster computing system designed specifically for processing large-scale spatial data. The system

[jira] [Commented] (SEDONA-276) Add support for Spark 3.4

2023-04-21 Thread Martin Andersson (Jira)
[ https://issues.apache.org/jira/browse/SEDONA-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715026#comment-17715026 ] Martin Andersson commented on SEDONA-276: - Thank you all! It looks like adding support for Spark

[jira] [Commented] (SEDONA-276) Add support for Spark 3.4

2023-04-21 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SEDONA-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714951#comment-17714951 ] Adam Binford commented on SEDONA-276: - [~umartin] Oh that's not how I took it! Yeah that's pretty

[jira] [Commented] (SEDONA-276) Add support for Spark 3.4

2023-04-21 Thread Martin Andersson (Jira)
[ https://issues.apache.org/jira/browse/SEDONA-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714935#comment-17714935 ] Martin Andersson commented on SEDONA-276: - [~kimahriman] That wasn't meant as criticism, sorry!

[GitHub] [sedona] umartin commented on issue #821: How to define geometry data types such as polygon and point Spark SQL when using sedona

2023-04-21 Thread via GitHub
umartin commented on issue #821: URL: https://github.com/apache/sedona/issues/821#issuecomment-1517341461 I glad the reads are working. If you want to write geometries to Postgis you need to convert them to EWKB format first. Using the DataFrame API you could do something like this:

[GitHub] [sedona] cgqfzy commented on issue #821: How to define geometry data types such as polygon and point Spark SQL when using sedona

2023-04-21 Thread via GitHub
cgqfzy commented on issue #821: URL: https://github.com/apache/sedona/issues/821#issuecomment-1517318371 > Spark doesn't recognize the geometry type in Postgis. It will encode the geometry in EWKB format. When I tested on my machine it was even hex encoded. > > Does it work if you