[jira] [Created] (SEDONA-224) java.lang.NoSuchMethodError when loading GeoParquet files using Spark 3.0.x ~ 3.2.x
Kristin Cowalcijk created SEDONA-224:
Summary: java.lang.NoSuchMethodError when loading GeoParquet files
using Spark 3.0.x ~ 3.2.x
Key: SEDONA-224
URL: https://issues.apache.org/jira/browse/SEDONA-224
Project: Apache Sedona
Issue Type: Bug
Reporter: Kristin Cowalcijk
{{spark.read.format("geoparquet").load("/path/to/geoparquet.parquet")}} does
not work on Spark 3.0.x ~ 3.2.x, it raises an {{java.lang.NoSuchMethodError}}:
{code:scala}
spark.read.format("geoparquet").load("/path/to/example1.parquet")
22/12/29 15:53:44 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
java.lang.NoSuchMethodError:
org.apache.spark.sql.execution.datasources.parquet.ParquetToSparkSchemaConverter$.$lessinit$greater$default$3()Z
at
org.apache.spark.sql.execution.datasources.parquet.GeoParquetToSparkSchemaConverter.(GeoParquetSchemaConverter.scala:48)
at
org.apache.spark.sql.execution.datasources.parquet.GeoParquetFileFormat$.$anonfun$mergeSchemasInParallel$1(GeoParquetFileFormat.scala:265)
at
org.apache.spark.sql.execution.datasources.parquet.GeoParquetFileFormat$.$anonfun$mergeSchemasInParallel$1$adapted(GeoParquetFileFormat.scala:261)
at
org.apache.spark.sql.execution.datasources.parquet.GeoSchemaMergeUtils$.$anonfun$mergeSchemasInParallel$2(GeoSchemaMergeUtils.scala:69)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:863)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:863)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:131)
at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (SEDONA-223) Add ST_Split
[ https://issues.apache.org/jira/browse/SEDONA-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17652698#comment-17652698 ] Jia Yu commented on SEDONA-223: --- [~dougdennis] Assigned :) > Add ST_Split > > > Key: SEDONA-223 > URL: https://issues.apache.org/jira/browse/SEDONA-223 > Project: Apache Sedona > Issue Type: Improvement >Reporter: Doug Dennis >Assignee: Doug Dennis >Priority: Major > > Add the ST_Split function supporting splitting linestrings by (multi)points, > (multi)linestrings, and (multi)polygons as well as (multi)polygons by > (multi)linestrings. I did not see an exact match for this in JTS. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (SEDONA-223) Add ST_Split
[ https://issues.apache.org/jira/browse/SEDONA-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17652601#comment-17652601 ] Doug Dennis commented on SEDONA-223: I'd like to take a crack at this one. > Add ST_Split > > > Key: SEDONA-223 > URL: https://issues.apache.org/jira/browse/SEDONA-223 > Project: Apache Sedona > Issue Type: Improvement >Reporter: Doug Dennis >Priority: Major > > Add the ST_Split function supporting splitting linestrings by (multi)points, > (multi)linestrings, and (multi)polygons as well as (multi)polygons by > (multi)linestrings. I did not see an exact match for this in JTS. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (SEDONA-223) Add ST_Split
Doug Dennis created SEDONA-223: -- Summary: Add ST_Split Key: SEDONA-223 URL: https://issues.apache.org/jira/browse/SEDONA-223 Project: Apache Sedona Issue Type: Improvement Reporter: Doug Dennis Add the ST_Split function supporting splitting linestrings by (multi)points, (multi)linestrings, and (multi)polygons as well as (multi)polygons by (multi)linestrings. I did not see an exact match for this in JTS. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [SEDONA-212] Rework dependency management
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1366826375 > @Kimahriman I have updated the example projects. If you pull the latest change, it will pass the CI. > > I suppose you have read: https://sedona.apache.org/1.3.1-incubating/community/release-manager/ > > Now you can try to publish the snapshots: https://sedona.apache.org/1.3.1-incubating/community/release-manager/ > > Once you publish the snapshots, you can try them in the example projects by using sedona-1.3.2-incubating-SNAPSHOT as the version. Published 1.3.2-incubating-SNAPSHOT to the snapshot repo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #735: [SEDONA-212] Rework dependency management
Kimahriman commented on code in PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#discussion_r1058306335 ## .github/workflows/java.yml: ## @@ -3,7 +3,7 @@ name: Scala and Java build on: push: branches: - - master Review Comment: I always change this locally so that I can get the CI to run before making a PR, will change this back. Does that only apply to things running under the apache group? When it runs on my own GitHub actions does it use the same vms? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-sedona] Kontinuation commented on pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance
Kontinuation commented on PR #739: URL: https://github.com/apache/incubator-sedona/pull/739#issuecomment-1366459294 > @Kontinuation I just switched all versions to 1.3.1-incubating. If you pull the latest change, you should be fine. > > A quick question: does this solve [SEDONA-222](https://issues.apache.org/jira/browse/SEDONA-222)? It does not resolve SEDONA-222. I'm working on another branch to resolve issues related to GeoParquet, and this issue will be resolved in an upcoming PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on pull request #735: [SEDONA-212] Rework dependency management
jiayuasu commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1366450486 @Kimahriman I have updated the example projects. If you pull the latest change, it will pass the CI. I suppose you have read: https://sedona.apache.org/1.3.1-incubating/community/release-manager/ Now you can try to publish the snapshots: https://sedona.apache.org/1.3.1-incubating/community/release-manager/ Once you publish the snapshots, you can try them in the example projects by using sedona-1.3.2-incubating-SNAPSHOT as the version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #735: [SEDONA-212] Rework dependency management
jiayuasu commented on code in PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#discussion_r1058131076 ## .github/workflows/java.yml: ## @@ -3,7 +3,7 @@ name: Scala and Java build on: push: branches: - - master Review Comment: Please don't change it to `*`. CI is supposed to only test the master branch and pull requests because all repos under ASF share 150 VMs sponsored by GitHub and we want to leave some resources to other projects. ## .github/workflows/python.yml: ## @@ -3,7 +3,7 @@ name: Python build on: push: branches: - - master Review Comment: Same here. Use `master` please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@sedona.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
