[GitHub] [sedona] jiayuasu merged pull request #749: [SEDONA-221] Outer join throws NPE for null geometries.

2023-01-20 Thread GitBox
jiayuasu merged PR #749: URL: https://github.com/apache/sedona/pull/749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] jiayuasu merged pull request #748: [SEDONA-233] Incorrect results for several joins in a single stage

2023-01-20 Thread GitBox
jiayuasu merged PR #748: URL: https://github.com/apache/sedona/pull/748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] umartin opened a new pull request, #749: [SEDONA-221] Outer join throws NPE for null geometries.

2023-01-20 Thread GitBox
umartin opened a new pull request, #749: URL: https://github.com/apache/sedona/pull/749 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [sedona] umartin commented on a diff in pull request #748: [SEDONA-233] Incorrect results for several joins in a single stage

2023-01-19 Thread GitBox
umartin commented on code in PR #748: URL: https://github.com/apache/sedona/pull/748#discussion_r1081312568 ## core/src/main/java/org/apache/sedona/core/joinJudgement/DuplicatesFilter.java: ## @@ -0,0 +1,48 @@ +package org.apache.sedona.core.joinJudgement; + Review Comment:

[GitHub] [sedona] jiayuasu commented on a diff in pull request #748: [SEDONA-233] Incorrect results for several joins in a single stage

2023-01-18 Thread GitBox
jiayuasu commented on code in PR #748: URL: https://github.com/apache/sedona/pull/748#discussion_r1080791389 ## core/src/test/java/org/apache/sedona/core/spatialOperator/JoinQueryDeduplicationTest.java: ## @@ -0,0 +1,75 @@ +/* Review Comment: Could you please add another

[GitHub] [sedona] umartin opened a new pull request, #748: [SEDONA-233] Incorrect results for several joins in a single stage

2023-01-18 Thread GitBox
umartin opened a new pull request, #748: URL: https://github.com/apache/sedona/pull/748 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [sedona] jiayuasu merged pull request #747: [SEDONA-232] Upgrade CICD From Spark 3.2.0 to 3.2.3

2023-01-13 Thread GitBox
jiayuasu merged PR #747: URL: https://github.com/apache/sedona/pull/747 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] douglasdennis opened a new pull request, #747: [SEDONA-232] Upgrade CICD From Spark 3.2.0 to 3.2.3

2023-01-13 Thread GitBox
douglasdennis opened a new pull request, #747: URL: https://github.com/apache/sedona/pull/747 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [sedona] jiayuasu merged pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-13 Thread GitBox
jiayuasu merged PR #745: URL: https://github.com/apache/sedona/pull/745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] douglasdennis commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-12 Thread GitBox
douglasdennis commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1381198538 @jiayuasu I feel good about this PR. If the others do as well then I think it can merge. This code will probably have to be returned to at a later date once shapely decides how it

[GitHub] [sedona] douglasdennis commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-12 Thread GitBox
douglasdennis commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1381195068 @Imbruced Took me a little longer but here are some memory profiling results. TL;DR Serialization Average Memory Increment Results (MiB):

[GitHub] [sedona] jiayuasu commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-12 Thread GitBox
jiayuasu commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1381141376 @douglasdennis Is this PR ready to be merged? :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [sedona] douglasdennis commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
douglasdennis commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1374679892 > Do you have a chance to compare the memory footprint between wkb.loads approach and the changes after 29.12 and the current version ? I'd love too. Unfortunately, it won't be

[GitHub] [sedona] douglasdennis commented on a diff in pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
douglasdennis commented on code in PR #745: URL: https://github.com/apache/sedona/pull/745#discussion_r1064067552 ## python/sedona/utils/geometry_serde.py: ## @@ -217,70 +233,97 @@ def create_buffer_for_geom(geom_type: int, coord_type: int, size: int, num_coord

[GitHub] [sedona] douglasdennis commented on a diff in pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
douglasdennis commented on code in PR #745: URL: https://github.com/apache/sedona/pull/745#discussion_r1064067248 ## python/sedona/utils/geometry_serde.py: ## @@ -217,70 +233,97 @@ def create_buffer_for_geom(geom_type: int, coord_type: int, size: int, num_coord

[GitHub] [sedona] Imbruced commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
Imbruced commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1374664244 Do you have a chance to compare the memory footprint between wkb.loads approach and the changes after 29.12 and the current version ? -- This is an automated message from the Apache Git

[GitHub] [sedona] Imbruced commented on a diff in pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
Imbruced commented on code in PR #745: URL: https://github.com/apache/sedona/pull/745#discussion_r1064062557 ## python/sedona/utils/geometry_serde.py: ## @@ -217,70 +233,97 @@ def create_buffer_for_geom(geom_type: int, coord_type: int, size: int, num_coord

[GitHub] [sedona] Imbruced commented on a diff in pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
Imbruced commented on code in PR #745: URL: https://github.com/apache/sedona/pull/745#discussion_r1064062506 ## python/sedona/utils/geometry_serde.py: ## @@ -15,22 +15,32 @@ # specific language governing permissions and limitations # under the License. +import array

[GitHub] [sedona] Imbruced commented on a diff in pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
Imbruced commented on code in PR #745: URL: https://github.com/apache/sedona/pull/745#discussion_r1064062431 ## python/sedona/utils/geometry_serde.py: ## @@ -217,70 +233,97 @@ def create_buffer_for_geom(geom_type: int, coord_type: int, size: int, num_coord

[GitHub] [sedona] Imbruced commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-07 Thread GitBox
Imbruced commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1374663015 Is the shapely section refering to wkb.loads and wkb.dumps methods ? What do you think about letting user to chose if the Z/M support should be turned on ? (I guess that majority of use

[GitHub] [sedona] jiayuasu merged pull request #744: [SEDONA-156] Support spatial filter push-down for GeoParquetFileFormat

2023-01-07 Thread GitBox
jiayuasu merged PR #744: URL: https://github.com/apache/sedona/pull/744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] jiayuasu merged pull request #746: [SEDONA-230] rdd.saveAsGeoJSON should generate feature properties with field names

2023-01-06 Thread GitBox
jiayuasu merged PR #746: URL: https://github.com/apache/sedona/pull/746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] douglasdennis commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-06 Thread GitBox
douglasdennis commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1373879622 I stand corrected :) I should have been a little more thorough. I think empty geometries in collections should be supported then, since the standards expressly provide for them. --

[GitHub] [sedona] Kontinuation commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-06 Thread GitBox
Kontinuation commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1373869602 > @Kontinuation What's your take on dropping support for empty geometry inside of multi-geometries in the new serialization format? They don't have representation in any other form so

[GitHub] [sedona] douglasdennis commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-06 Thread GitBox
douglasdennis commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1373758835 @Kontinuation What's your take on dropping support for empty geometry inside of multi-geometries in the new serialization format? They don't have representation in any other form so

[GitHub] [sedona] Kontinuation commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-06 Thread GitBox
Kontinuation commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1373383865 LGTM. The refactored implementation looks much more pythonic while having better performance. It is also a great move to remove the expensive `has_z` calls, as shapely currently does

[GitHub] [sedona] jiayuasu opened a new pull request, #746: [SEDONA-230] rdd.saveAsGeoJSON should generate feature properties with field names

2023-01-05 Thread GitBox
jiayuasu opened a new pull request, #746: URL: https://github.com/apache/sedona/pull/746 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [sedona] jiayuasu merged pull request #742: [SEDONA-223] Add ST_Split

2023-01-05 Thread GitBox
jiayuasu merged PR #742: URL: https://github.com/apache/sedona/pull/742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] jiayuasu commented on pull request #742: [SEDONA-223] Add ST_Split

2023-01-05 Thread GitBox
jiayuasu commented on PR #742: URL: https://github.com/apache/sedona/pull/742#issuecomment-1373023036 Awesome. Will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [sedona] Kimahriman commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-05 Thread GitBox
Kimahriman commented on code in PR #742: URL: https://github.com/apache/sedona/pull/742#discussion_r1062985739 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,331 @@ +/** + * Licensed under the Apache License, Version 2.0 (the

[GitHub] [sedona] neontty commented on pull request #744: [SEDONA-156] Support spatial filter push-down for GeoParquetFileFormat

2023-01-05 Thread GitBox
neontty commented on PR #744: URL: https://github.com/apache/sedona/pull/744#issuecomment-1372809028 @Kontinuation your code is so clean! Thank you for putting in this work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [sedona] jiayuasu commented on pull request #745: [SEDONA-227] Python Serde Refactor

2023-01-05 Thread GitBox
jiayuasu commented on PR #745: URL: https://github.com/apache/sedona/pull/745#issuecomment-1372700871 @Kontinuation @Imbruced Please feel free to chime in with comments :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [sedona] jiayuasu commented on pull request #743: [SEDONA-228] Standardize logging modules

2023-01-05 Thread GitBox
jiayuasu commented on PR #743: URL: https://github.com/apache/sedona/pull/743#issuecomment-1372632857 Great explanation! Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [sedona] douglasdennis commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-05 Thread GitBox
douglasdennis commented on code in PR #742: URL: https://github.com/apache/sedona/pull/742#discussion_r1062589421 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,331 @@ +/** + * Licensed under the Apache License, Version 2.0 (the

[GitHub] [sedona] Kimahriman commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-05 Thread GitBox
Kimahriman commented on code in PR #742: URL: https://github.com/apache/sedona/pull/742#discussion_r1062394602 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,331 @@ +/** + * Licensed under the Apache License, Version 2.0 (the

[GitHub] [sedona] Kimahriman commented on pull request #743: [SEDONA-228] Standardize logging modules

2023-01-05 Thread GitBox
Kimahriman commented on PR #743: URL: https://github.com/apache/sedona/pull/743#issuecomment-1372116194 Do you mean for tests or for real use? For tests the logs should all go to `target/unit-tests.log`. Copied this from Spark so the stdout/err of the tests is less verbose and mostly

[GitHub] [sedona] douglasdennis opened a new pull request, #745: [SEDONA-227] Python Serde Refactor

2023-01-05 Thread GitBox
douglasdennis opened a new pull request, #745: URL: https://github.com/apache/sedona/pull/745 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [sedona] douglasdennis commented on pull request #742: [SEDONA-223] Add ST_Split

2023-01-05 Thread GitBox
douglasdennis commented on PR #742: URL: https://github.com/apache/sedona/pull/742#issuecomment-1371906517 @jiayuasu Added it in. Let me know if I was supposed to do it some other way. The output I see looks like this: `23/01/04 23:52:03 ERROR GeometrySplitter: Colinear sections

[GitHub] [sedona] Kontinuation commented on pull request #744: [SEDONA-156] Support spatial filter push-down for GeoParquetFileFormat

2023-01-04 Thread GitBox
Kontinuation commented on PR #744: URL: https://github.com/apache/sedona/pull/744#issuecomment-1371889883 [1], [2] was already handled in [PR 740](https://github.com/apache/sedona/pull/740/files#diff-5c93229890e07bc39a57465e812db752a82cd23ab68c82ee34669acb7aeee2c6). I'll add a

[GitHub] [sedona] jiayuasu commented on pull request #744: [SEDONA-156] Support spatial filter push-down for GeoParquetFileFormat

2023-01-04 Thread GitBox
jiayuasu commented on PR #744: URL: https://github.com/apache/sedona/pull/744#issuecomment-1371875015 @Kontinuation This looks good to me! Thank you! Can you please add some documentation? 1. Update GeoParquet read / write in [1] and [2] since the users no longer need to specify

[GitHub] [sedona] jiayuasu commented on pull request #742: [SEDONA-223] Add ST_Split

2023-01-04 Thread GitBox
jiayuasu commented on PR #742: URL: https://github.com/apache/sedona/pull/742#issuecomment-1371827972 @douglasdennis PR https://github.com/apache/sedona/pull/743 has been merged. If you pull from the upstream, you can actually now use log.error to handle the issue. Adam might need to make

[GitHub] [sedona] jiayuasu commented on pull request #743: [SEDONA-228] Standardize logging modules

2023-01-04 Thread GitBox
jiayuasu commented on PR #743: URL: https://github.com/apache/sedona/pull/743#issuecomment-1371827270 @Kimahriman One thing I just realized: We probably should just print logging to the console (stdout and stderr) directly. Otherwise the users won't be able to see the logging unless they

[GitHub] [sedona] jiayuasu merged pull request #743: [SEDONA-228] Standardize logging modules

2023-01-04 Thread GitBox
jiayuasu merged PR #743: URL: https://github.com/apache/sedona/pull/743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [sedona] jiayuasu commented on pull request #743: [SEDONA-228] Standardize logging modules

2023-01-04 Thread GitBox
jiayuasu commented on PR #743: URL: https://github.com/apache/sedona/pull/743#issuecomment-1371814275 @Kimahriman Thank you for your great explanation! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [sedona] Kontinuation opened a new pull request, #744: [SEDONA-156] Support spatial filter push-down for GeoParquetFileFormat

2023-01-04 Thread GitBox
Kontinuation opened a new pull request, #744: URL: https://github.com/apache/sedona/pull/744 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [incubator-sedona] Kimahriman commented on pull request #743: [SEDONA-228] Standardize logging modules

2023-01-03 Thread GitBox
Kimahriman commented on PR #743: URL: https://github.com/apache/incubator-sedona/pull/743#issuecomment-1370424624 > @Kimahriman I don't think it is a good idea to include logging dependencies in Sedona. Since Spark and Flink already package their own versions of log4j or slf4j, including

[GitHub] [incubator-sedona] jiayuasu commented on pull request #743: [SEDONA-228] Standardize logging modules

2023-01-02 Thread GitBox
jiayuasu commented on PR #743: URL: https://github.com/apache/incubator-sedona/pull/743#issuecomment-1369390537 @Kimahriman I don't think it is a good idea to include logging dependencies in Sedona. Since Spark and Flink already package their own versions of log4j or slf4j, including

[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
Kimahriman commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060183246 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
Kimahriman commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060179876 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] douglasdennis commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
douglasdennis commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060178239 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; Review

[GitHub] [incubator-sedona] douglasdennis commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
douglasdennis commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060177619 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] Kimahriman opened a new pull request, #743: [SEDONA-228] Standardize logging modules

2023-01-02 Thread GitBox
Kimahriman opened a new pull request, #743: URL: https://github.com/apache/incubator-sedona/pull/743 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
jiayuasu commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060159401 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
jiayuasu commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060159401 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
Kimahriman commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060157902 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
jiayuasu commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060157586 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; Review Comment:

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-02 Thread GitBox
jiayuasu commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1060157432 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] jiayuasu merged pull request #735: [SEDONA-212] Rework dependency management

2023-01-01 Thread GitBox
jiayuasu merged PR #735: URL: https://github.com/apache/incubator-sedona/pull/735 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #735: [SEDONA-212] Rework dependency management

2023-01-01 Thread GitBox
Kimahriman commented on code in PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#discussion_r1059797432 ## pom.xml: ## @@ -59,178 +59,198 @@ github -24.0 UTF-8 -provided +1.8 +${java.version} +

[GitHub] [incubator-sedona] douglasdennis commented on pull request #735: [SEDONA-212] Rework dependency management

2023-01-01 Thread GitBox
douglasdennis commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1368525288 For what it's worth and from a user's perspective, I think this looks awesome and I love the idea of an explicitly named shaded jar over the python adapter one. I just noted

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #735: [SEDONA-212] Rework dependency management

2023-01-01 Thread GitBox
jiayuasu commented on code in PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#discussion_r1059789647 ## pom.xml: ## @@ -59,178 +59,198 @@ github -24.0 UTF-8 -provided +1.8 +${java.version} +

[GitHub] [incubator-sedona] douglasdennis commented on a diff in pull request #742: [SEDONA-223] Add ST_Split

2023-01-01 Thread GitBox
douglasdennis commented on code in PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#discussion_r1059726043 ## common/src/main/java/org/apache/sedona/common/utils/GeometrySplitter.java: ## @@ -0,0 +1,321 @@ +package org.apache.sedona.common.utils; + +import

[GitHub] [incubator-sedona] douglasdennis commented on pull request #742: [SEDONA-223] Add ST_Split

2023-01-01 Thread GitBox
douglasdennis commented on PR #742: URL: https://github.com/apache/incubator-sedona/pull/742#issuecomment-1368394310 Ugh. I noticed that my VS Code seems to have gone wild on deleting leading white spaces. I hope that is ok. If the edits seem too messy though then I can try to revert them.

[GitHub] [incubator-sedona] douglasdennis opened a new pull request, #742: [SEDONA-223] Add ST_Split

2023-01-01 Thread GitBox
douglasdennis opened a new pull request, #742: URL: https://github.com/apache/incubator-sedona/pull/742 ## Did you read the Contributor Guide? - Yes ## Is this PR related to a JIRA ticket? - Yes, the URL of the associated JIRA ticket is

[GitHub] [incubator-sedona] jiayuasu merged pull request #741: [SEDONA-207] Remove dependency to spark from UnsafeGeometryBuffer in sedona-common

2022-12-31 Thread GitBox
jiayuasu merged PR #741: URL: https://github.com/apache/incubator-sedona/pull/741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [incubator-sedona] Kimahriman commented on pull request #741: [SEDONA-207] Remove dependency to spark from UnsafeGeometryBuffer in sedona-common

2022-12-31 Thread GitBox
Kimahriman commented on PR #741: URL: https://github.com/apache/incubator-sedona/pull/741#issuecomment-1368315058 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [incubator-sedona] Kontinuation commented on a diff in pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-31 Thread GitBox
Kontinuation commented on code in PR #739: URL: https://github.com/apache/incubator-sedona/pull/739#discussion_r1059692396 ## common/src/main/java/org/apache/sedona/common/geometrySerde/UnsafeGeometryBuffer.java: ## @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache Software

[GitHub] [incubator-sedona] Kontinuation opened a new pull request, #741: [SEDONA-207] Remove dependency to spark from UnsafeGeometryBuffer in sedona-common

2022-12-31 Thread GitBox
Kontinuation opened a new pull request, #741: URL: https://github.com/apache/incubator-sedona/pull/741 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-31 Thread GitBox
Kimahriman commented on code in PR #739: URL: https://github.com/apache/incubator-sedona/pull/739#discussion_r1059687208 ## common/src/main/java/org/apache/sedona/common/geometrySerde/UnsafeGeometryBuffer.java: ## @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache Software

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-30 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1368014562 Had an old shaded jar in my spark jars folder still, examples are working off of the published snapshot versions now -- This is an automated message from the Apache Git

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-30 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1367905193 > @Kimahriman Did you try the SNAPSHOT in example projects? Yeah I've been trying but running into some weird things that may just be issues with my local environment,

[GitHub] [incubator-sedona] jiayuasu commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-29 Thread GitBox
jiayuasu commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1367707550 @Kimahriman Did you try the SNAPSHOT in example projects? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [incubator-sedona] jiayuasu merged pull request #740: [SEDONA-226] Support reading and writing GeoParquet file metadata

2022-12-29 Thread GitBox
jiayuasu merged PR #740: URL: https://github.com/apache/incubator-sedona/pull/740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [incubator-sedona] jiayuasu merged pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-29 Thread GitBox
jiayuasu merged PR #739: URL: https://github.com/apache/incubator-sedona/pull/739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [incubator-sedona] Kontinuation opened a new pull request, #740: [SEDONA-226] Support reading and writing GeoParquet file metadata

2022-12-29 Thread GitBox
Kontinuation opened a new pull request, #740: URL: https://github.com/apache/incubator-sedona/pull/740 ## Notice This patch is based on https://github.com/apache/incubator-sedona/pull/739, thus PR 739 should be merged prior to this patch. ## Did you read the Contributor Guide?

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-28 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1366826375 > @Kimahriman I have updated the example projects. If you pull the latest change, it will pass the CI. > > I suppose you have read:

[GitHub] [incubator-sedona] Kimahriman commented on a diff in pull request #735: [SEDONA-212] Rework dependency management

2022-12-28 Thread GitBox
Kimahriman commented on code in PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#discussion_r1058306335 ## .github/workflows/java.yml: ## @@ -3,7 +3,7 @@ name: Scala and Java build on: push: branches: - - master Review Comment: I always change

[GitHub] [incubator-sedona] Kontinuation commented on pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-28 Thread GitBox
Kontinuation commented on PR #739: URL: https://github.com/apache/incubator-sedona/pull/739#issuecomment-1366459294 > @Kontinuation I just switched all versions to 1.3.1-incubating. If you pull the latest change, you should be fine. > > A quick question: does this solve

[GitHub] [incubator-sedona] jiayuasu commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-28 Thread GitBox
jiayuasu commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1366450486 @Kimahriman I have updated the example projects. If you pull the latest change, it will pass the CI. I suppose you have read:

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #735: [SEDONA-212] Rework dependency management

2022-12-28 Thread GitBox
jiayuasu commented on code in PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#discussion_r1058131076 ## .github/workflows/java.yml: ## @@ -3,7 +3,7 @@ name: Scala and Java build on: push: branches: - - master Review Comment: Please don't

[GitHub] [incubator-sedona] jiayuasu commented on pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-27 Thread GitBox
jiayuasu commented on PR #739: URL: https://github.com/apache/incubator-sedona/pull/739#issuecomment-1366440465 @Kontinuation I just switched all versions to 1.3.1-incubating. If you pull the latest change, you should be fine. A quick question: does this solve

[GitHub] [incubator-sedona] Kontinuation commented on pull request #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-27 Thread GitBox
Kontinuation commented on PR #739: URL: https://github.com/apache/incubator-sedona/pull/739#issuecomment-1366337573 Example project build failed since the [snapshot artifact for Sedona

[GitHub] [incubator-sedona] Kontinuation opened a new pull request, #739: [SEDONA-207] Implemented a new geometry serde for supporting Z/M dimensions and better performance

2022-12-27 Thread GitBox
Kontinuation opened a new pull request, #739: URL: https://github.com/apache/incubator-sedona/pull/739 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-26 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1365251416 Couriser dependencies for each module now: common: ``` com.fasterxml.jackson.core:jackson-annotations:2.12.2:default

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [SEDONA-212] Rework dependency management

2022-12-26 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1365247851 I think I have this in a good place and builds working, except for the examples. It looks like those don't use any local builds of sedona to build the examples, and just relies

[GitHub] [incubator-sedona] jiayuasu merged pull request #738: [DOCS] Add the news about Sedona 1.3.1-incubating and fix a bunch of typos

2022-12-23 Thread GitBox
jiayuasu merged PR #738: URL: https://github.com/apache/incubator-sedona/pull/738 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [incubator-sedona] jiayuasu opened a new pull request, #738: [DOCS] Add the news about Sedona 1.3.1-incubating and fix a bunch of typos

2022-12-23 Thread GitBox
jiayuasu opened a new pull request, #738: URL: https://github.com/apache/incubator-sedona/pull/738 ## Did you read the Contributor Guide? - Yes, I have read [Contributor Rules](https://sedona.apache.org/community/rule/) and [Contributor Development

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-22 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1362871967 Got an initial version of the shaded modules working. Need to see if tests still pass (mostly python ones, using the shaded spark module to run those now). After seeing how

[GitHub] [incubator-sedona] jiayuasu commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-21 Thread GitBox
jiayuasu commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1362510105 @Kimahriman @umartin Thank you both for the great suggestions. @Kimahriman Let's proceed with your proposal. In addition, let's create two completely separate modules

[GitHub] [incubator-sedona] jiayuasu merged pull request #737: [SEDONA-215] Add ST_ConcaveHull

2022-12-21 Thread GitBox
jiayuasu merged PR #737: URL: https://github.com/apache/incubator-sedona/pull/737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [incubator-sedona] umartin commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-19 Thread GitBox
umartin commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1357700094 > This was mostly intentional, I tried to include everything that was directly imported by the package to be more explicit, and not rely on expecting transitive dependencies

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-19 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1357614169 > * Dependencies with compile scope are inherited. So python-adapter doesn't need to depend on sql, core and common. Only sql is needed. Same goes for the other modules. Maybe

[GitHub] [incubator-sedona] umartin commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-19 Thread GitBox
umartin commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1357488467 Nice work! Over all this looks good to me. Just a few notes: * Dependencies with compile scope are inherited. So python-adapter doesn't need to depend on sql, core and common.

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #737: [SEDONA-215] Add ST_ConcaveHull

2022-12-18 Thread GitBox
jiayuasu commented on code in PR #737: URL: https://github.com/apache/incubator-sedona/pull/737#discussion_r1051745774 ## docs/archive/api/sql/GeoSparkSQL-Function.md: ## @@ -12,6 +12,20 @@ SELECT ST_Distance(polygondf.countyshape, polygondf.countyshape) FROM polygondf ```

[GitHub] [incubator-sedona] Kimahriman commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-18 Thread GitBox
Kimahriman commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1356792821 Yeah I think that mostly covers it. We can keep a shaded jar for Spark and/or Flink for people who need it, and that can either be the python-adapter module or a new

[GitHub] [incubator-sedona] awadhesh14 commented on a diff in pull request #737: [SEDONA-215] Add ST_ConcaveHull

2022-12-17 Thread GitBox
awadhesh14 commented on code in PR #737: URL: https://github.com/apache/incubator-sedona/pull/737#discussion_r1051539051 ## python/tests/sql/test_function.py: ## @@ -50,6 +50,22 @@ class TestPredicateJoin(TestBase): ] ) +def test_st_concave_hull(self): +

[GitHub] [incubator-sedona] awadhesh14 commented on a diff in pull request #737: [SEDONA-215] Add ST_ConcaveHull

2022-12-17 Thread GitBox
awadhesh14 commented on code in PR #737: URL: https://github.com/apache/incubator-sedona/pull/737#discussion_r1051538611 ## docs/archive/api/sql/GeoSparkSQL-Function.md: ## @@ -12,6 +12,20 @@ SELECT ST_Distance(polygondf.countyshape, polygondf.countyshape) FROM polygondf ```

[GitHub] [incubator-sedona] awadhesh14 commented on a diff in pull request #737: [SEDONA-215] Add ST_ConcaveHull

2022-12-17 Thread GitBox
awadhesh14 commented on code in PR #737: URL: https://github.com/apache/incubator-sedona/pull/737#discussion_r1051538585 ## docs/api/sql/Function.md: ## @@ -332,6 +332,22 @@ Result: ++ ``` +##

[GitHub] [incubator-sedona] jiayuasu commented on pull request #735: [WIP][SEDONA-212] Rework dependency management

2022-12-17 Thread GitBox
jiayuasu commented on PR #735: URL: https://github.com/apache/incubator-sedona/pull/735#issuecomment-1356683199 @Kimahriman Please feel free to correct me if I am wrong since I am not 100% confident about the current POM setting. 1. The reason why we provide a shaded jar for

[GitHub] [incubator-sedona] jiayuasu commented on a diff in pull request #737: [SEDONA-215] Add ST_ConcaveHull

2022-12-17 Thread GitBox
jiayuasu commented on code in PR #737: URL: https://github.com/apache/incubator-sedona/pull/737#discussion_r1051529391 ## python/tests/sql/test_function.py: ## @@ -50,6 +50,22 @@ class TestPredicateJoin(TestBase): ] ) +def test_st_concave_hull(self): +

  1   2   3   4   5   6   7   8   9   10   >