On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung <felixche...@apache.org> wrote:
> I’d strongly recommend the community to move towards the first release > with the WIP disclaimer > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > https://incubator.apache.org/policy/incubation.html#releases > > > As for the LGPL dependency specifically, a replacement will be needed? > To clarify, ok to note in the WIP disclaimer- so it can be released with this. > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes <jhug...@ccri.com> wrote: > >> Hi all, >> >> Has the fact that one of the dependencies is LGPL (GeoTools) been >> discussed / addressed? (See >> https://www.apache.org/legal/resolved.html#category-x) >> >> I'm asking since I don't know if the ASF has any recommended work >> arounds for shipping code with licenses that it does not approve of. >> >> Cheers, >> >> Jim >> >> On 11/23/20 1:41 PM, Felix Cheung wrote: >> > I can help review around Dev 13 to give a first pass. It should give >> you an >> > easier path to IPMC vote. >> > >> > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu <jiayu198...@gmail.com> wrote: >> > >> >> Hi Pawel and everyone, >> >> >> >> Let's do this in the first Sedona release. But can you please first >> fix the >> >> Python API for our Move-to-JTS PR, and then work on this one? If this >> >> Python RDD-DF Adapter PR might slow down our progress of releasing >> Sedona >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. >> >> >> >> @everyone >> >> Our top priority is to draw the first Sedona release ASAP. Users have >> been >> >> waiting for almost six months. Let's push hard to publish the first >> Sedona >> >> release to Maven Central and PyPI before Christmas. In order to make it >> >> happen, >> >> >> >> Finalize coding and documentation before Dec 6: >> >> 1. I believe the Move-to-JTS PR will be done in around one week. >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary >> >> 3. I will work on Sedona documentation. >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I >> will >> >> first create a branch for it to illustrate some necessary changes in >> Sedona >> >> SQL for Spark 2.4. >> >> >> >> Final walk-through before Dec 13 >> >> 1. Netanel can test the release management for Sedona. >> >> 2. Other committers can go through the docs, release notes >> >> >> >> Community voting before Dec 20 >> >> 1. Sedona community voting: before Dec 16 >> >> 2. Apache Incubator voting: before Dec 20 >> >> >> >> Push to Maven Central and PyPi before Dec 24 >> >> >> >> Please feel free to comment if you have any suggestions! >> >> >> >> Jia >> >> >> >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < >> pawel93kocin...@gmail.com> >> >> wrote: >> >> >> >>> Hi, >> >>> I saw some users reported need to improve Python RDD API in two >> >> scenarios: >> >>> - converting spatial flat join result to df >> >>> - saving spatial flat join result directly to external storage >> >>> >> >>> Currently SerDe between jvm and Python causes additional time needed >> to >> >>> compute the result. I have a local branch with tests where this >> >>> functionality is available (need 3-4 days to make it 100% ready), in >> two >> >>> above scenarios there will be almost no difference between Python and >> >> Scala >> >>> or Java API. Should I create PR to include this feature within the >> first >> >>> Sedona release ? >> >>> Regards, >> >>> Paweł >> >>> >> >>> pon., 16 lis 2020 o 08:29 Jia Yu <jiayu198...@gmail.com> napisał(a): >> >>> >> >>>> Dear all, >> >>>> >> >>>> Thanks for all your suggestions. >> >>>> >> >>>> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR >> >> and >> >>>> two JTS PRs. @Jim Hughes <jhug...@ccri.com> , @Paweł Kociński >> >>>> <pawel93kocin...@gmail.com> , I, and probably Martin from JTS will >> take >> >>>> care of these PRs in the coming days. >> >>>> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 >> >>>> (2) JTS PR: https://github.com/locationtech/jts/pull/633 >> >>>> https://github.com/locationtech/jts/pull/634 >> >>>> >> >>>> 2. To move forward with the first release, I have deleted the >> "SNAPSHOT" >> >>>> in my JTS 1.16 fork. >> >>>> Most likely, we have to move forward with my JTS 1.16 fork in the >> first >> >>>> Sedona release because of the conflict among JTStoGeoJSON, GeoTools, >> and >> >>>> JTS 1.17. >> >>>> So @Netanel Malka <netanel...@gmail.com> could you please do >> another >> >>>> dry-run on the Sedona first release on this Sedona branch: >> >> sedona-1.0-doc: >> >>>> https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc >> >>>> >> >>>> Thanks, >> >>>> Jia >> >>>> >> >>>> On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes <jhug...@ccri.com> >> wrote: >> >>>> >> >>>>> Hi Mo, >> >>>>> >> >>>>> I can definitely help. The first step will be for Jia to push a PR >> for >> >>>>> the JTS changes. (Since they are his changes, I cannot do this on >> his >> >>>>> behalf.) >> >>>>> >> >>>>> From talking to the lead JTS developer, he wanted to see the >> previous >> >>>>> PR (from months/a year+ ago) split up. I think the initial PR >> should >> >> be >> >>>>> used to discuss what changes are sensible for JTS and where we'll >> need >> >>>>> to push some of the changes to Sedona. >> >>>>> >> >>>>> Concretely, I noticed that the Sedona JTS fork changes the toString >> on >> >>>>> Geometry to include printing out the userData. I imagine that may >> >> cause >> >>>>> trouble for downstream JTS users, so it'd be good to find an >> >>>>> alternative. One suggestion would to be add a static method in >> Sedona >> >>>>> for printing a Geometry with its userData object. >> >>>>> >> >>>>> Cheers, >> >>>>> >> >>>>> Jim >> >>>>> >> >>>>> On 11/12/20 12:32 PM, Mohamed Sarwat wrote: >> >>>>>> Folks, >> >>>>>> >> >>>>>> I totally agree with Jim on that. Jim, would you like to take the >> >> lead >> >>>>> on that - I trust that you can bring this task to completion. Jia, >> >> would >> >>>>> you please let us know how we can incorporate the changes into the >> JTS >> >>>>> master branch? >> >>>>>> Thanks, >> >>>>>> >> >>>>>>> On Nov 12, 2020, at 10:10 AM, Jim Hughes <jhug...@ccri.com> >> wrote: >> >>>>>>> >> >>>>>>> Hi all, >> >>>>>>> >> >>>>>>> As a JTS committer, I have tried to request that the Sedona >> project >> >>>>> discuss the desired changes to JTS previously. I'd still encourage >> >> that. >> >>>>>>> JTS is an active project and I feel that maintaining a fork of JTS >> >> is >> >>>>> unnecessary and inappropriate. >> >>>>>>> Cheers, >> >>>>>>> >> >>>>>>> Jim >> >>>>>>> >> >>>>>>>> On 11/11/20 9:04 PM, Felix Cheung wrote: >> >>>>>>>> Ah. You will need to publish it in order for the dependency chain >> >> to >> >>>>> work >> >>>>>>>> on Maven Central >> >>>>>>>> >> >>>>>>>> However, since you are not the project owner there you might need >> >> to >> >>>>>>>> publish that under a different artifact id. >> >>>>>>>> >> >>>>>>>> In general, it would be best to avoid hard forking another >> project >> >>>>> like >> >>>>>>>> this. >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>> On Wed, Nov 11, 2020 at 1:05 PM Jia Yu <jiayu198...@gmail.com> >> >>>>> wrote: >> >>>>>>>>> Hi Netanel, >> >>>>>>>>> >> >>>>>>>>> That links to this git submodule: >> >>>>>>>>> >> >> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 >> >>>>>>>>> I can easily fix this by changing the version number here to >> >> 1.16.2 >> >>>>>>>>> excluding "SNAPSHOT": >> >>>>>>>>> >> >> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 >> >>>>>>>>> Will this solve the problem? >> >>>>>>>>> >> >>>>>>>>> On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka < >> >> netanel...@gmail.com >> >>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> Hi Folks, >> >>>>>>>>>> >> >>>>>>>>>> I tried to make a release (dry-run) following by >> >>>>>>>>>> publishing-maven-artifacts >> >>>>>>>>>> <https://infra.apache.org/publishing-maven-artifacts.html>, >> and >> >> I >> >>>>>>>>>> encountered an issue. >> >>>>>>>>>> >> >>>>>>>>>> On sedona-core, we have jts-core as a dependency with the >> >> SNAPSHOT >> >>>>>>>>>> version. >> >>>>>>>>>> (link >> >>>>>>>>>> < >> >>>>>>>>>> >> >> >> https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 >> >>>>>>>>>> ) >> >>>>>>>>>> >> >>>>>>>>>> As a prerequisite to the release process, we cannot have >> >>>>> dependencies in a >> >>>>>>>>>> SNAPSHOT version. >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> Do you have any clue about how to solve this? >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> On Mon, 9 Nov 2020 at 21:22, Netanel Malka < >> netan...@sela.co.il> >> >>>>> wrote: >> >>>>>>>>>>> OK. Thanks Felix. >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Updates: >> >>>>>>>>>>> >> >>>>>>>>>>> * >> >>>>>>>>>>> * Opened a ticket for INFRA to Enable Nexus Access For >> >>>>> Sedona< >> >>>>>>>>>>> https://issues.apache.org/jira/browse/INFRA-21085> >> >>>>>>>>>>> * Followed this< >> >>>>>>>>>>> https://infra.apache.org/publishing-maven-artifacts.html> >> guide >> >>>>> to test >> >>>>>>>>>>> the maven release process >> >>>>>>>>>>> * I hope to create a PR soon for adjusting the build to >> >>>>> deploy to >> >>>>>>>>>> the >> >>>>>>>>>>> ASF Nexus repository >> >>>>>>>>>>> * The key that signs the artifacts were created and >> tested. >> >>>>>>>>>>> >> >>>>>>>>>>> Do we want to create a candidate release for the current >> master >> >>>>> branch? >> >>>>>>>>>>> Netanel Malka, >> >>>>>>>>>>> Big Data Consultant >> >>>>>>>>>>> [Description: Description: Description: Description: >> >>>>>>>>>>> cid:image001.jpg@01C85203.36A2AF30] >> >>>>>>>>>>> ________________________________ >> >>>>>>>>>>> From: Felix Cheung <felixche...@apache.org> >> >>>>>>>>>>> Sent: Wednesday, November 4, 2020 19:57 >> >>>>>>>>>>> To: dev@sedona.apache.org >> >>>>>>>>>>> Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; >> >>>>> Zongsi >> >>>>>>>>>> Zhang >> >>>>>>>>>>> Subject: Re: First Sedona release >> >>>>>>>>>>> >> >>>>>>>>>>> 1) No you don’t need KEYS file in github only on the release >> >> share >> >>>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/ >> >>>>>>>>>>> >> >>>>>>>>>>> 2) as podling you add to >> >>>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/ >> >>>>>>>>>>> When you commit via svn you will be able to add a “directory” >> >> for >> >>>>> Sedona >> >>>>>>>>>>> 2a) for release, you basically do a svn rename to move from >> dev >> >> to >> >>>>>>>>>> release >> >>>>>>>>>>> “path” >> >>>>>>>>>>> >> >>>>>>>>>>> 3) if you have java based artifacts, yes. You will publish to >> >>>>> Nexus, >> >>>>>>>>>>> staging first and when release is signed off, you can click on >> >> the >> >>>>>>>>>>> interface to make it official, which then automatically sync >> to >> >>>>> Maven >> >>>>>>>>>>> central. >> >>>>>>>>>>> >> >>>>>>>>>>> Here is a script for example that does release signing and >> >>>>> publication >> >>>>>>>>>> to >> >>>>>>>>>>> Nexus (and staging before release) >> >>>>>>>>>>> >> >>>>>>>>>>> >> >> >> https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh >> >>>>>>>>>>> On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka < >> >>>>> netanel...@gmail.com >> >>>>>>>>>> <mailto: >> >>>>>>>>>>> netanel...@gmail.com>> wrote: >> >>>>>>>>>>> Hi, >> >>>>>>>>>>> >> >>>>>>>>>>> I followed the release-signing >> >>>>>>>>>>> <https://infra.apache.org/release-signing.html> doc and >> created >> >>>>> a key >> >>>>>>>>>> for >> >>>>>>>>>>> signing and hashing. >> >>>>>>>>>>> >> >>>>>>>>>>> I have a few questions: >> >>>>>>>>>>> >> >>>>>>>>>>> 1. Should the KEYS file also be added to the project root >> >>>>> directory >> >>>>>>>>>> on >> >>>>>>>>>>> Github? ( I saw it in Apache Ant) >> >>>>>>>>>>> 2. I saw in release-policy_upload-ci >> >>>>>>>>>>> < >> http://www.apache.org/legal/release-policy.html#upload-ci> >> >>>>> that we >> >>>>>>>>>>> need >> >>>>>>>>>>> to add a release candidate to >> >>>>>>>>>> https://dist.apache.org/repos/dist/*dev*/ >> >>>>>>>>>>> <TLP >> >>>>>>>>>>> name>/. However, there does not seem to be a directory >> with >> >>>>> Sedona as >> >>>>>>>>>>> the >> >>>>>>>>>>> TLP name. How may we be able to get a directory with that >> >>>>> name? (Also >> >>>>>>>>>>> for >> >>>>>>>>>>> the *release*) >> >>>>>>>>>>> 3. Do we need to push the artifacts also to ASF Nexus >> >>>>> Repository >> >>>>>>>>>> (beside >> >>>>>>>>>>> Maven Central)? >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks. >> >>>>>>>>>>> >> >>>>>>>>>>> On Mon, 2 Nov 2020 at 19:21, Netanel Malka < >> >> netanel...@gmail.com >> >>>>>>>>>> <mailto: >> >>>>>>>>>>> netanel...@gmail.com>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Thanks Felix. >> >>>>>>>>>>>> >> >>>>>>>>>>>> I would be delighted to help. >> >>>>>>>>>>>> I can start with the GPG. >> >>>>>>>>>>>> Can I test it on a some artifact, or I need to wait for >> the >> >>>>> first >> >>>>>>>>>>> release? >> >>>>>>>>>>>> On Mon, 2 Nov 2020 at 03:17, Felix Cheung < >> >>>>> felixche...@apache.org >> >>>>>>>>>>> <mailto:felixche...@apache.org>> wrote: >> >>>>>>>>>>>>> Great progress! >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> To add, >> >>>>>>>>>>>>> A) I’d strongly recommend the WIP disclaimer - it would be >> >> much >> >>>>>>>>>> easier >> >>>>>>>>>>> to >> >>>>>>>>>>>>> pass with in the first release >> >>>>>>>>>>>>> >> >> https://incubator.apache.org/policy/incubation.html#disclaimers >> >>>>>>>>>>>>> B) more info in signing, checksum >> >>>>>>>>>>>>> https://infra.apache.org/release-signing.html >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> C) signing key should be individual’s and (public key ) >> >>>>> published and >> >>>>>>>>>>> also >> >>>>>>>>>>>>> listed in KEYS file - KEYS file should be located next to >> the >> >>>>>>>>>> staging >> >>>>>>>>>>>>> (and >> >>>>>>>>>>>>> later release) location, see above >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> D) “correct place” - this is in reference to ASF officIal >> >>>>> staging >> >>>>>>>>>> server >> >>>>>>>>>>>>> http://www.apache.org/legal/release-policy.html#stage >> >>>>>>>>>>>>> And can be “uploaded” by committing to svn >> >>>>>>>>>>>>> http://www.apache.org/legal/release-policy.html#upload-ci >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> E) python / PyPI - >> >>>>>>>>>>>>> https://incubator.apache.org/guides/distribution.html#pypi >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Sun, Nov 1, 2020 at 2:17 PM Jia Yu <ji...@apache.org >> >> <mailto: >> >>>>>>>>>>> ji...@apache.org>> wrote: >> >>>>>>>>>>>>>> Hi Netanel, Pawel and other committers, >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> While Pawel is working on Python code of Sedona 1.0, let's >> >>>>> focus on >> >>>>>>>>>>>>> other >> >>>>>>>>>>>>>> parts required by the release. Netanel, can you help me >> with >> >>>>> all >> >>>>>>>>>> the >> >>>>>>>>>>> ASF >> >>>>>>>>>>>>>> incubator requirement items that are not DONE? >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> *Here is a checklist for our first Sedona release* >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> *ASF incubator requirement >> >>>>>>>>>>>>>> ( >> https://incubator.apache.org/guides/releasemanagement.html >> >>>>>>>>>>>>>> < >> https://incubator.apache.org/guides/releasemanagement.html >> >>> , >> >>>>> we >> >>>>>>>>>>>>> probably >> >>>>>>>>>>>>>> should read ASF release requirement as well):* >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> 1 .Include the word incubating in the release file name: >> >> DONE. >> >>>>>>>>>> Please >> >>>>>>>>>>>>> see >> >>>>>>>>>>>>>> the POM.xml in all directories. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> 2. Include an ASF LICENSE and NOTICE file: DONE. Please see >> >> the >> >>>>>>>>>> GitHub >> >>>>>>>>>>>>>> repo. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> 3. Have valid checksums or signatures: I believe signature >> >>>>> should >> >>>>>>>>>> be >> >>>>>>>>>>>>> done >> >>>>>>>>>>>>>> by the GPG key. Not sure about the checksum. I am also not >> >> sure >> >>>>>>>>>> about >> >>>>>>>>>>>>> the >> >>>>>>>>>>>>>> GPG key requirement of ASF. I use GPG key to sign releases >> of >> >>>>>>>>>> GeoSpark >> >>>>>>>>>>>>> in >> >>>>>>>>>>>>>> the past. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> 4. Be placed in the correct place on the ASF’s >> >> infrastructure: >> >>>>> we >> >>>>>>>>>>> should >> >>>>>>>>>>>>>> place our releases in two places: Maven, and PyPi. Not sure >> >>>>> how to >> >>>>>>>>>>>>> relate >> >>>>>>>>>>>>>> them to ASF. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> 5. Have a KEYS file to validate the release: this should be >> >> the >> >>>>>>>>>> public >> >>>>>>>>>>>>> key >> >>>>>>>>>>>>>> of our GPG key? >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> *Sedona requirement* >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> 1. Python path name, file headers, and jars >> >>>>>>>>>>>>>> 2. Project website docs: documentation should use the name, >> >>>>>>>>>> Sedona, in >> >>>>>>>>>>>>> all >> >>>>>>>>>>>>>> tutorials. We should also include the situation of GeoTools >> >>>>>>>>>>>>> dependencies. >> >>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>> Jia >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Wed, Oct 14, 2020 at 10:08 PM Jia Yu <ji...@apache.org >> >>>>> <mailto: >> >>>>>>>>>>> ji...@apache.org>> wrote: >> >>>>>>>>>>>>>>> Hi folks, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> We will be working on the first Sedona. Please see the >> JIRA >> >>>>>>>>>> ticket >> >>>>>>>>>>>>> here: >> >> >> https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues >> >>>>>>>>>>>>>>> Do you think there are any outstanding issues to be fixed >> as >> >>>>>>>>>> well? >> >>>>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>>>> Jia >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>> -- >> >>>>>>>>>>>> Best regards, >> >>>>>>>>>>>> Netanel Malka. >> >>>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> Best regards, >> >>>>>>>>>>> Netanel Malka. >> >>>>>>>>>>> >> >>>>>>>>>> -- >> >>>>>>>>>> Best regards, >> >>>>>>>>>> Netanel Malka. >> >>>>>>>>>> >> >>>>> >> >>
