[jira] [Closed] (SEDONA-3) Prepare the first Sedona release
[ https://issues.apache.org/jira/browse/SEDONA-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Yu closed SEDONA-3. --- Resolution: Fixed > Prepare the first Sedona release > > > Key: SEDONA-3 > URL: https://issues.apache.org/jira/browse/SEDONA-3 > Project: Apache Sedona > Issue Type: Task >Reporter: Jia Yu >Assignee: Jia Yu >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > > My idea is that in the first Sedona release (lets call it 1.0.0?), we will > only support Spark 3.0 and 2.4. > > We will have to change all file headers and classpaths. > > In addition, we also need to update the documentation website and explain the > GeoTools jar has to be downloaded by the users themselves. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: First Sedona release
https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer https://incubator.apache.org/policy/incubation.html#releases As for the LGPL dependency specifically, a replacement will be needed? To clarify, ok to note in the WIP disclaimer- so it can be released with this. On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes < jhug...@ccri.com> wrote: Hi all, Has the fact that one of the dependencies is LGPL (GeoTools) been discussed / addressed? (See https://www.apache.org/legal/resolved.html#category-x) I'm asking since I don't know if the ASF has any recommended work arounds for shipping code with licenses that it does not approve of. Cheers, Jim On 11/23/20 1:41 PM, Felix Cheung wrote: I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu < jiayu198...@gmail.com> wrote: Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < pawel93kocin...@gmail.com> wrote: Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external storage Currently SerDe between jvm and Python causes additional time needed to compute the result. I have a local branch with tests where this functionality is available (need 3-4 days to make it 100% ready), in two above scenarios there will be almost no difference between Python and Scala or Java API. Should I create PR to include this feature within the first Sedona release ? Regards, Paweł pon., 16 lis 2020 o 08:29 Jia Yu < jiayu198...@gmail.com> napisał(a): Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes < jhug...@ccri.com> wrote: Hi Mo, I can definitely help. The first step will be for Jia to push a PR for the JTS changes. (Since they are his changes, I cannot do this on his behalf.) From talking to the lead JTS developer, he wanted to see the previous PR (from months/a year+ ago) split up. I think the initial PR should be used to discuss what changes are sensible for JTS and where we'll need to push some of the changes to Sedona. Concretely, I noticed that the Sedona JTS fork changes the toString on Geometry to include printing out the userData. I imagine that may cause trouble for downstream JTS users, so it'd be good to find an alternative. One suggestion would to be add a static method in Sedona for printing a Geometry with its userData object. Cheers, Jim On 11/12/20 12:32 PM, Mohamed Sarwat wrote: Folks, I totally agree with Jim on that. Ji
Re: First Sedona release
t;>> >> > >>>>>> The current status: >>> >> > >>>>>> 1. Move to JTS PR has been merged to the master branch. If >>> JTS >>> >> 1.18 >>> >> > >> gets >>> >> > >>>>>> published in a few weeks, we will use the latest JTS. >>> Otherwise, >>> >> we >>> >> > >> still >>> >> > >>>>>> need to use my fork for this release. But Sedona API is now >>> >> > >> finalized. From >>> >> > >>>>>> the user perspective, use my fork or JTS official release >>> should >>> >> not >>> >> > >> make >>> >> > >>>>>> any difference. >>> >> > >>>>>> 2. Sedona doc update is in progress. I am half way there. >>> You can >>> >> > >> track >>> >> > >>>>>> the progress here: >>> >> > >> https://github.com/apache/incubator-sedona/pull/493 >>> >> > >>>>>> 3. I will create a separate branch to test Spark 2.4 over >>> this >>> >> > >> weekend. >>> >> > >>>>>> 4. Pawel is working on his improvement on RDD-SQL Python >>> adapter. >>> >> > >>>>>> >>> >> > >>>>>> Question: >>> >> > >>>>>> >>> >> > >>>>>> What is the most appropriate maven artifact name for Sedona >>> on >>> >> Spark >>> >> > >>>>>> 2.4? I used to put "sedona-sql_2.4". But it looks like >>> "_2.4" is >>> >> > >> usually >>> >> > >>>>>> reserved for specifying the Scala version. How about >>> >> > >> "sedona-sql-spark2"? >>> >> > >>>>>> Should we also use "sedona-sql-spark3" for Spark 3.0? >>> >> > >>>>>> >>> >> > >>>>>> Thanks, >>> >> > >>>>>> Jia >>> >> > >>>>>> >>> >> > >>>>>> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes >> > >>> >> > wrote: >>> >> > >>>>>> >>> >> > >>>>>>> Hi all, >>> >> > >>>>>>> >>> >> > >>>>>>> Felix, good to know that a WIP disclaimer is standard >>> practice >>> >> and >>> >> > >> will >>> >> > >>>>>>> let things move forward! >>> >> > >>>>>>> >>> >> > >>>>>>> Jia, I believe that page is explaining that a portion of the >>> >> code >>> >> > in >>> >> > >>>>>>> various GeoTools modules has other licenses on it. As such, >>> >> > gt-main >>> >> > >> is >>> >> > >>>>>>> mostly LGPL with some BSD code as well. >>> >> > >>>>>>> >>> >> > >>>>>>> Cheers, >>> >> > >>>>>>> >>> >> > >>>>>>> Jim >>> >> > >>>>>>> >>> >> > >>>>>>> On 11/23/2020 9:50 PM, Jia Yu wrote: >>> >> > >>>>>>>> Thank you, Felix. I will use the WIP disclaimer. >>> >> > >>>>>>>> >>> >> > >>>>>>>> To answer Jim's question, GeoTools components use different >>> >> > >> licenses: >>> >> > >>>>>>>> >>> >> https://docs.geotools.org/latest/userguide/welcome/license.html >>> >> > >>>>>>>> >>> >> > >>>>>>>> GT-main uses BSD, so its binary can be included in Sedona's >>> >> > release. >>> >> > >>>>>>>> Other components in GeoTools use LGPL, but Sedona only uses >>> >> them >>> >> > for >>> >> > >>>>>>> CRS >
Re: First Sedona release
t; JTS >> >> > >>> source code into Sedona-core in our 1.0.0 release. In the future >> >> > release >> >> > >>> (say Sedona 1.0.1), we can drop JTS source code and use their >> Maven >> >> > >>> release. JTS source code is dual-licensed under Eclipse Public >> >> License >> >> > >> 2.0 >> >> > >>> and Eclipse Distribution License 1.0 (a BSD Style License). So >> it is >> >> > safe >> >> > >>> to keep it in Sedona. >> >> > >>> >> >> > >>> What do you think? @Jim Hughes Is this a >> good >> >> > idea? >> >> > >>> >> >> > >>> Thanks, >> >> > >>> Jia >> >> > >>> >> >> > >>> On Fri, Dec 4, 2020 at 10:43 PM Jia Yu >> >> wrote: >> >> > >>> >> >> > >>>> Hi Netanel, >> >> > >>>> >> >> > >>>> So for Sedona SQL 1.0.0 on Spark 2.4, we can do >> >> > >>>> "sedona-sql_2.11-2.4-1.0.0-incubator" , right? >> >> > >>>> >> >> > >>>> Sedona 1.0 on Spark 2.4 and 3.0 will be compiled against Scala >> 2.11 >> >> > and >> >> > >>>> 2.12. I believe this can be done via different compilation >> target >> >> in >> >> > >> Maven. >> >> > >>>> I am currently looking at whether I can do conditional >> compilation >> >> > using >> >> > >>>> Maven (similar to C++ #ifdef) because there is a change in >> >> Aggregator >> >> > in >> >> > >>>> Spark 3.0. Otherwise I always need to maintain a separate branch >> >> for >> >> > >> Sedona >> >> > >>>> on Spark 2.4 >> >> > >>>> >> >> > >>>> It looks OK to me. >> >> > >>>> >> >> > >>>> On Fri, Dec 4, 2020 at 1:12 AM Netanel Malka < >> netanel...@gmail.com >> >> > >> >> > >> wrote: >> >> > >>>>> Hi, >> >> > >>>>> I think that we can follow the Apache Spark convention as you >> can >> >> see >> >> > >>>>> here >> >> > >>>>> < >> >> > >> >> >> https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.0.1/ >> >> > >. >> >> > >>>>> For example: >> >> > >>>>> sedona-sql_2.11-2.4, where 2.11 -> scala version and 2.4 -> >> spark >> >> > >> version >> >> > >>>>>What do you think? >> >> > >>>>> >> >> > >>>>> >> >> > >>>>> On Fri, 4 Dec 2020 at 10:34, Jia Yu >> >> wrote: >> >> > >>>>> >> >> > >>>>>> Dear all, >> >> > >>>>>> >> >> > >>>>>> The current status: >> >> > >>>>>> 1. Move to JTS PR has been merged to the master branch. If JTS >> >> 1.18 >> >> > >> gets >> >> > >>>>>> published in a few weeks, we will use the latest JTS. >> Otherwise, >> >> we >> >> > >> still >> >> > >>>>>> need to use my fork for this release. But Sedona API is now >> >> > >> finalized. From >> >> > >>>>>> the user perspective, use my fork or JTS official release >> should >> >> not >> >> > >> make >> >> > >>>>>> any difference. >> >> > >>>>>> 2. Sedona doc update is in progress. I am half way there. You >> can >> >> > >> track >> >> > >>>>>> the progress here: >> >> > >> https://github.com/apache/incubator-sedona/pull/493 >> >> > >>>>>> 3. I will create a separate branch to test Spark 2.4 over this >> >> > >> weekend. >> >> > >>>>>> 4. Pawel is working on his improvement on RDD-SQL Python >
Re: First Sedona release
>>>> "sedona-sql_2.11-2.4-1.0.0-incubator" , right? > >> > >>>> > >> > >>>> Sedona 1.0 on Spark 2.4 and 3.0 will be compiled against Scala > 2.11 > >> > and > >> > >>>> 2.12. I believe this can be done via different compilation target > >> in > >> > >> Maven. > >> > >>>> I am currently looking at whether I can do conditional > compilation > >> > using > >> > >>>> Maven (similar to C++ #ifdef) because there is a change in > >> Aggregator > >> > in > >> > >>>> Spark 3.0. Otherwise I always need to maintain a separate branch > >> for > >> > >> Sedona > >> > >>>> on Spark 2.4 > >> > >>>> > >> > >>>> It looks OK to me. > >> > >>>> > >> > >>>> On Fri, Dec 4, 2020 at 1:12 AM Netanel Malka < > netanel...@gmail.com > >> > > >> > >> wrote: > >> > >>>>> Hi, > >> > >>>>> I think that we can follow the Apache Spark convention as you > can > >> see > >> > >>>>> here > >> > >>>>> < > >> > >> > >> https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.0.1/ > >> > >. > >> > >>>>> For example: > >> > >>>>> sedona-sql_2.11-2.4, where 2.11 -> scala version and 2.4 -> > spark > >> > >> version > >> > >>>>>What do you think? > >> > >>>>> > >> > >>>>> > >> > >>>>> On Fri, 4 Dec 2020 at 10:34, Jia Yu > >> wrote: > >> > >>>>> > >> > >>>>>> Dear all, > >> > >>>>>> > >> > >>>>>> The current status: > >> > >>>>>> 1. Move to JTS PR has been merged to the master branch. If JTS > >> 1.18 > >> > >> gets > >> > >>>>>> published in a few weeks, we will use the latest JTS. > Otherwise, > >> we > >> > >> still > >> > >>>>>> need to use my fork for this release. But Sedona API is now > >> > >> finalized. From > >> > >>>>>> the user perspective, use my fork or JTS official release > should > >> not > >> > >> make > >> > >>>>>> any difference. > >> > >>>>>> 2. Sedona doc update is in progress. I am half way there. You > can > >> > >> track > >> > >>>>>> the progress here: > >> > >> https://github.com/apache/incubator-sedona/pull/493 > >> > >>>>>> 3. I will create a separate branch to test Spark 2.4 over this > >> > >> weekend. > >> > >>>>>> 4. Pawel is working on his improvement on RDD-SQL Python > adapter. > >> > >>>>>> > >> > >>>>>> Question: > >> > >>>>>> > >> > >>>>>> What is the most appropriate maven artifact name for Sedona on > >> Spark > >> > >>>>>> 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" > is > >> > >> usually > >> > >>>>>> reserved for specifying the Scala version. How about > >> > >> "sedona-sql-spark2"? > >> > >>>>>> Should we also use "sedona-sql-spark3" for Spark 3.0? > >> > >>>>>> > >> > >>>>>> Thanks, > >> > >>>>>> Jia > >> > >>>>>> > >> > >>>>>> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes > >> > wrote: > >> > >>>>>> > >> > >>>>>>> Hi all, > >> > >>>>>>> > >> > >>>>>>> Felix, good to know that a WIP disclaimer is standard practice > >> and > >> > >> will > >> > >>>>>>> let things move forward! > >> > >>>>>>> > >> > >>>>>>> Jia, I believe that page is explaining that a portion of
Re: First Sedona release
>> > >. >> > >>>>> For example: >> > >>>>> sedona-sql_2.11-2.4, where 2.11 -> scala version and 2.4 -> spark >> > >> version >> > >>>>>What do you think? >> > >>>>> >> > >>>>> >> > >>>>> On Fri, 4 Dec 2020 at 10:34, Jia Yu >> wrote: >> > >>>>> >> > >>>>>> Dear all, >> > >>>>>> >> > >>>>>> The current status: >> > >>>>>> 1. Move to JTS PR has been merged to the master branch. If JTS >> 1.18 >> > >> gets >> > >>>>>> published in a few weeks, we will use the latest JTS. Otherwise, >> we >> > >> still >> > >>>>>> need to use my fork for this release. But Sedona API is now >> > >> finalized. From >> > >>>>>> the user perspective, use my fork or JTS official release should >> not >> > >> make >> > >>>>>> any difference. >> > >>>>>> 2. Sedona doc update is in progress. I am half way there. You can >> > >> track >> > >>>>>> the progress here: >> > >> https://github.com/apache/incubator-sedona/pull/493 >> > >>>>>> 3. I will create a separate branch to test Spark 2.4 over this >> > >> weekend. >> > >>>>>> 4. Pawel is working on his improvement on RDD-SQL Python adapter. >> > >>>>>> >> > >>>>>> Question: >> > >>>>>> >> > >>>>>> What is the most appropriate maven artifact name for Sedona on >> Spark >> > >>>>>> 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" is >> > >> usually >> > >>>>>> reserved for specifying the Scala version. How about >> > >> "sedona-sql-spark2"? >> > >>>>>> Should we also use "sedona-sql-spark3" for Spark 3.0? >> > >>>>>> >> > >>>>>> Thanks, >> > >>>>>> Jia >> > >>>>>> >> > >>>>>> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes >> > wrote: >> > >>>>>> >> > >>>>>>> Hi all, >> > >>>>>>> >> > >>>>>>> Felix, good to know that a WIP disclaimer is standard practice >> and >> > >> will >> > >>>>>>> let things move forward! >> > >>>>>>> >> > >>>>>>> Jia, I believe that page is explaining that a portion of the >> code >> > in >> > >>>>>>> various GeoTools modules has other licenses on it. As such, >> > gt-main >> > >> is >> > >>>>>>> mostly LGPL with some BSD code as well. >> > >>>>>>> >> > >>>>>>> Cheers, >> > >>>>>>> >> > >>>>>>> Jim >> > >>>>>>> >> > >>>>>>> On 11/23/2020 9:50 PM, Jia Yu wrote: >> > >>>>>>>> Thank you, Felix. I will use the WIP disclaimer. >> > >>>>>>>> >> > >>>>>>>> To answer Jim's question, GeoTools components use different >> > >> licenses: >> > >>>>>>>> >> https://docs.geotools.org/latest/userguide/welcome/license.html >> > >>>>>>>> >> > >>>>>>>> GT-main uses BSD, so its binary can be included in Sedona's >> > release. >> > >>>>>>>> Other components in GeoTools use LGPL, but Sedona only uses >> them >> > for >> > >>>>>>> CRS >> > >>>>>>>> transformation. I already set the dependency scope to >> "provided" >> > in >> > >>>>>>>> Sedona's POM.xml. If a user wants to use CRS transformation in >> > >>>>>>> Sedona, they >> > >>>>>>>> will have to add some GeoTools library by themselves. >> > >>>>>>>> >> > >>>>&g
Re: First Sedona release
;> > >>>>>What do you think? >> > >>>>> >> > >>>>> >> > >>>>> On Fri, 4 Dec 2020 at 10:34, Jia Yu >> wrote: >> > >>>>> >> > >>>>>> Dear all, >> > >>>>>> >> > >>>>>> The current status: >> > >>>>>> 1. Move to JTS PR has been merged to the master branch. If JTS >> 1.18 >> > >> gets >> > >>>>>> published in a few weeks, we will use the latest JTS. Otherwise, >> we >> > >> still >> > >>>>>> need to use my fork for this release. But Sedona API is now >> > >> finalized. From >> > >>>>>> the user perspective, use my fork or JTS official release should >> not >> > >> make >> > >>>>>> any difference. >> > >>>>>> 2. Sedona doc update is in progress. I am half way there. You can >> > >> track >> > >>>>>> the progress here: >> > >> https://github.com/apache/incubator-sedona/pull/493 >> > >>>>>> 3. I will create a separate branch to test Spark 2.4 over this >> > >> weekend. >> > >>>>>> 4. Pawel is working on his improvement on RDD-SQL Python adapter. >> > >>>>>> >> > >>>>>> Question: >> > >>>>>> >> > >>>>>> What is the most appropriate maven artifact name for Sedona on >> Spark >> > >>>>>> 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" is >> > >> usually >> > >>>>>> reserved for specifying the Scala version. How about >> > >> "sedona-sql-spark2"? >> > >>>>>> Should we also use "sedona-sql-spark3" for Spark 3.0? >> > >>>>>> >> > >>>>>> Thanks, >> > >>>>>> Jia >> > >>>>>> >> > >>>>>> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes >> > wrote: >> > >>>>>> >> > >>>>>>> Hi all, >> > >>>>>>> >> > >>>>>>> Felix, good to know that a WIP disclaimer is standard practice >> and >> > >> will >> > >>>>>>> let things move forward! >> > >>>>>>> >> > >>>>>>> Jia, I believe that page is explaining that a portion of the >> code >> > in >> > >>>>>>> various GeoTools modules has other licenses on it. As such, >> > gt-main >> > >> is >> > >>>>>>> mostly LGPL with some BSD code as well. >> > >>>>>>> >> > >>>>>>> Cheers, >> > >>>>>>> >> > >>>>>>> Jim >> > >>>>>>> >> > >>>>>>> On 11/23/2020 9:50 PM, Jia Yu wrote: >> > >>>>>>>> Thank you, Felix. I will use the WIP disclaimer. >> > >>>>>>>> >> > >>>>>>>> To answer Jim's question, GeoTools components use different >> > >> licenses: >> > >>>>>>>> >> https://docs.geotools.org/latest/userguide/welcome/license.html >> > >>>>>>>> >> > >>>>>>>> GT-main uses BSD, so its binary can be included in Sedona's >> > release. >> > >>>>>>>> Other components in GeoTools use LGPL, but Sedona only uses >> them >> > for >> > >>>>>>> CRS >> > >>>>>>>> transformation. I already set the dependency scope to >> "provided" >> > in >> > >>>>>>>> Sedona's POM.xml. If a user wants to use CRS transformation in >> > >>>>>>> Sedona, they >> > >>>>>>>> will have to add some GeoTools library by themselves. >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung < >> > >> felixche...@apache.org> >> > >&g
Re: First Sedona release
progress. I am half way there. You can > > >> track > > >>>>>> the progress here: > > >> https://github.com/apache/incubator-sedona/pull/493 > > >>>>>> 3. I will create a separate branch to test Spark 2.4 over this > > >> weekend. > > >>>>>> 4. Pawel is working on his improvement on RDD-SQL Python adapter. > > >>>>>> > > >>>>>> Question: > > >>>>>> > > >>>>>> What is the most appropriate maven artifact name for Sedona on > Spark > > >>>>>> 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" is > > >> usually > > >>>>>> reserved for specifying the Scala version. How about > > >> "sedona-sql-spark2"? > > >>>>>> Should we also use "sedona-sql-spark3" for Spark 3.0? > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Jia > > >>>>>> > > >>>>>> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes > > wrote: > > >>>>>> > > >>>>>>> Hi all, > > >>>>>>> > > >>>>>>> Felix, good to know that a WIP disclaimer is standard practice > and > > >> will > > >>>>>>> let things move forward! > > >>>>>>> > > >>>>>>> Jia, I believe that page is explaining that a portion of the code > > in > > >>>>>>> various GeoTools modules has other licenses on it. As such, > > gt-main > > >> is > > >>>>>>> mostly LGPL with some BSD code as well. > > >>>>>>> > > >>>>>>> Cheers, > > >>>>>>> > > >>>>>>> Jim > > >>>>>>> > > >>>>>>> On 11/23/2020 9:50 PM, Jia Yu wrote: > > >>>>>>>> Thank you, Felix. I will use the WIP disclaimer. > > >>>>>>>> > > >>>>>>>> To answer Jim's question, GeoTools components use different > > >> licenses: > > >>>>>>>> https://docs.geotools.org/latest/userguide/welcome/license.html > > >>>>>>>> > > >>>>>>>> GT-main uses BSD, so its binary can be included in Sedona's > > release. > > >>>>>>>> Other components in GeoTools use LGPL, but Sedona only uses them > > for > > >>>>>>> CRS > > >>>>>>>> transformation. I already set the dependency scope to "provided" > > in > > >>>>>>>> Sedona's POM.xml. If a user wants to use CRS transformation in > > >>>>>>> Sedona, they > > >>>>>>>> will have to add some GeoTools library by themselves. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung < > > >> felixche...@apache.org> > > >>>>>>> wrote: > > >>>>>>>>> On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung < > > >> felixche...@apache.org > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> I’d strongly recommend the community to move towards the first > > >>>>>>> release > > >>>>>>>>>> with the WIP disclaimer > > >>>>>>>>>> > > >>>>>>>>>> > > >> > > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > >>>>>>>>>> https://incubator.apache.org/policy/incubation.html#releases > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> As for the LGPL dependency specifically, a replacement will be > > >>>>>>> needed? > > >>>>>>>>> To clarify, ok to note in the WIP disclaimer- so it can be > > released > > >>>>>>> with > > >>>>>>>>> this. > > >>>>>>>>> > > >>>>>>
Re: First Sedona release
wrote: > >>>>>> > >>>>>>> Hi all, > >>>>>>> > >>>>>>> Felix, good to know that a WIP disclaimer is standard practice and > >> will > >>>>>>> let things move forward! > >>>>>>> > >>>>>>> Jia, I believe that page is explaining that a portion of the code > in > >>>>>>> various GeoTools modules has other licenses on it. As such, > gt-main > >> is > >>>>>>> mostly LGPL with some BSD code as well. > >>>>>>> > >>>>>>> Cheers, > >>>>>>> > >>>>>>> Jim > >>>>>>> > >>>>>>> On 11/23/2020 9:50 PM, Jia Yu wrote: > >>>>>>>> Thank you, Felix. I will use the WIP disclaimer. > >>>>>>>> > >>>>>>>> To answer Jim's question, GeoTools components use different > >> licenses: > >>>>>>>> https://docs.geotools.org/latest/userguide/welcome/license.html > >>>>>>>> > >>>>>>>> GT-main uses BSD, so its binary can be included in Sedona's > release. > >>>>>>>> Other components in GeoTools use LGPL, but Sedona only uses them > for > >>>>>>> CRS > >>>>>>>> transformation. I already set the dependency scope to "provided" > in > >>>>>>>> Sedona's POM.xml. If a user wants to use CRS transformation in > >>>>>>> Sedona, they > >>>>>>>> will have to add some GeoTools library by themselves. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung < > >> felixche...@apache.org> > >>>>>>> wrote: > >>>>>>>>> On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung < > >> felixche...@apache.org > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I’d strongly recommend the community to move towards the first > >>>>>>> release > >>>>>>>>>> with the WIP disclaimer > >>>>>>>>>> > >>>>>>>>>> > >> > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > >>>>>>>>>> https://incubator.apache.org/policy/incubation.html#releases > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> As for the LGPL dependency specifically, a replacement will be > >>>>>>> needed? > >>>>>>>>> To clarify, ok to note in the WIP disclaimer- so it can be > released > >>>>>>> with > >>>>>>>>> this. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes > >>>>>>> wrote: > >>>>>>>>>>> Hi all, > >>>>>>>>>>> > >>>>>>>>>>> Has the fact that one of the dependencies is LGPL (GeoTools) > been > >>>>>>>>>>> discussed / addressed? (See > >>>>>>>>>>> https://www.apache.org/legal/resolved.html#category-x) > >>>>>>>>>>> > >>>>>>>>>>> I'm asking since I don't know if the ASF has any recommended > work > >>>>>>>>>>> arounds for shipping code with licenses that it does not > approve > >>>>>>> of. > >>>>>>>>>>> Cheers, > >>>>>>>>>>> > >>>>>>>>>>> Jim > >>>>>>>>>>> > >>>>>>>>>>> On 11/23/20 1:41 PM, Felix Cheung wrote: > >>>>>>>>>>>> I can help review around Dev 13 to give a first pass. It > should > >>>>>>> give > >>>>>>>>>>> you an > >>>>>>>>>>>> easier path to IPMC vote. > >>>>>>>>>>>
Re: First Sedona release
e Spark convention as you can see here < https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.0.1/>. For example: sedona-sql_2.11-2.4, where 2.11 -> scala version and 2.4 -> spark version What do you think? On Fri, 4 Dec 2020 at 10:34, Jia Yu wrote: Dear all, The current status: 1. Move to JTS PR has been merged to the master branch. If JTS 1.18 gets published in a few weeks, we will use the latest JTS. Otherwise, we still need to use my fork for this release. But Sedona API is now finalized. From the user perspective, use my fork or JTS official release should not make any difference. 2. Sedona doc update is in progress. I am half way there. You can track the progress here: https://github.com/apache/incubator-sedona/pull/493 3. I will create a separate branch to test Spark 2.4 over this weekend. 4. Pawel is working on his improvement on RDD-SQL Python adapter. Question: What is the most appropriate maven artifact name for Sedona on Spark 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" is usually reserved for specifying the Scala version. How about "sedona-sql-spark2"? Should we also use "sedona-sql-spark3" for Spark 3.0? Thanks, Jia On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes wrote: Hi all, Felix, good to know that a WIP disclaimer is standard practice and will let things move forward! Jia, I believe that page is explaining that a portion of the code in various GeoTools modules has other licenses on it. As such, gt-main is mostly LGPL with some BSD code as well. Cheers, Jim On 11/23/2020 9:50 PM, Jia Yu wrote: Thank you, Felix. I will use the WIP disclaimer. To answer Jim's question, GeoTools components use different licenses: https://docs.geotools.org/latest/userguide/welcome/license.html GT-main uses BSD, so its binary can be included in Sedona's release. Other components in GeoTools use LGPL, but Sedona only uses them for CRS transformation. I already set the dependency scope to "provided" in Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, they will have to add some GeoTools library by themselves. On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung < felixche...@apache.org> wrote: On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung < felixche...@apache.org wrote: I’d strongly recommend the community to move towards the first release with the WIP disclaimer https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer https://incubator.apache.org/policy/incubation.html#releases As for the LGPL dependency specifically, a replacement will be needed? To clarify, ok to note in the WIP disclaimer- so it can be released with this. On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: Hi all, Has the fact that one of the dependencies is LGPL (GeoTools) been discussed / addressed? (See https://www.apache.org/legal/resolved.html#category-x) I'm asking since I don't know if the ASF has any recommended work arounds for shipping code with licenses that it does not approve of. Cheers, Jim On 11/23/20 1:41 PM, Felix Cheung wrote: I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < pawel93kocin...@gmail.com> wrote: Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external st
Re: First Sedona release
t; Question: > >>>> > >>>> What is the most appropriate maven artifact name for Sedona on Spark > >>>> 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" is > usually > >>>> reserved for specifying the Scala version. How about > "sedona-sql-spark2"? > >>>> Should we also use "sedona-sql-spark3" for Spark 3.0? > >>>> > >>>> Thanks, > >>>> Jia > >>>> > >>>> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes wrote: > >>>> > >>>>> Hi all, > >>>>> > >>>>> Felix, good to know that a WIP disclaimer is standard practice and > will > >>>>> let things move forward! > >>>>> > >>>>> Jia, I believe that page is explaining that a portion of the code in > >>>>> various GeoTools modules has other licenses on it. As such, gt-main > is > >>>>> mostly LGPL with some BSD code as well. > >>>>> > >>>>> Cheers, > >>>>> > >>>>> Jim > >>>>> > >>>>> On 11/23/2020 9:50 PM, Jia Yu wrote: > >>>>>> Thank you, Felix. I will use the WIP disclaimer. > >>>>>> > >>>>>> To answer Jim's question, GeoTools components use different > licenses: > >>>>>> https://docs.geotools.org/latest/userguide/welcome/license.html > >>>>>> > >>>>>> GT-main uses BSD, so its binary can be included in Sedona's release. > >>>>>> Other components in GeoTools use LGPL, but Sedona only uses them for > >>>>> CRS > >>>>>> transformation. I already set the dependency scope to "provided" in > >>>>>> Sedona's POM.xml. If a user wants to use CRS transformation in > >>>>> Sedona, they > >>>>>> will have to add some GeoTools library by themselves. > >>>>>> > >>>>>> > >>>>>> On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung < > felixche...@apache.org> > >>>>> wrote: > >>>>>>> On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung < > felixche...@apache.org > >>>>>>> wrote: > >>>>>>> > >>>>>>>> I’d strongly recommend the community to move towards the first > >>>>> release > >>>>>>>> with the WIP disclaimer > >>>>>>>> > >>>>>>>> > >>>>> > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > >>>>>>>> https://incubator.apache.org/policy/incubation.html#releases > >>>>>>>> > >>>>>>>> > >>>>>>>> As for the LGPL dependency specifically, a replacement will be > >>>>> needed? > >>>>>>> To clarify, ok to note in the WIP disclaimer- so it can be released > >>>>> with > >>>>>>> this. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes > >>>>> wrote: > >>>>>>>>> Hi all, > >>>>>>>>> > >>>>>>>>> Has the fact that one of the dependencies is LGPL (GeoTools) been > >>>>>>>>> discussed / addressed? (See > >>>>>>>>> https://www.apache.org/legal/resolved.html#category-x) > >>>>>>>>> > >>>>>>>>> I'm asking since I don't know if the ASF has any recommended work > >>>>>>>>> arounds for shipping code with licenses that it does not approve > >>>>> of. > >>>>>>>>> Cheers, > >>>>>>>>> > >>>>>>>>> Jim > >>>>>>>>> > >>>>>>>>> On 11/23/20 1:41 PM, Felix Cheung wrote: > >>>>>>>>>> I can help review around Dev 13 to give a first pass. It should > >>>>> give > >>>>>>>>> you an > >>>>>>>>>> easier path to IPMC vote. > >>>>>>>>>> > >>>>>>>>
[GitHub] [incubator-sedona] jnh5y commented on a change in pull request #495: [SEDONA-3] Prepare the first Sedona release: Set up POM for uploading snapshots and releases
jnh5y commented on a change in pull request #495: URL: https://github.com/apache/incubator-sedona/pull/495#discussion_r540217296 ## File path: core/src/main/java/org/locationtech/jts/index/quadtree/Node.java ## @@ -0,0 +1,183 @@ + +/* + * Copyright (c) 2016 Vivid Solutions. + * + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License 2.0 + * and Eclipse Distribution License v. 1.0 which accompanies this distribution. + * The Eclipse Public License is available at http://www.eclipse.org/legal/epl-v20.html + * and the Eclipse Distribution License is available at + * + * http://www.eclipse.org/org/documents/edl-v10.php. + */ +package org.locationtech.jts.index.quadtree; + +import org.locationtech.jts.geom.Envelope; +import org.locationtech.jts.util.Assert; + +/** + * Represents a node of a {@link Quadtree}. Nodes contain + * items which have a spatial extent corresponding to the node's position + * in the quadtree. + * + * @version 1.7 + */ +public class Node Review comment: While I'm still interested in identifying options that'd allow Sedona to use a JTS release without forking/copying any of the code, I'd point that using the same FQCNs may lead to classpath / classloader issues. Basically, if you really have to do this, I'd suggest changing the package names. Again, reflection may be a winning, short term solution here! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: First Sedona release
www.apache.org/legal/resolved.html#category-x) I'm asking since I don't know if the ASF has any recommended work arounds for shipping code with licenses that it does not approve of. Cheers, Jim On 11/23/20 1:41 PM, Felix Cheung wrote: I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < pawel93kocin...@gmail.com> wrote: Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external storage Currently SerDe between jvm and Python causes additional time needed to compute the result. I have a local branch with tests where this functionality is available (need 3-4 days to make it 100% ready), in two above scenarios there will be almost no difference between Python and Scala or Java API. Should I create PR to include this feature within the first Sedona release ? Regards, Paweł pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: Hi Mo, I can definitely help. The first step will be for Jia to push a PR for the JTS changes. (Since they are his changes, I cannot do this on his behalf.) From talking to the lead JTS developer, he wanted to see the previous PR (from months/a year+ ago) split up. I think the initial PR should be used to discuss what changes are sensible for JTS and where we'll need to push some of the changes to Sedona. Concretely, I noticed that the Sedona JTS fork changes the toString on Geometry to include printing out the userData. I imagine that may cause trouble for downstream JTS users, so it'd be good to find an alternative. One suggestion would to be add a static method in Sedona for printing a Geometry with its userData object. Cheers, Jim On 11/12/20 12:32 PM, Mohamed Sarwat wrote: Folks, I totally agree with Jim on that. Jim, would you like to take the lead on that - I trust that you can bring this task to completion. Jia, would you please let us know how we can incorporate the changes into the JTS master branch? Thanks, On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: Hi all, As a JTS committer, I have tried to request that the Sedona project discuss the desired changes to JTS previously. I'd still encourage that. JTS is an active project and I feel that maintaining a fork of JTS is unnecessary and inappropriate. Cheers, Jim On 11/11/20 9:04 PM, Felix Cheung wrote: Ah. You will need to publish it in order for the dependency chain to work on
[GitHub] [incubator-sedona] jiayuasu opened a new pull request #495: [SEDONA-3] Prepare the first Sedona release: Set up POM for uploading snapshots and releases
jiayuasu opened a new pull request #495: URL: https://github.com/apache/incubator-sedona/pull/495 ## Is this PR related to a proposed Issue? https://issues.apache.org/jira/browse/SEDONA-3 ## What changes were proposed in this PR? 1. Remove JTS Git submodule, use JTS 1.17.1 official release with 5 additional JTS index files copied from JTS 1.18 (see my PR: https://github.com/locationtech/jts/pull/634) 2. Update the POM file accordingly 3. Move scalastyle_config.xml to the root just like what Spark does ## How was this patch tested? Successfully published SNAPSHOTs to Nexus: https://repository.apache.org/ You can verify this by searching sedona in the search box The following commands are used to cut the snapshots: https://gist.github.com/jiayuasu/849e1f3bf7a2dd11593ca27c14e9e92d ## Did this PR include necessary documentation updates? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: First Sedona release
gt;>>> >> wrote: >>>> >> >>>> >>> I’d strongly recommend the community to move towards the first >>>> release >>>> >>> with the WIP disclaimer >>>> >>> >>>> >>> >>>> >> >>>> https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer >>>> >>> https://incubator.apache.org/policy/incubation.html#releases >>>> >>> >>>> >>> >>>> >>> As for the LGPL dependency specifically, a replacement will be >>>> needed? >>>> >>> >>>> >> >>>> >> To clarify, ok to note in the WIP disclaimer- so it can be released >>>> with >>>> >> this. >>>> >> >>>> >> >>>> >> >>>> >>> On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes >>>> wrote: >>>> >>> >>>> >>>> Hi all, >>>> >>>> >>>> >>>> Has the fact that one of the dependencies is LGPL (GeoTools) been >>>> >>>> discussed / addressed? (See >>>> >>>> https://www.apache.org/legal/resolved.html#category-x) >>>> >>>> >>>> >>>> I'm asking since I don't know if the ASF has any recommended work >>>> >>>> arounds for shipping code with licenses that it does not approve >>>> of. >>>> >>>> >>>> >>>> Cheers, >>>> >>>> >>>> >>>> Jim >>>> >>>> >>>> >>>> On 11/23/20 1:41 PM, Felix Cheung wrote: >>>> >>>>> I can help review around Dev 13 to give a first pass. It should >>>> give >>>> >>>> you an >>>> >>>>> easier path to IPMC vote. >>>> >>>>> >>>> >>>>> >>>> >>>>> On Sun, Nov 22, 2020 at 10:50 PM Jia Yu >>>> >> wrote: >>>> >>>>>> Hi Pawel and everyone, >>>> >>>>>> >>>> >>>>>> Let's do this in the first Sedona release. But can you please >>>> first >>>> >>>> fix the >>>> >>>>>> Python API for our Move-to-JTS PR, and then work on this one? If >>>> this >>>> >>>>>> Python RDD-DF Adapter PR might slow down our progress of >>>> releasing >>>> >>>> Sedona >>>> >>>>>> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. >>>> >>>>>> >>>> >>>>>> @everyone >>>> >>>>>> Our top priority is to draw the first Sedona release ASAP. Users >>>> have >>>> >>>> been >>>> >>>>>> waiting for almost six months. Let's push hard to publish the >>>> first >>>> >>>> Sedona >>>> >>>>>> release to Maven Central and PyPI before Christmas. In order to >>>> make >>>> >> it >>>> >>>>>> happen, >>>> >>>>>> >>>> >>>>>> Finalize coding and documentation before Dec 6: >>>> >>>>>> 1. I believe the Move-to-JTS PR will be done in around one week. >>>> >>>>>> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if >>>> necessary >>>> >>>>>> 3. I will work on Sedona documentation. >>>> >>>>>> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala >>>> 2.11. >>>> >> I >>>> >>>> will >>>> >>>>>> first create a branch for it to illustrate some necessary >>>> changes in >>>> >>>> Sedona >>>> >>>>>> SQL for Spark 2.4. >>>> >>>>>> >>>> >>>>>> Final walk-through before Dec 13 >>>> >>>>>> 1. Netanel can test the release management for Sedona. >>>> >>>>>> 2. Other committers can go through the docs, release notes >>>> >>>>>> >>>> >>>>>> Community voting before Dec
Re: First Sedona release
Hi Netanel, So for Sedona SQL 1.0.0 on Spark 2.4, we can do "sedona-sql_2.11-2.4-1.0.0-incubator" , right? Sedona 1.0 on Spark 2.4 and 3.0 will be compiled against Scala 2.11 and 2.12. I believe this can be done via different compilation target in Maven. I am currently looking at whether I can do conditional compilation using Maven (similar to C++ #ifdef) because there is a change in Aggregator in Spark 3.0. Otherwise I always need to maintain a separate branch for Sedona on Spark 2.4 It looks OK to me. On Fri, Dec 4, 2020 at 1:12 AM Netanel Malka wrote: > Hi, > I think that we can follow the Apache Spark convention as you can see here > <https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.0.1/>. > For example: > sedona-sql_2.11-2.4, where 2.11 -> scala version and 2.4 -> spark version > > What do you think? > > > On Fri, 4 Dec 2020 at 10:34, Jia Yu wrote: > >> Dear all, >> >> The current status: >> 1. Move to JTS PR has been merged to the master branch. If JTS 1.18 gets >> published in a few weeks, we will use the latest JTS. Otherwise, we still >> need to use my fork for this release. But Sedona API is now finalized. From >> the user perspective, use my fork or JTS official release should not make >> any difference. >> 2. Sedona doc update is in progress. I am half way there. You can track >> the progress here: https://github.com/apache/incubator-sedona/pull/493 >> 3. I will create a separate branch to test Spark 2.4 over this weekend. >> 4. Pawel is working on his improvement on RDD-SQL Python adapter. >> >> Question: >> >> What is the most appropriate maven artifact name for Sedona on Spark 2.4? >> I used to put "sedona-sql_2.4". But it looks like "_2.4" is usually >> reserved for specifying the Scala version. How about "sedona-sql-spark2"? >> Should we also use "sedona-sql-spark3" for Spark 3.0? >> >> Thanks, >> Jia >> >> On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes wrote: >> >>> Hi all, >>> >>> Felix, good to know that a WIP disclaimer is standard practice and will >>> let things move forward! >>> >>> Jia, I believe that page is explaining that a portion of the code in >>> various GeoTools modules has other licenses on it. As such, gt-main is >>> mostly LGPL with some BSD code as well. >>> >>> Cheers, >>> >>> Jim >>> >>> On 11/23/2020 9:50 PM, Jia Yu wrote: >>> > Thank you, Felix. I will use the WIP disclaimer. >>> > >>> > To answer Jim's question, GeoTools components use different licenses: >>> > https://docs.geotools.org/latest/userguide/welcome/license.html >>> > >>> > GT-main uses BSD, so its binary can be included in Sedona's release. >>> > Other components in GeoTools use LGPL, but Sedona only uses them for >>> CRS >>> > transformation. I already set the dependency scope to "provided" in >>> > Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, >>> they >>> > will have to add some GeoTools library by themselves. >>> > >>> > >>> > On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung >>> wrote: >>> > >>> >> On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung >>> >> wrote: >>> >> >>> >>> I’d strongly recommend the community to move towards the first >>> release >>> >>> with the WIP disclaimer >>> >>> >>> >>> >>> >> >>> https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer >>> >>> https://incubator.apache.org/policy/incubation.html#releases >>> >>> >>> >>> >>> >>> As for the LGPL dependency specifically, a replacement will be >>> needed? >>> >>> >>> >> >>> >> To clarify, ok to note in the WIP disclaimer- so it can be released >>> with >>> >> this. >>> >> >>> >> >>> >> >>> >>> On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes >>> wrote: >>> >>> >>> >>>> Hi all, >>> >>>> >>> >>>> Has the fact that one of the dependencies is LGPL (GeoTools) been >>> >>>> discussed / addressed? (See >>> >>>> https://www.apache.org/legal/resolved.html#category-x) >>> >>>&g
Re: First Sedona release
Hi, I think that we can follow the Apache Spark convention as you can see here <https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.0.1/>. For example: sedona-sql_2.11-2.4, where 2.11 -> scala version and 2.4 -> spark version What do you think? On Fri, 4 Dec 2020 at 10:34, Jia Yu wrote: > Dear all, > > The current status: > 1. Move to JTS PR has been merged to the master branch. If JTS 1.18 gets > published in a few weeks, we will use the latest JTS. Otherwise, we still > need to use my fork for this release. But Sedona API is now finalized. From > the user perspective, use my fork or JTS official release should not make > any difference. > 2. Sedona doc update is in progress. I am half way there. You can track > the progress here: https://github.com/apache/incubator-sedona/pull/493 > 3. I will create a separate branch to test Spark 2.4 over this weekend. > 4. Pawel is working on his improvement on RDD-SQL Python adapter. > > Question: > > What is the most appropriate maven artifact name for Sedona on Spark 2.4? > I used to put "sedona-sql_2.4". But it looks like "_2.4" is usually > reserved for specifying the Scala version. How about "sedona-sql-spark2"? > Should we also use "sedona-sql-spark3" for Spark 3.0? > > Thanks, > Jia > > On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes wrote: > >> Hi all, >> >> Felix, good to know that a WIP disclaimer is standard practice and will >> let things move forward! >> >> Jia, I believe that page is explaining that a portion of the code in >> various GeoTools modules has other licenses on it. As such, gt-main is >> mostly LGPL with some BSD code as well. >> >> Cheers, >> >> Jim >> >> On 11/23/2020 9:50 PM, Jia Yu wrote: >> > Thank you, Felix. I will use the WIP disclaimer. >> > >> > To answer Jim's question, GeoTools components use different licenses: >> > https://docs.geotools.org/latest/userguide/welcome/license.html >> > >> > GT-main uses BSD, so its binary can be included in Sedona's release. >> > Other components in GeoTools use LGPL, but Sedona only uses them for CRS >> > transformation. I already set the dependency scope to "provided" in >> > Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, >> they >> > will have to add some GeoTools library by themselves. >> > >> > >> > On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung >> wrote: >> > >> >> On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung >> >> wrote: >> >> >> >>> I’d strongly recommend the community to move towards the first release >> >>> with the WIP disclaimer >> >>> >> >>> >> >> >> https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer >> >>> https://incubator.apache.org/policy/incubation.html#releases >> >>> >> >>> >> >>> As for the LGPL dependency specifically, a replacement will be needed? >> >>> >> >> >> >> To clarify, ok to note in the WIP disclaimer- so it can be released >> with >> >> this. >> >> >> >> >> >> >> >>> On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: >> >>> >> >>>> Hi all, >> >>>> >> >>>> Has the fact that one of the dependencies is LGPL (GeoTools) been >> >>>> discussed / addressed? (See >> >>>> https://www.apache.org/legal/resolved.html#category-x) >> >>>> >> >>>> I'm asking since I don't know if the ASF has any recommended work >> >>>> arounds for shipping code with licenses that it does not approve of. >> >>>> >> >>>> Cheers, >> >>>> >> >>>> Jim >> >>>> >> >>>> On 11/23/20 1:41 PM, Felix Cheung wrote: >> >>>>> I can help review around Dev 13 to give a first pass. It should give >> >>>> you an >> >>>>> easier path to IPMC vote. >> >>>>> >> >>>>> >> >>>>> On Sun, Nov 22, 2020 at 10:50 PM Jia Yu >> >> wrote: >> >>>>>> Hi Pawel and everyone, >> >>>>>> >> >>>>>> Let's do this in the first Sedona release. But can you please first >> >>>> fix the >> >>>>>> Python A
Re: First Sedona release
Dear all, The current status: 1. Move to JTS PR has been merged to the master branch. If JTS 1.18 gets published in a few weeks, we will use the latest JTS. Otherwise, we still need to use my fork for this release. But Sedona API is now finalized. From the user perspective, use my fork or JTS official release should not make any difference. 2. Sedona doc update is in progress. I am half way there. You can track the progress here: https://github.com/apache/incubator-sedona/pull/493 3. I will create a separate branch to test Spark 2.4 over this weekend. 4. Pawel is working on his improvement on RDD-SQL Python adapter. Question: What is the most appropriate maven artifact name for Sedona on Spark 2.4? I used to put "sedona-sql_2.4". But it looks like "_2.4" is usually reserved for specifying the Scala version. How about "sedona-sql-spark2"? Should we also use "sedona-sql-spark3" for Spark 3.0? Thanks, Jia On Tue, Nov 24, 2020 at 8:16 AM Jim Hughes wrote: > Hi all, > > Felix, good to know that a WIP disclaimer is standard practice and will > let things move forward! > > Jia, I believe that page is explaining that a portion of the code in > various GeoTools modules has other licenses on it. As such, gt-main is > mostly LGPL with some BSD code as well. > > Cheers, > > Jim > > On 11/23/2020 9:50 PM, Jia Yu wrote: > > Thank you, Felix. I will use the WIP disclaimer. > > > > To answer Jim's question, GeoTools components use different licenses: > > https://docs.geotools.org/latest/userguide/welcome/license.html > > > > GT-main uses BSD, so its binary can be included in Sedona's release. > > Other components in GeoTools use LGPL, but Sedona only uses them for CRS > > transformation. I already set the dependency scope to "provided" in > > Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, > they > > will have to add some GeoTools library by themselves. > > > > > > On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung > wrote: > > > >> On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung > >> wrote: > >> > >>> I’d strongly recommend the community to move towards the first release > >>> with the WIP disclaimer > >>> > >>> > >> > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > >>> https://incubator.apache.org/policy/incubation.html#releases > >>> > >>> > >>> As for the LGPL dependency specifically, a replacement will be needed? > >>> > >> > >> To clarify, ok to note in the WIP disclaimer- so it can be released with > >> this. > >> > >> > >> > >>> On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > >>> > >>>> Hi all, > >>>> > >>>> Has the fact that one of the dependencies is LGPL (GeoTools) been > >>>> discussed / addressed? (See > >>>> https://www.apache.org/legal/resolved.html#category-x) > >>>> > >>>> I'm asking since I don't know if the ASF has any recommended work > >>>> arounds for shipping code with licenses that it does not approve of. > >>>> > >>>> Cheers, > >>>> > >>>> Jim > >>>> > >>>> On 11/23/20 1:41 PM, Felix Cheung wrote: > >>>>> I can help review around Dev 13 to give a first pass. It should give > >>>> you an > >>>>> easier path to IPMC vote. > >>>>> > >>>>> > >>>>> On Sun, Nov 22, 2020 at 10:50 PM Jia Yu > >> wrote: > >>>>>> Hi Pawel and everyone, > >>>>>> > >>>>>> Let's do this in the first Sedona release. But can you please first > >>>> fix the > >>>>>> Python API for our Move-to-JTS PR, and then work on this one? If > this > >>>>>> Python RDD-DF Adapter PR might slow down our progress of releasing > >>>> Sedona > >>>>>> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > >>>>>> > >>>>>> @everyone > >>>>>> Our top priority is to draw the first Sedona release ASAP. Users > have > >>>> been > >>>>>> waiting for almost six months. Let's push hard to publish the first > >>>> Sedona > >>>>>> release to Maven Central and PyPI before Christmas. In order to make > >> it > >>>>
Re: First Sedona release
Hi all, Felix, good to know that a WIP disclaimer is standard practice and will let things move forward! Jia, I believe that page is explaining that a portion of the code in various GeoTools modules has other licenses on it. As such, gt-main is mostly LGPL with some BSD code as well. Cheers, Jim On 11/23/2020 9:50 PM, Jia Yu wrote: Thank you, Felix. I will use the WIP disclaimer. To answer Jim's question, GeoTools components use different licenses: https://docs.geotools.org/latest/userguide/welcome/license.html GT-main uses BSD, so its binary can be included in Sedona's release. Other components in GeoTools use LGPL, but Sedona only uses them for CRS transformation. I already set the dependency scope to "provided" in Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, they will have to add some GeoTools library by themselves. On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung wrote: On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung wrote: I’d strongly recommend the community to move towards the first release with the WIP disclaimer https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer https://incubator.apache.org/policy/incubation.html#releases As for the LGPL dependency specifically, a replacement will be needed? To clarify, ok to note in the WIP disclaimer- so it can be released with this. On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: Hi all, Has the fact that one of the dependencies is LGPL (GeoTools) been discussed / addressed? (See https://www.apache.org/legal/resolved.html#category-x) I'm asking since I don't know if the ASF has any recommended work arounds for shipping code with licenses that it does not approve of. Cheers, Jim On 11/23/20 1:41 PM, Felix Cheung wrote: I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < pawel93kocin...@gmail.com> wrote: Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external storage Currently SerDe between jvm and Python causes additional time needed to compute the result. I have a local branch with tests where this functionality is available (need 3-4 days to make it 100% ready), in two above scenarios there will be almost no difference between Python and Scala or Java API. Should I create PR to include this feature within the first Sedona release ? Regards, Paweł pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrot
Re: First Sedona release
Thx Jia. Then it falls under optional use https://www.apache.org/legal/resolved.html#optional Which is totally fine. On Mon, Nov 23, 2020 at 6:50 PM Jia Yu wrote: > Thank you, Felix. I will use the WIP disclaimer. > > To answer Jim's question, GeoTools components use different licenses: > https://docs.geotools.org/latest/userguide/welcome/license.html > > GT-main uses BSD, so its binary can be included in Sedona's release. > Other components in GeoTools use LGPL, but Sedona only uses them for CRS > transformation. I already set the dependency scope to "provided" in > Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, they > will have to add some GeoTools library by themselves. > > > On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung > wrote: > > > On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung > > wrote: > > > > > I’d strongly recommend the community to move towards the first release > > > with the WIP disclaimer > > > > > > > > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > > > > > https://incubator.apache.org/policy/incubation.html#releases > > > > > > > > > As for the LGPL dependency specifically, a replacement will be needed? > > > > > > > > > To clarify, ok to note in the WIP disclaimer- so it can be released with > > this. > > > > > > > > > > > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > > > > > >> Hi all, > > >> > > >> Has the fact that one of the dependencies is LGPL (GeoTools) been > > >> discussed / addressed? (See > > >> https://www.apache.org/legal/resolved.html#category-x) > > >> > > >> I'm asking since I don't know if the ASF has any recommended work > > >> arounds for shipping code with licenses that it does not approve of. > > >> > > >> Cheers, > > >> > > >> Jim > > >> > > >> On 11/23/20 1:41 PM, Felix Cheung wrote: > > >> > I can help review around Dev 13 to give a first pass. It should give > > >> you an > > >> > easier path to IPMC vote. > > >> > > > >> > > > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu > > wrote: > > >> > > > >> >> Hi Pawel and everyone, > > >> >> > > >> >> Let's do this in the first Sedona release. But can you please first > > >> fix the > > >> >> Python API for our Move-to-JTS PR, and then work on this one? If > this > > >> >> Python RDD-DF Adapter PR might slow down our progress of releasing > > >> Sedona > > >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > > >> >> > > >> >> @everyone > > >> >> Our top priority is to draw the first Sedona release ASAP. Users > have > > >> been > > >> >> waiting for almost six months. Let's push hard to publish the first > > >> Sedona > > >> >> release to Maven Central and PyPI before Christmas. In order to > make > > it > > >> >> happen, > > >> >> > > >> >> Finalize coding and documentation before Dec 6: > > >> >> 1. I believe the Move-to-JTS PR will be done in around one week. > > >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > > >> >> 3. I will work on Sedona documentation. > > >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala > 2.11. > > I > > >> will > > >> >> first create a branch for it to illustrate some necessary changes > in > > >> Sedona > > >> >> SQL for Spark 2.4. > > >> >> > > >> >> Final walk-through before Dec 13 > > >> >> 1. Netanel can test the release management for Sedona. > > >> >> 2. Other committers can go through the docs, release notes > > >> >> > > >> >> Community voting before Dec 20 > > >> >> 1. Sedona community voting: before Dec 16 > > >> >> 2. Apache Incubator voting: before Dec 20 > > >> >> > > >> >> Push to Maven Central and PyPi before Dec 24 > > >> >> > > >> >> Please feel free to comment if you have any suggestions! > > >> &g
Re: First Sedona release
Thank you, Felix. I will use the WIP disclaimer. To answer Jim's question, GeoTools components use different licenses: https://docs.geotools.org/latest/userguide/welcome/license.html GT-main uses BSD, so its binary can be included in Sedona's release. Other components in GeoTools use LGPL, but Sedona only uses them for CRS transformation. I already set the dependency scope to "provided" in Sedona's POM.xml. If a user wants to use CRS transformation in Sedona, they will have to add some GeoTools library by themselves. On Mon, Nov 23, 2020 at 6:24 PM Felix Cheung wrote: > On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung > wrote: > > > I’d strongly recommend the community to move towards the first release > > with the WIP disclaimer > > > > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > > > https://incubator.apache.org/policy/incubation.html#releases > > > > > > As for the LGPL dependency specifically, a replacement will be needed? > > > > > To clarify, ok to note in the WIP disclaimer- so it can be released with > this. > > > > > > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > > > >> Hi all, > >> > >> Has the fact that one of the dependencies is LGPL (GeoTools) been > >> discussed / addressed? (See > >> https://www.apache.org/legal/resolved.html#category-x) > >> > >> I'm asking since I don't know if the ASF has any recommended work > >> arounds for shipping code with licenses that it does not approve of. > >> > >> Cheers, > >> > >> Jim > >> > >> On 11/23/20 1:41 PM, Felix Cheung wrote: > >> > I can help review around Dev 13 to give a first pass. It should give > >> you an > >> > easier path to IPMC vote. > >> > > >> > > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu > wrote: > >> > > >> >> Hi Pawel and everyone, > >> >> > >> >> Let's do this in the first Sedona release. But can you please first > >> fix the > >> >> Python API for our Move-to-JTS PR, and then work on this one? If this > >> >> Python RDD-DF Adapter PR might slow down our progress of releasing > >> Sedona > >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > >> >> > >> >> @everyone > >> >> Our top priority is to draw the first Sedona release ASAP. Users have > >> been > >> >> waiting for almost six months. Let's push hard to publish the first > >> Sedona > >> >> release to Maven Central and PyPI before Christmas. In order to make > it > >> >> happen, > >> >> > >> >> Finalize coding and documentation before Dec 6: > >> >> 1. I believe the Move-to-JTS PR will be done in around one week. > >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > >> >> 3. I will work on Sedona documentation. > >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. > I > >> will > >> >> first create a branch for it to illustrate some necessary changes in > >> Sedona > >> >> SQL for Spark 2.4. > >> >> > >> >> Final walk-through before Dec 13 > >> >> 1. Netanel can test the release management for Sedona. > >> >> 2. Other committers can go through the docs, release notes > >> >> > >> >> Community voting before Dec 20 > >> >> 1. Sedona community voting: before Dec 16 > >> >> 2. Apache Incubator voting: before Dec 20 > >> >> > >> >> Push to Maven Central and PyPi before Dec 24 > >> >> > >> >> Please feel free to comment if you have any suggestions! > >> >> > >> >> Jia > >> >> > >> >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < > >> pawel93kocin...@gmail.com> > >> >> wrote: > >> >> > >> >>> Hi, > >> >>> I saw some users reported need to improve Python RDD API in two > >> >> scenarios: > >> >>> - converting spatial flat join result to df > >> >>> - saving spatial flat join result directly to external storage > >> >>> > >> >>> Currently SerDe between jvm and Python causes additional time needed > >> to > >> >>> c
Re: First Sedona release
On Mon, Nov 23, 2020 at 6:03 PM Felix Cheung wrote: > I’d strongly recommend the community to move towards the first release > with the WIP disclaimer > > https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer > > https://incubator.apache.org/policy/incubation.html#releases > > > As for the LGPL dependency specifically, a replacement will be needed? > To clarify, ok to note in the WIP disclaimer- so it can be released with this. > > On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > >> Hi all, >> >> Has the fact that one of the dependencies is LGPL (GeoTools) been >> discussed / addressed? (See >> https://www.apache.org/legal/resolved.html#category-x) >> >> I'm asking since I don't know if the ASF has any recommended work >> arounds for shipping code with licenses that it does not approve of. >> >> Cheers, >> >> Jim >> >> On 11/23/20 1:41 PM, Felix Cheung wrote: >> > I can help review around Dev 13 to give a first pass. It should give >> you an >> > easier path to IPMC vote. >> > >> > >> > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: >> > >> >> Hi Pawel and everyone, >> >> >> >> Let's do this in the first Sedona release. But can you please first >> fix the >> >> Python API for our Move-to-JTS PR, and then work on this one? If this >> >> Python RDD-DF Adapter PR might slow down our progress of releasing >> Sedona >> >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. >> >> >> >> @everyone >> >> Our top priority is to draw the first Sedona release ASAP. Users have >> been >> >> waiting for almost six months. Let's push hard to publish the first >> Sedona >> >> release to Maven Central and PyPI before Christmas. In order to make it >> >> happen, >> >> >> >> Finalize coding and documentation before Dec 6: >> >> 1. I believe the Move-to-JTS PR will be done in around one week. >> >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary >> >> 3. I will work on Sedona documentation. >> >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I >> will >> >> first create a branch for it to illustrate some necessary changes in >> Sedona >> >> SQL for Spark 2.4. >> >> >> >> Final walk-through before Dec 13 >> >> 1. Netanel can test the release management for Sedona. >> >> 2. Other committers can go through the docs, release notes >> >> >> >> Community voting before Dec 20 >> >> 1. Sedona community voting: before Dec 16 >> >> 2. Apache Incubator voting: before Dec 20 >> >> >> >> Push to Maven Central and PyPi before Dec 24 >> >> >> >> Please feel free to comment if you have any suggestions! >> >> >> >> Jia >> >> >> >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < >> pawel93kocin...@gmail.com> >> >> wrote: >> >> >> >>> Hi, >> >>> I saw some users reported need to improve Python RDD API in two >> >> scenarios: >> >>> - converting spatial flat join result to df >> >>> - saving spatial flat join result directly to external storage >> >>> >> >>> Currently SerDe between jvm and Python causes additional time needed >> to >> >>> compute the result. I have a local branch with tests where this >> >>> functionality is available (need 3-4 days to make it 100% ready), in >> two >> >>> above scenarios there will be almost no difference between Python and >> >> Scala >> >>> or Java API. Should I create PR to include this feature within the >> first >> >>> Sedona release ? >> >>> Regards, >> >>> Paweł >> >>> >> >>> pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): >> >>> >> >>>> Dear all, >> >>>> >> >>>> Thanks for all your suggestions. >> >>>> >> >>>> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR >> >> and >> >>>> two JTS PRs. @Jim Hughes , @Paweł Kociński >> >>>> , I, and probably Martin from JTS will >> take >> >>>> care of these PRs in the coming days. >> >>>> (1) Sedona
Re: First Sedona release
I’d strongly recommend the community to move towards the first release with the WIP disclaimer https://incubator.apache.org/policy/incubation.html#work_in_progress_disclaimer https://incubator.apache.org/policy/incubation.html#releases As for the LGPL dependency specifically, a replacement will be needed? On Mon, Nov 23, 2020 at 11:15 AM Jim Hughes wrote: > Hi all, > > Has the fact that one of the dependencies is LGPL (GeoTools) been > discussed / addressed? (See > https://www.apache.org/legal/resolved.html#category-x) > > I'm asking since I don't know if the ASF has any recommended work > arounds for shipping code with licenses that it does not approve of. > > Cheers, > > Jim > > On 11/23/20 1:41 PM, Felix Cheung wrote: > > I can help review around Dev 13 to give a first pass. It should give you > an > > easier path to IPMC vote. > > > > > > On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: > > > >> Hi Pawel and everyone, > >> > >> Let's do this in the first Sedona release. But can you please first fix > the > >> Python API for our Move-to-JTS PR, and then work on this one? If this > >> Python RDD-DF Adapter PR might slow down our progress of releasing > Sedona > >> before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > >> > >> @everyone > >> Our top priority is to draw the first Sedona release ASAP. Users have > been > >> waiting for almost six months. Let's push hard to publish the first > Sedona > >> release to Maven Central and PyPI before Christmas. In order to make it > >> happen, > >> > >> Finalize coding and documentation before Dec 6: > >> 1. I believe the Move-to-JTS PR will be done in around one week. > >> 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > >> 3. I will work on Sedona documentation. > >> 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I > will > >> first create a branch for it to illustrate some necessary changes in > Sedona > >> SQL for Spark 2.4. > >> > >> Final walk-through before Dec 13 > >> 1. Netanel can test the release management for Sedona. > >> 2. Other committers can go through the docs, release notes > >> > >> Community voting before Dec 20 > >> 1. Sedona community voting: before Dec 16 > >> 2. Apache Incubator voting: before Dec 20 > >> > >> Push to Maven Central and PyPi before Dec 24 > >> > >> Please feel free to comment if you have any suggestions! > >> > >> Jia > >> > >> On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński < > pawel93kocin...@gmail.com> > >> wrote: > >> > >>> Hi, > >>> I saw some users reported need to improve Python RDD API in two > >> scenarios: > >>> - converting spatial flat join result to df > >>> - saving spatial flat join result directly to external storage > >>> > >>> Currently SerDe between jvm and Python causes additional time needed to > >>> compute the result. I have a local branch with tests where this > >>> functionality is available (need 3-4 days to make it 100% ready), in > two > >>> above scenarios there will be almost no difference between Python and > >> Scala > >>> or Java API. Should I create PR to include this feature within the > first > >>> Sedona release ? > >>> Regards, > >>> Paweł > >>> > >>> pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): > >>> > >>>> Dear all, > >>>> > >>>> Thanks for all your suggestions. > >>>> > >>>> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR > >> and > >>>> two JTS PRs. @Jim Hughes , @Paweł Kociński > >>>> , I, and probably Martin from JTS will > take > >>>> care of these PRs in the coming days. > >>>> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > >>>> (2) JTS PR: https://github.com/locationtech/jts/pull/633 > >>>> https://github.com/locationtech/jts/pull/634 > >>>> > >>>> 2. To move forward with the first release, I have deleted the > "SNAPSHOT" > >>>> in my JTS 1.16 fork. > >>>> Most likely, we have to move forward with my JTS 1.16 fork in the > first > >>>> Sedona release because of the conflict among JTStoGeoJSON, GeoTools, > and > >>
Re: First Sedona release
Hi all, Has the fact that one of the dependencies is LGPL (GeoTools) been discussed / addressed? (See https://www.apache.org/legal/resolved.html#category-x) I'm asking since I don't know if the ASF has any recommended work arounds for shipping code with licenses that it does not approve of. Cheers, Jim On 11/23/20 1:41 PM, Felix Cheung wrote: I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński wrote: Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external storage Currently SerDe between jvm and Python causes additional time needed to compute the result. I have a local branch with tests where this functionality is available (need 3-4 days to make it 100% ready), in two above scenarios there will be almost no difference between Python and Scala or Java API. Should I create PR to include this feature within the first Sedona release ? Regards, Paweł pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: Hi Mo, I can definitely help. The first step will be for Jia to push a PR for the JTS changes. (Since they are his changes, I cannot do this on his behalf.) From talking to the lead JTS developer, he wanted to see the previous PR (from months/a year+ ago) split up. I think the initial PR should be used to discuss what changes are sensible for JTS and where we'll need to push some of the changes to Sedona. Concretely, I noticed that the Sedona JTS fork changes the toString on Geometry to include printing out the userData. I imagine that may cause trouble for downstream JTS users, so it'd be good to find an alternative. One suggestion would to be add a static method in Sedona for printing a Geometry with its userData object. Cheers, Jim On 11/12/20 12:32 PM, Mohamed Sarwat wrote: Folks, I totally agree with Jim on that. Jim, would you like to take the lead on that - I trust that you can bring this task to completion. Jia, would you please let us know how we can incorporate the changes into the JTS master branch? Thanks, On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: Hi all, As a JTS committer, I have tried to request that the Sedona project discuss the desired changes to JTS previously. I'd still encourage that. JTS is an active project and I feel that maintaining a fork of JTS is unnecessary and inappropriate. Cheers, Jim On 11/11/20 9:04 PM, Felix Cheung wrote: Ah. You will need to publish it in order for the dependency chain to work on Mav
Re: First Sedona release
I can help review around Dev 13 to give a first pass. It should give you an easier path to IPMC vote. On Sun, Nov 22, 2020 at 10:50 PM Jia Yu wrote: > Hi Pawel and everyone, > > Let's do this in the first Sedona release. But can you please first fix the > Python API for our Move-to-JTS PR, and then work on this one? If this > Python RDD-DF Adapter PR might slow down our progress of releasing Sedona > before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. > > @everyone > Our top priority is to draw the first Sedona release ASAP. Users have been > waiting for almost six months. Let's push hard to publish the first Sedona > release to Maven Central and PyPI before Christmas. In order to make it > happen, > > Finalize coding and documentation before Dec 6: > 1. I believe the Move-to-JTS PR will be done in around one week. > 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary > 3. I will work on Sedona documentation. > 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will > first create a branch for it to illustrate some necessary changes in Sedona > SQL for Spark 2.4. > > Final walk-through before Dec 13 > 1. Netanel can test the release management for Sedona. > 2. Other committers can go through the docs, release notes > > Community voting before Dec 20 > 1. Sedona community voting: before Dec 16 > 2. Apache Incubator voting: before Dec 20 > > Push to Maven Central and PyPi before Dec 24 > > Please feel free to comment if you have any suggestions! > > Jia > > On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński > wrote: > > > Hi, > > I saw some users reported need to improve Python RDD API in two > scenarios: > > - converting spatial flat join result to df > > - saving spatial flat join result directly to external storage > > > > Currently SerDe between jvm and Python causes additional time needed to > > compute the result. I have a local branch with tests where this > > functionality is available (need 3-4 days to make it 100% ready), in two > > above scenarios there will be almost no difference between Python and > Scala > > or Java API. Should I create PR to include this feature within the first > > Sedona release ? > > Regards, > > Paweł > > > > pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): > > > >> Dear all, > >> > >> Thanks for all your suggestions. > >> > >> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR > and > >> two JTS PRs. @Jim Hughes , @Paweł Kociński > >> , I, and probably Martin from JTS will take > >> care of these PRs in the coming days. > >> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > >> (2) JTS PR: https://github.com/locationtech/jts/pull/633 > >> https://github.com/locationtech/jts/pull/634 > >> > >> 2. To move forward with the first release, I have deleted the "SNAPSHOT" > >> in my JTS 1.16 fork. > >> Most likely, we have to move forward with my JTS 1.16 fork in the first > >> Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and > >> JTS 1.17. > >> So @Netanel Malka could you please do another > >> dry-run on the Sedona first release on this Sedona branch: > sedona-1.0-doc: > >> https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc > >> > >> Thanks, > >> Jia > >> > >> On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: > >> > >>> Hi Mo, > >>> > >>> I can definitely help. The first step will be for Jia to push a PR for > >>> the JTS changes. (Since they are his changes, I cannot do this on his > >>> behalf.) > >>> > >>> From talking to the lead JTS developer, he wanted to see the previous > >>> PR (from months/a year+ ago) split up. I think the initial PR should > be > >>> used to discuss what changes are sensible for JTS and where we'll need > >>> to push some of the changes to Sedona. > >>> > >>> Concretely, I noticed that the Sedona JTS fork changes the toString on > >>> Geometry to include printing out the userData. I imagine that may > cause > >>> trouble for downstream JTS users, so it'd be good to find an > >>> alternative. One suggestion would to be add a static method in Sedona > >>> for printing a Geometry with its userData object. > >>> > >>> Cheers, > >>> > >>> Jim > >>> > >>> On
Re: First Sedona release
Hi Pawel and everyone, Let's do this in the first Sedona release. But can you please first fix the Python API for our Move-to-JTS PR, and then work on this one? If this Python RDD-DF Adapter PR might slow down our progress of releasing Sedona before Christmas, we can postpone it to Sedona 1.0.1 or 1.1.0. @everyone Our top priority is to draw the first Sedona release ASAP. Users have been waiting for almost six months. Let's push hard to publish the first Sedona release to Maven Central and PyPI before Christmas. In order to make it happen, Finalize coding and documentation before Dec 6: 1. I believe the Move-to-JTS PR will be done in around one week. 2. Then we can accept Pawel' Python RDD-DF Adapter PR, if necessary 3. I will work on Sedona documentation. 4. @Netanel will work on Sedona support of Spark 2.4 and Scala 2.11. I will first create a branch for it to illustrate some necessary changes in Sedona SQL for Spark 2.4. Final walk-through before Dec 13 1. Netanel can test the release management for Sedona. 2. Other committers can go through the docs, release notes Community voting before Dec 20 1. Sedona community voting: before Dec 16 2. Apache Incubator voting: before Dec 20 Push to Maven Central and PyPi before Dec 24 Please feel free to comment if you have any suggestions! Jia On Sun, Nov 22, 2020 at 9:51 AM Paweł Kociński wrote: > Hi, > I saw some users reported need to improve Python RDD API in two scenarios: > - converting spatial flat join result to df > - saving spatial flat join result directly to external storage > > Currently SerDe between jvm and Python causes additional time needed to > compute the result. I have a local branch with tests where this > functionality is available (need 3-4 days to make it 100% ready), in two > above scenarios there will be almost no difference between Python and Scala > or Java API. Should I create PR to include this feature within the first > Sedona release ? > Regards, > Paweł > > pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): > >> Dear all, >> >> Thanks for all your suggestions. >> >> 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and >> two JTS PRs. @Jim Hughes , @Paweł Kociński >> , I, and probably Martin from JTS will take >> care of these PRs in the coming days. >> (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 >> (2) JTS PR: https://github.com/locationtech/jts/pull/633 >> https://github.com/locationtech/jts/pull/634 >> >> 2. To move forward with the first release, I have deleted the "SNAPSHOT" >> in my JTS 1.16 fork. >> Most likely, we have to move forward with my JTS 1.16 fork in the first >> Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and >> JTS 1.17. >> So @Netanel Malka could you please do another >> dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: >> https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc >> >> Thanks, >> Jia >> >> On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: >> >>> Hi Mo, >>> >>> I can definitely help. The first step will be for Jia to push a PR for >>> the JTS changes. (Since they are his changes, I cannot do this on his >>> behalf.) >>> >>> From talking to the lead JTS developer, he wanted to see the previous >>> PR (from months/a year+ ago) split up. I think the initial PR should be >>> used to discuss what changes are sensible for JTS and where we'll need >>> to push some of the changes to Sedona. >>> >>> Concretely, I noticed that the Sedona JTS fork changes the toString on >>> Geometry to include printing out the userData. I imagine that may cause >>> trouble for downstream JTS users, so it'd be good to find an >>> alternative. One suggestion would to be add a static method in Sedona >>> for printing a Geometry with its userData object. >>> >>> Cheers, >>> >>> Jim >>> >>> On 11/12/20 12:32 PM, Mohamed Sarwat wrote: >>> > Folks, >>> > >>> > I totally agree with Jim on that. Jim, would you like to take the lead >>> on that - I trust that you can bring this task to completion. Jia, would >>> you please let us know how we can incorporate the changes into the JTS >>> master branch? >>> > >>> > Thanks, >>> > >>> >> On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: >>> >> >>> >> Hi all, >>> >> >>> >> As a JTS committer, I have tried to request that the Sedona project >>> discuss the desired c
Re: First Sedona release
Hi, I saw some users reported need to improve Python RDD API in two scenarios: - converting spatial flat join result to df - saving spatial flat join result directly to external storage Currently SerDe between jvm and Python causes additional time needed to compute the result. I have a local branch with tests where this functionality is available (need 3-4 days to make it 100% ready), in two above scenarios there will be almost no difference between Python and Scala or Java API. Should I create PR to include this feature within the first Sedona release ? Regards, Paweł pon., 16 lis 2020 o 08:29 Jia Yu napisał(a): > Dear all, > > Thanks for all your suggestions. > > 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and > two JTS PRs. @Jim Hughes , @Paweł Kociński > , I, and probably Martin from JTS will take > care of these PRs in the coming days. > (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > (2) JTS PR: https://github.com/locationtech/jts/pull/633 > https://github.com/locationtech/jts/pull/634 > > 2. To move forward with the first release, I have deleted the "SNAPSHOT" > in my JTS 1.16 fork. > Most likely, we have to move forward with my JTS 1.16 fork in the first > Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and > JTS 1.17. > So @Netanel Malka could you please do another > dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: > https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc > > Thanks, > Jia > > On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: > >> Hi Mo, >> >> I can definitely help. The first step will be for Jia to push a PR for >> the JTS changes. (Since they are his changes, I cannot do this on his >> behalf.) >> >> From talking to the lead JTS developer, he wanted to see the previous >> PR (from months/a year+ ago) split up. I think the initial PR should be >> used to discuss what changes are sensible for JTS and where we'll need >> to push some of the changes to Sedona. >> >> Concretely, I noticed that the Sedona JTS fork changes the toString on >> Geometry to include printing out the userData. I imagine that may cause >> trouble for downstream JTS users, so it'd be good to find an >> alternative. One suggestion would to be add a static method in Sedona >> for printing a Geometry with its userData object. >> >> Cheers, >> >> Jim >> >> On 11/12/20 12:32 PM, Mohamed Sarwat wrote: >> > Folks, >> > >> > I totally agree with Jim on that. Jim, would you like to take the lead >> on that - I trust that you can bring this task to completion. Jia, would >> you please let us know how we can incorporate the changes into the JTS >> master branch? >> > >> > Thanks, >> > >> >> On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: >> >> >> >> Hi all, >> >> >> >> As a JTS committer, I have tried to request that the Sedona project >> discuss the desired changes to JTS previously. I'd still encourage that. >> >> >> >> JTS is an active project and I feel that maintaining a fork of JTS is >> unnecessary and inappropriate. >> >> >> >> Cheers, >> >> >> >> Jim >> >> >> >>> On 11/11/20 9:04 PM, Felix Cheung wrote: >> >>> Ah. You will need to publish it in order for the dependency chain to >> work >> >>> on Maven Central >> >>> >> >>> However, since you are not the project owner there you might need to >> >>> publish that under a different artifact id. >> >>> >> >>> In general, it would be best to avoid hard forking another project >> like >> >>> this. >> >>> >> >>> >> >>>> On Wed, Nov 11, 2020 at 1:05 PM Jia Yu >> wrote: >> >>>> >> >>>> Hi Netanel, >> >>>> >> >>>> That links to this git submodule: >> >>>> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 >> >>>> >> >>>> I can easily fix this by changing the version number here to 1.16.2 >> >>>> excluding "SNAPSHOT": >> >>>> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 >> >>>> >> >>>> Will this solve the problem? >> >>>> >> >>>> On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka >> >>>> wrote: >> >&g
Re: First Sedona release
This is a great cross community collaboration and fantastic outcome. Thank you. On Tue, Nov 17, 2020 at 11:33 AM Jia Yu wrote: > Dear all, > > According to my discussion with JTS committer Jim and Martin, both JTS PRs > could be partially or completely avoided by adopting the following methods: > > 1. For Check UserData in Geometry Equals > https://github.com/locationtech/jts/pull/633 , in Sedona RDD JoinQuery, I > will try to use HashMap as an intermediate step because HashMap allows > self-defined hash key. The new method will be added here: > > https://github.com/apache/incubator-sedona/blob/master/core/src/main/java/org/apache/sedona/core/spatialOperator/JoinQuery.java#L88 > 2. For Change Access Modifier and Add setter and getter > https://github.com/locationtech/jts/pull/634 , I will add a > package-private > RTree/QuadTree class in Sedona, which should be under the same folder of > JTS, to expose the internals of JTS indices. > > If both methods work, I believe we probably will not need to change JTS and > can directly use JTS in Maven Centrl. I will try out the solutions in the > next few days. > > Thanks, > Jia > > On Mon, Nov 16, 2020 at 2:30 PM Jim Hughes wrote: > > > Hi Jia, > > > > Thanks for putting up the PRs. Martin and I have commented on them. If > > you are interested in a more real-time discussion than the PRs, Martin > > and I are both in the JTS Gitter (https://gitter.im/locationtech/jts). > > > > To ask directly, please do not fork JTS. You will be unable to publish > > 1.16.2 artifacts on Maven central. Finding another way to do this will > > cause confusion. > > > > Cheers, > > > > Jim > > > > On 11/16/20 2:28 AM, Jia Yu wrote: > > > Dear all, > > > > > > Thanks for all your suggestions. > > > > > > 1. To completely solve the long-overdue JTS issue, I made a Sedona PR > and > > > two JTS PRs. @Jim Hughes , @Paweł Kociński > > > , I, and probably Martin from JTS will > take > > > care of these PRs in the coming days. > > > (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > > > (2) JTS PR: https://github.com/locationtech/jts/pull/633 > > > https://github.com/locationtech/jts/pull/634 > > > > > > 2. To move forward with the first release, I have deleted the > "SNAPSHOT" > > in > > > my JTS 1.16 fork. > > > Most likely, we have to move forward with my JTS 1.16 fork in the first > > > Sedona release because of the conflict among JTStoGeoJSON, GeoTools, > and > > > JTS 1.17. > > > So @Netanel Malka could you please do another > > > dry-run on the Sedona first release on this Sedona branch: > > sedona-1.0-doc: > > > https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc > > > > > > Thanks, > > > Jia > > > > > > On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: > > > > > >> Hi Mo, > > >> > > >> I can definitely help. The first step will be for Jia to push a PR > for > > >> the JTS changes. (Since they are his changes, I cannot do this on his > > >> behalf.) > > >> > > >> From talking to the lead JTS developer, he wanted to see the > previous > > >> PR (from months/a year+ ago) split up. I think the initial PR should > be > > >> used to discuss what changes are sensible for JTS and where we'll need > > >> to push some of the changes to Sedona. > > >> > > >> Concretely, I noticed that the Sedona JTS fork changes the toString on > > >> Geometry to include printing out the userData. I imagine that may > cause > > >> trouble for downstream JTS users, so it'd be good to find an > > >> alternative. One suggestion would to be add a static method in Sedona > > >> for printing a Geometry with its userData object. > > >> > > >> Cheers, > > >> > > >> Jim > > >> > > >> On 11/12/20 12:32 PM, Mohamed Sarwat wrote: > > >>> Folks, > > >>> > > >>> I totally agree with Jim on that. Jim, would you like to take the > lead > > >> on that - I trust that you can bring this task to completion. Jia, > would > > >> you please let us know how we can incorporate the changes into the JTS > > >> master branch? > > >>> Thanks, > > >>> > > >>>> On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: > > >>>> > > >&
Re: First Sedona release
Dear all, According to my discussion with JTS committer Jim and Martin, both JTS PRs could be partially or completely avoided by adopting the following methods: 1. For Check UserData in Geometry Equals https://github.com/locationtech/jts/pull/633 , in Sedona RDD JoinQuery, I will try to use HashMap as an intermediate step because HashMap allows self-defined hash key. The new method will be added here: https://github.com/apache/incubator-sedona/blob/master/core/src/main/java/org/apache/sedona/core/spatialOperator/JoinQuery.java#L88 2. For Change Access Modifier and Add setter and getter https://github.com/locationtech/jts/pull/634 , I will add a package-private RTree/QuadTree class in Sedona, which should be under the same folder of JTS, to expose the internals of JTS indices. If both methods work, I believe we probably will not need to change JTS and can directly use JTS in Maven Centrl. I will try out the solutions in the next few days. Thanks, Jia On Mon, Nov 16, 2020 at 2:30 PM Jim Hughes wrote: > Hi Jia, > > Thanks for putting up the PRs. Martin and I have commented on them. If > you are interested in a more real-time discussion than the PRs, Martin > and I are both in the JTS Gitter (https://gitter.im/locationtech/jts). > > To ask directly, please do not fork JTS. You will be unable to publish > 1.16.2 artifacts on Maven central. Finding another way to do this will > cause confusion. > > Cheers, > > Jim > > On 11/16/20 2:28 AM, Jia Yu wrote: > > Dear all, > > > > Thanks for all your suggestions. > > > > 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and > > two JTS PRs. @Jim Hughes , @Paweł Kociński > > , I, and probably Martin from JTS will take > > care of these PRs in the coming days. > > (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 > > (2) JTS PR: https://github.com/locationtech/jts/pull/633 > > https://github.com/locationtech/jts/pull/634 > > > > 2. To move forward with the first release, I have deleted the "SNAPSHOT" > in > > my JTS 1.16 fork. > > Most likely, we have to move forward with my JTS 1.16 fork in the first > > Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and > > JTS 1.17. > > So @Netanel Malka could you please do another > > dry-run on the Sedona first release on this Sedona branch: > sedona-1.0-doc: > > https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc > > > > Thanks, > > Jia > > > > On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: > > > >> Hi Mo, > >> > >> I can definitely help. The first step will be for Jia to push a PR for > >> the JTS changes. (Since they are his changes, I cannot do this on his > >> behalf.) > >> > >> From talking to the lead JTS developer, he wanted to see the previous > >> PR (from months/a year+ ago) split up. I think the initial PR should be > >> used to discuss what changes are sensible for JTS and where we'll need > >> to push some of the changes to Sedona. > >> > >> Concretely, I noticed that the Sedona JTS fork changes the toString on > >> Geometry to include printing out the userData. I imagine that may cause > >> trouble for downstream JTS users, so it'd be good to find an > >> alternative. One suggestion would to be add a static method in Sedona > >> for printing a Geometry with its userData object. > >> > >> Cheers, > >> > >> Jim > >> > >> On 11/12/20 12:32 PM, Mohamed Sarwat wrote: > >>> Folks, > >>> > >>> I totally agree with Jim on that. Jim, would you like to take the lead > >> on that - I trust that you can bring this task to completion. Jia, would > >> you please let us know how we can incorporate the changes into the JTS > >> master branch? > >>> Thanks, > >>> > >>>> On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: > >>>> > >>>> Hi all, > >>>> > >>>> As a JTS committer, I have tried to request that the Sedona project > >> discuss the desired changes to JTS previously. I'd still encourage > that. > >>>> JTS is an active project and I feel that maintaining a fork of JTS is > >> unnecessary and inappropriate. > >>>> Cheers, > >>>> > >>>> Jim > >>>> > >>>>> On 11/11/20 9:04 PM, Felix Cheung wrote: > >>>>> Ah. You will need to publish it in order for the dependency chain to > >> work > >>>>> on Maven C
Re: First Sedona release
Hi Jia, Thanks for putting up the PRs. Martin and I have commented on them. If you are interested in a more real-time discussion than the PRs, Martin and I are both in the JTS Gitter (https://gitter.im/locationtech/jts). To ask directly, please do not fork JTS. You will be unable to publish 1.16.2 artifacts on Maven central. Finding another way to do this will cause confusion. Cheers, Jim On 11/16/20 2:28 AM, Jia Yu wrote: Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: Hi Mo, I can definitely help. The first step will be for Jia to push a PR for the JTS changes. (Since they are his changes, I cannot do this on his behalf.) From talking to the lead JTS developer, he wanted to see the previous PR (from months/a year+ ago) split up. I think the initial PR should be used to discuss what changes are sensible for JTS and where we'll need to push some of the changes to Sedona. Concretely, I noticed that the Sedona JTS fork changes the toString on Geometry to include printing out the userData. I imagine that may cause trouble for downstream JTS users, so it'd be good to find an alternative. One suggestion would to be add a static method in Sedona for printing a Geometry with its userData object. Cheers, Jim On 11/12/20 12:32 PM, Mohamed Sarwat wrote: Folks, I totally agree with Jim on that. Jim, would you like to take the lead on that - I trust that you can bring this task to completion. Jia, would you please let us know how we can incorporate the changes into the JTS master branch? Thanks, On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: Hi all, As a JTS committer, I have tried to request that the Sedona project discuss the desired changes to JTS previously. I'd still encourage that. JTS is an active project and I feel that maintaining a fork of JTS is unnecessary and inappropriate. Cheers, Jim On 11/11/20 9:04 PM, Felix Cheung wrote: Ah. You will need to publish it in order for the dependency chain to work on Maven Central However, since you are not the project owner there you might need to publish that under a different artifact id. In general, it would be best to avoid hard forking another project like this. On Wed, Nov 11, 2020 at 1:05 PM Jia Yu wrote: Hi Netanel, That links to this git submodule: https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 I can easily fix this by changing the version number here to 1.16.2 excluding "SNAPSHOT": https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 Will this solve the problem? On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka wrote: Hi Folks, I tried to make a release (dry-run) following by publishing-maven-artifacts <https://infra.apache.org/publishing-maven-artifacts.html>, and I encountered an issue. On sedona-core, we have jts-core as a dependency with the SNAPSHOT version. (link < https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 ) As a prerequisite to the release process, we cannot have dependencies in a SNAPSHOT version. Do you have any clue about how to solve this? On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: OK. Thanks Felix. Updates: * * Opened a ticket for INFRA to Enable Nexus Access For Sedona< https://issues.apache.org/jira/browse/INFRA-21085> * Followed this< https://infra.apache.org/publishing-maven-artifacts.html> guide to test the maven release process * I hope to create a PR soon for adjusting the build to deploy to the ASF Nexus repository * The key that signs the artifacts were created and tested. Do we want to create a candidate release for the current master branch? Netanel Malka, Big Data Consultant [Description: Description: Description: Description: cid:image001.jpg@01C85203.36A2AF30] From: Felix Cheung Sent: Wednesday, November 4, 2020 19:57 To: dev@sedona.apache.org Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi Zhang Subject: Re: First Sedona release 1) No you don’t
Re: First Sedona release
Dear all, Thanks for all your suggestions. 1. To completely solve the long-overdue JTS issue, I made a Sedona PR and two JTS PRs. @Jim Hughes , @Paweł Kociński , I, and probably Martin from JTS will take care of these PRs in the coming days. (1) Sedona PR: https://github.com/apache/incubator-sedona/pull/488 (2) JTS PR: https://github.com/locationtech/jts/pull/633 https://github.com/locationtech/jts/pull/634 2. To move forward with the first release, I have deleted the "SNAPSHOT" in my JTS 1.16 fork. Most likely, we have to move forward with my JTS 1.16 fork in the first Sedona release because of the conflict among JTStoGeoJSON, GeoTools, and JTS 1.17. So @Netanel Malka could you please do another dry-run on the Sedona first release on this Sedona branch: sedona-1.0-doc: https://github.com/apache/incubator-sedona/tree/sedona-1.0-doc Thanks, Jia On Thu, Nov 12, 2020 at 11:36 AM Jim Hughes wrote: > Hi Mo, > > I can definitely help. The first step will be for Jia to push a PR for > the JTS changes. (Since they are his changes, I cannot do this on his > behalf.) > > From talking to the lead JTS developer, he wanted to see the previous > PR (from months/a year+ ago) split up. I think the initial PR should be > used to discuss what changes are sensible for JTS and where we'll need > to push some of the changes to Sedona. > > Concretely, I noticed that the Sedona JTS fork changes the toString on > Geometry to include printing out the userData. I imagine that may cause > trouble for downstream JTS users, so it'd be good to find an > alternative. One suggestion would to be add a static method in Sedona > for printing a Geometry with its userData object. > > Cheers, > > Jim > > On 11/12/20 12:32 PM, Mohamed Sarwat wrote: > > Folks, > > > > I totally agree with Jim on that. Jim, would you like to take the lead > on that - I trust that you can bring this task to completion. Jia, would > you please let us know how we can incorporate the changes into the JTS > master branch? > > > > Thanks, > > > >> On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: > >> > >> Hi all, > >> > >> As a JTS committer, I have tried to request that the Sedona project > discuss the desired changes to JTS previously. I'd still encourage that. > >> > >> JTS is an active project and I feel that maintaining a fork of JTS is > unnecessary and inappropriate. > >> > >> Cheers, > >> > >> Jim > >> > >>> On 11/11/20 9:04 PM, Felix Cheung wrote: > >>> Ah. You will need to publish it in order for the dependency chain to > work > >>> on Maven Central > >>> > >>> However, since you are not the project owner there you might need to > >>> publish that under a different artifact id. > >>> > >>> In general, it would be best to avoid hard forking another project like > >>> this. > >>> > >>> > >>>> On Wed, Nov 11, 2020 at 1:05 PM Jia Yu wrote: > >>>> > >>>> Hi Netanel, > >>>> > >>>> That links to this git submodule: > >>>> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 > >>>> > >>>> I can easily fix this by changing the version number here to 1.16.2 > >>>> excluding "SNAPSHOT": > >>>> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 > >>>> > >>>> Will this solve the problem? > >>>> > >>>> On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka > >>>> wrote: > >>>> > >>>>> Hi Folks, > >>>>> > >>>>> I tried to make a release (dry-run) following by > >>>>> publishing-maven-artifacts > >>>>> <https://infra.apache.org/publishing-maven-artifacts.html>, and I > >>>>> encountered an issue. > >>>>> > >>>>> On sedona-core, we have jts-core as a dependency with the SNAPSHOT > >>>>> version. > >>>>> (link > >>>>> < > >>>>> > https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 > >>>>> ) > >>>>> > >>>>> As a prerequisite to the release process, we cannot have > dependencies in a > >>>>> SNAPSHOT version. > >>>>> > >>>>> > >>>>> Do you have any clue about how to solve this? > >>>>> >
Re: First Sedona release
Hi Mo, I can definitely help. The first step will be for Jia to push a PR for the JTS changes. (Since they are his changes, I cannot do this on his behalf.) From talking to the lead JTS developer, he wanted to see the previous PR (from months/a year+ ago) split up. I think the initial PR should be used to discuss what changes are sensible for JTS and where we'll need to push some of the changes to Sedona. Concretely, I noticed that the Sedona JTS fork changes the toString on Geometry to include printing out the userData. I imagine that may cause trouble for downstream JTS users, so it'd be good to find an alternative. One suggestion would to be add a static method in Sedona for printing a Geometry with its userData object. Cheers, Jim On 11/12/20 12:32 PM, Mohamed Sarwat wrote: Folks, I totally agree with Jim on that. Jim, would you like to take the lead on that - I trust that you can bring this task to completion. Jia, would you please let us know how we can incorporate the changes into the JTS master branch? Thanks, On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: Hi all, As a JTS committer, I have tried to request that the Sedona project discuss the desired changes to JTS previously. I'd still encourage that. JTS is an active project and I feel that maintaining a fork of JTS is unnecessary and inappropriate. Cheers, Jim On 11/11/20 9:04 PM, Felix Cheung wrote: Ah. You will need to publish it in order for the dependency chain to work on Maven Central However, since you are not the project owner there you might need to publish that under a different artifact id. In general, it would be best to avoid hard forking another project like this. On Wed, Nov 11, 2020 at 1:05 PM Jia Yu wrote: Hi Netanel, That links to this git submodule: https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 I can easily fix this by changing the version number here to 1.16.2 excluding "SNAPSHOT": https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 Will this solve the problem? On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka wrote: Hi Folks, I tried to make a release (dry-run) following by publishing-maven-artifacts <https://infra.apache.org/publishing-maven-artifacts.html>, and I encountered an issue. On sedona-core, we have jts-core as a dependency with the SNAPSHOT version. (link < https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 ) As a prerequisite to the release process, we cannot have dependencies in a SNAPSHOT version. Do you have any clue about how to solve this? On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: OK. Thanks Felix. Updates: * * Opened a ticket for INFRA to Enable Nexus Access For Sedona< https://issues.apache.org/jira/browse/INFRA-21085> * Followed this< https://infra.apache.org/publishing-maven-artifacts.html> guide to test the maven release process * I hope to create a PR soon for adjusting the build to deploy to the ASF Nexus repository * The key that signs the artifacts were created and tested. Do we want to create a candidate release for the current master branch? Netanel Malka, Big Data Consultant [Description: Description: Description: Description: cid:image001.jpg@01C85203.36A2AF30] From: Felix Cheung Sent: Wednesday, November 4, 2020 19:57 To: dev@sedona.apache.org Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi Zhang Subject: Re: First Sedona release 1) No you don’t need KEYS file in github only on the release share https://dist.apache.org/repos/dist/dev/incubator/ 2) as podling you add to https://dist.apache.org/repos/dist/dev/incubator/ When you commit via svn you will be able to add a “directory” for Sedona 2a) for release, you basically do a svn rename to move from dev to release “path” 3) if you have java based artifacts, yes. You will publish to Nexus, staging first and when release is signed off, you can click on the interface to make it official, which then automatically sync to Maven central. Here is a script for example that does release signing and publication to Nexus (and staging before release) https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka netanel...@gmail.com>> wrote: Hi, I followed the release-signing <https://infra.apache.org/release-signing.html> doc and created a key for signing and hashing. I have a few questions: 1. Should the KEYS file also be added to the project root directory on Github? ( I saw it in Apache Ant) 2. I saw in release-policy_upload-ci <http://www.apache.org/legal/release-policy.html#upload-ci> that we need to add a release candidate to https://dist.apache.org/repos/dist/*dev*/ /. However, there does not seem to be a directory with Sedona as t
Re: First Sedona release
Folks, I totally agree with Jim on that. Jim, would you like to take the lead on that - I trust that you can bring this task to completion. Jia, would you please let us know how we can incorporate the changes into the JTS master branch? Thanks, > > On Nov 12, 2020, at 10:10 AM, Jim Hughes wrote: > > Hi all, > > As a JTS committer, I have tried to request that the Sedona project discuss > the desired changes to JTS previously. I'd still encourage that. > > JTS is an active project and I feel that maintaining a fork of JTS is > unnecessary and inappropriate. > > Cheers, > > Jim > >> On 11/11/20 9:04 PM, Felix Cheung wrote: >> Ah. You will need to publish it in order for the dependency chain to work >> on Maven Central >> >> However, since you are not the project owner there you might need to >> publish that under a different artifact id. >> >> In general, it would be best to avoid hard forking another project like >> this. >> >> >>> On Wed, Nov 11, 2020 at 1:05 PM Jia Yu wrote: >>> >>> Hi Netanel, >>> >>> That links to this git submodule: >>> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 >>> >>> I can easily fix this by changing the version number here to 1.16.2 >>> excluding "SNAPSHOT": >>> https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 >>> >>> Will this solve the problem? >>> >>> On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka >>> wrote: >>> >>>> Hi Folks, >>>> >>>> I tried to make a release (dry-run) following by >>>> publishing-maven-artifacts >>>> <https://infra.apache.org/publishing-maven-artifacts.html>, and I >>>> encountered an issue. >>>> >>>> On sedona-core, we have jts-core as a dependency with the SNAPSHOT >>>> version. >>>> (link >>>> < >>>> https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 >>> >>>> ) >>>> >>>> As a prerequisite to the release process, we cannot have dependencies in a >>>> SNAPSHOT version. >>>> >>>> >>>> Do you have any clue about how to solve this? >>>> >>>> >>>> On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: >>>> >>>>> OK. Thanks Felix. >>>>> >>>>> >>>>> Updates: >>>>> >>>>> * >>>>> * Opened a ticket for INFRA to Enable Nexus Access For Sedona< >>>>> https://issues.apache.org/jira/browse/INFRA-21085> >>>>> * Followed this< >>>>> https://infra.apache.org/publishing-maven-artifacts.html> guide to test >>>>> the maven release process >>>>> * I hope to create a PR soon for adjusting the build to deploy to >>>> the >>>>> ASF Nexus repository >>>>> * The key that signs the artifacts were created and tested. >>>>> >>>>> Do we want to create a candidate release for the current master branch? >>>>> >>>>> Netanel Malka, >>>>> Big Data Consultant >>>>> [Description: Description: Description: Description: >>>>> cid:image001.jpg@01C85203.36A2AF30] >>>>> >>>>> From: Felix Cheung >>>>> Sent: Wednesday, November 4, 2020 19:57 >>>>> To: dev@sedona.apache.org >>>>> Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi >>>> Zhang >>>>> Subject: Re: First Sedona release >>>>> >>>>> 1) No you don’t need KEYS file in github only on the release share >>>>> https://dist.apache.org/repos/dist/dev/incubator/ >>>>> >>>>> 2) as podling you add to >>>>> https://dist.apache.org/repos/dist/dev/incubator/ >>>>> When you commit via svn you will be able to add a “directory” for Sedona >>>>> >>>>> 2a) for release, you basically do a svn rename to move from dev to >>>> release >>>>> “path” >>>>> >>>>> 3) if you have java based artifacts, yes. You will publish to Nexus, >>>>> staging first and when release is signed off, you can click on the >>>>> interface to make it official, which t
Re: First Sedona release
Hi all, As a JTS committer, I have tried to request that the Sedona project discuss the desired changes to JTS previously. I'd still encourage that. JTS is an active project and I feel that maintaining a fork of JTS is unnecessary and inappropriate. Cheers, Jim On 11/11/20 9:04 PM, Felix Cheung wrote: Ah. You will need to publish it in order for the dependency chain to work on Maven Central However, since you are not the project owner there you might need to publish that under a different artifact id. In general, it would be best to avoid hard forking another project like this. On Wed, Nov 11, 2020 at 1:05 PM Jia Yu wrote: Hi Netanel, That links to this git submodule: https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 I can easily fix this by changing the version number here to 1.16.2 excluding "SNAPSHOT": https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 Will this solve the problem? On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka wrote: Hi Folks, I tried to make a release (dry-run) following by publishing-maven-artifacts <https://infra.apache.org/publishing-maven-artifacts.html>, and I encountered an issue. On sedona-core, we have jts-core as a dependency with the SNAPSHOT version. (link < https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 ) As a prerequisite to the release process, we cannot have dependencies in a SNAPSHOT version. Do you have any clue about how to solve this? On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: OK. Thanks Felix. Updates: * * Opened a ticket for INFRA to Enable Nexus Access For Sedona< https://issues.apache.org/jira/browse/INFRA-21085> * Followed this< https://infra.apache.org/publishing-maven-artifacts.html> guide to test the maven release process * I hope to create a PR soon for adjusting the build to deploy to the ASF Nexus repository * The key that signs the artifacts were created and tested. Do we want to create a candidate release for the current master branch? Netanel Malka, Big Data Consultant [Description: Description: Description: Description: cid:image001.jpg@01C85203.36A2AF30] From: Felix Cheung Sent: Wednesday, November 4, 2020 19:57 To: dev@sedona.apache.org Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi Zhang Subject: Re: First Sedona release 1) No you don’t need KEYS file in github only on the release share https://dist.apache.org/repos/dist/dev/incubator/ 2) as podling you add to https://dist.apache.org/repos/dist/dev/incubator/ When you commit via svn you will be able to add a “directory” for Sedona 2a) for release, you basically do a svn rename to move from dev to release “path” 3) if you have java based artifacts, yes. You will publish to Nexus, staging first and when release is signed off, you can click on the interface to make it official, which then automatically sync to Maven central. Here is a script for example that does release signing and publication to Nexus (and staging before release) https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka netanel...@gmail.com>> wrote: Hi, I followed the release-signing <https://infra.apache.org/release-signing.html> doc and created a key for signing and hashing. I have a few questions: 1. Should the KEYS file also be added to the project root directory on Github? ( I saw it in Apache Ant) 2. I saw in release-policy_upload-ci <http://www.apache.org/legal/release-policy.html#upload-ci> that we need to add a release candidate to https://dist.apache.org/repos/dist/*dev*/ /. However, there does not seem to be a directory with Sedona as the TLP name. How may we be able to get a directory with that name? (Also for the *release*) 3. Do we need to push the artifacts also to ASF Nexus Repository (beside Maven Central)? Thanks. On Mon, 2 Nov 2020 at 19:21, Netanel Malka netanel...@gmail.com>> wrote: Thanks Felix. I would be delighted to help. I can start with the GPG. Can I test it on a some artifact, or I need to wait for the first release? On Mon, 2 Nov 2020 at 03:17, Felix Cheung <mailto:felixche...@apache.org>> wrote: Great progress! To add, A) I’d strongly recommend the WIP disclaimer - it would be much easier to pass with in the first release https://incubator.apache.org/policy/incubation.html#disclaimers B) more info in signing, checksum https://infra.apache.org/release-signing.html C) signing key should be individual’s and (public key ) published and also listed in KEYS file - KEYS file should be located next to the staging (and later release) location, see above D) “correct place” - this is in reference to ASF officIal staging server http://www.apache.org/legal/rele
Re: First Sedona release
Ah. You will need to publish it in order for the dependency chain to work on Maven Central However, since you are not the project owner there you might need to publish that under a different artifact id. In general, it would be best to avoid hard forking another project like this. On Wed, Nov 11, 2020 at 1:05 PM Jia Yu wrote: > Hi Netanel, > > That links to this git submodule: > https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 > > I can easily fix this by changing the version number here to 1.16.2 > excluding "SNAPSHOT": > https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 > > Will this solve the problem? > > On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka > wrote: > >> Hi Folks, >> >> I tried to make a release (dry-run) following by >> publishing-maven-artifacts >> <https://infra.apache.org/publishing-maven-artifacts.html>, and I >> encountered an issue. >> >> On sedona-core, we have jts-core as a dependency with the SNAPSHOT >> version. >> (link >> < >> https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 >> > > > >> ) >> >> As a prerequisite to the release process, we cannot have dependencies in a >> SNAPSHOT version. >> >> >> Do you have any clue about how to solve this? >> >> >> On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: >> >> > OK. Thanks Felix. >> > >> > >> > Updates: >> > >> > * >> > * Opened a ticket for INFRA to Enable Nexus Access For Sedona< >> > https://issues.apache.org/jira/browse/INFRA-21085> >> > * Followed this< >> > https://infra.apache.org/publishing-maven-artifacts.html> guide to test >> > the maven release process >> > * I hope to create a PR soon for adjusting the build to deploy to >> the >> > ASF Nexus repository >> > * The key that signs the artifacts were created and tested. >> > >> > Do we want to create a candidate release for the current master branch? >> > >> > Netanel Malka, >> > Big Data Consultant >> > [Description: Description: Description: Description: >> > cid:image001.jpg@01C85203.36A2AF30] >> > >> > From: Felix Cheung >> > Sent: Wednesday, November 4, 2020 19:57 >> > To: dev@sedona.apache.org >> > Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi >> Zhang >> > Subject: Re: First Sedona release >> > >> > 1) No you don’t need KEYS file in github only on the release share >> > https://dist.apache.org/repos/dist/dev/incubator/ >> > >> > 2) as podling you add to >> > https://dist.apache.org/repos/dist/dev/incubator/ >> > When you commit via svn you will be able to add a “directory” for Sedona >> > >> > 2a) for release, you basically do a svn rename to move from dev to >> release >> > “path” >> > >> > 3) if you have java based artifacts, yes. You will publish to Nexus, >> > staging first and when release is signed off, you can click on the >> > interface to make it official, which then automatically sync to Maven >> > central. >> > >> > Here is a script for example that does release signing and publication >> to >> > Nexus (and staging before release) >> > >> > >> https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh >> > >> > >> > On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka > > > netanel...@gmail.com>> wrote: >> > Hi, >> > >> > I followed the release-signing >> > <https://infra.apache.org/release-signing.html> doc and created a key >> for >> > signing and hashing. >> > >> > I have a few questions: >> > >> >1. Should the KEYS file also be added to the project root directory >> on >> >Github? ( I saw it in Apache Ant) >> >2. I saw in release-policy_upload-ci >> ><http://www.apache.org/legal/release-policy.html#upload-ci> that we >> > need >> >to add a release candidate to >> https://dist.apache.org/repos/dist/*dev*/ >> > > >name>/. However, there does not seem to be a directory with Sedona as >> > the >> >TLP name. How may we be able to get a directory with that name? (Also >> > for >> >the *release*) >> >3. Do we ne
Re: First Sedona release
Hi Netanel, That links to this git submodule: https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 I can easily fix this by changing the version number here to 1.16.2 excluding "SNAPSHOT": https://github.com/jiayuasu/jts/blob/1.16.x/modules/core/pom.xml#L6 Will this solve the problem? On Wed, Nov 11, 2020 at 7:40 AM Netanel Malka wrote: > Hi Folks, > > I tried to make a release (dry-run) following by publishing-maven-artifacts > <https://infra.apache.org/publishing-maven-artifacts.html>, and I > encountered an issue. > > On sedona-core, we have jts-core as a dependency with the SNAPSHOT version. > (link > < > https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37 > > > ) > > As a prerequisite to the release process, we cannot have dependencies in a > SNAPSHOT version. > > > Do you have any clue about how to solve this? > > > On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: > > > OK. Thanks Felix. > > > > > > Updates: > > > > * > > * Opened a ticket for INFRA to Enable Nexus Access For Sedona< > > https://issues.apache.org/jira/browse/INFRA-21085> > > * Followed this< > > https://infra.apache.org/publishing-maven-artifacts.html> guide to test > > the maven release process > > * I hope to create a PR soon for adjusting the build to deploy to the > > ASF Nexus repository > > * The key that signs the artifacts were created and tested. > > > > Do we want to create a candidate release for the current master branch? > > > > Netanel Malka, > > Big Data Consultant > > [Description: Description: Description: Description: > > cid:image001.jpg@01C85203.36A2AF30] > > ____________ > > From: Felix Cheung > > Sent: Wednesday, November 4, 2020 19:57 > > To: dev@sedona.apache.org > > Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi > Zhang > > Subject: Re: First Sedona release > > > > 1) No you don’t need KEYS file in github only on the release share > > https://dist.apache.org/repos/dist/dev/incubator/ > > > > 2) as podling you add to > > https://dist.apache.org/repos/dist/dev/incubator/ > > When you commit via svn you will be able to add a “directory” for Sedona > > > > 2a) for release, you basically do a svn rename to move from dev to > release > > “path” > > > > 3) if you have java based artifacts, yes. You will publish to Nexus, > > staging first and when release is signed off, you can click on the > > interface to make it official, which then automatically sync to Maven > > central. > > > > Here is a script for example that does release signing and publication to > > Nexus (and staging before release) > > > > > https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh > > > > > > On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka > netanel...@gmail.com>> wrote: > > Hi, > > > > I followed the release-signing > > <https://infra.apache.org/release-signing.html> doc and created a key > for > > signing and hashing. > > > > I have a few questions: > > > >1. Should the KEYS file also be added to the project root directory on > >Github? ( I saw it in Apache Ant) > >2. I saw in release-policy_upload-ci > ><http://www.apache.org/legal/release-policy.html#upload-ci> that we > > need > >to add a release candidate to > https://dist.apache.org/repos/dist/*dev*/ > > >name>/. However, there does not seem to be a directory with Sedona as > > the > >TLP name. How may we be able to get a directory with that name? (Also > > for > >the *release*) > >3. Do we need to push the artifacts also to ASF Nexus Repository > (beside > >Maven Central)? > > > > > > Thanks. > > > > On Mon, 2 Nov 2020 at 19:21, Netanel Malka > netanel...@gmail.com>> wrote: > > > > > Thanks Felix. > > > > > > I would be delighted to help. > > > I can start with the GPG. > > > Can I test it on a some artifact, or I need to wait for the first > > release? > > > > > > > > > On Mon, 2 Nov 2020 at 03:17, Felix Cheung > <mailto:felixche...@apache.org>> wrote: > > > > > >> Great progress! > > >> > > >> To add, > > >> A) I’d strongly recommend the WIP disclaimer - it would be much easier > > to > > >> pass with
Re: First Sedona release
Hi Folks, I tried to make a release (dry-run) following by publishing-maven-artifacts <https://infra.apache.org/publishing-maven-artifacts.html>, and I encountered an issue. On sedona-core, we have jts-core as a dependency with the SNAPSHOT version. (link <https://github.com/apache/incubator-sedona/blob/2e60fc07b0eae78ccae3876d970e677fc9319c40/core/pom.xml#L37> ) As a prerequisite to the release process, we cannot have dependencies in a SNAPSHOT version. Do you have any clue about how to solve this? On Mon, 9 Nov 2020 at 21:22, Netanel Malka wrote: > OK. Thanks Felix. > > > Updates: > > * > * Opened a ticket for INFRA to Enable Nexus Access For Sedona< > https://issues.apache.org/jira/browse/INFRA-21085> > * Followed this< > https://infra.apache.org/publishing-maven-artifacts.html> guide to test > the maven release process > * I hope to create a PR soon for adjusting the build to deploy to the > ASF Nexus repository > * The key that signs the artifacts were created and tested. > > Do we want to create a candidate release for the current master branch? > > Netanel Malka, > Big Data Consultant > [Description: Description: Description: Description: > cid:image001.jpg@01C85203.36A2AF30] > > From: Felix Cheung > Sent: Wednesday, November 4, 2020 19:57 > To: dev@sedona.apache.org > Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi Zhang > Subject: Re: First Sedona release > > 1) No you don’t need KEYS file in github only on the release share > https://dist.apache.org/repos/dist/dev/incubator/ > > 2) as podling you add to > https://dist.apache.org/repos/dist/dev/incubator/ > When you commit via svn you will be able to add a “directory” for Sedona > > 2a) for release, you basically do a svn rename to move from dev to release > “path” > > 3) if you have java based artifacts, yes. You will publish to Nexus, > staging first and when release is signed off, you can click on the > interface to make it official, which then automatically sync to Maven > central. > > Here is a script for example that does release signing and publication to > Nexus (and staging before release) > > https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh > > > On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka netanel...@gmail.com>> wrote: > Hi, > > I followed the release-signing > <https://infra.apache.org/release-signing.html> doc and created a key for > signing and hashing. > > I have a few questions: > >1. Should the KEYS file also be added to the project root directory on >Github? ( I saw it in Apache Ant) >2. I saw in release-policy_upload-ci ><http://www.apache.org/legal/release-policy.html#upload-ci> that we > need >to add a release candidate to https://dist.apache.org/repos/dist/*dev*/ > name>/. However, there does not seem to be a directory with Sedona as > the >TLP name. How may we be able to get a directory with that name? (Also > for >the *release*) >3. Do we need to push the artifacts also to ASF Nexus Repository (beside >Maven Central)? > > > Thanks. > > On Mon, 2 Nov 2020 at 19:21, Netanel Malka netanel...@gmail.com>> wrote: > > > Thanks Felix. > > > > I would be delighted to help. > > I can start with the GPG. > > Can I test it on a some artifact, or I need to wait for the first > release? > > > > > > On Mon, 2 Nov 2020 at 03:17, Felix Cheung <mailto:felixche...@apache.org>> wrote: > > > >> Great progress! > >> > >> To add, > >> A) I’d strongly recommend the WIP disclaimer - it would be much easier > to > >> pass with in the first release > >> https://incubator.apache.org/policy/incubation.html#disclaimers > >> > >> B) more info in signing, checksum > >> https://infra.apache.org/release-signing.html > >> > >> C) signing key should be individual’s and (public key ) published and > also > >> listed in KEYS file - KEYS file should be located next to the staging > >> (and > >> later release) location, see above > >> > >> D) “correct place” - this is in reference to ASF officIal staging server > >> http://www.apache.org/legal/release-policy.html#stage > >> And can be “uploaded” by committing to svn > >> http://www.apache.org/legal/release-policy.html#upload-ci > >> > >> E) python / PyPI - > >> https://incubator.apache.org/guides/distribution.html#pypi > >> > >> > >> > >> On Sun, Nov 1, 2020 at 2:17 PM Jia Yu
Re: First Sedona release
OK. Thanks Felix. Updates: * * Opened a ticket for INFRA to Enable Nexus Access For Sedona<https://issues.apache.org/jira/browse/INFRA-21085> * Followed this<https://infra.apache.org/publishing-maven-artifacts.html> guide to test the maven release process * I hope to create a PR soon for adjusting the build to deploy to the ASF Nexus repository * The key that signs the artifacts were created and tested. Do we want to create a candidate release for the current master branch? Netanel Malka, Big Data Consultant [Description: Description: Description: Description: cid:image001.jpg@01C85203.36A2AF30] From: Felix Cheung Sent: Wednesday, November 4, 2020 19:57 To: dev@sedona.apache.org Cc: Jinxuan Wu; Mohamed Sarwat; Netanel Malka; Paweł Kociński; Zongsi Zhang Subject: Re: First Sedona release 1) No you don’t need KEYS file in github only on the release share https://dist.apache.org/repos/dist/dev/incubator/ 2) as podling you add to https://dist.apache.org/repos/dist/dev/incubator/ When you commit via svn you will be able to add a “directory” for Sedona 2a) for release, you basically do a svn rename to move from dev to release “path” 3) if you have java based artifacts, yes. You will publish to Nexus, staging first and when release is signed off, you can click on the interface to make it official, which then automatically sync to Maven central. Here is a script for example that does release signing and publication to Nexus (and staging before release) https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka mailto:netanel...@gmail.com>> wrote: Hi, I followed the release-signing <https://infra.apache.org/release-signing.html> doc and created a key for signing and hashing. I have a few questions: 1. Should the KEYS file also be added to the project root directory on Github? ( I saw it in Apache Ant) 2. I saw in release-policy_upload-ci <http://www.apache.org/legal/release-policy.html#upload-ci> that we need to add a release candidate to https://dist.apache.org/repos/dist/*dev*//. However, there does not seem to be a directory with Sedona as the TLP name. How may we be able to get a directory with that name? (Also for the *release*) 3. Do we need to push the artifacts also to ASF Nexus Repository (beside Maven Central)? Thanks. On Mon, 2 Nov 2020 at 19:21, Netanel Malka mailto:netanel...@gmail.com>> wrote: > Thanks Felix. > > I would be delighted to help. > I can start with the GPG. > Can I test it on a some artifact, or I need to wait for the first release? > > > On Mon, 2 Nov 2020 at 03:17, Felix Cheung > mailto:felixche...@apache.org>> wrote: > >> Great progress! >> >> To add, >> A) I’d strongly recommend the WIP disclaimer - it would be much easier to >> pass with in the first release >> https://incubator.apache.org/policy/incubation.html#disclaimers >> >> B) more info in signing, checksum >> https://infra.apache.org/release-signing.html >> >> C) signing key should be individual’s and (public key ) published and also >> listed in KEYS file - KEYS file should be located next to the staging >> (and >> later release) location, see above >> >> D) “correct place” - this is in reference to ASF officIal staging server >> http://www.apache.org/legal/release-policy.html#stage >> And can be “uploaded” by committing to svn >> http://www.apache.org/legal/release-policy.html#upload-ci >> >> E) python / PyPI - >> https://incubator.apache.org/guides/distribution.html#pypi >> >> >> >> On Sun, Nov 1, 2020 at 2:17 PM Jia Yu >> mailto:ji...@apache.org>> wrote: >> >> > Hi Netanel, Pawel and other committers, >> > >> > While Pawel is working on Python code of Sedona 1.0, let's focus on >> other >> > parts required by the release. Netanel, can you help me with all the ASF >> > incubator requirement items that are not DONE? >> > >> > *Here is a checklist for our first Sedona release* >> > >> > *ASF incubator requirement >> > (https://incubator.apache.org/guides/releasemanagement.html >> > <https://incubator.apache.org/guides/releasemanagement.html>, we >> probably >> > should read ASF release requirement as well):* >> > >> > 1 .Include the word incubating in the release file name: DONE. Please >> see >> > the POM.xml in all directories. >> > >> > 2. Include an ASF LICENSE and NOTICE file: DONE. Please see the GitHub >> > repo. >> > >> > 3. Have valid checksums or signatures: I believe signature should be &g
Re: First Sedona release
1) No you don’t need KEYS file in github only on the release share https://dist.apache.org/repos/dist/dev/incubator/ 2) as podling you add to https://dist.apache.org/repos/dist/dev/incubator/ When you commit via svn you will be able to add a “directory” for Sedona 2a) for release, you basically do a svn rename to move from dev to release “path” 3) if you have java based artifacts, yes. You will publish to Nexus, staging first and when release is signed off, you can click on the interface to make it official, which then automatically sync to Maven central. Here is a script for example that does release signing and publication to Nexus (and staging before release) https://github.com/apache/spark/blob/master/dev/create-release/release-build.sh On Wed, Nov 4, 2020 at 2:50 AM Netanel Malka wrote: > Hi, > > I followed the release-signing > <https://infra.apache.org/release-signing.html> doc and created a key for > signing and hashing. > > I have a few questions: > >1. Should the KEYS file also be added to the project root directory on >Github? ( I saw it in Apache Ant) >2. I saw in release-policy_upload-ci ><http://www.apache.org/legal/release-policy.html#upload-ci> that we > need >to add a release candidate to https://dist.apache.org/repos/dist/*dev*/ > name>/. However, there does not seem to be a directory with Sedona as > the >TLP name. How may we be able to get a directory with that name? (Also > for >the *release*) >3. Do we need to push the artifacts also to ASF Nexus Repository (beside >Maven Central)? > > > Thanks. > > On Mon, 2 Nov 2020 at 19:21, Netanel Malka wrote: > > > Thanks Felix. > > > > I would be delighted to help. > > I can start with the GPG. > > Can I test it on a some artifact, or I need to wait for the first > release? > > > > > > On Mon, 2 Nov 2020 at 03:17, Felix Cheung > wrote: > > > >> Great progress! > >> > >> To add, > >> A) I’d strongly recommend the WIP disclaimer - it would be much easier > to > >> pass with in the first release > >> https://incubator.apache.org/policy/incubation.html#disclaimers > >> > >> B) more info in signing, checksum > >> https://infra.apache.org/release-signing.html > >> > >> C) signing key should be individual’s and (public key ) published and > also > >> listed in KEYS file - KEYS file should be located next to the staging > >> (and > >> later release) location, see above > >> > >> D) “correct place” - this is in reference to ASF officIal staging server > >> http://www.apache.org/legal/release-policy.html#stage > >> And can be “uploaded” by committing to svn > >> http://www.apache.org/legal/release-policy.html#upload-ci > >> > >> E) python / PyPI - > >> https://incubator.apache.org/guides/distribution.html#pypi > >> > >> > >> > >> On Sun, Nov 1, 2020 at 2:17 PM Jia Yu wrote: > >> > >> > Hi Netanel, Pawel and other committers, > >> > > >> > While Pawel is working on Python code of Sedona 1.0, let's focus on > >> other > >> > parts required by the release. Netanel, can you help me with all the > ASF > >> > incubator requirement items that are not DONE? > >> > > >> > *Here is a checklist for our first Sedona release* > >> > > >> > *ASF incubator requirement > >> > (https://incubator.apache.org/guides/releasemanagement.html > >> > <https://incubator.apache.org/guides/releasemanagement.html>, we > >> probably > >> > should read ASF release requirement as well):* > >> > > >> > 1 .Include the word incubating in the release file name: DONE. Please > >> see > >> > the POM.xml in all directories. > >> > > >> > 2. Include an ASF LICENSE and NOTICE file: DONE. Please see the GitHub > >> > repo. > >> > > >> > 3. Have valid checksums or signatures: I believe signature should be > >> done > >> > by the GPG key. Not sure about the checksum. I am also not sure about > >> the > >> > GPG key requirement of ASF. I use GPG key to sign releases of GeoSpark > >> in > >> > the past. > >> > > >> > 4. Be placed in the correct place on the ASF’s infrastructure: we > should > >> > place our releases in two places: Maven, and PyPi. Not sure how to > >> relate > >> > them to ASF. > >> > > >
Re: First Sedona release
Hi, I followed the release-signing <https://infra.apache.org/release-signing.html> doc and created a key for signing and hashing. I have a few questions: 1. Should the KEYS file also be added to the project root directory on Github? ( I saw it in Apache Ant) 2. I saw in release-policy_upload-ci <http://www.apache.org/legal/release-policy.html#upload-ci> that we need to add a release candidate to https://dist.apache.org/repos/dist/*dev*//. However, there does not seem to be a directory with Sedona as the TLP name. How may we be able to get a directory with that name? (Also for the *release*) 3. Do we need to push the artifacts also to ASF Nexus Repository (beside Maven Central)? Thanks. On Mon, 2 Nov 2020 at 19:21, Netanel Malka wrote: > Thanks Felix. > > I would be delighted to help. > I can start with the GPG. > Can I test it on a some artifact, or I need to wait for the first release? > > > On Mon, 2 Nov 2020 at 03:17, Felix Cheung wrote: > >> Great progress! >> >> To add, >> A) I’d strongly recommend the WIP disclaimer - it would be much easier to >> pass with in the first release >> https://incubator.apache.org/policy/incubation.html#disclaimers >> >> B) more info in signing, checksum >> https://infra.apache.org/release-signing.html >> >> C) signing key should be individual’s and (public key ) published and also >> listed in KEYS file - KEYS file should be located next to the staging >> (and >> later release) location, see above >> >> D) “correct place” - this is in reference to ASF officIal staging server >> http://www.apache.org/legal/release-policy.html#stage >> And can be “uploaded” by committing to svn >> http://www.apache.org/legal/release-policy.html#upload-ci >> >> E) python / PyPI - >> https://incubator.apache.org/guides/distribution.html#pypi >> >> >> >> On Sun, Nov 1, 2020 at 2:17 PM Jia Yu wrote: >> >> > Hi Netanel, Pawel and other committers, >> > >> > While Pawel is working on Python code of Sedona 1.0, let's focus on >> other >> > parts required by the release. Netanel, can you help me with all the ASF >> > incubator requirement items that are not DONE? >> > >> > *Here is a checklist for our first Sedona release* >> > >> > *ASF incubator requirement >> > (https://incubator.apache.org/guides/releasemanagement.html >> > <https://incubator.apache.org/guides/releasemanagement.html>, we >> probably >> > should read ASF release requirement as well):* >> > >> > 1 .Include the word incubating in the release file name: DONE. Please >> see >> > the POM.xml in all directories. >> > >> > 2. Include an ASF LICENSE and NOTICE file: DONE. Please see the GitHub >> > repo. >> > >> > 3. Have valid checksums or signatures: I believe signature should be >> done >> > by the GPG key. Not sure about the checksum. I am also not sure about >> the >> > GPG key requirement of ASF. I use GPG key to sign releases of GeoSpark >> in >> > the past. >> > >> > 4. Be placed in the correct place on the ASF’s infrastructure: we should >> > place our releases in two places: Maven, and PyPi. Not sure how to >> relate >> > them to ASF. >> > >> > 5. Have a KEYS file to validate the release: this should be the public >> key >> > of our GPG key? >> > >> > *Sedona requirement* >> > >> > 1. Python path name, file headers, and jars >> > 2. Project website docs: documentation should use the name, Sedona, in >> all >> > tutorials. We should also include the situation of GeoTools >> dependencies. >> > >> > Thanks, >> > Jia >> > >> > >> > On Wed, Oct 14, 2020 at 10:08 PM Jia Yu wrote: >> > >> > > Hi folks, >> > > >> > > We will be working on the first Sedona. Please see the JIRA ticket >> here: >> > > >> > >> https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues >> > > >> > > Do you think there are any outstanding issues to be fixed as well? >> > > >> > > Thanks, >> > > Jia >> > > >> > >> > > > -- > Best regards, > Netanel Malka. > -- Best regards, Netanel Malka.
Re: First Sedona release
Thanks Felix. I would be delighted to help. I can start with the GPG. Can I test it on a some artifact, or I need to wait for the first release? On Mon, 2 Nov 2020 at 03:17, Felix Cheung wrote: > Great progress! > > To add, > A) I’d strongly recommend the WIP disclaimer - it would be much easier to > pass with in the first release > https://incubator.apache.org/policy/incubation.html#disclaimers > > B) more info in signing, checksum > https://infra.apache.org/release-signing.html > > C) signing key should be individual’s and (public key ) published and also > listed in KEYS file - KEYS file should be located next to the staging (and > later release) location, see above > > D) “correct place” - this is in reference to ASF officIal staging server > http://www.apache.org/legal/release-policy.html#stage > And can be “uploaded” by committing to svn > http://www.apache.org/legal/release-policy.html#upload-ci > > E) python / PyPI - > https://incubator.apache.org/guides/distribution.html#pypi > > > > On Sun, Nov 1, 2020 at 2:17 PM Jia Yu wrote: > > > Hi Netanel, Pawel and other committers, > > > > While Pawel is working on Python code of Sedona 1.0, let's focus on other > > parts required by the release. Netanel, can you help me with all the ASF > > incubator requirement items that are not DONE? > > > > *Here is a checklist for our first Sedona release* > > > > *ASF incubator requirement > > (https://incubator.apache.org/guides/releasemanagement.html > > <https://incubator.apache.org/guides/releasemanagement.html>, we > probably > > should read ASF release requirement as well):* > > > > 1 .Include the word incubating in the release file name: DONE. Please see > > the POM.xml in all directories. > > > > 2. Include an ASF LICENSE and NOTICE file: DONE. Please see the GitHub > > repo. > > > > 3. Have valid checksums or signatures: I believe signature should be done > > by the GPG key. Not sure about the checksum. I am also not sure about the > > GPG key requirement of ASF. I use GPG key to sign releases of GeoSpark in > > the past. > > > > 4. Be placed in the correct place on the ASF’s infrastructure: we should > > place our releases in two places: Maven, and PyPi. Not sure how to relate > > them to ASF. > > > > 5. Have a KEYS file to validate the release: this should be the public > key > > of our GPG key? > > > > *Sedona requirement* > > > > 1. Python path name, file headers, and jars > > 2. Project website docs: documentation should use the name, Sedona, in > all > > tutorials. We should also include the situation of GeoTools dependencies. > > > > Thanks, > > Jia > > > > > > On Wed, Oct 14, 2020 at 10:08 PM Jia Yu wrote: > > > > > Hi folks, > > > > > > We will be working on the first Sedona. Please see the JIRA ticket > here: > > > > > > https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues > > > > > > Do you think there are any outstanding issues to be fixed as well? > > > > > > Thanks, > > > Jia > > > > > > -- Best regards, Netanel Malka.
Re: First Sedona release
Great progress! To add, A) I’d strongly recommend the WIP disclaimer - it would be much easier to pass with in the first release https://incubator.apache.org/policy/incubation.html#disclaimers B) more info in signing, checksum https://infra.apache.org/release-signing.html C) signing key should be individual’s and (public key ) published and also listed in KEYS file - KEYS file should be located next to the staging (and later release) location, see above D) “correct place” - this is in reference to ASF officIal staging server http://www.apache.org/legal/release-policy.html#stage And can be “uploaded” by committing to svn http://www.apache.org/legal/release-policy.html#upload-ci E) python / PyPI - https://incubator.apache.org/guides/distribution.html#pypi On Sun, Nov 1, 2020 at 2:17 PM Jia Yu wrote: > Hi Netanel, Pawel and other committers, > > While Pawel is working on Python code of Sedona 1.0, let's focus on other > parts required by the release. Netanel, can you help me with all the ASF > incubator requirement items that are not DONE? > > *Here is a checklist for our first Sedona release* > > *ASF incubator requirement > (https://incubator.apache.org/guides/releasemanagement.html > <https://incubator.apache.org/guides/releasemanagement.html>, we probably > should read ASF release requirement as well):* > > 1 .Include the word incubating in the release file name: DONE. Please see > the POM.xml in all directories. > > 2. Include an ASF LICENSE and NOTICE file: DONE. Please see the GitHub > repo. > > 3. Have valid checksums or signatures: I believe signature should be done > by the GPG key. Not sure about the checksum. I am also not sure about the > GPG key requirement of ASF. I use GPG key to sign releases of GeoSpark in > the past. > > 4. Be placed in the correct place on the ASF’s infrastructure: we should > place our releases in two places: Maven, and PyPi. Not sure how to relate > them to ASF. > > 5. Have a KEYS file to validate the release: this should be the public key > of our GPG key? > > *Sedona requirement* > > 1. Python path name, file headers, and jars > 2. Project website docs: documentation should use the name, Sedona, in all > tutorials. We should also include the situation of GeoTools dependencies. > > Thanks, > Jia > > > On Wed, Oct 14, 2020 at 10:08 PM Jia Yu wrote: > > > Hi folks, > > > > We will be working on the first Sedona. Please see the JIRA ticket here: > > > https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues > > > > Do you think there are any outstanding issues to be fixed as well? > > > > Thanks, > > Jia > > >
Re: First Sedona release
Hi Netanel, Pawel and other committers, While Pawel is working on Python code of Sedona 1.0, let's focus on other parts required by the release. Netanel, can you help me with all the ASF incubator requirement items that are not DONE? *Here is a checklist for our first Sedona release* *ASF incubator requirement (https://incubator.apache.org/guides/releasemanagement.html <https://incubator.apache.org/guides/releasemanagement.html>, we probably should read ASF release requirement as well):* 1 .Include the word incubating in the release file name: DONE. Please see the POM.xml in all directories. 2. Include an ASF LICENSE and NOTICE file: DONE. Please see the GitHub repo. 3. Have valid checksums or signatures: I believe signature should be done by the GPG key. Not sure about the checksum. I am also not sure about the GPG key requirement of ASF. I use GPG key to sign releases of GeoSpark in the past. 4. Be placed in the correct place on the ASF’s infrastructure: we should place our releases in two places: Maven, and PyPi. Not sure how to relate them to ASF. 5. Have a KEYS file to validate the release: this should be the public key of our GPG key? *Sedona requirement* 1. Python path name, file headers, and jars 2. Project website docs: documentation should use the name, Sedona, in all tutorials. We should also include the situation of GeoTools dependencies. Thanks, Jia On Wed, Oct 14, 2020 at 10:08 PM Jia Yu wrote: > Hi folks, > > We will be working on the first Sedona. Please see the JIRA ticket here: > https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues > > Do you think there are any outstanding issues to be fixed as well? > > Thanks, > Jia >
First Sedona release
Hi folks, We will be working on the first Sedona. Please see the JIRA ticket here: https://issues.apache.org/jira/projects/SEDONA/issues/SEDONA-3?filter=allopenissues Do you think there are any outstanding issues to be fixed as well? Thanks, Jia