Re: Sedona 1.0 Maven release is staged
Hi Netanel, The reason why I use sbt is that, when I use Maven to run SparkSQL 3.0 + Sedona in IntelliJ, it always prompts an Exception like "NoClassFound: jackson-data-bind: noMappingException." This is because Spark 3.0 uses inconsistent com.fasterxml.jackson version (2.10.0 vs some Spark dependencies use 2.9.x). Maven always packages the 2.9.x for test no matter how I change the pom.xml in example projects. If I use SBT, all exceptions are gone. The "cannot reparse for SNAPSHOT" issue in SBT is a warning, not an error. so it is not a blocker. But I am not sure where another error comes from. In fact, I have tested all example projects (with Sedona SNAPSHOTs) in IntelliJ and CLI. I just added the GitHub action CI for the example projects. It also passed: https://github.com/apache/incubator-sedona/actions/runs/456989056 Do you mind pull the master branch again and run the three projects in CLI and IntelliJ? See https://github.com/apache/incubator-sedona/blob/master/.github/workflows/example.yml#L35 Thanks, Jia On Fri, Jan 1, 2021 at 1:14 AM Netanel Malka wrote: > Hi Jia, > I tried to build the Demo project and I have a dependency issue. > I tried using sbt assembly and from IntelliJ. > > For example, build error for Sedona SQL (None of them succeeded in build): > https://pastebin.pl/view/15882089 > > It looks like a bug in SBT regards to SNAPSHOT jars that don't handle: > https://github.com/sbt/sbt/issues/3404#issuecomment-400641680 > and also here: > https://github.com/sbt/sbt/issues/2687 > > There is any special reason why did you choose to use sbt instead of maven? > > > > > On Thu, 31 Dec 2020 at 23:27, Jia Yu wrote: > > > Dear Felix, > > > > Thanks for your great comments! > > > > I have fixed all three comments accordingly, Since we have disclaim-WIP, > I > > think we don't have to re-stage the releases. > > > > Thanks, > > Jia > > > > > > > > On Wed, Dec 30, 2020 at 10:54 PM Felix Cheung > > wrote: > > > > > Release review. > > > > > > Note: with DISCLAIMER-WIP, these are not blockers to release. > > > > > > website: > > > 1. the website can add more links as described in this section: > > > https://www.apache.org/foundation/marks/pmcs#navigation > > > 2. might need to add trademark attributions > > > https://www.apache.org/foundation/marks/pmcs#attributions > > > > > > project source: > > > 1. For LICENSE, we should add RAT check > > > https://creadur.apache.org/rat/#What_Is_Rat > > > 2. This file seems to have an irregular header: > > > /incubator-sedona/zeppelin/index.js > > > 3. These code file should have Apache v2 header: > > > /python/build_wheel.sh > > > /spark-version-converter.py > > > > > > > > > /python-adapter/src/test/scala/org/apache/sedona/python/wrapper/PythonTestSpec.scala > > > > > > > > > /python-adapter/src/test/scala/org/apache/sedona/python/wrapper/GeometrySample.scala > > > 4. > > > > > > > > > /docs/api/javadoc/core/org/apache/sedona/core/geometryObjects/GeoJSONWriterNew.html > > > /docs/api/javadoc/* > > > are these doc under /docs/api/javadoc auto generated? if that is the > > case, > > > we probably shouldn't have the HTML committed in the repo > > > 5. > > > /trunk/import/* > > > what is this for? > > > 6. should add a small section on how to build (or link to) the project > in > > > README.md > > > > > > > > > On Wed, Dec 30, 2020 at 6:24 PM Felix Cheung > > > wrote: > > > > > > > Yes > > > > https://incubator.apache.org/guides/distribution.html#pypi > > > > > > > > Reminder: This should not be posted until we have Sedona and IPMC > vote. > > > > > > > > > > > > On Wed, Dec 30, 2020 at 3:30 PM Jia Yu wrote: > > > > > > > >> Hi Pawel, > > > >> > > > >> The only Pypi guideline that I can find is Incubator distribution > > > >> guideline: https://incubator.apache.org/guides/distribution.html > > > >> > > > >> If you scroll down to the bottom, you will see the section. > > > >> > > > >> @Felix Cheung Felix, does Pawel need to be > > > >> aware of anything else, besides the incubator PyPi guideline? > > > >> > > > >> Thank you very much! > > > >> Jia > > > >> > > > >> On Wed, Dec 30, 2020 at 3:26 PM Jia Yu wrote: > > > >> > > > >> > Hi Netanel, > > > >> > > > > >> > I have created a few demo projects: > > > >> > https://sedona.staged.apache.org/tutorial/GeoSpark-Runnable-DEMO/ > > > >> > The source code is here: > > > >> > https://github.com/apache/incubator-sedona/tree/master/examples > > > >> > > > > >> > Does it look good? > > > >> > > > > >> > Thanks, > > > >> > Jia > > > >> > > > > >> > On Wed, Dec 30, 2020 at 1:06 PM Netanel Malka < > netan...@sela.co.il> > > > >> wrote: > > > >> > > > > >> >> Hi Jia, > > > >> >> That's great news! > > > >> >> > > > >> >> Happy and healthy New Year to everyone. > > > >> >> > > > >> >> They look fine to me. > > > >> >> Do you think it will be worth to use them within a demo project? > > (to > > > >> >> eliminate dependency issues) I can test it if we need it. > > > >> >> > > > >> >> BR, > > > >> >> > > > >> >> > > >
[GitHub] [incubator-sedona] jiayuasu merged pull request #504: Add the CI test for example projects
jiayuasu merged pull request #504: URL: https://github.com/apache/incubator-sedona/pull/504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu opened a new pull request #504: Add the CI test for example projects
jiayuasu opened a new pull request #504: URL: https://github.com/apache/incubator-sedona/pull/504 ## Is this PR related to a proposed Issue? ## What changes were proposed in this PR? Add the CI test for three example projects of Scala and Java ## How was this patch tested? ## Did this PR include necessary documentation updates? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: Sedona 1.0 Maven release is staged
Hi Jia, I tried to build the Demo project and I have a dependency issue. I tried using sbt assembly and from IntelliJ. For example, build error for Sedona SQL (None of them succeeded in build): https://pastebin.pl/view/15882089 It looks like a bug in SBT regards to SNAPSHOT jars that don't handle: https://github.com/sbt/sbt/issues/3404#issuecomment-400641680 and also here: https://github.com/sbt/sbt/issues/2687 There is any special reason why did you choose to use sbt instead of maven? On Thu, 31 Dec 2020 at 23:27, Jia Yu wrote: > Dear Felix, > > Thanks for your great comments! > > I have fixed all three comments accordingly, Since we have disclaim-WIP, I > think we don't have to re-stage the releases. > > Thanks, > Jia > > > > On Wed, Dec 30, 2020 at 10:54 PM Felix Cheung > wrote: > > > Release review. > > > > Note: with DISCLAIMER-WIP, these are not blockers to release. > > > > website: > > 1. the website can add more links as described in this section: > > https://www.apache.org/foundation/marks/pmcs#navigation > > 2. might need to add trademark attributions > > https://www.apache.org/foundation/marks/pmcs#attributions > > > > project source: > > 1. For LICENSE, we should add RAT check > > https://creadur.apache.org/rat/#What_Is_Rat > > 2. This file seems to have an irregular header: > > /incubator-sedona/zeppelin/index.js > > 3. These code file should have Apache v2 header: > > /python/build_wheel.sh > > /spark-version-converter.py > > > > > /python-adapter/src/test/scala/org/apache/sedona/python/wrapper/PythonTestSpec.scala > > > > > /python-adapter/src/test/scala/org/apache/sedona/python/wrapper/GeometrySample.scala > > 4. > > > > > /docs/api/javadoc/core/org/apache/sedona/core/geometryObjects/GeoJSONWriterNew.html > > /docs/api/javadoc/* > > are these doc under /docs/api/javadoc auto generated? if that is the > case, > > we probably shouldn't have the HTML committed in the repo > > 5. > > /trunk/import/* > > what is this for? > > 6. should add a small section on how to build (or link to) the project in > > README.md > > > > > > On Wed, Dec 30, 2020 at 6:24 PM Felix Cheung > > wrote: > > > > > Yes > > > https://incubator.apache.org/guides/distribution.html#pypi > > > > > > Reminder: This should not be posted until we have Sedona and IPMC vote. > > > > > > > > > On Wed, Dec 30, 2020 at 3:30 PM Jia Yu wrote: > > > > > >> Hi Pawel, > > >> > > >> The only Pypi guideline that I can find is Incubator distribution > > >> guideline: https://incubator.apache.org/guides/distribution.html > > >> > > >> If you scroll down to the bottom, you will see the section. > > >> > > >> @Felix Cheung Felix, does Pawel need to be > > >> aware of anything else, besides the incubator PyPi guideline? > > >> > > >> Thank you very much! > > >> Jia > > >> > > >> On Wed, Dec 30, 2020 at 3:26 PM Jia Yu wrote: > > >> > > >> > Hi Netanel, > > >> > > > >> > I have created a few demo projects: > > >> > https://sedona.staged.apache.org/tutorial/GeoSpark-Runnable-DEMO/ > > >> > The source code is here: > > >> > https://github.com/apache/incubator-sedona/tree/master/examples > > >> > > > >> > Does it look good? > > >> > > > >> > Thanks, > > >> > Jia > > >> > > > >> > On Wed, Dec 30, 2020 at 1:06 PM Netanel Malka > > >> wrote: > > >> > > > >> >> Hi Jia, > > >> >> That's great news! > > >> >> > > >> >> Happy and healthy New Year to everyone. > > >> >> > > >> >> They look fine to me. > > >> >> Do you think it will be worth to use them within a demo project? > (to > > >> >> eliminate dependency issues) I can test it if we need it. > > >> >> > > >> >> BR, > > >> >> > > >> >> > > >> >> Netanel Malka, > > >> >> Big Data Consultant > > >> >> [image: Description: Description: Description: Description: > > >> >> cid:image001.jpg@01C85203.36A2AF30] > > >> >> -- > > >> >> *From:* Paweł Kociński > > >> >> *Sent:* Wednesday, December 30, 2020 22:09 > > >> >> *To:* Jia Yu > > >> >> *Cc:* dev@sedona.apache.org; Jinxuan Wu; Mohamed Sarwat; Netanel > > >> Malka; > > >> >> Zongsi Zhang; Felix Cheung; Netanel Malka > > >> >> *Subject:* Re: Sedona 1.0 Maven release is staged > > >> >> > > >> >> Hi Jia, > > >> >> Merry Christmas and Happy New Year to everyone. > > >> >> I can definitely do that but I am not sure about the steps which I > > >> should > > >> >> follow to upload package under Apache project. Also should we sync > > this > > >> >> processes, I mean Maven and PyPI release ? > > >> >> > > >> >> Regards, > > >> >> Pawel > > >> >> > > >> >> śr., 30 gru 2020 o 08:43 Jia Yu napisał(a): > > >> >> > > >> >>> Dear all, > > >> >>> > > >> >>> Merry Christmas and Happy New Year! > > >> >>> > > >> >>> After months of hard work, we are finally close to the Sedona > > >> >>> 1.0.0-incubator release. I have staged Sedona 1.0.0 official > release > > >> on > > >> >>> Nexus. This is a thread for collecting final feedback rather than > a > > >> voting > > >> >>> thread. > > >> >>> > > >> >>> 1. I would like to ask all of
