Hi Netanel, The reason why I use sbt is that, when I use Maven to run SparkSQL 3.0 + Sedona in IntelliJ, it always prompts an Exception like "NoClassFound: jackson-data-bind: noMappingException." This is because Spark 3.0 uses inconsistent com.fasterxml.jackson version (2.10.0 vs some Spark dependencies use 2.9.x). Maven always packages the 2.9.x for test no matter how I change the pom.xml in example projects.
If I use SBT, all exceptions are gone. The "cannot reparse for SNAPSHOT" issue in SBT is a warning, not an error. so it is not a blocker. But I am not sure where another error comes from. In fact, I have tested all example projects (with Sedona SNAPSHOTs) in IntelliJ and CLI. I just added the GitHub action CI for the example projects. It also passed: https://github.com/apache/incubator-sedona/actions/runs/456989056 Do you mind pull the master branch again and run the three projects in CLI and IntelliJ? See https://github.com/apache/incubator-sedona/blob/master/.github/workflows/example.yml#L35 Thanks, Jia On Fri, Jan 1, 2021 at 1:14 AM Netanel Malka <netanel...@gmail.com> wrote: > Hi Jia, > I tried to build the Demo project and I have a dependency issue. > I tried using sbt assembly and from IntelliJ. > > For example, build error for Sedona SQL (None of them succeeded in build): > https://pastebin.pl/view/15882089 > > It looks like a bug in SBT regards to SNAPSHOT jars that don't handle: > https://github.com/sbt/sbt/issues/3404#issuecomment-400641680 > and also here: > https://github.com/sbt/sbt/issues/2687 > > There is any special reason why did you choose to use sbt instead of maven? > > > > > On Thu, 31 Dec 2020 at 23:27, Jia Yu <jiayu198...@gmail.com> wrote: > > > Dear Felix, > > > > Thanks for your great comments! > > > > I have fixed all three comments accordingly, Since we have disclaim-WIP, > I > > think we don't have to re-stage the releases. > > > > Thanks, > > Jia > > > > > > > > On Wed, Dec 30, 2020 at 10:54 PM Felix Cheung <felixche...@apache.org> > > wrote: > > > > > Release review. > > > > > > Note: with DISCLAIMER-WIP, these are not blockers to release. > > > > > > website: > > > 1. the website can add more links as described in this section: > > > https://www.apache.org/foundation/marks/pmcs#navigation > > > 2. might need to add trademark attributions > > > https://www.apache.org/foundation/marks/pmcs#attributions > > > > > > project source: > > > 1. For LICENSE, we should add RAT check > > > https://creadur.apache.org/rat/#What_Is_Rat > > > 2. This file seems to have an irregular header: > > > /incubator-sedona/zeppelin/index.js > > > 3. These code file should have Apache v2 header: > > > /python/build_wheel.sh > > > /spark-version-converter.py > > > > > > > > > /python-adapter/src/test/scala/org/apache/sedona/python/wrapper/PythonTestSpec.scala > > > > > > > > > /python-adapter/src/test/scala/org/apache/sedona/python/wrapper/GeometrySample.scala > > > 4. > > > > > > > > > /docs/api/javadoc/core/org/apache/sedona/core/geometryObjects/GeoJSONWriterNew.html > > > /docs/api/javadoc/* > > > are these doc under /docs/api/javadoc auto generated? if that is the > > case, > > > we probably shouldn't have the HTML committed in the repo > > > 5. > > > /trunk/import/* > > > what is this for? > > > 6. should add a small section on how to build (or link to) the project > in > > > README.md > > > > > > > > > On Wed, Dec 30, 2020 at 6:24 PM Felix Cheung <felixche...@apache.org> > > > wrote: > > > > > > > Yes > > > > https://incubator.apache.org/guides/distribution.html#pypi > > > > > > > > Reminder: This should not be posted until we have Sedona and IPMC > vote. > > > > > > > > > > > > On Wed, Dec 30, 2020 at 3:30 PM Jia Yu <ji...@apache.org> wrote: > > > > > > > >> Hi Pawel, > > > >> > > > >> The only Pypi guideline that I can find is Incubator distribution > > > >> guideline: https://incubator.apache.org/guides/distribution.html > > > >> > > > >> If you scroll down to the bottom, you will see the section. > > > >> > > > >> @Felix Cheung <felixche...@apache.org> Felix, does Pawel need to be > > > >> aware of anything else, besides the incubator PyPi guideline? > > > >> > > > >> Thank you very much! > > > >> Jia > > > >> > > > >> On Wed, Dec 30, 2020 at 3:26 PM Jia Yu <ji...@apache.org> wrote: > > > >> > > > >> > Hi Netanel, > > > >> > > > > >> > I have created a few demo projects: > > > >> > https://sedona.staged.apache.org/tutorial/GeoSpark-Runnable-DEMO/ > > > >> > The source code is here: > > > >> > https://github.com/apache/incubator-sedona/tree/master/examples > > > >> > > > > >> > Does it look good? > > > >> > > > > >> > Thanks, > > > >> > Jia > > > >> > > > > >> > On Wed, Dec 30, 2020 at 1:06 PM Netanel Malka < > netan...@sela.co.il> > > > >> wrote: > > > >> > > > > >> >> Hi Jia, > > > >> >> That's great news! > > > >> >> > > > >> >> Happy and healthy New Year to everyone. > > > >> >> > > > >> >> They look fine to me. > > > >> >> Do you think it will be worth to use them within a demo project? > > (to > > > >> >> eliminate dependency issues) I can test it if we need it. > > > >> >> > > > >> >> BR, > > > >> >> > > > >> >> > > > >> >> Netanel Malka, > > > >> >> Big Data Consultant > > > >> >> [image: Description: Description: Description: Description: > > > >> >> cid:image001.jpg@01C85203.36A2AF30] > > > >> >> ------------------------------ > > > >> >> *From:* Paweł Kociński <pawel93kocin...@gmail.com> > > > >> >> *Sent:* Wednesday, December 30, 2020 22:09 > > > >> >> *To:* Jia Yu > > > >> >> *Cc:* dev@sedona.apache.org; Jinxuan Wu; Mohamed Sarwat; Netanel > > > >> Malka; > > > >> >> Zongsi Zhang; Felix Cheung; Netanel Malka > > > >> >> *Subject:* Re: Sedona 1.0 Maven release is staged > > > >> >> > > > >> >> Hi Jia, > > > >> >> Merry Christmas and Happy New Year to everyone. > > > >> >> I can definitely do that but I am not sure about the steps which > I > > > >> should > > > >> >> follow to upload package under Apache project. Also should we > sync > > > this > > > >> >> processes, I mean Maven and PyPI release ? > > > >> >> > > > >> >> Regards, > > > >> >> Pawel > > > >> >> > > > >> >> śr., 30 gru 2020 o 08:43 Jia Yu <ji...@apache.org> napisał(a): > > > >> >> > > > >> >>> Dear all, > > > >> >>> > > > >> >>> Merry Christmas and Happy New Year! > > > >> >>> > > > >> >>> After months of hard work, we are finally close to the Sedona > > > >> >>> 1.0.0-incubator release. I have staged Sedona 1.0.0 official > > release > > > >> on > > > >> >>> Nexus. This is a thread for collecting final feedback rather > than > > a > > > >> voting > > > >> >>> thread. > > > >> >>> > > > >> >>> 1. I would like to ask all of you to take a look at the staged > > Maven > > > >> >>> releases (12 Sedona modules in total). Please learn how to > access > > > >> Nexus > > > >> >>> from [1][2] > > > >> >>> > > > >> >>> @Mohamed Sarwat <msar...@asu.edu> @Netanel Malka < > > > >> netanel...@gmail.com> Would > > > >> >>> you please take a look at the staged releases? > > > >> >>> > > > >> >>> @Felix Cheung <felixche...@apache.org> Felix, could you please > > > take > > > >> a > > > >> >>> look at the release as well? Your opinion is very important. If > > you > > > >> think > > > >> >>> everything is good, then I will call for a vote. > > > >> >>> > > > >> >>> 2. Python release on PyPi > > > >> >>> @Paweł Kociński <pawel93kocin...@gmail.com> Pawel, could you > > please > > > >> >>> publish Sedona 1.0.0-incubator-SNAPSHOT to PyPi? Then we will > use > > > >> that to > > > >> >>> call for a vote. > > > >> >>> > > > >> >>> I know many of you may be on vacation so no need to rush. > > > >> >>> > > > >> >>> [1] > > > >> >>> > > > >> > > > > > > https://central.sonatype.org/pages/releasing-the-deployment.html#locate-and-examine-your-staging-repository > > > >> >>> [2] Sedona staged website: https://sedona.staged.apache.org/ > > > >> >>> > > > >> >> > > > >> > > > > > > > > > > > > -- > Best regards, > Netanel Malka. >
