Did some testing with rc1 today. - Validated rc tarballs - Successfully built and ran the release - Tested SDAP ingest successfully - Tested the following SDAP endpoints: -/match_spark -/cdmssubset -/domsresults -/domslist -/list -/apidocs -/timeSeriesSpark
I performed the endpoint tests using the demo Jupyter notebook used in the quickstart guide and some test scripts I wrote for a project that uses SDAP. Riley On 2022/12/06 07:23:57 "Perez, Stepheny K (US 398F)" wrote: > Hi Nga, > > I’ve had success testing rc0 – the following analysis endpoints were used in > my tests: > > > * match_spark > * cdmssubsetter > * cdmsresults > > Has anyone tested the time series analysis endpoints? > > Thanks, > Stepheny > > From: Nga Chung <ngatch...@gmail.com> > Date: Monday, December 5, 2022 at 11:06 PM > To: dev@sdap.apache.org <dev@sdap.apache.org> > Subject: Re: [EXTERNAL] Re: [DISCUSS] Towards release 0.4.5a56 > Riley, thank you for adding all the missing ASF header and the NOTICE. > > Riley's changes have been merged and rc1 artifacts have been uploaded to > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc1/__;!!PvBDto6Hs4WbVuu7!dPruOg9-ncxdgZW_4Gb72i05RAtnn9YBnohlQjSwWdWTM6KCo5J3490Ih-RHUOkm66RM11OAKhc$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc1/__;!!PvBDto6Hs4WbVuu7!dPruOg9-ncxdgZW_4Gb72i05RAtnn9YBnohlQjSwWdWTM6KCo5J3490Ih-RHUOkm66RM11OAKhc$> > > > Has anyone had any success testing rc0? > > Julian, any suggestions on next steps? > > Thanks, > Nga > > On Thu, Dec 1, 2022 at 1:03 PM Kuttruff, Riley K (US 398F-Affiliate) > <riley.k.kuttr...@jpl.nasa.gov.invalid> wrote: > > > It appears some of the source files are missing the ASF header. I did a > > check for all the Python files across the repositories and added them where > > needed in a pair of PRs (nexus & ingester; nexusproto had no .py files > > missing the header). > > > > I still have yet to check for any non-python source files. > > > > Riley > > > > On 11/30/22, 4:29 PM, "Nga Chung" <nch...@apache.org> wrote: > > > > Hi everyone, > > > > So this email thread probably needs renaming or we can start a new one > > if > > we're proceeding with a Version 1.0.0 release instead of 0.4.5a56. > > > > Anyways, with help from many folks we now have a release candidate > > that can > > be found here: > > > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGjkf-Wmw$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGjkf-Wmw$> > > > > > > Instructions for building docker images from source can be found here: > > > > https://urldefense.us/v3/__https://incubator-sdap-nexus.readthedocs.io/en/latest/build.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGMdzQUnc$<https://urldefense.us/v3/__https:/incubator-sdap-nexus.readthedocs.io/en/latest/build.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGMdzQUnc$> > > Instructions for deploying locally to test can be found here: > > > > https://urldefense.us/v3/__https://incubator-sdap-nexus.readthedocs.io/en/latest/quickstart.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG-tQ0nvg$<https://urldefense.us/v3/__https:/incubator-sdap-nexus.readthedocs.io/en/latest/quickstart.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG-tQ0nvg$> > > Associated docker images can be found here: > > > > https://urldefense.us/v3/__https://hub.docker.com/search?q=apache*2Fsdap__;JQ!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG5KMpZEg$<https://urldefense.us/v3/__https:/hub.docker.com/search?q=apache*2Fsdap__;JQ!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG5KMpZEg$> > > > > Here's how I created rc0. I will get these instructions into github but > > wanted to get all this out first for your review. > > > > git clone --branch release/1.0.0 > > > > https://urldefense.us/v3/__https://github.com/apache/incubator-sdap-nexusproto.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGbmDU9OM$<https://urldefense.us/v3/__https:/github.com/apache/incubator-sdap-nexusproto.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGbmDU9OM$> > > > > cd incubator-sdap-nexusproto*/* > > > > git ls-files > /tmp/manifest.txt > > > > tar cvfz apache-sdap-nexusproto-1.0.0-src.tar.gz -T /tmp/manifest.txt > > > > gpg --armor --output apache-sdap-nexusproto-1.0.0-src.tar.gz.asc > > --detach-sig apache-sdap-nexusproto-1.0.0-src.tar.gz > > > > shasum -a 512 apache-sdap-nexusproto-1.0.0-src.tar.gz > > > apache-sdap-nexusproto-1.0.0-src.tar.gz.sha512 > > > > > > git clone --branch release/1.0.0 > > > > https://urldefense.us/v3/__https://github.com/apache/incubator-sdap-ingester.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGgkAuYFg$<https://urldefense.us/v3/__https:/github.com/apache/incubator-sdap-ingester.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGgkAuYFg$> > > > > cd incubator-sdap-ingester/ > > > > git ls-files > /tmp/manifest.txt > > > > tar cvfz apache-sdap-ingester-1.0.0-src.tar.gz -T /tmp/manifest.txt > > > > gpg --armor --output apache-sdap-ingester-1.0.0-src.tar.gz.asc > > --detach-sig > > apache-sdap-ingester-1.0.0-src.tar.gz > > > > shasum -a 512 apache-sdap-ingester-1.0.0-src.tar.gz > > > apache-sdap-ingester-1.0.0-src.tar.gz.sha512 > > > > > > git clone --branch release/1.0.0 > > > > https://urldefense.us/v3/__https://github.com/apache/incubator-sdap-nexus.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG19U3rPQ$<https://urldefense.us/v3/__https:/github.com/apache/incubator-sdap-nexus.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG19U3rPQ$> > > > > cd incubator-sdap-nexus/ > > > > git ls-files > /tmp/manifest.txt > > > > tar cvfz apache-sdap-nexus-1.0.0-src.tar.gz -T /tmp/manifest.txt > > > > gpg --armor --output apache-sdap-nexus-1.0.0-src.tar.gz.asc > > --detach-sig > > apache-sdap-nexus-1.0.0-src.tar.gz > > > > shasum -a 512 apache-sdap-nexus-1.0.0-src.tar.gz > > > apache-sdap-nexus-1.0.0-src.tar.gz.sha512 > > > > svn co > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG8upBcY0$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG8upBcY0$> > > sdap > > mkdir sdap/apache-sdap-1.0.0-rc0 > > cp incubator-sdap-nexusproto/apache-sdap-nexusproto-1.0.0-src.tar.gz* > > sdap/apache-sdap-1.0.0-rc0/. > > cp incubator-sdap-ingester/apache-sdap-ingester-1.0.0-src.tar.gz* > > sdap/apache-sdap-1.0.0-rc0/. > > cp incubator-sdap-nexus/apache-sdap-nexus-1.0.0-src.tar.gz* > > sdap/apache-sdap-1.0.0-rc0/. > > > > svn add apache-sdap-1.0.0-rc0 > > > > svn ci -m "Uploading release candidate Apache SDAP > > apache-sdap-1.0.0-rc0 to > > dev area" apache-sdap-1.0.0-rc0 > > > > > > Thanks, > > Nga > > > > On Wed, Nov 9, 2022 at 4:26 PM Julian Hyde <jhyde.apa...@gmail.com> > > wrote: > > > > > hanks for volunteering to be release manager! > > > > > > The artifacts to be voted on will be in the following directory (or > > > something very much like it): > > > > > > > > > > > > > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-0.4.5a56-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGsl1T6OQ$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-0.4.5a56-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGsl1T6OQ$> > > > > > > with > > > * “dev” being the place for candidate releases (to be replaced by > > “release” > > > when the release is final) > > > * “sdap” being the project name (prefixed with “incubator/“ while > > sdap is > > > incubating) > > > * “apache-sdap” being the component > > > * “0.4.5a56” being the version > > > * “rc0” being the release candidate label > > > > > > On release, you will copy the artifacts to > > > > > > > > > > > > > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/release/incubator/sdap/apache-sdap-0.4.5a56/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGQnZZiUU$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/release/incubator/sdap/apache-sdap-0.4.5a56/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGQnZZiUU$> > > > > > > Note that ‘dev’ became ‘release’ and the ‘rc0’ label was removed. > > And by > > > the time you release, you'll also need a KEYS file similar to the > > one in > > > the bRPC project: > > > > > > > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/release/incubator/brpc/KEYS__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG858uAsg$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/release/incubator/brpc/KEYS__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG858uAsg$> > > > > > > Browse > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGJkaorxg$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGJkaorxg$> > > and you’ll see that all > > > projects use this directory structure. > > > > > > Let’s look at the artifacts that were in Calcite’s last release. In > > the > > > directory > > > > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc2/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGkK8Q9VQ$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc2/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGkK8Q9VQ$> > > > sh > > > you’ll see the following files: > > > > > > * apache-calcite-1.31.0-src.tar.gz # source tar ball > > > * apache-calcite-1.31.0-src.tar.gz.asc # armored signature > > generated by > > > PGP > > > * apache-calcite-1.31.0-src.tar.gz.sha512 # SHA512 checksum of the > > > src.tar.gz file > > > > > > SDAP will need equivalent files. > > > > > > Now, how to write to dist.apache.org? That web server is a view > > onto an > > > ASF > > > source code repository managed by the Subversion source control > > system. ASF > > > uses it for content management of releases. > > > > > > First, install subversion. "sudo apt-get install subversion” or > > similar. > > > > > > Then checkout the tree: > > > > > > svn co > > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/trunk__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGTWMmcJY$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/trunk__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGTWMmcJY$> > > sdap > > > cd sdap > > > mkdir apache-sdap-0.4.5a56-rc0 > > > > > > Create some files, then check them in: > > > > > > svn add apache-sdap-0.4.5a56-rc0 > > > svn ci -m’Uploading release candidate Apache SDAP > > sdap-0.4.5a56-rc0 to > > > dev area’ apache-sdap-0.4.5a56-rc0 > > > > > > These instructions are from memory, so there might be a few mistakes. > > > Hopefully you get the general idea. Do some Google searches and > > you’ll > > > probably find the release instructions used by other projects. > > > > > > You'll need to log into subversion using your ASF username and > > password, > > > but I don’t remember the details. > > > > > > Be sure to write a ‘how to’ so that the next release manager can > > follow > > > your steps, and add it to the source code when you’re done. And > > maybe one > > > or two shell scripts. > > > > > > I also recommend that you create a bug with the title ‘Release SDAP > > > 0.4.5a56’. It will be a useful place to have discussions, link to > > other > > > bugs, release notes, etc. > > > > > > Julian > > > > > > > > > On Nov 9, 2022, at 3:32 PM, Nga Chung <nch...@apache.org> wrote: > > > > > > I'm going to be the release manager for this first release. Where > > exactly > > > do we upload the 3 .tar.gz (1 per repository) to? > > > > > > Thanks, > > > Nga > > > > > > On Thu, Nov 3, 2022 at 5:12 PM Julian Hyde <jh...@apache.org> wrote: > > > > > > Regarding testing. I recommend that the release manager creates a > > > recipe ("HOWTO") for the steps to create a release. One of those > > steps > > > is a manual smoke test (e.g. am I able to start the server and do x, > > > y, and z simple operations). > > > > > > Other people voting on the release can do their own smoke tests. > > > > > > But do bear in mind that if there are bugs, this does not prevent a > > > release. Clearly you don't want show-stopper bugs like code that > > > doesn't compile. > > > > > > Julian > > > > > > > > > On Wed, Nov 2, 2022 at 3:47 PM Perez, Stepheny K (US 398F) > > > <stepheny.k.pe...@jpl.nasa.gov.invalid> wrote: > > > > > > > > > Hi everyone, > > > > > > I’d like to start the conversation regarding our first official > > Apache > > > > > > release. From what I can tell, these are the major items that need > > to be > > > completed before we can move forward: > > > > > > > > > > > > 1. Identify a release manager. Any volunteers? > > > > > > 2. Create release tarball > > > > > > 3. Write release notes > > > > > > 4. Write installation instructions from source (Riley Kuttruff > > has > > > > > > started this work) > > > > > > > > > 5. Push docker images to Dockerhub (and update quickstart with > > > > > > these versions) > > > > > > > > > Another important task before moving forward would be testing the > > latest > > > > > > SDAP analysis image 0.4.5a56. I have personally used this version > > without > > > any issues. Has anyone else upgraded to this latest alpha version? > > > > > > > > > Thank you! > > > Stepheny > > > > > > > >