Did some testing with rc1 today.
- Validated rc tarballs
- Successfully built and ran the release
- Tested SDAP ingest successfully
- Tested the following SDAP endpoints:
-/match_spark
-/cdmssubset
-/domsresults
-/domslist
-/list
-/apidocs
-/timeSeriesSpark
I performed the endpoint tests using the demo Jupyter notebook used in the
quickstart guide and some test scripts I wrote for a project that uses SDAP.
Riley
On 2022/12/06 07:23:57 "Perez, Stepheny K (US 398F)" wrote:
> Hi Nga,
>
> I’ve had success testing rc0 – the following analysis endpoints were used in
> my tests:
>
>
> * match_spark
> * cdmssubsetter
> * cdmsresults
>
> Has anyone tested the time series analysis endpoints?
>
> Thanks,
> Stepheny
>
> From: Nga Chung <[email protected]>
> Date: Monday, December 5, 2022 at 11:06 PM
> To: [email protected] <[email protected]>
> Subject: Re: [EXTERNAL] Re: [DISCUSS] Towards release 0.4.5a56
> Riley, thank you for adding all the missing ASF header and the NOTICE.
>
> Riley's changes have been merged and rc1 artifacts have been uploaded to
> https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc1/__;!!PvBDto6Hs4WbVuu7!dPruOg9-ncxdgZW_4Gb72i05RAtnn9YBnohlQjSwWdWTM6KCo5J3490Ih-RHUOkm66RM11OAKhc$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc1/__;!!PvBDto6Hs4WbVuu7!dPruOg9-ncxdgZW_4Gb72i05RAtnn9YBnohlQjSwWdWTM6KCo5J3490Ih-RHUOkm66RM11OAKhc$>
>
>
> Has anyone had any success testing rc0?
>
> Julian, any suggestions on next steps?
>
> Thanks,
> Nga
>
> On Thu, Dec 1, 2022 at 1:03 PM Kuttruff, Riley K (US 398F-Affiliate)
> <[email protected]> wrote:
>
> > It appears some of the source files are missing the ASF header. I did a
> > check for all the Python files across the repositories and added them where
> > needed in a pair of PRs (nexus & ingester; nexusproto had no .py files
> > missing the header).
> >
> > I still have yet to check for any non-python source files.
> >
> > Riley
> >
> > On 11/30/22, 4:29 PM, "Nga Chung" <[email protected]> wrote:
> >
> > Hi everyone,
> >
> > So this email thread probably needs renaming or we can start a new one
> > if
> > we're proceeding with a Version 1.0.0 release instead of 0.4.5a56.
> >
> > Anyways, with help from many folks we now have a release candidate
> > that can
> > be found here:
> >
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGjkf-Wmw$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-1.0.0-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGjkf-Wmw$>
> >
> >
> > Instructions for building docker images from source can be found here:
> >
> > https://urldefense.us/v3/__https://incubator-sdap-nexus.readthedocs.io/en/latest/build.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGMdzQUnc$<https://urldefense.us/v3/__https:/incubator-sdap-nexus.readthedocs.io/en/latest/build.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGMdzQUnc$>
> > Instructions for deploying locally to test can be found here:
> >
> > https://urldefense.us/v3/__https://incubator-sdap-nexus.readthedocs.io/en/latest/quickstart.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG-tQ0nvg$<https://urldefense.us/v3/__https:/incubator-sdap-nexus.readthedocs.io/en/latest/quickstart.html__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG-tQ0nvg$>
> > Associated docker images can be found here:
> >
> > https://urldefense.us/v3/__https://hub.docker.com/search?q=apache*2Fsdap__;JQ!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG5KMpZEg$<https://urldefense.us/v3/__https:/hub.docker.com/search?q=apache*2Fsdap__;JQ!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG5KMpZEg$>
> >
> > Here's how I created rc0. I will get these instructions into github but
> > wanted to get all this out first for your review.
> >
> > git clone --branch release/1.0.0
> >
> > https://urldefense.us/v3/__https://github.com/apache/incubator-sdap-nexusproto.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGbmDU9OM$<https://urldefense.us/v3/__https:/github.com/apache/incubator-sdap-nexusproto.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGbmDU9OM$>
> >
> > cd incubator-sdap-nexusproto*/*
> >
> > git ls-files > /tmp/manifest.txt
> >
> > tar cvfz apache-sdap-nexusproto-1.0.0-src.tar.gz -T /tmp/manifest.txt
> >
> > gpg --armor --output apache-sdap-nexusproto-1.0.0-src.tar.gz.asc
> > --detach-sig apache-sdap-nexusproto-1.0.0-src.tar.gz
> >
> > shasum -a 512 apache-sdap-nexusproto-1.0.0-src.tar.gz >
> > apache-sdap-nexusproto-1.0.0-src.tar.gz.sha512
> >
> >
> > git clone --branch release/1.0.0
> >
> > https://urldefense.us/v3/__https://github.com/apache/incubator-sdap-ingester.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGgkAuYFg$<https://urldefense.us/v3/__https:/github.com/apache/incubator-sdap-ingester.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGgkAuYFg$>
> >
> > cd incubator-sdap-ingester/
> >
> > git ls-files > /tmp/manifest.txt
> >
> > tar cvfz apache-sdap-ingester-1.0.0-src.tar.gz -T /tmp/manifest.txt
> >
> > gpg --armor --output apache-sdap-ingester-1.0.0-src.tar.gz.asc
> > --detach-sig
> > apache-sdap-ingester-1.0.0-src.tar.gz
> >
> > shasum -a 512 apache-sdap-ingester-1.0.0-src.tar.gz >
> > apache-sdap-ingester-1.0.0-src.tar.gz.sha512
> >
> >
> > git clone --branch release/1.0.0
> >
> > https://urldefense.us/v3/__https://github.com/apache/incubator-sdap-nexus.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG19U3rPQ$<https://urldefense.us/v3/__https:/github.com/apache/incubator-sdap-nexus.git__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG19U3rPQ$>
> >
> > cd incubator-sdap-nexus/
> >
> > git ls-files > /tmp/manifest.txt
> >
> > tar cvfz apache-sdap-nexus-1.0.0-src.tar.gz -T /tmp/manifest.txt
> >
> > gpg --armor --output apache-sdap-nexus-1.0.0-src.tar.gz.asc
> > --detach-sig
> > apache-sdap-nexus-1.0.0-src.tar.gz
> >
> > shasum -a 512 apache-sdap-nexus-1.0.0-src.tar.gz >
> > apache-sdap-nexus-1.0.0-src.tar.gz.sha512
> >
> > svn co
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG8upBcY0$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG8upBcY0$>
> > sdap
> > mkdir sdap/apache-sdap-1.0.0-rc0
> > cp incubator-sdap-nexusproto/apache-sdap-nexusproto-1.0.0-src.tar.gz*
> > sdap/apache-sdap-1.0.0-rc0/.
> > cp incubator-sdap-ingester/apache-sdap-ingester-1.0.0-src.tar.gz*
> > sdap/apache-sdap-1.0.0-rc0/.
> > cp incubator-sdap-nexus/apache-sdap-nexus-1.0.0-src.tar.gz*
> > sdap/apache-sdap-1.0.0-rc0/.
> >
> > svn add apache-sdap-1.0.0-rc0
> >
> > svn ci -m "Uploading release candidate Apache SDAP
> > apache-sdap-1.0.0-rc0 to
> > dev area" apache-sdap-1.0.0-rc0
> >
> >
> > Thanks,
> > Nga
> >
> > On Wed, Nov 9, 2022 at 4:26 PM Julian Hyde <[email protected]>
> > wrote:
> >
> > > hanks for volunteering to be release manager!
> > >
> > > The artifacts to be voted on will be in the following directory (or
> > > something very much like it):
> > >
> > >
> > >
> > >
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-0.4.5a56-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGsl1T6OQ$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/apache-sdap-0.4.5a56-rc0/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGsl1T6OQ$>
> > >
> > > with
> > > * “dev” being the place for candidate releases (to be replaced by
> > “release”
> > > when the release is final)
> > > * “sdap” being the project name (prefixed with “incubator/“ while
> > sdap is
> > > incubating)
> > > * “apache-sdap” being the component
> > > * “0.4.5a56” being the version
> > > * “rc0” being the release candidate label
> > >
> > > On release, you will copy the artifacts to
> > >
> > >
> > >
> > >
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/release/incubator/sdap/apache-sdap-0.4.5a56/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGQnZZiUU$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/release/incubator/sdap/apache-sdap-0.4.5a56/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGQnZZiUU$>
> > >
> > > Note that ‘dev’ became ‘release’ and the ‘rc0’ label was removed.
> > And by
> > > the time you release, you'll also need a KEYS file similar to the
> > one in
> > > the bRPC project:
> > >
> > >
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/release/incubator/brpc/KEYS__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG858uAsg$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/release/incubator/brpc/KEYS__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtG858uAsg$>
> > >
> > > Browse
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGJkaorxg$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGJkaorxg$>
> > and you’ll see that all
> > > projects use this directory structure.
> > >
> > > Let’s look at the artifacts that were in Calcite’s last release. In
> > the
> > > directory
> > >
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc2/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGkK8Q9VQ$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc2/__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGkK8Q9VQ$>
> > > sh
> > > you’ll see the following files:
> > >
> > > * apache-calcite-1.31.0-src.tar.gz # source tar ball
> > > * apache-calcite-1.31.0-src.tar.gz.asc # armored signature
> > generated by
> > > PGP
> > > * apache-calcite-1.31.0-src.tar.gz.sha512 # SHA512 checksum of the
> > > src.tar.gz file
> > >
> > > SDAP will need equivalent files.
> > >
> > > Now, how to write to dist.apache.org? That web server is a view
> > onto an
> > > ASF
> > > source code repository managed by the Subversion source control
> > system. ASF
> > > uses it for content management of releases.
> > >
> > > First, install subversion. "sudo apt-get install subversion” or
> > similar.
> > >
> > > Then checkout the tree:
> > >
> > > svn co
> > https://urldefense.us/v3/__https://dist.apache.org/repos/dist/dev/incubator/sdap/trunk__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGTWMmcJY$<https://urldefense.us/v3/__https:/dist.apache.org/repos/dist/dev/incubator/sdap/trunk__;!!PvBDto6Hs4WbVuu7!aY2tIy4CD6wRO7JlwQ9_XSOpDkqvhu8QMq2ROLe907bKV-UqYPygNoR8NjtGTWMmcJY$>
> > sdap
> > > cd sdap
> > > mkdir apache-sdap-0.4.5a56-rc0
> > >
> > > Create some files, then check them in:
> > >
> > > svn add apache-sdap-0.4.5a56-rc0
> > > svn ci -m’Uploading release candidate Apache SDAP
> > sdap-0.4.5a56-rc0 to
> > > dev area’ apache-sdap-0.4.5a56-rc0
> > >
> > > These instructions are from memory, so there might be a few mistakes.
> > > Hopefully you get the general idea. Do some Google searches and
> > you’ll
> > > probably find the release instructions used by other projects.
> > >
> > > You'll need to log into subversion using your ASF username and
> > password,
> > > but I don’t remember the details.
> > >
> > > Be sure to write a ‘how to’ so that the next release manager can
> > follow
> > > your steps, and add it to the source code when you’re done. And
> > maybe one
> > > or two shell scripts.
> > >
> > > I also recommend that you create a bug with the title ‘Release SDAP
> > > 0.4.5a56’. It will be a useful place to have discussions, link to
> > other
> > > bugs, release notes, etc.
> > >
> > > Julian
> > >
> > >
> > > On Nov 9, 2022, at 3:32 PM, Nga Chung <[email protected]> wrote:
> > >
> > > I'm going to be the release manager for this first release. Where
> > exactly
> > > do we upload the 3 .tar.gz (1 per repository) to?
> > >
> > > Thanks,
> > > Nga
> > >
> > > On Thu, Nov 3, 2022 at 5:12 PM Julian Hyde <[email protected]> wrote:
> > >
> > > Regarding testing. I recommend that the release manager creates a
> > > recipe ("HOWTO") for the steps to create a release. One of those
> > steps
> > > is a manual smoke test (e.g. am I able to start the server and do x,
> > > y, and z simple operations).
> > >
> > > Other people voting on the release can do their own smoke tests.
> > >
> > > But do bear in mind that if there are bugs, this does not prevent a
> > > release. Clearly you don't want show-stopper bugs like code that
> > > doesn't compile.
> > >
> > > Julian
> > >
> > >
> > > On Wed, Nov 2, 2022 at 3:47 PM Perez, Stepheny K (US 398F)
> > > <[email protected]> wrote:
> > >
> > >
> > > Hi everyone,
> > >
> > > I’d like to start the conversation regarding our first official
> > Apache
> > >
> > > release. From what I can tell, these are the major items that need
> > to be
> > > completed before we can move forward:
> > >
> > >
> > >
> > > 1. Identify a release manager. Any volunteers?
> > >
> > > 2. Create release tarball
> > >
> > > 3. Write release notes
> > >
> > > 4. Write installation instructions from source (Riley Kuttruff
> > has
> > >
> > > started this work)
> > >
> > >
> > > 5. Push docker images to Dockerhub (and update quickstart with
> > >
> > > these versions)
> > >
> > >
> > > Another important task before moving forward would be testing the
> > latest
> > >
> > > SDAP analysis image 0.4.5a56. I have personally used this version
> > without
> > > any issues. Has anyone else upgraded to this latest alpha version?
> > >
> > >
> > > Thank you!
> > > Stepheny
> > >
> >
> >
>
