Yes, docker should give you a clean slate every time you use it (with
the exception of the gradle-cache volume to speed things up by reusing
the cached items), so I am not quite sure why it's causing a permission
issue with the home directory on your machine.
As for the passwords in a single text file issue, I think gradle offers
some facility to encrypt the file, but I've not used it personally.
So sorry you seem to be running into these issues. Personally, I've
found the docker-compose method to automate a lot of things I would
otherwise do manually and cuts down the time I need to create a release
significantly.
Francis
On 5/10/2021 7:46 pm, Julian Hyde wrote:
I thought the purpose of using Docker was to create a simpler, more
reproducible environment. I don't use Docker regularly (in fact, my
employer bans it from my work machine) and so, from my perspective,
any use of Docker makes the process more complicated.
I also hate having to put all of my most important passwords into a
single text file to make this process work.
Julian
On Tue, Oct 5, 2021 at 1:39 AM Francis Chuang <francischu...@apache.org> wrote:
Hey Julian,
Thanks for reporting that. I think the instructions in the howto needs
to be clearer regarding the steps to start asflike-release-environment.
I think we should put the actual commands used to start the mock servers
directly in there.
Hmm, the docker build should not write to your home directory at all. It
creates a separate volume to mount into the container and uses that as
the home directory in the container, so it should never touch your real
home directory.
I have been using it for the past few releases and it's always worked
for me. My only suggestion if you're still willing to try is run
`docker-compose down` from the root directory of the git repo to clear
all containers and to remove the `gradle-cache` from docker, then
starting over from there.
As for pushing to gitbox, I think you can use
-Pasf.git.pushRepositoryProvider=GITBOX, that's what I have for docker:
https://github.com/apache/calcite-avatica/blob/89e0deb510311b85b8c8bacde6d2ff70c309930e/docker.sh#L251
Francis
On 5/10/2021 7:24 pm, Julian Hyde wrote:
Francis,
That helped somewhat. I was able to get the Docker-powered server
running using your instructions. (From reading the HOWTO, I had no
idea that I was creating a server in a separate terminal.) I was
running Tomcat on port 80, and so I had to shut that down to prevent
port clashes.
But then:
$ ./gradlew prepareVote -Prc=0
Starting a Gradle Daemon (subsequent builds will be faster)
FAILURE: Build failed with an exception.
* What went wrong:
Gradle could not start your build.
Cannot create service of type DefaultConfigurationCache using
DefaultConfigurationCache constructor as there is a problem with parameter #10
of type ConfigurationCacheFingerprintController.
> Cannot create service of type
ConfigurationCacheFingerprintController using
ConfigurationCacheFingerprintController constructor as there is a
problem with parameter #5 of type FileCollectionFingerprinterRegistry.
> Cannot create service of type
FileCollectionFingerprinterRegistry using method
VirtualFileSystemServices$BuildSessionServices.createFileCollectionFingerprinterRegistry()
as there is a problem with parameter #1 of type
List<FileCollectionFingerprinter>.
> Could not create service of type CrossBuildFileHashCache
using BuildSessionServices.createCrossBuildFileHashCache().
> Failed to create parent directory
'/home/jhyde/dev/calcite-avatica/.gradle/6.8.1' when creating
directory '/home/jhyde/dev/calcite-avatica/.gradle/6.8.1/fileHashes'
I think the Docker build had created a .gradle directory as root, and
when I ran as myself, I couldn't write to it. And then I gave up
trying to do 'dry runs' and added the -Pasf flag to do it for real:
$ ./gradlew prepareVote -Prc=0 -Pasf
Task :pushRcTag FAILED
Build calcite-avatica FAILURE reason:
Execution failed for task ':pushRcTag':
Caused by: org.eclipse.jgit.api.errors.TransportException:
https://github.com/apache/calcite-avatica.git: not authorized
at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:180)
at
com.github.vlsi.gradle.release.jgit.dsl.GitExtensionsKt.push(GitExtensions.kt:132)
at
com.github.vlsi.gradle.release.GitPushTask$pushTag$1.invoke(GitPushTask.kt:54)
After an hour checking my github username and password, I found this:
$ git push origin test-tag-1
Username for 'https://github.com': julianhyde
Password for 'https://julianh...@github.com':
remote: Support for password authentication was removed on August 13,
2021. Please use a personal access token instead.
remote: Please see
https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/
for more information.
fatal: Authentication failed for 'https://github.com/apache/calcite-avatica/'
If the remote was
https://gitbox.apache.org/repos/asf/calcite-avatica.git or
g...@github.com:apache/calcite-avatica.git I would probably hve been
able to authenticate, but it seems to be hard-wired to
https://github.com/apache/calcite-avatica.
There's a whole day gone up in smoke. Sometimes the 'productivity
gains' of Gradle + Docker are pretty hard to discern.
Julian
On Mon, Oct 4, 2021 at 2:25 PM Francis Chuang <francischu...@apache.org> wrote:
Hey Julian,
I just tested everything in docker and it works if I do the following:
Terminal 1:
git clone https://github.com/vlsi/asflike-release-environment
cd asflike-release-environment
./recreate.sh calcite-avatica
... wait for the mock release environment to finish starting ...
Terminal 2:
git clone https://github.com/apache/calcite-avatica
cd calcite-avatica
docker-compose run -v ~/.gnupg:/.gnupg dry-run
This is the result:
--- snipped ---
> Task :createRcTag
Created tag avatica-1.19.0-rc0 ->
Ref[refs/tags/avatica-1.19.0-rc0=2df9697c97350c9e92f9336c2b8c1270076e45a9(-1)]
> Task :pushRcTag
Pushing tag to Git remote release-origin-test:
git://127.0.0.1/calcite-avatica.git
Message from release-origin-test:
refs/tags/avatica-1.19.0-rc0: OK,
2df9697c97350c9e92f9336c2b8c1270076e45a9 (fastForward)
> Task :generateVoteText
Listing SVN folders [] at
http://127.0.0.1/svn/dist/dev/calcite/apache-calcite-avatica-1.19.0-rc0
Fetching
http://127.0.0.1/svn/dist/dev/calcite/apache-calcite-avatica-1.19.0-rc0/apache-calcite-avatica-1.19.0-src.tar.gz.sha512@2
Please find draft vote text in /src/build/prepareVote/mail.txt
> Task :prepareVote
--- vote email snipped ---
Deprecated Gradle features were used in this build, making it
incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 5m 8s
105 actionable tasks: 105 executed
Francis
On 5/10/2021 7:40 am, Stamatis Zampetakis wrote:
I haven't done an avatica release so I don't know if using docker is must
but maybe running directly the prepareVote task could work:
./gradlew prepareVote -Prc=0 -Pasf
It may complain about missing properties. I have the following on my
~/.gradle/gradle.properties file:
useGpgCmd=true
signing.gnupg.executable=gpg
signing.gnupg.useLegacyGpg=false
signing.gnupg.keyName=D77C3383F1927570
signing.gnupg.passphrase=XXXX
asfSvnUsername=zabetak
asfSvnPassword=YYY
asfGitSourceUsername=zabetak
asfGitSourcePassword=ZZZZ
asfNexusUsername=zabetak
asfNexusPassword=YYY
Best,
Stamatis
On Mon, Oct 4, 2021 at 10:03 PM Julian Hyde <jh...@apache.org> wrote:
I'm still running into this 'Connection refused' error. It's a
show-stopper. I cannot make a release candidate unless this problem is
solved.
On Mon, Oct 4, 2021 at 10:58 AM Julian Hyde <jh...@apache.org> wrote:
This is the command I used:
docker-compose run -v ~/.gnupg:/.gnupg dry-run
I'm running Docker 18.06.1-ce on Ubuntu 20.04.3 LTS.
By the way, I was thinking of creating a "stage" branch in the Avatica
git repo. We no longer use a branch per release (which is fine) but
this means that the commit is not pushed to the git repo until after
the vote passes. I propose to push the RC commit to the "stage" branch
(and destructively force push if we need another RC) to give CI a
chance to run on the RC. After the vote passes I will push a tag.
Julian
On Mon, Oct 4, 2021 at 10:48 AM Vladimir Sitnikov
<sitnikov.vladi...@gmail.com> wrote:
I'm not sure which steps are you following, however, the same sequence
seems to work fine:
Commands:
https://github.com/vlsi/vlsi-release-plugins/blob/83c85c5faa4c7cd1fe0173b75c1cba5e60c3f209/.github/workflows/release-test.yml#L33-L60
Logs:
https://github.com/vlsi/vlsi-release-plugins/runs/3794304055?check_suite_focus=true
Vladimir
пн, 4 окт. 2021 г. в 20:15, Julian Hyde <jh...@apache.org>:
As release manager for the upcoming Avatica 1.19, I just tried to use
the docker-based dry-run. I got the following failure:
Build calcite-avatica FAILURE reason:
Execution failed for task ':initializeNexusStagingRepository':
java.io.UncheckedIOException: java.net.ConnectException:
Failed to connect to /127.0.0.1:8080
at
de.marcphilipp.gradle.nexus.internal.NexusClient.findStagingProfileId(NexusClient.kt:84)
at
de.marcphilipp.gradle.nexus.InitializeNexusStagingRepository.determineStagingProfileId(InitializeNexusStagingRepository.kt:113)
at
de.marcphilipp.gradle.nexus.InitializeNexusStagingRepository.access$determineStagingProfileId(InitializeNexusStagingRepository.kt:38)
I don't want to spend a morning debugging Docker port-mappings so I'm
moving on.
Julian
