Cleanup 1.14.x STATUS vote for back ports
I am inching closer to taking on the RM duties so it would be good if people started to clean up the STATUS file and get things merged. Looking at what is in there ... these items all seem Bindings related so don't they just need the 2 votes? Did someone just forget to move them to the Approved section? * r1878379, r1883719, r1883722, r1884610 Distinguish configure scripts on release mode and non release mode. Justification: Building process should not be prevented by swig installed in users' environment when users use release tar balls. Votes: +1: futatuki, stsp * r1886708 swig-py: Fix dependency of make copy-swig-py target. Justification: Fix a regression introduced between 1.13.0 and 1.14.0. Votes: +1: futatuki, jun66j5 * r1885199 swig-py: Skip some tests on Python 2 if default encoding is 'utf-8'. Justification: Fix check-swig-py not working at all since r1889654 merged from r1889487 (r1889487 depends on r1885199). Votes: +1: jun66j5, futatuki Maybe this one too? * r1887704 Follow up to r1865987, r1866588: Unbreak a msgid. Justification: subversion.pot should be built correctly. Votes: +1: futatuki, jun66j5 Also brane had already given his +1 to this one if the CRLF problems were fixed. His vote could be added and this could be approved: * r1881534 (without CRLF problem) Fix issue #4864 "build/ac-macros/macosx.m4: workaround AC_RUN_IFELSE" Justification: Unblocks cross-compiling SVN. Notes: Replacement for veto-blocked r1881534 group (see below) without the inconsistent line endings that instigated said veto blockage. Branch: 1.14.x-r1881534-no-crlf Votes: +1: hartmannathan, stsp I would like to see the bindings fixes merged so I can get a full build and test suite run of the 1.14.x branch before taking on the RM duties. Thanks Mark
Re: Questions on Release Management Process
Mark Phippard wrote on Wed, 23 Mar 2022 11:36 +00:00: > On Tue, Mar 22, 2022 at 11:51 PM Daniel Shahaf > wrote: >> >> Mark Phippard wrote on Mon, Mar 21, 2022 at 16:46:55 -0400: >> > On Mon, Mar 21, 2022 at 4:31 PM Stefan Sperling wrote: >> > > On Mon, Mar 21, 2022 at 12:44:44PM -0400, Mark Phippard wrote: >> > > > Problem 1: Rolling the tarballs >> > > > >> > > > The process creates the tarballs but fails near the end. It looks GPG >> > > > related? >> > > >> > > >INFO:root:Building Unix tarballs >> > > >INFO:root:Moving artifacts and calculating checksums >> > > >Traceback (most recent call last): >> > > > File "trunk/tools/dist/release.py", line 1916, in >> > > >main() >> > > > File "trunk/tools/dist/release.py", line 1912, in main >> > > >args.func(args) >> > > > File "trunk/tools/dist/release.py", line 983, in roll_tarballs >> > > >download_file(KEYS, filepath, None) >> > > > File "trunk/tools/dist/release.py", line 289, in download_file >> > > >response = urlopen(url) >> > > > File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen >> > > >return opener.open(url, data, timeout) >> > > > File "/usr/lib/python2.7/urllib2.py", line 435, in open >> > > >response = meth(req, response) >> > > > File "/usr/lib/python2.7/urllib2.py", line 548, in http_response >> > > >'http', request, response, code, msg, hdrs) >> > > > File "/usr/lib/python2.7/urllib2.py", line 473, in error >> > > >return self._call_chain(*args) >> > > > File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain >> > > >result = func(*args) >> > > > File "/usr/lib/python2.7/urllib2.py", line 556, in >> > > > http_error_default >> > > >raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) >> > > >urllib2.HTTPError: HTTP Error 404: Not Found >> > > >> > > It seems ASF have removed the KEYS file our script is trying to fetch. >> > > See http://people.apache.org/keys/ where it says "Project group files are >> > > no longer created." >> > > >> > > It looks like what the script wants to do here is obtain a copy of >> > > the Subversion project's KEYS file and store it along with release >> > > artifacts. If we want to keep doing this we will have to maintain >> > > our own KEYS file on the website, I suppose. Otherwise, we could >> > > decide to no longer provide such a file and remove relevant code >> > > from the script. Not sure what is better. >> > >> > Since I do not know what this was used for, maybe someone can help get >> > us to a decision and update the script? Daniel Shahaf, if you see >> > this, I think of you as the resident expert on this stuff. Any thoughts? >> >> The design goals here are two: >> >> 1. When someone with commit access to N ASF projects updates their PGP >> key, they shouldn't have to do O(N) work to update N KEYS files. They >> should have to do either O(1) work or, ideally, nothing at all. >> >> 2. Releases should snapshot the keys that are current at the time they >> are generated, in order to remain verifiable in archive.a.o even if the >> keys in question are later removed from LDAP (by root@ as part of >> a manual password reset, or by the committer via id.a.o). >> >> The ASF-wide "generate group keys" scripts addressed #1. They used to >> generate two files, one with only the full committers' keys and one with >> both full and partial committers' keys. We used the former (to reduce >> attack surface) until it stopped getting generated. >> >> Copying the file to subversion-.KEYS addressed #2. > > Thanks for responding and please forgive some basic questions. My > "knowledge" of this begins and ends with running gpg -ba to sign > releases and copying and pasting the output to an email. > > It sounds like the KEYS file is a list of all of the GPG Keys for ASF > committers? > Yes. It's basically the output of «gpg --armor --export $args» where $args is developers' keys, as recently updated from public keyservers, with cosmetic text added before each block. > What is it used for during the RM process? Is it so that the RM can > validate signatures that come in? Does it get included in the release > somehow? > It's uploaded to dist/ and users are pointed to it. The idea is to establish a trust path for users between "whoever controls the SSL certificate on subversion.apache.org:443" and the particular PGP keys that signed a particular artifact. I assume the RM should verify signatures too, though HACKING doesn't call this out. > >> >> So, what to do? >> >> - We could talk to some ASF-wide list (comdev, infra, site-dev@) about >> resuming generation of group keys files. >> >> (If you do this, someone will probably ask you to comply with >> a markdown document that calls itself policy. I don't believe that >> document is Foundation policy, and even if it is, our practice has >> been +1ed by an Officer of the Foundation.) >> >> - We
Re: Pristines-on-demand: printing progress notifications
Finished and committed: https://svn.apache.org/r1899173 . This looks and feels much better to me now. Example: $ svn revert -R subversion/ Fetching text bases ..done Reverted 'subversion/svn/cat-cmd.c' Reverted 'subversion/svn/diff-cmd.c' Reverted [...] - Julian
Re: Pristines-on-demand: printing progress notifications
Updated and cleaned-up patch 'hydrating-notifications-2.patch' attached, for interest. Still TODO: update test expectations. hydrating-notifications-2.patch Description: Binary data
Re: Pristines-on-demand: printing progress notifications
Daniel Sahlberg wrote: > How does svn cat handle any other informative or warning messages? I don't think 'svn cat' prints any other informative or warning messages. It occurs to me now that anything on stderr is generally assumed to indicate an error, in tests in the test suite. We need to update test expectations anyway (about 30 tests fail when I add the notification to the other commands); this is just an additional point.
Re: Pristines-on-demand: printing progress notifications
Den tors 24 mars 2022 kl 11:34 skrev Julian Foad : > For 'svn diff' especially, if we don't print the notifications, then we > miss out on informing the user during one of the times when it could be > particularly valuable to them. (They are waiting for diff output, which > previously in svn used to come quickly.) > > It would be ugly to print these notifications in the stdout stream. > Diff output format rules do allow it to contain new informative lines > outside the diff hunks, but this kind of notification would not be > useful or nice for consumers of the diff. > I didn't think of these command before but I agree it might be confusing to have these notifications in the middle of the actual data stream. Good catch! > Print progress notifications to stderr? That is what some other > command-line tools do. Some even have command-line flags to control > where it is sent. One precedent we have is warnings, e.g. in "svn > proplist nonexistent-target valid-target" a warning about > 'nonexistent-target' goes to stderr while the primary output for > 'valid-target' goes to stdout. I am not sure how I feel about printing > them to stderr. I could be persuaded either way (print them to stderr or > not at all). > stderr sounds good to me, but I'm maybe not the best judge. At least there is a possibility for the user to separate the different streams. > Anybody want to recommend what we should do for 'cat' and 'diff'? > > For the time being I am going to make 'svn' suppress hydrating > notifications for 'cat' and 'diff'. We will at least get the benefit for > the other subcommands. > How does svn cat handle any other informative or warning messages? This is only a concern for command-line clients. GUIs have other ways to > separate the progress notifications from the primary output. Therefore > this is all a question of what to do in the client layer, the 'svn' > executable; the library will continue to call the notification call-back > if the client supplies one. > +1 /Daniel
Re: Pristines-on-demand: printing progress notifications
For 'svn diff' especially, if we don't print the notifications, then we miss out on informing the user during one of the times when it could be particularly valuable to them. (They are waiting for diff output, which previously in svn used to come quickly.) It would be ugly to print these notifications in the stdout stream. Diff output format rules do allow it to contain new informative lines outside the diff hunks, but this kind of notification would not be useful or nice for consumers of the diff. Print progress notifications to stderr? That is what some other command-line tools do. Some even have command-line flags to control where it is sent. One precedent we have is warnings, e.g. in "svn proplist nonexistent-target valid-target" a warning about 'nonexistent-target' goes to stderr while the primary output for 'valid-target' goes to stdout. I am not sure how I feel about printing them to stderr. I could be persuaded either way (print them to stderr or not at all). Anybody want to recommend what we should do for 'cat' and 'diff'? For the time being I am going to make 'svn' suppress hydrating notifications for 'cat' and 'diff'. We will at least get the benefit for the other subcommands. This is only a concern for command-line clients. GUIs have other ways to separate the progress notifications from the primary output. Therefore this is all a question of what to do in the client layer, the 'svn' executable; the library will continue to call the notification call-back if the client supplies one. - Julian
Re: Pristines-on-demand: printing progress notifications
Den tors 24 mars 2022 kl 09:53 skrev Julian Foad : > Daniel Sahlberg wrote: > >> I'll put it on my todo list, but I can't promise when I find time to > >> to that. > > I only meant to ask you to clarify what you meant: whether you are > reporting for TSVN 1.14 or a TSVN 1.15-dev build (based on the > pristines-on-demand branch or otherwise; I don't know if anyone has made > any such builds); and whether you meant it was reporting bytes-progress > specifically on hydrating or just on *some* data transfers. > Yeah, sorry. I meant that plain 1.14 on 1.14 WCs is reporting bytes transferred. /Daniel
Re: Pristines-on-demand: printing progress notifications
I made a mistake in: > 'svn cat' already suppresses these notifications. CORRECTION: 'svn cat' currently prints these notifications. 'svn diff' currently prints these notifications. --> TODO: stop printing them here. It seems the only change needed here is to make both 'svn diff' AND 'svn cat' suppress those notifications.
Re: Pristines-on-demand: printing progress notifications
Daniel Sahlberg wrote: >> I'll put it on my todo list, but I can't promise when I find time to >> to that. I only meant to ask you to clarify what you meant: whether you are reporting for TSVN 1.14 or a TSVN 1.15-dev build (based on the pristines-on-demand branch or otherwise; I don't know if anyone has made any such builds); and whether you meant it was reporting bytes-progress specifically on hydrating or just on *some* data transfers. * * * * About '--quiet' and when to print these notifications (in the command-line client 'svn'). Commands whose primary purpose is to output data probably should not display the notifications. (If they accept flags like '--quiet' or '--verbose', those are for adjusting their primary information output, not progress notifications. We historically do not have any debug-level verbosity controls on commands like 'diff' and 'cat'. I don't think we need to add such controls before or as part of adding these notifications to the other commands.) 'svn cat' already suppresses these notifications. 'svn diff' currently prints these notifications. --> TODO: stop printing them here. (Those are the only data-output commands that may hydrate.) The other commands probably should print the notifications by default and suppress them when '--quiet' is added. 'svn revert --quiet' already suppresses those notifications. 'svn co/up/sw --quiet' already suppresses those notifications. 'svn resolve[d] --quiet' already suppresses those notifications. Conflict resolver in 'svn merge/up/sw --quiet' already suppresses those notifications. It seems the only change needed here is to make 'svn diff' suppress those notifications. - Julian
Re: Pristines-on-demand: printing progress notifications
Den tors 24 mars 2022 kl 09:04 skrev Julian Foad : > Daniel, can you confirm if you mean you are using a 1.14-based TSVN, on > a 1.15 working copy, and it is showing bytes-transferred feedback during > the hydrating? I had assumed it would need to be rebuilt and modified to > do that. If that part Just Works already, that's great news. I'll put it on my todo list, but I can't promise when I find time to to that. /Daniel
Re: Pristines-on-demand: printing progress notifications
Marl Phippard wrote: > Yes, and I agree --quiet should suppress. [...TSVN] hooked into our > notifications to be providing an update on bytes transferred. [...] I > just assumed they would work ... maybe they still do? Daniel Sahlberg wrote: >> Yes, I like this very much. The more feedback [...] the less >> risk/chance of users believing something is wrong. --quiet also seems >> like a good idea, then powerusers can limit the amount of feedback. OK, seems I should proceed. Thanks for the comments. >> I'm primarily using TortoiseSVN and I can confirm that it is >> displaying the number of bytes trasfered, but I havn't digged through >> the code to find where these notifications come from. Daniel, can you confirm if you mean you are using a 1.14-based TSVN, on a 1.15 working copy, and it is showing bytes-transferred feedback during the hydrating? I had assumed it would need to be rebuilt and modified to do that. If that part Just Works already, that's great news. >> I would want to see the "hydrating" message also in TSVN, but I >> assume it would not be too difficult to hook into these notifications >> (if they are indeed a different kind). Certainly TSVN would need a small update to make it display the new notifications (hydrating start, hydrating file, hydrating end). - Julian
Re: Pristines-on-demand: printing progress notifications
Den ons 23 mars 2022 kl 23:58 skrev Mark Phippard : > On Wed, Mar 23, 2022 at 5:51 PM Julian Foad wrote: > > > I thought maybe we would like to show this detail by default, and > > suppress it when '--quiet' is passed. (Not implemented in this demo > patch.) > > > > I was also mildly surprised to see that the fetches are not necessarily > > all grouped together at the start of a high-level operation. Clearly > > it's doing the 'hydrate' once per explicit target. Actually I remember > > seeing this in earlier review, then forgetting as it didn't seem > > important. I don't know if we care. I suppose it is unavoidable that a > > client's operations might sometimes be grouped like this. Even if the > > client library API allows the client to pass all targets in one call, a > > client in general may choose to call the library function once per > target. > > > > If I were the user, and particularly if I were in a use case where just > > one huge file is involved, then I would also like to have feedback on > > the (percentage) feedback through the file fetch. It doesn't look like > > we have a good way to do that at the moment, but please correct me if > > I'm wrong. > > > > Do we like this? > Yes, I like this very much. The more feedback an application provides, especially with long-running processes, the less risk/chance of users believing something is wrong. --quiet also seems like a good idea, then powerusers can limit the amount of feedback. Yes, and I agree --quiet should suppress. It would be good to > understand the progress feedback a bit more. I have not used > TortoiseSVN in a long time but I recall it hooked into our > notifications to be providing an update on bytes transferred. I think > they would expect that here too. I realize that is kind of separate > from adding these notifications and honestly I thought those > notifications for bytes transferred came from a lower level RA layer > so I just assumed they would work ... maybe they still do? > I'm primarily using TortoiseSVN and I can confirm that it is displaying the number of bytes trasfered, but I havn't digged through the code to find where these notifications come from. I would want to see the "hydrating" message also in TSVN, but I assume it would not be too difficult to hook into these notifications (if they are indeed a different kind). Kind regards, Daniel Sahlberg
