RE: [VOTE] Release Apache Daffodil 3.1.0-rc1

2021-05-12 Thread Interrante, John A (GE Research, US)
+1

I checked (for most items, using WSL2/Ubuntu 20.04 on my work laptop):

[OK] rpm works on Fedora 24 (ran some daffodil CLI commands by hand)
[OK] hash and signature of src.zip is correct
[OK] signature of git tag is correct
[OK] source release matches git tag (minus KEYS file)
[OK] no unexpected binaries in source
[OK] source compiles and all tests pass
[OK] RAT check passes
[OK] jars built from source have the same content as helper binary jars
[OK] src, bins, and jars include correct LICENSE/NOTICE/DISCLAIMER

John

-Original Message-
From: Steve Lawrence  
Sent: Wednesday, May 12, 2021 1:48 PM
To: dev@daffodil.apache.org
Subject: EXT: [VOTE] Release Apache Daffodil 3.1.0-rc1

Hi all,

I'd like to call a vote to release Apache Daffodil 3.1.0-rc1.

All distribution packages, including signatures, digests, etc. can be found at:

https://dist.apache.org/repos/dist/dev/daffodil/3.1.0-rc1/

Staging artifacts can be found at:

https://repository.apache.org/content/repositories/orgapachedaffodil-1020/

This release has been signed with PGP key 36F3494B033AE661, corresponding to 
slawre...@apache.org, which is included in the KEYS file here:

https://downloads.apache.org/daffodil/KEYS

The release candidate has been tagged in git with v3.1.0-rc1.

For reference, here is a list of all closed JIRAs tagged with 3.1.0:

https://s.apache.org/daffodil-issues-3.1.0

For a summary of the changes in this release, see:

https://daffodil.apache.org/releases/3.1.0/

Please review and vote. The vote will be open for at least 72 hours (Saturday, 
15 May 2021, 2pm EST).

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)



[VOTE] Release Apache Daffodil 3.1.0-rc1

2021-05-12 Thread Steve Lawrence
Hi all,

I'd like to call a vote to release Apache Daffodil 3.1.0-rc1.

All distribution packages, including signatures, digests, etc. can be
found at:

https://dist.apache.org/repos/dist/dev/daffodil/3.1.0-rc1/

Staging artifacts can be found at:

https://repository.apache.org/content/repositories/orgapachedaffodil-1020/

This release has been signed with PGP key 36F3494B033AE661,
corresponding to slawre...@apache.org, which is included in the KEYS
file here:

https://downloads.apache.org/daffodil/KEYS

The release candidate has been tagged in git with v3.1.0-rc1.

For reference, here is a list of all closed JIRAs tagged with 3.1.0:

https://s.apache.org/daffodil-issues-3.1.0

For a summary of the changes in this release, see:

https://daffodil.apache.org/releases/3.1.0/

Please review and vote. The vote will be open for at least 72 hours
(Saturday, 15 May 2021, 2pm EST).

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)



Re: release 3.1.0 critical bugs still outstanding

2021-05-12 Thread Steve Lawrence
Yeah, ASF doesn't *require* an isolated installation. This link talks
about securing keys:

  https://infra.apache.org/release-signing.html#where

ASF only recommends removable media or isolated installation, but
doesn't require it. As long as your laptop is secure and your key is
properly protected with a good passphrase and permissions, you should be
fine.

On 5/12/21 12:27 PM, Beckerle, Mike wrote:
> Hmmm. I think isolation just means not group or world readable.
> 
> On Unix/Linux chmod go-rwx.
> The equivalent on Windows I'm not sure of.
> 
> From: Interrante, John A (GE Research, US) 
> Sent: Wednesday, May 12, 2021 12:22 PM
> To: dev@daffodil.apache.org 
> Subject: release 3.1.0 critical bugs still outstanding
> 
> I looked at the instructions for generating an Apache code signing key.  I 
> believe I can do everything on my work laptop except for one thing - storing 
> my $GPGHOME/.gnupg directory with my code signing key in encrypted and 
> isolated storage.  My work laptop's disk is encrypted (per GE security 
> policy) but it doesn't offer isolation unless there's a clever way of 
> enforcing isolation?
> 
> I would need to plug a separate USB device into my work laptop and it would 
> have to be a Yubico YubiKey 5 or a Yubico YubiKey 5C since GE's data guardian 
> software doesn't allow most USB devices to work (YubiKey devices are an 
> exception).
> 
> Do you know of anyone who uses a YukiKey device to generate and/or store 
> their GPG code signing key and how to use a YubiKey device in the release 
> signing workflow?  I found those articles but they don't say much about using 
> the YubiKey in a release signing workflow similar to ours...
> 
> https://support.yubico.com/hc/en-us/articles/360016614840-Code-Signing-with-the-YubiKey-on-Windows
> https://ocramius.github.io/blog/yubikey-for-ssh-gpg-git-and-local-login/
> https://eclipsesource.com/blogs/2016/11/25/yubikey-code-signing-with-a-smart-card/
> 
> John
> 
> -Original Message-
> From: Steve Lawrence 
> Sent: Tuesday, May 11, 2021 10:23 AM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
> 
> Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't take 
> too long to get it merged.
> 
> I'll volunteer to be release manager 3.1.0. Probably for the best since I'm 
> probably the most familiar with the process/script, and it's possible 
> something could be broken with this being the first non-incubator release.
> 
> Though, I would recommend that other PMC/committers go through the process to 
> create and publish a gpg key so that when it does come time to do a release, 
> at least that part is out of the way.
> 
> 
> On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
>> Yes, we will be in good shape for the 3.1.0 release after we update the CLI 
>> help information, which I'd like to complete today.  We've also got at least 
>> 4-5 days to add validation.md and anything else we want to the daffodil-site 
>> before the earliest possible official release announcement could go out.
>>
>> I'm taking this Thursday and Friday off which is a consideration in 
>> volunteering to be the release manager.  I'd have to generate a signing key, 
>> add its public part to the KEYS file, commit it to the Daffodil release 
>> distribution SVN repository, send the fingerprint to the Apache key server, 
>> build a release candidate, and start a vote before I go out of town on 
>> Thursday.  Steve, perhaps I should wait for the following release unless you 
>> think I'd be able to do all these steps quickly within a few hours.
>>
>> John
>>
>> -Original Message-
>> From: Beckerle, Mike 
>> Sent: Monday, May 10, 2021 2:43 PM
>> To: dev@daffodil.apache.org
>> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>>
>> I agree we should do the release.
>>
>> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of 
>> refactoring here, 30 files touched, changes in diagnostic behavior, etc. 
>> Perhaps best to put it off until after the 3.1.0 release.
>>
>>
>>
>>
>> 
>> From: Steve Lawrence 
>> Sent: Monday, May 10, 2021 1:59 PM
>> To: dev@daffodil.apache.org 
>> Subject: Re: release 3.1.0 critical bugs still outstanding
>>
>> We have fixed the later two mentioned issues. The current list of critical 
>> issues is now:
>>
>> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
>> DAFFODIL-2400: New SAX API causes performance degredations.
>> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
>>
>> I agree the the first two can likely be postponed without issue. The last 
>> one doesn't even seem critical to me, unless there are very important 
>> formats that require this functions that I'm not aware of. I suggest we also 
>> postpone that ticket as well.
>>
>> If others agree, I think we are ready for the 3.1.0 release?
>>
>> Does any want to volunteer 

Re: release 3.1.0 critical bugs still outstanding

2021-05-12 Thread Beckerle, Mike
Hmmm. I think isolation just means not group or world readable.

On Unix/Linux chmod go-rwx.
The equivalent on Windows I'm not sure of.

From: Interrante, John A (GE Research, US) 
Sent: Wednesday, May 12, 2021 12:22 PM
To: dev@daffodil.apache.org 
Subject: release 3.1.0 critical bugs still outstanding

I looked at the instructions for generating an Apache code signing key.  I 
believe I can do everything on my work laptop except for one thing - storing my 
$GPGHOME/.gnupg directory with my code signing key in encrypted and isolated 
storage.  My work laptop's disk is encrypted (per GE security policy) but it 
doesn't offer isolation unless there's a clever way of enforcing isolation?

I would need to plug a separate USB device into my work laptop and it would 
have to be a Yubico YubiKey 5 or a Yubico YubiKey 5C since GE's data guardian 
software doesn't allow most USB devices to work (YubiKey devices are an 
exception).

Do you know of anyone who uses a YukiKey device to generate and/or store their 
GPG code signing key and how to use a YubiKey device in the release signing 
workflow?  I found those articles but they don't say much about using the 
YubiKey in a release signing workflow similar to ours...

https://support.yubico.com/hc/en-us/articles/360016614840-Code-Signing-with-the-YubiKey-on-Windows
https://ocramius.github.io/blog/yubikey-for-ssh-gpg-git-and-local-login/
https://eclipsesource.com/blogs/2016/11/25/yubikey-code-signing-with-a-smart-card/

John

-Original Message-
From: Steve Lawrence 
Sent: Tuesday, May 11, 2021 10:23 AM
To: dev@daffodil.apache.org
Subject: EXT: Re: release 3.1.0 critical bugs still outstanding

Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't take 
too long to get it merged.

I'll volunteer to be release manager 3.1.0. Probably for the best since I'm 
probably the most familiar with the process/script, and it's possible something 
could be broken with this being the first non-incubator release.

Though, I would recommend that other PMC/committers go through the process to 
create and publish a gpg key so that when it does come time to do a release, at 
least that part is out of the way.


On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
> Yes, we will be in good shape for the 3.1.0 release after we update the CLI 
> help information, which I'd like to complete today.  We've also got at least 
> 4-5 days to add validation.md and anything else we want to the daffodil-site 
> before the earliest possible official release announcement could go out.
>
> I'm taking this Thursday and Friday off which is a consideration in 
> volunteering to be the release manager.  I'd have to generate a signing key, 
> add its public part to the KEYS file, commit it to the Daffodil release 
> distribution SVN repository, send the fingerprint to the Apache key server, 
> build a release candidate, and start a vote before I go out of town on 
> Thursday.  Steve, perhaps I should wait for the following release unless you 
> think I'd be able to do all these steps quickly within a few hours.
>
> John
>
> -Original Message-
> From: Beckerle, Mike 
> Sent: Monday, May 10, 2021 2:43 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>
> I agree we should do the release.
>
> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of 
> refactoring here, 30 files touched, changes in diagnostic behavior, etc. 
> Perhaps best to put it off until after the 3.1.0 release.
>
>
>
>
> 
> From: Steve Lawrence 
> Sent: Monday, May 10, 2021 1:59 PM
> To: dev@daffodil.apache.org 
> Subject: Re: release 3.1.0 critical bugs still outstanding
>
> We have fixed the later two mentioned issues. The current list of critical 
> issues is now:
>
> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
> DAFFODIL-2400: New SAX API causes performance degredations.
> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
>
> I agree the the first two can likely be postponed without issue. The last one 
> doesn't even seem critical to me, unless there are very important formats 
> that require this functions that I'm not aware of. I suggest we also postpone 
> that ticket as well.
>
> If others agree, I think we are ready for the 3.1.0 release?
>
> Does any want to volunteer to be the release manager? I've done it a handful 
> of times so don't mind, but it might be good to get others experience 
> depending on availability. By this point, the workflow is pretty well 
> documented here:
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
>
>
> On 5/3/21 5:25 PM, Beckerle, Mike wrote:
>> Of the 4 remaining "critical bugs or improvements" I think we should 
>> postpone and release note these first two:
>>
>>   *
>>
>>   *   Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New 
>> SAX API 

release 3.1.0 critical bugs still outstanding

2021-05-12 Thread Interrante, John A (GE Research, US)
I looked at the instructions for generating an Apache code signing key.  I 
believe I can do everything on my work laptop except for one thing - storing my 
$GPGHOME/.gnupg directory with my code signing key in encrypted and isolated 
storage.  My work laptop's disk is encrypted (per GE security policy) but it 
doesn't offer isolation unless there's a clever way of enforcing isolation?

I would need to plug a separate USB device into my work laptop and it would 
have to be a Yubico YubiKey 5 or a Yubico YubiKey 5C since GE's data guardian 
software doesn't allow most USB devices to work (YubiKey devices are an 
exception).  

Do you know of anyone who uses a YukiKey device to generate and/or store their 
GPG code signing key and how to use a YubiKey device in the release signing 
workflow?  I found those articles but they don't say much about using the 
YubiKey in a release signing workflow similar to ours...

https://support.yubico.com/hc/en-us/articles/360016614840-Code-Signing-with-the-YubiKey-on-Windows
https://ocramius.github.io/blog/yubikey-for-ssh-gpg-git-and-local-login/
https://eclipsesource.com/blogs/2016/11/25/yubikey-code-signing-with-a-smart-card/

John

-Original Message-
From: Steve Lawrence  
Sent: Tuesday, May 11, 2021 10:23 AM
To: dev@daffodil.apache.org
Subject: EXT: Re: release 3.1.0 critical bugs still outstanding

Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't take 
too long to get it merged.

I'll volunteer to be release manager 3.1.0. Probably for the best since I'm 
probably the most familiar with the process/script, and it's possible something 
could be broken with this being the first non-incubator release.

Though, I would recommend that other PMC/committers go through the process to 
create and publish a gpg key so that when it does come time to do a release, at 
least that part is out of the way.


On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
> Yes, we will be in good shape for the 3.1.0 release after we update the CLI 
> help information, which I'd like to complete today.  We've also got at least 
> 4-5 days to add validation.md and anything else we want to the daffodil-site 
> before the earliest possible official release announcement could go out.
> 
> I'm taking this Thursday and Friday off which is a consideration in 
> volunteering to be the release manager.  I'd have to generate a signing key, 
> add its public part to the KEYS file, commit it to the Daffodil release 
> distribution SVN repository, send the fingerprint to the Apache key server, 
> build a release candidate, and start a vote before I go out of town on 
> Thursday.  Steve, perhaps I should wait for the following release unless you 
> think I'd be able to do all these steps quickly within a few hours.
> 
> John
> 
> -Original Message-
> From: Beckerle, Mike 
> Sent: Monday, May 10, 2021 2:43 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
> 
> I agree we should do the release.
> 
> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of 
> refactoring here, 30 files touched, changes in diagnostic behavior, etc. 
> Perhaps best to put it off until after the 3.1.0 release.
> 
> 
> 
> 
> 
> From: Steve Lawrence 
> Sent: Monday, May 10, 2021 1:59 PM
> To: dev@daffodil.apache.org 
> Subject: Re: release 3.1.0 critical bugs still outstanding
> 
> We have fixed the later two mentioned issues. The current list of critical 
> issues is now:
> 
> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
> DAFFODIL-2400: New SAX API causes performance degredations.
> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
> 
> I agree the the first two can likely be postponed without issue. The last one 
> doesn't even seem critical to me, unless there are very important formats 
> that require this functions that I'm not aware of. I suggest we also postpone 
> that ticket as well.
> 
> If others agree, I think we are ready for the 3.1.0 release?
> 
> Does any want to volunteer to be the release manager? I've done it a handful 
> of times so don't mind, but it might be good to get others experience 
> depending on availability. By this point, the workflow is pretty well 
> documented here:
> 
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
> 
> 
> On 5/3/21 5:25 PM, Beckerle, Mike wrote:
>> Of the 4 remaining "critical bugs or improvements" I think we should 
>> postpone and release note these first two:
>>
>>   *
>>
>>   *   Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New 
>> SAX API causes performance degradations.
>>  *It is a mystery why the SAX API is slower. The whole point of SAX 
>> is lighter weight.
>>   *   Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - 
>> disallow doctype decls in all XML and XSD we read in.
>>  *   Assigned to Mike Beckerle. Unlikely