Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher
Besides those 2 bugs, I think we should also merge https://github.com/apache/daffodil/pull/490 which adds a new charset (EBCDIC). We should roll this forward if susmita can't take it up. From: Beckerle, Mike Sent: Tuesday, April 20, 2021 3:49 PM To: dev@daffodil.apache.org Subject: Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher Looking at the critical JIRA tickets, they are: DAFFODIL-1422<https://issues.apache.org/jira/browse/DAFFODIL-1422> disallow doctype decls in all XML & XSD that we read in DAFFODIL-2473<https://issues.apache.org/jira/browse/DAFFODIL-2473> Need bit-wise AND, OR, NOT, and shift operations DAFFODIL-2183<https://issues.apache.org/jira/browse/DAFFODIL-2183> Unparse nilled complex element fails. DAFFODIL-1598<https://issues.apache.org/jira/browse/DAFFODIL-1598> Unparser: For strings that truncate, the dfdl:valueLength function cannot suspend DAFFODIL-2399<https://issues.apache.org/jira/browse/DAFFODIL-2399> Error diagnostics output even though there is an infoset DAFFODIL-2400<https://issues.apache.org/jira/browse/DAFFODIL-2400> New SAX API causes performance degredations. DAFFODIL-1971<https://issues.apache.org/jira/browse/DAFFODIL-1971> Statement order of evaluation not per DFDL Spec I'm going to suggest how we handle each of these for the 3.1.0 release. I end up with 2 bugs to fix. Keep in mind this is discussion fodder. DAFFODIL-1422<https://issues.apache.org/jira/browse/DAFFODIL-1422> disallow doctype decls in all XML & XSD that we read in * release note it - try to fix for 3.2.0 DAFFODIL-2473<https://issues.apache.org/jira/browse/DAFFODIL-2473> Need bit-wise AND, OR, NOT, and shift operations * change to priority major - we need better documentation of the use case driving this. DAFFODIL-2183<https://issues.apache.org/jira/browse/DAFFODIL-2183> Unparse nilled complex element fails. * Fix. This is actually a pretty bad bug, and some solutions to problems that have come up in real schemas need to use nilled complex elements in the solution. DAFFODIL-1598<https://issues.apache.org/jira/browse/DAFFODIL-1598> Unparser: For strings that truncate, the dfdl:valueLength function cannot suspend *Release Note, leave open. I question if this is critical. Change priority? DAFFODIL-2399<https://issues.apache.org/jira/browse/DAFFODIL-2399> Error diagnostics output even though there is an infoset *Fix. This is actually a pretty bad bug. DAFFODIL-2400<https://issues.apache.org/jira/browse/DAFFODIL-2400> New SAX API causes performance degredations. * Release Note. Change priority to Major. I think this should not be critical as SAX is a relatively new feature, and while SAX is a performance-oriented feature, it's not like this is a regression on current functionality. DAFFODIL-1971<https://issues.apache.org/jira/browse/DAFFODIL-1971> Statement order of evaluation not per DFDL Spec * Release Note. Comments? From: Interrante, John A (GE Research, US) Sent: Monday, April 12, 2021 2:52 PM To: dev@daffodil.apache.org Subject: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher I found the list of open and in-progress issues sorted by priority (https://issues.apache.org/jira/issues/?jql=project%20%3D%20DAFFODIL%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC). I looked at all the critical issues (there are 8 in this list) and I don't think we have to hold up 3.1.0 for them although it would be nice if some got fixed first (e.g., DAFFODIL-1422 disallowing doctype decls for security and DAFFODIL-2400 fixing SAX API conformance and performance). However, I would like to merge the new Runtime2 backend in time for the 3.1.0 release of Daffodil in order to 1) accelerate development by avoiding the need to rebase the runtime2 branch on the master branch periodically, and 2) attract new developers to help me build out the runtime2 code further. I need to finish a review of my outstanding pull request, merge it, rebase runtime2 on the master branch, submit a pull request to merge runtime2, and complete that PR’s review, so about 1-2 more weeks. John -Original Message- From: Steve Lawrence Sent: Monday, April 12, 2021 1:33 PM To: dev@daffodil.apache.org Subject: EXT: Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher I would also like to see the updates to SAX merged--there's some important API conformance and performance fixes that are worth getting in. There's an open pull request (PR #478) that I think is close to being ready to be merged. Of the open critical issues, I think DAFFODIL-2399 might be worth blocking if
Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher
Looking at the critical JIRA tickets, they are: DAFFODIL-1422<https://issues.apache.org/jira/browse/DAFFODIL-1422> disallow doctype decls in all XML & XSD that we read in DAFFODIL-2473<https://issues.apache.org/jira/browse/DAFFODIL-2473> Need bit-wise AND, OR, NOT, and shift operations DAFFODIL-2183<https://issues.apache.org/jira/browse/DAFFODIL-2183> Unparse nilled complex element fails. DAFFODIL-1598<https://issues.apache.org/jira/browse/DAFFODIL-1598> Unparser: For strings that truncate, the dfdl:valueLength function cannot suspend DAFFODIL-2399<https://issues.apache.org/jira/browse/DAFFODIL-2399> Error diagnostics output even though there is an infoset DAFFODIL-2400<https://issues.apache.org/jira/browse/DAFFODIL-2400> New SAX API causes performance degredations. DAFFODIL-1971<https://issues.apache.org/jira/browse/DAFFODIL-1971> Statement order of evaluation not per DFDL Spec I'm going to suggest how we handle each of these for the 3.1.0 release. I end up with 2 bugs to fix. Keep in mind this is discussion fodder. DAFFODIL-1422<https://issues.apache.org/jira/browse/DAFFODIL-1422> disallow doctype decls in all XML & XSD that we read in * release note it - try to fix for 3.2.0 DAFFODIL-2473<https://issues.apache.org/jira/browse/DAFFODIL-2473> Need bit-wise AND, OR, NOT, and shift operations * change to priority major - we need better documentation of the use case driving this. DAFFODIL-2183<https://issues.apache.org/jira/browse/DAFFODIL-2183> Unparse nilled complex element fails. * Fix. This is actually a pretty bad bug, and some solutions to problems that have come up in real schemas need to use nilled complex elements in the solution. DAFFODIL-1598<https://issues.apache.org/jira/browse/DAFFODIL-1598> Unparser: For strings that truncate, the dfdl:valueLength function cannot suspend *Release Note, leave open. I question if this is critical. Change priority? DAFFODIL-2399<https://issues.apache.org/jira/browse/DAFFODIL-2399> Error diagnostics output even though there is an infoset *Fix. This is actually a pretty bad bug. DAFFODIL-2400<https://issues.apache.org/jira/browse/DAFFODIL-2400> New SAX API causes performance degredations. * Release Note. Change priority to Major. I think this should not be critical as SAX is a relatively new feature, and while SAX is a performance-oriented feature, it's not like this is a regression on current functionality. DAFFODIL-1971<https://issues.apache.org/jira/browse/DAFFODIL-1971> Statement order of evaluation not per DFDL Spec * Release Note. Comments? From: Interrante, John A (GE Research, US) Sent: Monday, April 12, 2021 2:52 PM To: dev@daffodil.apache.org Subject: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher I found the list of open and in-progress issues sorted by priority (https://issues.apache.org/jira/issues/?jql=project%20%3D%20DAFFODIL%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC). I looked at all the critical issues (there are 8 in this list) and I don't think we have to hold up 3.1.0 for them although it would be nice if some got fixed first (e.g., DAFFODIL-1422 disallowing doctype decls for security and DAFFODIL-2400 fixing SAX API conformance and performance). However, I would like to merge the new Runtime2 backend in time for the 3.1.0 release of Daffodil in order to 1) accelerate development by avoiding the need to rebase the runtime2 branch on the master branch periodically, and 2) attract new developers to help me build out the runtime2 code further. I need to finish a review of my outstanding pull request, merge it, rebase runtime2 on the master branch, submit a pull request to merge runtime2, and complete that PR’s review, so about 1-2 more weeks. John -Original Message- From: Steve Lawrence Sent: Monday, April 12, 2021 1:33 PM To: dev@daffodil.apache.org Subject: EXT: Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher I would also like to see the updates to SAX merged--there's some important API conformance and performance fixes that are worth getting in. There's an open pull request (PR #478) that I think is close to being ready to be merged. Of the open critical issues, I think DAFFODIL-2399 might be worth blocking if we were close to a fix. But I don't think anyone is actively working on it and I don't believe we ever tracked down what the issue was or what the right fix was. Hopefully that can be fixed by 3.2.0--I've seen that issue a few times and it's very confusing. - Steve On 4/12/21 1:01 PM, John Wass wrote: >> I believe we need to do a release very soon regardless of these 96 >> issues > > I woul
[Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher
I found the list of open and in-progress issues sorted by priority (https://issues.apache.org/jira/issues/?jql=project%20%3D%20DAFFODIL%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC). I looked at all the critical issues (there are 8 in this list) and I don't think we have to hold up 3.1.0 for them although it would be nice if some got fixed first (e.g., DAFFODIL-1422 disallowing doctype decls for security and DAFFODIL-2400 fixing SAX API conformance and performance). However, I would like to merge the new Runtime2 backend in time for the 3.1.0 release of Daffodil in order to 1) accelerate development by avoiding the need to rebase the runtime2 branch on the master branch periodically, and 2) attract new developers to help me build out the runtime2 code further. I need to finish a review of my outstanding pull request, merge it, rebase runtime2 on the master branch, submit a pull request to merge runtime2, and complete that PR’s review, so about 1-2 more weeks. John -Original Message- From: Steve Lawrence Sent: Monday, April 12, 2021 1:33 PM To: dev@daffodil.apache.org Subject: EXT: Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher I would also like to see the updates to SAX merged--there's some important API conformance and performance fixes that are worth getting in. There's an open pull request (PR #478) that I think is close to being ready to be merged. Of the open critical issues, I think DAFFODIL-2399 might be worth blocking if we were close to a fix. But I don't think anyone is actively working on it and I don't believe we ever tracked down what the issue was or what the right fix was. Hopefully that can be fixed by 3.2.0--I've seen that issue a few times and it's very confusing. - Steve On 4/12/21 1:01 PM, John Wass wrote: >> I believe we need to do a release very soon regardless of these 96 >> issues > > I would like DAFFODIL- 2482 to get into it; > https://github.com/apache/daffodil/pull/520 > > Will increase priority on wrapping this up. > > > > On Mon, Apr 12, 2021 at 12:43 PM Beckerle, Mike < > mbecke...@owlcyberdefense.com> wrote: > >> I'd like to discuss our need to create a new release of Daffodil, >> which would be 3.1.0. >> >> We have added enough new functionality certainly to justify a release. >> There are important features already complete, there is the new >> Runtime2 backend, etc. >> >> The challenge is that we have 96 JIRA tickets specifically for bugs >> that are marked "major" or above in priority. 6 are marked critical, >> so 90 are "major". (I am excluding all "improvement" and >> "new-feature" tickets in this count. Just bugs.) Obviously we're not >> going to fix 96 issues super quickly. >> >> Some people advocate a set of criteria for releases which stipulate >> there can be no critical/blocker issues, and no major issues, only minor >> issues. >> However, the status of critical/major/minor on our JIRA tickets is >> subjective, most bugs are found and reported by us. >> >> Exactly two bugs have "votes" more than zero, which reflects that >> we've not been using the votes field to prioritize anything, but >> perhaps we should use votes moving forward, rather than bumping >> priorities up and down based on our subjective assessment of importance. >> >> I believe we need to do a release very soon regardless of these 96 issues. >> In scrolling through them, evaluating them as "are they more >> important than doing our first TLP release", none of them rise to >> that level of importance to me. >> >> Most of these issues are part of release 3.0.0 and before that as >> well, so >> 3.1.0 would still be an improvement. >> >> One way to deal with the critical issues is to specifically discuss >> them in a release note. >> >> Please let's discuss openly. What do you believe must be in 3.1.0, >> that we would hold up a release over? >> >> -mike beckerle >> >> >> >
Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher
I would also like to see the updates to SAX merged--there's some important API conformance and performance fixes that are worth getting in. There's an open pull request (PR #478) that I think is close to being ready to be merged. Of the open critical issues, I think DAFFODIL-2399 might be worth blocking if we were close to a fix. But I don't think anyone is actively working on it and I don't believe we ever tracked down what the issue was or what the right fix was. Hopefully that can be fixed by 3.2.0--I've seen that issue a few times and it's very confusing. - Steve On 4/12/21 1:01 PM, John Wass wrote: >> I believe we need to do a release very soon regardless of these 96 issues > > I would like DAFFODIL- 2482 to get into it; > https://github.com/apache/daffodil/pull/520 > > Will increase priority on wrapping this up. > > > > On Mon, Apr 12, 2021 at 12:43 PM Beckerle, Mike < > mbecke...@owlcyberdefense.com> wrote: > >> I'd like to discuss our need to create a new release of Daffodil, which >> would be 3.1.0. >> >> We have added enough new functionality certainly to justify a release. >> There are important features already complete, there is the new Runtime2 >> backend, etc. >> >> The challenge is that we have 96 JIRA tickets specifically for bugs that >> are marked "major" or above in priority. 6 are marked critical, so 90 are >> "major". (I am excluding all "improvement" and "new-feature" tickets in >> this count. Just bugs.) Obviously we're not going to fix 96 issues super >> quickly. >> >> Some people advocate a set of criteria for releases which stipulate there >> can be no critical/blocker issues, and no major issues, only minor issues. >> However, the status of critical/major/minor on our JIRA tickets is >> subjective, most bugs are found and reported by us. >> >> Exactly two bugs have "votes" more than zero, which reflects that we've >> not been using the votes field to prioritize anything, but perhaps we >> should use votes moving forward, rather than bumping priorities up and down >> based on our subjective assessment of importance. >> >> I believe we need to do a release very soon regardless of these 96 issues. >> In scrolling through them, evaluating them as "are they more important than >> doing our first TLP release", none of them rise to that level of importance >> to me. >> >> Most of these issues are part of release 3.0.0 and before that as well, so >> 3.1.0 would still be an improvement. >> >> One way to deal with the critical issues is to specifically discuss them >> in a release note. >> >> Please let's discuss openly. What do you believe must be in 3.1.0, that >> we would hold up a release over? >> >> -mike beckerle >> >> >> >
Re: [Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher
> I believe we need to do a release very soon regardless of these 96 issues I would like DAFFODIL- 2482 to get into it; https://github.com/apache/daffodil/pull/520 Will increase priority on wrapping this up. On Mon, Apr 12, 2021 at 12:43 PM Beckerle, Mike < mbecke...@owlcyberdefense.com> wrote: > I'd like to discuss our need to create a new release of Daffodil, which > would be 3.1.0. > > We have added enough new functionality certainly to justify a release. > There are important features already complete, there is the new Runtime2 > backend, etc. > > The challenge is that we have 96 JIRA tickets specifically for bugs that > are marked "major" or above in priority. 6 are marked critical, so 90 are > "major". (I am excluding all "improvement" and "new-feature" tickets in > this count. Just bugs.) Obviously we're not going to fix 96 issues super > quickly. > > Some people advocate a set of criteria for releases which stipulate there > can be no critical/blocker issues, and no major issues, only minor issues. > However, the status of critical/major/minor on our JIRA tickets is > subjective, most bugs are found and reported by us. > > Exactly two bugs have "votes" more than zero, which reflects that we've > not been using the votes field to prioritize anything, but perhaps we > should use votes moving forward, rather than bumping priorities up and down > based on our subjective assessment of importance. > > I believe we need to do a release very soon regardless of these 96 issues. > In scrolling through them, evaluating them as "are they more important than > doing our first TLP release", none of them rise to that level of importance > to me. > > Most of these issues are part of release 3.0.0 and before that as well, so > 3.1.0 would still be an improvement. > > One way to deal with the critical issues is to specifically discuss them > in a release note. > > Please let's discuss openly. What do you believe must be in 3.1.0, that > we would hold up a release over? > > -mike beckerle > > >
[Discuss] creating Release 3.1.0 and 96 JIRA tickets marked "Major" or higher
I'd like to discuss our need to create a new release of Daffodil, which would be 3.1.0. We have added enough new functionality certainly to justify a release. There are important features already complete, there is the new Runtime2 backend, etc. The challenge is that we have 96 JIRA tickets specifically for bugs that are marked "major" or above in priority. 6 are marked critical, so 90 are "major". (I am excluding all "improvement" and "new-feature" tickets in this count. Just bugs.) Obviously we're not going to fix 96 issues super quickly. Some people advocate a set of criteria for releases which stipulate there can be no critical/blocker issues, and no major issues, only minor issues. However, the status of critical/major/minor on our JIRA tickets is subjective, most bugs are found and reported by us. Exactly two bugs have "votes" more than zero, which reflects that we've not been using the votes field to prioritize anything, but perhaps we should use votes moving forward, rather than bumping priorities up and down based on our subjective assessment of importance. I believe we need to do a release very soon regardless of these 96 issues. In scrolling through them, evaluating them as "are they more important than doing our first TLP release", none of them rise to that level of importance to me. Most of these issues are part of release 3.0.0 and before that as well, so 3.1.0 would still be an improvement. One way to deal with the critical issues is to specifically discuss them in a release note. Please let's discuss openly. What do you believe must be in 3.1.0, that we would hold up a release over? -mike beckerle
