Re: Output SVRL from Schematron Validator
Thanks. > Do we need API-level access to this? E.g. in SAPI/JAPI? I would imagine so. Yeah good call, Ill add it. On Mon, Apr 5, 2021 at 1:31 PM Beckerle, Mike wrote: > I looked at the PR for this feature. I think it's fine to have the CLI > provide an option with a file to write it to, and API-wise, if we decide we > have to expose this, then a parseResult.validationResult.raw member, or > like that, is fine with me. > > Do we need API-level access to this? E.g. in SAPI/JAPI? I would imagine so. > > From: John Wass > Sent: Monday, March 29, 2021 1:55 PM > To: dev@daffodil.apache.org > Subject: Re: Output SVRL from Schematron Validator > > The thought with the OutputStream was it would be dumped directly to a file > or log or stdout, definitely more of a logging effect than for more > processing, since the structured results from a validator are already > returned as ValidationResult. That idea looks and sounds worse today that > it did initially. > > > What about if each ParseResult has a member > > Ah, what if the ParseResult hangs on to the ValidationResult and makes it > accessible that way? > > def validationResult(): Option[ValidationResult] > > To support this ValidationResult would become a trait which lets validator > implementations attach custom data and interfaces to the result, which > clients can get to through the ParseResult accessor. > > Something like this; > https://github.com/jw3/daffodil/tree/validator_result_refactor > > Thoughts? > > > On Fri, Mar 26, 2021 at 10:30 AM Steve Lawrence > wrote: > > > What about if each ParseResult has a member that's something like > > > > val validationData: Option[AnyRef] > > > > Each validator can optionally return some validation data which is then > > store in this member. The user could then access this validation data > > through the ParseResult and cast it to what it should be, as documented > > by the validator. > > > > This allows each validator a way provide whatever additional data they > > want in whatever format makes the most sense for them. > > > > There's the downside that a user needs to know how to cast this AnyRef > > based on which validator was used. But a similar issue exists if this is > > just an InputStream--you still need to know how to interpret that > > InputStream data. But with this approach, it lets a Validator return > > complex structures that can provide richer information than an > > InputStream could. > > > > On 3/26/21 10:16 AM, John Wass wrote: > > > Reference implementation here > > > https://github.com/jw3/daffodil/tree/validator_outputstream > > > > > > Currently has changes sketched in from the parse result on down. Need > to > > > wire things in through DP and CLI yet. > > > > > > Haven't thought of an alternative that works yet. > > > > > > > > > On Tue, Mar 23, 2021 at 12:59 PM John Wass wrote: > > > > > >> Looking at DAFFODIL-2482 that came up due to a gap that's blocking > > >> integration of the schematron validation functionality into some > > workflows > > >> that require the full SVRL output, not just the pass/fail status. > > >> > > >> So what needs to happen here is the SVRL that we currently just parse > > for > > >> errors and discard needs to be output in a predictable way. I've > tried a > > >> couple things intent on minimizing the footprint of the impl but > coming > > up > > >> empty mainly due to violating the reusable validator principle. > > >> > > >> So another unminimized approach would be to provide an additional > stream > > >> to all validators for raw output to be written, the implementation of > > that > > >> stream is determined by configuration from the DataProcessor. The new > > >> output stream is passed at validation-time, which requires changing > the > > >> signature of the validate call to accept this output stream in > addition > > to > > >> the existing input stream (or we could add another interface, but I'm > > not > > >> convinced of the usefulness of that currently). > > >> > > >> Looking for some thoughts on this approach. > > >> > > >> > > >> [1] https://issues.apache.org/jira/browse/DAFFODIL-2482 > > >> > > >> > > > > > > > >
Re: Output SVRL from Schematron Validator
I looked at the PR for this feature. I think it's fine to have the CLI provide an option with a file to write it to, and API-wise, if we decide we have to expose this, then a parseResult.validationResult.raw member, or like that, is fine with me. Do we need API-level access to this? E.g. in SAPI/JAPI? I would imagine so. From: John Wass Sent: Monday, March 29, 2021 1:55 PM To: dev@daffodil.apache.org Subject: Re: Output SVRL from Schematron Validator The thought with the OutputStream was it would be dumped directly to a file or log or stdout, definitely more of a logging effect than for more processing, since the structured results from a validator are already returned as ValidationResult. That idea looks and sounds worse today that it did initially. > What about if each ParseResult has a member Ah, what if the ParseResult hangs on to the ValidationResult and makes it accessible that way? def validationResult(): Option[ValidationResult] To support this ValidationResult would become a trait which lets validator implementations attach custom data and interfaces to the result, which clients can get to through the ParseResult accessor. Something like this; https://github.com/jw3/daffodil/tree/validator_result_refactor Thoughts? On Fri, Mar 26, 2021 at 10:30 AM Steve Lawrence wrote: > What about if each ParseResult has a member that's something like > > val validationData: Option[AnyRef] > > Each validator can optionally return some validation data which is then > store in this member. The user could then access this validation data > through the ParseResult and cast it to what it should be, as documented > by the validator. > > This allows each validator a way provide whatever additional data they > want in whatever format makes the most sense for them. > > There's the downside that a user needs to know how to cast this AnyRef > based on which validator was used. But a similar issue exists if this is > just an InputStream--you still need to know how to interpret that > InputStream data. But with this approach, it lets a Validator return > complex structures that can provide richer information than an > InputStream could. > > On 3/26/21 10:16 AM, John Wass wrote: > > Reference implementation here > > https://github.com/jw3/daffodil/tree/validator_outputstream > > > > Currently has changes sketched in from the parse result on down. Need to > > wire things in through DP and CLI yet. > > > > Haven't thought of an alternative that works yet. > > > > > > On Tue, Mar 23, 2021 at 12:59 PM John Wass wrote: > > > >> Looking at DAFFODIL-2482 that came up due to a gap that's blocking > >> integration of the schematron validation functionality into some > workflows > >> that require the full SVRL output, not just the pass/fail status. > >> > >> So what needs to happen here is the SVRL that we currently just parse > for > >> errors and discard needs to be output in a predictable way. I've tried a > >> couple things intent on minimizing the footprint of the impl but coming > up > >> empty mainly due to violating the reusable validator principle. > >> > >> So another unminimized approach would be to provide an additional stream > >> to all validators for raw output to be written, the implementation of > that > >> stream is determined by configuration from the DataProcessor. The new > >> output stream is passed at validation-time, which requires changing the > >> signature of the validate call to accept this output stream in addition > to > >> the existing input stream (or we could add another interface, but I'm > not > >> convinced of the usefulness of that currently). > >> > >> Looking for some thoughts on this approach. > >> > >> > >> [1] https://issues.apache.org/jira/browse/DAFFODIL-2482 > >> > >> > > > >
Re: Output SVRL from Schematron Validator
The thought with the OutputStream was it would be dumped directly to a file or log or stdout, definitely more of a logging effect than for more processing, since the structured results from a validator are already returned as ValidationResult. That idea looks and sounds worse today that it did initially. > What about if each ParseResult has a member Ah, what if the ParseResult hangs on to the ValidationResult and makes it accessible that way? def validationResult(): Option[ValidationResult] To support this ValidationResult would become a trait which lets validator implementations attach custom data and interfaces to the result, which clients can get to through the ParseResult accessor. Something like this; https://github.com/jw3/daffodil/tree/validator_result_refactor Thoughts? On Fri, Mar 26, 2021 at 10:30 AM Steve Lawrence wrote: > What about if each ParseResult has a member that's something like > > val validationData: Option[AnyRef] > > Each validator can optionally return some validation data which is then > store in this member. The user could then access this validation data > through the ParseResult and cast it to what it should be, as documented > by the validator. > > This allows each validator a way provide whatever additional data they > want in whatever format makes the most sense for them. > > There's the downside that a user needs to know how to cast this AnyRef > based on which validator was used. But a similar issue exists if this is > just an InputStream--you still need to know how to interpret that > InputStream data. But with this approach, it lets a Validator return > complex structures that can provide richer information than an > InputStream could. > > On 3/26/21 10:16 AM, John Wass wrote: > > Reference implementation here > > https://github.com/jw3/daffodil/tree/validator_outputstream > > > > Currently has changes sketched in from the parse result on down. Need to > > wire things in through DP and CLI yet. > > > > Haven't thought of an alternative that works yet. > > > > > > On Tue, Mar 23, 2021 at 12:59 PM John Wass wrote: > > > >> Looking at DAFFODIL-2482 that came up due to a gap that's blocking > >> integration of the schematron validation functionality into some > workflows > >> that require the full SVRL output, not just the pass/fail status. > >> > >> So what needs to happen here is the SVRL that we currently just parse > for > >> errors and discard needs to be output in a predictable way. I've tried a > >> couple things intent on minimizing the footprint of the impl but coming > up > >> empty mainly due to violating the reusable validator principle. > >> > >> So another unminimized approach would be to provide an additional stream > >> to all validators for raw output to be written, the implementation of > that > >> stream is determined by configuration from the DataProcessor. The new > >> output stream is passed at validation-time, which requires changing the > >> signature of the validate call to accept this output stream in addition > to > >> the existing input stream (or we could add another interface, but I'm > not > >> convinced of the usefulness of that currently). > >> > >> Looking for some thoughts on this approach. > >> > >> > >> [1] https://issues.apache.org/jira/browse/DAFFODIL-2482 > >> > >> > > > >
Re: Output SVRL from Schematron Validator
What about if each ParseResult has a member that's something like val validationData: Option[AnyRef] Each validator can optionally return some validation data which is then store in this member. The user could then access this validation data through the ParseResult and cast it to what it should be, as documented by the validator. This allows each validator a way provide whatever additional data they want in whatever format makes the most sense for them. There's the downside that a user needs to know how to cast this AnyRef based on which validator was used. But a similar issue exists if this is just an InputStream--you still need to know how to interpret that InputStream data. But with this approach, it lets a Validator return complex structures that can provide richer information than an InputStream could. On 3/26/21 10:16 AM, John Wass wrote: > Reference implementation here > https://github.com/jw3/daffodil/tree/validator_outputstream > > Currently has changes sketched in from the parse result on down. Need to > wire things in through DP and CLI yet. > > Haven't thought of an alternative that works yet. > > > On Tue, Mar 23, 2021 at 12:59 PM John Wass wrote: > >> Looking at DAFFODIL-2482 that came up due to a gap that's blocking >> integration of the schematron validation functionality into some workflows >> that require the full SVRL output, not just the pass/fail status. >> >> So what needs to happen here is the SVRL that we currently just parse for >> errors and discard needs to be output in a predictable way. I've tried a >> couple things intent on minimizing the footprint of the impl but coming up >> empty mainly due to violating the reusable validator principle. >> >> So another unminimized approach would be to provide an additional stream >> to all validators for raw output to be written, the implementation of that >> stream is determined by configuration from the DataProcessor. The new >> output stream is passed at validation-time, which requires changing the >> signature of the validate call to accept this output stream in addition to >> the existing input stream (or we could add another interface, but I'm not >> convinced of the usefulness of that currently). >> >> Looking for some thoughts on this approach. >> >> >> [1] https://issues.apache.org/jira/browse/DAFFODIL-2482 >> >> >
Re: Output SVRL from Schematron Validator
Reference implementation here https://github.com/jw3/daffodil/tree/validator_outputstream Currently has changes sketched in from the parse result on down. Need to wire things in through DP and CLI yet. Haven't thought of an alternative that works yet. On Tue, Mar 23, 2021 at 12:59 PM John Wass wrote: > Looking at DAFFODIL-2482 that came up due to a gap that's blocking > integration of the schematron validation functionality into some workflows > that require the full SVRL output, not just the pass/fail status. > > So what needs to happen here is the SVRL that we currently just parse for > errors and discard needs to be output in a predictable way. I've tried a > couple things intent on minimizing the footprint of the impl but coming up > empty mainly due to violating the reusable validator principle. > > So another unminimized approach would be to provide an additional stream > to all validators for raw output to be written, the implementation of that > stream is determined by configuration from the DataProcessor. The new > output stream is passed at validation-time, which requires changing the > signature of the validate call to accept this output stream in addition to > the existing input stream (or we could add another interface, but I'm not > convinced of the usefulness of that currently). > > Looking for some thoughts on this approach. > > > [1] https://issues.apache.org/jira/browse/DAFFODIL-2482 > >
Output SVRL from Schematron Validator
Looking at DAFFODIL-2482 that came up due to a gap that's blocking integration of the schematron validation functionality into some workflows that require the full SVRL output, not just the pass/fail status. So what needs to happen here is the SVRL that we currently just parse for errors and discard needs to be output in a predictable way. I've tried a couple things intent on minimizing the footprint of the impl but coming up empty mainly due to violating the reusable validator principle. So another unminimized approach would be to provide an additional stream to all validators for raw output to be written, the implementation of that stream is determined by configuration from the DataProcessor. The new output stream is passed at validation-time, which requires changing the signature of the validate call to accept this output stream in addition to the existing input stream (or we could add another interface, but I'm not convinced of the usefulness of that currently). Looking for some thoughts on this approach. [1] https://issues.apache.org/jira/browse/DAFFODIL-2482
