Steve,
Thank you for your quick response. We will have to give some thought
as
to how we
can get the results needed for the XML payload validation. Maybe we
can
pull out the element with
RuntimeProperties = “stringAsXML=true” attribute and validate it where
we parse the overall binary
Input.
A few points:
1. The exception generated is not from our code but from the call to
validator.validate(new DOMSource(document));
2. If, when envisioning the embedded XML feature, the validation of the
payload
would not take place in the InfosetOutputter, why does the example provided
extend InfossetOutputter?
What am I missing?
Thank you for your time,
Mark T
.
-----Original Message-----
From: Steve Lawrence <[email protected]>
Sent: Friday, May 13, 2022 5:00 AM
To: [email protected]
Subject: EXT :Re: Re: Re: Re: Extending JAPI for processing binary input with an
XML Payload
There isn't currently a way for InfosetOutputter's to add to the diagnostics. In
order to do so, an outputter would need access to the PState (where diagnostics
are stored), which would require an API change. I'm not sure we'd want to give
direct access to the PState though--some discussions would be needed to figure
out what that API would look like to get diagnostics out of a outputter and into
the PState.
Also, note that throwing an Exception, whether it be a SAXException or something
else, won't really give the behavior your want. That exception will just bubble
up and lead to an "Abort" error about an unexpected exception being thrown.
The one exception you could throw is a SchemaDefinitionError and that will be
added to diagnostics. But it will immediately stop processing and you will not
get a complete infoset. This is different than normal validation that Daffodil
does which happens at the end of processing and does not prevent the creation of
an infoset.
When we first envisioned this embedded XML feature, we imagined that the
validation of the payload would not take place in the InfosetOutputter, but
instead would happen after the parse completed in some other process that is
purely for validation. This process would use a schema that combines the DFDL
schema with the schema for the embedded XML so that it would validate correctly.
If you really wanted to do it all in Daffodil, maybe another option would be to
create a custom Validator. This Validator would need to find/extract/unescape
the payload XML (the XML that the Validator gets is not the same as what the
InfosetOutputter creates), and then you can do whatever validation is necessary.
The Validator API does support returning errors/warnings from validation and
adding them to the diagnostics.
On 5/12/22 9:55 PM, Thompson, Mark M [US] (DS) wrote:
Hello again from XML Payload world,
- We have succeeded in implementing an extension of
XMLInfosetOutputter that will parse and validation a binary file
with an XML payload. It will validate and report if the
binary fields do not conform to the DFDL schema facet restrictions.
- We have also implemented a routine to validate the XML
payload against a schema which will report if the payload is
malformed. An exception (SAXException) is generated in this case.
What is not clear is how one gets the Diagnostics to be
populated with this info or, if they are populated, how to make
the Diagnostics available to the routine that called
DataProcessor.Parse() in the first place. The Diagnostics are available
here for the validation of the binary inputs against the
corresponding schema but not those reported for the XML payload.
Do I need to extend the ErrorHandler class and use the
warning and/or error methods to populate Diagnostics as part of the
Exception handling for code below?
The following is an excerpt of the code that seems to do
what I want except for making the SAXException info available.
document = db.parse(new InputSource (new StringReader (xmlPayload)));
// load a WXS schema, represented by a Schema instance
StreamSource schemaFile = new StreamSource(new
File(schemaFileName));
Schema schema = s_factory.newSchema(schemaFile);
// create a Validator instance, which can be used
to validate an instance document
javax.xml.validation.Validator validator =
schema.newValidator();
System.out.println("**** validateXMLPayload - Try
to validate XML Payload.");
// validate the DOM tree
validator.validate(new DOMSource(document));
If anyone can provide a clue as to what I’m missing or
invalid assumptions, I would greatly appreciate it.
Thank you for your time,
Mark T
-----Original Message-----
From: Steve Lawrence <[email protected] <mailto:[email protected]>>
Sent: Thursday, April 28, 2022 4:36 PM
To: [email protected] <mailto:[email protected]>
Subject: EXT :Re: Re: Re: Extending JAPI for processing binary input
with an XML Payload It looks like our InfosetOutputter API isn't very
Java friendly, at least once you start needing to get information out
of the ERD. I'd open a ticket, but it looks like JIRA is down at the
moment.
I think the way to fix your issue is to just use straight reference
equality comparison with NodeInfo.String(), e.g.:
if (simple.erd().optPrimType().get() == NodeInfo.String()) { ...
} The instance that NodeInfo.String() returns is essentially a
singleton to represent DFDL String primitives, which is also what is
returned from optPrimtType().get().
I'm not sure why our InfosetOutputter uses isInstanceOf[String.Kind]
instead of just using reference equality with NodeInfo.String. Might
have been old code that never got updated...
On 4/28/22 1:24 PM, Thompson, Mark M [US] (DS) wrote:
Hello all,
We are attempting to generate a Java class for this XML payload by
extending XMLInfosetOutputter using XMLTextInfosetOUtputter
As a guide.
We're having trouble coming up with a Java equivalent for the
StartSimple() method code:
if
(simple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) {
We've tried the following with no luck:
if (simple.erd().optPrimType().get() instanceof
NodeInfo.String().Kind) {
and
if
(simple.erd().optPrimType().get().isInstanceOf(NodeInfo.String().???)
)
{
Any input or suggestions are greatly appreciated.
Thank you for your time,
Mark T
-----Original Message-----
From: Steve Lawrence <[email protected]
<mailto:[email protected] <mailto:[email protected]>>>
Sent: Thursday, April 21, 2022 4:31 AM
To: [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
Subject: EXT :Re: Re: Extending JAPI for processing binary input with
an XML Payload
> Contributing to Daffodil is likely not an option for my company.
That's sad to hear, and I hope you can try to persuade your company to change
this position. Opensource works best with many contributors from many different
organizations. And if your company depends on Daffodil, contributing back to it
can help ensure its success and continued growth.
> Based on my limited understanding, this seems to suggest that I
would have to add another infoset type to make use of the new class
If you're using the CLI, this is correct. Infoset inputters and
outputters must currently be hardcoded in. If you're using the API to
invoke Daffodil this isn't a problem, but if you're using the CLI
then you must modify the CLI to support your new classes. This
hasn't been an issue so far since anytime a new one is needed we just
added it to Daffodil.
Part of the reason for this is that creating InfosetInputters/Outputters and
getting the result is very specific to the infoset representation.
It's probably possible that the API could be made more generic, and then we
could use a ServiceLoader or something to support pluggable infoset
inputters/outputters, but that hasn't be a priority so far.
On 4/20/22 10:08 PM, Thompson, Mark M [US] (DS) wrote:
Hi Steve,
I have been looking into your suggestions and researching the existing
code and here
Is where I'm at:
- Contributing to Daffodil is likely not an option for my company.
- I've been looking into XMLTextInfosetInputter and
ScalaXMLInfosetOutputter for insight.
What I don't understand (Java and Scala newby) is, having extended
InfosetOutputter with a new class,
the Daffodil parse operation magically has access to this new class. I
see, in main.scala, where
XMLTextInfosetOutputter and ScalaXMLInfosetOutputter are instantiated
based on command line
Input for infoset type. Based on my limited understanding, this seems
to suggest that I would have
To add another infoset type to make use of the new class.
Can someone clear up my misunderstanding or point to some info that will
cause the scales to fall
From my eyes?
Thank you for your time,
Mark T
-----Original Message-----
From: Steve Lawrence <[email protected]
<mailto:[email protected] <mailto:[email protected]>>>
Sent: Friday, April 8, 2022 6:31 AM
To: [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
Subject: EXT :Re: Extending JAPI for processing binary input with an
XML Payload
I've put a decent amount of thought into this issue, I'll add some additional
info that might help out...
> for example, the startSimple() method. It would be helpful to have an idea
what a given method does normally when there is no class extension.
The InfosetOutputter class is an abstract class with no default
implementations, so an implementation of all functions, including startSimple,
is required--InfosetInputters don't do anything by default.
The InfosetOutputters that are bundled with Daffodil implement all of these
functions to create the kind of infoset that is output (e.g. XML text, Scala
Nodes, JSON). If you want to take a look at how our bundled implementations
handle their logic, they are all in this folder:
https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/m
<https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/m>
a
<https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/ma
>
i
n/scala/org/apache/daffodil/infoset
If you plan to keep your InfosetOutputter outside of the Daffodil codebase,
you'll need to implement all of the functions/logic yourself.
Our current implementions are mostly private, so extending and overriding them
to do you you want is likely not possible.
A good place to start would be to pick one that that's close to what you want
and modify the various functions as described in the example implementation in
the runtime properties proposal:
https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Run
<https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Run>
t
<https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Runt
>
i
me+Properties
Alternatively, it might be easier to just add the support to our
existing bundled InfosetOutputters to support this capability, and
contribute that work to Daffodil. Not only will this probably be
easier since you can reuse alot of the existing logic, but our
ability to provide support greatly increases when it's related to
things being added to Daffodil. This feature is a request we've seen
multiple times--it would be a very welcome addition and we'd be happy
help get it added to Daffodil.
On 4/8/22 8:42 AM, Mike Beckerle wrote:
Mark,
There definitely are javadoc jars, and the javadoc is also online
here (for release 3.3.0)
https://daffodil.apache.org/docs/latest/javadoc/
<https://daffodil.apache.org/docs/latest/javadoc/>
<https://daffodil.apache.org/docs/latest/javadoc/
<https://daffodil.apache.org/docs/latest/javadoc/>>
<https://daffodil.apache.org/docs/latest/javadoc/
<https://daffodil.apache.org/docs/latest/javadoc/
<https://daffodil.apache.org/docs/latest/javadoc/>>>
The source and javadoc jars are normally pulled down using maven or
sbt. But you can find them directly on maven central also.
For example for the daffodil-runtime1 jar, all the jars, compiled
binary, sources, and the javadoc jar is at
https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime
<https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime>
1
<https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1
<https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1>>
_
2
.12/3.3.0/
<https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtim
e
1
_
2.12/3.3.0/>
On Thu, Apr 7, 2022 at 10:07 PM Thompson, Mark M [US] (DS)
<[email protected] <mailto:[email protected]
<mailto:[email protected] <mailto:[email protected]
<mailto:[email protected] <mailto:[email protected]>>>> wrote:
Greetings to All,____
__ __
Caveat: I am new to Daffodil and DFDL. I’m familiar with
the
basic functionality and have successfully written____
DFDL schemas to handle text and binary
inputs
to the parse/unparse cycle (separate assignments). ____
__ __
I am working on a task that requires the processing of binary files
containing an XML payload. The goal is to be____
able to validate the raw (un-escapified) XML against an
XML
schema during the parsing phase. I have been researching____
the Daffodil JAPI InfosetOutputter family of classes as a
first step. I’ve also read Steve Lawrence’s write-up on the
____
RuntimePropertiies proposal. What I have yet to grasp or
find
is what InfosetOutputter functionality is to be extended
.____
I’ve searched high and low but I have yet to find any
detailed information on what basic functionality is provided
by,____
for example, the startSimple() method. It would be helpful to have an
idea
what a given method does normally when____
there is no class extension.____
__ __
Can anyone recommend a place to start or a source of info on the
InfosetOutputter family of classes?____
__ __
I am working in NetBeans and every class I find seems to
indicate that:____
Javadoc not found. Either Javadoc documentation for this item does not
exist
or ____
you have not added specified Javadoc in the Java
Platform Manager or the Library ____
Manager.____
__ __
Are there Javadoc(s) available corresponding to the
Daffodil
JAPI classes?____
__ __
It feels like I’m missing some basic configuration setup
settings or files. Any guidance or suggestions will____
be greatly appreciated.____
__ __
Thank you for your time,____
Mark M. Thompson____
Northrop Grumman Defense Systems____
Software Engineer____
(818) 712-7439____
__ __
