This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil.git
The following commit(s) were added to refs/heads/main by this push:
new b05b82f48 Fix unidoc configuration to build javadoc
b05b82f48 is described below
commit b05b82f481641c2e50472e8408ec79ca646bc14f
Author: Steve Lawrence <[email protected]>
AuthorDate: Mon Aug 4 14:47:41 2025 -0400
Fix unidoc configuration to build javadoc
On Scala 3 unidoc ignores java files because Scala 3 scaladoc only
supports .tasty files--it does not support including .java files in the
API. Our API is all java, so we modify the unidocAllSources setting to
remove the Scala 3 logic and just find all scoped sources. This includes
both .scala and .java files, so we filter to only include our .java API
files. Also update the comments to make it clear we cannot support
scaladoc, and since that is unlikely to ever be supported, this removes
all files and configurations specific to scaladoc generation.
Also restores the package-info.java for the root
org.apache.daffodil.japi package, renaming and updating examples to use
org.apache.daffodil.api.
DAFFODIL-3012
---
build.sbt | 51 ++-
daffodil-core/root-doc.txt | 9 -
.../java/org/apache/daffodil/api/package-info.java | 355 +++++++++++++++++++++
project/Rat.scala | 3 -
4 files changed, 379 insertions(+), 39 deletions(-)
diff --git a/build.sbt b/build.sbt
index ce6e1a5fc..0a85e02ae 100644
--- a/build.sbt
+++ b/build.sbt
@@ -410,24 +410,21 @@ lazy val ratSettings = Seq(
)
/**
- * Filter to include only the doc files in our supported API classes
+ * For maximum compatability with Java users, our API is written entirely in
Java. The
+ * following configures unidoc to convert all our .java files to javadoc
*
- * @param sources - the sequence of files to filter
- * @return - the filtered sequence of files
- */
-def apiDocSourceFilter(sources: Seq[File]): Seq[File] = sources.filter {
source =>
- source.toString.contains("src/main/java")
-}
-
-/**
- * Previously we used JavaUnidoc to generate javadoc and ScalaUnidoc to
generate scaladoc.
- * But JavaUnidoc requires the genjavadoc plugin, which does not work on Scala
3.
- * So we can only use ScalaUnidoc. But this doesn't necessarily mean we must
generate
- * scaladoc. ScalaUnidoc will generate scaladoc if there is at least one
.scala file,
- * but uses javadoc if all source files are .java. Our public API is 100%
java, so we
- * add a filter to only include files in src/main/java so we generate javadoc.
- * This does mean scaladoc is no longer available, but javadoc can be
understood by
- * both Scala and Java devs.
+ * Note that that we do not use JavaUnidoc because that uses the genjavadoc
plugin to
+ * convert .scala files to .java and runs javadoc on the result. Our API is
already all
+ * .java so we do not need this converstion.
+ *
+ * Instead, we use ScalaUnidoc to generate javadoc. This works because all of
our API sources
+ * are .java files, which causes unidoc to generate documentation using
javadoc instead of
+ * scaladoc. Note that we need to change the logic of unidocAllSources because
by default it
+ * only includes .tasty files when run with Scala 3--we change it to instead
find scoped source
+ * files (i.e. .scala and .java files) and filter to include only .java files.
+ *
+ * Note that Scala 3 does not support .java files, so we are unlikely to ever
be able to
+ * generate scaladoc-style output in addition to the javadoc-style currently
supported.
*/
lazy val unidocSettings =
Seq(
@@ -438,22 +435,22 @@ lazy val unidocSettings =
),
ScalaUnidoc / unidoc / unidocProjectFilter :=
inProjects(udf, core),
- ScalaUnidoc / unidoc / scalacOptions := Seq(
- "-doc-title",
- "Apache Daffodil " + version.value + " Scala API",
- "-doc-root-content",
- (core / baseDirectory).value + "/root-doc.txt"
- ),
ScalaUnidoc / unidoc / javacOptions := Seq(
"-windowtitle",
- "Apache Daffodil " + version.value + " Java API",
+ "Apache Daffodil " + version.value + " API",
"-doctitle",
- "<h1>Apache Daffodil " + version.value + " Java API</h1>",
+ "<h1>Apache Daffodil " + version.value + " API</h1>",
"-notimestamp",
"-quiet"
),
- ScalaUnidoc / unidoc / unidocAllSources :=
- (ScalaUnidoc / unidoc / unidocAllSources).value.map(apiDocSourceFilter)
+ ScalaUnidoc / unidoc / unidocAllSources := Def
+ .taskDyn {
+ val scopeFilter = (ScalaUnidoc / unidoc / unidocScopeFilter).value
+ val scopedSources = sources.all(scopeFilter)
+ scopedSources
+ }
+ .value
+ .map(_.filter(_.getName.endsWith(".java")))
)
lazy val genTunablesDocSettings = Seq(
diff --git a/daffodil-core/root-doc.txt b/daffodil-core/root-doc.txt
deleted file mode 100644
index 3e9bb41d1..000000000
--- a/daffodil-core/root-doc.txt
+++ /dev/null
@@ -1,9 +0,0 @@
-This is the documentation for the Apache Daffodil Scala API.
-
-=== Package structure ===
-
-[[org.apache.daffodil.api]] - Provides the classes necessary to compile DFDL
schemas, parse and unparse files using the compiled objects, and retrieve
results and parsing diagnostics
-
-[[org.apache.daffodil.udf]] - Provides the classes necessary to create User
Defined Functions to extend the DFDL expression language
-
-[[org.apache.daffodil.runtime1.layers.api]] - Provides the classes necessary
to create custom Layer extensions to DFDL.
\ No newline at end of file
diff --git
a/daffodil-core/src/main/java/org/apache/daffodil/api/package-info.java
b/daffodil-core/src/main/java/org/apache/daffodil/api/package-info.java
new file mode 100644
index 000000000..0d7527535
--- /dev/null
+++ b/daffodil-core/src/main/java/org/apache/daffodil/api/package-info.java
@@ -0,0 +1,355 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Provides the classes necessary to compile DFDL schemas, parse and
+ * unparse files using the compiled objects, and retrieve results and
+ * parsing diagnostics
+ *
+ * <h2>Overview</h2>
+ *
+ * The {@link org.apache.daffodil.api.Daffodil} object is a factory object to
create a
+ * {@link org.apache.daffodil.api.Compiler}. The {@link
org.apache.daffodil.api.Compiler} provides
+ * a method to compile a provided DFDL schema into a {@link
org.apache.daffodil.api.ProcessorFactory},
+ * which creates a {@link org.apache.daffodil.api.DataProcessor}:
+ *
+ * <pre>
+ * {@code
+ * Compiler c = Daffodil.compiler();
+ * ProcessorFactory pf = c.compileFile(file);
+ * DataProcessor dp = pf.onPath("/");
+ * }</pre>
+ *
+ * The {@link org.apache.daffodil.api.DataProcessor} provides the necessary
functions to parse and
+ * unparse data, returning a {@link org.apache.daffodil.api.ParseResult} or
+ * {@link org.apache.daffodil.api.UnparseResult}, respectively. These contain
information about the
+ * parse/unparse, such as whether or not the processing succeeded with any
diagnostic information.
+ *
+ * The {@link org.apache.daffodil.api.DataProcessor} also provides two
functions that can be used to
+ * perform parsing/unparsing via the SAX API. The first creates a
+ * {@link org.apache.daffodil.api.DaffodilParseXMLReader} which is used for
parsing, and the
+ * second creates a {@link
org.apache.daffodil.api.DaffodilUnparseContentHandler} which is used for
+ * unparsing.
+ *
+ * <pre>
+ * {@code
+ * DaffodilParseXMLReader xmlReader = dp.newXMLReaderInstance();
+ * DaffodilUnparseContentHandler unparseContentHandler =
dp.newContentHandlerInstance(output);
+ * }</pre>
+ *
+ * The {@link org.apache.daffodil.api.DaffodilParseXMLReader} has several
methods that allow one to
+ * set properties and handlers (such as ContentHandlers or ErrorHandlers) for
the reader. One can
+ * use any contentHandler/errorHandler as long as they extend the
+ * {@link org.xml.sax.ContentHandler} and {@link org.xml.sax.ErrorHandler}
interfaces
+ * respectively. One can also set properties for the {@link
org.apache.daffodil.api.DaffodilParseXMLReader}
+ * using {@link
org.apache.daffodil.api.DaffodilParseXMLReader#setProperty(java.lang.String,
+ * java.lang.Object)}.
+ *
+ * The following properties can be set as follows:
+ *
+ * <p><i>The constants below have literal values starting with
+ * "urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:sax:" and ending with
"BlobDirectory",
+ * "BlobPrefix" and "BlobSuffix" respectively.</i></p>
+ *
+ * <pre>
+ * {@code
+ *
xmlReader.setProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_BLOBDIRECTORY(),
+ * Paths.get(System.getProperty("java.io.tmpdir"))); // value type:
java.nio.file.Paths
+ * xmlReader.setProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_BLOBPREFIX(),
"daffodil-sax-"); // value type String
+ * xmlReader.setProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_BLOBSUFFIX(),
".bin"); // value type String
+ * }
+ * </pre>
+ *
+ * The properties can be retrieved using the same variables with
+ * {@link
org.apache.daffodil.api.DaffodilParseXMLReader#getProperty(java.lang.String)}
and casting
+ * to the appropriate type as listed above.
+ *
+ * The following handlers can be set as follows:
+ * <pre>
+ * {@code
+ * xmlReader.setContentHandler(contentHandler);
+ * xmlReader.setErrorHandler(errorHandler);
+ * }
+ * </pre>
+ *
+ * The handlers above must implement the following interfaces respectively:
+ * <pre>
+ * {@code
+ * org.xml.sax.ContentHandler
+ * org.xml.sax.ErrorHandler
+ * }
+ * </pre>
+ *
+ * The {@link org.apache.daffodil.api.ParseResult} can be found as a property
within the
+ * {@link org.apache.daffodil.api.DaffodilParseXMLReader} using this uri:
+ * "urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:sax:ParseResult" or
+ * {@link
org.apache.daffodil.api.DaffodilParseXMLReader#DAFFODIL_SAX_URN_PARSERESULT}.
+ *
+ * In order for a successful unparse to happen, the SAX API requires the
+ * unparse to be kicked off by a parse call to any {@link
org.xml.sax.XMLReader} implementation that
+ * has the {@link org.apache.daffodil.api.DaffodilUnparseContentHandler}
registered as its content
+ * handler. To retrieve the {@link org.apache.daffodil.api.UnparseResult}, one
can use
+ * {@link
org.apache.daffodil.api.DaffodilUnparseContentHandler#getUnparseResult()} once
the
+ * XMLReader.parse run is complete.
+ *
+ * <h3>Parse</h3>
+ *
+ * <h4>DataProcessor Parse</h4>
+ *
+ * The {@link
org.apache.daffodil.api.DataProcessor#parse(org.apache.daffodil.api.InputSourceDataInputStream,
+ * org.apache.daffodil.api.infoset.InfosetOutputter)} method accepts input
data to parse in the form
+ * of a {@link org.apache.daffodil.api.InputSourceDataInputStream} and an
+ * {@link org.apache.daffodil.api.infoset.InfosetOutputter} to determine the
output representation
+ * of the infoset (e.g. Scala XML Nodes, JDOM2 Documents, etc.):
+ *
+ * <pre>
+ * {@code
+ * JDOMInfosetOutputter jdomOutputter= new JDOMInfosetOutputter();
+ * InputSourceDataInputStream is = new InputSourceDataInputStream(data);
+ * ParseResult pr = dp.parse(is, jdomOutputter);
+ * Document doc = jdomOutputter.getResult();
+ * }</pre>
+ *
+ * The {@link
org.apache.daffodil.api.DataProcessor#parse(org.apache.daffodil.api.InputSourceDataInputStream,
+ * org.apache.daffodil.api.infoset.InfosetOutputter)} method is thread-safe
and may be called multiple
+ * times without the need to create other data processors. However,
+ * {@link org.apache.daffodil.api.infoset.InfosetOutputter}'s are not thread
safe, requiring a
+ * unique instance per thread. An {@link
org.apache.daffodil.api.infoset.InfosetOutputter} should
+ * call {@link org.apache.daffodil.api.infoset.InfosetOutputter#reset()}
before reuse (or a new one
+ * should be allocated). For example:
+ *
+ * <pre>
+ * {@code
+ * JDOMInfosetOutputter jdomOutputter = new JDOMInfosetOutputter();
+ * for (File f : inputFiles) {
+ * jdomOutputter.reset();
+ * InputSourceDataInputStream is = new InputSourceDataInputStream(new
FileInputStream(f)));
+ * ParseResult pr = dp.parse(is, jdomOutputter);
+ * Document doc = jdomOutputter.getResult();
+ * }
+ * }</pre>
+ *
+ * One can repeat calls to parse() using the same InputSourceDataInputStream
to continue parsing
+ * where the previous parse ended. For example:
+ *
+ * <pre>
+ * {@code
+ * InputSourceDataInputStream is = new InputSourceDataInputStream(dataStream);
+ * JDOMInfosetOutputter jdomOutputter = new JDOMInfosetOutputter();
+ * boolean keepParsing = true;
+ * while (keepParsing && is.hasData()) {
+ * jdomOutputter.reset();
+ * ParseResult pr = dp.parse(is, jdomOutputter);
+ * ...
+ * keepParsing = !pr.isError();
+ * }
+ * }</pre>
+ *
+ * <h4>SAX Parse</h4>
+ * The {@link org.apache.daffodil.api.DaffodilParseXMLReader#parse(
+ * org.apache.daffodil.api.InputSourceDataInputStream)} method accepts input
data to parse in
+ * the form of a {@link org.apache.daffodil.api.InputSourceDataInputStream}.
The output
+ * representation of the infoset, as well as how parse errors are handled, are
dependent on the
+ * content handler and the error handler provided to the {@link
org.apache.daffodil.api.DaffodilParseXMLReader}.
+ * For example the {@link org.jdom2.input.sax.SAXHandler} provides a JDOM
representation, whereas
+ * other ContentHandlers may output directly to a {@link java.io.OutputStream}
or {@link java.io.Writer}.
+ *
+ * <pre>
+ * {@code
+ * SAXHandler contentHandler = new SAXHandler();
+ * xmlReader.setContentHandler(contentHandler);
+ * InputSourceDataInputStream is = new InputSourceDataInputStream(data);
+ * xmlReader.parse(is);
+ * ParseResult pr = (ParseResult)
xmlReader.getProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_PARSERESULT());
+ * Document doc = saxHandler.getDocument();
+ * }</pre>
+ *
+ * The The {@link org.apache.daffodil.api.DaffodilParseXMLReader#parse(
+ * org.apache.daffodil.api.InputSourceDataInputStream)} method is not
thread-safe and may
+ * only be called again/reused once a parse operation is completed. This can
be done multiple
+ * times without the need to create new DaffodilParseXMLReaders,
ContentHandlers or ErrorHandlers.
+ * It might be necessary to reset whatever ContentHandler is used (or allocate
a new one). A
+ * thread-safe implementation would require unique instances of the
DaffodilParseXMLReader and its
+ * components. For example:
+ *
+ * <pre>
+ * {@code
+ * SAXHandler contentHandler = new SAXHandler();
+ * xmlReader.setContentHandler(contentHandler);
+ * for (File f : inputFiles) {
+ * contentHandler.reset();
+ * InputSourceDataInputStream is = new InputSourceDataInputStream(new
FileInputStream(f));
+ * xmlReader.parse(is);
+ * ParseResult pr = (ParseResult)
xmlReader.getProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_PARSERESULT());
+ * Document doc = saxHandler.getDocument();
+ * }
+ * }
+ * </pre>
+ *
+ * The value of the supported features cannot be changed during a parse, and
the parse will run
+ * with the value of the features as they were when the parse was kicked off.
To run a parse with
+ * different feature values, one must wait until the running parse finishes,
set the feature values
+ * using the XMLReader's setFeature and run the parse again.
+ *
+ * One can repeat calls to parse() using the same InputSourceDataInputStream
to continue parsing
+ * where the previous parse ended. For example:
+ *
+ * <pre>
+ * {@code
+ * InputSourceDataInputStream is = new InputSourceDataInputStream(dataStream);
+ * SAXHandler contentHandler = new SAXHandler();
+ * xmlReader.setContentHandler(contentHandler);
+ * Boolean keepParsing = true;
+ * while (keepParsing && is.hasData()) {
+ * contentHandler.reset();
+ * xmlReader.parse(is);
+ * ParseResult pr =
xmlReader.getProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_PARSERESULT());
+ * ...
+ * keepParsing = !pr.isError();
+ * }
+ * }
+ * </pre>
+ *
+ * <h3>Unparse</h3>
+ *
+ * <h4>DataProcessor Unparse</h4>
+ *
+ * The same {@link org.apache.daffodil.api.DataProcessor} used for parse can
be used to unparse an
+ * infoset via the {@link
org.apache.daffodil.api.DataProcessor#unparse(org.apache.daffodil.api.infoset.InfosetInputter,
+ * java.nio.channels.WritableByteChannel)} method. An {@link
org.apache.daffodil.api.infoset.InfosetInputter}
+ * provides the infoset to unparse, with the unparsed data written to the
+ * provided {@link java.nio.channels.WritableByteChannel}. For example:
+ *
+ * <pre>
+ * {@code
+ * JDOMInfosetInputter jdomInputter = new JDOMInfosetInputter(doc);
+ * UnparseResult ur = dp.unparse(jdomInputter, wbc)
+ * }</pre>
+ *
+ * <h4>SAX Unparse</h4>
+ *
+ * In order to kick off an unparse via the SAX API, one must register the
+ * {@link org.apache.daffodil.api.DaffodilUnparseContentHandler} as the
ContentHandler for an
+ * XMLReader implementation. The call to the
+ * {@link
org.apache.daffodil.api.DataProcessor#newContentHandlerInstance(java.nio.channels.WritableByteChannel)}
+ * method must be provided with the {@link
java.nio.channels.WritableByteChannel}, where the unparsed
+ * data ought to be written to. Any XMLReader implementation is permissible,
as long as they have
+ * XML Namespace support.
+ *
+ * <pre>
+ * {@code
+ * ByteArrayInputStream is = new ByteArrayInputStream(data);
+ * ByteArrayOutputStream os = new ByteArrayOutputStream();
+ * WritableByteChannel wbc = java.nio.channels.Channels.newChannel(os);
+ * DaffodilUnparseContentHandler unparseContentHandler =
dp.newContentHandlerInstance(wbc);
+ * try {
+ * XMLReader xmlReader =
SAXParserFactory.newInstance().newSAXParser().getXMLReader();
+ * xmlReader.setContentHandler(unparseContentHandler)
+ * xmlReader.parse(is)
+ * } catch (ParserConfigurationException | SAXException e) {
+ * ...
+ * } catch (DaffodilUnparseErrorSAXException | DaffodilUnhandledSAXException
e) {
+ * ...
+ * }
+ * }
+ * </pre>
+ *
+ * The call to the XMLReader.parse method must be wrapped in a try/catch, as
+ * {@link org.apache.daffodil.api.DaffodilUnparseContentHandler} relies on
throwing an exception to
+ * end processing in the case of any errors/failures.
+ * There are two kinds of errors to expect
+ * {@link
org.apache.daffodil.api.exceptions.DaffodilUnparseErrorSAXException}, for the
case when the
+ * {@link org.apache.daffodil.api.UnparseResult#isError()} is true, and
+ * {@link org.apache.daffodil.api.exceptions.DaffodilUnhandledSAXException},
for any other errors.
+ *
+ * In the case of an {@link
org.apache.daffodil.api.exceptions.DaffodilUnhandledSAXException},
+ * {@link
org.apache.daffodil.api.DaffodilUnparseContentHandler#getUnparseResult()} will
return null.
+ *
+ * <pre>
+ * {@code
+ * try {
+ * xmlReader.parse(new InputSource(is));
+ * } catch (DaffodilUnparseErrorSAXException | DaffodilUnhandledSAXException
e) {
+ * ...
+ * }
+ * UnparseResult ur = unparseContentHandler.getUnparseResult();
+ * }
+ * </pre>
+ *
+ *
+ * <h3>Failures and Diagnostics</h3>
+ *
+ * It is possible that failures could occur during the creation of the
+ * {@link org.apache.daffodil.api.ProcessorFactory}, {@link
org.apache.daffodil.api.DataProcessor},
+ * {@link org.apache.daffodil.api.ParseResult}, or {@link
org.apache.daffodil.api.UnparseResult}. However,
+ * rather than throwing an exception on error (e.g. invalid DFDL schema, parse
error, etc), these classes extend
+ * {@link org.apache.daffodil.api.WithDiagnostics}, which is used to determine
if an error occurred,
+ * and any diagnostic information (see {@link
org.apache.daffodil.api.Diagnostic}) related to the step.
+ * Thus, before continuing, one must check {@link
org.apache.daffodil.api.WithDiagnostics#isError}.
+ * For example:
+ *
+ * <pre>
+ * {@code
+ * ProcessorFactor pf = c.compile(files);
+ * if (pf.isError()) {
+ * java.util.List<Diagnostic> diags = pf.getDiagnostics();
+ * for (Diagnostic d : diags) {
+ * System.out.println(d.toString());
+ * }
+ * return -1;
+ * }
+ * }</pre>
+ *
+ * <h3>Saving and Reloading Parsers</h3>
+ *
+ * In some cases, it may be beneficial to save a parser and reload it.
+ * For example, when starting up, it may be quicker to reload an
+ * already compiled parser than to compile it from scratch. To save a
+ * {@link org.apache.daffodil.api.DataProcessor}:
+ *
+ * <pre>
+ * {@code
+ * DataProcessor dp = pf.onPath("/");
+ * dp.save(saveFile);
+ * }</pre>
+ *
+ * And to restore a saved {@link org.apache.daffodil.api.DataProcessor}:
+ *
+ * <pre>
+ * {@code
+ * DataProcessor dp = Daffodil.reload(saveFile);
+ * }</pre>
+ *
+ * And use like below:
+ * <pre>
+ * {@code
+ * ParseResult pr = dp.parse(data);
+ * }</pre>
+ *
+ * or
+ *
+ * <pre>
+ * {@code
+ * DaffodilParseXMLReader xmlReader = dp.newXMLReaderInstance();
+ * ... // setting appropriate handlers
+ * xmlReader.parse(data);
+ * ParseResult pr =
xmlReader.getProperty(DaffodilParseXMLReader.DAFFODIL_SAX_URN_PARSERESULT());
+ * }</pre>
+ *
+ */
+
+package org.apache.daffodil.api;
diff --git a/project/Rat.scala b/project/Rat.scala
index 5e52eb794..64fd406bf 100644
--- a/project/Rat.scala
+++ b/project/Rat.scala
@@ -26,9 +26,6 @@ object Rat {
// IntelliJ files
file(".idea"),
- // scaladoc related, has no way to include a license
- file("daffodil-core/root-doc.txt"),
-
// version file does not contain a license header
file("VERSION"),
