[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[ https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635005#comment-17635005 ] Daniel Stieglitz edited comment on NIFI-6498 at 11/16/22 9:17 PM: -- [~pvillard] [~joewitt] It looks like a Github PR was never submitted. Can I submit one using the attached changes? was (Author: JIRAUSER294662): [~pvillard] It looks like a Github PR was never submitted. Can I submit one using the attached changes? > XMLTransform processor: XSLT doesn't give detail error message with line > number > --- > > Key: NIFI-6498 > URL: https://issues.apache.org/jira/browse/NIFI-6498 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.10.0, 1.9.2 >Reporter: Raymond >Priority: Major > Attachments: > 0001-Add-support-for-Saxon-error-messages-and-functions.patch, > 0001-Add-support-for-Saxon-error-messages-and-functions.patch > > > When something is wrong with the XSLT in the XMLTransform processor you get > an error, but this is only a general error without any details or the line > number where the parsing error occurs. For example when using Saxon from > example I get the following error: > Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element > type "xsl:tomplate" must be terminated by the matching end-tag > "". > When using NiFi the following error is printed: > 11:07:43 CEST > ERROR > 2b4c1dab-3a87-175f-f817-3f3c72af4102 > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform > StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default, > section=1], offset=0, > length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380] > due to IOException thrown from > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException: > java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported: org.apache.nifi.processor.exception.ProcessException: IOException > thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: > java.io.IOException: java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported > The error in NiFi gives almost no information for debugging. Would be a big > improvement if the Saxon error message is catched and add to the NiFi error > message (bulletin board). > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[ https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922485#comment-16922485 ] Nico Verwer edited comment on NIFI-6498 at 9/10/19 7:47 AM: If someone picks this up: Could you please move the version of Saxon-HE to 9.9.1-4? (I removed the earlier suggestion to set a Saxon property, which is better done through the processor's properties, see above.) was (Author: nverwer): If someone picks this up: Could you please move the version of Saxon-HE to 9.9.1-4? I removed the earlier suggestion to set a Saxon property, which is better done through the processor's properties. > XMLTransform processor: XSLT doesn't give detail error message with line > number > --- > > Key: NIFI-6498 > URL: https://issues.apache.org/jira/browse/NIFI-6498 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.9.2 >Reporter: Raymond >Priority: Major > > When something is wrong with the XSLT in the XMLTransform processor you get > an error, but this is only a general error without any details or the line > number where the parsing error occurs. For example when using Saxon from > example I get the following error: > Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element > type "xsl:tomplate" must be terminated by the matching end-tag > "". > When using NiFi the following error is printed: > 11:07:43 CEST > ERROR > 2b4c1dab-3a87-175f-f817-3f3c72af4102 > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform > StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default, > section=1], offset=0, > length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380] > due to IOException thrown from > TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException: > java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported: org.apache.nifi.processor.exception.ProcessException: IOException > thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: > java.io.IOException: java.util.concurrent.ExecutionException: > javax.xml.transform.TransformerConfigurationException: > net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error > reported > The error in NiFi gives almost no information for debugging. Would be a big > improvement if the Saxon error message is catched and add to the NiFi error > message (bulletin board). > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[
https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922485#comment-16922485
]
Nico Verwer edited comment on NIFI-6498 at 9/10/19 6:50 AM:
If someone picks this up: Could you please move the version of Saxon-HE to
9.9.1-4?
I removed the earlier suggestion to set a Saxon property, which is better done
through the processor's properties.
was (Author: nverwer):
If someone picks this up: Could you please move the version of Saxon-HE to
9.9.1-4?
And also do
{code:java}
factory.setFeature("http://saxon.sf.net/feature/allow-external-functions;,
true);{code}
> XMLTransform processor: XSLT doesn't give detail error message with line
> number
> ---
>
> Key: NIFI-6498
> URL: https://issues.apache.org/jira/browse/NIFI-6498
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.9.2
>Reporter: Raymond
>Priority: Major
>
> When something is wrong with the XSLT in the XMLTransform processor you get
> an error, but this is only a general error without any details or the line
> number where the parsing error occurs. For example when using Saxon from
> example I get the following error:
> Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element
> type "xsl:tomplate" must be terminated by the matching end-tag
> "".
> When using NiFi the following error is printed:
> 11:07:43 CEST
> ERROR
> 2b4c1dab-3a87-175f-f817-3f3c72af4102
> TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform
> StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default,
> section=1], offset=0,
> length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380]
> due to IOException thrown from
> TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException:
> java.util.concurrent.ExecutionException:
> javax.xml.transform.TransformerConfigurationException:
> net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error
> reported: org.apache.nifi.processor.exception.ProcessException: IOException
> thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]:
> java.io.IOException: java.util.concurrent.ExecutionException:
> javax.xml.transform.TransformerConfigurationException:
> net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error
> reported
> The error in NiFi gives almost no information for debugging. Would be a big
> improvement if the Saxon error message is catched and add to the NiFi error
> message (bulletin board).
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[
https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922473#comment-16922473
]
Nico Verwer edited comment on NIFI-6498 at 9/10/19 6:47 AM:
This response has been updated with refactored code fragments on 2019-09-10.
Saxon uses an ErrorListener when compiling a stylesheet. The
net.sf.saxon.lib.StandardErrorListener writes messages to System.err, if
nothing else is specified. Where they end up is unclear to me.
To send the error messages to Nifi's log-file, change the following in
TransformXML.java:
{code:java}
private Templates newTemplates(final ProcessContext context, final String
path) throws TransformerConfigurationException, LookupFailureException {
final ComponentLog logger = getLogger();
final Boolean secureProcessing =
context.getProperty(SECURE_PROCESSING).asBoolean();
TransformerFactory factory = TransformerFactory.newInstance();
final boolean isFilename = context.getProperty(XSLT_FILE_NAME).isSet();
if (secureProcessing) {
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
// don't be overly DTD-unfriendly forcing
http://apache.org/xml/features/disallow-doctype-decl
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-parameter-entities;,
false);
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-general-entities;,
false);
}// Set all dynamic properties that are boolean-valued Saxon
features,
// for example "http://saxon.sf.net/feature/allow-external-functions;.
for (final Map.Entry entry :
context.getProperties().entrySet()) {
if (entry.getKey().isDynamic() &&
entry.getKey().getName().startsWith("http://saxon.sf.net/;)) {
factory.setFeature(entry.getKey().getName(),
Boolean.parseBoolean(entry.getValue()));
}
}ErrorListener errorListener = new ErrorListener() {
@Override
public void warning(TransformerException exception) throws
TransformerException {
logger.warn(exception.getMessageAndLocation());
}
@Override
public void error(TransformerException exception) throws
TransformerException {
logger.error(exception.getMessageAndLocation());
}
@Override
public void fatalError(TransformerException exception) throws
TransformerException {
logger.log(LogLevel.FATAL, exception.getMessageAndLocation());
}
};
factory.setErrorListener(errorListener);if(isFilename) {
return factory.newTemplates(new StreamSource(path));
} else {
final String coordinateKey =
lookupService.get().getRequiredKeys().iterator().next();
final Optional attributeValue =
lookupService.get().lookup(Collections.singletonMap(coordinateKey, path));
if (attributeValue.isPresent() &&
StringUtils.isNotBlank(attributeValue.get())) {
return factory.newTemplates(new StreamSource(new
ByteArrayInputStream(attributeValue.get().getBytes(StandardCharsets.UTF_8;
} else {
throw new TransformerConfigurationException("No XSLT definition
is associated to " + path + " in the lookup controller service.");
}
}
}
{code}
The important part is the addition of an ErrorListener. Then, exceptions are
caught and the collected messages are added to the exception.
I have also added the possibility to set binary Saxon properties like
"http://saxon.sf.net/feature/allow-external-functions; from the processor's
properties.
The above will send compile-time messages to the Nifi logging, but we need to
do something similar for run-time messages emitted by . This
requires the following change inside the onTrigger() method:
{code:java}
public void process(final InputStream rawIn, final OutputStream
out) throws IOException {
try (final InputStream in = new BufferedInputStream(rawIn))
{
final Templates templates;
if (cache != null) {
templates = cache.get(path);
} else {
templates = newTemplates(context, path);
}final Transformer transformer
= templates.newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT,
(indentOutput ? "yes" : "no"));
setMessageLogger(transformer);// Pass all dynamic
properties to the transformer
for (final Map.Entry entry
: context.getProperties().entrySet()) {
[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[
https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922485#comment-16922485
]
Nico Verwer edited comment on NIFI-6498 at 9/9/19 8:25 AM:
---
If someone picks this up: Could you please move the version of Saxon-HE to
9.9.1-4?
And also do
{code:java}
factory.setFeature("http://saxon.sf.net/feature/allow-external-functions;,
true);{code}
was (Author: nverwer):
And if someone picks this up: Could you please move the version of Saxon-HE to
9.9.1-4?
> XMLTransform processor: XSLT doesn't give detail error message with line
> number
> ---
>
> Key: NIFI-6498
> URL: https://issues.apache.org/jira/browse/NIFI-6498
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.9.2
>Reporter: Raymond
>Priority: Major
>
> When something is wrong with the XSLT in the XMLTransform processor you get
> an error, but this is only a general error without any details or the line
> number where the parsing error occurs. For example when using Saxon from
> example I get the following error:
> Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element
> type "xsl:tomplate" must be terminated by the matching end-tag
> "".
> When using NiFi the following error is printed:
> 11:07:43 CEST
> ERROR
> 2b4c1dab-3a87-175f-f817-3f3c72af4102
> TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform
> StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default,
> section=1], offset=0,
> length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380]
> due to IOException thrown from
> TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException:
> java.util.concurrent.ExecutionException:
> javax.xml.transform.TransformerConfigurationException:
> net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error
> reported: org.apache.nifi.processor.exception.ProcessException: IOException
> thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]:
> java.io.IOException: java.util.concurrent.ExecutionException:
> javax.xml.transform.TransformerConfigurationException:
> net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error
> reported
> The error in NiFi gives almost no information for debugging. Would be a big
> improvement if the Saxon error message is catched and add to the NiFi error
> message (bulletin board).
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (NIFI-6498) XMLTransform processor: XSLT doesn't give detail error message with line number
[
https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922473#comment-16922473
]
Nico Verwer edited comment on NIFI-6498 at 9/4/19 1:29 PM:
---
Saxon uses an ErrorListener when compiling a stylesheet. The
net.sf.saxon.lib.StandardErrorListener writes messages to System.err, if
nothing else is specified. Where they end up is unclear to me.
To send the error messages to Nifi's log-file, change the following in
TransformXML.java:
{code:java}
private Templates newTemplates(final ProcessContext context, final String
path) throws TransformerConfigurationException, LookupFailureException {
final Boolean secureProcessing =
context.getProperty(SECURE_PROCESSING).asBoolean();
TransformerFactory factory = TransformerFactory.newInstance();
final boolean isFilename = context.getProperty(XSLT_FILE_NAME).isSet();
StringBuffer messages = new StringBuffer();if
(secureProcessing) {
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
// don't be overly DTD-unfriendly forcing
http://apache.org/xml/features/disallow-doctype-decl
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-parameter-entities;,
false);
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-general-entities;,
false);
}
ErrorListener errorListener = new ErrorListener() {
@Override
public void warning(TransformerException exception) throws
TransformerException {
messages.append("Warning:
"+exception.getMessageAndLocation()+"\n");
}
@Override
public void error(TransformerException exception) throws
TransformerException {
messages.append("Error:
"+exception.getMessageAndLocation()+"\n");
}
@Override
public void fatalError(TransformerException exception) throws
TransformerException {
messages.append("Fatal Error:
"+exception.getMessageAndLocation()+"\n");
}
};
factory.setErrorListener(errorListener);
try {
if(isFilename) {
return factory.newTemplates(new StreamSource(path));
} else {
final String coordinateKey =
lookupService.get().getRequiredKeys().iterator().next();
final Optional attributeValue =
lookupService.get().lookup(Collections.singletonMap(coordinateKey, path));
if (attributeValue.isPresent() &&
StringUtils.isNotBlank(attributeValue.get())) {
return factory.newTemplates(new StreamSource(new
ByteArrayInputStream(attributeValue.get().getBytes(StandardCharsets.UTF_8;
} else {
throw new TransformerConfigurationException("No XSLT
definition is associated to " + path + " in the lookup controller service.");
}
}
} catch (TransformerConfigurationException tce) {
throw new TransformerConfigurationException("Errors compiling XSLT:
"+messages.toString(), tce);
}
}
{code}
The important part is the addition of an ErrorListener. Then, exceptions are
caught and the collected messages are added to the exception.
I would make a pull request if I could, but I have not been able to get Nifi
working in Eclipse on my PC (XPS15, i7, 16GB RAM). Eclipse just keeps mavening
things forever. :(
Maybe someone can test this code fragment and add it to Nifi?
was (Author: nverwer):
Saxon uses an ErrorListener when compiling a stylesheet. The
net.sf.saxon.lib.StandardErrorListener writes messages to System.err, if
nothing else is specified. Where they end up is unclear to me.
To send the error messages to Nifi's log-file, change the following in
TransformXML.java:
{code:java}
private Templates newTemplates(final ProcessContext context, final String
path) throws TransformerConfigurationException, LookupFailureException {
final Boolean secureProcessing =
context.getProperty(SECURE_PROCESSING).asBoolean();
TransformerFactory factory = TransformerFactory.newInstance();
final boolean isFilename = context.getProperty(XSLT_FILE_NAME).isSet();
StringBuffer messages = new StringBuffer();if
(secureProcessing) {
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
// don't be overly DTD-unfriendly forcing
http://apache.org/xml/features/disallow-doctype-decl
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-parameter-entities;,
false);
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-general-entities;,
false);
