[jira] [Commented] (NIFI-11156) ValidateXml Processor - validatexml.invalid.error = Validation failed
[
https://issues.apache.org/jira/browse/NIFI-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689240#comment-17689240
]
Adam Taft commented on NIFI-11156:
--
{quote}> adjust one of the existing TestValidateXml methods to assert that the
attribute message contains a stable keyword from an expected error, such as the
word lineNumber.{quote}
Need to be careful here. Validation can fail without a "line number". And
there's a chance of catching IOException too. I'd think the unit test is just
looking for the pattern "Validation failed: ${reason}" and that's probably good
enough.
> ValidateXml Processor - validatexml.invalid.error = Validation failed
> -
>
> Key: NIFI-11156
> URL: https://issues.apache.org/jira/browse/NIFI-11156
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.19.1
> Environment: CentOS 7.5, RedHat 7.9
> Temurin - 11.0.17+8
>Reporter: Bilal
>Priority: Minor
>
> _NiFi_ environment was upgraded from 1.15.1 to 1.19.1. The following
> situation has occurred with the _ValidateXml_ processor:
> _ValidateXml_ processor works correctly on 1.15.1 and
> _validatexml.invalid.error_ attribute gives an detail information about the
> error.
> _ValidateXml_ processor works on 1.19.1 but _validatexml.invalid.error_
> attribute does not give an detail information about the _error.
> validatexml.invalid.error_ attriubute only contains “{_}Validation failed{_}”
> text.
> Similar problem was talked in cloudera community and Eduu said that:
> ===
> Hi
> [@ChuckE|https://community.cloudera.com/t5/user/viewprofilepage/user-id/98065]
> using NiFi 1.15.2 seems to add the error detail in the
> "validatexml.invalid.error" attribute.
> For example --> "cvc-minLength-valid: Value '' with length = '0' is not
> facet-valid with respect to minLength '1' for type 'HotelCode"
> This seems to be an issue starting from NiFi 1.16.3, maybe a bug?
> You can use NiFi 1.15.2 or custom code as
> [@SAMSAL|https://community.cloudera.com/t5/user/viewprofilepage/user-id/80381]
> suggested.
> ===
> Ref: [How to get the reason for invalid
> XML|[https://community.cloudera.com/t5/Support-Questions/How-to-get-the-reason-for-invalid-XML/m-p/348767/highlight/true#M235443]]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11156) ValidateXml Processor - validatexml.invalid.error = Validation failed
[
https://issues.apache.org/jira/browse/NIFI-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688774#comment-17688774
]
Adam Taft commented on NIFI-11156:
--
One more (maybe too obvious) solution just came to me. Arguably this might even
be the best and smallest thing we could do.
If we update StandardSchemaValidator to include the proper nested message when
it throws, then we'd be back to square one. Specifically, line 56 becomes:
throw new ProcessingException("Validation failed: " + e.getMessage(), e);
In my opinion, since StandardSchemaValidator and ProcessingException
effectively masked the existing error, it's ideal that these classes are the
ones made to fix the issue. The one line change above does it.
Additionally, as a response to the comment about "parsing" the error attribute.
The use case here is not about parsing the message, it's about (as stated)
logging the error message. Logging, or generating a support ticket, or
evaluating the flowfile in a queue, etc. The validation messages are only human
readable, yes, but that's the point.
A classic use case for this is when the failure condition results in a ticket
generated for a help system. If you just have some generic phrase, is at is
today, "Validation failed", it gives no more context for debugging or triaging
the incident.
At that point, the only thing you can do with the flowfile is write it to disk
or somehow capture the content so that you can run it through xmllint or
equivalent external from NiFi. That's the pain being avoided with this fix,
very much about logging or viewing the validation error message.
> ValidateXml Processor - validatexml.invalid.error = Validation failed
> -
>
> Key: NIFI-11156
> URL: https://issues.apache.org/jira/browse/NIFI-11156
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.19.1
> Environment: CentOS 7.5, RedHat 7.9
> Temurin - 11.0.17+8
>Reporter: Bilal
>Priority: Minor
>
> _NiFi_ environment was upgraded from 1.15.1 to 1.19.1. The following
> situation has occurred with the _ValidateXml_ processor:
> _ValidateXml_ processor works correctly on 1.15.1 and
> _validatexml.invalid.error_ attribute gives an detail information about the
> error.
> _ValidateXml_ processor works on 1.19.1 but _validatexml.invalid.error_
> attribute does not give an detail information about the _error.
> validatexml.invalid.error_ attriubute only contains “{_}Validation failed{_}”
> text.
> Similar problem was talked in cloudera community and Eduu said that:
> ===
> Hi
> [@ChuckE|https://community.cloudera.com/t5/user/viewprofilepage/user-id/98065]
> using NiFi 1.15.2 seems to add the error detail in the
> "validatexml.invalid.error" attribute.
> For example --> "cvc-minLength-valid: Value '' with length = '0' is not
> facet-valid with respect to minLength '1' for type 'HotelCode"
> This seems to be an issue starting from NiFi 1.16.3, maybe a bug?
> You can use NiFi 1.15.2 or custom code as
> [@SAMSAL|https://community.cloudera.com/t5/user/viewprofilepage/user-id/80381]
> suggested.
> ===
> Ref: [How to get the reason for invalid
> XML|[https://community.cloudera.com/t5/Support-Questions/How-to-get-the-reason-for-invalid-XML/m-p/348767/highlight/true#M235443]]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11156) ValidateXml Processor - validatexml.invalid.error = Validation failed
[
https://issues.apache.org/jira/browse/NIFI-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688714#comment-17688714
]
Adam Taft commented on NIFI-11156:
--
Without arguing about the API decisions happening in all the xml processing
code (which is a different discussion), agreed that unrolling the exception
chain that starts with ProcessException is probably the smallest thing that
ValidateXML could do. Somewhere around line 200 of ValidateXML.java you'd want
to unroll the chain that is now an extra layer deep because of the
ProcessException wrapper. Hopefully you'd find your way down to a meaningful
SAXException, which includes the more useful error message via #getMessage().
The ideal situation is that we wouldn't be passing around validation errors as
exceptions at all. I know this design originates from the JDK, but ideally one
could pass a callback or some other reference to the SchemaValidator in order
to do something better than just rethrowing checked exceptions as unchecked,
which is all the "value" that ProcessingException is apparently adding. But
that's probably a different overhaul and effort though, especially if we don't
want to change anything related to SchemaValidator and StandardSchemaValidator
nor any runtime exceptions that they throw. SchemaValidator could, for example,
return a ValidationResult object, so that callers don't have to trap exceptions
into AtomicBoolean state to decide if it "passed" or "failed" validation.
That being said, rolling out the exception chain to eventually call
getMessage() on the bottom-most exception is probably the most localized update
to ValidateXML that would restore the original behavior. I agree with David on
this assessment, probably the recommended approach for where it's at.
If on the other hand, SchemaValidator wanted to be improved, adding a return
value to it would be the most ideal situation, so that callers can definitively
know if validation was successful and if not what type of error they might be
dealing with. Using exceptions for control flow sucks.
> ValidateXml Processor - validatexml.invalid.error = Validation failed
> -
>
> Key: NIFI-11156
> URL: https://issues.apache.org/jira/browse/NIFI-11156
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.19.1
> Environment: CentOS 7.5, RedHat 7.9
> Temurin - 11.0.17+8
>Reporter: Bilal
>Priority: Minor
>
> _NiFi_ environment was upgraded from 1.15.1 to 1.19.1. The following
> situation has occurred with the _ValidateXml_ processor:
> _ValidateXml_ processor works correctly on 1.15.1 and
> _validatexml.invalid.error_ attribute gives an detail information about the
> error.
> _ValidateXml_ processor works on 1.19.1 but _validatexml.invalid.error_
> attribute does not give an detail information about the _error.
> validatexml.invalid.error_ attriubute only contains “{_}Validation failed{_}”
> text.
> Similar problem was talked in cloudera community and Eduu said that:
> ===
> Hi
> [@ChuckE|https://community.cloudera.com/t5/user/viewprofilepage/user-id/98065]
> using NiFi 1.15.2 seems to add the error detail in the
> "validatexml.invalid.error" attribute.
> For example --> "cvc-minLength-valid: Value '' with length = '0' is not
> facet-valid with respect to minLength '1' for type 'HotelCode"
> This seems to be an issue starting from NiFi 1.16.3, maybe a bug?
> You can use NiFi 1.15.2 or custom code as
> [@SAMSAL|https://community.cloudera.com/t5/user/viewprofilepage/user-id/80381]
> suggested.
> ===
> Ref: [How to get the reason for invalid
> XML|[https://community.cloudera.com/t5/Support-Questions/How-to-get-the-reason-for-invalid-XML/m-p/348767/highlight/true#M235443]]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10016) Unannounced change to flow.json.gz
[ https://issues.apache.org/jira/browse/NIFI-10016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539869#comment-17539869 ] Adam Taft commented on NIFI-10016: -- I don't disagree with this and sympathize with your perspective. From a devops / system admin point of reference, this is a pretty jarring change. And it's going to be confusing to know what to do with it. Like, once flow.json.gz is created, is it safe to delete flow.xml.gz? Or should you hold onto it, just in case? This is definitely going to cause a stumble with system admins, power users, people who have written/maintained backup scripts, etc. Also unfortunately, the reference to flow.json.gz is pretty sparse right now, minus the NiFi Admin Guide you reference. But for completeness, the term for this change is in the "Versioned Components" family of NiFi capabilities. This change is on the path of simplifying AllTheWays in which NiFi can serialize a dataflow definition. So hopefully this is a good thing. But the messaging could have probably been better. Here's a reference to the JIRA ticket and matching github PR, for completeness: https://issues.apache.org/jira/browse/NIFI-9069 https://github.com/apache/nifi/pull/5514 If you have suggestions for the Admin Guide that would better reflect the change, please do try and issue a pull request or make some comments here perhaps. But hopefully the links above will help tie some of the information and rationale together. > Unannounced change to flow.json.gz > -- > > Key: NIFI-10016 > URL: https://issues.apache.org/jira/browse/NIFI-10016 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration, Documentation Website >Affects Versions: 1.16.0, 1.16.1 >Reporter: Daniel Salwerowicz >Priority: Trivial > Attachments: 2022-05-11--12-35-22.png, 2022-05-11--12-37-29.png > > > As far as I am aware there is no information anywhere that flow.xml.gz is now > being replaced by flow.json.gz. > While it might not be that big of a deal to some I was very concerned that > neither changelog, nor migration guide from 1.15 to 1.16 mention that this > has occurred. > The only way I noticed that was because during update I needed to look up > something in nifi.properties file and noticed a configuration option for > flow.json.gz. > This lead me to search for what it is, when the change happened etc. > The only place in documentation where I can see flow.json.gz being mentioned > is in the Admin Guide, which itself pulled a move comparable to ministry of > truth from 1984 and just acts like flow.json.gz was always used in NiFi, no > mention about change or what to do with old flow.xml.gz files. > I think that both changelog and migration guide should at the very least > mention that switch to flow.json.gz has occured, why it happened, and how to > deal with old flow.xml.gz when upgrading. > If I am wrong and there is mention of flow.json.gz anywhere in the official > channels, then I apologize, and ask for link to them so I can read more. > It was very surprising to come across it when I wasn't expecting it after > reading both changelog and migration guide. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (NIFI-9276) Add verification to ListSFTP processor
[ https://issues.apache.org/jira/browse/NIFI-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462024#comment-17462024 ] Adam Taft commented on NIFI-9276: - Is this issue closed? The github pull request has apparently been merged. https://github.com/apache/nifi/pull/5453 > Add verification to ListSFTP processor > -- > > Key: NIFI-9276 > URL: https://issues.apache.org/jira/browse/NIFI-9276 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > Though processor > [verification|https://cwiki.apache.org/confluence/display/NIFI/Component+Configuration+Verification] > is not yet exposed via the UI, config verification steps could be added to > ListSFTP that would in the future allow quick turnaround in verifying its > configuration. For example, it would be useful to be able to quickly see how > many files in the configured directory on the remote server match if the > "Path Filter Regex" property is configured. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-8745) NullPointerException in Bootstrap
[
https://issues.apache.org/jira/browse/NIFI-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Taft updated NIFI-8745:
Description:
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
A fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check (after fixing the
NPE) whether the thrown exception would be logged to the bootstrap log or be
output to standard out, like the above stack trace is currently. Ideally, any
exception thrown by the bootstrap code should be caught and logged, not output
to the console.
It might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
was:
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
A fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check whether the
thrown exception would be logged to the bootstrap log or be output to standard
out, like the above stack trace is currently. Ideally, any exception thrown by
the bootstrap code should be caught and logged, not output to the console.
It might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
> NullPointerException in Bootstrap
> -
>
> Key: NIFI-8745
> URL: https://issues.apache.org/jira/browse/NIFI-8745
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
> Environment: JDK 1.8 - Ubuntu Server
>Reporter: Adam Taft
>Priority: Major
>
> I have seen this NullPointerException several times recently. This is
> apparently coming from the bootstrap component. But really unfortunately, the
> stacktrace is being echoed to standard out/err and not captured in any log.
> {{
> java.lang.NullPointerException
> at
> org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
> at
> org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
> at
>
[jira] [Updated] (NIFI-8745) NullPointerException in Bootstrap
[
https://issues.apache.org/jira/browse/NIFI-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Taft updated NIFI-8745:
Description:
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
A fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check whether the
thrown exception would be logged to the bootstrap log or be output to standard
out, like the above stack trace is currently. Ideally, any exception thrown by
the bootstrap code should be caught and logged, not output to the console.
I might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
was:
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
This fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check whether the
thrown exception would be logged to the bootstrap log or be output to standard
out, like the above stack trace is currently. Ideally, any exception thrown by
the bootstrap code should be caught and logged, not output to the console.
I might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
> NullPointerException in Bootstrap
> -
>
> Key: NIFI-8745
> URL: https://issues.apache.org/jira/browse/NIFI-8745
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
> Environment: JDK 1.8 - Ubuntu Server
>Reporter: Adam Taft
>Priority: Major
>
> I have seen this NullPointerException several times recently. This is
> apparently coming from the bootstrap component. But really unfortunately, the
> stacktrace is being echoed to standard out/err and not captured in any log.
> {{
> java.lang.NullPointerException
> at
> org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
> at
> org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
> at
>
[jira] [Updated] (NIFI-8745) NullPointerException in Bootstrap
[
https://issues.apache.org/jira/browse/NIFI-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Taft updated NIFI-8745:
Description:
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
A fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check whether the
thrown exception would be logged to the bootstrap log or be output to standard
out, like the above stack trace is currently. Ideally, any exception thrown by
the bootstrap code should be caught and logged, not output to the console.
It might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
was:
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
A fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check whether the
thrown exception would be logged to the bootstrap log or be output to standard
out, like the above stack trace is currently. Ideally, any exception thrown by
the bootstrap code should be caught and logged, not output to the console.
I might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
> NullPointerException in Bootstrap
> -
>
> Key: NIFI-8745
> URL: https://issues.apache.org/jira/browse/NIFI-8745
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
> Environment: JDK 1.8 - Ubuntu Server
>Reporter: Adam Taft
>Priority: Major
>
> I have seen this NullPointerException several times recently. This is
> apparently coming from the bootstrap component. But really unfortunately, the
> stacktrace is being echoed to standard out/err and not captured in any log.
> {{
> java.lang.NullPointerException
> at
> org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
> at
> org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
> at
>
[jira] [Created] (NIFI-8745) NullPointerException in Bootstrap
Adam Taft created NIFI-8745:
---
Summary: NullPointerException in Bootstrap
Key: NIFI-8745
URL: https://issues.apache.org/jira/browse/NIFI-8745
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.13.2
Environment: JDK 1.8 - Ubuntu Server
Reporter: Adam Taft
I have seen this NullPointerException several times recently. This is
apparently coming from the bootstrap component. But really unfortunately, the
stacktrace is being echoed to standard out/err and not captured in any log.
{{
java.lang.NullPointerException
at
org.apache.nifi.bootstrap.BootstrapCodec.communicate(BootstrapCodec.java:44)
at
org.apache.nifi.bootstrap.NiFiListener$Listener$2.run(NiFiListener.java:122)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
}}
This fix on line 44 to prevent the NPE should be obvious, but the cause is
unknown. I speculate that the system being run on is scanned (probably a
malware / open port scanner) for compliance reasons. However, I honestly don't
know anything more at this point as to a direct cause.
There is an IOException that can potentially be thrown if the codec
communication is invalid (line 46). It would be good to check whether the
thrown exception would be logged to the bootstrap log or be output to standard
out, like the above stack trace is currently. Ideally, any exception thrown by
the bootstrap code should be caught and logged, not output to the console.
I might be wise to consider this a non-error. An open port scanner (as
hypothesized above) should be able to run against the bootstrap listener
without causing a stack trace at all. Maybe just an "INFO" logging, that some
non-NiFi client connected. It's unlikely that a stacktrace adds much value to
the diagnosing of a random socket client. Maybe the INFO logging could include
remote IP and port of connecting client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-8702) ListFile produces invalid datetime attributes
Adam Taft created NIFI-8702:
---
Summary: ListFile produces invalid datetime attributes
Key: NIFI-8702
URL: https://issues.apache.org/jira/browse/NIFI-8702
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.13.2
Reporter: Adam Taft
The ListFile processor produces date-time attribute strings that are
non-conforming to the ISO-8601 standard. This affects the flowfile attributes
created by the processor: 'file.creationTime', 'file.lastAccessTime' and
'file.lastModifiedTime'.
An example output attribute value looks like: 2021-06-14T18:19:20+
This is non-conforming to the ISO-8601 format, because it mixes "basic" syntax
with "extended" syntax. The 'basic' ISO-8601 format does not include separators
designed for human readability (like dashes, colons, etc.). The 'extended'
syntax is designed for human consumption and includes the separators.
The problem is that ListFile produces attributes that _mix_ basic and extended
format, as shown above, in the offset component. e.g. the offset is missing the
required colon separator.
The above attribute example should instead be formatted as such:
2021-06-14T18:19:20+00:00
Note the colon in the offset component "00:00". The colon is required because
the other segments are using extended format (with separators), and a
consistent representation must be kept.
The test for the output from ListFile should simply be whether the attributes
can be parsed by DateTimeFormatter.ISO_OFFSET_DATE_TIME without throwing an
exception.
Some references:
[https://stackoverflow.com/questions/38252867/what-is-the-right-iso8601-format]
[https://en.wikipedia.org/wiki/ISO_8601#General_principles]
{quote}Representations can be done in one of two formats – a basic format with
a minimal number of separators or an extended format with separators added to
enhance human readability.
{quote}
[https://bugs.openjdk.java.net/browse/JDK-8176547]
The goal of a fix would be to allow downstream processors to properly parse and
interpret fully conformant ISO-8601 attributes using standard
java.time.DateTimeFormatter parsers. The current situation requires a custom
parsing operation (or string hacking) to properly parse with java.time, which
is not ideal.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6394) Ability to export contents of a List Queue
[ https://issues.apache.org/jira/browse/NIFI-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088160#comment-17088160 ] Adam Taft commented on NIFI-6394: - This PR doesn't support CVS and friends, but it's the correct start to that functionality. Once this parameter is exposed as a REST parameter, it would be possible to build a REST-client that can perform the needed transformation to an output representation. https://github.com/apache/nifi/pull/4221 > Ability to export contents of a List Queue > -- > > Key: NIFI-6394 > URL: https://issues.apache.org/jira/browse/NIFI-6394 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Tim Chermak >Assignee: Adam Taft >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When in the List Queue window, we would like to see a new feature enabling > the ability to export the detail contents of the List Queue to XLS or CSV, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-6394) Ability to export contents of a List Queue
[ https://issues.apache.org/jira/browse/NIFI-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088160#comment-17088160 ] Adam Taft edited comment on NIFI-6394 at 4/20/20, 11:26 PM: This PR doesn't support CVS export, but it's the correct start to that functionality. Once this parameter is exposed as a REST parameter, it would be possible to build a REST-client that can perform the needed transformation to an output representation. [https://github.com/apache/nifi/pull/4221] was (Author: taftster): This PR doesn't support CVS and friends, but it's the correct start to that functionality. Once this parameter is exposed as a REST parameter, it would be possible to build a REST-client that can perform the needed transformation to an output representation. https://github.com/apache/nifi/pull/4221 > Ability to export contents of a List Queue > -- > > Key: NIFI-6394 > URL: https://issues.apache.org/jira/browse/NIFI-6394 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Tim Chermak >Assignee: Adam Taft >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When in the List Queue window, we would like to see a new feature enabling > the ability to export the detail contents of the List Queue to XLS or CSV, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6882) PutFile throws NullPointerException if destination directory doesn't exist
[
https://issues.apache.org/jira/browse/NIFI-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017360#comment-17017360
]
Adam Taft commented on NIFI-6882:
-
[~mattyb149] - Can this fix still make it into 1.11.0 (RC2)? Or is it too late
for that? The fix for this is trivial (add a call to File#absolutePath), but
[~JoshE] 's work on this included some additional unit tests to ensure correct
behavior.
If you think we can land this onto 1.11.0, I think we can expedite today even.
> PutFile throws NullPointerException if destination directory doesn't exist
> --
>
> Key: NIFI-6882
> URL: https://issues.apache.org/jira/browse/NIFI-6882
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.10.0
>Reporter: Mark Payne
>Assignee: Josh P
>Priority: Critical
>
> I have a PutFile processor that is configured to create directories if they
> don't exist. I have the directory set to "heap-dumps", which does not exist.
> When data came into the processor, it started throwing the following NPE:
> {code}
> 2019-11-18 16:46:14,879 ERROR [Timer-Driven Process Thread-2]
> o.a.nifi.processors.standard.PutFile
> PutFile[id=0b173f77-d60a-1ce8-d59e-4a9aeff49dbf] Penalizing
> StandardFlowFileRecord[uuid=38a6849b-965b-46b2-96d3-fe37a1c22cf8,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1574113470589-2, container=default,
> section=2], offset=0,
> length=1139822320],offset=0,name=heap.bin.gz,size=1139822320] and
> transferring to failure due to java.lang.NullPointerException:
> java.lang.NullPointerException
> java.lang.NullPointerException: null
> at java.nio.file.Files.provider(Files.java:97)
> at java.nio.file.Files.exists(Files.java:2385)
> at
> org.apache.nifi.processors.standard.PutFile.onTrigger(PutFile.java:243)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-6394) Ability to export contents of a List Queue
[ https://issues.apache.org/jira/browse/NIFI-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Taft reassigned NIFI-6394: --- Assignee: Adam Taft > Ability to export contents of a List Queue > -- > > Key: NIFI-6394 > URL: https://issues.apache.org/jira/browse/NIFI-6394 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Tim Chermak >Assignee: Adam Taft >Priority: Minor > > When in the List Queue window, we would like to see a new feature enabling > the ability to export the detail contents of the List Queue to XLS or CSV, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-6742) H2 test.mv.db left after build
[ https://issues.apache.org/jira/browse/NIFI-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Taft reassigned NIFI-6742: --- Assignee: Adam Taft > H2 test.mv.db left after build > -- > > Key: NIFI-6742 > URL: https://issues.apache.org/jira/browse/NIFI-6742 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Adam Taft >Assignee: Adam Taft >Priority: Trivial > > DBCPServiceTest.java leaves a test file called `test.mv.db` in the home > directory when building NiFi. This is an H2 database file which should > ideally be cleaned up after building. > Recommend using a JUnit TemporaryFolder or a maven target directory which > would enable more automatic cleanup of the temporary file. > There are two references in the unit test to this H2 database file at lines: > 360, 408 > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/src/test/java/org/apache/nifi/dbcp/DBCPServiceTest.java] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6742) H2 test.mv.db left after build
Adam Taft created NIFI-6742: --- Summary: H2 test.mv.db left after build Key: NIFI-6742 URL: https://issues.apache.org/jira/browse/NIFI-6742 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.9.2 Reporter: Adam Taft DBCPServiceTest.java leaves a test file called `test.mv.db` in the home directory when building NiFi. This is an H2 database file which should ideally be cleaned up after building. Recommend using a JUnit TemporaryFolder or a maven target directory which would enable more automatic cleanup of the temporary file. There are two references in the unit test to this H2 database file at lines: 360, 408 [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/src/test/java/org/apache/nifi/dbcp/DBCPServiceTest.java] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6394) Ability to export contents of a List Queue
[ https://issues.apache.org/jira/browse/NIFI-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944870#comment-16944870 ] Adam Taft commented on NIFI-6394: - There is a hard coded limit that constrains this view to 100 flowfiles max. This limit can be seen here: [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardConnectionDAO.java#L361] One starting place as a solution here would be to expose this value as a parameter to the REST API, allowing for options in the UI to "show all flowfiles in queue". It could potentially also open up avenues for exporting the flowfile list as a CSV data file, for dataflow O debugging. > Ability to export contents of a List Queue > -- > > Key: NIFI-6394 > URL: https://issues.apache.org/jira/browse/NIFI-6394 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Tim Chermak >Priority: Minor > > When in the List Queue window, we would like to see a new feature enabling > the ability to export the detail contents of the List Queue to XLS or CSV, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-6202) NiFI flow states one file struck in process and not getting cleared
[ https://issues.apache.org/jira/browse/NIFI-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944864#comment-16944864 ] Adam Taft edited comment on NIFI-6202 at 10/4/19 10:04 PM: --- Somewhat related is NIFI-6394, which implies the ability to view more than 100 files in the queue as well. The cause of the 100 limit is hard coded here: [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardConnectionDAO.java#L361] This value could possible stand to be exposed as a parameter to the REST API, with the default being 100, but allowing the web UI to offer a "show all flowfiles" option in that view. was (Author: taftster): Somewhat related is NIFI-6394, which implies the ability to view more than 100 files in the queue as well. The core of the 100 limit is hard coded here: [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardConnectionDAO.java#L361] This value could possible stand to be exposed as a parameter to the REST API, with the default being 100, but allowing the web UI to offer a "show all flowfiles" option in that view. > NiFI flow states one file struck in process and not getting cleared > > > Key: NIFI-6202 > URL: https://issues.apache.org/jira/browse/NIFI-6202 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.8.0 >Reporter: sayee >Priority: Blocker > Attachments: defect image.PNG > > > Processing large files through NiFi: > 1. Running a large number of data files through NiFi processor causes 1 file > struck in Q and not getting cleared. > 2. only way to clean is delete flow files from repository > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6202) NiFI flow states one file struck in process and not getting cleared
[ https://issues.apache.org/jira/browse/NIFI-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944864#comment-16944864 ] Adam Taft commented on NIFI-6202: - Somewhat related is NIFI-6394, which implies the ability to view more than 100 files in the queue as well. The core of the 100 limit is hard coded here: [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardConnectionDAO.java#L361] This value could possible stand to be exposed as a parameter to the REST API, with the default being 100, but allowing the web UI to offer a "show all flowfiles" option in that view. > NiFI flow states one file struck in process and not getting cleared > > > Key: NIFI-6202 > URL: https://issues.apache.org/jira/browse/NIFI-6202 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.8.0 >Reporter: sayee >Priority: Blocker > Attachments: defect image.PNG > > > Processing large files through NiFi: > 1. Running a large number of data files through NiFi processor causes 1 file > struck in Q and not getting cleared. > 2. only way to clean is delete flow files from repository > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-4425) Establish new repository for NiFi containerization efforts and migrate code
[ https://issues.apache.org/jira/browse/NIFI-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181519#comment-16181519 ] Adam Taft commented on NIFI-4425: - In the minor points department, the name "containerization" is a bit hard/long (imho). Can we shorten? nifi-containers, nifi-container, nifi-cntr, nifi-c14n, nifi-virt > Establish new repository for NiFi containerization efforts and migrate code > --- > > Key: NIFI-4425 > URL: https://issues.apache.org/jira/browse/NIFI-4425 > Project: Apache NiFi > Issue Type: Task > Components: Docker, Tools and Build >Reporter: Aldrin Piri > > As per the discussion that took on the list: > https://lists.apache.org/thread.html/8f4f36fa85c04b02151f7ff0b8402bc350983c88b96dba843f7158bc@%3Cdev.nifi.apache.org%3E > There was a preference to establish a separate release process for NiFi > (inclusive of all subprojects) Dockerfiles and the preference was to make > this against a separate repository. > Accordingly, we should request a new repository. To keep things broadly > scoped, I would recommend calling this nifi-containerization such that we can > include not only Docker specific items but those that apply to this same > general usage of NiFi components. > Once established, we should then consolidate relevant container code from > their current repos to the newly created repo. > We will then need to establish/migrate Docker hub builds to match to the > directory structure we have established. > We will need to update the NiFi release guide to remove the sections > mentioning Docker and will need to create/augment the release docs to talk > about the procedure we will use for managing the Dockerfile release process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4057) Docker Image is twice as large as necessary
[
https://issues.apache.org/jira/browse/NIFI-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075490#comment-16075490
]
Adam Taft commented on NIFI-4057:
-
[~aldrin] Can you please look this over and target this for 1.4.0? Since
you've invested a lot in the NIFI docker image, I was hoping you could signoff
on these changes? This will reduce the docker image size by half. We should
get this in for the next minor release, if it checks out.
> Docker Image is twice as large as necessary
> ---
>
> Key: NIFI-4057
> URL: https://issues.apache.org/jira/browse/NIFI-4057
> Project: Apache NiFi
> Issue Type: Bug
> Components: Docker
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Jordan Moore
>Priority: Minor
>
> By calling {{chown}} as a secondary {{RUN}} command, you effectively double
> the size of image by creating a Docker layer of the same size as the
> extracted binary.
> See GitHub discussion:
> https://github.com/apache/nifi/pull/1372#issuecomment-307592287
> *Expectation*
> The resultant Docker image should be no larger than the Base image + the size
> required by extracting the Nifi binaries.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4057) Docker Image is twice as large as necessary
[
https://issues.apache.org/jira/browse/NIFI-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046054#comment-16046054
]
Adam Taft commented on NIFI-4057:
-
OK, indeed. JSP compilation is being handled by the Eclipse compiler. This is
mentioned in the NOTICE file. I'm thinking JRE will be fine. Let's try it out!
> Docker Image is twice as large as necessary
> ---
>
> Key: NIFI-4057
> URL: https://issues.apache.org/jira/browse/NIFI-4057
> Project: Apache NiFi
> Issue Type: Bug
> Components: Docker
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Jordan Moore
>Priority: Minor
>
> By calling {{chown}} as a secondary {{RUN}} command, you effectively double
> the size of image by creating a Docker layer of the same size as the
> extracted binary.
> See GitHub discussion:
> https://github.com/apache/nifi/pull/1372#issuecomment-307592287
> *Expectation*
> The resultant Docker image should be no larger than the Base image + the size
> required by extracting the Nifi binaries.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (NIFI-4057) Docker Image is twice as large as necessary
[
https://issues.apache.org/jira/browse/NIFI-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046050#comment-16046050
]
Adam Taft commented on NIFI-4057:
-
Yes, you're right. Using the JRE only version might be a good idea as well.
Maybe use openjdk:jre-8 as the base image?
Are there any JSPs etc. that are dependent on the JDK compiler? I don't think
so, but [~mcgilman] would know for sure
Since we're assigning UID=1000, how about GID=1000 to follow tradition? In
fact, I would just recommend we don't bother with the group creation at all,
because calling useradd will create a group with the same GID as the UID by
default.
> Docker Image is twice as large as necessary
> ---
>
> Key: NIFI-4057
> URL: https://issues.apache.org/jira/browse/NIFI-4057
> Project: Apache NiFi
> Issue Type: Bug
> Components: Docker
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Jordan Moore
>Priority: Minor
>
> By calling {{chown}} as a secondary {{RUN}} command, you effectively double
> the size of image by creating a Docker layer of the same size as the
> extracted binary.
> See GitHub discussion:
> https://github.com/apache/nifi/pull/1372#issuecomment-307592287
> *Expectation*
> The resultant Docker image should be no larger than the Base image + the size
> required by extracting the Nifi binaries.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (NIFI-4057) Docker Image is twice as large as necessary
[
https://issues.apache.org/jira/browse/NIFI-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046044#comment-16046044
]
Adam Taft commented on NIFI-4057:
-
This is a good catch and will be super easy to fix.
There are some other cleanup areas that might want to be considered in the
same patch. I'm wondering about:
- GID=50 already exists in openjdk:8 (staff). Why are we attempting to use
and rename an existing group? Just create a new group and be done with it?
- Consolidate the "Setup NiFi user" into a single layer. Combine any
groupadd, useradd, mkdir, chown statements into a single RUN layer.
- Update the deprecated MAINTAINER statement with "LABEL maintainer"
> Docker Image is twice as large as necessary
> ---
>
> Key: NIFI-4057
> URL: https://issues.apache.org/jira/browse/NIFI-4057
> Project: Apache NiFi
> Issue Type: Bug
> Components: Docker
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Jordan Moore
>Priority: Minor
>
> By calling {{chown}} as a secondary {{RUN}} command, you effectively double
> the size of image by creating a Docker layer of the same size as the
> extracted binary.
> See GitHub discussion:
> https://github.com/apache/nifi/pull/1372#issuecomment-307592287
> *Expectation*
> The resultant Docker image should be no larger than the Base image + the size
> required by extracting the Nifi binaries.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
