[jira] [Updated] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
[ https://issues.apache.org/jira/browse/NIFI-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11698: Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Adjust Github Workflow timeouts to allow more time for slow builds > -- > > Key: NIFI-11698 > URL: https://issues.apache.org/jira/browse/NIFI-11698 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Trivial > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Recent CI builds are occasionally failing due to slow worker nodes on certain > platforms. Adjust workflow timeout to allow more time for builds on these > platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
[ https://issues.apache.org/jira/browse/NIFI-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11698: Component/s: Tools and Build > Adjust Github Workflow timeouts to allow more time for slow builds > -- > > Key: NIFI-11698 > URL: https://issues.apache.org/jira/browse/NIFI-11698 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Trivial > Fix For: 2.0.0, 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Recent CI builds are occasionally failing due to slow worker nodes on certain > platforms. Adjust workflow timeout to allow more time for builds on these > platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
[ https://issues.apache.org/jira/browse/NIFI-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733273#comment-17733273 ] ASF subversion and git services commented on NIFI-11698: Commit 7ba33a742c7e1fbe2ed114e43b0c74e33d594f18 in nifi's branch refs/heads/main from Paul Grey [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7ba33a742c ] NIFI-11698 Changed GitHub MacOS and Windows timeouts to 150 mins This closes #7386 Signed-off-by: David Handermann > Adjust Github Workflow timeouts to allow more time for slow builds > -- > > Key: NIFI-11698 > URL: https://issues.apache.org/jira/browse/NIFI-11698 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > Recent CI builds are occasionally failing due to slow worker nodes on certain > platforms. Adjust workflow timeout to allow more time for builds on these > platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
[ https://issues.apache.org/jira/browse/NIFI-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733274#comment-17733274 ] ASF subversion and git services commented on NIFI-11698: Commit e47d18fed0915a919cb0bc057c5856a3c7436120 in nifi's branch refs/heads/support/nifi-1.x from Paul Grey [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e47d18fed0 ] NIFI-11698 Changed GitHub MacOS and Windows timeouts to 150 mins This closes #7386 Signed-off-by: David Handermann (cherry picked from commit 7ba33a742c7e1fbe2ed114e43b0c74e33d594f18) > Adjust Github Workflow timeouts to allow more time for slow builds > -- > > Key: NIFI-11698 > URL: https://issues.apache.org/jira/browse/NIFI-11698 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > Recent CI builds are occasionally failing due to slow worker nodes on certain > platforms. Adjust workflow timeout to allow more time for builds on these > platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7386: NIFI-11698 - adjust github ci workflow timeouts
exceptionfactory closed pull request #7386: NIFI-11698 - adjust github ci workflow timeouts URL: https://github.com/apache/nifi/pull/7386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11694) SAML logout signature verification failed
[
https://issues.apache.org/jira/browse/NIFI-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11694:
Summary: SAML logout signature verification failed (was: SAML logout
failed)
> SAML logout signature verification failed
> -
>
> Key: NIFI-11694
> URL: https://issues.apache.org/jira/browse/NIFI-11694
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Beat Fuellemann
>Assignee: David Handermann
>Priority: Major
>
> We activated SAML Authentication with the following configuration:
> {code:java}
> nifi.security.user.saml.request.signing.enabled=false
> nifi.security.user.saml.want.assertions.signed=true
> nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> nifi.security.user.saml.authentication.expiration=1 hours
> nifi.security.user.saml.single.logout.enabled=true
> nifi.security.user.saml.http.client.truststore.strategy=JDK
> nifi.security.user.saml.http.client.connect.timeout=30 secs
> nifi.security.user.saml.http.client.read.timeout=30 secs{code}
> Login works fine.
> But during logout, it looks that NIFI signs the request, even if we
> "request.signing.enabled=false". This causes the logout fail on the IdP.
> it gives us the following error:
> {code:java}
> 2023-06-15 06:38:35,629 INFO [NiFi Web Server-78]
> org.apache.nifi.web.api.AccessResource Logout Request
> [7b8370e8-752f-484e-8caa-5a8ce3f29caf] Identity [TX] started
> 2023-06-15 06:38:35,673 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256': supported
> 2023-06-15 06:38:35,674 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmlenc#sha256': supported
> 2023-06-15 06:38:35,676 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Resolved
> SignatureSigningParameters:
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signing credential with
> key algorithm: RSA
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature algorithm URI:
> http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature
> KeyInfoGenerator: present
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference digest method
> algorithm URI: http://www.w3.org/2001/04/xmlenc#sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference
> canonicalization algorithm URI: null
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Canonicalization
> algorithm URI: http://www.w3.org/2001/10/xml-exc-c14n#
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver HMAC output length: null
> 2023-06-15 06:38:35,678 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computing signature over input using
> private key of type RSA and JCA algorithm ID SHA256withRSA
> 2023-06-15 06:38:35,691 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computed signature: [3, e, 2, 0, d, 4,
> 0, 7, d, 8, 2, 6, 9, 7, a, f, c, 1, 0, 8, b, 9, 5, f, d, 0, a, 3, 2, 9, b, 9,
> 3, d, b, 5, 2, 4, 2, f, a, 9, 7, 1, 2, 3, d, 3, c, d, 9, 8, 1, 0, a, 5, 1, 8,
> 8, 6, 3, 3, 8, a, a, 7, f, 1, 8, 9, c, a, 3, 5, 7, b, 2, e, c, 2, 5, 3, 7, 1,
> 2, b, 2, 1, 4, 3, e, 6, f, 4, 8, 5, e, 1, d, 3, e, 1, a, 5, 1, a, f, 8, 2, f,
> a, 3, 8, a, 3, 2, f, 0, 6, d, e, 8, 7, b, 9, f, d, 2, 8, b, d, f, 8, 2, 7, 9,
> 3, 5, 1, d, c, 1, 2, e, 3, 4, 8, f, 3, 7, e, 6, 5, c, e, 3, 8, 3, 1, 2, a, 6,
> 5, 6, 1, 2, 8, c, 8, 3, 8, 3, a, a, 9, 6, 2, a, 8, 3, 2, 9, 2, 5, 9, 2, b, e,
> 6, d, 0, 0, e, 1, 8, 9, 2, 4, 0, 2, a, 5, c, b, 3, 1, b, 1, b, b, a, e, 0, f,
> 6, e, 8, 0, b, c, 9, 0, 0, f, c, 1, 7, 5, c, 4, d, b, 5, c, 1, 0, f, b, 3, d,
> 4, c, e, 5, 7, 4, 3, 8, f, b, 1, f, 1, d, a, a, 0, c, 8, e, d, b, 5, 0, 5, 9,
> 7, a, c, 8, 7, 9, 4, 4, d, f, 1, 3, 2, 9, 6, 6, 2, 4, 1, e, c, 8, 3, 7, 3, 2,
> 4, 9, a, 9, 4, 0, 3, c, 4, b, 2, f, 1, b, 9, b, 4, 3, 1, f, 6, d, 3, d, 4, 5,
> 0, f, 7, 8, d, 1, c, 1, 8, f, 2, 4, 8, 3, 3, 9, e, 3, 4, b, 5, 0, 9, 9, 1, 0,
> c, b, e, 3, 7, 9, 4, 4, d, 7, a, a, 4, 6, 6, 0, 1, b, c, 8, b, 4, c, 9, c, a,
> b, 2, b, e, d, 4, 4, 4, 0, a, b, 9, 4, 4, 4, 4, 9, e, a, b, 4, b, 0, 1, 4, 0,
> b, 7, 2, f, d, b, 8, a, a, 8, f, 8, e, 3, 8, 9, 0, c, 8, f, 3, 0, 6, 0, 9, 3,
> d, 5, c, 3,
[jira] [Commented] (NIFI-11694) SAML logout failed
[
https://issues.apache.org/jira/browse/NIFI-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733258#comment-17733258
]
David Handermann commented on NIFI-11694:
-
Thanks for describing the issue [~fube1].
As noted in the NiFi Admin Guide section on
[SAML|https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#saml],
the {{saml.request.signing.enabled}} property controls how NiFi generates and
renders the Service Provider Metadata XML. The property does not control how
NiFi verifies single logout requests or responses.
The SAML Identity Provider configuration controls the signing of Single Logout
Requests, and signature verification is a standard part of the process to avoid
malicious logout requests.
It is possible to disable SAML Single Logout, which will invalidate the local
NiFi session, without logging out of the identity provider.
> SAML logout failed
> --
>
> Key: NIFI-11694
> URL: https://issues.apache.org/jira/browse/NIFI-11694
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Beat Fuellemann
>Assignee: David Handermann
>Priority: Major
>
> We activated SAML Authentication with the following configuration:
> {code:java}
> nifi.security.user.saml.request.signing.enabled=false
> nifi.security.user.saml.want.assertions.signed=true
> nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> nifi.security.user.saml.authentication.expiration=1 hours
> nifi.security.user.saml.single.logout.enabled=true
> nifi.security.user.saml.http.client.truststore.strategy=JDK
> nifi.security.user.saml.http.client.connect.timeout=30 secs
> nifi.security.user.saml.http.client.read.timeout=30 secs{code}
> Login works fine.
> But during logout, it looks that NIFI signs the request, even if we
> "request.signing.enabled=false". This causes the logout fail on the IdP.
> it gives us the following error:
> {code:java}
> 2023-06-15 06:38:35,629 INFO [NiFi Web Server-78]
> org.apache.nifi.web.api.AccessResource Logout Request
> [7b8370e8-752f-484e-8caa-5a8ce3f29caf] Identity [TX] started
> 2023-06-15 06:38:35,673 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256': supported
> 2023-06-15 06:38:35,674 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmlenc#sha256': supported
> 2023-06-15 06:38:35,676 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Resolved
> SignatureSigningParameters:
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signing credential with
> key algorithm: RSA
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature algorithm URI:
> http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature
> KeyInfoGenerator: present
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference digest method
> algorithm URI: http://www.w3.org/2001/04/xmlenc#sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference
> canonicalization algorithm URI: null
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Canonicalization
> algorithm URI: http://www.w3.org/2001/10/xml-exc-c14n#
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver HMAC output length: null
> 2023-06-15 06:38:35,678 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computing signature over input using
> private key of type RSA and JCA algorithm ID SHA256withRSA
> 2023-06-15 06:38:35,691 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computed signature: [3, e, 2, 0, d, 4,
> 0, 7, d, 8, 2, 6, 9, 7, a, f, c, 1, 0, 8, b, 9, 5, f, d, 0, a, 3, 2, 9, b, 9,
> 3, d, b, 5, 2, 4, 2, f, a, 9, 7, 1, 2, 3, d, 3, c, d, 9, 8, 1, 0, a, 5, 1, 8,
> 8, 6, 3, 3, 8, a, a, 7, f, 1, 8, 9, c, a, 3, 5, 7, b, 2, e, c, 2, 5, 3, 7, 1,
> 2, b, 2, 1, 4, 3, e, 6, f, 4, 8, 5, e, 1, d, 3, e, 1, a, 5, 1, a, f, 8, 2, f,
> a, 3, 8, a, 3, 2, f, 0, 6, d, e, 8, 7, b, 9, f, d, 2, 8, b, d, f, 8, 2, 7, 9,
> 3, 5, 1, d, c, 1, 2, e, 3, 4, 8, f, 3, 7, e, 6, 5, c, e, 3, 8, 3, 1, 2, a, 6,
> 5, 6, 1, 2, 8, c, 8, 3, 8, 3, a, a, 9, 6, 2, a, 8, 3, 2, 9, 2, 5, 9, 2, b, e,
> 6, d, 0, 0, e, 1, 8, 9, 2, 4, 0, 2, a, 5, c, b, 3, 1, b, 1, b, b, a, e, 0, f,
> 6, e, 8, 0, b, c, 9, 0
[jira] [Assigned] (NIFI-11694) SAML logout failed
[
https://issues.apache.org/jira/browse/NIFI-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann reassigned NIFI-11694:
---
Assignee: David Handermann
> SAML logout failed
> --
>
> Key: NIFI-11694
> URL: https://issues.apache.org/jira/browse/NIFI-11694
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Beat Fuellemann
>Assignee: David Handermann
>Priority: Major
>
> We activated SAML Authentication with the following configuration:
> {code:java}
> nifi.security.user.saml.request.signing.enabled=false
> nifi.security.user.saml.want.assertions.signed=true
> nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> nifi.security.user.saml.authentication.expiration=1 hours
> nifi.security.user.saml.single.logout.enabled=true
> nifi.security.user.saml.http.client.truststore.strategy=JDK
> nifi.security.user.saml.http.client.connect.timeout=30 secs
> nifi.security.user.saml.http.client.read.timeout=30 secs{code}
> Login works fine.
> But during logout, it looks that NIFI signs the request, even if we
> "request.signing.enabled=false". This causes the logout fail on the IdP.
> it gives us the following error:
> {code:java}
> 2023-06-15 06:38:35,629 INFO [NiFi Web Server-78]
> org.apache.nifi.web.api.AccessResource Logout Request
> [7b8370e8-752f-484e-8caa-5a8ce3f29caf] Identity [TX] started
> 2023-06-15 06:38:35,673 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256': supported
> 2023-06-15 06:38:35,674 DEBUG [NiFi Web Server-78]
> o.o.xmlsec.algorithm.AlgorithmRegistry Runtime support eval for algorithm URI
> 'http://www.w3.org/2001/04/xmlenc#sha256': supported
> 2023-06-15 06:38:35,676 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Resolved
> SignatureSigningParameters:
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signing credential with
> key algorithm: RSA
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature algorithm URI:
> http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Signature
> KeyInfoGenerator: present
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference digest method
> algorithm URI: http://www.w3.org/2001/04/xmlenc#sha256
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Reference
> canonicalization algorithm URI: null
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver Canonicalization
> algorithm URI: http://www.w3.org/2001/10/xml-exc-c14n#
> 2023-06-15 06:38:35,677 DEBUG [NiFi Web Server-78]
> o.o.x.i.BasicSignatureSigningParametersResolver HMAC output length: null
> 2023-06-15 06:38:35,678 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computing signature over input using
> private key of type RSA and JCA algorithm ID SHA256withRSA
> 2023-06-15 06:38:35,691 DEBUG [NiFi Web Server-78]
> o.opensaml.security.crypto.SigningUtil Computed signature: [3, e, 2, 0, d, 4,
> 0, 7, d, 8, 2, 6, 9, 7, a, f, c, 1, 0, 8, b, 9, 5, f, d, 0, a, 3, 2, 9, b, 9,
> 3, d, b, 5, 2, 4, 2, f, a, 9, 7, 1, 2, 3, d, 3, c, d, 9, 8, 1, 0, a, 5, 1, 8,
> 8, 6, 3, 3, 8, a, a, 7, f, 1, 8, 9, c, a, 3, 5, 7, b, 2, e, c, 2, 5, 3, 7, 1,
> 2, b, 2, 1, 4, 3, e, 6, f, 4, 8, 5, e, 1, d, 3, e, 1, a, 5, 1, a, f, 8, 2, f,
> a, 3, 8, a, 3, 2, f, 0, 6, d, e, 8, 7, b, 9, f, d, 2, 8, b, d, f, 8, 2, 7, 9,
> 3, 5, 1, d, c, 1, 2, e, 3, 4, 8, f, 3, 7, e, 6, 5, c, e, 3, 8, 3, 1, 2, a, 6,
> 5, 6, 1, 2, 8, c, 8, 3, 8, 3, a, a, 9, 6, 2, a, 8, 3, 2, 9, 2, 5, 9, 2, b, e,
> 6, d, 0, 0, e, 1, 8, 9, 2, 4, 0, 2, a, 5, c, b, 3, 1, b, 1, b, b, a, e, 0, f,
> 6, e, 8, 0, b, c, 9, 0, 0, f, c, 1, 7, 5, c, 4, d, b, 5, c, 1, 0, f, b, 3, d,
> 4, c, e, 5, 7, 4, 3, 8, f, b, 1, f, 1, d, a, a, 0, c, 8, e, d, b, 5, 0, 5, 9,
> 7, a, c, 8, 7, 9, 4, 4, d, f, 1, 3, 2, 9, 6, 6, 2, 4, 1, e, c, 8, 3, 7, 3, 2,
> 4, 9, a, 9, 4, 0, 3, c, 4, b, 2, f, 1, b, 9, b, 4, 3, 1, f, 6, d, 3, d, 4, 5,
> 0, f, 7, 8, d, 1, c, 1, 8, f, 2, 4, 8, 3, 3, 9, e, 3, 4, b, 5, 0, 9, 9, 1, 0,
> c, b, e, 3, 7, 9, 4, 4, d, 7, a, a, 4, 6, 6, 0, 1, b, c, 8, b, 4, c, 9, c, a,
> b, 2, b, e, d, 4, 4, 4, 0, a, b, 9, 4, 4, 4, 4, 9, e, a, b, 4, b, 0, 1, 4, 0,
> b, 7, 2, f, d, b, 8, a, a, 8, f, 8, e, 3, 8, 9, 0, c, 8, f, 3, 0, 6, 0, 9, 3,
> d, 5, c, 3, 5, 6, a, 6, e, 1, d, 5, c, 5, a, 4, 9, 2, 3, c, d, 5, 6, 8, f, 1,
> 3, f, c, 4, 5, 4, 4, 9
[jira] [Updated] (NIFI-11699) Fix dynamic properties in SnowflakeComputingConnectionPool
[
https://issues.apache.org/jira/browse/NIFI-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-11699:
---
Description:
NIFI-11151 removed {{SENSITIVE.*}} legacy dynamic properties from
{{AbstractDBCPConnectionPool}} in favour of
{{@SupportsSensitiveDynamicProperties}} which should be used by new child
processors.
The dynamic property handling was moved to {{DBCPConnectionPool}} but it is
missing from {{SnowflakeComputingConnectionPool}} which was also an existing
implementation.
As it was released in 1.21.0 and there has been no bug report, I assume the
feature is not really used in the Snowflake controller service and for this
reason I would move forward and add back the dynamic properties with
{{@SupportsSensitiveDynamicProperties}} instead of the legacy {{SENSITIVE.*}}.
was:
NIFI-11151 removed {{SENSITIVE.*}} legacy dynamic properties from
{{AbstractDBCPConnectionPool}} in favour of
{{@SupportsSensitiveDynamicProperties}} which should be used by new child
processors.
The dynamic property handling was moved to {{DBCPConnectionPool}} but it is
missing from {{SnowflakeComputingConnectionPool}} which was also an existing
implementation.
As it has already been released in 1.21.0 and there has been no bug report, I
assume the feature is not really used in the Snowflake controller service and
for this reason I would move forward and add back the dynamic properties with
{{@SupportsSensitiveDynamicProperties}} instead of {{SENSITIVE.*}}.
> Fix dynamic properties in SnowflakeComputingConnectionPool
> --
>
> Key: NIFI-11699
> URL: https://issues.apache.org/jira/browse/NIFI-11699
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> NIFI-11151 removed {{SENSITIVE.*}} legacy dynamic properties from
> {{AbstractDBCPConnectionPool}} in favour of
> {{@SupportsSensitiveDynamicProperties}} which should be used by new child
> processors.
> The dynamic property handling was moved to {{DBCPConnectionPool}} but it is
> missing from {{SnowflakeComputingConnectionPool}} which was also an existing
> implementation.
> As it was released in 1.21.0 and there has been no bug report, I assume the
> feature is not really used in the Snowflake controller service and for this
> reason I would move forward and add back the dynamic properties with
> {{@SupportsSensitiveDynamicProperties}} instead of the legacy {{SENSITIVE.*}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11699) Fix dynamic properties in SnowflakeComputingConnectionPool
[
https://issues.apache.org/jira/browse/NIFI-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-11699:
---
Status: Patch Available (was: Open)
> Fix dynamic properties in SnowflakeComputingConnectionPool
> --
>
> Key: NIFI-11699
> URL: https://issues.apache.org/jira/browse/NIFI-11699
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> NIFI-11151 removed {{SENSITIVE.*}} legacy dynamic properties from
> {{AbstractDBCPConnectionPool}} in favour of
> {{@SupportsSensitiveDynamicProperties}} which should be used by new child
> processors.
> The dynamic property handling was moved to {{DBCPConnectionPool}} but it is
> missing from {{SnowflakeComputingConnectionPool}} which was also an existing
> implementation.
> As it has already been released in 1.21.0 and there has been no bug report, I
> assume the feature is not really used in the Snowflake controller service and
> for this reason I would move forward and add back the dynamic properties with
> {{@SupportsSensitiveDynamicProperties}} instead of {{SENSITIVE.*}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] turcsanyip opened a new pull request, #7387: NIFI-11699: Fixed dynamic properties in SnowflakeComputingConnectionPool
turcsanyip opened a new pull request, #7387: URL: https://github.com/apache/nifi/pull/7387 Also fixed driver class name handling (caused IT test failure). Other test fixes to make IT tests work. # Summary [NIFI-11699](https://issues.apache.org/jira/browse/NIFI-11699) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11699) Fix dynamic properties in SnowflakeComputingConnectionPool
Peter Turcsanyi created NIFI-11699:
--
Summary: Fix dynamic properties in SnowflakeComputingConnectionPool
Key: NIFI-11699
URL: https://issues.apache.org/jira/browse/NIFI-11699
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
NIFI-11151 removed {{SENSITIVE.*}} legacy dynamic properties from
{{AbstractDBCPConnectionPool}} in favour of
{{@SupportsSensitiveDynamicProperties}} which should be used by new child
processors.
The dynamic property handling was moved to {{DBCPConnectionPool}} but it is
missing from {{SnowflakeComputingConnectionPool}} which was also an existing
implementation.
As it has already been released in 1.21.0 and there has been no bug report, I
assume the feature is not really used in the Snowflake controller service and
for this reason I would move forward and add back the dynamic properties with
{{@SupportsSensitiveDynamicProperties}} instead of {{SENSITIVE.*}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
lordgamez commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1231548116
##
extensions/aws/processors/PutS3Object.cpp:
##
@@ -77,7 +78,31 @@ void PutS3Object::onSchedule(const
std::shared_ptr &contex
use_virtual_addressing_ = !*use_path_style_access;
}
+ context->getProperty(MultipartThreshold.getName(), multipart_threshold_);
+ if (multipart_threshold_ > getMaxUploadSize() || multipart_threshold_ <
getMinPartSize()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Multipart Threshold is not
between the valid 5MB and 5GB range!");
+ }
+ logger_->log_debug("PutS3Object: Multipart Threshold %" PRIu64,
multipart_threshold_);
+ context->getProperty(MultipartPartSize.getName(), multipart_size_);
+ if (multipart_size_ > getMaxUploadSize() || multipart_size_ <
getMinPartSize()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Multipart Part Size is not
between the valid 5MB and 5GB range!");
+ }
+ logger_->log_debug("PutS3Object: Multipart Size %" PRIu64, multipart_size_);
+
+
+ multipart_upload_ageoff_interval_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadAgeOffInterval.getName()).getMilliseconds();
+ logger_->log_debug("PutS3Object: Multipart Upload Ageoff Interval %" PRIu64
" ms", multipart_upload_ageoff_interval_.count());
+
+ multipart_upload_max_age_threshold_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadMaxAgeThreshold.getName()).getMilliseconds();
+ logger_->log_debug("PutS3Object: Multipart Upload Max Age Threshold %"
PRIu64 " ms", multipart_upload_max_age_threshold_.count());
+
fillUserMetadata(context);
+
+ std::string multipart_temp_dir;
+ context->getProperty(TemporaryDirectoryMultipartState.getName(),
multipart_temp_dir);
+
+
+ s3_wrapper_.initailizeMultipartUploadStateStorage(multipart_temp_dir,
getUUIDStr());
Review Comment:
First I implemented it with the state manager, but there was a problem with
that. The state manager only commits the state at session commit and if the
process is killed or the upload fails and the session is rolled back then the
states between the part uploads are lost. Even when I tried to commit the state
manager manually it caused an exception at the end of the trigger because the
session commit could not commit (or rollback in case of a failure) an already
commited session. It may be the same issue in NiFi because they also use a
separate temporary directory for this state management.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
[ https://issues.apache.org/jira/browse/NIFI-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-11698: - Status: Patch Available (was: In Progress) > Adjust Github Workflow timeouts to allow more time for slow builds > -- > > Key: NIFI-11698 > URL: https://issues.apache.org/jira/browse/NIFI-11698 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > Recent CI builds are occasionally failing due to slow worker nodes on certain > platforms. Adjust workflow timeout to allow more time for builds on these > platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 opened a new pull request, #7386: NIFI-11698 - adjust github ci workflow timeouts
greyp9 opened a new pull request, #7386: URL: https://github.com/apache/nifi/pull/7386 ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
[ https://issues.apache.org/jira/browse/NIFI-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733229#comment-17733229 ] Paul Grey commented on NIFI-11698: -- https://github.com/apache/nifi/actions/runs/5282041774 Windows build timed out after two hours. MacOS build is also close to existing timeout. > Adjust Github Workflow timeouts to allow more time for slow builds > -- > > Key: NIFI-11698 > URL: https://issues.apache.org/jira/browse/NIFI-11698 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Trivial > > Recent CI builds are occasionally failing due to slow worker nodes on certain > platforms. Adjust workflow timeout to allow more time for builds on these > platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11698) Adjust Github Workflow timeouts to allow more time for slow builds
Paul Grey created NIFI-11698: Summary: Adjust Github Workflow timeouts to allow more time for slow builds Key: NIFI-11698 URL: https://issues.apache.org/jira/browse/NIFI-11698 Project: Apache NiFi Issue Type: Improvement Reporter: Paul Grey Assignee: Paul Grey Recent CI builds are occasionally failing due to slow worker nodes on certain platforms. Adjust workflow timeout to allow more time for builds on these platforms to complete. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11696) Upgrade Bouncy Castle to 1.74
[ https://issues.apache.org/jira/browse/NIFI-11696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11696: Fix Version/s: 2.0.0 1.23.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Bouncy Castle to 1.74 > - > > Key: NIFI-11696 > URL: https://issues.apache.org/jira/browse/NIFI-11696 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.23.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Bouncy Castle [1.74|https://www.bouncycastle.org/releasenotes.html#r1rv74] > includes a number of bug fixes and feature improvements over previous > versions. > Bouncy Castle 1.72 and 1.73 included the defunct SIKE algorithm, which added > multiple megabytes to the provider library. Version 1.74 removed this > algorithm, minimizing the size impact of the new version. > Bouncy Castle 1.74 also resolves CVE-2023-33201 related to LDAP certificate > store handling. Apache NiFi does not use the X509LDAPCertStoreSpi class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11696) Upgrade Bouncy Castle to 1.74
[ https://issues.apache.org/jira/browse/NIFI-11696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733225#comment-17733225 ] ASF subversion and git services commented on NIFI-11696: Commit 6b19ab8eaa91129b02f48d961435951355dd9be2 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6b19ab8eaa ] NIFI-11696 Upgraded Bouncy Castle from 1.71 to 1.74 - Adjusted nifi-repository-encryption to remove dependency on Bouncy Castle Provider - Updated Google Cloud Provider dependencies to remove exclusions and dependencies on Bouncy Castle that no longer apply to current versions Signed-off-by: Matt Burgess This closes #7384 > Upgrade Bouncy Castle to 1.74 > - > > Key: NIFI-11696 > URL: https://issues.apache.org/jira/browse/NIFI-11696 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 1h > Remaining Estimate: 0h > > Bouncy Castle [1.74|https://www.bouncycastle.org/releasenotes.html#r1rv74] > includes a number of bug fixes and feature improvements over previous > versions. > Bouncy Castle 1.72 and 1.73 included the defunct SIKE algorithm, which added > multiple megabytes to the provider library. Version 1.74 removed this > algorithm, minimizing the size impact of the new version. > Bouncy Castle 1.74 also resolves CVE-2023-33201 related to LDAP certificate > store handling. Apache NiFi does not use the X509LDAPCertStoreSpi class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11696) Upgrade Bouncy Castle to 1.74
[ https://issues.apache.org/jira/browse/NIFI-11696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733222#comment-17733222 ] ASF subversion and git services commented on NIFI-11696: Commit 22008c23319fdd6bc494bdf8f2c9723957e7bb02 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=22008c2331 ] NIFI-11696 Upgraded Bouncy Castle from 1.71 to 1.74 - Adjusted nifi-repository-encryption to remove dependency on Bouncy Castle Provider - Updated Google Cloud Provider dependencies to remove exclusions and dependencies on Bouncy Castle that no longer apply to current versions Signed-off-by: Matt Burgess > Upgrade Bouncy Castle to 1.74 > - > > Key: NIFI-11696 > URL: https://issues.apache.org/jira/browse/NIFI-11696 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 50m > Remaining Estimate: 0h > > Bouncy Castle [1.74|https://www.bouncycastle.org/releasenotes.html#r1rv74] > includes a number of bug fixes and feature improvements over previous > versions. > Bouncy Castle 1.72 and 1.73 included the defunct SIKE algorithm, which added > multiple megabytes to the provider library. Version 1.74 removed this > algorithm, minimizing the size impact of the new version. > Bouncy Castle 1.74 also resolves CVE-2023-33201 related to LDAP certificate > store handling. Apache NiFi does not use the X509LDAPCertStoreSpi class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] turcsanyip commented on a diff in pull request #7076: NIFI-11294 - Add support for component state as checkpointing strategy
turcsanyip commented on code in PR #7076:
URL: https://github.com/apache/nifi/pull/7076#discussion_r1206003566
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/ConsumeAzureEventHub.java:
##
@@ -403,36 +420,76 @@ public void stopClient() {
getLogger().warn("Event Processor Client stop failed", e);
}
eventProcessorClient = null;
-processSessionFactory = null;
-readerFactory = null;
-writerFactory = null;
}
}
-protected EventProcessorClient createClient(final ProcessContext context) {
-namespaceName =
context.getProperty(NAMESPACE).evaluateAttributeExpressions().getValue();
+protected EventProcessorClient createClient(final ProcessContext context,
final ProcessSessionFactory processSessionFactory) {
+final String namespaceName =
context.getProperty(NAMESPACE).evaluateAttributeExpressions().getValue();
final String eventHubName =
context.getProperty(EVENT_HUB_NAME).evaluateAttributeExpressions().getValue();
final String consumerGroup =
context.getProperty(CONSUMER_GROUP).evaluateAttributeExpressions().getValue();
+final String identifier = UUID.randomUUID().toString();
-final String containerName =
defaultIfBlank(context.getProperty(STORAGE_CONTAINER_NAME).evaluateAttributeExpressions().getValue(),
eventHubName);
-final String storageConnectionString =
createStorageConnectionString(context);
-final BlobContainerAsyncClient blobContainerAsyncClient = new
BlobContainerClientBuilder()
-.connectionString(storageConnectionString)
-.containerName(containerName)
-.buildAsyncClient();
-final BlobCheckpointStore checkpointStore = new
BlobCheckpointStore(blobContainerAsyncClient);
+CheckpointStore checkpointStore;
+final Map legacyPartitionEventPosition;
+
+final CheckpointStrategy checkpointStrategy =
CheckpointStrategy.valueOf(
+context.getProperty(CHECKPOINT_STRATEGY).getValue()
+);
+if (checkpointStrategy == CheckpointStrategy.AZURE_BLOB_STORAGE) {
+final String containerName =
defaultIfBlank(context.getProperty(STORAGE_CONTAINER_NAME).evaluateAttributeExpressions().getValue(),
eventHubName);
+final String storageConnectionString =
createStorageConnectionString(context);
+final BlobContainerAsyncClient blobContainerAsyncClient =
getBlobContainerAsyncClient(containerName, storageConnectionString);
+checkpointStore =
getCheckpointStoreFromBlobContainer(blobContainerAsyncClient);
+legacyPartitionEventPosition =
getLegacyPartitionEventPosition(blobContainerAsyncClient, consumerGroup);
+} else {
+checkpointStore = new ComponentStateCheckpointStore(
+identifier,
+new ComponentStateCheckpointStore.State() {
+public StateMap getState() throws IOException {
+final ProcessSession session =
processSessionFactory.createSession();
+return session.getState(Scope.CLUSTER);
+}
+
+public boolean replaceState(StateMap oldValue,
Map newValue) throws IOException {
+final ProcessSession session =
processSessionFactory.createSession();
+if (!session.replaceState(oldValue, newValue,
Scope.CLUSTER)) {
Review Comment:
`ProcessSession.replaceState()` does not provide fully proper optimistic
locking because the state can be changed by another session between
`replaceState()` and `commit()`. So even if `replaceState()` returns `true`,
the state change may be omitted at commit time due to a concurrent update.
I suggest using `ProcessContext.getStateManager().replace()` instead which
has the right optimistic locking semantics which is required from a checkpoint
store implementation.
Please note `ProcessContext.getStateManager().replace()` cannot initialize
the state currently (see also
[NIFI-11595](https://issues.apache.org/jira/browse/NIFI-11595)) so it needs to
be created with `setState()` before using `replace()`. E.g. in `@OnScheduled`
like this:
```
@OnScheduled
public void onScheduled(ProcessContext context) throws IOException {
if (getNodeTypeProvider().isPrimary()) {
final StateManager stateManager = context.getStateManager();
final StateMap state = stateManager.getState(Scope.CLUSTER);
if (!state.getStateVersion().isPresent()) {
stateManager.setState(new HashMap<>(), Scope.CLUSTER);
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
[jira] [Updated] (NIFI-11595) StateProvider.replace() cannot create the initial state
[
https://issues.apache.org/jira/browse/NIFI-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-11595:
Fix Version/s: 1.latest
2.latest
> StateProvider.replace() cannot create the initial state
> ---
>
> Key: NIFI-11595
> URL: https://issues.apache.org/jira/browse/NIFI-11595
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{StateProvider.replace()}} method properly works when the state already
> exists and persisted in the storage. However, it cannot create the state at
> first run.
> As a workaround, {{setState()}} needs to be used but it does not provide the
> same compare-and-swap mechanism as {{replace()}} so it is the caller
> responsibility to handle concurrency.
> To lift this responsibility from the clients and also to provide a more
> consistent API, {{replace()}} should support creating the initial state. It
> should be able to move the state "from nothing to X" not only "from X1 to X2"
> and also providing the same compare-and-swap logic.
> Affected {{StateProvider}} implementations:
> - {{ZooKeeperStateProvider}}
> - {{RedisStateProvider}}
> - {{KubernetesConfigMapStateProvider}}
> - {{WriteAheadLocalStateProvider}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on a diff in pull request #7384: NIFI-11696 Upgrade Bouncy Castle from 1.71 to 1.74
greyp9 commented on code in PR #7384:
URL: https://github.com/apache/nifi/pull/7384#discussion_r1231459842
##
nifi-commons/nifi-repository-encryption/src/main/java/org/apache/nifi/repository/encryption/AesGcmByteArrayRepositoryEncryptor.java:
##
@@ -74,9 +73,17 @@ protected byte[] encrypt(final byte[] record, final String
recordId, final Strin
try {
final byte[] encryptedRecord = cipher.doFinal(record);
final byte[] serializedMetadata = getMetadata(keyId,
cipher.getIV(), encryptedRecord.length);
-return Arrays.concatenate(serializedMetadata, encryptedRecord);
+return concatenate(serializedMetadata, encryptedRecord);
Review Comment:
Thanks, I missed that commons is an implicit dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11697) Update snappy-java to 1.1.10.1
[ https://issues.apache.org/jira/browse/NIFI-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11697: Fix Version/s: 2.0.0 1.23.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Update snappy-java to 1.1.10.1 > -- > > Key: NIFI-11697 > URL: https://issues.apache.org/jira/browse/NIFI-11697 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 2.0.0, 1.23.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update snappy-java to 1.1.10.1 for important fixes, release notes here: > https://github.com/xerial/snappy-java/releases/tag/v1.1.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11697) Update snappy-java to 1.1.10.1
[ https://issues.apache.org/jira/browse/NIFI-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733209#comment-17733209 ] ASF subversion and git services commented on NIFI-11697: Commit 6a129be1149a6c441618db6b22557875e474b426 in nifi's branch refs/heads/main from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6a129be114 ] NIFI-11697: Bump snappy-java from 1.1.10.0 to 1.1.10.1 Bumps [snappy-java](https://github.com/xerial/snappy-java) from 1.1.10.0 to 1.1.10.1. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/v1.1.10.0...v1.1.10.1) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:production ... Signed-off-by: dependabot[bot] Signed-off-by: Matt Burgess This closes #7383 > Update snappy-java to 1.1.10.1 > -- > > Key: NIFI-11697 > URL: https://issues.apache.org/jira/browse/NIFI-11697 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > Update snappy-java to 1.1.10.1 for important fixes, release notes here: > https://github.com/xerial/snappy-java/releases/tag/v1.1.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dependabot[bot] commented on pull request #7383: NIFI-11697: Bump snappy-java from 1.1.10.0 to 1.1.10.1
dependabot[bot] commented on PR #7383: URL: https://github.com/apache/nifi/pull/7383#issuecomment-1593612949 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 closed pull request #7383: NIFI-11697: Bump snappy-java from 1.1.10.0 to 1.1.10.1
mattyb149 closed pull request #7383: NIFI-11697: Bump snappy-java from 1.1.10.0 to 1.1.10.1 URL: https://github.com/apache/nifi/pull/7383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #7383: NIFI-11697: Bump snappy-java from 1.1.10.0 to 1.1.10.1
mattyb149 commented on PR #7383: URL: https://github.com/apache/nifi/pull/7383#issuecomment-1593612695 +1 LGTM, Github Actions completed successfully indicating no regressions. Adding Jira case to commit and merging to main and support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11697) Update snappy-java to 1.1.10.1
[ https://issues.apache.org/jira/browse/NIFI-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733208#comment-17733208 ] ASF subversion and git services commented on NIFI-11697: Commit dd97bea2ed18684fb271d5cf3f80af1e2e46d4b8 in nifi's branch refs/heads/support/nifi-1.x from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dd97bea2ed ] NIFI-11697: Bump snappy-java from 1.1.10.0 to 1.1.10.1 Bumps [snappy-java](https://github.com/xerial/snappy-java) from 1.1.10.0 to 1.1.10.1. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/v1.1.10.0...v1.1.10.1) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:production ... Signed-off-by: dependabot[bot] Signed-off-by: Matt Burgess > Update snappy-java to 1.1.10.1 > -- > > Key: NIFI-11697 > URL: https://issues.apache.org/jira/browse/NIFI-11697 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > > Update snappy-java to 1.1.10.1 for important fixes, release notes here: > https://github.com/xerial/snappy-java/releases/tag/v1.1.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11697) Update snappy-java to 1.1.10.1
[ https://issues.apache.org/jira/browse/NIFI-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11697: Status: Patch Available (was: In Progress) > Update snappy-java to 1.1.10.1 > -- > > Key: NIFI-11697 > URL: https://issues.apache.org/jira/browse/NIFI-11697 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > > Update snappy-java to 1.1.10.1 for important fixes, release notes here: > https://github.com/xerial/snappy-java/releases/tag/v1.1.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11697) Update snappy-java to 1.1.10.1
[ https://issues.apache.org/jira/browse/NIFI-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-11697: --- Assignee: Matt Burgess > Update snappy-java to 1.1.10.1 > -- > > Key: NIFI-11697 > URL: https://issues.apache.org/jira/browse/NIFI-11697 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > > Update snappy-java to 1.1.10.1 for important fixes, release notes here: > https://github.com/xerial/snappy-java/releases/tag/v1.1.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11697) Update snappy-java to 1.1.10.1
Matt Burgess created NIFI-11697: --- Summary: Update snappy-java to 1.1.10.1 Key: NIFI-11697 URL: https://issues.apache.org/jira/browse/NIFI-11697 Project: Apache NiFi Issue Type: Improvement Reporter: Matt Burgess Fix For: 1.latest, 2.latest Update snappy-java to 1.1.10.1 for important fixes, release notes here: https://github.com/xerial/snappy-java/releases/tag/v1.1.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6435) ConvertAvroToORC Should support Date and Timestamp type
[
https://issues.apache.org/jira/browse/NIFI-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess resolved NIFI-6435.
Resolution: Won't Fix
The version of Avro used in ConvertAvroToORC does not support logical types
such as date and timestamp. The entire Hive 1 NAR has been deprecated and
removed from the Apache NiFi binaries. As you mentioned PutORC is the
replacement, but if the Hive 3 NAR is not available in Apache NiFi binary it
can be downloaded using
https://repository.apache.org/service/local/repositories/releases/content/org/apache/nifi/nifi-hive3-nar/
as the base for you to select the corresponding version and find the NAR
artifact.
> ConvertAvroToORC Should support Date and Timestamp type
> ---
>
> Key: NIFI-6435
> URL: https://issues.apache.org/jira/browse/NIFI-6435
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: archon gum
>Priority: Major
>
> h1. From AVRO
> data
>
> {code:java}
> {
> "date_type": 17897, // 2019-01-01
> "timestamp_type": 154630080 // 2019-01-01 00:00:00 +
> }
> {code}
>
> schema
>
> {code:java}
> {
> "name": "test_types",
> "type": "record",
> "fields": [
> {
>"name": "date_type",
>"type": "int",
>"logicalType": "date"
> },
> {
>"name": "timestamp_type",
>"type": "long",
>"logicalType": "timestamp-millis" // or others
> }
> ]
> }
> {code}
> or schema
>
> {code:java}
> {
> "name": "test_types",
> "type": "record",
> "fields": [
> {
>"name": "date_type",
>"type": {
>"type": "int",
>"logicalType": "date"
>}
> },
> {
>"name": "timestamp_type",
>"type": {
>"type": "long",
>"logicalType": "timestamp-millis" // or others
>}
> }
> ]
> }
> {code}
>
> h1. To ORC
> schema should be
> {code:java}
> struct
> {code}
>
> ---
>
> Update: 2019-07-15
>
> h1. PutORC
> [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/nifi/processors/orc/PutORC.java]
>
> This processor can do the trick.
> *PutORC*: Read from RecordReader(eg JsonReader, AvroReader), convert to ORC
> and put the ORC file to HDFS.
> *PutORC* can convert Avro logical type such like:
> From Avro schema
> {code:java}
> {
> "name": "test_types",
> "type": "record",
> "fields": [
> {
>"name": "date_type",
>"type": {
>"type": "int",
>"logicalType": "date"
>}
> },
> {
>"name": "timestamp_type",
>"type": {
>"type": "long",
>"logicalType": "timestamp-millis"
>}
> }
> ]
> }{code}
> And the result ORC schema would be:
> {code:java}
> struct{code}
>
> h1. Question?
> The *PutORC* still in development? Because *nifi-hive3-processors* module is
> not including in build.(At least in NiFi-1.9.2)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7384: NIFI-11696 Upgrade Bouncy Castle from 1.71 to 1.74
exceptionfactory commented on code in PR #7384:
URL: https://github.com/apache/nifi/pull/7384#discussion_r1231428904
##
nifi-commons/nifi-repository-encryption/src/main/java/org/apache/nifi/repository/encryption/AesGcmByteArrayRepositoryEncryptor.java:
##
@@ -74,9 +73,17 @@ protected byte[] encrypt(final byte[] record, final String
recordId, final Strin
try {
final byte[] encryptedRecord = cipher.doFinal(record);
final byte[] serializedMetadata = getMetadata(keyId,
cipher.getIV(), encryptedRecord.length);
-return Arrays.concatenate(serializedMetadata, encryptedRecord);
+return concatenate(serializedMetadata, encryptedRecord);
Review Comment:
I considered that, but this approach avoids introducing an additional
dependency on commons-lang3 for this library, so that's the reason for the
manual approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-11023) MiNiFi - Delay in command response under heavy load causes command execution issues
[ https://issues.apache.org/jira/browse/NIFI-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan resolved NIFI-11023. Resolution: Fixed > MiNiFi - Delay in command response under heavy load causes command execution > issues > --- > > Key: NIFI-11023 > URL: https://issues.apache.org/jira/browse/NIFI-11023 > Project: Apache NiFi > Issue Type: Improvement > Components: MiNiFi >Reporter: Csaba Bejan >Assignee: Csaba Bejan >Priority: Major > Fix For: 1.20.0 > > Time Spent: 50m > Remaining Estimate: 0h > > When MiNiFi is under heavy load the minifi service response to a command from > bootstrap can be delayed as message written to the socket doesn't arrive at > the bootstrap side. > This is a problem for example when MiNiFi is under heavy load and receives a > shutdown command. The shutdown response is delayed while the shutdown process > is underway. After a while as the response doesn't arrive for bootstrap it > considers the command execution as failed and stops. Meanwhile the long > running bootstrap realises that the minifi process stopped for some reason > and restarts it. So in this case the shutdown command is basically broken as > minifi can't be shut down. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11023) MiNiFi - Delay in command response under heavy load causes command execution issues
[ https://issues.apache.org/jira/browse/NIFI-11023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan updated NIFI-11023: --- Fix Version/s: 1.20.0 > MiNiFi - Delay in command response under heavy load causes command execution > issues > --- > > Key: NIFI-11023 > URL: https://issues.apache.org/jira/browse/NIFI-11023 > Project: Apache NiFi > Issue Type: Improvement > Components: MiNiFi >Reporter: Csaba Bejan >Assignee: Csaba Bejan >Priority: Major > Fix For: 1.20.0 > > Time Spent: 50m > Remaining Estimate: 0h > > When MiNiFi is under heavy load the minifi service response to a command from > bootstrap can be delayed as message written to the socket doesn't arrive at > the bootstrap side. > This is a problem for example when MiNiFi is under heavy load and receives a > shutdown command. The shutdown response is delayed while the shutdown process > is underway. After a while as the response doesn't arrive for bootstrap it > considers the command execution as failed and stops. Meanwhile the long > running bootstrap realises that the minifi process stopped for some reason > and restarts it. So in this case the shutdown command is basically broken as > minifi can't be shut down. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on a diff in pull request #7384: NIFI-11696 Upgrade Bouncy Castle from 1.71 to 1.74
greyp9 commented on code in PR #7384:
URL: https://github.com/apache/nifi/pull/7384#discussion_r1231418270
##
nifi-commons/nifi-repository-encryption/src/main/java/org/apache/nifi/repository/encryption/AesGcmByteArrayRepositoryEncryptor.java:
##
@@ -74,9 +73,17 @@ protected byte[] encrypt(final byte[] record, final String
recordId, final Strin
try {
final byte[] encryptedRecord = cipher.doFinal(record);
final byte[] serializedMetadata = getMetadata(keyId,
cipher.getIV(), encryptedRecord.length);
-return Arrays.concatenate(serializedMetadata, encryptedRecord);
+return concatenate(serializedMetadata, encryptedRecord);
Review Comment:
Maybe use `org.apache.commons.lang3.ArrayUtils.addAll()` here, to avoid the
need for a helper method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #7384: NIFI-11696 Upgrade Bouncy Castle from 1.71 to 1.74
mattyb149 commented on PR #7384: URL: https://github.com/apache/nifi/pull/7384#issuecomment-1593554591 +1 LGTM will merge when Github Actions finishes :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11682) Update keys not quoted in PutDatabaseRecord
[ https://issues.apache.org/jira/browse/NIFI-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11682: Status: Patch Available (was: In Progress) > Update keys not quoted in PutDatabaseRecord > --- > > Key: NIFI-11682 > URL: https://issues.apache.org/jira/browse/NIFI-11682 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > When update keys are used in PutDatabaseRecord (either by declaring them in > the Update Keys property or being auto-determined from the database > metadata), they are not being quoted correctly when 'Quote Column > Identifiers' is set to 'true'. This happens for Upsert, Update, and Insert > Ignore statements. > This was fixed for Upsert in NIFI-8043 but a regression was introduced via > NIFI-9607 when the quoting was removed inadvertently. Testing has revealed > this issue exists for Upsert, Update, and Insert Ignore statements, so should > be fixed for all of them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 opened a new pull request, #7385: NIFI-11682: Quote update keys in PutDatabaseRecord when specified
mattyb149 opened a new pull request, #7385: URL: https://github.com/apache/nifi/pull/7385 # Summary [NIFI-11682](https://issues.apache.org/jira/browse/NIFI-11682) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-11682` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-11682` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-2136) Make base image an argument for all Docker images
[ https://issues.apache.org/jira/browse/MINIFICPP-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2136. - Resolution: Done > Make base image an argument for all Docker images > - > > Key: MINIFICPP-2136 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2136 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > Currently only the alpine based Docker image can be parameterized with a > custom base image, but that should be available for all Docker images. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2121) Investigate if we need the semaphore for intraprocess signaling
[ https://issues.apache.org/jira/browse/MINIFICPP-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2121. - Resolution: Done > Investigate if we need the semaphore for intraprocess signaling > --- > > Key: MINIFICPP-2121 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2121 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2125) Yield time is not always respected
[ https://issues.apache.org/jira/browse/MINIFICPP-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2125. - Resolution: Fixed > Yield time is not always respected > -- > > Key: MINIFICPP-2125 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2125 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > Running a simple EventDriven processor with debug logs enabled reveals that > we are continously checking if the processor has work to do (periodically > busy looping). > > On a quick glance this is due to using nanosec clock and then rounding the > results to milliseconds. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1576: MINIFICPP-2121 - Use std::atomic_flag instead of semaphore
szaszm closed pull request #1576: MINIFICPP-2121 - Use std::atomic_flag instead of semaphore URL: https://github.com/apache/nifi-minifi-cpp/pull/1576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1588: MINIFICPP-2136 Make base image an argument for all Docker images
szaszm closed pull request #1588: MINIFICPP-2136 Make base image an argument for all Docker images URL: https://github.com/apache/nifi-minifi-cpp/pull/1588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11696) Upgrade Bouncy Castle to 1.74
[ https://issues.apache.org/jira/browse/NIFI-11696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11696: Status: Patch Available (was: Open) > Upgrade Bouncy Castle to 1.74 > - > > Key: NIFI-11696 > URL: https://issues.apache.org/jira/browse/NIFI-11696 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Bouncy Castle [1.74|https://www.bouncycastle.org/releasenotes.html#r1rv74] > includes a number of bug fixes and feature improvements over previous > versions. > Bouncy Castle 1.72 and 1.73 included the defunct SIKE algorithm, which added > multiple megabytes to the provider library. Version 1.74 removed this > algorithm, minimizing the size impact of the new version. > Bouncy Castle 1.74 also resolves CVE-2023-33201 related to LDAP certificate > store handling. Apache NiFi does not use the X509LDAPCertStoreSpi class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7384: NIFI-11696 Upgrade Bouncy Castle from 1.71 to 1.74
exceptionfactory opened a new pull request, #7384: URL: https://github.com/apache/nifi/pull/7384 # Summary [NIFI-11696](https://issues.apache.org/jira/browse/NIFI-11696) Upgrades Bouncy Castle dependencies from 1.71 to [1.74](https://www.bouncycastle.org/releasenotes.html#r1rv74). Bouncy Castle 1.74 does not include the defunct SIKE algorithm, which avoids expanding the library size as occurred in versions 1.72 and 1.73. Additional changes include adjusting `nifi-repository-encryption` classes to use `commons-codec` for hexadecimal decoding instead of Bouncy Castle, and removing exclusions and dependencies for Google Cloud Provider libraries that are not necessary in recent versions. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11696) Upgrade Bouncy Castle to 1.74
David Handermann created NIFI-11696: --- Summary: Upgrade Bouncy Castle to 1.74 Key: NIFI-11696 URL: https://issues.apache.org/jira/browse/NIFI-11696 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 1.latest, 2.latest Bouncy Castle [1.74|https://www.bouncycastle.org/releasenotes.html#r1rv74] includes a number of bug fixes and feature improvements over previous versions. Bouncy Castle 1.72 and 1.73 included the defunct SIKE algorithm, which added multiple megabytes to the provider library. Version 1.74 removed this algorithm, minimizing the size impact of the new version. Bouncy Castle 1.74 also resolves CVE-2023-33201 related to LDAP certificate store handling. Apache NiFi does not use the X509LDAPCertStoreSpi class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dependabot[bot] opened a new pull request, #7383: Bump snappy-java from 1.1.10.0 to 1.1.10.1
dependabot[bot] opened a new pull request, #7383: URL: https://github.com/apache/nifi/pull/7383 Bumps [snappy-java](https://github.com/xerial/snappy-java) from 1.1.10.0 to 1.1.10.1. Release notes Sourced from https://github.com/xerial/snappy-java/releases";>snappy-java's releases. v1.1.10.1 What's Changed 🐛 Bug Fixes Fixed several vulnerabilities by https://github.com/aidanchiu1112";>@aidanchiu1112: CVE-2023-34453 https://github.com/xerial/snappy-java/security/advisories/GHSA-pqr6-cmr2-h8hf";>Integer overflow in shuffle CVE-2023-34454 https://github.com/xerial/snappy-java/security/advisories/GHSA-fjpj-2g6w-x25r";>Integer overflow in compress CVE-2023-34455 https://github.com/xerial/snappy-java/security/advisories/GHSA-qcwq-55hx-v3vh";>Unchecked chunk length internal: Fix commit message by https://github.com/xerial";>@xerial in https://redirect.github.com/xerial/snappy-java/pull/447";>xerial/snappy-java#447 internal: Fix CI target branch by https://github.com/xerial";>@xerial in https://redirect.github.com/xerial/snappy-java/pull/449";>xerial/snappy-java#449 Fix typo by https://github.com/aidanchiu1112";>@aidanchiu1112 in https://redirect.github.com/xerial/snappy-java/pull/457";>xerial/snappy-java#457 CI Fix to Prevent Checks Dealing with Large Array Sizes by https://github.com/aidanchiu1112";>@aidanchiu1112 in https://redirect.github.com/xerial/snappy-java/pull/459";>xerial/snappy-java#459 🔗 Dependency Updates Update native libraries by https://github.com/github-actions";>@github-actions in https://redirect.github.com/xerial/snappy-java/pull/445";>xerial/snappy-java#445 Update native libraries by https://github.com/github-actions";>@github-actions in https://redirect.github.com/xerial/snappy-java/pull/450";>xerial/snappy-java#450 Update scalafmt-core to 3.7.4 by https://github.com/xerial-bot";>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/454";>xerial/snappy-java#454 Update sbt to 1.9.0 by https://github.com/xerial-bot";>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/455";>xerial/snappy-java#455 🛠 Internal Updates Trigger native lib build on PR by https://github.com/imsudiproy";>@imsudiproy in https://redirect.github.com/xerial/snappy-java/pull/444";>xerial/snappy-java#444 internal: Run CI tests on native file change by https://github.com/xerial";>@xerial in https://redirect.github.com/xerial/snappy-java/pull/446";>xerial/snappy-java#446 internal: Run CI tests for update-native-libs branch by https://github.com/xerial";>@xerial in https://redirect.github.com/xerial/snappy-java/pull/448";>xerial/snappy-java#448 intertnal: Fix CI watch target files by https://github.com/xerial";>@xerial in https://redirect.github.com/xerial/snappy-java/pull/451";>xerial/snappy-java#451 Update airframe-log to 23.5.6 by https://github.com/xerial-bot";>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/453";>xerial/snappy-java#453 New Contributors https://github.com/imsudiproy";>@imsudiproy made their first contribution in https://redirect.github.com/xerial/snappy-java/pull/444";>xerial/snappy-java#444 https://github.com/github-actions";>@github-actions made their first contribution in https://redirect.github.com/xerial/snappy-java/pull/445";>xerial/snappy-java#445 https://github.com/aidanchiu1112";>@aidanchiu1112 made their first contribution in https://redirect.github.com/xerial/snappy-java/pull/457";>xerial/snappy-java#457 Full Changelog: https://github.com/xerial/snappy-java/compare/v1.1.10.0...v1.1.10.1";>https://github.com/xerial/snappy-java/compare/v1.1.10.0...v1.1.10.1 Commits https://github.com/xerial/snappy-java/commit/737f3973ff392c351da2744820a23a0521c6cc77";>737f397 CI Fix to Prevent Checks Dealing with Large Array Sizes (https://redirect.github.com/xerial/snappy-java/issues/459";>#459) https://github.com/xerial/snappy-java/commit/d0042551e4a3509a725038eb9b2ad1f683674d94";>d004255 Merge pull request from GHSA-fjpj-2g6w-x25r https://github.com/xerial/snappy-java/commit/3bf67857fcf70d9eea56eed4af7c925671e8eaea";>3bf6785 Merge pull request from GHSA-qcwq-55hx-v3vh https://github.com/xerial/snappy-java/commit/820e2e074c58748b41dbd547f4edba9e108ad905";>820e2e0 Merge pull request from GHSA-pqr6-cmr2-h8hf https://github.com/xerial/snappy-java/commit/27e2ce0fb09c92d2f8948626e32a0655b2b4ddc0";>27e2ce0 Fix typo (https://redirect.github.com/xerial/snappy-java/issues/457";>#457) https://github.com/xerial/snappy-java/commit/f32d5b0d7c1abca6b76249ecff80a2dcc1a1b415";>f32d5b0 Update sbt to 1.9.0 (https://redirect.github.com/xerial/snappy-java/issues/455";>#455) https://github.com/xerial/snappy-java/commit/326f0b45ee8b53eb4163fd8d639b4da0b0b6772d";>326f0b4 Update scalafmt-core to 3.7.4 (https://redirect.github.com/xerial/snappy-java/issues/454";>#454)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1540: MINIFICPP-2082 Move RocksDB stats to RepositoryMetrics
szaszm commented on code in PR #1540:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1540#discussion_r1159265202
##
extensions/rocksdb-repos/DatabaseContentRepository.cpp:
##
@@ -286,6 +286,32 @@ uint64_t
DatabaseContentRepository::getRepositoryEntryCount() const {
})).value_or(0);
}
+std::optional
DatabaseContentRepository::getRocksDbStats() const {
+ RocksDbStats stats;
+ auto opendb = db_->open();
+ if (!opendb) {
+return stats;
+ }
+
+ std::string table_readers;
+ opendb->GetProperty("rocksdb.estimate-table-readers-mem", &table_readers);
+ try {
+stats.table_readers_size = std::stoull(table_readers);
+ } catch (const std::exception&) {
+logger_->log_error("Could not retrieve valid
'rocksdb.estimate-table-readers-mem' property value from rocksdb content
repository!");
Review Comment:
A warning may be enough, since this doesn't prevent normal operation of the
agent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1577: MINIFICPP-2020 - Protect MINIFI_HOME from mutual access
szaszm commented on code in PR #1577:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1577#discussion_r1231227171
##
libminifi/include/utils/FileMutex.h:
##
@@ -0,0 +1,58 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include "utils/gsl.h"
+
+#ifdef WIN32
+#include
+#endif
+
+namespace org::apache::nifi::minifi::utils {
+
+class FileMutex {
+ public:
+ explicit FileMutex(std::filesystem::path path);
+ ~FileMutex() {
+gsl_Expects(!file_handle_.has_value());
+ }
+
+ FileMutex(const FileMutex&) = delete;
+ FileMutex(FileMutex&&) = delete;
+ FileMutex& operator=(const FileMutex&) = delete;
+ FileMutex& operator=(FileMutex&&) = delete;
+
+ void lock();
+ void unlock();
+
+ private:
+ std::filesystem::path path_;
+
+ std::mutex mtx_;
+#ifdef WIN32
+ std::optional file_handle_;
+#else
+ std::optional file_handle_;
+#endif
Review Comment:
Could we make it a pidfile, and print the PID of the existing other process,
for a more helpful error message?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1586: MINIFICPP-2127 Add multipart upload support for PutS3Object processor
szaszm commented on code in PR #1586:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1586#discussion_r1231135508
##
extensions/aws/tests/MultipartUploadStateStorageTest.cpp:
##
@@ -0,0 +1,141 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "TestBase.h"
+#include "Catch.h"
+#include "s3/MultipartUploadStateStorage.h"
+
+namespace org::apache::nifi::minifi::test {
+
+class MultipartUploadStateStorageTestFixture {
+ public:
+ MultipartUploadStateStorageTestFixture()
+: upload_storage_(test_controller.createTempDirectory().string(),
"test_id") {
+ }
+
+ protected:
+ TestController test_controller;
+ minifi::aws::s3::MultipartUploadStateStorage upload_storage_;
+};
+
+TEST_CASE_METHOD(MultipartUploadStateStorageTestFixture, "Store and get
current key state", "[s3StateStorage]") {
+ REQUIRE(upload_storage_.getState("test_bucket", "key") == std::nullopt);
+ minifi::aws::s3::MultipartUploadState state;
+ state.upload_id = "id1";
+ state.uploaded_parts = 2;
+ state.uploaded_size = 100_MiB;
+ state.part_size = 50_MiB;
+ state.full_size = 200_MiB;
+ state.upload_time = Aws::Utils::DateTime::CurrentTimeMillis();
+ state.uploaded_etags = {"etag1", "etag2"};
+ upload_storage_.storeState("test_bucket", "key", state);
+ REQUIRE(*upload_storage_.getState("test_bucket", "key") == state);
+}
+
+TEST_CASE_METHOD(MultipartUploadStateStorageTestFixture, "Get key upload state
from multiple keys and buckets", "[s3StateStorage]") {
+ minifi::aws::s3::MultipartUploadState state1;
+ state1.upload_id = "id1";
+ state1.uploaded_parts = 3;
+ state1.uploaded_size = 150_MiB;
+ state1.part_size = 50_MiB;
+ state1.full_size = 200_MiB;
+ state1.upload_time = Aws::Utils::DateTime::CurrentTimeMillis();
+ state1.uploaded_etags = {"etag1", "etag2", "etag3"};
+ upload_storage_.storeState("old_bucket", "key1", state1);
Review Comment:
Why did you use `old_bucket` here? Isn't this meant to test that storing
states under the same bucket and key overwrites the old one?
##
extensions/aws/processors/PutS3Object.cpp:
##
@@ -77,7 +78,31 @@ void PutS3Object::onSchedule(const
std::shared_ptr &contex
use_virtual_addressing_ = !*use_path_style_access;
}
+ context->getProperty(MultipartThreshold.getName(), multipart_threshold_);
+ if (multipart_threshold_ > getMaxUploadSize() || multipart_threshold_ <
getMinPartSize()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Multipart Threshold is not
between the valid 5MB and 5GB range!");
+ }
+ logger_->log_debug("PutS3Object: Multipart Threshold %" PRIu64,
multipart_threshold_);
+ context->getProperty(MultipartPartSize.getName(), multipart_size_);
+ if (multipart_size_ > getMaxUploadSize() || multipart_size_ <
getMinPartSize()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Multipart Part Size is not
between the valid 5MB and 5GB range!");
+ }
+ logger_->log_debug("PutS3Object: Multipart Size %" PRIu64, multipart_size_);
+
+
+ multipart_upload_ageoff_interval_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadAgeOffInterval.getName()).getMilliseconds();
+ logger_->log_debug("PutS3Object: Multipart Upload Ageoff Interval %" PRIu64
" ms", multipart_upload_ageoff_interval_.count());
+
+ multipart_upload_max_age_threshold_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadMaxAgeThreshold.getName()).getMilliseconds();
+ logger_->log_debug("PutS3Object: Multipart Upload Max Age Threshold %"
PRIu64 " ms", multipart_upload_max_age_threshold_.count());
+
Review Comment:
We should wrap in int64_t and use PRId64. See tick count representation of
standard types here: https://en.cppreference.com/w/cpp/chrono/duration
```suggestion
logger_->log_debug("PutS3Object: Multipart Upload Ageoff Interval %"
PRId64 " ms", int64_t{multipart_upload_ageoff_interval_.count()});
multipart_upload_max_age_threshold_ =
minifi::utils::getRequiredPropertyOrThrow(*context,
MultipartUploadMaxAgeThreshold.getName()).getMilliseconds();
logger_->log_debug("PutS3Object: Multipart Upload Max Age Threshold %"
PRId64 " ms", int64_t{multipart_upload_max_age_threshold_.count()});
```
##
[jira] [Commented] (NIFI-10948) QueryElasticsearchHTTP doesnt work on Opensearch (and possibly newer Elasticsearch versions)
[
https://issues.apache.org/jira/browse/NIFI-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733130#comment-17733130
]
Chris Sampson commented on NIFI-10948:
--
I would probably suggest a feature request ticket to create an OpenSearch
compatible Controller Service, i.e. an alternative implementaion of the
{{ElasticsearchClientService}} API interface to the existing
{{ElasticsearchClientServiceImpl}} that connects to Elastic-based Elasticsearch
instances for the REST-based Elasticsearch processors.
It would hopefully be sufficient to retain the same processors but allow for
different connections (controller services) to OpenSearch vs. Elasticsearch.
There could, of course, be instances where functionality is only available in
one service and not the other where service-specific processors would be
needed, but the basic set of Put and Get/Search processors would hopefully be
fine with just different Controller Services configured.
> QueryElasticsearchHTTP doesnt work on Opensearch (and possibly newer
> Elasticsearch versions)
>
>
> Key: NIFI-10948
> URL: https://issues.apache.org/jira/browse/NIFI-10948
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.15.3
> Environment: Nifi 1.15.3 on centos machine
>Reporter: David Vassallo
>Priority: Minor
> Attachments: image-2022-12-05-16-34-20-513.png
>
>
> The QueryElasticsearchHTTP processor will return a "Null Pointer Exception"
> when running agasint Opensearch and probably newer versions of Elasticsearch.
> I tracked down the null pointer to this line:
>
> [https://github.com/apache/nifi/blob/0ebc6d31489e975dcbbe078fa572332ef8ffa9e2/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/QueryElasticsearchHttp.java#L418]
>
> The processor is looking for the "_type" field which is no longer in the
> right place. In opensearch v 1.x and above (possibly Elasticsearch too, but I
> dont have the means to test this), the field has moved to be a child of the
> "@metadata" field:
> !image-2022-12-05-16-34-20-513.png!
>
> If you try to circumvent this issue by populating the "Type" property of the
> processor, you will get an HTTP 400 Bad Message response since the processor
> builds a URL with the "size" and "from" arguments as part of the URL which
> are not supported by Opensearch/Elasticsearch in more recent versions, so you
> are forced to leave this property empty but then run into the null problem
> described above
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194: URL: https://github.com/apache/nifi/pull/7194#discussion_r1231194122 ## nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/pom.xml: ## @@ -75,6 +75,11 @@ com.fasterxml.jackson.dataformat jackson-dataformat-csv + +com.github.pjfanning +excel-streaming-reader Review Comment: This dependency can be removed now that the ExcelReader is in a different module. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11690) Propagate SupportSensitiveDynamicProperties Information to C2 Component Manifest
[ https://issues.apache.org/jira/browse/NIFI-11690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-11690: --- Fix Version/s: 2.0.0 1.23.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Propagate SupportSensitiveDynamicProperties Information to C2 Component > Manifest > > > Key: NIFI-11690 > URL: https://issues.apache.org/jira/browse/NIFI-11690 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Support for component sensitive dynamic properties was recently added via a > new annotation [1]. This information is available in > "extension-manifest.xml", and should also be made available in C2 component > manifest. > [1] > nifi-api/src/main/java/org/apache/nifi/annotation/behavior/SupportsSensitiveDynamicProperties.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11690) Propagate SupportSensitiveDynamicProperties Information to C2 Component Manifest
[ https://issues.apache.org/jira/browse/NIFI-11690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733127#comment-17733127 ] ASF subversion and git services commented on NIFI-11690: Commit b6d4099a1499caa6d1cc577a0ea88d97789276e2 in nifi's branch refs/heads/support/nifi-1.x from greyp9 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b6d4099a14 ] NIFI-11690 - include extension info for SupportsSensitiveDynamicPrope… (#7379) * NIFI-11690 - include extension info for SupportsSensitiveDynamicProperties in C2 manifest * NIFI-11690 - include extension info for SupportsSensitiveDynamicProperties in C2 manifest * NIFI-11690 - include extension info for SupportsSensitiveDynamicProperties in C2 manifest > Propagate SupportSensitiveDynamicProperties Information to C2 Component > Manifest > > > Key: NIFI-11690 > URL: https://issues.apache.org/jira/browse/NIFI-11690 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Time Spent: 1h > Remaining Estimate: 0h > > Support for component sensitive dynamic properties was recently added via a > new annotation [1]. This information is available in > "extension-manifest.xml", and should also be made available in C2 component > manifest. > [1] > nifi-api/src/main/java/org/apache/nifi/annotation/behavior/SupportsSensitiveDynamicProperties.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11690) Propagate SupportSensitiveDynamicProperties Information to C2 Component Manifest
[ https://issues.apache.org/jira/browse/NIFI-11690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733122#comment-17733122 ] ASF subversion and git services commented on NIFI-11690: Commit e4ad7574e716b5e09ebdc3f7a17f1db91d7534b7 in nifi's branch refs/heads/main from greyp9 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e4ad7574e7 ] NIFI-11690 - include extension info for SupportsSensitiveDynamicPrope… (#7379) * NIFI-11690 - include extension info for SupportsSensitiveDynamicProperties in C2 manifest * NIFI-11690 - include extension info for SupportsSensitiveDynamicProperties in C2 manifest * NIFI-11690 - include extension info for SupportsSensitiveDynamicProperties in C2 manifest > Propagate SupportSensitiveDynamicProperties Information to C2 Component > Manifest > > > Key: NIFI-11690 > URL: https://issues.apache.org/jira/browse/NIFI-11690 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Assignee: Paul Grey >Priority: Minor > Time Spent: 1h > Remaining Estimate: 0h > > Support for component sensitive dynamic properties was recently added via a > new annotation [1]. This information is available in > "extension-manifest.xml", and should also be made available in C2 component > manifest. > [1] > nifi-api/src/main/java/org/apache/nifi/annotation/behavior/SupportsSensitiveDynamicProperties.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] bbende merged pull request #7379: NIFI-11690 - include extension info for SupportsSensitiveDynamicPrope…
bbende merged PR #7379: URL: https://github.com/apache/nifi/pull/7379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bbende commented on pull request #7379: NIFI-11690 - include extension info for SupportsSensitiveDynamicPrope…
bbende commented on PR #7379: URL: https://github.com/apache/nifi/pull/7379#issuecomment-1593290321 +1 looks good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1545: MINIFICPP-55 Implement test coverage report generation
szaszm commented on code in PR #1545: URL: https://github.com/apache/nifi-minifi-cpp/pull/1545#discussion_r1231163899 ## .github/workflows/ci.yml: ## @@ -156,6 +159,14 @@ jobs: run: cd build && make shellcheck - id: flake8_check run: cd build && make flake8 + - id: coverage Review Comment: My bad, I misread the condition to be OR instead of AND. That's fine then, sorry for the confusion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10948) QueryElasticsearchHTTP doesnt work on Opensearch (and possibly newer Elasticsearch versions)
[ https://issues.apache.org/jira/browse/NIFI-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733085#comment-17733085 ] David Tippett commented on NIFI-10948: -- I can confirm that `_type` has been deprecated in the 2.x.x versions of OpenSearch. I am wondering if we should turn this minor issue into a feature request for a set of OpenSearch processors? OpenSearch is sticking to semantic versioning but will not be compatible with Elasticsearch long term and this is just the first canary. > QueryElasticsearchHTTP doesnt work on Opensearch (and possibly newer > Elasticsearch versions) > > > Key: NIFI-10948 > URL: https://issues.apache.org/jira/browse/NIFI-10948 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.15.3 > Environment: Nifi 1.15.3 on centos machine >Reporter: David Vassallo >Priority: Minor > Attachments: image-2022-12-05-16-34-20-513.png > > > The QueryElasticsearchHTTP processor will return a "Null Pointer Exception" > when running agasint Opensearch and probably newer versions of Elasticsearch. > I tracked down the null pointer to this line: > > [https://github.com/apache/nifi/blob/0ebc6d31489e975dcbbe078fa572332ef8ffa9e2/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/QueryElasticsearchHttp.java#L418] > > The processor is looking for the "_type" field which is no longer in the > right place. In opensearch v 1.x and above (possibly Elasticsearch too, but I > dont have the means to test this), the field has moved to be a child of the > "@metadata" field: > !image-2022-12-05-16-34-20-513.png! > > If you try to circumvent this issue by populating the "Type" property of the > processor, you will get an HTTP 400 Bad Message response since the processor > builds a URL with the "size" and "from" arguments as part of the URL which > are not supported by Opensearch/Elasticsearch in more recent versions, so you > are forced to leave this property empty but then run into the null problem > described above -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10784) implement a processor of Apache IoTDB Query
[ https://issues.apache.org/jira/browse/NIFI-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733069#comment-17733069 ] David Handermann commented on NIFI-10784: - Backported to support branch for subsequent 1.23.0 release version. > implement a processor of Apache IoTDB Query > - > > Key: NIFI-10784 > URL: https://issues.apache.org/jira/browse/NIFI-10784 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Zhizhou.Li >Assignee: Zhizhou.Li >Priority: Major > Fix For: 2.0.0, 1.23.0 > > Original Estimate: 168h > Time Spent: 1h 40m > Remaining Estimate: 166h 20m > > The PutIoTDBRecord process is in NIFI-10234. > This issue is the QueryIoTDBRecord process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10784) implement a processor of Apache IoTDB Query
[ https://issues.apache.org/jira/browse/NIFI-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733067#comment-17733067 ] ASF subversion and git services commented on NIFI-10784: Commit 105fcb90e261894711ea1cf2bd089cf96f29facf in nifi's branch refs/heads/support/nifi-1.x from lizhizhou [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=105fcb90e2 ] NIFI-10784 Added QueryIoTDBRecord Processor This closes #6844 Co-authored-by: David Handermann Signed-off-by: David Handermann (cherry picked from commit 7cb86dd42d52923bae57e2dc0763799f68f5) > implement a processor of Apache IoTDB Query > - > > Key: NIFI-10784 > URL: https://issues.apache.org/jira/browse/NIFI-10784 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Zhizhou.Li >Assignee: Zhizhou.Li >Priority: Major > Fix For: 2.0.0 > > Original Estimate: 168h > Time Spent: 1h 40m > Remaining Estimate: 166h 20m > > The PutIoTDBRecord process is in NIFI-10234. > This issue is the QueryIoTDBRecord process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11635) Upgrade IoTDB to 1.1.1
[ https://issues.apache.org/jira/browse/NIFI-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11635: Fix Version/s: 1.23.0 > Upgrade IoTDB to 1.1.1 > -- > > Key: NIFI-11635 > URL: https://issues.apache.org/jira/browse/NIFI-11635 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0, 1.23.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Update Apache IoTDB to > [1.1.1|https://dlcdn.apache.org/iotdb/1.1.1/RELEASE_NOTES.md] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10784) implement a processor of Apache IoTDB Query
[ https://issues.apache.org/jira/browse/NIFI-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10784: Fix Version/s: 1.23.0 > implement a processor of Apache IoTDB Query > - > > Key: NIFI-10784 > URL: https://issues.apache.org/jira/browse/NIFI-10784 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Zhizhou.Li >Assignee: Zhizhou.Li >Priority: Major > Fix For: 2.0.0, 1.23.0 > > Original Estimate: 168h > Time Spent: 1h 40m > Remaining Estimate: 166h 20m > > The PutIoTDBRecord process is in NIFI-10234. > This issue is the QueryIoTDBRecord process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11635) Upgrade IoTDB to 1.1.1
[ https://issues.apache.org/jira/browse/NIFI-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733068#comment-17733068 ] ASF subversion and git services commented on NIFI-11635: Commit c21c0e27ebb91d6f49a1dfea079d778b0c6e359c in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c21c0e27eb ] NIFI-11635 Upgraded IoTDB from 1.0.1 to 1.1.1 - Excluded transitive test dependencies to avoid SNAPSHOT versions Signed-off-by: Pierre Villard This closes #7382. (cherry picked from commit 4638d2e2515e1c6da99947c451dfcd15abbf601f) > Upgrade IoTDB to 1.1.1 > -- > > Key: NIFI-11635 > URL: https://issues.apache.org/jira/browse/NIFI-11635 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Update Apache IoTDB to > [1.1.1|https://dlcdn.apache.org/iotdb/1.1.1/RELEASE_NOTES.md] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1545: MINIFICPP-55 Implement test coverage report generation
szaszm commented on code in PR #1545: URL: https://github.com/apache/nifi-minifi-cpp/pull/1545#discussion_r1231019414 ## cmake/CodeCoverage.cmake: ## @@ -0,0 +1,748 @@ +# Copyright (c) 2012 - 2017, Lars Bilke +# All rights reserved. +# Review Comment: We need LICENSE+NOTICE changes for this. Please check which license is below, and add its name to LICENSE as well. ## .github/workflows/ci.yml: ## @@ -156,6 +159,14 @@ jobs: run: cd build && make shellcheck - id: flake8_check run: cd build && make flake8 + - id: coverage Review Comment: The condition seems no different from all of our typical runs, so it may be better to just omit it. That is if the build time overhead is low enough. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (MINIFICPP-2130) Custom cache eviction strategy for GitHub Actions
[ https://issues.apache.org/jira/browse/MINIFICPP-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi reassigned MINIFICPP-2130: Assignee: Gábor Gyimesi > Custom cache eviction strategy for GitHub Actions > - > > Key: MINIFICPP-2130 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2130 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Gábor Gyimesi >Priority: Major > > GitHub Actions caches have a 10 GB storage limit for the repository, and it > evicts older caches automatically. Because the caches are immutable, we're > forced to create a new, separate cache key for each run. We need a cleanup > job that cleans up older than latest cache entries for each branch, remove > caches for closed pull requests, and if we're near the limit, start removing > non-main / pull request caches, since pull requests can fall back to the main > branch cache. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] timeabarna commented on pull request #7315: NIFI-3065 Per Process Group logging
timeabarna commented on PR #7315: URL: https://github.com/apache/nifi/pull/7315#issuecomment-1592917330 Hello @markap14, Thanks for your review, I've modified the PR based on your recommendations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11635) Upgrade IoTDB to 1.1.1
[ https://issues.apache.org/jira/browse/NIFI-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-11635: -- Fix Version/s: 2.0.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade IoTDB to 1.1.1 > -- > > Key: NIFI-11635 > URL: https://issues.apache.org/jira/browse/NIFI-11635 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Update Apache IoTDB to > [1.1.1|https://dlcdn.apache.org/iotdb/1.1.1/RELEASE_NOTES.md] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11635) Upgrade IoTDB to 1.1.1
[ https://issues.apache.org/jira/browse/NIFI-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733013#comment-17733013 ] ASF subversion and git services commented on NIFI-11635: Commit 4638d2e2515e1c6da99947c451dfcd15abbf601f in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4638d2e251 ] NIFI-11635 Upgraded IoTDB from 1.0.1 to 1.1.1 - Excluded transitive test dependencies to avoid SNAPSHOT versions Signed-off-by: Pierre Villard This closes #7382. > Upgrade IoTDB to 1.1.1 > -- > > Key: NIFI-11635 > URL: https://issues.apache.org/jira/browse/NIFI-11635 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Fix For: 1.latest, 2.latest > > Time Spent: 40m > Remaining Estimate: 0h > > Update Apache IoTDB to > [1.1.1|https://dlcdn.apache.org/iotdb/1.1.1/RELEASE_NOTES.md] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11695) ChangecaptureMysql Support for encrypted binlogs
Santhosh created NIFI-11695: --- Summary: ChangecaptureMysql Support for encrypted binlogs Key: NIFI-11695 URL: https://issues.apache.org/jira/browse/NIFI-11695 Project: Apache NiFi Issue Type: Improvement Reporter: Santhosh ChangecaptureMysql reads mysql binlogs and make them available via flowfiles to downstream processors. Starting from Mysql 8.0, Mysql can encrypt Binary logs. This is to request for support of Nifi to have a mechanism to be read encrypted Binary logs in Mysql instance (8.0) through ChangeCaptureMysql processor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1583: MINIFICPP-1719 Replace LibreSSL with OpenSSL 3.1
lordgamez commented on code in PR #1583:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1583#discussion_r1230573291
##
README.md:
##
@@ -151,6 +151,10 @@ and rebuild.
* Lua and development headers -- Required if Lua support is enabled
* libgps-dev -- Required if building libGPS support
* Zlib headers
+* perl -- Required for OpenSSL configuration
+* NASM -- Required for OpenSSL only on Windows
+
+**NOTE** On Windows if Strawberry Perl is used the
`${StrawberryPerlRoot}\c\bin` directory should not be part of the %PATH%
variable as Strawberry Perl's patch.exe will be found as the patch executable
in the configure phase instead if the git patch executable.
Review Comment:
Updated in 1a573202fd6a65332b4f7d8e8f3b6b3ecac2673a
##
NOTICE:
##
Review Comment:
Updated in 1a573202fd6a65332b4f7d8e8f3b6b3ecac2673a
##
cmake/GoogleCloudCpp.cmake:
##
@@ -40,3 +40,9 @@ FetchContent_MakeAvailable(google-cloud-cpp)
if (CMAKE_CXX_COMPILER_ID STREQUAL "Clang" AND CMAKE_CXX_COMPILER_VERSION
VERSION_GREATER_EQUAL "14.0.0" )
target_compile_options(google_cloud_cpp_common PUBLIC
-Wno-error=deprecated-pragma)
endif()
+
+if (WIN32)
+target_compile_options(google_cloud_cpp_storage PUBLIC /wd4996)
+else()
+target_compile_options(google_cloud_cpp_storage PUBLIC
-Wno-error=deprecated-declarations)
Review Comment:
Added in 1a573202fd6a65332b4f7d8e8f3b6b3ecac2673a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
