[jira] [Resolved] (NIFI-12274) Evaluate Apache Infrastructure Hosted GitHub Runners
[ https://issues.apache.org/jira/browse/NIFI-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-12274. - Resolution: Later > Evaluate Apache Infrastructure Hosted GitHub Runners > > > Key: NIFI-12274 > URL: https://issues.apache.org/jira/browse/NIFI-12274 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The Apache Infrastructure Team has begun testing self-hosted runners for > GitHub Action workflows that provide better performance than standard runners. > The self-hosted runners require specific run configuration settings that > apply to workflows within the context of the project repository, as opposed > to forks. The workflow configuration should support targeted execution on > self-hosted runners for testing and comparison of results. > https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=INFRA&title=ASF+Infra+provided+self-hosted+runners -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-5851) Add MFA to *S3Object Processors
[ https://issues.apache.org/jira/browse/NIFI-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-5851. Resolution: Won't Do > Add MFA to *S3Object Processors > --- > > Key: NIFI-5851 > URL: https://issues.apache.org/jira/browse/NIFI-5851 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Lukas Heusser >Priority: Major > Labels: features, security > > Add the possibility to add MFA to the *S3Object processors. Due to security > this is a quiet important feature for customers. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-3723) Provide standalone tool to perform provenance repository encryption translation
[
https://issues.apache.org/jira/browse/NIFI-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-3723.
Resolution: Won't Do
> Provide standalone tool to perform provenance repository encryption
> translation
> ---
>
> Key: NIFI-3723
> URL: https://issues.apache.org/jira/browse/NIFI-3723
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Tools and Build
>Affects Versions: 1.2.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: encryption, provenance, security
>
> The NiFi Toolkit should provide a standalone tool to handle translation of
> the provenance repository from encrypted -> unencrypted or vice-versa .
> * Unencrypted -> encrypted: Attempt to create a
> {{EventIdFirstSchemaRecordReader}} instance to read the existing events into
> memory and write them back using {{EncryptedSchemaRecordWriter}}
> * Encrypted -> unencrypted: Attempt to create a
> {{EncryptedSchemaRecordReader}} instance to read the existing events into
> memory and write them back using {{EventIdFirstSchemaRecordWriter}} or
> {{ByteArraySchemaRecordWriter}} depending on the repository implementation
> class. This depends on the key(s) for the key IDs used still being available
> via {{nifi.properties}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-4868) Unable to initialise `HBase_1_1_2_ClientService `
[
https://issues.apache.org/jira/browse/NIFI-4868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-4868.
Resolution: Won't Fix
> Unable to initialise `HBase_1_1_2_ClientService `
> --
>
> Key: NIFI-4868
> URL: https://issues.apache.org/jira/browse/NIFI-4868
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Tools and Build
>Affects Versions: 1.1.1, 1.3.0, 1.4.0
>Reporter: Naveen Nain
>Priority: Major
> Labels: security
>
> I'm creating a HBASE connection controller service to connect to secured
> HBASE. I have defined the kerbrose priclpal and keytab. But I'm enabling the
> controller service, It's giving me this error:
> {quote}2018-02-12 14:16:04,791 INFO [StandardProcessScheduler Thread-4]
> o.a.nifi.hbase.HBase_1_1_2_ClientService
> HBase_1_1_2_ClientService[id=88ee1856-0161-1000-019f-35d668a0300e] HBase
> Security Enabled, logging in as principal hbase/m111.server.e...@embs.com
> with keytab /Users/naveen/Downloads/nifi-1.0.0/conf/hbase.keytab
> 2018-02-12 14:16:05,388 INFO [StandardProcessScheduler Thread-4]
> o.a.nifi.hbase.HBase_1_1_2_ClientService
> HBase_1_1_2_ClientService[id=88ee1856-0161-1000-019f-35d668a0300e]
> Successfully logged in as principal hbase/* with keytab
> /Users/naveen/Downloads/nifi-1.0.0/conf/hbase.keytab
> 2018-02-12 14:16:05,389 INFO [StandardProcessScheduler Thread-4]
> o.a.h.h.zookeeper.RecoverableZooKeeper Process
> identifier=hconnection-0x6d8fedd8 connecting to ZooKeeper
> ensemble=*:2181,**:2181,***:2181
> 2018-02-12 14:16:08,073 ERROR [StandardProcessScheduler Thread-4]
> o.a.n.c.s.StandardControllerServiceNode
> HBase_1_1_2_ClientService[id=88ee1856-0161-1000-019f-35d668a0300e] Failed to
> invoke @OnEnabled method due to java.io.IOException:
> java.lang.reflect.InvocationTargetException
> 2018-02-12 14:16:08,075 ERROR [StandardProcessScheduler Thread-4]
> o.a.n.c.s.StandardControllerServiceNode
> java.io.IOException: java.lang.reflect.InvocationTargetException
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
> ~[hbase-client-1.1.2.jar:1.1.2]
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> ~[hbase-client-1.1.2.jar:1.1.2]
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> ~[hbase-client-1.1.2.jar:1.1.2]
> at
> org.apache.nifi.hbase.HBase_1_1_2_ClientService$1.run(HBase_1_1_2_ClientService.java:232)
> ~[nifi-hbase_1_1_2-client-service-1.0.0.jar:1.0.0]
> at
> org.apache.nifi.hbase.HBase_1_1_2_ClientService$1.run(HBase_1_1_2_ClientService.java:229)
> ~[nifi-hbase_1_1_2-client-service-1.0.0.jar:1.0.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.8.0_141]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_141]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> ~[hadoop-common-2.6.2.jar:na]
> at
> org.apache.nifi.hbase.HBase_1_1_2_ClientService.createConnection(HBase_1_1_2_ClientService.java:229)
> ~[nifi-hbase_1_1_2-client-service-1.0.0.jar:1.0.0]
> at
> org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:178)
> ~[nifi-hbase_1_1_2-client-service-1.0.0.jar:1.0.0]
> at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source)
> ~[na:na]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_141]
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
> ~[na:na]
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
> ~[na:na]
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
> ~[na:na]
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
> ~[na:na]
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:348)
> ~[na:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_141]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_141]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_141]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.8.0_141
[jira] [Resolved] (NIFI-7120) Improve secure hasher architecture
[
https://issues.apache.org/jira/browse/NIFI-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-7120.
Assignee: (was: Andy LoPresto)
Resolution: Won't Do
> Improve secure hasher architecture
> --
>
> Key: NIFI-7120
> URL: https://issues.apache.org/jira/browse/NIFI-7120
> Project: Apache NiFi
> Issue Type: Epic
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Priority: Major
> Labels: architecture, hashing, refactor, security
>
> In fixing NIFI-7079, the {{Argon2SecureHasher}} was introduced. The concept
> of a secure hasher and the varying implementations should be coordinated for
> cleaner reuse and structure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-7125) Extend SecureHasher interface to provide implementation-specific "full hash output" string representation
[
https://issues.apache.org/jira/browse/NIFI-7125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-7125.
Resolution: Won't Do
> Extend SecureHasher interface to provide implementation-specific "full hash
> output" string representation
> -
>
> Key: NIFI-7125
> URL: https://issues.apache.org/jira/browse/NIFI-7125
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Security
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Priority: Major
> Labels: hashing, security
>
> The current {{SecureHasher}} interface only provides methods which output the
> "pure" hash of the input (in raw, hex, and Base64 encodings). There should be
> additional methods which provide the output in the "full" representation,
> including the algorithm indicator, cost parameters, and salt.
> Example: {{$2a$10$ABCDEFGHIJKLMNOPQRSTUV$}}
> It may be sufficient to simply output the full string representation, or a
> new POJO datatype may be necessary. The benefit of such container should be
> weighed against the need for algorithm-specific containers in the event the
> fields are not sufficiently generic.
> The interface should also provide a {{boolean matches(String input, String
> hash)}} method to determine if an existing input value matches a generated
> ("full") hash output.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-5470) Allow Initial Admin Identity to have full read/write access to new instance flow
[
https://issues.apache.org/jira/browse/NIFI-5470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-5470.
Resolution: Won't Do
> Allow Initial Admin Identity to have full read/write access to new instance
> flow
>
>
> Key: NIFI-5470
> URL: https://issues.apache.org/jira/browse/NIFI-5470
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Security
>Affects Versions: 1.7.1
>Reporter: Andy LoPresto
>Priority: Major
> Labels: access_control, authorization, security, user
>
> As noted in the [Apache NiFi Admin Guide -- Initial Admin
> Identity|https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#initial-admin-identity],
> when a user configures a new secure NiFi instance, they must populate an
> *Initial Admin Identity* in {{authorizers.xml}}. However, if this is a
> instance, the IAI user does not have any access to the flow itself.
> {quote}
> For a brand new secure flow, providing the "Initial Admin Identity" gives
> that user access to get into the UI and to manage users, groups and policies.
> But if that user wants to start modifying the flow, they need to grant
> themselves policies for the root process group. The system is unable to do
> this automatically because in a new flow the UUID of the root process group
> is not permanent until the flow.xml.gz is generated. If the NiFi instance is
> an upgrade from an existing flow.xml.gz or a 1.x instance going from unsecure
> to secure, then the "Initial Admin Identity" user is automatically given the
> privileges to modify the flow.
> {quote}
> I believe there can be a workaround to determine the root process group UUID
> and grant the IAI user access automatically on startup. When starting a new
> instance, I can see the {{flow.xml.gz}} file persisted to disk with a
> generated root process group ID before granting the IAI user any additional
> permissions.
> Once the empty {{flow.xml.gz}} is persisted to disk and the root process
> group ID determined, the IAI user should be automatically granted write
> permissions to that group.
> {code}
>
>
> 10
> 5
>
>
> de37762f-0164-1000-ca28-13cc9d45f41b
> NiFi Flow
>
>
>
>
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-6823) Investigate adding 2FA to NiFi
[ https://issues.apache.org/jira/browse/NIFI-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-6823. Assignee: (was: Nathan Gough) Resolution: Won't Do > Investigate adding 2FA to NiFi > -- > > Key: NIFI-6823 > URL: https://issues.apache.org/jira/browse/NIFI-6823 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Reporter: Nathan Gough >Priority: Major > Labels: 2fa, 2sv, authentication, security, two-factor > > It would be great to see two-factor authentication options added to NiFi. I > believe there can be two solutions which can be investigated and added: > # External identity providers which offer 2FA/2SV (easy) > ## Allow a NiFi administrator to enforce 2SV for NiFi users > # A native solution (hard) > ## Will require some level of native user tracking to track shared keys per > user for TOTP. Currently NiFi stores basically no details of users or their > passwords. > ### Secure hashing of passwords using bcrypt/scrypt or some other password > hashing mechanism > ## Uses an open source library to handle TOTP > ### Must allow users to initialize, enter and verify TOTP values -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-5363) Enhance NiFi Toolkit to handle NiFi Registry
[
https://issues.apache.org/jira/browse/NIFI-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-5363.
Assignee: (was: Andy LoPresto)
Resolution: Implemented
> Enhance NiFi Toolkit to handle NiFi Registry
>
>
> Key: NIFI-5363
> URL: https://issues.apache.org/jira/browse/NIFI-5363
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: encryption, registry, security, tls, toolkit
>
> The Apache NiFi Toolkit contains tools for encrypting the sensitive
> configuration values in {{nifi.properties}},
> {{login-identity-providers.xml}}, and {{authorizers.xml}}, as well as
> generating TLS material for the {{keystore.jks}}, {{truststore.jks}}, and
> {{nifi.properties}} files. The toolkit should be enhanced to handle the NiFi
> Registry as well, in order to make deploying a secure registry instance
> easier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-4483) Admin toolkit improvements
[
https://issues.apache.org/jira/browse/NIFI-4483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-4483.
Assignee: (was: Andy LoPresto)
Resolution: Won't Do
> Admin toolkit improvements
> --
>
> Key: NIFI-4483
> URL: https://issues.apache.org/jira/browse/NIFI-4483
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Affects Versions: 1.4.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: documentation, tools
>
> The Admin Toolkit components (notify and node-manager specifically) and their
> documentation have some inconsistencies that were discovered during PR 2206
> review for NIFI-.
> {code}
> -Improve documentation & help message
> -bootstrap & home dir should not both be required
> -Test specs don't have fully defined resources
> -Line 292 should set activeUrls list to empty list instead of null by
> default
> -Check for extra '-'
> -Default HTTPS port 8081 -> 8443
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-3740) Hostname validation error message can be unclear if SAN fails but CN matches hostname
[
https://issues.apache.org/jira/browse/NIFI-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-3740.
Resolution: Won't Do
> Hostname validation error message can be unclear if SAN fails but CN matches
> hostname
> -
>
> Key: NIFI-3740
> URL: https://issues.apache.org/jira/browse/NIFI-3740
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Priority: Major
> Labels: certificate, hostname, security
>
> As reported on the mailing list, the error message can be very confusing if
> the hostname matches the certificate CN but not the SAN.
> {quote}
> On Apr 23, 2017, at 4:42 PM, Joe Gresock wrote:
> Just to follow up -- apparently if the Subject Alternate Name is set
> incorrectly, it will result in this error. Apparently the CN is ignored if
> the SAN is set on the cert.
> On Sat, Apr 22, 2017 at 12:08 PM, Joe Gresock wrote:
> I've been banging my head against the wall on this one.. is there a good
> way to further debug this RPG error? The hostname clearly matches the
> certificate CN.
> 2017-04-22 12:04:35,932 WARN [Remote Process Group
> 68ed2275-894d-3d75-b457-9d28a1b680e0:
> https://ip-172-31-33-37.ec2.internal:8443/nifi Thread-1]
> o.a.n.remote.StandardRemoteProcessGroup
> Unable to connect to RemoteProcessGroup[https://ip-
> 172-31-33-37.ec2.internal:8443/nifi] due to
> javax.net.ssl.SSLPeerUnverifiedException:
> Host name '*ip-172-31-33-37.ec2.internal*' does not match the certificate
> subject provided by the peer (CN=*ip-172-31-33-37.ec2.internal*, OU=LZ,
> O=LZS, L=Jessup, ST=Maryland, C=US)
> {quote}
> The exception thrown by the code under discussion should differentiate
> between the reasons the verification failed so a more helpful error message
> can be displayed to the user/in the logs.
> See [RFC 2818|https://tools.ietf.org/html/rfc2818#section-3] for more
> information.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-1475) Enforce desired key length validation check in Scrypt
[
https://issues.apache.org/jira/browse/NIFI-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-1475.
Assignee: (was: Andy LoPresto)
Resolution: Won't Do
> Enforce desired key length validation check in Scrypt
> -
>
> Key: NIFI-1475
> URL: https://issues.apache.org/jira/browse/NIFI-1475
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Priority: Minor
> Labels: encryption, security
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The {{Scrypt}} primitive and {{ScryptCipherProvider}} should enforce
> validation on the desired key length value {{dkLen}}.
> Add unit tests to exercise validation in {{ScryptGroovyTest}} and {{
> ScryptCipherProviderGroovyTest}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-7395) Make SecureHasher KDF implementations available to HashAttribute and HashContent processors
[
https://issues.apache.org/jira/browse/NIFI-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-7395.
Resolution: Won't Do
> Make SecureHasher KDF implementations available to HashAttribute and
> HashContent processors
> ---
>
> Key: NIFI-7395
> URL: https://issues.apache.org/jira/browse/NIFI-7395
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions, Security
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Priority: Major
> Labels: ProcessorStatus, hash, kdf, security
>
> The {{HashService}} and {{HashAlgorithm}} classes should be updated to make
> the Argon2, Bcrypt, Scrypt, and PBKDF2 hashes available to the
> {{CryptographicHashAttribute}} and {{CryptographicHashContent}} processors.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-3890) Create Key Management Controller Service
[
https://issues.apache.org/jira/browse/NIFI-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-3890.
Resolution: Abandoned
> Create Key Management Controller Service
>
>
> Key: NIFI-3890
> URL: https://issues.apache.org/jira/browse/NIFI-3890
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: encryption, key-management, metadata, security
>
> Similar to the {{KeyProvider}} interface and various backing implementations
> for the {{EncryptedWriteAheadProvenanceRepository}}, a generic
> {{KeyManagementControllerService}} should be made available so that other
> controller services, reporting tasks, and processors can reference it and
> retrieve the necessary keys to perform their cryptographic operations.
> It can be backed by a variety of implementations (file-based, static,
> database, HSM, etc.).
> It should have extensive auditing and granular access controls to restrict
> both user interaction and component interaction.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-1364) Audit OCSP certificate validation
[ https://issues.apache.org/jira/browse/NIFI-1364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-1364. Resolution: Information Provided > Audit OCSP certificate validation > - > > Key: NIFI-1364 > URL: https://issues.apache.org/jira/browse/NIFI-1364 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Affects Versions: 0.4.1 >Reporter: Andy LoPresto >Assignee: Nathan Gough >Priority: Major > Labels: certificate, ocsp, security > Time Spent: 2h 40m > Remaining Estimate: 0h > > While upgrading the version of BouncyCastle libraries used, I had to re-write > the OCSP certificate validation code because BC split the PKIX code into a > separate module and renamed many classes & methods. During this re-write, I > made the code compile using the new logic, but I am unsure that OCSP > validation needs to occur outside of the SSL/TLS negotiation, or that the > current mechanism is correct. > Questions: > * Can we use Java's built-in OCSP validation? [1][2] > * Is the current mechanism correct, where a local cache is used with custom > internal classes representing OCSP requests and statuses, and it queries a > pre-specified OCSP responder as opposed to the per-certificate OCSP responder > listed in each certificate's Authority Information Access OCSP URI [3]? I > think this design decision stems from a legacy environment which may not > apply to current use cases. > More information: [4] > [1] https://blogs.oracle.com/xuelei/entry/enable_ocsp_checking > [2] > https://stackoverflow.com/questions/8506661/check-x509-certificate-revocation-status-in-spring-security-before-authenticatin > [3] > https://blog.ivanristic.com/2014/02/checking-ocsp-revocation-using-openssl.html > [4] > https://raymii.org/s/articles/OpenSSL_Manually_Verify_a_certificate_against_an_OCSP.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-7674) Toolkit diagnostic for Cluster configuration
[ https://issues.apache.org/jira/browse/NIFI-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-7674. Resolution: Won't Do > Toolkit diagnostic for Cluster configuration > > > Key: NIFI-7674 > URL: https://issues.apache.org/jira/browse/NIFI-7674 > Project: Apache NiFi > Issue Type: Sub-task > Components: Configuration Management, Tools and Build >Affects Versions: 1.11.4 >Reporter: Veda Kadam >Assignee: Veda Kadam >Priority: Major > Labels: cluster, keystore, security, tls, tls-toolkit > > Refer NIFI-7673 for problems addressed. > # Cluster TLS configuration verification > # Priority 1 > # Addresses A, B, C, D, J, K > # Description: Verifies all nodes in the cluster have the ability to verify > and communicate with each other (TLS only) > # Steps > # Run on each node > # Start simple Jetty server using keystore & truststore from nifi.properties > on each node (using API port) > # Possible to run listening on multiple ports (API, CC, S2S, LB) > # Connect to embedded or external ZK and retrieve all cluster node hostnames > # Attempt to ping NiFi hostnames to resolve DNS/validate firewall > # Make simple request from each node to each other node and verify mTLS > # PKIX path building > # Cipher suite availability > # TLS protocol version availability > # Ports open -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-7675) Toolkit diagnostic should verify user provisions and policies
[ https://issues.apache.org/jira/browse/NIFI-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-7675. Resolution: Won't Do > Toolkit diagnostic should verify user provisions and policies > - > > Key: NIFI-7675 > URL: https://issues.apache.org/jira/browse/NIFI-7675 > Project: Apache NiFi > Issue Type: Sub-task > Components: Configuration Management, Tools and Build >Affects Versions: 1.11.4 >Reporter: Veda Kadam >Assignee: Veda Kadam >Priority: Major > Labels: permission, policy, security, tls, tls-toolkit, user > > Refer NIFI-7673 for addressed problems. > # Verify user/group provisioning, permissions & policies > # Priority 2 > # Description: Verifies each node has the correct user & group configuration > files, the correct authorizer configuration, and can perform basic authorized > actions > # TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-3691) Provide utility to verify configured security settings and certificates
[ https://issues.apache.org/jira/browse/NIFI-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-3691: -- Assignee: (was: Andy LoPresto) > Provide utility to verify configured security settings and certificates > --- > > Key: NIFI-3691 > URL: https://issues.apache.org/jira/browse/NIFI-3691 > Project: Apache NiFi > Issue Type: New Feature > Components: Configuration >Reporter: Aldrin Piri >Priority: Major > Labels: certificate, keystore, security, tls-toolkit > > It would be helpful to provide a utility that could analyze > keystores/truststores to verify compatibility and expected behavior with > configured security settings such as two way SSL (right hostname, alias, > etc). The idea is that as a diagnostic tool, we could provide users with > some help to verify and troubleshoot any issues that may exist with > certificates outside of more expensive change/restart loops with NiFi. As a > follow-on, it would be helpful to get a listing of key properties about the > configured keystore/truststore or files provided. An extension of this might > additionally setup a client/server test with the utility between instances, > again, to verify correct operation without doing so in NiFi itself as > suggested by the parent ticket. > It would be nice to provide this as part of the NiFi release and accessible > via nifi.sh. By extension, the functionality could also appear in the TLS > toolkit. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-3766) Improve user experience when migrating from legacy provenance repository to encrypted
[
https://issues.apache.org/jira/browse/NIFI-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-3766.
Resolution: Abandoned
> Improve user experience when migrating from legacy provenance repository to
> encrypted
> -
>
> Key: NIFI-3766
> URL: https://issues.apache.org/jira/browse/NIFI-3766
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core Framework
>Reporter: Andy LoPresto
>Priority: Major
> Labels: encryption, provenance, security, ux
>
> As reported during a review of NIFI-3594 / PR 1686, [~YolandaMDavis] was able
> to generate some unhelpful stacktraces and UI error dialogs when switching
> from {{PersistentProvenanceRepository}} (the legacy implementation that most
> users have selected) to the new {{EncryptedWriteAheadProvenanceRepository}}.
> I had only tested {{WriteAheadProvenanceRepository}} to and from
> {{EncryptedWriteAheadProvenanceRepository}}.
> In addition, switching between {{EWAPR}} and {{WAPR}} causes a small
> exception but the application functions as expected with the exception that
> previously written provenance events are no longer available. However, when
> switching from {{PPR}} to {{EWAPR}}, new provenance events are not written or
> queryable. This is obviously untenable moving forward.
> For now, documentation has been provided explaining that existing
> repositories must be deleted from disk before switching implementations to
> allow continued indexing and querying under the new implementation.
> The user experience should be improved to suppress the stacktrace and error
> dialogs and smoothly handle the migration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-4881) Provide TLS "auto-secure" feature
[
https://issues.apache.org/jira/browse/NIFI-4881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-4881.
Resolution: Won't Fix
> Provide TLS "auto-secure" feature
> -
>
> Key: NIFI-4881
> URL: https://issues.apache.org/jira/browse/NIFI-4881
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Configuration Management, Core Framework
>Affects Versions: 1.5.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls
>
> As documented in the Apache NiFi Wiki Feature Roadmap (Security), I have
> wanted to implement for some time a feature where the administrator of a NiFi
> instance does not have to know the intricate details of TLS configuration in
> order to deploy a secure instance. What I propose is the following:
> * The administrator can set the TLS security settings to *high*, *medium*,
> and *low*
> * These settings have accompanying descriptions explaining that "*high*
> means most secure (with lower backwards compatibility)", "*medium* tries to
> strike a balance between security and compatibility", and "*low* allows for
> more widespread legacy compatibility with less emphasis on security".
> * The cipher suite lists and protocol versions for each would be downloaded
> from the [Mozilla TLS
> Observatory|https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_configurations],
> which publishes updated definitions for each of "modern", "intermediate",
> and "legacy" options at initial startup and at regular intervals.
> * For deployments in an airgapped or other environment without desired
> connectivity to this service, or where the source is preferred to be
> controlled by an organizational entity, an alternate service endpoint can be
> configured (for proof of concept, this could even be the NiFi instance itself
> reading a file from disk and returning JSON over an HTTP endpoint).
> * When a new definition is received or selected, the application framework
> would either:
> ** Restart the Jetty server automatically in a pre-configured timeframe
> ** Wait for all queues to empty and then restart
> ** Provide a visual alert (bulletin, etc.) that new definitions were
> received and a manual restart is required
> * This setting could be set in a variety of ways:
> ** Directly in {{nifi.properties}} before the first application launch
> ** Given as an argument to an enhanced TLS Toolkit (i.e.
> {{./bin/tls-toolkit.sh standalone ... -S high}}) and then the resulting
> {{nifi.properties}} placed in the correct location
> ** Through a UI configuration option (this would need to be restricted to
> the appropriate NiFi permissions and would require a Jetty server restart)
> * The definitions would "grow" with the ecosystem (i.e. as a new
> vulnerability is discovered or a new protocol version/cipher suite is made
> available, it is automatically added/removed from the definition, thus
> continually improving the security stance of the application without
> requiring active monitoring and input from an administrator)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-5211) Create JSON reader, writer, signer, and verifier
[
https://issues.apache.org/jira/browse/NIFI-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-5211.
Resolution: Won't Fix
> Create JSON reader, writer, signer, and verifier
>
>
> Key: NIFI-5211
> URL: https://issues.apache.org/jira/browse/NIFI-5211
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: Andy LoPresto
>Priority: Minor
> Labels: security, tls
>
> This piece should accept a TLS configuration object and serialize/deserialize
> it to/from JSON, and should provide a signature generation and verification
> service to ensure it is trusted and has not been manipulated.
> * JSON de/serialization
> * Read arbitrary JSON and verify an HMAC/SHA-512 signature before parsing
> * Generate a signature over JSON and persist in place
> * Derive the signature key from the master key in {{bootstrap.conf}} with a
> one-way transformation (i.e. {{HMAC/SHA-512("JSON TLS key", MK) -> TLSK}})
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-5212) Configure JettyServer with custom TLS protocols and cipher suites
[
https://issues.apache.org/jira/browse/NIFI-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-5212.
Assignee: (was: Andy LoPresto)
Resolution: Won't Fix
> Configure JettyServer with custom TLS protocols and cipher suites
> -
>
> Key: NIFI-5212
> URL: https://issues.apache.org/jira/browse/NIFI-5212
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Andy LoPresto
>Priority: Minor
> Labels: jetty, security, tls
>
> This piece should read a TLS configuration and configure the {{JettyServer}}
> and underlying {{SslContextFactory}} with the provided protocol versions and
> cipher suites, performing basic validation (to ensure at least one valid
> cipher suite and protocol, verifying against {{java.tls.disabledAlgorithms}},
> etc.).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-5210) Create service to retrieve TLS configurations from remote endpoint
[ https://issues.apache.org/jira/browse/NIFI-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-5210. Resolution: Won't Fix > Create service to retrieve TLS configurations from remote endpoint > -- > > Key: NIFI-5210 > URL: https://issues.apache.org/jira/browse/NIFI-5210 > Project: Apache NiFi > Issue Type: Sub-task > Components: Extensions >Affects Versions: 1.6.0 >Reporter: Andy LoPresto >Priority: Minor > Labels: security, tls > > One component of this system will be to retrieve the configurations from a > remote service (if the admin opts in). This piece should: > * communicate over HTTPS only > ** have a custom truststore? > * have a configurable URL > ** have a secondary URL? > * have a configurable polling interval -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-5458) Improve NiFi TLS and certificate management
[ https://issues.apache.org/jira/browse/NIFI-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-5458. Assignee: (was: Andy LoPresto) Resolution: Abandoned > Improve NiFi TLS and certificate management > --- > > Key: NIFI-5458 > URL: https://issues.apache.org/jira/browse/NIFI-5458 > Project: Apache NiFi > Issue Type: Epic > Components: Configuration, Configuration Management, Core Framework, > Docker, Security >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Priority: Major > Labels: certificate, cluster, encryption, security, tls, > tls-toolkit > > To securely deploy Apache NiFi requires substantial background knowledge, > applied familiarity with a disparate set of tools and operating systems, and > disjoint manual effort. The NiFi TLS Toolkit and Encrypt Config Toolkits aim > to help, but the former is designed for development/sandbox environments, not > integration with enterprise certificate authorities (CA). In addition, NiFi > requires tightly coupled security configuration when deploying in a cluster > environment, and dynamic horizontal scaling is difficult. > This epic will serve as an aggregator for all individual tickets related to > an ongoing, holistic effort to streamline, automate, and lower the barrier to > entry to configuring a secure NiFi deployment. > * Generating or acquiring signed certificates and converting them to the > proper format (JKS, PEM, P12, etc.) > * Integrating with external certificate providers > * Securing the sensitive configuration values > * Automating deployment of configuration values > * Encapsulating/delegating security configuration for containerization efforts > * Automating deployment of TLS cipher suites and protocol versions > * Automating mitigation of TLS vulnerabilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-5364) ConfigEncryptionTool should handle NiFi Registry
[
https://issues.apache.org/jira/browse/NIFI-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-5364.
Assignee: (was: Andy LoPresto)
Resolution: Incomplete
> ConfigEncryptionTool should handle NiFi Registry
>
>
> Key: NIFI-5364
> URL: https://issues.apache.org/jira/browse/NIFI-5364
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: encryption, keystore, registry, security
>
> The CET should encrypt sensitive properties for the NiFi Registry.
> The necessary improvements include:
> * the tool should respect the input filename and not generate a hard-coded
> {{nifi.properties}} file as output; if the input file is
> {{nifi-registry.properties}} and no output filename is provided as an
> argument, the output file should be {{nifi-registry.properties}} as well
> * Keys starting with {{nifi.registry.*}} should be detected (currently, the
> list of sensitive properties is hard-coded, so
> {{nifi.registry.security.keystorePasswd}} doesn't get encrypted, for example)
> * {{nifi.registry.sensitive.props.additional.keys}} should be detected
> * {{nifi.sensitive.props.additional.keys}} must be manually renamed
> * The encrypted output shows *3* protected when only *2* properties are.
> This is because {{nifi.sensitive.props.key}} was generated but did not
> persist (because it didn't already exist in {{nifi-registry.properties}})
> * {{nifi.registry.db.password}} should be detected
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-1471) Add accessor for parameterized CipherProviders
[
https://issues.apache.org/jira/browse/NIFI-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-1471.
Assignee: (was: Andy LoPresto)
Resolution: Won't Fix
> Add accessor for parameterized CipherProviders
> --
>
> Key: NIFI-1471
> URL: https://issues.apache.org/jira/browse/NIFI-1471
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Priority: Minor
> Labels: encryption, security
>
> Currently, all {{CipherProviders}} instantiated from
> {{CipherProviderFactory}} use the default constructor because the accessor
> does not accept parameters. Provide parameterized accessors for
> implementations which accept cost parameters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-1470) Add accessor for PBE and Keyed CipherProviders to CipherProviderFactory
[
https://issues.apache.org/jira/browse/NIFI-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-1470.
Assignee: (was: Andy LoPresto)
Resolution: Won't Fix
> Add accessor for PBE and Keyed CipherProviders to CipherProviderFactory
> ---
>
> Key: NIFI-1470
> URL: https://issues.apache.org/jira/browse/NIFI-1470
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Priority: Trivial
> Labels: encryption, security
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Currently the
> {{CipherProviderFactory#getCipherProvider(KeyDerivationFunction)}} method
> returns a {{CipherProvider}} instance without casting to the more specific
> interface. Provide more specific accessor methods for scenarios when the
> desired destination interface is known.
> * {{getPBECipherProvider(KeyDerivationFunction)}}
> * {{getKeyedCipherProvider(KeyDerivationFunction)}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12395) Update Jackson.bom.version to 2.16.0
[ https://issues.apache.org/jira/browse/NIFI-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12395: --- Assignee: David Handermann (was: Mike R) > Update Jackson.bom.version to 2.16.0 > > > Key: NIFI-12395 > URL: https://issues.apache.org/jira/browse/NIFI-12395 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.24.0, 1.23.2 >Reporter: Mike R >Assignee: David Handermann >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Update Jackson.bom.version to 2.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() [nifi-minifi-cpp]
arpadboda closed pull request #1703: MINIFICPP-2265 Implement AttributeRollingWindow and EL nextInt() URL: https://github.com/apache/nifi-minifi-cpp/pull/1703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2271 update AWS SDK and regions [nifi-minifi-cpp]
arpadboda closed pull request #1705: MINIFICPP-2271 update AWS SDK and regions URL: https://github.com/apache/nifi-minifi-cpp/pull/1705 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2274 Ship Markdown docs in binary packages [nifi-minifi-cpp]
arpadboda closed pull request #1709: MINIFICPP-2274 Ship Markdown docs in binary packages URL: https://github.com/apache/nifi-minifi-cpp/pull/1709 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12561) MergeContent processor retains text delimiter settings when switched to 'Do Not Use Delimiters' strategy
Mohammed Salim created NIFI-12561:
-
Summary: MergeContent processor retains text delimiter settings
when switched to 'Do Not Use Delimiters' strategy
Key: NIFI-12561
URL: https://issues.apache.org/jira/browse/NIFI-12561
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.24.0
Environment: NiFi Docker
Reporter: Mohammed Salim
In the MergeContent processor version 1.24.0, when configuring the processor to
use Text Delimiters (by setting values for Header, Footer, and Demarcator), and
then changing the Delimiter Strategy to "Do Not Use Delimiters", the previously
set values for the Text Delimiters Strategy are still being applied during
processing.
{_}Steps to Reproduce{_}:
# Configure the MergeContent processor with the Delimiter Strategy set to
"Text".
# Set values for Header, Footer, and Demarcator.
# Apply the configuration changes.
# Change the Delimiter Strategy to "Do Not Use Delimiters" and apply the
configuration.
# Observe that the processor still uses the old values for Header, Footer, and
Demarcator during processing, despite the Delimiter Strategy being set to "Do
Not Use Delimiters".
{_}Expected Behavior{_}: When changing the Delimiter Strategy to "Do Not Use
Delimiters", any previously set values for Text Delimiters should not be
applied.
{_}Actual Behavior{_}: The processor retains and applies the old values for
Text Delimiters even after changing the Delimiter Strategy to "Do Not Use
Delimiters".
{_}Environment{_}:
* NiFi Version: 1.24.0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12553) Status History from canvas context menu for components
[ https://issues.apache.org/jira/browse/NIFI-12553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12553: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Status History from canvas context menu for components > -- > > Key: NIFI-12553 > URL: https://issues.apache.org/jira/browse/NIFI-12553 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12553) Status History from canvas context menu for components
[ https://issues.apache.org/jira/browse/NIFI-12553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801943#comment-17801943 ] ASF subversion and git services commented on NIFI-12553: Commit 689f9909786d126843740f774e897f8bbd6a0e86 in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=689f990978 ] [NIFI-12553] - Context menu option for View Status History (#8193) * [NIFI-12553] - Context menu option for View Status History * Added View Status History to the canvas context menu for applicable component types * Added support for Node Status History from the flow menu * remove unused imports This closes #8193 > Status History from canvas context menu for components > -- > > Key: NIFI-12553 > URL: https://issues.apache.org/jira/browse/NIFI-12553 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12553] - Context menu option for View Status History [nifi]
mcgilman merged PR #8193: URL: https://github.com/apache/nifi/pull/8193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12560) Bulletin Board
Rob Fellows created NIFI-12560: -- Summary: Bulletin Board Key: NIFI-12560 URL: https://issues.apache.org/jira/browse/NIFI-12560 Project: Apache NiFi Issue Type: Sub-task Reporter: Rob Fellows Assignee: Rob Fellows -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) * Bulletin Board (NIFI-12560) * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) * Configure Reporting Task * Flow Analysis Rules * Registry Clients (NIFI-12486) * Import from Registry * Parameter Providers * Cluster * Flow Configuration History * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) * Policies (NIFI-12548) * Help * About * Show Upstream/Downstream * Align * List Queue * Empty [all] Queue * View Content * View State * Change Version * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) * Download Flow * Create RPG * Configure RPG * RPG Remote Ports * RPG Go To * Color * Move to Front * Copy/Paste * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Routing error handling * Prevent navigate to Service when configuration form is dirty * Introduce header in new pages to unify with canvas and offer better navigation. * Prompt user to save Parameter Context when Edit form is dirty [1] [https://github.com/apache/nifi/pull/8053] was: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) * Bulletin Board * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) * Configure Reporting Task * Flow Analysis Rules * Registry Clients (NIFI-12486) * Import from Registry * Parameter Providers * Cluster * Flow Configuration History * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) * Policies (NIFI-12548) * Help * About * Show Upstream/Downstream * Align * List Queue * Empty [all] Queue * View Content * View State * Change Version * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) * Download Flow * Create RPG * Configure RPG * RPG Remote Ports * RPG Go To * Color * Move to Front * Copy/Paste * Run unit tests are part of standard build * Update all API call
[jira] [Updated] (NIFI-12538) Add map Record Standalone Function
[
https://issues.apache.org/jira/browse/NIFI-12538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12538:
Fix Version/s: 1.25.0
2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add map Record Standalone Function
> --
>
> Key: NIFI-12538
> URL: https://issues.apache.org/jira/browse/NIFI-12538
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
> Fix For: 1.25.0, 2.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Adding a function that can be used to create a map of Strings.
> For example, if we have the following record:
>
> {code:java}
> {
> "firstName": "John",
> "lastName": "Snow"
> }{code}
>
> We could use the {{UpdateRecord}} processor with
> {code:java}
> /fullName => map("firstName", /firstName, "lastName", /lastName){code}
> And that would give us something like:
> {code:java}
> {
> "firstName": "John",
> "lastName": "Snow",
> "fullName": {"firstName": "John", "lastName": "Snow"}
> }{code}
> This function requires an even number of arguments and the record paths must
> represent simple field values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12538) Add map Record Standalone Function
[
https://issues.apache.org/jira/browse/NIFI-12538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801938#comment-17801938
]
ASF subversion and git services commented on NIFI-12538:
Commit 6ee0365043447f5770b186421b6371cb31a61a0a in nifi's branch
refs/heads/support/nifi-1.x from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6ee0365043 ]
NIFI-12538 Added mapOf Record Standalone Function
This closes #8182
Signed-off-by: David Handermann
(cherry picked from commit a7c9eccf4a2dbb2eed3487eaef477ac05624d291)
> Add map Record Standalone Function
> --
>
> Key: NIFI-12538
> URL: https://issues.apache.org/jira/browse/NIFI-12538
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Adding a function that can be used to create a map of Strings.
> For example, if we have the following record:
>
> {code:java}
> {
> "firstName": "John",
> "lastName": "Snow"
> }{code}
>
> We could use the {{UpdateRecord}} processor with
> {code:java}
> /fullName => map("firstName", /firstName, "lastName", /lastName){code}
> And that would give us something like:
> {code:java}
> {
> "firstName": "John",
> "lastName": "Snow",
> "fullName": {"firstName": "John", "lastName": "Snow"}
> }{code}
> This function requires an even number of arguments and the record paths must
> represent simple field values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12538) Add map Record Standalone Function
[
https://issues.apache.org/jira/browse/NIFI-12538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801935#comment-17801935
]
ASF subversion and git services commented on NIFI-12538:
Commit a7c9eccf4a2dbb2eed3487eaef477ac05624d291 in nifi's branch
refs/heads/main from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a7c9eccf4a ]
NIFI-12538 Added mapOf Record Standalone Function
This closes #8182
Signed-off-by: David Handermann
> Add map Record Standalone Function
> --
>
> Key: NIFI-12538
> URL: https://issues.apache.org/jira/browse/NIFI-12538
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Adding a function that can be used to create a map of Strings.
> For example, if we have the following record:
>
> {code:java}
> {
> "firstName": "John",
> "lastName": "Snow"
> }{code}
>
> We could use the {{UpdateRecord}} processor with
> {code:java}
> /fullName => map("firstName", /firstName, "lastName", /lastName){code}
> And that would give us something like:
> {code:java}
> {
> "firstName": "John",
> "lastName": "Snow",
> "fullName": {"firstName": "John", "lastName": "Snow"}
> }{code}
> This function requires an even number of arguments and the record paths must
> represent simple field values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12538 - Add map Record Standalone Function [nifi]
exceptionfactory closed pull request #8182: NIFI-12538 - Add map Record Standalone Function URL: https://github.com/apache/nifi/pull/8182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-12555) UI - Property with .dependsOn() incorrectly being displayed
[ https://issues.apache.org/jira/browse/NIFI-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801913#comment-17801913 ] Daniel Stieglitz edited comment on NIFI-12555 at 1/2/24 7:50 PM: - [~markbean] Can you give example(s) of the processor(s) you are seeing this behavior in? was (Author: JIRAUSER294662): [~markbean] Can you give examples of the processors you are seeing this behavior in? > UI - Property with .dependsOn() incorrectly being displayed > --- > > Key: NIFI-12555 > URL: https://issues.apache.org/jira/browse/NIFI-12555 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.24.0 >Reporter: Mark Bean >Priority: Major > > Consider PROPERTY_1 with: > ` > .dependsOn(PROPERTY_2, "foo") > ` > In the UI, when Property 2 is set to a value of "bar", Property 1 should not > be displayed. However, when the configuration is first opened, Property 1 is > displayed. The UI correctly updates only after re-selecting "bar" for > Property 2 - or even just opening its value and choosing "Cancel". In other > words, after taking any action to change any property, then Property 1 is > correctly removed from the UI. > This appears to be a problem only for the initial display of component > properties, and not when they are refreshed and/or dependency rules are > evaluated. > -It is possible this is related to changes introduced in NIFI-11593. Note > that the above behavior occurs even when PROPERTY_2 has specific allowed > values.- > The same behavior is seen in 1.21.0, and NIFI-11593 has a fix version of > 1.22.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12555) UI - Property with .dependsOn() incorrectly being displayed
[ https://issues.apache.org/jira/browse/NIFI-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801913#comment-17801913 ] Daniel Stieglitz commented on NIFI-12555: - [~markbean] Can you give examples of the processors you are seeing this behavior in? > UI - Property with .dependsOn() incorrectly being displayed > --- > > Key: NIFI-12555 > URL: https://issues.apache.org/jira/browse/NIFI-12555 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.24.0 >Reporter: Mark Bean >Priority: Major > > Consider PROPERTY_1 with: > ` > .dependsOn(PROPERTY_2, "foo") > ` > In the UI, when Property 2 is set to a value of "bar", Property 1 should not > be displayed. However, when the configuration is first opened, Property 1 is > displayed. The UI correctly updates only after re-selecting "bar" for > Property 2 - or even just opening its value and choosing "Cancel". In other > words, after taking any action to change any property, then Property 1 is > correctly removed from the UI. > This appears to be a problem only for the initial display of component > properties, and not when they are refreshed and/or dependency rules are > evaluated. > -It is possible this is related to changes introduced in NIFI-11593. Note > that the above behavior occurs even when PROPERTY_2 has specific allowed > values.- > The same behavior is seen in 1.21.0, and NIFI-11593 has a fix version of > 1.22.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12558) Upgrade Jagged to 0.3.0
[ https://issues.apache.org/jira/browse/NIFI-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-12558: Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Jagged to 0.3.0 > --- > > Key: NIFI-12558 > URL: https://issues.apache.org/jira/browse/NIFI-12558 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The Jagged library should be upgraded to version > [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both > main and support branches to resolve an issue with concurrent access to > KeyAgreement instances that presents itself when configuring the > DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12558) Upgrade Jagged to 0.3.0
[ https://issues.apache.org/jira/browse/NIFI-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801901#comment-17801901 ] ASF subversion and git services commented on NIFI-12558: Commit 7a0449b425690d391885c430ab1dade1164afeb3 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7a0449b425 ] NIFI-12558 Upgraded Jagged from 0.2.0 to 0.3.0 Signed-off-by: Matt Burgess This closes #8199 > Upgrade Jagged to 0.3.0 > --- > > Key: NIFI-12558 > URL: https://issues.apache.org/jira/browse/NIFI-12558 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The Jagged library should be upgraded to version > [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both > main and support branches to resolve an issue with concurrent access to > KeyAgreement instances that presents itself when configuring the > DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12558 Upgrade Jagged from 0.2.0 to 0.3.0 [nifi]
mattyb149 closed pull request #8199: NIFI-12558 Upgrade Jagged from 0.2.0 to 0.3.0 URL: https://github.com/apache/nifi/pull/8199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12558 Upgrade Jagged from 0.2.0 to 0.3.0 [nifi]
mattyb149 commented on PR #8199: URL: https://github.com/apache/nifi/pull/8199#issuecomment-1874424755 +1 LGTM, thanks for the upgrade! Merging to support/nifi-1.x and main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12558) Upgrade Jagged to 0.3.0
[ https://issues.apache.org/jira/browse/NIFI-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801900#comment-17801900 ] ASF subversion and git services commented on NIFI-12558: Commit 1a0a9b5bc09dada7ca79758ac268b21b3c06c342 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1a0a9b5bc0 ] NIFI-12558 Upgraded Jagged from 0.2.0 to 0.3.0 Signed-off-by: Matt Burgess > Upgrade Jagged to 0.3.0 > --- > > Key: NIFI-12558 > URL: https://issues.apache.org/jira/browse/NIFI-12558 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The Jagged library should be upgraded to version > [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both > main and support branches to resolve an issue with concurrent access to > KeyAgreement instances that presents itself when configuring the > DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12558) Upgrade Jagged to 0.3.0
[ https://issues.apache.org/jira/browse/NIFI-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-12558: Fix Version/s: 1.25.0 2.0.0 > Upgrade Jagged to 0.3.0 > --- > > Key: NIFI-12558 > URL: https://issues.apache.org/jira/browse/NIFI-12558 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The Jagged library should be upgraded to version > [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both > main and support branches to resolve an issue with concurrent access to > KeyAgreement instances that presents itself when configuring the > DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12524) AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade
[ https://issues.apache.org/jira/browse/NIFI-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-12524: Resolution: Fixed Status: Resolved (was: Patch Available) > AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade > > > Key: NIFI-12524 > URL: https://issues.apache.org/jira/browse/NIFI-12524 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0 > Environment: RHEL 8, OpenJDK11 Latest >Reporter: Shawn Weeks >Assignee: David Handermann >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Attachments: nifi-app.log > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems to be an issue with the migration from H2 to Xodus. After upgrading > from 1.23.2 to 1.24.0 NiFi fails to start and throws a null pointer exception > with the auditService Bean. See attached log snippet. > The cluster started life as NiFi 1.13.2 and has been steadily upgraded since > mid 2021. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12524) AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade
[ https://issues.apache.org/jira/browse/NIFI-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801899#comment-17801899 ] ASF subversion and git services commented on NIFI-12524: Commit 8ddc0f0741e9a456e60c8d43f09c5d348e1bc972 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8ddc0f0741 ] NIFI-12524 Fixed null handling for auditing Connect Details Signed-off-by: Matt Burgess This closes #8198 > AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade > > > Key: NIFI-12524 > URL: https://issues.apache.org/jira/browse/NIFI-12524 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0 > Environment: RHEL 8, OpenJDK11 Latest >Reporter: Shawn Weeks >Assignee: David Handermann >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Attachments: nifi-app.log > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems to be an issue with the migration from H2 to Xodus. After upgrading > from 1.23.2 to 1.24.0 NiFi fails to start and throws a null pointer exception > with the auditService Bean. See attached log snippet. > The cluster started life as NiFi 1.13.2 and has been steadily upgraded since > mid 2021. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12524 Fixed null handling for auditing Connect Details [nifi]
mattyb149 closed pull request #8198: NIFI-12524 Fixed null handling for auditing Connect Details URL: https://github.com/apache/nifi/pull/8198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12524) AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade
[ https://issues.apache.org/jira/browse/NIFI-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801898#comment-17801898 ] ASF subversion and git services commented on NIFI-12524: Commit e7f0efb99915c02e6aafd64acd1cf8d3e990f265 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e7f0efb999 ] NIFI-12524 Fixed null handling for auditing Connect Details Signed-off-by: Matt Burgess > AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade > > > Key: NIFI-12524 > URL: https://issues.apache.org/jira/browse/NIFI-12524 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0 > Environment: RHEL 8, OpenJDK11 Latest >Reporter: Shawn Weeks >Assignee: David Handermann >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Attachments: nifi-app.log > > Time Spent: 10m > Remaining Estimate: 0h > > Seems to be an issue with the migration from H2 to Xodus. After upgrading > from 1.23.2 to 1.24.0 NiFi fails to start and throws a null pointer exception > with the auditService Bean. See attached log snippet. > The cluster started life as NiFi 1.13.2 and has been steadily upgraded since > mid 2021. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12524 Fixed null handling for auditing Connect Details [nifi]
mattyb149 commented on PR #8198: URL: https://github.com/apache/nifi/pull/8198#issuecomment-1874417770 +1 LGTM, thanks for the fix! Merging to support/nifi-1.x and main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12559) Upgrade SSHJ to 0.38.0
[ https://issues.apache.org/jira/browse/NIFI-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801897#comment-17801897 ] ASF subversion and git services commented on NIFI-12559: Commit b8c8eafd034d107d9c912e3a5e2ad8023fe69d7b in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b8c8eafd03 ] NIFI-12559 Upgraded SSHJ from 0.37.0 to 0.38.0 - Removed Bouncy Castle JDK 1.5 version exclusions no longer required in SSHJ 0.38.0 Signed-off-by: Matt Burgess This closes #8200 > Upgrade SSHJ to 0.38.0 > -- > > Key: NIFI-12559 > URL: https://issues.apache.org/jira/browse/NIFI-12559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > SSHJ should be upgraded to version > [0.38.0|https://github.com/hierynomus/sshj#release-history] to incorporate > mitigations for the Terrapin vulnerability described in CVE-2023-48795, > applicable to SFTP Processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12559) Upgrade SSHJ to 0.38.0
[ https://issues.apache.org/jira/browse/NIFI-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-12559: Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade SSHJ to 0.38.0 > -- > > Key: NIFI-12559 > URL: https://issues.apache.org/jira/browse/NIFI-12559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > SSHJ should be upgraded to version > [0.38.0|https://github.com/hierynomus/sshj#release-history] to incorporate > mitigations for the Terrapin vulnerability described in CVE-2023-48795, > applicable to SFTP Processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12559 Upgrade SSHJ from 0.37.0 to 0.38.0 [nifi]
mattyb149 commented on PR #8200: URL: https://github.com/apache/nifi/pull/8200#issuecomment-1874412835 +1 LGTM, merging to support/nifi-1.x and main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12559 Upgrade SSHJ from 0.37.0 to 0.38.0 [nifi]
mattyb149 closed pull request #8200: NIFI-12559 Upgrade SSHJ from 0.37.0 to 0.38.0 URL: https://github.com/apache/nifi/pull/8200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12559) Upgrade SSHJ to 0.38.0
[ https://issues.apache.org/jira/browse/NIFI-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801896#comment-17801896 ] ASF subversion and git services commented on NIFI-12559: Commit dcc80805f99321bf861f72e6a7a8e7a885cd8b4d in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dcc80805f9 ] NIFI-12559 Upgraded SSHJ from 0.37.0 to 0.38.0 - Removed Bouncy Castle JDK 1.5 version exclusions no longer required in SSHJ 0.38.0 Signed-off-by: Matt Burgess > Upgrade SSHJ to 0.38.0 > -- > > Key: NIFI-12559 > URL: https://issues.apache.org/jira/browse/NIFI-12559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > SSHJ should be upgraded to version > [0.38.0|https://github.com/hierynomus/sshj#release-history] to incorporate > mitigations for the Terrapin vulnerability described in CVE-2023-48795, > applicable to SFTP Processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12531) Parameter references are removed after property migration
[
https://issues.apache.org/jira/browse/NIFI-12531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12531:
--
Affects Version/s: (was: 2.0.0)
> Parameter references are removed after property migration
> -
>
> Key: NIFI-12531
> URL: https://issues.apache.org/jira/browse/NIFI-12531
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M1
>Reporter: Zsihovszki Krisztina
>Assignee: Zsihovszki Krisztina
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Parameter references disappear from the properties when a flow is imported to
> NiFi 2.0.
> The problem was observed for InvokeHTTP processor when e.g. we extract "Host
> URL" property to a parameter. The flow definition contains the parameter
> reference properly:
>
> {code:java}
> "HTTP URL": "#{Server URL}",{code}
>
> but the property value is resolved in the imported flow and the connection
> between the parameter ("Server URL") and the processor is lost.
> The issue did not occur for other processors like ConvertRecord and it occurs
> in NiFi 2.0 only.
> Based on my investigation the root cause seems to be related to the property
> migration.
> During the property migration the raw values("#\{Server URL}") are resolved
> to effective values (e.g. http://host:port).
> In StandardProcessorNode.migrateProperties() the properties are overwritten (
> overwriteProperties) if the propertyConfig is modified.
> At this step the effective properties are used while I belive the raw
> properties should have been used.
> Another observed minor issue:
> In InvokeHTTP processor, the "Connection Timeout" is renamed to the same name
> (that triggered the above mentioned issue since it looked as if the processor
> properties were modified, property overwrite was needed)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12531) Parameter references are removed after property migration
[
https://issues.apache.org/jira/browse/NIFI-12531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801879#comment-17801879
]
ASF subversion and git services commented on NIFI-12531:
Commit 1a34208f65681bd13d5450407dcf6719daeeef37 in nifi's branch
refs/heads/main from krisztina-zsihovszki
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1a34208f65 ]
NIFI-12531 Parameter references are removed after property migration
Signed-off-by: Pierre Villard
This closes #8176.
> Parameter references are removed after property migration
> -
>
> Key: NIFI-12531
> URL: https://issues.apache.org/jira/browse/NIFI-12531
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M1, 2.0.0
>Reporter: Zsihovszki Krisztina
>Assignee: Zsihovszki Krisztina
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Parameter references disappear from the properties when a flow is imported to
> NiFi 2.0.
> The problem was observed for InvokeHTTP processor when e.g. we extract "Host
> URL" property to a parameter. The flow definition contains the parameter
> reference properly:
>
> {code:java}
> "HTTP URL": "#{Server URL}",{code}
>
> but the property value is resolved in the imported flow and the connection
> between the parameter ("Server URL") and the processor is lost.
> The issue did not occur for other processors like ConvertRecord and it occurs
> in NiFi 2.0 only.
> Based on my investigation the root cause seems to be related to the property
> migration.
> During the property migration the raw values("#\{Server URL}") are resolved
> to effective values (e.g. http://host:port).
> In StandardProcessorNode.migrateProperties() the properties are overwritten (
> overwriteProperties) if the propertyConfig is modified.
> At this step the effective properties are used while I belive the raw
> properties should have been used.
> Another observed minor issue:
> In InvokeHTTP processor, the "Connection Timeout" is renamed to the same name
> (that triggered the above mentioned issue since it looked as if the processor
> properties were modified, property overwrite was needed)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12531) Parameter references are removed after property migration
[
https://issues.apache.org/jira/browse/NIFI-12531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12531:
--
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Parameter references are removed after property migration
> -
>
> Key: NIFI-12531
> URL: https://issues.apache.org/jira/browse/NIFI-12531
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M1, 2.0.0
>Reporter: Zsihovszki Krisztina
>Assignee: Zsihovszki Krisztina
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Parameter references disappear from the properties when a flow is imported to
> NiFi 2.0.
> The problem was observed for InvokeHTTP processor when e.g. we extract "Host
> URL" property to a parameter. The flow definition contains the parameter
> reference properly:
>
> {code:java}
> "HTTP URL": "#{Server URL}",{code}
>
> but the property value is resolved in the imported flow and the connection
> between the parameter ("Server URL") and the processor is lost.
> The issue did not occur for other processors like ConvertRecord and it occurs
> in NiFi 2.0 only.
> Based on my investigation the root cause seems to be related to the property
> migration.
> During the property migration the raw values("#\{Server URL}") are resolved
> to effective values (e.g. http://host:port).
> In StandardProcessorNode.migrateProperties() the properties are overwritten (
> overwriteProperties) if the propertyConfig is modified.
> At this step the effective properties are used while I belive the raw
> properties should have been used.
> Another observed minor issue:
> In InvokeHTTP processor, the "Connection Timeout" is renamed to the same name
> (that triggered the above mentioned issue since it looked as if the processor
> properties were modified, property overwrite was needed)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12531 Parameter references are removed after property migration [nifi]
asfgit closed pull request #8176: NIFI-12531 Parameter references are removed after property migration URL: https://github.com/apache/nifi/pull/8176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12543) Introduce pages for managing Users/Groups
[ https://issues.apache.org/jira/browse/NIFI-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-12543: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Introduce pages for managing Users/Groups > - > > Key: NIFI-12543 > URL: https://issues.apache.org/jira/browse/NIFI-12543 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Introduce pages for managing Users/Groups. This includes creation, listing, > configuration, deleting, and viewing the access policies for users and user > groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12531 Parameter references are removed after property migration [nifi]
markap14 commented on PR #8176: URL: https://github.com/apache/nifi/pull/8176#issuecomment-1874240841 Thanks @krisztina-zsihovszki looks good to me as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12543) Introduce pages for managing Users/Groups
[ https://issues.apache.org/jira/browse/NIFI-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801842#comment-17801842 ] ASF subversion and git services commented on NIFI-12543: Commit 76f880588fd804af17f0c126fd2f55dca627d3e0 in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=76f880588f ] NIFI-12543: Users/User Groups (#8191) * NIFI-12543: - Users/User Groups. * NIFI-12543: - Users/User Groups Deletion. - Establishing routes for selection, editing, and access policies. * NIFI-12543: - User/User Group Creation. - User/User Group Configuration. - Renaming existing User State to Current User State. * NIFI-12543: - User access policies table. * NIFI-12543: - Sorting users/groups in the edit dialog. * NIFI-12543: - Addressing review feedback. This closes #8191 > Introduce pages for managing Users/Groups > - > > Key: NIFI-12543 > URL: https://issues.apache.org/jira/browse/NIFI-12543 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Introduce pages for managing Users/Groups. This includes creation, listing, > configuration, deleting, and viewing the access policies for users and user > groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12543: Users/User Groups [nifi]
rfellows merged PR #8191: URL: https://github.com/apache/nifi/pull/8191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12559) Upgrade SSHJ to 0.38.0
[ https://issues.apache.org/jira/browse/NIFI-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12559: Status: Patch Available (was: Open) > Upgrade SSHJ to 0.38.0 > -- > > Key: NIFI-12559 > URL: https://issues.apache.org/jira/browse/NIFI-12559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > SSHJ should be upgraded to version > [0.38.0|https://github.com/hierynomus/sshj#release-history] to incorporate > mitigations for the Terrapin vulnerability described in CVE-2023-48795, > applicable to SFTP Processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12559 Upgrade SSHJ from 0.37.0 to 0.38.0 [nifi]
exceptionfactory opened a new pull request, #8200: URL: https://github.com/apache/nifi/pull/8200 # Summary [NIFI-12559](https://issues.apache.org/jira/browse/NIFI-12559) Upgrades SSHJ from 0.37.0 to 0.38.0, incorporating mitigations for the Terrapin vulnerability described in CVE-2023-48795. This upgrade applies to SFTP Processors. Additional changes include removing exclusions for Bouncy Castle JDK 1.5 libraries that are no longer required for recent versions of SSHJ. This pull request is targeted to the main and branch and should be backported to the support branch. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12559) Upgrade SSHJ to 0.38.0
[ https://issues.apache.org/jira/browse/NIFI-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12559: Summary: Upgrade SSHJ to 0.38.0 (was: Upgraded SSHJ to 0.38.0) > Upgrade SSHJ to 0.38.0 > -- > > Key: NIFI-12559 > URL: https://issues.apache.org/jira/browse/NIFI-12559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: backport-needed > Fix For: 1.25.0, 2.0.0 > > > SSHJ should be upgraded to version > [0.38.0|https://github.com/hierynomus/sshj#release-history] to incorporate > mitigations for the Terrapin vulnerability described in CVE-2023-48795, > applicable to SFTP Processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12559) Upgraded SSHJ to 0.38.0
David Handermann created NIFI-12559: --- Summary: Upgraded SSHJ to 0.38.0 Key: NIFI-12559 URL: https://issues.apache.org/jira/browse/NIFI-12559 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 1.25.0, 2.0.0 SSHJ should be upgraded to version [0.38.0|https://github.com/hierynomus/sshj#release-history] to incorporate mitigations for the Terrapin vulnerability described in CVE-2023-48795, applicable to SFTP Processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12558) Upgrade Jagged to 0.3.0
[ https://issues.apache.org/jira/browse/NIFI-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12558: Status: Patch Available (was: Open) > Upgrade Jagged to 0.3.0 > --- > > Key: NIFI-12558 > URL: https://issues.apache.org/jira/browse/NIFI-12558 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Time Spent: 10m > Remaining Estimate: 0h > > The Jagged library should be upgraded to version > [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both > main and support branches to resolve an issue with concurrent access to > KeyAgreement instances that presents itself when configuring the > DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12558 Upgrade Jagged from 0.2.0 to 0.3.0 [nifi]
exceptionfactory opened a new pull request, #8199: URL: https://github.com/apache/nifi/pull/8199 # Summary [NIFI-12558](https://issues.apache.org/jira/browse/NIFI-12558) Upgrades Jagged libraries from 0.2.0 to [0.3.0](https://github.com/exceptionfactory/jagged/releases/tag/0.3.0) for the `EncryptContentAge` and `DecryptContentAge` Processors. Version 0.3.0 corrects an issue with concurrent usage of KeyAgreement objects that presents itself when configuring `DecryptContentAge` with more than one Concurrent Task. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-12558) Upgrade Jagged to 0.3.0
David Handermann created NIFI-12558: --- Summary: Upgrade Jagged to 0.3.0 Key: NIFI-12558 URL: https://issues.apache.org/jira/browse/NIFI-12558 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann The Jagged library should be upgraded to version [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both main and support branches to resolve an issue with concurrent access to KeyAgreement instances that presents itself when configuring the DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12558) Upgrade Jagged to 0.3.0
[ https://issues.apache.org/jira/browse/NIFI-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12558: Labels: backport-needed (was: ) > Upgrade Jagged to 0.3.0 > --- > > Key: NIFI-12558 > URL: https://issues.apache.org/jira/browse/NIFI-12558 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > > The Jagged library should be upgraded to version > [0.3.0|https://github.com/exceptionfactory/jagged/releases/tag/0.3.0] on both > main and support branches to resolve an issue with concurrent access to > KeyAgreement instances that presents itself when configuring the > DecryptContentAge Processor with more than one Concurrent Task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12524) AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade
[ https://issues.apache.org/jira/browse/NIFI-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12524: Fix Version/s: 1.25.0 2.0.0 > AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade > > > Key: NIFI-12524 > URL: https://issues.apache.org/jira/browse/NIFI-12524 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0 > Environment: RHEL 8, OpenJDK11 Latest >Reporter: Shawn Weeks >Assignee: David Handermann >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Attachments: nifi-app.log > > Time Spent: 10m > Remaining Estimate: 0h > > Seems to be an issue with the migration from H2 to Xodus. After upgrading > from 1.23.2 to 1.24.0 NiFi fails to start and throws a null pointer exception > with the auditService Bean. See attached log snippet. > The cluster started life as NiFi 1.13.2 and has been steadily upgraded since > mid 2021. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12524) AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade
[ https://issues.apache.org/jira/browse/NIFI-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12524: Affects Version/s: 1.24.0 Status: Patch Available (was: Open) > AuditService NullPointerException After 1.23.2 to 1.24.0 Upgrade > > > Key: NIFI-12524 > URL: https://issues.apache.org/jira/browse/NIFI-12524 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.24.0 > Environment: RHEL 8, OpenJDK11 Latest >Reporter: Shawn Weeks >Assignee: David Handermann >Priority: Major > Attachments: nifi-app.log > > Time Spent: 10m > Remaining Estimate: 0h > > Seems to be an issue with the migration from H2 to Xodus. After upgrading > from 1.23.2 to 1.24.0 NiFi fails to start and throws a null pointer exception > with the auditService Bean. See attached log snippet. > The cluster started life as NiFi 1.13.2 and has been steadily upgraded since > mid 2021. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-12524 Fixed null handling for auditing Connect Details [nifi]
exceptionfactory opened a new pull request, #8198: URL: https://github.com/apache/nifi/pull/8198 # Summary [NIFI-12524](https://issues.apache.org/jira/browse/NIFI-12524) Corrects handling of null property values in Flow Change Connect Details auditing objects. The original H2 database schema in NiFi 1.23 and required almost all columns to be populated, with the exception of the following fields in Connect Details: - Source Name - Destination Name - Relationship Changes include checking for null before attempting to set these properties on the Entity object, and unit tests to exercise both populated and null behavior. This corrects issues when migrating some existing Flow Configuration History from NiFi 1.23 and earlier. This pull request is targeted to the main branch, but is also ready for backporting to the support branch. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439542044
##
extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp:
##
@@ -0,0 +1,296 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "../PushGrafanaLokiREST.h"
+#include "MockGrafanaLoki.h"
+#include "SingleProcessorTestController.h"
+#include "Catch.h"
+#include "utils/StringUtils.h"
+#include "utils/TestUtils.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
+
+TEST_CASE("Url property is required", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::Url, ""));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/"));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::LogLineBatchSize, "1"));
+ REQUIRE_THROWS_AS(test_controller.trigger(), minifi::Exception);
+}
+
+TEST_CASE("Valid stream labels need to be set", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::Url, "localhost:10990"));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::LogLineBatchSize, "1"));
+ SECTION("Stream labels cannot be empty") {
+test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "");
+ }
+ SECTION("Stream labels need to be valid") {
+test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "invalidlabels,invalidlabels2");
+ }
+ REQUIRE_THROWS_AS(test_controller.trigger(), minifi::Exception);
+}
+
+TEST_CASE("Log Line Batch Size cannot be 0", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::Url, "localhost:10990"));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/"));
+ test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::LogLineBatchSize, "0");
+ REQUIRE_THROWS_AS(test_controller.trigger(), minifi::Exception);
+}
+
+class PushGrafanaLokiRESTTestFixture {
+ public:
+ PushGrafanaLokiRESTTestFixture()
+ : mock_loki_("10990"),
+
push_grafana_loki_rest_(std::make_shared("PushGrafanaLokiREST")),
+test_controller_(push_grafana_loki_rest_) {
+LogTestController::getInstance().setDebug();
+LogTestController::getInstance().setDebug();
+LogTestController::getInstance().setTrace();
+LogTestController::getInstance().setTrace();
+CHECK(test_controller_.plan->setProperty(push_grafana_loki_rest_,
PushGrafanaLokiREST::Url, "localhost:10990"));
+CHECK(test_controller_.plan->setProperty(push_grafana_loki_rest_,
PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/"));
+ }
+
+ void setProperty(const auto& property, const std::string& property_value) {
+CHECK(test_controller_.plan->setProperty(push_grafana_loki_rest_,
property, property_value));
+ }
+
+ void verifyLastRequestIsEmpty() {
+const auto& request = mock_loki_.getLastRequest();
+REQUIRE(request.IsNull());
+ }
+
+ void verifyTenantId(const std::string& tenant_id) {
+REQUIRE(mock_loki_.getLastTenantId() == tenant_id);
+ }
+
+ void verifyBasicAuthorization(const std::string&
expected_username_and_password) {
+auto last_authorization = mock_loki_.getLastAuthorization();
+std::string expected_authorization = "Basic ";
+REQUIRE(minifi::utils::StringUtils::startsWith(last_authorization,
expected_authorization))
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439541779
##
extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp:
##
@@ -0,0 +1,296 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "../PushGrafanaLokiREST.h"
+#include "MockGrafanaLoki.h"
+#include "SingleProcessorTestController.h"
+#include "Catch.h"
+#include "utils/StringUtils.h"
+#include "utils/TestUtils.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
+
+TEST_CASE("Url property is required", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::Url, ""));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/"));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::LogLineBatchSize, "1"));
Review Comment:
Removed the checks in 083c28ee59f96352e89d5675d635ea3b91bd9942
##
extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp:
##
@@ -0,0 +1,296 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "../PushGrafanaLokiREST.h"
+#include "MockGrafanaLoki.h"
+#include "SingleProcessorTestController.h"
+#include "Catch.h"
+#include "utils/StringUtils.h"
+#include "utils/TestUtils.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
+
+TEST_CASE("Url property is required", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::Url, ""));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/"));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::LogLineBatchSize, "1"));
+ REQUIRE_THROWS_AS(test_controller.trigger(), minifi::Exception);
+}
+
+TEST_CASE("Valid stream labels need to be set", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::Url, "localhost:10990"));
+ CHECK(test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::LogLineBatchSize, "1"));
+ SECTION("Stream labels cannot be empty") {
+test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "");
+ }
+ SECTION("Stream labels need to be valid") {
+test_controller.plan->setProperty(push_grafana_loki_rest,
PushGrafanaLokiREST::StreamLabels, "invalidlabels,invalidlabels2");
+ }
+ REQUIRE_THROWS_AS(test_controller.trigger(), minifi::Exception);
+}
+
+TEST_CASE("Log Line Batch Size cannot be 0", "[PushGrafanaLokiREST]") {
+ auto push_grafana_loki_rest =
std::make_shared("PushGrafanaLokiREST");
+ minifi::test::SingleProcessorTestController
test_controller(push_grafana_loki_rest);
+ CHECK(test_controller.plan->setPro
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439541507
##
extensions/grafana-loki/tests/MockGrafanaLoki.h:
##
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+#include "tests/CivetLibrary.h"
+#include "core/logging/Logger.h"
+#include "core/logging/LoggerConfiguration.h"
+#include "rapidjson/document.h"
+#include "rapidjson/writer.h"
+#include "rapidjson/stringbuffer.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
+
+class GrafanaLokiHandler : public CivetHandler {
+ public:
+ const rapidjson::Document& getLastRequest() const {
+return request_received_;
+ }
+
+ std::string getLastTenantId() const {
+return tenant_id_set_;
+ }
+
+ std::string getLastAuthorization() const {
+return authorization_set_;
+ }
+
+ private:
+ bool handlePost(CivetServer*, struct mg_connection* conn) override {
+tenant_id_set_.clear();
+authorization_set_.clear();
+const char *org_id = mg_get_header(conn, "X-Scope-OrgID");
+if (org_id != nullptr) {
+ tenant_id_set_ = org_id;
+}
+
+const char *authorization = mg_get_header(conn, "Authorization");
+if (authorization != nullptr) {
+ authorization_set_ = authorization;
+}
+
+std::array request;
+size_t chars_read = mg_read(conn, request.data(), 2048);
Review Comment:
Updated in 083c28ee59f96352e89d5675d635ea3b91bd9942
##
extensions/grafana-loki/tests/MockGrafanaLoki.h:
##
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+#include "tests/CivetLibrary.h"
+#include "core/logging/Logger.h"
+#include "core/logging/LoggerConfiguration.h"
+#include "rapidjson/document.h"
+#include "rapidjson/writer.h"
+#include "rapidjson/stringbuffer.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
+
+class GrafanaLokiHandler : public CivetHandler {
+ public:
+ const rapidjson::Document& getLastRequest() const {
+return request_received_;
+ }
+
+ std::string getLastTenantId() const {
+return tenant_id_set_;
+ }
+
+ std::string getLastAuthorization() const {
+return authorization_set_;
+ }
+
+ private:
+ bool handlePost(CivetServer*, struct mg_connection* conn) override {
+tenant_id_set_.clear();
+authorization_set_.clear();
+const char *org_id = mg_get_header(conn, "X-Scope-OrgID");
+if (org_id != nullptr) {
+ tenant_id_set_ = org_id;
+}
+
+const char *authorization = mg_get_header(conn, "Authorization");
+if (authorization != nullptr) {
+ authorization_set_ = authorization;
+}
+
+std::array request;
+size_t chars_read = mg_read(conn, request.data(), 2048);
+std::string json_str(request.data(), chars_read);
+request_received_.Parse(json_str.c_str());
+
+mg_printf(conn, "HTTP/1.1 204 OK\r\n");
+mg_printf(conn, "Content-length: 0");
+mg_printf(conn, "\r\n\r\n");
+return true;
+ }
+
+ rapidjson::Document request_received_;
+ std::string tenant_id_set_;
+ std::string authorization_set_;
+};
+
+class MockGrafanaLoki {
+ public:
+ explicit MockGrafanaLoki(std::string port) : port_(std::move(port)) {
+std::vector options;
+options.emplace_back("listening_ports");
+options.emplace_back(p
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439541257
##
extensions/grafana-loki/PushGrafanaLokiREST.cpp:
##
@@ -0,0 +1,396 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "PushGrafanaLokiREST.h"
+
+#include
+#include
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/ProcessorConfigUtils.h"
+#include "utils/StringUtils.h"
+#include "rapidjson/document.h"
+#include "rapidjson/stream.h"
+#include "rapidjson/writer.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki {
+
+void PushGrafanaLokiREST::LogBatch::add(const std::shared_ptr&
flowfile) {
+ gsl_Expects(state_manager_);
+ if (log_line_batch_wait_ && batched_flowfiles_.empty()) {
+start_push_time_ = std::chrono::steady_clock::now();
+std::unordered_map state;
+state["start_push_time"] =
std::to_string(std::chrono::duration_cast(start_push_time_.time_since_epoch()).count());
+logger_->log_debug("Saved start push time to state: {}",
state["start_push_time"]);
+state_manager_->set(state);
+ }
+ batched_flowfiles_.push_back(flowfile);
+}
+
+void PushGrafanaLokiREST::LogBatch::restore(const
std::shared_ptr& flowfile) {
+ batched_flowfiles_.push_back(flowfile);
+}
+
+std::vector>
PushGrafanaLokiREST::LogBatch::flush() {
+ gsl_Expects(state_manager_);
+ start_push_time_ = {};
+ auto result = batched_flowfiles_;
+ batched_flowfiles_.clear();
+ if (log_line_batch_wait_) {
+start_push_time_ = {};
+std::unordered_map state;
+logger_->log_debug("Reset start push time state");
+state["start_push_time"] = "0";
+state_manager_->set(state);
+ }
+ return result;
+}
+
+bool PushGrafanaLokiREST::LogBatch::isReady() const {
+ return (log_line_batch_size_ && batched_flowfiles_.size() >=
*log_line_batch_size_) || (log_line_batch_wait_ &&
std::chrono::steady_clock::now() - start_push_time_ >= *log_line_batch_wait_);
+}
+
+void
PushGrafanaLokiREST::LogBatch::setLogLineBatchSize(std::optional
log_line_batch_size) {
+ log_line_batch_size_ = log_line_batch_size;
+}
+
+void
PushGrafanaLokiREST::LogBatch::setLogLineBatchWait(std::optional
log_line_batch_wait) {
+ log_line_batch_wait_ = log_line_batch_wait;
+}
+
+void PushGrafanaLokiREST::LogBatch::setStateManager(core::StateManager*
state_manager) {
+ state_manager_ = state_manager;
+}
+
+void
PushGrafanaLokiREST::LogBatch::setStartPushTime(std::chrono::steady_clock::time_point
start_push_time) {
+ start_push_time_ = start_push_time;
+}
+
+const core::Relationship PushGrafanaLokiREST::Self("__self__", "Marks the
FlowFile to be owned by this processor");
+
+void PushGrafanaLokiREST::initialize() {
+ setSupportedProperties(Properties);
+ setSupportedRelationships(Relationships);
+}
+
+namespace {
+auto getSSLContextService(core::ProcessContext& context) {
+ if (auto ssl_context =
context.getProperty(PushGrafanaLokiREST::SSLContextService)) {
+return
std::dynamic_pointer_cast(context.getControllerService(*ssl_context));
+ }
+ return std::shared_ptr{};
+}
+
+std::string readLogLineFromFlowFile(const std::shared_ptr&
flow_file, core::ProcessSession& session) {
+ auto read_buffer_result = session.readBuffer(flow_file);
+ return {reinterpret_cast(read_buffer_result.buffer.data()),
read_buffer_result.buffer.size()};
+}
+} // namespace
+
+void PushGrafanaLokiREST::setUpStateManager(core::ProcessContext& context) {
+ auto state_manager = context.getStateManager();
+ if (state_manager == nullptr) {
+throw Exception(PROCESSOR_EXCEPTION, "Failed to get StateManager");
+ }
+ log_batch_.setStateManager(state_manager);
+
+ std::unordered_map state_map;
+ if (state_manager->get(state_map)) {
+auto it = state_map.find("start_push_time");
+if (it != state_map.end()) {
+ logger_->log_info("Restored start push time from processor state: {}",
it->second);
+ std::chrono::steady_clock::time_point
start_push_time{std::chrono::milliseconds{std::stoll(it->second)}};
+ log_batch_.setStartPushTime(start_push_time);
+}
+ }
+}
+
+void PushGrafanaLokiREST::setUpStreamLabels(core::ProcessConte
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439540650
##
extensions/grafana-loki/PushGrafanaLokiREST.cpp:
##
@@ -0,0 +1,396 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "PushGrafanaLokiREST.h"
+
+#include
+#include
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/ProcessorConfigUtils.h"
+#include "utils/StringUtils.h"
+#include "rapidjson/document.h"
+#include "rapidjson/stream.h"
+#include "rapidjson/writer.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki {
+
+void PushGrafanaLokiREST::LogBatch::add(const std::shared_ptr&
flowfile) {
+ gsl_Expects(state_manager_);
+ if (log_line_batch_wait_ && batched_flowfiles_.empty()) {
+start_push_time_ = std::chrono::steady_clock::now();
+std::unordered_map state;
+state["start_push_time"] =
std::to_string(std::chrono::duration_cast(start_push_time_.time_since_epoch()).count());
+logger_->log_debug("Saved start push time to state: {}",
state["start_push_time"]);
+state_manager_->set(state);
+ }
+ batched_flowfiles_.push_back(flowfile);
+}
+
+void PushGrafanaLokiREST::LogBatch::restore(const
std::shared_ptr& flowfile) {
+ batched_flowfiles_.push_back(flowfile);
+}
+
+std::vector>
PushGrafanaLokiREST::LogBatch::flush() {
+ gsl_Expects(state_manager_);
+ start_push_time_ = {};
+ auto result = batched_flowfiles_;
+ batched_flowfiles_.clear();
+ if (log_line_batch_wait_) {
+start_push_time_ = {};
+std::unordered_map state;
+logger_->log_debug("Reset start push time state");
+state["start_push_time"] = "0";
+state_manager_->set(state);
+ }
+ return result;
+}
+
+bool PushGrafanaLokiREST::LogBatch::isReady() const {
+ return (log_line_batch_size_ && batched_flowfiles_.size() >=
*log_line_batch_size_) || (log_line_batch_wait_ &&
std::chrono::steady_clock::now() - start_push_time_ >= *log_line_batch_wait_);
+}
+
+void
PushGrafanaLokiREST::LogBatch::setLogLineBatchSize(std::optional
log_line_batch_size) {
+ log_line_batch_size_ = log_line_batch_size;
+}
+
+void
PushGrafanaLokiREST::LogBatch::setLogLineBatchWait(std::optional
log_line_batch_wait) {
+ log_line_batch_wait_ = log_line_batch_wait;
+}
+
+void PushGrafanaLokiREST::LogBatch::setStateManager(core::StateManager*
state_manager) {
+ state_manager_ = state_manager;
+}
+
+void
PushGrafanaLokiREST::LogBatch::setStartPushTime(std::chrono::steady_clock::time_point
start_push_time) {
+ start_push_time_ = start_push_time;
+}
+
+const core::Relationship PushGrafanaLokiREST::Self("__self__", "Marks the
FlowFile to be owned by this processor");
+
+void PushGrafanaLokiREST::initialize() {
+ setSupportedProperties(Properties);
+ setSupportedRelationships(Relationships);
+}
+
+namespace {
+auto getSSLContextService(core::ProcessContext& context) {
+ if (auto ssl_context =
context.getProperty(PushGrafanaLokiREST::SSLContextService)) {
+return
std::dynamic_pointer_cast(context.getControllerService(*ssl_context));
+ }
+ return std::shared_ptr{};
+}
+
+std::string readLogLineFromFlowFile(const std::shared_ptr&
flow_file, core::ProcessSession& session) {
+ auto read_buffer_result = session.readBuffer(flow_file);
+ return {reinterpret_cast(read_buffer_result.buffer.data()),
read_buffer_result.buffer.size()};
+}
+} // namespace
+
+void PushGrafanaLokiREST::setUpStateManager(core::ProcessContext& context) {
+ auto state_manager = context.getStateManager();
+ if (state_manager == nullptr) {
+throw Exception(PROCESSOR_EXCEPTION, "Failed to get StateManager");
+ }
+ log_batch_.setStateManager(state_manager);
+
+ std::unordered_map state_map;
+ if (state_manager->get(state_map)) {
+auto it = state_map.find("start_push_time");
+if (it != state_map.end()) {
+ logger_->log_info("Restored start push time from processor state: {}",
it->second);
+ std::chrono::steady_clock::time_point
start_push_time{std::chrono::milliseconds{std::stoll(it->second)}};
+ log_batch_.setStartPushTime(start_push_time);
Review Comment:
Good catch, I would like to keep the state after restart, so
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439539966
##
extensions/grafana-loki/PushGrafanaLokiREST.cpp:
##
@@ -0,0 +1,396 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "PushGrafanaLokiREST.h"
+
+#include
+#include
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+#include "utils/ProcessorConfigUtils.h"
+#include "utils/StringUtils.h"
+#include "rapidjson/document.h"
+#include "rapidjson/stream.h"
+#include "rapidjson/writer.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki {
+
+void PushGrafanaLokiREST::LogBatch::add(const std::shared_ptr&
flowfile) {
+ gsl_Expects(state_manager_);
+ if (log_line_batch_wait_ && batched_flowfiles_.empty()) {
+start_push_time_ = std::chrono::steady_clock::now();
+std::unordered_map state;
+state["start_push_time"] =
std::to_string(std::chrono::duration_cast(start_push_time_.time_since_epoch()).count());
+logger_->log_debug("Saved start push time to state: {}",
state["start_push_time"]);
+state_manager_->set(state);
+ }
+ batched_flowfiles_.push_back(flowfile);
+}
+
+void PushGrafanaLokiREST::LogBatch::restore(const
std::shared_ptr& flowfile) {
+ batched_flowfiles_.push_back(flowfile);
+}
+
+std::vector>
PushGrafanaLokiREST::LogBatch::flush() {
+ gsl_Expects(state_manager_);
+ start_push_time_ = {};
+ auto result = batched_flowfiles_;
+ batched_flowfiles_.clear();
+ if (log_line_batch_wait_) {
+start_push_time_ = {};
+std::unordered_map state;
+logger_->log_debug("Reset start push time state");
+state["start_push_time"] = "0";
+state_manager_->set(state);
+ }
+ return result;
+}
+
+bool PushGrafanaLokiREST::LogBatch::isReady() const {
+ return (log_line_batch_size_ && batched_flowfiles_.size() >=
*log_line_batch_size_) || (log_line_batch_wait_ &&
std::chrono::steady_clock::now() - start_push_time_ >= *log_line_batch_wait_);
+}
+
+void
PushGrafanaLokiREST::LogBatch::setLogLineBatchSize(std::optional
log_line_batch_size) {
+ log_line_batch_size_ = log_line_batch_size;
+}
+
+void
PushGrafanaLokiREST::LogBatch::setLogLineBatchWait(std::optional
log_line_batch_wait) {
+ log_line_batch_wait_ = log_line_batch_wait;
+}
+
+void PushGrafanaLokiREST::LogBatch::setStateManager(core::StateManager*
state_manager) {
+ state_manager_ = state_manager;
+}
+
+void
PushGrafanaLokiREST::LogBatch::setStartPushTime(std::chrono::steady_clock::time_point
start_push_time) {
+ start_push_time_ = start_push_time;
+}
+
+const core::Relationship PushGrafanaLokiREST::Self("__self__", "Marks the
FlowFile to be owned by this processor");
+
+void PushGrafanaLokiREST::initialize() {
+ setSupportedProperties(Properties);
+ setSupportedRelationships(Relationships);
+}
+
+namespace {
+auto getSSLContextService(core::ProcessContext& context) {
+ if (auto ssl_context =
context.getProperty(PushGrafanaLokiREST::SSLContextService)) {
+return
std::dynamic_pointer_cast(context.getControllerService(*ssl_context));
+ }
+ return std::shared_ptr{};
+}
+
+std::string readLogLineFromFlowFile(const std::shared_ptr&
flow_file, core::ProcessSession& session) {
+ auto read_buffer_result = session.readBuffer(flow_file);
+ return {reinterpret_cast(read_buffer_result.buffer.data()),
read_buffer_result.buffer.size()};
Review Comment:
Updated with the latter in 083c28ee59f96352e89d5675d635ea3b91bd9942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439539574
##
extensions/grafana-loki/PushGrafanaLokiREST.h:
##
@@ -0,0 +1,178 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+#include
+
+#include "controllers/SSLContextService.h"
+#include "core/Processor.h"
+#include "core/PropertyDefinition.h"
+#include "core/PropertyDefinitionBuilder.h"
+#include "core/PropertyType.h"
+#include "core/RelationshipDefinition.h"
+#include "client/HTTPClient.h"
+#include "core/StateManager.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki {
+
+class PushGrafanaLokiREST : public core::Processor {
+ public:
+ EXTENSIONAPI static constexpr const char* Description = "A Grafana Loki push
processor that uses the Grafana Loki REST API. The processor expects each flow
file to contain a single log line to be "
+ "pushed to Grafana
Loki, therefore it is usually used together with the TailFile processor.";
+
+ explicit PushGrafanaLokiREST(const std::string& name, const
utils::Identifier& uuid = {})
+ : Processor(name, uuid),
+log_batch_(logger_) {
+ }
+ ~PushGrafanaLokiREST() override = default;
+
+ EXTENSIONAPI static constexpr auto Url =
core::PropertyDefinitionBuilder<>::createProperty("Url")
+.withDescription("Url of the Grafana Loki server. For example
http://localhost:3100/.";)
+.isRequired(true)
+.build();
+ EXTENSIONAPI static constexpr auto StreamLabels =
core::PropertyDefinitionBuilder<>::createProperty("Stream Labels")
+.withDescription("Comma separated list of = labels to be sent
as stream labels.")
+.isRequired(true)
+.build();
+ EXTENSIONAPI static constexpr auto LogLineMetadataAttributes =
core::PropertyDefinitionBuilder<>::createProperty("Log Line Metadata
Attributes")
+.withDescription("Comma separated list of attributes to be sent as log
line metadata for a log line.")
+.build();
+ EXTENSIONAPI static constexpr auto TenantID =
core::PropertyDefinitionBuilder<>::createProperty("Tenant ID")
+.withDescription("The tenant ID used by default to push logs to Grafana
Loki. If omitted or empty it assumes Grafana Loki is running in single-tenant
mode and no X-Scope-OrgID header is sent.")
+.build();
+ EXTENSIONAPI static constexpr auto MaxBatchSize =
core::PropertyDefinitionBuilder<>::createProperty("Max Batch Size")
+.withDescription("The maximum number of flow files to process at a time.
If not set, or set to 0, all FlowFiles will be processed at once.")
+.withPropertyType(core::StandardPropertyTypes::UNSIGNED_LONG_TYPE)
+.withDefaultValue("100")
+.build();
+ EXTENSIONAPI static constexpr auto LogLineBatchWait =
core::PropertyDefinitionBuilder<>::createProperty("Log Line Batch Wait")
+.withDescription("Time to wait before sending a log line batch to Grafana
Loki, full or not. If this property and Log Line Batch Size are both unset, "
+ "the log batch of the current trigger will be sent
immediately.")
+.withPropertyType(core::StandardPropertyTypes::TIME_PERIOD_TYPE)
+.build();
+ EXTENSIONAPI static constexpr auto LogLineBatchSize =
core::PropertyDefinitionBuilder<>::createProperty("Log Line Batch Size")
+.withDescription("Number of log lines to send in a batch to Loki. If this
property and Log Line Batch Wait are both unset, "
+ "the log batch of the current trigger will be sent
immediately.")
+.withPropertyType(core::StandardPropertyTypes::UNSIGNED_INT_TYPE)
+.build();
+ EXTENSIONAPI static constexpr auto ConnectTimeout =
core::PropertyDefinitionBuilder<>::createProperty("Connection Timeout")
+.withDescription("Max wait time for connection to the Grafana Loki
service.")
+.withPropertyType(core::StandardPropertyTypes::TIME_PERIOD_TYPE)
+.withDefaultValue("5 s")
+.isRequired(true)
+.build();
+ EXTENSIONAPI static constexpr auto ReadTimeout =
core::PropertyDefinitionBuilder<>::createProperty("Read Timeout")
+.withDescription("Max wait time for response from remote service
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439539348
##
bootstrap.sh:
##
@@ -339,6 +339,9 @@ add_option PROMETHEUS_ENABLED ${TRUE} "ENABLE_PROMETHEUS"
add_option OPENSSL_ENABLED ${TRUE} "OPENSSL_OFF"
add_dependency OPENSSL_ENABLED "opensslbuild"
+add_option GRAFANA_LOKI_ENABLED ${TRUE} "ENABLE_GRAFANA_LOKI"
+set_dependency GRAFANA_LOKI_ENABLED HTTP_CURL_ENABLED
Review Comment:
It should be false, updated in 083c28ee59f96352e89d5675d635ea3b91bd9942
##
docker/test/integration/cluster/checkers/GrafanaLokiChecker.py:
##
@@ -0,0 +1,54 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import requests
+from typing import List
+from utils import wait_for
+
+
+class GrafanaLokiChecker:
+def __init__(self):
+self.url = "localhost:3100/loki/api/v1/query"
+
+def veify_log_lines_on_grafana_loki(self, lines: List[str], ssl: bool,
tenant_id: str):
+labels = '{job="minifi"}'
+prefix = "http://";
+if ssl:
+prefix = "https://";
+
+query_url = f"{prefix}{self.url}?query={labels}"
+
+headers = None
+if tenant_id:
+headers = {'X-Scope-OrgID': tenant_id}
+
+response = requests.get(query_url, verify=False, timeout=30,
headers=headers)
+if response.status_code >= 200 and response.status_code < 300:
Review Comment:
Updated in 083c28ee59f96352e89d5675d635ea3b91bd9942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12400) Remaining items to migrate UI to currently supported/active framework
[ https://issues.apache.org/jira/browse/NIFI-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-12400: --- Description: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) * Bulletin Board * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) * Configure Reporting Task * Flow Analysis Rules * Registry Clients (NIFI-12486) * Import from Registry * Parameter Providers * Cluster * Flow Configuration History * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) * Policies (NIFI-12548) * Help * About * Show Upstream/Downstream * Align * List Queue * Empty [all] Queue * View Content * View State * Change Version * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) * Download Flow * Create RPG * Configure RPG * RPG Remote Ports * RPG Go To * Color * Move to Front * Copy/Paste * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmation * Update API calls to use uiOnly flag * Routing error handling * Prevent navigate to Service when configuration form is dirty * Introduce header in new pages to unify with canvas and offer better navigation. * Prompt user to save Parameter Context when Edit form is dirty [1] [https://github.com/apache/nifi/pull/8053] was: The purpose of this Jira is to track all remaining items following the initial commit [1] for NIFI-11481. The description will be kept up to date with remaining features, tasks, and improvements. As each items is worked, a new sub task Jira will be created and referenced in this description. * Support Parameters in Properties with Allowable Values (NIFI-12401) * Summary (NIFI-12437) ** Remaining work not addressed in initial Jira: *** input ports (NIFI-12504) *** output ports (NIFI-12504) *** remote process groups (NIFI-12504) *** process groups (NIFI-12504) *** connections (NIFI-12504) *** System Diagnostics (NIFI-12505) *** support for cluster-specific ui elements (NIFI-12537) *** Add pagination (NIFI-12552) * Counters (NIFI-12415) * Bulletin Board * Provenance (NIFI-12445) ** Event Listing (NIFI-12445) ** Search (NIFI-12445) ** Event Dialog (NIFI-12445) ** Lineage (NIFI-12485) ** Replay from context menu (NIFI-12445) * Flow Analysis Rules * Registry Clients (NIFI-12486) * Import from Registry * Parameter Providers * Cluster * Flow Configuration History * Node Status History (NIFI-12553) * Status history for components from canvas context menu (NIFI-12553) * Users (NIFI-12543) * Policies (NIFI-12548) * Help * About * Show Upstream/Downstream * Align * List Queue * Empty [all] Queue * View Content * View State * Change Version * PG Version ** Start ** Commit ** Force Commit ** Show changes ** Revert changes ** Change Flow version ** Stop * Configure PG (NIFI-12417) * Process Group Services (NIFI-12425) ** Listing (NIFI-12425) ** Create (NIFI-12425) ** Configure (NIFI-12425) ** Delete (NIFI-12425) ** Enable (NIFI-12529) ** Disable (NIFI-12529) ** Improve layout and breadcrumbs * Configure Processor ** Service Link (NIFI-12425) ** Create inline Service (NIFI-12425) ** Parameter Link (NIFI-12502) ** Convert to Parameter (NIFI-12502) ** Fix issue with Property Editor width (NIFI-12547) * Download Flow * Create RPG * Configure RPG * RPG Remote Ports * RPG Go To * Color * Move to Front * Copy/Paste * Run unit tests are part of standard build * Update all API calls to consider disconnect node confirmatio
Re: [PR] [NIFI-12553] - Context menu option for View Status History [nifi]
mcgilman commented on code in PR #8193:
URL: https://github.com/apache/nifi/pull/8193#discussion_r1439514520
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -102,6 +103,8 @@ import { YesNoDialog } from
'../../../../ui/common/yes-no-dialog/yes-no-dialog.c
import { EditParameterDialog } from
'../../../../ui/common/edit-parameter-dialog/edit-parameter-dialog.component';
import { selectParameterSaving } from '../parameter/parameter.selectors';
import { ParameterService } from '../../service/parameter.service';
+import { navigateToViewStatusHistoryForComponent } from './flow.actions';
+import { dispatch } from 'd3';
Review Comment:
This appears to have been inadvertently imported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12553] - Context menu option for View Status History [nifi]
mcgilman commented on PR #8193: URL: https://github.com/apache/nifi/pull/8193#issuecomment-1874100197 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12526) QueryCassandra should not output FlowFiles as soon as the "available rows without fetching" is reached
[ https://issues.apache.org/jira/browse/NIFI-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12526: -- Resolution: Fixed Status: Resolved (was: Patch Available) > QueryCassandra should not output FlowFiles as soon as the "available rows > without fetching" is reached > -- > > Key: NIFI-12526 > URL: https://issues.apache.org/jira/browse/NIFI-12526 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > NIFI-5642 introduced the Max Rows Per Flow File and Output Batch Size > properties to QueryCassandra, but still uses the internal > "rowsAvailableWithoutFetching" variable (whose value comes from the Cassandra > ResultSet and defaults to 5000) as a trigger to stop processing rows for a > FlowFile. This can cause unexpected behavior, such as getting multiple > FlowFiles when only one is expected. > Also since those properties were added, the fragment.* attributes should be > added and populated accordingly (such as ExecuteSQL does). > NIFI-5642 also removes the Compression Type property, which might be ok for > 2.x but will cause all flows using this property in 1.x to become invalid. On > the support (1.x) branch we need to add the property back in, perhaps we can > keep it removed for 2.x but we'd want to remove the Cassandra Connect Points > property and such to force the user to use a Cassandra Connection controller > service. The reason to add the property back is if the Cassandra Contact > Points property is used instead of the Cassandra Connection controller > service, there is no way to set the Compression Type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12526) QueryCassandra should not output FlowFiles as soon as the "available rows without fetching" is reached
[ https://issues.apache.org/jira/browse/NIFI-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801799#comment-17801799 ] ASF subversion and git services commented on NIFI-12526: Commit f3b38ddd40e54bb85f8b582f62dd8c7a66322a3a in nifi's branch refs/heads/support/nifi-1.x from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f3b38ddd40 ] NIFI-12526: Fixed handling of Fetch Size in QueryCassandra, added fragment attributes Signed-off-by: Pierre Villard This closes #8183. > QueryCassandra should not output FlowFiles as soon as the "available rows > without fetching" is reached > -- > > Key: NIFI-12526 > URL: https://issues.apache.org/jira/browse/NIFI-12526 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > NIFI-5642 introduced the Max Rows Per Flow File and Output Batch Size > properties to QueryCassandra, but still uses the internal > "rowsAvailableWithoutFetching" variable (whose value comes from the Cassandra > ResultSet and defaults to 5000) as a trigger to stop processing rows for a > FlowFile. This can cause unexpected behavior, such as getting multiple > FlowFiles when only one is expected. > Also since those properties were added, the fragment.* attributes should be > added and populated accordingly (such as ExecuteSQL does). > NIFI-5642 also removes the Compression Type property, which might be ok for > 2.x but will cause all flows using this property in 1.x to become invalid. On > the support (1.x) branch we need to add the property back in, perhaps we can > keep it removed for 2.x but we'd want to remove the Cassandra Connect Points > property and such to force the user to use a Cassandra Connection controller > service. The reason to add the property back is if the Cassandra Contact > Points property is used instead of the Cassandra Connection controller > service, there is no way to set the Compression Type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12526) QueryCassandra should not output FlowFiles as soon as the "available rows without fetching" is reached
[ https://issues.apache.org/jira/browse/NIFI-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17801798#comment-17801798 ] ASF subversion and git services commented on NIFI-12526: Commit aaa812b1b5bb23451238aee6d8ce0d9a61127bee in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=aaa812b1b5 ] NIFI-12526: Fixed handling of Fetch Size in QueryCassandra, added fragment attributes Signed-off-by: Pierre Villard This closes #8183. > QueryCassandra should not output FlowFiles as soon as the "available rows > without fetching" is reached > -- > > Key: NIFI-12526 > URL: https://issues.apache.org/jira/browse/NIFI-12526 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.25.0, 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-5642 introduced the Max Rows Per Flow File and Output Batch Size > properties to QueryCassandra, but still uses the internal > "rowsAvailableWithoutFetching" variable (whose value comes from the Cassandra > ResultSet and defaults to 5000) as a trigger to stop processing rows for a > FlowFile. This can cause unexpected behavior, such as getting multiple > FlowFiles when only one is expected. > Also since those properties were added, the fragment.* attributes should be > added and populated accordingly (such as ExecuteSQL does). > NIFI-5642 also removes the Compression Type property, which might be ok for > 2.x but will cause all flows using this property in 1.x to become invalid. On > the support (1.x) branch we need to add the property back in, perhaps we can > keep it removed for 2.x but we'd want to remove the Cassandra Connect Points > property and such to force the user to use a Cassandra Connection controller > service. The reason to add the property back is if the Cassandra Contact > Points property is used instead of the Cassandra Connection controller > service, there is no way to set the Compression Type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12526: Fixed handling of Fetch Size in QueryCassandra, added fragment attributes [nifi]
asfgit closed pull request #8183: NIFI-12526: Fixed handling of Fetch Size in QueryCassandra, added fragment attributes URL: https://github.com/apache/nifi/pull/8183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12538 - Add map Record Standalone Function [nifi]
pvillard31 commented on PR #8182: URL: https://github.com/apache/nifi/pull/8182#issuecomment-1874077031 Thanks for the review @exceptionfactory, your comments make a lot of sense and I pushed a commit with the changes you recommended. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2261 Add processor for pushing logs to Grafana Loki through REST API [nifi-minifi-cpp]
lordgamez commented on code in PR #1695:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1695#discussion_r1439438776
##
extensions/grafana-loki/PushGrafanaLokiREST.h:
##
@@ -0,0 +1,178 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+#include
+
+#include "controllers/SSLContextService.h"
+#include "core/Processor.h"
+#include "core/PropertyDefinition.h"
+#include "core/PropertyDefinitionBuilder.h"
+#include "core/PropertyType.h"
+#include "core/RelationshipDefinition.h"
+#include "client/HTTPClient.h"
+#include "core/StateManager.h"
+
+namespace org::apache::nifi::minifi::extensions::grafana::loki {
+
+class PushGrafanaLokiREST : public core::Processor {
+ public:
+ EXTENSIONAPI static constexpr const char* Description = "A Grafana Loki push
processor that uses the Grafana Loki REST API. The processor expects each flow
file to contain a single log line to be "
+ "pushed to Grafana
Loki, therefore it is usually used together with the TailFile processor.";
+
+ explicit PushGrafanaLokiREST(const std::string& name, const
utils::Identifier& uuid = {})
+ : Processor(name, uuid),
+log_batch_(logger_) {
+ }
+ ~PushGrafanaLokiREST() override = default;
+
+ EXTENSIONAPI static constexpr auto Url =
core::PropertyDefinitionBuilder<>::createProperty("Url")
+.withDescription("Url of the Grafana Loki server. For example
http://localhost:3100/.";)
+.isRequired(true)
+.build();
+ EXTENSIONAPI static constexpr auto StreamLabels =
core::PropertyDefinitionBuilder<>::createProperty("Stream Labels")
+.withDescription("Comma separated list of = labels to be sent
as stream labels.")
+.isRequired(true)
+.build();
+ EXTENSIONAPI static constexpr auto LogLineMetadataAttributes =
core::PropertyDefinitionBuilder<>::createProperty("Log Line Metadata
Attributes")
+.withDescription("Comma separated list of attributes to be sent as log
line metadata for a log line.")
+.build();
+ EXTENSIONAPI static constexpr auto TenantID =
core::PropertyDefinitionBuilder<>::createProperty("Tenant ID")
+.withDescription("The tenant ID used by default to push logs to Grafana
Loki. If omitted or empty it assumes Grafana Loki is running in single-tenant
mode and no X-Scope-OrgID header is sent.")
+.build();
+ EXTENSIONAPI static constexpr auto MaxBatchSize =
core::PropertyDefinitionBuilder<>::createProperty("Max Batch Size")
+.withDescription("The maximum number of flow files to process at a time.
If not set, or set to 0, all FlowFiles will be processed at once.")
+.withPropertyType(core::StandardPropertyTypes::UNSIGNED_LONG_TYPE)
+.withDefaultValue("100")
+.build();
+ EXTENSIONAPI static constexpr auto LogLineBatchWait =
core::PropertyDefinitionBuilder<>::createProperty("Log Line Batch Wait")
+.withDescription("Time to wait before sending a log line batch to Grafana
Loki, full or not. If this property and Log Line Batch Size are both unset, "
+ "the log batch of the current trigger will be sent
immediately.")
+.withPropertyType(core::StandardPropertyTypes::TIME_PERIOD_TYPE)
+.build();
+ EXTENSIONAPI static constexpr auto LogLineBatchSize =
core::PropertyDefinitionBuilder<>::createProperty("Log Line Batch Size")
+.withDescription("Number of log lines to send in a batch to Loki. If this
property and Log Line Batch Wait are both unset, "
+ "the log batch of the current trigger will be sent
immediately.")
+.withPropertyType(core::StandardPropertyTypes::UNSIGNED_INT_TYPE)
+.build();
+ EXTENSIONAPI static constexpr auto ConnectTimeout =
core::PropertyDefinitionBuilder<>::createProperty("Connection Timeout")
+.withDescription("Max wait time for connection to the Grafana Loki
service.")
+.withPropertyType(core::StandardPropertyTypes::TIME_PERIOD_TYPE)
+.withDefaultValue("5 s")
+.isRequired(true)
+.build();
+ EXTENSIONAPI static constexpr auto ReadTimeout =
core::PropertyDefinitionBuilder<>::createProperty("Read Timeout")
+.withDescription("Max wait time for response from remote service
Re: [PR] MINIFICPP-2264 GenerateFlowFile: 'Custom Text' should be reevaluated … [nifi-minifi-cpp]
fgerlits commented on code in PR #1706:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1706#discussion_r1439395796
##
extensions/standard-processors/processors/GenerateFlowFile.cpp:
##
@@ -58,57 +58,69 @@ void generateData(std::vector& data, bool textData =
false) {
}
void GenerateFlowFile::onSchedule(core::ProcessContext& context,
core::ProcessSessionFactory&) {
- if (context.getProperty(FileSize.name, fileSize_)) {
-logger_->log_trace("File size is configured to be {}", fileSize_);
+ if (context.getProperty(FileSize.name, file_size_)) {
+logger_->log_trace("File size is configured to be {}", file_size_);
}
- if (context.getProperty(BatchSize.name, batchSize_)) {
-logger_->log_trace("Batch size is configured to be {}", batchSize_);
+ if (context.getProperty(BatchSize.name, batch_size_)) {
+logger_->log_trace("Batch size is configured to be {}", batch_size_);
}
- std::string value;
- if (context.getProperty(DataFormat.name, value)) {
-textData_ = (value == GenerateFlowFile::DATA_FORMAT_TEXT);
+ if (auto data_format = context.getProperty(DataFormat)) {
+textData_ = (*data_format == DATA_FORMAT_TEXT);
}
- if (context.getProperty(UniqueFlowFiles.name, uniqueFlowFile_)) {
-logger_->log_trace("Unique Flow files is configured to be {}",
uniqueFlowFile_);
+ if (context.getProperty(UniqueFlowFiles.name, unique_flow_file_)) {
+logger_->log_trace("Unique Flow files is configured to be {}",
unique_flow_file_);
}
std::string custom_text;
context.getProperty(CustomText, custom_text, nullptr);
if (!custom_text.empty()) {
-if (textData_ && !uniqueFlowFile_) {
- data_.assign(custom_text.begin(), custom_text.end());
+if (textData_ && !unique_flow_file_) {
+ non_unique_data_.assign(custom_text.begin(), custom_text.end());
return;
-} else {
- logger_->log_warn("Custom Text property is set, but not used!");
}
+logger_->log_warn("Custom Text property is set, but not used!");
}
- if (!uniqueFlowFile_) {
-data_.resize(gsl::narrow(fileSize_));
-generateData(data_, textData_);
+ if (!unique_flow_file_) {
+non_unique_data_.resize(gsl::narrow(file_size_));
+generateData(non_unique_data_, textData_);
+ }
+}
+
+// If the Data Format is text and if Unique FlowFiles is false, the custom
text has to be evaluated once per batch
+void GenerateFlowFile::regenerateNonUniqueData(core::ProcessContext& context) {
+ std::string custom_text;
+ context.getProperty(CustomText, custom_text, nullptr);
+ if (!custom_text.empty()) {
+if (textData_ && !unique_flow_file_) {
+ non_unique_data_.assign(custom_text.begin(), custom_text.end());
+ return;
+}
+logger_->log_warn("Custom Text property is set, but not used!");
Review Comment:
I think this would be simpler, and would avoid doing some unnecessary work:
```c++
if (!textData_ || unique_flow_file_) return;
std::string custom_text;
context.getProperty(CustomText, custom_text, nullptr);
if (!custom_text.empty()) {
non_unique_data_.assign(custom_text.begin(), custom_text.end());
}
```
We would lose the warning, but we have warned in `onSchedule()` already.
Also, if the expanded custom text is empty now, but it wasn't empty before,
I think the flow file should be empty, so
```c++
if (!textData_ || unique_flow_file_) return;
std::string custom_text;
context.getProperty(CustomText, custom_text, nullptr);
non_unique_data_.assign(custom_text.begin(), custom_text.end());
```
would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
