[jira] [Resolved] (NIFI-12983) Add support for Qdrant vector store
[ https://issues.apache.org/jira/browse/NIFI-12983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-12983. --- Fix Version/s: 2.0.0-M4 Resolution: Fixed > Add support for Qdrant vector store > --- > > Key: NIFI-12983 > URL: https://issues.apache.org/jira/browse/NIFI-12983 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Anush >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Qdrant - [https://github.com/qdrant/qdrant], is an open-source vector search > engine and database, governed by the Apache-2.0 license. > Qdrant ranks amongst the most performant and most used vector databases > available today. > - [https://qdrant.tech/benchmarks]/ > - [https://ossinsight.io/collections/vector-search-engine]/ > This is a proposal to add support for Qdrant via an extension in NiFi. > Associated pull-request: [https://github.com/apache/nifi/pull/8590] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13369) 2.0.0-M3 Zookeeper TLS connection issue
[
https://issues.apache.org/jira/browse/NIFI-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852549#comment-17852549
]
Pierre Villard commented on NIFI-13369:
---
We've upgraded the ZK client in NiFi to 3.9.2 but I doubt that would be the
reason for the error you're seeing.
Dumb question but same OS for ZK nodes and NiFi nodes?
> 2.0.0-M3 Zookeeper TLS connection issue
> ---
>
> Key: NIFI-13369
> URL: https://issues.apache.org/jira/browse/NIFI-13369
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 2.0.0-M3
> Environment: Ubuntu 22.04.
> NiFi 2.0.0-M3 / OpenJDK-21
> Zookeeper 3.8.4 / OpenJDK-11
>Reporter: Night Gryphon
>Priority: Major
>
> After upgrading from 2.0.0-M2 to M3 NiFi can't connect existing Zookeeper
> cluster using SSL/TLS. That blocks upgrade to M3.
> Looks like TLS version mismatch but NiFi don't have corresponding setting for
> zookeeper client TLS version.
> Below is the error log
> {code:java}
> 2024-06-05 20:21:14,543 INFO [epollEventLoopGroup-2-1]
> o.apache.zookeeper.ClientCnxnSocketNetty SSL handler added for channel: [id:
> 0x5e8f288a]
> 2024-06-05 20:21:14,544 INFO [epollEventLoopGroup-2-1]
> o.apache.zookeeper.ClientCnxnSocketNetty channel is connected: [id:
> 0x5e8f288a, L:/10.10.0.145:14916 - R:zk3.nifi-test/10.10.0.14
> 3:2182]
> 2024-06-05 20:21:14,549 ERROR [epollEventLoopGroup-2-1]
> o.apache.zookeeper.ClientCnxnSocketNetty Unexpected throwable
> io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException:
> Received fatal alert: protocol_version
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:801)
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert:
> protocol_version
> at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
> at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
> at
> java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
> at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287)
> at
> java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)
> at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
> at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736)
> at
> java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691)
> at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506)
> at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482)
> at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679)
> at
> io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:310)
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1445)
> at
> io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338)
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387)
> at
>
[jira] [Commented] (NIFI-13369) 2.0.0-M3 Zookeeper TLS connection issue
[
https://issues.apache.org/jira/browse/NIFI-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852535#comment-17852535
]
Pierre Villard commented on NIFI-13369:
---
Version of Zookeeper? Did you check the security settings of your JVM for Java
11 and Java 21? Why not also use Java 21 for Zookeeper?
> 2.0.0-M3 Zookeeper TLS connection issue
> ---
>
> Key: NIFI-13369
> URL: https://issues.apache.org/jira/browse/NIFI-13369
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 2.0.0-M3
> Environment: Ubuntu 22.04.
> NiFi 2.0.0-M3 / OpenJDK-21
> Zookeeper 3.8.4 / OpenJDK-11
>Reporter: Night Gryphon
>Priority: Major
>
> After upgrading from 2.0.0-M2 to M3 NiFi can't connect existing Zookeeper
> cluster using SSL/TLS. That blocks upgrade to M3.
> Looks like TLS version mismatch but NiFi don't have corresponding setting for
> zookeeper client TLS version.
> Below is the error log
> {code:java}
> 2024-06-05 20:21:14,543 INFO [epollEventLoopGroup-2-1]
> o.apache.zookeeper.ClientCnxnSocketNetty SSL handler added for channel: [id:
> 0x5e8f288a]
> 2024-06-05 20:21:14,544 INFO [epollEventLoopGroup-2-1]
> o.apache.zookeeper.ClientCnxnSocketNetty channel is connected: [id:
> 0x5e8f288a, L:/10.10.0.145:14916 - R:zk3.nifi-test/10.10.0.14
> 3:2182]
> 2024-06-05 20:21:14,549 ERROR [epollEventLoopGroup-2-1]
> o.apache.zookeeper.ClientCnxnSocketNetty Unexpected throwable
> io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException:
> Received fatal alert: protocol_version
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:801)
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert:
> protocol_version
> at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
> at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
> at
> java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
> at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287)
> at
> java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)
> at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
> at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736)
> at
> java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691)
> at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506)
> at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482)
> at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679)
> at
> io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:310)
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1445)
> at
> io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338)
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387)
> at
> io.netty.handler.codec.ByteToMessage
[jira] [Updated] (NIFI-13358) NiFi Registry - rework action menus for flows and bundles
[ https://issues.apache.org/jira/browse/NIFI-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13358: -- Status: Patch Available (was: Open) > NiFi Registry - rework action menus for flows and bundles > - > > Key: NIFI-13358 > URL: https://issues.apache.org/jira/browse/NIFI-13358 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The current action menus are focused on flows which is not great if the NiFi > Registry is also used for bundles. This issue is to rework the action menus > to better distinguish flows and bundles and implement proper actions in the > NiFi Registry UI to deal with bundles. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13358) NiFi Registry - rework action menus for flows and bundles
Pierre Villard created NIFI-13358: - Summary: NiFi Registry - rework action menus for flows and bundles Key: NIFI-13358 URL: https://issues.apache.org/jira/browse/NIFI-13358 Project: Apache NiFi Issue Type: Improvement Components: NiFi Registry Reporter: Pierre Villard Assignee: Pierre Villard The current action menus are focused on flows which is not great if the NiFi Registry is also used for bundles. This issue is to rework the action menus to better distinguish flows and bundles and implement proper actions in the NiFi Registry UI to deal with bundles. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Back pressure in a NiFi cluster
Hi Alexis, Backpressure is a node-level concept. If there is backpressure on one node, it does not impact other nodes. So the statement "it could be that 1 node is slower and reaches the threshold while the others run normally" is correct. HTH, Pierre Le mar. 4 juin 2024 à 14:54, Alexis Sarda-Espinosa a écrit : > Hello, > > If I have a NiFi cluster and a processor that is scheduled to run on all > nodes, a queue for said processor basically represents a queue per node, > right? And since the configured back pressure thresholds are also per node, > it could be that 1 node is slower and reaches the threshold while the > others run normally. If the processor is only back pressured in 1 node, > does that also halt scheduling for the same processor in the other nodes? > > Regards, > Alexis. > >
[jira] [Updated] (NIFI-13310) Fix duplicate rat plugin declaration in jolt transform module
[ https://issues.apache.org/jira/browse/NIFI-13310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13310: -- Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Fix duplicate rat plugin declaration in jolt transform module > - > > Key: NIFI-13310 > URL: https://issues.apache.org/jira/browse/NIFI-13310 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 20m > Remaining Estimate: 0h > > [WARNING] > [WARNING] Some problems were encountered while building the effective model > for org.apache.nifi:nifi-jolt-transform-json-ui:war:2.0.0-SNAPSHOT > [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but > found duplicate declaration of plugin org.apache.rat:apache-rat-plugin @ line > 209, column 21 > [WARNING] > [WARNING] It is highly recommended to fix these problems because they > threaten the stability of your build. > [WARNING] > [WARNING] For this reason, future Maven versions might no longer support > building such malformed projects. > [WARNING] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13077) On-demand Extension Provider
[
https://issues.apache.org/jira/browse/NIFI-13077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851973#comment-17851973
]
Pierre Villard commented on NIFI-13077:
---
As a side note, I think it is good to also keep a close eye on what [~bbende]
is doing in NIFI-13343.
I think we all want to go in the same direction. Let's not duplicate efforts :)
> On-demand Extension Provider
>
>
> Key: NIFI-13077
> URL: https://issues.apache.org/jira/browse/NIFI-13077
> Project: Apache NiFi
> Issue Type: Epic
> Components: Core Framework
> Reporter: Pierre Villard
>Priority: Major
>
> We currently have the concept of *ExternalResourceProvider* with two
> implementations (HDFS and NiFi Registry) that can be configured to list and
> download all NARs made available in those locations. Those implementations,
> if configured, would get started when NiFi starts and would download ALL of
> the available NARs, plus a background thread would check every five minutes
> for new NARs to be available and downloaded.
> The proposal here is to have a similar concept that would focus on extensions
> / components but instead of having a background thread and instead of having
> all of the components downloaded, the approach would be to plug this into the
> *ExtensionBuilder* and when a component cannot be instantiated (when loading
> a flow definition) with locally available components, then, instead of
> creating a ghost component, the Extension Providers would be queried with
> specific coordinates and if the provider makes the component available, then
> the NAR would be downloaded (alongside required dependencies if the NAR
> depends on another NAR).
> This approach already exists in the *Kafka Connect NiFi plugin* with the
> class {*}ExtensionClientDefinition{*}. By adopting this approach in NiFi,
> it’d be much easier to ship a much *smaller version of NiFi* and have NiFi
> download the required components based on flows that are being instantiated /
> deployed.
> The operation of downloading the NAR would not be blocking, meaning that we
> would still create a ghost component but after completion of the NAR(s)
> download and the loading of the components, the flows would be fully
> operational.
> It might be possible to show something similar as for the Python extensions
> where we show that the component is still in the process of downloading third
> party dependencies.
> While this is a great opportunity to reduce the size of the NiFi binary (and
> associated container image), it would not be great from a user perspective
> when designing flows because all of the NARs removed from the default image
> would no longer be visible in the list of available components when adding,
> for example, a processor to the canvas.
> Longer term we could imagine that the extension providers can also implement
> a listing API so that when showing the list of available components, we would
> show the list of the components available locally as well as the components
> available through the extensions providers. The listing of components could
> add another column to indicate the source of the component.
> This is something that is exposed for the Extension Bundles in the NiFi
> Registry (we also have the information about the NiFi API version that has
> been used for building the components so we could use this information to
> only list components that should be compatible from an API standpoint - same
> major version but lower or equal API version).
> The immediate goal though would be to introduce the concept of
> ExtensionProvider with the following APIs:
> {code:java}
> boolean isAvailableExtension(Coordinates)
> void downloadExtension(Coordinates)
> {code}
> Longer term we could also consider something like:
> {code:java}
> List listExtensions(){code}
> But we would need to figure out how a NAR can provide the information about
> the components that are inside of it. The NiFi Registry provides this
> information, but that would not be the case for a Maven based implementation
> for example.
> In nifi.properties we would have something looking like:
> {code:java}
> nifi.nar.extension.provider..{code}
> And we would loop through all the configured providers to find the
> appropriate NAR to download based on provided coordinates in the flow
> definition that is being instantiated (either from flow.json.gz, or an
> uploaded JSON flow definition, or when checking out a flow from a registry
> client).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-13337) Column sizes when listing flowfiles in queue
[ https://issues.apache.org/jira/browse/NIFI-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851970#comment-17851970 ] Pierre Villard commented on NIFI-13337: --- Makes sense, thanks for the details [~scottyaslan] > Column sizes when listing flowfiles in queue > > > Key: NIFI-13337 > URL: https://issues.apache.org/jira/browse/NIFI-13337 > Project: Apache NiFi > Issue Type: Sub-task > Reporter: Pierre Villard >Assignee: Matt Gilman >Priority: Major > Attachments: Screenshot 2024-06-02 at 13.59.28.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > !Screenshot 2024-06-02 at 13.59.28.png|width=964,height=224! > It seems like columns cannot be resized. If that's correct, I believe the > defaults could be a bit better. In particular the columns 'Position' and > 'UUID' seem overly large (although we know the maximum size), while the > column 'Filename' could be larger. > As a side note, there is now one more click to get to flow file content, > details, provenance, etc, as we now have to click to get a menu instead of > having icons. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13329) Fallback to raw viewer when formatted viewer fails
[ https://issues.apache.org/jira/browse/NIFI-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13329: -- Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Fallback to raw viewer when formatted viewer fails > -- > > Key: NIFI-13329 > URL: https://issues.apache.org/jira/browse/NIFI-13329 > Project: Apache NiFi > Issue Type: Bug > Reporter: Pierre Villard >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M4 > > Attachments: Screenshot 2024-05-30 at 11.19.23.png > > Time Spent: 40m > Remaining Estimate: 0h > > The formatted viewer (based on the mime.type attribute) recently became the > default. In case the MIME type is wrong, this would return an error when > opening the flowfile's content. It could be nice to catch the error and > fallback to the raw viewer. > !Screenshot 2024-05-30 at 11.19.23.png|width=611,height=150! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [JDK] NiFi 1.20.0 comaptibility with JDK21
Hi Pedro, It's hard to have a definitive answer on this as it also depends on what you're using in your flows but you may have some issues. I'd highly recommend using the latest of Apache NiFi 1.x (NiFi 1.26) if you want to run it with Java 21. Thanks, Pierre Le lun. 3 juin 2024 à 13:56, Pedro Soto Fernandez < pedro.soto.fernan...@nttdata.com> a écrit : > Hello, > > > > We intend to update our environments to JAVA21. > > We are currently working with Apache NiFi 1.20.0 on Docker with image > apache/nifi:1.20.0. > > Our JAVA update scope doesn’t include upgrading NiFi and so far we only > found reference in Nifi Release notes confirming NiFi 1.20.0 requires > JAVA11 or later but no specifics on JDK21. > > > > Before assuming NiFi 1.20.0 has no issue with JDK21, could you point us to > where we can find specific data regarding this and/or further steps we > would need take to ensure compatibility between NiFi 1.20.0 and JDK21? > > Thanks in advance. > > > > Best Regards, > > *Pedro Soto Fernández* >
[jira] [Created] (NIFI-13337) Column sizes when listing flowfiles in queue
Pierre Villard created NIFI-13337: - Summary: Column sizes when listing flowfiles in queue Key: NIFI-13337 URL: https://issues.apache.org/jira/browse/NIFI-13337 Project: Apache NiFi Issue Type: Sub-task Reporter: Pierre Villard Attachments: Screenshot 2024-06-02 at 13.59.28.png !Screenshot 2024-06-02 at 13.59.28.png|width=964,height=224! It seems like columns cannot be resized. If that's correct, I believe the defaults could be a bit better. In particular the columns 'Position' and 'UUID' seem overly large (although we know the maximum size), while the column 'Filename' could be larger. As a side note, there is now one more click to get to flow file content, details, provenance, etc, as we now have to click to get a menu instead of having icons. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13336) Update aws, azure, gcp, commons-cli/net/compress, fabric8, neo4j, and spring integration mail
[ https://issues.apache.org/jira/browse/NIFI-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13336: -- Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Update aws, azure, gcp, commons-cli/net/compress, fabric8, neo4j, and spring > integration mail > - > > Key: NIFI-13336 > URL: https://issues.apache.org/jira/browse/NIFI-13336 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 20m > Remaining Estimate: 0h > > com.amazonaws * 1.12.730 1.12.733 > com.azure azure-sdk-bom 1.2.23 1.2.24 > com.google.cloud libraries-bom 26.39.0 26.40.0 > commons-cli 1.7.0 1.8.0 > commons-net 3.10.0 3.11.0 > io.fabric8 * 6.12.1 6.13.0 > org.apache.commons commons-compress 1.26.1 1.26.2 > software.amazon.awssdk 2.25.60 2.25.63 > com.google.apis google-api-services-drive v3-rev20240327-2.0.0 > v3-rev20240521-2.0.0 > org.neo4j.driver neo4j-java-driver 5.20.0 5.21.0 > org.springframework.integration spring-integration-mail 6.2.4 6.2.5 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13329) Fallback to raw viewer when formatted viewer fails
[
https://issues.apache.org/jira/browse/NIFI-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851164#comment-17851164
]
Pierre Villard commented on NIFI-13329:
---
{quote}Would it make sense for this case to render an error message stating
that the content does not match the mime type? Or there was an error in
processing the content in formatted mode? The UI would render correctly and
show Formatted mode with the message. The user could then change the mode to
Original or Hex to see those views. It addresses the error page your seeing
above. And I think this case is more uncommon. Defaulting to Formatted and
having to manually change to Original when the content in the flowfile is wrong
is less annoying for the user than defaulting to Original and having to
manually change to Formatted when the content in the flowfile is correct.
{quote}
I think that makes perfect sense and is a very good behavior IMO.
> Fallback to raw viewer when formatted viewer fails
> --
>
> Key: NIFI-13329
> URL: https://issues.apache.org/jira/browse/NIFI-13329
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Pierre Villard
>Priority: Major
> Attachments: Screenshot 2024-05-30 at 11.19.23.png
>
>
> The formatted viewer (based on the mime.type attribute) recently became the
> default. In case the MIME type is wrong, this would return an error when
> opening the flowfile's content. It could be nice to catch the error and
> fallback to the raw viewer.
> !Screenshot 2024-05-30 at 11.19.23.png|width=611,height=150!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-13329) Fallback to raw viewer when formatted viewer fails
[ https://issues.apache.org/jira/browse/NIFI-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851150#comment-17851150 ] Pierre Villard commented on NIFI-13329: --- Agree that it'd be an improvement. Thanks for looking into it Matt. > Fallback to raw viewer when formatted viewer fails > -- > > Key: NIFI-13329 > URL: https://issues.apache.org/jira/browse/NIFI-13329 > Project: Apache NiFi > Issue Type: Bug > Reporter: Pierre Villard >Priority: Major > Attachments: Screenshot 2024-05-30 at 11.19.23.png > > > The formatted viewer (based on the mime.type attribute) recently became the > default. In case the MIME type is wrong, this would return an error when > opening the flowfile's content. It could be nice to catch the error and > fallback to the raw viewer. > !Screenshot 2024-05-30 at 11.19.23.png|width=611,height=150! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13329) Fallback to raw viewer when formatted viewer fails
[ https://issues.apache.org/jira/browse/NIFI-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851142#comment-17851142 ] Pierre Villard commented on NIFI-13329: --- Just built latest and getting the same (legacy and new UI). You can reproduce with GFF to funnel. GFF configured with 'custom text' = 'Hi', and 'MIME type' = 'application/json', then Run Once, then list flow files in connection and show content of the flow file. > Fallback to raw viewer when formatted viewer fails > -- > > Key: NIFI-13329 > URL: https://issues.apache.org/jira/browse/NIFI-13329 > Project: Apache NiFi > Issue Type: Bug > Reporter: Pierre Villard >Priority: Major > Attachments: Screenshot 2024-05-30 at 11.19.23.png > > > The formatted viewer (based on the mime.type attribute) recently became the > default. In case the MIME type is wrong, this would return an error when > opening the flowfile's content. It could be nice to catch the error and > fallback to the raw viewer. > !Screenshot 2024-05-30 at 11.19.23.png|width=611,height=150! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13329) Fallback to raw viewer when formatted viewer fails
[
https://issues.apache.org/jira/browse/NIFI-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851096#comment-17851096
]
Pierre Villard commented on NIFI-13329:
---
Hey [~mcgilman] - I'll retry with latest on main because last time I built was
probably a couple of weeks ago.
I'm trying to open a flow file content for which mime.type is application/json
but the content is not JSON. I get the below stacktrace:
{code:java}
2024-05-31 15:31:44,682 ERROR [NiFi Web Server-457]
o.a.nifi.web.ContentViewerController Content preparation failed for Content
Viewer [/nifi-standard-content-viewer-2.0.0-SNAPSHOT]
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'Hi': was
expecting (JSON String, Number, Array, Object or token 'null', 'true' or
'false')
at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION`
disabled); line: 1, column: 4]
at
com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2567)
at
com.fasterxml.jackson.core.JsonParser._constructReadException(JsonParser.java:2593)
at
com.fasterxml.jackson.core.JsonParser._constructReadException(JsonParser.java:2601)
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:765)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3659)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2747)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:867)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:753)
at
com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4992)
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4898)
at
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3885)
at
org.apache.nifi.web.StandardContentViewerController.doGet(StandardContentViewerController.java:90)
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:527)
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614)
at
org.eclipse.jetty.ee10.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1379)
at
org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736)
at
org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1614)
at
org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:365)
at
org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:58)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
at
org.springframework.security.web.FilterChainProxy
[jira] [Created] (NIFI-13329) Fallback to raw viewer when formatted viewer fails
Pierre Villard created NIFI-13329: - Summary: Fallback to raw viewer when formatted viewer fails Key: NIFI-13329 URL: https://issues.apache.org/jira/browse/NIFI-13329 Project: Apache NiFi Issue Type: Sub-task Reporter: Pierre Villard Attachments: Screenshot 2024-05-30 at 11.19.23.png The formatted viewer (based on the mime.type attribute) recently became the default. In case the MIME type is wrong, this would return an error when opening the flowfile's content. It could be nice to catch the error and fallback to the raw viewer. !Screenshot 2024-05-30 at 11.19.23.png|width=611,height=150! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13320) Upgrade Spring Boot to 3.3.0
[ https://issues.apache.org/jira/browse/NIFI-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13320: -- Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Spring Boot to 3.3.0 > > > Key: NIFI-13320 > URL: https://issues.apache.org/jira/browse/NIFI-13320 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0-M4 > > Time Spent: 20m > Remaining Estimate: 0h > > Spring Boot dependencies for NiFi Registry should be upgraded to > [3.3.0|https://github.com/spring-projects/spring-boot/releases/tag/v3.3.0] to > incorporate the latest set of improvements and dependency upgrades. The 3.3.0 > release provides the latest maintained version beyond the 3.2 series. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13301) Create ExtensionRegistryClient for External Extension Registry Interaction
[
https://issues.apache.org/jira/browse/NIFI-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851054#comment-17851054
]
Pierre Villard commented on NIFI-13301:
---
{quote}Although NiFi Registry has a number of configurable features, it does
not support clustered deployment for high availability, which would be a vital
part of such a marketplace.
{quote}
Can you elaborate on this [~exceptionfactory] ? Such a marketplace would likely
be read-only for the consumers of the artefacts so I don't see why you could
not have multiple NiFi Registry instances serving the same assets.
As a note, I filed NIFI-13077 which I think could be linked to this one.
> Create ExtensionRegistryClient for External Extension Registry Interaction
> --
>
> Key: NIFI-13301
> URL: https://issues.apache.org/jira/browse/NIFI-13301
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions, NiFi Registry
> Environment: Linux: Ubuntu 20.04 or 22.04
>Reporter: James Guzman (Medel)
>Assignee: James Guzman (Medel)
>Priority: Minor
>
> *Objective:* Improve/enable sharing/reuse of at least two features of Apache
> NiFi, so the community can have an easier time contributing their flows
> and/or custom processors into a NiFi Marketplace:
> * {*}Pre-Designed Flows{*}: (Approach 1) stored in a NiFi Registry or
> accessible location. (Approach 2) NiFi Registry can also substituted by
> NiFi's git registry client having the location of the NiFi Marketplace.
> Thus, we just have a git location and NiFi uses that to find/store flows.
> * *Components/Processors* built in *Java/Python* that anyone is free to use
> * *End to End Full Stack Applications Powered By NiFi or MiNiFi CPP?* The
> frontend could be various ones from PyQT, ReactJS, H2O.ai Wave, 3D Slicer,
> OHIF Viewer, etc: (Maybe this one can be subscription based where users get
> access if they invest in the monthly or yearly subscription?)
> *Potential Solution:* Apache NiFi builds an extension point for interacting
> with Extension Registries, similar to how it currently has
> *{{FlowRegistryClient}}* with implementations for NiFi Registry and now
> GitHub, there would be *{{ExtensionRegistryClient}}* with implementations for
> stuff like Nexus, NiFi Registry, and any other vendor ones, for example maybe
> Datavolo provides one, but basically in apache we don't want to build another
> application, we already have NiFi Registry. (Briefly Discussed with [~bbende]
> )
> * I will start by looking into *{{FlowRegistryClient}}* and then base
> *{{ExtensionRegistryClient}}* toward that approach. If I am understanding
> correctly, we would contribute an *{{ExtensionRegistryClient}}* feature into
> Apache NiFi that enables NiFi to integrate with other vendors (Datavolo,
> H2Oai, etc) group of processors, data flow templates and so on. I see where
> you are coming from with for Apache the goal being not to build another
> application. That application could be by the another and NiFi's
> *{{ExtensionRegistryClient}}* hooks up to their {*}RegistryServer{*}. (Heres
> the path I will take toward that goal).
>
> {*}Project Ownership{*}: There will be a clear line that the NiFi Marketplace
> is not owned by the Apache NiFi PMC, rather it will be managed and owned by a
> vendor in close alignment with NiFi, which we will announce at a later time.
>
> {*}Motivation (NiFi){*}: Some people in the community have shown interest in
> having a NiFi Processor Marketplace where they can contribute their category
> of processors. In particular, I saw some people interested in contributing
> their NiFi python processors. This NiFi Processor marketplace could be also
> applied toward the community interested in contributing custom NiFi java
> processors.
> * {*}Extra MiNiFi C+{+}{{+}}{*}: We could even add a section for MiNiFi C+
> custom processors. There is a part of the MiNiFi build process that brings in
> NiFi through building the JNI extension. So, that is a way to integrate NiFi
> Registry there too.
> I have briefly talked with [~bbende] and [~joewitt] asking them questions on
> ways to bring custom python processors into production. One of the routes was
> through leveraging NiFi Registry in connection with Apache NiFi to streamline
> integration. For my use case, I would leverage the NiFi Processor Marketplace
> to start contributing my AI/DL "Medical Imaging" Pipeline of custom NiFi
> Python Processors where I focused on my master thesis AI/DL for stroke
> diagnosis.
> *High Level De
[RESULT][VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0
Apache NiFi Community, I am pleased to announce that the 2.0.0 release of Apache NiFi NAR Maven Plugin passes: 4 +1 (binding) votes 1 +1 (non-binding) votes 0 0 votes 0 -1 votes Thanks to all who helped make this release possible! The release artifacts will be published in the next 24 hours. Here is the vote thread: https://lists.apache.org/thread/kbrhjf72p3brbc17z3h16kh06zhnzx5r Thanks, Pierre
Re: [VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0
+1 binding Le jeu. 23 mai 2024 à 19:40, Joe Witt a écrit : > +1 binding > > On Thu, May 23, 2024 at 10:33 AM Matt Burgess wrote: > > > +1 binding, verified the commit hash, applied the patch to NiFi main and > > verified the NAR Maven plugin works as expected. > > > > Thanks for RM'ing Pierre! > > > > On Wed, May 22, 2024 at 7:31 AM Pierre Villard < > > pierre.villard...@gmail.com> > > wrote: > > > > > Hello Apache NiFi Community, > > > > > > I am pleased to be calling this vote for the source release of Apache > > NiFi > > > NAR Maven Plugin 2.0.0. This new release is intended to be used with > NiFi > > > 2+. > > > > > > The source being voted upon, including signatures, digests, etc. can be > > > found at: > > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-nar-maven-plugin-2.0.0/ > > > > > > A helpful reminder on how the release candidate verification process > > works: > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+NAR+Maven+Plugin+release+candidate > > > > > > Please note that when trying this release, some changes are expected on > > the > > > NiFi side as proposed in the PR: > > https://github.com/apache/nifi/pull/8860 > > > > > > The Git tag is nifi-maven-2.0.0-RC1 > > > The Git commit ID is d0de49a34a1197988eaa007d28e6eb20ab6d701f > > > > > > > > > https://gitbox.apache.org/repos/asf?p=nifi-maven.git;a=commit;h=d0de49a34a1197988eaa007d28e6eb20ab6d701f > > > > > > Checksums of nifi-nar-maven-plugin-2.0.0-source-release.zip: > > > SHA256: > ae889c843a82c4823fca0bf5fedf275dbb812c2b11e25ae26b0bd4ad7d26ccae > > > SHA512: > > > > > > > > > c9afefb093d8b9568012d6697f302eb4a138f0a65a7045eaeb1bff551fb4bff23c805294700ca20984a3493eb61fe80a39d8b5b393afc85cf3f6ab1a28df831c > > > > > > Release artifacts are signed with the following key: > > > https://people.apache.org/keys/committer/pvillard.asc > > > > > > KEYS file available here: > > > https://dist.apache.org/repos/dist/release/nifi/KEYS > > > > > > 5 issues were closed/resolved for this release: > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12354745 > > > > > > Release note highlights can be found here: > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiNARMavenPluginVersion2.0.0 > > > > > > The vote will be open for 72 hours. Please download the release > > > candidate and evaluate the necessary items including checking hashes, > > > signatures, build from source, and test. Then please vote: > > > > > > [ ] +1 Release this package as nifi-nar-maven-plugin-2.0.0 > > > [ ] +0 no opinion > > > [ ] -1 Do not release this package because … > > > > > >
[jira] [Updated] (NIFI-11658) Streamline using single parameter context for nested PGs
[ https://issues.apache.org/jira/browse/NIFI-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-11658: -- Fix Version/s: 1.27.0 > Streamline using single parameter context for nested PGs > > > Key: NIFI-11658 > URL: https://issues.apache.org/jira/browse/NIFI-11658 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 2.0.0-M1, 1.27.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > 1. when adding a new process group to the canvas, in the create PG view where > one should give the name of the process group, add a drop down menu to select > the parameter context to associate with the process group. It would default > to the parameter context associated to the parent process group (if there is > one specified and if the user has the permissions) > 2. in the configuration view of a process group, add a dedicated tab for > parameter context to have dedicated handling. In addition to the dropdown > menu for selecting a parameter context, add a checkbox to recursively apply > the change. This checkbox would only be used when clicking "Apply", it would > not be persisted. If checked, it means that the parameter context will be > applied to the process group as well to all the embedded process groups > (recursively) if and only if the user has the proper permissions on all > process groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13283) Exception thrown in PutKinesisFirehose processor
[ https://issues.apache.org/jira/browse/NIFI-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-13283. --- Fix Version/s: 2.0.0-M4 Resolution: Fixed > Exception thrown in PutKinesisFirehose processor > > > Key: NIFI-13283 > URL: https://issues.apache.org/jira/browse/NIFI-13283 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 2.0.0-M1, 2.0.0-M2, 2.0.0-M3 >Reporter: Evan Shelton >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When attempting to send data to Firehose an exception is thrown related to > attempting to access an ArrayList that hasn't been initialized -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13283) Exception thrown in PutKinesisFirehose processor
[ https://issues.apache.org/jira/browse/NIFI-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13283: -- Affects Version/s: 2.0.0-M2 2.0.0-M1 > Exception thrown in PutKinesisFirehose processor > > > Key: NIFI-13283 > URL: https://issues.apache.org/jira/browse/NIFI-13283 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 2.0.0-M1, 2.0.0-M2, 2.0.0-M3 >Reporter: Evan Shelton >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When attempting to send data to Firehose an exception is thrown related to > attempting to access an ArrayList that hasn't been initialized -- This message was sent by Atlassian Jira (v8.20.10#820010)
[VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0
Hello Apache NiFi Community, I am pleased to be calling this vote for the source release of Apache NiFi NAR Maven Plugin 2.0.0. This new release is intended to be used with NiFi 2+. The source being voted upon, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/nifi/nifi-nar-maven-plugin-2.0.0/ A helpful reminder on how the release candidate verification process works: https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+NAR+Maven+Plugin+release+candidate Please note that when trying this release, some changes are expected on the NiFi side as proposed in the PR: https://github.com/apache/nifi/pull/8860 The Git tag is nifi-maven-2.0.0-RC1 The Git commit ID is d0de49a34a1197988eaa007d28e6eb20ab6d701f https://gitbox.apache.org/repos/asf?p=nifi-maven.git;a=commit;h=d0de49a34a1197988eaa007d28e6eb20ab6d701f Checksums of nifi-nar-maven-plugin-2.0.0-source-release.zip: SHA256: ae889c843a82c4823fca0bf5fedf275dbb812c2b11e25ae26b0bd4ad7d26ccae SHA512: c9afefb093d8b9568012d6697f302eb4a138f0a65a7045eaeb1bff551fb4bff23c805294700ca20984a3493eb61fe80a39d8b5b393afc85cf3f6ab1a28df831c Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pvillard.asc KEYS file available here: https://dist.apache.org/repos/dist/release/nifi/KEYS 5 issues were closed/resolved for this release: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12354745 Release note highlights can be found here: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiNARMavenPluginVersion2.0.0 The vote will be open for 72 hours. Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote: [ ] +1 Release this package as nifi-nar-maven-plugin-2.0.0 [ ] +0 no opinion [ ] -1 Do not release this package because …
[jira] [Updated] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13268:
--
Component/s: Extensions
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 2.0.0-M3
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> nifi-1 | at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
> nifi-1 | at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)
> nifi-1 | at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) nifi-1 |
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.
[jira] [Updated] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13268:
--
Affects Version/s: 2.0.0-M3
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M3
>Reporter: Emin
>Assignee: Peter Turcsanyi
>Priority: Major
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> nifi-1 | at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
> nifi-1 | at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)
> nifi-1 | at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) nifi-1 |
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.j
[jira] [Updated] (NIFI-13267) Bump NiFi NAR Maven plugin version
[ https://issues.apache.org/jira/browse/NIFI-13267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13267: -- Status: Patch Available (was: Open) > Bump NiFi NAR Maven plugin version > -- > > Key: NIFI-13267 > URL: https://issues.apache.org/jira/browse/NIFI-13267 > Project: Apache NiFi > Issue Type: Task > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Once we have a new version of the NiFi NAR Maven Plugin, we'll need to bump > the version in the NiFi code base and make some changes to account for the > new extension types (Parameter Providers and Flow Analysis Rules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13274) Dependencies housekeeping for NiFi NAR Maven plugin
[ https://issues.apache.org/jira/browse/NIFI-13274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13274: -- Status: Patch Available (was: Open) > Dependencies housekeeping for NiFi NAR Maven plugin > --- > > Key: NIFI-13274 > URL: https://issues.apache.org/jira/browse/NIFI-13274 > Project: Apache NiFi > Issue Type: Task > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Fix For: nifi-nar-maven-plugin-2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Dependencies housekeeping for NiFi NAR Maven plugin > Also fixing the SNAPSHOT version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Release NiFi NAR Maven Plugin
My plan was to go with the latest for everything (including Java 21). This way, NiFi NAR Maven plugin 2.0.0 would be used only for NiFi 2.x. I actually think that the addition of the extension type "flow analysis rules" would break things if used in the 1.x line (unless I also update the enum in NiFi 1.x to include flow analysis rules as an extension type but not sure that would be a great approach). Le mar. 21 mai 2024 à 17:24, David Handermann a écrit : > Pierre, > > Thanks for highlighting the differences for Flow Analysis Rules. As > far as making this a 2.0.0 release version, do you anticipate > including any breaking changes? > > The current version of the plugin is built with Java 8, so this could > be updated to Java 21, which would be a breaking change, however, that > should be handled before the release process. > > If there are no breaking changes, I recommend a version 1.6.0 release > before a 2.0.0 release of the plugin. > > Regards, > David Handermann > > On Tue, May 21, 2024 at 10:19 AM Pierre Villard > wrote: > > > > Given that the flow analysis rules are something specific to NiFi 2, and > > given that this is an opportunity to do some house cleaning, I'll go > > directly to a 2.0.0 release directly and have it used only on the main > > branch. The things that have been added as improvements are not really > > required for 1.x anyway. > > > > Le mar. 21 mai 2024 à 15:40, David Handermann < > exceptionfact...@apache.org> > > a écrit : > > > > > Thanks Pierre, sounds good! > > > > > > Regards, > > > David Handermann > > > > > > On Tue, May 21, 2024 at 8:36 AM Pierre Villard > > > wrote: > > > > > > > > Thanks will go ahead with 1.6.0 release process > > > > > > > > Le lun. 20 mai 2024 à 19:30, Matt Burgess a > écrit > > > : > > > > > > > > > +1 for a 1.6.0 release > > > > > > > > > > On Mon, May 20, 2024 at 1:25 PM Bryan Bende > wrote: > > > > > > > > > > > Thanks Pierre. I agree it would be good to kick out a release. I > > > would > > > > > > lean towards 1.6.0 since the commits seem to be improvements > rather > > > > > > than bug fixes for a patch. > > > > > > > > > > > > On Mon, May 20, 2024 at 1:08 PM Pierre Villard > > > > > > wrote: > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > We just merged a couple of improvements for the NiFi NAR Maven > > > Plugin > > > > > and > > > > > > > another improvement has been merged some time ago for which the > > > > > community > > > > > > > expressed interest [1]. I think it could be a good time to > release > > > a > > > > > new > > > > > > > version, either as 1.5.2 or as 1.6.0. Once we have a new > release, > > > there > > > > > > > will be a need for updating the NiFi code base and I have a PR > > > ready > > > > > for > > > > > > > that [2]. > > > > > > > > > > > > > > I'm happy to help with the release process. > > > > > > > > > > > > > > Do you agree it is time for a new release or do you see > additional > > > > > > changes > > > > > > > we should make? > > > > > > > > > > > > > > Thanks, > > > > > > > Pierre > > > > > > > > > > > > > > [1] https://github.com/apache/nifi-maven/pull/35 > > > > > > > [2] https://issues.apache.org/jira/browse/NIFI-13267 > > > > > > > > > > > > > > >
[jira] [Updated] (NIFI-13273) Release NiFi NAR Maven Plugin 2.0.0
[ https://issues.apache.org/jira/browse/NIFI-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13273: -- Summary: Release NiFi NAR Maven Plugin 2.0.0 (was: Release NiFi NAR Maven Plugin 1.6.0) > Release NiFi NAR Maven Plugin 2.0.0 > --- > > Key: NIFI-13273 > URL: https://issues.apache.org/jira/browse/NIFI-13273 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: nifi-nar-maven-plugin-1.6.0 > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Fix For: nifi-nar-maven-plugin-1.6.0 > > > Release NiFi NAR Maven Plugin 1.6.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Release NiFi NAR Maven Plugin
Given that the flow analysis rules are something specific to NiFi 2, and given that this is an opportunity to do some house cleaning, I'll go directly to a 2.0.0 release directly and have it used only on the main branch. The things that have been added as improvements are not really required for 1.x anyway. Le mar. 21 mai 2024 à 15:40, David Handermann a écrit : > Thanks Pierre, sounds good! > > Regards, > David Handermann > > On Tue, May 21, 2024 at 8:36 AM Pierre Villard > wrote: > > > > Thanks will go ahead with 1.6.0 release process > > > > Le lun. 20 mai 2024 à 19:30, Matt Burgess a écrit > : > > > > > +1 for a 1.6.0 release > > > > > > On Mon, May 20, 2024 at 1:25 PM Bryan Bende wrote: > > > > > > > Thanks Pierre. I agree it would be good to kick out a release. I > would > > > > lean towards 1.6.0 since the commits seem to be improvements rather > > > > than bug fixes for a patch. > > > > > > > > On Mon, May 20, 2024 at 1:08 PM Pierre Villard > > > > wrote: > > > > > > > > > > Hi all, > > > > > > > > > > We just merged a couple of improvements for the NiFi NAR Maven > Plugin > > > and > > > > > another improvement has been merged some time ago for which the > > > community > > > > > expressed interest [1]. I think it could be a good time to release > a > > > new > > > > > version, either as 1.5.2 or as 1.6.0. Once we have a new release, > there > > > > > will be a need for updating the NiFi code base and I have a PR > ready > > > for > > > > > that [2]. > > > > > > > > > > I'm happy to help with the release process. > > > > > > > > > > Do you agree it is time for a new release or do you see additional > > > > changes > > > > > we should make? > > > > > > > > > > Thanks, > > > > > Pierre > > > > > > > > > > [1] https://github.com/apache/nifi-maven/pull/35 > > > > > [2] https://issues.apache.org/jira/browse/NIFI-13267 > > > > > > > >
[jira] [Created] (NIFI-13274) Dependencies housekeeping for NiFi NAR Maven plugin
Pierre Villard created NIFI-13274: - Summary: Dependencies housekeeping for NiFi NAR Maven plugin Key: NIFI-13274 URL: https://issues.apache.org/jira/browse/NIFI-13274 Project: Apache NiFi Issue Type: Task Reporter: Pierre Villard Assignee: Pierre Villard Fix For: nifi-nar-maven-plugin-1.6.0 Dependencies housekeeping for NiFi NAR Maven plugin Also fixing the SNAPSHOT version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13273) Release NiFi NAR Maven Plugin 1.6.0
Pierre Villard created NIFI-13273: - Summary: Release NiFi NAR Maven Plugin 1.6.0 Key: NIFI-13273 URL: https://issues.apache.org/jira/browse/NIFI-13273 Project: Apache NiFi Issue Type: Task Components: Tools and Build Affects Versions: nifi-nar-maven-plugin-1.6.0 Reporter: Pierre Villard Assignee: Pierre Villard Fix For: nifi-nar-maven-plugin-1.6.0 Release NiFi NAR Maven Plugin 1.6.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Release NiFi NAR Maven Plugin
Thanks will go ahead with 1.6.0 release process Le lun. 20 mai 2024 à 19:30, Matt Burgess a écrit : > +1 for a 1.6.0 release > > On Mon, May 20, 2024 at 1:25 PM Bryan Bende wrote: > > > Thanks Pierre. I agree it would be good to kick out a release. I would > > lean towards 1.6.0 since the commits seem to be improvements rather > > than bug fixes for a patch. > > > > On Mon, May 20, 2024 at 1:08 PM Pierre Villard > > wrote: > > > > > > Hi all, > > > > > > We just merged a couple of improvements for the NiFi NAR Maven Plugin > and > > > another improvement has been merged some time ago for which the > community > > > expressed interest [1]. I think it could be a good time to release a > new > > > version, either as 1.5.2 or as 1.6.0. Once we have a new release, there > > > will be a need for updating the NiFi code base and I have a PR ready > for > > > that [2]. > > > > > > I'm happy to help with the release process. > > > > > > Do you agree it is time for a new release or do you see additional > > changes > > > we should make? > > > > > > Thanks, > > > Pierre > > > > > > [1] https://github.com/apache/nifi-maven/pull/35 > > > [2] https://issues.apache.org/jira/browse/NIFI-13267 > > >
[jira] [Commented] (NIFI-13268) ConsumeGCPPubSub Stopped Working with M3
[
https://issues.apache.org/jira/browse/NIFI-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847969#comment-17847969
]
Pierre Villard commented on NIFI-13268:
---
The same problem seems to happen across GCP processors. I have the same error
with PutBigQuery.
> ConsumeGCPPubSub Stopped Working with M3
>
>
> Key: NIFI-13268
> URL: https://issues.apache.org/jira/browse/NIFI-13268
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Emin
>Priority: Major
>
> After updating from 1.26 to 2.0.0M3, getting the following for
> `ConsumeGCPPubSub`
>
> {{nifi-1 | 2024-05-18 23:00:02,157 INFO [NiFi Web Server-405]
> o.a.n.c.s.StandardProcessScheduler Running once
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,158 INFO [NiFi Web Server-405]
> o.a.n.controller.StandardProcessorNode Starting
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] nifi-1 | 2024-05-18
> 23:00:02,170 ERROR [Timer-Driven Process Thread-1]
> o.a.n.p.gcp.pubsub.ConsumeGCPubSub
> ConsumeGCPubSub[id=890ffc25-018f-1000-f9bc-b29966db3fc6] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | java.util.ServiceConfigurationError:
> io.grpc.ManagedChannelProvider:
> io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider Unable to get
> public no-arg constructor nifi-1 | at
> java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) nifi-1 | at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:679)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1240)
> nifi-1 | at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> nifi-1 | at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) nifi-1 |
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> nifi-1 | at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:67) nifi-1
> | at
> io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:101)
> nifi-1 | at
> io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:43)
> nifi-1 | at
> io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:44)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:404)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.(ChannelPool.java:107)
> nifi-1 | at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:85)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:243)
> nifi-1 | at
> com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:237)
> nifi-1 | at
> com.google.api.gax.rpc.ClientContext.create(ClientContext.java:230) nifi-1 |
> at
> com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:287)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.getSubscriber(ConsumeGCPubSub.java:286)
> nifi-1 | at
> org.apache.nifi.processors.gcp.pubsub.ConsumeGCPubSub.onScheduled(ConsumeGCPubSub.java:134)
> nifi-1 | at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> nifi-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
> nifi-1 | at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
> nifi-1 | at
> org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$10(StandardProcessorNode.java:1668)
> nifi-1 | at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)
> nifi-1 | at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) nifi-1 |
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$Sched
[DISCUSS] Release NiFi NAR Maven Plugin
Hi all, We just merged a couple of improvements for the NiFi NAR Maven Plugin and another improvement has been merged some time ago for which the community expressed interest [1]. I think it could be a good time to release a new version, either as 1.5.2 or as 1.6.0. Once we have a new release, there will be a need for updating the NiFi code base and I have a PR ready for that [2]. I'm happy to help with the release process. Do you agree it is time for a new release or do you see additional changes we should make? Thanks, Pierre [1] https://github.com/apache/nifi-maven/pull/35 [2] https://issues.apache.org/jira/browse/NIFI-13267
[jira] [Created] (NIFI-13267) Bump NiFi NAR Maven plugin version
Pierre Villard created NIFI-13267: - Summary: Bump NiFi NAR Maven plugin version Key: NIFI-13267 URL: https://issues.apache.org/jira/browse/NIFI-13267 Project: Apache NiFi Issue Type: Task Reporter: Pierre Villard Assignee: Pierre Villard Once we have a new version of the NiFi NAR Maven Plugin, we'll need to bump the version in the NiFi code base and make some changes to account for the new extension types (Parameter Providers and Flow Analysis Rules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13264) Attach extension-manifest.xml to build as an artifact
[ https://issues.apache.org/jira/browse/NIFI-13264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13264: -- Fix Version/s: nifi-nar-maven-plugin-1.5.2 Resolution: Fixed Status: Resolved (was: Patch Available) > Attach extension-manifest.xml to build as an artifact > - > > Key: NIFI-13264 > URL: https://issues.apache.org/jira/browse/NIFI-13264 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: nifi-nar-maven-plugin-1.5.1 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: nifi-nar-maven-plugin-1.5.2 > > Time Spent: 20m > Remaining Estimate: 0h > > The NAR Maven Plugin produces extension-manifest.xml and bundles it into the > resulting NAR file. It would beneficial to directly attach this file to the > build so that it can be published on releases and retrieved from a Maven > repository without retrieving the entire NAR which can be quite large. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12226) Maven plugin: Add flag to skip doc generation
[ https://issues.apache.org/jira/browse/NIFI-12226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-12226. --- Fix Version/s: nifi-nar-maven-plugin-1.5.2 Resolution: Fixed > Maven plugin: Add flag to skip doc generation > - > > Key: NIFI-12226 > URL: https://issues.apache.org/jira/browse/NIFI-12226 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nicolò Boschi >Priority: Minor > Fix For: nifi-nar-maven-plugin-1.5.2 > > Time Spent: 1h > Remaining Estimate: 0h > > In [LangStream/langstream|https://github.com/LangStream/langstream] we use > this plugin for producing NAR files. The NAR are then ingested by the > LangStream runtime and the documentation is not needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13258) NAR Plugin - add parameter providers and flow analysis rules extension types
[ https://issues.apache.org/jira/browse/NIFI-13258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13258: -- Status: Patch Available (was: Open) > NAR Plugin - add parameter providers and flow analysis rules extension types > > > Key: NIFI-13258 > URL: https://issues.apache.org/jira/browse/NIFI-13258 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Fix For: nifi-nar-maven-plugin-1.5.2 > > Time Spent: 10m > Remaining Estimate: 0h > > Improve the NiFi NAR Maven plugin to add Parameter Providers and Flow > Analysis Rules in the extension types. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13258) NAR Plugin - add parameter providers and flow analysis rules extension types
Pierre Villard created NIFI-13258: - Summary: NAR Plugin - add parameter providers and flow analysis rules extension types Key: NIFI-13258 URL: https://issues.apache.org/jira/browse/NIFI-13258 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: Pierre Villard Assignee: Pierre Villard Fix For: nifi-nar-maven-plugin-1.5.2 Improve the NiFi NAR Maven plugin to add Parameter Providers and Flow Analysis Rules in the extension types. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [discuss] What to do with the Cassandra components
Hey guys, what's the latest on this? Le sam. 23 mars 2024 à 01:49, Mike Thomsen a écrit : > Fair enough, Joe. > > Matt, > > I poked around your branch a little this evening. I agree with you and > David 100% now on the need for some sort of abstraction. I think the Graph > Bundle's model could serve as a good starting point for how to approach the > problem. The client drivers in that bundle do the heavy lifting of doing > the querying and passing the results back via a callback system to the > processors that call them to ensure that the processors don't know anything > other than they're getting back a Map of result data each iteration. > Thoughts? > > On Fri, Mar 22, 2024 at 2:48 PM Joe Witt wrote: > > > Mike, > > > > The bundles we include cannot have libs with know vulns and that last a > > very long time. That is a more pressing blocker. > > > > As noted top of thread we all recognize the importance of being able to > > integrate with Cassandra but including that must come with active mx > > especially as it relates to vulns. > > > > Thanks > > Joe > > > > On Fri, Mar 22, 2024 at 11:42 AM Mike Thomsen > > wrote: > > > > > The scope tag was probably copy pasta. You raise a valid point about > the > > > processor dependencies that completely slipped my mind. That said, I > > think > > > it's premature to remove the cassandra bundle until we have a working > > > replacement. I would consider that support a blocker for 2.X. > > > > > > On Fri, Mar 22, 2024 at 12:00 PM Matt Burgess > > wrote: > > > > > > > David beat me to it :) IMO the only NAR that should have any > > dependencies > > > > on Cassandra is the services NAR, not the processors or services API. > > > > > > > > On Fri, Mar 22, 2024 at 11:10 AM David Handermann < > > > > exceptionfact...@apache.org> wrote: > > > > > > > > > Mike, > > > > > > > > > > Thanks for sharing the branch, it is helpful to have that as a > > > > > reference example. Have you been able to exercise any of that > > approach > > > > > at runtime? > > > > > > > > > > Based on what is there right now, attempting to mark the DataStax > > > > > java-driver-core as provided does not look like it will work. It > may > > > > > pass unit tests, but runtime NAR class loading requires that > classes > > > > > be available in the same NAR, or in a parent NAR. That means when > > NiFi > > > > > tries to load the Controller Service interface, it must have access > > to > > > > > a version of the relevant Cassandra driver classes. By marking the > > > > > dependency as provided, it will not be available in the API NAR, > and > > > > > thus not available when loading the service interface. Including it > > in > > > > > the API NAR won't work either, because it conflicts with the > ScyllaDB > > > > > java-driver-core in the implementation NAR. > > > > > > > > > > This is the reason Matt and I highlighted for providing a layer of > > > > > abstraction at the Controller Service API level. > > > > > > > > > > Regards, > > > > > David Handermann > > > > > > > > > > On Fri, Mar 22, 2024 at 8:13 AM Mike Thomsen < > mikerthom...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > Work so far: > https://github.com/MikeThomsen/nifi/tree/cql-changes > > > > > > > > > > > > On Thu, Mar 21, 2024 at 9:52 AM Mike Thomsen < > > mikerthom...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > > > Matt/David, > > > > > > > > > > > > > > By this evening, I should be at a point where I can share my > > > branch. > > > > It > > > > > > > should be far enough along that y'all can see what I mean about > > how > > > > > most of > > > > > > > the changes really weren't that complicated. My sense is that > if > > we > > > > > > > collaborate on it, we can probably get it ready for a PR > within a > > > > week > > > > > or > > > > > > > two. > > > > > > > > > > > > > > It would probably be a good idea to plan to revisit the > Cassandra > > > > DMC's > > > > > > > design and make it more flexible. > > > > > > > > > > > > > > One nice thing about the new DataStax driver is that it > supports > > > > > > > configuration by a very detailed configuration file format, so > we > > > can > > > > > give > > > > > > > users that option + combine it with EL/parameters (I envision > an > > > > option > > > > > > > where the user puts EL in the file, we load the file, > preprocess > > > the > > > > > EL and > > > > > > > load that into the driver) > > > > > > > > > > > > > > On Wed, Mar 20, 2024 at 4:01 PM Mike Thomsen < > > > mikerthom...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> If it were that simple, they would probably have just gone > with > > > that > > > > > > >> solution. That said, the API is functionally vendor agnostic > at > > > this > > > > > point > > > > > > >> at the Java API level. So I see no need to add abstraction > above > > > > > that. I've > > > > > > >> got probably 2/3 of nifi-cassandra-bundle converted. Hitting a > > few > > > > > pain > > > > > > >>
[jira] [Resolved] (NIFI-13256) Python WriteHelloWorld Example has Incorrect Attributes
[ https://issues.apache.org/jira/browse/NIFI-13256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-13256. --- Fix Version/s: 2.0.0-M4 Resolution: Fixed > Python WriteHelloWorld Example has Incorrect Attributes > --- > > Key: NIFI-13256 > URL: https://issues.apache.org/jira/browse/NIFI-13256 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website >Affects Versions: 2.0.0-M2 >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0-M4 > > Time Spent: 20m > Remaining Estimate: 0h > > The Python Developer Guide includes an example WriteHelloWorld Processor > which defines a set instead of a dictionary for FlowFile attributes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release Apache NiFi 2.0.0-M3 (RC1)
+1 (binding) Went through the usual steps, deployed a secured cluster with a secured NiFi Registry instance on GCP and tested a bunch of flows that I have around. Thanks for RM'ing David! Le jeu. 16 mai 2024 à 15:26, Matt Burgess a écrit : > +1 (binding) > > Ran through release helper, verified various flows. Thanks for RM'ing > David! > > On Mon, May 13, 2024 at 11:48 PM David Handermann < > exceptionfact...@apache.org> wrote: > > > Team, > > > > I am pleased to be calling this vote for the source release of Apache > > NiFi 2.0.0-M3. > > > > Please review the following guide for how to verify a release candidate > > build: > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification > > > > The source being voted on and the convenience binaries are available > > on the Apache Distribution Repository: > > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-2.0.0-M3 > > > > The build artifacts are available on the Apache Nexus Repository: > > > > https://repository.apache.org/content/repositories/orgapachenifi-1269 > > > > Git Tag: nifi-2.0.0-M3-RC1 > > Git Commit ID: f2215c6522a5571189290760c55f0317f8562cbd > > GitHub Commit Link: > > > > > https://github.com/apache/nifi/commit/f2215c6522a5571189290760c55f0317f8562cbd > > > > Checksums of nifi-2.0.0-M3-source-release.zip > > > > SHA256: d471a0a9e4e04faf13bbe13c7d83be6f8002fba598bf0968a3c1b339802a185a > > SHA512: > > > cd0b8bbd3fe4ea7ebe8cdac6ac8a7afa97fd7b6a521c2cbcb2c0cdc94899b652bf3726c8fe432e16f44a9dc81907414bbb42e03113f0cd9fb51aa1de9cd727a7 > > > > Release artifacts are signed with the following key: > > > > https://people.apache.org/keys/committer/exceptionfactory.asc > > > > KEYS file is available on the Apache Distribution Repository: > > > > https://dist.apache.org/repos/dist/release/nifi/KEYS > > > > Issues resolved for this version: 411 > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12354155 > > > > Release note highlights can be found on the project wiki: > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.0.0-M3 > > > > The vote will be open for 72 hours. > > > > Please download the release candidate and evaluate the necessary items > > including checking hashes, signatures, build from source, and test. > > Then please vote: > > > > [] +1 Release this package as nifi-2.0.0-M3 > > [] +0 no opinion > > [] -1 Do not release this package because... > > >
Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.99.0 (RC4)
+1 (binding) Went through the usual steps Built on MacOS X. Tested a basic flow tailing a local file, doing some transformations and sending data with InvokeHTTP to a NiFi instance with FlowFile format. Thanks team for taking care of this release and the exciting upcoming 1.0. Le mer. 15 mai 2024 à 17:30, Ferenc Gerlits a écrit : > +1 (non-binding) > > verified signature and hashes > installed the Windows MSI and ran the service on Windows > compiled on Fedora 40 and ran ExecuteScript with lua > > It mostly worked as expected. I found two minor issues: > https://issues.apache.org/jira/browse/MINIFICPP-2374, > https://issues.apache.org/jira/browse/MINIFICPP-2375, but neither of > these is a showstopper, we'll fix them in the next release. > > Thank you for RMing! > Ferenc >
[jira] [Updated] (NIFI-13238) Checkstyle - rules for whitespace
[ https://issues.apache.org/jira/browse/NIFI-13238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13238: -- Status: Patch Available (was: Open) > Checkstyle - rules for whitespace > - > > Key: NIFI-13238 > URL: https://issues.apache.org/jira/browse/NIFI-13238 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Adding the following rules: > * WhitespaceAfter > * NoWhitespaceAfter > * NoWhitespaceBefore > * WhitespaceAround -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13238) Checkstyle - rules for whitespace
Pierre Villard created NIFI-13238: - Summary: Checkstyle - rules for whitespace Key: NIFI-13238 URL: https://issues.apache.org/jira/browse/NIFI-13238 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: Pierre Villard Assignee: Pierre Villard Adding the following rules: * WhitespaceAfter * NoWhitespaceAfter * NoWhitespaceBefore * WhitespaceAround -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13237) Checkstyle improvements
Pierre Villard created NIFI-13237: - Summary: Checkstyle improvements Key: NIFI-13237 URL: https://issues.apache.org/jira/browse/NIFI-13237 Project: Apache NiFi Issue Type: Epic Reporter: Pierre Villard Assignee: Pierre Villard Epic to track the addition of some rules in the checkstyle configuration of the NiFi project. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13210) Add Codecov Token to GitHub Actions
[
https://issues.apache.org/jira/browse/NIFI-13210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13210:
--
Fix Version/s: 2.0.0-M3
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add Codecov Token to GitHub Actions
> ---
>
> Key: NIFI-13210
> URL: https://issues.apache.org/jira/browse/NIFI-13210
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Codecov provides free services for unit test code coverage reporting and
> recent versions of the GitHub Action require a Codecov Token for reporting
> status. Apache Infra has assigned the {{CODECOV_TOKEN}} secret to the GitHub
> repository, so it can be used in GitHub workflows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: Azure lib issue in 1.26
This will likely motivate a 1.26.1 release. I should be able to do that next week as I'm traveling for the rest of this week. Thanks, Pierre Le jeu. 9 mai 2024 à 05:01, Joe Witt a écrit : > Eduardo > > Yes this was found/fixed today[1] > > Sorry for the trouble > > [1] https://issues.apache.org/jira/browse/NIFI-13181 > > Thanks > Joe > > On Wed, May 8, 2024 at 6:59 PM Eduardo Fontes > wrote: > > > Hi! > > > > Someone with errors like that em Azure components in Nifi 1.26? > > > > java.lang.NoSuchMethodError: > > 'com.microsoft.aad.msal4j.AbstractApplicationBase$Builder > > > > > com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(boolean)' > > > > I see thar msal lib was updated since 1.25. > > > > Best regards. > > Eduardo Fontes > > >
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12231: -- Fix Version/s: (was: 1.27.0) > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.23.2 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 40m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13133) Conduct Apache NiFi 1.26.0 Release
[ https://issues.apache.org/jira/browse/NIFI-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-13133. --- Resolution: Done > Conduct Apache NiFi 1.26.0 Release > -- > > Key: NIFI-13133 > URL: https://issues.apache.org/jira/browse/NIFI-13133 > Project: Apache NiFi > Issue Type: Task > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Fix For: 1.26.0 > > > Conduct Apache NiFi 1.26.0 Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[RESULT][VOTE] Release Apache NiFi 1.26.0 (RC1)
Apache NiFi Community, I am pleased to announce that the 1.26.0 release of Apache NiFi passes: 10 +1 (binding) votes 7 +1 (non-binding) votes 0 0 votes 0 -1 votes Thanks to all who helped make this release possible! The release artifacts will be published in the next 24 hours. Here is the vote thread: https://lists.apache.org/thread/8tqrzlfq289nf4hlm98ocsnqk88fkqo6 Thanks, Pierre
Re: [VOTE] Release Apache NiFi 1.26.0 (RC1)
+1 (binding) Went through the usual steps and verified the RC with a 3-nodes secured NiFi cluster + secured NiFi Registry and some flows. Closing the vote and proceeding with the release. Thanks, Pierre Le lun. 6 mai 2024 à 19:05, Dan S a écrit : > +1 (non-binding) > > Went through the helper guide and did a clean build > Verified signatures and hashes > Built on Linux 4.18.0-513.24.1.el8_9.x86_64 > Java version: Java openjdk version 17.0.11 > Apache Maven 3.9.6 > > Verified tickets > [NIFI-12785] InvokeHTTP handler should not urlencode HTTP URL > [NIFI-12842] InvokeHTTP version wrong encoding of % in URL > [NIFI-12677] ExcelReader documentation gives an example of a non-existent > strategy > > Thanks for RMing Pierre! > > On Mon, May 6, 2024 at 3:52 PM Joe Witt wrote: > > > +1 binding. > > > > Thanks Pierre! > > > > On Mon, May 6, 2024 at 8:08 AM Krisztina Zsihovszki > > wrote: > > > > > +1 (non-binding) > > > > > > - Verified signatures and hashes > > > - Ran build using Maven 3.8.5 on macOS 13.5 with 8.0.392-zulu > > > - Contrib check, unit tests passed > > > > > > Verified these tickets: > > > > > > [NIFI-12677] - ExcelReader documentation gives an example of a > > non-existent > > > strategy > > > [NIFI-12960] - Support reading password-protected files in ExcelReader > > > > > > > > > Thanks for RM'ing, Pierre! > > > > > > Krisztina > > > > > > On Mon, May 6, 2024 at 4:55 PM Michael Moser > wrote: > > > > > > > +1 (binding) > > > > > > > > verified signature, hashes, commit ID > > > > had a glance at the README, LICENSE, NOTICE files > > > > > > > > built on: > > > > Apache Maven 3.9.6 Maven home: > > > > /home/mwmoser/.m2/wrapper/dists/apache-maven-3.9.6/a53741d1 > > > > Java version: 1.8.0_412, vendor: Red Hat, Inc., runtime: > > > > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-2.el8.x86_64/jre > > > > Default locale: en_US, platform encoding: UTF-8 > > > > OS name: "linux", version: "4.18.0-513.24.1.el8_9.x86_64", arch: > > "amd64", > > > > family: "unix" > > > > > > > > Ran some simple flows and performed some operations against NiFi > > > Registry. > > > > > > > > Many thanks to Pierre for being RM. > > > > > > > > -- Mike > > > > > > > > > > > > On Mon, May 6, 2024 at 10:25 AM Peter Turcsanyi < > turcsa...@apache.org> > > > > wrote: > > > > > > > > > +1 (binding) > > > > > > > > > > Verified signatures and hashes. > > > > > Built NiFi on Ubuntu 22.04 with Java 8 (Temurin 1.8.0_412+8) and > Java > > > 11 > > > > > (Temurin 11.0.23+9), Maven 3.9.6 with empty local repo. > > > > > > > > > > Ran flows for validating: > > > > > - NIFI-12837 Add DFS setting to smb processors > > > > > - NIFI-12732 ListS3 should reset its tracking state after > > > configuration > > > > > change > > > > > - NIFI-12936 ListGCSBucket should reset its tracking state after > > > > > configuration change > > > > > - NIFI-12928 Add Simple Write strategy in PutAzureDataLakeStorage > > > > > > > > > > Thanks for RMing Pierre! > > > > > > > > > > Regards, > > > > > Peter Turcsanyi > > > > > > > > > > On Mon, May 6, 2024 at 4:09 PM Chris Sampson > > > > > wrote: > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > Ran through the release helper. Verified signatures and hashes. > > > > > > > > > > > > Full clean build of RC with Java openjdk 11.0.23 2024-04-16 on > > macOS > > > > > > Sonoma 14.4.1 using the Maven Wrapper 3.9.6. > > > > > > > > > > > > Ran a selection of processors/flows, primarily focussed on > > > > Elasticsearch > > > > > > integration and HTTP Request/Response. > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > --- > > > > > > Chris Sampson > > > > > > IT Consultant > > > > > > chris.samp..
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12231: -- Fix Version/s: 1.27.0 > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.23.2 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 2.0.0-M3, 1.27.0 > > Time Spent: 40m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12231: -- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.23.2 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 40m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12973) Add Process Group scope to Flow Analysis rules
[ https://issues.apache.org/jira/browse/NIFI-12973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-12973. --- Fix Version/s: 2.0.0-M3 Resolution: Fixed > Add Process Group scope to Flow Analysis rules > -- > > Key: NIFI-12973 > URL: https://issues.apache.org/jira/browse/NIFI-12973 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Pierre Villard >Assignee: Tamas Palfy >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 50m > Remaining Estimate: 0h > > I think it'd be useful to have an optional property in all Flow Analysis > rules to scope a rule execution to a specific process group (and its embedded > process groups). > Most NiFi users are using NiFi in a multi tenant way and are dedicating a > process group to a given team. Different rules may apply across different > teams hence the advantage of scoping a rule to a process group (when it makes > sense for the rule). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13008) CLI command to upgrade all instances of a versioned flow
[ https://issues.apache.org/jira/browse/NIFI-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13008: -- Fix Version/s: 1.27.0 (was: 1.26.0) > CLI command to upgrade all instances of a versioned flow > > > Key: NIFI-13008 > URL: https://issues.apache.org/jira/browse/NIFI-13008 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 2.0.0-M3, 1.27.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Add a CLI command allowing someone to upgrade all the instances of a given > versioned flow. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13138) Add extensions name and description in NiFi Registry
[ https://issues.apache.org/jira/browse/NIFI-13138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13138: -- Status: Patch Available (was: Open) > Add extensions name and description in NiFi Registry > > > Key: NIFI-13138 > URL: https://issues.apache.org/jira/browse/NIFI-13138 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Similar to NIFI-13050, it'd be extremely useful, for a given bundle, in NiFi > Registry, to display the included extensions in that bundle. For each > extension, we want to display the name and associated description. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13138) Add extensions name and description in NiFi Registry
Pierre Villard created NIFI-13138: - Summary: Add extensions name and description in NiFi Registry Key: NIFI-13138 URL: https://issues.apache.org/jira/browse/NIFI-13138 Project: Apache NiFi Issue Type: Improvement Components: NiFi Registry Reporter: Pierre Villard Assignee: Pierre Villard Similar to NIFI-13050, it'd be extremely useful, for a given bundle, in NiFi Registry, to display the included extensions in that bundle. For each extension, we want to display the name and associated description. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13137) Switch to Zulu for MacOS Java 8 action
[ https://issues.apache.org/jira/browse/NIFI-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13137: -- Status: Patch Available (was: Open) > Switch to Zulu for MacOS Java 8 action > -- > > Key: NIFI-13137 > URL: https://issues.apache.org/jira/browse/NIFI-13137 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.26.0 > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Temurin Java 8 is not available for macOS 14 on AArch64. The most > straightforward solution would be switching to some other JDK, such as Azul. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13137) Switch to Zulu for MacOS Java 8 action
Pierre Villard created NIFI-13137: - Summary: Switch to Zulu for MacOS Java 8 action Key: NIFI-13137 URL: https://issues.apache.org/jira/browse/NIFI-13137 Project: Apache NiFi Issue Type: Task Components: Tools and Build Affects Versions: 1.26.0 Reporter: Pierre Villard Assignee: Pierre Villard Temurin Java 8 is not available for macOS 14 on AArch64. The most straightforward solution would be switching to some other JDK, such as Azul. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[VOTE] Release Apache NiFi 1.26.0 (RC1)
Team, I am pleased to be calling this vote for the source release of Apache NiFi 1.26.0. Please review the following guide for how to verify a release candidate build: https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification The source being voted on the and the convenience binaries are available on the Apache Distribution Repository: https://dist.apache.org/repos/dist/dev/nifi/nifi-1.26.0 The build artifacts are available on the Apache Nexus Repository: https://repository.apache.org/content/repositories/orgapachenifi-1265 Git Tag: nifi-1.26.0-RC1 Git Commit ID: 338083f6850b61cd2651e41bbd73b3cb5330d98c GitHub Commit Link: https://github.com/apache/nifi/commit/338083f6850b61cd2651e41bbd73b3cb5330d98c Checksums of nifi-1.26.0-source-release.zip SHA256: 75ea201c12bb99672b1f075c9520b5f7df8b24e033ec47121848b351a0d47790 SHA512: 5b754d899ce8cff900d5871d44c2fda9224e69fe8a0fe9a7121f3359ed8881300e4d4d0b2fe5befea276e0495c7ebbed04cc307c18527aa7a1cea25a923a Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pvillard.asc KEYS file is available on the Apache Distribution Repository: https://dist.apache.org/repos/dist/release/nifi/KEYS Issues resolved for this version: 128 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12354185 Release note highlights can be found on the project wiki: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.26.0 The vote will be open for 72 hours. Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote: [] +1 Release this package as nifi-1.26.0 [] +0 no opinion [] -1 Do not release this package because...
[jira] [Created] (NIFI-13133) Conduct Apache NiFi 1.26.0 Release
Pierre Villard created NIFI-13133: - Summary: Conduct Apache NiFi 1.26.0 Release Key: NIFI-13133 URL: https://issues.apache.org/jira/browse/NIFI-13133 Project: Apache NiFi Issue Type: Task Reporter: Pierre Villard Assignee: Pierre Villard Fix For: 1.26.0 Conduct Apache NiFi 1.26.0 Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Release Timeline for NiFi 2.0.0-M3
I'm starting the process for the 1.26 RC as I don't believe we're waiting for any additional PRs there. Le jeu. 2 mai 2024 à 07:30, Joe Witt a écrit : > Matt > > You are referring to NIFI-12998 and it causing rebasing to be necessary for > outstanding PRs? > > The rebase should be quite straight forward for most cases. > > If you have any questions Im happy to help. > > I rebased a tremendous number of times to keep up with main changing while > the PR took several weeks to create and test and validate against previous > builds. Also rebased repeatedly during the review and feedback stage. As > a result I got to practice my rebase skills about 20 times across 50 or > more commits to main during that time. > > On your pr branch > > git fetch —all > git rebase upstream/main > git status > fix any conflicts > git add . > git rebase —continue > > If any conflicts it would likely be for new modules created in the pr which > would just be moving from a path in nifi-nar-bundles to > nifi-extension-bundles. > > If you have a pr in mind you have a question on for the rebase let me know. > > Thanks > Joe > > On Wed, May 1, 2024 at 9:51 PM Matt Burgess wrote: > > > One thing that was mentioned was an included Jira for "providing a > clearer > > distinction between framework and > > extensions," This involved moving extensions to a new module and for > > community folks this is causing problems with any > > new improvements in-flight before that. Seems like it needed more > > announcement than an upcoming RC due to the impact? > > > > On Mon, Apr 29, 2024 at 10:22 AM David Handermann < > > exceptionfact...@apache.org> wrote: > > > > > Team, > > > > > > We are getting closer to ready for a release candidate build. Based on > > > some conversations with those working on the user interface, there are > > > still a couple more items to complete this week. Getting a solid build > > > of the new UI into this next version will be very helpful, so getting > > > a few more of those issues resolved should put things in a good > > > position. > > > > > > Regards, > > > David Handermann > > > > > > On Wed, Apr 24, 2024 at 8:32 AM Joe Witt wrote: > > > > > > > > Also agreed there. Profile should be there to exclude perhaps but > > > include > > > > should be default. > > > > > > > > On Wed, Apr 24, 2024 at 6:30 AM David Handermann < > > > > exceptionfact...@apache.org> wrote: > > > > > > > > > Thanks for the replies thus far! > > > > > > > > > > With the goal of including the new UI, it seems like a good time to > > > > > move the module from an optional profile to part of the standard > > > > > build. That would make the release candidate build and review > process > > > > > simpler, not requiring the optional build profile. > > > > > > > > > > Regards, > > > > > David Handermann > > > > > > > > > > On Tue, Apr 23, 2024 at 2:42 PM Matt Gilman < > matt.c.gil...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > I agree. Including the updated UI and getting some feedback would > > be > > > > > great. > > > > > > > > > > > > On Mon, Apr 22, 2024 at 8:50 PM Joe Witt > > wrote: > > > > > > > > > > > > > Id be a big fan of including the new UI. > > > > > > > > > > > > > > On Mon, Apr 22, 2024 at 5:45 PM Pierre Villard < > > > > > > > pierre.villard...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Thanks David, I'm happy to take care of a 1.26 release at the > > > same > > > > > time. > > > > > > > > > > > > > > > > Regarding the M3 release, should the new UI be included and > > > > > instructions > > > > > > > > given on how to access it to start collecting feedback? I'm > > > under the > > > > > > > > impression that almost all of the work has been done, no? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Pierre > > > > > > > > > > > > > > > > Le mar. 23 avr.
[jira] [Resolved] (NIFI-13131) Dependency hygiene
[ https://issues.apache.org/jira/browse/NIFI-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-13131. --- Resolution: Fixed > Dependency hygiene > --- > > Key: NIFI-13131 > URL: https://issues.apache.org/jira/browse/NIFI-13131 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.26.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Related to NIFI-13108 it was asked to do some similar updates. > I will pick off the safest looking ones. The NiFi 1.x line does not have the > same dependency tree/pom structure protections now established in the 2.x > line from NIFI-12998 so such dependency hygiene efforts will likely require > more and more specific scrutiny and effort going forward. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12831) Add PutOpenSearchVector and QueryOpenSearchVector processors
[ https://issues.apache.org/jira/browse/NIFI-12831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-12831. --- Fix Version/s: 2.0.0-M3 Resolution: Fixed > Add PutOpenSearchVector and QueryOpenSearchVector processors > > > Key: NIFI-12831 > URL: https://issues.apache.org/jira/browse/NIFI-12831 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Create vector store specific put and query processors for OpenSearch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13097) Set Project Version in Python Extension Processors
[
https://issues.apache.org/jira/browse/NIFI-13097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-13097.
---
Resolution: Fixed
> Set Project Version in Python Extension Processors
> --
>
> Key: NIFI-13097
> URL: https://issues.apache.org/jira/browse/NIFI-13097
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Python Processors in the {{nifi-python-extensions}} module have the version
> field set to {{2.0.0-SNAPSHOT}} in the Processor Details section of the
> Python class. Although this works while the main branch remains on the
> snapshot version, it does not support applying the project version to these
> Processors using standard release processes.
> The version field can use a placeholder that will be populated during the
> Maven module build process to align the Processor version field with the
> released Maven module version, without other manual changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: Python Processor and relationships
Hi, Are you in a position to share the code of your python processor? Did you implement getRelationships(self) in your processor? Le ven. 26 avr. 2024, 06:52, Ashwini Kumar a écrit : > I created a Python Nifi Processor just as in the standard python > documentation. This is NOT an ExecuteScript processor.I am able to load the > python processor in Nifi but when I click the Configure Processor -> > Relationships tab, I see no relationships there. This renders the Python > processor useless. We cannot forward the resulting flowfile to any other > downstream processors. The Error I get when trying to do so is -> > "WriteHelloWorld' does not support any relationships.The basic example in > the documentation that I am using returns FlowFileTransformResult with a > success relationship but this being a scripted language, NIFI may not know > in advance what the relationships may be.Am I missing any steps in defining > relationships for python processors?Steps:Created python processor in > NifiLoaded it in NifiTried to connected it to PutFile processor Got message > stating that my Python processor does not define relationships. > Thanks,Ashwini >
Re: [DISCUSS] Release Timeline for NiFi 2.0.0-M3
Thanks David, I'm happy to take care of a 1.26 release at the same time. Regarding the M3 release, should the new UI be included and instructions given on how to access it to start collecting feedback? I'm under the impression that almost all of the work has been done, no? Thanks, Pierre Le mar. 23 avr. 2024, 02:03, David Handermann a écrit : > Team, > > We are approaching 300 Jira issues resolved for version 2.0.0-M3 [1], > including a number of important dependency upgrades and Python > framework improvements. > > There are several open pull requests that are getting close to > completion, which should be possible to include in an upcoming release > version. There is also a significant pull request [2] for NIFI-12998 > [3] that includes some important changes to module directory > hierarchy, providing a clearer distinction between framework and > extensions, and implementing significant cleanup of dependency > declarations across the repository. > > With these changes in view, we should be ready for another milestone > release soon after merging the above pull request. > > Another milestone release seems to be the best course of action for > now, providing the opportunity for additional review and testing > before a full 2.0.0 release version. > > I would be glad to handle Release Manager responsibilities for > 2.0.0-M3 when things are ready. > > Regards, > David Handermann > > [1] > https://issues.apache.org/jira/issues/?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%202.0.0-M3 > [2] https://github.com/apache/nifi/pull/8677 > [3] https://issues.apache.org/jira/browse/NIFI-12998 >
[jira] [Commented] (NIFI-13079) Is there a roadmap for bundle persistence provider?
[ https://issues.apache.org/jira/browse/NIFI-13079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839763#comment-17839763 ] Pierre Villard commented on NIFI-13079: --- I think it's better to discuss this in the Slack channels or via the email thread. The short answer is that it is absolutely used. I'm actually in the process of adding a lot of things so that it's better represented in the NiFi Registry. There are currently two persistence providers (local file system and S3), I filed a PR a long time ago to support GCS but never got merged, I may refresh that PR since I'm spending time on those topics lately. > Is there a roadmap for bundle persistence provider? > --- > > Key: NIFI-13079 > URL: https://issues.apache.org/jira/browse/NIFI-13079 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Igor Milavec >Priority: Minor > > Hi! > Based on the admin and user guides, it feels to me that the current status of > bundle persistence providers is in limbo? > The functionality is there, but it is not clear how to use it. I have found > documentation about how to configure NiFi Registry and how to upload a bundle > (using curl). I'm left wondering how this affects deployment, what are the > triggers for deployment, does it support SemVer, ... > Also, I could find no future plans for this feature. Is this being used at > all and are there plans to develop/support it in the future? > > Thank you! Igor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13077) On-demand Extension Provider
Pierre Villard created NIFI-13077:
-
Summary: On-demand Extension Provider
Key: NIFI-13077
URL: https://issues.apache.org/jira/browse/NIFI-13077
Project: Apache NiFi
Issue Type: Epic
Components: Core Framework
Reporter: Pierre Villard
We currently have the concept of ExternalResourceProvider with two
implementations (HDFS and NiFi Registry) that can be configured to list and
download all NARs made available in those locations. Those implementations, if
configured, would get started when NiFi starts and would download ALL of the
available NARs, plus a background thread would check every five minutes for new
NARs to be available and downloaded.
The proposal here is to have a similar concept that would focus on extensions /
components but instead of having a background thread and instead of having all
of the components downloaded, the approach would be to plug this into the
ExtensionBuilder and when a component cannot be instantiated (when loading a
flow definition) with locally available components, then, instead of creating a
ghost component, the Extension Providers would be queried with specific
coordinates and if the provider makes the component available, then the NAR
would be downloaded (alongside required dependencies if the NAR depends on
another NAR).
This approach already exists in the Kafka Connect NiFi plugin with the class
ExtensionClientDefinition. By adopting this approach in NiFi, it’d be much
easier to ship a much smaller version of NiFi and have NiFi download the
required components based on flows that are being instantiated / deployed.
The operation of downloading the NAR would not be blocking, meaning that we
would still create a ghost component but after completion of the NAR(s)
download and the loading of the components, the flows would be fully
operational.
It might be possible to show something similar as for the Python extensions
where we show that the component is still in the process of downloading third
party dependencies.
While this is a great opportunity to reduce the size of the NiFi binary (and
associated container image), it would not be great from a user perspective when
designing flows because all of the NARs removed from the default image would no
longer be visible in the list of available components when adding, for example,
a processor to the canvas.
Longer term we could imagine that the extension providers can also implement a
listing API so that when showing the list of available components, we would
show the list of the components available locally as well as the components
available through the extensions providers. The listing of components could add
another column to indicate the source of the component.
This is something that is exposed for the Extension Bundles in the NiFi
Registry (we also have the information about the NiFi API version that has been
used for building the components so we could use this information to only list
components that should be compatible from an API standpoint - same major
version but lower or equal API version).
The immediate goal though would be to introduce the concept of
ExtensionProvider with the following APIs:
{code:java}
boolean isAvailableExtension(Coordinates)
void downloadExtension(Coordinates)
{code}
Longer term we could also consider something like:
{code:java}
List listExtensions(){code}
But we would need to figure out how a NAR can provide the information about the
components that are inside of it. The NiFi Registry provides this information,
but that would not be the case for a Maven based implementation for example.
In nifi.properties we would have something looking like:
{code:java}
nifi.nar.extension.provider..{code}
And we would loop through all the configured providers to find the appropriate
NAR to download based on provided coordinates in the flow definition that is
being instantiated (either from flow.json.gz, or an uploaded JSON flow
definition, or when checking out a flow from a registry client).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13077) On-demand Extension Provider
[
https://issues.apache.org/jira/browse/NIFI-13077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13077:
--
Description:
We currently have the concept of ExternalResourceProvider with two
implementations (HDFS and NiFi Registry) that can be configured to list and
download all NARs made available in those locations. Those implementations, if
configured, would get started when NiFi starts and would download ALL of the
available NARs, plus a background thread would check every five minutes for new
NARs to be available and downloaded.
The proposal here is to have a similar concept that would focus on extensions /
components but instead of having a background thread and instead of having all
of the components downloaded, the approach would be to plug this into the
ExtensionBuilder and when a component cannot be instantiated (when loading a
flow definition) with locally available components, then, instead of creating a
ghost component, the Extension Providers would be queried with specific
coordinates and if the provider makes the component available, then the NAR
would be downloaded (alongside required dependencies if the NAR depends on
another NAR).
This approach already exists in the Kafka Connect NiFi plugin with the class
ExtensionClientDefinition. By adopting this approach in NiFi, it’d be much
easier to ship a much smaller version of NiFi and have NiFi download the
required components based on flows that are being instantiated / deployed.
The operation of downloading the NAR would not be blocking, meaning that we
would still create a ghost component but after completion of the NAR(s)
download and the loading of the components, the flows would be fully
operational.
It might be possible to show something similar as for the Python extensions
where we show that the component is still in the process of downloading third
party dependencies.
While this is a great opportunity to reduce the size of the NiFi binary (and
associated container image), it would not be great from a user perspective when
designing flows because all of the NARs removed from the default image would no
longer be visible in the list of available components when adding, for example,
a processor to the canvas.
Longer term we could imagine that the extension providers can also implement a
listing API so that when showing the list of available components, we would
show the list of the components available locally as well as the components
available through the extensions providers. The listing of components could add
another column to indicate the source of the component.
This is something that is exposed for the Extension Bundles in the NiFi
Registry (we also have the information about the NiFi API version that has been
used for building the components so we could use this information to only list
components that should be compatible from an API standpoint - same major
version but lower or equal API version).
The immediate goal though would be to introduce the concept of
ExtensionProvider with the following APIs:
{code:java}
boolean isAvailableExtension(Coordinates)
void downloadExtension(Coordinates)
{code}
Longer term we could also consider something like:
{code:java}
List listExtensions(){code}
But we would need to figure out how a NAR can provide the information about the
components that are inside of it. The NiFi Registry provides this information,
but that would not be the case for a Maven based implementation for example.
In nifi.properties we would have something looking like:
{code:java}
nifi.nar.extension.provider..{code}
And we would loop through all the configured providers to find the appropriate
NAR to download based on provided coordinates in the flow definition that is
being instantiated (either from flow.json.gz, or an uploaded JSON flow
definition, or when checking out a flow from a registry client).
was:
We currently have the concept of ExternalResourceProvider with two
implementations (HDFS and NiFi Registry) that can be configured to list and
download all NARs made available in those locations. Those implementations, if
configured, would get started when NiFi starts and would download ALL of the
available NARs, plus a background thread would check every five minutes for new
NARs to be available and downloaded.
The proposal here is to have a similar concept that would focus on extensions /
components but instead of having a background thread and instead of having all
of the components downloaded, the approach would be to plug this into the
ExtensionBuilder and when a component cannot be instantiated (when loading a
flow definition) with locally available components, then, instead of creating a
ghost component, the Extension Providers would be queried with specific
coordinates and if the provider makes the component available, then the NAR
would be downloaded
[jira] [Updated] (NIFI-12858) Processor change history on property hover
[ https://issues.apache.org/jira/browse/NIFI-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12858: -- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Processor change history on property hover > -- > > Key: NIFI-12858 > URL: https://issues.apache.org/jira/browse/NIFI-12858 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 2.0.0-M1, 1.24.0, 1.25.0, 2.0.0-M2 >Reporter: Chris Conklin >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Attachments: nifi1.23.2.png, nifi1.24.png > > Time Spent: 20m > Remaining Estimate: 0h > > The history details list on processors changed order after release 1.23.2. > Previous sort on history upon hovering over some property was latest update > first. > Starting with release 1.24 this order changed to oldest change first. > Since only 5 history lines appear this makes the feature not very useful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13070) Upgrade Netty to 4.1.109
[ https://issues.apache.org/jira/browse/NIFI-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13070: -- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Netty to 4.1.109 > > > Key: NIFI-13070 > URL: https://issues.apache.org/jira/browse/NIFI-13070 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty dependencies should be upgraded to > [4.1.109|https://netty.io/news/2024/04/15/4-1-109-Final.html] to incorporate > several bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13067) Upgrade Spring Security to 6.2.4
[ https://issues.apache.org/jira/browse/NIFI-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13067: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Spring Security to 6.2.4 > > > Key: NIFI-13067 > URL: https://issues.apache.org/jira/browse/NIFI-13067 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 10m > Remaining Estimate: 0h > > Spring Security dependencies should be upgraded to > [6.2.4|https://github.com/spring-projects/spring-security/releases/tag/6.2.4] > to incorporate several minor bug fixes and transitive dependency upgrades. As > part of the upgrade, OpenSAML dependencies should be upgraded to 4.3.1 and > AspectJ should be upgraded to 1.9.22. > Spring Boot for Registry can also be incremented to > [3.2.5|https://github.com/spring-projects/spring-boot/releases/tag/v3.2.5] to > align with Spring Security 6.2.4 and Spring Framework 6.1.6. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13066) Upgrade Bouncy Castle to 1.78.1
[ https://issues.apache.org/jira/browse/NIFI-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13066: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Bouncy Castle to 1.78.1 > --- > > Key: NIFI-13066 > URL: https://issues.apache.org/jira/browse/NIFI-13066 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Bouncy Castle libraries should be upgraded to version > [1.78.1|https://www.bouncycastle.org/releasenotes.html#r1rv78] to incorporate > several bug fixes, which include resolving several vulnerabilities that apply > to usage patterns outside of NiFi integration points. > Bouncy Castle 1.78.1 corrects a dependency relationship issue with bcpg and > bcutil that caused issues with OpenPGP usage in version 1.78. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Request for nifi custom processor development guidance in python
Hi, If you're using NiFi 2.0, then you can use Python. I recommend the below resources: https://www.youtube.com/watch?v=9Oi_6nFmbPg https://nifi.apache.org/documentation/nifi-2.0.0-M2/html/python-developer-guide.html HTH, Pierre Le mer. 17 avr. 2024 à 15:18, Usha Kiran Mahato a écrit : > Hi Team, > > I am Usha, a python developer. Recently I got a project to work on nifi. To > implement Client specific requirement, I have to build a custom processor > in nifi. Currently I use java to build that. > > As you know python is reach in library, I know jython is there but it > supports till python 2.7. So there are some limitations. > > Is there any way, so that I can build my custom processor in python instead > of java with the LTS python version. > > It will be very helpful to me. > > Thanking you > > Regards > Usha Kiran Mahato > email-ushakiranmaha...@gmail.com > Ph No.- +91 858 285 2477/ > +91 890 637 6213 >
[jira] [Updated] (NIFI-13052) CRON Driven components should be searchable
[ https://issues.apache.org/jira/browse/NIFI-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13052: -- Fix Version/s: 1.26.0 > CRON Driven components should be searchable > --- > > Key: NIFI-13052 > URL: https://issues.apache.org/jira/browse/NIFI-13052 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > By entering the keyword "timer" in the Search Bar, you can locate all > components with "Scheduling Strategy: Timer driven". > We should also be able to locate all components that have "Scheduling > Strategy: CRON driven". > This feature is documented in the User Guide under the "Search Components in > DataFlow" section. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release
I'm a +1 with this approach. Being able to use the Python extensions in MiNiFi cpp is great! Thanks, Pierre Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi a écrit : > Hi community, > > I'd like to initiate a discussion about the next release of MiNiFi C++. The > last release was more than seven months ago, and there have been many new > features, bug fixes and stability improvements committed to the development > branch since then: 107 tickets closed, over 122 commits as of today. > > I would be happy to take on RM duties for this release. > > Notable features and improvements since the 0.15.0 release: > > New notable features: > - Added support for using NiFi 2.0 Python processors in MiNiFi C++ > - This also includes several improvements to the previous MiNiFi style > python processors, like additional property options, custom relationships > and virtualenv support > - Added new python based multiplatform bootstrap script > - Added encryption support for sensitive properties in flow configuration > - Releasing Windows installer now can be done (and will be done) under the > Apache license > - Added support for service installation on MacOS > - Added C2 debug command to MiNiFi Controller > - Added support for setting MiNiFi properties from command line > - Added system load average field to C2 and Prometheus metrics > - Added support for manually configuring RocksDB options > - Added custom delimiter property for ListenTCP processor > - Added bandwidth limit properties to InvokeHTTP processor > - Added JSON flow configuration examples > > New processors: > - Added PutSmb, FetchSmb and ListSmb processor for SMB networking protocol > support > - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for pushing > logs to Grafana Loki > - Added JoltTransform to use Jolt JSON transformations > - Added SplitText processor > - Added AttributeRollingWindow processor > > Changes and improvements: > - Dropped support for disabling peer verification in InvokeHTTP > - Corrupt flow files are now filtered to avoid errors in the flow > - Using administrative yield duration instead of onschedule retry interval > in scheduling adjusting to NiFi's functionality > - Fixed high disk IO usage issue with MergeContent > - Fixed the site-to-site transfer or large files > - Fixed memory leak caused by unused loggers > - Fixed yielding processors to still respect scheduling period > > Upgraded dependencies: > - Upgraded OpenSSL to version 3.3.0 > - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions > - Upgraded libuvc to version 0.0.7 > - Upgraded docker base image to alpine:3.18 > - Upgraded Sol2 to version 3.3.0 > > With the current maturity level of the project and with the support for > NiFi 2.0 style python processors and json flow configuration, I suggest > releasing it as version 1.0.0-M1 milestone release. > > Do you agree it is time for a new release? Do you agree with the suggested > version? Are there any blockers that we should definitely include in this > release? > > Thanks, > Gabor >
[jira] [Updated] (NIFI-13052) CRON Driven components should be searchable
[ https://issues.apache.org/jira/browse/NIFI-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13052: -- Fix Version/s: 2.0.0-M3 Resolution: Fixed Status: Resolved (was: Patch Available) > CRON Driven components should be searchable > --- > > Key: NIFI-13052 > URL: https://issues.apache.org/jira/browse/NIFI-13052 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > By entering the keyword "timer" in the Search Bar, you can locate all > components with "Scheduling Strategy: Timer driven". > We should also be able to locate all components that have "Scheduling > Strategy: CRON driven". > This feature is documented in the User Guide under the "Search Components in > DataFlow" section. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13050) Add bundle dependencies info in NiFi Registry
[
https://issues.apache.org/jira/browse/NIFI-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13050:
--
Status: Patch Available (was: Open)
> Add bundle dependencies info in NiFi Registry
> -
>
> Key: NIFI-13050
> URL: https://issues.apache.org/jira/browse/NIFI-13050
> Project: Apache NiFi
> Issue Type: Improvement
> Components: NiFi Registry
> Reporter: Pierre Villard
> Assignee: Pierre Villard
>Priority: Major
>
> Follow-up improvement after NIFI-13048
> It'd be useful to display the dependencies for the bundles in NiFi Registry.
> The API endpoint is already exposed:
> {code:java}
> .../nifi-registry-api/bundles//versions/{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13050) Add bundle dependencies info in NiFi Registry
Pierre Villard created NIFI-13050:
-
Summary: Add bundle dependencies info in NiFi Registry
Key: NIFI-13050
URL: https://issues.apache.org/jira/browse/NIFI-13050
Project: Apache NiFi
Issue Type: Improvement
Components: NiFi Registry
Reporter: Pierre Villard
Assignee: Pierre Villard
Follow-up improvement after NIFI-13048
It'd be useful to display the dependencies for the bundles in NiFi Registry.
The API endpoint is already exposed:
{code:java}
.../nifi-registry-api/bundles//versions/{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13048) Add bundle build info in NiFi Registry
[ https://issues.apache.org/jira/browse/NIFI-13048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13048: -- Status: Patch Available (was: Open) > Add bundle build info in NiFi Registry > -- > > Key: NIFI-13048 > URL: https://issues.apache.org/jira/browse/NIFI-13048 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When dealing with bundles, we should display the build info as well as the > size of the bundle and the NiFi API version that has been used to build it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13048) Add bundle build info in NiFi Registry
[ https://issues.apache.org/jira/browse/NIFI-13048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13048: -- Description: When dealing with bundles, we should display the build info as well as the size of the bundle and the NiFi API version that has been used to build it. (was: When dealing with bundles, we should display the build info as well as the size of the bundle and the NiFi API version that has been used to build the it.) > Add bundle build info in NiFi Registry > -- > > Key: NIFI-13048 > URL: https://issues.apache.org/jira/browse/NIFI-13048 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > > When dealing with bundles, we should display the build info as well as the > size of the bundle and the NiFi API version that has been used to build it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13048) Add bundle build info in NiFi Registry
Pierre Villard created NIFI-13048: - Summary: Add bundle build info in NiFi Registry Key: NIFI-13048 URL: https://issues.apache.org/jira/browse/NIFI-13048 Project: Apache NiFi Issue Type: Improvement Components: NiFi Registry Reporter: Pierre Villard Assignee: Pierre Villard When dealing with bundles, we should display the build info as well as the size of the bundle and the NiFi API version that has been used to build the it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13046) Upgrade Solr dependencies to 8.11.3
[ https://issues.apache.org/jira/browse/NIFI-13046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-13046. --- Fix Version/s: 1.26.0 Resolution: Fixed > Upgrade Solr dependencies to 8.11.3 > --- > > Key: NIFI-13046 > URL: https://issues.apache.org/jira/browse/NIFI-13046 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade Solr dependencies to 8.11.3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13038) Upgrade Groovy to 4.0.21
[ https://issues.apache.org/jira/browse/NIFI-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13038: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Groovy to 4.0.21 > > > Key: NIFI-13038 > URL: https://issues.apache.org/jira/browse/NIFI-13038 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > Groovy dependencies should be upgraded to > [4.0.21|http://groovy-lang.org/changelogs/changelog-4.0.21.html] to > incorporate several transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13037) Upgrade Spring Framework to 5.3.34 on Support Branch
[ https://issues.apache.org/jira/browse/NIFI-13037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13037: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Spring Framework to 5.3.34 on Support Branch > > > Key: NIFI-13037 > URL: https://issues.apache.org/jira/browse/NIFI-13037 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, MiNiFi, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Spring Framework dependencies on the support branch should be upgraded to > version > [5.3.34|https://github.com/spring-projects/spring-framework/releases/tag/v5.3.34] > to resolve several flagged vulnerabilities. Spring Security should be > upgraded to 5.8.11 and Jetty should be upgraded to > [9.4.54|https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.54.v20240208] > to resolve CVE-2024-22201 related to HTTP/2 connection closing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13040) Upgrade Commons IO to 2.16.1
[ https://issues.apache.org/jira/browse/NIFI-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13040: -- Fix Version/s: 2.0.0-M3 1.26.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Commons IO to 2.16.1 > > > Key: NIFI-13040 > URL: https://issues.apache.org/jira/browse/NIFI-13040 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache Commons IO should be upgraded to > [2.16.1|https://commons.apache.org/proper/commons-io/changes-report.html#a2.16.1] > to incorporate a number of bug fixes and improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13036) Upgrade Logback to 1.5.5 and SLF4J to 2.0.13
[
https://issues.apache.org/jira/browse/NIFI-13036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-13036.
---
Resolution: Fixed
> Upgrade Logback to 1.5.5 and SLF4J to 2.0.13
>
>
> Key: NIFI-13036
> URL: https://issues.apache.org/jira/browse/NIFI-13036
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, MiNiFi, NiFi Registry
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Logback dependencies should be upgraded to
> [1.5.5|https://logback.qos.ch/news.html#1.5.5] and SLF4J should be upgraded
> to [2.0.13|https://www.slf4j.org/news.html#2.0.13] to incorporate several
> minor improvements.
> The Logback 1.5 release series requires Java 11, so it applies only to the
> main branch of Apache NiFi and cannot be backported to the support branch.
> Logback 1.5 decouples the core implementation from the {{logback-access}}
> module, which NiFi does not use, and is otherwise compatible with Logback 1.4.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13033) Upgrade Jetty to 12.0.8
[ https://issues.apache.org/jira/browse/NIFI-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13033: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Jetty to 12.0.8 > --- > > Key: NIFI-13033 > URL: https://issues.apache.org/jira/browse/NIFI-13033 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions, NiFi Registry >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > Jetty dependencies should be upgraded to > [12.0.8|https://github.com/jetty/jetty.project/releases/tag/jetty-12.0.8] to > incorporate various bug fixes and feature improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13032) Upgrade Spring Framework to 6.1.6
[
https://issues.apache.org/jira/browse/NIFI-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13032:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Upgrade Spring Framework to 6.1.6
> -
>
> Key: NIFI-13032
> URL: https://issues.apache.org/jira/browse/NIFI-13032
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, MiNiFi, NiFi Registry
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Spring Framework dependencies should be upgraded to
> [6.1.6|https://github.com/spring-projects/spring-framework/releases/tag/v6.1.6]
> to incorporate several bug fixes and improvements. Bug fixes include a
> resolution for CVE-2024-22262, which relates to parsing URLs using the Spring
> {{UriComponentsBuilder}} class.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13026) Add a read only mode to non secured NiFi Registry
[ https://issues.apache.org/jira/browse/NIFI-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13026: -- Resolution: Won't Do Status: Resolved (was: Patch Available) > Add a read only mode to non secured NiFi Registry > - > > Key: NIFI-13026 > URL: https://issues.apache.org/jira/browse/NIFI-13026 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Time Spent: 50m > Remaining Estimate: 0h > > There are scenarios where NiFi Registry may be deployed in a non secured way. > When doing so, as of now, all users access would be anonymous and all > permissions would be granted. This improvement is to allow for a read-only > mode when NiFi Registry is not secured so that all users can access NiFi > Registry but only perform READ actions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13027) Warn users for small files processing in PutIceberg
[ https://issues.apache.org/jira/browse/NIFI-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13027: -- Status: Patch Available (was: Open) > Warn users for small files processing in PutIceberg > --- > > Key: NIFI-13027 > URL: https://issues.apache.org/jira/browse/NIFI-13027 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > > While it can be a valid use case, it is a very bad idea to send a lot of > small flow files via the PutIceberg processor as it will generate a massive > amount of snapshot files. The recommendation is clearly to use a > MergeContent/MergeRecord processor before the PutIceberg processor to make > sure we limit the amount of individual files being sent to an Iceberg table. > While we can't force a user (this could be a flow analysis rule though) we > should let them know very clearly that what they're doing is likely a bad > idea and let them know what is the recommended way. However if the user is > sure they know what they're doing, they should be able to disable the warning. > This Jira is about adding: > * a property "Warn for small flow files" set to true by default > * a property "Minimum recommended file size" set to 10MB (depending on the > previous property, if set to true) > And if the warning is enabled and a processed flow file is below the limit, > then log a warning with the recommendation of using a Merge processor so that > a bulletin is generated and shown to the user. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13027) Warn users for small files processing in PutIceberg
Pierre Villard created NIFI-13027: - Summary: Warn users for small files processing in PutIceberg Key: NIFI-13027 URL: https://issues.apache.org/jira/browse/NIFI-13027 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Assignee: Pierre Villard While it can be a valid use case, it is a very bad idea to send a lot of small flow files via the PutIceberg processor as it will generate a massive amount of snapshot files. The recommendation is clearly to use a MergeContent/MergeRecord processor before the PutIceberg processor to make sure we limit the amount of individual files being sent to an Iceberg table. While we can't force a user (this could be a flow analysis rule though) we should let them know very clearly that what they're doing is likely a bad idea and let them know what is the recommended way. However if the user is sure they know what they're doing, they should be able to disable the warning. This Jira is about adding: * a property "Warn for small flow files" set to true by default * a property "Minimum recommended file size" set to 10MB (depending on the previous property, if set to true) And if the warning is enabled and a processed flow file is below the limit, then log a warning with the recommendation of using a Merge processor so that a bulletin is generated and shown to the user. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13026) Add a read only mode to non secured NiFi Registry
[ https://issues.apache.org/jira/browse/NIFI-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13026: -- Status: Patch Available (was: Open) > Add a read only mode to non secured NiFi Registry > - > > Key: NIFI-13026 > URL: https://issues.apache.org/jira/browse/NIFI-13026 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry > Reporter: Pierre Villard > Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > > There are scenarios where NiFi Registry may be deployed in a non secured way. > When doing so, as of now, all users access would be anonymous and all > permissions would be granted. This improvement is to allow for a read-only > mode when NiFi Registry is not secured so that all users can access NiFi > Registry but only perform READ actions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13026) Add a read only mode to non secured NiFi Registry
Pierre Villard created NIFI-13026: - Summary: Add a read only mode to non secured NiFi Registry Key: NIFI-13026 URL: https://issues.apache.org/jira/browse/NIFI-13026 Project: Apache NiFi Issue Type: Improvement Components: NiFi Registry Reporter: Pierre Villard Assignee: Pierre Villard There are scenarios where NiFi Registry may be deployed in a non secured way. When doing so, as of now, all users access would be anonymous and all permissions would be granted. This improvement is to allow for a read-only mode when NiFi Registry is not secured so that all users can access NiFi Registry but only perform READ actions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
