Re: Classloader issues

Condenser-processors-0.1.jar >> nifi-nar-utils-1.3.0.jar >> nifi-properties-1.3.0.jar >> nifi-runtime-1.3.0.jar >> slf4j-api-1.7.25.jar >> >> >>> On 7 Nov 2017, at 5:13 am, Bryan Bende wrote: >>> >>> Thanks for the poms. >>> >>

Re: Classloader issues

en.apache.org/POM/4.0.0 > http://maven.apache.org/xsd/maven-4.0.0.xsd";> > 4.0.0 > > > com.jidmu > JSONCondenser > 0.1 > > > nifi-JSONCondenser-nar > 0.1 > nar > > true > true > > > &

Re: Classloader issues

It is most likely an issue with the Maven configuration in one of your modules. Can you share your project, or the pom files for the processors, NAR, and bundle? Thanks, Bryan On Mon, Nov 6, 2017 at 12:20 PM, Phil H wrote: > Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-o

Re: FetchSFTP vs GetSFTP

could potentially have a different sftp host address in the queues. > > All together we have to pull from about 60 servers. If this doesn't work > out with the list/fetch I plan to have a micro acquisition cluster just > for gets. > > Ryan > > On Oct 31, 2017 4:26 PM,

Re: FetchSFTP vs GetSFTP

ch. > > Thanks again! > > > > On Tue, Oct 31, 2017 at 2:56 PM, Bryan Bende wrote: > >> Ryan, >> >> Personally I don't have experience running these processors at scale, >> but from a code perspective they are fundamentally different... >> &g

Re: FetchSFTP vs GetSFTP

Ryan, Personally I don't have experience running these processors at scale, but from a code perspective they are fundamentally different... GetSFTP is a source processor, meaning is not being fed by an upstream connection, so when it executes it can create a connection and retrieve up to max-sele

Re: Two issues relating to a processor I'm developing

Mike, Regarding the licensing, I believe LGPL is a no-go for Apache projects. Take a look here: https://www.apache.org/legal/resolved.html#category-x -Bryan On Sat, Oct 28, 2017 at 4:47 PM, Mike Thomsen wrote: > The processor breaks down a much larger file into a huge number of small > data p

Re: Multiple users or group as initial admin identity

Hi Fredrik, These are some good ideas. If we did support multiple initial admins, I would suggest it be done through multiple elements, rather than a comma separate list since commas are part of a DN which could be a single user. We already support this patter in the new user group provider:

Re: Syslog processing from cisco switches to Splunk

If you can provide an example message we can try to see why ListenSyslog says it is invalid. I'm not sure that will solve the issue, but would give you something else to try. On Thu, Oct 19, 2017 at 8:38 AM, Andrew Psaltis wrote: > Dave, > To clarify you are using the PutUDP processor, not the P

Re: Metric access for reporting tasks

vice/processor? > > > On Wed, Oct 11, 2017 at 8:48 PM, Bryan Bende wrote: > >> I just added you to the contributors list in JIRA so you should be >> able to assign things to yourself. >> >> I think initially putting all the metric services into the same NAR &

Re: Metric access for reporting tasks

that anyone who wants to implement their own service can do so without > including modules they don't need. > > By the way, if it's OK of course, could you please add me to the jira so > that the issue can be assigned to me once opened? Thank you! > > On Wed, 11 Oct 2017 at 1

Re: Metric access for reporting tasks

Omer, I think adding the new versions that implement the new MetricReporterService, and marking the old ones as deprecated makes sense. They could potentially be removed on a major future release like 2.0.0. Were you envisioning the DataDogMetricReportService and AmbariMetricReportingService livi

Re: Funnel Queue Slowness

Peter, The images didn’t come across for me, but since you mentioned that a failure queue is involved, is it possible all the flow files going to failure are being penalized which would cause them to not be processed immediately? -Bryan > On Oct 8, 2017, at 10:49 PM, Peter Wicks (pwicks) wrot

Re: ListenTcpRecord

; the length of the message. > > -Clay > > On Thu, Oct 5, 2017 at 8:22 AM, Bryan Bende wrote: > >> Have you tried using the template? >> >> https://gist.githubusercontent.com/bbende/fa2bff34e721fef21453986336664c >> b2/raw/db658c64f75fec47785ab63920ee23582bf1492f/m

Re: route flow based on variable

Ben, 1) Yes, the variables are hierarchical, so a variable at the root group would be visible to all components, unless there is a variable with the same name at a lower level which would override it. 2) I haven’t tried this, but I would expect that you should still be able to use RouteOnAttri

Re: RecordReader and RecordWriter: development work on ruleengine

Uwe, I don't think there is specific documentation on how to write code using the record readers and writers, but the best example to look at would be ConvertRecord ConvertRecord actually extends from AbstractRecordProcessor: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-s

Re: ListenTcpRecord

> the processor). I'd appreciate if you let me what I am doing wrong. > > thanks > Clay > > On Wed, Oct 4, 2017 at 9:14 AM, Bryan Bende wrote: > >> Hello, >> >> I wrote a post that shows an example of using ListenTCPRecord with a >> GrokReader to recei

Re: ListenTcpRecord

Hello, I wrote a post that shows an example of using ListenTCPRecord with a GrokReader to receive multi-line log messages. There is a link to a template of the flow at the very end. You could easily change the example so that PutTCP is sending a single JSON document, or an array of JSON documents

Re: [VOTE] Release Apache NiFi 1.4.0 (RC2)

+1 (binding) - Ran through the release helper and everything checked out. - Ran a couple of sample flows with no issues On Fri, Sep 29, 2017 at 9:46 AM, James Wing wrote: > Jeff, I agree the updated KEYS file has been published. Thanks. > > On Fri, Sep 29, 2017 at 6:00 AM, Jeff wrote: > >> Ja

Re: [VOTE] Release Apache NiFi 1.4.0

I think the reason for the upgrade issue was the following... Normally there is an automatic upgrade of component versions, with the following logic: - If the flow says you are using version X of a component, and during startup version X is not found, but version Y is found, and version Y is the

Re: Prometheus Reporting Task

Matt, That sounds great, looking forward to your contribution. I added you to the contributors list in JIRA so you should be able to assign the JIRA to yourself. Whenever your work is done you can submit a PR or attach a patch to the JIRA, and in either case mark the JIRA as "Patch Submitted".

Re: Reg. Nifi Clustering Load Balancing

Hello, You can run a standard HTTP load-balancer in front of ListenHTTP and have your producers use the URL of the load-balancer. Nginx or apache httpd can be used. Thanks, Bryan On Tue, Aug 29, 2017 at 11:40 AM, mayank rathi wrote: > Does this help? > > [image: Inline image 1] > > On Tue,

Re: determine if instance is a Cluster Coordinator or Primary Node

Mark, I don't believe there is currently anything like this in Authorizer API. You would likely have to build something similar to what processors have... In ProcessorInitializationContext they get access to a NodeType which tells them if they are currently primary or not. Then they can annotat

Re: Did something change with PropertyDescriptor validation?

Mike, I believe this is how it always worked and is just a mis-leading message from the mock framework. You have to have at least one validator and you can use Validator.VALID if you just want it to always be valid. -Bryan On Fri, Aug 25, 2017 at 1:33 PM, Mike Thomsen wrote: > I have this des

Re: Restricting Controller Service Class Definitions

Hi Michael, Generally processors are supposed to only know about an interface for the service, and then the framework makes the implementations of that interface available. So the processors shouldn't really know which specific implementations are available. I think there are a couple of things t

Re: how to execute code when processor is stopping

Ben, I apologize if I am not understanding the situation, but... In the case where your OnScheduled code is in a retry loop, if someone stops the processor it will call your OnUnscheduled code which will set the flag to bounce out of the loop. This sounds like what you want, right? In the case w

Re: get controller service's configuration

The way controller services are setup you have the following... - DBCPService interface (provides getConnection()) extends ControllerService interface (empty interface to indicate it is a CS) - DBCPConnectionPool extends AbstractControllerService implements DBCPService - Processor XYZ depends on D

Re: Removing unneeded nar files from NiFi standard distribution.

Toivo, Besides the Jetty NAR and the framework NAR, you should be able to remove most of the other NARs without any negative impact. The hope is that eventually most of these NARs can live in an extension repository and then people can pick and choose which processors to add to their distribution

Re: Avro Reader reading from FlowFile throws SchemaNotFoundException

Hello, I think I see what the problem is now... The exception in your second email is coming from the CSV writer which is set to get the schema from the "Schema Text" property, which is in turn set to ${avro.schema}. I believe what you showed in the section ### FLOWFILE ### is the content of the

Re: Avro Reader reading from FlowFile throws SchemaNotFoundException

Hello, I'm assuming this error came from #2 when you tried to use Schema Text set to ${avro.schema} ? The error means your flow file doesn't have an attribute called avro.schema, which it would need to have if you reference ${avro.schema}. What were the results using Embedded Avro Schema? That s

Re: How to Reference/Deploy RecordReader?

Hi Cam, Does the pom.xml of your NAR have a NAR dependency on standard-services-api-nar? org.apache.nifi nifi-standard-services-api-nar nar Using parquet as an example, your processors pom should have provided dependencies on the record modules you need to compile your code: https

Re: ERR_BAD_SSL_CLIENT_AUTH_CERT error after configuring secure cluster

Hello, I think you should only make one call to the toolkit which should generate a CA, the server certs, and the client cert all at the same time. The -C flag is for the client cert which you already had on the first call so I think it generated it already. By running it twice like above, the fi

Re: Sending Parquet files to S3

Hello, The PutParquet processor uses the Hadoop client to write to a filesystem. For example, to write to HDFS you would have a core-site.xml with a filesystem like: fs.defaultFS hdfs://yourhost And to write to a local filesystem you could have a core-site.xml with:

Re: RTC clarification

I agree with encouraging reviews from everyone, but I lean towards "binding" reviews coming from committers. If we allow any review to be binding, there could be completely different levels of review that occur... There could be someone who isn't a committer yet, but has been contributing already

Re: Custom NAR interfering with BundleUtils.findBundleForType

Scott, Thanks for providing the stacktrace... do any of your custom processors use the @DefaultSchedule annotation? and if so, do any of them set the tasks to a number less than 1? The exception you are getting is from some code that is preventing using 0 or negative number of tasks for a process

Re: Communication between flows

Steve, In 1.2.0 there were some new processors added called Wait/Notify... With those you could send your original JSON (before splitting) to a Wait processor and tell it to wait until the signal count is equal to the number of splits, then you could put a Notify processor right after PutMongo co

Re: HBase security label support

Mike, I don't know of any work being done or any JIRAs that exist for this, but seems like it would be good to support them. Most likely its just that no one has asked for it yet. I'd go ahead and create a JIRA, or if you were planning to incorporate it into the HBase record processors then that

Re: conversion from AVRO file format to Parquet file format

Rohith, Can you share more details about how you have configured PutParquet? What Record Reader are you using and what Schema Access Strategy? If your data is already in Avro then you would need to set the Record Reader to an AvroRecordReader. The AvroRecordReader can be configured to use the sch

Re: How to ingest files into HDFS via Apache NiFi from non-hadoop environment

Yes, I think running NiFi on edge nodes would make sense, this way they can access the public network to receive data, but also access HDFS on the private network. On Fri, Jun 23, 2017 at 4:24 PM, Mothi86 wrote: > Hi Bryan, > > Greetings and appreciate your instant reply. Data nodes are in priva

Re: How to ingest files into HDFS via Apache NiFi from non-hadoop environment

Hello, Every node where NiFi is running must be able to connect to the data node process on every node where HDFS is running. I believe the default port for the HDFS data node process is usually 50010. I'm assuming your 4 worker nodes are running HDFS, so NiFi would have to access those. -Bryan

Re: New processor location

Hi Michael, Those two points are definitely good ones to help make the decision. For the second point, it doesn't necessarily have to be an enormous set of dependencies, it could be just one dependency that the new processors need that is in conflict with what the standard processors are already u

Re: AbstractRecordProcessor

Hi Chris, I think a good place for an abstract record processor would be in the standard record utils under nifi-nar-bundles/nifi-extension-utils: https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-standard-record-utils We've taken a similar a

Re: NiFi 1.3.0 Remote ports "disabled" after a while?

Joe, Thanks for the info. I think this might be similar to https://issues.apache.org/jira/browse/NIFI-3900, but the fix didn't account for RPGs. I created this JIRA - https://issues.apache.org/jira/browse/NIFI-4075 -Bryan On Thu, Jun 15, 2017 at 10:14 AM, Joe Gresock wrote: > Btw, I was wrong

Re: NIFI-4064 High CPU with Funnels when no outgoing connections

Hi Wes, I added you to the contributors group in JIRA. Let me know if you still can't assign the ticket to yourself. -Bryan On Tue, Jun 13, 2017 at 11:45 AM, Wes Lawrence wrote: > Hello, > > I created NIFI-4064 since > I ran into a minor issue

Re: [VOTE] Release Apache NiFi 0.7.4

+1 (binding) Release this package as nifi-0.7.4 - Ran through release helper and everything checked out - Ran basic flow to test that binary works Minor note - The years in the NOTICE files still say 2014-2016. On Tue, Jun 6, 2017 at 11:30 AM, Joe Skora wrote: > +1 (binding) > > * Verified GPG

Re: [VOTE] Release Apache NiFi 1.3.0

+1 (binding) Release this package as nifi-1.3.0 - Ran through release helper and everything checked out - Ran some test flows with HDFS processors and verified they worked correctly On Tue, Jun 6, 2017 at 12:42 AM, James Wing wrote: > +1 (binding). I ran through the release helper -- verified s

Re: Example Processors that....

Hi Otto, Do you mean something specific by "files loaded into the system" or are you just referring to a processor having a property that points to an external file? There are a few examples of the latter in the code base, you can look for usage of FILE_EXISTS_VALIDATOR as an easy way to find the

Re: Zero-Downtime Deployments with Component Versioning

Hi Richard, I believe this is currently expected behavior, meaning that rolling deployments are not supported when changes to NARs are being made. There are two main issues... The first is that if nodes are running different versions of the framework NAR, they could potentially be running incom

Re: nifi-jms-processor not closing connection properly in "primary node only" execution mode

Dominik, Sorry for the delayed response... I think the issue is that currently the processor is started/schedule on all nodes, regardless of whether it is primary or not, but then only the instance on the primary node is triggered to run. So when the primary node changes from node1 to node2, not

Re: Duplicated processors when using nifi processors dependency

become irrelevant once the NAR dependencies are aligned and > they can be references as Java libraries? > > Thanks, > > James > > On Sun, May 21, 2017 at 10:28 AM, Bryan Bende wrote: > >> Currently the AWS NAR contains the processors, controller service API, and

Re: Duplicated processors when using nifi processors dependency

Currently the AWS NAR contains the processors, controller service API, and controller service implementation, all in one NAR. Typically the controller service API should be in its own NAR, so there should be something like: nifi-aws-bunde nifi-aws-service-api nifi-aws-service-ap

Re: NiFi 1.2.0 Record processors question

When a reader produces a record it attaches the schema it used to the record, but we currently don't have a way for a writer to use that schema when writing a record, although I think we do want to support that... something like a "Use Schema in Record" option as a choice in the 'Schema Access Stra

Re: Duplicated processors when using nifi processors dependency

Hi Robert, It sounds like your NAR is including nifi-aws-processors, and so is the regular AWS NAR. Previously it wasn't obvious this was happening, and when NiFi started up it was finding those processors in one of the NARs and skipping the other one. With component versioning it is now more spec

Re: [VOTE] Release Apache NiFi MiNiFi 0.2.0

+1 (binding) Release this package as minifi-0.2.0 Verified signatures and hashes, ran full build with contrib-check, verified licenses/notices in all binaries, and ran sample flow from MiNiFi to NiFi. On Wed, May 17, 2017 at 11:38 AM, Pierre Villard wrote: > +1 non-binding > > Signature, checksu

Re: Controller service question

Hi Yuri, You could definitely create a controller service to obtain the metadata. The controller service API doesn't provide anything to run a task on a timer, its meant to be something that is accessed when needed by a processor or reporting task, but you could certainly start a background threa

Re: NiFi JUnit test framework: How to pass through or cause to ignore validation of dynamic property

27;ll comb through the > standard validators, but I don't think there's an option for no validation. > > But, yeah, I kind of forgot to finish the job when I added dynamic property > handling. Thanks from drawing my attention to it. > > Russ > > > On 05/11/2017

Re: NiFi JUnit test framework: How to pass through or cause to ignore validation of dynamic property

Russell, I ran into this once before and I believe it happens when you have a property descriptor with no validator. Check your getSupportedDynamicPropertyDescriptor() method and see if you specified a validator. I think you at least need something like StandardValidators.NON_EMPTY_VALIDATOR whi

Re: Partitioning from actual Data (FlowFile) in NiFi

If your data is JSON, then you could extract the date field from the JSON before you convert to Avro by using EvaluateJsonPath. >From there lets say you have an attribute called "time" with the unix timestamp, you could use an UpdateAttribute processor to create attributes for each part of the tim

Latest Documentation

All, Just wanted to mention that you may need to clear your browser's cache when accessing the latest documentation at https://nifi.apache.org/docs.html. The structure of the links to the components changed during this release to support component versioning. Thanks, Bryan

[ANNOUNCE] Apache NiFi 1.2.0 release

Hello The Apache NiFi team would like to announce the release of Apache NiFi 1.2.0. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports highly configurable directed graphs of data routing, transformation, an

[RESULT][VOTE] Release Apache NiFi 1.2.0 (RC2)

Apache NiFi Community, I am pleased to announce that the 1.2.0 release of Apache NiFi passes with 11 +1 (binding) votes 11 +1 (non-binding) votes 1 0 vote 0-1 votes Thanks to all who helped make this release possible. Here is the PMC vote thread: https://mail-archives.apache.org

Re: Pulling out hair over slf4j...

.apache.httpcomponents:httpmime:jar:4.1.2:compile > [INFO] +- com.perfectsearchcorp:psjbase:jar:195:compile > [INFO] | +- org.fusesource:sigar:jar:1.6.4:compile > [INFO] | | \- log4j:log4j:jar:1.2.15:compile > [INFO] | +- jaxen:jaxen:jar:1.1.4:compile > [INFO] | \- org.databene:conti

Re: Pulling out hair over slf4j...

Russell, Assuming you have the standard project structure with a pom and then sub-modules for processors and NAR, does your root pom have a parent of nifi-nar-bundles and if so what version? Example: org.apache.nifi nifi-nar-bundles 1.2.0-SNAPSHOT It would also be helpful to see the output of

Re: [VOTE] Release Apache NiFi MiNiFI C++ 0.2.0 (RC2)

+1 (binding) Release this package as nifi-minifi-cpp-0.2.0 - Verified signature and hashes - Built on OSX - Successfully ran binary using provided sample config for s2s On Mon, May 8, 2017 at 2:26 AM, Koji Kawamura wrote: > +1 (non-binding) > > Full build and test finished successfully without

Apache NiFi 1.2.0 RC2 Release Helper Guide

Hello Apache NiFi community, Please find the associated guidance to help those interested in validating/verifying the release so they can vote. # Download latest KEYS file: https://dist.apache.org/repos/dist/dev/nifi/KEYS # Import keys file: gpg --import KEYS # [optional] Clear out local maven

[VOTE] Release Apache NiFi 1.2.0 (RC2)

Hello, I am pleased to be calling this vote for the source release of Apache NiFi nifi-1.2.0 (RC2). The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1104 The Git tag is nifi-1.2.0-RC2 The Git commit ID is 3a605a

[CANCEL] [VOTE] Release Apache NiFi 1.2.0

gt; > +1 binding, release as nifi-1.2.0 >> > >> > - confirmed keys, hashes, and commit >> > - built with tests and contrib-check >> > - ran various flows using the record readers/writers, new and existing >> > processors, controller services, etc. &

Apache NiFi 1.2.0 RC1 Release Helper Guide

Hello Apache NiFi community, Please find the associated guidance to help those interested in validating/verifying the release so they can vote. # Download latest KEYS file: https://dist.apache.org/repos/dist/dev/nifi/KEYS # Import keys file: gpg --import KEYS # [optional] Clear out local maven

[VOTE] Release Apache NiFi 1.2.0

Hello, I am pleased to be calling this vote for the source release of Apache NiFi nifi-1.2.0. The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1103 The Git tag is nifi-1.2.0-RC1 The Git commit ID is 243ed742badd

Re: Closing in on a NiFi 1.2.0 release?

The issues from yesterday have been resolved so I'll start kicking off another attempt at the RC process. On Wed, May 3, 2017 at 6:10 PM, Bryan Bende wrote: > Quick update... I ran into two issues that will need to be addressed to > create the RC. > > I've created JIR

Re: Closing in on a NiFi 1.2.0 release?

Quick update... I ran into two issues that will need to be addressed to create the RC. I've created JIRAs for them and tagged them as 1.2: https://issues.apache.org/jira/browse/NIFI-3795 https://issues.apache.org/jira/browse/NIFI-3793 On Wed, May 3, 2017 at 2:41 PM, Bryan Bende

Re: Closing in on a NiFi 1.2.0 release?

ot the most up-to-date revision. This component appears to have been >> modified >> >> >> I don't have a 1.1.0 instance around me at the moment but I vaguely >> remember being able to do that in the past. >> >> Can someone confirm this is new and ex

Re: Closing in on a NiFi 1.2.0 release?

ze by removing bundled dependencies. >>>>> - User Experience Improvements >>>>> - Ever wish that it was easier to align components on the canvas? Me >>>>> too...and now you can! >>>>> - We now provide deep links to any component(s)

Re: Not able to ingest the csv data to RDBS Database

In order for ReplaceText to reference ${city} and ${count} there need to be flow file attributes available for these values, and if you picked up a file from GetFile and went directly to ReplaceText then they wouldn't be there. You would likely have to do something like GetFile -> SplitText with l

Re: failed to replicate request GET /NIFI-API/flow/process-groups/root to saxx:9090

Pradeep, This means the NiFi you are accessing the UI from is trying to replicate a request to another NiFi in the cluster running on sabc.abcd.local:9090 and couldn't complete the replication due to a read timeout. Can you verify there is a NiFi running at sabc.abcd.local:9090? Can you access th

Re: how to write date to redis

Hello, I can't say for sure why your script isn't performing well, but I would guess that you would get better performance from a custom processor written in Java that wrapped a Redis client like Jedis. If you are interested in working on this I think it would be something the community would be

Re: Closing in on a NiFi 1.2.0 release?

, NIFI-3726 (CompareFuzzyHash processor) appears to be very close as well and has already had some review and updates made. Andy plans to do a final review and hopefully merge this as well. Thanks, Bryan On Tue, May 2, 2017 at 10:17 AM, Bryan Bende wrote: > All, > > Looks like we are cl

Re: Closing in on a NiFi 1.2.0 release?

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.2.0 If anyone knows of anything that should be highlighted, feel free to mention it here and I can update the notes. Thanks! -Bryan On Thu, Apr 27, 2017 at 5:52 PM, Bryan Bende wrote: > Looks like we are down to just a few

nifi-extension-utils

All, As part of NIFI-3724 which was merged to master earlier today, I made some slight refactoring to the organization of some utility code and wanted to mention it in case anyone is wondering where things moved... Previously under nifi-commons we had nifi-processor-utils and nifi-hadoop-utils wh

Re: whats new in nifi 1.2.0 / careful use of marks and being clear about non released features

Anshuman, You shouldn't have to download anything to use the NiFI NAR Maven Plugin. If you are building the latest NiFi code on the master branch then it will already be using version 1.2.0: https://github.com/apache/nifi/blob/master/pom.xml#L1780 If you are building your own NARs then you just

Re: Closing in on a NiFi 1.2.0 release?

Looks like we are down to just a few tickets, and all of them seem to have traction in terms of review and discussion. I'll keep an eye on the tickets and start trying to pull together anything I can to get ready for the RC process. Seems like we are close! On Tue, Apr 25, 2017 at 9:33 AM, Russel

Re: load balancer on nifi cluster

No there isn't auto-load balancing... That statement means that NiFi is not taking the graph of processors and running some processors on one node and some on another node (i.e. like Storm and other stream processing systems). What you see in the UI is running on every node and you have to design

Re: load balancer on nifi cluster

Correct, depending how you are using ListenSyslog, you need a load balancer that supports TCP or UDP. NGINX is one of them, although I am not sure if the free version supports TCP/UDP: https://www.nginx.com/resources/admin-guide/tcp-load-balancing/#upstream HAProxy supports TCP: http://www.hapr

Re: load balancer on nifi cluster

Hi Pradeep, Can you clarify what you are trying to load balance? How is data entering your flow? I'm going to assume you are talking about load balancing data coming in to the system, and not end users accessing the web UI, but correct me if I am wrong. Load balancing the data depends a lot on h

Re: Policy and UI

Mark, It is definitely possible that there is a browser issue there, although we generally support the most recent two versions of the major browsers [1], with the latest version of Chrome being around 56 and and Firefox around 52. Thanks, Bryan [1] https://nifi.apache.org/docs/nifi-docs/html/

Re: NiFi Processors bundle pom.xml deploying artifacts to Artifactory

Just following up on what Joe said, take a look at this section on the wiki: https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-Inheritance Thanks, Bryan On Fri, Mar 31, 2017 at 9:08 AM, Joe Witt wrote: > Michal > > The archetype and exist

Re: Processor missing when attempting Add Processor

It looks like the processors JAR has the service loader file in META_INF instead of META-INF... inflated: META_INF/services/org.apache.nifi.processor.Processor On Tue, Mar 28, 2017 at 8:44 PM, Russell Bateman wrote: > No problem. I'll upload a tarball as soon as I can get it rolled up. > > > On

Re: Commit d90cf84 seems to break build?

oad it seems as though its benefit may be marginal as per > https://docs.travis-ci.com/user/caching/#How-does-caching-work%3F > On Sat, Mar 25, 2017 at 11:04 Bryan Bende wrote: > >> Andre, >> >> Thanks for bringing this up. >> >> The standard prioritizers mo

Re: Commit d90cf84 seems to break build?

Andre, Thanks for bringing this up. The standard prioritizers moved from the standard bundle to the framework bundle, and sure enough the parent was still set as standard bundle. We had never built with a clean repo and were getting lucky, so I am glad you found this. In nifi-nar-bundles/nifi-f

Re: Closing in on a NiFi 1.2.0 release?

he JIRA whipping. > > Thanks > JOe > > On Mon, Mar 13, 2017 at 3:22 PM, Bryan Bende wrote: > > Just a quick update on this discussion... > > > > On Friday we were able to post an initial PR for the component > > versioning work [1]. > > > > I beli

Re: Error message in "ConsumeKafka_0_10" processor

ut value - Where do we do >> that? >> Settings for my Kafka cluster is zookeeper.connection.timeout.ms=6000 >> >> >> Thank you! >> __ >> >> Kind Regards, >> Anshuman Ghosh >> Contact - +49 179 9090964 >> >>

Re: Error message in "ConsumeKafka_0_10" processor

Anshuman, The error message is saying that NiFi tried to use a consumer where the session already timed-out. So for example, if the session timeout was 10 seconds, and NiFi had a consumer that hadn't been used in 12 seconds and then tried to use it, you would get this error because the session for

Re: List/Empty queue on a Cluster

Mark, I believe whichever node receives the request from the UI, it will replicate to the other nodes, and the other nodes will see an incoming request with two entities (the end user, and the node replicating the request). Those two entities then need to be authorized to see if they are allowed t

Re: [ANNOUNCE] Release of Apache NiFi nifi-nar-maven-plugin 1.2.0

; Whilst it is clear that this announce is about a Maven plugin for > Apache NiFi, there are no details as to what NiFi itself is about. > > Please can you add that information to future announce mails? > > Thanks. > > > On 20 March 2017 at 14:20, Bryan Bende wrote: >&g

[ANNOUNCE] Release of Apache NiFi nifi-nar-maven-plugin 1.2.0

Hello The Apache NiFi team would like to announce the release of Apache NiFi nifi-nar-maven-plugin 1.2.0. The Apache NiFi NAR Maven Plugin helps to build NiFi Archive bundles (NARs) to support the class loader isolation model in NiFi. More details on Apache NiFi can be found here: https://nifi.a

[RESULT][VOTE] Release Apache NiFi nifi-nar-maven-plugin-1.2.0

Apache NiFi Community, I am pleased to announce that the nifi-nar-maven-plugin-1.2.0 release passes with: 4 +1 (binding) votes 3 +1 (non-binding) votes 0 0 votes 0 -1 votes Thanks to all who helped make this release possible. Here is the PMC vote thread: https://mail-archives.apache.org/mod_mb

Re: [VOTE] Release Apache NiFi nifi-nar-maven-plugin-1.2.0

didn't). > > > On Tue, Mar 14, 2017 at 9:21 AM, Bryan Bende wrote: > >> Hello, >> >> I am pleased to be calling this vote for the source release of Apache >> NiFi nifi-nar-maven-plugin-1.2.0. >> >> The source zip, including signatures, d

Re: Primary Node and Isolated Processors

Hi Mark, In 1.x the primary node is elected automatically by voting through ZooKeeper, this gives automatic failover should the primary node go down. In general, the primary node is typically used to schedule source processors in cases where it wouldn't make sense to retrieve the same data on all

Re: Kafkaesque Output port

Just wanted to throw out a couple of other ideas... I ran into a similar situation and ended up creating a web-service at the core (HandleHttpRequest/HandleHttpResponse) where the edge instances could poll for the latest instructions [1][2]. This works well when theres basically one new piece of i

Apache NiFi nifi-nar-maven-plugin-1.2.0 Release Helper Guide

Hello Apache NiFi community, Please find the associated guidance to help those interested in validating/verifying the release so they can vote. # Download latest KEYS file: https://dist.apache.org/repos/dist/dev/nifi/KEYS # Import keys file: gpg --import KEYS # [optional] Clear out local maven

<    1   2   3   4   5   6   7   8   9   >