Re: Preferred schema registry

2019-01-16 Thread Andrew Grande
Isn't AvroSchemaRegistry an embedded one to NiFi? I.e. it's not a comparable alternative to an external dedicated schema registry. Andrew On Wed, Jan 16, 2019 at 7:43 AM Mike Thomsen wrote: > I think I figured out what it was. We backed out a change and it was > fungible with an earlier version

Re: Question about ephemeral NiFi (inside a Docker container) with ZooKeeper

2019-01-16 Thread Erik Anderson
Ok, I guess the better question to as is If NiFi is running in an ephemeral environment or mode such as kubernetes and docker with NO external volumes how can NiFi be backed up+recovered in event of a tragedy? Why not a simple S3 endpoint integration? Erik Anderson Bloomberg On Wed, Jan 16,

Re: Preferred schema registry

2019-01-16 Thread Mike Thomsen
I think I figured out what it was. We backed out a change and it was fungible with an earlier version of the schema. Therefore it appears that it didn't create a new version of the schema. When I made a more meaningful change, it created a new 4th version. Thanks, Mike On Wed, Jan 16, 2019 at 10

Re: Preferred schema registry

2019-01-16 Thread Bryan Bende
Mike, Do you have anymore details about why the Hortonworks schema registry stopped working? I have used it before and didn’t have any issues. Thanks Bryan On Tue, Jan 15, 2019 at 8:38 PM dan young wrote: > We used the AvroSchemaRegistry > > Dano > > On Tue, Jan 15, 2019, 12:51 PM Mike Thoms

Re: A question about [MergeContent] processor

2019-01-16 Thread Bryan Bende
Hello, The last release (1.8.0) introduced a new feature called load balanced connections which can be used to converge data from all nodes down to one node. You would make the connection right before merge content load balance to a single node. Thanks, Bryan On Tue, Jan 15, 2019 at 11:17 PM Ji

Re: Question about ephemeral NiFi (inside a Docker container) with ZooKeeper

2019-01-16 Thread Bryan Bende
Hello, ZooKeeper is not used at all in stand alone mode. In a cluster it used for leader election, as well as to store state for processors that maintain state, such as source processors that keep track of a time stamp or id. -Bryan On Wed, Jan 16, 2019 at 7:49 AM Erik Anderson wrote: > Our N

RE: Obtaining names of processor properties programmatically

2019-01-16 Thread Vitaly Krivoy
That's what I figured. To make up for it, I implemented the code which lists all processors in the release, then instantiates them one by one into a temporary process group (which is guaranteed to be unique by being named with a UUID), obtains the processor properties and then deletes the proces

Question about ephemeral NiFi (inside a Docker container) with ZooKeeper

2019-01-16 Thread Erik Anderson
Our NiFi is running in docker container. I am using some cool ideas from https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/start.sh#L39 Simply set environment variables, stop and restart the container and the systems are completely reconfigured. At the moment, I am using an e

QueryElasticsearchHttp processor over SSL

2019-01-16 Thread Dnyaneshwar Pawar
Hi All, We are trying to setup connection with ElasticSearch, HTTP connections are working fine, however, the HTTPS connections fails with hostname verification failure. We know this is the certificate issue, where server is using self signed certificate's with common hostname which does not