SQL request

2017-08-09 Thread Arnaud G
Hi all, I just have 2 small questions regarding SQL in Nifi. 1) What is the best way to execute arbitrary SQL request? For instance, I have a lot of use cases where I'm doing a couple of file transformation/deduplication that require to initiate copy of one table to another another with complex

Re: Kafka - Hadoop Kerberos issue

2017-05-18 Thread Arnaud G
were using the same Kerberos domain everything will be working fine. A. On Thu, May 18, 2017 at 9:22 AM, Arnaud G <greatpat...@gmail.com> wrote: > Hi again, > > Here are more details: > > 1) We are redoing the test on a test cluster which is plain 1.2 vanilla > (upgr

Re: Kafka - Hadoop Kerberos issue

2017-05-18 Thread Arnaud G
> "Additional Classpath Resources" property, and that there were no >> additional JARs added to NiFi's lib directory? >> >> Just want to ensure nothing else is on the classpath of the PutHDFS >> processor besides what would be expected. >> >> Thanks, >> >&

Re: Kafka - Hadoop Kerberos issue

2017-05-17 Thread Arnaud G
s, > > Bryan > > > On Wed, May 17, 2017 at 4:49 AM, Arnaud G <greatpat...@gmail.com> wrote: > > Hi! > > > > We are currently facing an issue and we have not yet find a potential > > solution. > > > > We are running 1.2 and we have are facing an

Kafka - Hadoop Kerberos issue

2017-05-17 Thread Arnaud G
Hi! We are currently facing an issue and we have not yet find a potential solution. We are running 1.2 and we have are facing an interaction between the Kafka 10.2 and PutHDFS processors. We are connecting to an external Kafka server that require SASL/PLAIN authentication. To enable this we are

Re: Queue incoherent state

2017-05-16 Thread Arnaud G
the FlowFile Repository > could have run out of disk space? > > Thanks > -Mark > > On May 16, 2017, at 3:34 AM, Arnaud G <greatpat...@gmail.com> wrote: > > Hi Matt, > > Thanks for your reply! > > I finally solved the problem by deleting all the content in

Re: Queue incoherent state

2017-05-16 Thread Arnaud G
ued? > > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-3897 > > On Fri, May 12, 2017 at 5:47 AM, Arnaud G <greatpat...@gmail.com> wrote: > >> Hi again! >> >> I currently have another issue with incoherent queue status. >> >> Following the upgrade t

Queue incoherent state

2017-05-12 Thread Arnaud G
Hi again! I currently have another issue with incoherent queue status. Following the upgrade to 1.2 of a cluster, I have a couple of queues that display through the GUI a high number of flowfiles. As the queue were no emptying despite tuning, I tried to list the content of the queue. This

Re: VolatileProvenanceRepository problem with 1.2?

2017-05-12 Thread Arnaud G
we could probably trim the property before using it. > > Pierre. > > > 2017-05-12 10:27 GMT+02:00 Arnaud G <greatpat...@gmail.com>: > >> Hi all! >> >> >> I was trying to solve an issue regarding the slowing down of the >> provenance repository by s

VolatileProvenanceRepository problem with 1.2?

2017-05-12 Thread Arnaud G
Hi all! I was trying to solve an issue regarding the slowing down of the provenance repository by switching to the volatile storage as documented in the documention and in various mailing list topics. However when I changed the configuration to:

Re: GetHDFS and triggering

2017-04-06 Thread Arnaud G
components/org. > apache.nifi.processors.hadoop.ListHDFS/index.html > [4] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.hadoop.FetchHDFS/index.html > [5] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.hive.ConvertA

GetHDFS and triggering

2017-04-05 Thread Arnaud G
Hi, I'm currently building a flow in Nifi and I'm trying to get the best way to do it in a reliable manner: The setup is the following: 1) Some files are copied in a folder in HDFS 2) An Hive external table point to this directory 3) The data of this table are then copied in an ORC table 4) The

Re: apache nifi 1.1.2 startup really slow on openstack environment

2017-04-05 Thread Arnaud G
Hi! We had the same problem on Openstack (on clustered and non clustered instances) and the problem can easily be solved by changing the entropy source to /dev/urandom in the Java security parameters. Regards, On Wed, Apr 5, 2017 at 1:07 PM, 尹文才 wrote: > Thanks Andre, you

Issue with a simple flow

2017-02-17 Thread Arnaud G
Hello, I'm trying something very simple but I cannot get to make it work properly (Nifi 1.1.1) You will see the flow here: https://goo.gl/photos/9ewQV6ovKDDZe2pw9 My goal is to pull a json file out of HDFS, get some information and store it back as a CSV something really trivial. My flow is

Re: Nifi taking forever to start

2017-02-15 Thread Arnaud G
esto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Feb 15, 2017, at 1:29 AM, Arnaud G <greatpat...@gmail.com> wrote: > > Hi guys! > > I'm trying to play with nifi (1.1.1) in a docker i

Nifi taking forever to start

2017-02-15 Thread Arnaud G
Hi guys! I'm trying to play with nifi (1.1.1) in a docker image. I tried different configuration (cluster, single node, secured, etc.), however whatever I try, Nifi takes forever to start (like 30-45 minutes). This not related to data as I observe this behavior even when I instantiate the docker