Re: DBCPConnectionPoolLookup

[John Fak]

Awesome I'll try it next week and if I can get it to work I'll let you
know

Thx !!!

On Fri, Oct 4, 2019, 6:24 PM Peter Turcsanyi  wrote:

> Hi John,
>
> You need to add dynamic properties to DBCPConnectionPoolLookup controller
> service. The name of the property corresponds to database.name attribute,
> the value is the DBCPConnectionPool that can be looked up (by the name of
> the property).
> So the attribute database.name ties back to the name of the dynamic
> property, not to the name of the DBCPConnectionPool component.
>
> Please find attached my simple example. It contains only one
> DBCPConnectionPool that can be looked up, but a second one can be added in
> the same way.
>
> Regards,
> Peter Turcsanyi
>
> On Fri, Oct 4, 2019 at 10:13 PM John Fak  wrote:
>
>> Does someone have a basic example of using this ?
>> I really dont understand it. And it seems useful.
>>
>> Does the attribute database.name tie back to the "Name" of the
>> DBCPConnectionPool ?
>>
>> How do you pass that dynamically ?  Via some updateattibute processor ?
>>
>> Be great to see a very basic flow on connecting to two or more sources
>> dynamically.
>>
>>
>>
>>

Re: Which JAR can I find org.apache.nifi.record.path.*

[Matt Burgess]

Eric,

I have, specifically for the unit tests. You can find the scaffolding
at [1] and the actual script at [2] (it's trivial but should have all
the boilerplate). Also it may interest you to know that I have put up
a Pull Request for the nifi-record-path addition to the scripting
bundle [3] :)

Regards,
Matt

[1] 
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/groovy/org/apache/nifi/record/script/ScriptedRecordSetWriterTest.groovy
[2] 
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/resources/groovy/test_record_writer_inline.groovy
[3] https://github.com/apache/nifi/pull/3790

On Fri, Oct 4, 2019 at 5:37 PM Eric Chaves  wrote:
>
> Hi Matt, thanks for letting me know.
>
> Have you ever wrote a ScriptedRecordSetWriter? I couldn't find any example of 
> it. Do you have any to share?
>
> Best regards,
>
> Em sex, 4 de out de 2019 às 18:00, Matt Burgess  
> escreveu:
>>
>> Eric,
>>
>> The RecordPath classes are in the nifi-record-path JAR, which is not
>> currently included in the scripting bundle. I have written up an
>> improvement Jira [1] to add this.
>>
>> Regards,
>> Matt
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-6741
>>
>> On Fri, Oct 4, 2019 at 3:30 PM Eric Chaves  wrote:
>> >
>> > Hi folks,
>> >
>> > I'd like to use the Record Path classes in a groovy script for a 
>> > InvokeScriptedProcessor but when I import those classes they aren't found. 
>> > Can anyone point me the JAR where they reside?
>> >
>> > The classes I'm importing are
>> >
>> > import org.apache.nifi.record.path.FieldValue
>> > import org.apache.nifi.record.path.RecordPath
>> > import org.apache.nifi.record.path.RecordPathResult
>> > import org.apache.nifi.record.path.util.compiledRecordPaths
>> >
>> > Thanks in advance,
>> >
>> > Eric

Re: DBCPConnectionPoolLookup

[Peter Turcsanyi]

Hi John,

You need to add dynamic properties to DBCPConnectionPoolLookup controller
service. The name of the property corresponds to database.name attribute,
the value is the DBCPConnectionPool that can be looked up (by the name of
the property).
So the attribute database.name ties back to the name of the dynamic
property, not to the name of the DBCPConnectionPool component.

Please find attached my simple example. It contains only one
DBCPConnectionPool that can be looked up, but a second one can be added in
the same way.

Regards,
Peter Turcsanyi

On Fri, Oct 4, 2019 at 10:13 PM John Fak  wrote:

> Does someone have a basic example of using this ?
> I really dont understand it. And it seems useful.
>
> Does the attribute database.name tie back to the "Name" of the
> DBCPConnectionPool ?
>
> How do you pass that dynamically ?  Via some updateattibute processor ?
>
> Be great to see a very basic flow on connecting to two or more sources
> dynamically.
>
>
>
>



6a4bf1bf-016b-1000-1fda-2e79e336695e
DBCP


47e5dacd-f1e4-30b4--
bd34a285-0243-32c3--

0.0
0.0




be2b62c1-c19c-3dec--
47e5dacd-f1e4-30b4--
1 GB
1

47e5dacd-f1e4-30b4--
5f285237-9d31-3f8b--
PROCESSOR

0 sec
1
DO_NOT_COMPRESS

LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE

success

47e5dacd-f1e4-30b4--
32dcf1a4-9cea-3cd0--
PROCESSOR

0


e14cd5b8-85ba-3fe2--
47e5dacd-f1e4-30b4--
1 GB
1

47e5dacd-f1e4-30b4--
c7e7e246-88f5-3b4d--
PROCESSOR

0 sec
1
DO_NOT_COMPRESS

LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE

success

47e5dacd-f1e4-30b4--
5f285237-9d31-3f8b--
PROCESSOR

0


042a173c-50cd-3376--
47e5dacd-f1e4-30b4--

nifi-record-serialization-services-nar
org.apache.nifi
1.10.0-SNAPSHOT




Schema Write Strategy

Schema Write Strategy



schema-cache

org.apache.nifi.serialization.RecordSchemaCacheService
schema-cache



schema-access-strategy

schema-access-strategy



schema-registry

org.apache.nifi.schemaregistry.services.SchemaRegistry
schema-registry



schema-name

schema-name



schema-version

schema-version



schema-branch

schema-branch



schema-text

schema-text



Date Format

Date Format



Time Format
   

Re: Which JAR can I find org.apache.nifi.record.path.*

[Eric Chaves]

Hi Matt, thanks for letting me know.

Have you ever wrote a ScriptedRecordSetWriter? I couldn't find any example
of it. Do you have any to share?

Best regards,

Em sex, 4 de out de 2019 às 18:00, Matt Burgess 
escreveu:

> Eric,
>
> The RecordPath classes are in the nifi-record-path JAR, which is not
> currently included in the scripting bundle. I have written up an
> improvement Jira [1] to add this.
>
> Regards,
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-6741
>
> On Fri, Oct 4, 2019 at 3:30 PM Eric Chaves  wrote:
> >
> > Hi folks,
> >
> > I'd like to use the Record Path classes in a groovy script for a
> InvokeScriptedProcessor but when I import those classes they aren't found.
> Can anyone point me the JAR where they reside?
> >
> > The classes I'm importing are
> >
> > import org.apache.nifi.record.path.FieldValue
> > import org.apache.nifi.record.path.RecordPath
> > import org.apache.nifi.record.path.RecordPathResult
> > import org.apache.nifi.record.path.util.compiledRecordPaths
> >
> > Thanks in advance,
> >
> > Eric
>

H2 DB locking error - Timeout trying to lock table "ACTION"

[Pat White]

Hi Folks,

We recently encountered an error on a prod cluster where Nifi couldn't
acquire an H2 database lock when the canvas UI was modified, 'Timeout
trying to lock table "ACTION"'. I wasn't able to track down who held the
lock, or even where the lock was at. Ended up restarting the cluster, which
did clear the issue and restored modification ability.

Has anyone seen issues like this, H2 DB lock timeouts?

I don't believe the issue was due to timeout, we made sure all users
stopped modification attempts so we could debug, so the canvas should have
been quiescent. I believe there was a lock held, either a stale lock or
someone had a pending modification that wasn't committed. I'd really like
to know where to look for this lock, to track. I see locks in
Nifi's database_repository, like nifi-flow-audit.lock.db
and nifi-user-keys.lock.db but don't believe these were at issue.

Related, see some threads for H2 DB where use of mvcc settings were
recommended to support multiple version concurrency, such as discussed in
http://www.h2database.com/html/advanced.html#mvcc.

Is this advisable for Nifi ?

Thanks for any help or information regarding this.

patw

Re: Which JAR can I find org.apache.nifi.record.path.*

[Matt Burgess]

Eric,

The RecordPath classes are in the nifi-record-path JAR, which is not
currently included in the scripting bundle. I have written up an
improvement Jira [1] to add this.

Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-6741

On Fri, Oct 4, 2019 at 3:30 PM Eric Chaves  wrote:
>
> Hi folks,
>
> I'd like to use the Record Path classes in a groovy script for a 
> InvokeScriptedProcessor but when I import those classes they aren't found. 
> Can anyone point me the JAR where they reside?
>
> The classes I'm importing are
>
> import org.apache.nifi.record.path.FieldValue
> import org.apache.nifi.record.path.RecordPath
> import org.apache.nifi.record.path.RecordPathResult
> import org.apache.nifi.record.path.util.compiledRecordPaths
>
> Thanks in advance,
>
> Eric

Re: HandleHTTPRequest 503 error issue

[Wyllys Ingersoll]

Yes, thanks.  Is there an ETA for 1.10?

On Fri, Oct 4, 2019 at 3:05 PM Peter Turcsanyi 
wrote:

> Hi Wyllys,
>
> It has been fixed in NIFI-6317
>  which will be shipped
> in the upcoming 1.10 release.
> Could you please retest your workload on that version when it is available?
>
> Regards,
> Peter Turcsanyi
>
> On Fri, Oct 4, 2019 at 6:28 PM Wyllys Ingersoll <
> wyllys.ingers...@keepertech.com> wrote:
>
>>
>> I believe this issue: https://issues.apache.org/jira/browse/NIFI-5522
>> is a regression in Nifi 1.9.2, it was marked as fixed in 1.8.0, but Im
>> seeing it a lot in 1.9.2
>>
>> I can recreate it fairly regularly when I am hitting the endpoint pretty
>> heavily and an error is encountered.  The process return 503 error (or
>> nothing at all) and cannot be stopped or restarted without rebooting the
>> entire node.
>>
>> -Wyllys Ingersoll
>>
>>

DBCPConnectionPoolLookup

[John Fak]

Does someone have a basic example of using this ?
I really dont understand it. And it seems useful.

Does the attribute database.name tie back to the "Name" of the
DBCPConnectionPool ?

How do you pass that dynamically ?  Via some updateattibute processor ?

Be great to see a very basic flow on connecting to two or more sources
dynamically.

RE: HandleHTTPRequest 503 error issue

[William Gosse]

I’ve seen the same issue in 1.92 as well and opened the following ticket for it:
https://issues.apache.org/jira/browse/NIFI-6661

From: Wyllys Ingersoll 
Sent: Friday, October 4, 2019 12:28 PM
To: users@nifi.apache.org
Subject: HandleHTTPRequest 503 error issue

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]


I believe this issue: 
https://issues.apache.org/jira/browse/NIFI-5522
  is a regression in Nifi 1.9.2, it was marked as fixed in 1.8.0, but Im seeing 
it a lot in 1.9.2

I can recreate it fairly regularly when I am hitting the endpoint pretty 
heavily and an error is encountered.  The process return 503 error (or nothing 
at all) and cannot be stopped or restarted without rebooting the entire node.

-Wyllys Ingersoll

Which JAR can I find org.apache.nifi.record.path.*

[Eric Chaves]

Hi folks,

I'd like to use the Record Path classes in a groovy script for a
InvokeScriptedProcessor but when I import those classes they aren't found.
Can anyone point me the JAR where they reside?

The classes I'm importing are

import org.apache.nifi.record.path.FieldValue
import org.apache.nifi.record.path.RecordPath
import org.apache.nifi.record.path.RecordPathResult
import org.apache.nifi.record.path.util.compiledRecordPaths

Thanks in advance,

Eric

Re: HandleHTTPRequest 503 error issue

[Peter Turcsanyi]

Hi Wyllys,

It has been fixed in NIFI-6317
 which will be shipped in
the upcoming 1.10 release.
Could you please retest your workload on that version when it is available?

Regards,
Peter Turcsanyi

On Fri, Oct 4, 2019 at 6:28 PM Wyllys Ingersoll <
wyllys.ingers...@keepertech.com> wrote:

>
> I believe this issue: https://issues.apache.org/jira/browse/NIFI-5522  is
> a regression in Nifi 1.9.2, it was marked as fixed in 1.8.0, but Im seeing
> it a lot in 1.9.2
>
> I can recreate it fairly regularly when I am hitting the endpoint pretty
> heavily and an error is encountered.  The process return 503 error (or
> nothing at all) and cannot be stopped or restarted without rebooting the
> entire node.
>
> -Wyllys Ingersoll
>
>

HandleHTTPRequest 503 error issue

[Wyllys Ingersoll]

I believe this issue: https://issues.apache.org/jira/browse/NIFI-5522  is a
regression in Nifi 1.9.2, it was marked as fixed in 1.8.0, but Im seeing it
a lot in 1.9.2

I can recreate it fairly regularly when I am hitting the endpoint pretty
heavily and an error is encountered.  The process return 503 error (or
nothing at all) and cannot be stopped or restarted without rebooting the
entire node.

-Wyllys Ingersoll