Matt,
Not sure if it's related.
I'm trying to use a timestamp column as Maximum-value Column, but it keeps
looping.
I have set Use Avro Logical Types = true on my QueryDatabase processor.
The original columns values are like this:
_Time-Stamp
----------------------------------
2017-12-01 14:35:56:204 - 02:00
2017-12-01 14:35:56:211 - 02:00
2017-12-01 15:25:35:945 - 02:00
2017-12-01 15:25:35:945 - 02:00
2017-12-01 15:28:23:046 - 02:00
So I'm converting to timestamp milis using CAST("_Time-Stamp" as
TIMESTAMP)"_Time-Stamp"
_Time-Stamp
-------------------------
2017-12-01 14:35:56.204
2017-12-01 14:35:56.211
2017-12-01 15:25:35.945
2017-12-01 15:25:35.945
2017-12-01 15:28:23.046
The state seems right:
KeyValueScope
pub."man_fabrica-cdc"@!@_time-stamp 2017-12-04 15:33:23.995
Cluster
Any clue?
Thank you!
Alberto
On Wed, Dec 6, 2017 at 4:06 PM, Alberto Bengoa <albe...@propus.com.br>
wrote:
> Matt,
>
> Perfect! Enabled and working now.
>
> Thank you!
>
> Cheers,
> Alberto
>
>
>
> On Wed, Dec 6, 2017 at 3:54 PM, Matt Burgess <mattyb...@apache.org> wrote:
>
>> Alberto,
>>
>> What version of NiFi are you using? As of version 1.1.0,
>> QueryDatabaseTable has a "Normalize Table/Column Names" property that
>> you can set to true, and it will replace all Avro-illegal characters
>> with underscores.
>>
>> Regards,
>> Matt
>>
>>
>> On Wed, Dec 6, 2017 at 12:06 PM, Alberto Bengoa <albe...@propus.com.br>
>> wrote:
>> > Hey Folks,
>> >
>> > I'm facing an odd situation with Nifi and Tables / Columns that have
>> hyphens
>> > on names (traceback below).
>> >
>> > I found on Avro Spec [1] that hyphens are not allowed, which makes
>> sense to
>> > have this error.
>> >
>> > There is any way to deal with this situation on Nifi instead of changing
>> > table/columns name or creating views to rename the hyphenated names?
>> >
>> > I'm getting this error on the first processor (QueryDatabaseTable) of my
>> > flow.
>> >
>> > Thanks!
>> >
>> > Alberto Bengoa
>> >
>> > [1] - https://avro.apache.org/docs/1.7.7/spec.html#Names
>> >
>> >
>> > 2017-12-06 14:37:25,809 ERROR [Timer-Driven Process Thread-2]
>> > o.a.n.p.standard.QueryDatabaseTable
>> > QueryDatabaseTable[id=9557387b-bbd6-1b2f-b68b-5a4458986794] Unable to
>> > execute SQL select query SELECT "_Change-Sequence" FROM
>> PUB.man_factory_cdc
>> > due to org.apache.nifi.processor.exception.ProcessException: Error
>> during
>> > database query or conversion of records to Avro.: {}
>> > org.apache.nifi.processor.exception.ProcessException: Error during
>> database
>> > query or conversion of records to Avro.
>> > at
>> > org.apache.nifi.processors.standard.QueryDatabaseTable.lambd
>> a$onTrigger$0(QueryDatabaseTable.java:289)
>> > at
>> > org.apache.nifi.controller.repository.StandardProcessSession
>> .write(StandardProcessSession.java:2526)
>> > at
>> > org.apache.nifi.processors.standard.QueryDatabaseTable.onTri
>> gger(QueryDatabaseTable.java:283)
>> > at
>> > org.apache.nifi.controller.StandardProcessorNode.onTrigger(S
>> tandardProcessorNode.java:1118)
>> > at
>> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>> .call(ContinuallyRunProcessorTask.java:147)
>> > at
>> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>> .call(ContinuallyRunProcessorTask.java:47)
>> > at
>> > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingA
>> gent$1.run(TimerDrivenSchedulingAgent.java:132)
>> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:
>> 308)
>> > at
>> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>> > at
>> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.run(ScheduledThreadPoolExecutor.java:294)
>> > at
>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> > at
>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Caused by: org.apache.avro.SchemaParseException: Illegal character in:
>> > _Change-Sequence
>> >
>>
>
>
