Re: Support for Elastic Search in Future releases
Shweta, I am also looking forward for ES support. In the mean time I inject data using PostHTTP processor and can try to help you if you have any questions. I even managed to create index-per-day because http processor URL property supports scripting language and I can format current date as a string. The only problem I have now is assembling data for bulk indexing Doing something wrong.. On Dec 18, 2015 6:48 AM, "Matthew Burgess" wrote: > Shweta, > > There is a Jira case for Elasticsearch processors: > > https://issues.apache.org/jira/browse/NIFI-1275 > > > I plan to work on these (with other folks in the NiFi community if > interested) very soon. > > Regards, > Matt > > > > On 12/18/15, 2:17 AM, "shweta" wrote: > > >Hi, > > > >I wanted to know if there are any plans to have custom processors > supporting > >Data ingestion/egestion > >for Elastic search just like there is on for SOLR. > > > >Thanks, > >Shweta > > > > > > > >-- > >View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/Support-for-Elastic-Search-in-Future-releases-tp5849.html > >Sent from the Apache NiFi Developer List mailing list archive at > Nabble.com. > >
Re: How to iterate through complex JSON objects.
The question is "Is NiFi supposed to be a full ETL tool"? On Dec 16, 2015 11:27 AM, "Angry Duck Studio" wrote: > Shweta, > > I think your issue demonstrates one of my minor complaints with NiFi -- > that you always have to think in terms of several little, built-in pieces > to get a simple job done. Sometimes it's fun, like a puzzle, but other > times, I don't feel like dealing with it. That's why I wrote this: > https://github.com/mring33621/scripting-for-nifi. A short, custom JS or > Groovy script could have handled your JSON data munging in a single stroke. > > -Matt > > On Tue, Dec 15, 2015 at 8:40 PM, shweta wrote: > > > Thanks Bryan!! Infact I followed the exact approach that you told. Just > > that > > I was clueless about using Mergecontent processor. So I wrote my custom > > script to combine the different outputs and executed it using Execute > > Stream > > command. > > Will try the same with Mergecontent. > > > > > > > > -- > > View this message in context: > > > http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-iterate-through-complex-JSON-objects-tp5776p5806.html > > Sent from the Apache NiFi Developer List mailing list archive at > > Nabble.com. > > >
Re: GetHTTP processor error
That would be nice addition.
On Tue, Dec 15, 2015 at 5:46 PM, Mark Payne wrote:
> That looks about right to me. But that seems a little bit painful. Perhaps
> we can make that easier by offering a new Property that allows you to just
> tell the Processor that it needs to do the URL Encoding for you. Would that
> make life easier for you?
>
> Thanks
> -Mark
>
>
> > On Dec 15, 2015, at 5:43 PM, Igor Kravzov
> wrote:
> >
> > Guys, I think I get it. Is this how it should be?
> >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=${literal('Matt
> >
> Dupe'):urlEncode()}&xmloutputversion=2&ts=${now():minus(60):format("-MM-dd
> > HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
> >
> > On Tue, Dec 15, 2015 at 5:38 PM, Joe Percivall <
> > joeperciv...@yahoo.com.invalid> wrote:
> >
> >> I think you're thinking about it the wrong way. You aren't actually
> using
> >> any attributes (attributes are on flowfiles, properties on processors),
> >> just utilizing the functions of expression language. So you can manually
> >> put in everything you would normally as a literal then reference the
> >> expression language functions in the expression using ${*expression*}.
> >>
> >> That being said, since GetHttp is a source processor you shouldn't be
> >> changing that URL all that often so you could just manually replace the
> >> space with "%20" (URL encoding of a space).
> >>
> >> Here is a link to a URL encoder/decoder:
> >> http://meyerweb.com/eric/tools/dencoder/
> >>
> >> Joe
> >> - - - - - -
> >> Joseph Percivall
> >> linkedin.com/in/Percivall
> >> e: joeperciv...@yahoo.com
> >>
> >>
> >>
> >>
> >> On Tuesday, December 15, 2015 5:32 PM, Igor Kravzov <
> >> igork.ine...@gmail.com> wrote:
> >> But how I define attribute "url" in GetHTTP processor? I tried to ad a
> >> property and got an error that property not defined or supported.
> >>
> >>
> >> On Tue, Dec 15, 2015 at 4:57 PM, Joe Percivall <
> >> joeperciv...@yahoo.com.invalid> wrote:
> >>
> >>> Hello Igor,
> >>>
> >>> The URL property for GetHTTP supports expression language. Check out
> this
> >>> function:
> >>>
> >>
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#urlencode
> >>>
> >>>
> >>> Hope that helps,
> >>> Joe
> >>>
> >>> - - - - - -
> >>> Joseph Percivall
> >>> linkedin.com/in/Percivall
> >>> e: joeperciv...@yahoo.com
> >>>
> >>>
> >>>
> >>>
> >>> On Tuesday, December 15, 2015 4:46 PM, Igor Kravzov <
> >>> igork.ine...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> Hi Joe,
> >>>
> >>> Another quick question. How can I encode some part of the URL? Like in
> >>> example bellow after searchpattern=
> >>> <
> >>>
> >>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(
> >>>>
> >>> space
> >>> between Matt Dupe needs to be encoded. Otherwise GetHTTP throws an
> >> error.
> >>>
> >>>
> >>>
> >>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=Matt
> >>> Dupe&xmloutputversion=2&ts=${now():minus(60):format(
> >>> <
> >>>
> >>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(
> >>>>
> >>> "-MM-dd HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
> >>>
> >>>
> >>> On Mon, Dec 14, 2015 at 6:16 PM, Joe Percivall <
> >>> joeperciv...@yahoo.com.invalid> wrote:
> >>>
> >>>> Glad I could help and thanks!
> >>>> - - - - - -
> >>>> Joseph Percivall
> >>>> linkedin.com/in/Percivall
> >>>> e: joeperciv...@yahoo.com
> >>>>
> >>>&
Re: GetHTTP processor error
Guys, I think I get it. Is this how it should be?
http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=${literal('Matt
Dupe'):urlEncode()}&xmloutputversion=2&ts=${now():minus(60):format("-MM-dd
HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
On Tue, Dec 15, 2015 at 5:38 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:
> I think you're thinking about it the wrong way. You aren't actually using
> any attributes (attributes are on flowfiles, properties on processors),
> just utilizing the functions of expression language. So you can manually
> put in everything you would normally as a literal then reference the
> expression language functions in the expression using ${*expression*}.
>
> That being said, since GetHttp is a source processor you shouldn't be
> changing that URL all that often so you could just manually replace the
> space with "%20" (URL encoding of a space).
>
> Here is a link to a URL encoder/decoder:
> http://meyerweb.com/eric/tools/dencoder/
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
>
>
>
>
> On Tuesday, December 15, 2015 5:32 PM, Igor Kravzov <
> igork.ine...@gmail.com> wrote:
> But how I define attribute "url" in GetHTTP processor? I tried to ad a
> property and got an error that property not defined or supported.
>
>
> On Tue, Dec 15, 2015 at 4:57 PM, Joe Percivall <
> joeperciv...@yahoo.com.invalid> wrote:
>
> > Hello Igor,
> >
> > The URL property for GetHTTP supports expression language. Check out this
> > function:
> >
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#urlencode
> >
> >
> > Hope that helps,
> > Joe
> >
> > - - - - - -
> > Joseph Percivall
> > linkedin.com/in/Percivall
> > e: joeperciv...@yahoo.com
> >
> >
> >
> >
> > On Tuesday, December 15, 2015 4:46 PM, Igor Kravzov <
> > igork.ine...@gmail.com> wrote:
> >
> >
> >
> > Hi Joe,
> >
> > Another quick question. How can I encode some part of the URL? Like in
> > example bellow after searchpattern=
> > <
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(
> > >
> > space
> > between Matt Dupe needs to be encoded. Otherwise GetHTTP throws an
> error.
> >
> >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=Matt
> > Dupe&xmloutputversion=2&ts=${now():minus(60):format(
> > <
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(
> > >
> > "-MM-dd HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
> >
> >
> > On Mon, Dec 14, 2015 at 6:16 PM, Joe Percivall <
> > joeperciv...@yahoo.com.invalid> wrote:
> >
> > > Glad I could help and thanks!
> > > - - - - - -
> > > Joseph Percivall
> > > linkedin.com/in/Percivall
> > > e: joeperciv...@yahoo.com
> > >
> > >
> > >
> > >
> > > On Monday, December 14, 2015 6:09 PM, Igor Kravzov <
> > igork.ine...@gmail.com>
> > > wrote:
> > > Thank you very much Joe. It worked.
> > > And congratulations.
> > >
> > >
> > > On Mon, Dec 14, 2015 at 6:00 PM, Joe Percivall <
> > > joeperciv...@yahoo.com.invalid> wrote:
> > >
> > > > Hello Igor,
> > > >
> > > > You're having trouble because you have a space in your format of the
> > > dates
> > > > which need to be URL encoded. You can use the EL method "urlencode"
> to
> > > get
> > > > a valid expression like such:
> > > >
> > > >
> > > >
> > >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format(
> > > "-MM-dd
> > > > HH:mm:ss"):urlEncode()}&tsTo=${now():format("-MM-dd
> > > > HH:mm:ss"):urlEncode()}
> > > >
> > > > Hope that helps,
> > > > Joe
> > > > - - - - - -
&g
Re: GetHTTP processor error
But how I define attribute "url" in GetHTTP processor? I tried to ad a
property and got an error that property not defined or supported.
On Tue, Dec 15, 2015 at 4:57 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:
> Hello Igor,
>
> The URL property for GetHTTP supports expression language. Check out this
> function:
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#urlencode
>
>
> Hope that helps,
> Joe
>
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
>
>
>
>
> On Tuesday, December 15, 2015 4:46 PM, Igor Kravzov <
> igork.ine...@gmail.com> wrote:
>
>
>
> Hi Joe,
>
> Another quick question. How can I encode some part of the URL? Like in
> example bellow after searchpattern=
> <
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(
> >
> space
> between Matt Dupe needs to be encoded. Otherwise GetHTTP throws an error.
>
>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=Matt
> Dupe&xmloutputversion=2&ts=${now():minus(60):format(
> <
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(
> >
> "-MM-dd HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
>
>
> On Mon, Dec 14, 2015 at 6:16 PM, Joe Percivall <
> joeperciv...@yahoo.com.invalid> wrote:
>
> > Glad I could help and thanks!
> > - - - - - -
> > Joseph Percivall
> > linkedin.com/in/Percivall
> > e: joeperciv...@yahoo.com
> >
> >
> >
> >
> > On Monday, December 14, 2015 6:09 PM, Igor Kravzov <
> igork.ine...@gmail.com>
> > wrote:
> > Thank you very much Joe. It worked.
> > And congratulations.
> >
> >
> > On Mon, Dec 14, 2015 at 6:00 PM, Joe Percivall <
> > joeperciv...@yahoo.com.invalid> wrote:
> >
> > > Hello Igor,
> > >
> > > You're having trouble because you have a space in your format of the
> > dates
> > > which need to be URL encoded. You can use the EL method "urlencode" to
> > get
> > > a valid expression like such:
> > >
> > >
> > >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format(
> > "-MM-dd
> > > HH:mm:ss"):urlEncode()}&tsTo=${now():format("-MM-dd
> > > HH:mm:ss"):urlEncode()}
> > >
> > > Hope that helps,
> > > Joe
> > > - - - - - -
> > > Joseph Percivall
> > > linkedin.com/in/Percivall
> > > e: joeperciv...@yahoo.com
> > >
> > >
> > >
> > >
> > > On Monday, December 14, 2015 5:31 PM, Igor Kravzov <
> > igork.ine...@gmail.com>
> > > wrote:
> > > Hi guys,
> > >
> > > Why I am getting the error bellow? I am constructing URL like this:
> > >
> > >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format(
> > > "-MM-dd
> > > HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
> > >
> > >
> > > Am I missing something? Thanks in advance.
> > >
> > >
> > > java.lang.IllegalArgumentException: Illegal character in query at index
> > > 143:
> > >
> > >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
> > > 17:03:46&tsTo=2015-12-14 17:13:46
> > > at java.net.URI.create(Unknown Source) ~[na:1.8.0_66]
> > > at org.apache.http.client.methods.HttpGet.(HttpGet.java:69)
> > > ~[httpclient-4.4.1.jar:4.4.1]
> > > at
> > org.apache.nifi.processors.standard.GetHTTP.onTrigger(GetHTTP.java:444)
> > > ~[na:na]
> > > at
> > >
> > >
> >
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
> > > ~[nifi-framework-core-0.4.0.jar:0.4.0]
> > > at
> > >
> > >
> >
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTas
Re: GetHTTP processor error
Hi Joe,
Another quick question. How can I encode some part of the URL? Like in
example bellow after searchpattern=
<http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(>
space
between Matt Dupe needs to be encoded. Otherwise GetHTTP throws an error.
http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=Matt
Dupe&xmloutputversion=2&ts=${now():minus(60):format(
<http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=$%7Bnow():minus(60):format(>
"-MM-dd HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
On Mon, Dec 14, 2015 at 6:16 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:
> Glad I could help and thanks!
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
>
>
>
>
> On Monday, December 14, 2015 6:09 PM, Igor Kravzov
> wrote:
> Thank you very much Joe. It worked.
> And congratulations.
>
>
> On Mon, Dec 14, 2015 at 6:00 PM, Joe Percivall <
> joeperciv...@yahoo.com.invalid> wrote:
>
> > Hello Igor,
> >
> > You're having trouble because you have a space in your format of the
> dates
> > which need to be URL encoded. You can use the EL method "urlencode" to
> get
> > a valid expression like such:
> >
> >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format(
> "-MM-dd
> > HH:mm:ss"):urlEncode()}&tsTo=${now():format("-MM-dd
> > HH:mm:ss"):urlEncode()}
> >
> > Hope that helps,
> > Joe
> > - - - - - -
> > Joseph Percivall
> > linkedin.com/in/Percivall
> > e: joeperciv...@yahoo.com
> >
> >
> >
> >
> > On Monday, December 14, 2015 5:31 PM, Igor Kravzov <
> igork.ine...@gmail.com>
> > wrote:
> > Hi guys,
> >
> > Why I am getting the error bellow? I am constructing URL like this:
> >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format(
> > "-MM-dd
> > HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
> >
> >
> > Am I missing something? Thanks in advance.
> >
> >
> > java.lang.IllegalArgumentException: Illegal character in query at index
> > 143:
> >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
> > 17:03:46&tsTo=2015-12-14 17:13:46
> > at java.net.URI.create(Unknown Source) ~[na:1.8.0_66]
> > at org.apache.http.client.methods.HttpGet.(HttpGet.java:69)
> > ~[httpclient-4.4.1.jar:4.4.1]
> > at
> org.apache.nifi.processors.standard.GetHTTP.onTrigger(GetHTTP.java:444)
> > ~[na:na]
> > at
> >
> >
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
> > ~[nifi-framework-core-0.4.0.jar:0.4.0]
> > at
> >
> >
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> > [nifi-framework-core-0.4.0.jar:0.4.0]
> > at
> >
> >
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> > [nifi-framework-core-0.4.0.jar:0.4.0]
> > at
> >
> >
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
> > [nifi-framework-core-0.4.0.jar:0.4.0]
> > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> > [na:1.8.0_66]
> > at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> > [na:1.8.0_66]
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> > Source) [na:1.8.0_66]
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> > Source) [na:1.8.0_66]
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> > [na:1.8.0_66]
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> > [na:1.8.0_66]
> > at java.lang.Thread.run(Unknown Source) [na:1.8.0_66]
> > Caused by: java.net.URISyntaxException: Illegal character in query at
> index
> > 143:
> >
> >
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
> > 17:03:46&tsTo=2015-12-14 17:13:46
> > at java.net.URI$Parser.fail(Unknown Source) ~[na:1.8.0_66]
> > at java.net.URI$Parser.checkChars(Unknown Source) ~[na:1.8.0_66]
> > at java.net.URI$Parser.parseHierarchical(Unknown Source) ~[na:1.8.0_66]
> > at java.net.URI$Parser.parse(Unknown Source) ~[na:1.8.0_66]
> > at java.net.URI.(Unknown Source) ~[na:1.8.0_66]
> > ... 14 common frames omitted
> >
>
Re: GetHTTP processor error
Thank you very much Joe. It worked.
And congratulations.
On Mon, Dec 14, 2015 at 6:00 PM, Joe Percivall <
joeperciv...@yahoo.com.invalid> wrote:
> Hello Igor,
>
> You're having trouble because you have a space in your format of the dates
> which need to be URL encoded. You can use the EL method "urlencode" to get
> a valid expression like such:
>
>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format("-MM-dd
> HH:mm:ss"):urlEncode()}&tsTo=${now():format("-MM-dd
> HH:mm:ss"):urlEncode()}
>
> Hope that helps,
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
>
>
>
>
> On Monday, December 14, 2015 5:31 PM, Igor Kravzov
> wrote:
> Hi guys,
>
> Why I am getting the error bellow? I am constructing URL like this:
>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format(
> "-MM-dd
> HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
>
>
> Am I missing something? Thanks in advance.
>
>
> java.lang.IllegalArgumentException: Illegal character in query at index
> 143:
>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
> 17:03:46&tsTo=2015-12-14 17:13:46
> at java.net.URI.create(Unknown Source) ~[na:1.8.0_66]
> at org.apache.http.client.methods.HttpGet.(HttpGet.java:69)
> ~[httpclient-4.4.1.jar:4.4.1]
> at org.apache.nifi.processors.standard.GetHTTP.onTrigger(GetHTTP.java:444)
> ~[na:na]
> at
>
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
> ~[nifi-framework-core-0.4.0.jar:0.4.0]
> at
>
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.4.0.jar:0.4.0]
> at
>
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.4.0.jar:0.4.0]
> at
>
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
> [nifi-framework-core-0.4.0.jar:0.4.0]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> [na:1.8.0_66]
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> [na:1.8.0_66]
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source) [na:1.8.0_66]
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source) [na:1.8.0_66]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.8.0_66]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.8.0_66]
> at java.lang.Thread.run(Unknown Source) [na:1.8.0_66]
> Caused by: java.net.URISyntaxException: Illegal character in query at index
> 143:
>
> http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
> 17:03:46&tsTo=2015-12-14 17:13:46
> at java.net.URI$Parser.fail(Unknown Source) ~[na:1.8.0_66]
> at java.net.URI$Parser.checkChars(Unknown Source) ~[na:1.8.0_66]
> at java.net.URI$Parser.parseHierarchical(Unknown Source) ~[na:1.8.0_66]
> at java.net.URI$Parser.parse(Unknown Source) ~[na:1.8.0_66]
> at java.net.URI.(Unknown Source) ~[na:1.8.0_66]
> ... 14 common frames omitted
>
GetHTTP processor error
Hi guys,
Why I am getting the error bellow? I am constructing URL like this:
http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=${now():minus(60):format("-MM-dd
HH:mm:ss")}&tsTo=${now():format("-MM-dd HH:mm:ss")}
Am I missing something? Thanks in advance.
java.lang.IllegalArgumentException: Illegal character in query at index
143:
http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
17:03:46&tsTo=2015-12-14 17:13:46
at java.net.URI.create(Unknown Source) ~[na:1.8.0_66]
at org.apache.http.client.methods.HttpGet.(HttpGet.java:69)
~[httpclient-4.4.1.jar:4.4.1]
at org.apache.nifi.processors.standard.GetHTTP.onTrigger(GetHTTP.java:444)
~[na:na]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
~[nifi-framework-core-0.4.0.jar:0.4.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
[nifi-framework-core-0.4.0.jar:0.4.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
[nifi-framework-core-0.4.0.jar:0.4.0]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
[nifi-framework-core-0.4.0.jar:0.4.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[na:1.8.0_66]
at java.util.concurrent.FutureTask.runAndReset(Unknown Source) [na:1.8.0_66]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
Source) [na:1.8.0_66]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) [na:1.8.0_66]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.8.0_66]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.8.0_66]
at java.lang.Thread.run(Unknown Source) [na:1.8.0_66]
Caused by: java.net.URISyntaxException: Illegal character in query at index
143:
http://api.twingly.com/analytics/Analytics.ashx?key=75744154-6ACB-3340-937A-9B5A59FA8F30&searchpattern=boycott&xmloutputversion=2&ts=2015-12-14
17:03:46&tsTo=2015-12-14 17:13:46
at java.net.URI$Parser.fail(Unknown Source) ~[na:1.8.0_66]
at java.net.URI$Parser.checkChars(Unknown Source) ~[na:1.8.0_66]
at java.net.URI$Parser.parseHierarchical(Unknown Source) ~[na:1.8.0_66]
at java.net.URI$Parser.parse(Unknown Source) ~[na:1.8.0_66]
at java.net.URI.(Unknown Source) ~[na:1.8.0_66]
... 14 common frames omitted
Re: MergeContent processor - set demarcator as new line
OH. Definitely I'll download 0.4.0 tomorrow and check :) On Sun, Dec 13, 2015 at 10:25 PM, Joe Witt wrote: > No. Just means you're on apache nifi 0.3.0 though. As Bryan pointed > out 0.4.0 (just released) this may be fixed. > > Thanks > Joe > > On Sun, Dec 13, 2015 at 7:45 PM, Igor K. wrote: > > I probably should mention that I use NiFi packaged by Hortonworks as > HDF. May > > it make a difference? > > > > > > > > -- > > View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/MergeContent-processor-set-demarcator-as-new-line-tp5747p5754.html > > Sent from the Apache NiFi Developer List mailing list archive at > Nabble.com. >
MergeContent processor - set demarcator as new line
I want to use MergeContent processor to merge tweets to bulk insert into
Elasticsearch index. For this I need command and tweets (each) to be separated
by \n.
This is how it should look like
{ action: { metadata }}\n
{ request body}\n
Which will be
{ "index" )\n
{ tweet1 }\n
{ twee2 }\n
When I put \n as demarcator the processor actually adds \n as a string instead
of new line separator. Is it possible to make it actual new line?
Also is it possible to leave or make footer empty?
Thanks in advance.
XML to JSON converter
Hi guys, In the current 0.3.0 NiFi release there is no XML to JSON converter processor. I needed one for a project I am working on and wrote one. Very simple one with just a couple lines of code using Java library form json.org http://mvnrepository.com/artifact/org.json/json/20151123 Do you guys have a plan for this type of convertor/processor? I will be glad to help and submit converter I wrote to to the NiFi project. Let me know what do you think. Igor
