Re: How to know the processors last running time?

Thank you for the help. On Mon, Mar 19, 2018 at 5:13 PM, Jorge Machado <jom...@me.com> wrote: > From Nifi default I don’t know any other way. You can always create a > entry on a table and write the information that you need into it. > > Jorge > > > On 19 Mar 2018,

Re: How to know the processors last running time?

> > > > > On 19 Mar 2018, at 07:57, prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > > i'm having group of processors to perform some functionalities. > > I have scheduled the starting processor to be run upon daily at particular > time by using cron expre

How to know the processors last running time?

i'm having group of processors to perform some functionalities. I have scheduled the starting processor to be run upon daily at particular time by using cron expression. I need to know that the processors last running time. just consider an example if i having getfile,generateflowfile processor

Is this possible to create table from JSON?

Hi All, I can able to get avro schema from json. It could be look like below. { "type" : "array", "items" : { "type" : "record", "name" : "AvroSchema", "fields" : [ { "name" : "Name", "type" : "String", "doc" : "Type inferred from 'S'" }, { "name" : "Age", "type" : "int", "doc" : "Type inferred

Re: Nifi: how to split logs?

Sally, Try this link ( https://community.hortonworks.com/articles/131320/using-partitionrecord-grokreaderjsonwriter-to-pars.html) . *GrokReader which helps to read log file with help of GrokExpression and in which you can parse by only error,info and warn data by using RouteOnAttribute.* Hope

Is this possible to access shared network files in NiFi?

i wants to process shared drive data into NiFi. I have shared file in which present in connected network.It prompts for credentials before access that file. Now i have to fetch that file from shared network(//hostname/shared/file) with use of credentials(username/pwd) and then process those

How to combine the row with column wise in nifi?

Am having below sample data. ,r1c2,r1c3,r1c4,r1c5 Date,r2c2,r2c3,r2c4,r2c5 07-08-2017,r3c2,r3c3,r3c4,r3c5 i need to convert those data like below. 07-08-2017,r1c2,r2c2,r3c2 07-08-2017,r1c3,r2c3,r3c3 07-08-2017,r1c4,r2c4,r3c4 07-08-2017,r1c5,r2c5,r3c5 i have tried those with nifi looping

Re: MaxHistoryNot Worked in logback-1.2.3?

Hi Aldrin, Thanks for your response. i wants to keep only 5 files each file should having size 100KB. If total size of the files reaches 500KB then it will delete oldest file and then write new file. I have tried SizeAndTimeBasedPolicy then it couldn't preferable for me to use with respect to

MaxHistoryNot Worked in logback-1.2.3?

I have tried below code to write logs based on SizeAndTimeBasedRollingPolicy . In that i can able to write files with respect to file size 100KB but not able to maintain only 2 maxHistory files.It always write new file after 100KB successfully written. Now i have using NiFI-1.3.0 which having

Re: how to use LookUpRecord processor?

ot;name": "ID", "type": "int" }, > { "name": "Age", "type": "int" }, > { "name": "Sex", "type": ["string", "null"] }, > ] > } > > Make sure yo

how to use LookUpRecord processor?

I tried to join two csv file based on id with respect to the below reference. How to join two CSVs with Apache Nifi i'm using NiFi-1.3.0 Now i have two csv files. 1.custom.csv No,Name,ID,Age1,Hik,2201,332,Kio,3300,22 2.gender.csv ID,Name,Sex2201,Hik,Male3300,Kio,FemaleI try to combine those

How to pass the username while download file from Web Url?

I have secured downloadable link to download the file. So configured the link in InvokeHTTP processor with StandardSSLContextService with the truststore certificate. In response, I get the HTML tags i.e., page source of the link. But using RestClient opensource if I logged into the link, data is

Re: How to update line with modified data in Jython?

owFile != None): flowFile = session.write(flowFile,PyStreamCallback(log)) flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename')) session.transfer(flowFile, REL_SUCCESS) Please let me know if I've misunderstood anything, and I will try to fix/improve the sc

How to update line with modified data in Jython?

I'm having one csv which contains lakhs of rows and below is sample lines.., 1,Ni,23,28-02-2015 12:22:33.2212-02 2,Fi,21,29-02-2015 12:22:34.3212-02 3,Us,33,30-03-2015 12:23:35-01 4,Uk,34,31-03-2015 12:24:36.332211-02 I need to get the last column of csv data which is in wrong datetime format. So

Re: How to transfer files between two windows machines using NiFi?

/nifi.apache.org/docs/nifi-docs/html/user- > guide.html#site-to-site > > > > On Jun 14, 2017, at 2:53 AM, prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > > Hi All, > > > i know that using "RoboCopy" and "Copy" to move the files

How to transfer files between two windows machines using NiFi?

Hi All, i know that using "RoboCopy" and "Copy" to move the files between two machines. I need to know how to configure those commands ExecuteStreamCommand processor in NiFi. Is there is any other option available without using ExecuteStream/ExecuteProcess processors? Thanks,

Re: How to ensure the rows moved into SQL?

ted > in the last time slice or with a special identifier range by checking > “max(id)” before the insert begins. > > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F6

How to ensure the rows moved into SQL?

Hi All, Since i need to know how to check all rows in csv moved in SQL Server. I have download csv files from HTTP. Just imagine i have 10 files i could move files one after another into SQL Server. i need to ensure if number of rows in csv moves correctly insert into SQL Server. For example

Re: How to perform bulk insert into SQLServer from one machine to another?

hat to issue your BULK INSERT statement. Regards, Matt [1] https://docs.microsoft.com/en-us/sql/t-sql/statements/bulk- insert-transact-sql On Thu, Jun 8, 2017 at 8:11 AM, prabhu Mahendran <prabhuu161...@gmail.com> wrote: > i have running nifi instance in one machine and have SQL Server in

How to perform bulk insert into SQLServer from one machine to another?

i have running nifi instance in one machine and have SQL Server in another machine. Here i can try to perform bulk insert operation with bulk insert Query in SQLserver. but i cannot able insert data from one machine and move it into SQL Server in another machine. If i run nifi and SQL Server in

How to find week of the year for the date value?

My Flow having attribute '05-05-2015'.And i need to find week of the date in NiFi. For example: if date is 05 then it is belongs to 1st week of the year. I shouldn't use Script/Program for find out week. i need to find out that using NiFi Expression language only. Can anyone suggest way to do

Re: some processors runs only once in NiFi

le. The screenshot > indicates that GetHTTP runs 61 times but it only creates output > FlowFile once because it's not modified. > > I believe that is what's happening. > > Thanks, > Koji > > On Wed, May 24, 2017 at 2:30 PM, prabhu Mahendran > <prabhuu161...@gmail.c

Re: some processors runs only once in NiFi

the version is the > latest, could you please provide some more details in terms of structure > and timestamps of the associated files causing the issue you are describing? > > > [1] https://en.wikipedia.org/wiki/HTTP_ETag > > On Tue, May 23, 2017 at 3:22 AM, prabhu Mahendran

some processors runs only once in NiFi

Since i have faced some unexpected behaviour's in NiFi. I don't know why those processors which doesn't run more than once. *For example:* *1.GetHTTP:* I have used GetHTTP processor for download files from "HTTP" Url. Initially i have scheduled 0 sec If i runs the processor it runs only once

Re: How to process files sequentially?

cfcb0b5911acefbb455edf0/raw/967a051177878a98f5ddb57653478c > 6091a7b23c/process-files-in-order-and-defrag.xml > > Thanks, > Koji > > On Fri, May 19, 2017 at 6:11 PM, prabhu Mahendran > <prabhuu161...@gmail.com> wrote: > > Hi Koji, > > > > Thanks for yo

Re: How to process files sequentially?

t; > Thanks, > Koji > > On Fri, May 19, 2017 at 1:52 PM, prabhu Mahendran > <prabhuu161...@gmail.com> wrote: > > I have approximately 1000 files in local drive.I need to move that files > > into SQL Server accordingly one after another. > > > > Since loc

Re: How to lock getfile upto putfile write into same file?

ave PutFile and > GetFile sharing a given file in the same NiFi as you can simply > connect the processors to move the data without have to leave the > confines of NiFi using file IO. > > Thanks > > On Thu, May 18, 2017 at 3:06 AM, prabhu Mahendran > <prabhuu161...@gmail.com&g

How to process files sequentially?

I have approximately 1000 files in local drive.I need to move that files into SQL Server accordingly one after another. Since local drive having files like file1.csv,file2.csv,..upto file1000.csv.I am sure that number of files in local drive may change dynamically. I can able to created template

Re: How to lock getfile upto putfile write into same file?

gt; > On GetFile side make sure that "ignore hidden files" is set to true. > > > > On Tue, May 16, 2017 at 1:29 AM prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > >> I have scheduled getfile processor to 0 sec to track the local folder. >> >&g

How to lock getfile upto putfile write into same file?

I have scheduled getfile processor to 0 sec to track the local folder. Issue I have faced: PutFile is appending few flowfiles into single file. Getfile has been configured with keepsourcefile as false. So getfile is fetching partial content before putfile writes into local location. How to

How to hold on fetch file upto putfile to be complete?

Am using FetchFile processor after PutFile. Since I am using Putfile with 'append' as conflict resolution strategy to merge similar lines to particular file. By overall, nearly 200 success status sent to fetchfile processor for only 5 completely appended file. This leads to fetch file with

How to use wait\notify Processor?

favorite I have running nifi instance 1.2.0. I just tried to use "Wait\Notify" Processor with following reference http://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/#why-mergecontent-dont-suffice Drag

Re: How to get ftp file according to Current date?

.com > wrote: > Hi, > > You can file a JIRA and ask for an improvement to add this feature to > GetFTP and ListFTP, or modify the processor on your side and re-generate > the NAR containing the processor. This should not be a huge change. > > 2017-04-24 9:11 GMT+02:00 prabhu M

Re: How to get ftp file according to Current date?

e and only keep the flow files with the filename you are > looking for. > > Pierre. > > 2017-04-21 13:29 GMT+02:00 prabhu Mahendran <prabhuu161...@gmail.com>: > >> I have tried that "GetFTP" processor in which downloads file from FTP >> accoding to the two a

How to get ftp file according to Current date?

I have tried that "GetFTP" processor in which downloads file from FTP accoding to the two attributes 1."FileFilterRegex" -Name of file in FTP 2."RemotePath"-Path of an FTP file. I wants to download the File from FTP Server only if it having today's date which is append with filename. *For

Re: How can datetime to month conversion failed in french language?

flows and processors allow, you may want to > specify the locale NiFi uses to execute either via environment variables or > using java command line parameters. > > Cheers > > On 12 Apr 2017 8:03 PM, "prabhu Mahendran" <prabhuu161...@gmail.com> > wrote: &

Re: MaxFileSize in timeBasedFileNamingAndTriggeringPolicy not work?

For your information,I have try that same procedure in NiFi-1.1.1 also. In that also NiFi have logback-classic-1.1.3 and logback-core-1.1.3 jar only present. In conf\logback.xml MaxFileSize and maxHistory in *TimeBasedRollingPolicy* not work. On Wed, Apr 12, 2017 at 3:58 PM, prabhu Mahendran

Re: MaxFileSize in timeBasedFileNamingAndTriggeringPolicy not work?

ixed with 1.1.7. > Unfortunately, it looks like NiFi 0.6.1 is using logback 1.1.3. > > [1] https://jira.qos.ch/browse/LOGBACK-747 > > On Tue, Apr 11, 2017 at 8:37 AM prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > >> In NiFi-0.6.1 ,i have try to reduce size of

Re: How can datetime to month conversion failed in french language?

output of the breakdown of the functions is 'Mai'.But in my local file contains 'May'. while processing 'May'(English) could be converted as 'Mai'(French). Is there is any expression language to convert French language into English? On Mon, Apr 10, 2017 at 8:02 PM, prabhu Mahendran <prabhuu

MaxFileSize in timeBasedFileNamingAndTriggeringPolicy not work?

In NiFi-0.6.1 ,i have try to reduce size of nifi-app.log to be stored in local directory. In that conf\logback.xml i have configured "MaxFileSize" to be 1MB.I think this only stores nifi-app.log should be under 1 MB Size only.But it doesn't do like that.It always store every logs.

Re: How can datetime to month conversion failed in french language?

t.4:substringBefore('-'):toDate('MMM')} *? > > Cheers > > On Mon, Apr 10, 2017 at 3:15 PM, prabhu Mahendran <prabhuu161...@gmail.com > > wrote: > >> Jeff, >> >> My actual data is in English(US). >> >> consider sample data, >> >> >> *1,12990,M

Re: How can datetime to month conversion failed in french language?

in NiFi myself. I'll let you know what I find. > What is the result of the EL you're using when you are trying it with > French? > > On Fri, Apr 7, 2017 at 1:03 AM prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > >> jeff, >> >> Thanks for your repl

Re: How can datetime to month conversion failed in french language?

(OS) it couldn't work. Can you suggest any way to solve that problem? On Fri, Apr 7, 2017 at 1:28 AM, Jeff <jtsw...@gmail.com> wrote: > What is the expression language statement that you're attempting to use? > > On Thu, Apr 6, 2017 at 3:12 AM prabhu Mahendran <prabhuu161...@

How can datetime to month conversion failed in french language?

In NiFi How JVM Check language of machine? is that take any default language like English(US) else System DateTime Selected language? I face issue while converting datetime format into Month using expression language with NiFi package installed with French OS. But it worked in English(US)

Is this possible to store the conf/archive directory in HDFS or FTP?

i have try to store backup operations for nifi in HDFS or FTP Directory. If flow has been any changes then it could be stored in "%NIFI_HOME%/conf/archive" and "%NIFI_HOME%/conf/flow.xml.gz" Now i try to store that changes in ftp directory. If any configuration has changes it will also stored

Re: Cannot get a connection, pool error Timeout waiting for idle object in PutSQL?

ax Connections as high as is prudent (at least > > the number of concurrent tasks using the controller service), and > > perhaps extending to Max Wait Time to 1 second or more, depending on > > how long you are willing for a task to wait for a connection to be > > ret

Re: How to access Controller Service created in UI into Root processors in NiFi-1.1.1?

No i am not running secure NiFi instance.Only viewed the url (http://:/nifi) in windows 8.1 On Wed, Mar 22, 2017 at 8:43 PM, Bryan Bende <bbe...@gmail.com> wrote: > Are you running a secure NiFi instance with certificates over https? > > On Wed, Mar 22, 2017 at 11:02 AM, p

Re: How to access Controller Service created in UI into Root processors in NiFi-1.1.1?

.apache.org/jira/browse/NIFI-3167 > > On Wed, Mar 22, 2017 at 2:16 AM, prabhu Mahendran <prabhuu161...@gmail.com > > wrote: > >> I haven't see the access policies in "Reporting Tasks".Look at below >> image. >> >> ​ >> ​But in your referral

Re: How to access Controller Service created in UI into Root processors in NiFi-1.1.1?

t 12:08 AM, Bryan Bende <bbe...@gmail.com> wrote: > Hello, > > I responded on stackoverflow: > > https://stackoverflow.com/questions/42853055/how-to- > access-controller-service-created-in-ui-into-root-processors-in-nifi-1-1 > > Thanks, > > Bryan > >

Re: How to specify priority attributes for seperate flowfiles?

rityAttributePrioritizer > --> > > rest of your flow > > > > > > Once again, assuming the file name is file_2017-03-03T010101.csv, the > > expression language would be something like: > > > > ${filename:toDate("'file_'-MM-dd'T'HHmmss'.csv'", "UT

Re: How to specify priority attributes for seperate flowfiles?

equesting a way to allow a DFM > to fine tune how GetFile build it's queues and control how to prioritise > the consumption of files. > > Would that be what you are looking after? > > Cheers > > > On 3 Mar 2017 15:55, "prabhu Mahendran" <prabhuu161...@gmail.com>

Re: How to specify priority attributes for seperate flowfiles?

e filename that indicates which file comes first? In > order for NiFi to prioritize these files, there has to be something > that tells NiFi what the priority is. > > On Wed, Mar 1, 2017 at 11:56 PM, prabhu Mahendran > <prabhuu161...@gmail.com> wrote: > > As you sugges

How to specify priority attributes for seperate flowfiles?

I need to use PrioritizeAttributePrioritizer in NiFi. i have observed that prioritizers in below reference. https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings if i receive 10 flowfiles then i need to set the priority value for every flow file to be unique. After that specify

How to find average of two lines in NiFi?

i need to find average of two values in seperate lines. My Csv file loo like this., Name,ID,Marks Mahi,1,90 Mahi,1,90 Andy,2,100 Andy,2,100 Now i need to store that average of 2 marks in database. "Average" column should add two marks and divide with 2 and store that result in SQL query

Re: Split csv file into multipart.

465 > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Feb 20, 2017, at 6:51 AM, prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > &

Re: How to avoid this splitting of single line as multi lines in SplitText?

ely, you could write a custom processor to > specifically handle the incoming data. > > Lee > > On Feb 14, 2017, at 11:01 PM, prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > > I have CSV file which contains following line. > > No,NAme,ID,Description > 1

Is this possible to increase speed of processors in NiFi?

Since each processor have concurrent tasks if i have set the concurrent tasks for processors then it boosts processing speed of processors.But it affects the System performance such as 100% Disk Usage,100% Memory Usage..etc Is there is any other way to speed up processors without use concurrent

Flowfile handling in C# is possible or not?

Since processors code written as java which handles flow file read and write with relationship transfer. Is it possible for flow file handling in .net application? Many thanks, prabhu

Re: How to send the success status to GetFile processor?

n, it does > allow for your use case to happen indirectly by monitoring a predefined > directory. So, one PG finishes and produces a file that is being put into a > directory monitored by another PG’s GetFile. > > Am I missing something? > > Cheers > Oleg > > On Feb 2

How to send the success status to GetFile processor?

Consider the below scenario: ProcessGroupA->ProcessGroupB Since my ProcessgroupA ends with ExecuteProcess processor that runs console application and save result into a directory. In ProcessGroupB, I will process each file in the saved directory using GetFile processor. Once, ProcessGroupA

Re: Convert xls to csv in Nifi.

nks, > Jeremy Dyer > > On Fri, Jan 27, 2017 at 5:50 AM, prabhu Mahendran <prabhuu161...@gmail.com > > wrote: > >> Can I expect the HSSF implementation to the existing PR? >> >> On Fri, Jan 27, 2017 at 10:06 AM, prabhu Mahendran < >> prabhuu161...@gmail.com&

Re: Data extraction for 100 columns is possible in NiFi?

at come with NiFi. Perhaps by using Extract Text to extract the header >> > columns, and then using ReplaceText >> > and perhaps a few other processors. Going down this route though is >> likely >> > to be incredibly inefficient, though, >> > and hard to

Data extraction for 100 columns is possible in NiFi?

I have a CSV data with 100 columns like below.., ,A,B,C,D,E,F,..[upto 100 Header columns] Date,A1,B1,C1,D1,E1,F1..[upto 100 Header columns] 30/01/2017 23:23:22,Majestic,32,2100.12[upto 100 data columns] In data having first 2 header lines and 3 rd line is data in which i

Re: How can i compare hours in which data having with current datetime hours?

ate/time minus 72 hours. > > Each line of text that matches will be routed to 'older.than.72.hours'. > Note, the outgoing FlowFiles may contain several lines of text > each, though. It does not split the output into individual lines. You can > do so if necessary via SplitText. &

Re: Convert xls to csv in Nifi.

Can I expect the HSSF implementation to the existing PR? On Fri, Jan 27, 2017 at 10:06 AM, prabhu Mahendran <prabhuu161...@gmail.com> wrote: > Jeremy,Thanks for your information. > > i think you will make effort to HSSF implementation in PR for convert (XLS > into CSV). Is

Re: Past month data insertion is possible

date is less than > 1483254000 (midnight of first day of current month), it is the previous > month (for my timezone).Thanks, > Lee > > > On Thu, Jan 26, 2017 at 10:42 PM, prabhu Mahendran < > prabhuu161...@gmail.com> wrote: > >> Hi Andy, >> >> i have alread

Re: Past month data insertion is possible

@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jan 26, 2017, at 8:40 PM, prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > > Hi All, > > I have data in which i need to compare month of d

Past month data insertion is possible

Hi All, I have data in which i need to compare month of data if it is previous month then it should be insert otherwise not. *Example:* *23.12.2016 12:02:23,Koji,24* 22.01.2016 01:21:22,Mahi,24 Now i need to get first column of data (23.12.2016 12:02:23) and then get month (12) on it.

Re: Convert xls to csv in Nifi.

ert .xlxs documents to > csv. The processor uses Apache POI XSSF implementation which only supports > xlxs while HSSF would be pre 2007 excel files. I think to your point I > should probably make an effort to add the HSSF implementation to the > existing PR. > > - Jeremy > > On Wed

Convert xls to csv in Nifi.

Hi All, i have look into below JIRA for conversion of my excel documents into csv file. https://issues.apache.org/jira/browse/NIFI-2613 i have apply patches in GitHub Pull request.And then i can able to convert my .xlxs documents into csv file. But while give .xls documents it can't converted

How can i compare hours in which data having with current datetime hours?

My input contains dateTime with hours like below., *No,DateTime,Name,ID* *1,23.01.2016 09:02:21,Mega,201* *2,03.01.2016 10:02:23,Hema,202* Now i need to get 2nd Column ["02.01.2016 10:02:23"] and then compare it with current dateTime with milliseconds. After that if 2nd Column is in between

Re: How to download HTTP Content with multiple years in GetHTTP Processor or any processor?

te URL, e.g: > http://example.com/?year=${name-of-loop-counter}. > > Thanks, > Koji > > On Mon, Jan 23, 2017 at 3:28 PM, prabhu Mahendran > <prabhuu161...@gmail.com> wrote: > > I have HTTP_URL which having file in year basis(2000-2010). > > >

How to download HTTP Content with multiple years in GetHTTP Processor or any processor?

I have HTTP_URL which having file in year basis(2000-2010). In GetHTTP Processor(NiFi-0.6.1), i have given below to url to download data for particular year like below., *http:\\myurl.com \year=2000* It can download the file which i need in particular year 2000. But now my

Re: How backup works when flow.xml size more than max storage?

And > after this point, archive mechanism stop working (Trying to create new > archive, but keep getting exception: no space left on device). > > In either case above, once flow.xml has grown to that size, some human > intervention would be needed. > Do those simulation look re

Re: How backup works when flow.xml size more than max storage?

/browse/NIFI-3373 > > Thanks, > Koji > > On Thu, Jan 19, 2017 at 2:21 PM, prabhu Mahendran > <prabhuu161...@gmail.com> wrote: > > Hi Koji, > > > > > > Thanks for your reply, > > > > Yes. Solution B may meet as I required. Currently if

Re: How backup works when flow.xml size more than max storage?

gt; archived even the flow.xml.gz exceeds archive storage size, since it > was able to be written to disk, which means the physical disk had > enough space. > > How do you think? > > Thanks! > Koji > > On Wed, Jan 18, 2017 at 3:27 PM, prabhu Mahendran > <p

How backup works when flow.xml size more than max storage?

i have check below properties used for the backup operations in Nifi-1.0.0 with respect to JIRA. https://issues.apache.org/jira/browse/NIFI-2145 *nifi.flow.configuration.archive.max.time=1 hours* *nifi.flow.configuration.archive.max.storage=1 MB* Since we have two backup operations first one is

Re: Delimiter splitting in ExtractText possible?

> > ultimately going. One caveat though is that ConvertCSVtoAvro seems to > only > > work with properly quoted and escaped CSV that conforms to RFC 4180. > > > > I'm just getting started with NiFi myself so not an expert or anything > but I > > hope that helps. >

Delimiter splitting in ExtractText possible?

Hi All, I have CSV unstructured data with comma as delimiter which contains 100 rows. Is it possible to extract the data's in csv file using comma as seperator in nifi processors. *See my Sample data 3 from 100 rows.* *No,Name,Age,PAN,City* *1,Siva,22,91230,Londan,* *2,,23,91231,UK*

Re: How to increase the processing speed of the ExtractText and ReplaceText Processor?

ould be true bottleneck; Index the DB well and use > many threads) > > > If my assumptions are incorrect, please let me know. > > Thanks, > Lee > > On Thu, Oct 20, 2016 at 1:43 AM, Kevin Verhoeven < > kevin.verhoe...@ds-iq.com> wrote: > >> I’m not clear on ho

How to increase the processing speed of the ExtractText and ReplaceText Processor?

Hi All, I have tried to perform the below operation. dat file(input)-->JSON-->SQL-->SQLServer GetFile-->SplitText-->SplitText-->ExtractText-->ReplaceText-->ConvertJsonToSQL-->PutSQL. My Input File(.dat)-->3,00,000 rows. *Objective:* Move the data from '.dat' file into SQLServer. I can able

Regarding ConsumeIMAP Processor.

Hi, I am new to the NIFI. I have just use Consume IMAP Processor to retrieve attachement from mail Server. If i use it then i can able to download attachement but that document having MIME type information with addition of EMail Data like below screenshot. I need to extract the exact data only

Re: Large dataset on hbase

this happens? Here there are no errors in both web interface(UI) and logs/nifi-app.log file. Thanks, Prabhu Mahendran On 12-Apr-2016 8:20 pm, "Bryan Bende" <bbe...@gmail.com> wrote: > > Is the output of your Pig script a single file that contains all the JSON > documents

Large dataset on hbase

correctly in smaller dataset. Can anyone please help me to solve this? Thanks in advance. Thanks & Regards, Prabhu Mahendran

Re: Sqoop Support in NIFI

. > > Simon > > On 30 Mar 2016, at 07:06, prabhu Mahendran <prabhuu161...@gmail.com> > wrote: > > For Below reasons i have choose Sqoop in NIFI Processor is the best method > to move data in Table Structure. > > If once move the Table from oracle or sql server

Re: Sqoop Support in NIFI

f the functionality you’ll find in Sqoop is in NiFi. > Unless you are looking to move terabytes at a time, then NiFi should be > able to handle most of what you would use sqoop for, so it would be very > interesting to hear more detail on your use case, and why you needed sqoop > on top of N

Re: Sqoop Support in NIFI

can > ‘drive’ NiFi externally (using the REST api) - if Sqoop can consume it. > Regards > Conrad > > > From: prabhu Mahendran <prabhuu161...@gmail.com> > Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> > Date: Tuesday, 29 March 2016 at 07:55 > To:

Re: Sqoop Support in NIFI

Hi Conrad, Thanks for Quick Response. Yeah.Combination of Execute SQL and Put HDFS works well instead of Sqoop.But is there any possible to use Sqoop(client) to do like this? Prabhu Mahendran On Tue, Mar 29, 2016 at 12:04 PM, Conrad Crampton < conrad.cramp...@secdata.com> wrote: >

Sqoop Support in NIFI

Hi, I am new to nifi. I have to know that "Is there is any Support for Sqoop with help of NIFI Processors?." And in which way to done the following case with help of Sqoop. Move data from oracle,SqlServer,MySql into HDFS and vice versa. Thanks, Prabhu Mahendran

StoreInKiteDataset Processor for Hive

, is there any Hive dependency required for storing it ? Best, Prabhu Mahendran