Re: Missing records from HDFS

2013-11-22 Thread ZORAIDA HIDALGO SANCHEZ
noviembre de 2013 07:31 Para: user@hadoop.apache.orgmailto:user@hadoop.apache.org user@hadoop.apache.orgmailto:user@hadoop.apache.org Asunto: Re: Missing records from HDFS what's your hadoop version? and which InputFormat are you used? these files under one directory or there are lots

Re: Missing records from HDFS

2013-11-22 Thread ZORAIDA HIDALGO SANCHEZ
@hadoop.apache.orgmailto:user@hadoop.apache.org Asunto: Re: Missing records from HDFS Thanks for your response Azuryy. My hadoop version: 2.0.0-cdh4.3.0 InputFormat: a custom class that extends from FileInputFormat(csv input format) These fiels are under the same directory, different files. My input path is configured

Re: Missing records from HDFS

2013-11-22 Thread Azuryy Yu
are processed. Files are of 70,5MB. Thanks, Zoraida.- De: zoraida zora...@tid.es Fecha: viernes, 22 de noviembre de 2013 08:59 Para: user@hadoop.apache.org user@hadoop.apache.org Asunto: Re: Missing records from HDFS Thanks for your response Azuryy. My hadoop version: 2.0.0

Re: Missing records from HDFS

2013-11-22 Thread ZORAIDA HIDALGO SANCHEZ
@hadoop.apache.org user@hadoop.apache.orgmailto:user@hadoop.apache.org Fecha: viernes, 22 de noviembre de 2013 12:19 Para: user@hadoop.apache.orgmailto:user@hadoop.apache.org user@hadoop.apache.orgmailto:user@hadoop.apache.org Asunto: Re: Missing records from HDFS I do think this is because of your

Re: Missing records from HDFS

2013-11-22 Thread Azuryy Yu
Asunto: Re: Missing records from HDFS I do think this is because of your RecorderReader, can you paste your code here? and give a piece of data example. please use pastebin if you want. On Fri, Nov 22, 2013 at 7:16 PM, ZORAIDA HIDALGO SANCHEZ zora...@tid.eswrote: One more thing, if we split

Missing records from HDFS

2013-11-20 Thread ZORAIDA HIDALGO SANCHEZ
Hi all, my job is not reading all the input records. In the input directory I have a set of files containing a total of 600 records but only 5997000 are processed. The Map Input Records counter says 5997000. I have tried downloading the files with a getmerge to check how many records would

Re: Missing records from HDFS

2013-11-20 Thread Azuryy Yu
what's your hadoop version? and which InputFormat are you used? these files under one directory or there are lots of subdirectory? how ddi you configure input path in your main? On Thu, Nov 21, 2013 at 12:25 AM, ZORAIDA HIDALGO SANCHEZ zora...@tid.eswrote: Hi all, my job is not reading