it's on s3. and it always happens. Ananth T Sarathy
On Wed, Aug 19, 2009 at 4:35 PM, Raghu Angadi <rang...@yahoo-inc.com> wrote: > Ananth T. Sarathy wrote: > >> Also, I just want to clear... the delay seems to at the intial >> >> (read = in.read(buf)) >> > > It the file on HDFS (over S3) or S3? > > Does it always happen? > > Raghu. > > > after the first time into the loop it flies... >> >> Ananth T Sarathy >> >> >> On Wed, Aug 19, 2009 at 1:58 PM, Raghu Angadi <rang...@yahoo-inc.com> >> wrote: >> >> Edward Capriolo wrote: >>> >>> On Wed, Aug 19, 2009 at 11:11 AM, Edward Capriolo < >>>> edlinuxg...@gmail.com >>>> >>>>> wrote: >>>>>> >>>>> It would be as fast as underlying filesystem goes. >>>>> >>>>>> I would not agree with that statement. There is overhead. >>>>>>> >>>>>> You might be misinterpreting my comment. There is of course some over >>> head >>> (at the least the procedure calls).. depending on you underlying >>> filesystem, >>> there could be extra buffer copies and CRC overhead. But none of that >>> explains transfer as slow as 1 MBps (if my interpretation of of results >>> is >>> correct). >>> >>> Raghu. >>> >>> >>>
