Thanks. Experimentally, I have found that changing the buffers sizes has no
effect, so that makes sense.
John
From: Arpit Agarwal [mailto:aagar...@hortonworks.com]
Sent: Tuesday, January 28, 2014 12:35 AM
To: user@hadoop.apache.org
Subject: Re: HDFS buffer sizes
Looks like
gt; any benefit to setting a larger bufferSize in FileSystem.create() and
> FileSystem.append()?
>
>
>
> *From:* Arpit Agarwal [mailto:aagar...@hortonworks.com]
> *Sent:* Friday, January 24, 2014 9:35 AM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: HDFS buffer
.apache.org>
Subject: Re: HDFS buffer sizes
HDFS does not appear to use dfs.stream-buffer-size.
On Thu, Jan 23, 2014 at 6:57 AM, John Lilley
mailto:john.lil...@redpoint.net>> wrote:
What is the interaction between dfs.stream-buffer-size and
dfs.client-write-packet-size?
I see that the
ZE_DEFAULT = 4096;
>
> Are there benefits to increasing this for large reads or writes?
>
> john
>
>
>
> *From:* Arpit Agarwal [mailto:aagar...@hortonworks.com]
> *Sent:* Thursday, January 23, 2014 3:31 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: HDFS buffer si
: user@hadoop.apache.org
Subject: Re: HDFS buffer sizes
HDFS does not appear to use dfs.stream-buffer-size.
On Thu, Jan 23, 2014 at 6:57 AM, John Lilley
mailto:john.lil...@redpoint.net>> wrote:
What is the interaction between dfs.stream-buffer-size and
dfs.client-write-packet-size?
I see th
HDFS does not appear to use dfs.stream-buffer-size.
On Thu, Jan 23, 2014 at 6:57 AM, John Lilley wrote:
> What is the interaction between dfs.stream-buffer-size and
> dfs.client-write-packet-size?
>
> I see that the default for dfs.stream-buffer-size is 4K. Does anyone have
> experience using
