On 5/30/06, Bruce Momjian pgman@candle.pha.pa.us wrote:
The crux of the matter was if its possible to use fwrite
as easy string combining mechanism and the answer is no,
because it's not lightweight enough.
So your patch to src/backend/storage/file/fd.c should be discarded? OK.
Yes. It
Marko Kreen wrote:
On 5/28/06, Martijn van Oosterhout kleptog@svana.org wrote:
With -lpthread
lock.enabled 323s
lock.disabled 50s
lock.unlocked 36s
I forgot to test with -lpthread, my bad. Indeed by default
something less expensive that full locking is going on.
The crux of the
Marko Kreen wrote:
On 5/28/06, Martijn van Oosterhout kleptog@svana.org wrote:
With -lpthread
lock.enabled 323s
lock.disabled 50s
lock.unlocked 36s
I forgot to test with -lpthread, my bad. Indeed by default
something less expensive that full locking is going on.
The
On 5/28/06, Martijn van Oosterhout kleptog@svana.org wrote:
With -lpthread
lock.enabled 323s
lock.disabled 50s
lock.unlocked 36s
I forgot to test with -lpthread, my bad. Indeed by default
something less expensive that full locking is going on.
The crux of the matter is though,
On 5/27/06, Tom Lane [EMAIL PROTECTED] wrote:
I wrote:
I'm off for a little visit with oprofile...
It seems the answer is that fwrite() does have pretty significant
per-call overhead, at least on Fedora Core 4.
That may be because of the locking ritual all stdio functions
like to do, even
Tom Lane wrote:
I wrote:
I'm off for a little visit with oprofile...
It seems the answer is that fwrite() does have pretty significant
per-call overhead, at least on Fedora Core 4. The patch I did yesterday
still ended up making an fwrite() call every few characters when dealing
with bytea
Marko Kreen [EMAIL PROTECTED] writes:
If you want to use fwrite as string operator, then maybe
should replace it with fwrite_unlocked?
ISTM that in a single-threaded application such as the backend,
it should be libc's responsibility to avoid such overhead, not
ours.
On 5/27/06, Tom Lane [EMAIL PROTECTED] wrote:
Marko Kreen [EMAIL PROTECTED] writes:
If you want to use fwrite as string operator, then maybe
should replace it with fwrite_unlocked?
ISTM that in a single-threaded application such as the backend,
it should be libc's responsibility to avoid such
On Sat, May 27, 2006 at 06:36:15PM +0300, Marko Kreen wrote:
ISTM that in a single-threaded application such as the backend,
it should be libc's responsibility to avoid such overhead, not
ours.
Obviously, except glibc guys seems to be philosophically
opposed to this, so apps need to work
On 5/27/06, Martijn van Oosterhout kleptog@svana.org wrote:
Given there is no way to know if you're running single threaded or not,
I don't think glibc can take chances like that.
There's CPP symbol _REENTRANT for that and in run time,
libc can detect call to pthread_create [1].
In any case,
On Sat, May 27, 2006 at 08:23:35PM +0300, Marko Kreen wrote:
On 5/27/06, Martijn van Oosterhout kleptog@svana.org wrote:
Given there is no way to know if you're running single threaded or not,
I don't think glibc can take chances like that.
There's CPP symbol _REENTRANT for that and in run
Tom Lane wrote:
Andreas Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
Looking at CopySendData, I wonder whether any traction could be gained
by trying not to call fwrite() once per character. I'm not sure how
much per-call overhead there is in that function. We've done a lot of
work
Andreas Pflug [EMAIL PROTECTED] writes:
Here are the results, with the copy patch:
psql \copy 1.4 GB from table, binary:
8.0 8.1 8.2dev
36s 34s 36s
psql \copy 6.6 GB from table, std:
8.0 8.1 8.2dev
375s 362s290s (second:283s)
Hmph. There's something strange going
Tom Lane wrote:
Andreas Pflug [EMAIL PROTECTED] writes:
Here are the results, with the copy patch:
psql \copy 1.4 GB from table, binary:
8.0 8.1 8.2dev
36s 34s 36s
psql \copy 6.6 GB from table, std:
8.0 8.1 8.2dev
375s362s290s (second:283s)
Hmph.
I wrote:
I can't test psql binary \copy just yet, but will look at applying your
recent patch so that case can be checked.
With patch applied:
$ time psql -c \\copy t to '/home/tgl/t.out2' bytea
real3m46.057s
user0m2.724s
sys 0m36.118s
$ time psql -c \\copy t to
I wrote:
I'm off for a little visit with oprofile...
It seems the answer is that fwrite() does have pretty significant
per-call overhead, at least on Fedora Core 4. The patch I did yesterday
still ended up making an fwrite() call every few characters when dealing
with bytea text output, because
Andreas Pflug [EMAIL PROTECTED] writes:
When dumping the table with psql \copy (non-binary), the resulting file
would be 6.6GB of size, taking about 5.5 minutes. Using psql \copy WITH
BINARY (modified psql as posted to -patches), the time was cut down to
21-22 seconds (filesize 1.4GB as
Tom Lane wrote:
Andreas Pflug [EMAIL PROTECTED] writes:
When dumping the table with psql \copy (non-binary), the resulting file
would be 6.6GB of size, taking about 5.5 minutes. Using psql \copy WITH
BINARY (modified psql as posted to -patches), the time was cut down to
21-22 seconds
Andreas Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
Looking at CopySendData, I wonder whether any traction could be gained
by trying not to call fwrite() once per character. I'm not sure how
much per-call overhead there is in that function. We've done a lot of
work trying to optimize
19 matches
Mail list logo
