Hi all,
I think the intent of the syntax / parser patch from Gavin and Jeff was to get
consensus from PG on the syntax prior to proceeding with the next chunk of work.
The next chunk of work is now well underway - with support for "ALTER TABLE"
and partitioning, along with fast inserts into the
Cool! It's been a while since we've done the same kind of thing :-)
- Luke
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of
> Heikki Linnakangas
> Sent: Saturday, February 23, 2008 5:30 PM
> To: pgsql-patches@postgresql.org
> Subject: [PATCHES] C
Nice!
- Luke
On 1/30/08 9:22 AM, "Gregory Stark" <[EMAIL PROTECTED]> wrote:
>
> Here's the WIP patch for doing prereading when doing bitmap index scans.
>
> I was performance testing it as I was developing it here:
>
> http://archives.postgresql.org/pgsql-hackers/2007-12/msg00395.php
>
> No
-
From: Simon Riggs [mailto:[EMAIL PROTECTED]
Sent: Sunday, October 28, 2007 04:48 PM Eastern Standard Time
To: CK.Tan
Cc: Luke Lonergan; Kenneth Marshall; pgsql-patches@postgresql.org; [EMAIL
PROTECTED]; [EMAIL PROTECTED]
Subject:Re: [PATCHES] updated hash functions for pos
We just applied this and saw a 5 percent speedup on a hash aggregation query
with four colums in a 'group by' clause run against a single TPC-H table
(lineitem).
CK - can you post the query?
- Luke
Msg is shrt cuz m on ma treo
-Original Message-
From: Simon Riggs [mailto:[EMAIL PRO
Hi All,
On 5/31/07 12:40 AM, "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote:
> BTW, we've been talking about the "L2 cache effect" but we don't really
> know for sure if the effect has anything to do with the L2 cache. But
> whatever it is, it's real.
The mailing list archives contain the ample
Hi Heikki,
On 5/17/07 10:28 AM, "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote:
> is also visible on larger scans that don't fit in cache with bigger I/O
> hardware, and this patch would increase the max. I/O throughput that we
> can handle on such hardware. I don't have such hardware available,
So, if I understand this correctly, we're calling Alloc and ContextAlloc 10
times for every row being summed?
There are approx 10M rows and the profile snippet below shows 100M calls to
each of those.
- Luke
On 11/24/06 4:46 PM, "Mark Kirkwood" <[EMAIL PROTECTED]> wrote:
> time seconds se
Bruce,
We have a bitmap index AM in Bizgres (on PG 8.1.3) that is functional and
achieves very worthwhile (10x+) performance and space objectives.
It is a large patch, and introduces the access method along with modifying
the appropriate executor nodes. The latter part was necessary because of
t
Martijn,
On 5/11/06 12:17 AM, "Martijn van Oosterhout" wrote:
> Did you test it? There are some cases where this might still leave some
> noticable overhead (high loop count). I'm just not sure if they occur
> all that often in practice...
I've sent it to our team for testing, let's see if we g
Nice one Martijn - we have immediate need for this, as one of our sizeable
queries under experimentation took 3 hours without EXPLAIN ANALYZE, then
over 20 hours with it...
- Luke
On 5/9/06 2:38 PM, "Martijn van Oosterhout" wrote:
> On Tue, May 09, 2006 at 05:16:57PM -0400, Rocco Altier wrote
Tom,
On 5/8/06 11:46 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> I made a table of 16M rows with an
> index over a random-data integer column. With a thoroughly disordered
> index (built on-the-fly as the random data was inserted), the time to
> VACUUM after deleting a small number of rows was 6
t; <[EMAIL PROTECTED]> wrote:
"Luke Lonergan" <[EMAIL PROTECTED]> writes:
> So you know, we=B9ve done some more work on the external sort to remove the
> =B3tape=B2 abstraction from the code, which makes a significant improvement.
Improvement where? That code's do
Title: Re: [PATCHES] WIP: further sorting speedup
Cool!
We’ll test this sometime soon and get back to you. We’re kind of jammed this week, hopefully we’ll get some time.
So you know, we’ve done some more work on the external sort to remove the “tape” abstraction from the code, which makes a s
Thanks for this Jim! We'll test it and report results.
- Luke
From: [EMAIL PROTECTED] on behalf of Jim C. Nasby
Sent: Sat 2/11/2006 9:26 PM
To: pgsql-patches@postgresql.org
Subject: [PATCHES] OS X shared memory documentation
I finally figured out what the is
Bruce,
On 12/27/05 10:20 AM, "Bruce Momjian" wrote:
> OK, original patch applied to HEAD and smaller version to 8.1.X, and
> regression test added, now attached.
Great, good catch.
Have you tested performance, before and after?
The only good way to test performance is using a fast enough I/O
Tom,
On 8/10/05 8:37 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> Luke, I dislike whacking people upside the head, but this discussion
> seems to presume that raw speed on Intel platforms is the only thing
> that matters. We have a few other concerns. Portability, readability,
> maintainability,
Alvaro,
On 8/10/05 9:46 AM, "Alvaro Herrera" <[EMAIL PROTECTED]> wrote:
> AFAIR he never claimed otherwise ... his point was that to gain that
> additional speedup, the code has to be made considerable "worse" (in
> maintenability terms.) Have you (or Alon) tried to port the rest of the
> speed
Simon,
> That part of the code was specifically written to take advantage of
> processing pipelines in the hardware, not because the actual theoretical
> algorithm for that approach was itself faster.
Yup, good point.
> Nobody's said what compiler/hardware they have been using, so since both
>
Tom,
> As best I can tell, my version of CopyReadAttributes is significantly
> quicker than Alon's, approximately balancing out the fact that my
> version of CopyReadLine is slower. I did the latter first, and would
> now be tempted to rewrite it in the same style as CopyReadAttributes,
> ie one
Andrew,
> Arguably this might exaggerate the effect quite significantly. Users
> will want to know the real time effect on a complete COPY. Depending on
> how much the pasing is in the total time your 20% improvement in parsing
> might only be a small fraction of 20% improvement in COPY.
Arguabl
Tom,
On 8/6/05 9:08 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> "Luke Lonergan" <[EMAIL PROTECTED]> writes:
>>> I had some difficulty in generating test cases that weren't largely
>>> I/O-bound, but AFAICT the patch as applied is about t
Tom,
My direct e-mails to you are apparently blocked, so I'll send this to the
list.
I've attached the case we use for load performance testing, with the data
generator modified to produce a single row version of the dataset.
I do believe that you/we will need to invert the processing loop to ge
Tom,
The previous timings were for a table with 15 columns of mixed type. We
also test with 1 column to make the parsing overhead more apparent. In the
case of 1 text column with 145MB of input data:
Your patch:
Time: 6612.599 ms
Alon's patch:
Time: 6119.244 ms
Alon's patch is 7.5% faste
Tom,
Thanks for finding the bugs and reworking things.
> I had some difficulty in generating test cases that weren't largely
> I/O-bound, but AFAICT the patch as applied is about the same speed
> as what you submitted.
You achieve the important objective of knocking the parsing stage down a
lot,
Joshua,
On 7/21/05 7:53 PM, "Joshua D. Drake" <[EMAIL PROTECTED]> wrote:
> Well I know that isn't true at least not with ANY of the Dells my
> customers have purchased in the last 18 months. They are still really,
> really slow.
That's too bad, can you cite some model numbers? SCSI?
> I have gr
Joshua,
On 7/21/05 5:08 PM, "Joshua D. Drake" <[EMAIL PROTECTED]> wrote:
> O.k. this strikes me as interesting, now we know that Compaq and Dell
> are borked for Linux. Is there a name brand server (read Enterprise)
> that actually does provide reasonable performance?
I think late model Dell (po
Cool!
At what rate does your disk setup write sequential data, e.g.:
time dd if=/dev/zero of=bigfile bs=8k count=50
(sized for 2x RAM on a system with 2GB)
BTW - the Compaq smartarray controllers are pretty broken on Linux from a
performance standpoint in our experience. We've had disastr
Good points on all, another element in the performance expectations is the
ratio of CPU speed to I/O subsystem speed, as Alon had hinted earlier.
This patch substantially (500%) improves the efficiency of parsing in the
COPY path, which, on a 3GHz P4 desktop with a commodity disk drive
represents
Bruce,
On 7/15/05 9:59 PM, "Bruce Momjian" wrote:
> Actually, mine returns ')' too for the last command. I didn't copy
> that into the email. How about the top tests? Notice I get an error on
> the first one without the backslash. Are you OK escaping '(' but not
> ')'? That might be a solu
Bruce,
> I found that parentheses in gawk regular expressions require backslashes
> so they are not treated as regex groupings:
>
> $ echo '('|awk '$0 ~ /(/ {print $0}'
> awk: cmd. line:1: fatal: Unmatched ( or \(: /(/
> $ echo '('|awk '$0 ~ /\(/ {print $0}'
> (
> Now, it seems closing parenthe
> If the StringInfo API were extended to use lengths...
Er, like appendBinaryStringInfo() perhaps? ;-)
Looks like what we need is there, it only lacks the offset adjustment of our
bytebuffer.
We should redo the copy.c with appendBinaryStringInfo() I think.
- Luke
---
Alon,
> Hmm, now that I look back at them I can't remember why I thought it is
> slower. Certainly using appendStringInfoCharMacro for every char is very
> slow, but I could probably use appendStringInfoString and it should be as
> fast as using the bytebuffer, they both do a straight forward mem
Alon,
> If I remember more relevant things I'll post them here.
>
> Hope that helps, please ask more if there are unclear things!
Good detail, very useful!
- Luke
---(end of broadcast)---
TIP 7: don't forget to increase your free space map sett
Andrew,
> What I would like to have is a high level description of
> . how the new text mode code differs from the old text mode code, and
> . which part of the change is responsible for how much performance gain.
>
> Maybe I have missed that in previous discussion, but this change is
> sufficien
We're porting this to 8.1, we should be able to get it done within the next
couple of days. The changes needed are to the way that the unmodified CSV
and binary code paths are preserved alongside the changed "delimited" text
COPY.
- Luke
---(end of broadcast)---
Note that this patch is against 8.0.2.
- Luke
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
FYI - we're prepping a proposal for just that. Stay tuned for Alon's
proposal.
Luke
On 6/1/05 11:43 PM, "Neil Conway" <[EMAIL PROTECTED]> wrote:
> On Thu, 2005-06-02 at 00:53 -0400, Alon Goldshuv wrote:
>> My thoughts were -- see how the responses are, and if people think
>> that this is a bet
Andrew,
> OK ... that seems fair enough. The next question is where the data being
> loaded comes from? pg_dump? How does load speed compare with using COPY's
> binary mode?
Oddly, our tests in the past have shown that binary is actually slower.
Luke
---(end of broadca
Andrew,
> I will be the first to admit that there are probably some very good
> possibilities for optimisation of this code. My impression though has been
> that in almost all cases it's fast enough anyway. I know that on some very
> modest hardware I have managed to load a 6m row TPC line-items t
40 matches
Mail list logo
