Re: [HACKERS] Implementing Bitmap Indexes
On Sat, Jan 29, 2005 at 01:56:12PM +0200, Victor Y. Yegorov wrote: 2) Queries, like where A = 1 or where A != 2 will require only 1 scan of the index, while where A 3 will require 2 stages: 1st create a list of values lesser then 3, 2nd --- do OR of all bitmaps for that values. For high cardinality attributes, this can take a lot of time; 3) Each bitmap is only a bitmap, so there should be an array of corresponding ctids pointers. Maybe, some more arrays (pages, don't know). For 2)nd --- there are techniques, allowing better performance for A 3 queries via increased storage space (see here for details: http://delab.csd.auth.gr/papers/ADBIS03mmnm.pdf) and increased reaction time for simple queries. I don't know, if they should be implemented, may later. Sorry if this is in the PDF but I didn't want to read 17 pages to find out... for the example where 1 = A = 4, couldn't you just do NOT (A = 3)? Granted, in this example it wouldn't matter, but it would be faster to do this if you asked for A 4. One downside is that you'd also have to consider the NULL bitmap, if the field is nullable. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Repleacement for src/port/snprintf.c
Greetings,
I would like to submit my changes to src/port/snprintf.c to
enable %n$ format placeholder replacement in snprintf() and
vsnprintf(). Additionally I implemented a trivial printf().
I also attach a diff for configure.in to include snprintf.o
in pgport but I am sure it is not the right thing to do.
Could someone give a hint on where I need to place such a
definition.
Best regards,
Nicolai Tufar
*** ./src/port/snprintf.c.orig 2005-01-20 01:27:22.0 +0200
--- ./src/port/snprintf.c 2005-01-24 03:09:31.0 +0200
***
*** 57,62
--- 57,66
typedef unsigned long ulong_long;
#endif
+ #ifndef NL_ARGMAX
+ #define NL_ARGMAX 4096
+ #endif
+
/*
**SNPRINTF, VSNPRINT -- counted versions of printf
**
***
*** 85,93
--- 89,115
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt,
va_list args);
+ int printf(const char *format, ...);
static void dopr(char *buffer, const char *format, va_list args);
int
+ printf(const char *fmt,...)
+ {
+ int len;
+ va_list args;
+ static char*buffer[4096];
+ char* p;
+
+ va_start(args, fmt);
+ len = vsnprintf((char*)buffer, (size_t)4096, fmt, args);
+ va_end(args);
+ p = (char*)buffer;
+ for(;*p;p++)
+ putchar(*p);
+ return len;
+ }
+
+ int
snprintf(char *str, size_t count, const char *fmt,...)
{
int len;
***
*** 124,129
--- 146,155
static char *output;
+ #define FMTSTR 1
+ #define FMTNUM 2
+ #define FMTFLOAT3
+ #define FMTCHAR 4
static void
dopr(char *buffer, const char *format, va_list args)
***
*** 139,145
--- 165,200
int ljust;
int len;
int zpad;
+ int i;
+ const char* format_save;
+ const char* fmtbegin;
+ int fmtpos = 1;
+ int realpos = 0;
+ int position;
+ static struct{
+ const char* fmtbegin;
+ const char* fmtend;
+ union{
+ void* value;
+ long_long numvalue;
+ double fvalue;
+ int charvalue;
+ };
+ int ljust;
+ int len;
+ int zpad;
+ int maxwidth;
+ int base;
+ int dosign;
+ chartype;
+ int precision;
+ int pointflag;
+ charfunc;
+ int realpos;
+ }fmtpar[NL_ARGMAX+1], *fmtparptr[NL_ARGMAX+1];
+
+ format_save = format;
output = buffer;
while ((ch = *format++))
{
***
*** 148,161
case '%':
ljust = len = zpad = maxwidth = 0;
longflag = longlongflag = pointflag = 0;
nextch:
ch = *format++;
switch (ch)
{
case 0:
! dostr(**end of format**, 0);
! *output = '\0';
! return;
case '-':
ljust = 1;
goto nextch;
--- 203,217
case '%':
ljust = len = zpad = maxwidth = 0;
longflag = longlongflag = pointflag = 0;
+ fmtbegin = format - 1;
+ realpos = 0;
+ position = 0;
nextch:
ch = *format++;
switch (ch)
{
case 0:
! goto performpr;
case '-':
ljust = 1;
goto nextch;
***
*** 174,180
--- 230,243
if (pointflag)
maxwidth = maxwidth *
10 + ch - '0';
else
+
Re: [HACKERS] [BUGS] Bug in create operator and/or initdb
I suspect that the right thing to do is to kill the inet type entirely, and replace it with a special case of cidr. (And possibly then to kill cidr and replace it with something that can be indexed more effectively.) Yes, which is actually what brought this to my attention. I'll be sending an rtree index implementation shortly for review/comments. For a replacement type, how important is it that it be completely compatible with the existing inet/cidr types? Is anyone actually using inet types with a non-cidr mask? I wouldn't think so, anyone I've spoken with has come up with other ways of managing that kind of info, because of, as you mentioned, it's lack of proper index methods. Kind Regards, John Hansen ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] Rtree index method for inet/cidr
Hi, Attached, an rtree implementation for inet/cidr values. It's beyond me to implement this as a patch, but if anyone has the time and energy to do so, feel free! Comments, positive and negative welcome - and appreciated :) Kind Regards, John Hansen Makefile Description: Makefile rtree_inet.c Description: rtree_inet.c rtree_inet.sql Description: rtree_inet.sql ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] Packaging begins in 4 hours ...
Any last minute show stoppers that needs a bit more time, let me know before 23:00 GMT Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Implementing Bitmap Indexes
On Sat, 29 Jan 2005 18:46:44 +, Mike Rylander [EMAIL PROTECTED] wrote : For on-disk bitmap indexes, yes. I don't see any reason this couldn't be done with GiST, perhaps even as a generalization of the index stuff in the int_array contrib module. I was thinking about playing with the core source of PostgreSQL and creating patches, GiST as I read has some limitations that I want to overcome. Speaking about my idea - I was thinking about implementing on-disk indexes, not in memory. I think having both of them would be great :-) -- **Pawel Niewiadomski**, new()foo-baz.com, http://new.foo-baz.com/ Virtual Qmail (http://v-q.foo-baz.com), qmail-patches (http://q-p.foo-baz.com) ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Packaging begins in 4 hours ...
Marc G. Fournier [EMAIL PROTECTED] writes: Any last minute show stoppers that needs a bit more time, let me know before 23:00 GMT AFAIK we're good to go. I've updated all the version files and release notes --- it's ready to tag and wrap. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Rtree index method for inet/cidr
John Hansen [EMAIL PROTECTED] writes: Attached, an rtree implementation for inet/cidr values. Er ... what about IPv6? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [BUGS] Bug in create operator and/or initdb
John Hansen [EMAIL PROTECTED] writes: I wouldn't think so, anyone I've spoken with has come up with other ways of managing that kind of info, because of, as you mentioned, it's lack of proper index methods. On the contrary I'm using it for something that isn't really what it was designed for precisely *because* of the index methods. What index access methods are you looking for that are lacking? db= explain select * from foo where foo_code '4.0.0.0/8'; QUERY PLAN -- Index Scan using foo_foo_code on foo (cost=0.00..34.56 rows=1695 width=229) Index Cond: ((foo_code '4.0.0.0/8'::cidr) AND (foo_code = '4.255.255.255'::cidr)) Filter: (foo_code '4.0.0.0/8'::cidr) (3 rows) -- greg ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [BUGS] Bug in create operator and/or initdb
On the contrary I'm using it for something that isn't really what it was designed for precisely *because* of the index methods. What index access methods are you looking for that are lacking? db= explain select * from foo where foo_code '4.0.0.0/8'; explain select * from foo where foo_code '220.244.179.214/32'; ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [BUGS] Bug in create operator and/or initdb
John Hansen [EMAIL PROTECTED] writes: On the contrary I'm using it for something that isn't really what it was designed for precisely *because* of the index methods. What index access methods are you looking for that are lacking? db= explain select * from foo where foo_code '4.0.0.0/8'; explain select * from foo where foo_code '220.244.179.214/32'; Note also that the btree optimization for depends on having a plan-time-constant righthand side; so it's useless for joins, for instance. I didn't look closely, but I'd suppose that rtree could help for searches without that constraint. Looking to the future, it might be better to base this on gist instead of rtree indexes --- gist is being worked on semi-actively, rtree isn't really being touched at all. But the immediate problem is that I don't think we can integrate this if it doesn't handle IPv6 addresses. We aren't going to want to backslide on having full IPv6 support. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [BUGS] Bug in create operator and/or initdb
Note also that the btree optimization for depends on having a plan-time-constant righthand side; so it's useless for joins, for instance. I didn't look closely, but I'd suppose that rtree could help for searches without that constraint. Indeed it can... tho rtree seems to be unable to do merge joins, so you only get an index scan on one of the joined tables. Which index is chosen has proven a bit hard to predict. Looking to the future, it might be better to base this on gist instead of rtree indexes --- gist is being worked on semi-actively, rtree isn't really being touched at all. Yea, rtree is also broken, tho I think andrew is going to attempt fixing it. But the immediate problem is that I don't think we can integrate this if it doesn't handle IPv6 addresses. We aren't going to want to backslide on having full IPv6 support. Right,. i'll be adding ipv6 support... ... John ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Packaging begins in 4 hours ...
On Sunday 30 January 2005 17:01, you wrote:
Marc G. Fournier [EMAIL PROTECTED] writes:
Any last minute show stoppers that needs a bit more time, let me know
before 23:00 GMT
AFAIK we're good to go. I've updated all the version files and release
notes --- it's ready to tag and wrap.
I thought I saw something this weekend about a new zic/timezone database being
released though having trouble finding out the details now... is this
something that should be backpatched into the 8.0.x series? Not that this
would be a show stopper, but it should be a simple update if anyone has
gumption to do it now.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Packaging begins in 4 hours ...
Robert Treat [EMAIL PROTECTED] writes: I thought I saw something this weekend about a new zic/timezone database being released though having trouble finding out the details now... is this something that should be backpatched into the 8.0.x series? Too late for 8.0.1. It's not like there won't be any more 8.0 releases ;-) regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
