Re: [GENERAL] Fedora 13 and yum.pgsqlrpms.org

2010-05-31 Thread Devrim GÜNDÜZ
On Sat, 2010-05-29 at 13:18 -0400, Jerry LeVan wrote:
 
 Will the repository be updated for Fedora 13 in the
 near future :)
 
 I had to disable the repo because yumex croaks when it
 cannot find the repository.

FWIW, I pushed Fedora-13 packages to my repository:

http://yum.pgrpms.org/news-fedora13-packages-released.php
http://yum.pgrpms.org/8.4/fedora/fedora-13-i386/repoview/
http://yum.pgrpms.org/8.4/fedora/fedora-13-x86_64/repoview/

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz


signature.asc
Description: This is a digitally signed message part


[GENERAL] Will you please help with my research on OSS work...

2010-05-31 Thread Geoff Breach

Hi All,

 I'm Geoff, a long time postgresql user and been a reasonably regular
participant on #gentoo and #postgresql on freenode for several years
now, helping out new folks where I can. I'm a student (phd) at the
University of Technology, Sydney in Australia. (I've also been a
subscriber to this list for at least thirty minutes now! :-) )

 I'm working on a thesis that examines the nature of OSS work as
represented by the people who actually DO the work (that's you!). I
hope that my approach will be somewhat unique because I'm a career geek
who can hold an intelligent technical conversation, not just some random
academic with a white coat and a clipboard!

 If you do any form of work on Open Source Software (and its variants,
(Free, Open, Free Libre, you name it) then I'd like to talk to you, please.

 I'm interested in talking to *anyone* who contributes by writing code,
testing code, using code, providing support online (here, on freenode,
elsewhere), in person, advocacy, moderating, organizing, advertising,
shooting the breeze or ANYTHING that somehow contributes to or benefits
OSS. No matter how small your contribution to OSS, your perspective is
important to me.

 In a nutshell, I'd like to chat online via some form of instant messaging
(irc is great, just about anything else is fine) for about an hour or two.
A few months later I'd like to catch up again to ask you to check my
interpretations and make corrections. I don't need to know your real name,
and I don't want demographic details.

 More details of what I need from you are at 
http://geoffbreach.com/consent


 Papers that I've already written about OSS are at 
http://geoffbreach.com/writing


 The backstory on some of that writing is at 
http://www.geoffbreach.com/research


 There's also a (nearly) infinite list of ways to contact me on
the 'consent' page I've listed above. I'm 'geoffb' on
irc.freenode.net, ge...@breach.com.au on email and MSN and
geoff.bre...@gmail.com on Jabber/GTalk. Please feel free to ask
me *anything* you need to know before (or after) you go ahead and
sign up. (I'm on freenode via a caching proxy so I can access
seamlessly from anywhere. If I don't respond immediately to your
PM it's because I'm /away. I'll be back soon!)

 Thanks (very much) in advance,

Geoff
(geoffb @ freenode)

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] vulnerability of COPY command

2010-05-31 Thread Wappler, Robert
On 2010-05-30, Martin Gainty wrote:
 
 i have mixed feelings about parameterised statements.
 
 On the one hand a parameterised statement would be more
 difficult for a Wireshark criminal to insert their own c**p
 into a database because they would have to know the schema
 a-priori for example they would have to know the names of the
 database, table and the datatypes of these 2 statements
 Insert DateField1 INTO DateTable VALUES(1) would gack as 1 is
 not a valid date
 Insert NumericField1 INTO NumericTable VALUES('A') would gack
 as well as A is not a valid number
 
 But thats only one line of defence..Struts..JSF and other
 Frameworks have field validators so a good design would
 involve 2 tiered defence
 1)The web folks would put some kind of JS function to
 disallow client know from enetering a invalid date

Sorry, but that isn't a security measure at all, you cannot even ensure
that it is executed. And if it is executed, it is executed on the client
side. It is at best useful for some usability niceties. It catches some
malformed input from non-malicious users, but any criminal just ignores
your JS and builds the request he wants.

 2)The back-end folks (me) would validate Database params in
 the servlet or the DTO as it makes its way to the VO and
 before any of the offending values get into the database
 

That is what PreparedStatement.set*(...) (and the non-java pendants)
does by properly escaping input and transforming it to a representation
which exactly represents the datatype and not any kind of executable
statement.

 The net effect is wireshark criminals are succeeding in
 making everyone's lives a living hell..the least we can do is
 take their pitchfork away!
 

That is not a problem with (prepared) statements at all, that is a
matter of protecting the wire. If you don't want your transmitted data
to be manipulated, you should use at least encryption or validated
signatures.

-- 
Robert...

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Alonso García , Bruno Elier
Hello,
I am migrating a client/server application from Debian Sarge to Debian 5.0 and 
I am finding problems with the client application. The facts are the following:
-The client application is an interface to a Postgresql DB so it uses libpq.
-The client application compiles properly in both Debian Sarge and Debian 5.0.
-The client application employs Libpq asynchronous command processing.
-The pseudo.code for the queries is the following:
PQconnectdb
PQsendQuery
PQflush
loop{
PQconsumeInput
PQisBusy
if not busy PQgetResult and leave
}
-The new DB server is postgresql 8.3.
-The old DB server is postgresql 7.4
-I am using the same SQL script to create the DB.
And the problems I am finding are the following:
-Queries from the client to the new DB server take a lot of time. 
-Queries from the client to the old DB server are fast.
-The same query takes 150 secs in one case an 1 sec in the other case.

¿Any ideas regarding the origin of this strange behaviour?¿Could it be the 
configuration of the new DB?
Thanks in advance.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Giles Lean

=?iso-8859-1?Q?Alonso_Garc=EDa_=2C_Bruno_Elier?= bealo...@indra.es wrote:

 And the problems I am finding are the following:
 -Queries from the client to the new DB server take a lot of time.
 -Queries from the client to the old DB server are fast.
 -The same query takes 150 secs in one case an 1 sec in the other case.

With that analysis, I'd be betting against it being a client problem.
(If you wanted, you might confirm that by pointing an old client at
the new server.)

I'd look into how the data was loaded into the new server and how
the database is configured: number of buffers, indexes, and whether
analyze has been run or not.

It would be strange indeed (possible, but very strange) to find
such a slowdown between 7.x and 8.x when the team is preparing
to push 9.0 out the door.  Surely it would have been known before;
therefore it's a practical certatinty that there is something
different about the configuration of your two servers.

Giles

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] 110,000,000 rows

2010-05-31 Thread Jasen Betts
On 2010-05-26, John Gage jsmg...@numericable.fr wrote:
 Please forgive this intrusion, and please ignore it, but how many  
 applications out there have 110,000,000 row tables?  I recently  
 multiplied 85,000 by 1,400 and said now way Jose.

census data would be one.
USA phone whitepages.
transaction records at a medium sized bank bank.

you're probably going to want to partition it somehow.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Craig Ringer

On 31/05/2010 5:41 PM, Giles Lean wrote:

=?iso-8859-1?Q?Alonso_Garc=EDa_=2C_Bruno_Elier?=bealo...@indra.es  wrote:


And the problems I am finding are the following:
-Queries from the client to the new DB server take a lot of time.
-Queries from the client to the old DB server are fast.
-The same query takes 150 secs in one case an 1 sec in the other case.


With that analysis, I'd be betting against it being a client problem.
(If you wanted, you might confirm that by pointing an old client at
the new server.)

I'd look into how the data was loaded into the new server and how
the database is configured: number of buffers, indexes, and whether
analyze has been run or not.

It would be strange indeed (possible, but very strange) to find
such a slowdown between 7.x and 8.x when the team is preparing
to push 9.0 out the door.  Surely it would have been known before;
therefore it's a practical certatinty that there is something
different about the configuration of your two servers.


... or that the planner is making a bad choice when it made a good one 
in 7.x . That's far from unheard of; the downside of a stats-based and 
very complex planner is that sometimes it doesn't make the perfect 
choice. Even with the same stats, etc, it's far from impossible that 7.x 
might hit a good plan when 8.x doesn't.


I mention this because the OP really needs to supply EXPLAIN ANALYZE 
results for the query run via psql (not their custom code) on both their 
7.x and 8.x servers.


--
Craig Ringer

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Alonso García , Bruno Elier
 With that analysis, I'd be betting against it being a client problem.
 (If you wanted, you might confirm that by pointing an old client at
 the new server.)

 I'd look into how the data was loaded into the new server and how
 the database is configured: number of buffers, indexes, and whether
 analyze has been run or not.

 It would be strange indeed (possible, but very strange) to find
 such a slowdown between 7.x and 8.x when the team is preparing
 to push 9.0 out the door.  Surely it would have been known before;
 therefore it's a practical certatinty that there is something
 different about the configuration of your two servers.

... or that the planner is making a bad choice when it made a good one 
in 7.x . That's far from unheard of; the downside of a stats-based and 
very complex planner is that sometimes it doesn't make the perfect 
choice. Even with the same stats, etc, it's far from impossible that 7.x 
might hit a good plan when 8.x doesn't.

I mention this because the OP really needs to supply EXPLAIN ANALYZE 
results for the query run via psql (not their custom code) on both their 
7.x and 8.x servers.

If I perform the query using pgadmin I get the same result in both versions 7.4 
and version 8.3. 
In fact I have written two test applications that perform the same query, one 
using the synchronous command processing (PQexec) an one using the asynchronous 
Command Processing (PQsendQuery / PQconsumeInput / PQisBusy / PQgetResult) and 
the results are:
- synchronous command processing takes less than two seconds to retrieve the 
result.
- asynchronous command processing takes more than 120 seconds to retrieve the 
result.
Both applications are connecting to the same DB so I don't know why I am 
getting different results. Well I know that PQIsBusy is returning true so I am 
not executing PQgetResult.
Bruno,

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] What Linux edition we should chose?

2010-05-31 Thread Michal Szymanski
Hi,
Currently we use Debian, but it chosen by our OS admnistrator. Now we
can change our OS and it is question what Linux edition will be the
best. We would like have access to new versions of Postgres as soon
as  possible, for Debian sometimes we had to wait many weeks for
official packages.

Regards
Michal Szymanski

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Craig Ringer

On 31/05/2010 10:34 PM, Alonso García , Bruno Elier wrote:


If I perform the query using pgadmin I get the same result in both versions 7.4 
and version 8.3.


Please post the output of EXPLAIN ANALYZE for each. See:

  http://wiki.postgresql.org/wiki/SlowQueryQuestions

and

  http://wiki.postgresql.org/wiki/Using_EXPLAIN

--
Craig Ringer

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Nilesh Govindarajan
On Mon, May 31, 2010 at 1:59 PM, Michal Szymanski dy...@poczta.onet.pl wrote:
 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

 Regards
 Michal Szymanski

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general



ArchLinux or Gentoo.

--
Nilesh Govindarajan
Facebook: nilesh.gr
Twitter: nileshgr
Website: www.itech7.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Scott Marlowe
On Mon, May 31, 2010 at 2:29 AM, Michal Szymanski dy...@poczta.onet.pl wrote:
 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

Pgsql is pretty easy to build from source.

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Craig Ringer

On 31/05/2010 10:34 PM, Alonso García , Bruno Elier wrote:

 If I perform the query using pgadmin I get the same result in both 
versions 7.4 and version 8.3.


Just re-read your post and realized you were probably saying that you 
get (effectively) the same EXPLAIN ANALYZE results from both, ie this 
isn't your problem.



In fact I have written two test applications that perform the same query, one 
using the synchronous command processing (PQexec) an one using the asynchronous 
Command Processing (PQsendQuery / PQconsumeInput / PQisBusy / PQgetResult) and 
the results are:
-  synchronous command processing takes less than two seconds to retrieve the 
result.


So PQexec works fine for you on both 7.4 and 8.3, producing a quick 
result no matter which server you run it against?



-  asynchronous command processing takes more than 120 seconds to retrieve the 
result.


You mean that this is where you have your problem, and it's fine on both 
versions when you use plain PQexec?


Consider using wireshark to examine the network traffic, and see if 
there's much activity during your long and slow PQconsumeInput / 
PQisBusy loop. The throughput analysis tool is handy for this.


--
Craig Ringer

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Andreas Kretschmer
Michal Szymanski dy...@poczta.onet.pl wrote:

 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon

With which distribution you are familiar?


 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

That's not true, and it's no problem to build PG from source.


Andreas
-- 
Really, I'm not out to destroy Microsoft. That will just be a completely
unintentional side effect.  (Linus Torvalds)
If I was god, I would recompile penguin with --enable-fly.   (unknown)
Kaufbach, Saxony, Germany, Europe.  N 51.05082°, E 13.56889°

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Szymon Guz
2010/5/31 Michal Szymanski dy...@poczta.onet.pl

 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

 Regards
 Michal Szymanski


Use whatever you want. I've been compiling PostgreSQL on Ubuntu, Debian and
Gentoo so far without any problems.
Choose the distribution that you're familiar with, take PostgreSQL sources
and compile as you wish.

regards
Szymon Guz


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Ivan Sergio Borgonovo
On Mon, 31 May 2010 08:47:25 -0600
Scott Marlowe scott.marl...@gmail.com wrote:

 On Mon, May 31, 2010 at 2:29 AM, Michal Szymanski
 dy...@poczta.onet.pl wrote:
  Hi,
  Currently we use Debian, but it chosen by our OS admnistrator.
  Now we can change our OS and it is question what Linux edition
  will be the best. We would like have access to new versions of
  Postgres as soon as  possible, for Debian sometimes we had to
  wait many weeks for official packages.
 
 Pgsql is pretty easy to build from source.

Yeah it is. But what is it going to be an upgrade process? On a
production box?
Any experience to share on upgrading from source on Debian?

-- 
Ivan Sergio Borgonovo
http://www.webthatworks.it


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Szymon Guz
2010/5/31 Ivan Sergio Borgonovo m...@webthatworks.it

 On Mon, 31 May 2010 08:47:25 -0600
 Scott Marlowe scott.marl...@gmail.com wrote:

  On Mon, May 31, 2010 at 2:29 AM, Michal Szymanski
  dy...@poczta.onet.pl wrote:
   Hi,
   Currently we use Debian, but it chosen by our OS admnistrator.
   Now we can change our OS and it is question what Linux edition
   will be the best. We would like have access to new versions of
   Postgres as soon as  possible, for Debian sometimes we had to
   wait many weeks for official packages.
 
  Pgsql is pretty easy to build from source.

 Yeah it is. But what is it going to be an upgrade process? On a
 production box?
 Any experience to share on upgrading from source on Debian?



Usually that's pretty easy: for upgrading the minor version (e.g. from 8.3.1
to 8.3.3) it should be enough to compile the new sources, stop server, run
`make install` and run the server with new binaries. Upgrading from 8.3 to
8.4 can be easily done using dump from current version. There is nothing
wrong to run the new and old postgres versions parallel so you can copy data
from one database to another.
There is also pgmigrator, but I haven't checked that yet.

Remember to make a database dump before the whole operation :)

regards
Szymon Guz


Re: [GENERAL] SELECT my_table.varchar FROM my_table

2010-05-31 Thread Richard Broersma
On Mon, May 31, 2010 at 7:48 AM, Jan Strube j...@deriva.de wrote:

 I accidentally encountered a feature in Postgres 8.3 that I couldn't find in
 the documentation while submitting a query like

 SELECT my_table.varchar FROM my_table

 which returns a concatenated string of all field values per row.
 I wonder where this is documented (and if it has something to do with
 composite types).

 Can anyone please explain?

I don't really know, but the result looks more like a single field
formatted as ROW-WISE rather than CSV.  The official way to get this
result is:

SELECT ROW( my_table.* ) FROM my_table;

http://www.postgresql.org/docs/9.0/static/sql-expressions.html#SQL-SYNTAX-ROW-CONSTRUCTORS


-- 
Regards,
Richard Broersma Jr.

Visit the Los Angeles PostgreSQL Users Group (LAPUG)
http://pugs.postgresql.org/lapug

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Tom Lane
Ivan Sergio Borgonovo m...@webthatworks.it writes:
 On Mon, 31 May 2010 08:47:25 -0600
 Scott Marlowe scott.marl...@gmail.com wrote:
 Pgsql is pretty easy to build from source.

 Yeah it is. But what is it going to be an upgrade process? On a
 production box?

If it makes you feel better, build your own RPMs (or
$package-style-of-choice).  This is actually a pretty good idea if you
are on a package-manager-based platform, as it makes it far simpler to
keep track of exactly what you've got installed.  It's generally not
hard to take the source package supplied by your distro and stick a
new minor-release source tarball into it.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Nilesh Govindarajan
You should use whatever you are comfortable with.
I would go with ArchLinux for its ease of use and making packages. RPM
and DPKG are much harder to build than ArchLinux's .pkg.tar.xz
Also, if you install some libraries like python clients or some
software depending on PgSql from the repositories in RPM/DPKG based
OS, you will have a tough time with the dependency stuff.
ArchLinux provides a PKGBUILD already from ABS (Arch Build System)
which contains all the dependency satisfiers. It serves two purposes-
depedency satisfaction and self compilation.

-- 
Nilesh Govindarajan
Facebook: nilesh.gr
Twitter: nileshgr
Website: www.itech7.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] SELECT my_table.varchar FROM my_table

2010-05-31 Thread Tom Lane
Richard Broersma richard.broer...@gmail.com writes:
 On Mon, May 31, 2010 at 7:48 AM, Jan Strube j...@deriva.de wrote:
 I accidentally encountered a feature in Postgres 8.3 that I couldn't find in
 the documentation while submitting a query like
 
 SELECT my_table.varchar FROM my_table
 
 which returns a concatenated string of all field values per row.
 I wonder where this is documented (and if it has something to do with
 composite types).
 
 Can anyone please explain?

 I don't really know, but the result looks more like a single field

It's equivalent to (my_table.*)::varchar.  We've seen enough people
confused by this (or the equivalent cases with text and name as
the target type) that I wonder if we should intentionally break the
symmetry and disable treating this case as a cast.  Although I do
rather wonder what the OP expected to happen here.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] SELECT my_table.varchar FROM my_table

2010-05-31 Thread Jan Strube


Am 31.05.2010 17:44, schrieb Tom Lane:

Richard Broersmarichard.broer...@gmail.com  writes:
   

On Mon, May 31, 2010 at 7:48 AM, Jan Strubej...@deriva.de  wrote:
 

I accidentally encountered a feature in Postgres 8.3 that I couldn't find in
the documentation while submitting a query like

SELECT my_table.varchar FROM my_table

which returns a concatenated string of all field values per row.
I wonder where this is documented (and if it has something to do with
composite types).

Can anyone please explain?
   
   

I don't really know, but the result looks more like a single field
 

It's equivalent to (my_table.*)::varchar.  We've seen enough people
confused by this (or the equivalent cases with text and name as
the target type) that I wonder if we should intentionally break the
symmetry and disable treating this case as a cast.  Although I do
rather wonder what the OP expected to happen here.
   


I didn't expect anything special, because my original statement was 
actually a typo. So I was just amazed that I didn't get an error.


Thanks for the explanation,
Jan


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Nested function invocation, but parameter does not exist

2010-05-31 Thread Wappler, Robert
Hi list,
I want to create an install script for a database. First a schema and
its elements are created in a second approach, some adjustments are
done, e.g. create rows, which can be referenced as defaults instead of
having NULL in the referenced column. Below is a minimum non-working
example.

The procedure create_default_ref_target() creates the actual row, which
should be referenced and has to return the automatically generated key.
The table reference should reference the row just generated, if there is
nothing else known. So the procedure alter_default_ref(int) alters the
table. But if alter_default_ref(int) is invoked, there is now parameter
$1. I do not really understand this. Invoking
create_default_ref_target() alone creates the row and returns a value.

Thanks for your help.

 Example:
CREATE TABLE referenced (id serial PRIMARY KEY, str text);
CREATE TABLE referencee (id serial PRIMARY KEY, ref int REFERENCES
referenced (id) NOT NULL);
CREATE OR REPLACE FUNCTION create_default_ref_target() RETURNS int
VOLATILE AS $$
   INSERT INTO referenced (str) VALUES ('default ref target')
RETURNING id;
$$ LANGUAGE SQL;
CREATE OR REPLACE FUNCTION alter_default_ref(_ref int) RETURNS void
VOLATILE AS $$
   ALTER TABLE referencee ALTER COLUMN ref SET DEFAULT $1;
$$ LANGUAGE SQL;

 Invocations:
SELECT alter_default_ref(create_default_ref_target());
ERROR:  there is no parameter $1
KONTEXT:  SQL function alter_default_ref statement 1
db= SELECT alter_default_ref(create_default_ref_target());
ERROR:  there is no parameter $1
KONTEXT:  SQL function alter_default_ref statement 1
db= SELECT * FROM referenced;
 id | str
+-
(0 Zeilen)

-- 
Robert...


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Luis Daniel Lucio Quiroz
Le lundi 31 mai 2010 10:23:51, Szymon Guz a écrit :
 2010/5/31 Ivan Sergio Borgonovo m...@webthatworks.it
 
  On Mon, 31 May 2010 08:47:25 -0600
  
  Scott Marlowe scott.marl...@gmail.com wrote:
   On Mon, May 31, 2010 at 2:29 AM, Michal Szymanski
   
   dy...@poczta.onet.pl wrote:
Hi,
Currently we use Debian, but it chosen by our OS admnistrator.
Now we can change our OS and it is question what Linux edition
will be the best. We would like have access to new versions of
Postgres as soon as  possible, for Debian sometimes we had to
wait many weeks for official packages.
   
   Pgsql is pretty easy to build from source.
  
  Yeah it is. But what is it going to be an upgrade process? On a
  production box?
  Any experience to share on upgrading from source on Debian?
 
 Usually that's pretty easy: for upgrading the minor version (e.g. from
 8.3.1 to 8.3.3) it should be enough to compile the new sources, stop
 server, run `make install` and run the server with new binaries. Upgrading
 from 8.3 to 8.4 can be easily done using dump from current version. There
 is nothing wrong to run the new and old postgres versions parallel so you
 can copy data from one database to another.
 There is also pgmigrator, but I haven't checked that yet.
 
 Remember to make a database dump before the whole operation :)
 
 regards
 Szymon Guz

Me as system architec, sysadmin and manager (gerencial power) jejej :)

we have choose Mandriva, it is quite easy to install and to maintain,  and  
speaking about packages there are many support in them, including PGSQL

LD

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] PG backup performance

2010-05-31 Thread Isabella Ghiurea

Hello PG list,
I 'm looking  for some tip, advice toimprove PG backups performance, 
presently  running
pg_dumpall compressed option on raid array 0 getting  aprox14GB writes 
in 45 min, I'm backing up aprox 200GB database cluster daily .
How can I improve this performance with the present hardware and PG 
version 8.3.6 , can I run parallel backups in PG ?

Thank you
Isabella

--
---
Isabella A. Ghiurea 


isabella.ghiu...@nrc-cnrc.gc.ca
Canadian Astronomy Data Centre |http://www.nrc-cnrc.gc.ca/eng/services/hia/data-centre.html 
National Research Council of Canada, Herzberg Institute of Astrophysics 
5071 West Saanich Road, Victoria BC V9E 2E7, Canada

Phone: 250 363-3446 fax: 250 363-0045


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Alonso García , Bruno Elier
So PQexec works fine for you on both 7.4 and 8.3, producing a quick 
result no matter which server you run it against?

Yes. If I use PQexec, both 7.4 and 8.3 produce a quick result but I if I use 
asynchronous command processing 8.3 produce a slow result whereas 7.4 works 
fine.

Consider using wireshark to examine the network traffic, and see if 
there's much activity during your long and slow PQconsumeInput / 
PQisBusy loop. The throughput analysis tool is handy for this.
I will be back with the results.
Bruno,

--
Craig Ringer

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Trouble with timestamp in PQprepare/PQexecPrepared

2010-05-31 Thread Viktor Pavlenko
Hello,

I'm using PG 8.1.11 on linux and having trouble inserting a timestamp
value (for starters :)) from a C++ program.

The table:

CREATE TABLE test_tbl (ts TIMESTAMP (6) NOT NULL);

Insert works from psql:

INSERT INTO test_tbl VALUES (to_timestamp('20100527101705806216', 
'MMDDHHMISSUS'));

C++ code:

--8

#define TIMESTAMPOID 1114

//conn is a connection

const char* stmt_name = test_insert;
const char* stmt = INSERT INTO test_tbl VALUES (to_timestamp($1, 
'MMDDHHMISSUS'));

Oid param_types[1];
param_types[0] = TIMESTAMPOID;

PGresult* res = PQprepare(conn, stmt_name, stmt, 1, param_types);

ExecStatusType res_status = PQresultStatus(res);
if (res_status != PGRES_COMMAND_OK) {
string msg = string(prepare ) + stmt_name +  failed with status  +
PQresStatus(res_status) + , error \ + PQresultErrorMessage(res) + 
\;
//print msg, exit
}

const char* ts = 20100527101705806216;

const char* param_values[1];
param_values[0] = ts;

int param_lengths[1];
param_lengths[0] = strlen(ts);

int param_formats[1];
param_formats[0] = 0;

res = PQexecPrepared(conn, stmt_name, 1, param_values, param_lengths,
 param_formats, 0);

res_status = PQresultStatus(res);
if (res_status != PGRES_COMMAND_OK) {
string msg = string(stmt_name) +  failed with status  +
PQresStatus(res_status) + , error \ + PQresultErrorMessage(res) + 
\;
//print msg, exit
}

--8

This fails with msg:

test_insert failed with status PGRES_FATAL_ERROR, error ERROR: 22007:
invalid input syntax for type timestamp: 20100527101705806216
LOCATION:  DateTimeParseError, datetime.c:3423


Please let me know what I'm doing wrong.

Thank you.

-- 
Viktor


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Nilesh Govindarajan
On Mon, May 31, 2010 at 8:17 PM, Scott Marlowe scott.marl...@gmail.com wrote:
 On Mon, May 31, 2010 at 2:29 AM, Michal Szymanski dy...@poczta.onet.pl 
 wrote:
 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

 Pgsql is pretty easy to build from source.

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general



And with ArchLinux its even more easy to keep track of its updates.
You don't have to make weird configurations. Just use a PKGBUILD
script from ABS (Arch Build System), change some settings, run makepkg
and your package is created with the default clean configuration. No
/usr/local stuff.

-- 
Nilesh Govindarajan
Facebook: nilesh.gr
Twitter: nileshgr
Website: www.itech7.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Libpq Asynchronous Command Processing

2010-05-31 Thread Tom Lane
=?iso-8859-1?Q?Alonso_Garc=EDa_=2C_Bruno_Elier?= bealo...@indra.es writes:
 Yes. If I use PQexec, both 7.4 and 8.3 produce a quick result but I if I use 
 asynchronous command processing 8.3 produce a slow result whereas 7.4 works 
 fine.

You're still being quite unclear.  Is this 7.4 libpq + 7.4 server
against 8.3 libpq + 8.3 server, or some cross-version combination?
Have you tried switching to the other library version?

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Trouble with timestamp in PQprepare/PQexecPrepared

2010-05-31 Thread Tom Lane
Viktor Pavlenko v...@cogeco.ca writes:
 I'm using PG 8.1.11 on linux and having trouble inserting a timestamp
 value (for starters :)) from a C++ program.

The first argument of to_timestamp is not a timestamp ...

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Trouble with timestamp in PQprepare/PQexecPrepared

2010-05-31 Thread Viktor Pavlenko
 TL == Tom Lane t...@sss.pgh.pa.us writes:

TL Viktor Pavlenko v...@cogeco.ca writes:
 I'm using PG 8.1.11 on linux and having trouble inserting a
 timestamp value (for starters :)) from a C++ program.

TL The first argument of to_timestamp is not a timestamp ...

TL regards, tom lane

Indeed! This is the fix:

#define VARCHAROID 1043
param_types[0] = VARCHAROID;

Thanks a lot for prompt reply.

-- 
Viktor


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Debian: upgrading from was: What Linux edition we should chose?

2010-05-31 Thread Ivan Sergio Borgonovo
On Mon, 31 May 2010 17:23:51 +0200
Szymon Guz mabew...@gmail.com wrote:

  Yeah it is. But what is it going to be an upgrade process? On a
  production box?
  Any experience to share on upgrading from source on Debian?

 Usually that's pretty easy: for upgrading the minor version (e.g.
 from 8.3.1 to 8.3.3) it should be enough to compile the new
 sources, stop server, run `make install` and run the server with
 new binaries. Upgrading from 8.3 to 8.4 can be easily done using
 dump from current version. There is nothing wrong to run the new
 and old postgres versions parallel so you can copy data from one
 database to another. There is also pgmigrator, but I haven't
 checked that yet.

That's clear but there are a bunch of small and possibly very
annoying details that make deploying in production a bit more
challenging than ./configure, make, make install.

I admit I only compiled postgres in my /home when I was developing
an extension. It is something I do rarely and never on production.

If I was thinking to upgrade on a debian box that is already running
a packaged version I'd have to understand how deal with debian
patches (I think most were related to paths where postgres expect to
find it's stuff).

Once I understand what all debian patches do I'll try to see if I
can avoid them all so that upgrading will be easier the next time.

I'll have to see how debian ./configure the package, I'll have to
replicate the init.d script for the newer version, take care of
making the 2 servers run temporarily on different ports... etc...

I could even think of making a .deb

I think about it I could even come up with a longer list of things I
should do.

I bet I'm not the first one that's going to upgrade Debian from
source. So someone may share his recipe and caveats.

I was actually thinking to test 9.0 in my /home on some real world
DB. That could be a chance to learn how to upgrade from source.

-- 
Ivan Sergio Borgonovo
http://www.webthatworks.it


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Installing version 8.4

2010-05-31 Thread Bob Pawley
During the install it was recommended that I run the scripts at
PostgresPlus\8.4ss\share\PostgreSQL\contrib

The folder contrib is present holding 78 files. But the PostgreSQL folder is 
missing.

Perhaps, that may be of some help?

Also I got the same error on my other computer that is running Windows XP. So 
if there is nothing wrong with the PostgreSQL installation it may well be some 
common element in both my computers.

Bob

[GENERAL] archive_command

2010-05-31 Thread Jun Wang
Where can I find an example shell script and windows batch file for
archive_command for backup?

Thanks.

Jack

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] archive_command

2010-05-31 Thread Bruce Momjian
Jun Wang wrote:
 Where can I find an example shell script and windows batch file for
 archive_command for backup?

There are no examples.  I can be any command you want, like 'COPY'.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Installing version 8.4

2010-05-31 Thread Ashesh Vashi
Hi Bob,

On Mon, May 31, 2010 at 10:57 PM, Bob Pawley rjpaw...@shaw.ca wrote:

  During the install it was recommended that I run the scripts at
 PostgresPlus\8.4ss\share\PostgreSQL\contrib

You should run the script against the output of pg_config.exe --sharedir,
which should be C:\Program Files (x64)\PostgresPlus\8.4SS\share\contrib on
your case. (and every windows).

 The folder contrib is present holding 78 files. But the PostgreSQL folder
 is missing.

 Perhaps, that may be of some help?

Thanks.
But, it is not. :-)


 Also I got the same error on my other computer that is running Windows XP.
 So if there is nothing wrong with the PostgreSQL installation it may well be
 some common element in both my computers.

Can you please share the log files from that machine too?
Are you using domain to setup the intranet?

The log file - you shared previously, does says that you're using existing
data directory.
If possible, please remove the existing data directory after uninstallation
and rerun the installer, and then please share the log file.
(NOTE: Please take back of the data directory, if it does contain some
important data).

--
Thanks  Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise Postgres Companyhttp://www.enterprisedb.com


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Devrim Gündüz
On Mon, 2010-05-31 at 21:14 +0530, Nilesh Govindarajan wrote:
 if you install some libraries like python clients or some
 software depending on PgSql from the repositories in RPM/DPKG based
 OS, you will have a tough time with the dependency stuff. 

Really?

-- 
Devrim Gündüz dev...@gunduz.org



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Devrim Gündüz
On Mon, 2010-05-31 at 08:47 -0600, Scott Marlowe wrote:
 Pgsql is pretty easy to build from source.

Right, but some sysadmins don't want to see development libraries on the
machines.
-- 
Devrim Gündüz dev...@gunduz.org


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Devrim Gündüz
On Mon, 2010-05-31 at 01:29 -0700, Michal Szymanski wrote:
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages. 

It is not many weeks actually -- it is just their QA policy.

Anyway, I've been running an RPM repository, which has up2date packages,
which are releases on the same date as PostgreSQL updates are releases.
You may want to consider it, if you are familiar with CentOS,RHEL or
Fedora:

http://yum.pgrpms.org

Regards,Currently we use Debian, but it chosen by our OS admnistrator.
Now we
can change our OS and it is question what Linux edition will be the
best. We would like have access to new versions of Postgres as soon
as  possible, for Debian sometimes we had to wait many weeks for
official packages.

-- 
Devrim Gündüz dev...@gunduz.org


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] PG backup performance

2010-05-31 Thread Andy Colson

On 05/31/2010 11:05 AM, Isabella Ghiurea wrote:

Hello PG list,
I 'm looking for some tip, advice toimprove PG backups performance,
presently running
pg_dumpall compressed option on raid array 0 getting aprox14GB writes in
45 min, I'm backing up aprox 200GB database cluster daily .
How can I improve this performance with the present hardware and PG
version 8.3.6 , can I run parallel backups in PG ?
Thank you
Isabella



Short answer, yes, you can.
Long answer, we need more info.

We need to know what the slow part is.

Are you CPU bound or IO bound?
Are you backing up over a network?
How many tables?  (well, big tables... how many really big tables).
How many cores/cpu's do you have?

I bet all 200GB hardly changes, are you sure you need to back it all up over 
and over again?  Have you thought of replication?

-Andy



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Nested function invocation, but parameter does not exist

2010-05-31 Thread Andy Colson

On 05/31/2010 11:00 AM, Wappler, Robert wrote:

Hi list,
I want to create an install script for a database. First a schema and
its elements are created in a second approach, some adjustments are
done, e.g. create rows, which can be referenced as defaults instead of
having NULL in the referenced column. Below is a minimum non-working
example.

The procedure create_default_ref_target() creates the actual row, which
should be referenced and has to return the automatically generated key.
The table reference should reference the row just generated, if there is
nothing else known. So the procedure alter_default_ref(int) alters the
table. But if alter_default_ref(int) is invoked, there is now parameter
$1. I do not really understand this. Invoking
create_default_ref_target() alone creates the row and returns a value.

Thanks for your help.

 Example:
CREATE TABLE referenced (id serial PRIMARY KEY, str text);
CREATE TABLE referencee (id serial PRIMARY KEY, ref int REFERENCES
referenced (id) NOT NULL);
CREATE OR REPLACE FUNCTION create_default_ref_target() RETURNS int
VOLATILE AS $$
INSERT INTO referenced (str) VALUES ('default ref target')
RETURNING id;
$$ LANGUAGE SQL;
CREATE OR REPLACE FUNCTION alter_default_ref(_ref int) RETURNS void
VOLATILE AS $$
ALTER TABLE referencee ALTER COLUMN ref SET DEFAULT $1;
$$ LANGUAGE SQL;

 Invocations:
SELECT alter_default_ref(create_default_ref_target());
ERROR:  there is no parameter $1
KONTEXT:  SQL function alter_default_ref statement 1
db=  SELECT alter_default_ref(create_default_ref_target());
ERROR:  there is no parameter $1
KONTEXT:  SQL function alter_default_ref statement 1
db=  SELECT * FROM referenced;
  id | str
+-
(0 Zeilen)



You don't understand what you wrote?  Or you didn't write it?  You dont 
understand the $1?  Its kinda a strange setup, but, I'm gonna guess what you 
need is:

CREATE OR REPLACE FUNCTION alter_default_ref(_ref int) RETURNS void
VOLATILE AS $$
ALTER TABLE referencee ALTER COLUMN ref SET DEFAULT _ref;
$$ LANGUAGE SQL;


or

CREATE OR REPLACE FUNCTION alter_default_ref(_ref int) RETURNS void
VOLATILE AS $$
execute 'ALTER TABLE referencee ALTER COLUMN ref SET DEFAULT ' || _ref;
$$ LANGUAGE SQL;


-Andy

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] PG backup performance

2010-05-31 Thread Isabella Ghiurea


Hi Andy,
Thank you , please, see bellow my answers:
Andy Colson wrote:


On 05/31/2010 11:05 AM, Isabella Ghiurea wrote:
 Hello PG list,
 I 'm looking for some tip, advice toimprove PG backups performance,
 presently running
 pg_dumpall compressed option on raid array 0 getting aprox14GB writes in
 45 min, I'm backing up aprox 200GB database cluster daily .
 How can I improve this performance with the present hardware and PG
 version 8.3.6 , can I run parallel backups in PG ?
 Thank you
 Isabella


Short answer, yes, you can.
Long answer, we need more info.

We need to know what the slow part is.

Are you CPU bound


CPU


or IO bound?
Are you backing up over a network?


No , on locally disks


How many tables?  (well, big tables... how many really big tables).

Around 20 big tables all of them in a one  separate schema on separate 
tables space from rest of other schemas. I  already start backing up 
individuals schema.
My big concern is IF I have to recover from backups will take me at 
least twice as much time around 1-2 days I expect.


How many cores/cpu's do you have?


4 quad core CPU server. How can I make PG to use  multi threading?



I bet all 200GB hardly changes, are you sure you need to back it all 
up over and over again?  Have you thought of replication?


Yes, but waiting for a  more robust build in replication  PG version 
aka  PG 9.1.

Isabella


--
---
Isabella A. Ghiurea 


isabella.ghiu...@nrc-cnrc.gc.ca
Canadian Astronomy Data Centre |http://www.nrc-cnrc.gc.ca/eng/services/hia/data-centre.html 
National Research Council of Canada, Herzberg Institute of Astrophysics 
5071 West Saanich Road, Victoria BC V9E 2E7, Canada

Phone: 250 363-3446 fax: 250 363-0045


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] PG backup performance

2010-05-31 Thread Andy Colson

On 05/31/2010 02:45 PM, Isabella Ghiurea wrote:


Hi Andy,
Thank you , please, see bellow my answers:
Andy Colson wrote:


On 05/31/2010 11:05 AM, Isabella Ghiurea wrote:
 Hello PG list,
 I 'm looking for some tip, advice toimprove PG backups performance,
 presently running
 pg_dumpall compressed option on raid array 0 getting aprox14GB
writes in
 45 min, I'm backing up aprox 200GB database cluster daily .
 How can I improve this performance with the present hardware and PG
 version 8.3.6 , can I run parallel backups in PG ?
 Thank you
 Isabella


Short answer, yes, you can.
Long answer, we need more info.

We need to know what the slow part is.

Are you CPU bound


CPU


or IO bound?
Are you backing up over a network?


No , on locally disks


How many tables? (well, big tables... how many really big tables).


Around 20 big tables all of them in a one separate schema on separate
tables space from rest of other schemas. I already start backing up
individuals schema.
My big concern is IF I have to recover from backups will take me at
least twice as much time around 1-2 days I expect.


How many cores/cpu's do you have?


4 quad core CPU server. How can I make PG to use multi threading?



I bet all 200GB hardly changes, are you sure you need to back it all
up over and over again? Have you thought of replication?


Yes, but waiting for a more robust build in replication PG version aka
PG 9.1.
Isabella





cool.. my second astronomer :-)

You are cpu bound because pg_dump will use one cpu, and if you have pg_dump 
compress, it uses the same cpu.  But if you use a pipe (pg_dump | gzip) then 
each gets a cpu.  And instead of using pg_dumpall, which dumps one db at a 
time, use pg_dump and run several in parallel.

try something like:

pg_dump big1 | gzip  big1.sql  pg_dump big2 | gzip  big2.sql

This will dump two at a time using 4 cpu's.

Have a backup file for each big table will also make restore faster, because 
you can restore them all at the same time, where as pg_dumpall will restore one 
at a time.

The compression is probably going to take the most time... if your data is not 
very compressible then you'll get a big speed boost by dropping the compress.

-Andy

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] PG backup performance

2010-05-31 Thread Andy Colson

On 05/31/2010 02:45 PM, Isabella Ghiurea wrote:



 I 'm looking for some tip, advice to improve PG backups performance,




Yep, I thought I recalled a conversation like this before, this might have some 
interesting info:

http://archives.postgresql.org/pgsql-performance/2010-03/msg00132.php


-Andy

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] Insert or Update a path from textbox...

2010-05-31 Thread david.catasus
Hi,

I'm just try to insert or update an actually table with Microsoft .NET platform 
VS2005.

The problem is that de \ dissapear when I make the insert or Update.

If i debug the object has all detailed path...so Why is not saved on the table.
The type of column is character (100).

So:

I have: C:\EVOL\Carga\
I obtain on table of the postgresql: c:EVOLCarga.

Is an X-File?

If anybody can has any idea...

Thanks.

David Catasús


Re: [GENERAL] Insert or Update a path from textbox...

2010-05-31 Thread Alban Hertroys
On 31 May 2010, at 23:27, david.cata...@1as.es wrote:

 Hi,
  
 I'm just try to insert or update an actually table with Microsoft .NET 
 platform VS2005.
  
 The problem is that de \ dissapear when I make the insert or Update.

Postgres is interpreting those backslashes as escape characters. Either escape 
them or turn on standard_conforming_strings.

That said, if you're having this problem your queries are probably vulnerable 
to SQL injection too, they're certainly not parameterised or Postgres would 
have done the escaping for you.

Alban Hertroys

--
If you can't see the forest for the trees,
cut the trees and you'll see there is no forest.


!DSPAM:737,4c04394a10151125916774!



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Out of Memory and Configuration Problems (Big Computer)

2010-05-31 Thread Tom Wilcox
I am having difficulties. I have rerun my update that uses the python 
functions..


(1) UPDATE nlpg.match_data SET org = normalise(org);

And some other similar queries on neighbouring fields in the table. They 
have all now worked. Without any changes to the configuration. I have 
done one thing in an attempt to minimise the risk of memory leak 
normalise() I added toks = None to the end of the normalise() 
function. However this was done after query (1) succeeded on the rerun.


Why would I get inconsistent behaviour? Would it have anything to do 
with SQL Server running on the same machine (although not actually doing 
anything at the moment - just idle server running in background).


Tangent: Is there any way to increase the memory allocated to postgres 
by Windows using Job Objects?


Cheers,
Tom

On 29/05/2010 18:55, Bill Moran wrote:

On 5/28/10 8:43:48 PM, Tom Wilcox wrote:

I ran this query:

EXPLAIN ANALYZE UPDATE nlpg.match_data SET org = org;

And I got this result:

Seq Scan on match_data  (cost=0.00..9762191.68 rows=32205168 width=206)
(actual time=76873.592..357450.519 rows=2961 loops=1)
Total runtime: 8028212.367 ms


That would seem to indicate that the problem is in your Python
functions.

Some ideas for next steps:
* Perhaps it's just a few rows that have data in them that the
  function has difficulty with.  Add some debugging/logging to
  the function and see if the row it bombs on has anything unusual
  in it (such as a very large text field)
* While large, that function is fairly simplistic.  You may want
  to consider rewriting it as an SQL function, which should be
  more efficient in any event.




On 28 May 2010 19:39, Tom Wilcox hungry...@googlemail.com
mailto:hungry...@googlemail.com wrote:

Oops. Sorry about that.

I am having this problem with multiple queries however I am
confident that a fair number may involve the custom plpython
normalise function which I have made myself. I didn't think it
would be complicated enough to produce a memory problem.. here it 
is:


-- Normalises common address words (i.e. 'Ground' maps to 'grd')
CREATE OR REPLACE FUNCTION normalise(s text) RETURNS text AS $$
ADDR_FIELD_DELIM = ' '

# Returns distinct list without null or empty elements
def distinct_str(list):
 seen = set()
 return [x for x in list if x not in seen and not seen.add(x)
and x!=None and len(x)0]

# normalise common words in given address string
def normalise(match_data):
 if match_data==None: return ''
 import re
 # Tokenise
 toks = distinct_str(re.split(r'\s', match_data.lower()))
 out = ''
 for tok in toks:
 ## full word replace
 if tok == 'house' : out += 'hse'+ADDR_FIELD_DELIM
 elif tok == 'ground' : out += 'grd'+ADDR_FIELD_DELIM
 elif tok == 'gnd' : out += 'grd'+ADDR_FIELD_DELIM
 elif tok == 'front' : out += 'fnt'+ADDR_FIELD_DELIM
 elif tok == 'floor' : out += 'flr'+ADDR_FIELD_DELIM
 elif tok == 'floors' : out += 'flr'+ADDR_FIELD_DELIM
 elif tok == 'flrs' : out += 'flr'+ADDR_FIELD_DELIM
 elif tok == 'fl' : out += 'flr'+ADDR_FIELD_DELIM
 elif tok == 'basement' : out += 'bst'+ADDR_FIELD_DELIM
 elif tok == 'subbasement' : out += 'sbst'+ADDR_FIELD_DELIM
 elif tok == 'bsmt' : out += 'bst'+ADDR_FIELD_DELIM
 elif tok == 'lbst' : out += 'lower bst'+ADDR_FIELD_DELIM
 elif tok == 'street' : out += 'st'+ADDR_FIELD_DELIM
 elif tok == 'road' : out += 'rd'+ADDR_FIELD_DELIM
 elif tok == 'lane' : out += 'ln'+ADDR_FIELD_DELIM
 elif tok == 'rooms' : out += 'rm'+ADDR_FIELD_DELIM
 elif tok == 'room' : out += 'rm'+ADDR_FIELD_DELIM
 elif tok == 'no' : pass
 elif tok == 'number' : pass
 elif tok == 'and' : out += ''+ADDR_FIELD_DELIM
 elif tok == 'rear' : out += 'rr'+ADDR_FIELD_DELIM
 elif tok == 'part' : out += 'pt'+ADDR_FIELD_DELIM
 elif tok == 'south' : out += 's'+ADDR_FIELD_DELIM
 elif tok == 'sth' : out += 's'+ADDR_FIELD_DELIM
 elif tok == 'north' : out += 'n'+ADDR_FIELD_DELIM
 elif tok == 'nth' : out += 'n'+ADDR_FIELD_DELIM
 elif tok == 'west' : out += 'w'+ADDR_FIELD_DELIM
 elif tok == 'wst' : out += 'w'+ADDR_FIELD_DELIM
 elif tok == 'east' : out += 'e'+ADDR_FIELD_DELIM
 elif tok == 'est' : out += 'e'+ADDR_FIELD_DELIM
 elif tok == 'first' : out += '1st'+ADDR_FIELD_DELIM
 elif tok == 'second' : out += '2nd'+ADDR_FIELD_DELIM
 elif tok == 'third' : out += '3rd'+ADDR_FIELD_DELIM
 elif tok == 'fourth' : out += '4th'+ADDR_FIELD_DELIM
 elif tok == 'fifth' : out += '5th'+ADDR_FIELD_DELIM
 elif tok == 'sixth' : out += '6th'+ADDR_FIELD_DELIM

Re: [GENERAL] Installing version 8.4

2010-05-31 Thread Bob Pawley



From: Ashesh Vashi 
Sent: Monday, May 31, 2010 11:33 AM
To: Bob Pawley 
Cc: Postgresql ; adrian.kla...@gmail.com 
Subject: Re: Installing version 8.4


Hi Bob,


On Mon, May 31, 2010 at 10:57 PM, Bob Pawley rjpaw...@shaw.ca wrote:

  During the install it was recommended that I run the scripts at
  PostgresPlus\8.4ss\share\PostgreSQL\contrib
You should run the script against the output of pg_config.exe --sharedir, 
which should be C:\Program Files (x64)\PostgresPlus\8.4SS\share\contrib on 
your case. (and every windows).
  The folder contrib is present holding 78 files. But the PostgreSQL folder is 
missing.

  Perhaps, that may be of some help?
Thanks.
But, it is not. :-) 

  Also I got the same error on my other computer that is running Windows XP. So 
if there is nothing wrong with the PostgreSQL installation it may well be some 
common element in both my computers.
Can you please share the log files from that machine too?
Are you using domain to setup the intranet?

I am keeping it local for now.


The log file - you shared previously, does says that you're using existing data 
directory.
If possible, please remove the existing data directory after uninstallation and 
rerun the installer, and then please share the log file.
(NOTE: Please take back of the data directory, if it does contain some 
important data).
Before the last install I removed all of the PostgreSQL and PostgresPlus 
folders as I thought that was where the existing data directory resided.

If not there where do I look for it??

Bob



--
Thanks  Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise Postgres Company





Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Alan Hodgson
On Monday 31 May 2010, Devrim Gündüz dev...@gunduz.org wrote:
 On Mon, 2010-05-31 at 21:14 +0530, Nilesh Govindarajan wrote:
  if you install some libraries like python clients or some
  software depending on PgSql from the repositories in RPM/DPKG based
  OS, you will have a tough time with the dependency stuff.
 
 Really?
 

Depends. If you build a compat- RPM to supply the original system-provided 
client libpq.so it will usually satisfy their package requirements. If you 
don't, then you might find yourself needing to rebuild other packages to 
coexist with upgraded PostgreSQL versions. Neither option is terribly 
difficult to accommodate.

-- 
No animals were harmed in the recording of this episode. We tried but that 
damn monkey was just too fast.

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] archive_command

2010-05-31 Thread Kenichiro Tanaka

Hello

In my environment,archive_command works fine with this command.

archive_command = 'COPY %p C:\\Program 
Files\\PostgreSQL\\8.4\\data\\archive\\%f'


(Is this what you want to know?)

 Where can I find an example shell script and windows batch file for
 archive_command for backup?

 Thanks.

 Jack




--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Nilesh Govindarajan
On Tue, Jun 1, 2010 at 4:40 AM, Alan Hodgson ahodg...@simkin.ca wrote:
 On Monday 31 May 2010, Devrim Gündüz dev...@gunduz.org wrote:
 On Mon, 2010-05-31 at 21:14 +0530, Nilesh Govindarajan wrote:
  if you install some libraries like python clients or some
  software depending on PgSql from the repositories in RPM/DPKG based
  OS, you will have a tough time with the dependency stuff.

 Really?


 Depends. If you build a compat- RPM to supply the original system-provided
 client libpq.so it will usually satisfy their package requirements. If you
 don't, then you might find yourself needing to rebuild other packages to
 coexist with upgraded PostgreSQL versions. Neither option is terribly
 difficult to accommodate.

 --
 No animals were harmed in the recording of this episode. We tried but that
 damn monkey was just too fast.

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general



@Devrim: You got the reason from @Alan.
Self compilation has the advantage of custom gcc flags like -O3 -march
-msse, etc. which can improve performance.
Building RPMs is not a task that everyone can do. It requires
extensive reading about rpmbuild and writing the specfile.
So if you install directly from source without RPM, it won't satisfy
the libpq.so dependency, so you cannot install applications using yum.
This is not the case with Arch PKGBUILD, because the PKGBUILD is just
a bash script.

-- 
Nilesh Govindarajan
Facebook: nilesh.gr
Twitter: nileshgr
Website: www.itech7.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Clemens Schwaighofer
Hi,

I run debian/testing since years and it is the best in my opinion.
Besides the fact that new versions come in quite fast (after the wait
phase from unstable to testing) the upgrade for major versions (eg 8.3
to 8.4) is very simple as it does not override the old files but does
a parallel install.

This is something I do miss from the RPM versions. Because if you do
not dump the data before you upgrade, you are quit screwed.

On Mon, May 31, 2010 at 17:29, Michal Szymanski dy...@poczta.onet.pl wrote:
 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

 Regards
 Michal Szymanski

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general




-- 
★ Clemens 呉 Schwaighofer
★ IT Engineer/Web Producer/Planning
★ E-Graphics Communications SP Digital
★ 6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN
★ Tel: +81-(0)3-3545-7706
★ Fax: +81-(0)3-3545-7343
★ http://www.e-gra.co.jp


This e-mail is intended only for the named person or entity to which
it is addressed and contains valuable business information that is 
privileged, confidential and/or otherwise protected from disclosure. 
If you received this e-mail in error, any review, use, dissemination,
distribution or copying of this e-mail is strictly prohibited.   
Please notify us immediately of the error via e-mail to 
disclai...@tbwaworld.com and please delete the e-mail from your system, 
retaining no copies in any media.
We appreciate your cooperation.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Nilesh Govindarajan
On Tue, Jun 1, 2010 at 6:38 AM, Clemens Schwaighofer
clemens_schwaigho...@e-gra.co.jp wrote:
 Hi,

 I run debian/testing since years and it is the best in my opinion.
 Besides the fact that new versions come in quite fast (after the wait
 phase from unstable to testing) the upgrade for major versions (eg 8.3
 to 8.4) is very simple as it does not override the old files but does
 a parallel install.

 This is something I do miss from the RPM versions. Because if you do
 not dump the data before you upgrade, you are quit screwed.

 On Mon, May 31, 2010 at 17:29, Michal Szymanski dy...@poczta.onet.pl wrote:
 Hi,
 Currently we use Debian, but it chosen by our OS admnistrator. Now we
 can change our OS and it is question what Linux edition will be the
 best. We would like have access to new versions of Postgres as soon
 as  possible, for Debian sometimes we had to wait many weeks for
 official packages.

 Regards
 Michal Szymanski

 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general




 --
 ★ Clemens 呉 Schwaighofer
 ★ IT Engineer/Web Producer/Planning
 ★ E-Graphics Communications SP Digital
 ★ 6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN
 ★ Tel: +81-(0)3-3545-7706
 ★ Fax: +81-(0)3-3545-7343
 ★ http://www.e-gra.co.jp


 This e-mail is intended only for the named person or entity to which
 it is addressed and contains valuable business information that is
 privileged, confidential and/or otherwise protected from disclosure.
 If you received this e-mail in error, any review, use, dissemination,
 distribution or copying of this e-mail is strictly prohibited.
 Please notify us immediately of the error via e-mail to
 disclai...@tbwaworld.com and please delete the e-mail from your system, 
 retaining no copies in any media.
 We appreciate your cooperation.


 --
 Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-general



Nope; you're wrong. Even RPM doesn't remove the data. But its always
safer to keep a backup.

-- 
Nilesh Govindarajan
Facebook: nilesh.gr
Twitter: nileshgr
Website: www.itech7.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Bruce Momjian
Craig Ringer wrote:
 Igor wrote:
  Hi All,
  
  Is there an easy way to add c++ files to my simple pgsql module ? My 
  Makefile 
  is as follows - 
  
  ===
  MODULES = pg_uservars
  DATA_built = pg_uservars.sql
  PGXS := $(shell pg_config --pgxs)
  include $(PGXS)
  ===
  
  I've got pg_uservars.c and hv.cc and I'd like to compile hv.cc via g++. 
  I'm aware of c++ name [de]mangling, just looking if there's a standard way 
  of 
  using C++ when it comes to pgxs.  
 
 It should just work. Simply make sure to follow the usual rules for
 calling into C++ code from C and vice versa:
 
 - Use extern C linkage for all functions that must be accessible by
   dlopen(), and preferably also for any functions that you might take
   a function pointer to and pass to C code
 
 - Never return new()'d memory that might be free()'d by the C code; use
   malloc()
 
 - Never delete() memory that was malloc()'d by the C code; use free()
 
 - Never let an exception propagate into the C code; use a catch-all
   block at the top level of all extern C functions
 
 ... and probably other things I've missed.

That is great new information.  I have created a new documentation
section called Using C++ for Extensibility, and listed you as the
author in the CVS commit;  patch attached.  Thanks. 

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

Index: doc/src/sgml/extend.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/extend.sgml,v
retrieving revision 1.38
diff -c -c -r1.38 extend.sgml
*** doc/src/sgml/extend.sgml	3 Apr 2010 07:22:53 -	1.38
--- doc/src/sgml/extend.sgml	1 Jun 2010 02:29:31 -
***
*** 273,276 
--- 273,322 
xoper;
xindex;
  
+   sect1 id=extend-how
+titleUsing C++ for Extensibility/title
+ 
+indexterm zone=extend-Cpp
+ primaryC++/primary
+/indexterm
+ 
+para
+ It is possible to use a compiler in C++ mode to build
+ productnamePostgreSQL/productname extensions;  you must simply
+ follow the standard methods for dynamically linking to C executables:
+ 
+ itemizedlist
+  listitem
+   para
+ Use literalextern C/ linkage for all functions that must
+ be accessible by functiondlopen()/.  This is also necessary
+ for any functions that might be passed as pointers between
+ the backend and C++ code.
+   /para
+  /listitem
+  listitem
+   para
+Use functionmalloc()/ to allocate any memory that might be
+freed by the backend C code (don't pass functionnew()/-allocated
+memory).
+   /para
+  /listitem
+  listitem
+   para
+Use functionfree()/ to free memory allocated by the backend
+C code (do not use functiondelete()/ for such cases).
+   /para
+  /listitem
+  listitem
+   para
+Prevent exceptions from propagating into the C code (use a
+catch-all block at the top level of all literalextern C/
+functions).
+   /para
+  /listitem
+ /itemizedlist
+/para
+ 
+   /sect1
+ 
   /chapter

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Schwaighofer Clemens
On Tue, Jun 1, 2010 at 11:30, Nilesh Govindarajan li...@itech7.com wrote:
 Nope; you're wrong. Even RPM doesn't remove the data. But its always
 safer to keep a backup.

I am not talking about removing the data I am talking of not beeing
able to access it because the database itself is still in the old
version.

Unless you use the migrate script, which just started to appear, you
had to dump the data, to the rpm upgrade and import the data.

I really prefer the debian way where I can run them parallel and
therefore test everything before I do a switchover.

-- 
★ Clemens 呉 Schwaighofer
★ IT Engineer/Web Producer/Planning
★ E-Graphics Communications SP Digital
★ 6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN
★ Tel: +81-(0)3-3545-7706
★ Fax: +81-(0)3-3545-7343
★ http://www.e-gra.co.jp


This e-mail is intended only for the named person or entity to which
it is addressed and contains valuable business information that is 
privileged, confidential and/or otherwise protected from disclosure. 
If you received this e-mail in error, any review, use, dissemination,
distribution or copying of this e-mail is strictly prohibited.   
Please notify us immediately of the error via e-mail to 
disclai...@tbwaworld.com and please delete the e-mail from your system, 
retaining no copies in any media.
We appreciate your cooperation.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 That is great new information.  I have created a new documentation
 section called Using C++ for Extensibility, and listed you as the
 author in the CVS commit;  patch attached.  Thanks. 

Too bad two out of the four pieces of advice are wrong (how many pieces
of memory managed by the backend are allocated directly with malloc?).
The other two are not wrong as far as they go, but they're certainly
woefully inadequate, because no interesting backend extension is going
to be able to get along without calling back into the core code.

Personally I would reduce this section to

para
 Don't.
/para

I don't think it is worth our time to try to support people who run into
the inevitable memory management and error handling incompatibilities.
Nor are they likely to be happy at the end of the experience, if we
blithely tell them up front that it'll work.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  That is great new information.  I have created a new documentation
  section called Using C++ for Extensibility, and listed you as the
  author in the CVS commit;  patch attached.  Thanks. 
 
 Too bad two out of the four pieces of advice are wrong (how many pieces
 of memory managed by the backend are allocated directly with malloc?).
 The other two are not wrong as far as they go, but they're certainly
 woefully inadequate, because no interesting backend extension is going
 to be able to get along without calling back into the core code.

Good point.  I assumed others would chime in to improve this.

 Personally I would reduce this section to
 
   para
Don't.
   /para
 
 I don't think it is worth our time to try to support people who run into
 the inevitable memory management and error handling incompatibilities.
 Nor are they likely to be happy at the end of the experience, if we
 blithely tell them up front that it'll work.

Well, I would have avoided this mine-trap except we have this 9.0
release note item:

   Allow use of productnameC++/ functions in backend code (Kurt
   Harriman, Peter Eisentraut)

I figure if we don't provide some guidance, things will be even worse.

I have updated the docs to mention palloc/pfree instead;  applied patch
attached.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +
Index: doc/src/sgml/extend.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/extend.sgml,v
retrieving revision 1.40
diff -c -c -r1.40 extend.sgml
*** doc/src/sgml/extend.sgml	1 Jun 2010 02:35:37 -	1.40
--- doc/src/sgml/extend.sgml	1 Jun 2010 02:53:30 -
***
*** 296,309 
   /listitem
   listitem
para
!Use functionmalloc()/ to allocate any memory that might be
 freed by the backend C code (don't pass functionnew()/-allocated
 memory).
/para
   /listitem
   listitem
para
!Use functionfree()/ to free memory allocated by the backend
 C code (do not use functiondelete()/ for such cases).
/para
   /listitem
--- 296,309 
   /listitem
   listitem
para
!Use functionpalloc()/ to allocate any memory that might be
 freed by the backend C code (don't pass functionnew()/-allocated
 memory).
/para
   /listitem
   listitem
para
!Use functionpfree()/ to free memory allocated by the backend
 C code (do not use functiondelete()/ for such cases).
/para
   /listitem

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Tom Lane wrote:
 Personally I would reduce this section to
  Don't.

 Well, I would have avoided this mine-trap except we have this 9.0
 release note item:
Allow use of productnameC++/ functions in backend code (Kurt
Harriman, Peter Eisentraut)

I'd be interested to see a section like this written by someone who'd
actually done a nontrivial C++ extension and lived to tell the tale.
As is, this is so incomplete that my opinion is it's worse than useless.
It gives people the impression that writing an extension in C++ will
be easy.  When they find out it isn't, we'll get the blame.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  Tom Lane wrote:
  Personally I would reduce this section to
 Don't.
 
  Well, I would have avoided this mine-trap except we have this 9.0
  release note item:
 Allow use of productnameC++/ functions in backend code (Kurt
 Harriman, Peter Eisentraut)
 
 I'd be interested to see a section like this written by someone who'd
 actually done a nontrivial C++ extension and lived to tell the tale.
 As is, this is so incomplete that my opinion is it's worse than useless.
 It gives people the impression that writing an extension in C++ will
 be easy.  When they find out it isn't, we'll get the blame.

So should I just comment it out and then when someone gets serious we
can use it as a starting point for them?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Well, I would have avoided this mine-trap except we have this 9.0
 release note item:
 Allow use of productnameC++/ functions in backend code (Kurt
 Harriman, Peter Eisentraut)

 So should I just comment it out and then when someone gets serious we
 can use it as a starting point for them?

Sure.  While you're at it, tone down the release-note item.  It should
read more like Take some steps towards allowing use ..., because C++
keywords in the header files surely were not the only stumbling block.

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  Well, I would have avoided this mine-trap except we have this 9.0
  release note item:
Allow use of productnameC++/ functions in backend code (Kurt
Harriman, Peter Eisentraut)
 
  So should I just comment it out and then when someone gets serious we
  can use it as a starting point for them?
 
 Sure.  While you're at it, tone down the release-note item.  It should
 read more like Take some steps towards allowing use ..., because C++
 keywords in the header files surely were not the only stumbling block.

OK, done with attached, applied patch.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +
Index: doc/src/sgml/extend.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/extend.sgml,v
retrieving revision 1.41
diff -c -c -r1.41 extend.sgml
*** doc/src/sgml/extend.sgml	1 Jun 2010 02:54:37 -	1.41
--- doc/src/sgml/extend.sgml	1 Jun 2010 03:17:46 -
***
*** 273,278 
--- 273,280 
xoper;
xindex;
  
+ !-- Use this someday when C++ is easier to use. bjm 2010-05-31
+ 
sect1 id=extend-Cpp
 titleUsing C++ for Extensibility/title
  
***
*** 318,322 
--- 320,325 
 /para
  
/sect1
+ --
  
   /chapter
Index: doc/src/sgml/release-9.0.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/release-9.0.sgml,v
retrieving revision 2.22
diff -c -c -r2.22 release-9.0.sgml
*** doc/src/sgml/release-9.0.sgml	17 May 2010 17:46:13 -	2.22
--- doc/src/sgml/release-9.0.sgml	1 Jun 2010 03:17:50 -
***
*** 2519,2532 
  
   listitem
para
!Allow use of productnameC++/ functions in backend code (Kurt
 Harriman, Peter Eisentraut)
/para
  
para
!This removes keyword conflicts that previously made productnameC++/
!usage difficult in backend code. literalextern C { }/ might still
!be necessary.
/para
   /listitem
  
--- 2519,2534 
  
   listitem
para
!Simplify use of productnameC++/ functions in backend code (Kurt
 Harriman, Peter Eisentraut)
/para
  
para
!While this removes keyword conflicts that previously made
!productnameC++/ usage difficult in backend code, there are
!still other complexities when using productnameC++/ for backend
!functions. literalextern C { }/ is still necessary in
!some cases.
/para
   /listitem
  

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] INSERTing lots of data

2010-05-31 Thread Greg Smith

Joachim Worringen wrote:
my Python application (http://perfbase.tigris.org) repeatedly needs to 
insert lots of data into an exsting, non-empty, potentially large 
table. Currently, the bottleneck is with the Python application, so I 
intend to multi-thread it. Each thread should work on a part of the 
input file.


You are wandering down a path followed by pgloader at one point:  
http://pgloader.projects.postgresql.org/#toc6 and one that I fought with 
briefly as well.  Simple multi-threading can be of minimal help in 
scaling up insert performance here, due to the Python issues involved 
with the GIL.  Maybe we get Dimitri to chime in here, he did more of 
this than I did.


Two thoughts.  First, build a test performance case assuming it will 
fail to scale upwards, looking for problems.  If you get lucky, great, 
but don't assume this will work--it's proven more difficult than is 
obvious in the past for others.


Second, if you do end up being throttled by the GIL, you can probably 
build a solution for Python 2.6/3.0 using the multiprocessing module for 
your use case:  http://docs.python.org/library/multiprocessing.html


--
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com   www.2ndQuadrant.us


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Greg Smith

Michal Szymanski wrote:

Currently we use Debian, but it chosen by our OS admnistrator. Now we
can change our OS and it is question what Linux edition will be the
best. We would like have access to new versions of Postgres as soon
as  possible, for Debian sometimes we had to wait many weeks for
official packages.
  


Yes, Debian QA can take a couple of weeks for things to reach you after 
release.  From some perspectives that's considered a good thing.  If the 
update is to fix a security bug, it's possible that's a problem 
instead.  In that rare case, you can always learn to build your own 
packages.


Ultimately, if your true priority is access to new versions of Postgres 
as soon as possible, you can do that on any Linux distribution by 
building from source and potentially packaging the result up as if it 
were a standard packages.  That should be way, way down on the list of 
things that factor into what version of Linux you deploy though.  If 
you've got support from your administration team using Debian, I think 
you'd be crazy to switch to another OS just to speed up getting newer 
versions of PostgreSQL.  Put a little time into learning how to build 
your own packages instead, to work around this one perceived flaw, and 
you'll be way ahead of the mess that comes with switching distributions 
altogether.


--
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com   www.2ndQuadrant.us


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] server-side extension in c++

2010-05-31 Thread Craig Ringer

On 01/06/10 10:48, Tom Lane wrote:


Too bad two out of the four pieces of advice are wrong (how many pieces
of memory managed by the backend are allocated directly with malloc?).
The other two are not wrong as far as they go, but they're certainly
woefully inadequate, because no interesting backend extension is going
to be able to get along without calling back into the core code.


It's a lot like mixing C++ with Symbian's longjump-based error handling. 
It's possible, just ugly, and requires error-handling boundaries to be 
carefully thought out.


Rather than saying don't mix new/delete and malloc/free I should've 
said always be sure to release memory with the matching function to 
that which allocated it, thus covering palloc too. Not that you 
generally need to worry too much about palloc'd memory.



Personally I would reduce this section to

para
Don't.
/para


Sometimes you need or want to expose capabilities of a C++ library. So 
long as you do so with proper encapsulation of the C++ functionality, so 
that the only interfaces Pg sees are C, there's really no problem.



Nor are they likely to be happy at the end of the experience, if we
blithely tell them up front that it'll work.


I've had no issues using C++ libraries in Pg server-side code. It *does* 
work. You just need to be careful where your error-handling and memory 
management style boundaries lie.


--
Craig Ringer

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Adrian von Bidder
On Tuesday 01 June 2010 03.08:06 Clemens Schwaighofer wrote:
 Besides the fact that new versions come in quite fast (after the wait
 phase from unstable to testing)

... and you can always mix testing and unstable.  If your testing 
installation is not too old, usually not much fiddling with dependencies is 
involved.  Reading the apt_preferences manual page and some other Debian 
documentation to understand how packages are moved between 
experimental/unstable/testing is strongly recommended, though.

I'd hesitate to use unstable for a production machine, though.  Even testing 
is not always a good idea, since security support for testing is not quite 
as good as for stable.

(shameless plug, since we're speaking of Debian ...:
http://blog.fortytwo.ch/archives/84-Order-Your-Debian-Swirl-Umbrella-Now.html)

cheers
-- vbi

-- 
Why on earth should we teach children
that they are not allowed to share the toys.
-- Patrick Harvie, Member of the Scottish Parliament
   Speaking at Debconf7


signature.asc
Description: This is a digitally signed message part.


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Adrian von Bidder
Heyho!

On Tuesday 01 June 2010 06.01:02 Greg Smith wrote:
 Put a little time into learning how to build 
 your own packages instead, to work around this one perceived flaw, and 
 you'll be way ahead of the mess that comes with switching distributions 
 altogether.

Note that we can always use more people to help Debian, too.  If you feel 
Debian is too slow uploading new pg versions [1]: perhaps you can change 
this by helping Martin preparing and testing new packages.  Packaging stuff 
for Debian is not magic, it's just Makefiles, Perl/shell scripts and stuff 
like this.  (And I, too, like the parallel installation of different pg 
versions and was very much missing it when I was forced to work on SuSE for 
some business stuff...)

cheers
-- vbi


[1] (you did look at unstable and experimental?  The stable releases are 
quite slow, that's true.)



-- 
featured link: http://www.pool.ntp.org


signature.asc
Description: This is a digitally signed message part.


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Devrim GÜNDÜZ
Hi,

On Tue, 2010-06-01 at 06:59 +0530, Nilesh Govindarajan wrote:
 Self compilation has the advantage of custom gcc flags like -O3 -march
 -msse, etc. which can improve performance.

I started to think that you have zero idea about building binary
packages.

 Building RPMs is not a task that everyone can do. It requires
 extensive reading about rpmbuild and writing the specfile.

Really?

http://people.planetpostgresql.org/devrim/index.php?/archives/44-How-To-Build-Your-Own-PostgreSQL-and-related-software-RPMs-on-CentOSRHELFedora.html

I can't see anything except svn co and make build there. Do you still
need an extensive reading?

 So if you install directly from source without RPM, it won't satisfy
 the libpq.so dependency, so you cannot install applications using yum.
 This is not the case with Arch PKGBUILD, because the PKGBUILD is just
 a bash script. 

Now I'm sure that you don't have any idea about PostgreSQL RPM packages:

http://svn.pgrpms.org/browser/rpm/redhat/8.4/compat-postgresql/EL-5

might give you a clue.

You may think that ArchLinux is fine for you, but please pick up correct
arguments for RPMs first.

Devrim - The RPM Packager
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz


signature.asc
Description: This is a digitally signed message part


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Devrim GÜNDÜZ
On Tue, 2010-06-01 at 07:20 +0200, Adrian von Bidder wrote:
 Packaging stuff  for Debian is not magic, it's just Makefiles,
 Perl/shell scripts and stuff like this. 

Given that *even I* ( :P ) could build a few 8.2 .deb packages for my
previous employer, I also want to confirm that building .debs are not
that hard.
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz


signature.asc
Description: This is a digitally signed message part


Re: [GENERAL] What Linux edition we should chose?

2010-05-31 Thread Nilesh Govindarajan
2010/6/1 Devrim GÜNDÜZ dev...@gunduz.org:
 Hi,

 On Tue, 2010-06-01 at 06:59 +0530, Nilesh Govindarajan wrote:
 Self compilation has the advantage of custom gcc flags like -O3 -march
 -msse, etc. which can improve performance.

 I started to think that you have zero idea about building binary
 packages.

 Building RPMs is not a task that everyone can do. It requires
 extensive reading about rpmbuild and writing the specfile.

 Really?

 http://people.planetpostgresql.org/devrim/index.php?/archives/44-How-To-Build-Your-Own-PostgreSQL-and-related-software-RPMs-on-CentOSRHELFedora.html

 I can't see anything except svn co and make build there. Do you still
 need an extensive reading?

 So if you install directly from source without RPM, it won't satisfy
 the libpq.so dependency, so you cannot install applications using yum.
 This is not the case with Arch PKGBUILD, because the PKGBUILD is just
 a bash script.

 Now I'm sure that you don't have any idea about PostgreSQL RPM packages:

 http://svn.pgrpms.org/browser/rpm/redhat/8.4/compat-postgresql/EL-5

 might give you a clue.

 You may think that ArchLinux is fine for you, but please pick up correct
 arguments for RPMs first.

 Devrim - The RPM Packager
 --
 Devrim GÜNDÜZ
 PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
 PostgreSQL RPM Repository: http://yum.pgrpms.org
 Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
 http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz



@Devrim, I may be wrong at sometimes, because I have not done any
qualification research on this. All I have learned from Google and
self experience.
I run my site (see my signature) on a self managed VPS. I was using
the default PGSQL RPM from the fedora repository, the site was getting
way slow. So I compiled all the stuff apache, php and postgresql with
custom gcc flags, which improved performance like hell.
This may not apply to all, its my experience; not an illusion because
I asked other site contributors also about the speed, they said it was
much better.

-- 
Nilesh Govindarajan
Facebook: nilesh.gr
Twitter: nileshgr
Website: www.itech7.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general