Re: [BUGS] [ADMIN] Repeatable crash in pg_dump (with -d2 info)
On Mon, Aug 27, 2012 at 9:58 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Jan 17, 2012 at 04:46:50PM -0500, David Schnur wrote: I finally had time to test this further on a variety of systems, and was unable to reproduce on any non-Windows platform. The dump even works fine on Windows XP; just not Windows 7. This prompted me to do a little more research, and this time I found this thread from Sept. 2011: http://postgresql.1045698.n5.nabble.com/ BUG-6233-pg-dump-hangs-with-Access-Violation-C005-td4851598.html From Tom Lane in the above thread: Hmm. I can see how that would happen if you're using one of the Windows environments wherein malloc's done inside libpq have to be free'd inside libpq. (The PQExpBuffer support code is in libpq...) However, the flaw in that explanation is that it would basically mean pg_dump doesn't work at all on Windows, at least not if you have any user-defined functions, and probably some other cases too because there seem to be multiple instances of the dubious coding. It's a bit hard to believe that nobody's noticed that before. This appears to describe exactly the issue I'm encountering, and my build is in fact linked against the static runtime. I guess the reason this hasn't come up sooner is because most Windows users either use the 'official' binaries rather than compiling from source, or link against the dynamic runtime. Is this something I could expect to be fixed in the near future, or is it enough of an edge case that I should come up with some solution or work-around on my own? Thanks, Late reply, but I don't see any way we could fix this easily. To me it seems like mostly a case of chasing down all the places where this happens. It's not impossible to do; it's just a bunch of work that nobody's gotten excited about doing yet. We've fixed similar issues in many other cases, IIUC. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] [ADMIN] Repeatable crash in pg_dump (with -d2 info)
Robert Haas robertmh...@gmail.com writes: On Mon, Aug 27, 2012 at 9:58 AM, Bruce Momjian br...@momjian.us wrote: From Tom Lane in the above thread: Hmm. I can see how that would happen if you're using one of the Windows environments wherein malloc's done inside libpq have to be free'd inside libpq. (The PQExpBuffer support code is in libpq...) Late reply, but I don't see any way we could fix this easily. To me it seems like mostly a case of chasing down all the places where this happens. It's not impossible to do; it's just a bunch of work that nobody's gotten excited about doing yet. We've fixed similar issues in many other cases, IIUC. Well, the problem with what I suspect you're thinking of is that even after we fixed all the existing trouble spots, it would keep on breaking. Unless we found some mechanical way to warn about unsafe coding; which in itself would be more work than I want to put into this. However, a plan B occurs to me: what about preventing pg_dump from using libpq's copy of PQExpBuffer? IIUC, a copy of the PQExpBuffer code statically linked into pg_dump would be free of this problem. There's not so much code there that this would be an intolerable price to pay. (Besides, we could probably arrange for the extra copy to happen only on Windows.) regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] [ADMIN] Repeatable crash in pg_dump (with -d2 info)
On Tue, Jan 17, 2012 at 04:46:50PM -0500, David Schnur wrote: I finally had time to test this further on a variety of systems, and was unable to reproduce on any non-Windows platform. The dump even works fine on Windows XP; just not Windows 7. This prompted me to do a little more research, and this time I found this thread from Sept. 2011: http://postgresql.1045698.n5.nabble.com/ BUG-6233-pg-dump-hangs-with-Access-Violation-C005-td4851598.html From Tom Lane in the above thread: Hmm. I can see how that would happen if you're using one of the Windows environments wherein malloc's done inside libpq have to be free'd inside libpq. (The PQExpBuffer support code is in libpq...) However, the flaw in that explanation is that it would basically mean pg_dump doesn't work at all on Windows, at least not if you have any user-defined functions, and probably some other cases too because there seem to be multiple instances of the dubious coding. It's a bit hard to believe that nobody's noticed that before. This appears to describe exactly the issue I'm encountering, and my build is in fact linked against the static runtime. I guess the reason this hasn't come up sooner is because most Windows users either use the 'official' binaries rather than compiling from source, or link against the dynamic runtime. Is this something I could expect to be fixed in the near future, or is it enough of an edge case that I should come up with some solution or work-around on my own? Thanks, Late reply, but I don't see any way we could fix this easily. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] [ADMIN] Repeatable crash in pg_dump (with -d2 info)
I finally had time to test this further on a variety of systems, and was unable to reproduce on any non-Windows platform. The dump even works fine on Windows XP; just not Windows 7. This prompted me to do a little more research, and this time I found this thread from Sept. 2011: http://postgresql.1045698.n5.nabble.com/BUG-6233-pg-dump-hangs-with-Access-Violation-C005-td4851598.html From Tom Lane in the above thread: Hmm. I can see how that would happen if you're using one of the Windows environments wherein malloc's done inside libpq have to be free'd inside libpq. (The PQExpBuffer support code is in libpq...) However, the flaw in that explanation is that it would basically mean pg_dump doesn't work at all on Windows, at least not if you have any user-defined functions, and probably some other cases too because there seem to be multiple instances of the dubious coding. It's a bit hard to believe that nobody's noticed that before. This appears to describe exactly the issue I'm encountering, and my build is in fact linked against the static runtime. I guess the reason this hasn't come up sooner is because most Windows users either use the 'official' binaries rather than compiling from source, or link against the dynamic runtime. Is this something I could expect to be fixed in the near future, or is it enough of an edge case that I should come up with some solution or work-around on my own? Thanks, David On Mon, Nov 28, 2011 at 8:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hmph. I still can't reproduce this here, which seems to mean one of three things: 1. We fixed this since 8.3.15 (still doesn't seem likely). 2. It's specific to Windows. 3. It's specific to the pg_dump build you are using.
Re: [BUGS] [ADMIN] Repeatable crash in pg_dump (with -d2 info)
David Schnur dnsch...@gmail.com writes: I probably can't get a stack trace, but I was able to reproduce it with just that function. Without the function, pg_dump works fine. I can DROP the function, pg_dump works, then add it back again and pg_dump crashes. Hmph. I still can't reproduce this here, which seems to mean one of three things: 1. We fixed this since 8.3.15 (still doesn't seem likely). 2. It's specific to Windows. 3. It's specific to the pg_dump build you are using. I wonder whether anyone else can get this to happen on the strength of David's description? Here are my steps: initdb -A md5 --no-locale -E UTF8 -U testuser -D C:\Users\David\Desktop\testdb -W pg_ctl -U testuser -D C:\Users\David\Desktop\testdb start CREATE DATABASE testdb OWNER testuser; CREATE OR REPLACE FUNCTION datastore_unpack( data_times TIMESTAMP WITH TIME ZONE[], data_values DOUBLE PRECISION[], OUT data_time TIMESTAMP WITH TIME ZONE, OUT data_value DOUBLE PRECISION ) RETURNS SETOF RECORD AS $$ SELECT $1[rowx] AS data_time, $2[rowx] AS data_value FROM generate_series(1, array_upper($1, 1)) AS rowx; $$ LANGUAGE SQL STABLE; pg_dump -v -F c -x -O -f C:\Users\David\Desktop\dumptest -U testuser testdb Here's the output from pg_dump: pg_dump: reading schemas pg_dump: reading user-defined functions pg_dump: reading user-defined types pg_dump: reading procedural languages pg_dump: reading user-defined aggregate functions pg_dump: reading user-defined operators pg_dump: reading user-defined operator classes pg_dump: reading user-defined text search parsers pg_dump: reading user-defined text search templates pg_dump: reading user-defined text search dictionaries pg_dump: reading user-defined text search configurations pg_dump: reading user-defined operator families pg_dump: reading user-defined conversions pg_dump: reading user-defined tables pg_dump: reading table inheritance information pg_dump: reading rewrite rules pg_dump: reading type casts pg_dump: finding inheritance relationships pg_dump: reading column info for interesting tables pg_dump: flagging inherited columns in subtables pg_dump: reading indexes pg_dump: reading constraints pg_dump: reading triggers pg_dump: reading dependency data pg_dump: saving encoding = UTF8 pg_dump: saving standard_conforming_strings = off pg_dump: saving database definition And here's the output from the postmaster: LOG: statement: SET DATESTYLE = ISO LOG: statement: SET extra_float_digits TO 2 LOG: statement: SET synchronize_seqscans TO off LOG: statement: BEGIN LOG: statement: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE LOG: statement: SET search_path = pg_catalog LOG: statement: SELECT tableoid, oid, nspname, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = nspowner) as rolname, nspacl FROM pg_namespace LOG: statement: SELECT tableoid, oid, proname, prolang, pronargs, proargtypes, prorettype, proacl, pronamespace,(SELECT rolname FROM pg_catalog.pg_roles WHERE oid = proowner) as rolname FROM pg_proc WHERE NOT proisagg AND pronamespace != (select oid from pg_namespace where nspname = 'pg_catalog') LOG: statement: SELECT tableoid, oid, typname, typnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = typowner) as rolname, typinput::oid as typinput, typoutput::oid as typoutput, typelem, typrelid, CASE WHEN typrelid = 0 THEN ' '::char ELSE (SELECT relkind FROM pg_class WHERE oid = typrelid) END as typrelkind, typtype, typisdefined, typname[0] = '_' AND typelem != 0 AND (SELECT typarray FROM pg_type te WHERE oid = pg_type.typelem) = oid AS isarray FROM pg_type LOG: statement: SELECT tableoid, oid, lanname, lanpltrusted, lanplcallfoid, lanvalidator, lanacl, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = lanowner) as lanowner FROM pg_language WHERE lanispl ORDER BY oid LOG: statement: SELECT tableoid, oid, proname as aggname, pronamespace as aggnamespace, pronargs, proargtypes, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = proowner) as rolname, proacl as aggacl FROM pg_proc WHERE proisagg AND pronamespace != (select oid from pg_namespace where nspname = 'pg_catalog') LOG: statement: SELECT tableoid, oid, oprname, oprnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = oprowner) as rolname, oprcode::oid as oprcode FROM pg_operator LOG: statement: SELECT tableoid, oid, opcname, opcnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = opcowner) as rolname FROM pg_opclass LOG: statement: SELECT tableoid, oid, prsname, prsnamespace, prsstart::oid, prstoken::oid, prsend::oid, prsheadline::oid, prslextype::oid FROM pg_ts_parser LOG: statement: SELECT tableoid, oid, tmplname, tmplnamespace, tmplinit::oid, tmpllexize::oid FROM pg_ts_template LOG: statement: SELECT tableoid, oid, dictname, dictnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = dictowner) as rolname, dicttemplate, dictinitoption FROM
