Re: minimal wal_level on subscriber

2020-01-16 Thread Arnaud L.
Le 16/01/2020 à 19:58, Laurenz Albe a écrit : On Thu, 2020-01-16 at 12:09 +0100, Arnaud L. wrote: is it OK to set wal_level to minimal on the subscriber side of the logical replication ? Yes, if you don't want physical backups. You mean online physical backups ? I could still shutdown

minimal wal_level on subscriber

2020-01-16 Thread Arnaud L.
Hi list, is it OK to set wal_level to minimal on the subscriber side of the logical replication ? Thanks ! Regards -- Arnaud

Re: psql \copy hanging

2019-10-10 Thread Arnaud L.
Le 08/10/2019 à 16:59, Arnaud L. a écrit : Yes, I already did that, and unfortunately there's no activy. There is absolutely no activity in procmon using psql.exe as a filter. process-xp is not giving more information, processor usage is 0%. My apologies, I obviously did something wrong last

Re: psql \copy hanging

2019-10-08 Thread Arnaud L.
Le 08/10/2019 à 16:55, Daniel Verite a écrit : Arnaud L. wrote: Anyway, it hung using this syntax during last night's run. I'll give it another try tonight just to be sure. When psql.exe is hanging, maybe you could use a tool like Process Monitor [1] or Process Explorer [2] to get

Re: psql \copy hanging

2019-10-08 Thread Arnaud L.
Le 08/10/2019 à 16:03, Adrian Klaver a écrit : This is going to be hard to troubleshoot if you change your commands. Previously you had: psql -h %MYPGSERVER% -a -f myscript.sql %MYPGDB% Changing more then one thing at a time makes it that much more difficult to isolate the issue. Yes, true.

Re: psql \copy hanging

2019-10-08 Thread Arnaud L.
Le 08/10/2019 à 12:55, Daniel Verite a écrit : Testing this with 11.5, it works for me. Make sure you're running the latest minor release (on the client side in this case), because a related fix was issued last February. For the 11 branch it was in version 11.2. OK, my bad, backslashes in a

Re: psql \copy hanging

2019-10-08 Thread Arnaud L.
Le 08/10/2019 à 12:55, Daniel Verite a écrit : Arnaud L. wrote: As a side note, COPY (...) TO STDOUT \g 'somefile' does not work in a script file (i.e. it does not work if the command is passed in a file via the -f argument). The command runs fine, no error is raised either

Re: psql \copy hanging

2019-10-08 Thread Arnaud L.
Le 08/10/2019 à 09:28, Pavel Stehule a écrit : you can write simple C application with COPY API https://www.postgresql.org/docs/12/libpq-copy.html Unfortunately, I don't know C. Then you can eliminate or ensure locality of problem. more, you can use server side copy. Superuser can read data

Re: psql \copy hanging

2019-10-08 Thread Arnaud L.
Le 07/10/2019 à 16:36, Adrian Klaver a écrit : So you are saying that you have not run the problematic line by itself? It hung during last night's run. I had modified my batch script to run the \copy commands separately, i.e. it now reads as : psql -h myserver -a mydb <

Re: psql \copy hanging

2019-10-07 Thread Arnaud L.
Le 07/10/2019 à 16:36, Adrian Klaver a écrit : So you are saying that you have not run the problematic line by itself? It depends what you mean by that. I've run this line by itself many times. Everytime the script has failed in fact. But until today I had not splitted the batch script to

Re: psql \copy hanging

2019-10-07 Thread Arnaud L.
Le 04/10/2019 à 19:08, Adrian Klaver a écrit : On 10/4/19 12:19 AM, Arnaud L. wrote: OK I can do that. I thought I nailed it down to this line because it started failing when this line was ~5th in the script, and it kept failing on that very same line after I moved it at the very end

Re: psql \copy hanging

2019-10-04 Thread Arnaud L.
Le 03/10/2019 à 16:32, Adrian Klaver a écrit : I may have missed it before, but where is the Postgres server located? On the same local area network. Not on the computer running the script (so direct COPY TO is not an option). Given that this seems to be some sort of resource issue and

Re: psql \copy hanging

2019-10-03 Thread Arnaud L.
Le 03/10/2019 à 15:54, Adrian Klaver a écrit : On 10/2/19 11:51 PM, Arnaud L. wrote: Well, this problem is still bugging me, and this time I've tried with a local file. Unfortunately, it did not help. To further rule out filesystem problems, I first took care to delete the target files before

Bad estimates on GIN bigint[] index

2019-09-06 Thread Arnaud L.
Le 03/09/2019 à 15:43, Tom Lane a écrit : "Arnaud L." writes: -> Bitmap Index Scan on planet_osm_ways_nodes_idx (cost=0.00..11190.36 rows=1420982 width=0) (actual time=0.268..0.268 rows=1 loops=1) Index Cond: (nodes && '{1}'::bigint[]) The planner should

Re: Slow statement using parallelism after 9.6>11 upgrade

2019-09-04 Thread Arnaud L.
Le 04/09/2019 à 09:04, Arnaud L. a écrit : Le 03/09/2019 à 15:43, Tom Lane a écrit : The planner should be able to do better than that, given up-to-date statistics on the "nodes" column. Tom, I can confirm that with up to date statistics the planner is still lost. I did a REINDEX t

Re: Slow statement using parallelism after 9.6>11 upgrade

2019-09-04 Thread Arnaud L.
Le 04/09/2019 à 09:04, Arnaud L. a écrit : Tom, I can confirm that with up to date statistics the planner is still lost. I did a REINDEX to rule out a broken index and the estimate is still in the 100k+ range. Sorry, I meant 1M+ range. EXPLAIN ANALYZE select id from planet_osm_ways WHERE

Re: Slow statement using parallelism after 9.6>11 upgrade

2019-09-04 Thread Arnaud L.
Le 03/09/2019 à 15:43, Tom Lane a écrit : "Arnaud L." writes: We have upgraded our database from 9.6 to 11 (and updated PostGIS from 2.3 to 2.5 as well). ... Have you re-ANALYZEd the database? The problem with this query seems to be the spectacularly awful rowcount est

Re: Slow statement using parallelism after 9.6>11 upgrade

2019-09-03 Thread Arnaud L.
Le 03/09/2019 à 17:02, Imre Samu a écrit : > We have upgraded our database from 9.6 to 11 This is now the latest PG ?   PG11.5? Yes, PostgreSQL 11.5, compiled by Visual C++ build 1914, 64-bit. - the osm2pgsql has an own parallelizations ...  ( osm2pgsql --number-processes .. )    so be

Re: Slow statement using parallelism after 9.6>11 upgrade

2019-09-03 Thread Arnaud L.
Le 03/09/2019 à 16:39, Paul Ramsey a écrit : On Sep 3, 2019, at 7:06 AM, Arnaud L. wrote: I've setup parallel_tuple_cost to 1.0 parallel_setup_cost to 5000.0 for the time being which solves this specific problem. These value don't look very sensible though, they are very high compared

Re: Slow statement using parallelism after 9.6>11 upgrade

2019-09-03 Thread Arnaud L.
Le 03/09/2019 à 15:43, Tom Lane a écrit : "Arnaud L." writes: We have upgraded our database from 9.6 to 11 (and updated PostGIS from 2.3 to 2.5 as well). ... Have you re-ANALYZEd the database? The problem with this query seems to be the spectacularly awful rowcount estimate h

Slow statement using parallelism after 9.6>11 upgrade

2019-09-03 Thread Arnaud L.
Hi list We have upgraded our database from 9.6 to 11 (and updated PostGIS from 2.3 to 2.5 as well). We are using it among other things to store an OpenStreetMap database, populated by the osm2pgsql program and updated on a daily basis. The process used to take ~1h30 minutes before the

Re: psql \copy hanging

2019-08-28 Thread Arnaud L.
Le 28/08/2019 à 09:43, Luca Ferrari a écrit : I don't want to be pedantic, but I would have tried with a single change at a time. And my bet is: the local file would do the trick (i.e., it is a weird share problem). You're not don't worry. This process is quite important in our workflow (not

Re: psql \copy hanging

2019-08-28 Thread Arnaud L.
Le 27/08/2019 à 13:17, Arnaud L. a écrit : I move the offending line at the end of the script, so it will run some minutes later, maybe this will be enough. OK, so this was enough for last night's schedule to run without problem. I still don't get it so I'm not satisfied with this solution

Re: psql \copy hanging

2019-08-27 Thread Arnaud L.
Le 27/08/2019 à 13:13, Luca Ferrari a écrit : My personal experience with shares and network that are not so much stable is that they can block, and that is why I'm suggesting to try on the local filesystem to see if that is a share-related problem or a resource problem. Also scheduling the

Re: psql \copy hanging

2019-08-27 Thread Arnaud L.
Le 27/08/2019 à 10:57, Luca Ferrari a écrit : On Tue, Aug 27, 2019 at 10:48 AM Arnaud L. wrote: I can run the script just fine during working hours. I meant thru your scheduler (cron or something). Yes, it runs fine too. There is indeed something happening on the server at this specific

Re: psql \copy hanging

2019-08-27 Thread Arnaud L.
Le 27/08/2019 à 10:00, Luca Ferrari a écrit : On Tue, Aug 27, 2019 at 9:54 AM Arnaud L. wrote: Any other idea ? I'll change the lines order for tonight's run, but that is not what I'd call a solution... Does it hangs against the same line content or the same line number? Are you able to run

Re: psql \copy hanging

2019-08-27 Thread Arnaud L.
Le 22/08/2019 à 10:47, Arnaud L. a écrit : On Windows, I have an unattended script that exports data overnight by issuing psql commands like : psql -f mycommands.sql whith mycommands.sql containing a bunch of \copy lines like : \copy (SELECT * FROM someview) TO '\\windowsserver\windowsshare

Re: psql \copy hanging

2019-08-26 Thread Arnaud L.
Le 22/08/2019 à 16:23, Arnaud L. a écrit : Le 22/08/2019 à 16:21, Adrian Klaver a écrit : The only thing I can think of at the moment is to put the offending line in a separate script and see what happens. Then I think I'll first try to switch the line orders. I'll execute this action last

Re: psql \copy hanging

2019-08-23 Thread Arnaud L.
Le 22/08/2019 à 20:00, Jerry Sievers a écrit : Is the overnight run being done as the same DB user you are testing with? If not, then perhaps the automated user deviates in resource settings and/or permissions, in turn resulting in sluggish performance caused by thrashing and/or inferior query

Re: psql \copy hanging

2019-08-22 Thread Arnaud L.
Le 22/08/2019 à 16:21, Adrian Klaver a écrit : The only thing I can think of at the moment is to put the offending line in a separate script and see what happens. Then I think I'll first try to switch the line orders. I'll execute this action last and see if it changes anything. If it

Re: psql \copy hanging

2019-08-22 Thread Arnaud L.
Le 22/08/2019 à 16:00, Adrian Klaver a écrit : The first thing I think of when seeing this sort of behavior on Windows is anti-virus software. Doing you have any running on the machine? You say it always hangs on the same line. Is that line doing something different from the others? Say writing

Re: Upgrading PostgreSQL under Windows

2019-03-27 Thread Arnaud L.
Le 27/03/2019 à 07:02, Kumar Prince NCS a écrit : We are  Upgrading PostgreSQL under Windows from v9.3 to 9.5. Following steps from https://www.postgresql.org/docs/10/pgupgrade.html We are using PostgreSQL database for confluence application. While running Step 9 , there is command to set

Re: How to compare dates from two tables with blanks values

2018-12-19 Thread Arnaud L.
Le 19/12/2018 à 11:41, Mike Martin a écrit : cast(nullif(d.joineddate,NULL) as timestamp) != cast(nullif(s.joineddate,'') as timestamp) Try with d.joineddate IS DISTINCT FROM s.joineddate https://www.postgresql.org/docs/current/functions-comparison.html Cheers -- Arnaud

Re: BDR and PostgreSQL 12 and Windows support

2018-11-16 Thread Arnaud L.
Le 16/11/2018 à 15:28, Ravi Krishna a écrit : A better question should be, are there any production users of PG on Windows :-) And the answer is yes, there are some ! Cheers -- Arnaud

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-26 Thread Arnaud L.
Le 25-06-18 à 18:47, chiru r a écrit : Please suggest Schema/Data conversion opensource tools from MySQL to PostgreSQL. Hi. I used this php script which did a pretty good job : https://github.com/AnatolyUss/FromMySqlToPostgreSql -- Regards

Re: VBA to connect to postgresql from MS Access

2018-06-01 Thread Arnaud L.
Le 01-06-18 à 14:21, Łukasz Jarych a écrit : ODBC;DSN=PostgreSQL35W;DATABASE=AccessTest;SERVER=localhost;PORT=5432;*CA=d;A7=100;B0=255;B1=8190;BI=0;C2=;CX=1c305008b;A1=7.4* When i was searching code for this in internet i found only: DRIVER={PostgreSQL