Fwd: iscsi performance

2019-04-17 Thread Mahmoud Moharam
-- Forwarded message - From: Mahmoud Moharam Date: Mon, Apr 15, 2019 at 5:02 PM Subject: iscsi performance To: Pgsql-admin Dear all I have a postgres db have a big data so I seek for any enhancement in performance and I think for iscsi disks so the question is :- is iscsi will i

Re: Out of Memory errors are frustrating as heck!

2019-04-17 Thread Gunther
Hi guys. I don't want to be pushy, but I found it strange that after so much lively back and forth getting to the bottom of this, suddenly my last nights follow-up remained completely without reply. I wonder if it even got received. For those who read their emails with modern readers (I know I

AW: Best Filesystem for PostgreSQL

2019-04-17 Thread Stephan Schmidt
my Question was meant for a Debian 9 environment with heavy read/wright load and very high requirements towards Performance and data Consistency Stephan Von: Bruce Momjian Gesendet: Wednesday, April 17, 2019 11:07:21 PM An: Stephan Schmidt Cc: pgsql-performa

Re: Best Filesystem for PostgreSQL

2019-04-17 Thread Gunther Schadow
On 4/17/2019 18:03, Imre Samu wrote: test:    PostgreSQL v10.3 + Linux 5.0 File-System Benchmarks: Btrfs vs. EXT4 vs. F2FS vs. XFS https://www.phoronix.com/scan.php?page=article&item=linux-50-filesystems&num=3 So looks like XFS won. I like XFS for its ease of use especially when growing. Any

Re: Best Filesystem for PostgreSQL

2019-04-17 Thread Imre Samu
> best filesystem to run PostgreSQL on, in Terms of Performance ... test:PostgreSQL v10.3 + Linux 5.0 File-System Benchmarks: Btrfs vs. EXT4 vs. F2FS vs. XFS https://www.phoronix.com/scan.php?page=article&item=linux-50-filesystems&num=3 Imre Stephan Schmidt ezt írta (időpont: 2019. ápr. 1

Re: Best Filesystem for PostgreSQL

2019-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2019 at 08:59:13PM +, Stephan Schmidt wrote: > What would be the best filesystem to run PostgreSQL on, in Terms of > Performance > and data Integrity? Uh, which operating system? If it is Linux, many people like ext4 or xfs. Some like zfs. ext3/ext2 are not recommended due

Best Filesystem for PostgreSQL

2019-04-17 Thread Stephan Schmidt
What would be the best filesystem to run PostgreSQL on, in Terms of Performance and data Integrity? Best regards, stephan

Re: Block / Page Size Optimization

2019-04-17 Thread Bruce Momjian
On Mon, Apr 15, 2019 at 06:19:06PM +0200, Tomas Vondra wrote: > On Mon, Apr 08, 2019 at 11:09:07AM -0400, Gunther wrote: > > What really is the theoretical issue with the file system block size? > > Where does -- in theory -- the benefit come from of using an XFS block > > size of 8 kB, or even inc

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Laurenz Albe
laurent.decha...@orange.com wrote: > There is something in documentation that says that there won't be parallelism > if " The client sends an Execute message with a non-zero fetch count." > I am not sure what this sentence means. The JDBC driver sends an "Execute" message to the server. https://ww

RE: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread laurent.dechambe
Hello Justin and thank you for your clues. Finally I found that putting blank to the option that limits the number of rows to retrieve (which is normal for this kind of tool) allows PostgreSQL to parallelize the query. On jdbc it seems this is equivalent to write : statement. setMaxRows(0); //

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Justin Pryzby
On Wed, Apr 17, 2019 at 09:51:02AM +, laurent.decha...@orange.com wrote: > > 2019-04-17 11:30:42 CEST;35895;thedbuser;thedb;0;LOG: 0: execute > : SELECT COUNT(1) FROM big_table > 2019-04-17 11:30:42 CEST;35895;thedbuser;thedb;0;LOCATION: > exec_execute_message, postgres.c:1959

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Gunther Schadow
On 4/17/2019 4:33, Thomas Kellerer wrote: A CTE would prevent parallelism. You mean like always? His SELECT count(1) FROM BigTable would be parallel if run alone but as WITH Data AS (SELECT count(1) FROM BigTable) SELECT * FROM Data nothing would be parallel any more? How about: SELECT * F

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Gunther Schadow
By the way On 4/17/2019 7:26, laurent.decha...@orange.com wrote: I can see whether there is parallelism with pg_top or barely top on the server. PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 38584 postgres 20 0 8863828 8.153g 8.151g R 100.0 3.2 1:23.01 p

RE: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread laurent.dechambe
Auto explain shows that in both cases there are workers planned, but with DBeaver they are not launched. Here's what I get with auto_explain : 2019-04-17 14:46:09 CEST;54882;thedbuser;thedb;0;LOG: 0: duration: 0.095 ms 2019-04-17 14:46:09 CEST;54882;thedbuser;thedb;0;LOCATION:

RE: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread laurent.dechambe
I can see whether there is parallelism with pg_top or barely top on the server. PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 38584 postgres 20 0 8863828 8.153g 8.151g R 100.0 3.2 1:23.01 postgres 10 root 20 0 0 0 0 S 0.3 0.0 8

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Andreas Kretschmer
Am 17.04.19 um 11:51 schrieb laurent.decha...@orange.com: Here are the logs (with log_error_verbosity = verbose) : 2019-04-17 11:30:42 CEST;35895;thedbuser;thedb;0;LOG:  0: execute : SELECT COUNT(1) FROM big_table 2019-04-17 11:30:42 CEST;35895;thedbuser;thedb;0;LOCATION: e

RE: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread laurent.dechambe
As answered to Andreas Kretschmer all settings are identical. I have made some other tests, even testing a basic jdbc program (open connection, execute statement, display result, close connection) Here are the logs (with log_error_verbosity = verbose) : 2019-04-17 11:30:42 CEST;35895;thedbuser

RE: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread laurent.dechambe
Thanks for the tip. I have compared all settings and they are identical. Very strange. -Message d'origine- De : Andreas Kretschmer [mailto:andr...@a-kretschmer.de] Envoyé : mercredi 17 avril 2019 10:07 À : pgsql-performance@lists.postgresql.org Objet : Re: Pg10 : Client Configuration for

Sv: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Andreas Joseph Krogh
På onsdag 17. april 2019 kl. 08:30:28, skrev mailto:laurent.decha...@orange.com>>: Hi, I am working on PostgreSQL 10.5 and I have a discrepancy between clients regarding parallelism feature. For a simple query (say a simple SELECT COUNT(*) FROM BIG_TABLE), I can see PostgreSQL use parallelism

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Thomas Kellerer
laurent.decha...@orange.com schrieb am 17.04.2019 um 08:30: > I am working on PostgreSQL 10.5 and I have a discrepancy between clients > regarding parallelism feature. > > For a simple query (say a simple SELECT COUNT(*) FROM BIG_TABLE), I > can see PostgreSQL use parallelism when the query is la

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Andreas Kretschmer
Am 17.04.19 um 08:30 schrieb laurent.decha...@orange.com: SELECT current_setting('max_parallel_workers_per_gather') gives 10 from my session. Is there a client configuration that prevents from using parallelism ? unlikely. if i were you, i would compare all settings, using the different c

Re: Out of Memory errors are frustrating as heck!

2019-04-17 Thread Gavin Flower
On 17/04/2019 18:01, Jean-David Beyer wrote: On 4/16/19 6:39 PM, Gavin Flower wrote: I suspect that most things will run a little better with some swap space. Not always. $ free total used free shared buffers cached Mem: 16254616 13120960 3133656 20