Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-06-04 Thread Andres Freund
Hi Dave, Ted, All, On 2014-05-23 16:42:47 +1000, Dave Chinner wrote: > On Tue, Apr 29, 2014 at 01:57:14AM +0200, Andres Freund wrote: > > Hi Dave, > > > > On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: > > > ping? > > > > I'd replied at http://marc.info/?l=linux-mm&m=139730910307321&w=2 > >

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-05-22 Thread Dave Chinner
On Tue, Apr 29, 2014 at 01:57:14AM +0200, Andres Freund wrote: > Hi Dave, > > On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: > > ping? > > I'd replied at http://marc.info/?l=linux-mm&m=139730910307321&w=2 I missed it, sorry. I've had a bit more time to look at this behaviour now and tweaked

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-04-28 Thread Andres Freund
Hi Dave, On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: > ping? I'd replied at http://marc.info/?l=linux-mm&m=139730910307321&w=2 As an additional note: > On Wed, Apr 09, 2014 at 07:20:09PM +1000, Dave Chinner wrote: > > I'm not sure how you were generating the behaviour you reported, but >

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-04-28 Thread Dave Chinner
ping? On Wed, Apr 09, 2014 at 07:20:09PM +1000, Dave Chinner wrote: > On Wed, Mar 26, 2014 at 08:11:13PM +0100, Andres Freund wrote: > > Hi, > > > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > > top#1 concern is frequent slow read()s that happen while another proces

Re: Postgresql performance problems with IO latency, especially during fsync()

2014-04-12 Thread Andres Freund
Hi Dave, On 2014-04-09 19:20:09 +1000, Dave Chinner wrote: > On Wed, Mar 26, 2014 at 08:11:13PM +0100, Andres Freund wrote: > > So, the average read time is less than one ms (SSD, and about 50% cached > > workload). But once another backend does the fsync(), read latency > > skyrockets. > > > > A

Re: Postgresql performance problems with IO latency, especially during fsync()

2014-04-09 Thread Dave Chinner
On Wed, Mar 26, 2014 at 08:11:13PM +0100, Andres Freund wrote: > Hi, > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > top#1 concern is frequent slow read()s that happen while another process > calls fsync(), even though we'd be perfectly fine if that fsync() took > ag

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Fernando Luis Vazquez Cao
(2014/03/28 0:50), Jan Kara wrote: On Wed 26-03-14 22:55:18, Andres Freund wrote: On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Jan Kara
Hello, On Wed 26-03-14 20:11:13, Andres Freund wrote: > At LSF/MM there was a slot about postgres' problems with the kernel. Our > top#1 concern is frequent slow read()s that happen while another process > calls fsync(), even though we'd be perfectly fine if that fsync() took > ages. > The "conc

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Jan Kara
On Wed 26-03-14 22:55:18, Andres Freund wrote: > On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: > > On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: > > > Hi, > > > > > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > > > top#1 concern is frequent slow read(

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 4:11 PM, Andy Lutomirski wrote: > On Wed, Mar 26, 2014 at 3:35 PM, David Lang wrote: >> On Wed, 26 Mar 2014, Andy Lutomirski wrote: >> > I'm not sure I understand the request queue stuff, but here's an idea. > The block core contains this little bit of code:

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 3:35 PM, David Lang wrote: > On Wed, 26 Mar 2014, Andy Lutomirski wrote: > I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little bit of code: >>> >>> >>> I haven't read enough of the code yet, to comment intell

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread David Lang
On Wed, 26 Mar 2014, Andy Lutomirski wrote: I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little bit of code: I haven't read enough of the code yet, to comment intelligently ;) My little patch doesn't seem to help. I'm either changing t

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 2:55 PM, Andres Freund wrote: > On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: >> On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: >> > Hi, >> > >> > At LSF/MM there was a slot about postgres' problems with the kernel. Our >> > top#1 concern is frequent slow re

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andres Freund
On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: > On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: > > Hi, > > > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > > top#1 concern is frequent slow read()s that happen while another process > > calls fsync(), eve

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: > Hi, > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > top#1 concern is frequent slow read()s that happen while another process > calls fsync(), even though we'd be perfectly fine if that fsync() took > ages. > Th

Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andres Freund
Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent slow read()s that happen while another process calls fsync(), even though we'd be perfectly fine if that fsync() took ages. The "conclusion" of that part was that it'd be very useful to have a de