Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

Thanks Yves for the clarification!

It used to be very important to pre-warm EBS before running benchmarks
in order to get consistent results.

Then at re:Invent 2015, the AWS engineers said that it is not needed
anymore, which IMO is a lot less work for us to do benchmarking in
AWS, because pre-warming a multi-TB EBS vol is very time consuming,
and the I/Os were not free.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html


On Thu, May 26, 2016 at 11:41 AM, Yves Dorfsman  wrote:
> On 2016-05-26 09:03, Artem Tomyuk wrote:
>> Why no? Or you missed something?
>
> I think Rayson is correct, but the double negative makes it hard to read:
>
> "So no EBS pre-warming does not apply to EBS volumes created from snapshots."
>
> Which I interpret as:
> So, "no EBS pre-warming", does not apply to EBS volumes created from 
> snapshots.
>
> Which is correct, you sitll have to warm your EBS when created from sanpshots 
> (to get the data from S3 to the filesystem).
>
>
> --
> http://yves.zioup.com
> gpg: 4096R/32B0F416
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

Thanks Artem.

So no EBS pre-warming does not apply to EBS volumes created from snapshots.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html


On Thu, May 26, 2016 at 10:52 AM, Artem Tomyuk <ad...@leboutique.com> wrote:
> Please look at the official doc.
>
> "New EBS volumes receive their maximum performance the moment that they are
> available and do not require initialization (formerly known as pre-warming).
> However, storage blocks on volumes that were restored from snapshots must be
> initialized (pulled down from Amazon S3 and written to the volume) before
> you can access the block"
>
> Quotation from:
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html
>
> 2016-05-26 17:47 GMT+03:00 Rayson Ho <raysonlo...@gmail.com>:
>>
>> On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk <ad...@leboutique.com>
>> wrote:
>>>
>>>
>>> 2016-05-26 16:50 GMT+03:00 Rayson Ho <raysonlo...@gmail.com>:
>>>>
>>>> Amazon engineers said that EBS pre-warming is not needed anymore.
>>>
>>>
>>> but still if you will skip this step you wont get much performance on ebs
>>> created from snapshot.
>>
>>
>>
>> IIRC, that's not what Amazon engineers said. Is that from your personal
>> experience, and if so, when did you do the test??
>>
>> Rayson
>>
>> ==
>> Open Grid Scheduler - The Official Open Source Grid Engine
>> http://gridscheduler.sourceforge.net/
>> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>>
>>
>>
>


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk <ad...@leboutique.com> wrote:

>
> 2016-05-26 16:50 GMT+03:00 Rayson Ho <raysonlo...@gmail.com>:
>
>> Amazon engineers said that EBS pre-warming is not needed anymore.
>
>
> but still if you will skip this step you wont get much performance on ebs
> created from snapshot.
>


IIRC, that's not what Amazon engineers said. Is that from your personal
experience, and if so, when did you do the test??

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

On Thu, May 26, 2016 at 9:00 AM, Artem Tomyuk <ad...@leboutique.com> wrote:
>
> But still strong recommendation to pre-warm your ebs in any case,
especially if they created from snapshot.

That used to be true. However, at AWS re:Invent 2015, Amazon engineers said
that EBS pre-warming is not needed anymore.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



> 2016-05-26 15:53 GMT+03:00 Yves Dorfsman <y...@zioup.com>:
>>
>> On 2016-05-25 19:08, Rayson Ho wrote:
>> > Actually, when "EBS-Optimized" is on, then the instance gets dedicated
>> > bandwidth to EBS.
>>
>> Hadn't realised that, thanks.
>> Is the EBS bandwidth then somewhat limited depending on the type of
instance too?
>>
>> --
>> http://yves.zioup.com
>> gpg: 4096R/32B0F416
>>
>>
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>
>

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

Actually, when "EBS-Optimized" is on, then the instance gets dedicated
bandwidth to EBS.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



On Wed, May 25, 2016 at 7:56 PM, Yves Dorfsman <y...@zioup.com> wrote:

> Indeed, old-style disk EBS vs new-style SSd EBS.
>
> Be aware that EBS traffic is considered as part of the total "network"
> traffic, and each type of instance has different limits on maximum network
> throughput. Those difference are very significant, do tests on the same
> volume
> between two different type of instances, both with enough cpu and memory
> for
> the I/O to be the bottleneck, you will be surprised!
>
>
> On 2016-05-25 17:02, Rayson Ho wrote:
> > There are many factors that can affect EBS performance. For example, the
> type
> > of EBS volume, the instance type, whether EBS-optimized is turned on or
> not, etc.
> >
> > Without the details, then there is no apples to apples comparsion...
> >
> > Rayson
> >
> > ==
> > Open Grid Scheduler - The Official Open Source Grid Engine
> > http://gridscheduler.sourceforge.net/
> > http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> >
> >
> >
> > On Wed, May 25, 2016 at 6:34 PM, Tory M Blue <tmb...@gmail.com
> > <mailto:tmb...@gmail.com>> wrote:
> >>
> >> We are starting some testing in AWS, with EC2, EBS backed setups.
> >>
> >> What I found interesting today, was a single EBS 1TB volume, gave me
> >> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
> >> volumes), gave me something like 31MB/s (test after test after test).
> >>
> >> I'm wondering what you folks are using inside of Amazon (not
> >> interested in RDS at the moment).
> >>
> >> Thanks
> >> Tory
> >>
> >>
> >> --
> >> Sent via pgsql-performance mailing list (
> pgsql-performance@postgresql.org
> > <mailto:pgsql-performance@postgresql.org>)
> >> To make changes to your subscription:
> >> http://www.postgresql.org/mailpref/pgsql-performance
>
>
> --
> http://yves.zioup.com
> gpg: 4096R/32B0F416
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

There are many factors that can affect EBS performance. For example, the
type of EBS volume, the instance type, whether EBS-optimized is turned on
or not, etc.

Without the details, then there is no apples to apples comparsion...

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



On Wed, May 25, 2016 at 6:34 PM, Tory M Blue  wrote:
>
> We are starting some testing in AWS, with EC2, EBS backed setups.
>
> What I found interesting today, was a single EBS 1TB volume, gave me
> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
> volumes), gave me something like 31MB/s (test after test after test).
>
> I'm wondering what you folks are using inside of Amazon (not
> interested in RDS at the moment).
>
> Thanks
> Tory
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

[PERFORM] cache false-sharing in lwlocks

[Rayson Ho]

Hi,

LWLockPadded is either 16 or 32 bytes, so modern systems (e.g. Core2
or AMD Opteron [1]) with cacheline size of 64 bytes can get
false-sharing in lwlocks.

I changed LWLOCK_PADDED_SIZE in src/backend/storage/lmgr/lwlock.c to
64, and ran sysbench OLTP read-only benchmark, and got a slight
improvement in throughput:

Hardware: single-socket Core 2, quad-core, Q6600  @ 2.40GHz
Software: Linux 2.6.28-17, glibc 2.9, gcc 4.3.3

PostgreSQL: 8.5alpha3

sysbench parameters: sysbench --num-threads=4 --max-requests=0
--max-time=120 --oltp-read-only=on --test=oltp

original: 3227, 3243, 3243
after: 3256, 3255, 3253

So there is a speedup of 1.005 or what other people usually call it, a
0.5% improvement.

However, it's a single socket machine, so all the cache traffic does
not need to go off-chip. Can someone with a multi-socket machine help
me run some test so that we can get a better idea of how this change
(patch attached) performs in bigger systems??

Thanks,
Rayson

P.S. And I just googled and found similar discussions about padding
LWLOCK_PADDED_SIZE, but the previous work was done on an IBM POWER
system, and the benchmark used was apachebench. IMO, the setup was too
complex to measure a small performance improvement in PostgreSQL.

[1] Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™ ccNUMA
Multiprocessor Systems Application Note
Index: src/backend/storage/lmgr/lwlock.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/storage/lmgr/lwlock.c,v
retrieving revision 1.55
diff -u -4 -r1.55 lwlock.c
--- src/backend/storage/lmgr/lwlock.c   2 Jan 2010 16:57:52 -   1.55
+++ src/backend/storage/lmgr/lwlock.c   11 Jan 2010 16:40:03 -
@@ -57,9 +57,11 @@
  *
  * LWLock is between 16 and 32 bytes on all known platforms, so these two
  * cases are sufficient.
  */
-#define LWLOCK_PADDED_SIZE (sizeof(LWLock) = 16 ? 16 : 32)
+/* #define LWLOCK_PADDED_SIZE  (sizeof(LWLock) = 16 ? 16 : 32) */
+
+#define LWLOCK_PADDED_SIZE 64
 
 typedef union LWLockPadded
 {
LWLock  lock;

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance