Re: Replaying the mysqld.log file from production onto QA???

[Baron Schwartz]

The mysqld.log isn't in an executable format, but mk-query-digest can
understand it and convert it into a "slow query log" format.  Then you
can use mk-upgrade to validate that your queries produce the same
result.  Good for you for thinking to do this -- most people just
upgrade and then panic when something doesn't work right.

On Wed, Sep 8, 2010 at 2:22 PM, Nunzio Daveri  wrote:
> Hello all, I am trying to "REPLAY" the mysqld.log file from a production 5.1.4
> server onto a QA server with 5.5 running and was wondering if there is an easy
> way of replaying the mysqld.log file?  Just want to make sure all of the kinds
> of inserts, updates, selects and deletes work just as well on the 5.5 box, esp
> since we are turning on replication and copying tables from another server 
> onto
> this server.  Any ideas please?
>
> TIA...
>
> Nunzio
>
>
>
>



-- 
Baron Schwartz
Percona Inc 
Consulting, Training, Support & Services for MySQL

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org

Re: Replaying the mysqld.log file from production onto QA???

[Johnny Withers]

Would something like: http://www.maatkit.org/doc/mk-log-player.html

Work? I know the timings
won't be the same as how the queries actually came in, but  it could be
a start,

JW


On Fri, Sep 10, 2010 at 8:38 AM, Johan De Meersman wrote:

> On Fri, Sep 10, 2010 at 1:56 PM, Shawn Green (MySQL) <
> shawn.l.gr...@oracle.com> wrote:
>
> > He already did! Those are the logs he needs to replay. He has the logs
> > already but needs tools to extract the commands and repeat them as a load
> > test.
> >
> > Do you have any techniques you can share?
> >
>
> Alas, no. I suspect the rather expensive Avalanches we got might be able
> to,
> but that's not particularly helpful to him :-)
>
> I wrote a little thing many years ago that pulled Apache logs from
> production over SSH and multithreaded them at the staging server, but SQL
> stuff would be much more time-critical - can't select before the insert :-)
>
>
>
> --
> Bier met grenadyn
> Is als mosterd by den wyn
> Sy die't drinkt, is eene kwezel
> Hy die't drinkt, is ras een ezel
>



-- 
-
Johnny Withers
601.209.4985
joh...@pixelated.net

RE: Replaying the mysqld.log file from production onto QA???

[Jerry Schwartz]

If possible, use web server logging to get the timings.

As I explained in my previous essay, this is not at all simple or easy.

If you run a benchmark more than once, be sure to reboot the system between 
runs.

Regards,

Jerry Schwartz
Global Information Incorporated
195 Farmington Ave.
Farmington, CT 06032

860.674.8796 / FAX: 860.674.8341
E-mail: je...@gii.co.jp
Web site: www.the-infoshop.com

>-Original Message-
>From: Shawn Green (MySQL) [mailto:shawn.l.gr...@oracle.com]
>Sent: Friday, September 10, 2010 7:56 AM
>To: Johan De Meersman
>Cc: Nunzio Daveri; andrew.2.mo...@nokia.com; mysql@lists.mysql.com
>Subject: Re: Replaying the mysqld.log file from production onto QA???
>
>Hello Johan,
>
>On 9/10/2010 7:47 AM, Johan De Meersman wrote:
>> On Thu, Sep 9, 2010 at 4:12 PM, Nunzio Daveri 
>> wrote:
>>
>>> So.. I am trying to mimic replaying production like queries so joins, temp
>>> tables etc... are stuff I am trying to test as well.  Just doing a dump 
>>> and
>>> import is no more than export and importing, I also want to test selects,
>>> updates :-)  Thanks for replying :-)
>>>
>>
>>
>> Then you'll have to first activate full logging on your production server.
>> This may take quite some diskpace, and IOs, so keep the logfiles on 
>> separate
>> spindles.
>>
>> Extracting the queries from the log shouldn't be all that hard; but there's
>> too little timing information in there to do a time-true replay - and I
>> don't know of any app that does that, either.
>>
>
>He already did! Those are the logs he needs to replay. He has the logs
>already but needs tools to extract the commands and repeat them as a
>load test.
>
>Do you have any techniques you can share?
>
>--
>Shawn Green
>MySQL Principal Technical Support Engineer
>Oracle USA, Inc.
>Office: Blountville, TN
>
>--
>MySQL General Mailing List
>For list archives: http://lists.mysql.com/mysql
>To unsubscribe:http://lists.mysql.com/mysql?unsub=je...@gii.co.jp





-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org

Re: Replaying the mysqld.log file from production onto QA???

[Johan De Meersman]

On Fri, Sep 10, 2010 at 1:56 PM, Shawn Green (MySQL) <
shawn.l.gr...@oracle.com> wrote:

> He already did! Those are the logs he needs to replay. He has the logs
> already but needs tools to extract the commands and repeat them as a load
> test.
>
> Do you have any techniques you can share?
>

Alas, no. I suspect the rather expensive Avalanches we got might be able to,
but that's not particularly helpful to him :-)

I wrote a little thing many years ago that pulled Apache logs from
production over SSH and multithreaded them at the staging server, but SQL
stuff would be much more time-critical - can't select before the insert :-)



-- 
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

Re: Replaying the mysqld.log file from production onto QA???

[Shawn Green (MySQL)]

Hello Johan,

On 9/10/2010 7:47 AM, Johan De Meersman wrote:

On Thu, Sep 9, 2010 at 4:12 PM, Nunzio Daveri wrote:


So.. I am trying to mimic replaying production like queries so joins, temp
tables etc... are stuff I am trying to test as well.  Just doing a dump and
import is no more than export and importing, I also want to test selects,
updates :-)  Thanks for replying :-)




Then you'll have to first activate full logging on your production server.
This may take quite some diskpace, and IOs, so keep the logfiles on separate
spindles.

Extracting the queries from the log shouldn't be all that hard; but there's
too little timing information in there to do a time-true replay - and I
don't know of any app that does that, either.



He already did! Those are the logs he needs to replay. He has the logs 
already but needs tools to extract the commands and repeat them as a 
load test.


Do you have any techniques you can share?

--
Shawn Green
MySQL Principal Technical Support Engineer
Oracle USA, Inc.
Office: Blountville, TN

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org

Re: Replaying the mysqld.log file from production onto QA???

[Johan De Meersman]

On Thu, Sep 9, 2010 at 4:12 PM, Nunzio Daveri wrote:

> So.. I am trying to mimic replaying production like queries so joins, temp
> tables etc... are stuff I am trying to test as well.  Just doing a dump and
> import is no more than export and importing, I also want to test selects,
> updates :-)  Thanks for replying :-)
>


Then you'll have to first activate full logging on your production server.
This may take quite some diskpace, and IOs, so keep the logfiles on separate
spindles.

Extracting the queries from the log shouldn't be all that hard; but there's
too little timing information in there to do a time-true replay - and I
don't know of any app that does that, either.




>
> Nunzio
>
>
>
>
> 
> From: "andrew.2.mo...@nokia.com" 
> To: nunziodav...@yahoo.com; mysql@lists.mysql.com
> Sent: Wed, September 8, 2010 4:19:19 PM
> Subject: Re: Replaying the mysqld.log file from production onto QA???
>
>  Try using a mysqldump.
>
>
> - Reply message -
> From: "ext Nunzio Daveri" 
> Date: Wed, Sep 8, 2010 19:25
> Subject: Replaying the mysqld.log file from production onto QA???
> To: "mysql@lists.mysql.com" 
>
> Hello all, I am trying to "REPLAY" the mysqld.log file from a production
> 5.1.4
> server onto a QA server with 5.5 running and was wondering if there is an
> easy
> way of replaying the mysqld.log file?  Just want to make sure all of the
> kinds
> of inserts, updates, selects and deletes work just as well on the 5.5 box,
> esp
> since we are turning on replication and copying tables from another server
> onto
> this server.  Any ideas please?
>
> TIA...
>
> Nunzio
>
>
>
>



-- 
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

Re: Replaying the mysqld.log file from production onto QA???

[Nunzio Daveri]

So.. I am trying to mimic replaying production like queries so joins, temp 
tables etc... are stuff I am trying to test as well.  Just doing a dump and 
import is no more than export and importing, I also want to test selects, 
updates :-)  Thanks for replying :-)

Nunzio





From: "andrew.2.mo...@nokia.com" 
To: nunziodav...@yahoo.com; mysql@lists.mysql.com
Sent: Wed, September 8, 2010 4:19:19 PM
Subject: Re: Replaying the mysqld.log file from production onto QA???

 Try using a mysqldump.


- Reply message -
From: "ext Nunzio Daveri" 
Date: Wed, Sep 8, 2010 19:25
Subject: Replaying the mysqld.log file from production onto QA???
To: "mysql@lists.mysql.com" 

Hello all, I am trying to "REPLAY" the mysqld.log file from a production 5.1.4 
server onto a QA server with 5.5 running and was wondering if there is an easy 
way of replaying the mysqld.log file?  Just want to make sure all of the kinds 
of inserts, updates, selects and deletes work just as well on the 5.5 box, esp 
since we are turning on replication and copying tables from another server onto 
this server.  Any ideas please?

TIA...

Nunzio


  

Re: Replaying the mysqld.log file from production onto QA???

[andrew.2.moore]

Try using a mysqldump.

- Reply message -
From: "ext Nunzio Daveri" 
Date: Wed, Sep 8, 2010 19:25
Subject: Replaying the mysqld.log file from production onto QA???
To: "mysql@lists.mysql.com" 

Hello all, I am trying to "REPLAY" the mysqld.log file from a production 5.1.4
server onto a QA server with 5.5 running and was wondering if there is an easy
way of replaying the mysqld.log file?  Just want to make sure all of the kinds
of inserts, updates, selects and deletes work just as well on the 5.5 box, esp
since we are turning on replication and copying tables from another server onto
this server.  Any ideas please?

TIA...

Nunzio