Re: UDF behaves non-deterministic
Hi everybody, it looks like the reason for the problem was me not handling string arguments properly (I did not use the provided lengths, but relied on string being null-terminated, it's in the doc, but ...). It seems this became a problem specifically in the parallel situation, misleading me into believing it had to do with this. Thanks to everybody for help! Even though the solution was not directly provided, the comments made me think about my code and so were still helpfull. Stefan On Monday 05 November 2012 15:08:51 Michael Dykman wrote: > C is not an inherently thread-safe language. Several of the standard > library functions use static data, which gets stepped on during concurrent > operation. Many of those do have thread-safe equivalents on many platforms > such as strtok/strtok_r (the latter being the safe one). > > If you are confident you are not using statics or globals in your code > directly, you will need to identify each function you do call. Start by > reading the man page for that function (if it's in the C stdlib, there is a > man page for it) which should tell you if it is safe or not; for those > which are not, the man page will likely suggest a threadsafe alternative if > one is available. If none are available, you might have to consider a > mutex. > > - michael dykman > > On Mon, Nov 5, 2012 at 9:28 AM, Stefan Kuhn wrote: > > Hi Dan, > > > > thanks for your answer. The UDF only contains functions (the one called > > in sql plus two functions called in it). There are no variables outside > > them and nothing is declared static. All variables inside the functions > > are declared just like "double x=0;" etc. I am not an expert on C, but my > > understanding is that these values are separate for each call of the > > function and don't influence each other. Do you have a suggestion what I > > should look for in my c code? Or do I need to make the code thread-safe > > in that sense that concurrent executions are prevented by monitors or > > semaphors or so (no idea about what this is called in c)? > > Stefan > > > > >The first thing I would do is examine your UDF and ensure that it is > > >thread-safe. No global variables, no static variables within functions, > > >etc. Also make sure that any libc functions you call that are documented > > > > as > > > > >non-threadsafe are wrapped by a mutex or otherwise protected against > > >multiple simultaneous access. > > > > > >http://dev.mysql.com/doc/refman/5.5/en/adding-udf.html > > > > > >As for debugging, you should be able to write things to stderr which > > > will show up in the mysql logfile, or you could open your own logfile > > > and write to that. > > > > -- > > Dan Nelson > > dnel...@allantgroup.com > > > > > > > > > > -- > > MySQL General Mailing List > > For list archives: http://lists.mysql.com/mysql > > To unsubscribe:http://lists.mysql.com/mysql -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
RE: How to verify mysqldump files
A variant on that... 1. pre-validate slave's consistency using pt-table-checksum 2. dump slave, wipe clean, restore 3. RE-validate slave's consistency using pt-table-checksum > -Original Message- > From: Stillman, Benjamin [mailto:bstill...@limitedbrands.com] > Sent: Wednesday, November 07, 2012 7:09 AM > To: 'Gary'; mysql@lists.mysql.com > Subject: RE: How to verify mysqldump files > > In the past when I used mysqldump, I used a slave database for backups and > periodically testing restores. > > My process for testing: > - Stop the slave process (so the db doesn't get updated). > - Run the backup. > - Create restore_test database. > - Restore the backup to the restore_test database. > - Use mysqldbcompare to compare the two databases. > - Drop restore_test database. > - Start the slave process. > > I have this scripted so it just runs and emails me the results. > > Useful link: > http://dev.mysql.com/doc/workbench//en/mysqldbcompare.html > > > > > -Original Message- > From: Gary [mailto:listgj-my...@yahoo.co.uk] > Sent: Wednesday, November 07, 2012 7:52 AM > To: mysql@lists.mysql.com > Subject: How to verify mysqldump files > > Can anyone suggest how I could verify that the files created by mysqldump > are "okay"? They are being created for backup purposes, and the last thing > I want to do is find out that the backups themselves are in some way > corrupt. > > I know I can check the output of the command itself, but what if.. I don't > know... if there are problems with the disc it writes to, or something like > that. Is there any way to check whether the output file is "valid" in the > sense that it is complete and syntactically correct? > > -- > GaryPlease do NOT send me 'courtesy' replies off-list. > > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe:http://lists.mysql.com/mysql > > > > > Notice: This communication may contain privileged and/or confidential > information. If you are not the intended recipient, please notify the > sender by email, and immediately delete the message and any attachments > without copying or disclosing them. LBI may, for any reason, intercept, > access, use, and disclose any information that is communicated by or > through, or which is stored on, its networks, applications, services, and > devices. > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe:http://lists.mysql.com/mysql -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
RE: How to verify mysqldump files
In the past when I used mysqldump, I used a slave database for backups and periodically testing restores. My process for testing: - Stop the slave process (so the db doesn't get updated). - Run the backup. - Create restore_test database. - Restore the backup to the restore_test database. - Use mysqldbcompare to compare the two databases. - Drop restore_test database. - Start the slave process. I have this scripted so it just runs and emails me the results. Useful link: http://dev.mysql.com/doc/workbench//en/mysqldbcompare.html -Original Message- From: Gary [mailto:listgj-my...@yahoo.co.uk] Sent: Wednesday, November 07, 2012 7:52 AM To: mysql@lists.mysql.com Subject: How to verify mysqldump files Can anyone suggest how I could verify that the files created by mysqldump are "okay"? They are being created for backup purposes, and the last thing I want to do is find out that the backups themselves are in some way corrupt. I know I can check the output of the command itself, but what if.. I don't know... if there are problems with the disc it writes to, or something like that. Is there any way to check whether the output file is "valid" in the sense that it is complete and syntactically correct? -- GaryPlease do NOT send me 'courtesy' replies off-list. -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql Notice: This communication may contain privileged and/or confidential information. If you are not the intended recipient, please notify the sender by email, and immediately delete the message and any attachments without copying or disclosing them. LBI may, for any reason, intercept, access, use, and disclose any information that is communicated by or through, or which is stored on, its networks, applications, services, and devices. -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
Re: How to verify mysqldump files
2012/11/7 Ananda Kumar > you can use checksum to make sure there are not corruption in the file That would work for the file integrity itself not for the data integrity _in_ the file. As Claudio suggested, probably going thru the whole recovery process from time to time is the best way to make sure the backup'ed data is correct. Manuel.
Re: How to verify mysqldump files
you can use checksum to make sure there are not corruption in the file On Wed, Nov 7, 2012 at 6:39 PM, Claudio Nanni wrote: > Gary, > > It is always a good practice to test the whole solution backup/restore. > So nothing is better than testing a restore, actually it should be a > periodic procedure. > As for the validity of the file usually is delegated to the operating > system. > If you want to check it yourself you may create an algorithm that analyses > some patterns in the dump file to recognize that it is correct, > starting may be from one that is working as 'valid' sample. > > Cheers > > Claudio > > > > 2012/11/7 Gary > > > Can anyone suggest how I could verify that the files created by > > mysqldump are "okay"? They are being created for backup purposes, and > > the last thing I want to do is find out that the backups themselves are > > in some way corrupt. > > > > I know I can check the output of the command itself, but what if.. I > > don't know... if there are problems with the disc it writes to, or > > something like that. Is there any way to check whether the output file > > is "valid" in the sense that it is complete and syntactically correct? > > > > -- > > GaryPlease do NOT send me 'courtesy' replies off-list. > > > > > > -- > > MySQL General Mailing List > > For list archives: http://lists.mysql.com/mysql > > To unsubscribe:http://lists.mysql.com/mysql > > > > > > > -- > Claudio >
Re: How to verify mysqldump files
Gary, It is always a good practice to test the whole solution backup/restore. So nothing is better than testing a restore, actually it should be a periodic procedure. As for the validity of the file usually is delegated to the operating system. If you want to check it yourself you may create an algorithm that analyses some patterns in the dump file to recognize that it is correct, starting may be from one that is working as 'valid' sample. Cheers Claudio 2012/11/7 Gary > Can anyone suggest how I could verify that the files created by > mysqldump are "okay"? They are being created for backup purposes, and > the last thing I want to do is find out that the backups themselves are > in some way corrupt. > > I know I can check the output of the command itself, but what if.. I > don't know... if there are problems with the disc it writes to, or > something like that. Is there any way to check whether the output file > is "valid" in the sense that it is complete and syntactically correct? > > -- > GaryPlease do NOT send me 'courtesy' replies off-list. > > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe:http://lists.mysql.com/mysql > > -- Claudio
