Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
On Mon, Jan 2, 2012 at 04:00:19PM -0500, Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of lun ene 02 17:28:33 -0300 2012: it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Probably because other filesystems do allow you to fsync directories. In fact for some cases they _require_ it ... remember the fiasco when MTA writers were told that they needed to fsync their queue dirs in order for all queued email to persist? Yeah, the long and the short of it is that if the filesystem won't accept an fsync on a directory, we have to assume that it doesn't need it and will manage metadata persistence safely without prodding. The only real question here is whether an EINVAL could mean something besides fsync on directory is not accepted. If there are any scenarios where it represents a transient/fixable error, then we'd want to report it. It's far from clear to me that there are any though. What it could mean in general is not at issue, because we know the target is a directory that we just created moments before. I assume this never got resolved. Should it be changed to ignore EINVAL? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Hi Tom, Thanks for the solution. CIFS worked with fsync flag by ingnoring EINVAL in copydir.c. I tested fsync with 8.2.2 version of PostgreSQL, it worked fine without EINVAL patch. I wanted to know is something changed in version 9.0.4 of postgreSQL. As fsync flag was not working with PostgreSQL version 9.0.4 without applying the patch. Regards, Anjali From: Magnus Hagander mag...@hagander.net To: Tom Lane t...@sss.pgh.pa.us Cc: anjali_...@yahoo.co.in; pgsql-bugs@postgresql.org Sent: Tuesday, 3 January 2012 2:00 AM Subject: Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS On Mon, Jan 2, 2012 at 21:28, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 21:14, Tom Lane t...@sss.pgh.pa.us wrote: I'm wondering what's your basis for asserting we don't support CIFS in general? It's probably not terribly bulletproof, but any worse than NFS? Yes, it is a lot worse than NFS from experience. I can't find a reference to it anywhere now, but IIRC there are bigger issues - with blocksizes, with syncing not properly, with write ordering. Hmm. I searched the list archives and couldn't find any previous discussion of such things, but that may just prove that no one thinks it's worth attempting. Yeah, I don't think it was in our archives, it was somewhere else. And as a disclaime r- it may have been about the win32 cifs *client*. It was at the time just talking windows cifs client - windows cifs server. Anyway the immediate question is which errnos are reasonable for copydir to ignore. Just looking at the standard's description of fsync's error conditions: The fsync() function shall fail if: [EBADF] The fildes argument is not a valid descriptor. [EINTR] The fsync() function was interrupted by a signal. [EINVAL] The fildes argument does not refer to a file on which this operation is possible. [EIO] An I/O error occurred while reading from or writing to the file system. it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Agreed. But do we really want to accept this with fsync=on? It basically means fsync=maybe, no? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Hi Tom, Thanks for the solution. CIFS worked with fsync flag by ingnoring EINVAL in copydir.c. I tested fsync with 8.2.2 version of PostgreSQL, it worked fine without EINVAL patch. I wanted to know is something changed in version 9.0.4 of postgreSQL. As fsync flag was not working with PostgreSQL version 9.0.4 without applying the patch. Regards, Anjali From: Magnus Hagander mag...@hagander.net To: Tom Lane t...@sss.pgh.pa.us Cc: anjali_...@yahoo.co.in; pgsql-bugs@postgresql.org Sent: Tuesday, 3 January 2012 2:00 AM Subject: Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS On Mon, Jan 2, 2012 at 21:28, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 21:14, Tom Lane t...@sss.pgh.pa.us wrote: I'm wondering what's your basis for asserting we don't support CIFS in general? It's probably not terribly bulletproof, but any worse than NFS? Yes, it is a lot worse than NFS from experience. I can't find a reference to it anywhere now, but IIRC there are bigger issues - with blocksizes, with syncing not properly, with write ordering. Hmm. I searched the list archives and couldn't find any previous discussion of such things, but that may just prove that no one thinks it's worth attempting. Yeah, I don't think it was in our archives, it was somewhere else. And as a disclaime r- it may have been about the win32 cifs *client*. It was at the time just talking windows cifs client - windows cifs server. Anyway the immediate question is which errnos are reasonable for copydir to ignore. Just looking at the standard's description of fsync's error conditions: The fsync() function shall fail if: [EBADF] The fildes argument is not a valid descriptor. [EINTR] The fsync() function was interrupted by a signal. [EINVAL] The fildes argument does not refer to a file on which this operation is possible. [EIO] An I/O error occurred while reading from or writing to the file system. it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Agreed. But do we really want to accept this with fsync=on? It basically means fsync=maybe, no? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
[BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
The following bug has been logged on the website: Bug reference: 6372 Logged by: Anjali Arora Email address: anjali_...@yahoo.co.in PostgreSQL version: 9.0.4 Operating system: Cent OS Description: The 8.2.2 postgres version works well with default fsync value on CIFS 2.6.32. This problem seems to be specific to postgres 9.0.4 release Error Message: PST ERROR: could not fsync file base/16409: Invalid argument Dec 30 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812 2011-12-30 03:00:26.511 PST STATEMENT: CREATE DATABASE KazDB Please help to make it work. -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
On Mon, Jan 2, 2012 at 17:27, anjali_...@yahoo.co.in wrote: The following bug has been logged on the website: Bug reference: 6372 Logged by: Anjali Arora Email address: anjali_...@yahoo.co.in PostgreSQL version: 9.0.4 Operating system: Cent OS Description: The 8.2.2 postgres version works well with default fsync value on CIFS 2.6.32. It does not, really. It may appear to, but it does not. This problem seems to be specific to postgres 9.0.4 release Error Message: PST ERROR: could not fsync file base/16409: Invalid argument Dec 30 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812 2011-12-30 03:00:26.511 PST STATEMENT: CREATE DATABASE KazDB Please help to make it work. PostgreSQL does not support data directory over CIFS. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 17:27, anjali_...@yahoo.co.in wrote: PST ERROR: could not fsync file base/16409: Invalid argument Dec 30 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812 2011-12-30 03:00:26.511 PST STATEMENT: CREATE DATABASE KazDB The specific error seems to be coming from copydir.c's attempt to fsync a directory. We are already ignoring EBADF there, and could presumably fix at least this symptom if we ignored EINVAL. PostgreSQL does not support data directory over CIFS. I'm wondering what's your basis for asserting we don't support CIFS in general? It's probably not terribly bulletproof, but any worse than NFS? regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
On Mon, Jan 2, 2012 at 21:14, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 17:27, anjali_...@yahoo.co.in wrote: PST ERROR: could not fsync file base/16409: Invalid argument Dec 30 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812 2011-12-30 03:00:26.511 PST STATEMENT: CREATE DATABASE KazDB The specific error seems to be coming from copydir.c's attempt to fsync a directory. We are already ignoring EBADF there, and could presumably fix at least this symptom if we ignored EINVAL. Sure, we could - and I guess if you're running over CIFS, reliability might not be the biggest concern in the first place... PostgreSQL does not support data directory over CIFS. I'm wondering what's your basis for asserting we don't support CIFS in general? It's probably not terribly bulletproof, but any worse than NFS? Yes, it is a lot worse than NFS from experience. I can't find a reference to it anywhere now, but IIRC there are bigger issues - with blocksizes, with syncing not properly, with write ordering. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 21:14, Tom Lane t...@sss.pgh.pa.us wrote: I'm wondering what's your basis for asserting we don't support CIFS in general? It's probably not terribly bulletproof, but any worse than NFS? Yes, it is a lot worse than NFS from experience. I can't find a reference to it anywhere now, but IIRC there are bigger issues - with blocksizes, with syncing not properly, with write ordering. Hmm. I searched the list archives and couldn't find any previous discussion of such things, but that may just prove that no one thinks it's worth attempting. Anyway the immediate question is which errnos are reasonable for copydir to ignore. Just looking at the standard's description of fsync's error conditions: The fsync() function shall fail if: [EBADF] The fildes argument is not a valid descriptor. [EINTR] The fsync() function was interrupted by a signal. [EINVAL] The fildes argument does not refer to a file on which this operation is possible. [EIO] An I/O error occurred while reading from or writing to the file system. it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
On Mon, Jan 2, 2012 at 21:28, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 21:14, Tom Lane t...@sss.pgh.pa.us wrote: I'm wondering what's your basis for asserting we don't support CIFS in general? It's probably not terribly bulletproof, but any worse than NFS? Yes, it is a lot worse than NFS from experience. I can't find a reference to it anywhere now, but IIRC there are bigger issues - with blocksizes, with syncing not properly, with write ordering. Hmm. I searched the list archives and couldn't find any previous discussion of such things, but that may just prove that no one thinks it's worth attempting. Yeah, I don't think it was in our archives, it was somewhere else. And as a disclaime r- it may have been about the win32 cifs *client*. It was at the time just talking windows cifs client - windows cifs server. Anyway the immediate question is which errnos are reasonable for copydir to ignore. Just looking at the standard's description of fsync's error conditions: The fsync() function shall fail if: [EBADF] The fildes argument is not a valid descriptor. [EINTR] The fsync() function was interrupted by a signal. [EINVAL] The fildes argument does not refer to a file on which this operation is possible. [EIO] An I/O error occurred while reading from or writing to the file system. it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Agreed. But do we really want to accept this with fsync=on? It basically means fsync=maybe, no? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Magnus Hagander mag...@hagander.net writes: On Mon, Jan 2, 2012 at 21:28, Tom Lane t...@sss.pgh.pa.us wrote: it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Agreed. But do we really want to accept this with fsync=on? It basically means fsync=maybe, no? Well, given the number of cases that the code already ignores when isdir is true, I don't think that argument holds much water at all. However, I'm not real eager to change this just on the basis of the CIFS case. If we find another filesystem that returns the same errno, though, I would vote to change it. regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Excerpts from Tom Lane's message of lun ene 02 17:28:33 -0300 2012: Anyway the immediate question is which errnos are reasonable for copydir to ignore. Just looking at the standard's description of fsync's error conditions: The fsync() function shall fail if: [EBADF] The fildes argument is not a valid descriptor. [EINTR] The fsync() function was interrupted by a signal. [EINVAL] The fildes argument does not refer to a file on which this operation is possible. [EIO] An I/O error occurred while reading from or writing to the file system. it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Probably because other filesystems do allow you to fsync directories. In fact for some cases they _require_ it ... remember the fiasco when MTA writers were told that they needed to fsync their queue dirs in order for all queued email to persist? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of lun ene 02 17:28:33 -0300 2012: it seems like EINVAL is a considerably more reasonable thing to return than EBADF, if the filesystem is trying to tell you that it won't fsync a directory. So I'm a bit surprised this question hasn't come up for other filesystems. Probably because other filesystems do allow you to fsync directories. In fact for some cases they _require_ it ... remember the fiasco when MTA writers were told that they needed to fsync their queue dirs in order for all queued email to persist? Yeah, the long and the short of it is that if the filesystem won't accept an fsync on a directory, we have to assume that it doesn't need it and will manage metadata persistence safely without prodding. The only real question here is whether an EINVAL could mean something besides fsync on directory is not accepted. If there are any scenarios where it represents a transient/fixable error, then we'd want to report it. It's far from clear to me that there are any though. What it could mean in general is not at issue, because we know the target is a directory that we just created moments before. regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
