Re: [OT] Lying hard drives [was Re: shutil.copyfile is incomplete (truncated)]

2013-04-12 Thread Chris Angelico
On Sat, Apr 13, 2013 at 1:17 PM, Steven D'Aprano wrote: > On Sat, 13 Apr 2013 03:33:29 +0100, Nobody wrote: >> If you want to wait for the data written to be written to the physical >> disk (in order to obtain specific behaviour with respect to an unclean >> shutdown), use f.flush() followed by os

[OT] Lying hard drives [was Re: shutil.copyfile is incomplete (truncated)]

2013-04-12 Thread Steven D'Aprano
On Sat, 13 Apr 2013 03:33:29 +0100, Nobody wrote: > On Fri, 12 Apr 2013 00:06:21 +, Steven D'Aprano wrote: > >>> The close method is defined and flushing and closing a file, so it >>> should not return until that's done. >> >> But note that "done" in this case means "the file system thinks i

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Chris Angelico
On Sat, Apr 13, 2013 at 12:33 PM, Nobody wrote: > But most of the time, there's no point. If you actually care about what > happens in the event of an unclean shutdown, you typically also need to > sync the directory, otherwise the file's contents will get sync'd but the > file's very existence mi

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Nobody
On Fri, 12 Apr 2013 00:06:21 +, Steven D'Aprano wrote: >> The close method is defined and flushing and closing a file, so it >> should not return until that's done. > > But note that "done" in this case means "the file system thinks it is > done", not *actually* done. Hard drives, especially

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread 88888 Dihedral
Steven D'Aprano於 2013年4月12日星期五UTC+8上午8時06分21秒寫道: > On Thu, 11 Apr 2013 19:55:53 +, Neil Cerutti wrote: > > > > > On 2013-04-11, Rob Schneider wrote: > > >> Thanks. Yes, there is a close function call before the copy is > > >> launched. No other writes. Does Python wait for file close com

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Roy Smith
In article , Rob Schneider wrote: > f.close Well, there's your problem. You're not calling close. You forgot the ()'s after the function name! -- http://mail.python.org/mailman/listinfo/python-list

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Roy Smith
In article <51674ffc$0$29977$c3e8da3$54964...@news.astraweb.com>, Steven D'Aprano wrote: > On Thu, 11 Apr 2013 19:55:53 +, Neil Cerutti wrote: > > > On 2013-04-11, Rob Schneider wrote: > >> Thanks. Yes, there is a close function call before the copy is > >> launched. No other writes. Does

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Roy Smith
In article , Cameron Simpson wrote: > 45046 is exactly 11 * 4096. I'd say your I/O is using 4KB blocks, > and the last partial block (to make it up to 47970) didn't get > written (at the OS level). Yeah, this sounds like a good diagnosis. BTW, the dtruss command I recommended in my earlier pos

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Terry Jan Reedy
On 4/12/2013 3:32 AM, Chris Angelico wrote: On Fri, Apr 12, 2013 at 4:25 PM, Rob Schneider wrote: The close method is defined and flushing and closing a file, so it should not return until that's done. What command are you using to create the temp file? re command to write the file: f

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Roy Smith
In article , Rob Schneider wrote: > Source (correct one) is 47,970 bytes. Target after copy of 45,056 bytes. > I've tried changing what gets written to change the file size. It is usually > this sort of difference. > > The file system is Mac OS Extended Journaled (default as out of the box).

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Mark Lawrence
On 12/04/2013 13:07, Rob Schneider wrote: On Friday, 12 April 2013 10:22:21 UTC+1, Chris Angelico wrote: On Fri, Apr 12, 2013 at 7:18 PM, Rob Schneider wrote: f.close Yep, there's the problem! See my previous post for details. Change this to: f.close() and you should be sorted. Chris

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Rob Schneider
On Friday, 12 April 2013 10:22:21 UTC+1, Chris Angelico wrote: > On Fri, Apr 12, 2013 at 7:18 PM, Rob Schneider wrote: > > > f.close > > > > Yep, there's the problem! See my previous post for details. Change this to: > > > > f.close() > > > > and you should be sorted. > > > > Chris

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Chris Angelico
On Fri, Apr 12, 2013 at 7:18 PM, Rob Schneider wrote: > f.close Yep, there's the problem! See my previous post for details. Change this to: f.close() and you should be sorted. ChrisA -- http://mail.python.org/mailman/listinfo/python-list

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Rob Schneider
On Friday, 12 April 2013 09:26:21 UTC+1, Cameron Simpson wrote: > > | > > Question: is the size of the incomplete file a round number? (Like > > | > > a multiple of a decent sized power of 2>) > > [...] > > | Source (correct one) is 47,970 bytes. Target after copy of 45,056 > > | bytes. I've

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Cameron Simpson
On 11Apr2013 23:32, Rob Schneider wrote: | > > Question: is the size of the incomplete file a round number? (Like | > > a multiple of a decent sized power of 2>) [...] | Source (correct one) is 47,970 bytes. Target after copy of 45,056 | bytes. I've tried changing what gets written to change the

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Ned Deily
In article <6eeabeb2-e6dd-49fc-bd64-8de539651...@googlegroups.com>, Rob Schneider wrote: > > The file system is Mac OS Extended Journaled (default as out of the box). > It shows file size 45,056 on both source and target, which is the file size > of the flawed target, and is not what Finder show

Re: shutil.copyfile is incomplete (truncated)

2013-04-12 Thread Chris Angelico
On Fri, Apr 12, 2013 at 4:25 PM, Rob Schneider wrote: > >> The close method is defined and flushing and closing a file, so >> >> it should not return until that's done. >> >> >> >> What command are you using to create the temp file? >> >> > > re command to write the file: > f=open(fn,'w') > ... th

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Rob Schneider
> The file system is Mac OS Extended Journaled (default as out of the box). I ran a repair disk .. .while it found and fixed what it called "minor" problems, it did something. However, the repair did not fix the problem. I just ran the program again and the source is 47,970 bytes and target af

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Rob Schneider
> > > Or that the filesystem may be full? Of course, that's usually obvious > > > more widely when it happens... > > > > > > Question: is the size of the incomplete file a round number? (Like > > > a multiple of a decent sized power of 2>) > > > > Also on what OS X file system type does t

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Rob Schneider
> I would consider the chance that the disk may be faulty, or the file > > system is corrupt. Does the problem go away if you write to a different > > file system or a different disk? > It's a relatively new MacBook Pro with a solid state disk. I've not noticed any other disk problems. I di

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Rob Schneider
> The close method is defined and flushing and closing a file, so > > it should not return until that's done. > > > > What command are you using to create the temp file? > > re command to write the file: f=open(fn,'w') ... then create HTML text in a string f.write(html) f.close -- http:/

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Ned Deily
In article <20130412011550.ga80...@cskk.homeip.net>, Cameron Simpson wrote: > Or that the filesystem may be full? Of course, that's usually obvious > more widely when it happens... > > Question: is the size of the incomplete file a round number? (Like > a multiple of a decent sized power of 2>)

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Cameron Simpson
On 12Apr2013 00:06, Steven D'Aprano wrote: | On Thu, 11 Apr 2013 19:55:53 +, Neil Cerutti wrote: | > On 2013-04-11, Rob Schneider wrote: | >> Thanks. Yes, there is a close function call before the copy is | >> launched. No other writes. Does Python wait for file close command to | >> comple

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Steven D'Aprano
On Thu, 11 Apr 2013 19:55:53 +, Neil Cerutti wrote: > On 2013-04-11, Rob Schneider wrote: >> Thanks. Yes, there is a close function call before the copy is >> launched. No other writes. Does Python wait for file close command to >> complete before proceeding? > > The close method is defined

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Neil Cerutti
On 2013-04-11, Rob Schneider wrote: > Thanks. Yes, there is a close function call before the copy is > launched. No other writes. Does Python wait for file close > command to complete before proceeding? The close method is defined and flushing and closing a file, so it should not return until th

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Rob Schneider
Thanks. Yes, there is a close function call before the copy is launched. No other writes. Does Python wait for file close command to complete before proceeding? -- http://mail.python.org/mailman/listinfo/python-list

Re: shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Neil Cerutti
On 2013-04-11, Rob Schneider wrote: > Using Python 2.7.2 on OSX, I have created a file in temp space, > then use the function "shutil.copyfile(fn,loc+fname)" from "fn" > to "loc+fname". > > At the destination location, the file is truncated. About 10% > of the file is lost. Original file is uncha

shutil.copyfile is incomplete (truncated)

2013-04-11 Thread Rob Schneider
Using Python 2.7.2 on OSX, I have created a file in temp space, then use the function "shutil.copyfile(fn,loc+fname)" from "fn" to "loc+fname". At the destination location, the file is truncated. About 10% of the file is lost. Original file is unchanged. I added calls to "statinfo" immediately