Re: Create new files as sparse on NT systems. (2nd try)

2003-02-03 Thread Igor Pechtchanski
On Mon, 3 Feb 2003, Vaclav Haisman wrote: > This is a little bit improved version of my previous post. > By default creation of sparse files is disabled. It can be enabled by CYGWIN > option sparse_files. > > Vaclav Haisman > > 2003-02-03 Vaclav Haisman <[EMAIL PROTECTED]> > [snip] > * env

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-03 Thread Vaclav Haisman
I haven't noticed until you pointed it out. This can be easily corrected. Vaclav Haisman > I don't know if it matters, but the rest of the entries in the parse_thing > table are alphabetically ordered... > Igor > --

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-13 Thread Christopher Faylor
On Mon, Feb 03, 2003 at 02:18:03PM +0100, Vaclav Haisman wrote: > >This is a little bit improved version of my previous post. >By default creation of sparse files is disabled. It can be enabled by CYGWIN >option sparse_files. > >Vaclav Haisman > > >2003-02-03 Vaclav Haisman <[EMAIL PROTECTED]> >

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-13 Thread Vaclav Haisman
On Thu, 13 Feb 2003, Christopher Faylor wrote: > This is YA case where I don't think that a CYGWIN environment variable option is > justified. > > UNIX has a method for producing sparse files. If this is desired functionality, > Cygwin should mimic that not invent a new way of doing things. > > c

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Vaclav Haisman
> UNIX has a method for producing sparse files. If this is desired functionality, > Cygwin should mimic that not invent a new way of doing things. > > cgf Hi, I have prepared another patch that implements parse files for Cygwin. It is smaller and, I think, even better than the previous. No new CY

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Christopher Faylor
On Mon, Feb 17, 2003 at 05:37:20PM +0100, Vaclav Haisman wrote: >> UNIX has a method for producing sparse files. If this is desired functionality, >> Cygwin should mimic that not invent a new way of doing things. >> >> cgf > >Hi, >I have prepared another patch that implements parse files for Cygwi

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Vaclav Haisman
> This looks pretty good but the cygwin convention is to use wincap > settings for this kind of thing rather than using is_winnt. So, please > add a wincap capability to accomplish this. > > cgf Modified as suggested. Vaclav Haisman 2003-02-17 Vaclav Haisman <[EMAIL PROTECTED]> * win

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Max Bowsher
> On Mon, Feb 17, 2003 at 05:37:20PM +0100, Vaclav Haisman wrote: >> I have prepared another patch that implements sparse files for >> Cygwin. It is smaller and, I think, even better than the previous. >> No new CYGWIN options. >> 2003-02-17 Vaclav Haisman <[EMAIL PROTECTED]> >> >> * include/win

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Vaclav Haisman
> Is it wise to set *all* new files to sparse? Surely if this was actually > advantageous, Windows would do it anyway? From MSDN: "Note It is up to the > application to maintain sparseness by writing zeros with > FSCTL_SET_ZERO_DATA." I.e., this will gain nothing unless the application > knows ab

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Max Bowsher
Vaclav Haisman wrote: >> Is it wise to set *all* new files to sparse? Surely if this was >> actually advantageous, Windows would do it anyway? From MSDN: "Note >> It is up to the application to maintain sparseness by writing zeros >> with FSCTL_SET_ZERO_DATA." I.e., this will gain nothing unless th

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Vaclav Haisman
> That's nice. Did you read the bit above where I quoted MSDN? Merely setting > the file as sparse will NOT SAVE SPACE on Windows. So, no space gain, and a > performance penalty of untested magnitude. I see only disadvantages. > > > Max. Oh yes, it will. Some applications do lseek() on rather lon

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Christopher Faylor
On Mon, Feb 17, 2003 at 07:20:42PM +0100, Vaclav Haisman wrote: >>That's nice. Did you read the bit above where I quoted MSDN? Merely >>setting the file as sparse will NOT SAVE SPACE on Windows. So, no >>space gain, and a performance penalty of untested magnitude. I see >>only disadvantages. >

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Vaclav Haisman
> Btw, now that I've said that it occurred to me to check > GetVolumeInformation. There is apparently a FILE_SUPPORTS_SPARSE_FILES > flag available. That's the ultimate way to deal with this rather than > adding a wincap, I believe. Check (pc->fs.flags & > FILE_SUPPORTS_SPARSE_FILES) in fhandler

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Christopher Faylor
On Mon, Feb 17, 2003 at 07:48:57PM +0100, Vaclav Haisman wrote: >>Btw, now that I've said that it occurred to me to check >>GetVolumeInformation. There is apparently a FILE_SUPPORTS_SPARSE_FILES >>flag available. That's the ultimate way to deal with this rather than >>adding a wincap, I believe.

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-17 Thread Vaclav Haisman
> This is consistent with the way the rest of cygwin works, however. The > same argument could be applied to testing for ntsec. If this was an issue > then we should be changing the fs information to reflect reparse points. > > cgf I am not sure what is the conclusion here. Should I make it chec

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Christopher Faylor
On Mon, Feb 17, 2003 at 08:23:21PM +0100, Vaclav Haisman wrote: >> This is consistent with the way the rest of cygwin works, however. The >> same argument could be applied to testing for ntsec. If this was an issue >> then we should be changing the fs information to reflect reparse points. > >I a

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Vaclav Haisman
I've moved the whole thing into fhandler_disk_file::open and made it conditional on FILE_SUPPORTS_SPARSE_FILES. I had to add new method to path_conv to get access to flags member of fs_info. This patch includes wincap changes but they can be left out as the rest of the patch doesn't rely on them a

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Max Bowsher
Two things - First: Please, please don't make this the default! Once a file is sparsified, it cannot be unsparsified except by copying the contents to a new file! This seems like an optimization for a corner case is trying to cause a global change. And: FSCTL_SET_SPARSE, used in the patch, is *

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Christopher Faylor
On Tue, Feb 18, 2003 at 10:08:56PM -, Max Bowsher wrote: >Two things - First: > >Please, please don't make this the default! Once a file is sparsified, it >cannot be unsparsified except by copying the contents to a new file! This >seems like an optimization for a corner case is trying to cause

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Max Bowsher
Christopher Faylor wrote: > On Tue, Feb 18, 2003 at 10:08:56PM -, Max Bowsher wrote: >> Two things - First: >> >> Please, please don't make this the default! Once a file is >> sparsified, it cannot be unsparsified except by copying the contents >> to a new file! This seems like an optimization

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Christopher Faylor
On Tue, Feb 18, 2003 at 10:21:26PM -, Max Bowsher wrote: >Christopher Faylor wrote: >> On Tue, Feb 18, 2003 at 10:08:56PM -, Max Bowsher wrote: >>> Two things - First: >>> >>> Please, please don't make this the default! Once a file is >>> sparsified, it cannot be unsparsified except by copy

RE: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Gary R Van Sickle
[snip] > >What kind of program would actually benefit from sparse > files? And shouldn't > >it be the responsibility of that program to request them? > > IIRC, linux creates sparse files automatically when you do an lseek to > a position beyond EOF. I believe that Windows is similar. > No, Windo

RE: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Vaclav Haisman
On Tue, 18 Feb 2003, Gary R Van Sickle wrote: > "Note: It is up to the application to maintain sparseness by writing zeros > with FSCTL_SET_ZERO_DATA", sez the Platform docs. In this respect Windows are ahead of any recent Unix system. I wasn't able find any Unix/Posix syscall that would allow t

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Max Bowsher
Vaclav Haisman wrote: > On Tue, 18 Feb 2003, Gary R Van Sickle wrote: > >> "Note: It is up to the application to maintain sparseness by writing >> zeros with FSCTL_SET_ZERO_DATA", sez the Platform docs. > > In this respect Windows are ahead of any recent Unix system. I wasn't > able find any Unix

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Vaclav Haisman
> Could you do some tests, so we have more than conjecture to go on? > > What programs actually *benefit* from sparseness? My primary motivation to do this is that I use P2P sharing program called BitTorrent. This program is written in Python and I run it in Cygwin. This program first creates who

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Max Bowsher
Vaclav Haisman wrote: >> Could you do some tests, so we have more than conjecture to go on? >> >> What programs actually *benefit* from sparseness? > > My primary motivation to do this is that I use P2P sharing program > called BitTorrent. This program is written in Python and I run it in > Cygwin.

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Vaclav Haisman
On Wed, 19 Feb 2003, Max Bowsher wrote: > Well, why not have BitTorrent set the file as sparse? Because it runs as Cygwin app which is Unix-like environment. There is no way to set files sparse in Unix because all files are sparse if the file systems supports it. Vaclav Haisman

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-18 Thread Christopher Faylor
On Wed, Feb 19, 2003 at 02:19:50AM +0100, Vaclav Haisman wrote: >On Wed, 19 Feb 2003, Max Bowsher wrote: >>Well, why not have BitTorrent set the file as sparse? > >Because it runs as Cygwin app which is Unix-like environment. There is >no way to set files sparse in Unix because all files are spars

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-19 Thread Joe Buehler
Christopher Faylor wrote: Because it runs as Cygwin app which is Unix-like environment. There is no way to set files sparse in Unix because all files are sparse if the file systems supports it. ...which is, coincidentally enough, why I was interested in the patch. It seems like a win to me -

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-19 Thread Christopher Faylor
On Wed, Feb 19, 2003 at 10:39:17AM -0500, Joe Buehler wrote: >Christopher Faylor wrote: > >>>Because it runs as Cygwin app which is Unix-like environment. There is >>>no way to set files sparse in Unix because all files are sparse if the >>>file systems supports it. >> >>...which is, coincidentall

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-19 Thread Corinna Vinschen
On Wed, Feb 19, 2003 at 12:57:38PM -0500, Christopher Faylor wrote: > On Wed, Feb 19, 2003 at 10:39:17AM -0500, Joe Buehler wrote: > >One thing to investigate would be what happens when Windows trys to mmap() > >a sparse file. It doesn't bother a UNIX box, but Windows? Perhaps that > >is what Bit

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-19 Thread Corinna Vinschen
On Wed, Feb 19, 2003 at 08:41:35PM +0100, Corinna Vinschen wrote: > On Wed, Feb 19, 2003 at 12:57:38PM -0500, Christopher Faylor wrote: > > On Wed, Feb 19, 2003 at 10:39:17AM -0500, Joe Buehler wrote: > > >One thing to investigate would be what happens when Windows trys to mmap() > > >a sparse file

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-19 Thread Corinna Vinschen
On Wed, Feb 19, 2003 at 09:19:25PM +0100, Corinna Vinschen wrote: > > I'm going to test that *shiver*. > > Well... as a first result: I tried to write two 4K blocks with a > lseek(16K), creating a 24K file which should use only 8K on disk. > > It didn't work. > > Unfortunaltely, NTFS5 seem to s

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-20 Thread Corinna Vinschen
On Tue, Feb 18, 2003 at 10:51:31PM +0100, Vaclav Haisman wrote: > 2003-02-17 Vaclav Haisman <[EMAIL PROTECTED]> > > * include/winioctl.h (FSCTL_SET_SPARSE): Define. > > 2003-02-18 Vaclav Haisman <[EMAIL PROTECTED]> > > * wincap.h (wincaps::supports_sparse_files): New flag. >

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-20 Thread Corinna Vinschen
On Wed, Feb 19, 2003 at 10:06:37PM +0100, Corinna Vinschen wrote: > Since st_blocks contains the number of blocks allocated, according to > the Linux man page and SUSv3, shouldn't we change st_blocks to reflect > the value of GetCompressedFileSize() now? I've checked in a patch so st_blocks is now

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-20 Thread Christopher Faylor
On Thu, Feb 20, 2003 at 11:16:06AM +0100, Corinna Vinschen wrote: >On Tue, Feb 18, 2003 at 10:51:31PM +0100, Vaclav Haisman wrote: >> 2003-02-17 Vaclav Haisman <[EMAIL PROTECTED]> >> >> * include/winioctl.h (FSCTL_SET_SPARSE): Define. >> >> 2003-02-18 Vaclav Haisman <[EMAIL PROTECTED]> >>

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-20 Thread Corinna Vinschen
On Thu, Feb 20, 2003 at 10:25:00AM -0500, Christopher Faylor wrote: > On Thu, Feb 20, 2003 at 11:16:06AM +0100, Corinna Vinschen wrote: > >I've applied that patch now. If anybody still has problems with it, > >please feel free to post a *testcase* which shows the problem. > > Um. I was still hop

Re: Create new files as sparse on NT systems. (2nd try)

2003-02-20 Thread Christopher Faylor
On Thu, Feb 20, 2003 at 04:52:22PM +0100, Corinna Vinschen wrote: >However, since some people were objecting so doggedly, I was thinking >that committing this patch would give a hint that a testcase(tm) is more >useful than just musing. Everybody who wants this patch removed can >accomplish this b