Re: random writing access to a file in Python

Dennis Lee Bieber wrote: > On 27 Aug 2006 15:06:07 -0700, Paul Rubin > declaimed the following in comp.lang.python: > > > >>I think that's not so bad, though probably still not optimal. 85 GB >>divided by 18 hours is 1.3 MB/sec, which means if the program is >>reading

Re: random writing access to a file in Python

Claudio Grondi <[EMAIL PROTECTED]> writes: > >>The Windows XP SP 2 '/> sort' (sorting of four Gigs of 20 byte records > >>took 12 CPU and 18 usual hours) > Ok, I see - the misunderstanding is, that there were 4.294.967.296 > records each 20 bytes long, what makes the actual file 85.899.345.920

Re: random writing access to a file in Python

Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>The Windows XP SP 2 '/> sort' (sorting of four Gigs of 20 byte records >>took 12 CPU and 18 usual hours) has, from what I could observe on the >>task manager, done the job in only two runs of 'copying' : > > > That is terrible; o

Re: random writing access to a file in Python

Claudio Grondi <[EMAIL PROTECTED]> writes: > The Windows XP SP 2 '/> sort' (sorting of four Gigs of 20 byte records > took 12 CPU and 18 usual hours) has, from what I could observe on the > task manager, done the job in only two runs of 'copying' : That is terrible; on a reasonably fast machine th

Re: random writing access to a file in Python

Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>Does it mean, that in case of very large files: >> the size of available memory for the sorting operation (making it >>possible to work on larger chunks of data in memory) has less impact >>on the actual sorting speed than >> t

Re: random writing access to a file in Python

Claudio Grondi <[EMAIL PROTECTED]> writes: > Does it mean, that in case of very large files: >the size of available memory for the sorting operation (making it > possible to work on larger chunks of data in memory) has less impact > on the actual sorting speed than >the speed of the data tr

Re: random writing access to a file in Python

Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>>Try the standard Unix/Linux sort utility. Use the --buffer-size=SIZE >>>to tell it how much memory to use. >> >>I am on Windows and it seems, that Windows XP SP2 'sort' can work with >>the file, but not without a temporary file a

Re: random writing access to a file in Python

Claudio Grondi <[EMAIL PROTECTED]> writes: > > Try the standard Unix/Linux sort utility. Use the --buffer-size=SIZE > > to tell it how much memory to use. > I am on Windows and it seems, that Windows XP SP2 'sort' can work with > the file, but not without a temporary file and space for the resulti

Re: random writing access to a file in Python

Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>Is there a ready to use (free, best Open Source) tool able to sort >>lines (each line appr. 20 bytes long) of a XXX GByte large text file >>(i.e. in place) taking full advantage of available memory to speed up >>the process as much

Re: random writing access to a file in Python

Claudio Grondi <[EMAIL PROTECTED]> writes: > Is there a ready to use (free, best Open Source) tool able to sort > lines (each line appr. 20 bytes long) of a XXX GByte large text file > (i.e. in place) taking full advantage of available memory to speed up > the process as much as possible? Try the

Re: random writing access to a file in Python

Claudio Grondi wrote: > I was smart enough to try it first on a very small file wondering what > was happening. Python documentation and even Google search after 'random > file access in Python' were not helpful as there was no example and no > hint available. one would think that the repeated

Re: random writing access to a file in Python

> Is there a ready to use (free, best Open Source) tool able to sort lines > (each line appr. 20 bytes long) of a XXX GByte large text file (i.e. in > place) taking full advantage of available memory to speed up the process > as much as possible? Sounds like an occasion to use a merge-sort. Th

Re: random writing access to a file in Python

> === > if I want to write, I don't open for reading (plus or not plus) > === I was thinking the same thing. I know python is only following C, which is good, but the flags

Re: random writing access to a file in Python

Dennis Lee Bieber wrote: > On Fri, 25 Aug 2006 16:39:14 +0200, Claudio Grondi > <[EMAIL PROTECTED]> declaimed the following in comp.lang.python: > > >>The core of my problem was ... trying to use 'wb' or 'w+b' ... (stupid >>me ...) > > > Ouch... How many times did you have to restore tha

Re: random writing access to a file in Python

Tim Peters wrote: > [Claudio Grondi] > >> I have a 250 Gbyte file (occupies the whole hard drive space) > > > Then where is Python stored ;-)? > >> and want to change only eight bytes in this file at a given offset of >> appr. 200 >> Gbyte (all other data in that file should remain unchanged).

Re: random writing access to a file in Python

[Claudio Grondi] > I have a 250 Gbyte file (occupies the whole hard drive space) Then where is Python stored ;-)? > and want to change only eight bytes in this file at a given offset of appr. > 200 > Gbyte (all other data in that file should remain unchanged). > > How can I do that in Python? S

random writing access to a file in Python

I have a 250 Gbyte file (occupies the whole hard drive space) and want to change only eight bytes in this file at a given offset of appr. 200 Gbyte (all other data in that file should remain unchanged). How can I do that in Python? Claudio Grondi -- http://mail.python.org/mailman/listinfo/pyt