Re: dynamic allocation file buffer

2008-09-15 Thread Francesc
On 12 Set, 14:39, "Aaron \"Castironpi\" Brady" <[EMAIL PROTECTED]> wrote: > > A consideration of other storage formats such as HDF5 might > > be appropriate: > > >http://hdf.ncsa.uiuc.edu/HDF5/whatishdf5.html > > > There are, of course, HDF5 tools available for Python. > > PyTablescame up within th

Re: dynamic allocation file buffer

2008-09-12 Thread Aaron "Castironpi" Brady
On Sep 12, 4:34 am, Paul Boddie <[EMAIL PROTECTED]> wrote: > On 12 Sep, 08:30, Steven D'Aprano > > <[EMAIL PROTECTED]> wrote: > > > Which is why I previously said that XML was not well suited for random > > access. > > Maybe not. No, it's not. Element trees are, which if I just would have said or

Re: dynamic allocation file buffer

2008-09-12 Thread Paul Boddie
On 12 Sep, 08:30, Steven D'Aprano <[EMAIL PROTECTED]> wrote: > > Which is why I previously said that XML was not well suited for random > access. Maybe not. A consideration of other storage formats such as HDF5 might be appropriate: http://hdf.ncsa.uiuc.edu/HDF5/whatishdf5.html There are, of cou

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron "Castironpi" Brady
On Sep 12, 1:30 am, Steven D'Aprano <[EMAIL PROTECTED]> wrote: > On Thu, 11 Sep 2008 22:40:01 -0700, Dennis Lee Bieber wrote: > > On 12 Sep 2008 03:37:51 GMT, Steven D'Aprano > > <[EMAIL PROTECTED]> declaimed the following in > > comp.lang.python: > > >> I'm pretty sure you're wrong. XML can be use

Re: dynamic allocation file buffer

2008-09-11 Thread Steven D'Aprano
On Thu, 11 Sep 2008 22:40:01 -0700, Dennis Lee Bieber wrote: > On 12 Sep 2008 03:37:51 GMT, Steven D'Aprano > <[EMAIL PROTECTED]> declaimed the following in > comp.lang.python: > > >> I'm pretty sure you're wrong. XML can be used for serialization, but >> that doesn't mean it is only sequential

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron "Castironpi" Brady
On Sep 11, 10:37 pm, Steven D'Aprano <[EMAIL PROTECTED]> wrote: > On Thu, 11 Sep 2008 10:20:41 -0700, Aaron \"Castironpi\" Brady wrote: > > XML is the wrong word for the example I was thinking of (as was already > > pointed out in another thread).  XML is by definition sequential. > > I'm pretty su

Re: dynamic allocation file buffer

2008-09-11 Thread Steven D'Aprano
On Thu, 11 Sep 2008 10:20:41 -0700, Aaron \"Castironpi\" Brady wrote: > XML is the wrong word for the example I was thinking of (as was already > pointed out in another thread). XML is by definition sequential. I'm pretty sure you're wrong. XML can be used for serialization, but that doesn't me

Re: dynamic allocation file buffer

2008-09-11 Thread Paul Boddie
On 11 Sep, 19:31, "Aaron \"Castironpi\" Brady" <[EMAIL PROTECTED]> wrote: > > An acquaintance suggests that defragmentation would be a useful > service to provide along with memory management too, which also > requires an index. I presume that you mean efficient access to large amounts of data in

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron "Castironpi" Brady
On Sep 11, 5:35 am, Paul Boddie <[EMAIL PROTECTED]> wrote: > On 11 Sep, 10:34, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > > > > And as I said before, the only use case for *huge* XML files I've ever > > seen used in practice is to store large streams of record-style data; > > I can imagine that t

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron "Castironpi" Brady
On Sep 11, 2:40 am, Steven D'Aprano <[EMAIL PROTECTED]> wrote: > On Wed, 10 Sep 2008 11:59:35 -0700, Aaron \"Castironpi\" Brady wrote: > > On Sep 10, 5:24 am, Steven D'Aprano > > <[EMAIL PROTECTED]> wrote: > >> On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: > >> > Steven D'Aprano wrote: >

Re: dynamic allocation file buffer

2008-09-11 Thread Paul Boddie
On 11 Sep, 10:34, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > And as I said before, the only use case for *huge* XML files I've ever > seen used in practice is to store large streams of record-style data; I can imagine that the manipulation of the persistent form of large graph structures might b

Re: dynamic allocation file buffer

2008-09-11 Thread Fredrik Lundh
Steven D'Aprano wrote: I'm no longer *claiming* anything, I'm *asking* whether random access to a 4GB XML file is something that is credible or useful. It is my understanding that XML is particularly ill-suited to random access once the amount of data is too large to fit in RAM. An XML file

Re: dynamic allocation file buffer

2008-09-11 Thread Steven D'Aprano
On Wed, 10 Sep 2008 11:59:35 -0700, Aaron \"Castironpi\" Brady wrote: > On Sep 10, 5:24 am, Steven D'Aprano > <[EMAIL PROTECTED]> wrote: >> On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: >> > Steven D'Aprano wrote: >> >> >> You've created a solution to a problem which (probably) only aff

Re: dynamic allocation file buffer

2008-09-10 Thread Paul Boddie
On Sep 10, 5:03 am, George Sakkis <[EMAIL PROTECTED]> wrote: > > So at best (i.e. if it actually makes any sense; I didn't read it), > this is an ANNouncement of a pre-alpha piece of code. ANN posts rarely > attract replies, even when they are about production/stable software. To be fair, at least

Re: dynamic allocation file buffer

2008-09-10 Thread Aaron "Castironpi" Brady
On Sep 9, 10:03 pm, George Sakkis <[EMAIL PROTECTED]> wrote: > On Sep 9, 5:59 pm, castironpi <[EMAIL PROTECTED]> wrote: > > > I will try my idea again.  I want to talk to people about a > > module I want to write and I will take the time to explain it. > > I think it's a "cool idea" that a lot of p

Re: dynamic allocation file buffer

2008-09-10 Thread Aaron "Castironpi" Brady
On Sep 10, 5:24 am, Steven D'Aprano <[EMAIL PROTECTED]> wrote: > On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: > > Steven D'Aprano wrote: > > >> You've created a solution to a problem which (probably) only affects a > >> very small number of people, at least judging by your use-cases. Wh

Re: dynamic allocation file buffer

2008-09-10 Thread Steven D'Aprano
On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: > Steven D'Aprano wrote: > >> You've created a solution to a problem which (probably) only affects a >> very small number of people, at least judging by your use-cases. Who >> has a 4GB XML file > > Getting 4GB XML files from, say, logging

Re: dynamic allocation file buffer

2008-09-10 Thread Fredrik Lundh
Steven D'Aprano wrote: You've created a solution to a problem which (probably) only affects a very small number of people, at least judging by your use-cases. Who has a 4GB XML file Getting 4GB XML files from, say, logging processes or databases that can render their output as XML is not tha

Re: dynamic allocation file buffer

2008-09-09 Thread George Sakkis
On Sep 9, 5:59 pm, castironpi <[EMAIL PROTECTED]> wrote: > I will try my idea again.  I want to talk to people about a > module I want to write and I will take the time to explain it. > I think it's a "cool idea" that a lot of people, forgiving the > slang, could benefit from.   > > (snipped) > >

Re: dynamic allocation file buffer

2008-09-09 Thread castironpi
On Sep 9, 5:58 pm, Steven D'Aprano <[EMAIL PROTECTED] cybersource.com.au> wrote: > On Tue, 09 Sep 2008 14:59:19 -0700, castironpi wrote: > > I will try my idea again.  I want to talk to people about a module I > > want to write and I will take the time to explain it.  I think it's a > > "cool idea"

Re: dynamic allocation file buffer

2008-09-09 Thread castironpi
On Sep 9, 5:44 pm, Larry Bates <[EMAIL PROTECTED]> wrote: > castironpi wrote: > > I will try my idea again.  I want to talk to people about a module I > > want to write and I will take the time to explain it.  I think it's a > > "cool idea" that a lot of people, forgiving the slang, could benefit >

Re: dynamic allocation file buffer

2008-09-09 Thread Steven D'Aprano
On Tue, 09 Sep 2008 14:59:19 -0700, castironpi wrote: > I will try my idea again. I want to talk to people about a module I > want to write and I will take the time to explain it. I think it's a > "cool idea" that a lot of people, forgiving the slang, could benefit > from. What are its flaws?

Re: dynamic allocation file buffer

2008-09-09 Thread Larry Bates
castironpi wrote: I will try my idea again. I want to talk to people about a module I want to write and I will take the time to explain it. I think it's a "cool idea" that a lot of people, forgiving the slang, could benefit from. What are its flaws? A user has a file he is using either 1/ to

dynamic allocation file buffer

2008-09-09 Thread castironpi
I will try my idea again. I want to talk to people about a module I want to write and I will take the time to explain it. I think it's a "cool idea" that a lot of people, forgiving the slang, could benefit from. What are its flaws? A user has a file he is using either 1/ to persist binary data