koara wrote:
>>> I would recommend you to use a database since it meets your
>>> requirements (off-memory, fast, persistent). The bsdddb module
>>> (berkeley db) even gives you a dictionary like interface.
>>> http://www.python.org/doc/lib/module-bsddb.html
>> Standard SQL databases can work for th
> > I would recommend you to use a database since it meets your
> > requirements (off-memory, fast, persistent). The bsdddb module
> > (berkeley db) even gives you a dictionary like interface.
> >http://www.python.org/doc/lib/module-bsddb.html
>
> Standard SQL databases can work for this, but gener
Rafael Darder Calvo wrote:
>> > > Please recommend a module that allows persistent set/dict storage +
>> > > fast query that best fits my problem,
>> >
>> > What is the problem you are trying to solve? How many keys do you have?
>>
>> Corpus processing. There are in the order of billions to tens of
> > > Please recommend a module that allows persistent set/dict storage +
> > > fast query that best fits my problem,
> >
> > What is the problem you are trying to solve? How many keys do you have?
>
> Corpus processing. There are in the order of billions to tens of
> billions keys (64bit integers)
Hello Steven,
On Jun 10, 5:29 pm, Steven D'Aprano
<[EMAIL PROTECTED]> wrote:
> > ...
> How do you know it won't fit in main memory if you don't know the
> overhead? A guess? You've tried it and your computer crashed?
exactly
> > Please recommend a module that allows persistent set/dict storage +
On Sun, 10 Jun 2007 07:27:56 -0700, koara wrote:
> What is the best to go about using a large set (or dictionary) that
> doesn't fit into main memory? What is Python's (2.5 let's say)
> overhead for storing int in the set, and how much for storing int ->
> int mapping in the dict?
How do you know
What is the best to go about using a large set (or dictionary) that
doesn't fit into main memory? What is Python's (2.5 let's say)
overhead for storing int in the set, and how much for storing int ->
int mapping in the dict?
Please recommend a module that allows persistent set/dict storage +
fast
