Now this one removes, the usage of different dicts, removes the
whitespace, and uses just a whole dict, then becomes 7.6kb
That's a big difference to someone parsing files, and utilizing memory
storage areas.
--
Best Regards,
David Hutto
CEO: http://www.hitwebdevelopment.com
{0:0,1:1,2:2,3:3,4:4
On 17/10/12 13:22, Alexander wrote:
On Tue, Oct 16, 2012 at 20:43 EST, Mark Lawrence
wrote:
For the record Access is not a database, or so some geezer called Alex
Martelli reckons http://code.activestate.com/lists/python-list/48130/, so
please don't shoot the messenger:)
Cheers.
Mark Lawrence.
On Oct 16, 2012, at 12:57 PM, Abhishek Pratap wrote:
> Hi Guys
>
> For my problem I need to store 400-800 million 20 characters keys in a
> dictionary and do counting. This data structure takes about 60-100 Gb
> of RAM.
> I am wondering if there are slick ways to map the dictionary to a file
> o
On 17/10/12 12:30, Dwight Hutto wrote:
My inexperienced advice would be to begin with the storage areas
available. I would begin by eliminating certain things such as:
x = {'one_entry' : 1}
into
x = {'one_entry':1}
Those two entries are exactly the same. The presence of absence of
spaces,
On Tue, Oct 16, 2012 at 12:57 PM, Abhishek Pratap
wrote:
> Hi Guys
>
> For my problem I need to store 400-800 million 20 characters keys in a
> dictionary and do counting. This data structure takes about 60-100 Gb
> of RAM.
> I am wondering if there are slick ways to map the dictionary to a file
>
On 17/10/2012 03:22, Alexander wrote:
On Tue, Oct 16, 2012 at 20:43 EST, Mark Lawrence
wrote:
For the record Access is not a database, or so some geezer called Alex
Martelli reckons http://code.activestate.com/lists/python-list/48130/, so
please don't shoot the messenger:)
Cheers.
Mark Lawrence
On Tue, Oct 16, 2012 at 7:22 PM, Alexander wrote:
> On Tue, Oct 16, 2012 at 20:43 EST, Mark Lawrence
> wrote:
>> For the record Access is not a database, or so some geezer called Alex
>> Martelli reckons http://code.activestate.com/lists/python-list/48130/, so
>> please don't shoot the messenger:
On Tue, Oct 16, 2012 at 20:43 EST, Mark Lawrence
wrote:
> For the record Access is not a database, or so some geezer called Alex
> Martelli reckons http://code.activestate.com/lists/python-list/48130/, so
> please don't shoot the messenger:)
> Cheers.
> Mark Lawrence.
Mark I don't believe your re
On 17/10/2012 01:03, Alan Gauld wrote:
You just need to set it up, load the data and use it - probably around
50 lines of SQL... And you don't need anything fancy for a single table
database - Access, SQLite, even FoxPro...
For the record Access is not a database, or so some geezer called Al
On 16/10/12 17:57, Abhishek Pratap wrote:
For my problem I need to store 400-800 million 20 characters keys in a
dictionary and do counting. This data structure takes about 60-100 Gb
of RAM.
Thats a lot of records but without details of what kind of counting you
plan on we can't give specific
On 16 October 2012 21:03, Prasad, Ramit wrote:
> Abhishek Pratap wrote:
>> Sent: Tuesday, October 16, 2012 11:57 AM
>> To: tutor@python.org
>> Subject: [Tutor] managing memory large dictionaries in python
>>
>> Hi Guys
>>
>> For my problem I need to
On 10/16/2012 01:03 PM, Prasad, Ramit wrote:
Abhishek Pratap wrote:
Sent: Tuesday, October 16, 2012 11:57 AM
To: tutor@python.org
Subject: [Tutor] managing memory large dictionaries in python
Hi Guys
For my problem I need to store 400-800 million 20 characters keys in a
dictionary and do
Abhishek Pratap wrote:
> Sent: Tuesday, October 16, 2012 11:57 AM
> To: tutor@python.org
> Subject: [Tutor] managing memory large dictionaries in python
>
> Hi Guys
>
> For my problem I need to store 400-800 million 20 characters keys in a
> dictionary and do counting. T
Hi Guys
For my problem I need to store 400-800 million 20 characters keys in a
dictionary and do counting. This data structure takes about 60-100 Gb
of RAM.
I am wondering if there are slick ways to map the dictionary to a file
on disk and not store it in memory but still access it as dictionary
o
14 matches
Mail list logo
