In article ,
=?GB2312?B?zPC5zw==?= wrote:
>Well, Database is not proper because 1. the table is very big (~10^9
>rows) 2. we should support very fast *simple* query that is to get
>value corresponding to single key (~10^7 queries / second).
>
>Currently, I have implemented a specific algorithm to
甜瓜 wrote:
> Well, Database is not proper because 1. the table is very big (~10^9
> rows) 2. we should support very fast *simple* query that is to get
> value corresponding to single key (~10^7 queries / second).
Ah, crypto rainbow tables.
John Nagle
--
Many thanks for your kind reply. As you mentioned, a sparse array may
be the best choice.
Storing offset rather than payload itself can greatly save memory space.
1e7 queries per second is my ideal aim. But 1e6 must be achieved.
Currently I have implemented 5e6 on one PC (without incremental
index
On 25-3-2010 10:55, 甜瓜 wrote:
> Thank you irmen. I will take a look at pytable.
> FYI, let me explain the case clearly.
>
> Originally, my big data table is simply array of Item:
> struct Item
> {
> long id;// used as key
> BYTE payload[LEN]; // corresponding value with fixed lengt
Irmen de Jong :
> On 3/25/10 4:28 AM, 甜瓜 wrote:
>>
>> Howdy,
>>
>> Recently, I am finding a good library for build index on binary data.
>> Xapian& Lucene for python binding focus on text digestion rather than
>> binary data. Could anyone give me some reco
Thank you Rubin! Let me have a look at Judy. It seems good at first glance.
--
ShenLei
2010/3/25 Paul Rubin :
> 甜瓜 writes:
>> Well, Database is not proper because 1. the table is very big (~10^9
>> rows) 2. we should support very fast *simple* query that is to get
>> value corresponding to singl
甜瓜 writes:
> Well, Database is not proper because 1. the table is very big (~10^9
> rows) 2. we should support very fast *simple* query that is to get
> value corresponding to single key (~10^7 queries / second).
Just one numeric key/value pair in each row? What's wrong with
universal hashing?
On 3/25/10 4:28 AM, 甜瓜 wrote:
Howdy,
Recently, I am finding a good library for build index on binary data.
Xapian& Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data no ma
ext digestion rather than
>> binary data. Could anyone give me some recommendation? Is there any
>> library for indexing binary data no matter whether it is written in
>> python?
>>
>> In my case, there is a very big datatable which stores structured
>> binar
En Thu, 25 Mar 2010 00:28:58 -0300, 甜瓜
escribió:
Recently, I am finding a good library for build index on binary data.
Xapian & Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary
Howdy,
Recently, I am finding a good library for build index on binary data.
Xapian & Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data no matter whether it is written in
python?
11 matches
Mail list logo