You may want to consider how often you will be accessing all 10,000 files in one query.
It is my understanding that with a key/value store such as Riak, it's a good idea to analyze what your common queries will be as long as your frequency of reading vs writing, and then to structure your data so that when you do a write, you precompute and denormalize as needed to satisfy those queries. For less frequent queries, you use map/reduce as needed to get those results, then perhaps determine whether and for how long to cache those results. This is of course different than the managed indexes given by other styles of data stores, but is part of the tradeoff involved. -- Matthew Scott ElevenCraft, Inc. http://11craft.com/ +1 360 389-2512 On Sat, Sep 4, 2010 at 11:51, John Axel Eriksson <[email protected]> wrote: > Yes, I've thought of this but as I understand it there is a bit of a > problem attaching a large amount of links to a key which > would be necessary here am I right? If I had 10 000 files in riak that > would mean 10 000 links attached to the "listing" key. > > 4 sep 2010 kl. 20.44 skrev Matthew Scott: > > > On Sat, Sep 4, 2010 at 11:31, John Axel Eriksson <[email protected]> wrote: > >> Listing keys in a bucket has been described as "bad" and something you use >> in development but >> not in production. I'm just starting out on Riak so I'm a newbie... >> >> I'm thinking of building an application using Riak as filestorage and >> possibly much more than that, but it would >> at least store lots of files with, perhaps, metadata attached. How would I >> then list files for display in a webapp if >> I don't use key listing? >> > > I'm still very much a Riak newb myself, but I'll take a shot at answering > this one by suggesting the use of links. > > From what I understand, links some quantity limits when you attach a large > number, but adding and removing them is an inexpensive operation. (Someone > fact check me on that please :) > > - Create a key called 'listing', perhaps even in its own bucket to prevent > namespace collision. > > - Create links from that 'listing' key to metadata keys. Remember that you > can attach a tag to each link to differentiate different types of links, > such as "metadata". > > - The metadata keys' values would contain file metadata in JSON form, and > in turn have a link to the file contents key, tagged "contents". > > - Remember to attach the proper mime type to the key containing your file > contents. > > - To get a file listing, do a map/reduce starting at "listing", following > its "metadata" links and grabbing those values, then reduce to sort by a > key. > > - To get file contents, follow the "contents" link from the metadata key. > > -- > Matthew Scott > ElevenCraft, Inc. > http://11craft.com/ > +1 360 389-2512 > > > > >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
