Re: updating the default bucket-type to support Solr indexing?

2014-03-24 Thread Ryan Zezeski
There is support for indexing CRDTs. Field definitions are defined in the default schema: https://github.com/basho/yokozuna/blob/develop/priv/default_schema.xml#L96 On Mon, Mar 24, 2014 at 12:33 PM, Paul Walk wrote: > Thanks Luke. > > If I might be allowed a follow-on question, what is the eff

Re: updating the default bucket-type to support Solr indexing?

2014-03-24 Thread Ryan Zezeski
Actually, a bucket type is not required. To associate an index with a bucket you simply set the `search_index` property. Bucket types provide a method for many buckets to inherit the same properties. Setting the search_index property on the bucket type is a way to index multiple buckets under one i

Re: Riak Search 2.0

2014-03-24 Thread Eric Redmond
jsonpp, and the python app are just convenience tools to format the json output in the pretty way. Just don't use it to see the raw output. Eric On Mar 24, 2014, at 10:23 AM, "Sapre, Meghna A" wrote: > I see my index in the bucket properties, but the search query still doesn’t > have anythi

Re: Storing Photos in riak

2014-03-24 Thread Matthew Von-Maszewski
To date, only leveldb. Will be expanding the matrix to bitcask this week for the first time. Not sure where memory backend fits with that particular test scenario. Will ask around. Matthew On Mar 24, 2014, at 2:38 PM, István wrote: > Hi Matthew, > > Just a quick question. Do you test Ria

Re: Storing Photos in riak

2014-03-24 Thread István
Hi Matthew, Just a quick question. Do you test Riak with memory backend with 130k objects or just LevelDB? Regards, Istvan On Mon, Mar 24, 2014 at 5:55 AM, Matthew Von-Maszewski wrote: > Ingo, > > We regularly test Riak and its leveldb backend with 130k objects as part of > our performance and

Re: What happens when a node goes down?

2014-03-24 Thread Jeff Peck
Thank you, this has been very helpful. I appreciate all of the information and the quick responses. - Jeff On Mar 24, 2014, at 2:20 PM, Seth Thomas wrote: > To further elaborate: > > When a nodes fails (or is simply too slow) for a PUT request, that data will > be placed on the first fallba

Re: What happens when a node goes down?

2014-03-24 Thread Seth Thomas
To further elaborate: When a nodes fails (or is simply too slow) for a PUT request, that data will be placed on the first fallback in the preflist as per [1]. For a GET the request only needs to full fill the R[2] value which for an N of 3, is 2. So the GET request would succeed by simply takin

Re: What happens when a node goes down?

2014-03-24 Thread Jeff Peck
Aha, so if a node is detected to go down, but the cluster is not currently receiving any requests (i.e. on a development cluster that is used to store data that is only requested periodically or in batches, etc.) then there would not be any increased I/O unless it is manually ("administratively

Re: What happens when a node goes down?

2014-03-24 Thread Seth Thomas
As immediately as the cluster can detect the node is no longer serving requests[1]. There will likely be increased network and IO among the remaining nodes as they will be picking up the slack. That said, the data is not permanently reshuffled at that point - only such time as it is administrati

Re: Storing Photos in riak

2014-03-24 Thread Matthew Von-Maszewski
Ingo, Maximum size involves more components of Riak than my specialty area of leveldb. So I did some asking around the engineering room. The word back is: - 1M is a typical, recommended maximum, - if you care about worst case latencies (tail latencies), should stay below 100K - if you really,

Re: What happens when a node goes down?

2014-03-24 Thread Jeff Peck
Does that happen immediately? I am basically trying to understand: When a physical node goes down (let's say it is temporarily restarted, or down for even a couple hours due to some sort of failure), will that cause an increase in disk and network bandwidth at the moment that it goes down as dat

Re: What happens when a node goes down?

2014-03-24 Thread Seth Thomas
Data is redistributed temporarily (indefinitely) until the primary node comes back online. So primary ownership of data would not be changed but your keys could be living on another physical node if any of the primary replicas were down. So to answer your question directly: Yes (in the narrowes

Re: What happens when a node goes down?

2014-03-24 Thread Jeff Peck
Thank you. So, does that mean that no redistribution of data would occur unless the node is manually removed? - Jeff On Mar 24, 2014, at 1:31 PM, Seth Thomas wrote: > Jeff, > > When a node is no longer responding a process called hinted handoff[1] takes > over and ensure that your N (replic

Re: What happens when a node goes down?

2014-03-24 Thread Seth Thomas
Jeff, When a node is no longer responding a process called hinted handoff[1] takes over and ensure that your N (replication) value is met by allowing other nodes to temporarily take responsibility for the vnodes of the downed node. This node can return to the cluster and will resume operations

RE: Riak Search 2.0

2014-03-24 Thread Sapre, Meghna A
I see my index in the bucket properties, but the search query still doesn't have anything. The objects are in the bucket. $ curl "$RIAK_HOST/search/famous?wt=json&q=name_s:lion*" | python -mjson.tool % Total% Received % Xferd Average Speed TimeTime Time Current

Re: Storing Photos in riak

2014-03-24 Thread Ingo Rockel
Hi Matthew, thanks for the Infos. What is the maximum size of objects which I can store into riak without the need of splitting them and without suffering from performance issues? Best, Ingo Am 24.03.2014 13:55, schrieb Matthew Von-Maszewski: Ingo, We regularly test Riak and its l

Re: updating the default bucket-type to support Solr indexing?

2014-03-24 Thread Paul Walk
Thanks Luke. If I might be allowed a follow-on question, what is the effect of adding an index to a 'typed bucket-type'. For example, if I define a bucket_type as follows: ./riak-admin bucket-type create map_bucket_type '{"props":{"search_index":"my_index","datatype":"map"}}' Are the members

What happens when a node goes down?

2014-03-24 Thread Jeff Peck
Is there a description of what happens internally when a node goes down? I am curious if any there would be any sort of reshuffling or redistribution of data in the remaining vnodes? Or would the node simply be unavailable until restarted? Thanks, Jeff __

Re: updating the default bucket-type to support Solr indexing?

2014-03-24 Thread Luke Bakken
Hi Paul, You are correct, a new bucket type must be created for Riak 2.0 search indexes. -- Luke Bakken CSE lbak...@basho.com On Mon, Mar 24, 2014 at 2:27 AM, Paul Walk wrote: > I'm experimenting with the technology preview of Riak 2.0, using an > existing Ruby web application which uses the

Re: Storing Photos in riak

2014-03-24 Thread Matthew Von-Maszewski
Ingo, We regularly test Riak and its leveldb backend with 130k objects as part of our performance and qualification tests. Works just fine. The one thing to remember is that with N=3 redundancy, the bandwidth between the nodes will approach 3 times the incoming user bandwidth. Make sure your

Storing Photos in riak

2014-03-24 Thread Ingo Rockel
Hi List, I'm currently evaluating the possibilities to migrate our mysql-based image storage to something which actually scales and is easier to maintain. Among different distributed file systems (like ceph, openstack swift) I was looking at riak as we already have a riak cluster for storing

updating the default bucket-type to support Solr indexing?

2014-03-24 Thread Paul Walk
I'm experimenting with the technology preview of Riak 2.0, using an existing Ruby web application which uses the official Ruby client gem (1.4.3). My understanding is that if I have not specified particular bucket-types, then my buckets are implicitly using a 'default' bucket type. So, in order