Riak Recap, June 3rd, 2016

[Alexander Sicular]

Hello All,

Just in time for the weekend we bring you a rundown of... the mailing list!

As always, we want to hear from you. Let me know if you’re working on or
speaking about something Riak related and would like a shoutout. I’ll get
your talk, project or blog post in the next Riak Recap.

## Community Code and Walkthroughs


   - Dan Cunnington shows us how to monitor Riak from NodeJS via an
   Appmetrics integration [0].



##Community Events


   -

   Basho will be at Spark Summit SF next week [1]. Stop by and say hi.



## Jobs at Basho

Interested in working on distributed computing related problems or helping
our community with theirs? Consider working at Basho.



   -

   Client Services Engineer (USA) [2].
   -

   Developer Advocate (London and US) [3].


Yes, we're remote friendly.


## Recently Answered


   -

   Guillaume was able to confirm that Solr is a resource hog and indeed
   needs resources to operate within tolerable limits [4].
   -

   Luke shares some Solr links with Alex to help grok relevancy queries [5].
   -

   Seema lets us know that Basho is no longer supporting FreeBSD [6].
   -

   Luke helps Alex get to the bottom of some Stanchion/RiakS2 (formerly
   Riak CS) related questions. Yes, Stanchion is an oracle and should only
   have one running process in your cluster [7].
   -

   Robert has a question about Erlang versions [8]. Sargun points out that
   kerl [9] is a great way to manage multiple Erlang versions on the same box
   and Luke notes that Riak packages come bundled with the correct Erlang
   version [10].
   -

   Vikram was having issues with data availability during different failure
   scenarios in a three node cluster [11]. DeadZen and I point out that this
   is most likely a quorum situation and how to ensure you always have access
   to your data going forward. Incidentally, always being able to access your
   data is exactly what Riak was designed to do.
   -

   Nick answers Richards question on the Riak Erlang clients
   thread-safeness [12]. Yes, connections may be shared in the
   riak-erlang-client [13].
   -

   Johnny and Alex are looking for information on changing a clusters ring
   size [14]. Although it is possible in 2.x, the feature is more or less
   deprecated. Changing the ring size is not without caveats and performance
   considerations [15].
   -

   Vikram is concerned with how Riak distributes data in a cluster [16] and
   Sam provides a number of great links that review the subject [17].
   -

   Norman is interested in using Riak as a session store backing in a
   NodeJS/Express application [18]. Riak is indeed quite adept at storing
   session data and Luke points out [19] a possible way forward by using the
   connect-memcached lib as a starting point by replacing its memchached
   client lib with the Riak nodejs client lib [20].



## Open Discussions


   -

   Kyle is looking for information on Riak TS drop table functionality and
   HTTP auth support [21].
   -

   Guillaume kicks off a conversation on custom Solr configs [22] and Luke
   points to an open issue on github reviewing some considerations [23].
   -

   Anil is looking for ways to copy Solr indecies from one machine to
   another [24].
   -

   Murali is looking for information around Riak S2 (formerly CS)
   performance on a single node [25].
   -

   Vladimir asks the list for help with the following warning “perflists do
   not satisfy the n_val” [26].
   -

   Alex is looking for advice regarding whether or not Riak TS would be
   better suited in certain use cases over Riak KV for his social network
   project [27].
   -

   Michael is looking for some guidance on properly sizing a Riak S2
   cluster [28].
   -

   Psterk is looking for help on using hadoops’s distcpy to copy files from
   Riak S2 to hadoop [29][30].



Do have yourselves a great weekend,

-Alexander Sicular

Solution Architect, Basho

@siculars


[0] https://developer.ibm.com/open/2016/05/20/riak-monitoring-appmetrics/

[1] https://spark-summit.org/2016/

[2]
http://bashojobs.theresumator.com/apply/0CTNKU/Client-Services-Engineer-Remote

[3] http://bashojobs.theresumator.com/apply/nVY7Ea/Developer-Advocate-EMEA

[4]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018484.html

[5]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018481.html

[6]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018482.html

[7]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018455.html

[8]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018456.html

[9] https://github.com/kerl/kerl

[10]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018459.html

[11]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018468.html

[12]
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-May/018475.html

[13] https://github.com/basho/riak-erlang-client

[14]

Re: Put failure: too many siblings

[Luke Bakken]

Hi Vladyslav,

If you recognize the full name of the object raising the sibling
warning, it is most likely a manifest object. Sometimes, during hinted
handoff, you can see these messages. They should resolve after handoff
completes.

Please see the documentation for the transfer-limit command as well:

http://docs.basho.com/riak/kv/2.1.4/using/admin/riak-admin/#transfer-limit

--
Luke Bakken
Engineer
lbak...@basho.com


On Fri, Jun 3, 2016 at 2:55 AM, Vladyslav Zakhozhai
 wrote:
> Hi.
>
> I have a trouble with PUT to Riak CS cluster. During this process I
> periodically see the following message in Riak error.log:
>
> 2016-06-03 11:15:55.201 [error]
> <0.15536.142>@riak_kv_vnode:encode_and_put:2253 Put failure: too many
> siblings for object OBJECT_NAME (101)
>
> and also
>
> 2016-06-03 12:41:50.678 [error]
> <0.20448.515>@riak_api_pb_server:handle_info:331 Unrecognized message
> {7345880,{error,{too_many_siblings,101}}}
>
> Here OBJECT_NAME - is the name of object in Riak which has too many
> siblings.
>
> I definitely sure that this objects are static. Nobody deletes is, nobody
> rewrites it. I have no idea why more than 100 siblings of this object
> occurs.
>
> The following effect of this issue occurs:
>
> Great amount of keys are loaded to RAM. I almost out of RAM (Do each sibling
> has it own key or key duplicate?).
> Nodes are slow - adding new nodes are too slow
> Presence of "too many siblings" affects ownership handoffs
>
> So I have several questions:
>
> Do hinted or ownership handoffs can affect siblings count (I mean can
> siblings be created during ownership of hinted handoffs)
> Is there any workaround of this issue. Do I need remove siblings manually or
> it removes during merges, read repairs and so on
>
>
> My configuration:
>
> riak from basho's packages - 2.1.3-1
> riak cs from basho's packages - 2.1.0-1
> 24 riak/riak-cs nodes
> 32 GB RAM per node
> AAE is disabled
>
>
> I appreciate you help.
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: using riak to store user sessions

[Alex Moore]

Hi Norman,

It's still the case. Riak KV was designed to be highly available, and
highly scalable key/value store with predictable low latency.  This lets
you scale to a few or as many machines as needed, while removing any single
failure points that a single-machine database or cache might impose.

We unfortunately don't have connectors for every language / platform /
framework combination, the explosive exponential swath of support would be
exhausting.  We do keep well maintained client libraries
, and
community libraries and plugins are always growing in number. I think
Luke's suggestion would provide a good way forward, and we'd be happy to
help with any questions you have along the way.

Thanks,
Alex

On Thu, Jun 2, 2016 at 4:51 PM Norman Khine  wrote:

> hi luke, thanks for the reply, in the riak document "Riak KV is uniquely
> architected to handle user and session data..." or is this no longer the
> case?
>
>
> On 2 June 2016 at 17:40, Luke Bakken  wrote:
> >
> > Hi Norman,
> >
> > A quick search turns up this Node.js module:
> > https://www.npmjs.com/package/express-session
> >
> > There is currently not a session store for Riak
> > (https://www.npmjs.com/package/express-session#compatible-session-stores
> )
> >
> > Since memcached operates as a key/value store, forking the connector
> > for that session store as a starting point would be your best bet.
> >
> > https://github.com/balor/connect-memcached
> >
> > You would then use the Riak Node.js client to fetch and store data:
> > https://github.com/basho/riak-nodejs-client
> >
> > --
> > Luke Bakken
> > Engineer
> > lbak...@basho.com
> >
> >
> > On Thu, Jun 2, 2016 at 6:19 AM, Norman Khine  wrote:
> > > hello, i am trying to setup a node.js/express application to use riak
> to
> > > store the logged in users' sessions as detailed
> > > http://basho.com/use-cases/session-data/
> > >
> > > i have setup the dev cluster on my machine and everything is running
> fine.
> > >
> > > what is the correct way to set this up?
> > >
> > > any advise much appreciated.
> > >
> > >
> > >
> > > --
> > > %>>> "".join( [ {'*':'@','^':'.'}.get(c,None) or
> chr(97+(ord(c)-83)%26) for
> > > c in ",adym,*)^zqf" ] )
> > >
> > > ___
> > > riak-users mailing list
> > > riak-users@lists.basho.com
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
>
>
>
>
> --
> %>>> "".join( [ {'*':'@','^':'.'}.get(c,None) or chr(97+(ord(c)-83)%26)
> for c in ",adym,*)^zqf" ] )
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Put failure: too many siblings

[Vladyslav Zakhozhai]

Hi.

I have a trouble with PUT to Riak CS cluster. During this process I
periodically see the following message in Riak error.log:

2016-06-03 11:15:55.201 [error]
<0.15536.142>@riak_kv_vnode:encode_and_put:2253 Put failure: too many
siblings for object OBJECT_NAME (101)

and also

2016-06-03 12:41:50.678 [error]
<0.20448.515>@riak_api_pb_server:handle_info:331 Unrecognized message
{7345880,{error,{too_many_siblings,101}}}

Here OBJECT_NAME - is the name of object in Riak which has too many
siblings.

I definitely sure that this objects are static. Nobody deletes is, nobody
rewrites it. I have no idea why more than 100 siblings of this object
occurs.

The following effect of this issue occurs:

   1. Great amount of keys are loaded to RAM. I almost out of RAM (Do each
   sibling has it own key or key duplicate?).
   2. Nodes are slow - adding new nodes are too slow
   3. Presence of "too many siblings" affects ownership handoffs

So I have several questions:

   1. Do hinted or ownership handoffs can affect siblings count (I mean can
   siblings be created during ownership of hinted handoffs)
   2. Is there any workaround of this issue. Do I need remove siblings
   manually or it removes during merges, read repairs and so on


My configuration:

   1. riak from basho's packages - 2.1.3-1
   2. riak cs from basho's packages - 2.1.0-1
   3. 24 riak/riak-cs nodes
   4. 32 GB RAM per node
   5. AAE is disabled


I appreciate you help.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com