Re: riak_core 2.0.0rc1
Karolis, While there are no planned changes for 2.0.0 in riak_core specifically, at this time, we always suggest waiting until the final release to use it in production. Additionally, Riak Core 2.0.0, like many other Riak Core releases may not be 100% backwards compatible. We encourage testing your application with the 2.0.0rc1 release. Please let us know if you find any issues. Cheers, Jordan On Mon, Jul 28, 2014 at 11:24 PM, Karolis Petrauskas wrote: > Hello, > > What is the status of riak_core in 2.0.0rc1? Is it resonable to start > using it instead of 1.4.10 for production? Or are there big changes > pending till the final release? > > Karolis > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RE: Riak and Spring
Hi,
Thanks, for your suggestions.
I've done something similar, but injected the client config directly into the
client bean in the application configuration (not via the
HTTPClientConfigFactory). E.g:
@ComponentScan
@EnableAutoConfiguration
public class Application {
@Value( "${riak.pool.host}" ) private String poolHost;
@Value( "${riak.pool.port}" ) private int poolPort;
@Value( "${riak.pool.timeout.connection}" ) private long
poolTimeoutConnection;
@Value( "${riak.pool.timeout.idle}" ) private long poolTimeoutIdle;
@Value( "${riak.pool.timeout.request}" ) private int poolTimeoutRequest;
@Value( "${riak.pool.buffersize}" ) private int poolBufferSize;
@Bean
public static PropertyPlaceholderConfigurer properties(){
PropertyPlaceholderConfigurer ppc = new PropertyPlaceholderConfigurer();
ClassPathResource[] resources = new ClassPathResource[ ]{ new
ClassPathResource( "app.properties" ) };
ppc.setLocations(resources);
ppc.setIgnoreUnresolvablePlaceholders(true);
return ppc;
}
@Bean
@Scope("singleton")
IRiakClient riakClient() throws Exception{
//create a single client connection configuration using the
property file values
Configuration config = (new PBClientConfig.Builder())
.withConnectionTimeoutMillis(poolTimeoutConnection)
.withIdleConnectionTTLMillis(poolTimeoutIdle)
.withRequestTimeoutMillis(poolTimeoutRequest)
.withHost(poolHost)
.withPort(poolPort)
.withSocketBufferSizeKb(poolBufferSize)
.build();
//create the singleton Riak client using the connection
configuration
IRiakClient client = RiakFactory.newClient(config);
return client;
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
I'm not aware of any obvious benefits of one method over the other. If anyone
can enlighten me please do so :)
Thanks,
Simon.
From: Mark Richard Thomas [mailto:mark.tho...@equifax.com]
Sent: 06 August 2014 09:50
To: Simon Hartley; riak-users@lists.basho.com
Subject: RE: Riak and Spring
Hello
1) Create a riakClient bean (applicationContext.xml)
2) create a HTTPClientConfigFactory bean in which you can specify the host
and port:
package com.webapp.util;
import org.apache.http.client.HttpClient;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import com.basho.riak.client.raw.http.HTTPClientConfig;
@Component
public class HTTPClientConfigFactory implements FactoryBean {
@Value("${${environment}.host}")
private String host;
@Value("${${environment}.port}")
private int port;
@Autowired
HttpClient httpClient;
public HTTPClientConfig getObject() throws Exception {
return new
HTTPClientConfig.Builder().withHttpClient(httpClient)
.withHost(host).withPort(port).build();
}
@Override
public Class getObjectType() {
return HTTPClientConfig.class;
}
@Override
public boolean isSingleton() {
return true;
}
@Override
protected void finalize() throws Throwable {
httpClient.getConnectionManager().shutdown();
}
}
3) Use the RiakClient within a service:
@Autowired
private IRiakClient client;
public Name getName(String sname) {
try {
Name name = client.fetchBucket(NAMEBUCKET).execute()
.fetch(sname, Name.class).execute();
if (name == null) {
throw new ResourceNotFoundException(sname);
}
return name;
} catch (UnresolvedConflictException | RiakRetryFailedException
| ConversionException e) {
throw new ServiceUnavailableException(sname);
RE: Riak and Spring
Hello
1) Create a riakClient bean (applicationContext.xml)
2) create a HTTPClientConfigFactory bean in which you can specify the host
and port:
package com.webapp.util;
import org.apache.http.client.HttpClient;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import com.basho.riak.client.raw.http.HTTPClientConfig;
@Component
public class HTTPClientConfigFactory implements FactoryBean {
@Value("${${environment}.host}")
private String host;
@Value("${${environment}.port}")
private int port;
@Autowired
HttpClient httpClient;
public HTTPClientConfig getObject() throws Exception {
return new
HTTPClientConfig.Builder().withHttpClient(httpClient)
.withHost(host).withPort(port).build();
}
@Override
public Class getObjectType() {
return HTTPClientConfig.class;
}
@Override
public boolean isSingleton() {
return true;
}
@Override
protected void finalize() throws Throwable {
httpClient.getConnectionManager().shutdown();
}
}
3) Use the RiakClient within a service:
@Autowired
private IRiakClient client;
public Name getName(String sname) {
try {
Name name = client.fetchBucket(NAMEBUCKET).execute()
.fetch(sname, Name.class).execute();
if (name == null) {
throw new ResourceNotFoundException(sname);
}
return name;
} catch (UnresolvedConflictException | RiakRetryFailedException
| ConversionException e) {
throw new ServiceUnavailableException(sname);
}
}
Mark Thomas | Software Engineer | Equifax UK
p: +44 (0)208 941 0573
m: +44 (0)7908 798 270
e: mark.tho...@equifax.com
Equifax Ltd, Capital House, 25 Chapel Street, London, NW1 5DS
From: riak-users [mailto:riak-users-boun...@lists.basho.com] On Behalf Of Simon
Hartley
Sent: 30 July 2014 15:37
To: riak-users@lists.basho.com
Subject: [IE]:Riak and Spring
Hi,
I'm looking at using a Riak backend in a new component.
My current client is a Spring shop. I'm not all that knowledgable about the
intricaies of Spring , so this question many not make sense, but I am hoping to
integrate Riak in a Spring friendly way.
I was looking at the
spring-projects/spring-data-keyvalue
project, but that's been deprecated in favour of
spring-projects/spring-data-riak
which is empty.
Does anyone know of a Spring-Riak integration project that is high quality and
currently active - preferably with excellent documentation and examples ;)
Thanks,
Simon.
Confidentiality: The contents of this e-mail and any attachments transmitted
with it are intended to be confidential to the intended recipient; and may be
privileged or otherwise protected from disclosure. If you are not an intended
recipient of this e-mail, do not duplicate or redistribute it by any means.
Please delete it and any attachments and notify the sender that you have
received it in error. This e-mail is sent by a William Hill PLC group company.
The William Hill group companies include, among others, William Hill PLC
(registered number 4212563), William Hill Organization Limited (registered
number 278208), William Hill US HoldCo Inc, WHG (International) Limited
(registered number 99191) and WHG Trading Limited (registered number 101439).
Each of William Hill PLC, William Hill Organization Limited is registered in
England and Wales and has its registered office at Greenside House, 50 Station
Road, Wood Green, London N22 7TP. William Hill U.S. HoldCo, Inc. is 160
Greentree Drive, Suite 101, Dover 19904, Kent, Delaware, United States of
America. Each of WHG (International) Limited and WHG Trading Limited is
registered in Gibraltar and has its registered office at 6/1 Waterport Place,
Gibraltar. Unless specifically indicated otherwise, the contents of this e-mail
are subject to contract; and are not an official statement, and do not
necessarily represent the views, of William Hill PLC, its subsidiaries or
affiliated companies. Please note that neither William Hill PLC, nor its
subsidiaries and affiliated companies ca
Re: repair-2i stops with "bad argument in call to eleveldb:async_write"
Simon: The data scan for that partition seems to be taking more than 5
minutes to collect a batch of 1000 items, so the 2i repair process is
giving up on it before it has a chance to finish. You can reduce the
likelihood of this happening by configuring the batch parameter to
something small. In the riak_kv section of the configuration file, set
this:
{riak_kv, [
{aae_2i_batch_size, 10},
...
Let us know if that allows it to finish the repair. You should still look
into what may be causing the slowness. A combination of slow disks or very
large data sets might do it.
On Fri, Aug 1, 2014 at 5:24 AM, Russell Brown wrote:
> Hi Simon,
> Sorry for the delays. I’m on vacation for a couple of days. Will pick this
> up on Monday.
>
> Cheers
>
> Russell
>
> On 1 Aug 2014, at 09:56, Effenberg, Simon
> wrote:
>
> > Hi Russell, @basho
> >
> > any updates on this? We still have the issues with 2i (repair is also
> > still not possible) and searching for the 2i indexes is reproducable
> > creating (for one range I tested) 3 different values.
> >
> > I would love to provide anything you need to debug that issue.
> >
> > Cheers
> > Simon
> >
> > On Wed, Jul 30, 2014 at 09:22:56AM +, Effenberg, Simon wrote:
> >> Great. Thanks Russell..
> >>
> >> if you need me to do something.. feel free to ask.
> >>
> >> Cheers
> >> Simon
> >>
> >> On Wed, Jul 30, 2014 at 10:19:56AM +0100, Russell Brown wrote:
> >>> Thanks Simon,
> >>>
> >>> I’m going to spend a some time on this day.
> >>>
> >>> Cheers
> >>>
> >>> Russell
> >>>
> >>> On 30 Jul 2014, at 10:05, Effenberg, Simon
> wrote:
> >>>
> Hi Russel,
>
> still one machine out of 13 is on wheezy and the rest on squeeze but
> the
> software is the same and basho is providing even the erlang stuff. So
> their should no real difference inside the application.
>
> And the errors are almost the same (except the async_write/read
> difference).
>
> I paste them:
>
> -- node 1 ---
>
> 2014-07-30 06:16:07.728 UTC [info]
> <0.14871.336>@riak_kv_2i_aae:next_partition:160 Finished 2i repair:
> Total partitions: 1
> Finished partitions: 1
> Speed: 100
> Total 2i items scanned: 0
> Total tree objects: 0
> Total objects fixed: 0
> With errors:
> Partition: 12559779695812446953312916531172001681702912
> Error: index_scan_timeout
>
>
> 2014-07-30 06:16:07.728 UTC [error] <0.1525.0> gen_server <0.1525.0>
> terminated with reason: bad argument in call to
> eleveldb:async_write(#Ref<0.0.324.211123>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97
> ,116,105,111,110,95,115,101,99,114,...>>,...}], []) in
> eleveldb:write/3 line 155
> 2014-07-30 06:16:07.728 UTC [error] <0.1525.0> CRASH REPORT Process
> <0.1525.0> with 0 neighbours exited with reason: bad argument in call to
> eleveldb:async_write(#Ref<0.0.324.211123>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,11
>
> 1,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}], [])
> in eleveldb:write/3 line 155 in gen_server:terminate/6 line 747
> 2014-07-30 06:16:07.728 UTC [error] <0.1517.0> Supervisor
> {<0.1517.0>,poolboy_sup} had child riak_core_vnode_worker started with
> {riak_core_vnode_worker,start_link,undefined} at <0.1525.0> exit with
> reason bad argument in call
> to eleveldb:async_write(#Ref<0.0.324.211123>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}],
> []) in eleveldb:write/3 line 155 in context child_terminated
>
>
> -- node 2 ---
>
> 2014-07-30 06:16:07.791 UTC [info]
> <0.8083.314>@riak_kv_2i_aae:next_partition:160 Finished 2i repair:
> Total partitions: 1
> Finished partitions: 1
> Speed: 100
> Total 2i items scanned: 0
> Total tree objects: 0
> Total objects fixed: 0
> With errors:
> Partition: 622279994019798508141412682679979879462877528064
> Error: index_scan_timeout
>
>
> 2014-07-30 06:16:07.791 UTC [error] <0.1884.0> gen_server <0.1884.0>
> terminated with reason: bad argument in call to
> eleveldb:async_write(#Ref<0.0.318.96628>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,
> 116,105,111,110,95,115,101,99,114,...>>,...}], []) in
> eleveldb:write/3 line 155
> 2014-07-30 06:16:07.791 UTC [error] <0.1884.0> CRASH REPORT Process
> <0.1884.0> with 0 neighbours exited with reason: bad argument in call to
> eleveldb:async_write(#Ref<0.0.318.96628>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,111
>
> ,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}], [])
> in eleveldb:write/3 line 155 in gen_server:terminate/6 line 747
> 2014-07-30 06:16:07.792 UTC [error] <0.1875.0> Supervisor
> {<0.1875.0>,poolboy_sup} had c
repair-2i not working as expected?
Hi, We have had a problem now twice on Ubuntu 12.04 with MDM arrays (all nodes Raid 1 with 2x2TB disks) where once a disk fails 2i queries don't return the expected result, the only solution we have been able to apply is by replacing the affected node. After doing an AAE clean up, repaired partitions and repaired 2i the problem still persists, would this be an indication of a problem with repair-2i not doing what is supposed to do? Once a disk is gone, MDM says the array consist of only one disk and the server seems fine, keys are fine and no data is lost but it seems like 2i once corrupted is never back to a correct state unless the problematic node is replaced. We have had this problem with 1.4.7 (or 8) and now with 1.4.10 Any thoughts? Best regards, Guido. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: repair-2i stops with "bad argument in call to eleveldb:async_write"
Simon,
If you want to get more verbose logging information, you could perform the
following to change the logging level, to debug, then run `repair-2i`, and
finally switching back to the normal logging level.
- `riak attach`
- `(riak@nodename)1> SetDebug = fun() -> {node(),
lager:set_loglevel(lager_file_backend, "/var/log/riak/console.log", debug)}
end.`
- `(riak@nodename)2> rp(rpc:multicall(erlang, apply, [SetDebug,[]])).`
(don't forget the period at the end of these statements)
- Hit CTRL+C twice to quit from the node
You can then revert back to the normal `info` logging level by running the
following command via `riak attach`:
- `riak attach`
- `(riak@nodename)1> SetInfo = fun() -> {node(),
lager:set_loglevel(lager_file_backend, "/var/log/riak/console.log", info)} end.`
- `(riak@nodename)2> rp(rpc:multicall(erlang, apply, [SetInfo,[]])).`
(don't forget the period at the end of these statements)
- Hit CTRL+C twice to quit from a the node
Please also see the docs for info on `riak attach` monitoring of repairs.
http://docs.basho.com/riak/1.4.9/ops/running/recovery/repairing-partitions/#Monitoring-Repairs
Repairs can also be monitored using the `riak-admin transfers` command.
http://docs.basho.com/riak/1.4.9/ops/running/recovery/repairing-partitions/#Running-a-Repair
Best Regards,
Bryan Hunt
Bryan Hunt - Client Services Engineer - Basho Technologies Limited - Registered
Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
On 6 Aug 2014, at 12:53, Effenberg, Simon wrote:
> Hi Engel,
>
> I tried it yesterday but it was the same:
>
> 2014-08-05 17:53:14.728 UTC [info]
> <0.24306.9>@riak_kv_2i_aae:repair_partition:257 Acquired lock on partition
> 548063113999088594326381812268606132370974703616
> 2014-08-05 17:53:14.728 UTC [info]
> <0.24306.9>@riak_kv_2i_aae:repair_partition:259 Repairing indexes in
> partition 548063113999088594326381812268606132370974703616
> 2014-08-05 17:53:14.753 UTC [info]
> <0.24306.9>@riak_kv_2i_aae:create_index_data_db:324 Creating temporary
> database of 2i data in /var/lib/riak/anti_entropy/2i/tmp_db
> 2014-08-05 17:53:14.772 UTC [info]
> <0.24306.9>@riak_kv_2i_aae:create_index_data_db:361 Grabbing all index data
> for partition 548063113999088594326381812268606132370974703616
> 2014-08-05 17:58:14.773 UTC [info]
> <0.24305.9>@riak_kv_2i_aae:next_partition:160 Finished 2i repair:
>Total partitions: 1
>Finished partitions: 1
>Speed: 100
>Total 2i items scanned: 0
>Total tree objects: 0
>Total objects fixed: 0
> With errors:
> Partition: 548063113999088594326381812268606132370974703616
> Error: index_scan_timeout
>
> Can't we use some erlang commands to execute parts of this manually to check
> where the timeout actually happens? Or at least who is timing out?
>
> Cheers
> Simon
>
> On Tue, Aug 05, 2014 at 10:21:57AM -0400, Engel Sanchez wrote:
>> Simon: The data scan for that partition seems to be taking more than 5
>> minutes to collect a batch of 1000 items, so the 2i repair process is
>> giving up on it before it has a chance to finish. You can reduce the
>> likelihood of this happening by configuring the batch parameter to
>> something small. In the riak_kv section of the configuration file, set
>> this:
>> {riak_kv, [
>> {aae_2i_batch_size, 10},
>> ...
>> Let us know if that allows it to finish the repair. You should still look
>> into what may be causing the slowness. A combination of slow disks or
>> very large data sets might do it.
>>
>> On Fri, Aug 1, 2014 at 5:24 AM, Russell Brown
>> wrote:
>>
>> Hi Simon,
>> Sorry for the delays. I'm on vacation for a couple of days. Will pick
>> this up on Monday.
>>
>> Cheers
>> Russell
>> On 1 Aug 2014, at 09:56, Effenberg, Simon
>> wrote:
>>
>>> Hi Russell, @basho
>>>
>>> any updates on this? We still have the issues with 2i (repair is also
>>> still not possible) and searching for the 2i indexes is reproducable
>>> creating (for one range I tested) 3 different values.
>>>
>>> I would love to provide anything you need to debug that issue.
>>>
>>> Cheers
>>> Simon
>>>
>>> On Wed, Jul 30, 2014 at 09:22:56AM +, Effenberg, Simon wrote:
Great. Thanks Russell..
if you need me to do something.. feel free to ask.
Cheers
Simon
On Wed, Jul 30, 2014 at 10:19:56AM +0100, Russell Brown wrote:
> Thanks Simon,
>
> I'm going to spend a some time on this day.
>
> Cheers
>
> Russell
>
> On 30 Jul 2014, at 10:05, Effenberg, Simon
>> wrote:
>
>> Hi Russel,
>>
>> still one machine out of 13 is on wheezy and the rest on squeeze
>> but the
>> software is the same and basho is providing even the erlang stuff.
>> So
>> their should no real difference inside the application.
>>
>> And the errors are almost the same (except the async_write/read
>> differen
Re: repair-2i stops with "bad argument in call to eleveldb:async_write"
Hi Engel,
I tried it yesterday but it was the same:
2014-08-05 17:53:14.728 UTC [info]
<0.24306.9>@riak_kv_2i_aae:repair_partition:257 Acquired lock on partition
548063113999088594326381812268606132370974703616
2014-08-05 17:53:14.728 UTC [info]
<0.24306.9>@riak_kv_2i_aae:repair_partition:259 Repairing indexes in partition
548063113999088594326381812268606132370974703616
2014-08-05 17:53:14.753 UTC [info]
<0.24306.9>@riak_kv_2i_aae:create_index_data_db:324 Creating temporary database
of 2i data in /var/lib/riak/anti_entropy/2i/tmp_db
2014-08-05 17:53:14.772 UTC [info]
<0.24306.9>@riak_kv_2i_aae:create_index_data_db:361 Grabbing all index data for
partition 548063113999088594326381812268606132370974703616
2014-08-05 17:58:14.773 UTC [info]
<0.24305.9>@riak_kv_2i_aae:next_partition:160 Finished 2i repair:
Total partitions: 1
Finished partitions: 1
Speed: 100
Total 2i items scanned: 0
Total tree objects: 0
Total objects fixed: 0
With errors:
Partition: 548063113999088594326381812268606132370974703616
Error: index_scan_timeout
Can't we use some erlang commands to execute parts of this manually to check
where the timeout actually happens? Or at least who is timing out?
Cheers
Simon
On Tue, Aug 05, 2014 at 10:21:57AM -0400, Engel Sanchez wrote:
>Simon: The data scan for that partition seems to be taking more than 5
>minutes to collect a batch of 1000 items, so the 2i repair process is
>giving up on it before it has a chance to finish. You can reduce the
>likelihood of this happening by configuring the batch parameter to
>something small. In the riak_kv section of the configuration file, set
>this:
>{riak_kv, [
> {aae_2i_batch_size, 10},
> ...
>Let us know if that allows it to finish the repair. You should still look
>into what may be causing the slowness. A combination of slow disks or
>very large data sets might do it.
>
>On Fri, Aug 1, 2014 at 5:24 AM, Russell Brown
>wrote:
>
> Hi Simon,
> Sorry for the delays. I'm on vacation for a couple of days. Will pick
> this up on Monday.
>
> Cheers
> Russell
> On 1 Aug 2014, at 09:56, Effenberg, Simon
> wrote:
>
> > Hi Russell, @basho
> >
> > any updates on this? We still have the issues with 2i (repair is also
> > still not possible) and searching for the 2i indexes is reproducable
> > creating (for one range I tested) 3 different values.
> >
> > I would love to provide anything you need to debug that issue.
> >
> > Cheers
> > Simon
> >
> > On Wed, Jul 30, 2014 at 09:22:56AM +, Effenberg, Simon wrote:
> >> Great. Thanks Russell..
> >>
> >> if you need me to do something.. feel free to ask.
> >>
> >> Cheers
> >> Simon
> >>
> >> On Wed, Jul 30, 2014 at 10:19:56AM +0100, Russell Brown wrote:
> >>> Thanks Simon,
> >>>
> >>> I'm going to spend a some time on this day.
> >>>
> >>> Cheers
> >>>
> >>> Russell
> >>>
> >>> On 30 Jul 2014, at 10:05, Effenberg, Simon
> wrote:
> >>>
> Hi Russel,
>
> still one machine out of 13 is on wheezy and the rest on squeeze
> but the
> software is the same and basho is providing even the erlang stuff.
> So
> their should no real difference inside the application.
>
> And the errors are almost the same (except the async_write/read
> difference).
>
> I paste them:
>
> -- node 1 ---
>
> 2014-07-30 06:16:07.728 UTC [info]
> <0.14871.336>@riak_kv_2i_aae:next_partition:160 Finished 2i repair:
> Total partitions: 1
> Finished partitions: 1
> Speed: 100
> Total 2i items scanned: 0
> Total tree objects: 0
> Total objects fixed: 0
> With errors:
> Partition: 12559779695812446953312916531172001681702912
> Error: index_scan_timeout
>
>
> 2014-07-30 06:16:07.728 UTC [error] <0.1525.0> gen_server
> <0.1525.0> terminated with reason: bad argument in call to
> eleveldb:async_write(#Ref<0.0.324.211123>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97
> ,116,105,111,110,95,115,101,99,114,...>>,...}], []) in
> eleveldb:write/3 line 155
> 2014-07-30 06:16:07.728 UTC [error] <0.1525.0> CRASH REPORT Process
> <0.1525.0> with 0 neighbours exited with reason: bad argument in call to
> eleveldb:async_write(#Ref<0.0.324.211123>, <<>>,
> [{put,<<131,104,2,109,0,0,0,20,99,11
>
> 1,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}],
> []) in eleveldb:write/3 line 155 in gen_server:terminate/6 line 747
> >
