Re: Riak 2.1.4 crashes with Out of Memory Error

How much ram is in this device? What is the ring_size setting in Riak.conf? Thank you, Matthew Sent from my iPad > On May 4, 2017, at 4:08 AM, Magnus Kessler wrote: > >> On 4 May 2017 at 09:56, Arulappan, Jerald (Jerald) wrote: >> Hi Magnus Kessler, >>

Re: Object not found after successful PUT on S3 API

h using leveldb instead of bitcask is additional > latency, but so far this has not been a problem for us. > > Daniel > > > On Fri, Apr 28, 2017 at 10:15 AM, Matthew Von-Maszewski <matth...@basho.com > <mailto:matth...@basho.com>> wrote: > Daniel, > > S

Re: Object not found after successful PUT on S3 API

Daniel, Something is wrong. All instances of leveldb within a node share the total memory configuration. The memory is equally divided between all active vnodes. It is possible to create an OOM situation if total RAM is low and vnodes count per node is high relative to RAM size. The best

Re: How Riak Handle Request?

"with a scheduler per thread" Slight correction: with a scheduler per logical CPU core. If you have other applications on the same box, e.g. java code, you might want to reduce the number of schedulers. See https://github.com/basho/leveldb/wiki/riak-tuning-2

Re: anti_entropy and solr taking up suspiciously large amounts of space

ing to create a bunch of extra disk operations. But I am being honest. --> You could reduce only the anti_entropy disk usage by changing the "anti_entropy" setting in riak.conf from "active" to "passive". But this does nothing for solr. Matthew > On Mar 22, 2

Re: Unable to compile Riak on Raspberry Pi 3

A fix for that problem in leveldb/util/perf_count.cc already exists on the "develop" branch of the github repository basho/leveldb. Download it and rebuild. Sent from my iPad > On Apr 1, 2017, at 4:08 AM, cocos wrote: > > Hello, > > currenty I'm trying to install

Re: anti_entropy and solr taking up suspiciously large amounts of space

Rohit, If you restart the node does the elevated anti_entropy size decline after the restart? Matthew > On Mar 21, 2017, at 8:00 PM, Rohit Sanbhadti > wrote: > > Running Riak 2.2.0 on Ubuntu 16.04.1, we’ve noticed that anti_entropy is > taking up way too much

Re: AAE Off

Performance gains on write intensive applications. > On Feb 28, 2017, at 11:18 AM, al so wrote: > > Why would anyone disable AAE in riak 2.x? > ___ > riak-users mailing list > riak-users@lists.basho.com >

Re: Truncated bit-cask files

hat the real problem is memory unhappiness. Matthew > On Feb 14, 2017, at 3:34 PM, Arun Rajagopalan <arun.v.rajagopa...@gmail.com> > wrote: > > Hi Matthew, Magnus > > I have attached the log files for your review > > Thanks > Arun > > > On Tue,

Re: Truncated bit-cask files

Arun, The AAE code uses leveldb for its storage of anti-entropy data, no matter which backend holds the user data. Therefore the error below suggests corruption within leveldb files (which is not impossible, but becoming really rare except with bad hardware or full disks). Before wiping out

Re: Reg:Continuous Periodic crashes after long operation

FYI: this is the function that is crashing: get_uint32_measurement(Request, #internal{os_type = {unix, linux}}) -> {ok,F} = file:open("/proc/loadavg",[read,raw]), %% <--- crash line {ok,D} = file:read(F,24), ok = file:close(F),

Re: memory usage of Riak.

Alex, Which backend are you using? Leveldb's memory usage does not show up within Erlang. Maybe that is what you are experiencing? Matthew Sent from my iPad > On Jan 26, 2017, at 5:47 AM, Alex Feng wrote: > > Hi Riak Users, > > One of my riak nodes, it has 4G

Re: Crash Log: yz_anti_entropy

> Setting ring_size to 8 got rid of crashing. I'll have to do a bit more > reading on this setting I suppose. I have a much more memory-constrained > virtual machine running on my local desktop running with just the default > install settings and no crashing. > > Thanks! > &

Re: Crash Log: yz_anti_entropy

Damion, Add the following settings within riak.conf: leveldb.limited_developer_mem = on ring_size = 8 Erase all data / vnodes and start over. Matthew > On Jan 19, 2017, at 8:51 AM, Junk, Damion A wrote: > > Hi Magnus - > > I've tried a wide range of parameters for

Re: Reaping Tombstones

> <mailto:luca.favate...@erlang-solutions.com>> wrote: > On 30 December 2016 at 15:06, Matthew Von-Maszewski <matth...@basho.com > <mailto:matth...@basho.com>> wrote: > > Greetings, > > > > I am not able to answer your tombstone questions. That q

Re: Reaping Tombstones

Greetings, I am not able to answer your tombstone questions. That question needs a better expert. Just wanted to point out that Riak now has global expiry in both the leveldb and bitcask backends. That might be a quicker solution for your frequent delete operations:

Re: RiakTS TTL

1. The global expiry module is an external C++ module that is open source. There is no definition at this time for an Erlang callback, but the design supports it. You can patch the open source code now. 2. The TTL has two components: when the record is written and number of minutes until

Re: Seeking Backup solution for live nodes

AM, DeadZen <dead...@deadzen.com> wrote: > > there a backup tool that uses this yet? or is this meant more to be used with > snapshots provided through xfs/zfs? > > On Monday, September 26, 2016, Matthew Von-Maszewski <matth...@basho.com > <mailto:matth...@ba

Re: Seeking Backup solution for live nodes

Here are notes on the new hot backup: https://github.com/basho/leveldb/wiki/mv-hot-backup This sound like what you need? Matthew Sent from my iPad > On Sep 26, 2016, at 5:39 AM, Niels Christian Sorensen > wrote: > > Hi, > > We use Riak-kv Enterprise Edition

Re: speeding up bulk loading

Travis, I cannot address the crash error you see in the logs. Someone else will have to review that problem. I want to point you to this wiki article: https://github.com/basho/leveldb/wiki/riak-tuning-2 The article details how you can potentially increase Riak's throughput by restricting

Re: Rick Time-Series and time to live, ttl

Anatoly, The first time series TTL feature is due in Riak TS 1.4. That release is looking good for late August. The 1.4 release establishes a global TTL (also known as expiry policy) for all data. A future release will allow individual policies per Riak TS table (yeah, I am coding on that

Re: Recovering Riak data if it can no longer load in memory

cing the memory footprint in dev, my > concern was more around reacting to a possible production scenario where the > db stops responding due to memory overload. Understood now that moving to a > larger instance should be possible. Thanks again. > > On Tue, Jul 12, 2016 at 12:26

Re: Recovering Riak data if it can no longer load in memory

It would be helpful if you described the physical characteristics of the servers: memory size, logical cpu count, etc. Google created leveldb to be highly reliable in the face of crashes. If it is not restarting, that suggests to me that you have a low memory condition that is not able to

[ANN] leveldb expiry beta1 available

Basho is releasing a beta test version of a very rudimentary implementation for expiry within leveldb today. This code is NOT for production use. Download and usage notes [1] exist on Basho's leveldb wiki [2]. Please email all feedback either to this mailing list [3] or directly to

Re: How to increase Riak write performance for sequential alpha-numeric keys

/basho/leveldb/wiki/mv-timed-grooming2 > On May 5, 2016, at 7:59 PM, Matthew Von-Maszewski <matth...@basho.com> wrote: > > Alex, > > I am making a guess. I would be better able to support this guess with data > from leveldb LOG files of one server. > > The

Re: How to increase Riak write performance for sequential alpha-numeric keys

Alex, I am making a guess. I would be better able to support this guess with data from leveldb LOG files of one server. The performance difference between the sequential keys and the reverse of the sequential keys is informative. The reverse ordered keys essentially become more like "random

Re: How to increase Riak write performance for sequential alpha-numeric keys

Alex, This is why I was requesting you run "riak-debug" on one server. I would like to look at the leveldb LOG files before guessing and/or offering other solutions. I have a guess but would like to confirm it with supporting evidence from the LOG file. Matthew > On May 5, 2016, at 10:11

Re: Very slow acquisition time (99 percentile) while fast median times

; > <http://www.awesomescreenshot.com/image/1219821/1b292613c051da86df5696034c114b14> > > I think i'll try to add a 6th node that don't rely on network disks and see > what's going on. > > G. > > > On 03/05/2016 22:47, Matthew Von-Maszewski wro

Re: Very slow acquisition time (99 percentile) while fast median times

b page will be slightly different than your Riak 2.1.4 installation: /usr/lib/riak/lib/eleveldb-/priv/ Matthew > On May 3, 2016, at 1:01 PM, Matthew Von-Maszewski <matth...@basho.com> wrote: > > Guillaume, > > I have reviewed the debug package for your riak1 server. There

Re: Very slow acquisition time (99 percentile) while fast median times

Guillaume, I have reviewed the debug package for your riak1 server. There are two potential areas of follow-up: 1. You are running our most recent Riak 2.1.4 which has eleveldb 2.0.17. We have seen a case where a recent feature in eleveldb 2.0.17 caused too much cache flushing, impacting

Re: Ok, I am stumped. Losing data or riak stop

..just two separate > directories). I checked all of the ../leveldb/* directories. All LOG files > are identical, and no errors in any of them. > > I will try to build another Vagrant machine with the default riak.conf and > see if I can get this to repeat. It is almost as if the KV pairs

Re: Ok, I am stumped. Losing data or riak stop

Joe, Are there any error messages in the leveldb LOG and/or LOG.old files? These files are located within each vnode's directory, likely /var/lib/riak/data/leveldb/*/LOG* on your machine. The LOG files are not to be confused with 000xxx.log files. The lower case *.log files are the recovery

Re: riak crash

Raviraj, Please run 'riak-debug'. This is in the bin directory along with 'riak start' and 'riak-admin'. riak-debug will produce a file named similar to /home/user/r...@10.0.0.15-riak-debug.tar.gz You should email that file to me directly,

Re: Leveldb segfault during Riak startup

.1, now waiting to see if it makes a difference. > > I installed the binary packages from Basho repo. > > > > ________ > From: Matthew Von-Maszewski <matth...@basho.com> > Sent: 31 December 2015 18:25 > To: Antti Kuusela > Cc: Luke

Re: Connection multiplexing in the Erlang client

Paulo, There is nothing to prevent you from "stacking" multiple protocol buffer request packets into the TCP chain. None of our published clients support this usage, but it is known to work. You would always get the response packets in the order you sent request packets. The benefit is that

Re: Leveldb segfault during Riak startup

I also failed to ask two basic questions: 1. did this failure start after your upgrade to 2.1.3, or happen prior to upgrade also? 2. did you use a Basho package for Centos 7, or did you build from source code? Matthew > On Dec 31, 2015, at 6:06 AM, Antti Kuusela

Re: Leveldb segfault during Riak startup

Would you forward the riak.conf for your setup and paste the first 30 lines from a LOG file from any leveldb vnode a server that has experienced the crash. Example lines from a LOG file are: 2015/12/27-11:56:53.559717 7f02a47b0700Version: 2.0.10

Re: Latency of Riak KV drops with small size server goes long

Try 2 times the ring size. Long term max latency increasing could be due to disk overhead. Sent from my iPhone > On Nov 8, 2015, at 5:51 PM, mtakahashi-ivi > wrote: > > Could someone advise me? > > I changed concurrency of benchmark from 300 to 16. > >

Re: Ownership handoff timed out

Hi, There was a known eleveldb bug with handoff receiving that could cause a timeout. But it does not sound like bug fits your symptoms. However, I am willing to verify my diagnosis. I would need you to gather the LOG files from all vnodes on the RECEIVING side (or at least from the vnode

Re: Ownership handoff timed out

;v.zakhoz...@smartweb.com.ua> wrote: > > Hi, > > Matthew thank for you answer. eleveldb LOGs are attached. > Here is LOGs from 2 eleveldb nodes (eggeater was not restarted; what about > rattlesnake I'm not sure). > > On Thu, Oct 29, 2015 at 5:24 PM Matthew Von-Maszewski

Re: Limiting LevelDB memory in Riak 1.4

See this: https://docs.basho.com/riak/1.4.7/ops/advanced/backends/leveldb/ It is the 1.4 docs relating to memory management for leveldb. Basic answer is that you multiple max_open_files times 4 megabytes times number of vnodes on the server. Matthew > On Oct 8, 2015, at 1:32 PM, Ricardo

Re: riak 2.1.1 : Erlang crash dump

Girish, This feels like a sibling explosion to me. I cannot help prove or fix it. Writing this paragraph as bait for others to help. Matthew Sent from my iPad > On Oct 3, 2015, at 8:34 PM, Girish Shankarraman > wrote: > > Thank you for the response, Jon. > > So

Re: riak handoff of partition error

Amao Wang, Would you send the riak-diag from the machine receiving the handoff, 10.21.136.93? Thank you, Matthew On Jul 30, 2015, at 5:35 AM, changmao wang wang.chang...@gmail.com wrote: Hi Riak-users group, I have found some errors related to handoff of partition in

Re: Monitoring protobuffs and CPU usage on a Riak KV cluster

Two ideas about lots of cpu with nothing happening: - is Riak entropy setting active. Weekly it will rebuild hash trees - does this activity occur after a handoff? Sometimes leveldb needs time to catch up afterward in current releases (fix coming soon). Matthew Sent from my iPad On Aug

Re: LevelDB Question

Joe, You want to start here: http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/#Tiered-Storage http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/#Tiered-Storage In addition to the 3 configuration lines given in that section, you need a fourth:

Re: LevelDB Question

All, The “Implementation Details” section of the referenced webpage is about 2 years behind Basho’s changes to leveldb. I already have the action item to rewrite it. Apologies for any confusion. The tiered storage section has an accurate, cumulative table. Matthew On Aug 5, 2015, at

Re: Riak 2.0.2 leveldb uses more disk space than Riak 1.4.x?

-sequential-tuning https://github.com/basho/leveldb/wiki/mv-sequential-tuning Matthew On Jul 16, 2015, at 2:06 PM, Wes Jossey weston.jos...@gmail.com wrote: Nope. Never did. On Jul 16, 2015, at 13:32, Matthew Von-Maszewski matth...@basho.com wrote: Did you ever get a reply

Re: Riak LevelDB Deletion Problem

Antonio, Here is a detailed discussion of the Riak / leveldb delete scenario: https://github.com/basho/leveldb/wiki/mv-aggressive-delete Pay close attention to the section titled “Update April 6, 2014”. This explains why as much as 4.2G bytes per vnode might remain within leveldb after

Re: Riak LevelDB Deletion Problem

2015-07-14 18:32 GMT+01:00 Matthew Von-Maszewski matth...@basho.com mailto:matth...@basho.com: Antonio, A Riak delete operation happens in these steps: - Riak writes a “tombstone” value for the key to the N vnodes that contain it (this is a new record) - Riak by default, waits 3

Re: Riak LevelDB Deletion Problem

compaction. Thanks for all the help, Antonio 2015-07-14 17:43 GMT+01:00 Matthew Von-Maszewski matth...@basho.com mailto:matth...@basho.com: Antonio, Here is a detailed discussion of the Riak / leveldb delete scenario: https://github.com/basho/leveldb/wiki/mv-aggressive-delete

Re: LevelDB

it to work. Setting the filesystem rights to 777 got it to start. I thought starting Riak as root would allow writing to the filesystem, but I guess not. Do most people run riak as an OS user riak? What is best practice here? Thanks for all your help. From: Matthew Von-Maszewski matth

Re: LevelDB

the documentation updated if it works. Matthew On Jun 8, 2015, at 10:37 AM, Joe Olson technol...@nododos.com wrote: Here you go. Thanks for the help. My riak.conf is mostly stock, except for the LevelDB changes. From: Matthew Von-Maszewski matth...@basho.com To: Mr

Re: LevelDB

) From: Joe Olson technol...@nododos.com To: Matthew Von-Maszewski matth...@basho.com Sent: Monday, June 8, 2015 11:22:24 AM Subject: Re: LevelDB I think our email just crossed each other. Let me try again. From: Joe Olson technol...@nododos.com To: Matthew Von-Maszewski matth

Re: LevelDB

for the LevelDB changes. From: Matthew Von-Maszewski matth...@basho.com To: Mr. Technology technol...@nododos.com Cc: riak-users riak-users@lists.basho.com Sent: Saturday, June 6, 2015 9:08:00 AM Subject: Re: LevelDB There is a script macro that puts paths together during program

Re: LevelDB

There is a script macro that puts paths together during program start. I will go look at that. It would help if you would send your riak.conf file. Matthew On Jun 5, 2015, at 7:48 PM, Mr. Technology technol...@nododos.com wrote: After installing Riak 2.1.1 on a 5 node Centos7 based test

Re: Riak Nodes Crashing

-entropy, since I did not want to change too many variables at the same time. I will do that today. Question is, was this a coincidence or do you think it's possible the search indexing was causing the OOM errors? Satish On Sat, Dec 6, 2014 at 6:59 AM, Matthew Von-Maszewski matth

Re: Riak Nodes Crashing

percentage than the other 4. Is this normal? On Mon, Dec 8, 2014 at 9:06 AM, Matthew Von-Maszewski matth...@basho.com wrote: Satish, This additional information continues to support my suspicion that the memory management is not fully accounting for your number of open files. A large

Re: Riak Nodes Crashing

, 2014 at 2:20 PM, ender extr...@gmail.com wrote: Hey Matthew it just crashed again. This time I got the syslog and leveldb logs right away. On Fri, Dec 5, 2014 at 11:43 AM, Matthew Von-Maszewski matth...@basho.com wrote: Satish, Here is a key line from /var/log/messages: Dec 5 06:52

Re: Riak Nodes Crashing

version: 1.4.7 5 Nodes in cluster RAM: 30GB The leveldb logs are attached. On Thu, Dec 4, 2014 at 1:34 PM, Matthew Von-Maszewski matth...@basho.com wrote: Satish, Some questions: - what version of Riak are you running? logs suggest 1.4.7 - how many nodes in your cluster? - what

Re: Riak Nodes Crashing

went ahead and changed max_open_files in app.config to to 150 from 315. Satish On Fri, Dec 5, 2014 at 11:29 AM, Matthew Von-Maszewski matth...@basho.com wrote: Satish, The key system log varies by Linux platform. Yes, /var/log/messages may hold some key clues. Again, be sure

Re: Riak Nodes Crashing

Satish, Some questions: - what version of Riak are you running? logs suggest 1.4.7 - how many nodes in your cluster? - what is the physical memory (RAM size) of each node? - would you send the leveldb LOG files from one of the crashed servers: tar -czf satish_LOG.tgz

Re: Riak unexpectedly crashes

What version of Riak? What backend are you using (bitcask, memory, leveldb)? Matthew On Nov 27, 2014, at 10:30 AM, Ivaylo Panitchkov ipanitch...@hibernum.com wrote: Hi Alexander, Yes, they are set properly in /etc/security/limits.conf Ivaylo On Thu, Nov 27, 2014 at 9:44 AM,

Re: Riak unexpectedly crashes

with bitcask and the problem occurred again few days ago. Thanks, Ivaylo On Thu, Nov 27, 2014 at 10:37 AM, Matthew Von-Maszewski matth...@basho.com wrote: What version of Riak? What backend are you using (bitcask, memory, leveldb)? Matthew On Nov 27, 2014, at 10:30 AM, Ivaylo

Re: Compaction errors when new node is added

Timo, You need to upgrade to 1.4.10. There is a known leveldb bug in 1.4.9 that fits your symptoms: - 1.4.10 Release Notes: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md - leveldb bug: https://github.com/basho/leveldb/issues/135 Matthew On Nov 23, 2014, at 8:36 AM, Timo

Re: {read_block_error,0} with level_db

There have been zero errors during the leveldb::ReadBlock call. On Jul 30, 2014, at 12:19 PM, Bryan br...@go-factory.net wrote: We are diagnosing a weird intermittent problem where we are getting an {error, disconnected} exception from the riak erlang client. There is not a lot of

Re: RIAK 1.4.6 - Mass key deletion

, Matthew Von-Maszewski wrote: Edgar, This is indirectly related to you key deletion discussion. I made changes recently to the aggressive delete code. The second section of the following (updated) web page discusses the adjustments: https://github.com/basho/leveldb/wiki/Mv

Re: Tag 2.0.0rc1 ??

I believe he meant end of next week. That seems more inline with the general goal. On Jul 18, 2014, at 4:42 PM, Daurnimator q...@daurnimator.com wrote: On 14 July 2014 16:40, Jared Morrow ja...@basho.com wrote: I'd personally be very disappointed if something wasn't in your hands by the

Re: leveldb Hot Threads in 1.4.9?

Responses inline. On Jul 7, 2014, at 10:54 PM, Tom Lanyon tom+r...@oneshoeco.com wrote: Hi Matthew, On Sunday, 6 July 2014 at 3:04, Matthew Von-Maszewski wrote: Tom, Basho prides itself on quickly responding to all user queries. I have failed that tradition in this case. Please

Re: leveldb Hot Threads in 1.4.9?

at 10:45, Tom Lanyon wrote: No problem, Matthew. Appreciate you taking a look when you have time. Regards, Tom On Tuesday, 24 June 2014 at 9:45, Matthew Von-Maszewski wrote: Tom, I have been distracted today and on a plane tomorrow. I apologize for the delayed response. It may

Re: leveldb Hot Threads in 1.4.9?

this happens? sort /var/lib/riak/*/LOG combined_log Substitute your actual data path for /var/lib/riak. Matthew Von-Maszewski On Jun 22, 2014, at 22:07, Tom Lanyon tom+r...@oneshoeco.com wrote: Could someone please confirm whether 1.4.9 includes Hot Threads in leveldb? The release notes

Re: leveldb issues.

Running repair now may have detected damage done to your data long ago. Repair reads every file and tests the CRC on every block in the file. Two known issues might have caused the original corruption: https://github.com/basho/leveldb/wiki/mv-verify-compactions or

Re: leveldb issues.

Yes. On Jun 19, 2014, at 1:47 PM, Theo Schlossnagle je...@omniti.com wrote: To be clear, you're considering a13160da8adeca96d80388cc77cb88ad5301aeaa the latest? On Thu, Jun 19, 2014 at 12:36 PM, Matthew Von-Maszewski matth...@basho.com wrote: Running repair now may have detected

Re: Performance slows down with write heavy use

Jie, 4G ram is small for your ring size and node count. My first recommendation is that you reduce your ring size to 32 {ring_creation_size, 32}. Then change the max_open_files setting of eleveldb to 40 {max_open_files, 50} and block_size to 32768 {sst_block_size, 32768}. These three

Re: Performance slows down with write heavy use

Jie, A colleague points out that I typed 40 in one place and 50 in another. 40 is the appropriate setting. Matthew On Apr 15, 2014, at 8:41 AM, Matthew Von-Maszewski matth...@basho.com wrote: Jie, 4G ram is small for your ring size and node count. My first recommendation

Re: RIAK 1.4.6 - Mass key deletion

of the machines after the anti-entropy data removal and consequent restart (5th of April). https://cloudup.com/cB0a15lCMeS Best regards! On 8 April 2014 23:44, Matthew Von-Maszewski matth...@basho.com wrote: No. I do not see a problem with your plan. But ... I would prefer to see

Re: RIAK 1.4.6 - Mass key deletion

each one with ~120G of anti-entropy data and 600G of level-db data. This seems to be quite a lot no? My total amount of keys is ~2.5 Billions. Best regards, Edgar On 6 April 2014 23:30, Matthew Von-Maszewski matth...@basho.com wrote: Edgar, This is indirectly related to you key

Re: RIAK 1.4.6 - Mass key deletion

a lot no? My total amount of keys is ~2.5 Billions. Best regards, Edgar On 6 April 2014 23:30, Matthew Von-Maszewski matth...@basho.com wrote: Edgar, This is indirectly related to you key deletion discussion. I made changes recently to the aggressive delete code. The second section

Re: RIAK 1.4.6 - Mass key deletion

*.sst files has ~3.7M Hope this info gives you some more help! Best regards, and again, thanks a lot Edgar On 8 April 2014 13:24, Matthew Von-Maszewski matth...@basho.com wrote: Argh. Missed where you said you had upgraded. Ok it will proceed with getting you comparison numbers

Re: RIAK 1.4.6 - Mass key deletion

days to complete the operation) so I'll come back and give you some feedback (hopefully good) on the next Monday! Again, thanks a lot, You've been very helpful. Edgar On 8 April 2014 15:47, Matthew Von-Maszewski matth...@basho.com wrote: Edgar, The test I have running currently has

Re: RIAK 1.4.6 - Mass key deletion

the next weekend (normally it takes about two days to complete the operation) so I'll come back and give you some feedback (hopefully good) on the next Monday! Again, thanks a lot, You've been very helpful. Edgar On 8 April 2014 15:47, Matthew Von-Maszewski matth...@basho.com wrote: Edgar

Re: RIAK 1.4.6 - Mass key deletion

, Matthew Von-Maszewski matth...@basho.com wrote: Riak 2.0 is coming. Hold your mass delete until then. The bug is within Google's original leveldb architecture. Riak 2.0 sneaks around to get the disk space freed. Matthew On Feb 18, 2014, at 11:10 AM, Edgar Veiga edgarmve

Re: RIAK 1.4.6 - Mass key deletion

, at 4:29 PM, Edgar Veiga edgarmve...@gmail.com wrote: Matthew, thanks again for the response! That said, I'll wait again for the 2.0 (and maybe buy some bigger disks :) Best regards On 6 April 2014 15:02, Matthew Von-Maszewski matth...@basho.com wrote: Edgar, In Riak 1.4

Re: LevelDB parameter planning - max_open_files

Oleksiy, Go to step 6: Compare Step 2 and Step 5 …. There is a link to an Excel spreadsheet at the end of the sentence The above calculations are automated in this memory model spreadsheet.. Forget the text and use the spreadsheet (memory model spreadsheet). Much of that text is still

Re: LevelDB parameter planning - max_open_files

! On 4 April 2014 18:14, Matthew Von-Maszewski matth...@basho.com wrote: Oleksiy, Go to step 6: Compare Step 2 and Step 5 …. There is a link to an Excel spreadsheet at the end of the sentence The above calculations are automated in this memory model spreadsheet.. Forget the text

Re: Change data directory

Are you using leveldb or bitcask back end? What is your desired data directory? I will create an example. Sent from my iPhone On Apr 4, 2014, at 3:05 PM, Nabil Hassein na...@betterpath.com wrote: Hello all, I'm trying to change /etc/riak/app.config to store data in a directory of my

Re: Storing Photos in riak

Ingo, We regularly test Riak and its leveldb backend with 130k objects as part of our performance and qualification tests. Works just fine. The one thing to remember is that with N=3 redundancy, the bandwidth between the nodes will approach 3 times the incoming user bandwidth. Make sure your

Re: Storing Photos in riak

from performance issues? Best, Ingo Am 24.03.2014 13:55, schrieb Matthew Von-Maszewski: Ingo, We regularly test Riak and its leveldb backend with 130k objects as part of our performance and qualification tests. Works just fine. The one thing to remember is that with N=3

Re: Storing Photos in riak

. Do you test Riak with memory backend with 130k objects or just LevelDB? Regards, Istvan On Mon, Mar 24, 2014 at 5:55 AM, Matthew Von-Maszewski matth...@basho.com wrote: Ingo, We regularly test Riak and its leveldb backend with 130k objects as part of our performance and qualification

Re: Cleaning up bucket after basho_bench run

-aggressive-delete Matthew Von-Maszewski On Mar 22, 2014, at 1:53, István lecc...@gmail.com wrote: All good, all the keys are gone! :) I am just waiting Riak to free up the space. It seems it is not instant... Or I am missing something. I need to read up on how LevelDB actually frees up space. I

Re: Cleaning up bucket after basho_bench run

, Istvan On Sat, Mar 22, 2014 at 5:33 AM, Matthew Von-Maszewski matth...@basho.com wrote: Leveldb, as written by Google, does not actively clean up delete tombstones or prior data records with the same key. The old data and tombstones stay on disk until they happen to participate

Re: Riak 1.4.2 crashing

Do you have active entropy enabled: {anti_entropy, {on, []}} in app.config file ? If so, please check file count there also: {anti_entropy_data_dir, … Matthew On Mar 1, 2014, at 1:54 PM, Bryan Hughes br...@go-factory.net wrote: Howdy, I am having a problem with Riak crashing (note,

Re: RIAK 1.4.6 - Mass key deletion

February 2014 03:18, Matthew Von-Maszewski matth...@basho.com wrote: Riak 2.0 is coming. Hold your mass delete until then. The bug is within Google's original leveldb architecture. Riak 2.0 sneaks around to get the disk space freed. I'm interested to know what happens if someone deletes a lot

Re: RIAK 1.4.6 - Mass key deletion

Which Riak backend are you using: bitcask, leveldb, multi? Matthew On Feb 18, 2014, at 10:17 AM, Edgar Veiga edgarmve...@gmail.com wrote: Hi all! I have a fairly trivial question regarding mass deletion on a riak cluster, but firstly let me give you just some context. My cluster is

Re: RIAK 1.4.6 - Mass key deletion

. But that release is not quite ready for production usage. What do you hope to achieve by the mass delete? Matthew On Feb 18, 2014, at 10:29 AM, Edgar Veiga edgarmve...@gmail.com wrote: Sorry, forgot that info! It's leveldb. Best regards On 18 February 2014 15:27, Matthew Von-Maszewski matth

Re: RIAK 1.4.6 - Mass key deletion

:S On 18 February 2014 15:45, Matthew Von-Maszewski matth...@basho.com wrote: Edgar, The first concern I have is that leveldb's delete does not free disk space. Others have executed mass delete operations only to discover they are now using more disk space instead of less. Here

Re: Riak Search and Yokozuna Backup Strategy

On Jan 27, 2014, at 3:06 PM, Elias Levy fearsome.lucid...@gmail.com wrote: On Mon, Jan 27, 2014 at 11:57 AM, Matthew Von-Maszewski matth...@basho.com wrote: Google's designed leveldb to always assume it was not cleanly shutdown. If the startup can read the most recent MANIFEST file

Re: rocksdb (fb leveldb fork) as riak backend?

Basho's leveldb requirements have lead to different optimizations. Facebook has a captive hardware environment and usage case that does not match ours. I am not saying their changes are better or worse, only different. Basho needs: - multiple databases running simultaneously: 6 to 64 -

Re: rocksdb (fb leveldb fork) as riak backend?

P.S. I failed to include the link to the wiki page that discusses each of Basho's optimizations for the Riak environment: https://github.com/basho/leveldb/wiki On Jan 27, 2014, at 8:49 PM, Matthew Von-Maszewski matth...@basho.com wrote: Basho's leveldb requirements have lead to different

Re: Single node causing cluster to be extremely slow (leveldb)

Sean, I did some math based upon the app.config and LOG files. I am guessing that you are starting to thrash your file cache. This theory should be easy to prove / disprove. On that one node, change the cache_size and max_open_files to: cache_size 68435456 max_open_files 425 If I am

Re: Single node causing cluster to be extremely slow (leveldb)

Sean, Also you mentioned concern about +S 6:6. 2i queries in 1.4 added sorting. Another heavy 2i user noticed that the sorting need more CPU for Erlang. They were happier after removing the +S. And finally, those 2i queries that return millions of results … how long do those queries take

  1   2   >