Hi Alex, That is an interesting question! I haven't seen a request like that in our backlog, so feel free to open a new issue [1]. I'm curious: why not use something like sys.getsizeof [2]?
[1] https://github.com/basho/riak-python-client/issues [2] http://stackoverflow.com/questions/449560/how-do-i-determine-the-size-of-an-object-in-python Matt Brender | Developer Advocacy Lead Basho Technologies t: @mjbrender On Mon, Apr 13, 2015 at 7:26 AM, Alex De la rosa <alex.rosa....@gmail.com> wrote: > Hi Bryan, > > Thanks for your answer; i don't know how to code in erlang, so all my system > relies on Python. > > Following Ciprian's curl suggestion, I tried to compare it with this python > code during the weekend: > > Map object: > curl -I >> 1058 bytes > print sys.getsizeof(obj.value) >> 3352 bytes > > Standard object: > curl -I >> 9718 bytes > print sys.getsizeof(obj.encoded_data) >> 9755 bytes > > The standard object seems pretty accurate in both approaches even the image > binary data was only 5kbs (I assume some overhead here) > > The map object is about 3x the difference between curl and getting the > object via Python. > > Not so sure if this is a realistic way to measure their growth (moreover > because the objects i would need this monitorization are Maps, not unaltered > binary data that I can know the size before storing it). > > Would it be possible in some way that the Python get() function would return > something like "obj.content-lenght" returning the size is currently taking? > that would be a pretty nice feature. > > Thanks! > Alex > > On Mon, Apr 13, 2015 at 12:47 PM, bryan hunt <bh...@basho.com> wrote: >> >> Alex, >> >> >> Maps and Sets are stored just like a regular Riak object, but using a >> particular data structure and object serialization format. As you have >> observed, there is an overhead, and you want to monitor the growth of these >> data structures. >> >> It is possible to write a MapReduce map function (in Erlang) which >> retrieves a provided object by type/bucket/id and returns the size of it's >> data. Would such a thing be of use? >> >> It would not be hard to write such a module, and I might even have some >> code for doing so if you are interested. There are also reasonably good >> examples in our documentation - >> http://docs.basho.com/riak/latest/dev/advanced/mapreduce >> >> I haven't looked at the Python PB API in a while, but I'm reasonably >> certain it supports the invocation of MapReduce jobs. >> >> Bryan >> >> >> On 10 Apr 2015, at 13:51, Alex De la rosa <alex.rosa....@gmail.com> wrote: >> >> Also, I forgot, i'm most interested on bucket_types instead of simple riak >> buckets. Being able how my mutable data inside a MAP/SET has grown. >> >> For a traditional standard bucket I can calculate the size of what I'm >> sending before, so Riak won't get data bigger than 1MB. Problem arise in >> MAPS/SETS that can grown. >> >> Thanks, >> Alex >> >> On Fri, Apr 10, 2015 at 2:47 PM, Alex De la rosa <alex.rosa....@gmail.com> >> wrote: >>> >>> Well... using the HTTP Rest API would make no sense when using the PB >>> API... would be extremely costly to maintain, also it may include some extra >>> bytes on the transport. >>> >>> I would be interested on being able to know the size via Python itself >>> using the PB API as I'm doing. >>> >>> Thanks anyway, >>> Alex >>> >>> On Fri, Apr 10, 2015 at 1:58 PM, Ciprian Manea <cipr...@basho.com> wrote: >>>> >>>> Hi Alex, >>>> >>>> You can always query the size of a riak object using `curl` and the REST >>>> API: >>>> >>>> i.e. curl -I <riak-node-ip>:8098/buckets/test/keys/demo >>>> >>>> >>>> Regards, >>>> Ciprian >>>> >>>> On Thu, Apr 9, 2015 at 12:11 PM, Alex De la rosa >>>> <alex.rosa....@gmail.com> wrote: >>>>> >>>>> Hi there, >>>>> >>>>> I'm using the python client (by the way). >>>>> >>>>> obj = RIAK.bucket('my_bucket').get('my_key') >>>>> >>>>> Is there any way to know the actual size of an object stored in Riak? >>>>> to make sure something mutable (like a set) didn't added up to more than >>>>> 1MB >>>>> in storage size. >>>>> >>>>> Thanks! >>>>> Alex >>>>> >>>>> _______________________________________________ >>>>> riak-users mailing list >>>>> riak-users@lists.basho.com >>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>> >>>> >>> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ᐧ _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
