Re: Unstructured object format.

2015-07-22 Thread Atri Sharma
+1 as well. I wonder if it makes sense to further enhance the header fields to have more metadata readily available (so that we can support more JSONB like operators in a much efficient manner). On Wed, Jul 22, 2015 at 3:37 PM, Atri Sharma wrote: > +1 from me too > > On Wed, Jul 22, 2015 at 3:3

Re: Unstructured object format.

2015-07-22 Thread Atri Sharma
+1 from me too On Wed, Jul 22, 2015 at 3:34 PM, Sergi Vladykin wrote: > Ok, looks good to me. > > Sergi > > 2015-07-22 10:46 GMT+03:00 Dmitriy Setrakyan : > > > On Tue, Jul 21, 2015 at 10:15 PM, Atri Sharma > wrote: > > > > > So does that mean that local hashmap is not controlled with all the >

Re: Unstructured object format.

2015-07-22 Thread Sergi Vladykin
Ok, looks good to me. Sergi 2015-07-22 10:46 GMT+03:00 Dmitriy Setrakyan : > On Tue, Jul 21, 2015 at 10:15 PM, Atri Sharma wrote: > > > So does that mean that local hashmap is not controlled with all the heavy > > locks that are present around the cache? > > > > Yes Atri, you are right. The dat

Re: Unstructured object format.

2015-07-22 Thread Dmitriy Setrakyan
On Tue, Jul 21, 2015 at 10:15 PM, Atri Sharma wrote: > So does that mean that local hashmap is not controlled with all the heavy > locks that are present around the cache? > Yes Atri, you are right. The data is stored in local hash map to avoid touching a distributed cache whenever serializing o

Re: Unstructured object format.

2015-07-21 Thread Alexey Goncharuk
2015-07-21 22:15 GMT-07:00 Atri Sharma : > So does that mean that local hashmap is not controlled with all the heavy > locks that are present around the cache? > Metadata cache is basically populated once and then never changes, so only first get goes to the cache which is guarded by gateways and

Re: Unstructured object format.

2015-07-21 Thread Atri Sharma
So does that mean that local hashmap is not controlled with all the heavy locks that are present around the cache? On 22 Jul 2015 07:31, "Alexey Goncharuk" wrote: > Metadata cache access is backed by a local hash map, so the real cost is a > String object hashcode which is cached in the String ob

Re: Unstructured object format.

2015-07-21 Thread Alexey Goncharuk
Metadata cache access is backed by a local hash map, so the real cost is a String object hashcode which is cached in the String object and a hashmap lookup by an integer key. On the other hand, the marshaller is still pluggable and after the ticket is completed, it should be fairly easy to impleme

Re: Unstructured object format.

2015-07-21 Thread Sergi Vladykin
I think O(N) reasoning does not make a real sense here since N is always small, lets not fool ourselves. To my mind operation cost of cache access (with all busy locks...), hashCode/equals and stuff like that has much bigger impact here. Do we still have a pluggable marshaller? Can my approach be i

Re: Unstructured object format.

2015-07-20 Thread Alexey Goncharuk
Currently an index-enabled serialized object form has the following layout (simplified): [object fields][field1Offset,field1Length, field2Offset,field2Length,...,fieldNOffset,fieldNLength] where fields order is determined upon the first object serialization and stored in metadata cache which is a

Re: Unstructured object format.

2015-07-16 Thread Atri Sharma
Keep in mind that JSONB's performance comes from the fact that it uses server encoding, is binary represented and can have GIN indexes on top of it. Not sure if Ignite's marshalling approach keeps those features as well. On Thu, Jul 16, 2015 at 1:20 PM, Sergi Vladykin wrote: > HSTORE and JSONB a

Re: Unstructured object format.

2015-07-16 Thread Sergi Vladykin
HSTORE and JSONB appeared to have similar format in Postgresql (because they was developed by the same people). They noticed that they switched off of using key length sorting because they sometimes need lexicographical order but this is irrelevant for us. Sergi 2015-07-16 10:43 GMT+03:00 Atri Sh

Re: Unstructured object format.

2015-07-16 Thread Atri Sharma
Are you referring to JSONB here? On Thu, Jul 16, 2015 at 1:10 PM, Sergi Vladykin wrote: > Guys, specially Alexey G. > > I've attended PostgreSQL conference and there was a talk about unstructured > data format. > They had an interesting idea of serialized layout close enough to ours, I'm > not s