Re: [go-nuts] Re: weak hash for (MongoDb like) data server

[Basile Starynkevitch]

On Thursday, February 16, 2017 at 9:11:08 PM UTC+1, Ian Lance Taylor wrote:

> identified one approach using finalizers.  That approach will work 
> today as Go's garbage collector never moves items.  
>

Thanks Ian for your help. FWIW, I just have coded this objvalmo.go file and 
function 

FindOrMakeObjectById 
 
commit 15a7fbc61e141 

 so 
I hope to have understood your advice correctly.

Regards.

Basile Starynkevitch
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: weak hash for (MongoDb like) data server

[Ian Lance Taylor]

On Wed, Feb 15, 2017 at 9:42 PM, Basile Starynkevitch
 wrote:
>
> On Wednesday, February 15, 2017 at 9:01:58 PM UTC+1, Tamás Gulácsi wrote:
>
>>
>> Why do you need this?
>> You want the GC do the housekeeping for you, but I'm sure you won't be
>> happy with the result, as the GC's policy differs from what you await...
>
>
> What make you believe I won't be happy with the GC's policy. I'm quite open
> on that. My understanding is that the GC's policy is what I want.
>>
>>
>> What kind of eviction policy would you want?
>> Storing the data in an append-only file (or LevelDB), the deleted items in
>> SQLite, evicting regularly and on close?
>
>
> Persistence to disk will only happen explicitly at shutdown (process exit)
> time. So the file aspect don't matter much for my Q1.
>
> So my Q1 is basically: how can I make a weak hash table (in the sense I have
> described, of having an association from strong keys -pairs of uint64- to
> weak pointers to items).

Hi Basile.

Go does not support any sort of weak pointer.  You've already
identified one approach using finalizers.  That approach will work
today as Go's garbage collector never moves items.  There are no
current plans to ever move items, but it is possible that some Go
implementation will do so, in which case hiding the pointer as a
uintptr will break.  (By the way, Go's garbage collector is precise,
so there is no need to invert the bits in the uintptr or anything like
that--simply storing a pointer value in a uintptr is enough to hide it
from the garbage collector.)

Other than that, I think you will have to implement garbage collection
yourself.  That doesn't sound too hard with your data structure, which
doesn't seem to have cycles.  Basically traverse the data structure
copying data from the current map to a new one, then replace the
official map with the new one.  With judicious locking this could even
be done concurrently.

Sorry I don't have anything else to suggest.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: weak hash for (MongoDb like) data server

[Basile Starynkevitch]

On Thursday, February 16, 2017 at 6:42:05 AM UTC+1, Basile Starynkevitch 
wrote:
>
>
>
> On Wednesday, February 15, 2017 at 9:01:58 PM UTC+1, Tamás Gulácsi wrote:
>
>
>> Why do you need this? 
>> You want the GC do the housekeeping for you, but I'm sure you won't be 
>> happy with the result, as the GC's policy differs from what you await...
>>
>
> What make you believe I won't be happy with the GC's policy. I'm quite 
> open on that. My understanding is that the GC's policy is what I want.
>
>>
>> What kind of eviction policy would you want?
>> Storing the data in an append-only file (or LevelDB), the deleted items 
>> in SQLite, evicting regularly and on close?
>>
>
> Persistence to disk will only happen explicitly at shutdown (process exit) 
> time. So the file aspect don't matter much for my Q1.
>
> So my Q1 is basically: how can I make a weak hash table (in the sense I 
> have described, of having an association from strong keys -pairs of uint64- 
> to weak pointers to items).
>
> BTW, I have found https://github.com/fx5/weakref/blob/master/weakref.go 
but I am not sure to understand how it works (and even if it works). 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: weak hash for (MongoDb like) data server

[Basile Starynkevitch]

On Wednesday, February 15, 2017 at 9:01:58 PM UTC+1, Tamás Gulácsi wrote:


> Why do you need this? 
> You want the GC do the housekeeping for you, but I'm sure you won't be 
> happy with the result, as the GC's policy differs from what you await...
>

What make you believe I won't be happy with the GC's policy. I'm quite open 
on that. My understanding is that the GC's policy is what I want.

>
> What kind of eviction policy would you want?
> Storing the data in an append-only file (or LevelDB), the deleted items in 
> SQLite, evicting regularly and on close?
>

Persistence to disk will only happen explicitly at shutdown (process exit) 
time. So the file aspect don't matter much for my Q1.

So my Q1 is basically: how can I make a weak hash table (in the sense I 
have described, of having an association from strong keys -pairs of uint64- 
to weak pointers to items).
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: weak hash for (MongoDb like) data server

[Tamás Gulácsi]

> The id of data items is the only way to refer to data items on the 
> protocol side. So I need a *weak association* between an id and the data 
> item (if any) of that id. I do not want to keep (in some ordinary Go map, 
> for example) the association between ids and their data items (otherwise, 
> no data item would be garbage collected). I accept the fact that eventually 
> (and perhaps quite later), a data item which is not reachable from the root 
> data item would be garbage collected.
>
> I was thinking of using unsafe.Pointer-s with runtime.SetFinalizer. A 
> simplistic approach could be to use a Go map (module variable) from pairs 
> of uint64_t to such unsafe pointers. At data item creation, I would 
> register in that map, using the pair of uint64_t-s as key, the pointer of 
> that data item suitably converted to an unsafe.Pointer. I would also pass 
> to SetFinalizer some finalization routine which would unregister (delete) 
> from that map. That scheme would work only if Go garbage collector does not 
> move objects (i.e. struct-s). If Go's GC is a conservative GC à la Boehm 
> GC , I'll probably even better "hide" the 
> pointer (e.g. by keeping not a Pointer, but some uintptr suitably 
> "hidden" e.g. by complementing all its bits à la GC_HIDE_POINTERS 
>  in Boehm 
> GC). But that scheme would work only if Go's garbage collector is not 
> moving values (e.g. struct-s).* Is Go's garbage collector guaranteed to 
> be some marking *(not moving or generational copying, à la Ocaml) *one?* 
> FWIW, I am familiar with the terminology of the GC handbook 
> .
>

Why do you need this? 
You want the GC do the housekeeping for you, but I'm sure you won't be 
happy with the result, as the GC's policy differs from what you await...

What kind of eviction policy would you want?
Storing the data in an append-only file (or LevelDB), the deleted items in 
SQLite, evicting regularly and on close?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.