Re: [go-nuts] How does golang handles DNS caching
Probably 1. You're using connection pool in gorm 2. Gorm is using golang's connection pool 3. The connection gets cached/reused while domain's IP changes (as the address is same but ip changes and the change can't be easily detected) So the solution would probably be to do a lookup in a separated goroutine, every X seconds using net.LookupIP (20 seconds should be good). Then connecting directly to that IP instead of FQDN using gorm. BTW: If you're using mysql you should also add skip-name-resolve to server config as it'd overload DNS server if you're making a lot of connections and is totally unnecessary. środa, 7 czerwca 2023 o 20:42:17 UTC+2 Ian Lance Taylor napisał(a): > On Wed, Jun 7, 2023 at 5:13 AM Kishan Pandey wrote: > > > > My application is using gorm to connect to the database FQDN. If I am > making changes to DNS record for the FQDN , the application needs to be > restarted to take the DNS change. I think it is making the DNS query only > at the application start time but not afterward. > > > > I would like to know how does golang manages the DNS changes. I looked > at gorm issue and found one comment stating that gorm does not manage DNS > cache its usage system default. > > The Go standard library doesn't cache DNS lookups. If you look up a > name twice, it will generate two different DNS requests, or two > different calls to getaddrinfo. That is, Go expects to be contacting > a caching resolver. Of course, a Go program can do a DNS lookup and > cache the results outside of the standard library. > > So while I don't know what problem you are encountering, it doesn't > seem likely to be in the Go standard library. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/24fb2c2b-26cb-40b7-adde-d717db9bfcefn%40googlegroups.com.
[go-nuts] Multiple modules, single git repo. That makes sense?
Hi Guys, is it reasonable to publish multiple modules inside single github repo, eg. https://github.com/user/tools/a https://github.com/user/tools/b https://github.com/user/tools/c That seems to be working, but could it create any issues in the future (eg. with version control if I import 2 such modules in single file and then update one of these mods). -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/57bd6137-245d-474b-b524-9e75d2ec4ce6n%40googlegroups.com.
[go-nuts] Non-mutable / readonly slices?
Hi Guys, writing a quick K/V caching system and have a performance related
question.
Will be operating on []byte, for the system to be thread safe we need to
create a copy of data before each SET so we're sure that the slice which
gets added to pool can't be modified. This shouldn't be problematic, but
there's a problem with GET.
For the system to be thread-safe we need to isolate the returned slice from
the data pool so if we do, eg. x := Get("key_abc") we won't create issues
by later doing x[1]='a'. So we'll need to create a copy of whole key for
each GET. We will have very few SETs but millions of GETs. So is it
possible for the slice to be not-mutable like strings are or you can
suggest some other approach?
Thanks,
Slawomir.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/d4059458-7307-4789-91eb-7354bd998fc1n%40googlegroups.com.
[go-nuts] Detecting JSON changes
Hi Guys,
I have 2 json files, A, B. Now i want to detect changes on the root level.
File A: {"a":{"b":1, "c":2}, "x":false, ... }
File B: {"a":{"b":1, "c":2}, "x":true, ... }
I want to be able to see that x is different between A and B and a stayed
the same. What would be the easiest approach?
Thanks
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/7e88ac99-b6ff-49af-95a9-c1c5d7cf9236n%40googlegroups.com.
Re: [go-nuts] Structures / mutex question
Thanks, very good point about including the structure by-value.
As for using the structure via pointer
> You don't need to rlock the global mutex. Even if another goroutine
appends to
> the slice and slice gets reallocated, this code will still be pointing to
the correct element.
I'd assume I'll still need the global RLock, as len will not be
synchronized. And moreover if i shrink the array by 1 and then append one
element, i'll be changing the pointer in-place which could result in some
pseudo-random memory location being accessed.
niedziela, 9 października 2022 o 18:38:36 UTC+2 bse...@computer.org
napisał(a):
> On Sun, Oct 9, 2022 at 5:49 AM Slawomir Pryczek
> wrote:
>
>> Hi Guys,
>> wanted to see if i making correct assumptions regarding mutexes and
>> structures
>>
>> var globalMutex sync.Mutex{}
>> type abc struct {
>> a int
>> b int
>> mu sync.Mutex{}
>> }
>>
>> 1. First is self explanatory, array of structures all needs to be
>> protected by mutex
>> x := []abc{}
>> x = append(x, abc{}) // <- This needs to be done inside globalMutex.Lock
>>
>
> There are some things to consider for this usage. You need the global
> mutex to lock the slice when appending. If append needs to reallocate a new
> slice, using a *sync.Mutex will work, but any goroutines working on the
> elements of the slice will be working on stale copies of it. That is, if
> one goroutine appends to the slice while another is modifying its elements,
> the modification to the elements may be lost in the new copy of the slice.
>
> In short: you should not use struct abc as value if you're ever going to
> append to the slice.
>
> If you are not going to append, then embed *sync.Mutex, otherwise you will
> be copying the mutex.
>
>
>> x[0].a = 10 // <- This needs to be done inside globalMutex.Lock, as the
>> array/slice is holding the data (?)
>>
>
> In general, you can lock x[0] for this without locking the global mutex,
> so other goroutines can update other elements of the array. This, if only
> you are not appending to the slice.
>
>
>> - Reading x[0].a would require globalMutex.*RLock*
>>
>
> It would require x[0].RLock(), if you are not appending to the slice.
>
>
>>
>> - Mu is not necessary
>>
>
> You can do this with a global mutex, but that will prevent concurrency for
> goroutines working on different slice elements.
>
>
>>
>> 2. Array of pointer to structures
>> y := []*abc{}
>> _tmp := &abc{}
>> y = append(y, _tmp) <- This needs to be put inside globalMutex.Lock
>> Mutex
>>
>
> The above is correct.
>
>
>> y[0].b = 1 <- This can be put inside globalMutex.RLock and we also
>> need mu.Lock (as it's sufficient to read abc pointer and the pointer will
>> never change after the element gets added to the list)
>>
>
> You don't need to rlock the global mutex. Even if another goroutine
> appends to the slice and slice gets reallocated, this code will still be
> pointing to the correct element.
>
>
>> - Reading y[0].b would require RLock on both globalMutex and mu
>>
>
> It would require rlock on y[0] only.
>
>
>>
>> Or maybe it's better to just use generic read-write locks for everything
>> and don't bother?
>>
>> Thanks
>>
>>
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/ffc7c10e-ddb9-4bc5-a545-2a1ce7b22bfbn%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/golang-nuts/ffc7c10e-ddb9-4bc5-a545-2a1ce7b22bfbn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/4c03668d-58b7-4629-8c27-001473005145n%40googlegroups.com.
[go-nuts] Structures / mutex question
Hi Guys,
wanted to see if i making correct assumptions regarding mutexes and
structures
var globalMutex sync.Mutex{}
type abc struct {
a int
b int
mu sync.Mutex{}
}
1. First is self explanatory, array of structures all needs to be protected
by mutex
x := []abc{}
x = append(x, abc{}) // <- This needs to be done inside globalMutex.Lock
x[0].a = 10 // <- This needs to be done inside globalMutex.Lock, as the
array/slice is holding the data (?)
- Reading x[0].a would require globalMutex.*RLock*
- Mu is not necessary
2. Array of pointer to structures
y := []*abc{}
_tmp := &abc{}
y = append(y, _tmp) <- This needs to be put inside globalMutex.Lock Mutex
y[0].b = 1 <- This can be put inside globalMutex.RLock and we also need
mu.Lock (as it's sufficient to read abc pointer and the pointer will never
change after the element gets added to the list)
- Reading y[0].b would require RLock on both globalMutex and mu
Or maybe it's better to just use generic read-write locks for everything
and don't bother?
Thanks
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/ffc7c10e-ddb9-4bc5-a545-2a1ce7b22bfbn%40googlegroups.com.
[go-nuts] Re: Is overwriting contents of pointer field the expected behavior of json.Unmarshal?
>From unmarshal doc: 1. To unmarshal JSON into a pointer, Unmarshal first handles the case of the JSON being the JSON literal null. In that case, Unmarshal sets the pointer to nil. Otherwise, Unmarshal unmarshals the JSON *into the value pointed at by the pointer*. *If the pointer is nil, Unmarshal allocates a new value for it to point to.* 2. To unmarshal a JSON array into a slice, Unmarshal resets the slice length to zero and then appends each element to the slice. The problem with it is that it's really hard to describe what it really does without looking at code. When the slice size is reset to zero it means that only the length of this slice will be set to 0. The underlying data is still there in memory and the capacity is still X. Now during slice expansion, the data is not reinitialized to 0 if that slice is still able to fit another element (number of elements < current capacity) because the pointer was initialized, previously. So during first unmarshal a new element is created, with V=0, P=nil. Unmarshal will set the pointer to some place in memory (1) so it can store int value in there. After first Unmarshal you have a slice newElems with length 1 and capacity 4 with V=10 and P=0xSOMETHING. The pointer is there so the pointed data will be overwritten for element [0], as described by documentation in section (1) There's a problem with section (2) because that's not exactly what happens during append. There was a proposition to re-initialize the slice element before each element is "appended" by unmarshal (so it'd work like "real" append does and be in sync with the doc) but it was rejected (and that's very good as it broken some code). Real behaviour of unmarshal can maybe be described better by saying that it allocates slice elements with zeroinit and overwrites slice elements when allocation isn't needed. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6bb2c42b-dbae-44db-af95-16633c327cf8n%40googlegroups.com.
Re: [go-nuts] Re: Cannot instantiate T(ype) passed as pointer T?
Thanks, actually that's what I was looking for. Don't have much experience
using generics (like 2 days) but looks rather inconvinient to do that. This
pattern is convinient for you and you're using it in your code? You think
there are plans to introducing some feature which will allow to work with
*interfaces & generics "normally" (without additional boilerplate?)
czwartek, 21 lipca 2022 o 08:25:44 UTC+2 axel.wa...@googlemail.com
napisał(a):
> I'm not *entirely* sure what you are asking, but does this help, perchance?
> https://gotipplay.golang.org/p/DmxDIdHrV8o
>
> On Thu, Jul 21, 2022 at 8:10 AM Slawomir Pryczek
> wrote:
>
>> Actually this is better code for the question
>> https://gotipplay.golang.org/p/GQlB2tyyj53
>>
>> czwartek, 21 lipca 2022 o 08:08:51 UTC+2 Slawomir Pryczek napisał(a):
>>
>>> https://gotipplay.golang.org/p/ojY3RRJRJgy
>>>
>>> func GenericData[T DATA, T1 any]() {
>>> }
>>>
>>> When DATA is interface of pointer type, so methods can modify the
>>> underlying structure...
>>>
>>> // T1 <- any type, so can instantiate but can't increment as there's no
>>> interface
>>> var tt T1
>>> fmt.Println(tt)
>>>
>>> var d T // T <- DATA interface type, so can't initialize but can
>>> instantiate
>>> can't instantiate because the object will never initialize (nil)
>>>
>>> Can i create new object of type DATA somehow?
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/4b36ca5e-ab1e-4468-85e5-0467efa34ca9n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/golang-nuts/4b36ca5e-ab1e-4468-85e5-0467efa34ca9n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/b95015d2-98d7-4670-b03c-ea321bc98e78n%40googlegroups.com.
[go-nuts] Re: Cannot instantiate T(ype) passed as pointer T?
Actually this is better code for the question
https://gotipplay.golang.org/p/GQlB2tyyj53
czwartek, 21 lipca 2022 o 08:08:51 UTC+2 Slawomir Pryczek napisał(a):
> https://gotipplay.golang.org/p/ojY3RRJRJgy
>
> func GenericData[T DATA, T1 any]() {
> }
>
> When DATA is interface of pointer type, so methods can modify the
> underlying structure...
>
> // T1 <- any type, so can instantiate but can't increment as there's no
> interface
> var tt T1
> fmt.Println(tt)
>
> var d T // T <- DATA interface type, so can't initialize but can
> instantiate
> can't instantiate because the object will never initialize (nil)
>
> Can i create new object of type DATA somehow?
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/4b36ca5e-ab1e-4468-85e5-0467efa34ca9n%40googlegroups.com.
[go-nuts] Cannot instantiate T(ype) passed as pointer T?
https://gotipplay.golang.org/p/ojY3RRJRJgy
func GenericData[T DATA, T1 any]() {
}
When DATA is interface of pointer type, so methods can modify the
underlying structure...
// T1 <- any type, so can instantiate but can't increment as there's no
interface
var tt T1
fmt.Println(tt)
var d T // T <- DATA interface type, so can't initialize but can
instantiate
can't instantiate because the object will never initialize (nil)
Can i create new object of type DATA somehow?
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/a5f33c45-c41f-47f6-98db-314716b70c3bn%40googlegroups.com.
[go-nuts] Re: Best way of implementing generic binary search?
Thanks everyone, was able to make this working by passing a function with
same type.
wtorek, 19 lipca 2022 o 15:17:39 UTC+2 Brian Candler napisał(a):
> You don't need a compare method on your type. You can just pass a func(T,
> T) bool to your search function.
>
> In fact, the standard library already has this, except implemented in
> terms of slice indexes only, rather than a slice of generic type.
> https://pkg.go.dev/sort#Search
>
> You could always make a generic Search[T] as a counterpart to
> SearchFloat64s, SearchInts and SearchStrings. Since the source of those
> existing functions is simple, this is left as an exercise for the reader :-)
>
> https://cs.opensource.google/go/go/+/refs/tags/go1.18.4:src/sort/search.go;l=76-103
>
> On Tuesday, 19 July 2022 at 13:53:33 UTC+1 Slawomir Pryczek wrote:
>
>> Hi Guys, is it possible to implement generic, efficient binary search
>> using generics or interfaces. So i'll have some index, and data inside
>> single struct and then could just define a comparison function between 2
>> variables of same type index which will return bool.
>>
>> Will have 20-30 million datapoints
>>
>> type abc struct {
>> index uint32
>> data1 []byte
>> data2 []string
>> }
>> type bcd struct {
>> index [4]byte
>> data1 []byte
>> data2 []string
>> }
>>
>> a = []abc{...}
>> b = []bcd{...}
>> find(a, 217621)
>> find(b, [4]byte{1,2,54,11})
>>
>> Currently i have this, which is probably incorrect:
>> type comparable[TC any] interface {
>> compare(TC, TC) bool
>> }
>>
>> func bin[T comparable](data []T, find T) int {
>> }
>>
>> Is that even possible to do efficiently or i should just go with writing
>> separated code for each struct type ?
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/25fab0ac-9399-4c91-88b1-88183574ad71n%40googlegroups.com.
Re: [go-nuts] Issues with time.Time
Have you tried starting a simple thread which will update some variable every 100ms (for example), and then just get value of this variable? Using atomic functions. I believe memcached was 'caching' calls to get time this way and it won't probably be very accurate (as we don't have hard quarantees for doing sleep(..)) but even if it'd be +2 seconds off that shouldn't make an issue. poniedziałek, 18 lipca 2022 o 21:34:19 UTC+2 Ian Lance Taylor napisał(a): > On Sun, Jul 17, 2022 at 6:20 AM Andrew Phillips > wrote: > > > > I've discovered a few problems with time.Time but thought I better > discuss here first before creating issues at > https://github.com/golang/go/issues (in case I missed something obvious > :). These are mainly performance issues, but there is also the serious > problem that it is easy to accidentally disable "monotonic" time > differences (reverting to "wall" time) and that you cannot obtain a > monotonic time with *Location == nil (UTC). > > > > This all began with real software that generates (using time.Now()) and > retains millions of timestamps (in memory) used for various purposes > including the deletions of old data by comparing these times with "now". > Due to the large number of calls to time.Now() performance is important, > accounting for a significant proportion of processing time. > > > > The first optimization made (some time ago) was to replace calls to > time.Now() with time.Now.UTC() in an attempt to reduce the load on the > garbage collector. (Calling the UTC() method sets the *Location field to > nil which obviates the GC need to follow the pointer I believe.) > > I wouldn't expect that change to make a big difference, but it should > help a little. What would help more is converting your stored > time.Time values by calling the UnixNano method, as that would give > you a value with no pointers at all. However, that would of course > lose the monotonic time reading. > > > > Recent benchmarking revealed that calling calling time.Now().UTC() takes > more than twice as long as calling time.Now(). Moreover, time.Now() has > some code that is effectively "dead" (never executed) which could reduce > that it by another 30% -- there is an if statement that will always return > false until after the current time is well into the future (after 2150 or > so). > > I'm a bit surprised by these measurements. Can you share your > benchmark code? Note that this is an area where microbenchmarks can > be fairly misleading. Real code does not call time.Now in a tight > loop without other memory writes. > > > > A further issue I found from inspecting the code for the UTC() method is > that calling it clears the hasMonotonic flag and subsequent time > comparisons use "wall" time - this affects many calls to > ContextWithDeadline(). The only discussion I can find on why calling UTC() > should clear the hasMonotonic flag is this comment from Russ Cox [ > https://github.com/golang/go/issues/18991#issuecomment-306209288] which > does not explain what tests are fixed or even what is being tested. > > The fact that UTC strips the monotonic time is documented at > https://pkg.go.dev/time, which says "Because t.In, t.Local, and t.UTC > are used for their effect on the interpretation of the wall time, they > also strip any monotonic clock reading from their results." > > > > I find it particularly worrisome that you can easily and inadvertently > turn off "monotonic" time comparisons. > > > > Some changes I might propose are: > > 1. Removal of the dead code in time.Now() > > The code is not dead, it just won't fire yet. On amd64 it's four > instructions including a jump that should always be predicted > correctly. We would need to see some very clear evidence that those > four instructions weigh heavily compared to the couple of hundred > instructions required to fetch the current time value (on > linux-amd64). > > > 2. Calling UTC() should not clear the hasMonotonic flag > > That would be a backward incompatible change. We can't do that. > > > 3. Passing a non-monotonic time to contextWithDeadline should panic > > That would be a backward incompatible change. We can't do that. > Programs construct deadlines in all sorts of ways. > > > 4. Add a new function (eg, time.NowUTC()) to efficiently get the current > monotonic, UTC time > > We could do that. We'd want to see some clear evidence that this > makes a difference for multiple real programs. My guess, without any > data, would be that in the vast majority of programs time.Time values > are not a significant percentage of memory. And the default value of > the pointer returned by time.Now will be a pointer to the data section > rather than the heap, so the GC should discard the pointer with a few > comparisons. So it's not obvious why it would be a significant source > of GC time. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this
[go-nuts] Best way of implementing generic binary search?
Hi Guys, is it possible to implement generic, efficient binary search using
generics or interfaces. So i'll have some index, and data inside single
struct and then could just define a comparison function between 2 variables
of same type index which will return bool.
Will have 20-30 million datapoints
type abc struct {
index uint32
data1 []byte
data2 []string
}
type bcd struct {
index [4]byte
data1 []byte
data2 []string
}
a = []abc{...}
b = []bcd{...}
find(a, 217621)
find(b, [4]byte{1,2,54,11})
Currently i have this, which is probably incorrect:
type comparable[TC any] interface {
compare(TC, TC) bool
}
func bin[T comparable](data []T, find T) int {
}
Is that even possible to do efficiently or i should just go with writing
separated code for each struct type ?
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/a7e769c3-70a7-4aa6-96a5-453a55db3e60n%40googlegroups.com.
[go-nuts] Keeping self-referencing URLs outside of module's code
Hi Guys, have a golang module with multiple sub-modules, which i want to use "properly" in some other projects. The issue is that the path is very long as that module is a part of a bigger project and it's inconvinient to repeat it everywhere over and over. Module path: github.com/slawomir-pryczek/HSServer/go/src/handler_socket2 https://github.com/slawomir-pryczek/HSServer/tree/master/go/src/handler_socket2 1. So the first idea was just to put handler_socket2 instead of the path inside go.mod. Everything was working correctly within the module, but it produced issue during import: "module declares its path as: handler_socket2 but was required as: github.com/slawomir-pryczek/HSServer/go/src/handler_socket2". Is it possible to get rid of this somehow? So if people will write some module and decide to publish it later they can just publish it without modifying half of their projects files? 2. The second idea was to declare the module with "proper" path, but replace handler_socket2 with ../handler_socket2 (link below). https://github.com/slawomir-pryczek/HSServer/blob/master/go/src/handler_socket2/go.mod Which again produces an error during import: go: github.com/slawomir-pryczek/HSServer/go/src/handler_socket2@v0.0.0-20220713073931-8de251cbe13e requires handler_socket2@v0.0.0: malformed module path "handler_socket2": missing dot in first path element So is there any way of not including self-referencing github URL over and over again if you just want to reference the sub-module of the module you're in? It's like writing a website and including local, absolute server paths for every image. Then if i want to fork a module or create v2 it'll get a different URL and i'll have to edit almost everything, which again will make the forked code to have to include unneeded changes vs the base. Thanks -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b35f3aa1-1988-45da-a2b1-4551782780e0n%40googlegroups.com.
Re: [go-nuts] Modules... why it has to be so painfull?
Thanks for the help, so i think i get it fully now.
@Nick: Sure works great if you're making a project with github modules.
Having local modules is possible. But so cumbersome people would be
occupied with managing files rather than writing code (if you have >20 of
them).
@Carla: Actually i think the process is convinient now. Thanks for the
suggestion. Actually tried that before but failed as how it works is little
misleading because from reading the docs I got an impression that you need
to keep the go.mod inside app's main package. While module should be
created one level lower, then you can have local packages shared between
projects and you're not forced to be bound to your main package for imports.
Not the modules itself but the way of handling the issue or using modules
most people are suggesting are broken. The docs give impression i should
convert everything to modules while like @Carla said we should just say
create /src dir with module called src, put everything there, prefix
imports with "src" and if some code is stable/mature enough you could
publish it by converting to a "proper", hosted module later.
Not sure why almost everyone is mentioning local proxies, replace or some
other complex stuff and documentation is so complex, when you can just do
it like that. Was sitting on this module stuff and docs for it for 3 days,
probably more than 15h just to realize this is so simple and i was going in
totally wrong direction.
czwartek, 8 kwietnia 2021 o 16:22:56 UTC+2 m8il...@gmail.com napisał(a):
> > >>One solution is apparently to use a module like a giant mono repo, aka
> gopath>
> Still it requires for this replace
>
> You do not need replace, unless you are amalgamating multiple modules.
>
> You can just create a folder named e.g. library with a .go file; package
> lib
>
> Then in your other folders with package main, .go files:
>
> You can
>
> import (
> lib "{moduleName}"/library
> )
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/81bc87fc-5c02-4d4f-ae2d-6fc6da7607f3n%40googlegroups.com.
Re: [go-nuts] Modules... why it has to be so painfull?
Thanks for all replies. Actually the docs are good (imo) just these solutions proposed in docs are a horror. Inside root folder of the module... so im unable to share packages between projects and code structure gets horrible, because now instead of nicely organized 2 levels of nesting i have 3 levels and i need to repeat name of the project 5 times inside every of 25 different root packages. Maybe for small apps. If i'm having eg. slab allocator used by both webserver and k-v storage it'd be good to put it outside and not having half of the modules in other apps imported from webserver and other half from k-v storage. It introduces unneeded complexity if package pretends to have something in common with something which is totally unrelated just because it's easier to put it in there. Next i'll have to do v2 of webserver when API of some of packages will change, so the version number won't reflect version number of my app... but changes to some single package in it. Putting main app dir/files as a parent of general purpose packages is a bad idea. Having tax app to be a parent of decimals package is like having a 3d graph library to be a parent of network I/O package. There is no relation so it should be separated for readability. >Have you tried go mod edit -replace github.com/foo/bar=/path/to/local/bar ? I'll have to do hundreds of such edits and then re-edit it each time anything changes (package rename, moving files, etc.) My point is why this module approach broken the ability to organize code conviniently and logically... If we're in /src/foo and there is /src/bar and we do import "bar" in foo... it should just (what a surprise) import the local bar :D And another issue with this approach is that... should we put this abomination inside git? So really, everyone who downloads my code will have to waste 4 hours of his life replacing imports or he'll be forced to do edits inside C:\Users\slawomir\go or maybe he'll edit in F:\users\michael\my\nice\go\path and put it in repo so i'll have to change it back :D I know it's easier said than done but we have all this nice things like goroutines, GC, channels, peemption... and everything is so simple. Just to manually babysit package manager when previous model was working? Even having the user go to command line and issue module init for something without remote imports is a step backwards, because it's a waste of time, added complexity and totally unneeded. Having like tens of pages of "instructions" and blog posts telling users how to configure package manager just to run 10 lines of code if we want to organize it well is a step forward? With GOPATH you did one simple thing and it was working. Just once not 1000 times over and over, for each package or after anything in your dev enviroment changed. So not sure why everyone should be forced to migrate to this cumbersome model when we have nicely organized, self containted projects and the only issue is versioning for the PART of projects which are actually ... libraries, not applications. >One solution is apparently to use a module like a giant mono repo, aka gopath Still it requires for this replace and keeping absolute paths everywhere. It's not even possible to put your project on github this way because it won't compile without configuring everything. That's a huge regression because you'll have to copy-paste all shared code inside each individual app folder if you don't want to publish 30 packages as separate modules or put unrelated things together. Each solution is sub-optimal because separation can't just be done right in sane amount of time in this model because PM isn't able to conviniently handle local packages nor initialization nor changes in the packages... My point is, that it's so inconvinient that you'll organize your code badly just to save time... GOPATH is much better approach for bigger apps and it shouldn't be made obsolete. At least not if the only solution is putting things where they don't belong or setting up proxies to pretend local files are remote and then re-running some command line tools each time you add a line in one of these... środa, 7 kwietnia 2021 o 23:55:03 UTC+2 m8il...@gmail.com napisał(a): > >> https://golang.org/ref/mod > > The how to code docs need improving for packages but once you initialise a > module. You can create folders to hold different packages within the root > folder containing the .mod file. I'm not sure modules are not a regression > compared to gopath in some respects though? One solution is apparently to > use a module like a giant mono repo, aka gopath. I never used gopath > extensively myself. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.
[go-nuts] Modules... why it has to be so painfull?
Hey Guys, I'm struggling with the new "modules" approach and after checking several help files it seems it's inconvinient beyond belief. Basically i have an app: ...src/ /myapp/main.go package main /pool/pool.go package pool And i can't even include package pool into main without manually initializing module, then installing it, and then when i do any change to pool i'll have to re-get / re-install the package, so it even breaks things as simple as conviniently building a basic app with >1 shared package. Question is why it's no longer possible to break the project into independent packages and easily re-use them. I have eg. a webserver which implements each operation as separate package, and also it has slab allocator which other projects are using. And it works great. Maybe someday i'd like to convert the slab allocator into separated module, but why forcing users to do so much unnecessary work and pretend everyone wants to expose everything they're writing as module from earliest stage of the project? Is there any way to retain this kind of structure without countless hours wasted on manually initializing modules and other completely pointless maintenance tasks. Previously i was just able to create a package in gopath and use it everywhere, refactoring was very easy and i could easily split any project into multiple packages. Actually go was so good because refactoring and reorganizing code was so easy. Now it seems that's no longer possible and in docs I found info that gopath approach will be obsolete, so i'm trying to go with the new one. Which seems so painfull... Also read several posts about local packages. Complexity of this is beyond ridiculous. Really i need to install a local proxy or use some special directives to allow my appa and appb to use packagec? Really it needs to be so complex and so user unfriendly so instead of writing code we'll be thinking about setting up proxies and configuring dependencies just to share some code between 2 local apps because this has to be done via HTTP or special configuration? Anyone has an idea for a reasonable solution which will allow easy refactoring and code organization in packages, in this new model? Thanks, Slawomir. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/bb6d0bd0-e411-4c93-a1ee-5eec44e1bc48n%40googlegroups.com.
Re: [go-nuts] Writing data without synchronization
Thanks for very insightful posts. That's actually what i was interested in, as i was wondering if x86 is so advanced to invalidate caches on its own, or it's taken care by software/compiler or if that's just pure coincidence that this code actually works :) Actually as i'm doing persistent connections i'll just read everything once, on-connect and then keep a copy per thread while doing the synchronization "normally". Basically i'm not into micro-optimizations too much, in production code i see that synchronization of short code leads to starvation that's why i try to avoid it, but yes probably redesigning code would be better than using some "hacks" which can be unsafe. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/686c344a-fe16-4889-b2c8-e7d14d6525ae%40googlegroups.com.
[go-nuts] Writing data without synchronization
Hi Guys, was wondering about some things related to multithread code. 1. If data that is accessed or changed - it needs to go into CPU cache first, and AFAIK to caches whole memory block not just this single area of memory on which we're operating. When we're doing writes without synchronization to something which is tightly packed, how does it work that cache isn't poisoned, and the result is still correct, even if each cpu core is having its own L1 and L2 caches? Are such patterns, when single thread is only operating on single cell(s) of memory, considered thread safe for slices and in general? https://play.golang.org/p/rF8jPEGDZzk 2. For lazy initialization double-check pattern. I found on several sources that it is safe, still race detector complains about data races (which is correct because there is race, which is done on purpose) - https://software.intel.com/en-us/node/506123 And of course this code seems to be working as expected https://play.golang.org/p/gUzWHr-1K9H 3. Now question is, can i disable these warnings somehow so race detector won't complain about safe code because i have ton of these and it's obfuscating usable output? 4. Im not sure about instruction ordering, will it always be the case that threads that see initialized=true will also see myconfig AFTER the update, because myconfig isn't synchronized. Would it be better to just use pointer instead and check for nil value or even atomic.Value? 5. How these patterns are working with other CPUs, are they considered unsafe on anything other than intel architecture? So im writing some code which will do hundreds of thousands of operations per second and these operations need to read config (probably in some cases at multiple places). You think using just atomic.Value (Store/Load) will have impact when i run it on machine with eg. 96 cores? Maybe RW mutex would be better? What you think? I could just run some synthetic benchmarks but for these usually seem to not have much use in real life scenarios for multithreaded code. Thanks -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/eeadca01-1ad5-48e0-9ace-5afb19def2f3%40googlegroups.com.
[go-nuts] Re: Suggestions for dynamic per object lock
I think what would work is declaring some number of mutexes, in array like
48. Then you can compute CRC32 of the token, and lock based on modulo. But
yes, probably best would be to change underlying code and just partitioning
the work by hash.
const concurrency = 48
type concurrentChecks struct {
mutex [concurrency]sync.Mutex
}
func (c *concurrentChecks) addCheck(token string) {
mutex_num := crc32.ChecksumIEEE([]byte(token)) % concurrency
c.mutex[mutex_num].Lock()
}
func (c *concurrentChecks) removeCheck(token string) {
mutex_num := crc32.ChecksumIEEE([]byte(token)) % concurrency
c.mutex[mutex_num].Unlock()
}
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/e4df25db-28e9-4ffb-aacd-b290d6295e1d%40googlegroups.com.
[go-nuts] Re: General thoughts about new proposals
Oh gosh, i'd rather call it a disaster :D You written something bigger in C++ or Java? If yes you really want to go back to all this mess, where you're not even sure what the "=" or ">" operator is doing? IMO this is totally unfit for code people are writing nowdays. You can no longer afford a luxury of having fragmented, unreadable code because codebases are huge and time to market is low. Sure, it's little easier to write, then after a month when some small change is needed you realize it's better to write it all from 0. W dniu piątek, 5 lipca 2019 13:42:55 UTC+2 użytkownik jessie@rococoglobaltech.com napisał: > > is there a hope for generics like this? > > gen[A]ToStringer > gen[string].ToString() > > > > > > Noong Huwebes, Hulyo 4, 2019 ng 6:02:45 PM UTC+8, si Slawomir Pryczek ay > sumulat: >> >> Following this group for couple years and I think that from some time the >> community is in some kind of crisis, because it seems that go1 is so good >> that there's a lack of some feature which will distinct go1 from go2, so >> everyone's trying to invent some code breaking change which will "distinct" >> 1 from 2 just for the sake of breaking backward-compatibility. There are no >> real issues with language design, so people seems to be "inventing" >> problems ;) >> >> So we're having a flood of different specs, which are trying to "fix" >> something that's not broken by adding needless complexity or adding a >> "feature" from C or Java which is more complicated than the whole go1 when >> we look at it for more than 2 minutes. And go1 is so good it seems so easy >> to introduce couple of very bad ideas into the language because in the end >> it'll still looks nice. >> >> Generics, operator overloading, memory ownership, try-catch, >> precalculated functions, and the list could go on-and-on. There's C++, >> everyone's favourite "missing feature" is already there ... probably that's >> why it's such a delight to write anything in it with >1 person in team ;) >> And if you "just" miss try/catch and generics it's called java. So much >> effor went into making go simple to read and develop, and to remove all >> "dust" which C++ gathered over the years, now I think so much thinking goes >> into bringing it all back. I think when creating specs people are totally >> missing the point... we thankfully don't have to deal with overloading or 3 >> different kind of for-loops so we can focus on algorithms because code is >> easy to read. Since when replacing 3 different loop keywords for 3 >> different conditional keywords plus adding code fragmentation sounds like a >> good idea? So when reading single line, we'll have to check specs and maybe >> 4 other places in the file to be kind-of-sure what it does, like it happens >> in C++, just to save a newline... >> >> It's really not about the specs, but the amount of support every change >> usually gets, seems just for the sake of changing something. I'm afraid >> some of these could be introduced, and the language will be going towards >> becoming some kind of bad c++ clone. And we could end up with something >> even as bad and unstable as node-js very quickly, because it seems that >> currently google is the only force which keeps potential feature creep in >> check. Really surprising how fast people forgot how horrible try/catch or >> generics are. And (especially for generics and overloading) - how >> unreadable and unmaintainable code using it really is. For sure there's >> room for improvement by inventing some new ways of doing things... not >> forcefully porting bad, decades old, error prone "ways of thinking" from >> C'ish languages, so we can spare a newline here and there or avoid typing 3 >> chars... >> >> So just my 2c for keeping simplicity. And if go2 can compile go1 code >> without changes, that's actually a feature not a "problem" and anyone can >> create overly complicated system, so yes simplicity isn't a "problem" as >> well ;) >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9b054633-465c-467a-a498-be499bc0146d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Re: Does the code have any concurrent problem?
Interesting point because output would be random but still n could be
accessed by only one thread at a time, after the change ... so technically
it's different kind of a race (race due to the order in which operations
are scheduled because they can not be safely serialized, not due to
simultaneous concurrent access due to technical error).
But sure, changing it to eg, 3 loops with are incrementing n 100k times
would be much better example ;)
W dniu piątek, 5 lipca 2019 13:59:10 UTC+2 użytkownik Robert Engels napisał:
>
> Even if you do this, the code still has a “race” as there is no way to
> logically reason as to the eventual outcome or purpose. You can take out
> all of the concurrent controls and you’ll have the same program/result
> (although the race detector will report an issue) - no claims can be made
> about any of the observed values.
>
> On Jul 5, 2019, at 6:42 AM, Slawomir Pryczek > wrote:
>
> Sure it has a race issue, because in this example you're mixing atomic
> operations with mutex, when you want your code to be thread safe you need
> to do access to the variable using mutex or using atomic in all cases.
> Because both ways work differently and are not "interchargable" so you
> can't cover access to same object using a mix of 2 ways.
>
> You can fix your code like this:
> 1a. change m.RLock() to m.Lock()/m.Unlock() near atomic.AddInt32(&n, 1)
> because it's writing operation so you need write mutex
> 1b. add m.Lock()/m.Unlock near n=0 because it's writing operation as well
> The, you can remove atomic ops, because now all the code is covered by
> mutexes
>
> 2. Change n=0 to atomic.StoreInt32(&n, 0)
> Then you can remove all mutexes, because access is covered by atomic ops
>
>
>
>
>
> W dniu piątek, 5 lipca 2019 12:40:32 UTC+2 użytkownik White Pure napisał:
>>
>> Hi,
>> I wrote some code like below these days, and my colleagues and me are
>> not sure if the code has any concurrent problem.
>> Can someone help to figure out if the code has any race problem?
>> Thanks very much!
>>
>> Code:
>>
>>> package main
>>
>>
>>> import (
>>
>> "sync"
>>
>> "sync/atomic"
>>
>> )
>>
>>
>>> func main() {
>>
>> var n int32
>>
>> var m sync.RWMutex
>>
>> go func() {
>>
>> for {
>>
>> atomic.LoadInt32(&n)
>>
>> }
>>
>> }()
>>
>> go func() {
>>
>> for {
>>
>> m.RLock()
>>
>> atomic.AddInt32(&n, 1)
>>
>> m.RUnlock()
>>
>> }
>>
>> }()
>>
>> go func() {
>>
>> for {
>>
>> m.Lock()
>>
>> n = 0
>>
>> m.Unlock()
>>
>> }
>>
>> }()
>>
>> // do something to keep goroutines running here
>>
>> ..
>>
>> }
>>
>>
>> Playground link: https://play.golang.org/p/fRa09l3VQob
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golan...@googlegroups.com .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ed3e8643-8781-438d-8191-1c60ee7473c5%40googlegroups.com
>
> <https://groups.google.com/d/msgid/golang-nuts/ed3e8643-8781-438d-8191-1c60ee7473c5%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/2c33216c-eac9-40b8-92d9-b25a3455bb83%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: readonly []byte required
Not sure if that's a good idea. Strings are immutable so you can pass
string to function by reference, in thread safe manner. So if you pass same
string to 2 threads and one of them modify it - you'll really have
allocate+copy+modify so you're never touching the original thing so you
can't have a race.
Now if go introduced readolny []byte, we'd probably want to change all io
functions to use that... and this would create a lot of confusion, because
it'd be required to cast []byte to readonly version. And this casted
variable wouldn't be readonly nor thread-safe at all, because underlying
memory could still be changed using the original slice. So we'd need
allocate+memcopy for readonly cast to do a network write.
Or maybe there could be readonly []byte version for each io op. But it'd
result only in many unnecessary OS calls.
So what is needed in this case (i guess) is a reusable buffer anyway, and then
you can do a copy to a byte buffer without a cast and allocation
tstr := "abc"
t := make([]byte, 10, 10)
copy(t, tstr)
W dniu piątek, 5 lipca 2019 04:27:58 UTC+2 użytkownik Ally Dale napisał:
>
> []byte is mutable in Go, but string is readonly, so []byte(string) in Go
> is actually allocate&memcopy, and this operation is costly.
> Sometimes, we need a readonly []byte parameter but we have string only,
> and we have no choice but allocate&memcopy to make a new []byte variable
> from string.
> Eg: net.Send([]byte("ping")) is costly, but net.Send(readonly
> []byte("ping")) can be cheaply.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/804e9ecf-4d2e-4ffd-8cca-1a9c91928bb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Does the code have any concurrent problem?
Sure it has a race issue, because in this example you're mixing atomic
operations with mutex, when you want your code to be thread safe you need
to do access to the variable using mutex or using atomic in all cases.
Because both ways work differently and are not "interchargable" so you
can't cover access to same object using a mix of 2 ways.
You can fix your code like this:
1a. change m.RLock() to m.Lock()/m.Unlock() near atomic.AddInt32(&n, 1)
because it's writing operation so you need write mutex
1b. add m.Lock()/m.Unlock near n=0 because it's writing operation as well
The, you can remove atomic ops, because now all the code is covered by
mutexes
2. Change n=0 to atomic.StoreInt32(&n, 0)
Then you can remove all mutexes, because access is covered by atomic ops
W dniu piątek, 5 lipca 2019 12:40:32 UTC+2 użytkownik White Pure napisał:
>
> Hi,
> I wrote some code like below these days, and my colleagues and me are
> not sure if the code has any concurrent problem.
> Can someone help to figure out if the code has any race problem?
> Thanks very much!
>
> Code:
>
>> package main
>
>
>> import (
>
> "sync"
>
> "sync/atomic"
>
> )
>
>
>> func main() {
>
> var n int32
>
> var m sync.RWMutex
>
> go func() {
>
> for {
>
> atomic.LoadInt32(&n)
>
> }
>
> }()
>
> go func() {
>
> for {
>
> m.RLock()
>
> atomic.AddInt32(&n, 1)
>
> m.RUnlock()
>
> }
>
> }()
>
> go func() {
>
> for {
>
> m.Lock()
>
> n = 0
>
> m.Unlock()
>
> }
>
> }()
>
> // do something to keep goroutines running here
>
> ..
>
> }
>
>
> Playground link: https://play.golang.org/p/fRa09l3VQob
>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/ed3e8643-8781-438d-8191-1c60ee7473c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] The "leave "if err != nil" alone?" anti-proposal
It seems I'm hearing that a lot. If C++/Java is so great and go is so "primitive"... well what all these people are still doing in here? ;) And i'd take "primitive" if err!=nil over exceptions any time of the day, at least code is streamlined, ordered and easy maintainable this way. Then i can even add any new code which supports error handling inside block that properly does error handling without looking at previous code, another advantage of this "primitive" approach. With go i can take anything and move it around almost freely. I hate try/catch both in C++ and in Java. I'm not even sure why we need another complex, general purpose, inconvinient language. If people would need all that "great" C++ stuff they could easily ... use it? That's the best we can do, take decades old way of looking at things because it allows developers to be lazy even if it results in code which doesn't handle errors in optimal manner, is hard to maintain and millions of flies can't be wrong? All these "modern" OOP ideas are making code we write today so unreadable and fragmented it's impossible to write anything in any sane amount of time because of size of codebases we need to maintain nowdays. And you want to bring that craziness back? If people are missing proven, dominant methods... i think the correct way is just going back to C++, D or Java. They had all of these "modern" "features" 10 years ago ;) W dniu sobota, 29 czerwca 2019 22:20:47 UTC+2 użytkownik Robert Engels napisał: > > I think you are confusing with ‘doing nothing as compared to the ‘try’ > proposal’ with ‘do nothing’. > > Go error handling is primitive at best, and people want a solution, just > not the ‘try’ proposal - which is the basis of my comment email - go back > to what is proven to work - Java and C++ are the dominant languages in use > today, adding their exception based error handling to Go is ‘the way to Go” > :) > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/61cfc96d-92e0-46a1-8579-47324ec8e5f7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] General thoughts about new proposals
Following this group for couple years and I think that from some time the community is in some kind of crisis, because it seems that go1 is so good that there's a lack of some feature which will distinct go1 from go2, so everyone's trying to invent some code breaking change which will "distinct" 1 from 2 just for the sake of breaking backward-compatibility. There are no real issues with language design, so people seems to be "inventing" problems ;) So we're having a flood of different specs, which are trying to "fix" something that's not broken by adding needless complexity or adding a "feature" from C or Java which is more complicated than the whole go1 when we look at it for more than 2 minutes. And go1 is so good it seems so easy to introduce couple of very bad ideas into the language because in the end it'll still looks nice. Generics, operator overloading, memory ownership, try-catch, precalculated functions, and the list could go on-and-on. There's C++, everyone's favourite "missing feature" is already there ... probably that's why it's such a delight to write anything in it with >1 person in team ;) And if you "just" miss try/catch and generics it's called java. So much effor went into making go simple to read and develop, and to remove all "dust" which C++ gathered over the years, now I think so much thinking goes into bringing it all back. I think when creating specs people are totally missing the point... we thankfully don't have to deal with overloading or 3 different kind of for-loops so we can focus on algorithms because code is easy to read. Since when replacing 3 different loop keywords for 3 different conditional keywords plus adding code fragmentation sounds like a good idea? So when reading single line, we'll have to check specs and maybe 4 other places in the file to be kind-of-sure what it does, like it happens in C++, just to save a newline... It's really not about the specs, but the amount of support every change usually gets, seems just for the sake of changing something. I'm afraid some of these could be introduced, and the language will be going towards becoming some kind of bad c++ clone. And we could end up with something even as bad and unstable as node-js very quickly, because it seems that currently google is the only force which keeps potential feature creep in check. Really surprising how fast people forgot how horrible try/catch or generics are. And (especially for generics and overloading) - how unreadable and unmaintainable code using it really is. For sure there's room for improvement by inventing some new ways of doing things... not forcefully porting bad, decades old, error prone "ways of thinking" from C'ish languages, so we can spare a newline here and there or avoid typing 3 chars... So just my 2c for keeping simplicity. And if go2 can compile go1 code without changes, that's actually a feature not a "problem" and anyone can create overly complicated system, so yes simplicity isn't a "problem" as well ;) -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5048648f-0c00-49dd-ab7d-f0b1f95ad3f8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: `on err` alternative to `try()` has traction...?
On one side, it's 1000x better than the other error handling specs, at least it isn't going to turn code into unreadable, fragmented mess. On the other, Aston seems to have a point, it's just replacing one-liner... and it's not that great at all because with "if" you know what it's doing without reading the spec. My point is it doesn't fix anything, it doesn't provide any clear benefit... it's an attempt to fix something which works great and is clearly not broken. So why complicate the language with a new keyword which has really no purpose. Maybe adding a warning about unhandled errors to VET would be better idea (which would probably be complicated to do properly, but at least it'll have some real, positive effect on code quality). W dniu wtorek, 2 lipca 2019 21:57:24 UTC+2 użytkownik Liam napisał: > > This proposal has attracted modest attention from the Go team... > https://github.com/golang/go/issues/32611 > > It suggests: > > err := f() > on err, > > on err, return err// any type can be tested for non-zero > on err, return fmt.Errorf(...) > > on err, fmt.Println(err) // doesn't stop the function > on err, continue // retry in a loop > > on err, goto label// labeled handler invocation > on err, hname // named handler invocation > > > > And offers these possible extensions: > > on err, os.IsNotExist(err): > on err, err == io.EOF: > on err, err.(*os.PathError): // doesn't panic if not a match > > on err, : > on err: // this pair provides if/else in 2 lines > > on err := f(), // for assignment with single lvalue > > > > Other punctuation is possible, e.g. on (err) > > Now if we could just convince the Go gods to prototype this along with > try() in 1.14 :-) > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/10352c41-6c98-46e0-9da3-15240d8cde08%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Interesting public commentary on Go...
Come on, open your minds a little. Once every 5 years it doesn't hurt to learn some new, better ways of doing things. And if someone wants to write java code untill he dies, then there's a great method of doing that called "stick to java" ;) That same kind of thinking already marginalized nodejs. The code written in it is horrible because almost everyone is trying to emulate c++ and java instead of taking one week to learn the language a bit and JS is so flexible you can actually do that to some extent. Not sure why some people think that every language on the planet needs to be a bad clone of c++ or java probably putting "java" in its name doesn't help. Thanks GOD it's golang, not go++, we'd be doomed. Seriously. For me - against generics because of amount of complexity and issues it'd introduce into the language. Would be cool to see a fork implementing it because maybe it can be done "nicely", however i doubt it... It brings to mind Rust's multithreading paradigms. That's actually great stuff... but totally unfit for integrating with golang (for this im preety sure). Hopefully, if generics will be implemented it won't be integrated too tightly so i won't have to get back to writing code which is more jav'ish that it needs to be. Because, that's a crazy idea... if i liked how java works more i'd just use java ;) And introducing operator overloading into this beautiful design will be like putting readability back into medieval ages... -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d0ab80ae-ad07-43e3-9a87-32a61940c639%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] time.Now.UnixNano() incorrect under windows7?
Hi Guys, so i have this small go program which works fine under linux... but there's some very strange issue with getting microsecond-precision time under windows7. https://play.golang.org/p/N9F7xpx7hEr It won't run properly under playground so let me just paste here so you can see the behaviour im observing, the format is milliseconds [dot] microseconds [space] raw value Linux: Took: 0 . 0 107 Took: 0 . 108 108383 Took: 0 . 204 204381 Took: 0 . 300 300110 Took: 0 . 395 395757 Took: 0 . 499 499740 Took: 0 . 597 597277 Took: 0 . 692 692953 Took: 0 . 788 788634 Took: 0 . 884 884278 Took: 0 . 979 979904 Took: 1 . 75 1075548 Took: 1 . 171 1171267 Windows: Took: 0 . 0 raw: 0 Took: 0 . 0 raw: 0 Took: 0 . 0 raw: 0 Took: 0 . 0 raw: 0 Took: 0 . 0 raw: 0 Took: 0 . 0 raw: 0 Took: 1 . 0 raw: 100 Took: 1 . 0 raw: 100 Took: 1 . 0 raw: 100 Took: 1 . 0 raw: 100 Took: 1 . 0 raw: 100 Took: 1 . 0 raw: 100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 2 . 0 raw: 2000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 3 . 0 raw: 3000100 Took: 4 . 0 raw: 4000200 Took: 4 . 0 raw: 4000200 Took: 4 . 0 raw: 4000200 So in linux the microsecond part is incremented correctly. In windows the milliseconds part is incremented correctly but for the microseconds part it seems it's just adding 0.1 microsecond for each millisecond elapsed, so the readings are off. Any idea why it's working this way, can it be fixed somehow? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Golang closures, counter-intuitive behaviour
Hi Guys,
When looking at this code below, you can see that the function will get
LAST value of i which is incremented inside the loop, but t is somehow
copied and taken inside a function so closure is created and it is bound to
current value of t, so we'll have its current value in function. This is
strange as t and i is defined on the same code block level (t will be
discarded at the same time as i, after the loop).
I always thought that t would need to be passed explicitly to goroutine to
retain current value (which i did). But why assigning value in a loop is
treated differently in this regard than a straight assignment? Any URL
where such "special cases" are documented? Isn't it kind of design flaw,
because this seem very confusing that loop is creating single value and
simple assignment is creating multiple copies? Maybe you consider changing
this in go2?
https://play.golang.org/p/6vx_qDOk51g
10 1
10 0
10 9
10 8
10 7
10 6
10 5
10 4
10 3
10 2
Thanks,
Slawomir.
-
package main
import (
"fmt"
"time"
)
func main() {
for i := 0; i < 10; i++ {
t := i
go func() {
time.Sleep(time.Millisecond * 100)
fmt.Println(i, t)
}()
}
time.Sleep(time.Millisecond * 1000)
}
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Sort a huge slice of data around 2GB
It should be very simple if you have additional 2G of memory. You divide the data to X parts where X is power of 2 and X needs to be less than number of cores available. Eg. for 2000MB it can be 250x8. Then you sort it in paralell using built-in sorting function and at the end you just concatenate the arrays by scanning them and picking smallest element as you move forward. You turn 250x8 => 500x4 => 1000x2 => 2000. It should take about 15-20 minutes to write, and the sorting would probably take 5-6 minutes instead of half hour for 8 threads. In the concat phase you can also paralellize the process more, by getting the average of middle elements, then finding position where element is less than this value and just breaking the array, at this point into 2 sepatate arrays. Eg. if you're doing the last step and you see the element in Arr1 at position 50 is 91821 and at the same position in Arr2 you have 1782713 => average is 937267 you're using binary search to find position of 937267 or anything that is closest in both arrays - then you can break the arrays to Arr11 +Arr12 / Arr21 + Arr22 and you just concat Arr11 c Arr21 and Arr21 c Arr22. But that probably would take more time and is not necessairly worth the effor. W dniu środa, 29 listopada 2017 15:19:13 UTC+1 użytkownik Subramanian K napisał: > > Hi > > I am using native sort provided part of the package and to process 48MB of > slice data, it takes ~45sec. > To run 2GB of data it takes really long time, I am trying to split these > to buckets and make it run concurrently, finally need to collate results of > all these small sorted buckets. > > Do we have any sort package which can sort huge data swiftly? > > Regards, > Subu. K > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: concurrent write-only to map ok?
I think using standard sync'ed map may be bad idea for this use case (write-only access), especially taking into account that it's optimized for read access and stable keys, so each write will acquire mutex anyway. if nothing is read from that map during the threads run, it should be probably much better to assign single map for every thread and at the end, copy/merge the results to single "output" using mutex or rwmutex. This way you can acquire mutex once per thread, acquiring it for every write from multiple threads (which is what sync/map will do) is a performance killer. W dniu piątek, 13 października 2017 20:05:56 UTC+2 użytkownik Alex Buchanan napisał: > > Basically, I want to spawn a goroutine per object, objects have unique > IDs, and I want each routine to write its results to a shared map. Nothing > will be reading from the map while the goroutines are running. > > Is this safe? > > Thanks > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Speeding up a concurrent "simple" web server
AB is using HTTP/1.0, so it isn't able to do keepalives and it may be the case that you're really benchmarking your TCP stack, instead of your webserver... as TCP connect is probably the bottleneck here :) Not sure how it works under the hood, but enabling tcp_reuse, and tcp_recycle might improve the results for ab, which are irrelevant anyway. Basically this seem related to HTTP/1.0 being outdated, so it isn't able to handle large amount of connections between 2 points well. We had same characteristics, seen in logs - untill we enabled keepalive and HTTP/1.1 for our micro-services. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Implementing a custom TCP client
Hi, your problem is that you need message length to be embedded somewhere,
preferable at the message beginning because otherwise >1 message could be
merged into single packet and you won't know where it is supposed to end.
Eg. if you send small messages 100 times, more messages will be merged into
larger TCP packet, so you'll only receive data eg. 10 times, but you still
need to know there is 100 requests there, to send 100 responses. Also
single message can be split into several packets / will require >1 read
call to get fully.
If you have no length (nor unique suffix) you can't know if message ended
and you need to start processing it, or there are link issues and you
should return a timeout error and maybe re-try. I have project like this on
github. It's golang TCP server and php TCP
client https://github.com/slawomir-pryczek/HSServer
Writing client is much easier, and TCP for server is implemented in single
file basically
(
https://github.com/slawomir-pryczek/HSServer/blob/master/go/src/handler_socket2/handler_socket.go
) ... so you can take a look ;) Only things which are not so obvious at
first is that you need to know exactly how long the message is, when to
finish reading and start sending, and you need some code to detect errors
so you can close connection (and later re-open it, instead of sending more
data when your link is in failed state, if there are some issues).
If you have 2 way blocking communication (C -> S, S->C) and you'll begin
sending your response back BEFORE receiving full (single) request from the
client, then you'll break the process because client could get blocked on
Send while Server will also block because it's already sending BUT client
is not ready to receive... so you'll have a deadlock as both C and S will
try to send data at the same time.
W dniu środa, 20 września 2017 14:26:41 UTC+2 użytkownik newbg...@gmail.com
napisał:
>
> Hi all!
> I already asked this on reddit
> <https://www.reddit.com/r/golang/comments/712epf/help_implementing_custom_protocol_client/>,
>
> but I figured might as well ask it here.
> Im trying to implement a custom TCP client. This protocol allows request
> from both the client and the server
>
>
> Scenario 1:
>
> Client ---REQUEST--> Server
>
> Client <-RESPONSE--- Server
>
>
> Scenario 2:
>
> Client <--REQUEST Server
>
> Client ---RESPONSE--> Server
>
>
> Here's
> <https://gist.github.com/anonymous/8246c2ec615fc101f567ce536f5d284a#file-client-go-L88>
> simplified
> form of the client. This approach uses a goroutine to read the *net.Conn,
> *parse the string to determine the type,
> and send the string to the appropriate channel for that type. Typical
> usage of this client is as follows
>
> c := client.New("localhost:")
> c.Connect()
> c.Send("message")
>
> On small workloads, it works fine, however calling Send around 100 times a
> second times out.
> I can confirm using wireshark that the server indeed sent a response,
> however it seems like the readLoop wasn't able read it and send it to the
> channel.
> I would like to ask if this approach is reasonable. Id like to ask anyone
> out there have implemented a client golang library thats able to both send
> and receive requests from the server. Basically a bidirectional tcp
> client. I know *nats *is similar to this, however their protocol parsing
> is too complex for me. Any new approach/advise will be nice. Thanks!
>
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Any ideas about slow io performance on Windows?
Are you sure you have this issue on write, not on previous read? W dniu środa, 4 stycznia 2017 16:24:13 UTC+1 użytkownik lixi...@gmail.com napisał: > > I'm working on a TUN/TAP library on Windows. ( > https://github.com/lixin9311/water) > It's basically completed, but I have encountered IO performance issue. > Here is a simple demo, just redirect everything from the TAP to another PC > via UDP. > https://gist.github.com/lixin9311/847b51e44898deabe34794d223c1a7c7 > I use a Linux PC and a Windows PC. > > > https://gist.github.com/lixin9311/847b51e44898deabe34794d223c1a7c7#file-tapdemo-go-L39 > > This line of code takes a lot of time, sometimes even 10s. > Do you have any ideas about this? > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Extending objects / methods
Thanks for the post, yes basically i don't want to inherit old methods...
the member approach would work for me but im thinking if that won't be
slow, probably will do some tests... basically im trying to find some way
of doing this without need of additional struct with just one member...
type xadvanced *x
defining type xadvanced (with 0 methods), adding some methods to xadvanced,
which will access methods of "parent" type x ... by doing something like
"pointer conversion" in C, but I didn't found a way of doing this ... so
that's maybe not possible
W dniu piątek, 2 grudnia 2016 16:39:31 UTC+1 użytkownik parais...@gmail.com
napisał:
>
> The declared type does not inherit any methods
>> <https://golang.org/ref/spec#Method_declarations> bound to the existing
>> type, but the method set <https://golang.org/ref/spec#Method_sets> of an
>> interface type or of elements of a composite type remains unchanged:
>
>
> https://golang.org/ref/spec#Type_declarations
>
> You need to use another strategy like struct embedding which *may *or *may
> not* work depending to your use case
>
> A field declared with a type but no explicit field name is an *anonymous
>> field*, also called an *embedded* field or an embedding of the type in
>> the struct. An embedded type must be specified as a type name T or as a
>> pointer to a non-interface type name *T, and T itself may not be a
>> pointer type. The unqualified type name acts as the field name.
>
>
> type xadvanced struct {
> *x
> }
> func(x *xadvanced)increment(){
>x.y ++
> fmt.Println(x.y)
> }
>
> I'd advise you do read the spec at least once, it's short and concise .
>
>
> Le vendredi 2 décembre 2016 16:25:50 UTC+1, Slawomir Pryczek a écrit :
>>
>> Hi Guys,
>>
>> i want to make something like this
>>
>> type si struct {
>>
>> s *sync_task.Sync_task
>>
>> }
>>
>>
>> func (this *si) Process() {
>>
>>
>>... some code here ...
>>
>>this.Process();
>>
>> }
>>
>>
>>
>> Basically i want to extend object in another package... and this works.
>> Now i'd just want to extend it without creating "real" struct, code
>>
>> type x struct {
>> y int
>> }
>> type xadvanced *x
>>
>> func (this *x) increment() {
>> this.y++
>> fmt.Println(this.y)
>> }
>>
>> test2 := xadvanced(&test)
>> test2.increment() ---> ERROR
>>
>> https://play.golang.org/
>>
>> What im doing wrong and how to access methods of x object when having
>> pointer to xadvanced...
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Extending objects / methods
Hi Guys,
i want to make something like this
type si struct {
s *sync_task.Sync_task
}
func (this *si) Process() {
... some code here ...
this.Process();
}
Basically i want to extend object in another package... and this works. Now
i'd just want to extend it without creating "real" struct, code
type x struct {
y int
}
type xadvanced *x
func (this *x) increment() {
this.y++
fmt.Println(this.y)
}
test2 := xadvanced(&test)
test2.increment() ---> ERROR
https://play.golang.org/
What im doing wrong and how to access methods of x object when having
pointer to xadvanced...
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Memcached replacement in GO!
Interesting, thanks for the link... wasn't able to find this in google when i was looking for some working extension the last time... we'll try it out for some services :) W dniu czwartek, 17 listopada 2016 02:51:38 UTC+1 użytkownik Hirotaka Yamamoto (ymmt2005) napisał: > > > Basically the memcache*d* extension for PHP was buggy > > yeah, and that's the reason I wrote our own memcached client for PHP5 & 7. > We believe it's very stable as we do not experience even a single failure > in our > cloud service for years. > > Please try out! > https://github.com/cybozu/php-yrmcds/ > > 2016年11月17日木曜日 8時58分48秒 UTC+9 Slawomir Pryczek: >> >> >> Here is nice page describing the issue: >> http://developers.nk.pl/tag/gc/ >> >> >The reality in nk.pl was that evictions of valid items happened hundreds >> >of times per second even though there were 50% of expired items, >> >because there is no inherent correlation between TTL and the pattern of >> usage. >> >> So basically if you have a plenty of items with short TTL, like 10-30 >> seconds, and then some items that need to live long, eg. 10 minutes, then >> the short living items will pile up at the beginning of the list, and >> memory will never be reclaimed, because GC will not pass the "barrier" >> created by long living items at the end (it starts from the end), because >> it'll clean only first expired item(s) found. In recent memcached release >> there is some mechanism to traverse all items from time to time... not sure >> how good it works, >> >> https://github.com/memcached/memcached/wiki/ReleaseNotes1431 >> >> Basically the memcache*d* extension for PHP was buggy and that's one of >> the reason for writing my project ... >> >> >> W dniu środa, 16 listopada 2016 22:52:55 UTC+1 użytkownik Jesper Louis >> Andersen napisał: >>> >>> >>> >>> On Wed, Nov 16, 2016 at 3:10 PM Slawomir Pryczek >>> wrote: >>> >>>> >>>> - Much better Garbage Collection mechanism, that isn't skipping items >>>> with short TTLs like LRU, so memory is not occupied with garbage data even >>>> if item TTLs differ a lot. >>>> >>>> >>> Could you expand on what this problem is? It sounds like a subtlety many >>> people implementing LRU will hit and overlook. Perhaps give an example of >>> what happens. >>> >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Memcached replacement in GO!
Here is nice page describing the issue: http://developers.nk.pl/tag/gc/ >The reality in nk.pl was that evictions of valid items happened hundreds >of times per second even though there were 50% of expired items, >because there is no inherent correlation between TTL and the pattern of usage. So basically if you have a plenty of items with short TTL, like 10-30 seconds, and then some items that need to live long, eg. 10 minutes, then the short living items will pile up at the beginning of the list, and memory will never be reclaimed, because GC will not pass the "barrier" created by long living items at the end (it starts from the end), because it'll clean only first expired item(s) found. In recent memcached release there is some mechanism to traverse all items from time to time... not sure how good it works, https://github.com/memcached/memcached/wiki/ReleaseNotes1431 Basically the memcache*d* extension for PHP was buggy and that's one of the reason for writing my project ... W dniu środa, 16 listopada 2016 22:52:55 UTC+1 użytkownik Jesper Louis Andersen napisał: > > > > On Wed, Nov 16, 2016 at 3:10 PM Slawomir Pryczek > wrote: > >> >> - Much better Garbage Collection mechanism, that isn't skipping items >> with short TTLs like LRU, so memory is not occupied with garbage data even >> if item TTLs differ a lot. >> >> > Could you expand on what this problem is? It sounds like a subtlety many > people implementing LRU will hit and overlook. Perhaps give an example of > what happens. > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Memcached replacement in GO!
It's meant for different purpose i think (eg. CAS, inc/dec, php client, fast item reclaim)... Main purpose for this project is to allow easy operations on items when there's a lot of paralell access to the same keys, so there's eg. a very easy way of atomically modifying a value using just callback (that isn't something that you can easily do using eg. memcached), easy way of doing caching with automatic stampeding herd prevention, some operations that will make managing dynamic lists of items easier, large items, etc. Generally it's what we were missing in memcached, for our company projects... > > 2016. november 16., szerda 15:10:30 UTC+1 időpontban Slawomir Pryczek a > következőt írta: >> >> Hi Guys, I wrote memcached like PHP-client and GO-server, with additional >> features, >> > > Why not https://github.com/golang/groupcache ? > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: httputil.ReverseProxy adding 100+ ms of latency on localhost - any ideas?
If you're on a beefy machine with recent linux, maybe MTU of loopback interface is set to 65k, setting it to default 1500 would help i think... ifconfig lo mtu 1500 up https://www.cyberciti.biz/faq/centos-rhel-redhat-fedora-debian-linux-mtu-size/ W dniu środa, 16 listopada 2016 06:01:50 UTC+1 użytkownik Tom napisał: > > Hi guys, > > I've been working on a HTTP reverse proxy that has an auth/authorization > function. It all works well, however I have noticed significant additional > latency - and I cannot work out why. From what I can tell, me hitting my > service from chrome on localhost should work just fine - I have a very > beefy machine! (8core 3.4Ghz Amd64) > > Anyone have any insights into httputil.ReverseProxy, or have any ideas > where to begin? > > The method is here: > https://github.com/twitchyliquid64/pushtart/blob/master/src/pushtart/webproxy/handler.go#L61 > > Cheers, > Tom > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Memcached replacement in GO!
Hi Guys, I wrote memcached like PHP-client and GO-server, with additional features, it is in production for a long time in my company (>6 months)... now it's open so if you're looking for some modern caching mechanism for PHP I invite everyone to use it... best of all, it works on windows and linux, has snapshots and rebalance and doesn't require to install anything for PHP, just the server. https://github.com/slawomir-pryczek/FlatDB/releases Why you should use it? - Much better Garbage Collection mechanism, that isn't skipping items with short TTLs like LRU, so memory is not occupied with garbage data even if item TTLs differ a lot. - It doesn't require PHP extension (memcached ext. is buggy in PHP7) - It allows modern caching patterns that prevent stampeding herd - Very easy to use atomic operations based on generating value using callbacks - Optimal caching and network settings (NODELAY, allow to set compression by ip range) Let me know if you have any feedback! -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
