Re: [go-nuts] How to generalize parameters for a function

[Kurtis Rader]

On Tue, Mar 14, 2023 at 7:07 PM Frank Jüdes  wrote:

> Thank you very much Ian!
> I am using a pointer to the map because i want to pass the map by
> reference, not by value.
> Those maps can become pretty large and are being handed down through a
> couple of function calls and i don't want them to be copied over and over
> again.
>

Maps are a special-case. You can't pass them "by value" in the sense you
mean because a "map" value is a very tiny structure that contains a pointer
to the actual map. Possibly the oldest public discussion of the fact maps
are reference type is this blog post: https://go.dev/blog/maps. Using a
pointer does, in fact, avoid copying that very small data structure but is
not necessary to keep from copying the entire map.


> I read the document you referenced, but am not able to understand…
> So i can use interface{} or any just to generalize simple types?
> Why is the compiler accepting then a p_Values map[string]interface{} as a
> parameter of a function? - What would be a compatible data-type to pass as
> this parameter?
>

The compiler accepts a type of "map[string]interface{}" because that is
well defined and obviously useful, Functions, such as those in the fmt
package, would not be useful if a function could not deal with the
"interface{}" type directly or indirectly. Your confusion is very common
and arises from the misconception that the Go map implementation will
magically convert map values from a concrete type (e.g., "int") to the
"any" type if that is what is needed in a particular context. One reason it
doesn't do so is the same reason it doesn't auto-magically convert any
other map value type; e.g., "int" to "float".

-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD8eKMBwJgQ_Qvn163mZz3%3D4tZ18Lpugp5fO_M1hSj68RA%40mail.gmail.com.

[go-nuts] Re: what the asyncPreempt happens in flamegraph mean?

[Landon Cooper]

Hi Cheng, I'm not sure I understand your question, but here's what little I 
know about the preempt function.

runtime.asyncPreempt is used to implement asynchronous preemption. 
Basically, the go runtime scheduler occasionally suspends the execution of 
a goroutine to check if it has been running too long. If it has, it 
"preempts" the routine and passes control to another goroutine. This 
prevents long running goroutines from stealing all the resources from other 
routines.

The following information is probably considered "dangerous" because it 
depends on low level knowledge that is probably going to change in the 
future..

There is a function in the runtime package called GOMAXPROCS which gives 
you some control over the number of system threads used by the go 
scheduler. You might be able to get a tiny bit of control over how often 
the preempt function is called by changing this value.

There's also a LockOSThread function which you can call to lock the routine 
to a thread and prevent the current goroutine from being preempted (the OS 
may still preempt the thread). It's counterpart, UnlockOSThread unlocks the 
routine from the thread so that it can again be used by the scheduler to 
run different goroutines.
On Tuesday, March 14, 2023 at 8:18:34 PM UTC-6 cheng dong wrote:

> here is the go env
>
> `
> GO111MODULE="on"
> GOARCH="amd64"
> GOBIN=""
> GOCACHE="/home/dongcheng/.cache/go-build"
> GOENV="/home/dongcheng/.config/go/env"
> GOEXE=""
> GOEXPERIMENT=""
> GOFLAGS=""
> GOHOSTARCH="amd64"
> GOHOSTOS="linux"
> GOINSECURE=""
> GONOPROXY=""
> GONOSUMDB=""
> GOOS="linux"
> GOPRIVATE=""
> GOPROXY="https://goproxy.cn,direct;
> GOROOT="/usr/local/go1.18"
> GOSUMDB="sum.golang.org"
> GOTMPDIR=""
> GOTOOLDIR="/usr/local/go1.18/pkg/tool/linux_amd64"
> GOVCS=""
> GOVERSION="go1.18.1"
> GCCGO="gccgo"
> GOAMD64="v1"
> AR="ar"
> CC="gcc"
> CXX="g++"
> CGO_ENABLED="1"
> GOWORK=""
> CGO_CFLAGS="-g -O2"
> CGO_CPPFLAGS=""
> CGO_CXXFLAGS="-g -O2"
> CGO_FFLAGS="-g -O2"
> CGO_LDFLAGS="-g -O2"
> PKG_CONFIG="pkg-config"
> GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 
> -fdebug-prefix-map=/tmp/go-build3091739208=/tmp/go-build 
> -gno-record-gcc-switches"
> `
>
> attachment is the profile data
> On Wednesday, March 15, 2023 at 10:11:40 AM UTC+8 cheng dong wrote:
>
>> my code run some logic every second in a Tick function, sometimes i could 
>> be very slow(like 5x time), i profile the process and found most of time is 
>> doing runtime.asyncPreempt. however in "source view", these asyncPreempt 
>> disappeared.
>>
>> [image: profile315.jpg]
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/96317d77-d01d-48c8-9a2a-8af472750396n%40googlegroups.com.

[go-nuts] Re: what the asyncPreempt happens in flamegraph mean?

[cheng dong]

here is the go env

`
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dongcheng/.cache/go-build"
GOENV="/home/dongcheng/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct;
GOROOT="/usr/local/go1.18"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go1.18/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 
-fdebug-prefix-map=/tmp/go-build3091739208=/tmp/go-build 
-gno-record-gcc-switches"
`

attachment is the profile data
On Wednesday, March 15, 2023 at 10:11:40 AM UTC+8 cheng dong wrote:

> my code run some logic every second in a Tick function, sometimes i could 
> be very slow(like 5x time), i profile the process and found most of time is 
> doing runtime.asyncPreempt. however in "source view", these asyncPreempt 
> disappeared.
>
> [image: profile315.jpg]
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1e135467-0901-4c3c-99a0-fb7949e140f7n%40googlegroups.com.


preempt.out
Description: GNU Zip compressed data

Re: [go-nuts] How to generalize parameters for a function

[Frank Jüdes]

Thank you very much Ian!
I am using a pointer to the map because i want to pass the map by 
reference, not by value. 
Those maps can become pretty large and are being handed down through a 
couple of function calls and i don't want them to be copied over and over 
again.

I read the document you referenced, but am not able to understand…
So i can use interface{} or any just to generalize simple types? 
Why is the compiler accepting then a p_Values map[string]interface{} as a 
parameter of a function? - What would be a compatible data-type to pass as 
this parameter?

Thank you very much in advance for your help.
Best regards from Charleston (WV),
 Frank/2


Ian Lance Taylor schrieb am Dienstag, 14. März 2023 um 21:40:27 UTC-4:

> On Tue, Mar 14, 2023 at 6:27 PM Frank Jüdes  wrote:
> >
> > i still somewhat new with Go and need a little tutoring: I have to 
> create Excel files, using the excelize package and wrote a little function 
> to store data from a map[string]int64 into a column within the worksheet:
> >
> > // 
> -
> > // Store a slice of int64 data items into the cells of a column
> > // 
> -
> > func SetColumnValuesInt64(xf *excelize.File,
> > p_SheetName string,
> > p_StartRow int,
> > p_Column rune,
> > p_CellStyle *excelize.Style,
> > p_Keys *UTL_StringSlice.T_StringSlice,
> > p_Values *map[string]int64) {
> > Row := p_StartRow
> > StyleID,_ := xf.NewStyle(p_CellStyle)
> > for _,Key := range(*p_Keys) {
> > CellAddress := fmt.Sprintf("%c%d",p_Column,Row)
> > xf.SetCellValue(p_SheetName,CellAddress,(*p_Values)[Key])
> > xf.SetCellStyle(p_SheetName,CellAddress,CellAddress,StyleID)
> > Row++
> > } // END for
> > } // END SetColumnValuesInt64
> >
> > Basically the function iterates through a slice of strings, containing 
> the key-values for the data-map and then calls the SetCellValue function 
> from the excelize package to store the number into the cell.
> > Now i have another set of data, stored into a map[string]float64 …
> > So i duplicated the above function, just replacing the red line with
> > p_Values *map[string]float64) {
> >
> > Basically it is the same code, just the parameter-type is different. I 
> looked into the definition of the function SetCellValue and saw that it is 
> using the empty interface as parameter for the value, so i tried to define 
> my function with
> > p_Values *map[string]interface{}) {
> >
> > The function compiles fine, but if i try to use it with a 
> *map[string]int64 as a parameter, the compiler objectst with the message 
> »cannot use Amounts (variable of type *map[string]int64) as type 
> *map[string]interface{} in argument to ExcelFormats.SetColumnValues«
> >
> > Can somebody please give me hint what i am doing wrong?
>
> First I'll note that it's quite unusual to use a pointer to a map.
> Maps are reference types. If you pass a map to a function, and that
> function adds values to the map, the caller will also see the new map
> entries.
>
> That said, the error message seems clear enough: a function that
> expects a pointer to map[string]any can't take an argument that is a
> pointer to map[string]int64. The types any and int64 are different
> types. You can't interchange them. See
> https://go.dev/doc/faq#covariant_types for a related issue.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/48a30be4-9229-4c6c-9503-12cbaa90c35an%40googlegroups.com.

[go-nuts] Re: How to generalize parameters for a function

[Landon Cooper]

Hi Frank, the issue is that the compiler won't accept a map[string]float64 
as a map[string]interface{} because they are defined as different types.
It probably feels to you as though the code would work okay because you 
expect to take things out of the map, check their type and then use them 
appropriately.
However, the compiler doesn't attempt to check that's really what you're 
doing..

Instead, the compiler has to assume your code might try to use the map to 
place things into it.
If you ask for a map[string]interface{}, are given a map[string]float64, 
and then try to put tulips into it, the code will fail.

This is an example of type variance, for which Ian has linked a much nicer 
and more concise answer than I could hope to type out.
On Tuesday, March 14, 2023 at 7:27:05 PM UTC-6 Frank Jüdes wrote:

> Hi Friends,
>
> i still somewhat new with Go and need a little tutoring: I have to create 
> Excel files, using the *excelize* package and wrote a little function to 
> store data from a map[string]int64 into a column within the worksheet:
>
> // 
> -
> // Store a slice of int64 data items into the cells of a column
> // 
> -
> func SetColumnValuesInt64(xf *excelize.File,
>   p_SheetName string,
>   p_StartRow int, 
>   p_Column rune, 
>   p_CellStyle *excelize.Style,
>   p_Keys *UTL_StringSlice.T_StringSlice, 
>   p_Values *map[string]int64) {
>Row := p_StartRow
>StyleID,_ := xf.NewStyle(p_CellStyle)
>for _,Key := range(*p_Keys) {
>  CellAddress := fmt.Sprintf("%c%d",p_Column,Row)
>  xf.SetCellValue(p_SheetName,CellAddress,(*p_Values)[Key])
>  xf.SetCellStyle(p_SheetName,CellAddress,CellAddress,StyleID)
> Row++
>} // END for 
> } // END SetColumnValuesInt64
>
> Basically the function iterates through a slice of strings, containing the 
> key-values for the data-map and then calls the *SetCellValue* function 
> from the *excelize* package to store the number into the cell.
> Now i have another set of data, stored into a map[string]*float64 …*
> So i duplicated the above function, just replacing the red line with
>   p_Values *map[string]float64) {
>
> Basically it is the same code, just the parameter-type is different. I 
> looked into the definition of the function *SetCellValue  *and saw that 
> it is using the empty interface as parameter for the value, so i tried to 
> define my function with
>   p_Values *map[string]interface{}) {
>
> The function compiles fine, but if i try to use it with a 
> *map[string]int64 as a parameter, the compiler objectst with the message 
> »*cannot 
> use Amounts (variable of type *map[string]int64) as type 
> *map[string]interface{} in argument to ExcelFormats.SetColumnValues*«
>
> Can somebody please give me hint what i am doing wrong?
>
> Thank you very much in advance for your help.
>
> Best regards from Charleston (WV),
>  Frank/2
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/715e8be0-0e36-41e5-8966-3c64c6233031n%40googlegroups.com.

Re: [go-nuts] How to generalize parameters for a function

[Ian Lance Taylor]

On Tue, Mar 14, 2023 at 6:27 PM Frank Jüdes  wrote:
>
> i still somewhat new with Go and need a little tutoring: I have to create 
> Excel files, using the excelize package and wrote a little function to store 
> data from a map[string]int64 into a column within the worksheet:
>
> // 
> -
> // Store a slice of int64 data items into the cells of a column
> // 
> -
> func SetColumnValuesInt64(xf *excelize.File,
>   p_SheetName string,
>   p_StartRow int,
>   p_Column rune,
>   p_CellStyle *excelize.Style,
>   p_Keys *UTL_StringSlice.T_StringSlice,
>   p_Values *map[string]int64) {
>Row := p_StartRow
>StyleID,_ := xf.NewStyle(p_CellStyle)
>for _,Key := range(*p_Keys) {
>  CellAddress := fmt.Sprintf("%c%d",p_Column,Row)
>  xf.SetCellValue(p_SheetName,CellAddress,(*p_Values)[Key])
>  xf.SetCellStyle(p_SheetName,CellAddress,CellAddress,StyleID)
> Row++
>} // END for
> } // END SetColumnValuesInt64
>
> Basically the function iterates through a slice of strings, containing the 
> key-values for the data-map and then calls the SetCellValue function from the 
> excelize package to store the number into the cell.
> Now i have another set of data, stored into a map[string]float64 …
> So i duplicated the above function, just replacing the red line with
>   p_Values *map[string]float64) {
>
> Basically it is the same code, just the parameter-type is different. I looked 
> into the definition of the function SetCellValue  and saw that it is using 
> the empty interface as parameter for the value, so i tried to define my 
> function with
>   p_Values *map[string]interface{}) {
>
> The function compiles fine, but if i try to use it with a *map[string]int64 
> as a parameter, the compiler objectst with the message »cannot use Amounts 
> (variable of type *map[string]int64) as type *map[string]interface{} in 
> argument to ExcelFormats.SetColumnValues«
>
> Can somebody please give me hint what i am doing wrong?

First I'll note that it's quite unusual to use a pointer to a map.
Maps are reference types.  If you pass a map to a function, and that
function adds values to the map, the caller will also see the new map
entries.

That said, the error message seems clear enough: a function that
expects a pointer to map[string]any can't take an argument that is a
pointer to map[string]int64.  The types any and int64 are different
types.  You can't interchange them.  See
https://go.dev/doc/faq#covariant_types for a related issue.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVnv5VE%3DF-Hdd7vMPUzQHqUDJNPtD_SDK2TSL3q-hcDcg%40mail.gmail.com.

[go-nuts] How to generalize parameters for a function

[Frank Jüdes]

Hi Friends,

i still somewhat new with Go and need a little tutoring: I have to create 
Excel files, using the *excelize* package and wrote a little function to 
store data from a map[string]int64 into a column within the worksheet:

// 
-
// Store a slice of int64 data items into the cells of a column
// 
-
func SetColumnValuesInt64(xf *excelize.File,
  p_SheetName string,
  p_StartRow int, 
  p_Column rune, 
  p_CellStyle *excelize.Style,
  p_Keys *UTL_StringSlice.T_StringSlice, 
  p_Values *map[string]int64) {
   Row := p_StartRow
   StyleID,_ := xf.NewStyle(p_CellStyle)
   for _,Key := range(*p_Keys) {
 CellAddress := fmt.Sprintf("%c%d",p_Column,Row)
 xf.SetCellValue(p_SheetName,CellAddress,(*p_Values)[Key])
 xf.SetCellStyle(p_SheetName,CellAddress,CellAddress,StyleID)
Row++
   } // END for 
} // END SetColumnValuesInt64

Basically the function iterates through a slice of strings, containing the 
key-values for the data-map and then calls the *SetCellValue* function from 
the *excelize* package to store the number into the cell.
Now i have another set of data, stored into a map[string]*float64 …*
So i duplicated the above function, just replacing the red line with
  p_Values *map[string]float64) {

Basically it is the same code, just the parameter-type is different. I 
looked into the definition of the function *SetCellValue  *and saw that it 
is using the empty interface as parameter for the value, so i tried to 
define my function with
  p_Values *map[string]interface{}) {

The function compiles fine, but if i try to use it with a *map[string]int64 
as a parameter, the compiler objectst with the message »*cannot use Amounts 
(variable of type *map[string]int64) as type *map[string]interface{} in 
argument to ExcelFormats.SetColumnValues*«

Can somebody please give me hint what i am doing wrong?

Thank you very much in advance for your help.

Best regards from Charleston (WV),
 Frank/2

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ff6cbe1d-df62-42c3-aff0-e62b6631f772n%40googlegroups.com.

Re: [go-nuts] Why the extra call to `seed` in this situation?

[Kwaku Biney]

Yeah this actually makes sense

```

On Wednesday, March 15, 2023 at 12:11:11 AM UTC Ian Lance Taylor wrote:

> On Tue, Mar 14, 2023 at 3:47 PM Kwaku Biney  wrote:
> >
> > i was digging in the rand package and came across NewSource(seed) in the 
> randpackage which creates a new source in case you don't want the global 
> source set by the package. It returns a Source interface. For some reason, 
> there's a seed method on the Source as well for which Rand implements, 
> where you pass the seed value. But in the NewSource implementation, there's 
> an rng.Seed() which gets called. Is there a reason why these two seed() 
> calls are necessary and if so, anyone know the difference?
> > https://github.com/golang/go/blob/master/src/math/rand/rand.go (edited)
>
> A Source requires a seed to do anything at all, so NewSource takes a
> seed value. It's possible to change an existing Source to start
> producing a new sequence of random numbers starting from a new seed,
> so Source has a Seed method. Sure, we could force people to create a
> new Source if they want to start a new sequence at a new seed, but
> it's easy enough to not require that.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/774ce47b-871c-4dbb-bb07-44143ad8e18en%40googlegroups.com.

Re: [go-nuts] Why the extra call to `seed` in this situation?

[Ian Lance Taylor]

On Tue, Mar 14, 2023 at 3:47 PM Kwaku Biney  wrote:
>
> i was digging in the rand package and came across NewSource(seed) in the 
> randpackage which creates a new source in case you don't want the global 
> source set by the package. It returns a Source interface. For some reason, 
> there's a seed method on the Source as well for which Rand implements, where 
> you pass the seed value. But in the NewSource implementation, there's an 
> rng.Seed() which gets called. Is there a reason why these two seed() calls 
> are necessary and if so, anyone know the difference?
> https://github.com/golang/go/blob/master/src/math/rand/rand.go (edited)

A Source requires a seed to do anything at all, so NewSource takes a
seed value.  It's possible to change an existing Source to start
producing a new sequence of random numbers starting from a new seed,
so Source has a Seed method.  Sure, we could force people to create a
new Source if they want to start a new sequence at a new seed, but
it's easy enough to not require that.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXGJeDhkb_VjC4pkaUo3tU5d2j4QMBJLRx%2B74nn3hBzzQ%40mail.gmail.com.

[go-nuts] Why the extra call to `seed` in this situation?

[Kwaku Biney]

i was digging in the rand package and came across NewSource(seed) in the 
randpackage which creates a new source in case you don't want the global 
source set by the package. It returns a Source interface. For some reason, 
there's a seed method on the Source as well for which Rand implements, 
where you pass the seed value. But in the NewSource implementation, there's 
an rng.Seed() which gets called. Is there a reason why these two seed() calls 
are necessary and if so, anyone know the difference?
https://github.com/golang/go/blob/master/src/math/rand/rand.go (edited) 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/458b100c-dd01-4521-b391-a80117239f8dn%40googlegroups.com.

Re: [go-nuts] How to encode/decode string to binary in golang

[Brian Candler]

Presumably you are looking at something like 
this: 
https://www.etsi.org/deliver/etsi_ts/129200_129299/129229/13.01.00_60/ts_129229v130100p.pdf



*6.3.10 SIP-Authenticate AVP*



*The SIP-Authenticate AVP is of type OctetString and contains specific 
parts of the data portion of the WWW-Authenticate or Proxy-Authenticate SIP 
headers that are to be present in a SIP response.It shall contain, binary 
encoded, the concatenation of the authentication challenge RAND and the 
token AUTN. See3GPP TS 33.203 [3] for further details about RAND and AUTN. 
The Authentication Information has a fixed length of32 octets; the 16 most 
significant octets shall contain the RAND, the 16 least significant octets 
shall contain the AUTN.*

I believe what it's saying is: in the [textual] SIP WWW-Authenticate 
header, there are some fields RAND and AUTN which are encoded in some 
textual form (hex? base64? Check the details of the auth scheme)

You need to convert those to their underlying binary data, giving you two 
[16]byte blobs, then concatenate them to give 32 bytes of data.

If so, you had roughly the right idea in the first place (except s2 was 
less than 16 bytes).  You could concatenate the hex representations and 
then decode, as you did; or I would be inclined to decode them separately:
https://go.dev/play/p/851IrsExn76

The result is [32]byte, which is 16 bytes from one field and 16 bytes from 
the other.  You just use it as-is for the AVP value; this is the "binary 
encoded" value.  It doesn't mean you should turn it into a string of ASCII 
"1" and "0" characters.

On Tuesday, 14 March 2023 at 11:12:11 UTC Brian Candler wrote:

> On Tuesday, 14 March 2023 at 09:26:05 UTC Van Fury wrote:
>
> Am developing a server (diameter) which will response with an AVP 
> SIP-Authenticate. 
> In the specification
>
> " The SIP-Authenticate AVP is of type OctetString and It shall contain, 
> binary encoded, the concatenation of the authentication challenge RAND and 
> the token AUTN" 
>
>
> I believe "binary encoded" here just means []byte.  Concatenate them using 
> append().
>
>  
>
>
> The RAND and the AUTN in this case are hexadecimal strings s1 and s2.
>
>
> I don't believe either is a "hexadecimal string" in the sense of being 
> encoded using the ASCII symbols "0" to "9" and "a" to "f".  They may 
> conventionally be *displayed* in this format, e.g. for debugging purposes, 
> but internally and on-the-wire I think they are just sequences of bytes.
>
> NOTE: my experience is with RADIUS rather than Diameter, so I *could* be 
> wrong.  I'm just outlining what I *expect* to be the case; I haven't gone 
> reading through Diameter RFCs.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/de06f8ad-b81f-4fc2-bca0-9a356e92f956n%40googlegroups.com.

Re: [go-nuts] How to encode/decode string to binary in golang

[Brian Candler]

On Tuesday, 14 March 2023 at 09:26:05 UTC Van Fury wrote:

Am developing a server (diameter) which will response with an AVP 
SIP-Authenticate. 
In the specification

" The SIP-Authenticate AVP is of type OctetString and It shall contain, 
binary encoded, the concatenation of the authentication challenge RAND and 
the token AUTN" 


I believe "binary encoded" here just means []byte.  Concatenate them using 
append().

 


The RAND and the AUTN in this case are hexadecimal strings s1 and s2.


I don't believe either is a "hexadecimal string" in the sense of being 
encoded using the ASCII symbols "0" to "9" and "a" to "f".  They may 
conventionally be *displayed* in this format, e.g. for debugging purposes, 
but internally and on-the-wire I think they are just sequences of bytes.

NOTE: my experience is with RADIUS rather than Diameter, so I *could* be 
wrong.  I'm just outlining what I *expect* to be the case; I haven't gone 
reading through Diameter RFCs.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1f5df4b2-b060-4fe7-9c97-4eb7c3a27d43n%40googlegroups.com.

Re: [go-nuts] How to encode/decode string to binary in golang

[Van Fury]

Am developing a server (diameter) which will response with an AVP
SIP-Authenticate.
In the specification

" The SIP-Authenticate AVP is of type OctetString and It shall contain,
binary encoded, the concatenation of the authentication challenge RAND and
the token AUTN"
The RAND and the AUTN in this case are hexadecimal strings s1 and s2.

My problem is how to encode s1 and s2 concatenated and then be able to
decode at the server side too.




On Tue, Mar 14, 2023 at 9:13 AM Marcello H  wrote:

> You can also use gob for that.
>
> Here are 2 functions from my library that can help.
>
> import "encoding/gob"
>
> /*
> encodeGob encodes a model to gob bytes.
> */
> func encodeGob(data any) ([]byte, error) {
> var (
> buf bytes.Buffer
> enc = gob.NewEncoder() // Will write to network.
> )
>
> err := enc.Encode(data)
>
> return buf.Bytes(), errors.Wrap(err, "encodeGob")
> }
>
> /*
> decodeGob decodes gob bytes to a model.
> */
> func decodeGob(decVal []byte, data any) (err error) {
> buf := bytes.NewBuffer(decVal)
> dec := gob.NewDecoder(buf)
> err = dec.Decode(data)
>
> return errors.Wrap(err, "decodeGob")
> }
>
> Op dinsdag 14 maart 2023 om 00:06:31 UTC+1 schreef robert engels:
>
>> Am arbitrary byte can be encoded in 2 HEX characters - only a 2x increase
>> in size not 8x.
>>
>> On Mar 13, 2023, at 2:35 PM, Volker Dobler  wrote:
>>
>> On Monday, 13 March 2023 at 17:41:42 UTC+1 Van Fury wrote:
>>
>> Relating to my previous question, I have been reading but it is still not
>> clear to me what raw binary is, how is it different from
>> text formatted binary (fmt.Sprintf("%b", s1s2Byte))?
>>
>> "text formated binary" takes a stream of bytes  and for each bit in
>> this stream output "0" or "1" as letters, i.e. the bytes 48==0x30
>> and 49==0x31. An eightfold blowup in size.
>> Try understanding how _text_ is encoded as bytes.
>>
>> Computers work on streams of bytes.
>> The meaning/interpretation of a single byte can vary:
>> Not all number can be stored in a single byte and
>> text has to be encoded by bytes too.
>> Please forget about "raw binary", there is no such thing.
>> String variables are a stream of bytes and do not need any encoding,
>> especially not something like "raw binary" that doesn't exist.
>> A stream of bytes can be _printed_ (encoded) as decimal, hexadecimal,
>> octal, base64 or even as bit ('0' and '1'). sometimes this is necessary
>> because the underlying protocol cannot transmit possible byte values.
>> When dealing with "binary" streams this cannot happen (it happens e.g.
>> when using JSON to transmit data).
>> You have to make clear what the output/encoding space should be
>> (and why!): There is a inconsistency between wanting "binary" and
>> a textual bit stream (which is not "binary" but text) and hexadecimal
>> encoding (also not binary but text).
>> You manoeuvred yourself in a corner by focusing on the solution
>> instead of the actual problem.
>>
>> V.
>>
>>
>> In my problem above I need to encode s1+s2 to raw binary before sending
>> the result to the server
>> which then decodes the raw binary back to s1+s2.
>>
>>
>>
>> On Mon, Mar 13, 2023 at 5:39 PM Van Fury  wrote:
>>
>> Do you mean encoding should be
>> rawdata := fmt.Print("%016d%s%s", len(s1), s1,s2)
>> or
>> rawdata := fmt.Printf("%016d%s%s", len(s1), s1,s2)
>>
>>
>>
>> On Monday, March 13, 2023 at 4:36:50 PM UTC+2 Alex Howarth wrote:
>>
>> You might be looking for strconv.ParseUint()
>> https://pkg.go.dev/strconv#ParseUint
>>
>> https://go.dev/play/p/qAO9LfLD41D
>>
>> On Mon, 13 Mar 2023 at 07:24, Van Fury  wrote:
>>
>>
>> Sorry I did not frame my question properly but what I would like to do is
>> to
>> encode concatenated s1 and s2 into raw binary and then decode the raw
>> binary
>> back to s1 and s2.
>>
>>
>> On Friday, March 10, 2023 at 11:36:09 PM UTC+2 Alex Howarth wrote:
>>
>> If s1 and s2 are a fixed length then you can just slice up the decoded
>> string based on the lengths. If they are of a variable length, you'll need
>> a separator in the input string to later split on when decoded (s3 := s1 +
>> ":" + s2 etc)?
>>
>> On Fri, 10 Mar 2023 at 10:33, Van Fury  wrote:
>>
>> Hi,
>>
>> I have two hexadecimal string values and would like to concatenate the
>> two strings and
>>
>>1. encode the result to binary
>>2. decode the resulting binary back to hexadecimal string
>>
>> I did the following but I was finding it difficult to decode the result
>> back. I ignore error check in this case.
>>
>> What i did so far:
>>
>> s1 := "1d28ed66824aa2593e1f2a4cf740343f"
>>
>> s2 := "dee2bd5dde763885944bc9d65419"
>>
>> s3 := s1 + s2
>>
>> s1s2Byte, _ := hex.DecodeString(s3)
>>
>> randAutnBin := fmt.Sprintf("%b", s1s2Byte)
>>
>> result:
>>
>> [11101 101000 11101101 1100110 1010 1001010 10100010 1011001 10
>> 1 101010 1001100 0111 100 110100 11 1100 11100010
>> 1001 1011101 1100 1110110 111000 1101 10010100 1001011 11001001
>> 11010110 1010100 11001]
>>
>> I would like to decode 

Re: [go-nuts] How to encode/decode string to binary in golang

[Marcello H]

You can also use gob for that.

Here are 2 functions from my library that can help.

import "encoding/gob"

/*
encodeGob encodes a model to gob bytes.
*/
func encodeGob(data any) ([]byte, error) {
var (
buf bytes.Buffer
enc = gob.NewEncoder() // Will write to network.
)

err := enc.Encode(data)

return buf.Bytes(), errors.Wrap(err, "encodeGob")
}

/*
decodeGob decodes gob bytes to a model.
*/
func decodeGob(decVal []byte, data any) (err error) {
buf := bytes.NewBuffer(decVal)
dec := gob.NewDecoder(buf)
err = dec.Decode(data)

return errors.Wrap(err, "decodeGob")
}

Op dinsdag 14 maart 2023 om 00:06:31 UTC+1 schreef robert engels:

> Am arbitrary byte can be encoded in 2 HEX characters - only a 2x increase 
> in size not 8x.
>
> On Mar 13, 2023, at 2:35 PM, Volker Dobler  wrote:
>
> On Monday, 13 March 2023 at 17:41:42 UTC+1 Van Fury wrote:
>
> Relating to my previous question, I have been reading but it is still not 
> clear to me what raw binary is, how is it different from
> text formatted binary (fmt.Sprintf("%b", s1s2Byte))?
>
> "text formated binary" takes a stream of bytes  and for each bit in
> this stream output "0" or "1" as letters, i.e. the bytes 48==0x30
> and 49==0x31. An eightfold blowup in size.
> Try understanding how _text_ is encoded as bytes.
>  
> Computers work on streams of bytes.
> The meaning/interpretation of a single byte can vary:
> Not all number can be stored in a single byte and
> text has to be encoded by bytes too.
> Please forget about "raw binary", there is no such thing.
> String variables are a stream of bytes and do not need any encoding,
> especially not something like "raw binary" that doesn't exist.
> A stream of bytes can be _printed_ (encoded) as decimal, hexadecimal,
> octal, base64 or even as bit ('0' and '1'). sometimes this is necessary
> because the underlying protocol cannot transmit possible byte values.
> When dealing with "binary" streams this cannot happen (it happens e.g.
> when using JSON to transmit data).
> You have to make clear what the output/encoding space should be
> (and why!): There is a inconsistency between wanting "binary" and
> a textual bit stream (which is not "binary" but text) and hexadecimal
> encoding (also not binary but text).
> You manoeuvred yourself in a corner by focusing on the solution
> instead of the actual problem.
>
> V.
>  
>
> In my problem above I need to encode s1+s2 to raw binary before sending 
> the result to the server
> which then decodes the raw binary back to s1+s2.
>
>
>
> On Mon, Mar 13, 2023 at 5:39 PM Van Fury  wrote:
>
> Do you mean encoding should be
> rawdata := fmt.Print("%016d%s%s", len(s1), s1,s2) 
> or
> rawdata := fmt.Printf("%016d%s%s", len(s1), s1,s2) 
>
>
>
> On Monday, March 13, 2023 at 4:36:50 PM UTC+2 Alex Howarth wrote:
>
> You might be looking for strconv.ParseUint() 
> https://pkg.go.dev/strconv#ParseUint
>
> https://go.dev/play/p/qAO9LfLD41D
>
> On Mon, 13 Mar 2023 at 07:24, Van Fury  wrote:
>
>
> Sorry I did not frame my question properly but what I would like to do is 
> to
> encode concatenated s1 and s2 into raw binary and then decode the raw 
> binary
> back to s1 and s2. 
>
>
> On Friday, March 10, 2023 at 11:36:09 PM UTC+2 Alex Howarth wrote:
>
> If s1 and s2 are a fixed length then you can just slice up the decoded 
> string based on the lengths. If they are of a variable length, you'll need 
> a separator in the input string to later split on when decoded (s3 := s1 + 
> ":" + s2 etc)?
>
> On Fri, 10 Mar 2023 at 10:33, Van Fury  wrote:
>
> Hi,
>
> I have two hexadecimal string values and would like to concatenate the two 
> strings and
>
>1. encode the result to binary
>2. decode the resulting binary back to hexadecimal string
>
> I did the following but I was finding it difficult to decode the result 
> back. I ignore error check in this case.
>
> What i did so far:
>
> s1 := "1d28ed66824aa2593e1f2a4cf740343f" 
>
> s2 := "dee2bd5dde763885944bc9d65419"
>
> s3 := s1 + s2 
>
> s1s2Byte, _ := hex.DecodeString(s3)
>
> randAutnBin := fmt.Sprintf("%b", s1s2Byte)
>
> result:
>
> [11101 101000 11101101 1100110 1010 1001010 10100010 1011001 10 
> 1 101010 1001100 0111 100 110100 11 1100 11100010 
> 1001 1011101 1100 1110110 111000 1101 10010100 1001011 11001001 
> 11010110 1010100 11001]
>
> I would like to decode the binary result back the hexadecimal string to 
> get s1 and s2.
>
> Any help?
>
> Van
>
> -- 
>
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/ad3a981b-cf22-47cd-9fe6-8db83a097b42n%40googlegroups.com
>  
> 
> .
>
>
> -- 
> You received 

Re: [go-nuts] what's the most popular kafka library to use

[Vasiliy Tolstov]

I'm try all kafka libraries and my list:

   1. github.com/twmb/franz-go/kgo - the best, no problems is around 1.5-2
   years
   2. github.com/segmentio/kafka-go (cool but have some errors 1.5 years
   ago, so i broke my production cluster with it)



-- 
Vasiliy Tolstov,
e-mail: v.tols...@selfip.ru


На 13 марта 2023 г., 20:43:05, Eli Lindsey  написали:

> Is there an obvious choice here? what are people using now days (March
> 2023)
>
>
> I don’t think there’s an obvious choice, there’s three or four main libs,
> any of which may be good for different uses.
>
> Some color on confluent-kafka-go - it's a wrapper around librdkafka.
> librdkafka is the main Kafka C lib and underpins many/most of the non-JVM
> language bindings. From that angle, if you’re looking for popularity it
> will surpass sarama. It also tends to have more of the esoteric or
> long-tail features implemented, though I haven’t done a feature comparison
> in awhile and it looks like that’s not relevant to you anyways. However it
> comes with the big caveat/potential pain point of requiring cgo.
>
> segmentio/kafka-go is also worth considering. It’s my main goto if
> avoiding cgo.
>
> -eli
>
> On Mar 13, 2023, at 12:48 PM, ami malimovka 
> wrote:
>
> Currently I'm interested only in consuming/producing.
>
> I'm aware of *confluent-kafka-go *and of *sarama, *and it looks like
> sarama has more github swag, but other than that I have no idea which one
> is better/more popular.
>
> Is there an obvious choice here? what are people using now days (March
> 2023)
>
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/1f3e2106-9254-453c-905e-d51076210fbbn%40googlegroups.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3C38837C-3811-4ABF-A6D8-E349FE947FB9%40siliconsprawl.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CACaajQv%2B%2BHJ-6Yo78oX%2BbNq7ZOt7tUzsZhZVb7XMPzwLxZN52g%40mail.gmail.com.