On Wednesday, October 18, 2023 at 3:12:29 AM UTC+13 Marc-Antoine Ruel wrote:
Sorry I wasn't clear. The static libraries are in a subdirectory because the user should not care and these libraries are effectively third party code. Actually, you were totally clear. Sorry if my comment didn't make sense. I get that the user doesn't need to worry about the third_party directory with the header and pre-built static libraries. And I saw how you arrange to pick up the right ones in the code. My point was that from previous experience, "go get" would prune away directories that do not contain go source code, after it performs the clone from the remote, into the module cache. So I would have expected that when someone uses your library as a dependency, it would have omitted the needed third_party directory, for not having any Go source files in it. But when I tested it just now, it does seem to retain the subdirectory and link correctly. Maybe this was improved as of recent Go releases. In earlier versions of the Go tool, I would have either included a dummy Go source file in the subdirectory, or just kept the non-go files in the root with the rest of the Go source. Maybe someone else can clarify if this workflow has been improved? This declares the generic code available everywhere: https://github.com/periph/d2xx/blob/main/d2xx.go One set of files declare the import that is OS and CPU arch appropriate, e.g. https://github.com/periph/d2xx/blob/main/d2xx_linux_amd64.go Then another set of two files defines the OS specific code. In practice, POSIX and windows; https://github.com/periph/d2xx/blob/main/d2xx_posix.go https://github.com/periph/d2xx/blob/main/d2xx_windows.go A CGO_ENABLED=0 build gets instead redirected to https://github.com/periph/d2xx/blob/main/d2xx_posix_no_cgo.go but not on windows because this package uses dynamic linking on windows. Adapt as appropriate in your use case. I hope this helps. M-A Le lun. 16 oct. 2023, 21 h 13, Justin Israel <justin...@gmail.com> a écrit : On Tuesday, October 17, 2023 at 10:40:33 AM UTC+13 Marc-Antoine Ruel wrote: I second Richard's suggestion. I used the same trick for https://github.com/periph/d2xx. This git repository contains a minimalist shim for the .a libraries and nothing else. It's designed to compile on any platform. It falls back to a scaffolding if the corresponding .a library is not present for the OS-arch combination or if CGO is disabled. It makes testing easier. Then a proper package under https://github.com/periph/host/tree/main/ftdi exposes a higher level abstraction for the user. With only headers and static libs in the thirdparty directory, is this package "go-gettable"? Will go make the subdirectory available in that case? It usually ignores if there is no Go source files. M-A Le lun. 16 oct. 2023, à 14 h 48, Mike Schinkel <mi...@newclarity.net> a écrit : Hi Jan, I'm going to go out on a limb here and suggest looking at using Go's `embed` feature? https://blog.jetbrains.com/go/2021/06/09/how-to-use-go-embed-in-go-1-16/ I have not tried it with C libraries so there may be numerous reasons why it may not work for your needs. For example, I do not know what is required to *"install"* the libraries so that might be your sticking point that embed would not address. But if building on the user's machine is the real problem and you can distribute pre-build binaries then this might work for you. Basically Go reads a `//go:embed` comment, converts your files into Go source code, compiles that code, and then provides a package that allows you to write those files to disk from an your Go app before you need to load the libraries. Maybe this will work for you? If yes, would love to hear back how it worked out. -Mike On Monday, October 16, 2023 at 8:40:54 AM UTC-4 Bruno Albuquerque wrote: I guess you switched things here. Shared libraries (.so) need to be available at engine. Static libraries (.a) are bagged into the binary. -Bruno On Sun, Oct 15, 2023, 3:55 PM Jan <pfe...@gmail.com> wrote: Thanks Tamas, this is useful information. One of my libraries is using a `.a` library -- as opposed to `.so`, which is another level of complexity, since the library has to be available in runtime, not only in compile time -- and I'm going to follow your "incantation" suggestion. On Sunday, October 15, 2023 at 7:55:55 AM UTC+2 Tamás Gulácsi wrote: Neither I see a convenient way. BUT if you add a .go file into the directories where your precompiled libraries live, then "go get" will download them too (and only those dirs that have .go files in it). So your next mission is to prepare the right #cgo CFLAGS LDFLAGS incantations to use those libraries. Jan a következőt írta (2023. október 14., szombat, 8:37:48 UTC+2): Thanks Tamás, I may not be understanding correctly, but after taking a look at github.com/godror/godror, and the odpi subdirectory, I see it is including all the `.c` files on the fly <https://github.com/godror/godror/blob/main/odpi/embed/dpi.c>. A couple of reasons immediately come to mind, that make this not a generically valid option: * ODPI library is all C code (see src <https://github.com/godror/godror/tree/main/odpi/src>) so it works including in Go: my dependencies are C++/Rust code, for which I write a small C wrapper (or for Rust just `extern "C"`). Also architecture dependent compilation is different in C++/C/Rust code ... * The C++ libraries I'm including have sub-dependencies of themselves (one of which is llvm). It uses Bazel to organize it, and to manually move all the required C++ files to a directory would be years of work :) Plus would require me to slave to maintain things in sync. * The C++ libraries take hours to compile ... I don't want to impose this to users of my libraries. I think the only way to work this out is distributing the pre-compiled C++/Rust libraries, so the Go simply refer to them (and we get the fast compilation times from Go). But then, how to best distribute them in an automatic fashion, so that users don't need to one by one figure out how to install them (and clean up them later if they are no longer using...) ? Or maybe there is another convenient way I'm not seeing ? On Thursday, October 12, 2023 at 6:39:34 PM UTC+2 Tamás Gulácsi wrote: Can't you build (make go build for you) those libraries? For example, see github.com/godror/godror just includes the sources of that third party library in an "odpi" subdir, and with ``` /* #cgo CFLAGS: -I./odpi/include -I./odpi/src -I./odpi/embed #include "dpi.c" */ import "C" ``` it is compiled automatically. Caveat: for "go get" to download a directory, it must include a sth.go file ("require.go" in the odpi/* subdirs). But it may work that your precompiled libs in a subdir, with a mock .go file gets downloaded. But how will it found by your lib/app ? Tamás Jan a következőt írta (2023. október 12., csütörtök, 8:14:41 UTC+2): Thanks Richard. Indeed, as you pointed out the downside is the bloating of the git repo, but it makes sense. But does the user have to manually clone the repository and move the `.a` file to, let's say, /usr/local/lib, or does a simple `go get` magically does everything ? On Thursday, October 12, 2023 at 2:29:21 AM UTC+2 Richard Wilkes wrote: It isn't a great solution, but I currently include the built library files and necessary headers in the git repo alongside the Go code. You can see an example here https://github.com/richardwilkes/unison/tree/main/internal/skia where I include the skia library I built for use in my UI framework, unison. The main downside of this is bloating the git repo with the binary .a and .dll files... but I've not found a better way to handle it. glfw, which unison also depends upon, does something very similar. - Rich On Wednesday, October 11, 2023 at 2:58:23 AM UTC-7 Jan wrote: hi, I'm developing a couple of ML framework/libraries for Go that depend on C/C++ code. Once C-libraries dependencies are installed, the CGO integration work great. Now, for end-users that just want to use these Go libraries, having to figure out how to manually build and install those C/C++/Rust dependencies is a hassle -- sadly each one with a different type of build system. I offer pre-built `.tgz` files (for a limited set of architectures) with the required `.h` and `.a/.so` files along the releases, which facilitates. But it's still a hassle to install -- and no auto-uninstall if someone is no longer using the Go module. I was wondering if others have figured out how to handle this in a nicer way. Is there a recommended way to distribute prebuilt CGO dependencies ? I like how Python wheels (`.whl` files) solve the issue, by including the pre-built libraries in a sub-directory of the python library installation. I was hoping there would be a similar way (maybe with a separate tool) to integrate with `go.mod`. Any pointers, thoughts or suggestions are very appreciated. Thanks! Jan ps: While searching I saw similar questions, but none that exactly answered this aspect of distribution. Just in case, apologies if it's a duplicate question. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a00b3a7b-200d-44ce-8060-913d9ac3c6cen%40googlegroups.com <https://groups.google.com/d/msgid/golang-nuts/a00b3a7b-200d-44ce-8060-913d9ac3c6cen%40googlegroups.com?utm_medium=email&utm_source=footer> . -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ca2d8979-f741-4b81-9099-8672bc5e2312n%40googlegroups.com <https://groups.google.com/d/msgid/golang-nuts/ca2d8979-f741-4b81-9099-8672bc5e2312n%40googlegroups.com?utm_medium=email&utm_source=footer> . -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/146f101d-8ab4-40ec-a03d-ea23aa84bf74n%40googlegroups.com <https://groups.google.com/d/msgid/golang-nuts/146f101d-8ab4-40ec-a03d-ea23aa84bf74n%40googlegroups.com?utm_medium=email&utm_source=footer> . -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ff63d201-724b-45db-a942-851449bfa446n%40googlegroups.com.
