Re: [RFC] Go (golang) packaging

2013-01-04 Thread Reinhard Tartler
On Thu, Jan 3, 2013 at 12:17 PM, Alastair McKinstry
alastair.mckins...@sceal.ie wrote:
 On 2013-01-03 08:41, Reinhard Tartler wrote:
 On Wed, Jan 2, 2013 at 10:26 PM, Wouter Verhelst wou...@debian.org wrote:
 On Wed, Jan 02, 2013 at 01:05:46PM +0100, Guillem Jover wrote:
 - Private dependencies, as they leak to rdeps. When a library uses
   another library privately this dependency gets linked in directly
   in all other rdeps, when that library stop depending on that
   private dependency, all rdeps need to be rebuilt.
 Strictly speaking, if you're only using static libraries this is not
 really true; once you've compiled something against a static library,
 the static library might change in whatever way it sees fit, the
 compiled binary will continue to work, with or without recompilation.
 Consider this from the application perspective: Say an application
 links against a library libfoo.a. At some point, libfoo decides to
 include compression support, and requires functionality from libz. No
 problem for the library package maintainer; he just adds a
 build-dependency  on libz-dev, and uploads the package. At some point
 the security team needs to update the application and finds the
 package to FTBFS because libz is missing. The solution, of course, is
 now to extend the build-dependencies of the application package.
 However, this is not obvious and in any case more effort than a
 binNMU.

 Yes, there are compile-time dependencies for any static library. We do
 need to track
 these. In practice we already have a mechanism in pkg-config, but this
 is (I believe)
 not properly formalised in Debian.

and generally pretty broken: see e.g. #622931

IIRC, the pkg-config maintainer dislikes static linking and the
situation is that many if not most .pc files in Debian do not fully
declare all dependencies that would be required for static linking.


 In the case you mention, if libfoo now depends on libz, adding a build
 dependency
 on libz-dev fixes the problem with libfoo.so as it will automatically
 pull in libz.so

Yeah, which does not really scale IMO. And doesn't solve the issue
that the application still needs to track the exact version of all
libraries that were used for linking, e.g. using the Built-Using
header.

 However, the packager should _also_ provide a pkg-config file and this
 will have
 a list of the dependencies , so
 LIBS:=` pkg-config --static -libs foo`
 does the right thing, and the updated libfoo-dev package will include
 -lz on the libs line.

 I think we should do the following:

 (1) pkg-config files for libraries, in particular all those that ship
 static libs, to be a
 release goal for jessie.

I would vote against this. It is really not worth the trouble.




-- 
regards,
Reinhard


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caj0ccebgukghydrxwfgn3h6rc+3kmczwxkea8mjru6geewk...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Reinhard Tartler
On Wed, Jan 2, 2013 at 10:26 PM, Wouter Verhelst wou...@debian.org wrote:
 On Wed, Jan 02, 2013 at 01:05:46PM +0100, Guillem Jover wrote:
 - Private dependencies, as they leak to rdeps. When a library uses
   another library privately this dependency gets linked in directly
   in all other rdeps, when that library stop depending on that
   private dependency, all rdeps need to be rebuilt.

 Strictly speaking, if you're only using static libraries this is not
 really true; once you've compiled something against a static library,
 the static library might change in whatever way it sees fit, the
 compiled binary will continue to work, with or without recompilation.

Consider this from the application perspective: Say an application
links against a library libfoo.a. At some point, libfoo decides to
include compression support, and requires functionality from libz. No
problem for the library package maintainer; he just adds a
build-dependency  on libz-dev, and uploads the package. At some point
the security team needs to update the application and finds the
package to FTBFS because libz is missing. The solution, of course, is
now to extend the build-dependencies of the application package.
However, this is not obvious and in any case more effort than a
binNMU.

 This isn't true if you're using a mix of shared and static libraries, of
 course.

mixing shared and static libraries makes the situation no less
complicated, that's true.


-- 
regards,
Reinhard


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAJ0cceZZYy2QcsFKO4uu5B4zm+uz3r7Ou0PAWH==dz5abe4...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Florian Weimer
* Wouter Verhelst:

 Strictly speaking, if you're only using static libraries this is not
 really true; once you've compiled something against a static library,
 the static library might change in whatever way it sees fit, the
 compiled binary will continue to work, with or without recompilation. 

My main worry is that, for example, a fix in another, otherwise
unrelated dependency prompts a rebuild, and this picks up behavioral
changes which haven't been visible before, but lingering in the static
library.  Essentially, we end up with non-reproducible builds.

The way we currently do QA, it is important that we do not release
many packages with statically linked copies of outdated libraries.  We
wouldn't have much chance spotting the impact of such lingering
changes once they are materializing.  Non-reproducible builds are also
a software freedom issue (practically speaking).


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87a9sqzfbe@mid.deneb.enyo.de



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Michael Stapelberg
Hi Reinhard,

Reinhard Tartler siret...@gmail.com writes:
 Consider this from the application perspective: Say an application
 links against a library libfoo.a. At some point, libfoo decides to
 include compression support, and requires functionality from libz. No
 problem for the library package maintainer; he just adds a
 build-dependency  on libz-dev, and uploads the package. At some point
 the security team needs to update the application and finds the
 package to FTBFS because libz is missing. The solution, of course, is
 now to extend the build-dependencies of the application package.
 However, this is not obvious and in any case more effort than a
 binNMU.
Thanks for your example. It occurs to me that the following way deals
with the problem:

The hypothetical “foo” package is packaged as “golang-foo-dev”.
The hypothetical application is packaged as “barapp”, which
Build-Depends on “golang-foo-dev”.

When golang-foo-dev starts depending on additional libraries, say
“golang-zlib-dev”, it not only adds that library to its Build-Depends,
but also to its Depends. This is okay because you cannot use
“golang-foo-dev” without having “golang-zlib-dev” installed (as you
explained), and normal users don’t get an unnecessary dependency because
“barapp” doesn’t Depend (only Build-Depends) on either of these
libraries anyway.

I hope my explanation was clear, the tl;dr is “static golang libraries
should mirror Build-Depends into Depends”. What do you think?

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6r4m2ldgo@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Alastair McKinstry
On 2013-01-03 08:41, Reinhard Tartler wrote:
 On Wed, Jan 2, 2013 at 10:26 PM, Wouter Verhelst wou...@debian.org wrote:
 On Wed, Jan 02, 2013 at 01:05:46PM +0100, Guillem Jover wrote:
 - Private dependencies, as they leak to rdeps. When a library uses
   another library privately this dependency gets linked in directly
   in all other rdeps, when that library stop depending on that
   private dependency, all rdeps need to be rebuilt.
 Strictly speaking, if you're only using static libraries this is not
 really true; once you've compiled something against a static library,
 the static library might change in whatever way it sees fit, the
 compiled binary will continue to work, with or without recompilation.
 Consider this from the application perspective: Say an application
 links against a library libfoo.a. At some point, libfoo decides to
 include compression support, and requires functionality from libz. No
 problem for the library package maintainer; he just adds a
 build-dependency  on libz-dev, and uploads the package. At some point
 the security team needs to update the application and finds the
 package to FTBFS because libz is missing. The solution, of course, is
 now to extend the build-dependencies of the application package.
 However, this is not obvious and in any case more effort than a
 binNMU.

Yes, there are compile-time dependencies for any static library. We do
need to track
these. In practice we already have a mechanism in pkg-config, but this
is (I believe)
not properly formalised in Debian.

In the case you mention, if libfoo now depends on libz, adding a build
dependency
on libz-dev fixes the problem with libfoo.so as it will automatically
pull in libz.so
However, the packager should _also_ provide a pkg-config file and this
will have
a list of the dependencies , so
LIBS:=` pkg-config --static -libs foo`
does the right thing, and the updated libfoo-dev package will include
-lz on the libs line.

I think we should do the following:

(1) pkg-config files for libraries, in particular all those that ship
static libs, to be a
release goal for jessie.

It would be useful / interesting if pkg-config information could be used
to generate dependencies
for the libfoo-dev package: ie. if libz is _really_ needed for libfoo,
then libfoo-dev should depend
on libz-dev, to provide libz.a. But this is harder than it looks.

Consider the case of netcdf. This ships a libnetcdf.a static library,
and this may be used in scientific
codes for optimisation as well as reliability. But in modern terms,
libnetcdf, which reads netcdf files,  may also accept a URL as well as a
filename. In this case, it uses libcurl to retrieve the file, but pulls
in a huge trace of packages, from gnutls, ssl , pam packages, psql, msql
, etc in order to
get authentication.  In practice this means I might build my scientific
binary with
libnetcdf.a static and dynamically link libcurl.so (which may never be
called in my
actual workloads) to avoid pulling in the whole distro.

So recursively pulling in all potentially needed static libs is not an
option, but examining all
binaries to find what static libs were used to generate it might be
problematic.
 

 This isn't true if you're using a mix of shared and static libraries, of
 course.
 mixing shared and static libraries makes the situation no less
 complicated, that's true.



regards
Alastair

-- 
Alastair McKinstry  , alast...@sceal.ie , mckins...@debian.org
http://blog.sceal.ie

Anyone who believes exponential growth can go on forever in a finite world
is either a madman or an economist - Kenneth Boulter, Economist.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/50e568b3.6070...@sceal.ie



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Michael Stapelberg
Hi Florian,

Florian Weimer f...@deneb.enyo.de writes:
 My main worry is that, for example, a fix in another, otherwise
 unrelated dependency prompts a rebuild, and this picks up behavioral
 changes which haven't been visible before, but lingering in the static
 library.  Essentially, we end up with non-reproducible builds.
Could you provide an example please? I don’t understand how this is
different with static linking than with dynamic linking yet.

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6obh6ld9b@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Florian Weimer
* Michael Stapelberg:

 Florian Weimer f...@deneb.enyo.de writes:
 My main worry is that, for example, a fix in another, otherwise
 unrelated dependency prompts a rebuild, and this picks up behavioral
 changes which haven't been visible before, but lingering in the static
 library.  Essentially, we end up with non-reproducible builds.

 Could you provide an example please? I don’t understand how this is
 different with static linking than with dynamic linking yet.

With dynamic linking, you pick up the behavior change along with
apt-get upgrade, so I expect that we get much more testing of the
combination during development.

Technically, you are correct that we generally do not do purely static
linking—preprocessor macros can change, and so can the value of enum
constants.  And with C++, there are many, many more language features
which escape dynamic linking.  But I think the lingering change
phenomenon is still a valid concern with static linking.

(One approach which doesn't seem to have been discussed so far is
compile-on-installation, à la common-lisp-controller.  Not sure if
this is a good idea.  Linking-on-installation does not appear to be
sufficient.)


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/874niyze8w@mid.deneb.enyo.de



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Sune Vuorela
On 2013-01-03, Alastair McKinstry alastair.mckins...@sceal.ie wrote:
 (1) pkg-config files for libraries, in particular all those that ship
 static libs, to be a
 release goal for jessie.

rather get rid of static libs.

 It would be useful / interesting if pkg-config information could be used
 to generate dependencies

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=534966

/Sune


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/slrnkearrs.me.nos...@sshway.ssh.pusling.com



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Florian Weimer
* Sune Vuorela:

 On 2013-01-03, Alastair McKinstry alastair.mckins...@sceal.ie wrote:
 (1) pkg-config files for libraries, in particular all those that ship
 static libs, to be a
 release goal for jessie.

 rather get rid of static libs.

We might want to extend static libraries with LTO data one day.
(We could even ship performance-criticial programs ready for LTO.)
Of course, this exacerbates the rebuild issue.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87fw2i5vw8@mid.deneb.enyo.de



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Michael Stapelberg
Hi Florian,

Florian Weimer f...@deneb.enyo.de writes:
 Could you provide an example please? I don’t understand how this is
 different with static linking than with dynamic linking yet.

 With dynamic linking, you pick up the behavior change along with
 apt-get upgrade, so I expect that we get much more testing of the
 combination during development.
I’m still not sure I understand you. Could you please provide an
example? Participating in this discussion makes no sense for me until I
understand your point.

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6licalbt8@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Florian Weimer
* Michael Stapelberg:

 Hi Florian,

 Florian Weimer f...@deneb.enyo.de writes:
 Could you provide an example please? I don’t understand how this is
 different with static linking than with dynamic linking yet.

 With dynamic linking, you pick up the behavior change along with
 apt-get upgrade, so I expect that we get much more testing of the
 combination during development.

 I’m still not sure I understand you. Could you please provide an
 example? Participating in this discussion makes no sense for me until I
 understand your point.

Okay.  Suppose you statically link into your application a copy of
OpenSSL and use it to make TLS connections, in a way that causes
OpenSSL to pick the highest supported TLS version.  We upgrade OpenSSL
and add TLSv1.2 support to it (or we remove RC4 support).  It turns
out that the application connects to a server which is broken and
refuses to handshake at version 1.2.  This means that if we recompile
the application, linking against the current OpenSSL version and
incidentally enabling TLSv1.2 support, it ceases to work.

With dynamic linking, we'd see this immediately, as soon as OpenSSL
upgraded (say, because it has entered testing).  With static linking,
we might never see it at all because a recompilation is necessary to
expose the non-working combination.

One might argue that the static case is actually better because it is
more predictable, but our post-release support model is heavily
dependent on minimal changes (because we cannot do full QA
post-release).  Such minimal changes are impossible with static
linking, unless we aggressively recompile and relink reverse
dependencies as soon as possible, so that we have no lingering
outdated copies in a release.

I hope this clarifies things a bit.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87bod65v7x@mid.deneb.enyo.de



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Paul Wise
On Thu, Jan 3, 2013 at 8:04 PM, Florian Weimer wrote:

 One might argue that the static case is actually better because it is
 more predictable, but our post-release support model is heavily
 dependent on minimal changes (because we cannot do full QA
 post-release).  Such minimal changes are impossible with static
 linking, unless we aggressively recompile and relink reverse
 dependencies as soon as possible, so that we have no lingering
 outdated copies in a release.

 I hope this clarifies things a bit.

With Built-Using, we get a way to rebuild packages that embed parts of
other packages:

http://www.debian.org/doc/debian-policy/ch-relationships.html#s-built-using

Not sure if the buildd stuff will automatically schedule rebuilds or
if we will notice due to britney knowing about b-u and blocking
testing migrations, anyone got more details about b-u?

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6Hwa6BiFqJ_NdR=EN4nR5pVsd45gz3Za7=avu0kx_z...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Ansgar Burchardt
Paul Wise p...@debian.org writes:
 With Built-Using, we get a way to rebuild packages that embed parts of
 other packages:

 http://www.debian.org/doc/debian-policy/ch-relationships.html#s-built-using

 Not sure if the buildd stuff will automatically schedule rebuilds or
 if we will notice due to britney knowing about b-u and blocking
 testing migrations, anyone got more details about b-u?

Built-Using only instructs dak to keep additional source packages in the
archive.  This is done to comply with licenses that require the full
source to be available and with the DFSG requirement to provide source
when, for example, linking static libraries, using *-source packages
(e.g. gcc-avr uses the source from gcc-*-source) or embedding binaries
from other packages (e.g. debian-installer).

Nothing else uses it so far, but using it to make sure the references
source packages are not too outdated seems useful. Otherwise any upload
in stable might introduce significant changes (for example gcc-avr might
use a new upstream release of gcc-*-source).

Ansgar


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87licabeaf@eisei.43-1.org



Re: [RFC] Go (golang) packaging

2013-01-03 Thread Wouter Verhelst
On Thu, Jan 03, 2013 at 09:41:15AM +0100, Reinhard Tartler wrote:
 On Wed, Jan 2, 2013 at 10:26 PM, Wouter Verhelst wou...@debian.org wrote:
  On Wed, Jan 02, 2013 at 01:05:46PM +0100, Guillem Jover wrote:
  - Private dependencies, as they leak to rdeps. When a library uses
another library privately this dependency gets linked in directly
in all other rdeps, when that library stop depending on that
private dependency, all rdeps need to be rebuilt.
 
  Strictly speaking, if you're only using static libraries this is not
  really true; once you've compiled something against a static library,
  the static library might change in whatever way it sees fit, the
  compiled binary will continue to work, with or without recompilation.
 
 Consider this from the application perspective: Say an application
 links against a library libfoo.a. At some point, libfoo decides to
 include compression support, and requires functionality from libz.

Ah, yes, but you're talking about /adding/ a dependency. Guillem's
example was about removing one. There's a difference.

-- 
Copyshops should do vouchers. So that next time some bureaucracy requires you
to mail a form in triplicate, you can mail it just once, add a voucher, and
save on postage.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130104010625.ge2...@grep.be



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Bernhard,

Bernhard R. Link brl...@debian.org writes:
 Please also consider codesearch-golang. Especially with longer package
 names not everything can show the full name so the beginning should be
 the more important information.
Given that we already have python-* and ruby-*, I’d find golang-* more
consistent. Which specific tools/places do you have in mind that have to
truncate package names?

 (and of course just “codesearch” for the binaries).

 I assume s/binaries/sources/? And I'd suggest to just not policy the
No, I really meant binaries, as in “cgrep”, “cindex” and “csearch” in
this specific case.

 Another question: Have you considered asking for a archive Section for
 those packages? I guess with no special section yet all those packages
 would be section libdevel as they are for static linking only, wouldn't
 they?
As pabs has pointed out, I did that, but the general rule of thumb is
that we want to have lots of packages first and an archive section
second.

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x67gnwngtf@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Shawn,

Shawn shawnland...@gmail.com writes:
 Henceforth when a go program depends on a go library, those go
 libraries are ALWAYS compiled in statically. Static linking causes
 many problems for distributions like Debian, and therefore this
Can you please tell us which specific problems are caused by static
linking for Debian? I’m not aware of any.

I’m somewhat hesitant to say “Let’s just use gccgo in Debian” because
static linking is one of the advantages of Go (IMO, but also a shared
feeling by the community I think) and the gc compiler is fast, which is
another advertised advantage.

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x64nj0ngow@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Vincent Bernat
 ❦  2 janvier 2013 09:15 CET, Michael Stapelberg stapelb...@debian.org :

 Henceforth when a go program depends on a go library, those go
 libraries are ALWAYS compiled in statically. Static linking causes
 many problems for distributions like Debian, and therefore this
 Can you please tell us which specific problems are caused by static
 linking for Debian? I’m not aware of any.

Fixin a bug (including a security bug) in a package means recompiling
all packages depending on this package.
-- 
panic(CPU too expensive - making holiday in the ANDES!);
2.2.16 /usr/src/linux/arch/mips/kernel/traps.c


pgp53DWVMB_Ew.pgp
Description: PGP signature


Re: [RFC] Go (golang) packaging

2013-01-02 Thread Bernhard R. Link
* Michael Stapelberg stapelb...@debian.org [130102 09:13]:
 Given that we already have python-* and ruby-*, I’d find golang-* more
 consistent.

We also have lib*-perl.
Ruby seems to have changed recently from lib*-ruby to ruby-*. (Does
anyone know of the reason that changed? For I only remember all the
reasons for the old naming scheme and see no reasons in favor of the new)

 Which specific tools/places do you have in mind that have to
 truncate package names?

Most GUIs will truncate depending on the size of the window.

  (and of course just “codesearch” for the binaries).
 
  I assume s/binaries/sources/? And I'd suggest to just not policy the
 No, I really meant binaries, as in “cgrep”, “cindex” and “csearch” in
 this specific case.

And what is the name of the binary package those programs will be in?

  Another question: Have you considered asking for a archive Section for
  those packages? I guess with no special section yet all those packages
  would be section libdevel as they are for static linking only, wouldn't
  they?
 As pabs has pointed out, I did that, but the general rule of thumb is
 that we want to have lots of packages first and an archive section
 second.
Too avoid packaging mistakes and to have the wording ready I'd suggest
to already prepare the wording (I guess in the section later will be
everything that is is now in libdevel plus golong toolchain packages,
while everything providing programs to be in their respective section).

Bernhard R. Link


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130102090022.ga5...@client.brlink.eu



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Bernhard,

Bernhard R. Link brl...@debian.org writes:
  (and of course just “codesearch” for the binaries).
 
  I assume s/binaries/sources/? And I'd suggest to just not policy the
 No, I really meant binaries, as in “cgrep”, “cindex” and “csearch” in
 this specific case.

 And what is the name of the binary package those programs will be in?
“codesearch”, as I wrote.

  Another question: Have you considered asking for a archive Section for
  those packages? I guess with no special section yet all those packages
  would be section libdevel as they are for static linking only, wouldn't
  they?
 As pabs has pointed out, I did that, but the general rule of thumb is
 that we want to have lots of packages first and an archive section
 second.
 Too avoid packaging mistakes and to have the wording ready I'd suggest
 to already prepare the wording (I guess in the section later will be
 everything that is is now in libdevel plus golong toolchain packages,
 while everything providing programs to be in their respective section).
Which wording do you mean?

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x61ue4ne3d@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Guillem Jover
On Wed, 2013-01-02 at 09:15:59 +0100, Michael Stapelberg wrote:
 Shawn shawnland...@gmail.com writes:
  Henceforth when a go program depends on a go library, those go
  libraries are ALWAYS compiled in statically. Static linking causes
  many problems for distributions like Debian, and therefore this

 Can you please tell us which specific problems are caused by static
 linking for Debian? I’m not aware of any.

Well, that's surprising, I would have thought this has been discussed
at lenght in the past, in any case here are several reasons (I'm most
probably forgetting others):

  * Changes in libraries might require recompiling all of their rdeps.
This includes:
- Bug fixes (security or otherwise, as pointed out in the thread).
- Performance improvements.
- Private dependencies, as they leak to rdeps. When a library uses
  another library privately this dependency gets linked in directly
  in all other rdeps, when that library stop depending on that
  private dependency, all rdeps need to be rebuilt.
  * We might need to keep the sources for all library instances that
have been linked into any other package, to comply with license
conditions (see the Built-Using comment on this thread too).
  * Larger dependency chains. Because private libraries need to be
pulled in even by indirect dependencies, the sources and static
libraries need to be available at compile time, this means the
libfoo-dev dependencies and Build-Depends chains are going to
be larger.
  * Produces huge binaries (and packages), which use more memory that
cannot be shared across processes (unlike shared libraries).

The first point alone, make such things a PITA to manage in a
distribution, and are a waste of buildd and people's time who need
to coordinate those things. The rest imply a waste of resources or
added complexity which could be avoided.

 I’m somewhat hesitant to say “Let’s just use gccgo in Debian” because
 static linking is one of the advantages of Go (IMO, but also a shared
 feeling by the community I think) and the gc compiler is fast, which is
 another advertised advantage.

While it's true that static linking can produce varying degrees of faster
binaries depending on the conditions (architecture, amount of libraries,
etc), I don't think it's worth the tradeoff for the general case and I'm
having a hard time seeing a new compiled system language that does not
support shared libraries as a serious contender.

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130102120546.ga31...@gaara.hadrons.org



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Florian Weimer
* Paul Wise:

 On Wed, Jan 2, 2013 at 12:14 AM, Michael Stapelberg wrote:

 Only when not using the “official” compiler (gc), e.g. gccgo has support
 for dynamic linking.

 Then we should use gccgo until the official compiler supports this.

gccgo supports dynamic linking, but Go 1 API changes may break the
ABI.  Struct sizes and and field offsets are compiled into reverse
dependencies, but the Go 1 API promise does not prevent changes in
this area.

There are ways to solve this (sizes and offsets have to be provided by
the dynamic linker), but there is a performance impact.  I think this
has been done for GCJ.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/8738yjsrru@mid.deneb.enyo.de



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Dmitrijs Ledkovs
On 1 January 2013 19:47, Michael Stapelberg stapelb...@debian.org wrote:
 Hi Dmitrijs,

 Dmitrijs Ledkovs x...@debian.org writes:
 What about multiarch?
 I tried to address this on the wiki page, see
 http://wiki.debian.org/MichaelStapelberg/GoPackaging#Multi-Arch.2Fcross-compiling


I was more concerned about future-proof, e.g. in debian we have
distinct armel  armhf and currently with multiarch I can
cross-compile binaries for either target from my amd64 build-machine.
In debian we have far more ports than any other distros. It would be
nice if go could use gnu and/or debian triplets instead of one more
incomplete set for the same thing.

 Essentially, I currently believe that multi-arch does not make sense for
 go, since we are only dealing with static binaries. E.g. to run an i386
 program, you don’t need any additional libraries:


I understand that users running the resulting binaries typically do
not need anything.
My interest in multi-arch support is for fast cross-compilation, i.e.
the same reason pre-compiled libraries are shipped for native
compilation.
I could use chroots, but having all those packages as M-A: same
would be lovely... which brings the argument back into the circle that
upstream doesn't define unique per-arch paths (at least not for arm*)

reference: https://wiki.ubuntu.com/MultiarchCross

Regards,

Dmitrijs.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/canbhlujnv-_h6+buolbwr-ae4+oispqw+q8epojupadwzbj...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Guillem,

Thanks for your explanations, most points make sense to me. Two
questions remain:

Guillem Jover guil...@debian.org writes:
 - Private dependencies, as they leak to rdeps. When a library uses
   another library privately this dependency gets linked in directly
   in all other rdeps, when that library stop depending on that
   private dependency, all rdeps need to be rebuilt.
I’m not entirely sure I get your point. Why exactly do the rdeps need to
be rebuilt? Could you maybe provide an example to clarify?

   * We might need to keep the sources for all library instances that
 have been linked into any other package, to comply with license
 conditions (see the Built-Using comment on this thread too).
How is this any different from a “normal” dependency with a shared
library? Also, what license conditions do you mean specifically?

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6sj6jn1qm@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Sune Vuorela
On 2013-01-02, Michael Stapelberg stapelb...@debian.org wrote:
   * We might need to keep the sources for all library instances that
 have been linked into any other package, to comply with license
 conditions (see the Built-Using comment on this thread too).
 How is this any different from a ???normal??? dependency with a shared
 library? Also, what license conditions do you mean specifically?

A 'normal' dependency with a shared library does not actually include
the actual code from the shared library in the application, only the
interface specification.

A static linking does include the actual code in the application.

Given the library is actually part of the distributed binary, we also
need to provide this code to comply with a) a copyleft license and b)
the dfsg.

/Sune



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/slrnke8eb5.me.nos...@sshway.ssh.pusling.com



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Matthias Klose
Am 02.01.2013 03:54, schrieb Paul Wise:
 On Wed, Jan 2, 2013 at 12:14 AM, Michael Stapelberg wrote:
 
 Only when not using the “official” compiler (gc), e.g. gccgo has support
 for dynamic linking.
 
 Then we should use gccgo until the official compiler supports this.
 
 AFAIK not, but I will add this to the list of questions I want to ask on
 the Go mailing lists.
 
 Ok, please let us know about the results.

Calling gc the official compiler seems to be misleading. gccgo in wheezy
supports the Go API 1.0, and the standard library 1.0.3, supports dynamic
linking, supports multiarch, makes the distinction for armel/armhf, and supports
cross-compilation.  I don't mind having a second compiler in the archives like
clang, but please make sure that you are not bound to one.  How does your
packaging proposal work with gccgo as the compiler? Did you consider making gc
an alternative (like cc, c++)?

  Matthias



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/50e4418e.3040...@debian.org



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Jakub Wilk

* Matthias Klose d...@debian.org, 2013-01-02, 15:17:

I don't mind having a second compiler in the archives


How generous of you!

--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130102143229.ga2...@jwilk.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Matthias,

Matthias Klose d...@debian.org writes:
 Calling gc the official compiler seems to be misleading. gccgo in
That’s why I called it “official”, not official.

 wheezy supports the Go API 1.0, and the standard library 1.0.3,
 supports dynamic linking, supports multiarch, makes the distinction
 for armel/armhf, and supports cross-compilation.  I don't mind having
 a second compiler in the archives like clang, but please make sure
 that you are not bound to one.  How does your packaging proposal work
 with gccgo as the compiler? Did you consider making gc an alternative
 (like cc, c++)?
I am trying to evaluate how this would work. Here is what I tried:

mkdir -p /tmp/golang
export GOPATH=/tmp/golang
go get github.com/mstap/godebiancontrol
go install -v -compiler=gccgo github.com/mstap/godebiancontrol
ls -hlR /tmp/golang/pkg/

This results in the following file structure:

/tmp/golang/pkg/:
total 0
drwxr-xr-x 3 michael staff 60 2013-01-02 15:59 gccgo
drwxr-xr-x 3 michael staff 60 2013-01-02 15:59 linux_amd64

/tmp/golang/pkg/gccgo:
total 0
drwxr-xr-x 3 michael staff 60 2013-01-02 15:59 github.com

/tmp/golang/pkg/gccgo/github.com:
total 0
drwxr-xr-x 2 michael staff 60 2013-01-02 15:59 mstap

/tmp/golang/pkg/gccgo/github.com/mstap:
total 24K
-rw-r--r-- 1 michael staff 23K 2013-01-02 15:59 libgodebiancontrol.a

/tmp/golang/pkg/linux_amd64:
total 0
drwxr-xr-x 3 michael staff 60 2013-01-02 15:59 github.com

/tmp/golang/pkg/linux_amd64/github.com:
total 0
drwxr-xr-x 2 michael staff 60 2013-01-02 15:59 mstap

/tmp/golang/pkg/linux_amd64/github.com/mstap:
total 36K
-rw-r--r-- 1 michael staff 34K 2013-01-02 15:59 godebiancontrol.a

Now, obviously this is not dynamically linked. Neither does it use
different host triplets.

How do I make gccgo use dynamic linking?

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6pq1nmxxg@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread shawnlandden


Michael Stapelberg stapelb...@debian.org wrote:

Hi Matthias,

Matthias Klose d...@debian.org writes:
 Calling gc the official compiler seems to be misleading. gccgo in
That’s why I called it “official”, not official.

 wheezy supports the Go API 1.0, and the standard library 1.0.3,
 supports dynamic linking, supports multiarch, makes the distinction
 for armel/armhf, and supports cross-compilation.  I don't mind having
 a second compiler in the archives like clang, but please make sure
 that you are not bound to one.  How does your packaging proposal work
 with gccgo as the compiler? Did you consider making gc an alternative
 (like cc, c++)?
I am trying to evaluate how this would work. Here is what I tried:

/tmp/golang/pkg/linux_amd64/github.com/mstap:
total 36K
-rw-r--r-- 1 michael staff 34K 2013-01-02 15:59 godebiancontrol.a

Now, obviously this is not dynamically linked. Neither does it use
different host triplets.

How do I make gccgo use dynamic linking?

I am not sure how or if lld works on .a files, but I am pretty sure that this 
binary is linked against libc6 and libgo1. If I invoke gccgo directly that is 
the result.
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/800ddf68-acaf-4189-b5fe-674386639...@email.android.com



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Shawn,

shawnland...@gmail.com writes:
 I am not sure how or if lld works on .a files, but I am pretty sure
 that this binary is linked against libc6 and libgo1. If I invoke gccgo
 directly that is the result.
If by “this binary” you mean godebiancontrol.a, then no:

ldd /tmp/golang/pkg/linux_amd64/github.com/mstap/godebiancontrol.a
not a dynamic executable


-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6mwwrmud3@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread shawnlandden


Michael Stapelberg stapelb...@debian.org wrote:

Hi Shawn,

shawnland...@gmail.com writes:
 I am not sure how or if lld works on .a files, but I am pretty sure
 that this binary is linked against libc6 and libgo1. If I invoke
gccgo
 directly that is the result.
If by “this binary” you mean godebiancontrol.a, then no:

ldd /tmp/golang/pkg/linux_amd64/github.com/mstap/godebiancontrol.a
   not a dynamic executable
.a files are just archives ( ar(1) i believe) of a bunch of .o files. They are 
not linked at all, either dynamically or statically. As gccgo uses libc6 and 
libgo1 and does not compile those every time you use it, you need libc6-dev and 
go Dev files to make any use of these .a files. You have to look at the 
resulting binaries.

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/151dbbd0-ce6e-4289-8e27-21e8dbd03...@email.android.com



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Michael Stapelberg
Hi Shawn,

shawnland...@gmail.com writes:
 [...]
 use of these .a files. You have to look at the resulting binaries.
But in this discussion we are talking about building _library_ packages,
not binaries. I would like to focus on the question of how to build a Go
package (such as github.com/mstap/godebiancontrol) as a shared library,
so that it can later be used in a dynamically linked binary.

-- 
Best regards,
Michael


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6hamzmneb@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-02 Thread Wouter Verhelst
On Wed, Jan 02, 2013 at 01:05:46PM +0100, Guillem Jover wrote:
 - Private dependencies, as they leak to rdeps. When a library uses
   another library privately this dependency gets linked in directly
   in all other rdeps, when that library stop depending on that
   private dependency, all rdeps need to be rebuilt.

Strictly speaking, if you're only using static libraries this is not
really true; once you've compiled something against a static library,
the static library might change in whatever way it sees fit, the
compiled binary will continue to work, with or without recompilation. 

This isn't true if you're using a mix of shared and static libraries, of
course.

[...]
   * Produces huge binaries (and packages), which use more memory that
 cannot be shared across processes (unlike shared libraries).

Also, if a significant body of code exists that uses a particular
library, this will have a significant influence on mirror archive size,
and installed size of Debian systems.

[...]
-- 
Copyshops should do vouchers. So that next time some bureaucracy requires you
to mail a form in triplicate, you can mail it just once, add a voucher, and
save on postage.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130102212637.ga18...@grep.be



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Paul Wise
On Tue, Jan 1, 2013 at 9:44 PM, Michael Stapelberg wrote:

 Any feedback is appreciated. Please read the wiki page before you
 comment, it contains more rationale than this email. Thanks.

Since golang apparently doesn't support dynamic linking, every package
built against a golang library will have to include an appropriate
Built-Using header. You will probably also want a lintian test for
this to autoreject anything without this header.

Upstream should switch to using the GNU triplet instead of linux_arm,
which is clearly not unique enough.

Is it possible to:

Support dynamic linking? That would avoid the whole rebuild-the-world
thing that statically-linked languages have.

Not require the full source code for libraries? C and other libraries
only require the interface headers.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caktje6h1ijosmhd1o23ou9qvhv_jjp3ocvobxjdwxxfqsk1...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Michael Stapelberg
Hi Paul,

 Since golang apparently doesn't support dynamic linking, every package
 built against a golang library will have to include an appropriate
 Built-Using header. You will probably also want a lintian test for
 this to autoreject anything without this header.
Thanks, I was not aware of this yet. Will have a closer look and
integrate it into my Wiki page.

 Upstream should switch to using the GNU triplet instead of linux_arm,
 which is clearly not unique enough.
I think so, too. Not sure if they think that’s doable at this point,
will ask them.

 Is it possible to:
 
 Support dynamic linking? That would avoid the whole rebuild-the-world
 thing that statically-linked languages have.
Only when not using the “official” compiler (gc), e.g. gccgo has support
for dynamic linking.

 Not require the full source code for libraries? C and other libraries
 only require the interface headers.
AFAIK not, but I will add this to the list of questions I want to ask on
the Go mailing lists.

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6fw2kop7h@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Dmitrijs Ledkovs
On 1 January 2013 15:44, Michael Stapelberg
michael+deb...@stapelberg.de wrote:
 Hi,

 I am co-maintainer of the golang package and spent a few hours on trying
 to figure out how to best create Debian packages for libraries and
 programs which are implemented in Go.

 I have documented my thoughts, conclusions and example packaging on:
 http://wiki.debian.org/MichaelStapelberg/GoPackaging

 Essentially, I propose that /usr/lib/gocode is used on Debian to store
 the “src” and “pkg” folders which contain the .go files and compiled
 versions (respectively) of Go libraries.


What about multiarch?

/usr/lib/gocode/ for sources
/usr/lib/$triplet/gocode/ for compiled versions

This assumes that sources are identical across all architectures.

Regards,

Dmitrijs.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/canbhluhh_c35qr3hqkha2apkxhps-9rsqmgqbrqufx8xwt4...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Michael Stapelberg
Hi Dmitrijs,

Dmitrijs Ledkovs x...@debian.org writes:
 What about multiarch?
I tried to address this on the wiki page, see
http://wiki.debian.org/MichaelStapelberg/GoPackaging#Multi-Arch.2Fcross-compiling

Essentially, I currently believe that multi-arch does not make sense for
go, since we are only dealing with static binaries. E.g. to run an i386
program, you don’t need any additional libraries:

$ file helloworld
helloworld: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), 
statically linked, not stripped
$ uname -a
Linux midna 3.5.0 #1 SMP Sun Jul 22 11:15:13 CEST 2012 x86_64 GNU/Linux
$ ./helloworld 
hello

 /usr/lib/gocode/ for sources
 /usr/lib/$triplet/gocode/ for compiled versions
Go’s directory layout already includes the architecture, see
http://wiki.debian.org/MichaelStapelberg/GoPackaging#Resulting_package_file_lists

I have a feeling that you have not actually read the wiki page :-).

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6d2xookwt@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Simon McVittie
On 01/01/13 16:14, Michael Stapelberg wrote:
 Hi Paul,
 Support dynamic linking? That would avoid the whole rebuild-the-world
 thing that statically-linked languages have.
 Only when not using the “official” compiler (gc), e.g. gccgo has support
 for dynamic linking.

Where does gccgo look for Go sources (src)?

Where does gccgo look for Go static libraries (pkg)?

Where does gccgo look for Go dynamic libraries? (Presumably the same
places where gcc looks for C dynamic libraries?)

 Not require the full source code for libraries? C and other libraries
 only require the interface headers.
 AFAIK not, but I will add this to the list of questions I want to ask on
 the Go mailing lists.

If a significant amount of code is going to be written in Go, it would
be useful to understand the classes of source change that do and don't
constitute an API break (well-understood for C, somewhat well-understood
for C++), and the classes of source change that do and don't constitute
an ABI break for gccgo.

Michael additionally wrote:
 I propose to patch
 Go in Debian so that it always appends /usr/lib/gocode when reading
 the list of paths from $GOPATH.

Add /usr/local/lib/[$TUPLE/]gocode as more important than
/usr/lib/[$TUPLE/]gocode, perhaps? That's how /usr/local usually works
in Debian.

 Essentially, I currently believe that multi-arch does not make sense
 for go, since we are only dealing with static binaries.

For multi-arch runtime libraries (in-scope for wheezy), sure, it's not
useful because there are no runtime libraries.

For multi-arch headers and development libraries (not formally in-scope
for wheezy, but some simpler packages already support it, and it's
likely to be a goal for jessie), it's useful to be multiarch so an
i386/amd64 system can have both golang-libhello:amd64 and
golang-libhello:i386, so that it can build both helloworld:amd64 and
helloworld:i386.

If golang has its own partial solution for multiarch but that solution
is actually insufficient for what we need (linux_arm for both armel and
armhf, etc.) and upstream are unwilling to make it sufficient, then I
don't see anything particularly wrong with
/usr/lib/arm-linux-gnueabi/golang/linux_arm/libhello.a. Sure, it's
redundant, but it works...

[Dmitrijs wrote:]
 /usr/lib/gocode/ for sources

/usr/[local/]share/gocode if they're arch-indep, surely? Or if they
*can* vary by arch but usually don't (like C headers),
/usr/local/lib/$TUPLE/gocode:/usr/local/share/gocode:/usr/lib/$TUPLE/gocode:/usr/share/gocode
or something?

 Furthermore, the package names should be e.g. “golang-codesearch” for
 the library code.google.com/p/codesearch (and of course just
 “codesearch” for the binaries).

Should the package with the headers and static library be
golang-codesearch-dev or something, in the hope that when Go grows the
ability to build dynamic libraries, the dynamic libraries can be called
golang-codesearch?

 For cross-compiling, it doesn’t matter which pre-compiled packages
 you have installed (e.g. linux_amd64) because the .go source files
 are shipped, too. go build will just re-compile the libraries then:

How cheap is recompiling everything? If it's sufficiently cheap, you
could just ship complete source and ignore the static libraries
entirely; or if it isn't, you'll benefit from proper multiarch.

S


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/50e33e9c.4000...@debian.org



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Michael Stapelberg
Hi Simon,

Simon McVittie s...@debian.org writes:
 Where does gccgo look for Go sources (src)?
 Where does gccgo look for Go static libraries (pkg)?
 Where does gccgo look for Go dynamic libraries? (Presumably the same
 places where gcc looks for C dynamic libraries?)
I can’t really answer these questions in depth, but the way I usually
invoke it is by using go(1)’s -compiler option, in which case the lookup
works the same as with the “original” go compiler.

I’m not sure whether Debian should support gccgo at all. Will ask the Go
mailing lists on what they think.

 Michael additionally wrote:
 I propose to patch
 Go in Debian so that it always appends /usr/lib/gocode when reading
 the list of paths from $GOPATH.

 Add /usr/local/lib/[$TUPLE/]gocode as more important than
 /usr/lib/[$TUPLE/]gocode, perhaps? That's how /usr/local usually works
 in Debian.
Mentioning /usr/local is a good point, but I’m not sure whether it’s
really a good idea: in the Go community, setting $GOPATH to a directory
within your home directory (e.g. ~/gocode) is normal and what everyone
uses. To install a third-party library, you just “go get” its URL, e.g.

go get code.google.com/p/codesearch

Which will then download the library to
~/gocode/src/code.google.com/p/codesearch.

Nobody will have libraries in /usr/local, because there is no facility
(no Makefiles) to install them there. The local directory is rather
specified by the user in $GOPATH — that is the idiomatic way.

 For multi-arch headers and development libraries (not formally in-scope
 for wheezy, but some simpler packages already support it, and it's
 likely to be a goal for jessie), it's useful to be multiarch so an
 i386/amd64 system can have both golang-libhello:amd64 and
 golang-libhello:i386, so that it can build both helloworld:amd64 and
 helloworld:i386.
You cannot currently cross-compile an i386 binary on an amd64 host with
the Go packages we have in Debian. I consider this a separate issue.
Essentially, when we have a golang-go package that contains a version of
the compiler which has been built to support cross-compiling for a
particular architecture (say golang-go-i386), you can cross-compile
using the source code of the libraries, a pre-compiled version is not
necessary.

To state this clearly: The benefit of shipping the pkg/ files in Debian
packages at all is to avoid re-compilation every time the user builds
code that depends on the library. We could just as well leave out the
pkg/ files and everything would still work — just slower.

 Should the package with the headers and static library be
 golang-codesearch-dev or something, in the hope that when Go grows the
 ability to build dynamic libraries, the dynamic libraries can be called
 golang-codesearch?
Yeah, I suppose we should name them -dev. AFAICT, there is some movement
into the shared library direction, see
https://groups.google.com/d/msg/golang-nuts/VS-Pt38097Q/MXMRFc7G26IJ
https://codereview.appspot.com/6926049/

 How cheap is recompiling everything? If it's sufficiently cheap, you
 could just ship complete source and ignore the static libraries
 entirely; or if it isn't, you'll benefit from proper multiarch.
See above. We could ship without the static libraries, but having them
speeds up compilation noticably in big projects.

-- 
Best regards,
Michael


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/x6a9ssocx8@midna.zekjur.net



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Paul Wise
On Wed, Jan 2, 2013 at 12:14 AM, Michael Stapelberg wrote:

 Only when not using the “official” compiler (gc), e.g. gccgo has support
 for dynamic linking.

Then we should use gccgo until the official compiler supports this.

 AFAIK not, but I will add this to the list of questions I want to ask on
 the Go mailing lists.

Ok, please let us know about the results.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6G3V1NCPyKnZt7By=sydcn_qh-kwnbsveenjkkhmuc...@mail.gmail.com



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Shawn
On Tue, Jan 1, 2013 at 6:54 PM, Paul Wise p...@debian.org wrote:

 On Wed, Jan 2, 2013 at 12:14 AM, Michael Stapelberg wrote:

  Only when not using the “official” compiler (gc), e.g. gccgo has support
  for dynamic linking.

 Then we should use gccgo until the official compiler supports this.


go also doesn't support building as a shared library, because
#1 the go scheduler would have to have differn't bootstrapping code hooked
up (for gccgo it would be in libgo) and
#2  go uses split stacks[1].
But when it doesn't support building as a shared library, it will probably
be done with gccgo first.


  AFAIK not, but I will add this to the list of questions I want to ask on
  the Go mailing lists.

 Ok, please let us know about the results.


Henceforth when a go program depends on a go library, those go libraries
are ALWAYS compiled in statically. Static linking causes many problems for
distributions like Debian, and therefore this presents a significant hurdle
to quality packaging of go in Debian. Source-only distribution is perhaps
possible, but then AFAIK go library writers are not necessarily as careful
as C library writers to maintain source compatibility.

[1] http://gcc.gnu.org/wiki/SplitStacks


 -Shawn Landden


Re: [RFC] Go (golang) packaging

2013-01-01 Thread Bernhard R. Link
* Michael Stapelberg michael+deb...@stapelberg.de [130101 14:45]:
 Furthermore, the package names should be e.g. “golang-codesearch” for
 the library code.google.com/p/codesearch

Please also consider codesearch-golang. Especially with longer package
names not everything can show the full name so the beginning should be
the more important information.

 (and of course just “codesearch” for the binaries).

I assume s/binaries/sources/? And I'd suggest to just not policy the
source package names at all (as there really is no one rule who fits
them all and there is no reason to have a rule there at all).

Another question: Have you considered asking for a archive Section for
those packages? I guess with no special section yet all those packages
would be section libdevel as they are for static linking only, wouldn't
they?

Bernhard R. Link


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130102071555.ga3...@client.brlink.eu



Re: [RFC] Go (golang) packaging

2013-01-01 Thread Paul Wise
On Wed, Jan 2, 2013 at 3:15 PM, Bernhard R. Link wrote:

 Another question: Have you considered asking for a archive Section for
 those packages? I guess with no special section yet all those packages
 would be section libdevel as they are for static linking only, wouldn't
 they?

http://wiki.debian.org/MichaelStapelberg/GoPackaging#TODO

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6FiyuoNc-GyaoO=jnygh2prp2pasb7qvkhh5whqp+q...@mail.gmail.com