Re: Go (golang) packaging, part 2

[Barry Warsaw]

Okay, fortunately, no bands are practicing tonight and no kids need homework
help, so let's see if I can answer some of these questions. :)

On Feb 07, 2013, at 08:54 AM, Paul Wise wrote:

>On Thu, Feb 7, 2013 at 8:19 AM, Barry Warsaw wrote:
>
>> Speaking with many hats on, I think Debian Python has done a very admirable
>> job of integrating the Python ecosystem with Debian.
>
>One of the pain points for users (I've had folks ask me this face-to-face)
>with that stuff was site-packages vs dist-packages. With your various Python
>hats on, can you explain why not just use "packages" instead of
>"site-packages" and "dist-packages"?

Fundamentally, this comes down to a conflict between Python's historical
defaults and Debian's interpretation of the FHS.  Let me just stipulate that
I'm not casting blame, or saying that anybody is doing anything wrong.  I'm
not interested in that discussion, though I've had it many times.  It is what
it is.

Old timers like me will remember the days when *nix systems reserved
/usr/local for stuff you downloaded and installed from source (i.e. most
everything on a usable system :).  There was no /opt or FHS.  This was
codified in the first auto-configuration scripts.  I don't remember when
Python adopted the configure regime, but as long as I can remember (going back
at least to 1994), a default build-from-source of Python installed into
/usr/local.  When site-packages was added,
/usr/local/lib/pythonX.Y/site-packages was the most logical place to put it.

Predating my involvement with Debian, I remember problem reports where
developers of Python, and others who install from source for various reasons,
would break their systems when they used the wrong Python executable to
install third party packages outside of the Debian packaging system.  This was
because Debian allowed /usr/local/lib/pythonX.Y/site-packages to be used for
third party packages installed outside the Debian packaging system, using the
*system* Python, i.e. /usr/bin/python.  This meant that if I installed
something for /usr/bin/python into /usr/local/lib/pythonX.Y/site-packages it
could easily break my /usr/local/bin/python, and possibly vice versa.

I think it was at a Pycon years ago that Matthias and I discussed this
problem.  At the time (and probably still so), it didn't seem like either
Debian or Python was going to change its policy, so we had to find a way to
avoid the conflict and let both communities live in peace.  Matthias's
solution was the use of dist-packages for Debian's system Python, which would
be ignored by a /usr/local/bin Python.  Also, system Python would ignore
/usr/local/lib/pythonX.Y/site-packages (but not .../dist-packages), thus
avoiding all conflict.  It seemed elegant at the time, and I still think this
is a reasonable compromise, even though it does cause some tooling problems,
which have to be patched in Debian.

>The right way (IMO) would have been to put site packages in
>/usr/local/lib/pythonX.Y/packages and dist ones in
>/usr/lib/pythonX.Y/packages. Right now I have
>/usr/local/lib/pythonX.Y/dist-packages and /usr/lib/pythonX.Y/dist-packages,
>why is /usr/local dist-packages instead of site-packages? /usr/local is
>clearly not the location for distro installed packages.

That was my position, i.e. that system Python shouldn't have any path from
/usr/local on sys.path, but that was very strongly (at the time) disputed by
Debian users.  To be fair, the Debian users at the time (and maybe still do)
say that the right solution is for a default from-source build of Python to
install into /opt/local and not /usr/local, but again, that would conflict
with years of established use by upstream.

That's the historical background as I remember it anyway.

>Why did Debian have to invent /usr/share/pyshared and symlink farms in
>/usr/lib/pythonX.Y instead of upstream having something like that in
>the default install and search paths?

Because upstream doesn't really care (or didn't until my PEPs 3147 and 3149 in
Python 3.2) about multiple versions of packages co-existing in harmony, and
because upstream Python requires .pyc files to live next to (FSVO, see below)
the .py files.

Debian was the first place that I recall where multiple versions of Python
could be co-installed.  Let's say you have both Python 2.6 and 2.7 installed,
and you have a module called foo.py that is source-level compatible with both.
The problem is that Python has never guaranteed that .pyc files would be
compatible across Python versions.  It's never said they wouldn't be, but in
practice the byte code cached in .pyc files always changes, due to new
features or bug fixes in the interpreter between major version numbers.

So in Debian you have a situation where you want to share foo.py across all
supported and installed Pythons, but where you cannot share .pyc files because
they aren't compatible.  You want to share .py files 1) to keep package sizes
smaller, 2) to consume less disk space, 3) because you don't actua

Re: Go (golang) packaging, part 2

[Matthias Klose]

Am 07.02.2013 01:54, schrieb Paul Wise:
> On Thu, Feb 7, 2013 at 8:19 AM, Barry Warsaw wrote:
> 
>> Speaking with many hats on, I think Debian Python has done a very admirable
>> job of integrating the Python ecosystem with Debian.
> 
> One of the pain points for users (I've had folks ask me this
> face-to-face) with that stuff was site-packages vs dist-packages. With
> your various Python hats on, can you explain why not just use
> "packages" instead of "site-packages" and "dist-packages"?

Sure, anything else than "site-packages" would have worked.

> The right
> way (IMO) would have been to put site packages in
> /usr/local/lib/pythonX.Y/packages and dist ones in
> /usr/lib/pythonX.Y/packages. Right now I have
> /usr/local/lib/pythonX.Y/dist-packages and
> /usr/lib/pythonX.Y/dist-packages, why is /usr/local dist-packages
> instead of site-packages? /usr/local is clearly not the location for
> distro installed packages.

This came up several times (e.g. [1], [2]). See [3] for the rationale for the
naming and the extra directories in /usr/local.

> Why did Debian have to invent /usr/share/pyshared and symlink farms in
> /usr/lib/pythonX.Y instead of upstream having something like that in
> the default install and search paths?

Having separate copies of .py files would have been an option. Splitting .so and
.py files across directories not, because having different directories for third
party packages in python2.x can break the import behaviour, does break split-out
debug information, doesn't work for gir files, and more. Please search back the
archives until 2003, if you are really interested in this.

> The location of .pyc files that are built at install time doesn't feel
> FHS-correct to me, /var/cache/python/X.Y/ seems better.

The location is correct if you include these files in the packages. So why
change it if you regenerate them? Byte-compilation and pyshared is only a means
to make pure python code independent of the python version. This is changed in
Python3 upstream [4]. Please address any outstanding issues upstream, then maybe
provide a backport.

> Debian's Python build helper tools are still breeding like rabbits,
> there is a new one in experimental. I guess because the current ones
> dh_python2/dh_python3 don't handle packages that contain only code
> that runs on both python2 and python3 without changes.

I don't have any issue to build packages using dh_python2 and dh_python3. Please
file a bug report if you do have such issues.  It is my understanding that
dh_python2/dh_python3 are stable and should still be available in jessie, but
then using the underlying pybuild system.  If you want to improve the tools,
please do. Sarcasm doesn't do any improvement by itself.  Please join Piotr.

  Matthias

[1] https://lists.debian.org/debian-python/2008/03/msg00021.html
[2] https://lists.debian.org/debian-python/2009/02/msg2.html
[3] https://lists.debian.org/debian-devel/2009/02/msg00431.html
[4] http://www.python.org/dev/peps/pep-3147/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/51130693.4050...@debian.org

Re: Go (golang) packaging, part 2

[Paul Wise]

On Thu, Feb 7, 2013 at 8:19 AM, Barry Warsaw wrote:

> Speaking with many hats on, I think Debian Python has done a very admirable
> job of integrating the Python ecosystem with Debian.

One of the pain points for users (I've had folks ask me this
face-to-face) with that stuff was site-packages vs dist-packages. With
your various Python hats on, can you explain why not just use
"packages" instead of "site-packages" and "dist-packages"? The right
way (IMO) would have been to put site packages in
/usr/local/lib/pythonX.Y/packages and dist ones in
/usr/lib/pythonX.Y/packages. Right now I have
/usr/local/lib/pythonX.Y/dist-packages and
/usr/lib/pythonX.Y/dist-packages, why is /usr/local dist-packages
instead of site-packages? /usr/local is clearly not the location for
distro installed packages.

Why did Debian have to invent /usr/share/pyshared and symlink farms in
/usr/lib/pythonX.Y instead of upstream having something like that in
the default install and search paths?

The location of .pyc files that are built at install time doesn't feel
FHS-correct to me, /var/cache/python/X.Y/ seems better.

Debian's Python build helper tools are still breeding like rabbits,
there is a new one in experimental. I guess because the current ones
dh_python2/dh_python3 don't handle packages that contain only code
that runs on both python2 and python3 without changes.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6GZ4O1yv38kNkneu_jnDx7=r-dtx16uudiebvzneoz...@mail.gmail.com

Re: Go (golang) packaging, part 2

[Russ Allbery]

Barry Warsaw  writes:

> Where things get tricky is if you have multiple applications that need
> different versions of its dependencies.  Say Debian has python-foo 1.2
> which application Bar needs, but application Baz needs python-foo 2.0.
> Despite years of discussion, in Debian, Ubuntu, and upstream, we really
> don't have a good multi-version story.  I think that's forgivable though
> because it's a Really Hard Problem and nobody's stepped up to pay a team
> of 5 gurus for the 25 man years of non-stop work it would probably take
> to solve (both technically and socially ;).  This problem isn't
> particularly limited to Python.

I keep being tempted to go off on a rant about how we have all of these
modern, sophisticated, much more expressive programming languages, and yet
still none of them handle ABI versioning as well as C does.  Normal
versioning problems that we just take for granted in C, such as allowing
the coexistence of two versions of the same library with different ABIs on
the system at the same time without doing the equivalent of static
linking, vary between ridiculously difficult and completely impossible in
every other programming language that I'm aware of (with the partial
exception of C++ where it is possible to use the same facilities as C,
just considerably more difficult).

In fact, most new languages seem to be *regressing* here.  Both Perl and
Python were already fairly bad at this, Java bailed on the problem and
shoved it under the carpet of a version of static linking, and now Go is
even worse than they are and is explicitly embracing static linking.
Sigh.

It's immensely frustrating that this doesn't appear to be an interesting
problem to language designers, and even brand-new, sophisticated languages
that are trying to be at the cutting edge of language design do not make
any serious attempt at solving the ABI versioning problem.

Everyone could stand to learn something from the REST and web services
community, where ABI versioning is a well-known and widely accepted
problem that regularly prompts considerable discussion and careful thought
and a variety of useful proposals for addressing it with different
tradeoffs.  It's kind of sad when one's brand-new, cutting-edge
programming language does a worse job at a foundational requirement of
language ecosystems than the Facebook API.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/8738x9ynes@windlord.stanford.edu

Re: Go (golang) packaging, part 2

[Barry Warsaw]

On Feb 06, 2013, at 03:26 PM, Roland Mas wrote:

>I can only speak about Python and Perl, but I don't remember *ever* having
>been told to use their deployment system instead of the packaged versions of
>the interpreter and modules.  The closest I've seen is something like "if
>you're running CentOS or RHEL, then you'll need this plethora of modules that
>are not packaged, so please use our language-specific system to install them
>instead".

Speaking with many hats on, I think Debian Python has done a very admirable
job of integrating the Python ecosystem with Debian.

It's never going to be perfect, and there are certainly difficult outliers,
but in most cases, things work pretty well.  OTOH, Python itself has several
strategies for dealing with difficult situations, with probably the most
notable being virtualenv (and upstream's built-in venv in Python 3.3+).  You
can even mix and match, i.e. if some Debian packaged versions are fine but you
need one or two missing or out-of-date packages from the Cheeseshop, you can
`virtualenv --site-system-packages` that generally works pretty well.

Where things get tricky is if you have multiple applications that need
different versions of its dependencies.  Say Debian has python-foo 1.2 which
application Bar needs, but application Baz needs python-foo 2.0.  Despite
years of discussion, in Debian, Ubuntu, and upstream, we really don't have a
good multi-version story.  I think that's forgivable though because it's a
Really Hard Problem and nobody's stepped up to pay a team of 5 gurus for the
25 man years of non-stop work it would probably take to solve (both
technically and socially ;).  This problem isn't particularly limited to
Python.

I'm also encouraged by the albeit slow work on upstream packaging technology
(PEPs and code) to help improve the overall packaging story.  I had many
discussions with various packaging folks, and good developers from Fedora and
other distros, about ways to make it easier to integrate PyPI packaging with
distros, Linux-based or otherwise.  I thought we'd made a lot of good
progress, but some of the drivers moved on to other things.  I'm hoping Nick
Coghlan's efforts at Pycon 2013 can help motivate a revival of this work[1].

Cheers,
-Barry

[1]
https://us.pycon.org/2013/community/openspaces/packaginganddistributionminisummit/


signature.asc
Description: PGP signature

Re: socket-based activation has unmaintainable security?

[Russell Coker]

On Thu, 7 Feb 2013, Thomas Goirand  wrote:
> > I think he's referring to allowing processes which require listening to a
> > port under 1024 to run without superuser privileges. I believe our
> > implementation on Debian (e.g. Apache) is to have the process start as
> > root, start listening, and then setuid to an unprivileged user.
> 
> Which would be the wrong way of doing things / wrong reason
> for using root as running user, since you can set the
> CAP_NET_BIND_SERVICE capability... (man capabilities ...)

Such capabilities allow the process to bind to all low ports, which usually 
isn't what you desire.  If you want to permit a daemon to bind to exactly one 
reserved port and no others then it seems that the options are systemd (if the 
daemon supports socket based activation) and SE Linux.

-- 
My Main Blog http://etbe.coker.com.au/
My Documents Bloghttp://doc.coker.com.au/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201302071028.28586.russ...@coker.com.au

Re: socket-based activation has unmaintainable security?

[Andrey Rahmatullin]

On Wed, Feb 06, 2013 at 03:20:09PM -0600, Serge Hallyn wrote:
> > > > > Do we finally have mechanisms to start processes without root but with
> > > > > elevated capabilities?
> > > > We also need fallback for non Capability-capable supported kernels
> > > > (wow that's an awkward sentence)
> > > Not to mention non-xattr-backed filesystems.
> > xattrs is only one of possible mechanisms but as we don't have it either,
> > its shortcomings are probably not worth mentioning.
> For posix capabilities attached to files xattrs are currently the
> only means.  That's what I assumed this was referring to.
I suppose there can be other theoretical means to do that with a
privileged helper (and /sbin/init is privileged, for example). 

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Re: socket-based activation has unmaintainable security?

[Serge Hallyn]

Quoting Andrey Rahmatullin (w...@wrar.name):
> On Wed, Feb 06, 2013 at 12:30:28PM -0600, Serge Hallyn wrote:
> > > > Do we finally have mechanisms to start processes without root but with
> > > > elevated capabilities?
> > > We also need fallback for non Capability-capable supported kernels
> > > (wow that's an awkward sentence)
> > Not to mention non-xattr-backed filesystems.
> xattrs is only one of possible mechanisms but as we don't have it either,
> its shortcomings are probably not worth mentioning.

For posix capabilities attached to files xattrs are currently the
only means.  That's what I assumed this was referring to.

-serge


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130206212009.GA18337@sergelap

Bug#699922: ITP: pyobfsproxy -- pluggable transport proxy for Tor (Python implementation)

[Jérémy Bobbio]

Package: wnpp
Severity: wishlist
Owner: Jérémy Bobbio 

* Package name: pyobfsproxy
  Version : 0.1
  Upstream Author : George Kadianakis 
* URL : 
https://git.torproject.org/pluggable-transports/pyobfsproxy.git
* License : BSD-3-clause
  Programming Lang: Python
  Description : pluggable transport proxy for Tor (Python implementation)

pyobfsproxy is a tool that attempts to circumvent censorship by transforming
the Tor traffic between the client and the bridge. This way censors, who
usually monitor traffic between the client and the bridge, will see
innocent-looking transformed traffic instead of the actual Tor traffic.

It is written in Python and is compliant with the Tor pluggable transports
specification.

-- 
Jérémy Bobbio.''`. 
lu...@debian.org: :Ⓐ  :  # apt-get install anarchism
`. `'` 
  `-   


signature.asc
Description: Digital signature

Bug#699921: ITP: pyptlib -- library for Tor pluggable transports written in Python

[Jérémy Bobbio]

Package: wnpp
Severity: wishlist
Owner: Jérémy Bobbio 

* Package name: pyptlib
  Version : 0.1
  Upstream Author : George Kadianakis 
* URL : https://git.torproject.org/pluggable-transports/pyptlib.git
* License : BSD-3-clause
  Programming Lang: Python
  Description : library for Tor pluggable transports written in Python

pyptlib is a little Python library which understands the Tor pluggable
ransport managed-proxy protocol. It is useful for developers writing
Tor pluggable transports in Python.

-- 
Jérémy Bobbio.''`. 
lu...@debian.org: :Ⓐ  :  # apt-get install anarchism
`. `'` 
  `-   


signature.asc
Description: Digital signature

Re: socket-based activation has unmaintainable security?

[Игорь Пашев]

2013/2/6 Chow Loong Jin :
> On 07/02/2013 01:35, Thomas Goirand wrote:
>>> >
>> Which would be the wrong way of doing things / wrong reason
>> for using root as running user, since you can set the
>> CAP_NET_BIND_SERVICE capability... (man capabilities ...)
>
> Yeah, I figured as much, but isn't that a Linuxism?

illumos (OpenSolaris) kernel has "privileges" [1]
So some services (can't remember which) are not started as root at all.

[1] http://illumos.org/man/5/privileges


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/call-q8y0ui+tv7fzsyjyuo1vsyydd2xaehvmhywu4oy6tov...@mail.gmail.com

Re: socket-based activation has unmaintainable security?

[Andrey Rahmatullin]

On Wed, Feb 06, 2013 at 12:30:28PM -0600, Serge Hallyn wrote:
> > > Do we finally have mechanisms to start processes without root but with
> > > elevated capabilities?
> > We also need fallback for non Capability-capable supported kernels
> > (wow that's an awkward sentence)
> Not to mention non-xattr-backed filesystems.
xattrs is only one of possible mechanisms but as we don't have it either,
its shortcomings are probably not worth mentioning.

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Re: socket-based activation has unmaintainable security?

[Serge Hallyn]

Quoting Jonathan Dowland (j...@debian.org):
> On 6 Feb 2013, at 17:37, Andrey Rahmatullin  wrote:
> 
> > Do we finally have mechanisms to start processes without root but with
> > elevated capabilities?
> 
> We also need fallback for non Capability-capable supported kernels
> (wow that's an awkward sentence)

Not to mention non-xattr-backed filesystems.

Every time I've been in a discussion like this, that ends up being
the reason not to pursue it.

-serge


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130206183028.GB19018@sergelap

Re: socket-based activation has unmaintainable security?

[Chow Loong Jin]

On 07/02/2013 01:35, Thomas Goirand wrote:
>> >
> Which would be the wrong way of doing things / wrong reason
> for using root as running user, since you can set the
> CAP_NET_BIND_SERVICE capability... (man capabilities ...)

Yeah, I figured as much, but isn't that a Linuxism?

-- 
Kind regards,
Loong Jin



signature.asc
Description: OpenPGP digital signature

Re: socket-based activation has unmaintainable security?

[Jonathan Dowland]

On 6 Feb 2013, at 17:37, Andrey Rahmatullin  wrote:

> Do we finally have mechanisms to start processes without root but with
> elevated capabilities?

We also need fallback for non Capability-capable supported kernels (wow that's 
an awkward sentence)

--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/a2ec83dc-57b2-474b-b56b-5d0225a0c...@debian.org

Re: socket-based activation has unmaintainable security?

[Andrey Rahmatullin]

On Thu, Feb 07, 2013 at 01:35:13AM +0800, Thomas Goirand wrote:
> >>> socket-activation in systemd _helps_ security in that you can give an
> >>> unprivlidged process a listening port under 1024. (using a privileged
> >>> configuration file)
> >> Privileged vs. unprivileged port is not really a secuitry improvement.
> > I think he's referring to allowing processes which require listening to a 
> > port
> > under 1024 to run without superuser privileges. I believe our 
> > implementation on
> > Debian (e.g. Apache) is to have the process start as root, start listening, 
> > and
> > then setuid to an unprivileged user.
> >
> Which would be the wrong way of doing things / wrong reason
> for using root as running user, since you can set the
> CAP_NET_BIND_SERVICE capability... (man capabilities ...)
Do we finally have mechanisms to start processes without root but with
elevated capabilities?

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Re: socket-based activation has unmaintainable security?

[Thomas Goirand]

On 02/06/2013 05:03 PM, Chow Loong Jin wrote:
> On 06/02/2013 16:27, Martin Wuertele wrote:
>> * Shawn  [2013-02-05 18:43]:
>>
>>> socket-activation in systemd _helps_ security in that you can give an
>>> unprivlidged process a listening port under 1024. (using a privileged
>>> configuration file)
>> Privileged vs. unprivileged port is not really a secuitry improvement.
> I think he's referring to allowing processes which require listening to a port
> under 1024 to run without superuser privileges. I believe our implementation 
> on
> Debian (e.g. Apache) is to have the process start as root, start listening, 
> and
> then setuid to an unprivileged user.
>
Which would be the wrong way of doing things / wrong reason
for using root as running user, since you can set the
CAP_NET_BIND_SERVICE capability... (man capabilities ...)

Thomas

P.S: I know this since the nice talk from Luciano last summer at debconf! :)


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/51129451.3070...@debian.org

Re: Go (golang) packaging, part 2

[Hilko Bengen]

* Roland Mas:

> Hilko Bengen, 2013-02-06 14:46:11 +0100 :
>
> [...]
>
>> I am pretty sure that if you asked about packaging software in the
>> Python, Perl, Ruby, Java, Lua communities, you would get recommendations
>> to not use Debian packages at all and get pointers to what the
>> respective community considers a solution to the packaging problem (if
>> they see it as a problem at all).
>
>   I can only speak about Python and Perl, but I don't remember *ever*
> having been told to use their deployment system instead of the packaged
> versions of the interpreter and modules.  The closest I've seen is
> something like "if you're running CentOS or RHEL, then you'll need this
> plethora of modules that are not packaged, so please use our
> language-specific system to install them instead".

I have heard exactly what I described at Perl conferences more than
once. The root cause may have been that some versions of RHEL still ship
Perl 5.8.something and ancient, broken versions of some modules, but I
had the impression that some (not all) people over-generalized this view
to every Linux distribution, including Debian.

If you need or want to run the current stable Perl (5.16.2) and the
latest-greatest modules on wheezy, it's going to be in your best
interest to use things like perlbrew and local::lib (which are both
shipped with Debian).

That upstream's preferences may differ from ours is not even a problem,
as long as no-one tries to enforce his values on users. I don't see such
attempts in the Perl and Go communities.

>   So it is possible for a language community to work with the
> distributors rather than against.

And it's possible for Debian to at least *try* to work with any language
community.

Simply calling people "idiots" when one hasn't yet understood the other
community's values/interests/position does not help, of course. ;-)

Cheers,
-Hilko


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87halplbvt@msgid.hilluzination.de

Re: Building and using shared libraries using gccgo

[Enrico Tassi]

On Tue, Feb 05, 2013 at 04:36:44PM +0100, Joachim Breitner wrote:
> At least to me my work on Haskell in Debian feels more than pretending,
> and from personal experience with the creators of the language, I have
> strong doubts that they are Idiots.

They are not, they are very smart, but they are academic people
with a very little idea of what it means to build a real software
distribution like Debian.  I love type systems to, but whenever I talk
about static linking with in the academic context, they think it just
saves few KB of disk space.

I can't assert the same for go's designers, but for different reasons
they ended up incurring in the same design flaw.  They want a binary to
be shipped on their production servers, and be sure that no matter how
crappy they are, it will work (no missing .so, no .so ABI problem).
Or at least, this is what I've understood.

Here we build a realistic system where a security update can be
made just by recompiling and pullig 1 package, not recompiling the whole
archive.  Static linking imposes so.

To me it is just a problem of scale. You see the problems of static
linking only on a reasonably large scale. And Debian is huge...

Ciao
-- 
Enrico Tassi


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130206143639.GA7528@birba

Re: Go (golang) packaging, part 2

[Roland Mas]

Hilko Bengen, 2013-02-06 14:46:11 +0100 :

[...]

> I am pretty sure that if you asked about packaging software in the
> Python, Perl, Ruby, Java, Lua communities, you would get recommendations
> to not use Debian packages at all and get pointers to what the
> respective community considers a solution to the packaging problem (if
> they see it as a problem at all).

  I can only speak about Python and Perl, but I don't remember *ever*
having been told to use their deployment system instead of the packaged
versions of the interpreter and modules.  The closest I've seen is
something like "if you're running CentOS or RHEL, then you'll need this
plethora of modules that are not packaged, so please use our
language-specific system to install them instead".

  So it is possible for a language community to work with the
distributors rather than against.  The fact that some communities
(insert your favourite pet peeve here) choose not to shouldn't be
construed as an impossibility.

Roland.
-- 
Roland Mas

Au royaume des aveugles, les borgnes sont mal vus.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/87r4ktfrhc@polymir.internal.placard.fr.eu.org

Re: Go (golang) packaging, part 2

[Hilko Bengen]

* Neil Williams:

> If what you want is complete separation, why is there even a long
> running thread on integration?

Sorry if I failed to make myself clear:

I want excellent Debian packages of the compiler/runtime/tools *and*
libraries *and* still make it possible for our users to use upstream's
half-a-package-manager if they so desire. Complete separation is
something entirely different.

> Then why bother discussing packaging Go if it isn't going to be
> packaged, it's just going to invent it's own little ghetto in
> /usr/local?

We already have directory structures in place in /usr/local for several
other languages and "software ecosystems":

$ ls /usr/local/lib/
eclipse   ocaml  python2.6  python3.1  site_ruby
luarocks  perl   python2.7  python3.2
$ ls /usr/local/share
applications emacs  games  man   sgml   xml
ca-certificates  fonts  hibernate  perl  texmf  zsh

I am pretty sure that if you asked about packaging software in the
Python, Perl, Ruby, Java, Lua communities, you would get recommendations
to not use Debian packages at all and get pointers to what the
respective community considers a solution to the packaging problem (if
they see it as a problem at all). This likely involve replacing the
language compiler/runtime itself.

So what? Just take the freedom given to you through the software
licenses and ignore those parts of their view of how things ought to
work and work with the useful parts instead.

Cheers,
-Hilko


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87y5f1lfmk@msgid.hilluzination.de

Re: Go (golang) packaging, part 2

[Russ Allbery]

Neil Williams  writes:

> If Go wants to be packaged, it complies by the requirements of
> packaging. If it wants to live the life of a hermit and disappear up
> itself, that's fine but then it doesn't get the privilege of interacting
> with the rest of Debian. It's just a user download.

Debian packaging isn't a reward that we give to upstream software authors
in exchange for doing things right.  Most software gets Debian packages
because people who use Debian want to use it and don't like dealing with
unpackaged software.  Upstream authors are often indifferent.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/878v71dbbl@windlord.stanford.edu

Re: Go (golang) packaging, part 2

[Jon Dowland]

On Wed, Feb 06, 2013 at 09:23:02AM +, Neil Williams wrote:
> Then don't package Go at all and leave it entirely outside the realm of dpkg
> - no dependencies allowed in either direction, no files created outside
> /usr/local for any reason, no contamination of the apt or dpkg cache data. If
> what you want is complete separation, why is there even a long running thread
> on integration?

That's one possible solution, and a low-risk one at that. The others carry the
risk of doing the job badly, especially if there is not enough resource to
implement them going forwards.

> Then why bother discussing packaging Go if it isn't going to be packaged,
> it's just going to invent it's own little ghetto in /usr/local?

That seems pretty perjorative. The reason Go has "invented it's own little
ghetto" is to solve the distribution problem. The reason they want to solve it
is because, despite our best efforts, we haven't solved it. Pretending we have
done helps nobody. From Go's perspective, we are a bit player. There's a
pattern of misplaced arrogance and pride that permeates -devel from time to
time whenever difficult integration discussions come up (systemd included) that
really doesn't help. Let's not overstate our importance in the wider world, it
does not help us one bit. Step by step we'll just close our doors to the rest
of the Universe and slide further into irrelevance.

> If Go wants to be packaged, it complies by the requirements of packaging.

Go doesn't want anything: It's a programming language and environment, not a
sentient being. The authors of Go are probably not that bothered about it being
packaged, seeing as they've put energy into solving the distribution problem
themselves, at the same time making it more difficult to distribution-package.
The people who want to see it packaged are people who want to see Debian users
be able to conveniently interact with Go-land.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130206094410.GA322@debian

Re: socket-based activation has unmaintainable security?

[Russ Allbery]

Chow Loong Jin  writes:

> I think he's referring to allowing processes which require listening to
> a port under 1024 to run without superuser privileges. I believe our
> implementation on Debian (e.g. Apache) is to have the process start as
> root, start listening, and then setuid to an unprivileged user.

For INN, quite some time ago, I wrote a setuid helper program that did
nothing but bind the port for its parent process.  I know there are a few
other implementations of the same idea (I think Ian Jackson has a generic
one that's packaged in Debian).  I've always been surprised that more
long-running daemons that for one reason or another need to set up their
own listening ports don't use that technique.  I think it's more secure
than starting the whole complex daemon as root and then dropping
privileges.

It's not completely trivial, since you have to use two different
techniques depending on whether the OS uses BSD-style sockets or
STREAMS-style sockets (in BSD-style sockets you can bind in the child and
the parent sees the results, but with STREAMS-style sockets you pass the
file descriptor back to the parent), but it was only 219 lines of C
(without comments) for the helper program and another 30 or so for the
library to call it, and most of the helper program size is argument
parsing and verification.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87halpdbwb@windlord.stanford.edu

Re: Go (golang) packaging, part 2

[Neil Williams]

On Tue, 05 Feb 2013 23:44:30 +0100
Hilko Bengen  wrote:

> * Adam Borowski:
> 
> >> The worst case scenario IMHO is some people invest a lot of time to
> >> make the Debianized-Go stuff quite divergent from upstream, people's
> >> expectations of how things behave in Go-land are broken when they
> >> access Go-via-Debian
> >
> > Just think what would happen if every single language and environment
> > would be allowed its own packaging system. This way lies madness.
> 
> Not if there's a clear separation between the language's solution and
> dpkg-land. Mind you, almost every language already has its own packaging
> system or two. I really don't see any big deal here.

Then don't package Go at all and leave it entirely outside the realm of
dpkg - no dependencies allowed in either direction, no files created
outside /usr/local for any reason, no contamination of the apt or dpkg
cache data. If what you want is complete separation, why is there even
a long running thread on integration?

> > Yes, packaging needs to be beaten into shape, even if this goes
> > against upstream wishes.
> 
> Just put up a boundary between dpkg and everything else. This is
> actually quite easy because Debian packages generally don't install
> anything to /usr/local. From there it's just a maatter of ensuring that
> each packaging system respects the line. And I expect that that's
> exactly what is going to happen in the case of the go(1) tool.

Then why bother discussing packaging Go if it isn't going to be
packaged, it's just going to invent it's own little ghetto
in /usr/local?

If Go wants to be packaged, it complies by the requirements of
packaging. If it wants to live the life of a hermit and disappear up
itself, that's fine but then it doesn't get the privilege of
interacting with the rest of Debian. It's just a user download.

-- 


Neil Williams
=
http://www.linux.codehelp.co.uk/



pgpKfyWEDztC5.pgp
Description: PGP signature

Re: socket-based activation has unmaintainable security?

[Chow Loong Jin]

On 06/02/2013 16:27, Martin Wuertele wrote:
> * Shawn  [2013-02-05 18:43]:
> 
>> socket-activation in systemd _helps_ security in that you can give an
>> unprivlidged process a listening port under 1024. (using a privileged
>> configuration file)
> 
> Privileged vs. unprivileged port is not really a secuitry improvement.

I think he's referring to allowing processes which require listening to a port
under 1024 to run without superuser privileges. I believe our implementation on
Debian (e.g. Apache) is to have the process start as root, start listening, and
then setuid to an unprivileged user.

-- 
Kind regards,
Loong Jin



signature.asc
Description: OpenPGP digital signature

Re: socket-based activation has unmaintainable security?

[Martin Wuertele]

* Shawn  [2013-02-05 18:43]:

> socket-activation in systemd _helps_ security in that you can give an
> unprivlidged process a listening port under 1024. (using a privileged
> configuration file)

Privileged vs. unprivileged port is not really a secuitry improvement.

Yours Martin


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130206082757.gc24...@anguilla.debian.or.at