Re: [gentoo-dev] arm64
On 25/01/15 03:57, Tom Gall wrote: Hi All, This is sort of a CFP in some ways but not quite that formal. I’ve been throttled back on arm64 for a bit as the hardware I’ve had access to has all been painfully remote and configured in ways that was less than optimal for massive key wording efforts. That’s about to change. So if there are others out there who have the same interest be great to coordinate efforts and get this moving along so we have arm64 stages and start to pull together install instructions for the varies pieces of hardware starting to show up. Best, Tom I have interest and hopefully I'll get some hardware (assuming the seller fixes his shopify instance...) lu
Re: [gentoo-dev] Things one could be upset about
On 01/22/2015 10:19, Peter Stuge wrote: Joshua Kinard wrote: Using seed stage3 stages I built 6 months ago (but never released due to getting sidetracked), I run into errors like this: !!! Multiple package instances within a single package slot have been pulled !!! into the dependency graph, resulting in a slot conflict: dev-lang/perl:0 (dev-lang/perl-5.20.1-r4:0/5.20::gentoo, ebuild scheduled for merge) pulled in by =dev-lang/perl-5.20* required by (virtual/perl-ExtUtils-ParseXS-3.240.0:0/0::gentoo, ebuild scheduled for merge) ^ ^ (and 16 more with the same problem) (dev-lang/perl-5.18.2-r2:0/5.18::gentoo, ebuild scheduled for merge) pulled in by dev-lang/perl:0/5.18=[-build(-)] required by (dev-perl/libintl-perl-1.230.0:0/0::gentoo, installed) =dev-lang/perl-5.18* required by (virtual/perl-ExtUtils-Manifest-1.630.0-r1:0/0::gentoo, installed) ^ ^ (and 2 more with the same problems) It's hard to read mess like that and trace down the offending package, fix it, and make catalyst happy. Lots of dev-perl packages have specific minor version dependencies on dev-lang/perl, maybe because sometimes the package is included in perl and sometimes not. It's a f*ing mess. You have to look up all your installed dev-perl packages manually and find which ones are either too old to know about perl-5.20 or not compatible with it, and then you have to unmerge those manually. In the past, it's been possible to have Portage deal with the updates to Perl, but only as long as you hit all of the packages in the same update run to satisfy the dependency chain. Newer portage seems to not do that anymore. But that output is horrible. Even with the color coding, it's not directly apparent which package is the problem package. I once had a Perl update issue bad enough that I removed all perl packages entirely, then remerged them from scratch. Took a while, but it fixed things. Kinda defeats the purpose of catalyst in the first place. The proper way is to build stage1+2+3 yourself, then this mess doesn't happen. But like you I too cheat a little, and have to deal with the mess. Well, I was trying to do it the right way by going stage1 - stage2 - stage3. I was using a stage3 that I built over the summer as the seed stage for the new stage1 when I started running into problems with Perl. I finally fixed that, got stage1 built, then got bit by Bug #447126 while trying to build the stage2. So now, I have to start a stage2 run, then after the unpack (but before catalyst drops into the chroot), edit the chroot's make.conf and remove sandbox from FEATURES, which is apparently part of the problem. Just irritating. And I know I'm earning no sympathy when I point out that my build machines (an Octane and an Onyx2) aren't the fastest things on the planet, nor the most power efficient (1 kW between the both of them). But I'd at least like to waste that power on actual compile jobs, not watching emerge's little spinner all the time as I try to fix various dependency bugs or other oddities that seemingly came out of nowhere (because the summertime stage runs were flawless in execution). -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org 4096R/D25D95E3 2011-03-28 The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between. --Emperor Turhan, Centauri Republic
RE: [gentoo-dev] Figuring out the solution to in-network-sandbox distcc
Hi, Michał Górny wrote: I see two generic approaches possible here: 1. proxying distcc from within the build environment, or 2. moving distcc-spawned processes back to parent's namespace. distcc client/server solution - The most obvious solution to me is to employ a client/server model where a system-wide daemon is running, parsing /etc/distcc/hosts and doing all the network activity. [...] It is not only distcc. Please don't forget things like sys-devel/icecream. -Thomas
[gentoo-portage-dev] [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy
Add a minimal SOCKSv5-over-UNIX-socket proxy to Portage, and start it
whenever ebuilds are started with network-sandbox enabled. Pass the
socket address in PORTAGE_SOCKS5_PROXY and DISTCC_SOCKS_PROXY variables.
The proxy can be used to escape the network sandbox whenever network
access is really desired, e.g. in distcc.
The proxy supports connecting to IPv6 IPv4 TCP hosts. UDP and socket
binding are not supported. SOCKSv5 authentication schemes are not
supported (UNIX sockets provide a security layer).
---
bin/save-ebuild-env.sh | 5 +-
bin/socks5-server.py | 218 +
.../package/ebuild/_config/special_env_vars.py | 2 +-
pym/portage/package/ebuild/doebuild.py | 7 +
pym/portage/util/socks5.py | 45 +
5 files changed, 274 insertions(+), 3 deletions(-)
create mode 100644 bin/socks5-server.py
create mode 100644 pym/portage/util/socks5.py
diff --git a/bin/save-ebuild-env.sh b/bin/save-ebuild-env.sh
index c6bffb5..477ed28 100644
--- a/bin/save-ebuild-env.sh
+++ b/bin/save-ebuild-env.sh
@@ -92,7 +92,7 @@ __save_ebuild_env() {
# portage config variables and variables set directly by portage
unset ACCEPT_LICENSE BAD BRACKET BUILD_PREFIX COLS \
- DISTCC_DIR DISTDIR DOC_SYMLINKS_DIR \
+ DISTCC_DIR DISTCC_SOCKS5_PROXY DISTDIR DOC_SYMLINKS_DIR \
EBUILD_FORCE_TEST EBUILD_MASTER_PID \
ECLASS_DEPTH ENDCOL FAKEROOTKEY \
GOOD HILITE HOME \
@@ -105,7 +105,8 @@ __save_ebuild_env() {
PORTAGE_DOHTML_WARN_ON_SKIPPED_FILES \
PORTAGE_NONFATAL PORTAGE_QUIET \
PORTAGE_SANDBOX_DENY PORTAGE_SANDBOX_PREDICT \
- PORTAGE_SANDBOX_READ PORTAGE_SANDBOX_WRITE PREROOTPATH \
+ PORTAGE_SANDBOX_READ PORTAGE_SANDBOX_WRITE \
+ PORTAGE_SOCKS5_PROXY PREROOTPATH \
QA_INTERCEPTORS \
RC_DEFAULT_INDENT RC_DOT_PATTERN RC_ENDCOL RC_INDENTATION \
ROOT ROOTPATH RPMDIR TEMP TMP TMPDIR USE_EXPAND \
diff --git a/bin/socks5-server.py b/bin/socks5-server.py
new file mode 100644
index 000..c079018
--- /dev/null
+++ b/bin/socks5-server.py
@@ -0,0 +1,218 @@
+#!/usr/bin/env python
+# SOCKSv5 proxy server for network-sandbox
+# Copyright 2015 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import asyncore
+import errno
+import socket
+import struct
+import sys
+
+
+class ProxyConnection(asyncore.dispatcher_with_send):
+ _addr = None
+ _connected = False
+ _family = socket.AF_INET
+ _proxy_conn = None
+
+ def __init__(self, proxy_conn):
+ self._proxy_conn = proxy_conn
+ asyncore.dispatcher_with_send.__init__(self)
+ self.create_socket(self._family, socket.SOCK_STREAM)
+
+ def start_connection(self, host, port):
+ try:
+ self.connect((host, port))
+ except:
+ self.handle_error()
+
+ def handle_read(self):
+ buf = self.recv(4096)
+ self._proxy_conn.send(buf)
+
+ def handle_connect(self):
+ self._connected = True
+ self._proxy_conn.send_connected(self._family,
self.getsockname())
+
+ def handle_close(self):
+ if self._connected:
+ self._proxy_conn.remote_closed()
+
+ def handle_error(self):
+ e, v, tb = sys.exc_info()
+ if isinstance(v, socket.gaierror) or isinstance(v,
socket.herror):
+ self.close()
+ self._proxy_conn.send_failure(self._family,
errno.EHOSTUNREACH)
+ elif isinstance(e, OSError):
+ self.close()
+ self._proxy_conn.send_failure(self._family, v.errno)
+ else:
+ raise
+
+
+class ProxyConnectionV6(ProxyConnection):
+ _family = socket.AF_INET6
+
+
+class ProxyHandler(asyncore.dispatcher_with_send):
+ _my_buf = b''
+ _my_conn = None
+ _my_state = 0
+ _my_addr = None
+
+ def handle_read(self):
+ rd = self.recv(4096)
+
+ self._my_buf += rd
+ if self._my_state == 0: # waiting for hello
+ if len(self._my_buf) = 3:
+ vers, method_no = struct.unpack('!BB',
self._my_buf[:2])
+ if vers != 0x05:
+ self.close()
+ return
+ if len(self._my_buf) = 2 + method_no:
+ for method in
self._my_buf[2:2+method_no]:
+ if method == 0x00:
+ break
+
Re: [gentoo-dev] arm64
On 01/24/15 21:57, Tom Gall wrote: Hi All, This is sort of a CFP in some ways but not quite that formal. I’ve been throttled back on arm64 for a bit as the hardware I’ve had access to has all been painfully remote and configured in ways that was less than optimal for massive key wording efforts. That’s about to change. So if there are others out there who have the same interest be great to coordinate efforts and get this moving along so we have arm64 stages and start to pull together install instructions for the varies pieces of hardware starting to show up. Best, Tom I'll jump in on the arm64 craze. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
[gentoo-portage-dev] [PATCH v3] Support escaping network-sandbox through SOCKSv5 proxy
Add a minimal SOCKSv5-over-UNIX-socket proxy to Portage, and start it
whenever ebuilds are started with network-sandbox enabled. Pass the
socket address in PORTAGE_SOCKS5_PROXY and DISTCC_SOCKS_PROXY variables.
The proxy can be used to escape the network sandbox whenever network
access is really desired, e.g. in distcc.
The proxy supports connecting to IPv6 IPv4 TCP hosts. UDP and socket
binding are not supported. SOCKSv5 authentication schemes are not
supported (UNIX sockets provide a security layer).
---
New in v3:
- added __nonzero__ for py2,
- added BlockingIOError handling w/ py2 compat,
- added unlinking of socket on server exit.
---
bin/save-ebuild-env.sh | 5 +-
bin/socks5-server.py | 233 +
.../package/ebuild/_config/special_env_vars.py | 2 +-
pym/portage/package/ebuild/doebuild.py | 7 +
pym/portage/util/socks5.py | 48 +
5 files changed, 292 insertions(+), 3 deletions(-)
create mode 100644 bin/socks5-server.py
create mode 100644 pym/portage/util/socks5.py
diff --git a/bin/save-ebuild-env.sh b/bin/save-ebuild-env.sh
index c6bffb5..477ed28 100644
--- a/bin/save-ebuild-env.sh
+++ b/bin/save-ebuild-env.sh
@@ -92,7 +92,7 @@ __save_ebuild_env() {
# portage config variables and variables set directly by portage
unset ACCEPT_LICENSE BAD BRACKET BUILD_PREFIX COLS \
- DISTCC_DIR DISTDIR DOC_SYMLINKS_DIR \
+ DISTCC_DIR DISTCC_SOCKS5_PROXY DISTDIR DOC_SYMLINKS_DIR \
EBUILD_FORCE_TEST EBUILD_MASTER_PID \
ECLASS_DEPTH ENDCOL FAKEROOTKEY \
GOOD HILITE HOME \
@@ -105,7 +105,8 @@ __save_ebuild_env() {
PORTAGE_DOHTML_WARN_ON_SKIPPED_FILES \
PORTAGE_NONFATAL PORTAGE_QUIET \
PORTAGE_SANDBOX_DENY PORTAGE_SANDBOX_PREDICT \
- PORTAGE_SANDBOX_READ PORTAGE_SANDBOX_WRITE PREROOTPATH \
+ PORTAGE_SANDBOX_READ PORTAGE_SANDBOX_WRITE \
+ PORTAGE_SOCKS5_PROXY PREROOTPATH \
QA_INTERCEPTORS \
RC_DEFAULT_INDENT RC_DOT_PATTERN RC_ENDCOL RC_INDENTATION \
ROOT ROOTPATH RPMDIR TEMP TMP TMPDIR USE_EXPAND \
diff --git a/bin/socks5-server.py b/bin/socks5-server.py
new file mode 100644
index 000..45cf76b
--- /dev/null
+++ b/bin/socks5-server.py
@@ -0,0 +1,233 @@
+#!/usr/bin/env python
+# SOCKSv5 proxy server for network-sandbox
+# Copyright 2015 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import asyncore
+import errno
+import os
+import socket
+import struct
+import sys
+
+
+if sys.hexversion 0x0300:
+ from io import BlockingIOError
+
+
+class ProxyConnection(asyncore.dispatcher_with_send):
+ _addr = None
+ _connected = False
+ _family = socket.AF_INET
+ _proxy_conn = None
+
+ def __init__(self, proxy_conn):
+ self._proxy_conn = proxy_conn
+ asyncore.dispatcher_with_send.__init__(self)
+ self.create_socket(self._family, socket.SOCK_STREAM)
+
+ def start_connection(self, host, port):
+ try:
+ self.connect((host, port))
+ except:
+ self.handle_error()
+
+ def handle_read(self):
+ try:
+ buf = self.recv(4096)
+ except BlockingIOError:
+ return
+ self._proxy_conn.send(buf)
+
+ def handle_connect(self):
+ self._connected = True
+ self._proxy_conn.send_connected(self._family,
self.getsockname())
+
+ def handle_close(self):
+ if self._connected:
+ self._proxy_conn.remote_closed()
+
+ def handle_error(self):
+ e, v, tb = sys.exc_info()
+ if isinstance(v, socket.gaierror) or isinstance(v,
socket.herror):
+ self.close()
+ self._proxy_conn.send_failure(self._family,
errno.EHOSTUNREACH)
+ elif isinstance(e, OSError):
+ self.close()
+ self._proxy_conn.send_failure(self._family, v.errno)
+ else:
+ raise
+
+
+class ProxyConnectionV6(ProxyConnection):
+ _family = socket.AF_INET6
+
+
+class ProxyHandler(asyncore.dispatcher_with_send):
+ _my_buf = b''
+ _my_conn = None
+ _my_state = 0
+ _my_addr = None
+
+ def handle_read(self):
+ try:
+ rd = self.recv(4096)
+ except BlockingIOError:
+ return
+
+ self._my_buf += rd
+ if self._my_state == 0: # waiting for hello
+ if len(self._my_buf) = 3:
+ vers, method_no = struct.unpack('!BB',
self._my_buf[:2])
+
Re: [gentoo-dev] arm64
On 20:57 Sat 24 Jan , Tom Gall wrote:
Hi All,
This is sort of a CFP in some ways but not quite that formal. I’ve been
throttled back on arm64 for a bit as the hardware I’ve had access to has all
been painfully remote and configured in ways that was less than optimal for
massive key wording efforts.
That’s about to change.
So if there are others out there who have the same interest be great to
coordinate efforts and get this moving along so we have arm64 stages and
start to pull together install instructions for the varies pieces of hardware
starting to show up.
Hi
I've already keyworded a few ebuilds.. but unfortunately the hardware
I'm using now can't be available for end user
the arm64 profile is still experimental.. massive USEs have been masked [1]
tests/verification are required.
[1] ${PORTDIR}/profiles/arch/arm64/use.mask
--
Yixun Lan (dlan)
Gentoo Linux Developer
GPG Key ID AABEFD55
Re: [gentoo-dev] arm64
Least speaking for myself I can help you out starting Feb 15th, presuming all the stars are in alignment. If someone else doesn’t help you before, please mark it on your calendar and bug me again then cause I’m sure I’ll forget! Best, Tom On Jan 25, 2015, at 11:43 AM, Sebastian Pipping sp...@gentoo.org wrote: Hi! I got a bug report for arm64 against the test suite of uriparser. If I could get a temporary arm64 shell somewhere, that could help me understand the issue. Best, Sebastian
Re: [gentoo-dev] arm64
Hi! I got a bug report for arm64 against the test suite of uriparser. If I could get a temporary arm64 shell somewhere, that could help me understand the issue. Best, Sebastian
Re: [gentoo-portage-dev] [PATCH v3] Support escaping network-sandbox through SOCKSv5 proxy
On Sun, 25 Jan 2015 17:23:38 -0800 Zac Medico zmed...@gentoo.org wrote: On 01/25/2015 02:34 PM, Michał Górny wrote: diff --git a/bin/socks5-server.py b/bin/socks5-server.py new file mode 100644 index 000..45cf76b --- /dev/null +++ b/bin/socks5-server.py @@ -0,0 +1,233 @@ +#!/usr/bin/env python +# SOCKSv5 proxy server for network-sandbox +# Copyright 2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +import asyncore +import errno +import os +import socket +import struct +import sys + + +if sys.hexversion 0x0300: + from io import BlockingIOError + + +class ProxyConnection(asyncore.dispatcher_with_send): + _addr = None + _connected = False + _family = socket.AF_INET + _proxy_conn = None You've defined these as class variables, but they should be instance variables (initialized in the constructor). Since the class is a singleton, it works either way, but it's poor style to use class variables like this. +class ProxyHandler(asyncore.dispatcher_with_send): + _my_buf = b'' + _my_conn = None + _my_state = 0 + _my_addr = None These class variables should also be changed to instance variables. Yeah, I meant to ask you the same... Do they need to be class wide (multiple instances refer to the same exact variables) or just instance wide ones. But I hadn't looked it over in enough detail yet. And LOOSE the _my prefix. -- Brian Dolbec dolsen
Re: [gentoo-dev] Re: Things one could be upset about
On Sun, 25 Jan 2015 04:29:43 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: Alexey Mishustin posted on Sat, 24 Jan 2015 21:54:06 +0400 as excerpted: 2015-01-20 14:42 GMT+04:00 Róbert Čerňanský ope...@tightmail.com: I somehow thought that edit the overgrowing package.use file upon each emerge world annoys anyone the same as me. But for me this is one of the most useful and convenient options in Gentoo. Yes, I do edit package.use almost every emerge world. And I like to do it. And I don't want to delegate this right to any program - portage, or any other. Agreed that I don't want to (and won't) delegate that decision, but almost every emerge world? Not here. So ??? I do edit package.use (or my global USE flags) occasionally, but not as often as the above implies. What might I be doing different? Well, here's what I do: 1) I try to sync and update deep newuse @world once a week, tho sometimes it's every two weeks (but sometimes it's daily). I suppose if people only get to it every couple months, they'll have more [...] So maybe it's simply that I update frequently enough, tho I /do/ run ~arch as well, which changes much faster than stable, and I even run More frequent updates is most likely the reason that you do not have to edit use flags every time. Running testing perhaps does not increase the editing frequency because dependency changes are the same regardles of how many bumps a package has. For example in testing you'll get following updates of package foo: foo-1.1, ~foo-1.2, ~foo-1.3, foo-1.4. In stable, I would get: foo-1.1, foo-1.4. If dependency changes in 1.3, both of us could have to edit USE flags once. I update every 2-4 months (only GLSAs in between) therefore my experience is that I have to edit it every time (not for GLSAs of course). 2) My global USE= starts with -*. I got tired of worrying about what [...] 3) I don't normally distinguish between local and global USE flags. I normally treat them all as global and set them globally in my make.conf use file[1]. When I encounter a new USE flag, because of the -* in USE, it's off by default. If I decide I want it off, no problem, it's off. If I decide I want it on, I run an equery hasuse flag to see if any other package uses it. If nothing else uses it, [...] Similarly, if I encounter a new USE flag that's on already, obviously some other package I use is already using it and the entry is in my use file or it wouldn't be on already, due to the -* in that use file. That's a strong hint what I'm likely to want the default to be. If I decide I want it off anyway, I run an equery hasuse flag [...] So for all flags, if I want the default off, due to the -* in my use file, it's off. If I want the default on, it's in my use file, turning it on. 4) The result is that my package.use files contain ONLY non-default entries. More or less same here, except -* as the default. I trust developers that they are choosing wisely in profile and ebuilds. ;-) If not, I see it in emerge -av output anyway and can disable/enable particular flag. But generally I have vast majority of flags in make.conf and exceptions in package.use. When I set such an entry, I prefix a comment line with the date and an explanation of WHY the entry needs to be there, different from my normal default settings. Sometimes, it's due to a bug, and a couple versions later I can go thru and test with that entry commented, to see if the bug is fixed, yet. Other times it's due to a use-dep from some other package I have installed. Both qt and kde tend to have This where we get to the point. If you set USE flag because of a bug or because of dependency it is not because you *want to* but because you *have to* (or portage *needs to*). Therefore you need to add a comment WHY you did it. For example I have -doc in make.conf because I do *not want* docs globally. But I have 'dev-lang/python doc' in package.use because I develop in Python and *want* the documentation for it. Such entry does not need a comment, because I simply know what I want. Another example. I have -python globally and have been using app-mobilephone/gammu. Now I want to emerge app-mobilephone/wammu. But it needs +python for gammu, so I *have to* set 'app-mobilephone/gammu python' in package.use. In this case I also add a comment which explains the reason because it is not what *I want* it is what *portage needs*. Once this dependency is gone (either because wammu stops requiring it or I unmerge it) it is on me to detect it and remove the entry. So effectively I manage portage's stuff in my configuration file. Regardless of why it's there, however, because only non-default entries are in package.use, they're the obvious exception. You somehow do not distinguish between those two types of exceptions explained in examples above. And as exceptions, they don't tend to change that often. =:^) Generally, They might change as
Re: [gentoo-dev] Figuring out the solution to in-network-sandbox distcc
Dnia 2015-01-21, o godz. 11:05:34 Michał Górny mgo...@gentoo.org napisał(a): Generic proxy solution -- The simplest solution so far seems to be setting a generic SOCKS proxy inside the build environment, and wrapping distcc so that it will use it for network access. Unless we do some extra magic which don't want to do, this means that other apps can also abuse the proxy to reach outside sandbox. However, network-sandbox is not really a security feature, so I don't think that is important. At least as long as we don't export it globally :). Of course, software is a problem. We'd need at least some SOCKS server for Portage (at least a very simple one), and as far as I'm aware distcc does not support SOCKS directly, so tsocks in addition to that. So finally went this way instead. I've implemented a simple SOCKSv5 server over UNIX sockets [1] and wrote a patch adding SOCKSv5 support to distcc [2,3]. With the two patches, everything works perfectly for me :). [1]:http://article.gmane.org/gmane.linux.gentoo.portage.devel/5142 [2]:https://code.google.com/p/distcc/issues/detail?id=149 [3]:https://bugs.gentoo.org/show_bug.cgi?id=537616 -- Best regards, Michał Górny pgpuy8pRvnWQr.pgp Description: OpenPGP digital signature
[gentoo-portage-dev] [PATCH] SOCKSv5: report bound socket name
Report bound socket name as requested by the protocol. Supports both
IPv4 and IPv6 sockets.
---
bin/socks5-server.py | 21 +
1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/bin/socks5-server.py b/bin/socks5-server.py
index 4795dcc..78a6751 100644
--- a/bin/socks5-server.py
+++ b/bin/socks5-server.py
@@ -11,6 +11,7 @@ import sys
class ProxyConnection(asyncore.dispatcher_with_send):
+ _family = None
_proxy_conn = None
def __init__(self, host, port, proxy_conn):
@@ -18,6 +19,7 @@ class ProxyConnection(asyncore.dispatcher_with_send):
asyncore.dispatcher_with_send.__init__(self)
# TODO: how to support IPv6? ugly fail-then-reinit?
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+ self._family = socket.AF_INET
self.connect((host, port))
def handle_read(self):
@@ -25,7 +27,7 @@ class ProxyConnection(asyncore.dispatcher_with_send):
self._proxy_conn.send(buf)
def handle_connect(self):
- self._proxy_conn.send_connected()
+ self._proxy_conn.send_connected(self._family,
self.getsockname())
def handle_close(self):
self._proxy_conn.close()
@@ -126,9 +128,20 @@ class ProxyHandler(asyncore.dispatcher_with_send):
if self._my_conn is not None:
self._my_conn.close()
- def send_connected(self):
- repl = struct.pack('!LH', 0x05, 0x00, 0x00, 0x01,
- 0x, 0x)
+ def send_connected(self, family, addr):
+ if family == socket.AF_INET:
+ host, port = addr
+ bin_host = socket.inet_aton(host)
+
+ repl = struct.pack('!4sH', 0x05, 0x00, 0x00, 0x01,
+ bin_host, port)
+ elif family == socket.AF_INET6:
+ host, port = addr
+ bin_host = socket.inet_pton(family, host)
+
+ repl = struct.pack('!16sH', 0x05, 0x00, 0x00, 0x04,
+ bin_host, port)
+
self.send(repl)
self._my_state = 3
--
2.2.2
Re: [gentoo-portage-dev] [PATCH] Support escaping network-sandbox through SOCKSv5 proxy
Dnia 2015-01-25, o godz. 12:29:54 Michał Górny mgo...@gentoo.org napisał(a): Add a minimal SOCKSv5-over-UNIX-socket proxy to Portage, and start it whenever ebuilds are started with network-sandbox enabled. Pass the socket address in PORTAGE_SOCKS5_PROXY and DISTCC_SOCKS_PROXY variables. The proxy can be used to escape the network sandbox whenever network access is really desired, e.g. in distcc. The proxy currently supports IPv4 only, and does not report bound address (reports 0.0.0.0:0). No authentication is supported (UNIX sockets provide a security layer). Resubmitted with a number of fixes as: [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy -- Best regards, Michał Górny pgpPYFbov0PUJ.pgp Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH] SOCKSv5: report bound socket name
Dnia 2015-01-25, o godz. 13:19:37 Michał Górny mgo...@gentoo.org napisał(a): Report bound socket name as requested by the protocol. Supports both IPv4 and IPv6 sockets. Now included (with fixes) in: [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy -- Best regards, Michał Górny pgpJdlrhXUsla.pgp Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy
On 01/25/2015 06:00 AM, Michał Górny wrote: diff --git a/bin/socks5-server.py b/bin/socks5-server.py new file mode 100644 index 000..c079018 --- /dev/null +++ b/bin/socks5-server.py @@ -0,0 +1,218 @@ +#!/usr/bin/env python +# SOCKSv5 proxy server for network-sandbox +# Copyright 2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +import asyncore +import errno +import socket +import struct +import sys + + +class ProxyConnection(asyncore.dispatcher_with_send): + _addr = None + _connected = False + _family = socket.AF_INET + _proxy_conn = None + + def __init__(self, proxy_conn): + self._proxy_conn = proxy_conn + asyncore.dispatcher_with_send.__init__(self) + self.create_socket(self._family, socket.SOCK_STREAM) + + def start_connection(self, host, port): + try: + self.connect((host, port)) + except: + self.handle_error() This except handler should at least allow SystemExit and KeyboardInterrupt to raise. diff --git a/pym/portage/package/ebuild/_config/special_env_vars.py b/pym/portage/package/ebuild/_config/special_env_vars.py index 6bb3c95..905d5e7 100644 --- a/pym/portage/package/ebuild/_config/special_env_vars.py +++ b/pym/portage/package/ebuild/_config/special_env_vars.py @@ -71,7 +71,7 @@ environ_whitelist += [ PORTAGE_PYM_PATH, PORTAGE_PYTHON, PORTAGE_PYTHONPATH, PORTAGE_QUIET, PORTAGE_REPO_NAME, PORTAGE_REPOSITORIES, PORTAGE_RESTRICT, - PORTAGE_SIGPIPE_STATUS, + PORTAGE_SIGPIPE_STATUS, PORTAGE_SOCKS5_PROXY, PORTAGE_TMPDIR, PORTAGE_UPDATE_ENV, PORTAGE_USERNAME, PORTAGE_VERBOSE, PORTAGE_WORKDIR_MODE, PORTAGE_XATTR_EXCLUDE, PORTDIR, PORTDIR_OVERLAY, PREROOTPATH, The DISTCC_SOCKS_PROXY variable should also be added to the whitelist. Other than these 2 minor issues, the patch looks to me. I guess there's no point in using portage's event loop instead of asyncore, since we want the proxy to drop privileges, and therefore it can't run in the main portage process. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy
On 01/25/2015 06:00 AM, Michał Górny wrote: diff --git a/bin/socks5-server.py b/bin/socks5-server.py new file mode 100644 index 000..c079018 --- /dev/null +++ b/bin/socks5-server.py @@ -0,0 +1,218 @@ +#!/usr/bin/env python +# SOCKSv5 proxy server for network-sandbox +# Copyright 2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +import asyncore +import errno +import socket +import struct +import sys + + +class ProxyConnection(asyncore.dispatcher_with_send): + _addr = None + _connected = False + _family = socket.AF_INET + _proxy_conn = None + + def __init__(self, proxy_conn): + self._proxy_conn = proxy_conn + asyncore.dispatcher_with_send.__init__(self) + self.create_socket(self._family, socket.SOCK_STREAM) + + def start_connection(self, host, port): + try: + self.connect((host, port)) + except: + self.handle_error() + + def handle_read(self): + buf = self.recv(4096) + self._proxy_conn.send(buf) The self.recv calls should probably handle BlockingIOError, since the docs say it can be raised even though select.select() or select.poll() has reported the socket ready for reading. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy
On 01/25/2015 06:00 AM, Michał Górny wrote: + def __bool__(self):a + return self.socket_path is not None You should also implement __nonzero__ for python-2.x. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH v2] Support escaping network-sandbox through SOCKSv5 proxy
Dnia 2015-01-25, o godz. 13:43:14 Zac Medico zmed...@gentoo.org napisał(a): On 01/25/2015 06:00 AM, Michał Górny wrote: diff --git a/bin/socks5-server.py b/bin/socks5-server.py new file mode 100644 index 000..c079018 --- /dev/null +++ b/bin/socks5-server.py @@ -0,0 +1,218 @@ +#!/usr/bin/env python +# SOCKSv5 proxy server for network-sandbox +# Copyright 2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +import asyncore +import errno +import socket +import struct +import sys + + +class ProxyConnection(asyncore.dispatcher_with_send): + _addr = None + _connected = False + _family = socket.AF_INET + _proxy_conn = None + + def __init__(self, proxy_conn): + self._proxy_conn = proxy_conn + asyncore.dispatcher_with_send.__init__(self) + self.create_socket(self._family, socket.SOCK_STREAM) + + def start_connection(self, host, port): + try: + self.connect((host, port)) + except: + self.handle_error() This except handler should at least allow SystemExit and KeyboardInterrupt to raise. handle_error() has conditional exception reraising code. diff --git a/pym/portage/package/ebuild/_config/special_env_vars.py b/pym/portage/package/ebuild/_config/special_env_vars.py index 6bb3c95..905d5e7 100644 --- a/pym/portage/package/ebuild/_config/special_env_vars.py +++ b/pym/portage/package/ebuild/_config/special_env_vars.py @@ -71,7 +71,7 @@ environ_whitelist += [ PORTAGE_PYM_PATH, PORTAGE_PYTHON, PORTAGE_PYTHONPATH, PORTAGE_QUIET, PORTAGE_REPO_NAME, PORTAGE_REPOSITORIES, PORTAGE_RESTRICT, - PORTAGE_SIGPIPE_STATUS, + PORTAGE_SIGPIPE_STATUS, PORTAGE_SOCKS5_PROXY, PORTAGE_TMPDIR, PORTAGE_UPDATE_ENV, PORTAGE_USERNAME, PORTAGE_VERBOSE, PORTAGE_WORKDIR_MODE, PORTAGE_XATTR_EXCLUDE, PORTDIR, PORTDIR_OVERLAY, PREROOTPATH, The DISTCC_SOCKS_PROXY variable should also be added to the whitelist. There's a regexp for DISTCC_* below. Other than these 2 minor issues, the patch looks to me. I guess there's no point in using portage's event loop instead of asyncore, since we want the proxy to drop privileges, and therefore it can't run in the main portage process. To be honest, I didn't even think about it. Asyncore seemed like the Python way of doing non-blocking socket I/O. -- Best regards, Michał Górny pgpVHjkCecvHb.pgp Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH v3] Support escaping network-sandbox through SOCKSv5 proxy
On 01/25/2015 02:34 PM, Michał Górny wrote: diff --git a/bin/socks5-server.py b/bin/socks5-server.py new file mode 100644 index 000..45cf76b --- /dev/null +++ b/bin/socks5-server.py @@ -0,0 +1,233 @@ +#!/usr/bin/env python +# SOCKSv5 proxy server for network-sandbox +# Copyright 2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +import asyncore +import errno +import os +import socket +import struct +import sys + + +if sys.hexversion 0x0300: + from io import BlockingIOError + + +class ProxyConnection(asyncore.dispatcher_with_send): + _addr = None + _connected = False + _family = socket.AF_INET + _proxy_conn = None You've defined these as class variables, but they should be instance variables (initialized in the constructor). Since the class is a singleton, it works either way, but it's poor style to use class variables like this. +class ProxyHandler(asyncore.dispatcher_with_send): + _my_buf = b'' + _my_conn = None + _my_state = 0 + _my_addr = None These class variables should also be changed to instance variables. -- Thanks, Zac
[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2015-01-25 23:59 UTC
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2015-01-25 23:59 UTC. Removals: dev-ruby/tmail 2015-01-21 05:05:30 mrueg dev-ruby/refe 2015-01-21 05:05:57 mrueg dev-ruby/mysql-ruby 2015-01-21 05:06:19 mrueg dev-ruby/gem_plugin 2015-01-21 05:06:43 mrueg dev-ruby/directory_watcher 2015-01-21 05:07:06 mrueg dev-ruby/awesome_nested_set 2015-01-21 05:07:35 mrueg Additions: net-analyzer/apinger2015-01-19 19:26:14 jer dev-lang/go-bootstrap 2015-01-20 03:39:35 williamh media-plugins/vdr-satip 2015-01-20 11:40:00 hd_brummy dev-perl/Data-Types 2015-01-20 13:21:40 chainsaw dev-perl/DateTime-Tiny 2015-01-20 13:30:09 chainsaw dev-perl/MongoDB2015-01-20 14:32:34 chainsaw dev-python/paramunittest2015-01-21 23:04:03 alunduil dev-python/mando2015-01-21 23:07:06 alunduil dev-python/radon2015-01-21 23:09:00 alunduil sci-geosciences/opencpn-plugin-br24radar2015-01-21 23:19:37 mschiff sci-geosciences/opencpn-plugin-climatology 2015-01-21 23:26:57 mschiff sci-geosciences/opencpn-plugin-launcher 2015-01-21 23:27:29 mschiff sci-geosciences/opencpn-plugin-logbookkonni 2015-01-21 23:28:01 mschiff sci-geosciences/opencpn-plugin-objsearch2015-01-21 23:28:38 mschiff sci-geosciences/opencpn-plugin-ocpndebugger 2015-01-21 23:29:16 mschiff sci-geosciences/opencpn-plugin-statusbar2015-01-21 23:29:52 mschiff sci-geosciences/opencpn-plugin-weatherfax 2015-01-21 23:30:27 mschiff sci-geosciences/opencpn-plugin-weather_routing 2015-01-21 23:31:05 mschiff sci-geosciences/opencpn-plugin-wmm 2015-01-21 23:31:38 mschiff dev-python/elasticsearch-py 2015-01-22 14:54:13 vapier dev-php/ming-php2015-01-22 17:25:49 grknight app-portage/cpuinfo2cpuflags2015-01-23 23:09:54 mgorny dev-ruby/spy2015-01-24 22:36:31 mrueg dev-ruby/power_assert 2015-01-25 07:28:30 graaff dev-ruby/vcr2015-01-25 09:39:27 graaff -- Robin Hugh Johnson Gentoo Linux Developer E-Mail : robb...@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 Removed Packages: dev-ruby/tmail,removed,mrueg,2015-01-21 05:05:30 dev-ruby/refe,removed,mrueg,2015-01-21 05:05:57 dev-ruby/mysql-ruby,removed,mrueg,2015-01-21 05:06:19 dev-ruby/gem_plugin,removed,mrueg,2015-01-21 05:06:43 dev-ruby/directory_watcher,removed,mrueg,2015-01-21 05:07:06 dev-ruby/awesome_nested_set,removed,mrueg,2015-01-21 05:07:35 Added Packages: net-analyzer/apinger,added,jer,2015-01-19 19:26:14 dev-lang/go-bootstrap,added,williamh,2015-01-20 03:39:35 media-plugins/vdr-satip,added,hd_brummy,2015-01-20 11:40:00 dev-perl/Data-Types,added,chainsaw,2015-01-20 13:21:40 dev-perl/DateTime-Tiny,added,chainsaw,2015-01-20 13:30:09 dev-perl/MongoDB,added,chainsaw,2015-01-20 14:32:34 dev-python/paramunittest,added,alunduil,2015-01-21 23:04:03 dev-python/mando,added,alunduil,2015-01-21 23:07:06 dev-python/radon,added,alunduil,2015-01-21 23:09:00 sci-geosciences/opencpn-plugin-br24radar,added,mschiff,2015-01-21 23:19:37 sci-geosciences/opencpn-plugin-climatology,added,mschiff,2015-01-21 23:26:57 sci-geosciences/opencpn-plugin-launcher,added,mschiff,2015-01-21 23:27:29 sci-geosciences/opencpn-plugin-logbookkonni,added,mschiff,2015-01-21 23:28:01 sci-geosciences/opencpn-plugin-objsearch,added,mschiff,2015-01-21 23:28:38 sci-geosciences/opencpn-plugin-ocpndebugger,added,mschiff,2015-01-21 23:29:16 sci-geosciences/opencpn-plugin-statusbar,added,mschiff,2015-01-21 23:29:52 sci-geosciences/opencpn-plugin-weatherfax,added,mschiff,2015-01-21 23:30:27 sci-geosciences/opencpn-plugin-weather_routing,added,mschiff,2015-01-21 23:31:05 sci-geosciences/opencpn-plugin-wmm,added,mschiff,2015-01-21 23:31:38 dev-python/elasticsearch-py,added,vapier,2015-01-22 14:54:13 dev-php/ming-php,added,grknight,2015-01-22 17:25:49 app-portage/cpuinfo2cpuflags,added,mgorny,2015-01-23 23:09:54 dev-ruby/spy,added,mrueg,2015-01-24 22:36:31 dev-ruby/power_assert,added,graaff,2015-01-25 07:28:30 dev-ruby/vcr,added,graaff,2015-01-25 09:39:27 Done.
Re: [gentoo-portage-dev] [PATCH] Allow IPC networking during pkg_config pkg_info
On 01/25/2015 03:42 AM, Michał Górny wrote: --- pym/portage/package/ebuild/doebuild.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pym/portage/package/ebuild/doebuild.py b/pym/portage/package/ebuild/doebuild.py index 0d71f01..050f6c4 100644 --- a/pym/portage/package/ebuild/doebuild.py +++ b/pym/portage/package/ebuild/doebuild.py @@ -86,7 +86,7 @@ _unsandboxed_phases = frozenset([ # phases in which IPC with host is allowed _ipc_phases = frozenset([ - setup, pretend, + setup, pretend, config, info, preinst, postinst, prerm, postrm, ]) LGTM. -- Thanks, Zac
