Re: How to get debian ci test passed for proxy application

[Roger Shimizu]

Dear Antonio,

Thanks for your informative email!
With the upload last night, I confirm now the issue got solved [0].

[0] 
https://ci.debian.net/data/autopkgtest/unstable/amd64/s/shadowsocks-libev/20171212_175154/log.gz

On Tue, Dec 5, 2017 at 3:14 AM, Antonio Terceiro  wrote:
>
> You do not need anything Restrictions: to be able to start a daemon or
> listen on a local TCP port.

Thanks for your confirmation!
Now I removed the Restrictions for isolation-*.

> | $ ./tests/test.sh
> | running test:  python tests/test.py -c tests/aes.json
> |  2017-12-04 16:07:02 INFO: UDP relay enabled
> |  2017-12-04 16:07:02 INFO: initializing ciphers... aes-256-cfb
> |  2017-12-04 16:07:02 ERROR: bind: Address already in use
> |  2017-12-04 16:07:02 ERROR: bind() error
> |  2017-12-04 16:07:02 INFO: initializing ciphers... aes-256-cfb
> |  2017-12-04 16:07:02 INFO: listening at 127.0.0.1:1081
> |  2017-12-04 16:07:02 INFO: initializing ciphers... aes-256-cfb
> |  2017-12-04 16:07:02 INFO: UDP relay enabled
> |  2017-12-04 16:07:02 INFO: listening at 127.0.0.1:1082
>
> 
>
> | *   Trying 127.0.0.1...
> | * TCP_NODELAY set
> |   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
> |  Dload  Upload   Total   SpentLeft  
> Speed
> |   0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
>  0* SOCKS5 communication to www.google.com:80
> |  2017-12-04 16:07:04 INFO: connect to www.google.com:80
> | * SOCKS5 request granted.
> | * Connected to 127.0.0.1 (127.0.0.1) port 1081 (#0)
> 
> | > GET / HTTP/1.1
> | > Host: www.google.com
> | > User-Agent: curl/7.57.0
> | > Accept: */*
> | >
> |  2017-12-04 16:07:04 ERROR: remote_recv_cb_recv: Connection reset by peer
> | * Empty reply from server
> |   0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
>  0
> | * Connection #0 to host 127.0.0.1 left intact
> | curl: (52) Empty reply from server
> |  2017-12-04 16:07:04 INFO: closed gracefully
> | test failed:  python tests/test.py -c tests/aes.json
>
> Why this happens? Because on autopkgtest, you assume the package is already
> *installed*. I assume the the daemon provided by the binary package is already
> listening to port 1081, so the test server starts on 1082.

Thanks for your hint!
Yes, previous failure on debci was because of installation of the
package, with the default config, so with the default port to open.

However the test opens both 1081 and 1082, which is expected.
The root cause is the application opens more than 1 port, and 8388 is
both listed in default config [1] and test config.
So here's the conflict.
After I add a patch [2] to change the port for the test from 8388 to
8389, now the test can be passed on debci [0].

[1] 
https://anonscm.debian.org/cgit/collab-maint/shadowsocks-libev.git/tree/debian/config.json
[2] 
https://anonscm.debian.org/cgit/collab-maint/shadowsocks-libev.git/tree/debian/patches/0001-Change-the-port-to-listen-from-8388-to-8389.patch

I'm happy that one more package is starting to enjoy the benefit of
continuous integration practice in debian.
It's really appreciated there's debci framework in debian.
Thanks for your work!

Cheers,
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: How to get debian ci test passed for proxy application

[Antonio Terceiro]

On Mon, Dec 04, 2017 at 04:59:12PM +0100, gregor herrmann wrote:
> On Mon, 04 Dec 2017 22:45:21 +0900, Roger Shimizu wrote:
> 
> > On Mon, Nov 27, 2017 at 5:36 AM, gregor herrmann  wrote:
> > > On Sun, 26 Nov 2017 18:42:22 +, James Cowgill wrote:
> > >> I think you might need a "Restrictions: isolation-container" to get
> > >> network access, but that's only a guess.
> > > That's not my experience. We have quite a few perl packages where the
> > > tests do something networky and haven't experienced problems on
> > > ci.debian.net (modulo failing requests to external resources but
> > > that's of course a different story).
> > Maybe out bound network activity is OK, but not for open tcp port to
> > listen, as James said.
> 
> Sorry for being not more clear in my first mail: we also start all
> kinds of daemons, either from the package itself or some other
> packages (mysql, mariadb, memcached, …), and the tests successfully
> connect to them.

You do not need anything Restrictions: to be able to start a daemon or
listen on a local TCP port.

I tried the package locally both under autopkgtest+lxc, and on my host
system, and got the same results as the ones in ci.debian.net, i.e.
stuff like

running test:  python tests/test.py -c tests/chacha20-ietf-poly1305.json
*   Trying 127.0.0.1...
* TCP_NODELAY set
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0* 
SOCKS5 communication to www.google.com:80
* SOCKS5 request granted.
* Connected to 127.0.0.1 (127.0.0.1) port 1081 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/7.57.0
> Accept: */*
>
* Empty reply from server
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host 127.0.0.1 left intact
curl: (52) Empty reply from server
test failed:  python tests/test.py -c tests/chacha20-ietf-poly1305.json


I tried messing with tests/test.py like this:

diff --git a/tests/test.py b/tests/test.py
index 0a1297b..ec8f3e3 100755
--- a/tests/test.py
+++ b/tests/test.py
@@ -60,9 +60,9 @@ if config.client_args:
 else:
 server_args.extend(config.client_args.split())

-p1 = Popen(server_args, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
-p2 = Popen(client_args, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
-p5 = Popen(tunnel_args, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
+p1 = Popen(server_args, stdin=PIPE, close_fds=True)
+p2 = Popen(client_args, stdin=PIPE, close_fds=True)
+p5 = Popen(tunnel_args, stdin=PIPE, close_fds=True)
 p3 = None
 p4 = None
 p3_fin = False

and discovered that the test server is not listening to the port you think it 
is:

| $ ./tests/test.sh 
| running test:  python tests/test.py -c tests/aes.json
|  2017-12-04 16:07:02 INFO: UDP relay enabled
|  2017-12-04 16:07:02 INFO: initializing ciphers... aes-256-cfb
|  2017-12-04 16:07:02 ERROR: bind: Address already in use
|  2017-12-04 16:07:02 ERROR: bind() error
|  2017-12-04 16:07:02 INFO: initializing ciphers... aes-256-cfb
|  2017-12-04 16:07:02 INFO: listening at 127.0.0.1:1081
|  2017-12-04 16:07:02 INFO: initializing ciphers... aes-256-cfb
|  2017-12-04 16:07:02 INFO: UDP relay enabled
|  2017-12-04 16:07:02 INFO: listening at 127.0.0.1:1082



| *   Trying 127.0.0.1...
| * TCP_NODELAY set
|   % Total% Received % Xferd  Average Speed   TimeTime Time  
Current
|  Dload  Upload   Total   SpentLeft  Speed
|   0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0* SOCKS5 communication to www.google.com:80
|  2017-12-04 16:07:04 INFO: connect to www.google.com:80
| * SOCKS5 request granted.
| * Connected to 127.0.0.1 (127.0.0.1) port 1081 (#0)

| > GET / HTTP/1.1
| > Host: www.google.com
| > User-Agent: curl/7.57.0
| > Accept: */*
| > 
|  2017-12-04 16:07:04 ERROR: remote_recv_cb_recv: Connection reset by peer
| * Empty reply from server
|   0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
| * Connection #0 to host 127.0.0.1 left intact
| curl: (52) Empty reply from server
|  2017-12-04 16:07:04 INFO: closed gracefully
| test failed:  python tests/test.py -c tests/aes.json

Why this happens? Because on autopkgtest, you assume the package is already
*installed*. I assume the the daemon provided by the binary package is already
listening to port 1081, so the test server starts on 1082.

In this case, I think you want connect to the daemon that is already running,
i.e. the one that is provided by the binary package, instead of some daemon
that is started by the test themselves.

the log from
http://debomatic-amd64.debian.net/distribution#unstable/shadowsocks-libev/3.1.1+ds-1/autopkgtest
seems 

Re: How to get debian ci test passed for proxy application

[gregor herrmann]

On Mon, 04 Dec 2017 22:45:21 +0900, Roger Shimizu wrote:

> On Mon, Nov 27, 2017 at 5:36 AM, gregor herrmann  wrote:
> > On Sun, 26 Nov 2017 18:42:22 +, James Cowgill wrote:
> >> I think you might need a "Restrictions: isolation-container" to get
> >> network access, but that's only a guess.
> > That's not my experience. We have quite a few perl packages where the
> > tests do something networky and haven't experienced problems on
> > ci.debian.net (modulo failing requests to external resources but
> > that's of course a different story).
> Maybe out bound network activity is OK, but not for open tcp port to
> listen, as James said.

Sorry for being not more clear in my first mail: we also start all
kinds of daemons, either from the package itself or some other
packages (mysql, mariadb, memcached, …), and the tests successfully
connect to them.
 
Maybe Antonio as the ci.d.n guru and maintainer can give an
authoritative answer :)


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: U2: Kite


signature.asc
Description: Digital Signature

Re: How to get debian ci test passed for proxy application

[Roger Shimizu]

Thanks all for your reply!

On Mon, Nov 27, 2017 at 3:42 AM, James Cowgill  wrote:
> Hi,
>
> On 26/11/17 17:00, Roger Shimizu wrote:
>
>> The last test.sh script invokes the test, which creates local proxy
>> listen to 127.0.0.1:1081, and then it calls curl command to get index
>> page of google via local proxy, 127.0.0.1:1081.
>>
>> My local test shows all pass, while debian ci test [1] shows a
>> connection timeout message.
>> So I'm wondering whether debian ci support network activity, and how
>> can I configure the test to get it passed.
>
> I think you might need a "Restrictions: isolation-container" to get
> network access, but that's only a guess.

Thanks for the hint!

I find a supporting document [2], which state this flag is to allow
open network TCP ports.

[2] https://people.debian.org/~mpitt/autopkgtest/README.package-tests.html

However, I tried "Restrictions: isolation-container", but it still failed [3]

[3] 
https://ci.debian.net/data/autopkgtest/unstable/amd64/s/shadowsocks-libev/20171201_180744/log.gz

So I'll try to use "Restrictions: isolation-machine" next.


On Mon, Nov 27, 2017 at 5:36 AM, gregor herrmann  wrote:
> On Sun, 26 Nov 2017 18:42:22 +, James Cowgill wrote:
>
>> > My local test shows all pass, while debian ci test [1] shows a
>> > connection timeout message.
>> > So I'm wondering whether debian ci support network activity, and how
>> > can I configure the test to get it passed.
>> I think you might need a "Restrictions: isolation-container" to get
>> network access, but that's only a guess.
>
> That's not my experience. We have quite a few perl packages where the
> tests do something networky and haven't experienced problems on
> ci.debian.net (modulo failing requests to external resources but
> that's of course a different story).

Maybe out bound network activity is OK, but not for open tcp port to
listen, as James said.


On Wed, Nov 29, 2017 at 5:45 PM, gustavo panizzo  wrote:
>
> I'd suggest to install apache as part of the tests and connect to
> localhost:80, that way it always works even if ci.debian.net moves to
> China or google goes down
>
> to test python-openstackclient; MySQL, RabbitMQ, Apache and others are
> installed, I haven't check its tests in a while but happy to help you

Thanks for your idea!
Maybe it'a a last resort, but I want to avoid currently.
It's too big for just a simple test.

And actually the package I'm try to make the test to work is helping
people to fight with censorship in mid- or far- east area.
So your example is not so proper IMHO.

Cheers,
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: How to get debian ci test passed for proxy application

[gustavo panizzo]

Hello

On Mon, Nov 27, 2017 at 02:00:00AM +0900, Roger Shimizu wrote:

Dear mentors list,

I maintain a proxy application, shadowsocks-libev.
I want to let it pass debian ci test. And I already confirm the test
all passed on my local environment, and debomatic [0].
However it failed on debian ci infrastructure [1].

[0] 
http://debomatic-amd64.debian.net/distribution#unstable/shadowsocks-libev/3.1.1+ds-1/autopkgtest
[1] https://ci.debian.net/packages/s/shadowsocks-libev/unstable/amd64/

For local test, it just need the commands below:

$ sudo apt install shadowsocks-libev curl dnsutils
$ git clone https://anonscm.debian.org/git/collab-maint/shadowsocks-libev.git
$ cd shadowsocks-libev
$ ./tests/test.sh

The last test.sh script invokes the test, which creates local proxy
listen to 127.0.0.1:1081, and then it calls curl command to get index
page of google via local proxy, 127.0.0.1:1081.

My local test shows all pass, while debian ci test [1] shows a
connection timeout message.
So I'm wondering whether debian ci support network activity, and how
can I configure the test to get it passed.


I'd suggest to install apache as part of the tests and connect to
localhost:80, that way it always works even if ci.debian.net moves to
China or google goes down

to test python-openstackclient; MySQL, RabbitMQ, Apache and others are
installed, I haven't check its tests in a while but happy to help you



Thank you!
--
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1



--
IRC: gfa
GPG: 0X44BB1BA79F6C6333

Re: How to get debian ci test passed for proxy application

[gregor herrmann]

On Sun, 26 Nov 2017 18:42:22 +, James Cowgill wrote:

> > My local test shows all pass, while debian ci test [1] shows a
> > connection timeout message.
> > So I'm wondering whether debian ci support network activity, and how
> > can I configure the test to get it passed.
> I think you might need a "Restrictions: isolation-container" to get
> network access, but that's only a guess.

That's not my experience. We have quite a few perl packages where the
tests do something networky and haven't experienced problems on
ci.debian.net (modulo failing requests to external resources but
that's of course a different story).


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Die Tontauben: jonny


signature.asc
Description: Digital Signature

Re: How to get debian ci test passed for proxy application

[James Cowgill]

Hi,

On 26/11/17 17:00, Roger Shimizu wrote:
> Dear mentors list,
> 
> I maintain a proxy application, shadowsocks-libev.
> I want to let it pass debian ci test. And I already confirm the test
> all passed on my local environment, and debomatic [0].
> However it failed on debian ci infrastructure [1].
> 
> [0] 
> http://debomatic-amd64.debian.net/distribution#unstable/shadowsocks-libev/3.1.1+ds-1/autopkgtest
> [1] https://ci.debian.net/packages/s/shadowsocks-libev/unstable/amd64/
> 
> For local test, it just need the commands below:
> 
> $ sudo apt install shadowsocks-libev curl dnsutils
> $ git clone https://anonscm.debian.org/git/collab-maint/shadowsocks-libev.git
> $ cd shadowsocks-libev
> $ ./tests/test.sh

Running autopkgtest as described on this page might help:
https://ci.debian.net/doc/file.MAINTAINERS.html

> The last test.sh script invokes the test, which creates local proxy
> listen to 127.0.0.1:1081, and then it calls curl command to get index
> page of google via local proxy, 127.0.0.1:1081.
> 
> My local test shows all pass, while debian ci test [1] shows a
> connection timeout message.
> So I'm wondering whether debian ci support network activity, and how
> can I configure the test to get it passed.

I think you might need a "Restrictions: isolation-container" to get
network access, but that's only a guess.

James



signature.asc
Description: OpenPGP digital signature

How to get debian ci test passed for proxy application

[Roger Shimizu]

Dear mentors list,

I maintain a proxy application, shadowsocks-libev.
I want to let it pass debian ci test. And I already confirm the test
all passed on my local environment, and debomatic [0].
However it failed on debian ci infrastructure [1].

[0] 
http://debomatic-amd64.debian.net/distribution#unstable/shadowsocks-libev/3.1.1+ds-1/autopkgtest
[1] https://ci.debian.net/packages/s/shadowsocks-libev/unstable/amd64/

For local test, it just need the commands below:

$ sudo apt install shadowsocks-libev curl dnsutils
$ git clone https://anonscm.debian.org/git/collab-maint/shadowsocks-libev.git
$ cd shadowsocks-libev
$ ./tests/test.sh

The last test.sh script invokes the test, which creates local proxy
listen to 127.0.0.1:1081, and then it calls curl command to get index
page of google via local proxy, 127.0.0.1:1081.

My local test shows all pass, while debian ci test [1] shows a
connection timeout message.
So I'm wondering whether debian ci support network activity, and how
can I configure the test to get it passed.

Thank you!
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1