Bug#830353: ruby-kgio: FTBFS: ERROR: Test "ruby2.3" failed.
Antonio Terceiro wrote:
> On Fri, Jul 08, 2016 at 06:53:29PM +, Eric Wong wrote:
> > Thanks for the report. Can you try the patch below?
>
> Didn't help.
>
> I can't, however, reproduce this locally on my workstation.
Thanks for following up.
> > I'd be interested to know what non-standard sysctl knobs are set
> > for kernel socket buffer sizes (in /proc/sys/net/{core,ipv4}/*mem*)
> ==> /proc/sys/net/ipv4/tcp_rmem <==
> 40961258291216777216
>
> ==> /proc/sys/net/ipv4/tcp_wmem <==
> 40961258291216777216
Yikes at the gigantic default (middle) values. Anyways, I've
set those locally and notice the problem with my previous patch:
I missed some spots where temporary strings/arrays were created
inside the tests themselves.
> In Debian we don't even have 2.2 anymore. The only things using kgio are
> unicorn and rainbows. Maybe if their dependency on kgio could be skipped on
> Ruby 2.3, we could drop kgio from Debian.
Yes, I still have to check and make sure I don't introduce
performance regressions and such when dropping kgio.
I still need to support Ruby 1.9.3 and 2.0.0 outside of Debian
(or on wheezy LTS/jessie)
The following ought to work (I tested locally with those sysctl
values) but I'll see about making the tests more robust against
larger kernel buffers.
---8<
--- a/test/lib_read_write.rb
+++ b/test/lib_read_write.rb
@@ -7,7 +7,12 @@ $-w = true
require 'kgio'
module LibReadWriteTest
- RANDOM_BLOB = File.open("/dev/urandom") { |fp| fp.read(10 * 1024 * 1024) }
+ RANDOM_BLOB = File.open("/dev/urandom") do |fp|
+nr = 31
+buf = fp.read(nr)
+# get roughly a 20MB block of random data
+(buf * (20 * 1024 * 1024 / nr)) + (buf * rand(123))
+ end
def teardown
@rd.close if defined?(@rd) && ! @rd.closed?
@@ -369,7 +374,7 @@ module LibReadWriteTest
@nr += 1
IO.select(nil, [self])
end
-buf = "." * 1024 * 1024 * 10
+buf = RANDOM_BLOB
thr = Thread.new { @wr.kgio_write(buf) }
Thread.pass until thr.stop?
readed = @rd.read(buf.size)
@@ -385,7 +390,7 @@ module LibReadWriteTest
@nr += 1
IO.select(nil, [self])
end
-buf = ["." * 1024] * 1024 * 10
+buf = [ RANDOM_BLOB, RANDOM_BLOB ]
buf_size = buf.inject(0){|c, s| c + s.size}
thr = Thread.new { @wr.kgio_writev(buf) }
Thread.pass until thr.stop?
--
EW
Bug#830353: ruby-kgio: FTBFS: ERROR: Test "ruby2.3" failed.
Control: retitle -1 ruby-kgio: FTBFS on Amazon EC2: ERROR: Test "ruby2.3"
failed.
Control: severity -1 important
On Fri, Jul 08, 2016 at 06:53:29PM +, Eric Wong wrote:
> Lucas Nussbaum wrote:
> > The full build log is available from:
> >
> > http://people.debian.org/~lucas/logs/2016/07/07/ruby-kgio_2.10.0-1_unstable_reb.normal.log
> >
> > A list of current common problems and possible solutions is available at
> > http://wiki.debian.org/qa.debian.org/FTBFS . You're welcome to contribute!
> >
> > About the archive rebuild: The rebuild was done on EC2 VM instances from
> > Amazon Web Services, using a clean, minimal and up-to-date chroot. Every
> > failed build was retried once to eliminate random failures.
>
> Thanks for the report. Can you try the patch below?
Didn't help.
I can't, however, reproduce this locally on my workstation.
I remember hitting this type of issue with kgio whenever I did rebuilds of all
Ruby packages on an Amazon EC2 instance, and then couldn't reproduce it
locally.
If you want to give a closer look I can give you access to the VM I use for
these builds, where I can consistently reproduce this.
> I'd be interested to know what non-standard sysctl knobs are set
> for kernel socket buffer sizes (in /proc/sys/net/{core,ipv4}/*mem*)
$ tail -n 1000 /proc/sys/net/{core,ipv4}/*mem*
==> /proc/sys/net/core/optmem_max <==
20480
==> /proc/sys/net/core/rmem_default <==
212992
==> /proc/sys/net/core/rmem_max <==
16777216
==> /proc/sys/net/core/wmem_default <==
212992
==> /proc/sys/net/core/wmem_max <==
16777216
==> /proc/sys/net/ipv4/igmp_max_memberships <==
20
==> /proc/sys/net/ipv4/tcp_mem <==
2954019 3938693 5908038
==> /proc/sys/net/ipv4/tcp_rmem <==
40961258291216777216
==> /proc/sys/net/ipv4/tcp_wmem <==
40961258291216777216
==> /proc/sys/net/ipv4/udp_mem <==
2954019 3938693 5908038
==> /proc/sys/net/ipv4/udp_rmem_min <==
4096
==> /proc/sys/net/ipv4/udp_wmem_min <==
4096
> You can also increase the 20M to 30M or whatever (otoh, that
> means using more memory during tests :<)
>
> I'm not comfortable calling setsockopt to set SO_{SND,RCV}BUF
> knobs since that can trigger VM deadlock bugs on some older
> kernels, too.
>
> Anyways, most of the kgio functionality is in Ruby 2.3 nowadays
> and kgio can probably go away when support for 2.2 can be
> dropped in gems.
In Debian we don't even have 2.2 anymore. The only things using kgio are
unicorn and rainbows. Maybe if their dependency on kgio could be skipped on
Ruby 2.3, we could drop kgio from Debian.
signature.asc
Description: PGP signature
Bug#830353: ruby-kgio: FTBFS: ERROR: Test "ruby2.3" failed.
Lucas Nussbaum wrote:
> The full build log is available from:
>
> http://people.debian.org/~lucas/logs/2016/07/07/ruby-kgio_2.10.0-1_unstable_reb.normal.log
>
> A list of current common problems and possible solutions is available at
> http://wiki.debian.org/qa.debian.org/FTBFS . You're welcome to contribute!
>
> About the archive rebuild: The rebuild was done on EC2 VM instances from
> Amazon Web Services, using a clean, minimal and up-to-date chroot. Every
> failed build was retried once to eliminate random failures.
Thanks for the report. Can you try the patch below?
I'd be interested to know what non-standard sysctl knobs are set
for kernel socket buffer sizes (in /proc/sys/net/{core,ipv4}/*mem*)
You can also increase the 20M to 30M or whatever (otoh, that
means using more memory during tests :<)
I'm not comfortable calling setsockopt to set SO_{SND,RCV}BUF
knobs since that can trigger VM deadlock bugs on some older
kernels, too.
Anyways, most of the kgio functionality is in Ruby 2.3 nowadays
and kgio can probably go away when support for 2.2 can be
dropped in gems.
8<
Subject: [PATCH] test: increase random blob size for giant socket buffers
Socket buffers on some systems may be too big to trigger
partial write/blocking behavior we need for our tests.
Increase it to 20M for now and see if we can fix an FTBFS
on larger machines: https://bugs.debian.org/830353
This should also speed up tests a little since urandom is
slow and reading 10M from it in the first place was lazy.
---
test/lib_read_write.rb | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/test/lib_read_write.rb b/test/lib_read_write.rb
index 26e7aef..f5c5bf8 100644
--- a/test/lib_read_write.rb
+++ b/test/lib_read_write.rb
@@ -7,7 +7,12 @@ $-w = true
require 'kgio'
module LibReadWriteTest
- RANDOM_BLOB = File.open("/dev/urandom") { |fp| fp.read(10 * 1024 * 1024) }
+ RANDOM_BLOB = File.open("/dev/urandom") do |fp|
+nr = 31
+buf = fp.read(nr)
+# get roughly a 20MB block of random data
+(buf * (20 * 1024 * 1024 / nr)) + (buf * rand(123))
+ end
def teardown
@rd.close if defined?(@rd) && ! @rd.closed?
--
EW
Bug#830353: ruby-kgio: FTBFS: ERROR: Test "ruby2.3" failed.
Source: ruby-kgio Version: 2.10.0-1 Severity: serious Tags: stretch sid User: debian...@lists.debian.org Usertags: qa-ftbfs-20160707 qa-ftbfs Justification: FTBFS on amd64 Hi, During a rebuild of all packages in sid, your package failed to build on amd64. Relevant part (hopefully): > /usr/bin/ruby2.3 /usr/bin/gem2deb-test-runner > > ┌──┐ > │ Checking Rubygems dependency resolution on ruby2.3 > │ > └──┘ > > GEM_PATH=debian/ruby-kgio/usr/share/rubygems-integration/2.3.0:/var/lib/gems/2.3.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0:/usr/share/rubygems-integration/2.3.0:/usr/share/rubygems-integration/all > ruby2.3 -e gem\ \"kgio\" > > ┌──┐ > │ Run tests for ruby2.3 from debian/ruby-tests.rake > │ > └──┘ > > RUBYLIB=/«PKGBUILDDIR»/debian/ruby-kgio/usr/lib/x86_64-linux-gnu/ruby/vendor_ruby/2.3.0:/«PKGBUILDDIR»/debian/ruby-kgio/usr/lib/ruby/vendor_ruby:. > > GEM_PATH=debian/ruby-kgio/usr/share/rubygems-integration/2.3.0:/var/lib/gems/2.3.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0:/usr/share/rubygems-integration/2.3.0:/usr/share/rubygems-integration/all > ruby2.3 -S rake -f debian/ruby-tests.rake > /usr/bin/ruby2.3 -I"test" > "/usr/lib/ruby/vendor_ruby/rake/rake_test_loader.rb" > "test/test_accept_class.rb" "test/test_accept_flags.rb" > "test/test_autopush.rb" "test/test_connect_fd_leak.rb" > "test/test_cross_thread_close.rb" "test/test_default_wait.rb" > "test/test_kgio_addr.rb" "test/test_no_dns_on_tcp_connect.rb" > "test/test_peek.rb" "test/test_pipe_popen.rb" "test/test_pipe_read_write.rb" > "test/test_poll.rb" "test/test_singleton_read_write.rb" "test/test_socket.rb" > "test/test_socketpair_read_write.rb" "test/test_syssend.rb" > "test/test_tcp6_client_read_server_write.rb" > "test/test_tcp_client_read_server_write.rb" "test/test_tcp_connect.rb" > "test/test_tcp_server.rb" "test/test_tcp_server_read_client_write.rb" > "test/test_tfo.rb" "test/test_tryopen.rb" > "test/test_unix_client_read_server_write.rb" "test/test_unix_connect.rb" > "test/test_unix_server.rb" "test/test_unix_server_read_client_write.rb" -v > /«PKGBUILDDIR»/test/test_connect_fd_leak.rb:13: warning: assigned but unused > variable - sock > /«PKGBUILDDIR»/test/test_kgio_addr.rb:11: warning: assigned but unused > variable - client > /«PKGBUILDDIR»/test/lib_read_write.rb:163: warning: assigned but unused > variable - e > /«PKGBUILDDIR»/test/lib_read_write.rb:200: warning: assigned but unused > variable - e > /«PKGBUILDDIR»/test/test_poll.rb:46: warning: assigned but unused variable - > foo > /«PKGBUILDDIR»/test/test_poll.rb:58: warning: assigned but unused variable - > res > /«PKGBUILDDIR»/test/test_tcp6_client_read_server_write.rb:4: warning: > assigned but unused variable - tmp > /«PKGBUILDDIR»/test/lib_server_accept.rb:18: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/lib_server_accept.rb:26: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/lib_server_accept.rb:34: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/lib_server_accept.rb:47: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/lib_server_accept.rb:60: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/lib_server_accept.rb:70: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/lib_server_accept.rb:80: warning: assigned but unused > variable - a > /«PKGBUILDDIR»/test/test_tryopen.rb:60: warning: assigned but unused variable > - file > /«PKGBUILDDIR»/test/test_unix_connect.rb:9: warning: method redefined; > discarding old foo > /«PKGBUILDDIR»/test/test_unix_connect.rb:9: warning: method redefined; > discarding old foo= > /«PKGBUILDDIR»/test/test_unix_connect.rb:10: warning: method redefined; > discarding old kgio_wait_writable > /«PKGBUILDDIR»/test/test_tcp_connect.rb:8: warning: previous definition of > kgio_wait_writable was here > Loaded suite /usr/lib/ruby/vendor_ruby/rake/rake_test_loader > Started > TesTcpClientReadServerWrite: > test_monster_trywrite: F > === > Failure: >expected to be kind_of? >but was > . > test_monster_trywrite(TesTcpClientReadServerWrite) > /«PKGBUILDDIR»/test/lib_read_write.rb:319:in `test_monster_trywrite' > 316: def test_monster_trywrite > 317: buf = RANDOM_BLOB.dup > 318: rv = @wr.kgio_trywrite(buf) > => 319: assert_kind_of String, rv > 320: assert rv.size < buf.size > 321: @rd.nonblock = false > 322: assert_equal(buf, @rd.read(buf.size - rv.size) + rv) > =
