Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package rubygem-puma for openSUSE:Factory checked in at 2021-05-15 01:24:40 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/rubygem-puma (Old) and /work/SRC/openSUSE:Factory/.rubygem-puma.new.2988 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "rubygem-puma" Sat May 15 01:24:40 2021 rev:44 rq:893206 version:5.3.1 Changes: -------- --- /work/SRC/openSUSE:Factory/rubygem-puma/rubygem-puma.changes 2021-03-15 10:56:25.737358538 +0100 +++ /work/SRC/openSUSE:Factory/.rubygem-puma.new.2988/rubygem-puma.changes 2021-05-15 01:24:59.682854811 +0200 @@ -1,0 +2,38 @@ +Fri May 14 16:22:43 UTC 2021 - Manuel Schnitzer <[email protected]> + +- updated to version 5.3.1 + + ## 5.3.1 / 2021-05-11 + + * Security + * Close keepalive connections after the maximum number of fast inlined requests (#2625) + + ## 5.3.0 / 2021-05-07 + + * Features + * Add support for Linux's abstract sockets ([#2564], [#2526]) + * Add debug to worker timeout and startup ([#2559], [#2528]) + * Print warning when running one-worker cluster ([#2565], [#2534]) + * Don't close systemd activated socket on pumactl restart ([#2563], [#2504]) + + * Bugfixes + * systemd - fix event firing ([#2591], [#2572]) + * Immediately unlink temporary files ([#2613]) + * Improve parsing of HTTP_HOST header ([#2605], [#2584]) + * Handle fatal error that has no backtrace ([#2607], [#2552]) + * Fix timing out requests too early ([#2606], [#2574]) + * Handle segfault in Ruby 2.6.6 on thread-locals ([#2567], [#2566]) + * Server#closed_socket? - parameter may be a MiniSSL::Socket ([#2596]) + * Define UNPACK_TCP_STATE_FROM_TCP_INFO in the right place ([#2588], [#2556]) + * request.rb - fix chunked assembly for ascii incompatible encodings, add test ([#2585], [#2583]) + + * Performance + * Reset peerip only if remote_addr_header is set ([#2609]) + * Reduce puma_parser struct size ([#2590]) + + * Refactor + * Refactor drain on shutdown ([#2600]) + * Micro optimisations in `wait_for_less_busy_worker` feature ([#2579]) + * Lots of test fixes + +------------------------------------------------------------------- Old: ---- puma-5.2.2.gem New: ---- puma-5.3.1.gem ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ rubygem-puma.spec ++++++ --- /var/tmp/diff_new_pack.IfINdp/_old 2021-05-15 01:25:00.122853265 +0200 +++ /var/tmp/diff_new_pack.IfINdp/_new 2021-05-15 01:25:00.122853265 +0200 @@ -24,7 +24,7 @@ # Name: rubygem-puma -Version: 5.2.2 +Version: 5.3.1 Release: 0 %define mod_name puma %define mod_full_name %{mod_name}-%{version} ++++++ puma-5.2.2.gem -> puma-5.3.1.gem ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/History.md new/History.md --- old/History.md 2021-03-02 17:07:50.000000000 +0100 +++ new/History.md 2021-05-11 16:54:42.000000000 +0200 @@ -1,3 +1,36 @@ +## 5.3.1 / 2021-05-11 + +* Security + * Close keepalive connections after the maximum number of fast inlined requests (#2625) + +## 5.3.0 / 2021-05-07 + +* Features + * Add support for Linux's abstract sockets ([#2564], [#2526]) + * Add debug to worker timeout and startup ([#2559], [#2528]) + * Print warning when running one-worker cluster ([#2565], [#2534]) + * Don't close systemd activated socket on pumactl restart ([#2563], [#2504]) + +* Bugfixes + * systemd - fix event firing ([#2591], [#2572]) + * Immediately unlink temporary files ([#2613]) + * Improve parsing of HTTP_HOST header ([#2605], [#2584]) + * Handle fatal error that has no backtrace ([#2607], [#2552]) + * Fix timing out requests too early ([#2606], [#2574]) + * Handle segfault in Ruby 2.6.6 on thread-locals ([#2567], [#2566]) + * Server#closed_socket? - parameter may be a MiniSSL::Socket ([#2596]) + * Define UNPACK_TCP_STATE_FROM_TCP_INFO in the right place ([#2588], [#2556]) + * request.rb - fix chunked assembly for ascii incompatible encodings, add test ([#2585], [#2583]) + +* Performance + * Reset peerip only if remote_addr_header is set ([#2609]) + * Reduce puma_parser struct size ([#2590]) + +* Refactor + * Refactor drain on shutdown ([#2600]) + * Micro optimisations in `wait_for_less_busy_worker` feature ([#2579]) + * Lots of test fixes + ## 5.2.2 / 2021-02-22 * Bugfixes @@ -180,6 +213,11 @@ * Support parallel tests in verbose progress reporting ([#2223]) * Refactor error handling in server accept loop ([#2239]) +## 4.3.8 / 2021-05-11 + +* Security + * Close keepalive connections after the maximum number of fast inlined requests (#2625) + ## 4.3.7 / 2020-11-30 * Bugfixes @@ -1708,8 +1746,36 @@ * Bugfixes * Your bugfix goes here <Most recent on the top, like GitHub> (#Github Number) +[#2564]:https://github.com/puma/puma/pull/2564 "PR by @MSP-Greg, merged 2021-04-24" +[#2526]:https://github.com/puma/puma/issues/2526 "Issue by @nerdrew, closed 2021-04-24" +[#2559]:https://github.com/puma/puma/pull/2559 "PR by @ylecuyer, merged 2021-03-11" +[#2528]:https://github.com/puma/puma/issues/2528 "Issue by @cjlarose, closed 2021-03-11" +[#2565]:https://github.com/puma/puma/pull/2565 "PR by @CGA1123, merged 2021-03-09" +[#2534]:https://github.com/puma/puma/issues/2534 "Issue by @nateberkopec, closed 2021-03-09" +[#2563]:https://github.com/puma/puma/pull/2563 "PR by @MSP-Greg, merged 2021-03-06" +[#2504]:https://github.com/puma/puma/issues/2504 "Issue by @fsateler, closed 2021-03-06" +[#2591]:https://github.com/puma/puma/pull/2591 "PR by @MSP-Greg, merged 2021-05-05" +[#2572]:https://github.com/puma/puma/issues/2572 "Issue by @josefbilendo, closed 2021-05-05" +[#2613]:https://github.com/puma/puma/pull/2613 "PR by @smcgivern, merged 2021-04-27" +[#2605]:https://github.com/puma/puma/pull/2605 "PR by @pascalbetz, merged 2021-04-26" +[#2584]:https://github.com/puma/puma/issues/2584 "Issue by @kaorihinata, closed 2021-04-26" +[#2607]:https://github.com/puma/puma/pull/2607 "PR by @calvinxiao, merged 2021-04-23" +[#2552]:https://github.com/puma/puma/issues/2552 "Issue by @feliperaul, opened 2021-02-09" +[#2606]:https://github.com/puma/puma/pull/2606 "PR by @wjordan, merged 2021-04-20" +[#2574]:https://github.com/puma/puma/issues/2574 "Issue by @darkhelmet, closed 2021-04-20" +[#2567]:https://github.com/puma/puma/pull/2567 "PR by @kddeisz, merged 2021-04-19" +[#2566]:https://github.com/puma/puma/issues/2566 "Issue by @kddeisz, closed 2021-04-19" +[#2596]:https://github.com/puma/puma/pull/2596 "PR by @MSP-Greg, merged 2021-04-18" +[#2588]:https://github.com/puma/puma/pull/2588 "PR by @dentarg, merged 2021-04-02" +[#2556]:https://github.com/puma/puma/issues/2556 "Issue by @gamecreature, closed 2021-04-02" +[#2585]:https://github.com/puma/puma/pull/2585 "PR by @MSP-Greg, merged 2021-03-26" +[#2583]:https://github.com/puma/puma/issues/2583 "Issue by @jboler, closed 2021-03-26" +[#2609]:https://github.com/puma/puma/pull/2609 "PR by @calvinxiao, merged 2021-04-26" +[#2590]:https://github.com/puma/puma/pull/2590 "PR by @calvinxiao, merged 2021-04-05" +[#2600]:https://github.com/puma/puma/pull/2600 "PR by @wjordan, merged 2021-04-30" +[#2579]:https://github.com/puma/puma/pull/2579 "PR by @ghiculescu, merged 2021-03-17" +[#2553]:https://github.com/puma/puma/pull/2553 "PR by @olivierbellone, merged 2021-02-10" [#2557]:https://github.com/puma/puma/pull/2557 "PR by @cjlarose, merged 2021-02-22" -[#2553]:https://github.com/puma/puma/pull/2553 "PR by @olivierbellone, merged 02-10-22" [#2550]:https://github.com/puma/puma/pull/2550 "PR by @MSP-Greg, merged 2021-02-05" [#2547]:https://github.com/puma/puma/pull/2547 "PR by @wildmaples, merged 2021-02-03" [#2543]:https://github.com/puma/puma/pull/2543 "PR by @MSP-Greg, merged 2021-02-01" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/README.md new/README.md --- old/README.md 2021-03-02 17:07:50.000000000 +0100 +++ new/README.md 2021-05-11 16:54:42.000000000 +0200 @@ -16,7 +16,7 @@ Puma processes requests using a C-optimized Ragel extension (inherited from Mongrel) that provides fast, accurate HTTP 1.1 protocol parsing in a portable way. Puma then serves the request using a thread pool. Each request is served in a separate thread, so truly concurrent Ruby implementations (JRuby, Rubinius) will use all available CPU cores. -Puma was designed to be the go-to server for [Rubinius](https://rubinius.com), but also works well with JRuby and MRI. +Originally designed as a server for [Rubinius](https://github.com/rubinius/rubinius), Puma also works well with Ruby (MRI) and JRuby. On MRI, there is a Global VM Lock (GVL) that ensures only one thread can run Ruby code at a time. But if you're doing a lot of blocking IO (such as HTTP calls to external APIs like Twitter), Puma still improves MRI's throughput by allowing IO waiting to be done in parallel. Binary files old/checksums.yaml.gz and new/checksums.yaml.gz differ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/architecture.md new/docs/architecture.md --- old/docs/architecture.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/architecture.md 2021-05-11 16:54:42.000000000 +0200 @@ -6,11 +6,12 @@ Puma is a threaded Ruby HTTP application server, processing requests across a TCP or UNIX socket. -Puma processes (there can be one or many) accept connections from the socket via a thread (in the `Reactor` class). The connection, once fully buffered and read, moves in to the `todo` list, where it will be picked up by a free/waiting thread in the threadpool (the `ThreadPool` class). + +Puma processes (there can be one or many) accept connections from the socket via a thread (in the [`Reactor`](../lib/puma/reactor.rb) class). The connection, once fully buffered and read, moves in to the `todo` list, where it will be picked up by a free/waiting thread in the threadpool (the [`ThreadPool`](../lib/puma/thread_pool.rb) class). Puma works in two main modes: cluster and single. In single mode, only one Puma process is booted. In cluster mode, a `master` process is booted, which prepares (and may boot) the application, and then uses the `fork()` system call to create 1 or more `child` processes. These `child` processes all listen to the same socket. The `master` process does not listen to the socket or process requests - its purpose is mostly to manage and listen for UNIX signals and possibly kill or boot `child` processes. -We sometimes call `child` processes (or Puma processes in `single` mode) _workers_, and we sometimes call the threads created by Puma's `ThreadPool` _worker threads_. +We sometimes call `child` processes (or Puma processes in `single` mode) _workers_, and we sometimes call the threads created by Puma's [`ThreadPool`](../lib/puma/thread_pool.rb) _worker threads_. ## How Requests Work @@ -18,8 +19,8 @@ * Upon startup, Puma listens on a TCP or UNIX socket. * The backlog of this socket is configured (with a default of 1024). This determines the size of the queue for unaccepted connections. Generally, this setting is unimportant and will never be hit in production use. If the backlog is full, the connection will be refused by the operating system. - * This socket backlog is distinct from the `backlog` of work as reported by `Puma.stats` or the control server. The backlog as reported by Puma is the number of connections in the process' `todo` set waiting for a thread from the `ThreadPool`. -* By default, a single, separate thread (created by the `Reactor` class) is used to read and buffer requests from the socket. + * This socket backlog is distinct from the `backlog` of work as reported by `Puma.stats` or the control server. The backlog as reported by Puma is the number of connections in the process' `todo` set waiting for a thread from the [`ThreadPool`](../lib/puma/thread_pool.rb). +* By default, a single, separate thread (created by the [`Reactor`](../lib/puma/reactor.rb) class) is used to read and buffer requests from the socket. * When at least one worker thread is available for work, the reactor thread listens to the socket and accepts a request, if one is waiting. * The reactor thread waits for the entire HTTP request to be received. * The time spent waiting for the HTTP request body to be received is exposed to the Rack app as `env['puma.request_body_wait']` (milliseconds). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/deployment.md new/docs/deployment.md --- old/docs/deployment.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/deployment.md 2021-05-11 16:54:42.000000000 +0200 @@ -16,7 +16,7 @@ ## Single vs Cluster mode -Puma was originally conceived as a thread-only webserver, but grew the ability to +Puma was originally conceived as a thread-only web server, but grew the ability to also use processes in version 2. To run `puma` in single mode (e.g. for a development environment) you will need to diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/jungle/rc.d/README.md new/docs/jungle/rc.d/README.md --- old/docs/jungle/rc.d/README.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/jungle/rc.d/README.md 2021-05-11 16:54:42.000000000 +0200 @@ -1,6 +1,6 @@ # Puma as a service using rc.d -Manage multilpe Puma servers as services on one box using FreeBSD's rc.d service. +Manage multiple Puma servers as services on one box using FreeBSD's rc.d service. ## Dependencies diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/kubernetes.md new/docs/kubernetes.md --- old/docs/kubernetes.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/kubernetes.md 2021-05-11 16:54:42.000000000 +0200 @@ -61,6 +61,6 @@ There is a subtle race condition between step 2 and 3: The replication controller does not synchronously remove the pod from the Services AND THEN call the pre-stop hook of the pod, but rather it asynchronously sends "remove this pod from your endpoints" requests to the Services and then immediately proceeds to invoke the pods' pre-stop hook. If the Service controller (typically something like nginx or haproxy) receives this request handles this request "too" late (due to internal lag or network latency between the replication and Service controllers) then it is possible that the Service controller will send one or more requests to a Puma process which has already shut down its listening socket. These requests will then fail with 5XX error codes. -The way Kubernetes works this way, rather than handling step 2 synchronously, is due to the CAP theorem: in a distributed system there is no way to guarantuee that any message will arrive promptly. In particular, waiting for all Service controllers to report back might get stuck for an indefinite time if one of them has already been terminated or if there has been a net split. A way to work around this is to add a sleep to the pre-stop hook of the same time as the `terminationGracePeriodSeconds` time. This will allow the Puma process to keep serving new requests during the entire grace period, although it will no longer receive new requests after all Service controllers have propagated the removal of the pod from their endpoint lists. Then, after `terminationGracePeriodSeconds`, the pod receives `SIGKILL` and closes down. If your process can't handle SIGKILL properly, for example because it needs to release locks in different services, you can also sleep for a shorter period (and/or increase `terminationGracePeriodSeconds`) as long as the time slept is longer than the time that your Service controllers take to propagate the pod removal. The downside of this workaround is that all pods will take at minimum the amount of time slept to shut down and this will increase the time required for your rolling deploy. +The way Kubernetes works this way, rather than handling step 2 synchronously, is due to the CAP theorem: in a distributed system there is no way to guarantee that any message will arrive promptly. In particular, waiting for all Service controllers to report back might get stuck for an indefinite time if one of them has already been terminated or if there has been a net split. A way to work around this is to add a sleep to the pre-stop hook of the same time as the `terminationGracePeriodSeconds` time. This will allow the Puma process to keep serving new requests during the entire grace period, although it will no longer receive new requests after all Service controllers have propagated the removal of the pod from their endpoint lists. Then, after `terminationGracePeriodSeconds`, the pod receives `SIGKILL` and closes down. If your process can't handle SIGKILL properly, for example because it needs to release locks in different services, you can also sleep for a shorter period (and/or increase `terminationGracePeriodSeconds`) as long as the time slept is longer than the time that your Service controllers take to propagate the pod removal. The downside of this workaround is that all pods will take at minimum the amount of time slept to shut down and this will increase the time required for your rolling deploy. More discussions and links to relevant articles can be found in https://github.com/puma/puma/issues/2343. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/plugins.md new/docs/plugins.md --- old/docs/plugins.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/plugins.md 2021-05-11 16:54:42.000000000 +0200 @@ -34,5 +34,5 @@ `config` runs when the server is being configured and is passed a `Puma::DSL` object that can be used to add additional configuration. -Any public methods in `Puma::Plugin` are the public API that any plugin may +Any public methods in [`Puma::Plugin`](../lib/puma/plugin.rb) are the public API that any plugin may use. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/restart.md new/docs/restart.md --- old/docs/restart.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/restart.md 2021-05-11 16:54:42.000000000 +0200 @@ -45,7 +45,7 @@ ### Supported configurations * Works in cluster mode only -* To support upgrading the application that Puma is serving, ensure `prune_bundler` is enabled and that `preload_app` is disabled (it is disabled by default). +* To support upgrading the application that Puma is serving, ensure `prune_bundler` is enabled and that `preload_app!` is disabled * Supported on all platforms where cluster mode is supported ### Client experience diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/stats.md new/docs/stats.md --- old/docs/stats.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/stats.md 2021-05-11 16:54:42.000000000 +0200 @@ -53,7 +53,7 @@ ### single mode and individual workers in cluster mode -When Puma is run in single mode, these stats ar available at the top level. When Puma is run in cluster mode, these stats are available within the `worker_status` array in a hash labeled `last_status`, in an array of hashes, one hash for each worker. +When Puma is run in single mode, these stats are available at the top level. When Puma is run in cluster mode, these stats are available within the `worker_status` array in a hash labeled `last_status`, in an array of hashes, one hash for each worker. * backlog: requests that are waiting for an available thread to be available. if this is above 0, you need more capacity [always true?] * running: how many threads are running diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/docs/systemd.md new/docs/systemd.md --- old/docs/systemd.md 2021-03-02 17:07:50.000000000 +0100 +++ new/docs/systemd.md 2021-05-11 16:54:42.000000000 +0200 @@ -8,7 +8,7 @@ ## Service Configuration Below is a sample puma.service configuration file for systemd, which -can be copied or symlinked to /etc/systemd/system/puma.service, or if +can be copied or symlinked to `/etc/systemd/system/puma.service`, or if desired, using an application or instance specific name. Note that this uses the systemd preferred "simple" type where the diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ext/puma_http11/http11_parser.h new/ext/puma_http11/http11_parser.h --- old/ext/puma_http11/http11_parser.h 2021-03-02 17:07:50.000000000 +0100 +++ new/ext/puma_http11/http11_parser.h 2021-05-11 16:54:42.000000000 +0200 @@ -29,8 +29,8 @@ typedef struct puma_parser { int cs; - size_t body_start; int content_len; + size_t body_start; size_t nread; size_t mark; size_t field_start; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/binder.rb new/lib/puma/binder.rb --- old/lib/puma/binder.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/binder.rb 2021-05-11 16:54:42.000000000 +0200 @@ -13,7 +13,7 @@ require 'puma/minissl' require 'puma/minissl/context_builder' - # Odd bug in 'pure Ruby' nio4r verion 2.5.2, which installs with Ruby 2.3. + # Odd bug in 'pure Ruby' nio4r version 2.5.2, which installs with Ruby 2.3. # NIO doesn't create any OpenSSL objects, but it rescues an OpenSSL error. # The bug was that it did not require openssl. # @todo remove when Ruby 2.3 support is dropped @@ -177,11 +177,19 @@ @listeners << [str, io] if io when "unix" path = "#{uri.host}#{uri.path}".gsub("%20", " ") + abstract = false + if str.start_with? 'unix://@' + raise "OS does not support abstract UNIXSockets" unless Puma.abstract_unix_socket? + abstract = true + path = "@#{path}" + end if fd = @inherited_fds.delete(str) + @unix_paths << path unless abstract io = inherit_unix_listener path, fd logger.log "* Inherited #{str}" elsif sock = @activated_sockets.delete([ :unix, path ]) + @unix_paths << path unless abstract || File.exist?(path) io = inherit_unix_listener path, sock logger.log "* Activated #{str}" else @@ -205,6 +213,7 @@ end end + @unix_paths << path unless abstract || File.exist?(path) io = add_unix_listener path, umask, mode, backlog logger.log "* #{log_msg} on #{str}" end @@ -258,14 +267,18 @@ end # Also close any unused activated sockets - @activated_sockets.each do |key, sock| - logger.log "* Closing unused activated socket: #{key.join ':'}" - begin - sock.close - rescue SystemCallError + unless @activated_sockets.empty? + fds = @ios.map(&:to_i) + @activated_sockets.each do |key, sock| + next if fds.include? sock.to_i + logger.log "* Closing unused activated socket: #{key.first}://#{key[1..-1].join ':'}" + begin + sock.close + rescue SystemCallError + end + # We have to unlink a unix socket path that's not being used + File.unlink key[1] if key.first == :unix end - # We have to unlink a unix socket path that's not being used - File.unlink key[1] if key[0] == :unix end end @@ -351,8 +364,6 @@ # Tell the server to listen on +path+ as a UNIX domain socket. # def add_unix_listener(path, umask=nil, mode=nil, backlog=1024) - @unix_paths << path unless File.exist? path - # Let anyone connect by default umask ||= 0 @@ -369,8 +380,7 @@ raise "There is already a server bound to: #{path}" end end - - s = UNIXServer.new(path) + s = UNIXServer.new path.sub(/\A@/, "\0") # check for abstract UNIXSocket s.listen backlog @ios << s ensure @@ -389,8 +399,6 @@ end def inherit_unix_listener(path, fd) - @unix_paths << path unless File.exist? path - s = fd.kind_of?(::TCPServer) ? fd : ::UNIXServer.for_fd(fd) @ios << s @@ -403,24 +411,24 @@ end def close_listeners - listeners.each do |l, io| - io.close unless io.closed? # Ruby 2.2 issue - uri = URI.parse(l) + @listeners.each do |l, io| + io.close unless io.closed? + uri = URI.parse l next unless uri.scheme == 'unix' unix_path = "#{uri.host}#{uri.path}" - File.unlink unix_path if unix_paths.include? unix_path + File.unlink unix_path if @unix_paths.include?(unix_path) && File.exist?(unix_path) end end def redirects_for_restart - redirects = listeners.map { |a| [a[1].to_i, a[1].to_i] }.to_h + redirects = @listeners.map { |a| [a[1].to_i, a[1].to_i] }.to_h redirects[:close_others] = true redirects end # @version 5.0.0 def redirects_for_restart_env - listeners.each_with_object({}).with_index do |(listen, memo), i| + @listeners.each_with_object({}).with_index do |(listen, memo), i| memo["PUMA_INHERIT_#{i}"] = "#{listen[1].to_i}:#{listen[0]}" end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/client.rb new/lib/puma/client.rb --- old/lib/puma/client.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/client.rb 2021-05-11 16:54:42.000000000 +0200 @@ -126,7 +126,7 @@ @parsed_bytes = 0 @ready = false @body_remain = 0 - @peerip = nil + @peerip = nil if @remote_addr_header @in_last_chunk = false if @buffer @@ -295,6 +295,7 @@ if remain > MAX_BODY @body = Tempfile.new(Const::PUMA_TMP_BASE) + @body.unlink @body.binmode @tempfile = @body else @@ -386,6 +387,7 @@ @prev_chunk = "" @body = Tempfile.new(Const::PUMA_TMP_BASE) + @body.unlink @body.binmode @tempfile = @body @chunked_content_length = 0 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/cluster/worker_handle.rb new/lib/puma/cluster/worker_handle.rb --- old/lib/puma/cluster/worker_handle.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/cluster/worker_handle.rb 2021-05-11 16:54:42.000000000 +0200 @@ -31,6 +31,10 @@ @stage == :booted end + def uptime + Time.now - started_at + end + def boot! @last_checkin = Time.now @stage = :booted diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/cluster.rb new/lib/puma/cluster.rb --- old/lib/puma/cluster.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/cluster.rb 2021-05-11 16:54:42.000000000 +0200 @@ -43,6 +43,7 @@ end def start_phased_restart + @events.fire_on_restart! @phase += 1 log "- Starting phased worker restart, phase: #{@phase}" @@ -317,7 +318,7 @@ stop_workers stop - + @events.fire_on_stopped! raise(SignalException, "SIGTERM") if @options[:raise_exception_on_sigterm] exit 0 # Clean exit, workers were stopped end @@ -332,16 +333,22 @@ # This is aligned with the output from Runner, see Runner#output_header log "* Workers: #{@options[:workers]}" - # Threads explicitly marked as fork safe will be ignored. - # Used in Rails, but may be used by anyone. - before = Thread.list.reject { |t| t.thread_variable_get(:fork_safe) } - if preload? + # Threads explicitly marked as fork safe will be ignored. Used in Rails, + # but may be used by anyone. Note that we need to explicit + # Process::Waiter check here because there's a bug in Ruby 2.6 and below + # where calling thread_variable_get on a Process::Waiter will segfault. + # We can drop that clause once those versions of Ruby are no longer + # supported. + fork_safe = ->(t) { !t.is_a?(Process::Waiter) && t.thread_variable_get(:fork_safe) } + + before = Thread.list.reject(&fork_safe) + log "* Restarts: (\u2714) hot (\u2716) phased" log "* Preloading application" load_and_bind - after = Thread.list.reject { |t| t.thread_variable_get(:fork_safe) } + after = Thread.list.reject(&fork_safe) if after.size > before.size threads = (after - before) @@ -382,6 +389,8 @@ log "Use Ctrl-C to stop" + single_worker_warning + redirect_io Plugins.fire_background @@ -403,12 +412,16 @@ begin booted = false + in_phased_restart = false + workers_not_booted = @options[:workers] while @status == :run begin if @phased_restart start_phased_restart @phased_restart = false + in_phased_restart = true + workers_not_booted = @options[:workers] end check_workers @@ -434,8 +447,9 @@ case req when "b" w.boot! - log "- Worker #{w.index} (PID: #{pid}) booted, phase: #{w.phase}" + log "- Worker #{w.index} (PID: #{pid}) booted in #{w.uptime.round(2)}s, phase: #{w.phase}" @next_check = Time.now + workers_not_booted -= 1 when "e" # external term, see worker method, Signal.trap "SIGTERM" w.instance_variable_set :@term, true @@ -453,6 +467,10 @@ log "! Out-of-sync worker list, no #{pid} worker" end end + if in_phased_restart && workers_not_booted.zero? + @events.fire_on_booted! + in_phased_restart = false + end rescue Interrupt @status = :stop @@ -470,6 +488,15 @@ private + def single_worker_warning + return if @options[:workers] != 1 || @options[:silence_single_worker_warning] + + log "! WARNING: Detected running cluster mode with 1 worker." + log "! Running Puma in cluster mode with a single worker is often a misconfiguration." + log "! Consider running Puma in single-mode (workers = 0) in order to reduce memory overhead." + log "! Set the `silence_single_worker_warning` option to silence this warning message." + end + # loops thru @workers, removing workers that exited, and calling # `#term` if needed def wait_workers @@ -499,7 +526,12 @@ def timeout_workers @workers.each do |w| if !w.term? && w.ping_timeout <= Time.now - log "! Terminating timed out worker: #{w.pid}" + details = if w.booted? + "(worker failed to check in within #{@options[:worker_timeout]} seconds)" + else + "(worker failed to boot within #{@options[:worker_boot_timeout]} seconds)" + end + log "! Terminating timed out worker #{details}: #{w.pid}" w.kill end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/configuration.rb new/lib/puma/configuration.rb --- old/lib/puma/configuration.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/configuration.rb 2021-05-11 16:54:42.000000000 +0200 @@ -193,6 +193,7 @@ :debug => false, :binds => ["tcp://#{DefaultTCPHost}:#{DefaultTCPPort}"], :workers => Integer(ENV['WEB_CONCURRENCY'] || 0), + :silence_single_worker_warning => false, :mode => :http, :worker_timeout => DefaultWorkerTimeout, :worker_boot_timeout => DefaultWorkerTimeout, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/const.rb new/lib/puma/const.rb --- old/lib/puma/const.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/const.rb 2021-05-11 16:54:42.000000000 +0200 @@ -100,8 +100,8 @@ # too taxing on performance. module Const - PUMA_VERSION = VERSION = "5.2.2".freeze - CODE_NAME = "Fettisdagsbulle".freeze + PUMA_VERSION = VERSION = "5.3.1".freeze + CODE_NAME = "Sweetnighter".freeze PUMA_SERVER_STRING = ['puma', PUMA_VERSION, CODE_NAME].join(' ').freeze @@ -235,7 +235,7 @@ EARLY_HINTS = "rack.early_hints".freeze - # Mininum interval to checks worker health + # Minimum interval to checks worker health WORKER_CHECK_INTERVAL = 5 # Illegal character in the key or value of response header diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/control_cli.rb new/lib/puma/control_cli.rb --- old/lib/puma/control_cli.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/control_cli.rb 2021-05-11 16:54:42.000000000 +0200 @@ -176,7 +176,9 @@ when 'tcp' TCPSocket.new uri.host, uri.port when 'unix' - UNIXSocket.new "#{uri.host}#{uri.path}" + # check for abstract UNIXSocket + UNIXSocket.new(@control_url.start_with?('unix://@') ? + "\0#{uri.host}#{uri.path}" : "#{uri.host}#{uri.path}") else raise "Invalid scheme: #{uri.scheme}" end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/dsl.rb new/lib/puma/dsl.rb --- old/lib/puma/dsl.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/dsl.rb 2021-05-11 16:54:42.000000000 +0200 @@ -482,6 +482,24 @@ @options[:workers] = count.to_i end + # Disable warning message when running in cluster mode with a single worker. + # + # Cluster mode has some overhead of running an additional 'control' process + # in order to manage the cluster. If only running a single worker it is + # likely not worth paying that overhead vs running in single mode with + # additional threads instead. + # + # There are some scenarios where running cluster mode with a single worker + # may still be warranted and valid under certain deployment scenarios, see + # https://github.com/puma/puma/issues/2534 + # + # Moving from workers = 1 to workers = 0 will save 10-30% of memory use. + # + # @note Cluster mode only. + def silence_single_worker_warning + @options[:silence_single_worker_warning] = true + end + # Code to run immediately before master process # forks workers (once on boot). These hooks can block if necessary # to wait for background operations unknown to Puma to finish before diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/error_logger.rb new/lib/puma/error_logger.rb --- old/lib/puma/error_logger.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/error_logger.rb 2021-05-11 16:54:42.000000000 +0200 @@ -23,7 +23,7 @@ new $stderr end - # Print occured error details. + # Print occurred error details. # +options+ hash with additional options: # - +error+ is an exception object # - +req+ the http request @@ -34,7 +34,7 @@ log title(options) end - # Print occured error details only if + # Print occurred error details only if # environment variable PUMA_DEBUG is defined. # +options+ hash with additional options: # - +error+ is an exception object diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/queue_close.rb new/lib/puma/queue_close.rb --- old/lib/puma/queue_close.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/queue_close.rb 2021-05-11 16:54:42.000000000 +0200 @@ -5,22 +5,22 @@ # Add a simple implementation for earlier Ruby versions. # module QueueClose - def initialize - @closed = false - super - end def close + num_waiting.times {push nil} @closed = true end def closed? - @closed + @closed ||= false end def push(object) - @closed ||= false - raise ClosedQueueError if @closed + raise ClosedQueueError if closed? super end alias << push + def pop(non_block=false) + return nil if !non_block && closed? && empty? + super + end end ::Queue.prepend QueueClose end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/request.rb new/lib/puma/request.rb --- old/lib/puma/request.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/request.rb 2021-05-11 16:54:42.000000000 +0200 @@ -148,8 +148,9 @@ res_body.each do |part| next if part.bytesize.zero? if chunked - str = part.bytesize.to_s(16) << line_ending << part << line_ending - fast_write io, str + fast_write io, (part.bytesize.to_s(16) << line_ending) + fast_write io, part # part may have different encoding + fast_write io, line_ending else fast_write io, part end @@ -230,7 +231,11 @@ # def normalize_env(env, client) if host = env[HTTP_HOST] - if colon = host.index(":") + # host can be a hostname, ipv4 or bracketed ipv6. Followed by an optional port. + if colon = host.rindex("]:") # IPV6 with port + env[SERVER_NAME] = host[0, colon+1] + env[SERVER_PORT] = host[colon+2, host.bytesize] + elsif !host.start_with?("[") && colon = host.index(":") # not hostname or IPV4 with port env[SERVER_NAME] = host[0, colon] env[SERVER_PORT] = host[colon+1, host.bytesize] else diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/server.rb new/lib/puma/server.rb --- old/lib/puma/server.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/server.rb 2021-05-11 16:54:42.000000000 +0200 @@ -137,8 +137,6 @@ # socket parameter may be an MiniSSL::Socket, so use to_io # if tcp_cork_supported? - UNPACK_TCP_STATE_FROM_TCP_INFO = "C".freeze - # 6 == Socket::IPPROTO_TCP # 3 == TCP_CORK # 1/0 == turn on/off @@ -168,12 +166,14 @@ end if closed_socket_supported? + UNPACK_TCP_STATE_FROM_TCP_INFO = "C".freeze + def closed_socket?(socket) - return false unless socket.kind_of? TCPSocket - return false unless @precheck_closing + skt = socket.to_io + return false unless skt.kind_of?(TCPSocket) && @precheck_closing begin - tcp_info = socket.getsockopt(Socket::IPPROTO_TCP, Socket::TCP_INFO) + tcp_info = skt.getsockopt(Socket::IPPROTO_TCP, Socket::TCP_INFO) rescue IOError, SystemCallError Thread.current.purge_interrupt_queue if Thread.current.respond_to? :purge_interrupt_queue @precheck_closing = false @@ -295,6 +295,9 @@ @thread_pool << client elsif shutdown || client.timeout == 0 client.timeout! + else + client.set_timeout(@first_data_timeout) + false end rescue StandardError => e client_error(e, client) @@ -308,6 +311,7 @@ sockets = [check] + @binder.ios pool = @thread_pool queue_requests = @queue_requests + drain = @options[:drain_on_shutdown] ? 0 : nil remote_addr_value = nil remote_addr_header = nil @@ -319,22 +323,23 @@ remote_addr_header = @options[:remote_address_header] end - while @status == :run + while @status == :run || (drain && shutting_down?) begin - ios = IO.select sockets + ios = IO.select sockets, nil, nil, (shutting_down? ? 0 : nil) + break unless ios ios.first.each do |sock| if sock == check break if handle_check else pool.wait_until_not_full - pool.wait_for_less_busy_worker( - @options[:wait_for_less_busy_worker].to_f) + pool.wait_for_less_busy_worker(@options[:wait_for_less_busy_worker]) io = begin sock.accept_nonblock rescue IO::WaitReadable next end + drain += 1 if shutting_down? client = Client.new io, @binder.env(sock) if remote_addr_value client.peerip = remote_addr_value @@ -349,6 +354,7 @@ end end + @events.debug "Drained #{drain} additional connections." if drain @events.fire :state, @status if queue_requests @@ -441,15 +447,20 @@ requests += 1 - check_for_more_data = @status == :run + # Closing keepalive sockets after they've made a reasonable + # number of requests allows Puma to service many connections + # fairly, even when the number of concurrent connections exceeds + # the size of the threadpool. It also allows cluster mode Pumas + # to keep load evenly distributed across workers, because clients + # are randomly assigned a new worker when opening a new connection. + # + # Previously, Puma would kick connections in this conditional back + # to the reactor. However, because this causes the todo set to increase + # in size, the wait_until_full mutex would never unlock, leaving + # any additional connections unserviced. + break if requests >= @max_fast_inline - if requests >= @max_fast_inline - # This will mean that reset will only try to use the data it already - # has buffered and won't try to read more data. What this means is that - # every client, independent of their request speed, gets treated like a slow - # one once every max_fast_inline requests. - check_for_more_data = false - end + check_for_more_data = @status == :run next_request_ready = with_force_shutdown(client) do client.reset(check_for_more_data) @@ -526,7 +537,8 @@ end if @leak_stack_on_error - [status, {}, ["Puma caught this error: #{e.message} (#{e.class})\n#{e.backtrace.join("\n")}"]] + backtrace = e.backtrace.nil? ? '<no backtrace available>' : e.backtrace.join("\n") + [status, {}, ["Puma caught this error: #{e.message} (#{e.class})\n#{backtrace}"]] else [status, {}, ["An unhandled lowlevel error occurred. The application logs may have details.\n"]] end @@ -550,28 +562,6 @@ $stdout.syswrite "#{pid}: === End thread backtrace dump ===\n" end - if @options[:drain_on_shutdown] - count = 0 - - while true - ios = IO.select @binder.ios, nil, nil, 0 - break unless ios - - ios.first.each do |sock| - begin - if io = sock.accept_nonblock - count += 1 - client = Client.new io, @binder.env(sock) - @thread_pool << client - end - rescue SystemCallError - end - end - end - - @events.debug "Drained #{count} additional connections." - end - if @status != :restart @binder.close end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma/thread_pool.rb new/lib/puma/thread_pool.rb --- old/lib/puma/thread_pool.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma/thread_pool.rb 2021-05-11 16:54:42.000000000 +0200 @@ -13,7 +13,7 @@ # a thread pool via the `Puma::ThreadPool#<<` operator where it is stored in a `@todo` array. # # Each thread in the pool has an internal loop where it pulls a request from the `@todo` array - # and proceses it. + # and processes it. class ThreadPool class ForceShutdown < RuntimeError end @@ -220,7 +220,7 @@ # then the `@todo` array would stay the same size as the reactor works # to try to buffer the request. In that scenario the next call to this # method would not block and another request would be added into the reactor - # by the server. This would continue until a fully bufferend request + # by the server. This would continue until a fully buffered request # makes it through the reactor and can then be processed by the thread pool. def wait_until_not_full with_mutex do @@ -240,11 +240,12 @@ # @version 5.0.0 def wait_for_less_busy_worker(delay_s) + return unless delay_s && delay_s > 0 + # Ruby MRI does GVL, this can result # in processing contention when multiple threads # (requests) are running concurrently return unless Puma.mri? - return unless delay_s > 0 with_mutex do return if @shutdown diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/puma.rb new/lib/puma.rb --- old/lib/puma.rb 2021-03-02 17:07:50.000000000 +0100 +++ new/lib/puma.rb 2021-05-11 16:54:42.000000000 +0200 @@ -23,6 +23,8 @@ # not in minissl.rb HAS_SSL = const_defined?(:MiniSSL, false) && MiniSSL.const_defined?(:Engine, false) + HAS_UNIX_SOCKET = Object.const_defined? :UNIXSocket + if HAS_SSL require 'puma/minissl' else @@ -37,6 +39,20 @@ HAS_SSL end + def self.abstract_unix_socket? + @abstract_unix ||= + if HAS_UNIX_SOCKET + begin + ::UNIXServer.new("\0puma.temp.unix").close + true + rescue ArgumentError # darwin + false + end + else + false + end + end + # @!attribute [rw] stats_object= def self.stats_object=(val) @get_stats = val diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/metadata new/metadata --- old/metadata 2021-03-02 17:07:50.000000000 +0100 +++ new/metadata 2021-05-11 16:54:42.000000000 +0200 @@ -1,14 +1,14 @@ --- !ruby/object:Gem::Specification name: puma version: !ruby/object:Gem::Version - version: 5.2.2 + version: 5.3.1 platform: ruby authors: - Evan Phoenix autorequire: bindir: bin cert_chain: [] -date: 2021-03-02 00:00:00.000000000 Z +date: 2021-05-11 00:00:00.000000000 Z dependencies: - !ruby/object:Gem::Dependency name: nio4r
