Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package fortio for openSUSE:Factory checked
in at 2022-09-21 14:43:12
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/fortio (Old)
and /work/SRC/openSUSE:Factory/.fortio.new.2083 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "fortio"
Wed Sep 21 14:43:12 2022 rev:21 rq:1005152 version:1.38.0
Changes:
--------
--- /work/SRC/openSUSE:Factory/fortio/fortio.changes 2022-09-14
13:45:10.157922526 +0200
+++ /work/SRC/openSUSE:Factory/.fortio.new.2083/fortio.changes 2022-09-21
14:44:11.422036955 +0200
@@ -1,0 +2,7 @@
+Wed Sep 21 08:29:20 UTC 2022 - ka...@b1-systems.de
+
+- Update to version 1.38.0:
+ * only do compression if requested explictly when forwarding/fetching (#625)
+ * add connection times histogram (#626)
+
+-------------------------------------------------------------------
Old:
----
fortio-1.37.1.tar.gz
New:
----
fortio-1.38.0.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ fortio.spec ++++++
--- /var/tmp/diff_new_pack.BgHovk/_old 2022-09-21 14:44:11.942038312 +0200
+++ /var/tmp/diff_new_pack.BgHovk/_new 2022-09-21 14:44:11.946038323 +0200
@@ -19,7 +19,7 @@
%define __arch_install_post export NO_BRP_STRIP_DEBUG=true
Name: fortio
-Version: 1.37.1
+Version: 1.38.0
Release: 0
Summary: Load testing library, command line tool, advanced echo server
and web UI
License: Apache-2.0
++++++ _service ++++++
--- /var/tmp/diff_new_pack.BgHovk/_old 2022-09-21 14:44:11.982038417 +0200
+++ /var/tmp/diff_new_pack.BgHovk/_new 2022-09-21 14:44:11.986038427 +0200
@@ -3,7 +3,7 @@
<param name="url">https://github.com/fortio/fortio</param>
<param name="scm">git</param>
<param name="exclude">.git</param>
- <param name="revision">v1.37.1</param>
+ <param name="revision">v1.38.0</param>
<param name="versionformat">@PARENT_TAG@</param>
<param name="changesgenerate">enable</param>
<param name="versionrewrite-pattern">v(.*)</param>
@@ -17,7 +17,7 @@
<param name="compression">gz</param>
</service>
<service name="go_modules" mode="disabled">
- <param name="archive">fortio-1.37.1.tar.gz</param>
+ <param name="archive">fortio-1.38.0.tar.gz</param>
</service>
</services>
++++++ _servicedata ++++++
--- /var/tmp/diff_new_pack.BgHovk/_old 2022-09-21 14:44:12.006038480 +0200
+++ /var/tmp/diff_new_pack.BgHovk/_new 2022-09-21 14:44:12.010038490 +0200
@@ -1,6 +1,6 @@
<servicedata>
<service name="tar_scm">
<param name="url">https://github.com/fortio/fortio</param>
- <param
name="changesrevision">0fa89821fab2d9451802ea6195b721d4a94d65a7</param></service></servicedata>
+ <param
name="changesrevision">c9eaf5e4579a36ca1e341a64873626fa49fb97b2</param></service></servicedata>
(No newline at EOF)
++++++ fortio-1.37.1.tar.gz -> fortio-1.38.0.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/README.md new/fortio-1.38.0/README.md
--- old/fortio-1.37.1/README.md 2022-09-13 19:31:18.000000000 +0200
+++ new/fortio-1.38.0/README.md 2022-09-19 23:06:27.000000000 +0200
@@ -52,13 +52,13 @@
The [releases](https://github.com/fortio/fortio/releases) page has binaries
for many OS/architecture combinations (see assets).
```shell
-curl -L
https://github.com/fortio/fortio/releases/download/v1.37.1/fortio-linux_amd64-1.37.1.tgz
\
+curl -L
https://github.com/fortio/fortio/releases/download/v1.38.0/fortio-linux_amd64-1.38.0.tgz
\
| sudo tar -C / -xvzpf -
# or the debian package
-wget
https://github.com/fortio/fortio/releases/download/v1.37.1/fortio_1.37.1_amd64.deb
-dpkg -i fortio_1.37.1_amd64.deb
+wget
https://github.com/fortio/fortio/releases/download/v1.38.0/fortio_1.38.0_amd64.deb
+dpkg -i fortio_1.38.0_amd64.deb
# or the rpm
-rpm -i
https://github.com/fortio/fortio/releases/download/v1.37.1/fortio-1.37.1-1.x86_64.rpm
+rpm -i
https://github.com/fortio/fortio/releases/download/v1.38.0/fortio-1.38.0-1.x86_64.rpm
# and more, see assets in release page
```
@@ -68,7 +68,7 @@
brew install fortio
```
-On Windows, download
https://github.com/fortio/fortio/releases/download/v1.37.1/fortio_win_1.37.1.zip
and extract `fortio.exe` to any location, then using the Windows Command
Prompt:
+On Windows, download
https://github.com/fortio/fortio/releases/download/v1.38.0/fortio_win_1.38.0.zip
and extract `fortio.exe` to any location, then using the Windows Command
Prompt:
```
fortio.exe server
```
@@ -116,7 +116,7 @@
<details>
<!-- use release/updateFlags.sh to update this section -->
<pre>
-???????????? 1.37.1 usage:
+???????????? 1.38.0 usage:
fortio command [flags] target
where command is one of: load (load testing), server (starts ui, rest api,
http-echo, redirect, proxies, tcp-echo and grpc ping servers), tcp-echo (only
@@ -360,6 +360,8 @@
| header | header(s) to add to the reply e.g. `&header=Foo:Bar&header=X:Y` |
| gzip | If `Accept-Encoding: gzip` is passed in headers by the
caller/client; and `gzip=true` is in the query args, all response will be
gzipped; or if `gzip=42.7` is passed, approximately 42.7% will|
+`delay`, `close` and `header` query arguments are also supported for the
`debug` endpoint which echoes back the request (gzip is always done if
`Accept-Encoding: gzip` is present, status is always 200, and the payload is
the echo back debug information).
+
You can set a default value for all these by passing
`-echo-server-default-params` to the server command line, for instance:
`fortio server -echo-server-default-params="delay=0.5s:50,1s:40&status=418"`
will make the server respond with http 418 and a delay of either 0.5s half of
the time, 1s 40% and no delay in 10% of the calls; unless any `?` query args is
passed by the client. Note that the quotes (") are for the shell to escape
the ampersand (&) but should not be put in a yaml nor the dynamicflag url
for instance.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/http_client.go
new/fortio-1.38.0/fhttp/http_client.go
--- old/fortio-1.37.1/fhttp/http_client.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/fhttp/http_client.go 2022-09-19 23:06:27.000000000
+0200
@@ -45,9 +45,9 @@
Fetch() (int, []byte, int)
// Close() cleans up connections and state - must be paired with
NewClient calls.
Close()
- // GetIPAddress() returns the occurrence of ip address used by this
client connection.
- // and how many sockets have been used
- GetIPAddress() (*stats.Occurrence, int)
+ // GetIPAddress() returns the occurrence of ip address used by this
client connection,
+ // and the connection time histogram (which includes the count).
+ GetIPAddress() (*stats.Occurrence, *stats.Histogram)
}
const (
@@ -90,6 +90,9 @@
log.Warnf("Invalid timeout %v, setting to %v",
h.HTTPReqTimeOut, HTTPReqTimeOutDefaultValue)
h.HTTPReqTimeOut = HTTPReqTimeOutDefaultValue
}
+ if h.Resolution <= 0 {
+ h.Resolution = 0.001
+ }
h.URLSchemeCheck()
return h
}
@@ -185,6 +188,10 @@
ConnReuseRange [2]int // range of max number of connection to
reuse for each thread.
// When false, re-resolve the DNS name when the connection breaks.
NoResolveEachConn bool
+ // Optional Offset Duration; to offset the histogram of the Connection
duration
+ Offset time.Duration
+ // Optional resolution divider for the Connection duration histogram.
In seconds. Defaults to 0.001 or 1 millisecond.
+ Resolution float64
}
// ResetHeaders resets all the headers, including the User-Agent: one (and the
Host: logical special header).
@@ -354,8 +361,8 @@
bodyContainsUUID bool // if body contains the "{uuid}" pattern
(lowercase)
logErrors bool
id int
- socketCount int
ipAddrUsage *stats.Occurrence
+ connectStats *stats.Histogram
}
// Close cleans up any resources used by NewStdClient.
@@ -443,9 +450,9 @@
return code, data, 0
}
-// GetIPAddress get the ip address that DNS resolves to when using stdClient.
-func (c *Client) GetIPAddress() (*stats.Occurrence, int) {
- return c.ipAddrUsage, c.socketCount
+// GetIPAddress get the ip address that DNS resolves to when using stdClient
and connection stats.
+func (c *Client) GetIPAddress() (*stats.Occurrence, *stats.Histogram) {
+ return c.ipAddrUsage, c.connectStats
}
// NewClient creates either a standard or fast client (depending on
@@ -483,6 +490,8 @@
id: o.ID,
logErrors: o.LogErrors,
ipAddrUsage: stats.NewOccurrence(),
+ // Keep track of timing for connection (re)establishment.
+ connectStats: stats.NewHistogram(o.Offset.Seconds(),
o.Resolution),
}
tr := http.Transport{
@@ -497,10 +506,11 @@
addr = o.Resolve + addr[strings.LastIndex(addr,
":"):]
}
var conn net.Conn
+ now := time.Now()
conn, err = (&net.Dialer{
Timeout: o.HTTPReqTimeOut,
}).DialContext(ctx, network, addr)
-
+ client.connectStats.Record(time.Since(now).Seconds())
if conn != nil {
newRemoteAddress := conn.RemoteAddr().String()
// No change when it wasn't set before (first
time) and when the value isn't actually changing either.
@@ -508,10 +518,8 @@
log.Infof("[%d] Standard client IP
address changed from %s to %s", client.id, req.RemoteAddr, newRemoteAddress)
}
req.RemoteAddr = newRemoteAddress
- client.socketCount++
client.ipAddrUsage.Record(req.RemoteAddr)
}
-
return conn, err
},
TLSHandshakeTimeout: o.HTTPReqTimeOut,
@@ -561,7 +569,7 @@
req []byte
dest net.Addr
socket net.Conn
- socketCount int
+ socketCount int // number of sockets attempts, same as the new
connectStats.Count() + DNS errors if any.
size int
code int
errorCount int
@@ -588,11 +596,12 @@
connReuseRange [2]int
connReuse int
reuseCount int
+ connectStats *stats.Histogram
}
-// GetIPAddress get ip address that DNS resolved to when using fast client.
-func (c *FastClient) GetIPAddress() (*stats.Occurrence, int) {
- return c.ipAddrUsage, c.socketCount
+// GetIPAddress get ip address that DNS resolved to when using fast client and
connection stats.
+func (c *FastClient) GetIPAddress() (*stats.Occurrence, *stats.Histogram) {
+ return c.ipAddrUsage, c.connectStats
}
// Close cleans up any resources used by FastClient.
@@ -657,6 +666,8 @@
http10: o.HTTP10, halfClose: o.AllowHalfClose, logErrors:
o.LogErrors, id: o.ID,
https: o.https, connReuseRange: o.ConnReuseRange, connReuse:
connReuse,
resolve: o.Resolve, noResolveEachConn: o.NoResolveEachConn,
ipAddrUsage: stats.NewOccurrence(),
+ // Keep track of timing for connection (re)establishment.
+ connectStats: stats.NewHistogram(o.Offset.Seconds(),
o.Resolution),
}
if o.https {
bc.tlsConfig, err = o.TLSOptions.TLSClientConfig()
@@ -751,14 +762,17 @@
}
d := &net.Dialer{Timeout: c.reqTimeout}
+ now := time.Now()
if c.https {
socket, err = tls.DialWithDialer(d, c.dest.Network(),
c.dest.String(), c.tlsConfig)
+ c.connectStats.Record(time.Since(now).Seconds())
if err != nil {
log.Errf("[%d] Unable to TLS connect to %v : %v", c.id,
c.dest, err)
return nil
}
} else {
socket, err = d.Dial(c.dest.Network(), c.dest.String())
+ c.connectStats.Record(time.Since(now).Seconds())
if err != nil {
log.Errf("[%d] Unable to connect to %v : %v", c.id,
c.dest, err)
return nil
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/http_forwarder.go
new/fortio-1.38.0/fhttp/http_forwarder.go
--- old/fortio-1.37.1/fhttp/http_forwarder.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/fhttp/http_forwarder.go 2022-09-19 23:06:27.000000000
+0200
@@ -247,6 +247,9 @@
// TODO make configurable, should be fine for now for
most but extreme -c values
MaxIdleConnsPerHost: 128, // must be more than incoming
parallelization; divided by number of fan out if using parallel mode
MaxIdleConns: 256,
+ // This avoids Accept-Encoding: gzip being added to
outgoing requests when no encoding accept is specified
+ // yet if passed by request, it will do gzip end to
end. Issue #624.
+ DisableCompression: true,
},
}
return client
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/http_forwarder_test.go
new/fortio-1.38.0/fhttp/http_forwarder_test.go
--- old/fortio-1.37.1/fhttp/http_forwarder_test.go 2022-09-13
19:31:18.000000000 +0200
+++ new/fortio-1.38.0/fhttp/http_forwarder_test.go 2022-09-19
23:06:27.000000000 +0200
@@ -43,6 +43,10 @@
if !bytes.Contains(data, []byte(payload)) {
t.Errorf("Result %s doesn't contain expected payload
echo back %q", DebugSummary(data, 1024), payload)
}
+ // Issue #624
+ if bytes.Contains(data, []byte("gzip")) {
+ t.Errorf("Result %s contains unexpected gzip (accept
encoding)", DebugSummary(data, 1024))
+ }
if !bytes.Contains(data, []byte("X-Fortio-Multi-Id: 1")) {
t.Errorf("Result %s doesn't contain expected
X-Fortio-Multi-Id: 1", DebugSummary(data, 1024))
}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/http_loglevel_test.go
new/fortio-1.38.0/fhttp/http_loglevel_test.go
--- old/fortio-1.37.1/fhttp/http_loglevel_test.go 2022-09-13
19:31:18.000000000 +0200
+++ new/fortio-1.38.0/fhttp/http_loglevel_test.go 2022-09-19
23:06:27.000000000 +0200
@@ -32,3 +32,8 @@
TestNoFirstChunkSizeInitially(t)
TestFetchAndOnBehalfOf(t)
}
+
+func TesWarningMode(t *testing.T) {
+ log.SetLogLevel(log.Warning)
+ TestHTTPRunner(t)
+}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/http_server.go
new/fortio-1.38.0/fhttp/http_server.go
--- old/fortio-1.37.1/fhttp/http_server.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/fhttp/http_server.go 2022-09-19 23:06:27.000000000
+0200
@@ -33,6 +33,7 @@
"fortio.org/fortio/dflag"
"fortio.org/fortio/fnet"
+ "fortio.org/fortio/jrpc"
"fortio.org/fortio/log"
"fortio.org/fortio/version"
"golang.org/x/net/http2"
@@ -81,11 +82,7 @@
return
}
log.Debugf("Read %d", len(data))
- dur := generateDelay(r.FormValue("delay"))
- if dur > 0 {
- log.LogVf("Sleeping for %v", dur)
- time.Sleep(dur)
- }
+ handleCommonArgs(w, r)
statusStr := r.FormValue("status")
var status int
if statusStr != "" {
@@ -93,34 +90,12 @@
} else {
status = http.StatusOK
}
- if log.LogDebug() {
- // TODO: this easily lead to contention - use 'thread local'
- rqNum := atomic.AddInt64(&EchoRequests, 1)
- log.Debugf("Request # %v", rqNum)
- }
- if generateClose(r.FormValue("close")) {
- log.Debugf("Adding Connection:close / will close socket")
- w.Header().Set("Connection", "close")
- }
gzip := strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") &&
generateGzip(r.FormValue("gzip"))
if gzip {
gwz := NewGzipHTTPResponseWriter(w)
defer gwz.Close()
w = gwz
}
- // process header(s) args, must be before size to compose properly
- for _, hdr := range r.Form["header"] {
- log.LogVf("Adding requested header %s", hdr)
- if len(hdr) == 0 {
- continue
- }
- s := strings.SplitN(hdr, ":", 2)
- if len(s) != 2 {
- log.Errf("invalid extra header '%s', expecting Key:
Value", hdr)
- continue
- }
- w.Header().Add(s[0], s[1])
- }
size := generateSize(r.FormValue("size"))
if size >= 0 {
log.LogVf("Writing %d size with %d status", size, status)
@@ -130,7 +105,7 @@
// echo back the Content-Type and Content-Length in the response
for _, k := range []string{"Content-Type", "Content-Length"} {
if v := r.Header.Get(k); v != "" {
- w.Header().Set(k, v)
+ jrpc.SetHeaderIfMissing(w.Header(), k, v)
}
}
w.WriteHeader(status)
@@ -139,8 +114,40 @@
}
}
+// handleCommonArgs common flags for debug and echo handlers.
+// Must be called after body is read.
+func handleCommonArgs(w http.ResponseWriter, r *http.Request) {
+ dur := generateDelay(r.FormValue("delay"))
+ if dur > 0 {
+ log.LogVf("Sleeping for %v", dur)
+ time.Sleep(dur)
+ }
+ if log.LogDebug() {
+ // Note this easily lead to contention, debug mode only (or low
qps).
+ rqNum := atomic.AddInt64(&EchoRequests, 1)
+ log.Debugf("Request # %v", rqNum)
+ }
+ if generateClose(r.FormValue("close")) {
+ log.Debugf("Adding Connection:close / will close socket")
+ w.Header().Set("Connection", "close")
+ }
+ // process header(s) args, must be before size to compose properly
+ for _, hdr := range r.Form["header"] {
+ log.LogVf("Adding requested header %s", hdr)
+ if len(hdr) == 0 {
+ continue
+ }
+ s := strings.SplitN(hdr, ":", 2)
+ if len(s) != 2 {
+ log.Errf("invalid extra header '%s', expecting Key:
Value", hdr)
+ continue
+ }
+ w.Header().Add(s[0], s[1])
+ }
+}
+
func writePayload(w http.ResponseWriter, status int, size int) {
- w.Header().Set("Content-Type", "application/octet-stream")
+ jrpc.SetHeaderIfMissing(w.Header(), "Content-Type",
"application/octet-stream")
w.Header().Set("Content-Length", strconv.Itoa(size))
w.WriteHeader(status)
n, err := w.Write(fnet.Payload[:size])
@@ -291,7 +298,7 @@
buf.WriteString("Host: ")
buf.WriteString(r.Host)
- var keys []string //nolint:prealloc // header is multi valued map,...
+ keys := make([]string, 0, len(r.Header))
for k := range r.Header {
keys = append(keys, k)
}
@@ -337,7 +344,8 @@
buf.WriteByte('\n')
}
}
- w.Header().Set("Content-Type", "text/plain; charset=UTF-8")
+ handleCommonArgs(w, r)
+ jrpc.SetHeaderIfMissing(w.Header(), "Content-Type", "text/plain;
charset=UTF-8")
if _, err = w.Write(buf.Bytes()); err != nil {
log.Errf("Error writing response %v to %v", err, r.RemoteAddr)
}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/http_test.go
new/fortio-1.38.0/fhttp/http_test.go
--- old/fortio-1.37.1/fhttp/http_test.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/fhttp/http_test.go 2022-09-19 23:06:27.000000000
+0200
@@ -1107,34 +1107,46 @@
}
}
+// TestDebugHandlerSortedHeaders tests the headers are sorted but
+// also tests post echo back and gzip handling.
func TestDebugHandlerSortedHeaders(t *testing.T) {
m, a := DynamicHTTPServer(false)
- m.HandleFunc("/debug", DebugHandler)
- url := fmt.Sprintf("http://localhost:%d/debug";, a.Port)
- o := HTTPOptions{URL: url, DisableFastClient: true}
+ m.Handle("/debug", Gzip(http.HandlerFunc(DebugHandler))) // same as in
Serve()
+ // Debug handler does respect the delay arg but not status, status is
always 200
+ url := fmt.Sprintf("http://localhost:%d/debug?delay=500ms&status=555";,
a.Port)
+ // Trigger transparent compression (which will add Accept-Encoding:
gzip header)
+ o := HTTPOptions{URL: url, DisableFastClient: true, Compression: true,
Payload: []byte("abcd")}
o.AddAndValidateExtraHeader("BBB: bbb")
o.AddAndValidateExtraHeader("CCC: ccc")
o.AddAndValidateExtraHeader("ZZZ: zzz")
o.AddAndValidateExtraHeader("AAA: aaa")
client, _ := NewClient(&o)
+ now := time.Now()
code, data, header := client.Fetch() // used to panic/bug #127
+ duration := time.Since(now)
t.Logf("TestDebugHandlerSortedHeaders result code %d, data len %d,
headerlen %d", code, len(data), header)
if code != http.StatusOK {
t.Errorf("Got %d instead of 200", code)
}
+ if duration < 500*time.Millisecond {
+ t.Errorf("Got %s instead of 500ms", duration)
+ }
// remove the first line ('???????????? version...') from the body
body := string(data)
i := strings.Index(body, "\n")
body = body[i+1:]
- expected := fmt.Sprintf("\nGET /debug HTTP/1.1\n\n"+
+ expected := fmt.Sprintf("\nPOST /debug?delay=500ms&status=555
HTTP/1.1\n\n"+
"headers:\n\n"+
"Host: localhost:%d\n"+
"Aaa: aaa\n"+
+ "Accept-Encoding: gzip\n"+
"Bbb: bbb\n"+
"Ccc: ccc\n"+
+ "Content-Length: 4\n"+
+ "Content-Type: application/octet-stream\n"+
"User-Agent: %s\n"+
"Zzz: zzz\n\n"+
- "body:\n\n\n", a.Port, jrpc.UserAgent)
+ "body:\n\nabcd\n", a.Port, jrpc.UserAgent)
if body != expected {
t.Errorf("Get body: %s not as expected: %s", body, expected)
}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/httprunner.go
new/fortio-1.38.0/fhttp/httprunner.go
--- old/fortio-1.37.1/fhttp/httprunner.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/fhttp/httprunner.go 2022-09-19 23:06:27.000000000
+0200
@@ -45,8 +45,10 @@
HTTPOptions
Sizes *stats.HistogramData
HeaderSizes *stats.HistogramData
- Sockets []int
- SocketCount int
+ Sockets []int64
+ SocketCount int64
+ // Connection Time stats
+ ConnectionStats *stats.HistogramData
// http code to abort the run on (-1 for connection or other socket
error)
AbortOn int
aborter *periodic.Aborter
@@ -85,7 +87,7 @@
// RunHTTPTest runs an http test and returns the aggregated stats.
//
-//nolint:funlen, gocognit, gocyclo
+//nolint:funlen, gocognit, gocyclo, maintidx
func RunHTTPTest(o *HTTPRunnerOptions) (*HTTPRunnerResults, error) {
o.RunType = "HTTP"
warmupMode := "parallel"
@@ -100,6 +102,12 @@
log.Infof("Starting http test for %s with %d threads at %.1f qps and %s
warmup%s",
o.URL, o.NumThreads, o.QPS, warmupMode, connReuseMsg)
r := periodic.NewPeriodicRunner(&o.RunnerOptions)
+ if o.HTTPOptions.Resolution <= 0 {
+ // Set both connect histogram params when Resolution isn't set
explicitly on the HTTP options
+ // (that way you can set the offet to 0 in connect and to
something else for the call)
+ o.HTTPOptions.Resolution = r.Options().Resolution
+ o.HTTPOptions.Offset = r.Options().Offset
+ }
defer r.Options().Abort()
numThreads := r.Options().NumThreads // can change during run for c > 2
n
o.HTTPOptions.Init(o.URL)
@@ -188,6 +196,8 @@
fm.Close()
_, _ = fmt.Fprintf(out, "Wrote profile data to %s.{cpu|mem}\n",
o.Profiler)
}
+ // Connection stats, aggregated
+ connectionStats := stats.NewHistogram(o.HTTPOptions.Offset.Seconds(),
o.HTTPOptions.Resolution)
// Numthreads may have reduced:
numThreads = total.RunnerResults.NumThreads
// But we also must cleanup all the created clients.
@@ -195,10 +205,11 @@
fmt.Fprintf(out, "# Socket and IP used for each connection:\n")
for i := 0; i < numThreads; i++ {
// Get the report on the IP address each thread use to send
traffic
- occurrence, currentSocketUsed :=
httpstate[i].client.GetIPAddress()
+ occurrence, connStats := httpstate[i].client.GetIPAddress()
+ currentSocketUsed := connStats.Count
httpstate[i].client.Close()
- fmt.Fprintf(out, "[%d] %3d socket used, resolved to %s\n", i,
currentSocketUsed, occurrence.PrintAndAggregate(total.IPCountMap))
-
+ fmt.Fprintf(out, "[%d] %3d socket used, resolved to %s ", i,
currentSocketUsed, occurrence.PrintAndAggregate(total.IPCountMap))
+ connStats.Counter.Print(out, "connection timing")
total.SocketCount += currentSocketUsed
total.Sockets = append(total.Sockets, currentSocketUsed)
// Q: is there some copying each time stats[i] is used?
@@ -210,6 +221,13 @@
}
total.sizes.Transfer(httpstate[i].sizes)
total.headerSizes.Transfer(httpstate[i].headerSizes)
+ connectionStats.Transfer(connStats)
+ }
+ total.ConnectionStats =
connectionStats.Export().CalcPercentiles(o.Percentiles)
+ if log.Log(log.Info) {
+ total.ConnectionStats.Print(out, "Connection time histogram
(s)")
+ } else if log.Log(log.Warning) {
+ connectionStats.Counter.Print(out, "Connection time (s)")
}
// Sort the ip address form largest to smallest based on its usage count
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/fhttp/httprunner_test.go
new/fortio-1.38.0/fhttp/httprunner_test.go
--- old/fortio-1.37.1/fhttp/httprunner_test.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/fhttp/httprunner_test.go 2022-09-19 23:06:27.000000000
+0200
@@ -58,7 +58,7 @@
if totalReq != httpOk {
t.Errorf("Mismatch between requests %d and ok %v", totalReq,
res.RetCodes)
}
- if res.SocketCount != res.RunnerResults.NumThreads {
+ if res.SocketCount != int64(res.RunnerResults.NumThreads) {
t.Errorf("%d socket used, expected same as thread# %d",
res.SocketCount, res.RunnerResults.NumThreads)
}
count := getIPUsageCount(res.IPCountMap)
@@ -126,7 +126,7 @@
if ngAfter > ngBefore2+8 {
t.Errorf("Goroutines after test %d, expected it to stay near
%d", ngAfter, ngBefore2)
}
- if res.SocketCount != res.RunnerResults.NumThreads {
+ if res.SocketCount != int64(res.RunnerResults.NumThreads) {
t.Errorf("%d socket used, expected same as thread# %d",
res.SocketCount, res.RunnerResults.NumThreads)
}
}
@@ -213,7 +213,7 @@
if totalReq != httpOk {
t.Errorf("Mismatch between requests %d and ok %v", totalReq,
res.RetCodes)
}
- if int64(res.SocketCount) != numReq {
+ if res.SocketCount != numReq {
t.Errorf("When closing, got %d while expected as many sockets
as requests %d", res.SocketCount, numReq)
}
}
@@ -502,14 +502,14 @@
t.Error(err)
}
- if res.SocketCount != (int)(expectedSocketReuse) {
+ if res.SocketCount != (int64)(expectedSocketReuse) {
t.Errorf("Expecting %f socket to be used, got %d",
expectedSocketReuse, res.SocketCount)
}
}
// Test when connection reuse range min != max.
// The actual socket count should always be 2 as the connection reuse
range varies between 5 and 9.
- expectedSocketReuse := 2
+ expectedSocketReuse := int64(2)
opts.ConnReuseRange = [2]int{5, 9}
// Check a few times that despite the range and random 2-9 we still
always get 2 connections
for i := 0; i < 5; i++ {
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/periodic/periodic.go
new/fortio-1.38.0/periodic/periodic.go
--- old/fortio-1.37.1/periodic/periodic.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/periodic/periodic.go 2022-09-19 23:06:27.000000000
+0200
@@ -105,12 +105,13 @@
return
}
a.stopRequested = true
- if a.hasStarted || !wait {
+ started := a.hasStarted
+ if started || !wait {
log.LogVf("ABORT Closing %v", a)
close(a.StopChan)
a.StopChan = nil
a.Unlock()
- if a.hasStarted {
+ if started {
log.LogVf("ABORT reading start channel")
// shouldn't block/hang, just purging/resetting
<-a.StartChan
@@ -171,9 +172,11 @@
// Note that this actually maps to gorountines and not actual threads
// but threads seems like a more familiar name to use for non go users
// and in a benchmarking context
- NumThreads int
+ NumThreads int
+ // List of percentiles to calculate.
Percentiles []float64
- Resolution float64
+ // Divider to apply to duration data in seconds. Defaults to 0.001 or 1
millisecond.
+ Resolution float64
// Where to write the textual version of the results, defaults to stdout
Out io.Writer `json:"-"`
// Extra data to be copied back to the results (to be saved/JSON
serialized)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/rapi/restHandler_test.go
new/fortio-1.38.0/rapi/restHandler_test.go
--- old/fortio-1.37.1/rapi/restHandler_test.go 2022-09-13 19:31:18.000000000
+0200
+++ new/fortio-1.38.0/rapi/restHandler_test.go 2022-09-19 23:06:27.000000000
+0200
@@ -117,7 +117,7 @@
if totalReq != httpOk {
t.Errorf("Mismatch between requests %d and ok %v (%+v)",
totalReq, res.RetCodes, res)
}
- if res.SocketCount != res.RunnerResults.NumThreads {
+ if res.SocketCount != int64(res.RunnerResults.NumThreads) {
t.Errorf("%d socket used, expected same as thread# %d",
res.SocketCount, res.RunnerResults.NumThreads)
}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/fortio-1.37.1/stats/stats.go
new/fortio-1.38.0/stats/stats.go
--- old/fortio-1.37.1/stats/stats.go 2022-09-13 19:31:18.000000000 +0200
+++ new/fortio-1.38.0/stats/stats.go 2022-09-19 23:06:27.000000000 +0200
@@ -196,7 +196,7 @@
Avg float64
StdDev float64
Data []Bucket
- Percentiles []Percentile
+ Percentiles []Percentile `json:"Percentiles,omitempty"`
}
// NewHistogram creates a new histogram (sets up the buckets).
++++++ vendor.tar.gz ++++++
