Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package mtail for openSUSE:Factory checked
in at 2021-11-22 23:04:37
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/mtail (Old)
and /work/SRC/openSUSE:Factory/.mtail.new.1895 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "mtail"
Mon Nov 22 23:04:37 2021 rev:2 rq:933157 version:3.0.0rc48
Changes:
--------
--- /work/SRC/openSUSE:Factory/mtail/mtail.changes 2021-10-11
15:31:58.438879282 +0200
+++ /work/SRC/openSUSE:Factory/.mtail.new.1895/mtail.changes 2021-11-22
23:05:35.093649999 +0100
@@ -1,0 +2,42 @@
+Mon Nov 22 19:35:22 UTC 2021 - mich...@stroeder.com
+
+- Update to version 3.0.0rc48:
+ * Fix small syntax problems in interoperability docs
+ * Bump lewagon/wait-on-check-action from 1.0.0 to 1.1.1
+ * Make one-shot Prometheus emit prog label and metric timestamp flags
+ * Bump codecov/codecov-action from 2.0.3 to 2.1.0
+ * Change the flag to -one_shot_format
+ * Add -prometheus_output to -one_shot
+ * Exit with 0 code when --one_shot is set
+ * Bump codecov/codecov-action from 2.0.2 to 2.0.3
+ * Bump lewagon/wait-on-check-action from 0.2 to 1.0.0
+ * Bump actions/github-script from 4.0.2 to 4.1
+ * Comment out permissions in automerge as they're incorrect.
+ * examples/apache_metrics.mtail: fix http_request_duration_seconds division
+ * Fix permissions for the publish action.
+ * Fix a typo.
+ * Update last workflows with minimum permissions.
+ * Add permissions to more workflows.
+ * Make a yaml-valid version of no permissions on the token.
+ * Set minimum permissions for benchmarking and uploading benchmark and CI
resulrts.
+ * Set minimum permissions on auto-review.
+ * Remove unnecessary comment.
+ * Set minimum required permissions on automerge action.
+ * Run security scan on pull requests, not pushes.
+ * Bump github.com/prometheus/common from 0.29.0 to 0.30.0
+ * Remove uninclusive language.
+ * Bump codecov/codecov-action from 2.0.1 to 2.0.2
+ * Bump codecov/codecov-action from 1 to 2.0.1
+ * doc fix
+
+-------------------------------------------------------------------
+Fri Oct 8 16:08:11 UTC 2021 - Michael Str??der <mich...@stroeder.com>
+
+- some more spec-cleanup
+
+-------------------------------------------------------------------
+Fri Oct 8 15:40:04 UTC 2021 - Michael Str??der <mich...@stroeder.com>
+
+- use %go_nostrip macro
+
+-------------------------------------------------------------------
Old:
----
mtail-3.0.0rc47.tar.gz
New:
----
mtail-3.0.0rc48.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ mtail.spec ++++++
--- /var/tmp/diff_new_pack.po76M7/_old 2021-11-22 23:05:35.677648039 +0100
+++ /var/tmp/diff_new_pack.po76M7/_new 2021-11-22 23:05:35.677648039 +0100
@@ -17,15 +17,14 @@
# nodebuginfo
-# Remove stripping of Go binaries.
-%define __arch_install_post export NO_BRP_STRIP_DEBUG=true
+%{go_nostrip}
# Project name when using go tooling.
%define project github.com/google/mtail
%bcond_without apparmor
Name: mtail
-Version: 3.0.0rc47
+Version: 3.0.0rc48
Release: 0
Summary: Tool for extracting metrics from application logs
License: Apache-2.0
@@ -37,9 +36,9 @@
Source3: %{name}.sysconfig
Source4: apparmor-usr.sbin.%{name}
BuildRequires: golang-packaging
-BuildRequires: pkg-config
-BuildRequires: golang(API) >= 1.16
-Requires(pre): %fillup_prereq
+BuildRequires: golang(API)
+Requires(post): %fillup_prereq
+Requires(pre): shadow
%if %{with apparmor}
BuildRequires: apparmor-abstractions
BuildRequires: apparmor-rpm-macros
++++++ _service ++++++
--- /var/tmp/diff_new_pack.po76M7/_old 2021-11-22 23:05:35.701647958 +0100
+++ /var/tmp/diff_new_pack.po76M7/_new 2021-11-22 23:05:35.701647958 +0100
@@ -3,10 +3,10 @@
<param name="url">git://github.com/google/mtail.git</param>
<param name="scm">git</param>
<param name="exclude">.git</param>
- <param name="revision">v3.0.0-rc47</param>
+ <param name="revision">3.0.0-rc48</param>
<param name="versionformat">@PARENT_TAG@</param>
<param name="changesgenerate">enable</param>
- <param name="versionrewrite-pattern">v(.+)-rc(.+)</param>
+ <param name="versionrewrite-pattern">(.+)-rc(.+)</param>
<param name="versionrewrite-replacement">\1rc\2</param>
</service>
<service name="recompress" mode="disabled">
++++++ _servicedata ++++++
--- /var/tmp/diff_new_pack.po76M7/_old 2021-11-22 23:05:35.717647905 +0100
+++ /var/tmp/diff_new_pack.po76M7/_new 2021-11-22 23:05:35.717647905 +0100
@@ -1,4 +1,4 @@
<servicedata>
<service name="tar_scm">
<param name="url">git://github.com/google/mtail.git</param>
- <param
name="changesrevision">e6d88064340b0f01252f26d86cf04c57bc010c71</param></service></servicedata>
\ No newline at end of file
+ <param
name="changesrevision">baede67a6de1067a1d101b2978d7954d3fcd9435</param></service></servicedata>
\ No newline at end of file
++++++ mtail-3.0.0rc47.tar.gz -> mtail-3.0.0rc48.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/ISSUE_TEMPLATE/bug_report.md
new/mtail-3.0.0rc48/.github/ISSUE_TEMPLATE/bug_report.md
--- old/mtail-3.0.0rc47/.github/ISSUE_TEMPLATE/bug_report.md 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/ISSUE_TEMPLATE/bug_report.md 2021-09-18
00:24:42.000000000 +0200
@@ -15,7 +15,7 @@
If your problem is with `mtail`, please include the commandline you started it
with, and the INFO log.
-See also [Reporting a
problem](https://github.com/google/mtail/blob/master/docs/Troubleshooting.md#reporting-a-problem).
+See also [Reporting a
problem](https://github.com/google/mtail/blob/main/docs/Troubleshooting.md#reporting-a-problem).
Thanks!
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/auto-review.yml
new/mtail-3.0.0rc48/.github/workflows/auto-review.yml
--- old/mtail-3.0.0rc47/.github/workflows/auto-review.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/auto-review.yml 2021-09-18
00:24:42.000000000 +0200
@@ -23,15 +23,20 @@
) && (
github.actor == 'jaqx0r'
)
+ permissions:
+ # wait on check
+ checks: read
+ # create review
+ pull-requests: write
steps:
- - uses: lewagon/wait-on-check-action@v0.2
+ - uses: lewagon/wait-on-check-action@v1.1.1
with:
ref: ${{ github.event.pull_request.head.sha }}
check-name: test
repo-token: ${{ github.token }}
wait-interval: 60
- - uses: "actions/github-script@v4.0.2"
+ - uses: "actions/github-script@v4.1"
with:
github-token: ${{ github.token }}
script: |
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/automerge.yml
new/mtail-3.0.0rc48/.github/workflows/automerge.yml
--- old/mtail-3.0.0rc47/.github/workflows/automerge.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/automerge.yml 2021-09-18
00:24:42.000000000 +0200
@@ -10,6 +10,19 @@
approve-and-merge-dependabot:
if: github.event.pull_request.user.login == 'dependabot[bot]' &&
contains(github.event.pull_request.labels.*.name, 'dependencies')
runs-on: ubuntu-latest
+ # TODO(jaq): The enable automerge permission is incorrect, pending solution
+ # in
https://github.community/t/what-permission-does-a-github-action-need-to-call-graphql-enablepullrequestautomerge/197708
+ # permissions:
+ # # wait-on-check requires check read
+ # checks: read
+ # # enable-automerge is a graphql query, not REST, so isn't documented,
+ # # except in a mention in
+ # #
https://github.blog/changelog/2021-02-04-pull-request-auto-merge-is-now-generally-available/
+ # # which says "can only be enabled by users with permissino to merge";
the
+ # # REST documentation says you need contents: write to perform a merge.
+ # contents: write
+ # # auto-approve-action requires write on actions
+ # actions: write
steps:
# Enable auto-merge *before* issuing an approval.
@@ -18,11 +31,11 @@
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
- - uses: lewagon/wait-on-check-action@v0.2
+ - uses: lewagon/wait-on-check-action@v1.1.1
with:
ref: ${{ github.event.pull_request.head.sha }}
check-name: test
- repo-token: ${{ github.token }}
+ repo-token: ${{ secrets.GITHUB_TOKEN }}
wait-interval: 60
# We "trust" dependabot updates (this still requires all other checks to
pass!)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/benchmark-done.yml
new/mtail-3.0.0rc48/.github/workflows/benchmark-done.yml
--- old/mtail-3.0.0rc47/.github/workflows/benchmark-done.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/benchmark-done.yml 2021-09-18
00:24:42.000000000 +0200
@@ -7,8 +7,13 @@
jobs:
comment:
runs-on: ubuntu-latest
+ permissions:
+ # list and download artifact
+ actions: read
+ # post benchmark results
+ pull-requests: write
steps:
- - uses: actions/github-script@v4.0.2
+ - uses: actions/github-script@v4.1
with:
script: |
var artifacts = await github.actions.listWorkflowRunArtifacts({
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/benchmark.yml
new/mtail-3.0.0rc48/.github/workflows/benchmark.yml
--- old/mtail-3.0.0rc47/.github/workflows/benchmark.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/benchmark.yml 2021-09-18
00:24:42.000000000 +0200
@@ -9,6 +9,14 @@
env:
GO111MODULE: "on"
GOPROXY: https://proxy.golang.org
+
+permissions:
+ # none-all, which doesn't exist, but
+ #
https://docs.github.com/en/actions/reference/authentication-in-a-workflow#using-the-github_token-in-a-workflow
+ # implies that the token still gets created. Elsewhere we learn that any
+ # permission not mentioned here gets turned to `none`.
+ actions: none
+
jobs:
benchmark:
runs-on: ubuntu-latest
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/ci-done.yml
new/mtail-3.0.0rc48/.github/workflows/ci-done.yml
--- old/mtail-3.0.0rc47/.github/workflows/ci-done.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/ci-done.yml 2021-09-18
00:24:42.000000000 +0200
@@ -7,8 +7,15 @@
jobs:
comment:
runs-on: ubuntu-latest
+ permissions:
+ # list and download
+ actions: read
+ # post results as comment
+ pull-requests: write
+ # publish creates a check run
+ checks: write
steps:
- - uses: actions/github-script@v4.0.2
+ - uses: actions/github-script@v4.1
with:
script: |
var artifacts = await github.actions.listWorkflowRunArtifacts({
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/ci.yml
new/mtail-3.0.0rc48/.github/workflows/ci.yml
--- old/mtail-3.0.0rc47/.github/workflows/ci.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/ci.yml 2021-09-18
00:24:42.000000000 +0200
@@ -4,11 +4,18 @@
tags:
- v*
branches:
- - master
- main
pull_request:
env:
GOPROXY: "https://proxy.golang.org";
+
+permissions:
+ # none-all, which doesn't exist, but
+ #
https://docs.github.com/en/actions/reference/authentication-in-a-workflow#using-the-github_token-in-a-workflow
+ # implies that the token still gets created. Elsewhere we learn that any
+ # permission not mentioned here gets turned to `none`.
+ actions: none
+
jobs:
test:
runs-on: ubuntu-latest
@@ -40,7 +47,7 @@
# secret merge commit of the PR itself.
https://help.github.com/en/actions/automating-your-workflow-with-github-actions/events-that-trigger-workflows#pull-request-event-pull_request
echo ${{ github.event.pull_request.head.sha }} >
test-results/sha-number
make --debug junit-regtest TESTCOVERPROFILE=coverprofile
- - uses: codecov/codecov-action@v1
+ - uses: codecov/codecov-action@v2.1.0
if: always()
with:
file: coverprofile
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/mtail-3.0.0rc47/.github/workflows/codeql-analysis.yml
new/mtail-3.0.0rc48/.github/workflows/codeql-analysis.yml
--- old/mtail-3.0.0rc47/.github/workflows/codeql-analysis.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/codeql-analysis.yml 2021-09-18
00:24:42.000000000 +0200
@@ -20,6 +20,10 @@
schedule:
- cron: '34 6 * * 3'
+permissions:
+ # https://github.com/github/codeql-action/issues/464
+ security-events: write
+
jobs:
analyze:
name: Analyze
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/golangci-lint.yml
new/mtail-3.0.0rc48/.github/workflows/golangci-lint.yml
--- old/mtail-3.0.0rc47/.github/workflows/golangci-lint.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/golangci-lint.yml 2021-09-18
00:24:42.000000000 +0200
@@ -4,9 +4,15 @@
tags:
- v*
branches:
- - master
- main
pull_request:
+
+permissions:
+ # golangci-lint does annotations, not comments
+ # No-one knows what an annotation is, but I suspect it's printing file:line:
msg to stdout.
+ # https://github.community/t/what-are-annotations/16173/2
+ checks: none
+
jobs:
golangci:
name: lint
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/oss-fuzz.yml
new/mtail-3.0.0rc48/.github/workflows/oss-fuzz.yml
--- old/mtail-3.0.0rc47/.github/workflows/oss-fuzz.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/oss-fuzz.yml 2021-09-18
00:24:42.000000000 +0200
@@ -6,6 +6,11 @@
- 'internal/runtime/compiler/parser/parser.y'
- 'Makefile'
- 'Dockerfile'
+
+permissions:
+ # Secret code for "the github token should have no tokens at all"
+ actions: none
+
jobs:
oss-fuzz:
runs-on: ubuntu-latest
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/release.yml
new/mtail-3.0.0rc48/.github/workflows/release.yml
--- old/mtail-3.0.0rc47/.github/workflows/release.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/release.yml 2021-09-18
00:24:42.000000000 +0200
@@ -6,6 +6,10 @@
tags:
- v*
+permissions:
+ # goreleaser writes to the releases api
+ contents: write
+
jobs:
goreleaser:
runs-on: ubuntu-latest
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/security-scan.yml
new/mtail-3.0.0rc48/.github/workflows/security-scan.yml
--- old/mtail-3.0.0rc47/.github/workflows/security-scan.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/security-scan.yml 2021-09-18
00:24:42.000000000 +0200
@@ -1,8 +1,7 @@
name: "Security Scan"
-# Run workflow each time code is pushed to your repository and on a schedule.
on:
- push:
+ pull_request:
schedule:
- cron: 25 1 * * 1
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/.github/workflows/stale.yml
new/mtail-3.0.0rc48/.github/workflows/stale.yml
--- old/mtail-3.0.0rc47/.github/workflows/stale.yml 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/.github/workflows/stale.yml 2021-09-18
00:24:42.000000000 +0200
@@ -3,6 +3,10 @@
schedule:
- cron: "30 1 * * *"
+permissions:
+ pull-requests: write
+ issues: write
+
jobs:
stale:
runs-on: ubuntu-latest
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/Dockerfile
new/mtail-3.0.0rc48/Dockerfile
--- old/mtail-3.0.0rc47/Dockerfile 2021-07-17 00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/Dockerfile 2021-09-18 00:24:42.000000000 +0200
@@ -21,12 +21,12 @@
LABEL org.opencontainers.image.ref.name="google/mtail" \
org.opencontainers.image.vendor="Google" \
org.opencontainers.image.title="mtail" \
- org.opencontainers.image.description="extract whitebox monitoring data
from application logs for collection in a timeseries database" \
+ org.opencontainers.image.description="extract internal monitoring data
from application logs for collection in a timeseries database" \
org.opencontainers.image.authors="Jamie Wilkinson (@jaqx0r)" \
org.opencontainers.image.licenses="Apache-2.0" \
org.opencontainers.image.version=$version \
org.opencontainers.image.revision=$commit_hash \
org.opencontainers.image.source=$vcs_url \
-
org.opencontainers.image.documentation="https://github.com/google/mtail/tree/master/docs";
\
+
org.opencontainers.image.documentation="https://github.com/google/mtail/tree/main/docs";
\
org.opencontainers.image.created=$build_date \
org.opencontainers.image.url="https://github.com/google/mtail";
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/ISSUE_TEMPLATE.md
new/mtail-3.0.0rc48/ISSUE_TEMPLATE.md
--- old/mtail-3.0.0rc47/ISSUE_TEMPLATE.md 2021-07-17 00:49:04.000000000
+0200
+++ new/mtail-3.0.0rc48/ISSUE_TEMPLATE.md 2021-09-18 00:24:42.000000000
+0200
@@ -6,7 +6,7 @@
If your problem is with `mtail`, please include the commandline you started it
with, and the INFO log.
-See also [Reporting a
problem](https://github.com/google/mtail/blob/master/docs/Troubleshooting.md#reporting-a-problem).
+See also [Reporting a
problem](https://github.com/google/mtail/blob/main/docs/Troubleshooting.md#reporting-a-problem).
Thanks!
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/README.md
new/mtail-3.0.0rc48/README.md
--- old/mtail-3.0.0rc47/README.md 2021-07-17 00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/README.md 2021-09-18 00:24:42.000000000 +0200
@@ -1,6 +1,6 @@
<img src="https://raw.githubusercontent.com/google/mtail/main/logo.png";
alt="mtail" title="mtail" align="right" width="140">
-# mtail - extract whitebox monitoring data from application logs for
collection into a timeseries database
+# mtail - extract internal monitoring data from application logs for
collection into a timeseries database
[](https://github.com/google/mtail/actions?query=workflow%3ACI+branch%3main)
[](http://godoc.org/github.com/google/mtail)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/cmd/mtail/main.go
new/mtail-3.0.0rc48/cmd/mtail/main.go
--- old/mtail-3.0.0rc47/cmd/mtail/main.go 2021-07-17 00:49:04.000000000
+0200
+++ new/mtail-3.0.0rc48/cmd/mtail/main.go 2021-09-18 00:24:42.000000000
+0200
@@ -15,6 +15,7 @@
"time"
"github.com/golang/glog"
+ "github.com/google/mtail/internal/exporter"
"github.com/google/mtail/internal/metrics"
"github.com/google/mtail/internal/mtail"
"github.com/google/mtail/internal/waker"
@@ -46,11 +47,12 @@
version = flag.Bool("version", false, "Print mtail version
information.")
// Compiler behaviour flags.
- oneShot = flag.Bool("one_shot", false, "Compile the programs, then
read the contents of the provided logs from start until EOF, print the values
of the metrics store and exit. This is a debugging flag only, not for
production use.")
- compileOnly = flag.Bool("compile_only", false, "Compile programs only,
do not load the virtual machine.")
- dumpAst = flag.Bool("dump_ast", false, "Dump AST of programs after
parse (to INFO log).")
- dumpAstTypes = flag.Bool("dump_ast_types", false, "Dump AST of programs
with type annotation after typecheck (to INFO log).")
- dumpBytecode = flag.Bool("dump_bytecode", false, "Dump bytecode of
programs (to INFO log).")
+ oneShot = flag.Bool("one_shot", false, "Compile the programs,
then read the contents of the provided logs from start until EOF, print the
values of the metrics store in the given format and exit. This is a debugging
flag only, not for production use.")
+ oneShotFormat = flag.String("one_shot_format", "json", "Format to use
with -one_shot. This is a debugging flag only, not for production use.
Supported formats: json, prometheus.")
+ compileOnly = flag.Bool("compile_only", false, "Compile programs
only, do not load the virtual machine.")
+ dumpAst = flag.Bool("dump_ast", false, "Dump AST of programs
after parse (to INFO log).")
+ dumpAstTypes = flag.Bool("dump_ast_types", false, "Dump AST of
programs with type annotation after typecheck (to INFO log).")
+ dumpBytecode = flag.Bool("dump_bytecode", false, "Dump bytecode of
programs (to INFO log).")
// VM Runtime behaviour flags.
syslogUseCurrentYear = flag.Bool("syslog_use_current_year", true,
"Patch yearless timestamps with the present year.")
@@ -167,6 +169,7 @@
mtail.MaxRegexpLength(*maxRegexpLength),
mtail.MaxRecursionDepth(*maxRecursionDepth),
}
+ eOpts := []exporter.Option{}
if *logRuntimeErrors {
opts = append(opts, mtail.LogRuntimeErrors)
}
@@ -203,9 +206,11 @@
}
if !*emitProgLabel {
opts = append(opts, mtail.OmitProgLabel)
+ eOpts = append(eOpts, exporter.OmitProgLabel())
}
if *emitMetricTimestamp {
opts = append(opts, mtail.EmitMetricTimestamp)
+ eOpts = append(eOpts, exporter.EmitTimestamp())
}
if *jaegerEndpoint != "" {
opts = append(opts, mtail.JaegerReporter(*jaegerEndpoint))
@@ -227,11 +232,34 @@
os.Exit(1) //nolint:gocritic // false positive
}
if *oneShot {
- err = store.WriteMetrics(os.Stdout)
- if err != nil {
- glog.Error(err)
+ switch *oneShotFormat {
+ case "prometheus":
+ e, err := exporter.New(ctx, nil, store, eOpts...)
+ if err != nil {
+ glog.Error(err)
+ cancel()
+ os.Exit(1) //nolint:gocritic // false positive
+ }
+ err = e.Write(os.Stdout)
+ if err != nil {
+ glog.Error(err)
+ cancel()
+ os.Exit(1) //nolint:gocritic // false positive
+ }
+ cancel()
+ os.Exit(0) //nolint:gocritic // false positive
+ case "json":
+ err = store.WriteMetrics(os.Stdout)
+ if err != nil {
+ glog.Error(err)
+ os.Exit(1) //nolint:gocritic // false positive
+ }
+ cancel()
+ os.Exit(0) //nolint:gocritic // false positive
+ default:
+ glog.Errorf("unsupported format: %q", *oneShotFormat)
+ cancel()
+ os.Exit(1) //nolint:gocritic // false positive
}
- cancel()
- os.Exit(1) //nolint:gocritic // false positive
}
}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/docs/Deploying.md
new/mtail-3.0.0rc48/docs/Deploying.md
--- old/mtail-3.0.0rc47/docs/Deploying.md 2021-07-17 00:49:04.000000000
+0200
+++ new/mtail-3.0.0rc48/docs/Deploying.md 2021-09-18 00:24:42.000000000
+0200
@@ -116,7 +116,7 @@
Alternatively, if you're using `scp` or some similiar method to copy the
programme files without a receiver, then either follow it with a `ssh $host
'killall -HUP mtail'` or use a tool like
[`inotifywait`](https://linux.die.net/man/1/inotifywait) in a side process next
to mtail to watch for changes and send the reload signal.
```shell
-inotifywait /etc/mtail/progs | while read event; do killall -HUP mtail; done
+inotifywait -m /etc/mtail/progs | while read event; do killall -HUP mtail; done
```
## Getting the Metrics Out
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/docs/Interoperability.md
new/mtail-3.0.0rc48/docs/Interoperability.md
--- old/mtail-3.0.0rc47/docs/Interoperability.md 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/docs/Interoperability.md 2021-09-18
00:24:42.000000000 +0200
@@ -27,7 +27,7 @@
## Prometheus Exporter Metrics
-https://prometheus.io/docs/instrumenting/writing_exporters/ describes useful
metrics for a Prometheus exporter to export. `mtail` does not follow that
guide, for these reasons.
+Prometheus' [writing exporters
documentation](https://prometheus.io/docs/instrumenting/writing_exporters/)
describes useful metrics for a Prometheus exporter to export. `mtail` does not
follow that guide, for these reasons.
The exporter model described in that document is for active proxies between an
application and Prometheus. The expectation is that when Prometheus scrapes
the proxy (the exporter) that it then performs its own scrape of the target
application, and translates the results back into the Prometheus exposition
format. The time taken to query the target application is what is exported as
`X_scrape_duration_seconds` and its availability as `X_up`.
@@ -35,7 +35,7 @@
TODO(jaq): Instead, mtail will export a histogram of the runtime per line of
each VM program.
-`mtail` doesn't export `mtail_up` or `mtail_scrape_duration_seconds` because
they are exactly equivalent* the synthetic metrics that Prometheus creates
automatically: https://prometheus.io/docs/concepts/jobs_instances/
+`mtail` doesn't export `mtail_up` or `mtail scrape_duration_seconds` because
they are exactly equivalent* to the [synthetic
metrics](https://prometheus.io/docs/concepts/jobs_instances/) that Prometheus
creates automatically.
\* The difference between a scrape duration measured in mtail versus
Prometheus would differ in the network round trip time, TCP setup time, and
send/receive queue time. For practical purposes you can ignore them as the
usefulness of a scrape duration metric is not in its absolute value, but how it
changes over time.
@@ -52,7 +52,7 @@
* syslog-ng supports output to [named
pipe](https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.30/administration-guide/44#TOPIC-1595018)
and [unix domain
socket](https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.30/administration-guide/54#TOPIC-1595060)
-Additionally, use a proper syslog to transmit and receive logs over the
network. `mtail` does not provide any transprot security, nor does TCP itself
guarantee that no loss of data will occur: the [RELP
spec](https://www.rsyslog.com/doc/v8-stable/tutorials/reliable_forwarding.html)
exists for the latter.
+Additionally, use a proper syslog to transmit and receive logs over the
network. `mtail` does not provide any transport security, nor does TCP itself
guarantee that no loss of data will occur: the [RELP
spec](https://www.rsyslog.com/doc/v8-stable/tutorials/reliable_forwarding.html)
exists for the latter.
*Recommendation*
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/docs/Testing.md
new/mtail-3.0.0rc48/docs/Testing.md
--- old/mtail-3.0.0rc47/docs/Testing.md 2021-07-17 00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/docs/Testing.md 2021-09-18 00:24:42.000000000 +0200
@@ -40,7 +40,7 @@
If you wish, send a PR containing your program, some sample input, and a golden
output to be run as a test in
-http://github.com/google/mtail/blob/master/ex_test.go to ensure that mtail
+http://github.com/google/mtail/blob/main/ex_test.go to ensure that mtail
never breaks your program (or that your program gets any updates if the
language changes.)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/docs/index.md
new/mtail-3.0.0rc48/docs/index.md
--- old/mtail-3.0.0rc47/docs/index.md 2021-07-17 00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/docs/index.md 2021-09-18 00:24:42.000000000 +0200
@@ -1,4 +1,4 @@
-mtail - extract whitebox monitoring data from application logs for collection
into a timeseries database
+mtail - extract internal monitoring data from application logs for collection
into a timeseries database
========================================================================================================
`mtail` is a tool for extracting metrics from application logs to be exported
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/examples/apache_metrics.mtail
new/mtail-3.0.0rc48/examples/apache_metrics.mtail
--- old/mtail-3.0.0rc47/examples/apache_metrics.mtail 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/examples/apache_metrics.mtail 2021-09-18
00:24:42.000000000 +0200
@@ -26,7 +26,7 @@
###
# HTTP Requests with histogram buckets.
#
-
http_request_duration_seconds[$server_port][$handler][$method][$code][$protocol]
= $time_us / 1000000
+
http_request_duration_seconds[$server_port][$handler][$method][$code][$protocol]
= $time_us / 1000000.0
###
# Sent/Received bytes.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/go.mod new/mtail-3.0.0rc48/go.mod
--- old/mtail-3.0.0rc47/go.mod 2021-07-17 00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/go.mod 2021-09-18 00:24:42.000000000 +0200
@@ -9,7 +9,7 @@
github.com/google/go-cmp v0.5.6
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.11.0
- github.com/prometheus/common v0.29.0
+ github.com/prometheus/common v0.30.0
go.opencensus.io v0.23.0
golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40
)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/go.sum new/mtail-3.0.0rc48/go.sum
--- old/mtail-3.0.0rc47/go.sum 2021-07-17 00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/go.sum 2021-09-18 00:24:42.000000000 +0200
@@ -169,8 +169,8 @@
github.com/prometheus/common v0.4.1/go.mod
h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.10.0/go.mod
h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
github.com/prometheus/common v0.26.0/go.mod
h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc=
-github.com/prometheus/common v0.29.0
h1:3jqPBvKT4OHAbje2Ql7KeaaSicDBCxMYwEJU1zRJceE=
-github.com/prometheus/common v0.29.0/go.mod
h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls=
+github.com/prometheus/common v0.30.0
h1:JEkYlQnpzrzQFxi6gnukFPdQ+ac82oRhzMcIduJu/Ug=
+github.com/prometheus/common v0.30.0/go.mod
h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod
h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod
h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.1.3/go.mod
h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/internal/exporter/prometheus.go
new/mtail-3.0.0rc48/internal/exporter/prometheus.go
--- old/mtail-3.0.0rc47/internal/exporter/prometheus.go 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/internal/exporter/prometheus.go 2021-09-18
00:24:42.000000000 +0200
@@ -6,12 +6,14 @@
import (
"expvar"
"fmt"
+ "io"
"strings"
"github.com/golang/glog"
"github.com/google/mtail/internal/metrics"
"github.com/google/mtail/internal/metrics/datum"
"github.com/prometheus/client_golang/prometheus"
+ "github.com/prometheus/common/expfmt"
)
var metricExportTotal = expvar.NewInt("metric_export_total")
@@ -96,6 +98,27 @@
})
}
+// Write is used to write Prometheus metrics to an io.Writer.
+func (e *Exporter) Write(w io.Writer) error {
+ reg := prometheus.NewRegistry()
+ err := reg.Register(e)
+ if err != nil {
+ return err
+ }
+ mfs, err := reg.Gather()
+ if err != nil {
+ return err
+ }
+ enc := expfmt.NewEncoder(w, expfmt.FmtText)
+ for _, mf := range mfs {
+ err := enc.Encode(mf)
+ if err != nil {
+ return err
+ }
+ }
+ return nil
+}
+
func promTypeForKind(k metrics.Kind) prometheus.ValueType {
switch k {
case metrics.Counter:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/mtail-3.0.0rc47/internal/exporter/prometheus_test.go
new/mtail-3.0.0rc48/internal/exporter/prometheus_test.go
--- old/mtail-3.0.0rc47/internal/exporter/prometheus_test.go 2021-07-17
00:49:04.000000000 +0200
+++ new/mtail-3.0.0rc48/internal/exporter/prometheus_test.go 2021-09-18
00:24:42.000000000 +0200
@@ -4,6 +4,7 @@
package exporter
import (
+ "bytes"
"context"
"math"
"strings"
@@ -275,5 +276,84 @@
cancel()
wg.Wait()
})
+ }
+}
+
+var writePrometheusTests = []struct {
+ name string
+ metrics []*metrics.Metric
+ expected string
+}{
+ {
+ "empty",
+ []*metrics.Metric{},
+ "",
+ },
+ {
+ "single",
+ []*metrics.Metric{
+ {
+ Name: "foo",
+ Program: "test",
+ Kind: metrics.Counter,
+ LabelValues: []*metrics.LabelValue{{Labels:
[]string{}, Value: datum.MakeInt(1, time.Unix(0, 0))}},
+ },
+ },
+ `# HELP foo defined at
+# TYPE foo counter
+foo 1
+`,
+ },
+ {
+ "multi",
+ []*metrics.Metric{
+ {
+ Name: "foo",
+ Program: "test",
+ Kind: metrics.Counter,
+ LabelValues: []*metrics.LabelValue{{Labels:
[]string{}, Value: datum.MakeInt(1, time.Unix(0, 0))}},
+ },
+ {
+ Name: "bar",
+ Program: "test",
+ Kind: metrics.Counter,
+ LabelValues: []*metrics.LabelValue{{Labels:
[]string{}, Value: datum.MakeInt(2, time.Unix(0, 0))}},
+ },
+ },
+ `# HELP bar defined at
+# TYPE bar counter
+bar 2
+# HELP foo defined at
+# TYPE foo counter
+foo 1
+`,
+ },
+}
+
+func TestWritePrometheus(t *testing.T) {
+ for _, tc := range writePrometheusTests {
+ tc := tc
+ t.Run(tc.name, func(t *testing.T) {
+ var wg sync.WaitGroup
+ ctx, cancel := context.WithCancel(context.Background())
+ ms := metrics.NewStore()
+ for _, metric := range tc.metrics {
+ testutil.FatalIfErr(t, ms.Add(metric))
+ }
+ opts := []Option{
+ Hostname("gunstar"),
+ OmitProgLabel(),
+ }
+ e, err := New(ctx, &wg, ms, opts...)
+ testutil.FatalIfErr(t, err)
+
+ var buf bytes.Buffer
+ err = e.Write(&buf)
+ testutil.FatalIfErr(t, err)
+ testutil.ExpectNoDiff(t, tc.expected, buf.String())
+
+ cancel()
+ wg.Wait()
+ })
}
}
++++++ vendor.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/vendor/github.com/prometheus/common/expfmt/encode.go
new/vendor/github.com/prometheus/common/expfmt/encode.go
--- old/vendor/github.com/prometheus/common/expfmt/encode.go 2021-09-06
12:21:43.696916000 +0200
+++ new/vendor/github.com/prometheus/common/expfmt/encode.go 2021-11-22
20:35:28.649705600 +0100
@@ -18,7 +18,7 @@
"io"
"net/http"
- "github.com/golang/protobuf/proto"
+ "github.com/golang/protobuf/proto" //nolint:staticcheck // Ignore
SA1019. Need to keep deprecated package for compatibility.
"github.com/matttproud/golang_protobuf_extensions/pbutil"
"github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg"
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/vendor/github.com/prometheus/common/expfmt/text_parse.go
new/vendor/github.com/prometheus/common/expfmt/text_parse.go
--- old/vendor/github.com/prometheus/common/expfmt/text_parse.go
2021-09-06 12:21:43.700916000 +0200
+++ new/vendor/github.com/prometheus/common/expfmt/text_parse.go
2021-11-22 20:35:28.649705600 +0100
@@ -24,7 +24,7 @@
dto "github.com/prometheus/client_model/go"
- "github.com/golang/protobuf/proto"
+ "github.com/golang/protobuf/proto" //nolint:staticcheck // Ignore
SA1019. Need to keep deprecated package for compatibility.
"github.com/prometheus/common/model"
)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/vendor/modules.txt new/vendor/modules.txt
--- old/vendor/modules.txt 2021-09-06 12:21:43.772915600 +0200
+++ new/vendor/modules.txt 2021-11-22 20:35:28.721705000 +0100
@@ -40,7 +40,7 @@
github.com/prometheus/client_golang/prometheus/testutil/promlint
# github.com/prometheus/client_model v0.2.0
github.com/prometheus/client_model/go
-# github.com/prometheus/common v0.29.0
+# github.com/prometheus/common v0.30.0
## explicit
github.com/prometheus/common/expfmt
github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg
![https://raw.githubusercontent.com/google/mtail/main/logo.png"](https://raw.githubusercontent.com/google/mtail/main/logo.png"){kind=link}
