Re: [PATCH v6] checkpatch: add support to check 'Fixes:' tag format
On Mon, May 04, 2020 at 11:34:31AM -0700, Joe Perches wrote:
> On Mon, 2020-05-04 at 16:20 +0800, Wang YanQing wrote:
> > According to submitting-patches.rst, 'Fixes:' tag has a little
> > stricter condition about the one line summary than normal git
> > commit description:
> > “...
> > Do not split the tag across multiple lines, tags are exempt from
> > the "wrap at 75 columns" rule in order to simplify parsing scripts
> > ...”
> >
> > And there is no sanity check for 'Fixes:' tag format in checkpatch
> > the same as GIT_COMMIT_ID for git commit description, so let's expand
> > the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
> >
> > The check supports below formats:
> > Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number
> > of pages it actually freed")
> > Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
> > devices"")
> > Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
> > Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
> > Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
> >
> > The check doesn't support below formats and it will emit diagnostics info
> > for them:
> > Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
> > Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
> > Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power
> > amplifiers)
> > Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU
> > devices nodes)
> > Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
> > work-queue”)
> > Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT
> > page fault handler"
> > Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
> > completion")
> > Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
> > Fixes: 03404e8ae652("IB/mlx5: Add support to dropless RQ")
> >
> > Because after GIT_COMMIT_ID supports 'Fixes:' tag format check, it could do
> > the same check as the UNKNOWN_COMMIT_ID, so we don't need UNKNOWN_COMMIT_ID
> > anymore and I decide to delete it.
> >
> > Note: this patch also fixes double quotation mark issue for normal git
> > commit description, and now it supports double quotation mark in
> > title line, for example:
> > Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
> > when checking usable channels"")
> >
> > Note: this patch also adds diagnostics info support for normal git commit
> > description format check.
>
> Thanks YanQing.
>
> All of this seems fine, but perhaps checkpatch's output message content
> could be a bit shorter. I'll give it a think for a while.
>
Hi! Joe
What are the status of these patches?
"[PATCH v2] checkpatch: fix can't check for too long invalid commit id"
"[PATCH v6] checkpatch: add support to check 'Fixes:' tag format"
"[PATCH v2] checkpatch: allow commit description spans across three lines"
This is a ping message, maybe you have forgot them?
Thanks.
Re: [PATCH v6] checkpatch: add support to check 'Fixes:' tag format
On Mon, May 04, 2020 at 11:45:02AM +0200, Markus Elfring wrote: > > > 4: Add support to check and emit sensible diagnostics > > info for description which spans across more than > > two lines. > > My software concerns are growing also for the preferred dependency management. > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?id=0e698dfa282211e414076f9dc7e83c1c288314fd#n138 > Hi! Markus Elfring This feature is a side effect of missing right '")' detection, so I think it is acceptable to add it. But I agree with you, this patch is a little big, I don't want to add more code into it. Joe, what is your opinion? Do you think this version is good enough to merge? Thanks!
[PATCH v6] checkpatch: add support to check 'Fixes:' tag format
According to submitting-patches.rst, 'Fixes:' tag has a little
stricter condition about the one line summary than normal git
commit description:
“...
Do not split the tag across multiple lines, tags are exempt from
the "wrap at 75 columns" rule in order to simplify parsing scripts
...”
And there is no sanity check for 'Fixes:' tag format in checkpatch
the same as GIT_COMMIT_ID for git commit description, so let's expand
the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
The check supports below formats:
Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of
pages it actually freed")
Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
devices"")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
The check doesn't support below formats and it will emit diagnostics info for
them:
Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power amplifiers)
Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU devices
nodes)
Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
work-queue”)
Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page
fault handler"
Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
completion")
Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
Fixes: 03404e8ae652("IB/mlx5: Add support to dropless RQ")
Because after GIT_COMMIT_ID supports 'Fixes:' tag format check, it could do
the same check as the UNKNOWN_COMMIT_ID, so we don't need UNKNOWN_COMMIT_ID
anymore and I decide to delete it.
Note: this patch also fixes double quotation mark issue for normal git
commit description, and now it supports double quotation mark in
title line, for example:
Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
when checking usable channels"")
Note: this patch also adds diagnostics info support for normal git commit
description format check.
Based on original patch by Joe Perches
Link:
https://lore.kernel.org/lkml/40bfc40958fca6e2cc9b86101153aa0715fac4f7.ca...@perches.com/
Signed-off-by: Wang YanQing
---
v6:
1: Fix indentation issue.
2: Fix a typo issue, reported by Markus Elfring.
3: Reword some diagnostics info.
4: Add support to check and emit sensible diagnostics
info for description which spans across more than
two lines.
5: Avoid the word "acrosses", suggested by Markus Elfring.
v5:
1: Rebased on '[PATCH v2] checkpatch: fix can't check for too long invalid
commit id'.
2: Fix wrong diagnostics info for below non-standard 'Fixes:' formats.
a: Fixes: 03404e8ae652("IB/mlx5: Add support to dropless RQ")
b: Fixes: da97e18458fb: ("perf_event: Add support for LSM and SELinux
checks")
3: Reword the diagnostics info for non-standard title format.
v4:
1: Add diagnostics info support, suggested by Joe Perches and Markus Elfring.
2: Delete UNKNOWN_COMMIT_ID and do the check in GIT_COMMIT_ID.
v3:
1: Fix a bug in short title line support.
v2:
1: Add support for double quotation mark in title line, suggested by Markus
Elfring.
2: Add support for short title line with/without ellipsis.
3: Add supported format examples and unsupported format examples in changelog.
4: Fix a little wording issue in changelog , suggested by Markus Elfring.
scripts/checkpatch.pl | 129 ++
1 file changed, 99 insertions(+), 30 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 9b47584..ef34716 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2818,51 +2818,109 @@ sub process {
$line !~
/^\s*(?:Link|Patchwork|http|https|BugLink|base-commit):/i &&
$line !~ /^This reverts commit [0-9a-f]{7,40}/ &&
($line =~ /\bcommit\s+[0-9a-f]{5,}\b/i ||
+$line =~ /\bfixes:\s+[0-9a-f]{5,}\b/i ||
($line =~ /(?:\s|^)[0-9a-f]{12,40}(?:[\s"'\(\[]|$)/i &&
- $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i &&
- $line !~ /\bfixes:\s*[0-9a-f]{12,40}/i))) {
+ $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i))) {
my $init_char = "c";
my $orig_commit = "";
+ my $prefix = "commit";
+ my $prefix_case = "[Cc]ommit";
my $short = 1;
[PATCH v2] checkpatch: allow commit description spans across three lines
The current GIT_COMMIT_ID will report error when the commit description
spans across three lines, for examples:
"...
To rehash a previous explanation given in commit 1c44ce560b4d ("net:
mscc: ocelot: fix vlan_filtering when enslaving to bridge before link is
up"), the switch driver operates the in a mode where a single VLAN can
be transmitted as untagged on a particular egress port. That is the
"native VLAN on trunk port" use case.
..."
The above changelog comes from commit 87b0f983f66f ("net: mscc: ocelot:
fix untagged packet drops when enslaving to vlan aware bridge").
"...
With the optimizations for TLB invalidation from commit 0cef77c7798a
("powerpc/64s/radix: flush remote CPUs out of single-threaded
mm_cpumask"), the scope of a TLBI (global vs. local) can now be
..."
The above changelog comes from commit cca19f0b684f ("powerpc/64s/radix: Fix
missing global invalidations when removing copro").
The total length of commit description ("commit"+"12+ SHA1"+("title line"))
exceeds 85 isn't uncommon thing, and it isn't uncommon thing that the ~85
characters span across three lines, see above examples.
This patch adds support to recognize commit description which spans across
three lines, then it will not emit error message for such situation.
Signed-off-by: Wang YanQing
---
Hi! Joe
I have tested with below command:
git log -1 --format=%H -i --grep=" commit " | \
while read commit ; do \
echo $commit; \
./scripts/checkpatch.pl --git $commit --types=GIT_COMMIT_ID --quiet
--nosummary --color=never; \
done
There are ~50 properly formed commit descriptions belong to this class, and I
haven't check for
the non-standard commit descriptions, for examples:
3403e56b41c176f6531a2a6d77d85b46fa34169c
a78945c357f58665d6a5da8a69e085898e831c70
87b0f983f66f23762921129fd35966eddc3f2dae
ac8517440344dbe598f7c1c23e686c800b563061
cca19f0b684f4ed6aabf6ad07ae3e15e77bfd78a
53406ed1bcfdabe4b5bc35e6d17946c6f9f563e2
This number isn't big, but they are all in properly formed format, so I think
we should support them
and avoid emitting false positive error report.
v2:
1: Reword the title line.
2: Reword the changelog.
3: Rewrite the implementation.
scripts/checkpatch.pl | 29 -
1 file changed, 28 insertions(+), 1 deletion(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index ef34716..8cfc3a9 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2900,6 +2900,16 @@ sub process {
$rawlines[$linenr] =~ /^\s*\("(.+)"\)/) {
$orig_desc = $1;
$has_parens_and_dqm = 1;
+ } elsif ($line =~ /\bcommit\s+[0-9a-f]{5,}\s*$/i &&
+defined $rawlines[$linenr] &&
+$rawlines[$linenr] =~ /^\s*\(".+$/ &&
+defined $rawlines[$linenr + 1] &&
+$rawlines[$linenr + 1] =~ /^\s*.+"\)/) {
+ $rawlines[$linenr] =~ /^\s*\("(.+)$/i;
+ $orig_desc = $1;
+ $rawlines[$linenr + 1] =~ /^\s*(.+)"\)/;
+ $orig_desc .= " " . $1;
+ $has_parens_and_dqm = 1;
} elsif ($line =~ /\b$prefix\s+[0-9a-f]{5,}\s+\(".+$/i
&&
defined $rawlines[$linenr] &&
$rawlines[$linenr] =~ /^\s*.+"\)/) {
@@ -2913,12 +2923,29 @@ sub process {
$acrosslines = 1;
$diagnostics .= "The $name spans across
lines.\n";
}
+ } elsif ($line =~ /\b$prefix\s+[0-9a-f]{5,}\s+\(".+$/i
&&
+defined $rawlines[$linenr] &&
+$rawlines[$linenr] !~ /^\s*.+"\)/ &&
+defined $rawlines[$linenr + 1] &&
+$rawlines[$linenr + 1] =~ /^\s*.+"\)/) {
+ $line =~ /\b$prefix\s+[0-9a-f]{5,}\s+\("(.+)$/i;
+ $orig_desc = $1;
+ $rawlines[$linenr] =~ /^\s*(.+)/;
+ $orig_desc .= " " . $1;
+ $rawlines[$linenr + 1] =~ /^\s*(.+)"\)/;
+ $orig_desc .= " " . $1;
+ $has_parens_and_dqm = 1;
+
+ if ($prefix eq "Fixes:") {
+ $acrosslines = 1;
+
Re: [PATCH v5] checkpatch: add support to check 'Fixes:' tag format
On Sun, May 03, 2020 at 03:51:39PM +0200, Markus Elfring wrote:
>
>
> …
> > + $diagnostics .= "The title is too
> > abbreviated, at least half of orignial commit title is necessary.\n";
>
> Will the word “original” be more appropriate here?
> (Why did you not integrate my previous patch review comment?)
Sorry, I miss it.
This version patch has indentation issue, I will fix the typo
with the indentation issue in next version (v6).
Thanks.
>
>
> …
> > + "Please use git commit description style
> > '$prefix <$sha1_length_min+ chars of sha1> (\"<$title>\")' - ie:
> > '${init_char}" . substr($prefix, 1) .
> > + " $id (\"$description\")'\n" .
> > $diagnostics . $herecurr);
>
> Can error diagnostics become multi-line?
The length of "$description" is unknown, it is difficult to
cook the error message into pretty format with any length of
"$description", so let's keep it in current way.
Thanks.
[PATCH v5] checkpatch: add support to check 'Fixes:' tag format
According to submitting-patches.rst, 'Fixes:' tag has a little
stricter condition about the one line summary than normal git
commit description:
“...
Do not split the tag across multiple lines, tags are exempt from
the "wrap at 75 columns" rule in order to simplify parsing scripts
...”
And there is no sanity check for 'Fixes:' tag format in checkpatch
the same as GIT_COMMIT_ID for git commit description, so let's expand
the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
The check supports below formats:
Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of
pages it actually freed")
Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
devices"")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
The check doesn't support below formats and it will emit diagnostics info for
them:
Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power amplifiers)
Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU devices
nodes)
Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
work-queue”)
Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page
fault handler"
Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
completion")
Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
Fixes: 03404e8ae652("IB/mlx5: Add support to dropless RQ")
Because after GIT_COMMIT_ID supports 'Fixes:' tag format check, it could do
the same check as the UNKNOWN_COMMIT_ID, so we don't need UNKNOWN_COMMIT_ID
anymore and I decide to delete it.
Note: this patch also fixes double quotation mark issue for normal git
commit description, and now it supports double quotation mark in
title line, for example:
Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
when checking usable channels"")
Note: this patch also adds diagnostics info support for normal git commit
description format check.
Based on original patch by Joe Perches
Link:
https://lore.kernel.org/lkml/40bfc40958fca6e2cc9b86101153aa0715fac4f7.ca...@perches.com/
Signed-off-by: Wang YanQing
---
Hi! Joe and all.
This is the v5 version, and I have tested it with below command:
$ git log -1 --no-merges --grep="Fixes:" --format=%H | \
while read commit ; do \
./scripts/checkpatch.pl --git $commit --types=GIT_COMMIT_ID --quiet
--nosummary; \
done
The result looks good enough already.
Thanks.
v5:
1: Rebased on '[PATCH v2] checkpatch: fix can't check for too long invalid
commit id'.
2: Fix wrong diagnostics info for below non-standard 'Fixes:' formats.
a: Fixes: 03404e8ae652("IB/mlx5: Add support to dropless RQ")
b: Fixes: da97e18458fb: ("perf_event: Add support for LSM and SELinux
checks")
3: Reword the diagnostics info for non-standard title format.
v4:
1: Add diagnostics info support, suggested by Joe Perches and Markus Elfring.
2: Delete UNKNOWN_COMMIT_ID and do the check in GIT_COMMIT_ID.
v3:
1: Fix a bug in short title line support.
v2:
1: Add support for double quotation mark in title line, suggested by Markus
Elfring.
2: Add support for short title line with/without ellipsis.
3: Add supported format examples and unsupported format examples in changelog.
4: Fix a little wording issue in changelog , suggested by Markus Elfring.
scripts/checkpatch.pl | 120 +-
1 file changed, 90 insertions(+), 30 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 9b47584..a1e4113 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2818,51 +2818,101 @@ sub process {
$line !~
/^\s*(?:Link|Patchwork|http|https|BugLink|base-commit):/i &&
$line !~ /^This reverts commit [0-9a-f]{7,40}/ &&
($line =~ /\bcommit\s+[0-9a-f]{5,}\b/i ||
+$line =~ /\bfixes:\s+[0-9a-f]{5,}\b/i ||
($line =~ /(?:\s|^)[0-9a-f]{12,40}(?:[\s"'\(\[]|$)/i &&
- $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i &&
- $line !~ /\bfixes:\s*[0-9a-f]{12,40}/i))) {
+ $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i))) {
my $init_char = "c";
my $orig_commit = "";
+ my $prefix = "commit";
+ my $prefix_case = "[Cc]ommit";
[PATCH] checkpatch: add support for title acrosses three lines in commit id description
The current GIT_COMMIT_ID will report error when the title in commit id
description acrosses three lines, take changelog of commit 87b0f983f66f
("net: mscc: ocelot: fix untagged packet drops when enslaving to vlan
aware bridge") as a example.
It make senses to support three lines title, because three lines could only
have about two lines length title, see above example, but I think we don't
need to add support for more than three lines length title, ~225 (75 * 3)
shoud be enough for any sensible title.
This patch adds support for three lines title in the commit id description,
and emit diagnostics info when the title acrosses more than three lines.
Signed-off-by: Wang YanQing
---
Hi! Joe
This is the third patch based on previous two patches.
Thanks.
scripts/checkpatch.pl | 30 +-
1 file changed, 21 insertions(+), 9 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index a1e4113..e9c46d3 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2899,15 +2899,27 @@ sub process {
$orig_desc = $1;
$has_parens_and_dqm = 1;
} elsif ($line =~ /\b$prefix\s+[0-9a-f]{5,}\s+\(".+$/i
&&
-defined $rawlines[$linenr] &&
-$rawlines[$linenr] =~ /^\s*.+"\)/) {
- $line =~ /\b$prefix\s+[0-9a-f]{5,}\s+\("(.+)$/i;
- $orig_desc = $1;
- $rawlines[$linenr] =~ /^\s*(.+)"\)/;
- $orig_desc .= " " . $1;
- $has_parens_and_dqm = 1;
-
- if ($prefix eq "Fixes:") {
+defined $rawlines[$linenr]) {
+if ($rawlines[$linenr] =~ /^\s*.+"\)/) {
+ $line =~
/\b$prefix\s+[0-9a-f]{5,}\s+\("(.+)$/i;
+ $orig_desc = $1;
+ $rawlines[$linenr] =~ /^\s*(.+)"\)/;
+ $orig_desc .= " " . $1;
+ $has_parens_and_dqm = 1;
+} elsif (defined $rawlines[$linenr + 1] &&
+ $rawlines[$linenr + 1] =~
/^\s*.+"\)/) {
+ $line =~
/\b$prefix\s+[0-9a-f]{5,}\s+\("(.+)$/i;
+ $orig_desc = $1;
+ $rawlines[$linenr] =~ /^\s*(.+)/;
+ $orig_desc .= " " . $1;
+ $rawlines[$linenr + 1] =~ /^\s*(.+)"\)/;
+ $orig_desc .= " " . $1;
+ $has_parens_and_dqm = 1;
+} else {
+ $diagnostics .= "Missing terminal '\")'
at the end of title or the title acrosses more than 3 lines.\n";
+}
+
+ if ($has_parens_and_dqm && $prefix eq "Fixes:")
{
$acrosslines = 1;
$diagnostics .= "The title acrosses
lines.\n";
}
--
1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH v4] checkpatch: add support to check 'Fixes:' tag format
On Sat, May 02, 2020 at 09:40:24PM +0200, Markus Elfring wrote:
>
>
> > + $diagnostics .= "Missing a pair of parentheses
> > '()' or a pair of double quotation marks (\"\").\n";
>
> Can such a message trigger any more thoughts and development ideas?
No, I don't think so. '(" ... ")' is the minimum interface between analyser
(checkpatch) and commit id description (normal commit id and 'Fixes:' tag)
about the title, it is very difficult if not impossible to guess the title
boundary and whether it is the *REAL* title that folllow the SHA1 without
this precondition, and it is more difficult to do it when we need to support
title which across lines in the normal commit id description.
At last I really doubt the benefit it brings deserves the complexity and the
current diagnostics info is enough clear for most situation.
Thanks.
[PATCH v2] checkpatch: fix can't check for too long invalid commit id
The current UNKNOWN_COMMIT_ID doesn't check for 41+ length commit id,
and although GIT_COMMIT_ID will check for 41+ length commit id, but
it willn't warn anything about it due to 41+ length commit will never
be defined.
This patch moves the unknown commit id check for normal commit description
to GIT_COMMIT_ID, and uses ERROR instead of WARN, because unknown commit
id is total useless to track change history in changelog, it deserves the
ERROR.
Signed-off-by: Wang YanQing
---
v2:
1: Fix annonying "Invalid commit id" reports for non commit id number string.
2: Fix indentaton issue, reported by Joe Perches.
3: Reword the error message in code, suggested by Joe Perches.
4: Delete unnecessary capture group in UNKNOWN_COMMIT_ID, suggested by Joe
Perches.
scripts/checkpatch.pl | 26 --
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 23a001a..9b47584 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2829,18 +2829,21 @@ sub process {
my $space = 1;
my $hasdesc = 0;
my $hasparens = 0;
+ my $hasprefix = 1;
my $id = '0123456789ab';
my $orig_desc = "commit description";
my $description = "";
+ my $sha1_length_min = 12;
if ($line =~ /\b(c)ommit\s+([0-9a-f]{5,})\b/i) {
$init_char = $1;
$orig_commit = lc($2);
} elsif ($line =~ /\b([0-9a-f]{12,40})\b/i) {
$orig_commit = lc($1);
+ $hasprefix = 0;
}
- $short = 0 if ($line =~ /\bcommit\s+[0-9a-f]{12,40}/i);
+ $short = 0 if ($line =~
/\bcommit\s+[0-9a-f]{$sha1_length_min,40}/i);
$long = 1 if ($line =~ /\bcommit\s+[0-9a-f]{41,}/i);
$space = 0 if ($line =~ /\bcommit [0-9a-f]/i);
$case = 0 if ($line =~
/\b[Cc]ommit\s+[0-9a-f]{5,40}[^A-F]/);
@@ -2865,10 +2868,21 @@ sub process {
($id, $description) = git_commit_info($orig_commit,
$id, $orig_desc);
+ if ($hasprefix && !defined($id)) {
+ if ($long) {
+ ERROR("GIT_COMMIT_ID",
+ "Invalid commit id '$orig_commit'
length '" . length($orig_commit) . "' exceeds allowed maxium of 40
+ ($sha1_length_min+ chars of sha1
is recommended).\n" . $herecurr);
+ } else {
+ ERROR("GIT_COMMIT_ID",
+ "Unknown commit id
'$orig_commit', maybe rebased or not pulled?\n" . $herecurr);
+ }
+ }
+
if (defined($id) &&
- ($short || $long || $space || $case || ($orig_desc
ne $description) || !$hasparens)) {
+ ($short || $space || $case || ($orig_desc ne
$description) || !$hasparens)) {
ERROR("GIT_COMMIT_ID",
- "Please use git commit description style
'commit <12+ chars of sha1> (\"\")' - ie: '${init_char}ommit $id
(\"$description\")'\n" . $herecurr);
+ "Please use git commit description style
'commit <$sha1_length_min+ chars of sha1> (\"\")' - ie:
'${init_char}ommit $id (\"$description\")'\n" . $herecurr);
}
}
@@ -2969,13 +2983,13 @@ sub process {
}
# check for invalid commit id
- if ($in_commit_log && $line =~
/(^fixes:|\bcommit)\s+([0-9a-f]{6,40})\b/i) {
+ if ($in_commit_log && $line =~ /^fixes:\s+([0-9a-f]{6,40})\b/i)
{
my $id;
my $description;
- ($id, $description) = git_commit_info($2, undef, undef);
+ ($id, $description) = git_commit_info($1, undef, undef);
if (!defined($id)) {
WARN("UNKNOWN_COMMIT_ID",
-"Unknown commit id '$2', maybe rebased or
not pulled?\n" . $herecurr);
+"Unknown commit id '$1', maybe rebased or
not pulled?\n" . $herecurr);
}
}
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] checkpatch: fix can't check for too long invalid commit id
The current UNKNOWN_COMMIT_ID doesn't check for 41+ length commit id,
and although GIT_COMMIT_ID will check for 41+ length commit id, but
it willn't warn anything about it due to 41+ length commit will never
be defined.
This patch moves the unknown commit id check for normal commit description
to GIT_COMMIT_ID, and uses ERROR instead of WARN, because unknown commit
id is total useless to track change history in changelog, it deserves the
ERROR.
Signed-off-by: Wang YanQing
---
After this patch, in another patch 'checkpatch: add support to check 'Fixes:'
tag format',
I will delete UNKNOWN_COMMIT_ID, because we don't need it anymore after
GIT_COMMIT_ID could
do the same check as UNKNOWN_COMMIT_ID for 'Fixes:' tag.
scripts/checkpatch.pl | 14 --
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 23a001a..143bb43 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2865,8 +2865,18 @@ sub process {
($id, $description) = git_commit_info($orig_commit,
$id, $orig_desc);
+ if (!defined($id)) {
+ if ($long) {
+ ERROR("GIT_COMMIT_ID",
+ "Unknown commit id '$orig_commit' is too
long, maybe misspelled?\n" . $herecurr);
+ } else {
+ ERROR("GIT_COMMIT_ID",
+ "Unknown commit id '$orig_commit', maybe
rebased or not pulled?\n" . $herecurr);
+ }
+ }
+
if (defined($id) &&
- ($short || $long || $space || $case || ($orig_desc
ne $description) || !$hasparens)) {
+ ($short || $space || $case || ($orig_desc ne
$description) || !$hasparens)) {
ERROR("GIT_COMMIT_ID",
"Please use git commit description style
'commit <12+ chars of sha1> (\"\")' - ie: '${init_char}ommit $id
(\"$description\")'\n" . $herecurr);
}
@@ -2969,7 +2979,7 @@ sub process {
}
# check for invalid commit id
- if ($in_commit_log && $line =~
/(^fixes:|\bcommit)\s+([0-9a-f]{6,40})\b/i) {
+ if ($in_commit_log && $line =~
/(^fixes:)\s+([0-9a-f]{6,40})\b/i) {
my $id;
my $description;
($id, $description) = git_commit_info($2, undef, undef);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v4] checkpatch: add support to check 'Fixes:' tag format
According to submitting-patches.rst, 'Fixes:' tag has a little
stricter condition about the one line summary than normal git
commit description:
“...
Do not split the tag across multiple lines, tags are exempt from
the "wrap at 75 columns" rule in order to simplify parsing scripts
...”
And there is no sanity check for 'Fixes:' tag format in checkpatch
the same as GIT_COMMIT_ID for git commit description, so let's expand
the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
The check supports below formats:
Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of
pages it actually freed")
Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
devices"")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
The check doesn't support below formats and it will emit diagnostics info for
them:
Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power amplifiers)
Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU devices
nodes)
Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
work-queue”)
Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page
fault handler"
Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
completion")
Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
Because after GIT_COMMIT_ID supports 'Fixes:' tag format check, it could do
the same check as the UNKNOWN_COMMIT_ID, so we don't need UNKNOWN_COMMIT_ID
anymore and I decide to delete it.
Note: this patch also fixes double quotation mark issue for normal git
commit description, and now it supports double quotation mark in
title line, for example:
Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
when checking usable channels"")
Note: this patch also adds diagnostics info support for normal git commit
description format check.
Based on original patch by Joe Perches
Link:
https://lore.kernel.org/lkml/40bfc40958fca6e2cc9b86101153aa0715fac4f7.ca...@perches.com/
Signed-off-by: Wang YanQing
---
v4:
1: Add diagnostics info support, suggested by Joe Perches and Markus Elfring.
2: Delete UNKNOWN_COMMIT_ID and do the check in GIT_COMMIT_ID.
v3:
1: Fix a bug in short title line support.
v2:
1: Add support for double quotation mark in title line, suggested by Markus
Elfring.
2: Add support for short title line with/without ellipsis.
3: Add supported format examples and unsupported format examples in changelog.
4: Fix a little wording issue in changelog , suggested by Markus Elfring.
scripts/checkpatch.pl | 115 +-
1 file changed, 85 insertions(+), 30 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 143bb43..b5768e0 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2818,48 +2818,93 @@ sub process {
$line !~
/^\s*(?:Link|Patchwork|http|https|BugLink|base-commit):/i &&
$line !~ /^This reverts commit [0-9a-f]{7,40}/ &&
($line =~ /\bcommit\s+[0-9a-f]{5,}\b/i ||
+$line =~ /\bfixes:\s+[0-9a-f]{5,}\b/i ||
($line =~ /(?:\s|^)[0-9a-f]{12,40}(?:[\s"'\(\[]|$)/i &&
- $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i &&
- $line !~ /\bfixes:\s*[0-9a-f]{12,40}/i))) {
+ $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i))) {
my $init_char = "c";
my $orig_commit = "";
+ my $prefix = "commit";
+ my $prefix_case = "[Cc]ommit";
my $short = 1;
my $long = 0;
my $case = 1;
my $space = 1;
my $hasdesc = 0;
- my $hasparens = 0;
+ my $has_parens_and_dqm = 0; # Double quotation mark
+ my $hasprefix = 1;
my $id = '0123456789ab';
my $orig_desc = "commit description";
my $description = "";
+ my $acrosslines = 0;
+ my $title = "title line";
+ my $desc_mismatch = 0;
+ my $diagnostics = "Diagnostics info:\n";
- if ($line =~ /\b(c)o
Re: [PATCH v3] checkpatch: add support to check 'Fixes:' tag format
On Fri, May 01, 2020 at 08:57:42AM -0700, Joe Perches wrote:
> On Fri, 2020-05-01 at 23:40 +0800, Wang YanQing wrote:
> > According to submitting-patches.rst, 'Fixes:' tag has a little
> > stricter condition about the one line summary than normal git
> > commit description:
> > “...
> > Do not split the tag across multiple lines, tags are exempt from
> > the "wrap at 75 columns" rule in order to simplify parsing scripts
> > ...”
> >
> > And there is no sanity check for 'Fixes:' tag format in checkpatch
> > the same as GIT_COMMIT_ID for git commit description, so let's expand
> > the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
> >
> > The check supports below formats:
> > Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number
> > of pages it actually freed")
> > Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
> > devices"")
> > Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
> > Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
> > Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
>
> Hi again YanQing.
>
> I think all the non-standard and incomplete forms
> should have a warning emitted.
Hi Joe Perches
I am not sure whether I get your words, you mean we need to emit warning
for incomplete title line format? For example:
Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
Thanks.
>
> > The check doesn't support below formats:
> > Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
> > Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
> > Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power
> > amplifiers)
> > Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU
> > devices nodes)
> > Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
> > work-queue”)
> > Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT
> > page fault handler"
> > Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
> > completion")
> > Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
>
>
> > Note: this patch also fixes double quotation mark issue for normal git
> > commit description, and now it supports double quotation mark in
> > title line, for example:
> > Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
> > when checking usable channels"")
>
> Nice.
>
[PATCH v3] checkpatch: add support to check 'Fixes:' tag format
According to submitting-patches.rst, 'Fixes:' tag has a little
stricter condition about the one line summary than normal git
commit description:
“...
Do not split the tag across multiple lines, tags are exempt from
the "wrap at 75 columns" rule in order to simplify parsing scripts
...”
And there is no sanity check for 'Fixes:' tag format in checkpatch
the same as GIT_COMMIT_ID for git commit description, so let's expand
the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
The check supports below formats:
Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of
pages it actually freed")
Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
devices"")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
The check doesn't support below formats:
Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power amplifiers)
Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU devices
nodes)
Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
work-queue”)
Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page
fault handler"
Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
completion")
Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
Note: this patch also fixes double quotation mark issue for normal git
commit description, and now it supports double quotation mark in
title line, for example:
Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
when checking usable channels"")
Based on original patch by Joe Perches
Link:
https://lore.kernel.org/lkml/40bfc40958fca6e2cc9b86101153aa0715fac4f7.ca...@perches.com/
Signed-off-by: Wang YanQing
---
v3:
1: Fix a bug in short title line support.
v2:
1: Add support for double quotation mark in title line, suggested by Markus
Elfring.
2: Add support for short title line with/without ellipsis.
3: Add supported format examples and unsupported format examples in changelog.
4: Fix a little wording issue in changelog , suggested by Markus Elfring.
scripts/checkpatch.pl | 68 +++
1 file changed, 49 insertions(+), 19 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 23a001a66006..a649b9f711b6 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2818,57 +2818,87 @@ sub process {
$line !~
/^\s*(?:Link|Patchwork|http|https|BugLink|base-commit):/i &&
$line !~ /^This reverts commit [0-9a-f]{7,40}/ &&
($line =~ /\bcommit\s+[0-9a-f]{5,}\b/i ||
+$line =~ /\bfixes:\s+[0-9a-f]{5,}\b/i ||
($line =~ /(?:\s|^)[0-9a-f]{12,40}(?:[\s"'\(\[]|$)/i &&
- $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i &&
- $line !~ /\bfixes:\s*[0-9a-f]{12,40}/i))) {
+ $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i))) {
my $init_char = "c";
my $orig_commit = "";
+ my $prefix = "commit";
+ my $prefix_case = "[Cc]ommit";
my $short = 1;
my $long = 0;
my $case = 1;
my $space = 1;
my $hasdesc = 0;
- my $hasparens = 0;
+ my $has_parens_and_dqm = 0; # Double quotation mark
my $id = '0123456789ab';
my $orig_desc = "commit description";
my $description = "";
+ my $acrosslines = 0;
+ my $title = "title line";
+ my $desc_mismatch = 0;
- if ($line =~ /\b(c)ommit\s+([0-9a-f]{5,})\b/i) {
+ if ($line =~ /\b(f)ixes:\s+([0-9a-f]{5,})\b/i) {
+ $init_char = $1;
+ $orig_commit = lc($2);
+ $prefix = "Fixes:";
+ $prefix_case = "Fixes:";
+ $init_char = "F";
+ $title = "a single line title (without line
breaks but ellipsis is fine!)";
+ } elsif ($line =~ /\b(c)ommit\s+([0-9a-f]{5,})\b/i) {
$in
[PATCH v2] checkpatch: add support to check 'Fixes:' tag format
According to submitting-patches.rst, 'Fixes:' tag has a little
stricter condition about the one line summary than normal git
commit description:
“...
Do not split the tag across multiple lines, tags are exempt from
the "wrap at 75 columns" rule in order to simplify parsing scripts
...”
And there is no sanity check for 'Fixes:' tag format in checkpatch
the same as GIT_COMMIT_ID for git commit description, so let's expand
the GIT_COMMIT_ID to add 'Fixes:' tag format check support.
The check supports below formats:
Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of
pages it actually freed")
Fixes: 85f7cd3a2aad ("Revert "media: Kconfig: better support hybrid TV
devices"")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered...")
Fixes: 878520ac45f9 ("ext4: save the error code which triggered")
Fixes: 277f27e2f277 ("SUNRPC/cache: Allow garbage collection ... ")
The check doesn't support below formats:
Fixes: f2c2e717642c ("usb: gadget: add raw-gadget interface"
Fixes: 6c73698904aa pinctrl: qcom: Introduce readl/writel accessors
Fixes: 3fd6e7d9a146 (ASoC: tas571x: New driver for TI TAS571x power amplifiers)
Fixes: 55697cbb44e4 ("arm64: dts: renesas: r8a779{65,80,90}: Add IPMMU devices
nodes)
Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a
work-queue”)
Fixes: cd758a9b57ee "KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page
fault handler"
Fixes: 9b1640686470 ("scsi: lpfc: Fix use-after-free mailbox cmd
completion")
Fixes: 03f6fc6de919 ('ASoC: rt5682: Add the soundwire support')
Note: this patch also fixes double quotation mark issue for normal git
commit description, and now it supports double quotation mark in
title line, for example:
Commit e33e2241e272 ("Revert "cfg80211: Use 5MHz bandwidth by default
when checking usable channels"")
Based on original patch by Joe Perches
Link:
https://lore.kernel.org/lkml/40bfc40958fca6e2cc9b86101153aa0715fac4f7.ca...@perches.com/
Signed-off-by: Wang YanQing
---
If we really need to support for another quotation character, I think it is
the single
quotation mark, we could do it in next patch.
v2:
1: Add support for double quotation mark in title line, suggested by Markus
Elfring.
2: Add support for shorter title line with/without ellipsis.
3: Add supported format examples and unsupported format examples in changelog.
4: Fix a little wording issue in changelog , suggested by Markus Elfring.
scripts/checkpatch.pl | 64 ++-
1 file changed, 45 insertions(+), 19 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 23a001a66006..d9c95f1de025 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2818,57 +2818,83 @@ sub process {
$line !~
/^\s*(?:Link|Patchwork|http|https|BugLink|base-commit):/i &&
$line !~ /^This reverts commit [0-9a-f]{7,40}/ &&
($line =~ /\bcommit\s+[0-9a-f]{5,}\b/i ||
+$line =~ /\bfixes:\s+[0-9a-f]{5,}\b/i ||
($line =~ /(?:\s|^)[0-9a-f]{12,40}(?:[\s"'\(\[]|$)/i &&
- $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i &&
- $line !~ /\bfixes:\s*[0-9a-f]{12,40}/i))) {
+ $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i))) {
my $init_char = "c";
my $orig_commit = "";
+ my $prefix = "commit";
+ my $prefix_case = "[Cc]ommit";
my $short = 1;
my $long = 0;
my $case = 1;
my $space = 1;
my $hasdesc = 0;
- my $hasparens = 0;
+ my $has_parens_and_dqm = 0; # Double quotation mark
my $id = '0123456789ab';
my $orig_desc = "commit description";
my $description = "";
+ my $acrosslines = 0;
+ my $title = "title line";
+ my $desc_mismatch = 0;
- if ($line =~ /\b(c)ommit\s+([0-9a-f]{5,})\b/i) {
+ if ($line =~ /\b(f)ixes:\s+([0-9a-f]{5,})\b/i) {
+ $init_char = $1;
+ $orig_commit = lc($2);
+ $prefix = "Fixes:";
+ $prefix_case = "Fixes:";
+ $init_char = "F";
+ $title = "a single line title (without line
breaks but ellipsis is fine!)";
+
Re: [PATCH] checkpatch: add support to check 'Fixes:' tag format
On Wed, Apr 29, 2020 at 07:40:21PM +0200, Markus Elfring wrote:
> > “...
> > Do not split the tag across multiple
> > lines, tags are exempt from the "wrap at 75 columns" rule in order to
> > simplify
> > parsing scripts
> > ...”
>
> Why do you not like the reformatting of the quotation so far
> (if such change descriptions should cope also with specific
> Unicode characters)?
>
> “…
> Do not split the tag across multiple lines, tags are exempt from
> the "wrap at 75 columns" rule in order to simplify parsing scripts.
> …”
>
>
Sigh. I will fix it, but I want to hear from Joe Perches before
next patch version.
>
> > And there is no sanity check for 'Fixes:' tag format in checkpatch the same
> > as GIT_COMMIT_ID for git commit description, so let's expand the
> > GIT_COMMIT_ID
> > to add 'Fixes:' tag format check support.
>
> I have got the impression that this wording might need another bit
> of fine-tuning.
The current wording is enough I think.
>
>
> > + "Please use git commit description style
> > '$prefix <12+ chars of sha1> (\"<$title>\")' - ie: '${init_char}" .
> > substr($prefix, 1) . " $id (\"$description\")'\n" . $herecurr);
>
> I imagine that the support for different quotation characters
> can become helpful, can't it?
No, we don't need to support other quotation character for 'Fixes:' tag
at least now. The submitting-patches.rst tells us the pretty format is:
“...
Fixes: %h (\"%s\")
...”
>
> Regards,
> Markus
Thanks.
[PATCH] checkpatch: add support to check 'Fixes:' tag format
According to submitting-patches.rst, 'Fixes:' tag has a little
stricter condition about the one line summary than normal git
commit description:
“...
Do not split the tag across multiple
lines, tags are exempt from the "wrap at 75 columns" rule in order to simplify
parsing scripts
...”
And there is no sanity check for 'Fixes:' tag format in checkpatch the same
as GIT_COMMIT_ID for git commit description, so let's expand the GIT_COMMIT_ID
to add 'Fixes:' tag format check support.
Based on original patch by Joe Perches
Link:
https://lore.kernel.org/lkml/40bfc40958fca6e2cc9b86101153aa0715fac4f7.ca...@perches.com/
Signed-off-by: Wang YanQing
---
scripts/checkpatch.pl | 36
1 file changed, 24 insertions(+), 12 deletions(-)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 23a001a..879dcf4 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2818,11 +2818,13 @@ sub process {
$line !~
/^\s*(?:Link|Patchwork|http|https|BugLink|base-commit):/i &&
$line !~ /^This reverts commit [0-9a-f]{7,40}/ &&
($line =~ /\bcommit\s+[0-9a-f]{5,}\b/i ||
+$line =~ /\bfixes:\s+[0-9a-f]{5,}\b/i ||
($line =~ /(?:\s|^)[0-9a-f]{12,40}(?:[\s"'\(\[]|$)/i &&
- $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i &&
- $line !~ /\bfixes:\s*[0-9a-f]{12,40}/i))) {
+ $line !~ /[\<\[][0-9a-f]{12,40}[\>\]]/i))) {
my $init_char = "c";
my $orig_commit = "";
+ my $prefix = "commit";
+ my $prefix_case = "[Cc]ommit";
my $short = 1;
my $long = 0;
my $case = 1;
@@ -2832,19 +2834,28 @@ sub process {
my $id = '0123456789ab';
my $orig_desc = "commit description";
my $description = "";
+ my $acrosslines = 0;
+ my $title = "title line";
- if ($line =~ /\b(c)ommit\s+([0-9a-f]{5,})\b/i) {
+ if ($line =~ /\b(f)ixes:\s+([0-9a-f]{5,})\b/i) {
+ $init_char = $1;
+ $orig_commit = lc($2);
+ $prefix = "Fixes:";
+ $prefix_case = "Fixes:";
+ $init_char = "F";
+ $title = "a single line title (without line
breaks)";
+ } elsif ($line =~ /\b(c)ommit\s+([0-9a-f]{5,})\b/i) {
$init_char = $1;
$orig_commit = lc($2);
} elsif ($line =~ /\b([0-9a-f]{12,40})\b/i) {
$orig_commit = lc($1);
}
- $short = 0 if ($line =~ /\bcommit\s+[0-9a-f]{12,40}/i);
- $long = 1 if ($line =~ /\bcommit\s+[0-9a-f]{41,}/i);
- $space = 0 if ($line =~ /\bcommit [0-9a-f]/i);
- $case = 0 if ($line =~
/\b[Cc]ommit\s+[0-9a-f]{5,40}[^A-F]/);
- if ($line =~
/\bcommit\s+[0-9a-f]{5,}\s+\("([^"]+)"\)/i) {
+ $short = 0 if ($line =~ /\b$prefix\s+[0-9a-f]{12,40}/i);
+ $long = 1 if ($line =~ /\b$prefix\s+[0-9a-f]{41,}/i);
+ $space = 0 if ($line =~ /\b$prefix [0-9a-f]/i);
+ $case = 0 if ($line =~
/\b$prefix_case\s+[0-9a-f]{5,40}[^A-F]/);
+ if ($line =~
/\b$prefix\s+[0-9a-f]{5,}\s+\("([^"]+)"\)/i) {
$orig_desc = $1;
$hasparens = 1;
} elsif ($line =~ /\bcommit\s+[0-9a-f]{5,}\s*$/i &&
@@ -2852,23 +2863,24 @@ sub process {
$rawlines[$linenr] =~ /^\s*\("([^"]+)"\)/) {
$orig_desc = $1;
$hasparens = 1;
- } elsif ($line =~
/\bcommit\s+[0-9a-f]{5,}\s+\("[^"]+$/i &&
+ } elsif ($line =~
/\b$prefix\s+[0-9a-f]{5,}\s+\("[^"]+$/i &&
defined $rawlines[$linenr] &&
$rawlines[$linenr] =~ /^\s*[^"]+"\)/) {
- $line =~
/\bcommit\s+[0-9a-f]{5,}\s+\("([^"]+)$/i;
+ $line =~
/\b$prefix\s+[0-9a-f]{5,}\s+\("([^"]+)$/i;
$orig_desc = $1;
Re: [PATCH v3] checkpatch: add dedicated checker for 'Fixes:' tag
On Tue, Apr 28, 2020 at 12:52:59PM +0200, Markus Elfring wrote: > > And there is no 'Fixes:' tag format checker in checkpatch > > I have taken another look at corresponding implementation details. > Will programming challenges get any more attention? > > > > to check the commit id length too, > > The mentioned script contains the following information. > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/scripts/checkpatch.pl?id=b240f960afb900e59112ebcfa5a759bb0a85a14e#n2818 > > # Check for git id commit length and improperly formed commit descriptions > > > > so let's add dedicated checker to check these conditions for 'Fixes:' tag. > > How do you think about to reconsider the usage of the word “checker” > at specific places? Yes, I will use the word "check" only in later version. > > > > + my $id = '0123456789ab'; > > + my $orig_desc = "commit description"; > > * Do you try to extend the existing software analysis approach > “GIT_COMMIT_ID”? > > * Would you like to avoid the development of duplicate Perl code? Fixes: lines don't need to have a "commit" prefix before the commit id, the description in normal commit id could across multiple lines, and we don't need to consider the $commit_log_possible_stack_dump for 'Fixes:' tag line. I mean it will make the GIT_COMMIT_ID code become harder to read and maintain. > > Regards, > Markus
Re: [PATCH v3] checkpatch: add dedicated checker for 'Fixes:' tag
On Tue, Apr 28, 2020 at 08:21:37AM +0200, Markus Elfring wrote:
> > ...
> > Do not split the tag across multiple
> > lines, tags are exempt from the "wrap at 75 columns" rule in order to
> > simplify
> > parsing scripts
> > ...
>
> I suggest to reformat the quotation.
>
> “…
> Do not split the tag across multiple lines, tags are exempt from
> the "wrap at 75 columns" rule in order to simplify parsing scripts.
> …”
>
>
> > And there is no 'Fixes:' tag format checker in checkpatch to check
> > the commit id length too, so let's add dedicated checker to check
> > these conditions for 'Fixes:' tag.
>
> It seems that there are further challenges to consider for an imperative
> wording
> in such a change description.
>
> How do you think about the following wording variant?
>
>The script “checkpatch.pl” did not provide a check for the commit
>identification length. Thus add a dedicated check.
>
>
> > v2-v3
>
> I would find a shorter version identification (without the arrow)
> also sufficient.
>
>
> > 1:No modification to GIT_COMMIT_ID checker.
>
> Would you like to add a space character for the item enumeration?
>
>
> >I make a mistake previously, …
>
> Would you like to use the word “made” here?
>
>
> > + my $id = '0123456789ab';
> > + my $orig_desc = "commit description";
>
> How much do these variable initialisations matter?
>
Well! Thanks and you are right, I will fix them in next version.
>
> > + $lines = 0 if ($1 =~ /^\(\"(?:.*)\"\)$/i);
>
> I wonder why you see a need to use a non-capturing group in such
> a regular expression (when no alternatives were specified there).
I guess it is a better perl programming practice that use non-capturing group
always when
you don't need to use the '$' to access it, it will make code a little faster I
guess.
>
>
> > + ERROR("FIXES_TAG",
> > + "Please use 'Fixes:' tag with commit
> > description style '<12+ chars of sha1> (\"\")', and the title
> > line doesn't across multiple lines - ie: '$fixes_tag_fmt'\n" . $herecurr);
>
> * Would we like to support any other quotation characters around
> the commit summary?
>
I think we don't need now and people could patch it easily when
we need others in future:)
> * I propose to split the error message.
> May it become multi-line?
Yes
>
> * How do you think about another wording variant?
>
> The title must be specified as a single line (without line breaks).
>
Ok
> * Would you like to point questionable commit titles out?
>
> Regards,
> Markus
No, I think your previous wording variant is clear enough.
Thanks again!
Re: [PATCH] selftests/bpf: fix compile errors with older glibc
On Thu, Apr 18, 2019 at 05:09:53AM +, Yonghong Song wrote:
>
>
> On 4/17/19 10:48 AM, Wang YanQing wrote:
> > The older glibc (for example, 2.23) doesn't handle __UAPI_DEF_*
> > in libc-compat.h properly, and it bring below compile errors:
>
> I have an even old glibc 2.17 and it still works. Not sure
> why it failed here. Could you explain more?
We will meet these errors with the combination of some versions of kernel
headers and
some versions of glibc headers.
After some research on the git history of glibc and kernel, I find the reason
behind
the scene is a little complex:
There are some same definitions between glibc's netinet/in.h and kernel's
linux/in6.h,
IPPROTO_HOPOPTS, etc.
These same definitions willn't bring trouble when we include both of them if
kernel and
glibc coordinates with each other well, but the reality is the coordination is
poor and
unsynchronous in history.
Kernel and glibc uses guard macros to detect whether need to export their
definitions,
linux/in6.h includes libc-compat.h which will check the guard macro,
_NETINET_IN_H, and
netinet/in.h includes bits/in.h which will check the guard macro,
_UAPI_LINUX_IN6_H
(glibc-2.19~821 6c82a2f8d7c8e21e39237225c819f182ae438db3 "Coordinate IPv6
definitions for Linux and glibc"),
the problem is in the installation process of kernel headers, the "_UAPI" in
_UAPI_LINUX_IN6_H
in linux/in6.h will be stripped due to commit 56c176c9cac9
("UAPI: strip the _UAPI prefix from header guards during header installation").
The good news is the glibc fix this the trouble by checking the guard macro,
_LINUX_IN6_H, too.
(glibc c9bd40daaee18cf1d9824e4a7ebaebe321e0a5a8 "Bug 20214: Fix linux/in6.h and
netinet/in.h sync.").
My environment still have this trouble:
lsb_release -a:
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 16.04.6 LTS
Release:16.04
Codename: xenial
dpkg -l | grep libc-dev:
ii libc-dev-bin 2.23-0ubuntu11 amd64GNU C
Library: Development binaries
ii linux-libc-dev:amd64 4.4.0-145.171amd64Linux
Kernel Headers for development.
I will send out the v2 which I will change some words in changelog.
Thanks for review.
[PATCH v2] fix compile errors due to unsync linux/in6.h and netinet/in.h
I meet below compile errors: " In file included from test_tcpnotify_kern.c:12: /usr/include/netinet/in.h:101:5: error: expected identifier IPPROTO_HOPOPTS = 0, /* IPv6 Hop-by-Hop options. */ ^ /usr/include/linux/in6.h:131:26: note: expanded from macro 'IPPROTO_HOPOPTS' ^ In file included from test_tcpnotify_kern.c:12: /usr/include/netinet/in.h:103:5: error: expected identifier IPPROTO_ROUTING = 43, /* IPv6 routing header. */ ^ /usr/include/linux/in6.h:132:26: note: expanded from macro 'IPPROTO_ROUTING' ^ In file included from test_tcpnotify_kern.c:12: /usr/include/netinet/in.h:105:5: error: expected identifier IPPROTO_FRAGMENT = 44, /* IPv6 fragmentation header. */ ^ /usr/include/linux/in6.h:133:26: note: expanded from macro 'IPPROTO_FRAGMENT' " The same compile errors are reported for test_tcpbpf_kern.c too. My environment: lsb_release -a: No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 16.04.6 LTS Release:16.04 Codename: xenial dpkg -l | grep libc-dev: ii libc-dev-bin 2.23-0ubuntu11 amd64GNU C Library: Development binaries ii linux-libc-dev:amd64 4.4.0-145.171amd64Linux Kernel Headers for development. The reason is linux/in6.h and netinet/in.h aren't synchronous about how to handle the same definitions, IPPROTO_HOPOPTS, etc. This patch fixes the compile errors by moving to before the . Signed-off-by: Wang YanQing --- Changes v1-v2: 1:Rewrite the changelog. tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c| 2 +- tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c index 74f73b3..c7c3240 100644 --- a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c +++ b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -9,7 +10,6 @@ #include #include #include -#include #include "bpf_helpers.h" #include "bpf_endian.h" #include "test_tcpbpf.h" diff --git a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c index edbca20..ec6db6e 100644 --- a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c +++ b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -9,7 +10,6 @@ #include #include #include -#include #include "bpf_helpers.h" #include "bpf_endian.h" #include "test_tcpnotify.h" -- 1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH] selftests/bpf: fix compile errors with older glibc
On Thu, Apr 18, 2019 at 05:09:53AM +, Yonghong Song wrote:
>
>
> On 4/17/19 10:48 AM, Wang YanQing wrote:
> > The older glibc (for example, 2.23) doesn't handle __UAPI_DEF_*
> > in libc-compat.h properly, and it bring below compile errors:
>
> I have an even old glibc 2.17 and it still works. Not sure
> why it failed here. Could you explain more?
We will meet these errors with the combination of some versions of kernel
headers and
some versions of glibc headers.
After some research on the git history of glibc and kernel, I find the reason
behind
the scene is a little complex:
There are some same definitions between glibc's netinet/in.h and kernel's
linux/in6.h,
IPPROTO_HOPOPTS, etc.
These same definitions willn't bring trouble when we include both of them if
kernel and
glibc coordinates with each other well, but the reality is the coordination is
poor and
unsynchronous in history.
Kernel and glibc uses guard macros to detect whether need to export their
definitions,
linux/in6.h includes libc-compat.h which will check the guard macro,
_NETINET_IN_H, and
netinet/in.h includes bits/in.h which will check the guard macro,
_UAPI_LINUX_IN6_H
(glibc-2.19~821 6c82a2f8d7c8e21e39237225c819f182ae438db3 "Coordinate IPv6
definitions for Linux and glibc"),
the problem is in the installation process of kernel headers, the "_UAPI" in
_UAPI_LINUX_IN6_H
in linux/in6.h will be stripped due to commit 56c176c9cac9
("UAPI: strip the _UAPI prefix from header guards during header installation").
The good news is the glibc fix this the trouble by checking the guard macro,
_LINUX_IN6_H, too.
(glibc c9bd40daaee18cf1d9824e4a7ebaebe321e0a5a8 "Bug 20214: Fix linux/in6.h and
netinet/in.h sync.").
My environment still have this trouble:
lsb_release -a:
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 16.04.6 LTS
Release:16.04
Codename: xenial
dpkg -l | grep libc-dev:
ii libc-dev-bin 2.23-0ubuntu11 amd64GNU C
Library: Development binaries
ii linux-libc-dev:amd64 4.4.0-145.171amd64Linux
Kernel Headers for development.
I will send out the v2 which I will change some words in changelog.
Thanks for review.
[PATCH] selftests/bpf: fix compile errors with older glibc
The older glibc (for example, 2.23) doesn't handle __UAPI_DEF_* in libc-compat.h properly, and it bring below compile errors: " In file included from test_tcpnotify_kern.c:12: /usr/include/netinet/in.h:101:5: error: expected identifier IPPROTO_HOPOPTS = 0, /* IPv6 Hop-by-Hop options. */ ^ /usr/include/linux/in6.h:131:26: note: expanded from macro 'IPPROTO_HOPOPTS' ^ In file included from test_tcpnotify_kern.c:12: /usr/include/netinet/in.h:103:5: error: expected identifier IPPROTO_ROUTING = 43, /* IPv6 routing header. */ ^ /usr/include/linux/in6.h:132:26: note: expanded from macro 'IPPROTO_ROUTING' ^ In file included from test_tcpnotify_kern.c:12: /usr/include/netinet/in.h:105:5: error: expected identifier IPPROTO_FRAGMENT = 44, /* IPv6 fragmentation header. */ ^ /usr/include/linux/in6.h:133:26: note: expanded from macro 'IPPROTO_FRAGMENT' " The same compile errors are reported for test_tcpbpf_kern.c too. This patch fixes the compile errors by moving to before the . Signed-off-by: Wang YanQing --- tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c| 2 +- tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c index 74f73b3..c7c3240 100644 --- a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c +++ b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -9,7 +10,6 @@ #include #include #include -#include #include "bpf_helpers.h" #include "bpf_endian.h" #include "test_tcpbpf.h" diff --git a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c index edbca20..ec6db6e 100644 --- a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c +++ b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -9,7 +10,6 @@ #include #include #include -#include #include "bpf_helpers.h" #include "bpf_endian.h" #include "test_tcpnotify.h" -- 1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, x32: Fix regression caused by commit 24dea04767e6
Commit 24dea04767e6 ("bpf, x32: remove ld_abs/ld_ind")
removed the 4 /* Extra space for skb_copy_bits buffer */
from _STACK_SIZE, but it didn't fix the concerned code
in emit_prologue and emit_epilogue, and this error will
bring very strange kernel runtime errors.
This patch fix it.
Fixes: 24dea04767e6 ("bpf, x32: remove ld_abs/ld_ind")
Signed-off-by: Wang YanQing
---
arch/x86/net/bpf_jit_comp32.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5579987..8f6cc71 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -1441,8 +1441,8 @@ static void emit_prologue(u8 **pprog, u32 stack_depth)
/* sub esp,STACK_SIZE */
EMIT2_off32(0x81, 0xEC, STACK_SIZE);
- /* sub ebp,SCRATCH_SIZE+4+12*/
- EMIT3(0x83, add_1reg(0xE8, IA32_EBP), SCRATCH_SIZE + 16);
+ /* sub ebp,SCRATCH_SIZE+12*/
+ EMIT3(0x83, add_1reg(0xE8, IA32_EBP), SCRATCH_SIZE + 12);
/* xor ebx,ebx */
EMIT2(0x31, add_2reg(0xC0, IA32_EBX, IA32_EBX));
@@ -1475,8 +1475,8 @@ static void emit_epilogue(u8 **pprog, u32 stack_depth)
/* mov edx,dword ptr [ebp+off]*/
EMIT3(0x8B, add_2reg(0x40, IA32_EBP, IA32_EDX), STACK_VAR(r0[1]));
- /* add ebp,SCRATCH_SIZE+4+12*/
- EMIT3(0x83, add_1reg(0xC0, IA32_EBP), SCRATCH_SIZE + 16);
+ /* add ebp,SCRATCH_SIZE+12*/
+ EMIT3(0x83, add_1reg(0xC0, IA32_EBP), SCRATCH_SIZE + 12);
/* mov ebx,dword ptr [ebp-12]*/
EMIT3(0x8B, add_2reg(0x40, IA32_EBP, IA32_EBX), -12);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, x32: Fix regression caused by commit 24dea04767e6
Commit 24dea04767e6 ("bpf, x32: remove ld_abs/ld_ind")
removed the 4 /* Extra space for skb_copy_bits buffer */
from _STACK_SIZE, but it didn't fix the concerned code
in emit_prologue and emit_epilogue, and this error will
bring very strange kernel runtime errors.
This patch fix it.
Fixes: 24dea04767e6 ("bpf, x32: remove ld_abs/ld_ind")
Signed-off-by: Wang YanQing
---
arch/x86/net/bpf_jit_comp32.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5579987..8f6cc71 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -1441,8 +1441,8 @@ static void emit_prologue(u8 **pprog, u32 stack_depth)
/* sub esp,STACK_SIZE */
EMIT2_off32(0x81, 0xEC, STACK_SIZE);
- /* sub ebp,SCRATCH_SIZE+4+12*/
- EMIT3(0x83, add_1reg(0xE8, IA32_EBP), SCRATCH_SIZE + 16);
+ /* sub ebp,SCRATCH_SIZE+12*/
+ EMIT3(0x83, add_1reg(0xE8, IA32_EBP), SCRATCH_SIZE + 12);
/* xor ebx,ebx */
EMIT2(0x31, add_2reg(0xC0, IA32_EBX, IA32_EBX));
@@ -1475,8 +1475,8 @@ static void emit_epilogue(u8 **pprog, u32 stack_depth)
/* mov edx,dword ptr [ebp+off]*/
EMIT3(0x8B, add_2reg(0x40, IA32_EBP, IA32_EDX), STACK_VAR(r0[1]));
- /* add ebp,SCRATCH_SIZE+4+12*/
- EMIT3(0x83, add_1reg(0xC0, IA32_EBP), SCRATCH_SIZE + 16);
+ /* add ebp,SCRATCH_SIZE+12*/
+ EMIT3(0x83, add_1reg(0xC0, IA32_EBP), SCRATCH_SIZE + 12);
/* mov ebx,dword ptr [ebp-12]*/
EMIT3(0x8B, add_2reg(0x40, IA32_EBP, IA32_EBX), -12);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] bpf, arm32: Correct check_imm24
imm24 is signed, so the right range is:
[-(1<<(24 - 1)), (1<<(24 - 1)) - 1]
Note:this patch also fix a typo.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes
v1-v2:
1:Rewrite the patch, I make a mistake, the v1 is wrong totally,
reported by Russell King.
I use the fix suggested by Russell King instead of myself which
use the exact number range [-8388608, 8388607].
2:Fix the error in changelog.
Thanks!
arch/arm/net/bpf_jit_32.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index caccc78..316bc08 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -84,7 +84,7 @@
*
* 1. First argument is passed using the arm 32bit registers and rest of the
* arguments are passed on stack scratch space.
- * 2. First callee-saved arugument is mapped to arm 32 bit registers and rest
+ * 2. First callee-saved argument is mapped to arm 32 bit registers and rest
* arguments are mapped to scratch space on stack.
* 3. We need two 64 bit temp registers to do complex operations on eBPF
* registers.
@@ -1199,8 +1199,8 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
s32 jmp_offset;
#define check_imm(bits, imm) do { \
- if imm) > 0) && ((imm) >> (bits))) || \
- (((imm) < 0) && (~(imm) >> (bits { \
+ if ((imm) >= (1 << ((bits) - 1)) || \
+ (imm) < -(1 << ((bits) - 1))) { \
pr_info("[%2d] imm=%d(0x%x) out of range\n",\
i, imm, imm); \
return -EINVAL; \
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] bpf, arm32: Correct check_imm24
imm24 is signed, so the right range is:
[-(1<<(24 - 1)), (1<<(24 - 1)) - 1]
Note:this patch also fix a typo.
Signed-off-by: Wang YanQing
---
Changes
v1-v2:
1:Rewrite the patch, I make a mistake, the v1 is wrong totally,
reported by Russell King.
I use the fix suggested by Russell King instead of myself which
use the exact number range [-8388608, 8388607].
2:Fix the error in changelog.
Thanks!
arch/arm/net/bpf_jit_32.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index caccc78..316bc08 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -84,7 +84,7 @@
*
* 1. First argument is passed using the arm 32bit registers and rest of the
* arguments are passed on stack scratch space.
- * 2. First callee-saved arugument is mapped to arm 32 bit registers and rest
+ * 2. First callee-saved argument is mapped to arm 32 bit registers and rest
* arguments are mapped to scratch space on stack.
* 3. We need two 64 bit temp registers to do complex operations on eBPF
* registers.
@@ -1199,8 +1199,8 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
s32 jmp_offset;
#define check_imm(bits, imm) do { \
- if imm) > 0) && ((imm) >> (bits))) || \
- (((imm) < 0) && (~(imm) >> (bits { \
+ if ((imm) >= (1 << ((bits) - 1)) || \
+ (imm) < -(1 << ((bits) - 1))) { \
pr_info("[%2d] imm=%d(0x%x) out of range\n",\
i, imm, imm); \
return -EINVAL; \
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] bpf, arm32: Fix inconsistent naming about emit_a32_lsr_r64|emit_a32_lsr_i64
The names for BPF_ALU64 | BPF_ARSH are emit_a32_arsh_*,
the names for BPF_ALU64 | BPF_LSH are emit_a32_lsh_*, but
the names for BPF_ALU64 | BPF_RSH are emit_a32_lsr_*.
For consistence reason, let's rename emit_a32_lsr_* to
emit_a32_rsh_*.
This patch also correct a wrong comment.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes v1-v2:
1:Rewords the changelog.
arch/arm/net/bpf_jit_32.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index b5030e1..caccc78 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -708,7 +708,7 @@ static inline void emit_a32_arsh_r64(const u8 dst[], const
u8 src[], bool dstk,
}
/* dst = dst >> src */
-static inline void emit_a32_lsr_r64(const u8 dst[], const u8 src[], bool dstk,
+static inline void emit_a32_rsh_r64(const u8 dst[], const u8 src[], bool dstk,
bool sstk, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -724,7 +724,7 @@ static inline void emit_a32_lsr_r64(const u8 dst[], const
u8 src[], bool dstk,
emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
}
- /* Do LSH operation */
+ /* Do RSH operation */
emit(ARM_RSB_I(ARM_IP, rt, 32), ctx);
emit(ARM_SUBS_I(tmp2[0], rt, 32), ctx);
emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_LSR, rt), ctx);
@@ -774,7 +774,7 @@ static inline void emit_a32_lsh_i64(const u8 dst[], bool
dstk,
}
/* dst = dst >> val */
-static inline void emit_a32_lsr_i64(const u8 dst[], bool dstk,
+static inline void emit_a32_rsh_i64(const u8 dst[], bool dstk,
const u32 val, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -1330,7 +1330,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
case BPF_ALU64 | BPF_RSH | BPF_K:
if (unlikely(imm > 63))
return -EINVAL;
- emit_a32_lsr_i64(dst, dstk, imm, ctx);
+ emit_a32_rsh_i64(dst, dstk, imm, ctx);
break;
/* dst = dst << src */
case BPF_ALU64 | BPF_LSH | BPF_X:
@@ -1338,7 +1338,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
break;
/* dst = dst >> src */
case BPF_ALU64 | BPF_RSH | BPF_X:
- emit_a32_lsr_r64(dst, src, dstk, sstk, ctx);
+ emit_a32_rsh_r64(dst, src, dstk, sstk, ctx);
break;
/* dst = dst >> src (signed) */
case BPF_ALU64 | BPF_ARSH | BPF_X:
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] bpf, arm32: Fix inconsistent naming about emit_a32_lsr_r64|emit_a32_lsr_i64
The names for BPF_ALU64 | BPF_ARSH are emit_a32_arsh_*,
the names for BPF_ALU64 | BPF_LSH are emit_a32_lsh_*, but
the names for BPF_ALU64 | BPF_RSH are emit_a32_lsr_*.
For consistence reason, let's rename emit_a32_lsr_* to
emit_a32_rsh_*.
This patch also correct a wrong comment.
Signed-off-by: Wang YanQing
---
Changes v1-v2:
1:Rewords the changelog.
arch/arm/net/bpf_jit_32.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index b5030e1..caccc78 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -708,7 +708,7 @@ static inline void emit_a32_arsh_r64(const u8 dst[], const
u8 src[], bool dstk,
}
/* dst = dst >> src */
-static inline void emit_a32_lsr_r64(const u8 dst[], const u8 src[], bool dstk,
+static inline void emit_a32_rsh_r64(const u8 dst[], const u8 src[], bool dstk,
bool sstk, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -724,7 +724,7 @@ static inline void emit_a32_lsr_r64(const u8 dst[], const
u8 src[], bool dstk,
emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
}
- /* Do LSH operation */
+ /* Do RSH operation */
emit(ARM_RSB_I(ARM_IP, rt, 32), ctx);
emit(ARM_SUBS_I(tmp2[0], rt, 32), ctx);
emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_LSR, rt), ctx);
@@ -774,7 +774,7 @@ static inline void emit_a32_lsh_i64(const u8 dst[], bool
dstk,
}
/* dst = dst >> val */
-static inline void emit_a32_lsr_i64(const u8 dst[], bool dstk,
+static inline void emit_a32_rsh_i64(const u8 dst[], bool dstk,
const u32 val, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -1330,7 +1330,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
case BPF_ALU64 | BPF_RSH | BPF_K:
if (unlikely(imm > 63))
return -EINVAL;
- emit_a32_lsr_i64(dst, dstk, imm, ctx);
+ emit_a32_rsh_i64(dst, dstk, imm, ctx);
break;
/* dst = dst << src */
case BPF_ALU64 | BPF_LSH | BPF_X:
@@ -1338,7 +1338,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
break;
/* dst = dst >> src */
case BPF_ALU64 | BPF_RSH | BPF_X:
- emit_a32_lsr_r64(dst, src, dstk, sstk, ctx);
+ emit_a32_rsh_r64(dst, src, dstk, sstk, ctx);
break;
/* dst = dst >> src (signed) */
case BPF_ALU64 | BPF_ARSH | BPF_X:
--
1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH] bpf, arm32: Correct check_imm24
On Thu, May 10, 2018 at 08:56:57AM +0100, Russell King - ARM Linux wrote:
> On Thu, May 10, 2018 at 11:20:13AM +0800, Wang YanQing wrote:
> > imm24 is signed, so the right range is:
> > [-(2<<(24 - 1)), (2<<(24 - 1)) - 1]
>
> 2 << (24 - 1) is the same as 1 << 24.
>
> > -#define check_imm(bits, imm) do { \
> > - if imm) > 0) && ((imm) >> (bits))) || \
> > - (((imm) < 0) && (~(imm) >> (bits { \
> > - pr_info("[%2d] imm=%d(0x%x) out of range\n",\
> > - i, imm, imm); \
> > +#define check_imm_range(min, max, imm) do {\
> > + if (imm < min || imm > max) { \
> > + pr_info("[%2d] imm=%d is out of range\n", \
> > + i, imm);\
> > return -EINVAL; \
> > } \
> > } while (0)
> > -#define check_imm24(imm) check_imm(24, imm)
> > +#define check_imm24(imm) check_imm_range(-16777216, 16777215, imm)
>
> How is this any different?
>
> If imm is 16777216, then "imm > max" in your version is true.
> In the original version, "imm > 0" is true, so we then test for
> "16777216 >> 24" being non-zero. That's also true, so the test
> condition fires.
>
> If imm is 16777215, then "imm > max" is false in your version.
> In the original version, the conditions also evaluate to false.
>
> For the -16777217 case, "imm < min" in your version is true.
> In the original version, "imm < 0" is true, so we then test for
> "~(-16777217) >> 24" being non-zero. This is the same as
> "16777216 >> 24" being non-zero, which is true so the condition
> fires.
>
> With -16777216, the same thing happens, both end up evaluating
> to false.
>
> So, the two cases end up producing identical results, and there
> is no actual effect from this change.
>
> However, your commit message is correct - there is a bug here.
> That's obvious when you mask the "imm" value with 0x00ff,
> and realise that an imm value of -16777216 ends up having the
> same value in the instruction as an imm value of 0. So, the
> range of "imm" is _half_ that.
>
> #define check_imm(bits, imm) do {\
> - if imm) > 0) && ((imm) >> (bits))) || \
> - (((imm) < 0) && (~(imm) >> (bits { \
> + if imm) > 0) && ((imm) >> (bits - 1))) || \
> + (((imm) < 0) && (~(imm) >> (bits - 1 { \
> pr_info("[%2d] imm=%d(0x%x) out of range\n",\
> i, imm, imm); \
>
> would fix it. Alternatively:
>
> #define check_imm(bits, imm) do {\
> - if imm) > 0) && ((imm) >> (bits))) || \
> - (((imm) < 0) && (~(imm) >> (bits { \
> + if ((imm) >= (1 << ((bits) - 1)) || \
> + (imm) < -(1 << ((bits) - 1))) { \
> pr_info("[%2d] imm=%d(0x%x) out of range\n",\
> i, imm, imm); \
>
> would also fix it.
Hi!
Sorry for confusion, I make a mistake here, the real fix I want to
submit is [8388607, -8388608], this range has the same effect as your
suggestion.
Will you fix it? or I resend another version?
Thanks.
Re: [PATCH] bpf, arm32: Correct check_imm24
On Thu, May 10, 2018 at 08:56:57AM +0100, Russell King - ARM Linux wrote:
> On Thu, May 10, 2018 at 11:20:13AM +0800, Wang YanQing wrote:
> > imm24 is signed, so the right range is:
> > [-(2<<(24 - 1)), (2<<(24 - 1)) - 1]
>
> 2 << (24 - 1) is the same as 1 << 24.
>
> > -#define check_imm(bits, imm) do { \
> > - if imm) > 0) && ((imm) >> (bits))) || \
> > - (((imm) < 0) && (~(imm) >> (bits { \
> > - pr_info("[%2d] imm=%d(0x%x) out of range\n",\
> > - i, imm, imm); \
> > +#define check_imm_range(min, max, imm) do {\
> > + if (imm < min || imm > max) { \
> > + pr_info("[%2d] imm=%d is out of range\n", \
> > + i, imm);\
> > return -EINVAL; \
> > } \
> > } while (0)
> > -#define check_imm24(imm) check_imm(24, imm)
> > +#define check_imm24(imm) check_imm_range(-16777216, 16777215, imm)
>
> How is this any different?
>
> If imm is 16777216, then "imm > max" in your version is true.
> In the original version, "imm > 0" is true, so we then test for
> "16777216 >> 24" being non-zero. That's also true, so the test
> condition fires.
>
> If imm is 16777215, then "imm > max" is false in your version.
> In the original version, the conditions also evaluate to false.
>
> For the -16777217 case, "imm < min" in your version is true.
> In the original version, "imm < 0" is true, so we then test for
> "~(-16777217) >> 24" being non-zero. This is the same as
> "16777216 >> 24" being non-zero, which is true so the condition
> fires.
>
> With -16777216, the same thing happens, both end up evaluating
> to false.
>
> So, the two cases end up producing identical results, and there
> is no actual effect from this change.
>
> However, your commit message is correct - there is a bug here.
> That's obvious when you mask the "imm" value with 0x00ff,
> and realise that an imm value of -16777216 ends up having the
> same value in the instruction as an imm value of 0. So, the
> range of "imm" is _half_ that.
>
> #define check_imm(bits, imm) do {\
> - if imm) > 0) && ((imm) >> (bits))) || \
> - (((imm) < 0) && (~(imm) >> (bits { \
> + if imm) > 0) && ((imm) >> (bits - 1))) || \
> + (((imm) < 0) && (~(imm) >> (bits - 1 { \
> pr_info("[%2d] imm=%d(0x%x) out of range\n",\
> i, imm, imm); \
>
> would fix it. Alternatively:
>
> #define check_imm(bits, imm) do {\
> - if imm) > 0) && ((imm) >> (bits))) || \
> - (((imm) < 0) && (~(imm) >> (bits { \
> + if ((imm) >= (1 << ((bits) - 1)) || \
> + (imm) < -(1 << ((bits) - 1))) { \
> pr_info("[%2d] imm=%d(0x%x) out of range\n",\
> i, imm, imm); \
>
> would also fix it.
Hi!
Sorry for confusion, I make a mistake here, the real fix I want to
submit is [8388607, -8388608], this range has the same effect as your
suggestion.
Will you fix it? or I resend another version?
Thanks.
[PATCH] bpf, arm32: Correct check_imm24
imm24 is signed, so the right range is:
[-(2<<(24 - 1)), (2<<(24 - 1)) - 1]
Note:this patch also fix a typo.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
arch/arm/net/bpf_jit_32.c | 13 ++---
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index c0b4124..19c6d77 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -84,7 +84,7 @@
*
* 1. First argument is passed using the arm 32bit registers and rest of the
* arguments are passed on stack scratch space.
- * 2. First callee-saved arugument is mapped to arm 32 bit registers and rest
+ * 2. First callee-saved argument is mapped to arm 32 bit registers and rest
* arguments are mapped to scratch space on stack.
* 3. We need two 64 bit temp registers to do complex operations on eBPF
* registers.
@@ -1198,15 +1198,14 @@ static int build_insn(const struct bpf_insn *insn,
struct jit_ctx *ctx)
u8 rd, rt, rm, rn;
s32 jmp_offset;
-#define check_imm(bits, imm) do { \
- if imm) > 0) && ((imm) >> (bits))) || \
- (((imm) < 0) && (~(imm) >> (bits { \
- pr_info("[%2d] imm=%d(0x%x) out of range\n",\
- i, imm, imm); \
+#define check_imm_range(min, max, imm) do {\
+ if (imm < min || imm > max) { \
+ pr_info("[%2d] imm=%d is out of range\n", \
+ i, imm);\
return -EINVAL; \
} \
} while (0)
-#define check_imm24(imm) check_imm(24, imm)
+#define check_imm24(imm) check_imm_range(-16777216, 16777215, imm)
switch (code) {
/* ALU operations */
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, arm32: Correct check_imm24
imm24 is signed, so the right range is:
[-(2<<(24 - 1)), (2<<(24 - 1)) - 1]
Note:this patch also fix a typo.
Signed-off-by: Wang YanQing
---
arch/arm/net/bpf_jit_32.c | 13 ++---
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index c0b4124..19c6d77 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -84,7 +84,7 @@
*
* 1. First argument is passed using the arm 32bit registers and rest of the
* arguments are passed on stack scratch space.
- * 2. First callee-saved arugument is mapped to arm 32 bit registers and rest
+ * 2. First callee-saved argument is mapped to arm 32 bit registers and rest
* arguments are mapped to scratch space on stack.
* 3. We need two 64 bit temp registers to do complex operations on eBPF
* registers.
@@ -1198,15 +1198,14 @@ static int build_insn(const struct bpf_insn *insn,
struct jit_ctx *ctx)
u8 rd, rt, rm, rn;
s32 jmp_offset;
-#define check_imm(bits, imm) do { \
- if imm) > 0) && ((imm) >> (bits))) || \
- (((imm) < 0) && (~(imm) >> (bits { \
- pr_info("[%2d] imm=%d(0x%x) out of range\n",\
- i, imm, imm); \
+#define check_imm_range(min, max, imm) do {\
+ if (imm < min || imm > max) { \
+ pr_info("[%2d] imm=%d is out of range\n", \
+ i, imm);\
return -EINVAL; \
} \
} while (0)
-#define check_imm24(imm) check_imm(24, imm)
+#define check_imm24(imm) check_imm_range(-16777216, 16777215, imm)
switch (code) {
/* ALU operations */
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, arm32: Fix inconsistent naming about emit_a32_lsr_r64|emit_a32_lsr_i64
The reasonable names for emit_a32_lsr_r64|emit_a32_lsr_i64 are
emit_a32_rsh_r64|emit_a32_rsh_i64.
This patch also correct a wrong comment.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
arch/arm/net/bpf_jit_32.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index 82689b9..c0b4124 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -708,7 +708,7 @@ static inline void emit_a32_arsh_r64(const u8 dst[], const
u8 src[], bool dstk,
}
/* dst = dst >> src */
-static inline void emit_a32_lsr_r64(const u8 dst[], const u8 src[], bool dstk,
+static inline void emit_a32_rsh_r64(const u8 dst[], const u8 src[], bool dstk,
bool sstk, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -724,7 +724,7 @@ static inline void emit_a32_lsr_r64(const u8 dst[], const
u8 src[], bool dstk,
emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
}
- /* Do LSH operation */
+ /* Do RSH operation */
emit(ARM_RSB_I(ARM_IP, rt, 32), ctx);
emit(ARM_SUBS_I(tmp2[0], rt, 32), ctx);
emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_LSR, rt), ctx);
@@ -774,7 +774,7 @@ static inline void emit_a32_lsh_i64(const u8 dst[], bool
dstk,
}
/* dst = dst >> val */
-static inline void emit_a32_lsr_i64(const u8 dst[], bool dstk,
+static inline void emit_a32_rsh_i64(const u8 dst[], bool dstk,
const u32 val, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -1330,7 +1330,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
case BPF_ALU64 | BPF_RSH | BPF_K:
if (unlikely(imm > 63))
return -EINVAL;
- emit_a32_lsr_i64(dst, dstk, imm, ctx);
+ emit_a32_rsh_i64(dst, dstk, imm, ctx);
break;
/* dst = dst << src */
case BPF_ALU64 | BPF_LSH | BPF_X:
@@ -1338,7 +1338,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
break;
/* dst = dst >> src */
case BPF_ALU64 | BPF_RSH | BPF_X:
- emit_a32_lsr_r64(dst, src, dstk, sstk, ctx);
+ emit_a32_rsh_r64(dst, src, dstk, sstk, ctx);
break;
/* dst = dst >> src (signed) */
case BPF_ALU64 | BPF_ARSH | BPF_X:
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, arm32: Fix inconsistent naming about emit_a32_lsr_r64|emit_a32_lsr_i64
The reasonable names for emit_a32_lsr_r64|emit_a32_lsr_i64 are
emit_a32_rsh_r64|emit_a32_rsh_i64.
This patch also correct a wrong comment.
Signed-off-by: Wang YanQing
---
arch/arm/net/bpf_jit_32.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index 82689b9..c0b4124 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -708,7 +708,7 @@ static inline void emit_a32_arsh_r64(const u8 dst[], const
u8 src[], bool dstk,
}
/* dst = dst >> src */
-static inline void emit_a32_lsr_r64(const u8 dst[], const u8 src[], bool dstk,
+static inline void emit_a32_rsh_r64(const u8 dst[], const u8 src[], bool dstk,
bool sstk, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -724,7 +724,7 @@ static inline void emit_a32_lsr_r64(const u8 dst[], const
u8 src[], bool dstk,
emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
}
- /* Do LSH operation */
+ /* Do RSH operation */
emit(ARM_RSB_I(ARM_IP, rt, 32), ctx);
emit(ARM_SUBS_I(tmp2[0], rt, 32), ctx);
emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_LSR, rt), ctx);
@@ -774,7 +774,7 @@ static inline void emit_a32_lsh_i64(const u8 dst[], bool
dstk,
}
/* dst = dst >> val */
-static inline void emit_a32_lsr_i64(const u8 dst[], bool dstk,
+static inline void emit_a32_rsh_i64(const u8 dst[], bool dstk,
const u32 val, struct jit_ctx *ctx) {
const u8 *tmp = bpf2a32[TMP_REG_1];
const u8 *tmp2 = bpf2a32[TMP_REG_2];
@@ -1330,7 +1330,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
case BPF_ALU64 | BPF_RSH | BPF_K:
if (unlikely(imm > 63))
return -EINVAL;
- emit_a32_lsr_i64(dst, dstk, imm, ctx);
+ emit_a32_rsh_i64(dst, dstk, imm, ctx);
break;
/* dst = dst << src */
case BPF_ALU64 | BPF_LSH | BPF_X:
@@ -1338,7 +1338,7 @@ static int build_insn(const struct bpf_insn *insn, struct
jit_ctx *ctx)
break;
/* dst = dst >> src */
case BPF_ALU64 | BPF_RSH | BPF_X:
- emit_a32_lsr_r64(dst, src, dstk, sstk, ctx);
+ emit_a32_rsh_r64(dst, src, dstk, sstk, ctx);
break;
/* dst = dst >> src (signed) */
case BPF_ALU64 | BPF_ARSH | BPF_X:
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, doc: clarification for the meaning of 'id'
For me, as a reader whose mother language isn't English, the old words bring a little difficulty to catch the meaning, this patch rewords the subsection in a more clarificatory way. This patch also add blank lines as separator at two places to improve readability. Signed-off-by: Wang YanQing <udkni...@gmail.com> --- Documentation/networking/filter.txt | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt index 5032e12..e6b4ebb 100644 --- a/Documentation/networking/filter.txt +++ b/Documentation/networking/filter.txt @@ -1142,6 +1142,7 @@ into a register from memory, the register's top 56 bits are known zero, while the low 8 are unknown - which is represented as the tnum (0x0; 0xff). If we then OR this with 0x40, we get (0x40; 0xbf), then if we add 1 we get (0x0; 0x1ff), because of potential carries. + Besides arithmetic, the register state can also be updated by conditional branches. For instance, if a SCALAR_VALUE is compared > 8, in the 'true' branch it will have a umin_value (unsigned minimum value) of 9, whereas in the 'false' @@ -1150,14 +1151,16 @@ BPF_JSGE) would instead update the signed minimum/maximum values. Information from the signed and unsigned bounds can be combined; for instance if a value is first tested < 8 and then tested s> 4, the verifier will conclude that the value is also > 4 and s< 8, since the bounds prevent crossing the sign boundary. + PTR_TO_PACKETs with a variable offset part have an 'id', which is common to all pointers sharing that same variable offset. This is important for packet range -checks: after adding some variable to a packet pointer, if you then copy it to -another register and (say) add a constant 4, both registers will share the same -'id' but one will have a fixed offset of +4. Then if it is bounds-checked and -found to be less than a PTR_TO_PACKET_END, the other register is now known to -have a safe range of at least 4 bytes. See 'Direct packet access', below, for -more on PTR_TO_PACKET ranges. +checks: after adding a variable to a packet pointer register A, if you then copy +it to another register B and then add a constant 4 to A, both registers will +share the same 'id' but the A will have a fixed offset of +4. Then if A is +bounds-checked and found to be less than a PTR_TO_PACKET_END, the register B is +now known to have a safe range of at least 4 bytes. See 'Direct packet access', +below, for more on PTR_TO_PACKET ranges. + The 'id' field is also used on PTR_TO_MAP_VALUE_OR_NULL, common to all copies of the pointer returned from a map lookup. This means that when one copy is checked and found to be non-NULL, all copies can become PTR_TO_MAP_VALUEs. -- 1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, doc: clarification for the meaning of 'id'
For me, as a reader whose mother language isn't English, the old words bring a little difficulty to catch the meaning, this patch rewords the subsection in a more clarificatory way. This patch also add blank lines as separator at two places to improve readability. Signed-off-by: Wang YanQing --- Documentation/networking/filter.txt | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt index 5032e12..e6b4ebb 100644 --- a/Documentation/networking/filter.txt +++ b/Documentation/networking/filter.txt @@ -1142,6 +1142,7 @@ into a register from memory, the register's top 56 bits are known zero, while the low 8 are unknown - which is represented as the tnum (0x0; 0xff). If we then OR this with 0x40, we get (0x40; 0xbf), then if we add 1 we get (0x0; 0x1ff), because of potential carries. + Besides arithmetic, the register state can also be updated by conditional branches. For instance, if a SCALAR_VALUE is compared > 8, in the 'true' branch it will have a umin_value (unsigned minimum value) of 9, whereas in the 'false' @@ -1150,14 +1151,16 @@ BPF_JSGE) would instead update the signed minimum/maximum values. Information from the signed and unsigned bounds can be combined; for instance if a value is first tested < 8 and then tested s> 4, the verifier will conclude that the value is also > 4 and s< 8, since the bounds prevent crossing the sign boundary. + PTR_TO_PACKETs with a variable offset part have an 'id', which is common to all pointers sharing that same variable offset. This is important for packet range -checks: after adding some variable to a packet pointer, if you then copy it to -another register and (say) add a constant 4, both registers will share the same -'id' but one will have a fixed offset of +4. Then if it is bounds-checked and -found to be less than a PTR_TO_PACKET_END, the other register is now known to -have a safe range of at least 4 bytes. See 'Direct packet access', below, for -more on PTR_TO_PACKET ranges. +checks: after adding a variable to a packet pointer register A, if you then copy +it to another register B and then add a constant 4 to A, both registers will +share the same 'id' but the A will have a fixed offset of +4. Then if A is +bounds-checked and found to be less than a PTR_TO_PACKET_END, the register B is +now known to have a safe range of at least 4 bytes. See 'Direct packet access', +below, for more on PTR_TO_PACKET ranges. + The 'id' field is also used on PTR_TO_MAP_VALUE_OR_NULL, common to all copies of the pointer returned from a map lookup. This means that when one copy is checked and found to be non-NULL, all copies can become PTR_TO_MAP_VALUEs. -- 1.8.5.6.2.g3d8a54e.dirty
[PATCH v5] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning. We don't
treat overlaps partition as a error:
"
Caizhiyong <caizhiy...@hisilicon.com> said:
Partition overlap was intentionally designed in this cmdline partition.
reference http://lists.infradead.org/pipermail/linux-mtd/2013-August/048092.html
"
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes
v4-v5:
1:Delete unnecessary blank line at begin and end of logs.
2:Delete newline character in log, pr_* will add newline
automatically.
v3-v4:
1:Fix grammar typo, reported by Randy Dunlap.
2:Delete unnecessary type casting, reported by Caizhiyong.
3:Record the reason why we can't treat overlapping partitions
as errors into changelog.
v2-v3:
1:Fix log one pair of overlaps partitions twice in cmdline_parts_verifier.
2:Fix out of bound access in cmdline_parts_verifier.
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
Hi! Jens Axboe
What is your suggestion?
block/partitions/cmdline.c | 57 ++
1 file changed, 57 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..60fb3df 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,62 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("Overlapping partitions are used in command line partitions.");
+ pr_warn("Don't use filesystems on overlapping partitions:");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i;
+ bool header = true;
+
+ for (; slot < state->limit && state->parts[slot].has_info; slot++) {
+ for (i = slot+1; i < state->limit && state->parts[i].has_info;
+i++) {
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +149,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v5] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning. We don't
treat overlaps partition as a error:
"
Caizhiyong said:
Partition overlap was intentionally designed in this cmdline partition.
reference http://lists.infradead.org/pipermail/linux-mtd/2013-August/048092.html
"
Signed-off-by: Wang YanQing
---
Changes
v4-v5:
1:Delete unnecessary blank line at begin and end of logs.
2:Delete newline character in log, pr_* will add newline
automatically.
v3-v4:
1:Fix grammar typo, reported by Randy Dunlap.
2:Delete unnecessary type casting, reported by Caizhiyong.
3:Record the reason why we can't treat overlapping partitions
as errors into changelog.
v2-v3:
1:Fix log one pair of overlaps partitions twice in cmdline_parts_verifier.
2:Fix out of bound access in cmdline_parts_verifier.
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
Hi! Jens Axboe
What is your suggestion?
block/partitions/cmdline.c | 57 ++
1 file changed, 57 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..60fb3df 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,62 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("Overlapping partitions are used in command line partitions.");
+ pr_warn("Don't use filesystems on overlapping partitions:");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i;
+ bool header = true;
+
+ for (; slot < state->limit && state->parts[slot].has_info; slot++) {
+ for (i = slot+1; i < state->limit && state->parts[i].has_info;
+i++) {
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +149,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v4] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning. We don't
treat overlaps partition as a error:
"
Caizhiyong <caizhiy...@hisilicon.com> said:
Partition overlap was intentionally designed in this cmdline partition.
reference http://lists.infradead.org/pipermail/linux-mtd/2013-August/048092.html
"
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes
v3-v4:
1:Fix grammar typo, reported by Randy Dunlap.
2:Delete unnecessary type casting, reported by Caizhiyong.
3:Record the reason why we can't treat overlapping partitions
as errors into changelog.
v2-v3:
1:Fix log one pair of overlaps partitions twice in cmdline_parts_verifier.
2:Fix out of bound access in cmdline_parts_verifier.
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
block/partitions/cmdline.c | 66 ++
1 file changed, 66 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..2fd1b99 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,71 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("\n");
+ pr_warn("Overlapping partitions are used in command line
partitions.\n");
+ pr_warn("Don't use filesystems on overlapping partitions:\n");
+}
+
+static inline void overlaps_warns_tailer(void)
+{
+ pr_warn("\n");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i;
+ bool header = true;
+
+ for (; slot < state->limit && state->parts[slot].has_info; slot++) {
+ for (i = slot+1; i < state->limit && state->parts[i].has_info;
+i++) {
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].\n",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+
+ if (!header)
+ overlaps_warns_tailer();
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +158,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v4] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning. We don't
treat overlaps partition as a error:
"
Caizhiyong said:
Partition overlap was intentionally designed in this cmdline partition.
reference http://lists.infradead.org/pipermail/linux-mtd/2013-August/048092.html
"
Signed-off-by: Wang YanQing
---
Changes
v3-v4:
1:Fix grammar typo, reported by Randy Dunlap.
2:Delete unnecessary type casting, reported by Caizhiyong.
3:Record the reason why we can't treat overlapping partitions
as errors into changelog.
v2-v3:
1:Fix log one pair of overlaps partitions twice in cmdline_parts_verifier.
2:Fix out of bound access in cmdline_parts_verifier.
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
block/partitions/cmdline.c | 66 ++
1 file changed, 66 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..2fd1b99 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,71 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("\n");
+ pr_warn("Overlapping partitions are used in command line
partitions.\n");
+ pr_warn("Don't use filesystems on overlapping partitions:\n");
+}
+
+static inline void overlaps_warns_tailer(void)
+{
+ pr_warn("\n");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i;
+ bool header = true;
+
+ for (; slot < state->limit && state->parts[slot].has_info; slot++) {
+ for (i = slot+1; i < state->limit && state->parts[i].has_info;
+i++) {
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].\n",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+
+ if (!header)
+ overlaps_warns_tailer();
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +158,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH] bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform
On Sat, Apr 28, 2018 at 01:29:17PM +0800, Wang YanQing wrote:
> On Sat, Apr 28, 2018 at 01:33:15AM +0200, Daniel Borkmann wrote:
> > On 04/28/2018 12:48 AM, Alexei Starovoitov wrote:
> > > On Thu, Apr 26, 2018 at 05:57:49PM +0800, Wang YanQing wrote:
> > >> All the testcases for BPF_PROG_TYPE_PERF_EVENT program type in
> > >> test_verifier(kselftest) report below errors on x86_32:
> > >> "
> > >> 172/p unpriv: spill/fill of different pointers ldx FAIL
> > >> Unexpected error message!
> > >> 0: (bf) r6 = r10
> > >> 1: (07) r6 += -8
> > >> 2: (15) if r1 == 0x0 goto pc+3
> > >> R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1
> > >> 3: (bf) r2 = r10
> > >> 4: (07) r2 += -76
> > >> 5: (7b) *(u64 *)(r6 +0) = r2
> > >> 6: (55) if r1 != 0x0 goto pc+1
> > >> R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1
> > >> R10=fp0,call_-1 fp-8=fp
> > >> 7: (7b) *(u64 *)(r6 +0) = r1
> > >> 8: (79) r1 = *(u64 *)(r6 +0)
> > >> 9: (79) r1 = *(u64 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=8
> > >>
> > >> 378/p check bpf_perf_event_data->sample_period byte load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (71) r0 = *(u8 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=1
> > >>
> > >> 379/p check bpf_perf_event_data->sample_period half load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (69) r0 = *(u16 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=2
> > >>
> > >> 380/p check bpf_perf_event_data->sample_period word load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (61) r0 = *(u32 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=4
> > >>
> > >> 381/p check bpf_perf_event_data->sample_period dword load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (79) r0 = *(u64 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=8
> > >> "
> > >>
> > >> This patch fix it, the fix isn't only necessary for x86_32, it will fix
> > >> the
> > >> same problem for other platforms too, if their size of bpf_user_pt_regs_t
> > >> can't divide exactly into 8.
> > >>
> > >> Signed-off-by: Wang YanQing <udkni...@gmail.com>
> > >> ---
> > >> Hi all!
> > >> After mainline accept this patch, then we need to submit a sync patch
> > >> to update the tools/include/uapi/linux/bpf_perf_event.h.
> > >>
> > >> Thanks.
> > >>
> > >> include/uapi/linux/bpf_perf_event.h | 2 +-
> > >> 1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/include/uapi/linux/bpf_perf_event.h
> > >> b/include/uapi/linux/bpf_perf_event.h
> > >> index eb1b9d2..ff4c092 100644
> > >> --- a/include/uapi/linux/bpf_perf_event.h
> > >> +++ b/include/uapi/linux/bpf_perf_event.h
> > >> @@ -12,7 +12,7 @@
> > >>
> > >> struct bpf_perf_event_data {
> > >> bpf_user_pt_regs_t regs;
> > >> -__u64 sample_period;
> > >> +__u64 sample_period __attribute__((aligned(8)));
> > >
> > > I don't think this necessary.
> > > imo it's a bug in pe_prog_is_valid_access
> > > that should have allowed 8-byte access to 4-byte aligned sample_period.
> > > The access rewritten by pe_prog_convert_ctx_access anyway,
> > > no alignment issues as far as I can see.
> >
> > Right, good point. Wang, could you give the below a test run:
> >
> > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > index 56ba0f2..95b9142 100644
> > --- a/kernel/trace/bpf_trace.c
> > +++ b/kernel/trace/bpf_trace.c
> > @@ -833,8 +833,14 @@ static bool pe_prog_is_valid_access(int off, int size,
> > enum bpf_access_type type
> > return false;
> > if (type != BPF_READ)
> > return false;
> > - if (off % size != 0)
> > - return false;
> > + if (off % size != 0) {
> > + if (sizeof(long) != 4)
> > + return false;
> > + if (size != 8)
> > + return false;
> > + if (off % size != 4)
> > + return false;
> > + }
> >
> > switch (off) {
> > case bpf_ctx_range(struct bpf_perf_event_data, sample_period):
> Hi all!
>
> I have tested this patch, but test_verifier reports the same errors
> for the five testcases.
>
> The reason is they all failed to pass the test of bpf_ctx_narrow_access_ok.
>
> Thanks.
Hi! Daniel Borkmann.
Do you have any plan to fix bpf_ctx_narrow_access_ok for these problems?
Thanks.
Re: [PATCH] bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform
On Sat, Apr 28, 2018 at 01:29:17PM +0800, Wang YanQing wrote:
> On Sat, Apr 28, 2018 at 01:33:15AM +0200, Daniel Borkmann wrote:
> > On 04/28/2018 12:48 AM, Alexei Starovoitov wrote:
> > > On Thu, Apr 26, 2018 at 05:57:49PM +0800, Wang YanQing wrote:
> > >> All the testcases for BPF_PROG_TYPE_PERF_EVENT program type in
> > >> test_verifier(kselftest) report below errors on x86_32:
> > >> "
> > >> 172/p unpriv: spill/fill of different pointers ldx FAIL
> > >> Unexpected error message!
> > >> 0: (bf) r6 = r10
> > >> 1: (07) r6 += -8
> > >> 2: (15) if r1 == 0x0 goto pc+3
> > >> R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1
> > >> 3: (bf) r2 = r10
> > >> 4: (07) r2 += -76
> > >> 5: (7b) *(u64 *)(r6 +0) = r2
> > >> 6: (55) if r1 != 0x0 goto pc+1
> > >> R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1
> > >> R10=fp0,call_-1 fp-8=fp
> > >> 7: (7b) *(u64 *)(r6 +0) = r1
> > >> 8: (79) r1 = *(u64 *)(r6 +0)
> > >> 9: (79) r1 = *(u64 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=8
> > >>
> > >> 378/p check bpf_perf_event_data->sample_period byte load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (71) r0 = *(u8 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=1
> > >>
> > >> 379/p check bpf_perf_event_data->sample_period half load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (69) r0 = *(u16 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=2
> > >>
> > >> 380/p check bpf_perf_event_data->sample_period word load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (61) r0 = *(u32 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=4
> > >>
> > >> 381/p check bpf_perf_event_data->sample_period dword load permitted FAIL
> > >> Failed to load prog 'Permission denied'!
> > >> 0: (b7) r0 = 0
> > >> 1: (79) r0 = *(u64 *)(r1 +68)
> > >> invalid bpf_context access off=68 size=8
> > >> "
> > >>
> > >> This patch fix it, the fix isn't only necessary for x86_32, it will fix
> > >> the
> > >> same problem for other platforms too, if their size of bpf_user_pt_regs_t
> > >> can't divide exactly into 8.
> > >>
> > >> Signed-off-by: Wang YanQing
> > >> ---
> > >> Hi all!
> > >> After mainline accept this patch, then we need to submit a sync patch
> > >> to update the tools/include/uapi/linux/bpf_perf_event.h.
> > >>
> > >> Thanks.
> > >>
> > >> include/uapi/linux/bpf_perf_event.h | 2 +-
> > >> 1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/include/uapi/linux/bpf_perf_event.h
> > >> b/include/uapi/linux/bpf_perf_event.h
> > >> index eb1b9d2..ff4c092 100644
> > >> --- a/include/uapi/linux/bpf_perf_event.h
> > >> +++ b/include/uapi/linux/bpf_perf_event.h
> > >> @@ -12,7 +12,7 @@
> > >>
> > >> struct bpf_perf_event_data {
> > >> bpf_user_pt_regs_t regs;
> > >> -__u64 sample_period;
> > >> +__u64 sample_period __attribute__((aligned(8)));
> > >
> > > I don't think this necessary.
> > > imo it's a bug in pe_prog_is_valid_access
> > > that should have allowed 8-byte access to 4-byte aligned sample_period.
> > > The access rewritten by pe_prog_convert_ctx_access anyway,
> > > no alignment issues as far as I can see.
> >
> > Right, good point. Wang, could you give the below a test run:
> >
> > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > index 56ba0f2..95b9142 100644
> > --- a/kernel/trace/bpf_trace.c
> > +++ b/kernel/trace/bpf_trace.c
> > @@ -833,8 +833,14 @@ static bool pe_prog_is_valid_access(int off, int size,
> > enum bpf_access_type type
> > return false;
> > if (type != BPF_READ)
> > return false;
> > - if (off % size != 0)
> > - return false;
> > + if (off % size != 0) {
> > + if (sizeof(long) != 4)
> > + return false;
> > + if (size != 8)
> > + return false;
> > + if (off % size != 4)
> > + return false;
> > + }
> >
> > switch (off) {
> > case bpf_ctx_range(struct bpf_perf_event_data, sample_period):
> Hi all!
>
> I have tested this patch, but test_verifier reports the same errors
> for the five testcases.
>
> The reason is they all failed to pass the test of bpf_ctx_narrow_access_ok.
>
> Thanks.
Hi! Daniel Borkmann.
Do you have any plan to fix bpf_ctx_narrow_access_ok for these problems?
Thanks.
[PATCH v3] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes
v2-v3:
1:Fix log one pair of overlaps partitions twice in cmdline_parts_verifier.
2:Fix out of bound access in cmdline_parts_verifier.
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
block/partitions/cmdline.c | 66 ++
1 file changed, 66 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..fefcc72 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,71 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("\n");
+ pr_warn("Overlaps partitions being used in command line partition.\n");
+ pr_warn("Don't use filesystems on overlaps partitions:\n");
+}
+
+static inline void overlaps_warns_tailer(void)
+{
+ pr_warn("\n");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i;
+ bool header = true;
+
+ for (; slot < state->limit && state->parts[slot].has_info; slot++) {
+ for (i = slot+1; i < state->limit && state->parts[i].has_info;
+i++) {
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].\n",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+
+ if (!header)
+ overlaps_warns_tailer();
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +158,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, (void *)state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v3] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning.
Signed-off-by: Wang YanQing
---
Changes
v2-v3:
1:Fix log one pair of overlaps partitions twice in cmdline_parts_verifier.
2:Fix out of bound access in cmdline_parts_verifier.
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
block/partitions/cmdline.c | 66 ++
1 file changed, 66 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..fefcc72 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,71 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("\n");
+ pr_warn("Overlaps partitions being used in command line partition.\n");
+ pr_warn("Don't use filesystems on overlaps partitions:\n");
+}
+
+static inline void overlaps_warns_tailer(void)
+{
+ pr_warn("\n");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i;
+ bool header = true;
+
+ for (; slot < state->limit && state->parts[slot].has_info; slot++) {
+ for (i = slot+1; i < state->limit && state->parts[i].has_info;
+i++) {
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].\n",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+
+ if (!header)
+ overlaps_warns_tailer();
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +158,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, (void *)state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning. We don't
treat overlaps partition as a error:
"
Caizhiyong <caizhiy...@hisilicon.com> said:
Partition overlap was intentionally designed in this cmdline partition.
reference http://lists.infradead.org/pipermail/linux-mtd/2013-August/048092.html
"
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
block/partitions/cmdline.c | 68 ++
1 file changed, 68 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..7d70046 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,73 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("\n");
+ pr_warn("Overlaps partitions being used in command line partition.\n");
+ pr_warn("Don't use filesystems on overlaps partitions:\n");
+}
+
+static inline void overlaps_warns_tailer(void)
+{
+ pr_warn("\n");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i, j = slot;
+ bool header = true;
+
+ for (; state->parts[slot].has_info; slot++) {
+ for (i = j; state->parts[i].has_info; i++) {
+ if (i == slot)
+ continue;
+
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].\n",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+
+ if (!header)
+ overlaps_warns_tailer();
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +160,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, (void *)state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will log a warning. We don't
treat overlaps partition as a error:
"
Caizhiyong said:
Partition overlap was intentionally designed in this cmdline partition.
reference http://lists.infradead.org/pipermail/linux-mtd/2013-August/048092.html
"
Signed-off-by: Wang YanQing
---
Changes
v1-v2:
1:Don't treat overlaps partition as a error, but log a warning.
block/partitions/cmdline.c | 68 ++
1 file changed, 68 insertions(+)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..7d70046 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,73 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static inline void overlaps_warns_header(void)
+{
+ pr_warn("\n");
+ pr_warn("Overlaps partitions being used in command line partition.\n");
+ pr_warn("Don't use filesystems on overlaps partitions:\n");
+}
+
+static inline void overlaps_warns_tailer(void)
+{
+ pr_warn("\n");
+}
+
+static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i, j = slot;
+ bool header = true;
+
+ for (; state->parts[slot].has_info; slot++) {
+ for (i = j; state->parts[i].has_info; i++) {
+ if (i == slot)
+ continue;
+
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ if (header) {
+ header = false;
+ overlaps_warns_header();
+ }
+ pr_warn("%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu].\n",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ }
+ }
+ }
+
+ if (!header)
+ overlaps_warns_tailer();
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -93,6 +160,7 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ cmdline_parts_verifier(1, (void *)state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will return error and log
a error message.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
block/partitions/cmdline.c | 63 +-
1 file changed, 62 insertions(+), 1 deletion(-)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..6754c2a 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,62 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static bool cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i, j = slot;
+ bool ret = true;
+
+ for (; state->parts[slot].has_info; slot++) {
+ for (i = j; state->parts[i].has_info; i++) {
+ if (i == slot)
+ continue;
+
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ pr_err("Invalid cmdline partition:"
+ "%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu]",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ ret = false;
+ break;
+ }
+ }
+
+ if (!ret)
+ break;
+ }
+
+ return ret;
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -70,6 +126,7 @@ int cmdline_partition(struct parsed_partitions *state)
sector_t disk_size;
char bdev[BDEVNAME_SIZE];
struct cmdline_parts *parts;
+ int ret;
if (cmdline) {
if (bdev_parts)
@@ -93,8 +150,12 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ if (!cmdline_parts_verifier(1, (void *)state))
+ ret = -1;
+ else
+ ret = 1;
strlcat(state->pp_buf, "\n", PAGE_SIZE);
- return 1;
+ return ret;
}
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] block: add verifier for cmdline partition
I meet strange filesystem corruption issue recently, the reason
is there are overlaps partitions in cmdline partition argument.
This patch add verifier for cmdline partition, then if there are
overlaps partitions, cmdline_partition will return error and log
a error message.
Signed-off-by: Wang YanQing
---
block/partitions/cmdline.c | 63 +-
1 file changed, 62 insertions(+), 1 deletion(-)
diff --git a/block/partitions/cmdline.c b/block/partitions/cmdline.c
index e333583..6754c2a 100644
--- a/block/partitions/cmdline.c
+++ b/block/partitions/cmdline.c
@@ -58,6 +58,62 @@ static int __init cmdline_parts_setup(char *s)
}
__setup("blkdevparts=", cmdline_parts_setup);
+static bool has_overlaps(sector_t from, sector_t size,
+sector_t from2, sector_t size2)
+{
+ sector_t end = from + size;
+ sector_t end2 = from2 + size2;
+
+ if (from >= from2 && from < end2)
+ return true;
+
+ if (end > from2 && end <= end2)
+ return true;
+
+ if (from2 >= from && from2 < end)
+ return true;
+
+ if (end2 > from && end2 <= end)
+ return true;
+
+ return false;
+}
+
+static bool cmdline_parts_verifier(int slot, struct parsed_partitions *state)
+{
+ int i, j = slot;
+ bool ret = true;
+
+ for (; state->parts[slot].has_info; slot++) {
+ for (i = j; state->parts[i].has_info; i++) {
+ if (i == slot)
+ continue;
+
+ if (has_overlaps(state->parts[slot].from,
+state->parts[slot].size,
+state->parts[i].from,
+state->parts[i].size)) {
+ pr_err("Invalid cmdline partition:"
+ "%s[%llu,%llu] overlaps with "
+ "%s[%llu,%llu]",
+ state->parts[slot].info.volname,
+ (u64)state->parts[slot].from << 9,
+ (u64)state->parts[slot].size << 9,
+ state->parts[i].info.volname,
+ (u64)state->parts[i].from << 9,
+ (u64)state->parts[i].size << 9);
+ ret = false;
+ break;
+ }
+ }
+
+ if (!ret)
+ break;
+ }
+
+ return ret;
+}
+
/*
* Purpose: allocate cmdline partitions.
* Returns:
@@ -70,6 +126,7 @@ int cmdline_partition(struct parsed_partitions *state)
sector_t disk_size;
char bdev[BDEVNAME_SIZE];
struct cmdline_parts *parts;
+ int ret;
if (cmdline) {
if (bdev_parts)
@@ -93,8 +150,12 @@ int cmdline_partition(struct parsed_partitions *state)
disk_size = get_capacity(state->bdev->bd_disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
+ if (!cmdline_parts_verifier(1, (void *)state))
+ ret = -1;
+ else
+ ret = 1;
strlcat(state->pp_buf, "\n", PAGE_SIZE);
- return 1;
+ return ret;
}
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v6] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF
only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment.
IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use
EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF
ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others
eBPF registers, R0-R10, are simulated through scratch space on stack.
The reasons behind the hardware registers allocation policy are:
1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit
for general eBPF 64bit register simulation.
2:We need at least 4 registers to simulate most eBPF ISA operations
on registers operands instead of on register operands.
3:We need to put BPF_REG_AX on hardware registers, or constant blinding
will degrade jit performance heavily.
Tested on PC (Intel(R) Core(TM) i5-5200U CPU).
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 83 PASSED, 0 FAILED.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 828 PASSED, 0 FAILED.
Above tests are all done in following two conditions separately:
1:bpf_jit_enable=1 and bpf_jit_harden=0
2:bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs in kselftest 100 times continuously for every condition,
the numbers are in format: total/times=avg.
The numbers that test_bpf reports show almost the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10674/100=106
test_pkt_access:PASS:ipv6:9130/100=91
test_pkt_access:PASS:ipv6:4855/100=48
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685
test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373
test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550
c:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:10558/100=105
test_pkt_access:PASS:ipv6:5092/100=50
test_xdp:PASS:ipv4:131902/100=1319
test_xdp:PASS:ipv6:77932/100=779
test_l4lb:PASS:ipv4:38924/100=389
test_l4lb:PASS:ipv6:57520/100=575
The numbers show we get 30%~50% improvement.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes v5-v6:
1:Add do {} while (0) to RETPOLINE_RAX_BPF_JIT for
consistence reason.
2:Clean up non-standard comments, reported by Daniel Borkmann.
3:Fix a memory leak issue, repoted by Daniel Borkmann.
Changes v4-v5:
1:Delete is_on_stack, BPF_REG_AX is the only one
on real hardware registers, so just check with
it.
2:Apply commit 1612a981b766 ("bpf, x64: fix JIT emission
for dead code"), suggested by Daniel Borkmann.
Changes v3-v4:
1:Fix changelog in commit.
I install llvm-6.0, then test_progs willn't report errors.
I submit another patch:
"bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on
x86_32 platform"
to fix another problem, after that patch, test_verifier willn't report
errors too.
2:Fix clear r0[1] twice unnecessarily in *BPF_IND|BPF_ABS* simulation.
Changes v2-v3:
1:Move BPF_REG_AX to real hardware registers for performance reason.
3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann.
4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann.
5:Some bug fixes and comments improvement.
Changes v1-v2:
1:Fix bug in emit_ia32_neg64.
2:Fix bug in emit_ia32_arsh_r64.
3:Delete filename in top level comment, suggested by Thomas Gleixner.
4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner.
5:Rewrite some words in changelog.
6:CodingSytle improvement and a little more comments.
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 30 +-
arch/x86/net/Makefile|9 +-
arch/x86/net/bpf_jit_comp32.c| 2553 ++
4 files changed, 2588 insertions(+), 6 deletions(-)
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index c07f492..d51a71d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -138,7 +138,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
[PATCH v6] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF
only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment.
IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use
EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF
ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others
eBPF registers, R0-R10, are simulated through scratch space on stack.
The reasons behind the hardware registers allocation policy are:
1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit
for general eBPF 64bit register simulation.
2:We need at least 4 registers to simulate most eBPF ISA operations
on registers operands instead of on register operands.
3:We need to put BPF_REG_AX on hardware registers, or constant blinding
will degrade jit performance heavily.
Tested on PC (Intel(R) Core(TM) i5-5200U CPU).
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 83 PASSED, 0 FAILED.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 828 PASSED, 0 FAILED.
Above tests are all done in following two conditions separately:
1:bpf_jit_enable=1 and bpf_jit_harden=0
2:bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs in kselftest 100 times continuously for every condition,
the numbers are in format: total/times=avg.
The numbers that test_bpf reports show almost the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10674/100=106
test_pkt_access:PASS:ipv6:9130/100=91
test_pkt_access:PASS:ipv6:4855/100=48
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685
test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373
test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550
c:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:10558/100=105
test_pkt_access:PASS:ipv6:5092/100=50
test_xdp:PASS:ipv4:131902/100=1319
test_xdp:PASS:ipv6:77932/100=779
test_l4lb:PASS:ipv4:38924/100=389
test_l4lb:PASS:ipv6:57520/100=575
The numbers show we get 30%~50% improvement.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing
---
Changes v5-v6:
1:Add do {} while (0) to RETPOLINE_RAX_BPF_JIT for
consistence reason.
2:Clean up non-standard comments, reported by Daniel Borkmann.
3:Fix a memory leak issue, repoted by Daniel Borkmann.
Changes v4-v5:
1:Delete is_on_stack, BPF_REG_AX is the only one
on real hardware registers, so just check with
it.
2:Apply commit 1612a981b766 ("bpf, x64: fix JIT emission
for dead code"), suggested by Daniel Borkmann.
Changes v3-v4:
1:Fix changelog in commit.
I install llvm-6.0, then test_progs willn't report errors.
I submit another patch:
"bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on
x86_32 platform"
to fix another problem, after that patch, test_verifier willn't report
errors too.
2:Fix clear r0[1] twice unnecessarily in *BPF_IND|BPF_ABS* simulation.
Changes v2-v3:
1:Move BPF_REG_AX to real hardware registers for performance reason.
3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann.
4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann.
5:Some bug fixes and comments improvement.
Changes v1-v2:
1:Fix bug in emit_ia32_neg64.
2:Fix bug in emit_ia32_arsh_r64.
3:Delete filename in top level comment, suggested by Thomas Gleixner.
4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner.
5:Rewrite some words in changelog.
6:CodingSytle improvement and a little more comments.
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 30 +-
arch/x86/net/Makefile|9 +-
arch/x86/net/bpf_jit_comp32.c| 2553 ++
4 files changed, 2588 insertions(+), 6 deletions(-)
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index c07f492..d51a71d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -138,7 +138,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_EXIT_T
[PATCH v5] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF
only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment.
IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use
EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF
ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others
eBPF registers, R0-R10, are simulated through scratch space on stack.
The reasons behind the hardware registers allocation policy are:
1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit
for general eBPF 64bit register simulation.
2:We need at least 4 registers to simulate most eBPF ISA operations
on registers operands instead of on register operands.
3:We need to put BPF_REG_AX on hardware registers, or constant blinding
will degrade jit performance heavily.
Tested on PC (Intel(R) Core(TM) i5-5200U CPU).
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 83 PASSED, 0 FAILED.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 828 PASSED, 0 FAILED.
Above tests are all done in following two conditions separately:
1:bpf_jit_enable=1 and bpf_jit_harden=0
2:bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs in kselftest 100 times continuously for every condition,
the numbers are in format: total/times=avg.
The numbers that test_bpf reports show almost the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10674/100=106
test_pkt_access:PASS:ipv6:9130/100=91
test_pkt_access:PASS:ipv6:4855/100=48
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685
test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373
test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550
c:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:10558/100=105
test_pkt_access:PASS:ipv6:5092/100=50
test_xdp:PASS:ipv4:131902/100=1319
test_xdp:PASS:ipv6:77932/100=779
test_l4lb:PASS:ipv4:38924/100=389
test_l4lb:PASS:ipv6:57520/100=575
The numbers show we get 30%~50% improvement.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes v4-v5:
1:Delete is_on_stack, BPF_REG_AX is the only one
on real hardware registers, so just check with
it.
2:Apply commit 1612a981b766 ("bpf, x64: fix JIT emission
for dead code"), suggested by Daniel Borkmann.
Changes v3-v4:
1:Fix changelog in commit.
I install llvm-6.0, then test_progs willn't report errors.
I submit another patch:
"bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on
x86_32 platform"
to fix another problem, after that patch, test_verifier willn't report
errors too.
2:Fix clear r0[1] twice unnecessarily in *BPF_IND|BPF_ABS* simulation.
Changes v2-v3:
1:Move BPF_REG_AX to real hardware registers for performance reason.
3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann.
4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann.
5:Some bug fixes and comments improvement.
Changes v1-v2:
1:Fix bug in emit_ia32_neg64.
2:Fix bug in emit_ia32_arsh_r64.
3:Delete filename in top level comment, suggested by Thomas Gleixner.
4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner.
5:Rewrite some words in changelog.
6:CodingSytle improvement and a little more comments.
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 26 +-
arch/x86/net/Makefile|9 +-
arch/x86/net/bpf_jit_comp32.c| 2527 ++
4 files changed, 2559 insertions(+), 5 deletions(-)
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 00fcf81..1f5fa2f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -137,7 +137,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_EXIT_THREAD
select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE
diff --git a/arch/x86/include/asm/nospec-branch.h
b/arch/x86/include/asm/nospec-branch.h
index f928ad9..a4c
[PATCH v5] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF
only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment.
IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use
EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF
ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others
eBPF registers, R0-R10, are simulated through scratch space on stack.
The reasons behind the hardware registers allocation policy are:
1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit
for general eBPF 64bit register simulation.
2:We need at least 4 registers to simulate most eBPF ISA operations
on registers operands instead of on register operands.
3:We need to put BPF_REG_AX on hardware registers, or constant blinding
will degrade jit performance heavily.
Tested on PC (Intel(R) Core(TM) i5-5200U CPU).
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 83 PASSED, 0 FAILED.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 828 PASSED, 0 FAILED.
Above tests are all done in following two conditions separately:
1:bpf_jit_enable=1 and bpf_jit_harden=0
2:bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs in kselftest 100 times continuously for every condition,
the numbers are in format: total/times=avg.
The numbers that test_bpf reports show almost the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10674/100=106
test_pkt_access:PASS:ipv6:9130/100=91
test_pkt_access:PASS:ipv6:4855/100=48
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685
test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373
test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550
c:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:10558/100=105
test_pkt_access:PASS:ipv6:5092/100=50
test_xdp:PASS:ipv4:131902/100=1319
test_xdp:PASS:ipv6:77932/100=779
test_l4lb:PASS:ipv4:38924/100=389
test_l4lb:PASS:ipv6:57520/100=575
The numbers show we get 30%~50% improvement.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing
---
Changes v4-v5:
1:Delete is_on_stack, BPF_REG_AX is the only one
on real hardware registers, so just check with
it.
2:Apply commit 1612a981b766 ("bpf, x64: fix JIT emission
for dead code"), suggested by Daniel Borkmann.
Changes v3-v4:
1:Fix changelog in commit.
I install llvm-6.0, then test_progs willn't report errors.
I submit another patch:
"bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on
x86_32 platform"
to fix another problem, after that patch, test_verifier willn't report
errors too.
2:Fix clear r0[1] twice unnecessarily in *BPF_IND|BPF_ABS* simulation.
Changes v2-v3:
1:Move BPF_REG_AX to real hardware registers for performance reason.
3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann.
4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann.
5:Some bug fixes and comments improvement.
Changes v1-v2:
1:Fix bug in emit_ia32_neg64.
2:Fix bug in emit_ia32_arsh_r64.
3:Delete filename in top level comment, suggested by Thomas Gleixner.
4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner.
5:Rewrite some words in changelog.
6:CodingSytle improvement and a little more comments.
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 26 +-
arch/x86/net/Makefile|9 +-
arch/x86/net/bpf_jit_comp32.c| 2527 ++
4 files changed, 2559 insertions(+), 5 deletions(-)
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 00fcf81..1f5fa2f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -137,7 +137,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_EXIT_THREAD
select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE
diff --git a/arch/x86/include/asm/nospec-branch.h
b/arch/x86/include/asm/nospec-branch.h
index f928ad9..a4c7ca4 100644
--- a/arch/x8
Re: [PATCH] bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform
On Sat, Apr 28, 2018 at 01:33:15AM +0200, Daniel Borkmann wrote:
> On 04/28/2018 12:48 AM, Alexei Starovoitov wrote:
> > On Thu, Apr 26, 2018 at 05:57:49PM +0800, Wang YanQing wrote:
> >> All the testcases for BPF_PROG_TYPE_PERF_EVENT program type in
> >> test_verifier(kselftest) report below errors on x86_32:
> >> "
> >> 172/p unpriv: spill/fill of different pointers ldx FAIL
> >> Unexpected error message!
> >> 0: (bf) r6 = r10
> >> 1: (07) r6 += -8
> >> 2: (15) if r1 == 0x0 goto pc+3
> >> R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1
> >> 3: (bf) r2 = r10
> >> 4: (07) r2 += -76
> >> 5: (7b) *(u64 *)(r6 +0) = r2
> >> 6: (55) if r1 != 0x0 goto pc+1
> >> R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1 R10=fp0,call_-1
> >> fp-8=fp
> >> 7: (7b) *(u64 *)(r6 +0) = r1
> >> 8: (79) r1 = *(u64 *)(r6 +0)
> >> 9: (79) r1 = *(u64 *)(r1 +68)
> >> invalid bpf_context access off=68 size=8
> >>
> >> 378/p check bpf_perf_event_data->sample_period byte load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (71) r0 = *(u8 *)(r1 +68)
> >> invalid bpf_context access off=68 size=1
> >>
> >> 379/p check bpf_perf_event_data->sample_period half load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (69) r0 = *(u16 *)(r1 +68)
> >> invalid bpf_context access off=68 size=2
> >>
> >> 380/p check bpf_perf_event_data->sample_period word load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (61) r0 = *(u32 *)(r1 +68)
> >> invalid bpf_context access off=68 size=4
> >>
> >> 381/p check bpf_perf_event_data->sample_period dword load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (79) r0 = *(u64 *)(r1 +68)
> >> invalid bpf_context access off=68 size=8
> >> "
> >>
> >> This patch fix it, the fix isn't only necessary for x86_32, it will fix the
> >> same problem for other platforms too, if their size of bpf_user_pt_regs_t
> >> can't divide exactly into 8.
> >>
> >> Signed-off-by: Wang YanQing <udkni...@gmail.com>
> >> ---
> >> Hi all!
> >> After mainline accept this patch, then we need to submit a sync patch
> >> to update the tools/include/uapi/linux/bpf_perf_event.h.
> >>
> >> Thanks.
> >>
> >> include/uapi/linux/bpf_perf_event.h | 2 +-
> >> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/include/uapi/linux/bpf_perf_event.h
> >> b/include/uapi/linux/bpf_perf_event.h
> >> index eb1b9d2..ff4c092 100644
> >> --- a/include/uapi/linux/bpf_perf_event.h
> >> +++ b/include/uapi/linux/bpf_perf_event.h
> >> @@ -12,7 +12,7 @@
> >>
> >> struct bpf_perf_event_data {
> >>bpf_user_pt_regs_t regs;
> >> - __u64 sample_period;
> >> + __u64 sample_period __attribute__((aligned(8)));
> >
> > I don't think this necessary.
> > imo it's a bug in pe_prog_is_valid_access
> > that should have allowed 8-byte access to 4-byte aligned sample_period.
> > The access rewritten by pe_prog_convert_ctx_access anyway,
> > no alignment issues as far as I can see.
>
> Right, good point. Wang, could you give the below a test run:
>
> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index 56ba0f2..95b9142 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -833,8 +833,14 @@ static bool pe_prog_is_valid_access(int off, int size,
> enum bpf_access_type type
> return false;
> if (type != BPF_READ)
> return false;
> - if (off % size != 0)
> - return false;
> + if (off % size != 0) {
> + if (sizeof(long) != 4)
> + return false;
> + if (size != 8)
> + return false;
> + if (off % size != 4)
> + return false;
> + }
>
> switch (off) {
> case bpf_ctx_range(struct bpf_perf_event_data, sample_period):
Hi all!
I have tested this patch, but test_verifier reports the same errors
for the five testcases.
The reason is they all failed to pass the test of bpf_ctx_narrow_access_ok.
Thanks.
Re: [PATCH] bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform
On Sat, Apr 28, 2018 at 01:33:15AM +0200, Daniel Borkmann wrote:
> On 04/28/2018 12:48 AM, Alexei Starovoitov wrote:
> > On Thu, Apr 26, 2018 at 05:57:49PM +0800, Wang YanQing wrote:
> >> All the testcases for BPF_PROG_TYPE_PERF_EVENT program type in
> >> test_verifier(kselftest) report below errors on x86_32:
> >> "
> >> 172/p unpriv: spill/fill of different pointers ldx FAIL
> >> Unexpected error message!
> >> 0: (bf) r6 = r10
> >> 1: (07) r6 += -8
> >> 2: (15) if r1 == 0x0 goto pc+3
> >> R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1
> >> 3: (bf) r2 = r10
> >> 4: (07) r2 += -76
> >> 5: (7b) *(u64 *)(r6 +0) = r2
> >> 6: (55) if r1 != 0x0 goto pc+1
> >> R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1 R10=fp0,call_-1
> >> fp-8=fp
> >> 7: (7b) *(u64 *)(r6 +0) = r1
> >> 8: (79) r1 = *(u64 *)(r6 +0)
> >> 9: (79) r1 = *(u64 *)(r1 +68)
> >> invalid bpf_context access off=68 size=8
> >>
> >> 378/p check bpf_perf_event_data->sample_period byte load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (71) r0 = *(u8 *)(r1 +68)
> >> invalid bpf_context access off=68 size=1
> >>
> >> 379/p check bpf_perf_event_data->sample_period half load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (69) r0 = *(u16 *)(r1 +68)
> >> invalid bpf_context access off=68 size=2
> >>
> >> 380/p check bpf_perf_event_data->sample_period word load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (61) r0 = *(u32 *)(r1 +68)
> >> invalid bpf_context access off=68 size=4
> >>
> >> 381/p check bpf_perf_event_data->sample_period dword load permitted FAIL
> >> Failed to load prog 'Permission denied'!
> >> 0: (b7) r0 = 0
> >> 1: (79) r0 = *(u64 *)(r1 +68)
> >> invalid bpf_context access off=68 size=8
> >> "
> >>
> >> This patch fix it, the fix isn't only necessary for x86_32, it will fix the
> >> same problem for other platforms too, if their size of bpf_user_pt_regs_t
> >> can't divide exactly into 8.
> >>
> >> Signed-off-by: Wang YanQing
> >> ---
> >> Hi all!
> >> After mainline accept this patch, then we need to submit a sync patch
> >> to update the tools/include/uapi/linux/bpf_perf_event.h.
> >>
> >> Thanks.
> >>
> >> include/uapi/linux/bpf_perf_event.h | 2 +-
> >> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/include/uapi/linux/bpf_perf_event.h
> >> b/include/uapi/linux/bpf_perf_event.h
> >> index eb1b9d2..ff4c092 100644
> >> --- a/include/uapi/linux/bpf_perf_event.h
> >> +++ b/include/uapi/linux/bpf_perf_event.h
> >> @@ -12,7 +12,7 @@
> >>
> >> struct bpf_perf_event_data {
> >>bpf_user_pt_regs_t regs;
> >> - __u64 sample_period;
> >> + __u64 sample_period __attribute__((aligned(8)));
> >
> > I don't think this necessary.
> > imo it's a bug in pe_prog_is_valid_access
> > that should have allowed 8-byte access to 4-byte aligned sample_period.
> > The access rewritten by pe_prog_convert_ctx_access anyway,
> > no alignment issues as far as I can see.
>
> Right, good point. Wang, could you give the below a test run:
>
> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index 56ba0f2..95b9142 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -833,8 +833,14 @@ static bool pe_prog_is_valid_access(int off, int size,
> enum bpf_access_type type
> return false;
> if (type != BPF_READ)
> return false;
> - if (off % size != 0)
> - return false;
> + if (off % size != 0) {
> + if (sizeof(long) != 4)
> + return false;
> + if (size != 8)
> + return false;
> + if (off % size != 4)
> + return false;
> + }
>
> switch (off) {
> case bpf_ctx_range(struct bpf_perf_event_data, sample_period):
Hi all!
I have tested this patch, but test_verifier reports the same errors
for the five testcases.
The reason is they all failed to pass the test of bpf_ctx_narrow_access_ok.
Thanks.
[PATCH v4] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF only. Classic BPF is supported because of the conversion by BPF core. Almost all instructions from eBPF ISA supported except the following: BPF_ALU64 | BPF_DIV | BPF_K BPF_ALU64 | BPF_DIV | BPF_X BPF_ALU64 | BPF_MOD | BPF_K BPF_ALU64 | BPF_MOD | BPF_X BPF_STX | BPF_XADD | BPF_W BPF_STX | BPF_XADD | BPF_DW It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment. IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others eBPF registers, R0-R10, are simulated through scratch space on stack. The reasons behind the hardware registers allocation policy are: 1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit for general eBPF 64bit register simulation. 2:We need at least 4 registers to simulate most eBPF ISA operations on registers operands instead of on register operands. 3:We need to put BPF_REG_AX on hardware registers, or constant blinding will degrade jit performance heavily. Tested on PC (Intel(R) Core(TM) i5-5200U CPU). Testing results on i5-5200U: 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed] 2) test_progs: Summary: 83 PASSED, 0 FAILED. 3) test_lpm: OK 4) test_lru_map: OK 5) test_verifier: Summary: 828 PASSED, 0 FAILED. Above tests are all done in following two conditions separately: 1:bpf_jit_enable=1 and bpf_jit_harden=0 2:bpf_jit_enable=1 and bpf_jit_harden=2 Below are some numbers for this jit implementation: Note: I run test_progs in kselftest 100 times continuously for every condition, the numbers are in format: total/times=avg. The numbers that test_bpf reports show almost the same relation. a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0 test_pkt_access:PASS:ipv4:15622/100=156 test_pkt_access:PASS:ipv4:10674/100=106 test_pkt_access:PASS:ipv6:9130/100=91 test_pkt_access:PASS:ipv6:4855/100=48 test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389 test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685 test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373 test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550 c:jit_enable=1 and jit_harden=2 test_pkt_access:PASS:ipv4:10558/100=105 test_pkt_access:PASS:ipv6:5092/100=50 test_xdp:PASS:ipv4:131902/100=1319 test_xdp:PASS:ipv6:77932/100=779 test_l4lb:PASS:ipv4:38924/100=389 test_l4lb:PASS:ipv6:57520/100=575 The numbers show we get 30%~50% improvement. See Documentation/networking/filter.txt for more information. Signed-off-by: Wang YanQing <udkni...@gmail.com> --- Changes v3-v4: 1:Fix changelog in commit. I install llvm-6.0, then test_progs willn't report errors. I submit another patch: "bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform" to fix another problem, after that patch, test_verifier willn't report errors too. 2:Fix clear r0[1] twice unnecessarily in *BPF_IND|BPF_ABS* simulation. Changes v2-v3: 1:Move BPF_REG_AX to real hardware registers for performance reason. 3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann. 4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann. 5:Some bug fixes and comments improvement. Changes v1-v2: 1:Fix bug in emit_ia32_neg64. 2:Fix bug in emit_ia32_arsh_r64. 3:Delete filename in top level comment, suggested by Thomas Gleixner. 4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner. 5:Rewrite some words in changelog. 6:CodingSytle improvement and a little more comments. Hi all! This is the fourth version of this patch, and I think it is good enough to launch, any more suggestion? Thanks. arch/x86/Kconfig |2 +- arch/x86/include/asm/nospec-branch.h | 26 +- arch/x86/net/Makefile|9 +- arch/x86/net/bpf_jit_comp32.c| 2530 ++ 4 files changed, 2562 insertions(+), 5 deletions(-) create mode 100644 arch/x86/net/bpf_jit_comp32.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 00fcf81..1f5fa2f 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -137,7 +137,7 @@ config X86 select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS - select HAVE_EBPF_JITif X86_64 + select HAVE_EBPF_JIT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_EXIT_THREAD select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index f928ad9..a4c7ca4 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -291,14 +291,17
[PATCH v4] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF only. Classic BPF is supported because of the conversion by BPF core. Almost all instructions from eBPF ISA supported except the following: BPF_ALU64 | BPF_DIV | BPF_K BPF_ALU64 | BPF_DIV | BPF_X BPF_ALU64 | BPF_MOD | BPF_K BPF_ALU64 | BPF_MOD | BPF_X BPF_STX | BPF_XADD | BPF_W BPF_STX | BPF_XADD | BPF_DW It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment. IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others eBPF registers, R0-R10, are simulated through scratch space on stack. The reasons behind the hardware registers allocation policy are: 1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit for general eBPF 64bit register simulation. 2:We need at least 4 registers to simulate most eBPF ISA operations on registers operands instead of on register operands. 3:We need to put BPF_REG_AX on hardware registers, or constant blinding will degrade jit performance heavily. Tested on PC (Intel(R) Core(TM) i5-5200U CPU). Testing results on i5-5200U: 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed] 2) test_progs: Summary: 83 PASSED, 0 FAILED. 3) test_lpm: OK 4) test_lru_map: OK 5) test_verifier: Summary: 828 PASSED, 0 FAILED. Above tests are all done in following two conditions separately: 1:bpf_jit_enable=1 and bpf_jit_harden=0 2:bpf_jit_enable=1 and bpf_jit_harden=2 Below are some numbers for this jit implementation: Note: I run test_progs in kselftest 100 times continuously for every condition, the numbers are in format: total/times=avg. The numbers that test_bpf reports show almost the same relation. a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0 test_pkt_access:PASS:ipv4:15622/100=156 test_pkt_access:PASS:ipv4:10674/100=106 test_pkt_access:PASS:ipv6:9130/100=91 test_pkt_access:PASS:ipv6:4855/100=48 test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389 test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685 test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373 test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550 c:jit_enable=1 and jit_harden=2 test_pkt_access:PASS:ipv4:10558/100=105 test_pkt_access:PASS:ipv6:5092/100=50 test_xdp:PASS:ipv4:131902/100=1319 test_xdp:PASS:ipv6:77932/100=779 test_l4lb:PASS:ipv4:38924/100=389 test_l4lb:PASS:ipv6:57520/100=575 The numbers show we get 30%~50% improvement. See Documentation/networking/filter.txt for more information. Signed-off-by: Wang YanQing --- Changes v3-v4: 1:Fix changelog in commit. I install llvm-6.0, then test_progs willn't report errors. I submit another patch: "bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform" to fix another problem, after that patch, test_verifier willn't report errors too. 2:Fix clear r0[1] twice unnecessarily in *BPF_IND|BPF_ABS* simulation. Changes v2-v3: 1:Move BPF_REG_AX to real hardware registers for performance reason. 3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann. 4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann. 5:Some bug fixes and comments improvement. Changes v1-v2: 1:Fix bug in emit_ia32_neg64. 2:Fix bug in emit_ia32_arsh_r64. 3:Delete filename in top level comment, suggested by Thomas Gleixner. 4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner. 5:Rewrite some words in changelog. 6:CodingSytle improvement and a little more comments. Hi all! This is the fourth version of this patch, and I think it is good enough to launch, any more suggestion? Thanks. arch/x86/Kconfig |2 +- arch/x86/include/asm/nospec-branch.h | 26 +- arch/x86/net/Makefile|9 +- arch/x86/net/bpf_jit_comp32.c| 2530 ++ 4 files changed, 2562 insertions(+), 5 deletions(-) create mode 100644 arch/x86/net/bpf_jit_comp32.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 00fcf81..1f5fa2f 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -137,7 +137,7 @@ config X86 select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS - select HAVE_EBPF_JITif X86_64 + select HAVE_EBPF_JIT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_EXIT_THREAD select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index f928ad9..a4c7ca4 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -291,14 +291,17 @@ static i
[PATCH] bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform
All the testcases for BPF_PROG_TYPE_PERF_EVENT program type in
test_verifier(kselftest) report below errors on x86_32:
"
172/p unpriv: spill/fill of different pointers ldx FAIL
Unexpected error message!
0: (bf) r6 = r10
1: (07) r6 += -8
2: (15) if r1 == 0x0 goto pc+3
R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1
3: (bf) r2 = r10
4: (07) r2 += -76
5: (7b) *(u64 *)(r6 +0) = r2
6: (55) if r1 != 0x0 goto pc+1
R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1 R10=fp0,call_-1
fp-8=fp
7: (7b) *(u64 *)(r6 +0) = r1
8: (79) r1 = *(u64 *)(r6 +0)
9: (79) r1 = *(u64 *)(r1 +68)
invalid bpf_context access off=68 size=8
378/p check bpf_perf_event_data->sample_period byte load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (71) r0 = *(u8 *)(r1 +68)
invalid bpf_context access off=68 size=1
379/p check bpf_perf_event_data->sample_period half load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (69) r0 = *(u16 *)(r1 +68)
invalid bpf_context access off=68 size=2
380/p check bpf_perf_event_data->sample_period word load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (61) r0 = *(u32 *)(r1 +68)
invalid bpf_context access off=68 size=4
381/p check bpf_perf_event_data->sample_period dword load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (79) r0 = *(u64 *)(r1 +68)
invalid bpf_context access off=68 size=8
"
This patch fix it, the fix isn't only necessary for x86_32, it will fix the
same problem for other platforms too, if their size of bpf_user_pt_regs_t
can't divide exactly into 8.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Hi all!
After mainline accept this patch, then we need to submit a sync patch
to update the tools/include/uapi/linux/bpf_perf_event.h.
Thanks.
include/uapi/linux/bpf_perf_event.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/uapi/linux/bpf_perf_event.h
b/include/uapi/linux/bpf_perf_event.h
index eb1b9d2..ff4c092 100644
--- a/include/uapi/linux/bpf_perf_event.h
+++ b/include/uapi/linux/bpf_perf_event.h
@@ -12,7 +12,7 @@
struct bpf_perf_event_data {
bpf_user_pt_regs_t regs;
- __u64 sample_period;
+ __u64 sample_period __attribute__((aligned(8)));
__u64 addr;
};
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf: fix misaligned access for BPF_PROG_TYPE_PERF_EVENT program type on x86_32 platform
All the testcases for BPF_PROG_TYPE_PERF_EVENT program type in
test_verifier(kselftest) report below errors on x86_32:
"
172/p unpriv: spill/fill of different pointers ldx FAIL
Unexpected error message!
0: (bf) r6 = r10
1: (07) r6 += -8
2: (15) if r1 == 0x0 goto pc+3
R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1
3: (bf) r2 = r10
4: (07) r2 += -76
5: (7b) *(u64 *)(r6 +0) = r2
6: (55) if r1 != 0x0 goto pc+1
R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1 R10=fp0,call_-1
fp-8=fp
7: (7b) *(u64 *)(r6 +0) = r1
8: (79) r1 = *(u64 *)(r6 +0)
9: (79) r1 = *(u64 *)(r1 +68)
invalid bpf_context access off=68 size=8
378/p check bpf_perf_event_data->sample_period byte load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (71) r0 = *(u8 *)(r1 +68)
invalid bpf_context access off=68 size=1
379/p check bpf_perf_event_data->sample_period half load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (69) r0 = *(u16 *)(r1 +68)
invalid bpf_context access off=68 size=2
380/p check bpf_perf_event_data->sample_period word load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (61) r0 = *(u32 *)(r1 +68)
invalid bpf_context access off=68 size=4
381/p check bpf_perf_event_data->sample_period dword load permitted FAIL
Failed to load prog 'Permission denied'!
0: (b7) r0 = 0
1: (79) r0 = *(u64 *)(r1 +68)
invalid bpf_context access off=68 size=8
"
This patch fix it, the fix isn't only necessary for x86_32, it will fix the
same problem for other platforms too, if their size of bpf_user_pt_regs_t
can't divide exactly into 8.
Signed-off-by: Wang YanQing
---
Hi all!
After mainline accept this patch, then we need to submit a sync patch
to update the tools/include/uapi/linux/bpf_perf_event.h.
Thanks.
include/uapi/linux/bpf_perf_event.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/uapi/linux/bpf_perf_event.h
b/include/uapi/linux/bpf_perf_event.h
index eb1b9d2..ff4c092 100644
--- a/include/uapi/linux/bpf_perf_event.h
+++ b/include/uapi/linux/bpf_perf_event.h
@@ -12,7 +12,7 @@
struct bpf_perf_event_data {
bpf_user_pt_regs_t regs;
- __u64 sample_period;
+ __u64 sample_period __attribute__((aligned(8)));
__u64 addr;
};
--
1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH v2] bpf, x86_32: add eBPF JIT compiler for ia32
On Wed, Apr 25, 2018 at 02:11:16AM +0200, Daniel Borkmann wrote: > On 04/19/2018 05:54 PM, Wang YanQing wrote: > > Testing results on i5-5200U: > > > > 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed] > > 2) test_progs: Summary: 81 PASSED, 2 FAILED. > >test_progs report "libbpf: incorrect bpf_call opcode" for > >test_l4lb_noinline and test_xdp_noinline, because there is > >no llvm-6.0 on my machine, and current implementation doesn't > >support BPF_PSEUDO_CALL, so I think we can ignore the two failed > >testcases. > > 3) test_lpm: OK > > 4) test_lru_map: OK > > 5) test_verifier: Summary: 823 PASSED, 5 FAILED > >test_verifier report "invalid bpf_context access off=68 size=1/2/4/8" > >for all the 5 FAILED testcases with/without jit, we need to fix the > >failed testcases themself instead of this jit. > > Can you elaborate further on these? Looks like this definitely needs > fixing on 32 bit. Would be great to get a better understanding of the > underlying bug(s) and properly fix them. > Hi Daniel Borkmann, here is the detailed log for failed testcases. linux: Gentoo 32 bit llvm: ~ # llc --version LLVM (http://llvm.org/): LLVM version 4.0.1 Optimized build. Default target: i686-pc-linux-gnu Host CPU: broadwell Registered Targets: amdgcn - AMD GCN GPUs bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) nvptx - NVIDIA PTX 32-bit nvptx64 - NVIDIA PTX 64-bit r600- AMD GPUs HD2XXX-HD6XXX x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 ~ # clang --version clang version 4.0.1 (tags/RELEASE_401/final) Target: i686-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm/4/bin kernel version:4.16.2 test program:test_verifier in kselftest condition:bpf_jit_enable=0,bpf_jit_harden=0 log: #172/p unpriv: spill/fill of different pointers ldx FAIL Unexpected error message! 0: (bf) r6 = r10 1: (07) r6 += -8 2: (15) if r1 == 0x0 goto pc+3 R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1 3: (bf) r2 = r10 4: (07) r2 += -76 5: (7b) *(u64 *)(r6 +0) = r2 6: (55) if r1 != 0x0 goto pc+1 R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1 R10=fp0,call_-1 fp-8=fp 7: (7b) *(u64 *)(r6 +0) = r1 8: (79) r1 = *(u64 *)(r6 +0) 9: (79) r1 = *(u64 *)(r1 +68) invalid bpf_context access off=68 size=8 #378/p check bpf_perf_event_data->sample_period byte load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (71) r0 = *(u8 *)(r1 +68) invalid bpf_context access off=68 size=1 #379/p check bpf_perf_event_data->sample_period half load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (69) r0 = *(u16 *)(r1 +68) invalid bpf_context access off=68 size=2 #380/p check bpf_perf_event_data->sample_period word load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (61) r0 = *(u32 *)(r1 +68) invalid bpf_context access off=68 size=4 #381/p check bpf_perf_event_data->sample_period dword load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (79) r0 = *(u64 *)(r1 +68) invalid bpf_context access off=68 size=8 test program:test_progs condition:bpf_jit_enable=0,bpf_jit_harden=0 bpf # ./test_progs test_pkt_access:PASS:ipv4 53 nsec test_pkt_access:PASS:ipv6 47 nsec test_xdp:PASS:ipv4 1281 nsec test_xdp:PASS:ipv6 749 nsec test_l4lb:PASS:ipv4 427 nsec test_l4lb:PASS:ipv6 562 nsec libbpf: incorrect bpf_call opcode <= caused by ./test_l4lb_noinline.o in function test_l4lb_all libbpf: incorrect bpf_call opcode <= caused by ././test_xdp_noinline.o in function test_xdp_noinline Thanks.
Re: [PATCH v2] bpf, x86_32: add eBPF JIT compiler for ia32
On Wed, Apr 25, 2018 at 02:11:16AM +0200, Daniel Borkmann wrote: > On 04/19/2018 05:54 PM, Wang YanQing wrote: > > Testing results on i5-5200U: > > > > 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed] > > 2) test_progs: Summary: 81 PASSED, 2 FAILED. > >test_progs report "libbpf: incorrect bpf_call opcode" for > >test_l4lb_noinline and test_xdp_noinline, because there is > >no llvm-6.0 on my machine, and current implementation doesn't > >support BPF_PSEUDO_CALL, so I think we can ignore the two failed > >testcases. > > 3) test_lpm: OK > > 4) test_lru_map: OK > > 5) test_verifier: Summary: 823 PASSED, 5 FAILED > >test_verifier report "invalid bpf_context access off=68 size=1/2/4/8" > >for all the 5 FAILED testcases with/without jit, we need to fix the > >failed testcases themself instead of this jit. > > Can you elaborate further on these? Looks like this definitely needs > fixing on 32 bit. Would be great to get a better understanding of the > underlying bug(s) and properly fix them. > Hi Daniel Borkmann, here is the detailed log for failed testcases. linux: Gentoo 32 bit llvm: ~ # llc --version LLVM (http://llvm.org/): LLVM version 4.0.1 Optimized build. Default target: i686-pc-linux-gnu Host CPU: broadwell Registered Targets: amdgcn - AMD GCN GPUs bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) nvptx - NVIDIA PTX 32-bit nvptx64 - NVIDIA PTX 64-bit r600- AMD GPUs HD2XXX-HD6XXX x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 ~ # clang --version clang version 4.0.1 (tags/RELEASE_401/final) Target: i686-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm/4/bin kernel version:4.16.2 test program:test_verifier in kselftest condition:bpf_jit_enable=0,bpf_jit_harden=0 log: #172/p unpriv: spill/fill of different pointers ldx FAIL Unexpected error message! 0: (bf) r6 = r10 1: (07) r6 += -8 2: (15) if r1 == 0x0 goto pc+3 R1=ctx(id=0,off=0,imm=0) R6=fp-8,call_-1 R10=fp0,call_-1 3: (bf) r2 = r10 4: (07) r2 += -76 5: (7b) *(u64 *)(r6 +0) = r2 6: (55) if r1 != 0x0 goto pc+1 R1=ctx(id=0,off=0,imm=0) R2=fp-76,call_-1 R6=fp-8,call_-1 R10=fp0,call_-1 fp-8=fp 7: (7b) *(u64 *)(r6 +0) = r1 8: (79) r1 = *(u64 *)(r6 +0) 9: (79) r1 = *(u64 *)(r1 +68) invalid bpf_context access off=68 size=8 #378/p check bpf_perf_event_data->sample_period byte load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (71) r0 = *(u8 *)(r1 +68) invalid bpf_context access off=68 size=1 #379/p check bpf_perf_event_data->sample_period half load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (69) r0 = *(u16 *)(r1 +68) invalid bpf_context access off=68 size=2 #380/p check bpf_perf_event_data->sample_period word load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (61) r0 = *(u32 *)(r1 +68) invalid bpf_context access off=68 size=4 #381/p check bpf_perf_event_data->sample_period dword load permitted FAIL Failed to load prog 'Permission denied'! 0: (b7) r0 = 0 1: (79) r0 = *(u64 *)(r1 +68) invalid bpf_context access off=68 size=8 test program:test_progs condition:bpf_jit_enable=0,bpf_jit_harden=0 bpf # ./test_progs test_pkt_access:PASS:ipv4 53 nsec test_pkt_access:PASS:ipv6 47 nsec test_xdp:PASS:ipv4 1281 nsec test_xdp:PASS:ipv6 749 nsec test_l4lb:PASS:ipv4 427 nsec test_l4lb:PASS:ipv6 562 nsec libbpf: incorrect bpf_call opcode <= caused by ./test_l4lb_noinline.o in function test_l4lb_all libbpf: incorrect bpf_call opcode <= caused by ././test_xdp_noinline.o in function test_xdp_noinline Thanks.
[PATCH v3] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF only. Classic BPF is supported because of the conversion by BPF core. Almost all instructions from eBPF ISA supported except the following: BPF_ALU64 | BPF_DIV | BPF_K BPF_ALU64 | BPF_DIV | BPF_X BPF_ALU64 | BPF_MOD | BPF_K BPF_ALU64 | BPF_MOD | BPF_X BPF_STX | BPF_XADD | BPF_W BPF_STX | BPF_XADD | BPF_DW It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment. IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others eBPF registers, R0-R10, are simulated through scratch space on stack. The reasons behind the hardware registers allocation policy are: 1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit for general eBPF 64bit register simulation. 2:We need at least 4 registers to simulate most eBPF ISA operations on registers operands instead of on register operands. 3:We need to put BPF_REG_AX on hardware registers, or constant blinding will degrade jit performance heavily. Tested on PC (Intel(R) Core(TM) i5-5200U CPU). Testing results on i5-5200U: 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed] 2) test_progs: Summary: 81 PASSED, 2 FAILED. 3) test_lpm: OK 4) test_lru_map: OK 5) test_verifier: Summary: 823 PASSED, 5 FAILED Note: these failed testcases are caused by unconcerned reasons, we need to fix the testcases themself: "invalid bpf_context access" and "libbpf: incorrect bpf_call opcode" Above tests are all done in following two conditions separately: 1:bpf_jit_enable=1 and bpf_jit_harden=0 2:bpf_jit_enable=1 and bpf_jit_harden=2 Below are some numbers for this jit implementation: Note: I run test_progs in kselftest 100 times continuously for every condition, the numbers are in format: total/times=avg. The numbers that test_bpf reports show almost the same relation. a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0 test_pkt_access:PASS:ipv4:15622/100=156 test_pkt_access:PASS:ipv4:10674/100=106 test_pkt_access:PASS:ipv6:9130/100=91 test_pkt_access:PASS:ipv6:4855/100=48 test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389 test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685 test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373 test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550 c:jit_enable=1 and jit_harden=2 test_pkt_access:PASS:ipv4:10558/100=105 test_pkt_access:PASS:ipv6:5092/100=50 test_xdp:PASS:ipv4:131902/100=1319 test_xdp:PASS:ipv6:77932/100=779 test_l4lb:PASS:ipv4:38924/100=389 test_l4lb:PASS:ipv6:57520/100=575 The numbers show we get 30%~50% improvement. See Documentation/networking/filter.txt for more information. Signed-off-by: Wang YanQing <udkni...@gmail.com> --- Changes v2-v3: 1:Move BPF_REG_AX to real hardware registers for performance reason. 3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann. 4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann. 5:Some bug fixes and comments improvement. Changes v1-v2: 1:Fix bug in emit_ia32_neg64. 2:Fix bug in emit_ia32_arsh_r64. 3:Delete filename in top level comment, suggested by Thomas Gleixner. 4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner. 5:Rewrite some words in changelog. 6:CodingSytle improvement and a little more comments. arch/x86/Kconfig |2 +- arch/x86/include/asm/nospec-branch.h | 26 +- arch/x86/net/Makefile|9 +- arch/x86/net/bpf_jit_comp32.c| 2533 ++ 4 files changed, 2565 insertions(+), 5 deletions(-) create mode 100644 arch/x86/net/bpf_jit_comp32.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 00fcf81..1f5fa2f 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -137,7 +137,7 @@ config X86 select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS - select HAVE_EBPF_JITif X86_64 + select HAVE_EBPF_JIT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_EXIT_THREAD select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index f928ad9..a4c7ca4 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -291,14 +291,17 @@ static inline void indirect_branch_prediction_barrier(void) *lfence *jmp spec_trap * do_rop: - *mov %rax,(%rsp) + *mov %rax,(%rsp) for x86_64 + *mov %edx,(%esp) for x86_32 *retq * * Without retpolines configured: * - *jmp *%rax + *jmp *%rax for x86_64 + *jmp *%e
[PATCH v3] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports eBPF only. Classic BPF is supported because of the conversion by BPF core. Almost all instructions from eBPF ISA supported except the following: BPF_ALU64 | BPF_DIV | BPF_K BPF_ALU64 | BPF_DIV | BPF_X BPF_ALU64 | BPF_MOD | BPF_K BPF_ALU64 | BPF_MOD | BPF_X BPF_STX | BPF_XADD | BPF_W BPF_STX | BPF_XADD | BPF_DW It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL at the moment. IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI. I use EAX|EDX|ECX|EBX as temporary registers to simulate instructions in eBPF ISA, and allocate ESI|EDI to BPF_REG_AX for constant blinding, all others eBPF registers, R0-R10, are simulated through scratch space on stack. The reasons behind the hardware registers allocation policy are: 1:MUL need EAX:EDX, shift operation need ECX, so they aren't fit for general eBPF 64bit register simulation. 2:We need at least 4 registers to simulate most eBPF ISA operations on registers operands instead of on register operands. 3:We need to put BPF_REG_AX on hardware registers, or constant blinding will degrade jit performance heavily. Tested on PC (Intel(R) Core(TM) i5-5200U CPU). Testing results on i5-5200U: 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed] 2) test_progs: Summary: 81 PASSED, 2 FAILED. 3) test_lpm: OK 4) test_lru_map: OK 5) test_verifier: Summary: 823 PASSED, 5 FAILED Note: these failed testcases are caused by unconcerned reasons, we need to fix the testcases themself: "invalid bpf_context access" and "libbpf: incorrect bpf_call opcode" Above tests are all done in following two conditions separately: 1:bpf_jit_enable=1 and bpf_jit_harden=0 2:bpf_jit_enable=1 and bpf_jit_harden=2 Below are some numbers for this jit implementation: Note: I run test_progs in kselftest 100 times continuously for every condition, the numbers are in format: total/times=avg. The numbers that test_bpf reports show almost the same relation. a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0 test_pkt_access:PASS:ipv4:15622/100=156 test_pkt_access:PASS:ipv4:10674/100=106 test_pkt_access:PASS:ipv6:9130/100=91 test_pkt_access:PASS:ipv6:4855/100=48 test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:138912/100=1389 test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:68542/100=685 test_l4lb:PASS:ipv4:61100/100=611 test_l4lb:PASS:ipv4:37302/100=373 test_l4lb:PASS:ipv6:101000/100=1010test_l4lb:PASS:ipv6:55030/100=550 c:jit_enable=1 and jit_harden=2 test_pkt_access:PASS:ipv4:10558/100=105 test_pkt_access:PASS:ipv6:5092/100=50 test_xdp:PASS:ipv4:131902/100=1319 test_xdp:PASS:ipv6:77932/100=779 test_l4lb:PASS:ipv4:38924/100=389 test_l4lb:PASS:ipv6:57520/100=575 The numbers show we get 30%~50% improvement. See Documentation/networking/filter.txt for more information. Signed-off-by: Wang YanQing --- Changes v2-v3: 1:Move BPF_REG_AX to real hardware registers for performance reason. 3:Using bpf_load_pointer instead of bpf_jit32.S, suggested by Daniel Borkmann. 4:Delete partial codes in 1c2a088a6626, suggested by Daniel Borkmann. 5:Some bug fixes and comments improvement. Changes v1-v2: 1:Fix bug in emit_ia32_neg64. 2:Fix bug in emit_ia32_arsh_r64. 3:Delete filename in top level comment, suggested by Thomas Gleixner. 4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner. 5:Rewrite some words in changelog. 6:CodingSytle improvement and a little more comments. arch/x86/Kconfig |2 +- arch/x86/include/asm/nospec-branch.h | 26 +- arch/x86/net/Makefile|9 +- arch/x86/net/bpf_jit_comp32.c| 2533 ++ 4 files changed, 2565 insertions(+), 5 deletions(-) create mode 100644 arch/x86/net/bpf_jit_comp32.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 00fcf81..1f5fa2f 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -137,7 +137,7 @@ config X86 select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS - select HAVE_EBPF_JITif X86_64 + select HAVE_EBPF_JIT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_EXIT_THREAD select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index f928ad9..a4c7ca4 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -291,14 +291,17 @@ static inline void indirect_branch_prediction_barrier(void) *lfence *jmp spec_trap * do_rop: - *mov %rax,(%rsp) + *mov %rax,(%rsp) for x86_64 + *mov %edx,(%esp) for x86_32 *retq * * Without retpolines configured: * - *jmp *%rax + *jmp *%rax for x86_64 + *jmp *%edx for x86_32 */ #if
Re: [PATCH v2] bpf, x86_32: add eBPF JIT compiler for ia32
On Wed, Apr 25, 2018 at 02:11:16AM +0200, Daniel Borkmann wrote:
> On 04/19/2018 05:54 PM, Wang YanQing wrote:
> > The JIT compiler emits ia32 bit instructions. Currently, It supports
> > eBPF only. Classic BPF is supported because of the conversion by BPF core.
> >
> > Almost all instructions from eBPF ISA supported except the following:
> > BPF_ALU64 | BPF_DIV | BPF_K
> > BPF_ALU64 | BPF_DIV | BPF_X
> > BPF_ALU64 | BPF_MOD | BPF_K
> > BPF_ALU64 | BPF_MOD | BPF_X
> > BPF_STX | BPF_XADD | BPF_W
> > BPF_STX | BPF_XADD | BPF_DW
> >
> > It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
> >
> > ia32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
> > and in these six registers, we can't treat all of them as real
> > general purpose registers in jit:
> > MUL instructions need EAX:EDX, shift instructions need ECX.
> >
> > So I decide to use stack to emulate all eBPF 64 registers, this will
> > simplify the implementation a lot, because we don't need to face the
> > flexible memory address modes on ia32, for example, we don't need to
> > write below code pattern for one BPF_ADD instruction:
> >
> > if (src is a register && dst is a register)
> > {
> >//one instruction encoding for ADD instruction
> > } else if (only src is a register)
> > {
> >//another different instruction encoding for ADD instruction
> > } else if (only dst is a register)
> > {
> >//another different instruction encoding for ADD instruction
> > } else
> > {
> >//src and dst are all on stack.
> >//move src or dst to temporary registers
> > }
> >
> > If the above example if-else-else-else isn't so painful, try to think
> > it for BPF_ALU64|BPF_*SHIFT* instruction which we need to use many
> > native instructions to emulate.
> >
> > Tested on my PC (Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
>
> Just out of plain curiosity, do you have a specific use case on the
> JIT for x86_32? Are you targeting this to be used for e.g. Atom or
> the like (sorry, don't really have a good overview where x86_32 is
> still in use these days)?
Yes, you are right that x86_32 isn't the BIG DEAL anymore, but they willn't
disappear completely in near future, the same as 32-bit linux kernel on
64bit machine.
For me, because I use 32-bit linux on development/hacking notebook for many
years,
I try to trace/perf the kernel, then I meet eBPF, it is strange that there isn't
a jit for it, so I try to fill the empty.
>
> > Testing results on i5-5200U:
> >
> > 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
> > 2) test_progs: Summary: 81 PASSED, 2 FAILED.
> >test_progs report "libbpf: incorrect bpf_call opcode" for
> >test_l4lb_noinline and test_xdp_noinline, because there is
> >no llvm-6.0 on my machine, and current implementation doesn't
> >support BPF_PSEUDO_CALL, so I think we can ignore the two failed
> >testcases.
> > 3) test_lpm: OK
> > 4) test_lru_map: OK
> > 5) test_verifier: Summary: 823 PASSED, 5 FAILED
> >test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
> >for all the 5 FAILED testcases with/without jit, we need to fix the
> >failed testcases themself instead of this jit.
>
> Can you elaborate further on these? Looks like this definitely needs
> fixing on 32 bit. Would be great to get a better understanding of the
> underlying bug(s) and properly fix them.
Ok! I will try to dig them out.
>
> > Above tests are all done with following flags settings discretely:
> > 1:bpf_jit_enable=1 and bpf_jit_harden=0
> > 2:bpf_jit_enable=1 and bpf_jit_harden=2
> >
> > Below are some numbers for this jit implementation:
> > Note:
> > I run test_progs in kselftest 100 times continuously for every testcase,
> > the numbers are in format: total/times=avg.
> > The numbers that test_bpf reports show almost the same relation.
> >
> > a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
> > test_pkt_access:PASS:ipv4:15622/100=156
> > test_pkt_access:PASS:ipv4:10057/100=100
> > test_pkt_access:PASS:ipv6:9130/100=91
> > test_pkt_access:PASS:ipv6:5055/100=50
> > test_xdp:PASS:ipv4:240198/100=2401
> > test_xdp:PASS:ipv4:145945/100=1459
> > test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
> > test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
> > test_l4lb:PASS:ipv6:101000/100=1010
Re: [PATCH v2] bpf, x86_32: add eBPF JIT compiler for ia32
On Wed, Apr 25, 2018 at 02:11:16AM +0200, Daniel Borkmann wrote:
> On 04/19/2018 05:54 PM, Wang YanQing wrote:
> > The JIT compiler emits ia32 bit instructions. Currently, It supports
> > eBPF only. Classic BPF is supported because of the conversion by BPF core.
> >
> > Almost all instructions from eBPF ISA supported except the following:
> > BPF_ALU64 | BPF_DIV | BPF_K
> > BPF_ALU64 | BPF_DIV | BPF_X
> > BPF_ALU64 | BPF_MOD | BPF_K
> > BPF_ALU64 | BPF_MOD | BPF_X
> > BPF_STX | BPF_XADD | BPF_W
> > BPF_STX | BPF_XADD | BPF_DW
> >
> > It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
> >
> > ia32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
> > and in these six registers, we can't treat all of them as real
> > general purpose registers in jit:
> > MUL instructions need EAX:EDX, shift instructions need ECX.
> >
> > So I decide to use stack to emulate all eBPF 64 registers, this will
> > simplify the implementation a lot, because we don't need to face the
> > flexible memory address modes on ia32, for example, we don't need to
> > write below code pattern for one BPF_ADD instruction:
> >
> > if (src is a register && dst is a register)
> > {
> >//one instruction encoding for ADD instruction
> > } else if (only src is a register)
> > {
> >//another different instruction encoding for ADD instruction
> > } else if (only dst is a register)
> > {
> >//another different instruction encoding for ADD instruction
> > } else
> > {
> >//src and dst are all on stack.
> >//move src or dst to temporary registers
> > }
> >
> > If the above example if-else-else-else isn't so painful, try to think
> > it for BPF_ALU64|BPF_*SHIFT* instruction which we need to use many
> > native instructions to emulate.
> >
> > Tested on my PC (Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
>
> Just out of plain curiosity, do you have a specific use case on the
> JIT for x86_32? Are you targeting this to be used for e.g. Atom or
> the like (sorry, don't really have a good overview where x86_32 is
> still in use these days)?
Yes, you are right that x86_32 isn't the BIG DEAL anymore, but they willn't
disappear completely in near future, the same as 32-bit linux kernel on
64bit machine.
For me, because I use 32-bit linux on development/hacking notebook for many
years,
I try to trace/perf the kernel, then I meet eBPF, it is strange that there isn't
a jit for it, so I try to fill the empty.
>
> > Testing results on i5-5200U:
> >
> > 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
> > 2) test_progs: Summary: 81 PASSED, 2 FAILED.
> >test_progs report "libbpf: incorrect bpf_call opcode" for
> >test_l4lb_noinline and test_xdp_noinline, because there is
> >no llvm-6.0 on my machine, and current implementation doesn't
> >support BPF_PSEUDO_CALL, so I think we can ignore the two failed
> >testcases.
> > 3) test_lpm: OK
> > 4) test_lru_map: OK
> > 5) test_verifier: Summary: 823 PASSED, 5 FAILED
> >test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
> >for all the 5 FAILED testcases with/without jit, we need to fix the
> >failed testcases themself instead of this jit.
>
> Can you elaborate further on these? Looks like this definitely needs
> fixing on 32 bit. Would be great to get a better understanding of the
> underlying bug(s) and properly fix them.
Ok! I will try to dig them out.
>
> > Above tests are all done with following flags settings discretely:
> > 1:bpf_jit_enable=1 and bpf_jit_harden=0
> > 2:bpf_jit_enable=1 and bpf_jit_harden=2
> >
> > Below are some numbers for this jit implementation:
> > Note:
> > I run test_progs in kselftest 100 times continuously for every testcase,
> > the numbers are in format: total/times=avg.
> > The numbers that test_bpf reports show almost the same relation.
> >
> > a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
> > test_pkt_access:PASS:ipv4:15622/100=156
> > test_pkt_access:PASS:ipv4:10057/100=100
> > test_pkt_access:PASS:ipv6:9130/100=91
> > test_pkt_access:PASS:ipv6:5055/100=50
> > test_xdp:PASS:ipv4:240198/100=2401
> > test_xdp:PASS:ipv4:145945/100=1459
> > test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
> > test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
> > test_l4lb:PASS:ipv6:101000/100=1010
[PATCH v2] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports
eBPF only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
ia32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
and in these six registers, we can't treat all of them as real
general purpose registers in jit:
MUL instructions need EAX:EDX, shift instructions need ECX.
So I decide to use stack to emulate all eBPF 64 registers, this will
simplify the implementation a lot, because we don't need to face the
flexible memory address modes on ia32, for example, we don't need to
write below code pattern for one BPF_ADD instruction:
if (src is a register && dst is a register)
{
//one instruction encoding for ADD instruction
} else if (only src is a register)
{
//another different instruction encoding for ADD instruction
} else if (only dst is a register)
{
//another different instruction encoding for ADD instruction
} else
{
//src and dst are all on stack.
//move src or dst to temporary registers
}
If the above example if-else-else-else isn't so painful, try to think
it for BPF_ALU64|BPF_*SHIFT* instruction which we need to use many
native instructions to emulate.
Tested on my PC (Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 81 PASSED, 2 FAILED.
test_progs report "libbpf: incorrect bpf_call opcode" for
test_l4lb_noinline and test_xdp_noinline, because there is
no llvm-6.0 on my machine, and current implementation doesn't
support BPF_PSEUDO_CALL, so I think we can ignore the two failed
testcases.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 823 PASSED, 5 FAILED
test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
for all the 5 FAILED testcases with/without jit, we need to fix the
failed testcases themself instead of this jit.
Above tests are all done with following flags settings discretely:
1:bpf_jit_enable=1 and bpf_jit_harden=0
2:bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs in kselftest 100 times continuously for every testcase,
the numbers are in format: total/times=avg.
The numbers that test_bpf reports show almost the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10057/100=100
test_pkt_access:PASS:ipv6:9130/100=91test_pkt_access:PASS:ipv6:5055/100=50
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:145945/100=1459
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
test_l4lb:PASS:ipv6:101000/100=1010 test_l4lb:PASS:ipv6:57779/100=577
c:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:12650/100=126
test_pkt_access:PASS:ipv6:7074/100=70
test_xdp:PASS:ipv4:147211/100=1472
test_xdp:PASS:ipv6:85783/100=857
test_l4lb:PASS:ipv4:53222/100=532
test_l4lb:PASS:ipv6:76322/100=763
Yes, the numbers are pretty when turn off jit_harden, if we want to speedup
jit_harden, then we need to move BPF_REG_AX to *real* register instead of stack
emulation, but when we do it, we need to face all the pain I describe above. We
can do it in next step.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes v1-v2:
1:Fix bug in emit_ia32_neg64.
2:Fix bug in emit_ia32_arsh_r64.
3:Delete filename in top level comment, suggested by Thomas Gleixner.
4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner.
5:Rewrite some words in changelog.
6:CodingSytle improvement and a little more comments.
Thanks.
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 26 +-
arch/x86/net/Makefile| 10 +-
arch/x86/net/bpf_jit32.S | 143 +++
arch/x86/net/bpf_jit_comp32.c| 2258 ++
5 files changed, 2434 insertions(+), 5 deletions(-)
create mode 100644 arch/x86/net/bpf_jit32.S
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 00fcf81..1f5fa2f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -137,7 +137,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ s
[PATCH v2] bpf, x86_32: add eBPF JIT compiler for ia32
The JIT compiler emits ia32 bit instructions. Currently, It supports
eBPF only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
ia32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
and in these six registers, we can't treat all of them as real
general purpose registers in jit:
MUL instructions need EAX:EDX, shift instructions need ECX.
So I decide to use stack to emulate all eBPF 64 registers, this will
simplify the implementation a lot, because we don't need to face the
flexible memory address modes on ia32, for example, we don't need to
write below code pattern for one BPF_ADD instruction:
if (src is a register && dst is a register)
{
//one instruction encoding for ADD instruction
} else if (only src is a register)
{
//another different instruction encoding for ADD instruction
} else if (only dst is a register)
{
//another different instruction encoding for ADD instruction
} else
{
//src and dst are all on stack.
//move src or dst to temporary registers
}
If the above example if-else-else-else isn't so painful, try to think
it for BPF_ALU64|BPF_*SHIFT* instruction which we need to use many
native instructions to emulate.
Tested on my PC (Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 81 PASSED, 2 FAILED.
test_progs report "libbpf: incorrect bpf_call opcode" for
test_l4lb_noinline and test_xdp_noinline, because there is
no llvm-6.0 on my machine, and current implementation doesn't
support BPF_PSEUDO_CALL, so I think we can ignore the two failed
testcases.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 823 PASSED, 5 FAILED
test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
for all the 5 FAILED testcases with/without jit, we need to fix the
failed testcases themself instead of this jit.
Above tests are all done with following flags settings discretely:
1:bpf_jit_enable=1 and bpf_jit_harden=0
2:bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs in kselftest 100 times continuously for every testcase,
the numbers are in format: total/times=avg.
The numbers that test_bpf reports show almost the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10057/100=100
test_pkt_access:PASS:ipv6:9130/100=91test_pkt_access:PASS:ipv6:5055/100=50
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:145945/100=1459
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
test_l4lb:PASS:ipv6:101000/100=1010 test_l4lb:PASS:ipv6:57779/100=577
c:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:12650/100=126
test_pkt_access:PASS:ipv6:7074/100=70
test_xdp:PASS:ipv4:147211/100=1472
test_xdp:PASS:ipv6:85783/100=857
test_l4lb:PASS:ipv4:53222/100=532
test_l4lb:PASS:ipv6:76322/100=763
Yes, the numbers are pretty when turn off jit_harden, if we want to speedup
jit_harden, then we need to move BPF_REG_AX to *real* register instead of stack
emulation, but when we do it, we need to face all the pain I describe above. We
can do it in next step.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing
---
Changes v1-v2:
1:Fix bug in emit_ia32_neg64.
2:Fix bug in emit_ia32_arsh_r64.
3:Delete filename in top level comment, suggested by Thomas Gleixner.
4:Delete unnecessary boiler plate text, suggested by Thomas Gleixner.
5:Rewrite some words in changelog.
6:CodingSytle improvement and a little more comments.
Thanks.
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 26 +-
arch/x86/net/Makefile| 10 +-
arch/x86/net/bpf_jit32.S | 143 +++
arch/x86/net/bpf_jit_comp32.c| 2258 ++
5 files changed, 2434 insertions(+), 5 deletions(-)
create mode 100644 arch/x86/net/bpf_jit32.S
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 00fcf81..1f5fa2f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -137,7 +137,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select
Re: [PATCH] bpf, x86_32: add eBPF JIT compiler for ia32 (x86_32)
On Wed, Apr 18, 2018 at 05:31:18PM +0800, Wang YanQing wrote:
> The JIT compiler emits ia32 bit instructions. Currently, It supports
> eBPF only. Classic BPF is supported because of the conversion by BPF core.
>
> Almost all instructions from eBPF ISA supported except the following:
> BPF_ALU64 | BPF_DIV | BPF_K
> BPF_ALU64 | BPF_DIV | BPF_X
> BPF_ALU64 | BPF_MOD | BPF_K
> BPF_ALU64 | BPF_MOD | BPF_X
> BPF_STX | BPF_XADD | BPF_W
> BPF_STX | BPF_XADD | BPF_DW
>
> It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
>
> IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
> and for these six registers, we can't treat all of them as real
> general purpose registers:
> MUL instructions need EAX:EDX, shift instructions need ECX, ESI|EDI
> for string manipulation instructions.
>
> So I decide to use stack to emulate all eBPF 64 registers, this will
> simplify the implementation very much, because we don't need to face
> the flexible memory address modes on ia32, for example, we don't need
> to write below codes for one BPF_ADD instruction:
> if (src_reg is a register && dst_reg is a register)
> {
>//one instruction encoding for ADD instruction
> } else if (only src is a register)
> {
>//another different instruction encoding for ADD instruction
> } else if (only dst is a register)
> {
>//another different instruction encoding for ADD instruction
> } else
> {
>//src and dst are all on stack.
>//another different instruction encoding for ADD instruction
> }
>
> If you think above if-else-else-else isn't so painful, try to think
> it for BPF_ALU64|BPF_*SHIFT* instruction:)
>
> Tested on my PC(Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
>
> Testing results on i5-5200U:
>
> 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
> 2) test_progs: Summary: 81 PASSED, 2 FAILED.
>test_progs report "libbpf: incorrect bpf_call opcode" for
>test_l4lb_noinline and test_xdp_noinline, because there is
>no llvm-6.0 on my machine, and current implementation doesn't
>support BPF_CALL, so I think we can ignore it.
> 3) test_lpm: OK
> 4) test_lru_map: OK
> 5) test_verifier: Summary: 823 PASSED, 5 FAILED
>test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
>for all the 5 FAILED testcases, and test_verifier report them when
>we turn off the jit, so I think the jit can do nothing to fix them.
>
> Above tests are all done with following flags enabled discretely:
> bpf_jit_enable=1 and bpf_jit_harden=2
>
> Below are some numbers for this jit implementation:
> Note:
> I run test_progs 100 times in loop for every testcase, the numbers
> are in format: total/times=avg. The numbers that test_bpf report
> almost show the same relation.
>
> a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
> test_pkt_access:PASS:ipv4:15622/100=156
> test_pkt_access:PASS:ipv4:10057/100=100
> test_pkt_access:PASS:ipv6:9130/100=91
> test_pkt_access:PASS:ipv6:5055/100=50
> test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:145945/100=1459
> test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
> test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
> test_l4lb:PASS:ipv6:101000/100=1010 test_l4lb:PASS:ipv6:57779/100=577
>
> c:jit_enable=0 and jit_harden=2b:jit_enable=1 and jit_harden=2
> test_pkt_access:PASS:ipv4:15214/100=152
> test_pkt_access:PASS:ipv4:12650/100=126
> test_pkt_access:PASS:ipv6:9132/100=91
> test_pkt_access:PASS:ipv6:7074/100=70
> test_xdp:PASS:ipv4:237252/100=2372 test_xdp:PASS:ipv4:147211/100=1472
> test_xdp:PASS:ipv6:135977/100=1359 test_xdp:PASS:ipv6:85783/100=857
> test_l4lb:PASS:ipv4:61324/100=613test_l4lb:PASS:ipv4:53222/100=532
> test_l4lb:PASS:ipv6:100833/100=1008 test_l4lb:PASS:ipv6:76322/100=763
>
> Yes, the numbers are pretty without turn on jit_harden, if we want to speedup
> jit_harden, then we need to move BPF_REG_AX to *real* register instead of
> stack
> emulation, but If we do it, we need to face all the pain I describe above. We
> can do it in next step.
>
> See Documentation/networking/filter.txt for more information.
>
> Signed-off-by: Wang YanQing <udkni...@gmail.com>
> ---
> arch/x86/Kconfig |2 +-
> arch/x86/include/asm/nospec-branch.h | 26 +-
> arch/x86/net/Makefile| 10 +-
> arch/x86/net/bpf_jit32.S | 147 +++
> arch/x86/net/bpf_jit_comp32.c| 2239
> ++
> 5 files changed, 2419 insertions(+), 5 deletions(-)
> create mode 100644 arch/x86/net/bpf_jit32.S
> create mode 100644 arch/x86/net/bpf_jit_comp32.c
Add CC to da...@davemloft.net
Re: [PATCH] bpf, x86_32: add eBPF JIT compiler for ia32 (x86_32)
On Wed, Apr 18, 2018 at 05:31:18PM +0800, Wang YanQing wrote:
> The JIT compiler emits ia32 bit instructions. Currently, It supports
> eBPF only. Classic BPF is supported because of the conversion by BPF core.
>
> Almost all instructions from eBPF ISA supported except the following:
> BPF_ALU64 | BPF_DIV | BPF_K
> BPF_ALU64 | BPF_DIV | BPF_X
> BPF_ALU64 | BPF_MOD | BPF_K
> BPF_ALU64 | BPF_MOD | BPF_X
> BPF_STX | BPF_XADD | BPF_W
> BPF_STX | BPF_XADD | BPF_DW
>
> It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
>
> IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
> and for these six registers, we can't treat all of them as real
> general purpose registers:
> MUL instructions need EAX:EDX, shift instructions need ECX, ESI|EDI
> for string manipulation instructions.
>
> So I decide to use stack to emulate all eBPF 64 registers, this will
> simplify the implementation very much, because we don't need to face
> the flexible memory address modes on ia32, for example, we don't need
> to write below codes for one BPF_ADD instruction:
> if (src_reg is a register && dst_reg is a register)
> {
>//one instruction encoding for ADD instruction
> } else if (only src is a register)
> {
>//another different instruction encoding for ADD instruction
> } else if (only dst is a register)
> {
>//another different instruction encoding for ADD instruction
> } else
> {
>//src and dst are all on stack.
>//another different instruction encoding for ADD instruction
> }
>
> If you think above if-else-else-else isn't so painful, try to think
> it for BPF_ALU64|BPF_*SHIFT* instruction:)
>
> Tested on my PC(Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
>
> Testing results on i5-5200U:
>
> 1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
> 2) test_progs: Summary: 81 PASSED, 2 FAILED.
>test_progs report "libbpf: incorrect bpf_call opcode" for
>test_l4lb_noinline and test_xdp_noinline, because there is
>no llvm-6.0 on my machine, and current implementation doesn't
>support BPF_CALL, so I think we can ignore it.
> 3) test_lpm: OK
> 4) test_lru_map: OK
> 5) test_verifier: Summary: 823 PASSED, 5 FAILED
>test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
>for all the 5 FAILED testcases, and test_verifier report them when
>we turn off the jit, so I think the jit can do nothing to fix them.
>
> Above tests are all done with following flags enabled discretely:
> bpf_jit_enable=1 and bpf_jit_harden=2
>
> Below are some numbers for this jit implementation:
> Note:
> I run test_progs 100 times in loop for every testcase, the numbers
> are in format: total/times=avg. The numbers that test_bpf report
> almost show the same relation.
>
> a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
> test_pkt_access:PASS:ipv4:15622/100=156
> test_pkt_access:PASS:ipv4:10057/100=100
> test_pkt_access:PASS:ipv6:9130/100=91
> test_pkt_access:PASS:ipv6:5055/100=50
> test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:145945/100=1459
> test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
> test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
> test_l4lb:PASS:ipv6:101000/100=1010 test_l4lb:PASS:ipv6:57779/100=577
>
> c:jit_enable=0 and jit_harden=2b:jit_enable=1 and jit_harden=2
> test_pkt_access:PASS:ipv4:15214/100=152
> test_pkt_access:PASS:ipv4:12650/100=126
> test_pkt_access:PASS:ipv6:9132/100=91
> test_pkt_access:PASS:ipv6:7074/100=70
> test_xdp:PASS:ipv4:237252/100=2372 test_xdp:PASS:ipv4:147211/100=1472
> test_xdp:PASS:ipv6:135977/100=1359 test_xdp:PASS:ipv6:85783/100=857
> test_l4lb:PASS:ipv4:61324/100=613test_l4lb:PASS:ipv4:53222/100=532
> test_l4lb:PASS:ipv6:100833/100=1008 test_l4lb:PASS:ipv6:76322/100=763
>
> Yes, the numbers are pretty without turn on jit_harden, if we want to speedup
> jit_harden, then we need to move BPF_REG_AX to *real* register instead of
> stack
> emulation, but If we do it, we need to face all the pain I describe above. We
> can do it in next step.
>
> See Documentation/networking/filter.txt for more information.
>
> Signed-off-by: Wang YanQing
> ---
> arch/x86/Kconfig |2 +-
> arch/x86/include/asm/nospec-branch.h | 26 +-
> arch/x86/net/Makefile| 10 +-
> arch/x86/net/bpf_jit32.S | 147 +++
> arch/x86/net/bpf_jit_comp32.c| 2239
> ++
> 5 files changed, 2419 insertions(+), 5 deletions(-)
> create mode 100644 arch/x86/net/bpf_jit32.S
> create mode 100644 arch/x86/net/bpf_jit_comp32.c
Add CC to da...@davemloft.net
[PATCH] bpf, x86_32: add eBPF JIT compiler for ia32 (x86_32)
The JIT compiler emits ia32 bit instructions. Currently, It supports
eBPF only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
and for these six registers, we can't treat all of them as real
general purpose registers:
MUL instructions need EAX:EDX, shift instructions need ECX, ESI|EDI
for string manipulation instructions.
So I decide to use stack to emulate all eBPF 64 registers, this will
simplify the implementation very much, because we don't need to face
the flexible memory address modes on ia32, for example, we don't need
to write below codes for one BPF_ADD instruction:
if (src_reg is a register && dst_reg is a register)
{
//one instruction encoding for ADD instruction
} else if (only src is a register)
{
//another different instruction encoding for ADD instruction
} else if (only dst is a register)
{
//another different instruction encoding for ADD instruction
} else
{
//src and dst are all on stack.
//another different instruction encoding for ADD instruction
}
If you think above if-else-else-else isn't so painful, try to think
it for BPF_ALU64|BPF_*SHIFT* instruction:)
Tested on my PC(Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 81 PASSED, 2 FAILED.
test_progs report "libbpf: incorrect bpf_call opcode" for
test_l4lb_noinline and test_xdp_noinline, because there is
no llvm-6.0 on my machine, and current implementation doesn't
support BPF_CALL, so I think we can ignore it.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 823 PASSED, 5 FAILED
test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
for all the 5 FAILED testcases, and test_verifier report them when
we turn off the jit, so I think the jit can do nothing to fix them.
Above tests are all done with following flags enabled discretely:
bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs 100 times in loop for every testcase, the numbers
are in format: total/times=avg. The numbers that test_bpf report
almost show the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10057/100=100
test_pkt_access:PASS:ipv6:9130/100=91test_pkt_access:PASS:ipv6:5055/100=50
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:145945/100=1459
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
test_l4lb:PASS:ipv6:101000/100=1010 test_l4lb:PASS:ipv6:57779/100=577
c:jit_enable=0 and jit_harden=2b:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:15214/100=152
test_pkt_access:PASS:ipv4:12650/100=126
test_pkt_access:PASS:ipv6:9132/100=91test_pkt_access:PASS:ipv6:7074/100=70
test_xdp:PASS:ipv4:237252/100=2372 test_xdp:PASS:ipv4:147211/100=1472
test_xdp:PASS:ipv6:135977/100=1359 test_xdp:PASS:ipv6:85783/100=857
test_l4lb:PASS:ipv4:61324/100=613test_l4lb:PASS:ipv4:53222/100=532
test_l4lb:PASS:ipv6:100833/100=1008 test_l4lb:PASS:ipv6:76322/100=763
Yes, the numbers are pretty without turn on jit_harden, if we want to speedup
jit_harden, then we need to move BPF_REG_AX to *real* register instead of stack
emulation, but If we do it, we need to face all the pain I describe above. We
can do it in next step.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 26 +-
arch/x86/net/Makefile| 10 +-
arch/x86/net/bpf_jit32.S | 147 +++
arch/x86/net/bpf_jit_comp32.c| 2239 ++
5 files changed, 2419 insertions(+), 5 deletions(-)
create mode 100644 arch/x86/net/bpf_jit32.S
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 00fcf81..1f5fa2f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -137,7 +137,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_EXIT_THREAD
select HAVE_FENTRY if X86_64 || DY
[PATCH] bpf, x86_32: add eBPF JIT compiler for ia32 (x86_32)
The JIT compiler emits ia32 bit instructions. Currently, It supports
eBPF only. Classic BPF is supported because of the conversion by BPF core.
Almost all instructions from eBPF ISA supported except the following:
BPF_ALU64 | BPF_DIV | BPF_K
BPF_ALU64 | BPF_DIV | BPF_X
BPF_ALU64 | BPF_MOD | BPF_K
BPF_ALU64 | BPF_MOD | BPF_X
BPF_STX | BPF_XADD | BPF_W
BPF_STX | BPF_XADD | BPF_DW
It doesn't support BPF_JMP|BPF_CALL with BPF_PSEUDO_CALL too.
IA32 has few general purpose registers, EAX|EDX|ECX|EBX|ESI|EDI,
and for these six registers, we can't treat all of them as real
general purpose registers:
MUL instructions need EAX:EDX, shift instructions need ECX, ESI|EDI
for string manipulation instructions.
So I decide to use stack to emulate all eBPF 64 registers, this will
simplify the implementation very much, because we don't need to face
the flexible memory address modes on ia32, for example, we don't need
to write below codes for one BPF_ADD instruction:
if (src_reg is a register && dst_reg is a register)
{
//one instruction encoding for ADD instruction
} else if (only src is a register)
{
//another different instruction encoding for ADD instruction
} else if (only dst is a register)
{
//another different instruction encoding for ADD instruction
} else
{
//src and dst are all on stack.
//another different instruction encoding for ADD instruction
}
If you think above if-else-else-else isn't so painful, try to think
it for BPF_ALU64|BPF_*SHIFT* instruction:)
Tested on my PC(Intel(R) Core(TM) i5-5200U CPU) and virtualbox.
Testing results on i5-5200U:
1) test_bpf: Summary: 349 PASSED, 0 FAILED, [319/341 JIT'ed]
2) test_progs: Summary: 81 PASSED, 2 FAILED.
test_progs report "libbpf: incorrect bpf_call opcode" for
test_l4lb_noinline and test_xdp_noinline, because there is
no llvm-6.0 on my machine, and current implementation doesn't
support BPF_CALL, so I think we can ignore it.
3) test_lpm: OK
4) test_lru_map: OK
5) test_verifier: Summary: 823 PASSED, 5 FAILED
test_verifier report "invalid bpf_context access off=68 size=1/2/4/8"
for all the 5 FAILED testcases, and test_verifier report them when
we turn off the jit, so I think the jit can do nothing to fix them.
Above tests are all done with following flags enabled discretely:
bpf_jit_enable=1 and bpf_jit_harden=2
Below are some numbers for this jit implementation:
Note:
I run test_progs 100 times in loop for every testcase, the numbers
are in format: total/times=avg. The numbers that test_bpf report
almost show the same relation.
a:jit_enable=0 and jit_harden=0b:jit_enable=1 and jit_harden=0
test_pkt_access:PASS:ipv4:15622/100=156
test_pkt_access:PASS:ipv4:10057/100=100
test_pkt_access:PASS:ipv6:9130/100=91test_pkt_access:PASS:ipv6:5055/100=50
test_xdp:PASS:ipv4:240198/100=2401 test_xdp:PASS:ipv4:145945/100=1459
test_xdp:PASS:ipv6:137326/100=1373 test_xdp:PASS:ipv6:67337/100=673
test_l4lb:PASS:ipv4:61100/100=611test_l4lb:PASS:ipv4:38137/100=381
test_l4lb:PASS:ipv6:101000/100=1010 test_l4lb:PASS:ipv6:57779/100=577
c:jit_enable=0 and jit_harden=2b:jit_enable=1 and jit_harden=2
test_pkt_access:PASS:ipv4:15214/100=152
test_pkt_access:PASS:ipv4:12650/100=126
test_pkt_access:PASS:ipv6:9132/100=91test_pkt_access:PASS:ipv6:7074/100=70
test_xdp:PASS:ipv4:237252/100=2372 test_xdp:PASS:ipv4:147211/100=1472
test_xdp:PASS:ipv6:135977/100=1359 test_xdp:PASS:ipv6:85783/100=857
test_l4lb:PASS:ipv4:61324/100=613test_l4lb:PASS:ipv4:53222/100=532
test_l4lb:PASS:ipv6:100833/100=1008 test_l4lb:PASS:ipv6:76322/100=763
Yes, the numbers are pretty without turn on jit_harden, if we want to speedup
jit_harden, then we need to move BPF_REG_AX to *real* register instead of stack
emulation, but If we do it, we need to face all the pain I describe above. We
can do it in next step.
See Documentation/networking/filter.txt for more information.
Signed-off-by: Wang YanQing
---
arch/x86/Kconfig |2 +-
arch/x86/include/asm/nospec-branch.h | 26 +-
arch/x86/net/Makefile| 10 +-
arch/x86/net/bpf_jit32.S | 147 +++
arch/x86/net/bpf_jit_comp32.c| 2239 ++
5 files changed, 2419 insertions(+), 5 deletions(-)
create mode 100644 arch/x86/net/bpf_jit32.S
create mode 100644 arch/x86/net/bpf_jit_comp32.c
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 00fcf81..1f5fa2f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -137,7 +137,7 @@ config X86
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
- select HAVE_EBPF_JITif X86_64
+ select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_EXIT_THREAD
select HAVE_FENTRY if X86_64 || DYNAMIC_FTRACE
diff --git a
[tip:perf/core] perf report: Provide libtraceevent with a kernel symbol resolver
Commit-ID: ea85ab24c502720d2eb3dec30bedb6df06d4900b
Gitweb: https://git.kernel.org/tip/ea85ab24c502720d2eb3dec30bedb6df06d4900b
Author: Wang YanQing <udkni...@gmail.com>
AuthorDate: Thu, 8 Mar 2018 11:28:50 +0800
Committer: Arnaldo Carvalho de Melo <a...@redhat.com>
CommitDate: Thu, 8 Mar 2018 11:30:51 -0300
perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of "perf
kmem record", we will get kernel symbol output.
This patch affect the output of "perf report" for the record data
generated by "perf kmem record" looks like below:
Before patch:
0.01% call_site=814e5828 ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096
gfp_flags=GFP_ATOMIC
0.01% call_site=81370b87 ptr=0x428a3060 bytes_req=32 bytes_alloc=32
gfp_flags=GFP_KERNEL|GFP_ZERO
After patch:
0.01% (aa_alloc_task_context+0x27) call_site=81370b87 ptr=0x428a3060
bytes_req=32 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
0.01% (__tty_buffer_request_room+0x88) call_site=814e5828
ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096 gfp_flags=GFP_ATOMIC
Signed-off-by: Wang YanQing <udkni...@gmail.com>
Acked-by: Jiri Olsa <jo...@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <a...@redhat.com>
Cc: Alexander Shishkin <alexander.shish...@linux.intel.com>
Cc: Jiri Olsa <jo...@redhat.com>
Cc: Peter Zijlstra <pet...@infradead.org>
Link: http://lkml.kernel.org/r/20180308032850.GA12383@udknight-ThinkPad-E550
Signed-off-by: Arnaldo Carvalho de Melo <a...@redhat.com>
---
tools/perf/builtin-report.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index c3603d4c0c57..971ccba85464 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1348,6 +1348,15 @@ repeat:
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
+ __func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
[tip:perf/core] perf report: Provide libtraceevent with a kernel symbol resolver
Commit-ID: ea85ab24c502720d2eb3dec30bedb6df06d4900b
Gitweb: https://git.kernel.org/tip/ea85ab24c502720d2eb3dec30bedb6df06d4900b
Author: Wang YanQing
AuthorDate: Thu, 8 Mar 2018 11:28:50 +0800
Committer: Arnaldo Carvalho de Melo
CommitDate: Thu, 8 Mar 2018 11:30:51 -0300
perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of "perf
kmem record", we will get kernel symbol output.
This patch affect the output of "perf report" for the record data
generated by "perf kmem record" looks like below:
Before patch:
0.01% call_site=814e5828 ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096
gfp_flags=GFP_ATOMIC
0.01% call_site=81370b87 ptr=0x428a3060 bytes_req=32 bytes_alloc=32
gfp_flags=GFP_KERNEL|GFP_ZERO
After patch:
0.01% (aa_alloc_task_context+0x27) call_site=81370b87 ptr=0x428a3060
bytes_req=32 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
0.01% (__tty_buffer_request_room+0x88) call_site=814e5828
ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096 gfp_flags=GFP_ATOMIC
Signed-off-by: Wang YanQing
Acked-by: Jiri Olsa
Tested-by: Arnaldo Carvalho de Melo
Cc: Alexander Shishkin
Cc: Jiri Olsa
Cc: Peter Zijlstra
Link: http://lkml.kernel.org/r/20180308032850.GA12383@udknight-ThinkPad-E550
Signed-off-by: Arnaldo Carvalho de Melo
---
tools/perf/builtin-report.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index c3603d4c0c57..971ccba85464 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1348,6 +1348,15 @@ repeat:
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
+ __func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
[PATCH v3] perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of
"perf kmem record", we will get kernel symbol output.
This patch affect the output of "perf report" for the record data
generated by "perf kmem record" looks like below:
Before patch:
0.01% call_site=814e5828 ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096
gfp_flags=GFP_ATOMIC
0.01% call_site=81370b87 ptr=0x428a3060 bytes_req=32 bytes_alloc=32
gfp_flags=GFP_KERNEL|GFP_ZERO
After patch:
0.01% (aa_alloc_task_context+0x27) call_site=81370b87 ptr=0x428a3060
bytes_req=32 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
0.01% (__tty_buffer_request_room+0x88) call_site=814e5828
ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096 gfp_flags=GFP_ATOMIC
Signed-off-by: Wang YanQing <udkni...@gmail.com>
Acked-by: Jiri Olsa <jo...@kernel.org>
---
Changes v2-v3:
1: Fix one WARNING reported by ./scripts/checkpatch.pl
Note: the ./scripts/checkpatch.pl report "WARNING: Possible unwrapped commit
description"
for the output of "perf report", I think we can avoid that warning,
because the
original output of "perf report" is unwrapped.
Changes v1-v2:
1: Add a before and after output of tools affected by the patch in the
changelog.
2: Rebase the patch to last kernel version.
tools/perf/builtin-report.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 1eedb18..677ff58 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1337,6 +1337,15 @@ int cmd_report(int argc, const char **argv)
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
+ __func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v3] perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of
"perf kmem record", we will get kernel symbol output.
This patch affect the output of "perf report" for the record data
generated by "perf kmem record" looks like below:
Before patch:
0.01% call_site=814e5828 ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096
gfp_flags=GFP_ATOMIC
0.01% call_site=81370b87 ptr=0x428a3060 bytes_req=32 bytes_alloc=32
gfp_flags=GFP_KERNEL|GFP_ZERO
After patch:
0.01% (aa_alloc_task_context+0x27) call_site=81370b87 ptr=0x428a3060
bytes_req=32 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
0.01% (__tty_buffer_request_room+0x88) call_site=814e5828
ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096 gfp_flags=GFP_ATOMIC
Signed-off-by: Wang YanQing
Acked-by: Jiri Olsa
---
Changes v2-v3:
1: Fix one WARNING reported by ./scripts/checkpatch.pl
Note: the ./scripts/checkpatch.pl report "WARNING: Possible unwrapped commit
description"
for the output of "perf report", I think we can avoid that warning,
because the
original output of "perf report" is unwrapped.
Changes v1-v2:
1: Add a before and after output of tools affected by the patch in the
changelog.
2: Rebase the patch to last kernel version.
tools/perf/builtin-report.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 1eedb18..677ff58 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1337,6 +1337,15 @@ int cmd_report(int argc, const char **argv)
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
+ __func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of
"perf kmem record", we will get kernel symbol output.
This patch affect the output of "perf report" for the record data
generated by "perf kmem record" looks like below:
Before patch:
0.01% call_site=814e5828 ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096
gfp_flags=GFP_ATOMIC
0.01% call_site=81370b87 ptr=0x428a3060 bytes_req=32 bytes_alloc=32
gfp_flags=GFP_KERNEL|GFP_ZERO
After patch:
0.01% (aa_alloc_task_context+0x27) call_site=81370b87 ptr=0x428a3060
bytes_req=32 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
0.01% (__tty_buffer_request_room+0x88) call_site=814e5828
ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096 gfp_flags=GFP_ATOMIC
Signed-off-by: Wang YanQing <udkni...@gmail.com>
Acked-by: Jiri Olsa <jo...@kernel.org>
---
Changes v1-v2:
1: Add a before and after output of tools affected by the patch in the
changelog.
2: Rebase the patch to last kernel version.
tools/perf/builtin-report.c | 8
1 file changed, 8 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 1eedb18..d866398 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1337,6 +1337,14 @@ int cmd_report(int argc, const char **argv)
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
__func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of
"perf kmem record", we will get kernel symbol output.
This patch affect the output of "perf report" for the record data
generated by "perf kmem record" looks like below:
Before patch:
0.01% call_site=814e5828 ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096
gfp_flags=GFP_ATOMIC
0.01% call_site=81370b87 ptr=0x428a3060 bytes_req=32 bytes_alloc=32
gfp_flags=GFP_KERNEL|GFP_ZERO
After patch:
0.01% (aa_alloc_task_context+0x27) call_site=81370b87 ptr=0x428a3060
bytes_req=32 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
0.01% (__tty_buffer_request_room+0x88) call_site=814e5828
ptr=0x99bb000 bytes_req=3616 bytes_alloc=4096 gfp_flags=GFP_ATOMIC
Signed-off-by: Wang YanQing
Acked-by: Jiri Olsa
---
Changes v1-v2:
1: Add a before and after output of tools affected by the patch in the
changelog.
2: Rebase the patch to last kernel version.
tools/perf/builtin-report.c | 8
1 file changed, 8 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 1eedb18..d866398 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1337,6 +1337,14 @@ int cmd_report(int argc, const char **argv)
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
__func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
--
1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Thu, Feb 08, 2018 at 01:20:31PM +0100, Jiri Olsa wrote: > On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote: > > So that beautifiers wanting to resolve kernel function addresses to > > names can do its work, and when we use "perf report" for output of > > "perf kmem record", we will get kernel symbol output. > > > > Signed-off-by: Wang YanQing <udkni...@gmail.com> > > Acked-by: Jiri Olsa <jo...@kernel.org> Hi! Arnaldo Carvalho de Melo What is the status of this patch now? Does the patch sanked to the bottom of your mailbox? Thanks!
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Thu, Feb 08, 2018 at 01:20:31PM +0100, Jiri Olsa wrote: > On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote: > > So that beautifiers wanting to resolve kernel function addresses to > > names can do its work, and when we use "perf report" for output of > > "perf kmem record", we will get kernel symbol output. > > > > Signed-off-by: Wang YanQing > > Acked-by: Jiri Olsa Hi! Arnaldo Carvalho de Melo What is the status of this patch now? Does the patch sanked to the bottom of your mailbox? Thanks!
[PATCH] bpf, doc: Correct one wrong value in "Register value tracking"
If we then OR this with 0x40, then the value of 6th bit (0th is first bit) become known, so the right mask is 0xbf instead of 0xcf. Signed-off-by: Wang YanQing <udkni...@gmail.com> --- Documentation/networking/filter.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt index 8781485..a4508ec 100644 --- a/Documentation/networking/filter.txt +++ b/Documentation/networking/filter.txt @@ -1134,7 +1134,7 @@ The verifier's knowledge about the variable offset consists of: mask and value; no bit should ever be 1 in both. For example, if a byte is read into a register from memory, the register's top 56 bits are known zero, while the low 8 are unknown - which is represented as the tnum (0x0; 0xff). If we -then OR this with 0x40, we get (0x40; 0xcf), then if we add 1 we get (0x0; +then OR this with 0x40, we get (0x40; 0xbf), then if we add 1 we get (0x0; 0x1ff), because of potential carries. Besides arithmetic, the register state can also be updated by conditional branches. For instance, if a SCALAR_VALUE is compared > 8, in the 'true' branch -- 1.8.5.6.2.g3d8a54e.dirty
[PATCH] bpf, doc: Correct one wrong value in "Register value tracking"
If we then OR this with 0x40, then the value of 6th bit (0th is first bit) become known, so the right mask is 0xbf instead of 0xcf. Signed-off-by: Wang YanQing --- Documentation/networking/filter.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt index 8781485..a4508ec 100644 --- a/Documentation/networking/filter.txt +++ b/Documentation/networking/filter.txt @@ -1134,7 +1134,7 @@ The verifier's knowledge about the variable offset consists of: mask and value; no bit should ever be 1 in both. For example, if a byte is read into a register from memory, the register's top 56 bits are known zero, while the low 8 are unknown - which is represented as the tnum (0x0; 0xff). If we -then OR this with 0x40, we get (0x40; 0xcf), then if we add 1 we get (0x0; +then OR this with 0x40, we get (0x40; 0xbf), then if we add 1 we get (0x0; 0x1ff), because of potential carries. Besides arithmetic, the register state can also be updated by conditional branches. For instance, if a SCALAR_VALUE is compared > 8, in the 'true' branch -- 1.8.5.6.2.g3d8a54e.dirty
[tip:perf/core] perf symbols: Using O_CLOEXEC in do_open
Commit-ID: 4c0d8d27954d9efb2a02ec9fc16f39b02f248bb7
Gitweb: https://git.kernel.org/tip/4c0d8d27954d9efb2a02ec9fc16f39b02f248bb7
Author: Wang YanQing <udkni...@gmail.com>
AuthorDate: Mon, 15 Jan 2018 13:04:48 +0800
Committer: Arnaldo Carvalho de Melo <a...@redhat.com>
CommitDate: Tue, 23 Jan 2018 09:49:28 -0300
perf symbols: Using O_CLOEXEC in do_open
I've meet a strange behavior with these commands on my gentoo box:
1: perf kmem record
2: CTRL-C to stop 1
3: perf report
4: "Enter", "Enter", "Run scripts for all samples",
"event_analyzing_sample".
Then 'perf report' says:
"
No kallsyms or vmlinux with build-id was found
/lib/modules/4.10.0+/build/vmlinux with build id not found,
continuing without symbols
".
It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux is
right for perf.data.
After digging, I found out the reason is that "perf report" generates
many open fds, then "script_browse" uses popen to run "perf script"
which run out of open files.
The gentoo box has a small default value for "max open files", 1024.
Yes, "ulimit -n " with a bigger number could fix it, but I think that
using O_CLOEXEC in do_open is a better way.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
Acked-by: Jiri Olsa <jo...@kernel.org>
Cc: Alexander Shishkin <alexander.shish...@linux.intel.com>
Cc: Peter Zijlstra <pet...@infradead.org>
Link: http://lkml.kernel.org/r/20180115050448.GA20759@udknight
[ Make sure O_CLOEXEC is available in old systems by adding a patch
just before this one, to keep this bisectable in such systems ]
Signed-off-by: Arnaldo Carvalho de Melo <a...@redhat.com>
---
tools/perf/util/dso.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index d5b6f7f..36ef45b 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -446,7 +446,7 @@ static int do_open(char *name)
char sbuf[STRERR_BUFSIZE];
do {
- fd = open(name, O_RDONLY);
+ fd = open(name, O_RDONLY|O_CLOEXEC);
if (fd >= 0)
return fd;
[tip:perf/core] perf symbols: Using O_CLOEXEC in do_open
Commit-ID: 4c0d8d27954d9efb2a02ec9fc16f39b02f248bb7
Gitweb: https://git.kernel.org/tip/4c0d8d27954d9efb2a02ec9fc16f39b02f248bb7
Author: Wang YanQing
AuthorDate: Mon, 15 Jan 2018 13:04:48 +0800
Committer: Arnaldo Carvalho de Melo
CommitDate: Tue, 23 Jan 2018 09:49:28 -0300
perf symbols: Using O_CLOEXEC in do_open
I've meet a strange behavior with these commands on my gentoo box:
1: perf kmem record
2: CTRL-C to stop 1
3: perf report
4: "Enter", "Enter", "Run scripts for all samples",
"event_analyzing_sample".
Then 'perf report' says:
"
No kallsyms or vmlinux with build-id was found
/lib/modules/4.10.0+/build/vmlinux with build id not found,
continuing without symbols
".
It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux is
right for perf.data.
After digging, I found out the reason is that "perf report" generates
many open fds, then "script_browse" uses popen to run "perf script"
which run out of open files.
The gentoo box has a small default value for "max open files", 1024.
Yes, "ulimit -n " with a bigger number could fix it, but I think that
using O_CLOEXEC in do_open is a better way.
Signed-off-by: Wang YanQing
Acked-by: Jiri Olsa
Cc: Alexander Shishkin
Cc: Peter Zijlstra
Link: http://lkml.kernel.org/r/20180115050448.GA20759@udknight
[ Make sure O_CLOEXEC is available in old systems by adding a patch
just before this one, to keep this bisectable in such systems ]
Signed-off-by: Arnaldo Carvalho de Melo
---
tools/perf/util/dso.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index d5b6f7f..36ef45b 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -446,7 +446,7 @@ static int do_open(char *name)
char sbuf[STRERR_BUFSIZE];
do {
- fd = open(name, O_RDONLY);
+ fd = open(name, O_RDONLY|O_CLOEXEC);
if (fd >= 0)
return fd;
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Fri, Jan 19, 2018 at 07:57:59PM +0100, Jiri Olsa wrote:
> On Thu, Jan 18, 2018 at 10:22:30PM +0800, Wang YanQing wrote:
> > On Thu, Jan 18, 2018 at 10:43:18AM +0100, Jiri Olsa wrote:
> > > On Wed, Jan 17, 2018 at 12:48:12AM +0800, Wang YanQing wrote:
> > > > On Mon, Jan 15, 2018 at 11:06:11AM +0100, Jiri Olsa wrote:
> > > > > On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote:
> > > > > > So that beautifiers wanting to resolve kernel function addresses to
> > > > > > names can do its work, and when we use "perf report" for output of
> > > > > > "perf kmem record", we will get kernel symbol output.
> > > > > >
> > > > > > Signed-off-by: Wang YanQing <udkni...@gmail.com>
> > > > > > ---
> > > > > > tools/perf/builtin-report.c | 9 +
> > > > > > 1 file changed, 9 insertions(+)
> > > > > >
> > > > > > diff --git a/tools/perf/builtin-report.c
> > > > > > b/tools/perf/builtin-report.c
> > > > > > index dd4df9a..7b65100 100644
> > > > > > --- a/tools/perf/builtin-report.c
> > > > > > +++ b/tools/perf/builtin-report.c
> > > > > > @@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
> > > > > > report.range_num = 1;
> > > > > > }
> > > > > >
> > > > > > + if (session->tevent.pevent &&
> > > > > > + pevent_set_function_resolver(session->tevent.pevent,
> > > > > > +machine__resolve_kernel_addr,
> > > > > > +>machines.host) < 0) {
> > > > > > + pr_err("%s: failed to set libtraceevent function
> > > > > > resolver\n",
> > > > > > + __func__);
> > > > > > + return -1;
> > > > > > + }
> > > > >
> > > > > why not calling the wrapper trace_event__register_resolver?
> > > > The reason is the same as builtin-script doesn't use
> > > > trace_event__register_resolver,
> > > > because we only use report and script to analyze offline perf.data, and
> > > > there could
> > > > be no tracepoints in perf.data.
> > >
> > > hum, I missed this functionality.. so we need this even if there
> > > are no tracepoints in the perf.data?
> >
> > In "perf report", when there are tracepoints in perf.data,
> > session->tevent.pevent
> > will be initialized in trace-event-read.c:trace_report, then "if
> > (session->tevent.pevent"
> > will become true, and we should calling pevent_set_function_resolver.
> >
> > But if we calling trace_event__register_resolver, then it will initialize
> > tevent.pevent
> > no matter whether there are tracepoints in perf.data.
>
> ok, should we call it from perf_session__read_header then?
> below perf_evlist__prepare_tracepoint_events perhaps
No, I think we can't do it in perf_session__read_header,
because perf_session__new calls perf_session__create_kernel_maps
to initialize machines.host after calling perf_session__open, and
pevent_set_function_resolver needs it.
Although it is possible to place pevent_set_function_resolver at
the tail of perf_session__new, but I don't think it is a better
choice for us, because tools like "perf kmem stat" doesn't need it,
"perf kmem stat" has itself way to prepare and display the symbols
of callsites.
I think the current calling place of pevent_set_function_resolver
for "perf report" is the right place.
Thanks.
> jirka
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Fri, Jan 19, 2018 at 07:57:59PM +0100, Jiri Olsa wrote:
> On Thu, Jan 18, 2018 at 10:22:30PM +0800, Wang YanQing wrote:
> > On Thu, Jan 18, 2018 at 10:43:18AM +0100, Jiri Olsa wrote:
> > > On Wed, Jan 17, 2018 at 12:48:12AM +0800, Wang YanQing wrote:
> > > > On Mon, Jan 15, 2018 at 11:06:11AM +0100, Jiri Olsa wrote:
> > > > > On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote:
> > > > > > So that beautifiers wanting to resolve kernel function addresses to
> > > > > > names can do its work, and when we use "perf report" for output of
> > > > > > "perf kmem record", we will get kernel symbol output.
> > > > > >
> > > > > > Signed-off-by: Wang YanQing
> > > > > > ---
> > > > > > tools/perf/builtin-report.c | 9 +
> > > > > > 1 file changed, 9 insertions(+)
> > > > > >
> > > > > > diff --git a/tools/perf/builtin-report.c
> > > > > > b/tools/perf/builtin-report.c
> > > > > > index dd4df9a..7b65100 100644
> > > > > > --- a/tools/perf/builtin-report.c
> > > > > > +++ b/tools/perf/builtin-report.c
> > > > > > @@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
> > > > > > report.range_num = 1;
> > > > > > }
> > > > > >
> > > > > > + if (session->tevent.pevent &&
> > > > > > + pevent_set_function_resolver(session->tevent.pevent,
> > > > > > +machine__resolve_kernel_addr,
> > > > > > +>machines.host) < 0) {
> > > > > > + pr_err("%s: failed to set libtraceevent function
> > > > > > resolver\n",
> > > > > > + __func__);
> > > > > > + return -1;
> > > > > > + }
> > > > >
> > > > > why not calling the wrapper trace_event__register_resolver?
> > > > The reason is the same as builtin-script doesn't use
> > > > trace_event__register_resolver,
> > > > because we only use report and script to analyze offline perf.data, and
> > > > there could
> > > > be no tracepoints in perf.data.
> > >
> > > hum, I missed this functionality.. so we need this even if there
> > > are no tracepoints in the perf.data?
> >
> > In "perf report", when there are tracepoints in perf.data,
> > session->tevent.pevent
> > will be initialized in trace-event-read.c:trace_report, then "if
> > (session->tevent.pevent"
> > will become true, and we should calling pevent_set_function_resolver.
> >
> > But if we calling trace_event__register_resolver, then it will initialize
> > tevent.pevent
> > no matter whether there are tracepoints in perf.data.
>
> ok, should we call it from perf_session__read_header then?
> below perf_evlist__prepare_tracepoint_events perhaps
No, I think we can't do it in perf_session__read_header,
because perf_session__new calls perf_session__create_kernel_maps
to initialize machines.host after calling perf_session__open, and
pevent_set_function_resolver needs it.
Although it is possible to place pevent_set_function_resolver at
the tail of perf_session__new, but I don't think it is a better
choice for us, because tools like "perf kmem stat" doesn't need it,
"perf kmem stat" has itself way to prepare and display the symbols
of callsites.
I think the current calling place of pevent_set_function_resolver
for "perf report" is the right place.
Thanks.
> jirka
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Thu, Jan 18, 2018 at 10:43:18AM +0100, Jiri Olsa wrote:
> On Wed, Jan 17, 2018 at 12:48:12AM +0800, Wang YanQing wrote:
> > On Mon, Jan 15, 2018 at 11:06:11AM +0100, Jiri Olsa wrote:
> > > On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote:
> > > > So that beautifiers wanting to resolve kernel function addresses to
> > > > names can do its work, and when we use "perf report" for output of
> > > > "perf kmem record", we will get kernel symbol output.
> > > >
> > > > Signed-off-by: Wang YanQing <udkni...@gmail.com>
> > > > ---
> > > > tools/perf/builtin-report.c | 9 +
> > > > 1 file changed, 9 insertions(+)
> > > >
> > > > diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
> > > > index dd4df9a..7b65100 100644
> > > > --- a/tools/perf/builtin-report.c
> > > > +++ b/tools/perf/builtin-report.c
> > > > @@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
> > > > report.range_num = 1;
> > > > }
> > > >
> > > > + if (session->tevent.pevent &&
> > > > + pevent_set_function_resolver(session->tevent.pevent,
> > > > +machine__resolve_kernel_addr,
> > > > +>machines.host) < 0) {
> > > > + pr_err("%s: failed to set libtraceevent function
> > > > resolver\n",
> > > > + __func__);
> > > > + return -1;
> > > > + }
> > >
> > > why not calling the wrapper trace_event__register_resolver?
> > The reason is the same as builtin-script doesn't use
> > trace_event__register_resolver,
> > because we only use report and script to analyze offline perf.data, and
> > there could
> > be no tracepoints in perf.data.
>
> hum, I missed this functionality.. so we need this even if there
> are no tracepoints in the perf.data?
In "perf report", when there are tracepoints in perf.data,
session->tevent.pevent
will be initialized in trace-event-read.c:trace_report, then "if
(session->tevent.pevent"
will become true, and we should calling pevent_set_function_resolver.
But if we calling trace_event__register_resolver, then it will initialize
tevent.pevent
no matter whether there are tracepoints in perf.data.
Thanks.
>
> The only related interface function I could found is pevent_find_function
> and it seems to be called only from within traceevent library.
>
> thanks,
> jirka
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Thu, Jan 18, 2018 at 10:43:18AM +0100, Jiri Olsa wrote:
> On Wed, Jan 17, 2018 at 12:48:12AM +0800, Wang YanQing wrote:
> > On Mon, Jan 15, 2018 at 11:06:11AM +0100, Jiri Olsa wrote:
> > > On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote:
> > > > So that beautifiers wanting to resolve kernel function addresses to
> > > > names can do its work, and when we use "perf report" for output of
> > > > "perf kmem record", we will get kernel symbol output.
> > > >
> > > > Signed-off-by: Wang YanQing
> > > > ---
> > > > tools/perf/builtin-report.c | 9 +
> > > > 1 file changed, 9 insertions(+)
> > > >
> > > > diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
> > > > index dd4df9a..7b65100 100644
> > > > --- a/tools/perf/builtin-report.c
> > > > +++ b/tools/perf/builtin-report.c
> > > > @@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
> > > > report.range_num = 1;
> > > > }
> > > >
> > > > + if (session->tevent.pevent &&
> > > > + pevent_set_function_resolver(session->tevent.pevent,
> > > > +machine__resolve_kernel_addr,
> > > > +>machines.host) < 0) {
> > > > + pr_err("%s: failed to set libtraceevent function
> > > > resolver\n",
> > > > + __func__);
> > > > + return -1;
> > > > + }
> > >
> > > why not calling the wrapper trace_event__register_resolver?
> > The reason is the same as builtin-script doesn't use
> > trace_event__register_resolver,
> > because we only use report and script to analyze offline perf.data, and
> > there could
> > be no tracepoints in perf.data.
>
> hum, I missed this functionality.. so we need this even if there
> are no tracepoints in the perf.data?
In "perf report", when there are tracepoints in perf.data,
session->tevent.pevent
will be initialized in trace-event-read.c:trace_report, then "if
(session->tevent.pevent"
will become true, and we should calling pevent_set_function_resolver.
But if we calling trace_event__register_resolver, then it will initialize
tevent.pevent
no matter whether there are tracepoints in perf.data.
Thanks.
>
> The only related interface function I could found is pevent_find_function
> and it seems to be called only from within traceevent library.
>
> thanks,
> jirka
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Mon, Jan 15, 2018 at 11:06:11AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote:
> > So that beautifiers wanting to resolve kernel function addresses to
> > names can do its work, and when we use "perf report" for output of
> > "perf kmem record", we will get kernel symbol output.
> >
> > Signed-off-by: Wang YanQing <udkni...@gmail.com>
> > ---
> > tools/perf/builtin-report.c | 9 +
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
> > index dd4df9a..7b65100 100644
> > --- a/tools/perf/builtin-report.c
> > +++ b/tools/perf/builtin-report.c
> > @@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
> > report.range_num = 1;
> > }
> >
> > + if (session->tevent.pevent &&
> > + pevent_set_function_resolver(session->tevent.pevent,
> > +machine__resolve_kernel_addr,
> > +>machines.host) < 0) {
> > + pr_err("%s: failed to set libtraceevent function resolver\n",
> > + __func__);
> > + return -1;
> > + }
>
> why not calling the wrapper trace_event__register_resolver?
The reason is the same as builtin-script doesn't use
trace_event__register_resolver,
because we only use report and script to analyze offline perf.data, and there
could
be no tracepoints in perf.data.
The wrapper trace_event__register_resolver always initialize tevent.pevent, and
calling
pevent_set_function_resolver which isn't necessary when there is no tracepoints
in the
perf.data.
>
> jirka
>
> > +
> > sort__setup_elide(stdout);
> >
> > ret = __cmd_report();
> > --
> > 1.8.5.6.2.g3d8a54e.dirty
> >
Re: [PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
On Mon, Jan 15, 2018 at 11:06:11AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 12:47:32PM +0800, Wang YanQing wrote:
> > So that beautifiers wanting to resolve kernel function addresses to
> > names can do its work, and when we use "perf report" for output of
> > "perf kmem record", we will get kernel symbol output.
> >
> > Signed-off-by: Wang YanQing
> > ---
> > tools/perf/builtin-report.c | 9 +
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
> > index dd4df9a..7b65100 100644
> > --- a/tools/perf/builtin-report.c
> > +++ b/tools/perf/builtin-report.c
> > @@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
> > report.range_num = 1;
> > }
> >
> > + if (session->tevent.pevent &&
> > + pevent_set_function_resolver(session->tevent.pevent,
> > +machine__resolve_kernel_addr,
> > +>machines.host) < 0) {
> > + pr_err("%s: failed to set libtraceevent function resolver\n",
> > + __func__);
> > + return -1;
> > + }
>
> why not calling the wrapper trace_event__register_resolver?
The reason is the same as builtin-script doesn't use
trace_event__register_resolver,
because we only use report and script to analyze offline perf.data, and there
could
be no tracepoints in perf.data.
The wrapper trace_event__register_resolver always initialize tevent.pevent, and
calling
pevent_set_function_resolver which isn't necessary when there is no tracepoints
in the
perf.data.
>
> jirka
>
> > +
> > sort__setup_elide(stdout);
> >
> > ret = __cmd_report();
> > --
> > 1.8.5.6.2.g3d8a54e.dirty
> >
Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open
On Mon, Jan 15, 2018 at 10:45:06AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> > I meet strange behavior with below commands on my gentoo box:
> > 1:perf kmem record
> > 2:CTRL-C to stop 1
> > 3:perf report
> > 4:"Enter", "Enter", "Run scripts for all samples",
> > "event_analyzing_sample".
> >
> > Then perf report:
> > "
> > No kallsyms or vmlinux with build-id was found
> > /lib/modules/4.10.0+/build/vmlinux with build id not found,
> > continuing without symbols
> > ".
> >
> > It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> > is right for perf.data.
> >
> > After digging, I find out the reason is "perf report" generates many
> > open fds, then "script_browse" uses popen to run "perf script" which
> > run out of open files. The gentoo box has a small default value for
> > "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> > fix it, but I think that using O_CLOEXEC in do_open is a better way.
>
> that seems right.. so why does kmem record open those
> files before calling cmd_record? should that be fixed?
The problem is "perf report", isn't "perf kmem record",
see the above reproduction steps again.
The problem is "perf report" generates may open fds with
dso:do_open for symbol analysis, and "perf report" uses
popen to run another command "perf script", then the cmd
"perf script" will run out of "max open files".
Thanks.
>
> thanks,
> jirka
>
> >
> > Signed-off-by: Wang YanQing <udkni...@gmail.com>
> > ---
> > tools/perf/util/dso.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> > index d5b6f7f..36ef45b 100644
> > --- a/tools/perf/util/dso.c
> > +++ b/tools/perf/util/dso.c
> > @@ -446,7 +446,7 @@ static int do_open(char *name)
> > char sbuf[STRERR_BUFSIZE];
> >
> > do {
> > - fd = open(name, O_RDONLY);
> > + fd = open(name, O_RDONLY|O_CLOEXEC);
> > if (fd >= 0)
> > return fd;
> >
> > --
> > 1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open
On Mon, Jan 15, 2018 at 10:45:06AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> > I meet strange behavior with below commands on my gentoo box:
> > 1:perf kmem record
> > 2:CTRL-C to stop 1
> > 3:perf report
> > 4:"Enter", "Enter", "Run scripts for all samples",
> > "event_analyzing_sample".
> >
> > Then perf report:
> > "
> > No kallsyms or vmlinux with build-id was found
> > /lib/modules/4.10.0+/build/vmlinux with build id not found,
> > continuing without symbols
> > ".
> >
> > It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> > is right for perf.data.
> >
> > After digging, I find out the reason is "perf report" generates many
> > open fds, then "script_browse" uses popen to run "perf script" which
> > run out of open files. The gentoo box has a small default value for
> > "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> > fix it, but I think that using O_CLOEXEC in do_open is a better way.
>
> that seems right.. so why does kmem record open those
> files before calling cmd_record? should that be fixed?
The problem is "perf report", isn't "perf kmem record",
see the above reproduction steps again.
The problem is "perf report" generates may open fds with
dso:do_open for symbol analysis, and "perf report" uses
popen to run another command "perf script", then the cmd
"perf script" will run out of "max open files".
Thanks.
>
> thanks,
> jirka
>
> >
> > Signed-off-by: Wang YanQing
> > ---
> > tools/perf/util/dso.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> > index d5b6f7f..36ef45b 100644
> > --- a/tools/perf/util/dso.c
> > +++ b/tools/perf/util/dso.c
> > @@ -446,7 +446,7 @@ static int do_open(char *name)
> > char sbuf[STRERR_BUFSIZE];
> >
> > do {
> > - fd = open(name, O_RDONLY);
> > + fd = open(name, O_RDONLY|O_CLOEXEC);
> > if (fd >= 0)
> > return fd;
> >
> > --
> > 1.8.5.6.2.g3d8a54e.dirty
[PATCH] perf:util:dso: Using O_CLOEXEC in do_open
I meet strange behavior with below commands on my gentoo box:
1:perf kmem record
2:CTRL-C to stop 1
3:perf report
4:"Enter", "Enter", "Run scripts for all samples",
"event_analyzing_sample".
Then perf report:
"
No kallsyms or vmlinux with build-id was found
/lib/modules/4.10.0+/build/vmlinux with build id not found,
continuing without symbols
".
It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
is right for perf.data.
After digging, I find out the reason is "perf report" generates many
open fds, then "script_browse" uses popen to run "perf script" which
run out of open files. The gentoo box has a small default value for
"max open files", 1024. Yes, "ulimit -n " with a bigger number could
fix it, but I think that using O_CLOEXEC in do_open is a better way.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
tools/perf/util/dso.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index d5b6f7f..36ef45b 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -446,7 +446,7 @@ static int do_open(char *name)
char sbuf[STRERR_BUFSIZE];
do {
- fd = open(name, O_RDONLY);
+ fd = open(name, O_RDONLY|O_CLOEXEC);
if (fd >= 0)
return fd;
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] perf:util:dso: Using O_CLOEXEC in do_open
I meet strange behavior with below commands on my gentoo box:
1:perf kmem record
2:CTRL-C to stop 1
3:perf report
4:"Enter", "Enter", "Run scripts for all samples",
"event_analyzing_sample".
Then perf report:
"
No kallsyms or vmlinux with build-id was found
/lib/modules/4.10.0+/build/vmlinux with build id not found,
continuing without symbols
".
It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
is right for perf.data.
After digging, I find out the reason is "perf report" generates many
open fds, then "script_browse" uses popen to run "perf script" which
run out of open files. The gentoo box has a small default value for
"max open files", 1024. Yes, "ulimit -n " with a bigger number could
fix it, but I think that using O_CLOEXEC in do_open is a better way.
Signed-off-by: Wang YanQing
---
tools/perf/util/dso.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index d5b6f7f..36ef45b 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -446,7 +446,7 @@ static int do_open(char *name)
char sbuf[STRERR_BUFSIZE];
do {
- fd = open(name, O_RDONLY);
+ fd = open(name, O_RDONLY|O_CLOEXEC);
if (fd >= 0)
return fd;
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] tools/lib/traceevent/event-parse: delete pevent_register_function
After commit 4263cece22e3da94f16fbbcf71ce3807946d3ef3
("perf tools: Stop reading the kallsyms data from perf.data"),
there is no users of pevent_register_function in tree, so we
could just delete it.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
tools/lib/traceevent/event-parse.c | 183 +
tools/lib/traceevent/event-parse.h | 3 -
2 files changed, 1 insertion(+), 185 deletions(-)
diff --git a/tools/lib/traceevent/event-parse.c
b/tools/lib/traceevent/event-parse.c
index 7ce724f..0d125a3 100644
--- a/tools/lib/traceevent/event-parse.c
+++ b/tools/lib/traceevent/event-parse.c
@@ -347,96 +347,6 @@ struct func_list {
char*mod;
};
-static int func_cmp(const void *a, const void *b)
-{
- const struct func_map *fa = a;
- const struct func_map *fb = b;
-
- if (fa->addr < fb->addr)
- return -1;
- if (fa->addr > fb->addr)
- return 1;
-
- return 0;
-}
-
-/*
- * We are searching for a record in between, not an exact
- * match.
- */
-static int func_bcmp(const void *a, const void *b)
-{
- const struct func_map *fa = a;
- const struct func_map *fb = b;
-
- if ((fa->addr == fb->addr) ||
-
- (fa->addr > fb->addr &&
-fa->addr < (fb+1)->addr))
- return 0;
-
- if (fa->addr < fb->addr)
- return -1;
-
- return 1;
-}
-
-static int func_map_init(struct pevent *pevent)
-{
- struct func_list *funclist;
- struct func_list *item;
- struct func_map *func_map;
- int i;
-
- func_map = malloc(sizeof(*func_map) * (pevent->func_count + 1));
- if (!func_map)
- return -1;
-
- funclist = pevent->funclist;
-
- i = 0;
- while (funclist) {
- func_map[i].func = funclist->func;
- func_map[i].addr = funclist->addr;
- func_map[i].mod = funclist->mod;
- i++;
- item = funclist;
- funclist = funclist->next;
- free(item);
- }
-
- qsort(func_map, pevent->func_count, sizeof(*func_map), func_cmp);
-
- /*
-* Add a special record at the end.
-*/
- func_map[pevent->func_count].func = NULL;
- func_map[pevent->func_count].addr = 0;
- func_map[pevent->func_count].mod = NULL;
-
- pevent->func_map = func_map;
- pevent->funclist = NULL;
-
- return 0;
-}
-
-static struct func_map *
-__find_func(struct pevent *pevent, unsigned long long addr)
-{
- struct func_map *func;
- struct func_map key;
-
- if (!pevent->func_map)
- func_map_init(pevent);
-
- key.addr = addr;
-
- func = bsearch(, pevent->func_map, pevent->func_count,
- sizeof(*pevent->func_map), func_bcmp);
-
- return func;
-}
-
struct func_resolver {
pevent_func_resolver_t *func;
void *priv;
@@ -448,10 +358,6 @@ struct func_resolver {
* @pevent: handle for the pevent
* @resolver: function to be used
* @priv: resolver function private state.
- *
- * Some tools may have already a way to resolve kernel functions, allow them to
- * keep using it instead of duplicating all the entries inside
- * pevent->funclist.
*/
int pevent_set_function_resolver(struct pevent *pevent,
pevent_func_resolver_t *func, void *priv)
@@ -489,7 +395,7 @@ void pevent_reset_function_resolver(struct pevent *pevent)
struct func_map *map;
if (!pevent->func_resolver)
- return __find_func(pevent, addr);
+ return NULL;
map = >func_resolver->map;
map->mod = NULL;
@@ -543,75 +449,6 @@ const char *pevent_find_function(struct pevent *pevent,
unsigned long long addr)
return map->addr;
}
-/**
- * pevent_register_function - register a function with a given address
- * @pevent: handle for the pevent
- * @function: the function name to register
- * @addr: the address the function starts at
- * @mod: the kernel module the function may be in (NULL for none)
- *
- * This registers a function name with an address and module.
- * The @func passed in is duplicated.
- */
-int pevent_register_function(struct pevent *pevent, char *func,
-unsigned long long addr, char *mod)
-{
- struct func_list *item = malloc(sizeof(*item));
-
- if (!item)
- return -1;
-
- item->next = pevent->funclist;
- item->func = strdup(func);
- if (!item->func)
- goto out_free;
-
- if (mod) {
- item->mod = strdup(mod);
- if (!item->mod)
- goto out_free_func;
- } else
- item->mod = NULL;
- item->ad
[PATCH] tools/lib/traceevent/event-parse: delete pevent_register_function
After commit 4263cece22e3da94f16fbbcf71ce3807946d3ef3
("perf tools: Stop reading the kallsyms data from perf.data"),
there is no users of pevent_register_function in tree, so we
could just delete it.
Signed-off-by: Wang YanQing
---
tools/lib/traceevent/event-parse.c | 183 +
tools/lib/traceevent/event-parse.h | 3 -
2 files changed, 1 insertion(+), 185 deletions(-)
diff --git a/tools/lib/traceevent/event-parse.c
b/tools/lib/traceevent/event-parse.c
index 7ce724f..0d125a3 100644
--- a/tools/lib/traceevent/event-parse.c
+++ b/tools/lib/traceevent/event-parse.c
@@ -347,96 +347,6 @@ struct func_list {
char*mod;
};
-static int func_cmp(const void *a, const void *b)
-{
- const struct func_map *fa = a;
- const struct func_map *fb = b;
-
- if (fa->addr < fb->addr)
- return -1;
- if (fa->addr > fb->addr)
- return 1;
-
- return 0;
-}
-
-/*
- * We are searching for a record in between, not an exact
- * match.
- */
-static int func_bcmp(const void *a, const void *b)
-{
- const struct func_map *fa = a;
- const struct func_map *fb = b;
-
- if ((fa->addr == fb->addr) ||
-
- (fa->addr > fb->addr &&
-fa->addr < (fb+1)->addr))
- return 0;
-
- if (fa->addr < fb->addr)
- return -1;
-
- return 1;
-}
-
-static int func_map_init(struct pevent *pevent)
-{
- struct func_list *funclist;
- struct func_list *item;
- struct func_map *func_map;
- int i;
-
- func_map = malloc(sizeof(*func_map) * (pevent->func_count + 1));
- if (!func_map)
- return -1;
-
- funclist = pevent->funclist;
-
- i = 0;
- while (funclist) {
- func_map[i].func = funclist->func;
- func_map[i].addr = funclist->addr;
- func_map[i].mod = funclist->mod;
- i++;
- item = funclist;
- funclist = funclist->next;
- free(item);
- }
-
- qsort(func_map, pevent->func_count, sizeof(*func_map), func_cmp);
-
- /*
-* Add a special record at the end.
-*/
- func_map[pevent->func_count].func = NULL;
- func_map[pevent->func_count].addr = 0;
- func_map[pevent->func_count].mod = NULL;
-
- pevent->func_map = func_map;
- pevent->funclist = NULL;
-
- return 0;
-}
-
-static struct func_map *
-__find_func(struct pevent *pevent, unsigned long long addr)
-{
- struct func_map *func;
- struct func_map key;
-
- if (!pevent->func_map)
- func_map_init(pevent);
-
- key.addr = addr;
-
- func = bsearch(, pevent->func_map, pevent->func_count,
- sizeof(*pevent->func_map), func_bcmp);
-
- return func;
-}
-
struct func_resolver {
pevent_func_resolver_t *func;
void *priv;
@@ -448,10 +358,6 @@ struct func_resolver {
* @pevent: handle for the pevent
* @resolver: function to be used
* @priv: resolver function private state.
- *
- * Some tools may have already a way to resolve kernel functions, allow them to
- * keep using it instead of duplicating all the entries inside
- * pevent->funclist.
*/
int pevent_set_function_resolver(struct pevent *pevent,
pevent_func_resolver_t *func, void *priv)
@@ -489,7 +395,7 @@ void pevent_reset_function_resolver(struct pevent *pevent)
struct func_map *map;
if (!pevent->func_resolver)
- return __find_func(pevent, addr);
+ return NULL;
map = >func_resolver->map;
map->mod = NULL;
@@ -543,75 +449,6 @@ const char *pevent_find_function(struct pevent *pevent,
unsigned long long addr)
return map->addr;
}
-/**
- * pevent_register_function - register a function with a given address
- * @pevent: handle for the pevent
- * @function: the function name to register
- * @addr: the address the function starts at
- * @mod: the kernel module the function may be in (NULL for none)
- *
- * This registers a function name with an address and module.
- * The @func passed in is duplicated.
- */
-int pevent_register_function(struct pevent *pevent, char *func,
-unsigned long long addr, char *mod)
-{
- struct func_list *item = malloc(sizeof(*item));
-
- if (!item)
- return -1;
-
- item->next = pevent->funclist;
- item->func = strdup(func);
- if (!item->func)
- goto out_free;
-
- if (mod) {
- item->mod = strdup(mod);
- if (!item->mod)
- goto out_free_func;
- } else
- item->mod = NULL;
- item->addr = addr;
-
-
[PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of
"perf kmem record", we will get kernel symbol output.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
tools/perf/builtin-report.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index dd4df9a..7b65100 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
+ __func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH] perf report: Provide libtraceevent with a kernel symbol resolver
So that beautifiers wanting to resolve kernel function addresses to
names can do its work, and when we use "perf report" for output of
"perf kmem record", we will get kernel symbol output.
Signed-off-by: Wang YanQing
---
tools/perf/builtin-report.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index dd4df9a..7b65100 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1317,6 +1317,15 @@ int cmd_report(int argc, const char **argv)
report.range_num = 1;
}
+ if (session->tevent.pevent &&
+ pevent_set_function_resolver(session->tevent.pevent,
+machine__resolve_kernel_addr,
+>machines.host) < 0) {
+ pr_err("%s: failed to set libtraceevent function resolver\n",
+ __func__);
+ return -1;
+ }
+
sort__setup_elide(stdout);
ret = __cmd_report();
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] ALSA: hda: Fix forget to free resource in error handling code path in hda_codec_driver_probe
When hda_codec_driver_probe meet error and return failure, we need
to free resource with patch_ops.free, or we will get resource leak.
Signed-off-by: Wang YanQing <udkni...@gmail.com>
---
Changes v1-v2:
1: Fix calling patch_ops.free after module_put, reported by Takashi Iwai.
Hi! Takashi Iwai
Because the only one calling site of patch_ops.free is in
hda_codec_driver_remove, and if hda_codec_driver_probe return
failure, we will no chance to call the hda_codec_driver_remove,
so I think we don't need to care below case:
"Or, for safety, we may put an internal flag to indicate that the codec
free got already called, and check it at before calling
patch_ops.free, too."
Thanks!
sound/pci/hda/hda_bind.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/sound/pci/hda/hda_bind.c b/sound/pci/hda/hda_bind.c
index 6efadbf..d361bb7 100644
--- a/sound/pci/hda/hda_bind.c
+++ b/sound/pci/hda/hda_bind.c
@@ -100,7 +100,7 @@ static int hda_codec_driver_probe(struct device *dev)
if (patch) {
err = patch(codec);
if (err < 0)
- goto error_module;
+ goto error_module_put;
}
err = snd_hda_codec_build_pcms(codec);
@@ -120,6 +120,9 @@ static int hda_codec_driver_probe(struct device *dev)
return 0;
error_module:
+ if (codec->patch_ops.free)
+ codec->patch_ops.free(codec);
+ error_module_put:
module_put(owner);
error:
--
1.8.5.6.2.g3d8a54e.dirty
[PATCH v2] ALSA: hda: Fix forget to free resource in error handling code path in hda_codec_driver_probe
When hda_codec_driver_probe meet error and return failure, we need
to free resource with patch_ops.free, or we will get resource leak.
Signed-off-by: Wang YanQing
---
Changes v1-v2:
1: Fix calling patch_ops.free after module_put, reported by Takashi Iwai.
Hi! Takashi Iwai
Because the only one calling site of patch_ops.free is in
hda_codec_driver_remove, and if hda_codec_driver_probe return
failure, we will no chance to call the hda_codec_driver_remove,
so I think we don't need to care below case:
"Or, for safety, we may put an internal flag to indicate that the codec
free got already called, and check it at before calling
patch_ops.free, too."
Thanks!
sound/pci/hda/hda_bind.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/sound/pci/hda/hda_bind.c b/sound/pci/hda/hda_bind.c
index 6efadbf..d361bb7 100644
--- a/sound/pci/hda/hda_bind.c
+++ b/sound/pci/hda/hda_bind.c
@@ -100,7 +100,7 @@ static int hda_codec_driver_probe(struct device *dev)
if (patch) {
err = patch(codec);
if (err < 0)
- goto error_module;
+ goto error_module_put;
}
err = snd_hda_codec_build_pcms(codec);
@@ -120,6 +120,9 @@ static int hda_codec_driver_probe(struct device *dev)
return 0;
error_module:
+ if (codec->patch_ops.free)
+ codec->patch_ops.free(codec);
+ error_module_put:
module_put(owner);
error:
--
1.8.5.6.2.g3d8a54e.dirty
Re: [PATCH] ALSA: hda: Fix resource leak issue in snd_hda_codec_build_controls and snd_hda_codec_parse_pcms
On Sun, Sep 03, 2017 at 11:27:31PM +0800, Wang YanQing wrote:
> When patch_ops.init, patch_ops.build_pcms and patch_ops.build_controls
> return failure, we need to free resource with patch_ops.free, or we will
> get resource leak.
>
> Signed-off-by: Wang YanQing <udkni...@gmail.com>
> ---
> sound/pci/hda/hda_codec.c | 7 ++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/sound/pci/hda/hda_codec.c b/sound/pci/hda/hda_codec.c
> index df6b57e..4e3e613 100644
> --- a/sound/pci/hda/hda_codec.c
> +++ b/sound/pci/hda/hda_codec.c
> @@ -2973,8 +2973,11 @@ int snd_hda_codec_build_controls(struct hda_codec
> *codec)
> err = codec->patch_ops.init(codec);
> if (!err && codec->patch_ops.build_controls)
> err = codec->patch_ops.build_controls(codec);
> - if (err < 0)
> + if (err < 0) {
> + if (codec->patch_ops.free)
> + codec->patch_ops.free(codec);
> return err;
> + }
>
> /* we create chmaps here instead of build_pcms */
> err = add_std_chmaps(codec);
> @@ -3170,6 +3173,8 @@ int snd_hda_codec_parse_pcms(struct hda_codec *codec)
> if (err < 0) {
> codec_err(codec, "cannot build PCMs for #%d (error %d)\n",
> codec->core.addr, err);
> + if (codec->patch_ops.free)
> + codec->patch_ops.free(codec);
> return err;
> }
>
> --
> 1.8.5.6.2.g3d8a54e.dirty
I don't know whether this new patch is ok for you, but I think that
we could allocate resources in patch_ops.init, patch_ops.build_pcms
and patch_ops.build_controls separately, so I think it isn't flexible
and elegant to free all resources in all the error path cases in them
separately, so maybe it is better to use patch_ops.free as the unique
point to release all resource.
Thanks.
Re: [PATCH] ALSA: hda: Fix resource leak issue in snd_hda_codec_build_controls and snd_hda_codec_parse_pcms
On Sun, Sep 03, 2017 at 11:27:31PM +0800, Wang YanQing wrote:
> When patch_ops.init, patch_ops.build_pcms and patch_ops.build_controls
> return failure, we need to free resource with patch_ops.free, or we will
> get resource leak.
>
> Signed-off-by: Wang YanQing
> ---
> sound/pci/hda/hda_codec.c | 7 ++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/sound/pci/hda/hda_codec.c b/sound/pci/hda/hda_codec.c
> index df6b57e..4e3e613 100644
> --- a/sound/pci/hda/hda_codec.c
> +++ b/sound/pci/hda/hda_codec.c
> @@ -2973,8 +2973,11 @@ int snd_hda_codec_build_controls(struct hda_codec
> *codec)
> err = codec->patch_ops.init(codec);
> if (!err && codec->patch_ops.build_controls)
> err = codec->patch_ops.build_controls(codec);
> - if (err < 0)
> + if (err < 0) {
> + if (codec->patch_ops.free)
> + codec->patch_ops.free(codec);
> return err;
> + }
>
> /* we create chmaps here instead of build_pcms */
> err = add_std_chmaps(codec);
> @@ -3170,6 +3173,8 @@ int snd_hda_codec_parse_pcms(struct hda_codec *codec)
> if (err < 0) {
> codec_err(codec, "cannot build PCMs for #%d (error %d)\n",
> codec->core.addr, err);
> + if (codec->patch_ops.free)
> + codec->patch_ops.free(codec);
> return err;
> }
>
> --
> 1.8.5.6.2.g3d8a54e.dirty
I don't know whether this new patch is ok for you, but I think that
we could allocate resources in patch_ops.init, patch_ops.build_pcms
and patch_ops.build_controls separately, so I think it isn't flexible
and elegant to free all resources in all the error path cases in them
separately, so maybe it is better to use patch_ops.free as the unique
point to release all resource.
Thanks.
