[OE-core] [PATCH 5/6] scripts/oe-git-archive: implement --notes

2017-03-24 Thread Markus Lehtonen
Option for adding git-notes to the commit. [YOCTO #10582] Signed-off-by: Markus Lehtonen --- scripts/oe-git-archive | 21 ++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/scripts/oe-git-archive b/scripts/oe-git-archive index 1805ab3..ab19cb9 100755 --- a

[OE-core] [PATCH 6/6] build-perf-test-wrapper.sh: store buildstats under git-notes

2017-03-24 Thread Markus Lehtonen
In order to dramatically reduce the data footprint of the result branches. This makes cloning/pulling result repositories a lot faster. Buildstats can still be accessed, when needed by doing git fetch refs/notes/buildstats/*:refs/notes/buildstats/* [YOCTO #10582] Signed-off-by: Markus Lehtonen

[OE-core] [PATCH 1/6] oe-build-perf-test: pack all buildstat in one file

2017-03-24 Thread Markus Lehtonen
re no more per-testcase data files to store permanently. [YOCTO #10582] Signed-off-by: Markus Lehtonen --- meta/lib/oeqa/buildperf/base.py | 42 --- meta/lib/oeqa/buildperf/test_basic.py | 24 +--- scripts/oe-build-perf-test| 1

[OE-core] [PATCH 0/6] oe-build-perf-test: re-organize buildstats archiving

2017-03-24 Thread Markus Lehtonen
git.openembedded.org/openembedded-core-contrib marquiz/buildperf/buildstats http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/buildstats Markus Lehtonen (6): oe-build-perf-test: pack all buildstat in one file oe-build-perf-test: sum rusage in buildstats scripts

[OE-core] [PATCH 1/2] oeqa.utils.commands: limit runCmd exception output

2017-03-24 Thread Markus Lehtonen
Make it possible to limit the length of output lines shown in runCmd exceptions. E.g when running bitbake we easily get thousands of lines of log output, where only the last few (tens) are interesting or relevant when an error occurs. Signed-off-by: Markus Lehtonen --- meta/lib/oeqa/utils

[OE-core] [PATCH 2/2] oeqa.buildperf: limit the length of error output

2017-03-24 Thread Markus Lehtonen
Limit the length of error logs to 40 lines. We don't need to show/archive thousands of lines of bitbake logs if an error occurs. Signed-off-by: Markus Lehtonen --- meta/lib/oeqa/buildperf/base.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oeqa/buildperf/ba

Re: [OE-core] [PATCH 3/7] sysklogd: Use update-alternatives

2018-09-02 Thread Markus Lehtonen
Hi, IIRC, the problem was that all the syslog packages were providing ${sysconfdir}/init.d/syslog which caused problems. And I don't think that has changed. Why would you want to install two syslog daemons on the system? You should be able to install busybox after syslog as busybox-syslog is p

Re: [OE-core] [PATCH 3/7] sysklogd: Use update-alternatives

2018-09-03 Thread Markus Lehtonen
Hmm, There should be rconflicts in place, in both way – i.e. busybox-syslog rconflicts with sysklogd and the other way around. Isn't there(?) - Markus On 03/09/2018, 19.34, "Khem Raj" wrote: On Sun, Sep 2, 2018 at 11:49 PM Markus Lehtonen wrote: > >

[OE-core] [PATCH] cantarell-fonts: fix SRC_URI

2018-09-06 Thread Markus Lehtonen
Fixes build for older git versions (e.g. in Centos 7) which are not able to redirect if the .git suffix is missing. Build got broken by 25bac865def42650c0c0323322158ba338e8b753 Signed-off-by: Markus Lehtonen --- meta/recipes-graphics/cantarell-fonts/cantarell-fonts_git.bb | 2 +- 1 file changed

[OE-core] [PATCH v4 0/8] support profile-optimized build for Python

2017-08-08 Thread Markus Lehtonen
edded-core-contrib/log/?h=marquiz/fixes-9338 Markus Lehtonen (8): python3-native: support profile optimized build python3: fix depends of python3-tests python3: add python3-profile-opt recipe devtools/images: add python-pgo-image python3: support profile optimized build python3: fix pr

[OE-core] [PATCH 1/5] lib/oe/package_manager: re-implement rpm feed signing

2017-08-11 Thread Markus Lehtonen
[YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/lib/oe/package_manager.py | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 518cf8dbe3..942f2dd903 100644 --- a/meta/lib/oe

[OE-core] [PATCH 0/5] Support signed RPM package feeds

2017-08-11 Thread Markus Lehtonen
ble in the git repository at: git://git.openembedded.org/openembedded-core-contrib marquiz/fixes-11209 http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-11209 Markus Lehtonen (5): lib/oe/package_manager: re-implement rpm feed signing dnf: rrecommend gnupg o

[OE-core] [PATCH 3/5] oeqa: fix dnf tests

2017-08-11 Thread Markus Lehtonen
Rename one dnf runtime test that it will recognized as a python module and thus also found by the oe test loader. Also, fix value of TEST_SUITES in dnf selftest so that all test dependencies are satisfied and the runtime test may be successfully run from there. Signed-off-by: Markus Lehtonen

[OE-core] [PATCH 2/5] dnf: rrecommend gnupg

2017-08-11 Thread Markus Lehtonen
This makes it possible to enable 'repo_gpgcheck' in dnf.conf. That is, do GPG signature check on repository metadata. Without gnupg dnf fails with "error: Invalid crypto engine." [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/dnf/dnf_2.5.1.bb | 2 +

[OE-core] [PATCH 5/5] dnf: expand dnf selftest to test signed package feeds

2017-08-11 Thread Markus Lehtonen
[YOCTO #12099] Signed-off-by: Markus Lehtonen --- meta-selftest/files/signing/key.passphrase | 1 + meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 12 meta/lib/oeqa/selftest/cases/runtime_test.py| 13 - 3 files changed, 25 insertions(+), 1

[OE-core] [PATCH 4/5] oeqa: fix temp file handling in dnf package feed test

2017-08-11 Thread Markus Lehtonen
Prevent stale temp files and a possible (if unlikely) race in tempfile usage. Signed-off-by: Markus Lehtonen --- meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py

Re: [OE-core] [PATCH 5/5] dnf: expand dnf selftest to test signed package feeds

2017-08-11 Thread Markus Lehtonen
On 11/08/2017, 14.11, "Alexander Kanavin" wrote: On 08/11/2017 01:51 PM, Markus Lehtonen wrote: > --- a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py > +++ b/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py > @@ -31,12 +31,24 @@ class

[OE-core] [PATCH v2 2/7] dnf: rrecommend gnupg

2017-08-11 Thread Markus Lehtonen
This makes it possible to enable 'repo_gpgcheck' in dnf.conf. That is, do GPG signature check on repository metadata. Without gnupg dnf fails with "error: Invalid crypto engine." [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/dnf/dnf_2.5.1.bb | 2 +

[OE-core] [PATCH v2 1/7] lib/oe/package_manager: re-implement rpm feed signing

2017-08-11 Thread Markus Lehtonen
[YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/lib/oe/package_manager.py | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 518cf8dbe3..942f2dd903 100644 --- a/meta/lib/oe

[OE-core] [PATCH v2 3/7] sign_package_feed.bbclass: install signing key into rootfs

2017-08-11 Thread Markus Lehtonen
If package-management is enabled. [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/classes/sign_package_feed.bbclass | 3 +++ 1 file changed, 3 insertions(+) diff --git a/meta/classes/sign_package_feed.bbclass b/meta/classes/sign_package_feed.bbclass index 71df03bab3..f03c4802d0 100644

[OE-core] [PATCH v2 0/7] Support signed RPM package feeds

2017-08-11 Thread Markus Lehtonen
at: git://git.openembedded.org/openembedded-core-contrib marquiz/fixes-11209 http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-11209 Markus Lehtonen (7): lib/oe/package_manager: re-implement rpm feed signing dnf: rrecommend gnupg sign_package_feed.bbclass: inst

[OE-core] [PATCH v2 4/7] package_manager.py: enable dnf's repo_gpgcheck if feed signing is enabled

2017-08-11 Thread Markus Lehtonen
If package feed signing is enabled enable repo gpg signature check for rpm repositories added via PACKAGE_FEED_URIS. This has the implication that all repositories added via this mechanism must be signed with the same key. [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/lib/oe

[OE-core] [PATCH v2 5/7] oeqa: fix dnf tests

2017-08-11 Thread Markus Lehtonen
Rename one dnf runtime test that it will recognized as a python module and thus also found by the oe test loader. Also, fix value of TEST_SUITES in dnf selftest so that all test dependencies are satisfied and the runtime test may be successfully run from there. Signed-off-by: Markus Lehtonen

[OE-core] [PATCH v2 7/7] dnf: expand dnf selftest to test signed package feeds

2017-08-11 Thread Markus Lehtonen
[YOCTO #12099] Signed-off-by: Markus Lehtonen --- meta-selftest/files/signing/key.passphrase | 1 + meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 3 ++- meta/lib/oeqa/selftest/cases/runtime_test.py| 12 +++- 3 files changed, 14 insertions(+), 2 deletions

[OE-core] [PATCH v2 6/7] oeqa: fix temp file handling in dnf package feed test

2017-08-11 Thread Markus Lehtonen
Prevent stale temp files and a possible (if unlikely) race in tempfile usage. Signed-off-by: Markus Lehtonen --- meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py

[OE-core] [PATCH v3 0/7] Support signed RPM package feeds

2017-08-11 Thread Markus Lehtonen
org/openembedded-core-contrib marquiz/fixes-11209 http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-11209 Markus Lehtonen (7): lib/oe/package_manager: re-implement rpm feed signing dnf: rrecommend gnupg sign_package_feed.bbclass: install signing key into roo

Re: [OE-core] [PATCH v2 7/7] dnf: expand dnf selftest to test signed package feeds

2017-08-14 Thread Markus Lehtonen
On 14/08/2017, 14.16, "Alexander Kanavin" wrote: On 08/11/2017 06:35 PM, Markus Lehtonen wrote: > > import re > -output_makecache = self.dnf('makecache') > +output_makecache = self.dnf('-y makecache')

[OE-core] [PATCH v4 0/8] Support signed RPM package feeds

2017-08-15 Thread Markus Lehtonen
rg/openembedded-core-contrib/log/?h=marquiz/fixes-11209 Markus Lehtonen (8): lib/oe/package_manager: re-implement rpm feed signing dnf: rrecommend gnupg signing-keys: delete unused tasks sign_package_feed.bbclass: install signing key into rootfs package_manager.py: enable dnf's repo_gp

[OE-core] [PATCH v4 2/8] dnf: rrecommend gnupg

2017-08-15 Thread Markus Lehtonen
This makes it possible to enable 'repo_gpgcheck' in dnf.conf. That is, do GPG signature check on repository metadata. Without gnupg dnf fails with "error: Invalid crypto engine." [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/dnf/dnf_2.6.3.bb | 2 +

[OE-core] [PATCH v4 1/8] lib/oe/package_manager: re-implement rpm feed signing

2017-08-15 Thread Markus Lehtonen
[YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/lib/oe/package_manager.py | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 518cf8dbe3..942f2dd903 100644 --- a/meta/lib/oe

[OE-core] [PATCH v4 3/8] signing-keys: delete unused tasks

2017-08-15 Thread Markus Lehtonen
In particular, do_unpack needs to be deleted as it was causing missing subpackages. Do_unpack cleans the builddir and if it was run after do_get_public_keys it cleaned up all keys causing no packages to be built. Signed-off-by: Markus Lehtonen --- meta/recipes-core/meta/signing-keys.bb | 8

[OE-core] [PATCH v4 4/8] sign_package_feed.bbclass: install signing key into rootfs

2017-08-15 Thread Markus Lehtonen
If package-management is enabled. [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/classes/sign_package_feed.bbclass | 3 +++ 1 file changed, 3 insertions(+) diff --git a/meta/classes/sign_package_feed.bbclass b/meta/classes/sign_package_feed.bbclass index 71df03bab3..f03c4802d0 100644

[OE-core] [PATCH v4 5/8] package_manager.py: enable dnf's repo_gpgcheck if feed signing is enabled

2017-08-15 Thread Markus Lehtonen
If package feed signing is enabled enable repo gpg signature check for rpm repositories added via PACKAGE_FEED_URIS. This has the implication that all repositories added via this mechanism must be signed with the same key. [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta/lib/oe

[OE-core] [PATCH v4 6/8] oeqa: fix dnf tests

2017-08-15 Thread Markus Lehtonen
Rename one dnf runtime test that it will recognized as a python module and thus also found by the oe test loader. Also, fix value of TEST_SUITES in dnf selftest so that all test dependencies are satisfied and the runtime test may be successfully run from there. Signed-off-by: Markus Lehtonen

[OE-core] [PATCH v4 8/8] dnf: expand dnf selftest to test signed package feeds

2017-08-15 Thread Markus Lehtonen
[YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta-selftest/files/signing/key.passphrase | 1 + meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 4 +++- meta/lib/oeqa/selftest/cases/runtime_test.py| 12 +++- 3 files changed, 15 insertions(+), 2 deletions

[OE-core] [PATCH v4 7/8] oeqa: fix temp file handling in dnf package feed test

2017-08-15 Thread Markus Lehtonen
Prevent stale temp files and a possible (if unlikely) race in tempfile usage. Signed-off-by: Markus Lehtonen --- meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py

Re: [OE-core] [PATCH] gpg_sign: perform rpm signing serially

2017-08-16 Thread Markus Lehtonen
I agree. I don't see reason for dropping parallelism completely. There is a real gain when running on beefier machines. Making it configurable would probably be best. Or just drop it to a saner value, like 20 or 10. - Markus On 16/08/2017, 2.53, "Mark Hatle" wrote: It would probably be

Re: [OE-core] [PATCH] gpg_sign: perform rpm signing serially

2017-08-17 Thread Markus Lehtonen
BB_NUMBER_THREADS as Andre suggested in another email. - Markus On 16/08/2017, 19.00, "Leonardo Sandoval" wrote: On Wed, 2017-08-16 at 15:28 +0300, Markus Lehtonen wrote: > I agree. I don't see reason for dropping parallelism completely. There is a real gain when running o

Re: [OE-core] [PATCH] gpg_sign: perform rpm signing serially

2017-08-18 Thread Markus Lehtonen
ds of subpackages you get different figures. - Markus On 17/08/2017, 17.52, "Leonardo Sandoval" wrote: On Thu, 2017-08-17 at 10:50 +0300, Markus Lehtonen wrote: > Hi, > > I quickly run some tests on a Xeon server, using glibc-locale as the recipe to build.

[OE-core] [PATCH] oeqa: increase verbosity of dnf commands in dnf packagefeed test

2017-08-18 Thread Markus Lehtonen
Makes diagnosing failures easier. [YOCTO #11209] Signed-off-by: Markus Lehtonen --- meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py b/meta-selftest/lib/oeqa/runtime

[OE-core] [PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats

2017-09-15 Thread Markus Lehtonen
Fix a misbehavior when some of the buildstats are missing. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index b5ad42bc8a..23081db173 100755 --- a/scripts/oe-build

[OE-core] [PATCH 4/7] scripts/oe-build-perf-report: use --hostname in --list

2017-09-15 Thread Markus Lehtonen
Makes it possible to list test results for certain host only, instead of always listing all results from all hosts. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe

[OE-core] [PATCH 5/7] scripts/oe-build-perf-report: remove dead code

2017-09-15 Thread Markus Lehtonen
Some leftover from an early prototype. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 4 1 file changed, 4 deletions(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index 89d9d6f6b5..4e067caba0 100755 --- a/scripts/oe-build-perf-report +++ b

[OE-core] [PATCH 7/7] scripts/oe-build-perf-report: tidy up html syntax

2017-09-15 Thread Markus Lehtonen
Fix some problems in the html syntax of the generated report: - prevent empty rows in the summary table - add one missing column in the results table Signed-off-by: Markus Lehtonen --- scripts/lib/build_perf/html/report.html | 36 ++--- 1 file changed, 20 insertions

[OE-core] [PATCH 3/7] scripts/oe-build-perf-report: fix handling of --history-length

2017-09-15 Thread Markus Lehtonen
Don't crash if 'left' revision is older than the range of commits specified with '--history-length'. In this case the 'left' revision takes precedence. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 2 +- 1 file changed, 1 insertion(+), 1 d

[OE-core] [PATCH 6/7] scripts/oe-build-perf-report: provide valid title in the html report

2017-09-15 Thread Markus Lehtonen
Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index 4e067caba0..8d730cd20a 100755 --- a/scripts/oe-build-perf-report +++ b/scripts/oe-build-perf

[OE-core] [PATCH 0/7] Fixes to oe-build-perf-report

2017-09-15 Thread Markus Lehtonen
-14 12:39:29 +0100) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/fixes http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/fixes Markus Lehtonen (7): scripts/oe-build-perf-report: fix dumping

[OE-core] [PATCH 2/7] scripts/oe-build-perf-report: add AggregateTestData class

2017-09-15 Thread Markus Lehtonen
Making the code a bit more readable. Signed-off-by: Markus Lehtonen --- scripts/lib/build_perf/report.py | 5 - scripts/oe-build-perf-report | 16 +--- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/scripts/lib/build_perf/report.py b/scripts/lib/build_perf

[OE-core] [PATCH 0/4] oe-build-perf-report: task and recipe details in html report

2017-09-15 Thread Markus Lehtonen
/openembedded-core-contrib/log/?h=marquiz/fixes-11381 Markus Lehtonen (4): scripts/buildstats-diff: move code to lib/buildstats.py scripts/oe-build-perf-report: summary of task resource usage scripts/buildstats-diff: move more code to lib/buildstats.py scripts/oe-build-perf-report: show recipe

[OE-core] [PATCH 1/4] scripts/buildstats-diff: move code to lib/buildstats.py

2017-09-15 Thread Markus Lehtonen
Move over code from buildstats-diff to new scripts/lib/buildstats.py module in order to share code related to buildstats processing. Also, refactor the code, introducing new classes to make the code readable, maintainable and easier to debug. [YOCTO #11381] Signed-off-by: Markus Lehtonen

[OE-core] [PATCH 2/4] scripts/oe-build-perf-report: summary of task resource usage

2017-09-15 Thread Markus Lehtonen
) [YOCTO #11381] Signed-off-by: Markus Lehtonen --- scripts/lib/build_perf/html/report.html | 60 +- scripts/lib/buildstats.py | 8 +++ scripts/oe-build-perf-report| 90 - 3 files changed, 133 insertions(+), 25 deletions

[OE-core] [PATCH 3/4] scripts/buildstats-diff: move more code to lib/buildstats.py

2017-09-15 Thread Markus Lehtonen
More refactoring of buildstats-diff script. Move recipe version comparison functionality to scripts/lib/buildstats.py. This patch also compasses some wording changes, i.e. changing 'package' to 'recipe'. [YOCTO #11382] Signed-off-by: Markus Lehtonen --- scripts/bu

[OE-core] [PATCH 4/4] scripts/oe-build-perf-report: show recipe version changes in html report

2017-09-15 Thread Markus Lehtonen
If buildstats are available (for a certain measurement), show recipe version changes between the two builds that are being compared. The information shown includes new and dropped recipes as well as changes in recipe version, revision or epoch. [YOCTO #11382] Signed-off-by: Markus Lehtonen

Re: [OE-core] [PATCH] sysklogd: add alternatives for klogd and syslogd

2018-10-26 Thread Markus Lehtonen
'why would you want to have multiple, alternative syslog daemons on the system? Wouldn't a better fix be to move the syslogd and klogd symlinks to the busybox-syslog package? It doesn't seem to make much sense to have a syslogd binary on the system without an init script(?) The original problem

[OE-core] [PATCH 2/2] scripts/contrib: add oe-build-perf-report-email

2017-03-31 Thread Markus Lehtonen
: - phantomjs - optipng [YOCTO #10931] Signed-off-by: Markus Lehtonen --- scripts/contrib/oe-build-perf-report-email.py | 266 ++ scripts/lib/build_perf/scrape-html-report.js | 56 ++ 2 files changed, 322 insertions(+) create mode 100755 scripts/contrib/oe-build-perf

[OE-core] [PATCH 0/2] Tools for visualizing build performance test results

2017-03-31 Thread Markus Lehtonen
: git://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/scripts http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/scripts Markus Lehtonen (2): scripts: add oe-build-perf-report script scripts/contrib: add oe-build-perf-report-email scripts

[OE-core] [PATCH 1/2] scripts: add oe-build-perf-report script

2017-03-31 Thread Markus Lehtonen
generating HTML reports so it requires python3-jinja2 to be installed on the system. [YOCTO #10931] Signed-off-by: Markus Lehtonen --- scripts/lib/build_perf/__init__.py | 31 ++ scripts/lib/build_perf/html.py | 19 + scripts/lib/build_perf/html

[OE-core] [PATCH 0/7] Build perf test enhancements

2017-04-03 Thread Markus Lehtonen
://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/scripts http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/scripts Markus Lehtonen (7): scripts/oe-build-perf-report: better guessing of args scripts/oe-build-perf-report: fix 'charts ready' conso

[OE-core] [PATCH 1/7] scripts/oe-build-perf-report: better guessing of args

2017-04-03 Thread Markus Lehtonen
When getting info from the latest commit, don't search all refs but only branches. We don't get correct data from refs/tags/* or refs/notest/*, for example. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) di

[OE-core] [PATCH 3/7] scripts/oe-build-perf-report: fix typo

2017-04-03 Thread Markus Lehtonen
Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index ca9cf1d..6f2db49 100755 --- a/scripts/oe-build-perf-report +++ b/scripts/oe-build-perf-report

[OE-core] [PATCH 2/7] scripts/oe-build-perf-report: fix 'charts ready' console message

2017-04-03 Thread Markus Lehtonen
h, in turn, caused oe-build-perf-report-email.py script to fail with a timeout. Signed-off-by: Markus Lehtonen --- scripts/lib/build_perf/html/measurement_chart.html | 6 +++--- scripts/lib/build_perf/html/report.html| 5 + 2 files changed, 4 insertions(+), 7 deletions(-) diff --

[OE-core] [PATCH 4/7] build-perf-test-wrapper.sh: make it possible to specify Git branch name

2017-04-03 Thread Markus Lehtonen
Support : format for the -c argument. This makes it possible to test older commits of a certain branch (not just the tip of it) so that the branch name will still be correctly recorded in the test report data. Signed-off-by: Markus Lehtonen --- scripts/contrib/build-perf-test-wrapper.sh | 31

[OE-core] [PATCH 5/7] build-perf-test-wrapper.sh: correctly check test script exit status

2017-04-03 Thread Markus Lehtonen
Test of the exit code was accidentally moved to wrong place when oe-git-archive was taken into use. Signed-off-by: Markus Lehtonen --- scripts/contrib/build-perf-test-wrapper.sh | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/scripts/contrib/build-perf-test

[OE-core] [PATCH 6/7] build-perf-test-wrapper.sh: support pushing to remote Git

2017-04-03 Thread Markus Lehtonen
Implement new '-P' option for spefifying a Git remote where to push results after committing to a local Git repository. Signed-off-by: Markus Lehtonen --- scripts/contrib/build-perf-test-wrapper.sh | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/con

[OE-core] [PATCH 7/7] build-perf-test-wrapper.sh: support sending email reports

2017-04-03 Thread Markus Lehtonen
Add new '-E' command line option for sending an email report to specified recipient. Signed-off-by: Markus Lehtonen --- scripts/contrib/build-perf-test-wrapper.sh | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/scripts/contrib/build-perf-test-

[OE-core] [PATCH v2 0/8] Build perf test enhancements

2017-04-04 Thread Markus Lehtonen
://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/scripts http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/scripts Markus Lehtonen (8): scripts/oe-build-perf-report: better guessing of args scripts/oe-build-perf-report: fix 'charts

[OE-core] [PATCH] oe-build-perf-report-email.py: fix one file path

2017-04-06 Thread Markus Lehtonen
Sending report email was not working correctly if the script was given an html report path that contained directory components. Signed-off-by: Markus Lehtonen --- scripts/contrib/oe-build-perf-report-email.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/contrib/oe

[OE-core] [PATCH 1/3] build-perf-test-wrapper.sh: support extra args for email script

2017-04-07 Thread Markus Lehtonen
Make it possible to provide (extra) command line arguments to the oe-build-perf-test-email script via a new environment variable OE_BUILD_PERF_REPORT_EMAIL_EXTRA_ARGS. Signed-off-by: Markus Lehtonen --- scripts/contrib/build-perf-test-wrapper.sh | 2 +- 1 file changed, 1 insertion(+), 1

[OE-core] [PATCH 0/3] Fixes to build perf test reporting

2017-04-07 Thread Markus Lehtonen
Three miscellaneous one-liners fixing minor issues in build performance test reporting scripts. Markus Lehtonen (3): build-perf-test-wrapper.sh: support extra args for email script oe-build-perf-report-email.py: use proper fallback email address scripts/oe-build-perf-report: improve

[OE-core] [PATCH 2/3] oe-build-perf-report-email.py: use proper fallback email address

2017-04-07 Thread Markus Lehtonen
Use properly formatted fallback email address instead of just the username. Signed-off-by: Markus Lehtonen --- scripts/contrib/oe-build-perf-report-email.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/contrib/oe-build-perf-report-email.py b/scripts/contrib/oe

[OE-core] [PATCH 3/3] scripts/oe-build-perf-report: improve guessing of args

2017-04-07 Thread Markus Lehtonen
Search remote branches, too, when finding the latest commit. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index 6f2db49d5c..ced5ff2e12 100755 --- a

[OE-core] [PATCH] oe-build-perf-report-email.py: use pwd for getting user name

2017-04-12 Thread Markus Lehtonen
executed from a cron script, for example. Signed-off-by: Markus Lehtonen --- scripts/contrib/oe-build-perf-report-email.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/scripts/contrib/oe-build-perf-report-email.py b/scripts/contrib/oe-build-perf-report-email.py index

[OE-core] [PATCH] oeqa.utils.metadata: cope with invalid lines in os-release file

2017-04-26 Thread Markus Lehtonen
Don't crash if every line in /etc/os-release does not adhere to the expected "key=val" format. E.g. CentOS 7 has empty lines in the file. Signed-off-by: Markus Lehtonen --- meta/lib/oeqa/utils/metadata.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git

Re: [OE-core] [PATCH] oeqa.utils.metadata: cope with invalid lines in os-release file

2017-04-27 Thread Markus Lehtonen
On 26/04/2017, 17.47, "Joshua Lock" wrote: On Wed, 2017-04-26 at 17:39 +0300, Markus Lehtonen wrote: > Don't crash if every line in /etc/os-release does not adhere to the > expected "key=val" format. E.g. CentOS 7 has empty lines in the file.

[OE-core] [PATCH v2 0/1] oeqa.utils.metadata: cope with invalid lines in os-release file

2017-04-27 Thread Markus Lehtonen
Total rewrite of the patch, as per Joshua's comments. Refactor and move get_os_release() from oeqa.utils.metadata to oe.lsb. Markus Lehtonen (1): oe.lsb: add get_os_release() meta/lib/oe/lsb.py | 33 - meta/lib/oeqa/utils/metadata.py

[OE-core] [PATCH v2 1/1] oe.lsb: add get_os_release()

2017-04-27 Thread Markus Lehtonen
Move get_os_release() from oeqa.utils.metadata to oe.lsb, merging the code with release_dict_osr() from oe.lsb. This removes some code duplication and makes get_os_release() more robust. Signed-off-by: Markus Lehtonen --- meta/lib/oe/lsb.py | 33

[OE-core] [PATCH] build-perf-test-wrapper.sh: support uploading test reports

2017-05-04 Thread Markus Lehtonen
Implement new '-R' command line option for specifying an rsync destination where to upload test reports. [YOCTO #5049] Signed-off-by: Markus Lehtonen --- scripts/contrib/build-perf-test-wrapper.sh | 30 +- 1 file changed, 25 insertions(+), 5 deletions(-)

[OE-core] [PATCH 1/2] oe-build-perf-report: accept parenthesis in tag names

2017-05-04 Thread Markus Lehtonen
Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index ced5ff2e12..876e3b1a76 100755 --- a/scripts/oe-build-perf-report +++ b/scripts/oe-build-perf

[OE-core] [PATCH 2/2] oe-build-perf-report: allow slashes in {branch} field in tag names

2017-05-04 Thread Markus Lehtonen
The target branch name might contain slashes. Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index 876e3b1a76..d460caae18 100755 --- a/scripts/oe-build-perf-report

[OE-core] [PATCH 2/5] oe-build-perf-report: use correct x-axis max value in html charts

2017-05-15 Thread Markus Lehtonen
Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index ced5ff2e12..fc32fbe39a 100755 --- a/scripts/oe-build-perf-report +++ b/scripts/oe-build-perf

[OE-core] [PATCH 1/5] oeqa.utils.git: use --verify in rev_parse()

2017-05-15 Thread Markus Lehtonen
We use rev-parse for turning git object names into SHA-1 and checking their existence. Using --verify option makes sure git-rev-parse does what we expect. Signed-off-by: Markus Lehtonen --- meta/lib/oeqa/utils/git.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib

[OE-core] [PATCH 0/5] Enhancements to build perf reporting

2017-05-15 Thread Markus Lehtonen
1b95c: python*-git: Upgrade to version 2.1.3 (2017-05-12 08:49:30 +0100) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib marquiz/fixes-11355 http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-11355 Markus Lehton

[OE-core] [PATCH 4/5] oe-build-perf-report: implement --dump-buildstats

2017-05-15 Thread Markus Lehtonen
For dumping buildstats from the test runs being reported. The output directory where buildstats are copied is 'oe-build-perf-buildstats/'. Buildstats can be then further analyzed with buildstats-diff script, for example. [YOCTO #11355] Signed-off-by: Markus Lehtonen --- scripts/oe-

[OE-core] [PATCH 3/5] oe-build-perf-report: two verbosity levels for --list

2017-05-15 Thread Markus Lehtonen
Signed-off-by: Markus Lehtonen --- scripts/oe-build-perf-report | 45 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report index fc32fbe39a..34591bec16 100755 --- a/scripts/oe

[OE-core] [PATCH 5/5] scripts/buildstats-diff: support optimized rusage values

2017-05-15 Thread Markus Lehtonen
Buildstats from oe-build-perf-test results have been optimized to not have child rusage values at all. There, rusage is the sum of parent and child rusage values. This patch makes buildstats-diff compatible with this format. [YOCTO #11355] Signed-off-by: Markus Lehtonen --- scripts/buildstats

[OE-core] [PATCH] scripts/create-pull-request: fix handling of '-t' option

2017-05-15 Thread Markus Lehtonen
Signed-off-by: Markus Lehtonen --- scripts/create-pull-request | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/create-pull-request b/scripts/create-pull-request index 46d65386a3..5bdea44482 100755 --- a/scripts/create-pull-request +++ b/scripts/create-pull-request

[OE-core] [PATCH v2 04/20] python-profile-opt: rename libpython

2017-05-16 Thread Markus Lehtonen
The python library needs to be renamed so that the automatic dependency generation/checking mechanism in bitbake does not get confused. Otherwise python-profile-opt will depend on libpython from the "normal" python package. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- .../pyt

[OE-core] [PATCH v2 02/20] python: add python-profile-opt recipe

2017-05-16 Thread Markus Lehtonen
. This patch enables the first step, i.e. building python with profile instrumentation enabled [YOCTO #9338] Signed-off-by: Markus Lehtonen --- .../python/python-profile-opt_2.7.13.bb| 17 ++ meta/recipes-devtools/python/python_2.7.13.bb | 26 +++---

[OE-core] [PATCH v2 06/20] python: make profile-optimized build possible

2017-05-16 Thread Markus Lehtonen
PYTHON_PROFILE_OPT = "1" in your local.conf. Be sure to get the profile data for python, preferably by running "bitbake python-pgo-image -c profile". [YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/python/python_2.7.13.bb | 25 -

[OE-core] [PATCH v2 07/20] python-pgo-image: exclude tests from the default profile target

2017-05-16 Thread Markus Lehtonen
] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/images/python-pgo-image.bb | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/meta/recipes-devtools/images/python-pgo-image.bb b/meta/recipes-devtools/images/python-pgo-image.bb index af9da20338..426d6acb45 100644

[OE-core] [PATCH v2 01/20] python-native: support profile optimized build

2017-05-16 Thread Markus Lehtonen
OFILE_TASK = "${S}/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck" [YOCTO #9338] Signed-off-by: Markus Lehtonen --- ...x-.so-loading-when-when-running-profile-t.patch | 26 ++ .../python/python-native_2.7.13.bb | 10 + 2 files ch

[OE-core] [PATCH v2 03/20] python: remove path hack from setup.py

2017-05-16 Thread Markus Lehtonen
headers. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- .../python/python/01-use-proper-tools-for-cross-build.patch| 10 -- 1 file changed, 10 deletions(-) diff --git a/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch b/meta/recipes-devtools/python

[OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Markus Lehtonen
-core-contrib marquiz/fixes-9338 http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-9338 Markus Lehtonen (20): python-native: support profile optimized build python: add python-profile-opt recipe python: remove path hack from setup.py python-profile-opt: rename

[OE-core] [PATCH v2 05/20] devtools/images: add python-pgo-image

2017-05-16 Thread Markus Lehtonen
specifying ${PYTHON_PROFILE_TASK}. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/images/python-pgo-image.bb | 62 1 file changed, 62 insertions(+) create mode 100644 meta/recipes-devtools/images/python-pgo-image.bb diff --git a/meta/recipes-devtools

[OE-core] [PATCH v2 14/20] python-pgo-image: profiling for python3

2017-05-16 Thread Markus Lehtonen
ay be specified with PYTHON3_PROFILE_TASK. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/images/python-pgo-image.bb | 57 +++- 1 file changed, 45 insertions(+), 12 deletions(-) diff --git a/meta/recipes-devtools/images/python-pgo-image.bb b/meta/rec

[OE-core] [PATCH v2 08/20] python: add python-tools subpackage

2017-05-16 Thread Markus Lehtonen
Useful in developing Python, e.g. in benchmarking. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/python/python-2.7-manifest.inc | 8 +--- meta/recipes-devtools/python/python_2.7.13.bb| 5 + 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a

[OE-core] [PATCH v2 17/20] python3: fix profile-optimized build of modules

2017-05-16 Thread Markus Lehtonen
Without this the pgo-related compiler flags are not used in cross-builds. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- ...-CFLAGS-for-extensions-when-cross-compili.patch | 56 ++ meta/recipes-devtools/python/python3_3.5.2.bb | 1 + 2 files changed, 57 insertions

[OE-core] [PATCH v2 18/20] python-pgo-image: exclude tests from the python3 profile target

2017-05-16 Thread Markus Lehtonen
test_urllib2_localnet: 1276.8s test_itertools: 944.7s test_tuple: 847.7s test_trace: 756.7s test_tarfile: 716.7s test_unicodedata: 709.9s test_decimal: 615.1s [YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/images/python-pgo-image.bb | 5 + 1 file

[OE-core] [PATCH v2 11/20] python3-native: support profile optimized build

2017-05-16 Thread Markus Lehtonen
This feature is virtually identical to that of python-native. Profile guided optimization is enabled by defining PYTHON3_NATIVE_PROFILE_OPT = "1" in local.conf. In addition, the profile task may be defined with PYTHON3_NATIVE_PROFILE_TASK. [YOCTO #9338] Signed-off-by: Markus Lehtonen

[OE-core] [PATCH v2 19/20] python3: add python3-tools subpackage

2017-05-16 Thread Markus Lehtonen
[YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/python/python-3.5-manifest.inc | 8 +--- meta/recipes-devtools/python/python3_3.5.2.bb| 5 + 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/meta/recipes-devtools/python/python-3.5

<    1   2   3   4   5   6   7   >