Hello community, here is the log from the commit of package mkdud for openSUSE:Factory checked in at 2018-11-18 23:30:34 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/mkdud (Old) and /work/SRC/openSUSE:Factory/.mkdud.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "mkdud" Sun Nov 18 23:30:34 2018 rev:24 rq:649287 version:1.38 Changes: -------- --- /work/SRC/openSUSE:Factory/mkdud/mkdud.changes 2018-08-28 09:24:43.308879951 +0200 +++ /work/SRC/openSUSE:Factory/.mkdud.new/mkdud.changes 2018-11-18 23:30:55.317573011 +0100 @@ -1,0 +2,16 @@ +Thu Nov 15 13:35:09 UTC 2018 - snw...@suse.de + +- merge gh#openSUSE/mkdud#21 +- add HOWTO describing some typical uses +- update git2log script +- fix typo +- 1.38 + +-------------------------------------------------------------------- +Thu Nov 15 13:09:18 UTC 2018 - snw...@suse.de + +- merge gh#openSUSE/mkdud#20 +- put 'module.config' files into module directory +- 1.37 + +-------------------------------------------------------------------- Old: ---- mkdud-1.36.tar.xz New: ---- mkdud-1.38.tar.xz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ mkdud.spec ++++++ --- /var/tmp/diff_new_pack.ORzmLR/_old 2018-11-18 23:30:55.849572369 +0100 +++ /var/tmp/diff_new_pack.ORzmLR/_new 2018-11-18 23:30:55.849572369 +0100 @@ -23,7 +23,7 @@ Summary: Create driver update from rpms License: GPL-3.0+ Group: Hardware/Other -Version: 1.36 +Version: 1.38 Release: 0 Source: %{name}-%{version}.tar.xz Url: https://github.com/wfeldt/mkdud ++++++ mkdud-1.36.tar.xz -> mkdud-1.38.tar.xz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/mkdud-1.36/HOWTO.md new/mkdud-1.38/HOWTO.md --- old/mkdud-1.36/HOWTO.md 1970-01-01 01:00:00.000000000 +0100 +++ new/mkdud-1.38/HOWTO.md 2018-11-15 14:35:09.000000000 +0100 @@ -0,0 +1,197 @@ +# Driver updates and you + +Driver updates provide a way to update kernel drivers and to influence the +installation workflow **during the installation** of SUSE Linux. +Kernel modules on the running system are updated with the usual update process (of the kernel rpm). + +They are described in the [Update Media Howto](http://ftp.suse.com/pub/people/hvogel/Update-Media-HOWTO/Update-Media-HOWTO.html) + +A driver update can be either an archive file or a directory (the unpacked archive). + + +## Use driver updates + +1. Use the `dud` boot option to pass a URL pointing to the driver update. For example: + + - `dud=https://example.com/foo.dud` + - `dud=disk:/foo.dud` + +2. Driver updates in special places are loaded automatically: + + - a file named `driverupdate` in the root directory of the installation medium + - a partition (e.g. on a USB stick) with file system label `OEMDRV` containing an unpacked driver update + - an unpacked driver update in the initrd of the installation medium + +Putting the driver update into the initrd provides a way to seamlessly integrate driver updates into otherwise +unchanged SUSE installation media. The `mksusecd` command has a dedicated `--initrd` option to make this easy: + +```sh +mksusecd --create new.iso --initrd foo.dud old.iso +``` + + +## Create driver updates + +To create driver updates use `mkdud`. You have to specify at least the product the driver update in intended for but it's +also nice to give it a descriptive name: + +```sh +mkdud --create foo.dud --dist leap15.0 --name "Support the new bar" bar.ko +``` + +Driver updates have to be properly signed to be accepted by the installer. If they are not the user will +see a dialog asking them for manual confirmation during the installation process. + +The signature can be detached (e.g. `foo.dud.asc` for `foo.dud`) or integrated. It's possible to use `mkdud --[detached-]sign` for this. +But typically some external infrastructure for signing is used. + +An interesting alternative is to use signed rpms for driver updates. This way you can get properly signed driver updates +out of the [openSUSE Build Service](https://build.opensuse.org) for example. + + +## Update and load kernel modules + +To create a driver update that updates kernel modules `bar.ko` and `foo.ko` for product `xxx`: + +```sh +mkdud --create foobar.dud --dist xxx bar.ko foo.ko +``` + +Note that you have to take module dependencies into account (they are not resolved automatically). + +Sometimes modules must be loaded in a specific order. For this, add a file `module.order` containing one module name (without `.ko`) per line: + +```sh +echo -e "foo\nbar" >module.order +mkdud --create foobar.dud --dist xxx bar.ko foo.ko module.order +``` + +If you have to prevent a (broken) kernel module to be loaded in the first +place tell the user to add `brokenmodules=foo` to the boot options. In the driver update that +provides a fixed version of `foo.ko` it is usually a good idea to remove this restriction so the module +will no get blocked in the target system. For this you can add a `brokenmodules` option to the driver update: + +```sh +mkdud --create foo.dud --dist xxx --config "brokenmodules=-foo" foo.ko +``` + + +## Update and load kernel modules with module parameters + +If you have to load an updated kernel module with additional module parameters you can use the `options` config option. +For example: + +```sh +mkdud --create foo.dud --dist xxx --config "options=foo.foo_option=1" foo.ko +``` + +will load `foo.ko` with `foo_option=1`. + + +## Set config (boot) options + +As shown in the examples above driver updates can also set boot options to influence the installation. For example: + +```sh +mkdud --create foo.dud --dist xxx --config "autoyast=https://example.com/foo.xml" --config "selfupdate=0" +``` + +Even loading further driver updates is possible: + +```sh +mkdud --create foo.dud --dist xxx --config "dud=https://example.com/bar.dud" +``` + + +## Update the installation system + +You can also replace or add any file in the installation system. For this, create a directory structure and put the file into it. +For example, this will add the content of the `/tmp/foo` directory to the installation system: + +```sh +mkdir -p /tmp/foo/usr/bin +cp /usr/bin/cat /tmp/foo/usr/bin +mkdud --create foo.dud --dist xxx --name "Replace cat" /tmp/foo +``` + +For an easier way to update complete packages see the next section. + + +## Update packages + +Packages might be updated in the installation system (when needed during installation) and in the target system (if they +are needed also later). Here's an example: + +```sh +mkdud --create foo.dud --dist xxx foo.rpm +``` + +This replaces `foo.rpm` in the installation system **and** the target system. The update in the target system is done +by creating a temporary add-on repository and adding the packages there. The repositories show up as `DriverUpdateX` in +the installer's package manager. + +You can fine-tune package updates with `mkdud`'s `--install` option. For example, to only update the installation system, not the +target system: + +```sh +mkdud --create foo.dud --dist xxx --install instsys foo.rpm +``` + + +## Adjust installation workflow + +Driver updates also prove hooks into the installation workflow. This is done by triggering shell scripts at specific points. They are + +- `update.pre`: right before yast will be started +- `update.post`: after packages have been installed +- `update.post2`: after everything has been done, right before the target system will be unmounted and rebooted + +Simply pass the scripts with the above exact names to `mkdud`: + +```sh +echo "systemctl enable foo" > update.post2 +mkdud --create foo.dud --dist xxx update.post2 +``` + +to enable the `foo` service. + + +## Execute commands + +Sometimes the commands in `update.pre` are run too late for your purpose. It is possible to run commands immediately (when the driver update has +been loaded). For this, the `exec` config option can be used. For example + +```sh +mkdud --create foo.dud --dist xxx --config "exec=rmmod foo" +``` + +would unload module `foo` immediately. + + +## Combine driver updates + +A driver update is not limited to a single action. You can specify all the described things in a single command line, repeating `mkdud` options +as needed. + +But if you are handed a number of driver updates and have to apply them all you might simplify things by combining them into a +single driver update: + +```sh +mkdud --create all.dud foo.dud bar.dud zap.dud +``` + + +## Debug problems + +Sometimes things just don't work as expected. Here's what to do: + +- use `mkdud --show foo.dud` to get an overview of what's inside the driver update +- make sure the product specified with `--dist` matches the intended product + +If you see the driver update loaded but it's apparently not applied, do this: + +- start the installation with the `startshell=1` boot option; this will open a shell instead of starting the installer (the regular +workflow continues when you exit this shell) +- there must be an `/update` directory with subdirectories for each driver update; if it's missing, your driver update has not been recognized +- check the correct product: the `UpdateDir` entry in `/linuxrc.config` contains the expected string +- have a look at `/var/log/linuxrc.log` to see what linuxrc did with the driver update so far diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/mkdud-1.36/VERSION new/mkdud-1.38/VERSION --- old/mkdud-1.36/VERSION 2018-08-27 14:50:19.000000000 +0200 +++ new/mkdud-1.38/VERSION 2018-11-15 14:35:09.000000000 +0100 @@ -1 +1 @@ -1.36 +1.38 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/mkdud-1.36/changelog new/mkdud-1.38/changelog --- old/mkdud-1.36/changelog 2018-08-27 14:50:19.000000000 +0200 +++ new/mkdud-1.38/changelog 2018-11-15 14:35:09.000000000 +0100 @@ -1,3 +1,13 @@ +2018-11-15: 1.38 + - merge gh#openSUSE/mkdud#21 + - add HOWTO describing some typical uses + - update git2log script + - fix typo + +2018-11-15: 1.37 + - merge gh#openSUSE/mkdud#20 + - put 'module.config' files into module directory + 2018-08-27: 1.36 - merge gh#openSUSE/mkdud#18 - look also at ~/.config/osc/oscrc for osc config @@ -77,6 +87,7 @@ - merge gh#openSUSE/mkdud#3 - Convert README to Markdown - Add information about openSUSE development + - Convert documentation to markdown 2015-04-30: 1.18 - support creating duds in iso9660 format @@ -102,7 +113,7 @@ 2015-01-21: 1.11 - merge gh#openSUSE/mkdud#1 - - Fix typo + - Fix typos 2014-10-13: 1.10 - added 'archive' Makefile target diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/mkdud-1.36/git2log new/mkdud-1.38/git2log --- old/mkdud-1.36/git2log 2018-08-27 14:50:19.000000000 +0200 +++ new/mkdud-1.38/git2log 2018-11-15 14:35:09.000000000 +0100 @@ -258,6 +258,8 @@ # # Only tags recognized by is_formatted_tag() are considered. # +# Tags inside merge commits are ignored. +# # The parsed logs is stored in $config->{log}, an array of log sections. # Each section is a hash with these keys: # - 'tags': array of tags for this section @@ -268,12 +270,21 @@ { my $log_entry; + # the end of the merge commit if in a merge + my $merge; + for (@{$config->{raw_log}}) { if(/^commit (\S+)( \((.*)\))?/) { my $commit = $1; my $tag_list = $3; my $xtag; + # we have reached the end of the merge commit + undef $merge if $merge && $commit =~ /^$merge/; + + # ignore tag info inside a merge commit + $tag_list = "" if $merge; + for my $t (split /, /, $tag_list) { if($t =~ /tag: (\S+)/) { my $tag = $1; @@ -293,6 +304,10 @@ $log_entry = { commit => $commit } if !$log_entry; } } + elsif(!$merge && /^Merge: (\S+)/) { + # remember end of merge + $merge = $1; + } push @{$log_entry->{lines}}, $_ if $log_entry; } @@ -575,7 +590,6 @@ my $merge; my $commit; - my $saved_commit; my $commits; for (@{$log->{lines}}) { @@ -593,67 +607,57 @@ if($merge) { $commit->{merge_ref} = $merge->{ref}; $commit->{date} = $merge->{date}; + $commit->{author} = $merge->{author}; # add to all commits so it's not lost when we re-arrange - $commit->{merge_msg} = $merge->{msg} if $merge->{msg}; - # saved entry no longer needed - undef $saved_commit; + $commit->{merge_msg} = $merge->{msg}; } next; } - if(/^Merge: (\S+)/) { + if(/^Merge: (\S+)/ && !$merge) { if($commit) { $merge = { merge_end => $1, ref => $commit->{ref} } unless $merge; - $saved_commit = pop @{$commits}; } - undef $commit; next; } if(/^Date:\s+(\S.*)/) { - if($commit) { - $commit->{date} = $1 if !$commit->{date}; - } - elsif($merge) { - $merge->{date} = $1 if !$merge->{date}; - } + $commit->{date} ||= $1 if $commit; + $merge->{date} ||= $1 if $merge; next; } if(/^Author:\s+(\S.*)/) { - $commit->{author} = $1 if $commit; - $merge->{author} = $1 if $merge && !$merge->{author}; + $commit->{author} ||= $1 if $commit; + $merge->{author} ||= $1 if $merge; next; } - if($commit) { - push @{$commit->{lines}}, $_ if s/^ //; - } - elsif($merge && !$merge->{msg}) { + if($merge) { if(/^ Merge pull request (#\d+) from (\S+)/) { if($config->{github_project}) { - $merge->{msg} = "merge gh#$config->{github_project}$1"; + push @{$merge->{msg}}, "merge gh#$config->{github_project}$1"; } else { - $merge->{msg} = "merge pr $2"; + push @{$merge->{msg}}, "merge pr $2"; } } - elsif(/^ Merge branch '([^']+)'/) { - $merge->{msg} = "merge branch $1"; + elsif(/^ Merge branch '([^']+)'( into)?/) { + push @{$merge->{msg}}, "merge branch $1" if $2 eq ""; + } + elsif(/^ Merge remote-tracking branch /) { + # ignore + } + elsif(s/^ //) { + push @{$commit->{lines}}, $_ unless /^# /; + } + } + elsif($commit) { + if(s/^ //) { + push @{$commit->{lines}}, $_ unless /^# /; } } - } - - # it can happen that there's a lonely merge commit left at the end - if($merge && $saved_commit) { - $saved_commit->{merge_ref} = $merge->{ref}; - $saved_commit->{date} = $merge->{date}; - $saved_commit->{author} = $merge->{author}; - $saved_commit->{merge_msg} = $merge->{msg} if $merge->{msg}; - $saved_commit->{formatted} = []; - - push @{$commits}, $saved_commit; } # Note: the individual steps below work on the array @$commits and modify @@ -671,6 +675,11 @@ my $tagged_merges = {}; for my $commit (@$commits) { + # skip leading empty lines + for (@{$commit->{lines}}) { + last if !/^\s*$/; + shift @{$commit->{lines}}; + } my $para_cnt = 0; my $delete_all = 0; my $delete_first = 0; @@ -735,7 +744,7 @@ for my $commit (@$commits) { next unless $commit->{formatted}; for (@{$commit->{formatted}}) { - s/(fate|bnc|bsc)\s*(#\d+)/\L$1\E$2/ig; + s/(fate|bnc|bsc|boo)\s*(#\d+)/\L$1\E$2/ig; } } @@ -751,10 +760,10 @@ $merge_logged->{$commit->{merge_ref}} = 1; if($commit->{merge_msg}) { if($opt_merge_msg_before) { - unshift @{$commit->{formatted}}, $commit->{merge_msg}; + unshift @{$commit->{formatted}}, @{$commit->{merge_msg}}; } else { - push @{$commit->{formatted}}, $commit->{merge_msg}; + push @{$commit->{formatted}}, @{$commit->{merge_msg}}; } } } @@ -802,6 +811,36 @@ } # step 8 + # - remove identical lines + + for my $commit (@$commits) { + next unless $commit->{formatted}; + my %k; + $commit->{formatted} = [ grep { !$k{$_}++ } @{$commit->{formatted}} ] + } + + # step 9 + # - remove shortened lines (that match the beginning of other lines) + + for my $commit (@$commits) { + next unless $commit->{formatted}; + + # return 1 if some other commit line starts with function arg + my $is_substr = sub { + my $str = $_[0]; + $str =~ s/\s*…$//; # github likes to shorten lines with ' …' + my $str_len = length $str; + for (@{$commit->{formatted}}) { + return 1 if $str_len < length($_) && $str eq substr($_, 0, $str_len); + } + + return 0; + }; + + $commit->{formatted} = [ grep { ! $is_substr->($_) } @{$commit->{formatted}} ] + } + + # step 10 # - add line breaks for my $commit (@$commits) { @@ -811,7 +850,7 @@ } } - # step 9 + # step 11 # - generate final log message # # note: non-(open)suse email addresses are replaced by $opt_default_email diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/mkdud-1.36/mkdud new/mkdud-1.38/mkdud --- old/mkdud-1.36/mkdud 2018-08-27 14:50:19.000000000 +0200 +++ new/mkdud-1.38/mkdud 2018-11-15 14:35:09.000000000 +0100 @@ -483,7 +483,7 @@ - kernel modules. - - 'module.order' files. See driver update documentation. + - 'module.order' and 'module.config' files. See driver update documentation. - 'update.pre', 'update.post', 'update.post2' scripts. See driver update documentation. @@ -838,7 +838,7 @@ return; } } - elsif(-f $_[0] && $_[0] =~ m#(^|/)(update\.(pre|post|post2)|module\.order)$#) { + elsif(-f $_[0] && $_[0] =~ m#(^|/)(update\.(pre|post|post2)|module\.(order|config))$#) { push @files, { type => $2, file => $_[0] }; return; @@ -1146,6 +1146,12 @@ system "cat '$_->{file}' >>$base/modules/module.order"; } + if($_->{type} eq 'module.config') { + $dud_ok = 1; + mkdir "$base/modules", 0755; + system "cat '$_->{file}' >>$base/modules/module.config"; + } + if($_->{type} eq 'module') { $dud_ok =1; mkdir "$base/modules", 0755; @@ -2009,6 +2015,7 @@ } $sect{modules} .= " module.order\n" if -f "$dir/modules/module.order"; + $sect{modules} .= " module.config\n" if -f "$dir/modules/module.config"; # ---------------------------- # packages