Bug#1064973: diffoscope fails with struct.error: unpack requires a buffer of 4 bytes

[Zbigniew Jędrzejewski-Szmek]

On Thu, Feb 29, 2024 at 11:19:50AM +, Chris Lamb wrote:
> The traceback/error is fixed in Git, pending upload:
>   
> https://salsa.debian.org/reproducible-builds/diffoscope/commit/466523ac6fbd1437635f8393fb93c37ff59341b3

Thanks!

Bug#1064973: diffoscope fails with struct.error: unpack requires a buffer of 4 bytes

[Zbigniew Jędrzejewski-Szmek]

Package: diffoscope
Version: 257
Severity: normal

The package is really diffoscope-257-1.fc40.x86_64.

$ diffoscope cache/rpms/meson-1.3.2-1.fc41/meson-1.3.2-1.fc41.noarch.rpm 
cache/build/meson-1.3.2-1.fc41/rebuild/meson-1.3.2-1.fc41.noarch.rpm
...
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/diffoscope/main.py", line 766, in main
sys.exit(run_diffoscope(parsed_args))
 ^^^
  File "/usr/lib/python3.12/site-packages/diffoscope/main.py", line 717, in 
run_diffoscope
difference = compare_root_paths(path1, path2)
 
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", 
line 69, in compare_root_paths
difference = compare_files(file1, file2)
 ^^^
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", 
line 149, in compare_files
return file1.compare(file2, source)
   
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 
532, in compare
difference = self._compare_using_details(other, source)
 ^^
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 
467, in _compare_using_details
details.extend(
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", 
line 197, in compare_pair
difference = compare_files(
 ^^
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", 
line 149, in compare_files
return file1.compare(file2, source)
   
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 
532, in compare
difference = self._compare_using_details(other, source)
 ^^
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 
467, in _compare_using_details
details.extend(
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", 
line 197, in compare_pair
difference = compare_files(
 ^^
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", 
line 149, in compare_files
return file1.compare(file2, source)
   
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 
532, in compare
difference = self._compare_using_details(other, source)
 ^^
  File 
"/usr/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 
433, in _compare_using_details
details.extend(self.compare_details(other, source))
   ^^^
  File "/usr/lib/python3.12/site-packages/diffoscope/comparators/python.py", 
line 52, in compare_details
describe_pyc(other.path),

  File "/usr/lib/python3.12/site-packages/diffoscope/comparators/python.py", 
line 65, in describe_pyc
return "\n".join(parse_pyc(f))
   ^^^
  File "/usr/lib/python3.12/site-packages/diffoscope/comparators/python.py", 
line 74, in parse_pyc
modtime = time.asctime(time.gmtime(struct.unpack("https://fedorapeople.org/~zbyszek/meson-1.3.2-1.fc41.noarch.rpm
https://fedorapeople.org/~zbyszek/meson-1.3.2-1.fc41.noarch.rpm.2

Zbyszek

Bug#954490: systemd: resolved.conf "allow-downgrade" doesn't work

[Zbigniew Jędrzejewski-Szmek]

On Sat, Apr 04, 2020 at 10:41:31AM +0300, Fanis Dokianakis wrote:
> I confirm that this bug exists after upgrading systemd. Systemd-resolved
> *sometimes* does not downgrade and SERVERFAILS on all domains that do not
> have a signature dns record.

That's not what "allow-downgrade" means. The downgrade happens when the
configured DNS server does not support DNSSEC, not when some domain has
an invalid signature.

> The error with resolvectl query is
> $ resolvectl query example.domain
> example.domain: resolve call failed: DNSSEC validation failed: no-signature

Please give an actual domain name that fails resolution. Not providing
a reproducer just makes this harder for anyone trying to resolve this.

Zbyszek

Bug#946456: systemd: Provide systemd-sysusers as an independent package

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 30, 2020 at 11:51:48PM -0300, Felipe Sateler wrote:
> On Thu, Jan 30, 2020 at 6:40 PM Michael Biebl  wrote:
> 
> > Hi Felipe
> >
> > Am 30.01.20 um 22:30 schrieb Felipe Sateler:
> > >
> > >
> > > On Thu, Jan 30, 2020 at 1:39 PM Michael Biebl  > > > wrote:
> > >
> > > Am 28.01.20 um 17:27 schrieb Ansgar:
> > > > On Tue, 2020-01-28 at 16:51 +0100, Michael Biebl wrote:
> > > >> Am 28.01.20 um 14:59 schrieb Ansgar:
> > > >>> I tried linking systemd-{sysusers,tmpfiles} statically against
> > > >>> systemd's private library earlier this month.  It increases the
> > > >>> binaries size by ~100 kB (compared to Installed-Size: 14.2 MB of
> > > >>> systemd that is just one percent).
> > > >>
> > > >> Is that 100K per binary?
> > > >
> > > > I checked my notes at it was 100 kB per binary: they are 212 kB
> > larger
> > > > (sysusers 51 kB → 137 kB, tmpfiles 84 kB → 212 kB); I tested with
> > > > systemd 243-8.
> > > >
> > > > It might be possible to make it a bit smaller if one was to somehow
> > > > link libsystemd0 for functions available there (libsystemd-shared
> > > > currently duplicates those).
> > >
> > >
> > > That is not possible. There is global state that is not to be shared.
> > > See https://github.com/systemd/systemd/pull/3516#issuecomment-227482524
> >
> > What's your thought on how to solve this libsystemd-shared issue should
> > we consider splitting out systemd-{sysusers,tmpfiles}
> >
> > - link statically (and carry a downstream patch for eternity)
> > - move libsystemd-shared to systemd-utils and risk the breakage that can
> > result from a partial/aborted upgrade
> > - copy, instead of move, the binaries + libsystemd-shared and make the
> > resulting systemd-utils package Conflict with systemd (instead of having
> > systemd depend on systemd-utils)
> > - something else?
> >
> 
> I tried linking statically the "can run without systemd-pid1 tools" with
> the attached patch.
> 
> Disk usage appears to increase by about 400 kb:
> % dpkg --info systemd_244.1-1_amd64.deb|grep Installed
> 
>  Installed-Size: 13908
> % dpkg --info ../systemd_244-3_amd64.deb|grep Installed
>  Installed-Size: 14319
> 
> Maybe upstream can be persuaded to merge something like this?
> 
> -- 
> 
> Saludos,
> Felipe Sateler

> diff --git a/meson.build b/meson.build
> index b8dff8cd94..39cef6b301 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1493,6 +1493,29 @@ make_autosuspend_rules_py = 
> find_program('tools/make-autosuspend-rules.py')
>  
>  
>  
> +
> +libutil = static_library(
> +'util',
> +[
> +'src/shared/acl-util.c',
> +'src/shared/enable-mempool.c',
> +'src/shared/id128-print.c',
> +'src/shared/pager.c',
> +'src/shared/path-lookup.c',
> +'src/shared/pretty-print.c',
> +'src/shared/spawn-ask-password-agent.c',
> +'src/shared/spawn-polkit-agent.c',
> +'src/shared/specifier.c',
> +'src/shared/sysctl-util.c',
> +'src/shared/sysctl-util.h',
> +'src/shared/tmpfile-util-label.c',
> +'src/shared/uid-range.c',
> +'src/shared/verbs.c',
> +] + id128_sources,
> +link_with: [libbasic, libsystemd_static],
> +include_directories: includes
> +)

Is creating a separate static library actually necessary?  What would
the results be if those binaries were linked to the static version of
libshared that we already provide support for? I hope the compiler
would be able to drop any unused objects and achieve the same size,
without having to maintain yet another list of files.

>  # binaries that have --help and are intended for use by humans,
>  # usually, but not always, installed in /bin.
>  public_programs = []
> @@ -1537,6 +1560,7 @@ test_dlopen = executable(
>  include_directories : includes,
>  link_with : [libbasic],
>  dependencies : [libdl],
> +b_lto: false,
>  build_by_default : want_tests != 'false')
>  
>  foreach tuple : [['myhostname', 'ENABLE_NSS_MYHOSTNAME'],
> @@ -2407,7 +2431,7 @@ install_data('src/sleep/sleep.conf',
>  exe = executable('systemd-sysctl',
>   'src/sysctl/sysctl.c',
>   include_directories : includes,
> - link_with : [libshared],
> + link_with : [libutil],

To upstream this, the change in behaviour must be configuration-time
selectable.

Zbyszek

Bug#947935: ITP: opentmpfiles -- standalone utility written to process systemd-style tmpfiles.d files

[Zbigniew Jędrzejewski-Szmek]

[resend]

On Thu, Jan 02, 2020 at 01:35:26PM +0100, Thomas Goirand wrote:
> * URL : https://github.com/OpenRC/opentmpfiles

I had a quick look at the repo. This is not a comprehensive review,
just a quick list of the most obvious items:

opentmpfiles doesn't implement quite a few options: --root, --user,
v, q, e, Q, f+, a, A, m, x, X.
The last two are very important: without support for excludes, data loss is
likely. The first is even more important obviously to avoid mucking with
the host system unexpectedly.

opentmpfiles seems to have no support for the BSD lock protocol to
exclude directories from cleanup, again potential data loss.

Time-based cleanup also seems to be completely missing. (Systemd
relies on that to not fill up temporary partitions with stale
data. Lack of time-based cleanup becomes very problematic for
long-running systems.)

(Note that implementing cleanup in shell is hard, because files must
be opened with NO_ATIME to avoid bumping the access times and actually
preventing time-based cleanup from happening.)

I don't see any support for specifier expansion, so operation on wrong
files and bogus contents when written to files are likely.

w appends instead of replacing files.

systemd-tmpfiles automatically excludes unmounted autofs mountpoints,
to avoid needlessly triggering mounts. I see no support for that.

systemd-tmpfiles has a lot of very careful code to avoid unintended
privilege escalation. For example, when doing a chmod+chown, the
access mask is reduced first to avoid a time when a bigger privilege
mask would be set for the wrong owner. The actual chmod is done at the
end so the kernel can adjust the suid bit and other permissions
appropriately.

Similarly, systemd-tmpfiles is very careful when following paths which
go through a directory owned by an unprivileged user (e.g. if we are
to chown /run/foo/bar/passwd, where /run/foo belongs to user foo, we
should refuse operation if /run/foo/bar is a symlink to /etc. The
check for this needs to be done with TOCTOU in mind, i.e. using *at()
variants of all syscalls so that the foo user cannot substitute the
link between the time /run/foo/bar ownership is checked and the chmod
operation on /run/foo/bar/passwd.). Doing this in shell is going to be
hard. The code in opentmpfiles seems to have no notion of ownership
checks, i.e. is open to unintended privilege escalation.

In summary, treating this implementation as a drop-in replacement is
not a good idea. In particular, important missing functionality would
immediately impair operation of systemd on systems where such
substitution would be done. Also, the parts that are not implemented
are the hard parts. Even though it seems that approx. half of
functionality is available, the implementation effort of that second
half would be much grater, and the use of shell makes it a daunting
task.

Zbyszek

Bug#947935: ITP: opentmpfiles -- standalone utility written to process systemd-style tmpfiles.d files

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 02, 2020 at 01:35:26PM +0100, Thomas Goirand wrote:
> * URL : https://github.com/OpenRC/opentmpfiles

I had a quick look at the repo. This is not a comprehensive review,
just a quick list of the most obvious items:

opentmpfiles doesn't implement quite a few options: --root, --user,
v, q, e, Q, f+, a, A, m, x, X.
The last two are very important: without support for excludes, data loss is
likely. The first is even more important obviously to avoid mucking with
the host system unexpectedly.

opentmpfiles seems to have no support for the BSD lock protocol to
exclude directories from cleanup, again potential data loss.

Time-based cleanup also seems to be completely missing. (Systemd
relies on that to not fill up temporary partitions with stale
data. Lack of time-based cleanup becomes very problematic for
long-running systems.)

(Note that implementing cleanup in shell is hard, because files must
be opened with NO_ATIME to avoid bumping the access times and actually
preventing time-based cleanup from happening.)

I don't see any support for specifier expansion, so operation on wrong
files and bogus contents when written to files are likely.

w appends instead of replacing files.

systemd-tmpfiles automatically excludes unmounted autofs mountpoints,
to avoid needlessly triggering mounts. I see no support for that.

systemd-tmpfiles has a lot of very careful code to avoid unintended
privilege escalation. For example, when doing a chmod+chown, the
access mask is reduced first to avoid a time when a bigger privilege
mask would be set for the wrong owner. The actual chmod is done at the
end so the kernel can adjust the suid bit and other permissions
appropriately.

Similarly, systemd-tmpfiles is very careful when following paths which
go through a directory owned by an unprivileged user (e.g. if we are
to chown /run/foo/bar/passwd, where /run/foo belongs to user foo, we
should refuse operation if /run/foo/bar is a symlink to /etc. The
check for this needs to be done with TOCTOU in mind, i.e. using *at()
variants of all syscalls so that the foo user cannot substitute the
link between the time /run/foo/bar ownership is checked and the chmod
operation on /run/foo/bar/passwd.). Doing this in shell is going to be
hard. The code in opentmpfiles seems to have no notion of ownership
checks, i.e. is open to unintended privilege escalation.

In summary, treating this implementation as a drop-in replacement is
not a good idea. In particular, important missing functionality would
immediately impair operation of systemd on systems where such
substitution would be done. Also, the parts that are not implemented
are the hard parts. Even though it seems that approx. half of
functionality is available, the implementation effort of that second
half would be much grater, and the use of shell makes it a daunting
task.

Zbyszek

Bug#891903: diffoscope: programming error in except clause

[Zbigniew Jędrzejewski-Szmek]

Package: diffoscope
Version: 91
Severity: normal

Dear Maintainer,

I'm trying to update diffoscope to version 91 in Fedora rawhide, and
the tests failing for reasons I haven't diagnosed yet (it builds fine
locally, but fails on an s390x builder). When the tests are failing, a
try..except clause is reached which normally wouldn't be reached, and the
code there is non-python3-compatible:

E   subprocess.CalledProcessError: Command '['objdump', 
'--line-numbers', '--disassemble', '--demangle', '--section=.text', 
'/tmp/tmp5ujok4po_diffoscope/0/2.o']' returned non-zero exit status 1.
diffoscope/feeders.py:94: CalledProcessError
During handling of the above exception, another exception occurred:
rlib1 = < 
/builddir/build/BUILD/diffoscope-91/tests/data/test1.rlib>
rlib2 = < 
/builddir/build/BUILD/diffoscope-91/tests/data/test2.rlib>
@pytest.fixture
def differences(rlib1, rlib2):
>   return rlib1.compare(rlib2).details
tests/comparators/test_rlib.py:53: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
diffoscope/comparators/utils/file.py:362: in compare
difference = self._compare_using_details(other, source)
diffoscope/comparators/utils/file.py:317: in _compare_using_details
details.extend(self.as_container.compare(other.as_container, 
no_recurse=no_recurse))
diffoscope/comparators/utils/container.py:174: in compare_pair
difference = compare_files(file1, file2, source=None, 
diff_content_only=no_recurse)
diffoscope/comparators/utils/compare.py:117: in compare_files
return file1.compare(file2, source)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = < alloc_system-d16b8f0e.0.o>
other = < alloc_system-d16b8f0e.0.o>, source = None
def compare(self, other, source=None):
if hasattr(self, 'compare_details') or self.as_container:
try:
difference = self._compare_using_details(other, source)
# no differences detected inside? let's at least do a binary 
diff
if difference is None:
difference = self.compare_bytes(other, source=source)
if difference is None:
return None
difference.add_comment(
"No file format specific differences found inside, "
"yet data differs ({})".format(self.magic_file_type),
)
except subprocess.CalledProcessError as e:
difference = self.compare_bytes(other, source=source)
if e.output:
>   output = re.sub(r'^', '', e.output.decode('utf-8', 
> errors='replace'), flags=re.MULTILINE)
E   AttributeError: 'str' object has no attribute 'decode'
diffoscope/comparators/utils/file.py:375: AttributeError

It seem pretty clear that .decode() is called on a str object, which cannot 
work.

The full log is at 
https://kojipkgs.fedoraproject.org//work/tasks/2918/25412918/build.log.

Zbyszek

Bug#808151: pr submitted

[Zbigniew Jędrzejewski-Szmek]

https://github.com/systemd/systemd/pull/6316

Bug#865528: don't escape dash (\x2d) when logging

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jun 22, 2017 at 11:21:08PM +0200, Michael Biebl wrote:
> Am 22.06.2017 um 16:34 schrieb 積丹尼 Dan Jacobson:
> > Well
> > $ systemctl status AA.device
> > ● AA.device
> >Loaded: loaded
> >Active: inactive (dead)
> > if the user doesn't carefully replace \ with \\ he will get the dead
> > message anyway. Wouldn't it be more accurate to say that that device
> > could not be found?
> 
> That's a separate issue. I'm not entirely sure why
> systemctl status does-not-exist.service
> and
> systemctl status does-not-exist.device
> differ regarding their behaviour. You'd probably need to ask upstream
> about that.

.device units normally don't have a file on disk and are created
automatically .service units without don't make much sense, so systemctl
was changed to print an error in that case to avoid confusion when
a typo is made.

Zbyszek

Bug#856447: diffoscope: test_icc::test_diff fails due to locale difference in output

[Zbigniew Jędrzejewski-Szmek]

On Fri, Mar 03, 2017 at 04:11:57PM +0800, Chris Lamb wrote:
>   https://bugs.debian.org/847595
>   
> https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=fcbcafa1b1cefb08a7ffbd8936f22c5bc6849991

Cool, thanks!

I added a note to our spec file, and we'll just wait for the fixes to
percolate through upstream releases.

Zbyszek

Bug#817193: diffoscope: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

On Sun, Feb 26, 2017 at 05:37:51PM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Feb 23, 2017 at 05:48:14PM +0800, Chris Lamb wrote:
> > tags 817193 + unreproducible
> > thanks
> > 
> > Chris Lamb wrote:
> > 
> > > Are you still any of these test failures with a recent diffoscope
> > > version? We have made a few locale changes recently so its likely
> > > that at least some of these are fixed.
> > 
> > As I can't reproduce this and there has been no reply I'm going to
> > go ahead and close it. Re-open if you can still demonstrate the issue
> > of course. :)
> 
> Hi,
> sorry for the late reply. Too many packages ;)
> 
> With diffoscope-77, I get the following failures:
> (Fedora rawhide amd64 mock, export LC_CTYPE=en_US.utf8 TZ=UTC)
> 
> === FAILURES 
> ===
> _ test_listing 
> _
> 
> differences = [, 
> ]
> 
> @skip_unless_tools_exist('cbfstool')
> def test_listing(differences):
> expected_diff = get_data('cbfs_listing_expected_diff')
> >   assert differences[0].unified_diff == expected_diff
> E   assert '@@ -1,3 +1,3...  31896\n' == '@@ -1,6 +1,3 ...  
> 31896\n'
> E - @@ -1,3 +1,3 @@
> E ?   ^
> E + @@ -1,6 +1,3 @@
> E ?   ^
> E + -32 kB, bootblocksize 0, romsize 32768, offset 0x0
> E + -alignment: 64 bytes, architecture: x86
> E + -
> EName   Offset Type Size
> E   -text   0x0raw  446
> E   -(empty)0x200  null 32152
> E   +text   0x0raw  671
> E   +(empty)0x300  null 31896
> 
> tests/comparators/test_cbfs.py:80: AssertionError
>  Captured stderr setup 
> -
> Created CBFS (capacity = 32664 bytes)
> Created CBFS (capacity = 32664 bytes)
> __ test_diff 
> ___
> 
> differences = []
> 
> @skip_unless_tools_exist('cd-iccdump')
> def test_diff(differences):
> expected_diff = get_data('icc_expected_diff')
> >   assert differences[0].unified_diff == expected_diff
> E   assert '@@ -1,20 +1,... [24 bytes]\n' == '@@ -1,20 +1,2... [24 
> bytes]\n'
> E   @@ -1,20 +1,20 @@
> Eicc:
> EHeader:
> E  Size   = 14684 bytes
> E  Version= 4.3
> E  Profile Kind   = display-device
> E  Colorspace = rgb
> E  Conn. Space= xyz
> E   -  Date, Time = 2016-02-15, 21:02:09
> E   +  Date, Time = 2016-02-15, 21:03:22
> E  Flags  = Not embedded profile, Use anywhere
> E  Dev. Attrbts   = reflective, glossy
> E  Rndrng Intnt   = perceptual
> E  Creator= lcms
> E   -  Profile ID = 0x0477fa4b
> E   +  Profile ID = 0x06017f17
> E
> Etag 00:
> E  sig'desc' [0x64657363]
> E  size   38
> E  type   'mluc' [0x6d6c7563]
> EText:
> E -ne_SU: sRGB [24 bytes]
> E ? -  -
> E +en_US: sRGB [24 bytes]
> E ?+  +
> 
> tests/comparators/test_icc.py:47: AssertionError
> == 2 failed, 249 passed, 42 skipped in 44.78 seconds 
> ===
> 
> The second issue looks like a real difference.
> 
> The first issue I think is just a slightly different diff context between
> the texts. I have seen something similar when generating patches using 
> different
> versions of git.

Actually no, the first also looks like a real difference. For some reason
the expected header is not printed:

$ "cbfstool" "/tmp/pytest-of-zbyszek/pytest-11/test_listing0/coreboot1" "print"
Name   Offset Type Size
text   0x0raw  446
(empty)0x200  null 32152

$ "cbfstool" "/tmp/pytest-of-zbyszek/pytest-11/test_listing0/coreboot2.rom" 
"print"
Name   Offset Type Size
text   0x0raw  671
(empty)0x300  null 31896

$ rpm -qf /bin/cbfstool
coreboot-utils-4.2-2.fc24.x86_64
coreboot-utils-4.5-2.fc26.x86_64 (the same)

Bug#817193: diffoscope: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

On Thu, Feb 23, 2017 at 05:48:14PM +0800, Chris Lamb wrote:
> tags 817193 + unreproducible
> thanks
> 
> Chris Lamb wrote:
> 
> > Are you still any of these test failures with a recent diffoscope
> > version? We have made a few locale changes recently so its likely
> > that at least some of these are fixed.
> 
> As I can't reproduce this and there has been no reply I'm going to
> go ahead and close it. Re-open if you can still demonstrate the issue
> of course. :)

Hi,
sorry for the late reply. Too many packages ;)

With diffoscope-77, I get the following failures:
(Fedora rawhide amd64 mock, export LC_CTYPE=en_US.utf8 TZ=UTC)

=== FAILURES ===
_ test_listing _

differences = [, 
]

@skip_unless_tools_exist('cbfstool')
def test_listing(differences):
expected_diff = get_data('cbfs_listing_expected_diff')
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1,3 +1,3...  31896\n' == '@@ -1,6 +1,3 ...  31896\n'
E - @@ -1,3 +1,3 @@
E ?   ^
E + @@ -1,6 +1,3 @@
E ?   ^
E + -32 kB, bootblocksize 0, romsize 32768, offset 0x0
E + -alignment: 64 bytes, architecture: x86
E + -
EName   Offset Type Size
E   -text   0x0raw  446
E   -(empty)0x200  null 32152
E   +text   0x0raw  671
E   +(empty)0x300  null 31896

tests/comparators/test_cbfs.py:80: AssertionError
 Captured stderr setup -
Created CBFS (capacity = 32664 bytes)
Created CBFS (capacity = 32664 bytes)
__ test_diff ___

differences = []

@skip_unless_tools_exist('cd-iccdump')
def test_diff(differences):
expected_diff = get_data('icc_expected_diff')
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1,20 +1,... [24 bytes]\n' == '@@ -1,20 +1,2... [24 bytes]\n'
E   @@ -1,20 +1,20 @@
Eicc:
EHeader:
E  Size = 14684 bytes
E  Version  = 4.3
E  Profile Kind = display-device
E  Colorspace   = rgb
E  Conn. Space  = xyz
E   -  Date, Time   = 2016-02-15, 21:02:09
E   +  Date, Time   = 2016-02-15, 21:03:22
E  Flags= Not embedded profile, Use anywhere
E  Dev. Attrbts = reflective, glossy
E  Rndrng Intnt = perceptual
E  Creator  = lcms
E   -  Profile ID   = 0x0477fa4b
E   +  Profile ID   = 0x06017f17
E
Etag 00:
E  sig  'desc' [0x64657363]
E  size 38
E  type 'mluc' [0x6d6c7563]
EText:
E -ne_SU:   sRGB [24 bytes]
E ? -  -
E +en_US:   sRGB [24 bytes]
E ?+  +

tests/comparators/test_icc.py:47: AssertionError
== 2 failed, 249 passed, 42 skipped in 44.78 seconds ===

The second issue looks like a real difference.

The first issue I think is just a slightly different diff context between
the texts. I have seen something similar when generating patches using different
versions of git.

Zbyszek

Bug#854515: submitting enjarify man page upstream

[Zbigniew Jędrzejewski-Szmek]

Package: enjarify
Version: 1:1.0.3-3

Hi,
I'm working on a Fedora package for enjarify [1], and I want to include the
man page [2] that Debian provides. We could pull it directly from the Debian
repo, but it'd be nice to have it "official". Could you submit the man
page upstream?

Thanks,
Zbyszek

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1415331
[2] 
https://anonscm.debian.org/cgit/android-tools/enjarify.git/plain/debian/enjarify.1

Bug#817193: failing tests with v62: test_listing, test_diff test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

I noticed that I was missing ppudmp from my installation, so tests
were skipped.  Once it is installed, the ppu tests also fail. Looks
like a programming error, PpuFile is File, not FilesystemFile from a
quick look.

=== FAILURES ===
_ test_identification __

file1 = < 
/builddir/build/BUILD/diffoscope/tests/data/test1.ppu>

@skip_unless_tools_exist('ppudump')
def test_identification(file1):
>   assert isinstance(file1, PpuFile)
E   assert isinstance(< 
/builddir/build/BUILD/diffoscope/tests/data/test1.ppu>, PpuFile)

tests/comparators/test_ppu.py:46: AssertionError
__ test_diff ___

differences = []

@skip_unless_tool_is_older_than('ppudump', ppudump_version, '3.0.0')
def test_diff(differences):
>   print(differences[0].unified_diff)
E   IndexError: list index out of range

tests/comparators/test_ppu.py:58: IndexError
__ test_compare_non_existing ___

monkeypatch = <_pytest.monkeypatch.monkeypatch object at 0xf3f7c52c>
file1 = < 
/builddir/build/BUILD/diffoscope/tests/data/test1.ppu>

@skip_unless_tool_is_older_than('ppudump', ppudump_version, '3.0.0')
def test_compare_non_existing(monkeypatch, file1):
>   assert_non_existing(monkeypatch, file1, has_null_source=False)

tests/comparators/test_ppu.py:64: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

monkeypatch = <_pytest.monkeypatch.monkeypatch object at 0xf3f7c52c>
fixture = < 
/builddir/build/BUILD/diffoscope/tests/data/test1.ppu>
has_null_source = False, has_details = True

def assert_non_existing(monkeypatch, fixture, has_null_source=True, 
has_details=True):
monkeypatch.setattr(Config(), 'new_file', True)
assert Config().new_file, "didnt get patched"

difference = fixture.compare(NonExistingFile('/nonexisting', fixture))

output_html(difference, print_func=print)
output_text(difference, print_func=print)

assert difference.source2 == '/nonexisting'
>   assert not has_details or len(difference.details) > 0
E   assert (not True or 0 > 0)
E+  where 0 = len([])
E+where [] = .details

tests/comparators/utils.py:79: AssertionError

Bug#817193: failing tests with v62: test_listing, test_diff test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

=== FAILURES ===
_ test_listing _

differences = [, 
]

@skip_unless_tools_exist('cbfstool')
def test_listing(differences):
expected_diff = open(data('cbfs_listing_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1,3 +1,3...  31896\n' == '@@ -1,6 +1,3 ...  31896\n'
E - @@ -1,3 +1,3 @@
E ?   ^
E + @@ -1,6 +1,3 @@
E ?   ^
E + -32 kB, bootblocksize 0, romsize 32768, offset 0x0
E + -alignment: 64 bytes, architecture: x86
E + -
EName   Offset Type Size
E   -text   0x0raw  446
E   -(empty)0x200  null 32152
E   +text   0x0raw  671
E   +(empty)0x300  null 31896

tests/comparators/test_cbfs.py:79: AssertionError
 Captured stdout setup -
--- /tmp/pytest-of-zbyszek/pytest-1/test_listing0/coreboot1
+++ /tmp/pytest-of-zbyszek/pytest-1/test_listing0/coreboot2.rom
├── cbfstool {} print
│ @@ -1,3 +1,3 @@
│  Name   Offset Type Size
│ -text   0x0raw  446
│ -(empty)0x200  null 32152
│ +text   0x0raw  671
│ +(empty)0x300  null 31896
├── text
│ @@ -1,6 +1,12 @@
│ +A common form of lorem ipsum reads:
│ +
│  Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod 
tempor
│  incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
│  nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
│  Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore 
eu
│  fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt 
in
│  culpa qui officia deserunt mollit anim id est laborum.
│ +
│ +"Lorem ipsum" text is derived from sections 1.10.32--3 of Cicero's De finibus
│ +bonorum et malorum (On the Ends of Goods and Evils, or alternatively [About]
│ +The Purposes of Good and Evil).
╵
 Captured stderr setup -
Created CBFS (capacity = 32664 bytes)
Created CBFS (capacity = 32664 bytes)
__ test_diff ___

differences = []

@skip_unless_tools_exist('cd-iccdump')
def test_diff(differences):
expected_diff = open(data('icc_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1,20 +1,... [24 bytes]\n' == '@@ -1,20 +1,2... [24 bytes]\n'
E   @@ -1,20 +1,20 @@
Eicc:
EHeader:
E  Size = 14684 bytes
E  Version  = 4.3
E  Profile Kind = display-device
E  Colorspace   = rgb
E  Conn. Space  = xyz
E   -  Date, Time   = 2016-02-15, 21:02:09
E   +  Date, Time   = 2016-02-15, 21:03:22
E  Flags= Not embedded profile, Use anywhere
E  Dev. Attrbts = reflective, glossy
E  Rndrng Intnt = perceptual
E  Creator  = lcms
E   -  Profile ID   = 0x0477fa4b
E   +  Profile ID   = 0x06017f17
E
Etag 00:
E  sig  'desc' [0x64657363]
E  size 38
E  type 'mluc' [0x6d6c7563]
EText:
E -ne_SU:   sRGB [24 bytes]
E ? -  -
E +en_US:   sRGB [24 bytes]
E ?+  +

tests/comparators/test_icc.py:45: AssertionError
== 2 failed, 208 passed, 38 skipped in 41.30 seconds ===

test_differences, which was failing before, now passes.

Zbyszek

Bug#832010: Please enable LZ4 compression

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jul 21, 2016 at 07:28:26PM +0200, Yuri D'Elia wrote:
> On Thu, Jul 21 2016, Michael Biebl  wrote:
> > And in Debian we build against libxz, so xz compression is used for core
> > files.
> 
> Indeed, and it's pretty slow.
> 
> > What would we gain by switching from xz to lz4. Can you provide numbers?
> 
> I don't have enough time to back it up comparing exactly the difference
> between LZ4 and XZ in coredump, but if you compare regular XZ to LZ4 you
> can expect anywhere between 10x and 50x improvement in compression
> speed while maintaining an acceptable compression ratio.

systemd itself includes a benchmark:

$ ./test-compress-benchmark
XZ/zeros: compressed & decompressed 57898278 bytes in 2.02s (27.37MiB/s), mean 
compresion 99.96%, skipped 0 bytes
LZ4/zeros: compressed & decompressed 2031958921 bytes in 2.00s (968.65MiB/s), 
mean compresion 99.60%, skipped 0 bytes
XZ/simple: compressed & decompressed 58918934 bytes in 2.01s (27.97MiB/s), mean 
compresion 99.95%, skipped 0 bytes
LZ4/simple: compressed & decompressed 2544957269 bytes in 2.00s (1213.26MiB/s), 
mean compresion 99.60%, skipped 0 bytes
XZ/random: compressed & decompressed 12258687 bytes in 2.08s (5.62MiB/s), mean 
compresion 44.33%, skipped 168354 bytes
LZ4/random: compressed & decompressed 2235751628 bytes in 2.00s (1065.61MiB/s), 
mean compresion 44.94%, skipped 24251355 bytes

You can also specify different test time as an argument.

Zbyszek

Bug#839224: python-systemd: "undefined symbol" error on armhf

[Zbigniew Jędrzejewski-Szmek]

Oops, I forgot I fixed this in git after tagging v232 (commit 8024fc6171).
I now added -Werror=implicit-function-declaration to compile flags so the
compilation should actually fail instead of just emitted a warning if the
same problem occurs hain.

You probably want to pull in most of those:

* 84fcfc05e5 setup.py: make build fail if undeclared symbols are used
* 911591a118 build-sys: import "pytest" instead of "py.test"
* 8024fc6171 _reader: use proper ifdef guard for sd_j_open_files_fd
* 35a5b281ad tests: add workaround for pre-232 system returning EINVAL on some 
flags
* b9767792b9 docs: fix sphinx format warning
* 177ac6d894 tests: skip fdstore tests if not implemented

Zbyszek

Bug#839224: python-systemd: "undefined symbol" error on armhf

[Zbigniew Jędrzejewski-Szmek]

On Fri, Sep 30, 2016 at 02:24:57PM +0200, Michael Biebl wrote:
> Am 30.09.2016 um 12:39 schrieb Patrick Scharrenberg:
> > Package: python-systemd
> > Version: 232-1~bpo8+1
> > Severity: normal
> > 
> > Dear Maintainer,
> > 
> > when trying to import JournalHandler on arm architecture I get the 
> > following error:
> > 
> >   root@raspbian:~# python -c "from systemd.journal import JournalHandler"
> >   Traceback (most recent call last):
> > File "", line 1, in 
> > File "/usr/lib/python2.7/dist-packages/systemd/journal.py", line 34, in 
> > 
> >   from ._reader import (_Reader, NOP, APPEND, INVALIDATE,
> >   ImportError: 
> > /usr/lib/python2.7/dist-packages/systemd/_reader.arm-linux-gnueabihf.so: 
> > undefined symbol: sd_journal_open_files_fd
> 
> ...
> 
> > Versions of packages python-systemd depends on:
> > ii  libc62.19-18+deb8u6
> > ii  libsystemd0  215-17+deb8u5
> 
> 
> 
> I guess we should bump the Build-Depends on libsystemd-dev to (>= 230)
> to ensure we build against a version with the new API. The version in
> stable was built against v215. Maybe we should build it against the bpo
> version.
> 
> I wonder if building against older versions, like v215, is supposed to
> work. Zbigniew?

There's is configuration code (based on libsystemd version as reported by
pkg-config), which decides which function calls can be used. It's most
likely that the libsystemd headers on the system on which compilation was
done were newer than the library on the target system. Other possibility
is that the configuration code is wrong. In the second case, there would
be warnings emitted during build that the function is not declared. It
would be interesting to see the logs.

Zbyszek

Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

On Mon, Aug 15, 2016 at 09:07:03PM +0100, Chris Lamb wrote:
> Zbigniew Jędrzejewski-Szmek wrote:
> 
> > I invoke the tests as "PYTHONPATH=. py.test-3 . -vv" locally. Above output
> > is from that.
> 
> Hm, cannot seem to reproduce.
> 
> Are you building inside a container, out of interest? I see some strange
> things building in some containers as they prevent writes to /etc/timezone
I see no difference between running on bare metal and the mock chroot.

PYTHONPATH=. py.test-3 . -vv -k test_differences
 test session starts 
=
platform linux -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- 
/usr/bin/python3
cachedir: .cache
rootdir: /home/zbyszek/python/diffoscope, inifile: 
plugins: cov-2.2.1
collected 233 items 

tests/comparators/test_dex.py::test_differences SKIPPED
tests/comparators/test_elf.py::test_differences_with_dbgsym SKIPPED
tests/comparators/test_epub.py::test_differences ERROR
tests/comparators/test_fsimage.py::test_differences ERROR

= ERRORS 

_ ERROR at setup of test_differences 


@pytest.fixture(autouse=True)
def set_locale():
>   diffoscope.set_locale()

tests/comparators/conftest.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
_ _ _

def set_locale():
"""Normalize locale so external tool gives us stable and properly
encoded output"""

>   1/0
E   ZeroDivisionError: division by zero

diffoscope/__init__.py:265: ZeroDivisionError
___ ERROR at setup of test_differences 
_

@pytest.fixture(autouse=True)
def set_locale():
>   diffoscope.set_locale()

tests/comparators/conftest.py:28: 
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
_ _ _

def set_locale():
"""Normalize locale so external tool gives us stable and properly
encoded output"""

>   1/0
E   ZeroDivisionError: division by zero

diffoscope/__init__.py:265: ZeroDivisionError
== 229 tests deselected by '-ktest_differences' 
=
== 2 skipped, 229 deselected, 2 error in 14.69 seconds 
==


Zbyszek

Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

On Mon, Aug 15, 2016 at 11:24:05AM +0100, Chris Lamb wrote:
> Hi Zbigniew,
> 
> > > https://kojipkgs.fedoraproject.org//work/tasks/890/13300890/build.log
> [..]
> > https://kojipkgs.fedoraproject.org//work/tasks/2994/13312994/build.log
> [..]
> > https://kojipkgs.fedoraproject.org//work/tasks/834/13300834/build.log)
> 
> Alas these links now 404. Could you let me know:
Hi,

unfortunately failed builds are purged from koji after some time. I periodically
forget about that.

> a) Whether you still see these errors (and link to fresher URLs?)
I see only a single error. It was also on that list. Others have been fixed:

=== FAILURES 

___ test_differences 


differences = [, ]>]>]

@pytest.mark.skipif(not guestfs_working(), reason='guestfs not working on 
the system')
@pytest.mark.skipif(tool_missing('qemu-img'), reason='missing qemu-img')
@pytest.mark.skipif(miss_guestfs, reason='guestfs is missing')
def test_differences(differences):
assert differences[0].source1 == 'test1.ext4.tar'
tarinfo = differences[0].details[0]
tardiff = differences[0].details[1]
encodingdiff = tardiff.details[0]
assert tarinfo.source1 == 'file list'
assert tarinfo.source2 == 'file list'
assert tardiff.source1 == './date.txt'
assert tardiff.source2 == './date.txt'
assert encodingdiff.source1 == 'encoding'
assert encodingdiff.source2 == 'encoding'
expected_diff = open(os.path.join(os.path.dirname(__file__), 
'../data/ext4_expected_diffs'), encoding='utf-8').read()
found_diff = tarinfo.unified_diff + tardiff.unified_diff + 
encodingdiff.unified_diff
>   assert expected_diff == found_diff
E   assert '@@ -1,3 +1,3...cii\n+utf-8\n' == '@@ -1,3 +1,3 ...cii\n+utf-8\n'
E   @@ -1,3 +1,3 @@
E - -drwxr-xr-x   0000 2015-12-02 
16:01:40.00 ./
E - +drwxr-xr-x   0000 2015-12-02 
16:03:11.00 ./
E -  drwx--   0000 2015-12-02 
16:00:55.00 ./lost+found/
E + -drwxr-xr-x   0 root (0) root (0)0 
2015-12-02 16:01:40.00 ./
E + +drwxr-xr-x   0 root (0) root (0)0 
2015-12-02 16:03:11.00 ./
E +  drwx--   0 root (0) root (0)0 
2015-12-02 16:00:55.00 ./lost+found/
E   --rw-rw-rw-   0 1234 1234   28 2015-12-02 
16:01:40.00 ./date.txt
E   +-r--r--r--   0 4321 4321   44 2015-12-02 
16:03:11.00 ./date.txt
E   @@ -1 +1 @@
E   -Wed Dec 2 17:01:40 CET 2015
E   +jeudi 3 décembre 2015, 06:03:11 (UTC+1400)
E   @@ -1 +1 @@
E   -us-ascii
E   +utf-8

tests/comparators/test_fsimage.py:85: AssertionError
=== 1 failed, 197 passed, 35 skipped in 61.54 
seconds ===

> b) How, exactly, you are invoking the tests. This matters because
>diffoscope does some "environment reset" (see ``set_locale``) but
>its not always called and probably isn't during tests. This might
>be deliberate, I haven't looked into it.

I invoke the tests as "PYTHONPATH=. py.test-3 . -vv" locally. Above output
is from that.

Let me know if I should provide additional information.

Zbyszek

Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

I "enabled" the cbfs tests by installing the appropriate dependency 
(coreboot-utils),
and one more test starting failing:

=== FAILURES ===
_ test_listing _

differences = [, 
]

@pytest.mark.skipif(tool_missing('cbfstool'), reason='missing cbfstool')
def test_listing(differences):
expected_diff = open(os.path.join(os.path.dirname(__file__), 
'../data/cbfs_listing_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1,3 +1,3...  31896\n' == '@@ -1,6 +1,3 ...  31896\n'
E - @@ -1,3 +1,3 @@
E ?   ^
E + @@ -1,6 +1,3 @@
E ?   ^
E + -32 kB, bootblocksize 0, romsize 32768, offset 0x0
E + -alignment: 64 bytes, architecture: x86
E + -
EName   Offset Type Size
E   -text   0x0raw  446
E   -(empty)0x200  null 32152
E   +text   0x0raw  671
E   +(empty)0x300  null 31896

tests/comparators/test_cbfs.py:79: AssertionError
 Captured stdout setup -
--- /tmp/pytest-of-zbyszek/pytest-15/test_listing0/coreboot1
+++ /tmp/pytest-of-zbyszek/pytest-15/test_listing0/coreboot2.rom
├── cbfstool {} print
│ @@ -1,3 +1,3 @@
│  Name   Offset Type Size
│ -text   0x0raw  446
│ -(empty)0x200  null 32152
│ +text   0x0raw  671
│ +(empty)0x300  null 31896
├── text
│ @@ -1,6 +1,12 @@
│ +A common form of lorem ipsum reads:
│ +
│  Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod 
tempor
│  incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
│  nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
│  Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore 
eu
│  fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt 
in
│  culpa qui officia deserunt mollit anim id est laborum.
│ +
│ +"Lorem ipsum" text is derived from sections 1.10.32--3 of Cicero's De finibus
│ +bonorum et malorum (On the Ends of Goods and Evils, or alternatively [About]
│ +The Purposes of Good and Evil).
╵
 Captured stderr setup -
Created CBFS (capacity = 32664 bytes)
Created CBFS (capacity = 32664 bytes)
== 1 failed, 202 passed, 29 skipped in 54.96 seconds ===

Unfortunately Fedora is missing a bunch of the tools required to
support all tests. In particular pdftk has been retired because of licensing
reasons 
[https://lists.fedoraproject.org/pipermail/devel/2014-March/196394.html],
so some tests will continue to have to be skipped.

Zbyszek

Bug#820631: [Reproducible-builds] Bug#820631: support for concatenated cpio archives

[Zbigniew Jędrzejewski-Szmek]

On Sun, Apr 10, 2016 at 10:34:59PM +0200, Reiner Herrmann wrote:
> Hi Zbigniew,
> 
> thanks for the report!
> I just had a short look at this issue.
> The problem why diffoscope doesn't see this change is that the initramfs
> contains actually multiple cpio archives, but diffoscope treats
> the files as single cpio archives:
> 
> > $ cpio -t < initrd 
> > .
> > early_cpio
> > kernel
> > kernel/x86
> > kernel/x86/microcode
> > kernel/x86/microcode/GenuineIntel.bin
> 
> And the files in this first/visible archive don't differ.
> 
> Right now it doesn't know that somewhere in the original file an additional
> archive is embedded.
> And so it falls back to a hexdump/binary diff, as the files clearly
> differ, but it didn't find any more readable difference.
> 
> To solve this, diffoscope either needs to learn about initramfs files,
> or (to have a more general solution) use binwalk for extracting data
> embedded into other files (this would require #819018 first).

Thanks for the quick diagnosis.
I think that using binwalk as a general solution sounds better. 
I'll wait then for diffoscope to acquire that capability.
In Fedora we're lucky, binwalk already provides python3 modules (in fact
only python3 modules, without python2 support).

Zbyszek

Bug#820631: (no subject)

[Zbigniew Jędrzejewski-Szmek]

Subject: diffoscope: wrongly reports "No differences found inside, yet data 
differs"
Package: diffoscope
Version: 51
Severity: normal

Dear Maintainer,

when comparing two initramfs images [1,2], diffoscope reports:
[1] http://in.waw.pl/~zbyszek/diffoscope/initramfs-4.5.0-302.fc24.x86_64.img
[2] http://in.waw.pl/~zbyszek/diffoscope/initrd

$ diffoscope initrd initramfs-4.5.0-302.fc24.x86_64.img
--- initrd
+++ initramfs-4.5.0-302.fc24.x86_64.img
│┄ No differences found inside, yet data differs
│ @@ -3,8 +3,8 @@
│  00186980: 9f3b 9832 2c2d 928f 22a1 0518 4c44 9e04  .;.2,-.."...LD..
│  00186990: 6e15 ca38 8218 fbd3 f8ed 1150 021f 3d72  n..8...P..=r
│  001869a0: 8043 a230 7d49 a55f a840 40d7 9242 4da1  .C.0}I._.@@..BM.
│  001869b0: dc08 8470 3e05 bdcc a6c4 980d d084 cf2e  ...p>...
│  001869c0: e951 5086 f574 94c6 74c6 b2c8 5ffa 31d2  .QP..t..t..._.1.
│  001869d0: 941e 42e6 81a7 a945 aae3 e639 ac96 a81e  ..BE...9
│  001869e0: 8335 0705 238e cc20 11a0 219b 4300 d00a  .5..#.. ..!.C...
│ -[ Too much input for diff (SHA1: 3df03b52054657ed2022f06be8d8e0eedac8a4fa) ]
│ +[ Too much input for diff (SHA1: e691a25646c1427ec8471a0e73584399ac69e929) ]

but in fact there's a difference and it can be found manually:
$ cat initramfs-4.5.0-302.fc24.x86_64.img| (mkdir a; cd a; cpio -i; zcat|cpio 
-i)
$ cat initrd | (mkdir b; cd b; cpio -i; zcat|cpio -i)
$ diff -u -r a b
diff -u -r a/lib/dracut/build-parameter.txt b/lib/dracut/build-parameter.txt
--- a/lib/dracut/build-parameter.txt2016-04-10 15:52:12.243191083 -0400
+++ b/lib/dracut/build-parameter.txt2016-04-10 15:52:22.350662307 -0400
@@ -1 +1 @@
--f
+
diff -u -r a/usr/lib/dracut/build-parameter.txt 
b/usr/lib/dracut/build-parameter.txt
--- a/usr/lib/dracut/build-parameter.txt2016-04-10 15:52:12.243191083 
-0400
+++ b/usr/lib/dracut/build-parameter.txt2016-04-10 15:52:22.350662307 
-0400
@@ -1 +1 @@
--f
+

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.3.3-303.fc23.x86_64 (SMP w/12 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages diffoscope depends on:
ii  python3-libarchive-c   2.1-3
ii  python3-magic  1:5.25-2
ii  python3-pkg-resources  20.3.1-1
pn  python3:any

Versions of packages diffoscope recommends:
ii  acl2.2.52-3
ii  binutils-multiarch 2.26-8
ii  bzip2  1.0.6-8
ii  caca-utils 0.99.beta19-2+b1
ii  colord 1.2.12-1
ii  cpio   2.11+dfsg-5
ii  default-jdk [java-sdk] 2:1.8-57
ii  enjarify   20151118-1
ii  fontforge-extras   0.3-4
ii  fp-utils   3.0.0+dfsg-4
ii  fp-utils-3.0.0 [fp-utils]  3.0.0+dfsg-4
ii  genisoimage9:1.1.11-3
ii  gettext0.19.7-2
ii  ghc7.10.3-7
ii  ghostscript9.19~dfsg-1+b1
ii  gnupg  1.4.20-5
ii  mono-utils 4.2.1.102+dfsg2-6
ii  openjdk-8-jdk [java-sdk]   8u77-b03-3+b1
ii  pdftk  2.02-3
ii  poppler-utils  0.38.0-2+b1
ii  python3-debian 0.1.27
ii  python3-guestfs1:1.32.2-4+b1
ii  python3-rpm4.12.0.1+dfsg1-3+b2
ii  python3-tlsh   3.4.4+20151206-1+b1
ii  rpm2cpio   4.12.0.1+dfsg1-3+b2
ii  sng1.1.0-1+b1
ii  sqlite33.12.1-1
ii  squashfs-tools 1:4.3-3
ii  unzip  6.0-20
ii  vim-common 2:7.4.1689-3
ii  xz-utils   5.1.1alpha+20120614-2.1

Versions of packages diffoscope suggests:
ii  libjs-jquery  1.12.3-1

-- no debconf information

Bug#817193: [Reproducible-builds] Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

On Fri, Mar 11, 2016 at 03:03:28PM +, Zbigniew Jędrzejewski-Szmek wrote:
> > On Mittwoch, 9. März 2016, Zbigniew Jędrzejewski-Szmek wrote:
> > > I'll be able link to build logs in koji (the Fedora build system) once
> > > python-libarchive-c passes review and I can build against it.

One more status update, sorry for the noise.

> Example of full test suite failing:
> https://kojipkgs.fedoraproject.org//work/tasks/890/13300890/build.log

Another, different, failure:
https://kojipkgs.fedoraproject.org//work/tasks/2994/13312994/build.log
(this one appears to be non-deterministic)

> (I also run with this test commented out, and had a bunch of different
> failures, not sure what is going on there:
> https://kojipkgs.fedoraproject.org//work/tasks/834/13300834/build.log)
This was caused by locale issues (?). With LC_CTYPE=en_US.utf8
I don't see this any more.

Zbyszek

Bug#817193: [Reproducible-builds] Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

> On Mittwoch, 9. März 2016, Zbigniew Jędrzejewski-Szmek wrote:
> > I'll be able link to build logs in koji (the Fedora build system) once
> > python-libarchive-c passes review and I can build against it.

Example of full test suite failing:
https://kojipkgs.fedoraproject.org//work/tasks/890/13300890/build.log

(I also run with this test commented out, and had a bunch of different
failures, not sure what is going on there:
https://kojipkgs.fedoraproject.org//work/tasks/834/13300834/build.log)

Zbyszek

Bug#817193: [Reproducible-builds] Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 09, 2016 at 03:03:53PM +0100, Holger Levsen wrote:
> control: reopen -1
> 
> Hi,
> 
> On Mittwoch, 9. März 2016, Zbigniew Jędrzejewski-Szmek wrote:
> > The tests still fail with v. 51:
> 
> ok, reopening, thanks for confirming!
>  
> > If I set TZ=UTC, I only get the first failure.
> 
> interesting…

I'll be able link to build logs in koji (the Fedora build system) once
python-libarchive-c passes review and I can build against it.

Zbyszek

Bug#817248: [Reproducible-builds] Bug#817248: diffoscope 51 not uploaded to pypi

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 09, 2016 at 01:07:13PM +, Mattia Rizzolo wrote:
> http://pypi.debian.net/diffoscope should be better
> http://ftp.debian.org/debian/pool/main/d/diffoscope/
Thanks. Both seem to work indeed (apart from pypi missing some tarballs of 
course).

Zbyszek

Bug#817248: diffoscope 51 not uploaded to pypi

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 09, 2016 at 01:09:17PM +0100, Holger Levsen wrote:
> package: diffoscope
> version: 51
> severity: minor
> x-debbugs-cc: "Zbigniew Jędrzejewski-Szmek" <zbys...@in.waw.pl>
> 
> Hi,
> 
> thanks for making us aware of this issue!
> 
> On Mittwoch, 9. März 2016, Zbigniew Jędrzejewski-Szmek wrote:
> > > FYI: I checked why I missed diffscope 49, 50, and 51.
> > > It seems to be a problem with pypi:
> > > https://pypi.python.org/pypi?%3Aaction=search=diffoscope=sear
> > > ch only lists diffoscope 48 as the lastest version. So either somebody
> > > forgot to upload the latest versions there, or it's not displaying them
> > > for some reason.
> > V. 49 is also on pypi, just doesn't show up in the search.
> 
> is there a commandline client to search on pypi? I'd like to setup an 
> automatic test which will notify us, when a new diffoscope upload has been 
> made to Debian, but not to pypi.

Not that I know of, but it should be easy enough to get
https://pypi.python.org/pypi/diffoscope/
and scape for the version (it's in the title of the page).

> Also: who can how upload to pypi?
No idea.

> > Is there a canonical place do download a tarball of the project apart from
> > pypi?
Too bad. It would be nice to enable tarball downloads on anonscm.d.o.

Zbyszek

Bug#817193: failing tests: test_gzip.py::test_metadata, test_ipk.py::test_metadata, test_java.py::test_diff

[Zbigniew Jędrzejewski-Szmek]

Package: diffoscope
Version: 48

Hi,

I'm trying to package diffoscope for Fedora and the tests listed
in $subject are failing. Seems to be some timezone issue:

= FAILURES 
==
_ test_differences 
__

differences = [, ]>]>]

@pytest.mark.skipif(not guestfs_working(), reason='guestfs not working on 
the system')
@pytest.mark.skipif(tool_missing('qemu-img'), reason='missing qemu-img')
@pytest.mark.skipif(miss_guestfs, reason='guestfs is missing')
def test_differences(differences):
assert differences[0].source1 == 'test1.ext4.tar'
tarinfo = differences[0].details[0]
tardiff = differences[0].details[1]
encodingdiff = tardiff.details[0]
assert tarinfo.source1 == 'file list'
assert tarinfo.source2 == 'file list'
assert tardiff.source1 == './date.txt'
assert tardiff.source2 == './date.txt'
assert encodingdiff.source1 == 'encoding'
assert encodingdiff.source2 == 'encoding'
expected_diff = open(os.path.join(os.path.dirname(__file__), 
'../data/ext4_expected_diffs'), encoding='utf-8').read()
found_diff = tarinfo.unified_diff + tardiff.unified_diff + 
encodingdiff.unified_diff
>   assert expected_diff == found_diff
E   assert '@@ -1,3 +1,3...cii\n+utf-8\n' == '@@ -1,3 +1,3 ...cii\n+utf-8\n'
E   @@ -1,3 +1,3 @@
E - -drwxr-xr-x   0000 2015-12-02 
16:01:40.00 ./
E - +drwxr-xr-x   0000 2015-12-02 
16:03:11.00 ./
E -  drwx--   0000 2015-12-02 
16:00:55.00 ./lost+found/
E + -drwxr-xr-x   0 root (0) root (0)0 
2015-12-02 16:01:40.00 ./
E + +drwxr-xr-x   0 root (0) root (0)0 
2015-12-02 16:03:11.00 ./
E +  drwx--   0 root (0) root (0)0 
2015-12-02 16:00:55.00 ./lost+found/
E   --rw-rw-rw-   0 1234 1234   28 2015-12-02 
16:01:40.00 ./date.txt
E   +-r--r--r--   0 4321 4321   44 2015-12-02 
16:03:11.00 ./date.txt
E   @@ -1 +1 @@
E   -Wed Dec 2 17:01:40 CET 2015
E   +jeudi 3 décembre 2015, 06:03:11 (UTC+1400)
E   @@ -1 +1 @@
E   -us-ascii
E   +utf-8

tests/comparators/test_fsimage.py:85: AssertionError
___ test_metadata 
___

differences = [, ]

def test_metadata(differences):
assert differences[0].source1 == 'metadata'
assert differences[0].source2 == 'metadata'
expected_diff = open(os.path.join(os.path.dirname(__file__), 
'../data/gzip_metadata_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1 +1 @@\..., from Unix\n' == '@@ -1 +1 @@\n..., from Unix\n'
E   @@ -1 +1 @@
E - -gzip compressed data, last modified: Tue Jun 23 06:12:28 2015, max 
compression, from Unix
E ?   -
E + -gzip compressed data, last modified: Tue Jun 23 10:12:28 2015, max 
compression, from Unix
E ?  +
E - +gzip compressed data, last modified: Tue Jun 23 06:12:28 2015, 
from Unix
E ?   -
E + +gzip compressed data, last modified: Tue Jun 23 10:12:28 2015, 
from Unix
E ?  +

tests/comparators/test_gzip.py:54: AssertionError
___ test_metadata 
___

differences = [, , ]>]>]>]

def test_metadata(differences):
assert differences[0].source1 == 'metadata'
expected_diff = open(os.path.join(os.path.dirname(__file__), 
'../data/ipk_metadata_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1 +1 @@\..., from Unix\n' == '@@ -1 +1 @@\n..., from Unix\n'
E   @@ -1 +1 @@
E - -gzip compressed data, last modified: Mon May 18 19:26:52 2015, 
from Unix
E ?  ^^
E + -gzip compressed data, last modified: Mon May 18 23:26:52 2015, 
from Unix
E ?  ^^
E - +gzip compressed data, last modified: Mon Jun  8 13:31:21 2015, 
from Unix
E ?   ^
E + +gzip compressed data, last modified: Mon Jun  8 17:31:21 2015, 
from Unix
E ?   ^

tests/comparators/test_ipk.py:52: AssertionError
_ test_diff 

Bug#807458: :807: warning [p 10, 2.8i, div `3tbd1, 1', 1.2i]: can't break line

[Zbigniew Jędrzejewski-Szmek]

On Fri, Jan 29, 2016 at 04:43:56PM +0100, Michael Biebl wrote:
> Control: severity -1 minor
> Control: tags -1 = confirmed
> 
> On Tue, 05 Jan 2016 05:11:54 +0800 =?utf-8?B?56mN5Li55bC8?= Dan Jacobson
>  wrote:
> > # su - nobody
> > No directory, logging in with HOME=/
> > nobody@jidanni6:/$  < /dev/null > /dev/null COLUMNS=80 man systemctl
> > :807: warning [p 10, 2.8i, div `3tbd1,1', 0.8i]: can't 
> > break line
> > :807: warning [p 10, 2.8i, div `3tbd1,1', 1.2i]: can't 
> > break line
> > :817: warning [p 10, 2.8i, div `3tbd3,1', 1.0i]: can't 
> > break line
> > :827: warning [p 10, 2.8i, div `3tbd5,1', 1.0i]: can't 
> > break line
> 
> I can confirm the issue with the instructions provided by Dan.
> lintian also sees those errors [1].
> 
> That said, this looks like a problem in the xml/xslt toolchain to me.
> I'm saying this as someone who has no idea about docbook.
> 
> I'm CCing Zbigniew, maybe he has some ideas. Is this reproducible on
> fedora as well? At least my Fedora VM suggests so:
> 
> # MANWIDTH=80 man systemd > /dev/null
Yes, I can reproduce this.
It seems that the example output in list-sockets is longer than 80 columns.
But is this an actual issue? less displays this just fine
(wrapping the long lines by default), and can be easily
switched to "truncate" them with -+s+ENTER.

Zbyszek

Bug#807458: :807: warning [p 10, 2.8i, div `3tbd1, 1', 1.2i]: can't break line

[Zbigniew Jędrzejewski-Szmek]

On Sat, Jan 30, 2016 at 08:36:04AM +0800, 積丹尼 Dan Jacobson wrote:
> It would be best to just reengineer your table not to generate the errors.

It's the output of an actual command, and the values in the
columns simply need that much space.

Zbyszek

Bug#806291: systemd: systemctl status ignores -n argument

[Zbigniew Jędrzejewski-Szmek]

On Sun, Dec 20, 2015 at 02:52:49PM +0100, Eduard Bloch wrote:
> Hallo,
> * Zbigniew Jędrzejewski-Szmek [Mon, Nov 30 2015, 01:25:49AM]:
> 
> > > [Service]
> > > Type=notify
> > > Restart=on-failure
> > > ExecStart=/bin/sh -c "echo OMG THEY KILLED KENNY >&2; exit 123"
> > > 
> > > and check the result:
> > > 
> > >  foo.service - SomeThing
> > >Loaded: loaded (/lib/systemd/system/foo.service; disabled; vendor 
> > > preset: enabled)
> > >Active: failed (Result: start-limit) since Sa 2015-11-28 11:28:39 CET; 
> > > 13s ago
> > >   Process: 6982 ExecStart=/bin/sh -c echo OMG THEY KILLED KENNY >&2; exit 
> > > 123 (code=exited, status=123)
> > >  Main PID: 6982 (code=exited, status=123)
> > > 
> > > Nov 28 11:28:39 idefix systemd[1]: Failed to start SomeThing.
> > > Nov 28 11:28:39 idefix systemd[1]: foo.service: Unit entered failed state.
> > > Nov 28 11:28:39 idefix systemd[1]: foo.service: Failed with result 
> > > 'exit-code'.
> > > Nov 28 11:28:39 idefix sh[6982]: OMG THEY KILLED KENNY
> > > Nov 28 11:28:39 idefix systemd[1]: foo.service: Service hold-off time 
> > > over, scheduling restart.
> > > Nov 28 11:28:39 idefix systemd[1]: Stopped SomeThing.
> > > Nov 28 11:28:39 idefix systemd[1]: foo.service: Start request repeated 
> > > too quickly.
> > > Nov 28 11:28:39 idefix systemd[1]: Failed to start SomeThing.
> > > Nov 28 11:28:39 idefix systemd[1]: foo.service: Unit entered failed state.
> > > Nov 28 11:28:39 idefix systemd[1]: foo.service: Failed with result 
> > > 'start-limit'.
> > > 
> > > 
> > > Looks good, huh? So what is the major difference to the script in the 
> > > original report?
> > > Is it maybe this?
> > > Type=notify
> > 
> > There's a race for short-lived programs - if the process has exited by
> > the time journald looks the sender up in /proc, it will not be able to
> > assign the message the right systemd unit. This means that the message
> > will be in logs, but it will not be shown by
> > 'systemctl status unit'/'journalctl -u unit'. This race is currently
> > unfixable, until kernel provides additional functionality to attach more
> > metadata to messages.
> 
> OMG. I was assuming that sd_notify is a mechanism designed to avoid such
> kind of BS. Is there a public kernel bug report anywhere to check for
> more details?

https://bugzilla.redhat.com/show_bug.cgi?id=963620, links to lkml discussions
start at comment 33.

kdbus would also solve this, you can easily search for that.

> Can I flush the log processing somehow from inside the unit process
> itself?
Not that I'm aware of.

> > > But anyhow, it's a secondary issue and probably deserves a second bug 
> > > report.
> > > The thing that botters me, see subject, is the broken -n option, adding 
> > > -n20 or
> > > -n2000 to systemctl-status call does not change anything. It keeps 
> > > displaying 10
> > > lines.
> > That's really strange.
> > What is the exact command line that you are using?
> > Does 'systemctl -n0 status ...' show any log lines?
> 
> Basically any command. Let's pick the next example with useful logs from
> "journalctl -b0" output...
> 
> $ systemctl status virtualbox
> ● virtualbox.service - LSB: VirtualBox Linux kernel module
>Loaded: loaded (/etc/init.d/virtualbox; bad; vendor preset: enabled)
>Active: failed (Result: exit-code) since Sa 2015-12-19 21:11:20 CET; 17h 
> ago
>  Docs: man:systemd-sysv-generator(8)
>   Process: 1312 ExecStart=/etc/init.d/virtualbox start (code=exited, 
> status=1/FAILURE)
> $ systemctl status -n200 virtualbox
> ● virtualbox.service - LSB: VirtualBox Linux kernel module
>Loaded: loaded (/etc/init.d/virtualbox; bad; vendor preset: enabled)
>Active: failed (Result: exit-code) since Sa 2015-12-19 21:11:20 CET; 17h 
> ago
>  Docs: man:systemd-sysv-generator(8)
>   Process: 1312 ExecStart=/etc/init.d/virtualbox start (code=exited, 
> status=1/FAILURE)
> $ journalctl -b0 -u virtualbox
> Hint: You are currently not seeing messages from other users and the system.
>   Users in the 'systemd-journal' group can see all messages. Pass -q to
>   turn off this notice.
> No journal files were opened due to insufficient permissions.
> 
> (same thing as root)
> $ journalctl -b0 -u virtualbox
> -- Logs begin at Mo 2015-07-27 19:58:37 CEST, end at So 2015-12-20 14:46:29 
> CET. --
> Dez 19 21:11:19 zombie systemd[1]: Starting LSB: VirtualBox Linux k

Bug#806291: systemd: systemctl status ignores -n argument

[Zbigniew Jędrzejewski-Szmek]

On Sat, Nov 28, 2015 at 11:37:55AM +0100, Eduard Bloch wrote:
> Hallo,
> * Michael Biebl [Thu, Nov 26 2015, 04:32:43PM]:
> > Am 26.11.2015 um 15:59 schrieb Michael Biebl:
> > > Works here. But I actually need a service which produces more then 10
> > > lines of output when started.
> > 
> > To illustrate what I did:
> > 
> > # cat /etc/systemd/system/output.service
> > [Unit]
> > Description=foo
> > [Service]
> > Type=oneshot
> > ExecStart=/bin/sh -c "for i in `seq 1 200`; do echo $i; done"
> > 
> > 
> > # systemctl start output.service
> > # systemct status -n 100
> > log message 102-200
> > Nov 26 16:28:27 pluto systemd[1]: Started foo.
> 
> You are demonstrating the good case, I am after a bad case. Let's try
> this:
> 
> [Unit]
> Description=SomeThing
> After=network.target
> [Install]
> WantedBy=multi-user.target
> 
> [Service]
> Type=notify
> Restart=on-failure
> ExecStart=/bin/sh -c "echo OMG THEY KILLED KENNY >&2; exit 123"
> 
> and check the result:
> 
>  foo.service - SomeThing
>Loaded: loaded (/lib/systemd/system/foo.service; disabled; vendor preset: 
> enabled)
>Active: failed (Result: start-limit) since Sa 2015-11-28 11:28:39 CET; 13s 
> ago
>   Process: 6982 ExecStart=/bin/sh -c echo OMG THEY KILLED KENNY >&2; exit 123 
> (code=exited, status=123)
>  Main PID: 6982 (code=exited, status=123)
> 
> Nov 28 11:28:39 idefix systemd[1]: Failed to start SomeThing.
> Nov 28 11:28:39 idefix systemd[1]: foo.service: Unit entered failed state.
> Nov 28 11:28:39 idefix systemd[1]: foo.service: Failed with result 
> 'exit-code'.
> Nov 28 11:28:39 idefix sh[6982]: OMG THEY KILLED KENNY
> Nov 28 11:28:39 idefix systemd[1]: foo.service: Service hold-off time over, 
> scheduling restart.
> Nov 28 11:28:39 idefix systemd[1]: Stopped SomeThing.
> Nov 28 11:28:39 idefix systemd[1]: foo.service: Start request repeated too 
> quickly.
> Nov 28 11:28:39 idefix systemd[1]: Failed to start SomeThing.
> Nov 28 11:28:39 idefix systemd[1]: foo.service: Unit entered failed state.
> Nov 28 11:28:39 idefix systemd[1]: foo.service: Failed with result 
> 'start-limit'.
> 
> 
> Looks good, huh? So what is the major difference to the script in the 
> original report?
> Is it maybe this?
> Type=notify

There's a race for short-lived programs - if the process has exited by
the time journald looks the sender up in /proc, it will not be able to
assign the message the right systemd unit. This means that the message
will be in logs, but it will not be shown by
'systemctl status unit'/'journalctl -u unit'. This race is currently
unfixable, until kernel provides additional functionality to attach more
metadata to messages.

So you should check with journalctl -u that the messages you think
should be shown are really attached to that unit.

> I can imagine that if the service dies before any dbus event was sent then the
> messages are printed early, followed by the spam I mentioned? And that makes
> them scroll out of "systemctl status" buffer?
> 
> But anyhow, it's a secondary issue and probably deserves a second bug report.
> The thing that botters me, see subject, is the broken -n option, adding -n20 
> or
> -n2000 to systemctl-status call does not change anything. It keeps displaying 
> 10
> lines.
That's really strange.
What is the exact command line that you are using?
Does 'systemctl -n0 status ...' show any log lines?
Do you have POSIXLY_CORRECT set?

Zbyszek

Bug#782426: unblock: systemd/215-15

[Zbigniew Jędrzejewski-Szmek]

On Mon, Apr 13, 2015 at 10:56:45PM +0200, Michael Biebl wrote:
 Hi Julien,
 
 Am 13.04.2015 um 22:18 schrieb Julien Cristau:
  On Mon, Apr 13, 2015 at 22:03:51 +0200, Michael Biebl wrote:
  
  ++static int manager_dispatch_ask_password_fd(sd_event_source *source,
  ++int fd, uint32_t revents, 
  void *userdata) {
  ++Manager *m = userdata;
  ++
  ++assert(m);
  ++
  ++flush_fd(fd);
  ++
  ++m-have_ask_password = have_ask_password();
  ++if (m-have_ask_password  0)
  ++/* Log error but continue. Negative have_ask_password
  ++ * is treated as unknown status. */
  ++log_error(Failed to list /run/systemd/ask-password: 
  %s, strerror(m-have_ask_password));
  
  shouldn't that be strerror(-m-have_ask_password)?
 
 I think you are right. Thanks for checking so carefully.
 
 I notice, that this line was changed later on in commit c33b3297 to
  log_error_errno(m-have_ask_password, Failed to list
 /run/systemd/ask-password: %m);
 
 That helper function does use inverted error numbers. So v219 is indeed
 not affected by this.
 
 In any case, CCed Zbigniew, the author of the patch, maybe he can comment.
Yes, looks like a bug.

 Julien, do you want me to make a followup upload fixing that (pending
 confirmation from Zbigniew)?
Can you cc me on the patch so I can push it to -stable or push it yourself?
217-stable at least is affected.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#768314: [pkg-cryptsetup-devel] Bug#768314: cryptsetup: Passphrase prompt rolls by without stopping (fwd)

[Zbigniew Jędrzejewski-Szmek]

On Tue, Jan 06, 2015 at 06:56:11PM +0100, Tomas Pospisek wrote:
 Hello Zbigniew,
 
 I was told on IRC in #debian-systemd:
 
   mbiebl_ tpo, I remember Zbigniew had a patch
 
 without wanting to stress anybody: could you maybe tell me what the
 status of that patch is? Are you considering it ready for inclusion
 in Debian's systemd? Is it possible that it would be ready and
 included prior to jessie's release?
systemd git should now avoid output when waiting for the password.
IIRC, my initial patch was followed up by a few cleanups, so it might
not be very backportable, but latest systemd in experimental should have
all the changes. I'd suggest that you test if that version works for you.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#772182: systemd: Systemd tries to mount swap partition twice at the same time

[Zbigniew Jędrzejewski-Szmek]

On Fri, Dec 05, 2014 at 02:41:59PM -0800, Benjamin Xiao wrote:
 Systemd tries to mount my swap partition twice at the same time. One of the
 mount attempts is triggered by systemd-fstab-generator and the other attempt
 is triggered by systemd-gpt-auto-generator. There is a race condition and
 one of them will fail with Device or resource busy in journalctl.
Yeah, it's a know bug. Fix should be upstream soonish.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#771397: v217: systemd-delta says diff failed with error code 1.

[Zbigniew Jędrzejewski-Szmek]

On Sat, Nov 29, 2014 at 08:35:16AM +0100, Alexandre Detiste wrote:
 Package: systemd
 Version: 217-1
 Severity: minor
 
 Dear Maintainer,
 
 Hi,
 
 I'm testing v217 in experimental.
 
 systemd-delta inserts diff failed with error code 1 in it's output
 after each diff.
Fixed upstream in v217-476-g820d3ac.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#771172: systemd-sysv-generator does not generate

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 27, 2014 at 09:06:11PM +0100, Salvo Tomaselli wrote:
  Running systemctl daemon-reload is sufficient to run the generator.
  You don't need to run it by hand and copying the files around.
 I wasn't even aware there was such a thing. rantI thought systemd was 
 supposed to be a drop in replacement. Which clearly is not./rant
 So I am rather clueless on systemd's usage.
drop in replacement is an impossible task. The logic on which systemd
is built is too different to sysvinit. The goal is provide a way to support
well-behaved existing init scripts, possibly with small modifications.

I added a man page upstream for systemd-sysv-generator
(http://cgit.freedesktop.org/systemd/systemd/commit/?id=f509443af5).
It's not very exhaustive, but provides the basic pointers. People who
care about sysv compat will hopefully extend it at some point.

  That's also true for native services fwiw.
 Oh this explains why I had to reboot after creating a .service file!
Actually you don't have to. Even running systemctl daemon-reload is usually
not necessary, since 'systemctl start xxx.service' will load the unit if
didn't previously know about it.

HTH,
Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#770504: systemd: log all members of cyclic dependencies (loops)

[Zbigniew Jędrzejewski-Szmek]

On Sat, Nov 22, 2014 at 01:20:13AM +0200, Uoti Urpala wrote:
 On Fri, 21 Nov 2014 20:20:45 + Simon McVittie s...@debian.org wrote:
  dependency: systemd logs that there was a loop, and which arbitrary point
  it tried to use to break it. However, it does not log (an example of) the
  whole path around the loop before it started trying to break it.
 
 IIRC it does log that. If you don't see it, make sure that is not caused
 by visible log level issues or such.

C.f. http://cgit.freedesktop.org/systemd/systemd/commit/?id=14fe721b5f6d

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#767267: systemctl --failed not documented

[Zbigniew Jędrzejewski-Szmek]

http://cgit.freedesktop.org/systemd/systemd/commit/?id=599b6322f1


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#768178: systemd: sysvinit wrapper breaks newly-installed services

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 06, 2014 at 01:25:32PM -0800, Josh Triplett wrote:
 On Wed, 5 Nov 2014 20:20:27 -0500 Zbigniew 
 =?utf-8?Q?J=C4=99drzejewski-Szmek?= zjedr...@gmu.edu wrote:
  On Wed, Nov 05, 2014 at 07:31:32PM +0100, Michael Biebl wrote:
   The default for sysv init scripts is RemainAfterExit=true [0], so even
   if there are no running processes, the service is marked as active.
   This is because systemd doesn't know, if the sysv init script is
   supposed to start a long running process or a just some one shot commands.
  Hm, would there be downsides to defaulting to RemainAfterExit=false
  for sysvinit scripts? Apart from the obvious one of changig established
  behaviuor and potentially breaking compatiblity with older systemds. 
  This would indeed seem to match sysvinit behaviour more closely, and
  would also make sysvinit scripts more similar to normal units, which
  default to RemainAfterExit=no.
 
 That would have the effect of marking scripts that don't run
 long-running processes as failed
No, I don't think so. If only they exit with status 0, they will not be
marked as failed.

 , and thus running start on them
 again would re-run the script.  That seems like an improvement, as long
 as systemd doesn't let the failure of scripts that don't launch a
 daemon prevent other scripts with LSB dependencies on the failing
 script from running.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#768178: systemd: sysvinit wrapper breaks newly-installed services

[Zbigniew Jędrzejewski-Szmek]

On Wed, Nov 05, 2014 at 08:12:39PM +, Ximin Luo wrote:
 All I care is that service x start works. It does not. This is
 correctly called systemd breaks existing software - it is breaking
 the sysvinit behaviour.

Let's look what LSB says:

http://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic/iniscrptact.html

 For all other
[meaning everything apart from 'status']
 init-script actions, the init script shall return an exit status of
 zero if the action was successful. Otherwise, the exit status shall
 be non-zero
[...]
 6 program is not configured

Esentially, your scripts tells systemd action was sucessful, and
systemd has no reason (or way) to doubt that.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#768178: systemd: sysvinit wrapper breaks newly-installed services

[Zbigniew Jędrzejewski-Szmek]

On Wed, Nov 05, 2014 at 07:31:32PM +0100, Michael Biebl wrote:
 The default for sysv init scripts is RemainAfterExit=true [0], so even
 if there are no running processes, the service is marked as active.
 This is because systemd doesn't know, if the sysv init script is
 supposed to start a long running process or a just some one shot commands.
Hm, would there be downsides to defaulting to RemainAfterExit=false
for sysvinit scripts? Apart from the obvious one of changig established
behaviuor and potentially breaking compatiblity with older systemds. 
This would indeed seem to match sysvinit behaviour more closely, and
would also make sysvinit scripts more similar to normal units, which
default to RemainAfterExit=no.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#767893: systemd cannot mount zfs filesystems from fstab

[Zbigniew Jędrzejewski-Szmek]

On Mon, Nov 03, 2014 at 05:51:57AM -0500, John Holland wrote:
 I created luks volumes, installed zfsonlinux.org packages, created a zpool 
 out of the luks volumes. When ZFS is managing the mounting of the fs's it 
 works. If I put a zfs filesystem in /etc/fstab the prompts to enter passwords 
 for the luks volumes during boot are mixed in with output.

So actually its not the ZFS volumes, but simply the luks unlocking that is the 
problem?
You say that the prompts are mixed with output, but do things work if you type 
in
the password blind?

You can either wait until systemd-217 (which fixes the overlapping output) or 
install
plymouth (which provides a graphical prompt which is not interrupted by text 
output).


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#767893: systemd cannot mount zfs filesystems from fstab

[Zbigniew Jędrzejewski-Szmek]

On Mon, Nov 03, 2014 at 03:36:49PM +0100, Michael Biebl wrote:
 http://changelog.complete.org/archives/9241-update-on-the-systemd-issue
 might be relevant for you.
Yeah, this might be relevant. When I read that ZFS has init scripts
with carefully-planned dependencies, I shudder. They clearly have a bug
with refusing to mount over non-empty directories, this seems like something
likely to bite people all the time.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#767350: journald: does not respect carriage return

[Zbigniew Jędrzejewski-Szmek]

On Fri, Oct 31, 2014 at 01:26:32PM -0700, Josh Triplett wrote:
 Avoid emitting incremental progress messages when not outputting to a
 terminal.
Exactly.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#767350: journald: does not respect carriage return

[Zbigniew Jędrzejewski-Szmek]

On Thu, Oct 30, 2014 at 06:49:32PM +0100, Andreas Cadhalpun wrote:
 Control: retitle -1 journald: does not respect carriage return
 Control: reassign -1 systemd 215-5+b1
 
 Hi Santiago,
 
 On 30.10.2014 13:12, Santiago Vila wrote:
 freshclam[7495]: Downloading daily-19559.cdiff [ 45%]#015Downloading
 daily-19559.cdiff [ 61%]#015Downloading daily-19559.cdiff [ 
 77%]#015Downloading daily-19559.cdiff [
 93%]#015Downloading daily-19559.cdiff [100%]#015Downloading 
 daily-19559.cdiff [100%]
 
 A percentage or a progress bar in a log file does not make much sense.
 This should be better replaced by just two messages.
 
 Download start.
 Download end.
 
 This only shows up in /var/log/syslog.
 
 freshclam's own log file, /var/log/clamav/freshclam.log correctly
 only contains the last part, i.e. 'Downloading daily-19559.cdiff
 [100%]', just as it would appear on the terminal.
 
 The problem is that the systemd journal does not respect the
 carriage return '\r' by deleting the current line content, but
 rather continues to append the message.
 In 'journalctl -u clamav-freshclam' this output is suppressed as
 'freshclam[2421]: [1.6K blob data]', but it is still forwarded to
 syslog and there the carriage return is replaced with '#015'.
 
 The fix for this would be for journald to correctly interpret the
 carriage return. Therefore I'm reassigning this bug to systemd.
Nope, there's no place for '\r' in logs. It only makes sense on
a console, where you show something and then erase it some time
later when the situation changes. Log file viewers don't show
logs in real time, so any usage of erase characters should be
replaced by simply logging the final version in the first place.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#766598: systemd: help for journalctl --until option is misleading

[Zbigniew Jędrzejewski-Szmek]

Thanks, patch applied upstream as 
http://cgit.freedesktop.org/systemd/systemd/commit/?id=7558251eef.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#766050: systemd: Please look at a read-only implementation of debug-shell.service

[Zbigniew Jędrzejewski-Szmek]

On Mon, Oct 20, 2014 at 11:01:08PM +0200, Richard Hartmann wrote:
 On Mon, Oct 20, 2014 at 9:01 PM, Martin Pitt mp...@debian.org wrote:
  Perhaps a better thing would be to always log systemd events/changes
  to tty9? There is no scrollback buffer while its logging in the
  background (i. e. you are looking at a different tty), but usually the
  last 20-something lines should give you an idea what it's hanging on.
There's ForwardToConsole and TTYPath. This is enough to have messages forwarded
to a tty [1]. Currently enabling this requires editing 
/etc/systemd/journald.conf,
but when the patches currently being discussed [2] on the mailing list are 
merged,
journald.conf should allow dropin snippets, so this should become even easier
to enable.

[1] 
http://www.freedesktop.org/software/systemd/man/journald.conf.html#ForwardToSyslog=
[2] http://lists.freedesktop.org/archives/systemd-devel/2014-October/024022.html

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#766039: please document debug-shell.service in README.Debian

[Zbigniew Jędrzejewski-Szmek]

On Mon, Oct 20, 2014 at 01:02:25PM +0200, Stefano Zacchiroli wrote:
 Package: systemd
 Version: 215-5+b1
 Severity: wishlist
 
 Please document in README.Debian information about how to enable the
 debug-shell service for debugging bootup/shutdown problems. Something like the
 following (credit: Tollef For Heen) would do:
 
  For bootup problems:
  systemctl enable debug-shell.service
  
  For shutdown problems
  systemctl start debug-shell.service
  
  This enables a debug shell on tty9.
  
  Then look at the output of systemctl list-jobs to figure out which bit is
  hanging on shutdown/startup and start digging from there.
systemd.debug-shell on the command-line is often a better option.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759607: netfilter-persistent: contains invalid dependency in unit file

[Zbigniew Jędrzejewski-Szmek]

Package: netfilter-persistent
Version: 1.0.1
Severity: normal

systemd[1]: [/lib/systemd/system/netfilter-persistent.service:5] Failed to add 
dependency on systemd-modules-load, ignoring: Invalid argument
systemd[1]: [/lib/systemd/system/netfilter-persistent.service:6] Failed to add 
dependency on systemd-modules-load, ignoring: Invalid argument

-- System Information:
Debian Release: jessie/sid
Architecture: amd64


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759607: netfilter-persistent: contains invalid dependency in unit file

[Zbigniew Jędrzejewski-Szmek]

On Fri, Aug 29, 2014 at 12:39:37AM +0200, Zbigniew Jędrzejewski-Szmek wrote:
 systemd[1]: [/lib/systemd/system/netfilter-persistent.service:5] Failed to 
 add dependency on systemd-modules-load, ignoring: Invalid argument
 systemd[1]: [/lib/systemd/system/netfilter-persistent.service:6] Failed to 
 add dependency on systemd-modules-load, ignoring: Invalid argument

Forgot to say:
Fix is to do:
-Requires=systemd-modules-load
-After=systemd-modules-load
+Requires=systemd-modules-load.service
+After=systemd-modules-load.service

Automatic appending of .service only works on the commandline.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756903: systemd: Boot hangs if filesystems unavailable

[Zbigniew Jędrzejewski-Szmek]

On Tue, Aug 26, 2014 at 12:44:07AM -0400, Stefan Monnier wrote:
  Exactly.  I hope the reasoning behind current defaults has been explained
  adequately.
 
 Not sure what you mean by adequately.  I understand your argument, but
 I disagree with it.  Do you understand my argument?
Yes, I understand your argument. We just disagree on the right behaviour.
The behaviour that was the subject of the original bug report is by
design. I don't think there's much to be gained by discussing this
futher here.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756903: systemd: Boot hangs if filesystems unavailable

[Zbigniew Jędrzejewski-Szmek]

On Sat, Aug 23, 2014 at 09:33:56AM -0400, Stefan Monnier wrote:
  Yes, you can configure such behaviour.
 
 I already have plenty of ways to configure the behavior I need.
 This discussion is about the default behavior.
Exactly. I hope the reasoning behind current defaults has been explained
adequately.

That said, it would be great to improve reporting if something like this
happens. Hopefully with #755581 fixed, emergency mode will be entered
successfully.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759020: python-llvm: version string contains 'tag: '

[Zbigniew Jędrzejewski-Szmek]

Package: python-llvm
Version: 0.12.6-2
Severity: normal
Tags: sid

It seems that __version__ string contains 'tag: ' prefix by mistake.
At least numba is tripped up by this, since it expects the normal 
major.minor.micro
sequence.

 import llvm
 llvm.__version__
'tag: 0.12.6'

$ dpkg -l python-llvm
ii  python-llvm  0.12.6-2  amd64 Python 
bindings for LLVM

-- System Information:
Debian Release: 6.0.9
  APT prefers oldstable
  APT policy: (990, 'oldstable'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.26-2-xen-686 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759021: python-llvm: please add support for python 3

[Zbigniew Jędrzejewski-Szmek]

Package: python-llvm
Version: 0.12.6-2
Severity: normal
Tags: sid

Upstream supports Python 3 (3.3 and 3.4 are listed in setup.py).
It would be great to have python3-llvm subpackage.

-- System Information:
Debian Release: 6.0.9
  APT prefers oldstable
  APT policy: (990, 'oldstable'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.26-2-xen-686 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756903: systemd: Boot hangs if filesystems unavailable

[Zbigniew Jędrzejewski-Szmek]

On Fri, Aug 22, 2014 at 10:30:47AM -0400, Stefan Monnier wrote:
  - If a mount fails, keep on booting.  And then do your best to try and
  bring this problem to the attention of someone (mentioning the
  nofail option in that same message).  Only stop the boot if the
  partition is explicitly marked as critical or stoponfail.
  Well, 'fail', which is the default, means just that.
 Not sure what that means here.  Does your that mean that which you
 just described or does it mean fail?
Treat all mounts as critical.

  Systemd tries to do the corrent and safe thing by default.
 
 I'd hope so, but here's my case:
 - a machine somewhat far away with an old and unimportant fstab entry
   that refers to a drive that's rarely connected.
 - after upgrading to systemd, the fstab entry caused systemd to stop the
   boot (presumably asking for the operator to do something on console).
 - with the boot stopped, I (the operator who is not on console, since
   it's a remote machine), I can't fix the fstab entry.
 
 In which way is it safe and correct to interrupt the boot in this case?
In the way that missing some mounts may indicate a serious problem and
could lead to incorrect behaviour or data loss.

 I can understand that making the mount wait (rather than just fail
 right away) might be made necessary by the fact that systemd changes the
 order in which operations are performed.  I.e. I understand why the
 change nb 1 might be needed.
 
 But that doesn't explain why change number 2 was needed.
 Apparently you think Michael's response explains it, but I failed to
 see how.
Michael's response explains why it is not possible for systemd to
distinguish which filesystems are critical, and which are
not. Sysvinit documentation is not particularly verbose about what happens
on error. mount(8) describes 'nofail', which implies that the opposite it
the default. I agree that stopping the boot is a change in semantics
to some degree, but this kind of change parallels changes to other
parts of the boot done by systemd, where status of things (services,
swap points, configuration settings) is checked and the boot is
stopped when something important fails. You can configure things
otherwise, but the default is to strictly obey dependencies.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#758902: systemd: Please make ^C interrupt systemd-fsck

[Zbigniew Jędrzejewski-Szmek]

On Fri, Aug 22, 2014 at 07:09:56PM +0200, Alberto Fuentes wrote:
 Package: systemd
 Version: 208-6
 Severity: important
 
 Please consider adding ctrl + C or whatever other means to cancel fsck during
 startup...
http://cgit.freedesktop.org/systemd/systemd/tree/TODO?id=HEAD#n562

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756903: systemd: Boot hangs if filesystems unavailable

[Zbigniew Jędrzejewski-Szmek]

On Fri, Aug 22, 2014 at 01:51:56PM -0400, Stefan Monnier wrote:
  In which way is it safe and correct to interrupt the boot in this case?
  In the way that missing some mounts may indicate a serious problem and
  could lead to incorrect behaviour or data loss.
 
 Haven't heard many complaints about that over the years, so it shouldn't
 be a super-top-priority goal, I think.
It is one of the mail goals of systemd to use the dependencies. In the
end this is what allows us to achieve reliability.

 In any case, that's not incompatible with the desire to boot enough for
 remote maintenance to be possible.
 
  stopped when something important fails. You can configure things
  otherwise, but the default is to strictly obey dependencies.
 
 To get the best of both worlds, I suggest that if a problem happens
 during boot, systemd doesn't just interrupt the boot, but instead tries
 to keep booting up to a fallback/safe state, so that maintenance can be
 done conveniently over the network, instead of having to be done on
 console which is all too often completely unworkable.
Yes, you can configure such behaviour. You can add OnFailure= and
OnFailureJobMode= to default.target (e.g. in
/etc/systemd/system/default.target.d/failure.conf') to launch some
target you define, and add e.g. sshd.service and other things to this
target. It is hard to do something like this in a general way though.

You can also add nofail where necessary.

In systemd git, there's a more general setting StartTimeoutAction= [1]
which makes it possible to configure an action that will fire also
when boot hangs wait for password input or similar.

[1] http://cgit.freedesktop.org/systemd/systemd/commit/?id=2928b0a863

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756903: systemd: Boot hangs if filesystems unavailable

[Zbigniew Jędrzejewski-Szmek]

On Sat, Aug 09, 2014 at 04:36:50PM -0400, Stefan Monnier wrote:
  Well, I consider the sysvinit behaviour buggy and unfortunately this
  lead to broken fstab configurations in the past.
 
 There are 2 changes here:
 1- systemd seems to *wait* for the device to be available, whereas the
old scripts just failed right away if the device was absent.
 2- if the mount fails, this is considered by default as a fatal error.
 
 Which of those 2 was buggy in sysvinit?  The lack of wait or the fact
 that it moved on upon failure?
Both :)

It is not uncommon for systems on SSD to boot to graphical login in ~2s,
which simply is not enough for all hardware, especially USB, to be
reliably detected. Since systemd mounts disks somewhere in the first
second, waiting for devices to show up is simply unavoidable.

You snipped Michaels response for the other issue. There's no way to
guess which filesystems listen in fstab are important, and which
can be skipped.

 I like the idea of mounting/fscking partitions without blocking other
 unrelated boot steps, so I'm OK with adding boot options to help systemd
 do a better job, but blocking the boot just because some random fstab
 entry has an error is really obnoxious (it took me a while to fix my
 machine after installing systemd-sysv because of such an fstab entry
 referring to a disk that was not connected.  Thank god there is
 break=mount!).
 
 Here are my suggestions:
 - In the absence of an explicit request to bootwait, systemd should
   not wait for a device to appear, or at least not anywhere near 90s,
   especially if it is holding back the whole rest of the boot process.
 - If a mount fails, keep on booting.  And then do your best to try and
   bring this problem to the attention of someone (mentioning the
   nofail option in that same message).  Only stop the boot if the
   partition is explicitly marked as critical or stoponfail.
Well, 'fail', which is the default, means just that. Systemd tries
to do the corrent and safe thing by default.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#754340: Unable to run fsck manually when instructed to do so

[Zbigniew Jędrzejewski-Szmek]

On Fri, Jul 18, 2014 at 06:18:28PM +0200, Bas Wijnen wrote:
 When booting into single user mode, things worked as expected.  I was
 unable to remount the fs read-only; the logs you requested are attached.
 (The mei_me messages have always been there, also with sysv init; I
 don't think they are related to the problem.)
Running fsck on a filesystem mounted in rw mode is a bad idea.
So instead of trying to fix things to remount the fs ro, fsck should
finish and succeed or fail *before* the filesystem is remounted rw.
This should sidestep the whole issue of remounting.
So I'd try to answer the question why the fs is mounted rw in the first
place.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#749832: systemd: ignores /run/do-not-hibernate, hibernates after kernel update

[Zbigniew Jędrzejewski-Szmek]

On Fri, May 30, 2014 at 09:00:12PM -0700, Nikolaus Rath wrote:
 On 05/30/2014 10:46 AM, Michael Biebl wrote:
  Am 30.05.2014 18:34, schrieb Nikolaus Rath:
 
  The problem with simply doing nothing on hibernate when
  /run/do-not-hibernate exists, is that this is also not visible to the
  user since there there is no API to communicate that to the user why
  hibernate did not succeed.
 
  Well, in my case I'm calling 'systemctl hibernate' from the command
  line, so a simple message to the console would do.
  
  Do you want to work on such a patch?
  You'd have to implement this check in systemd (or logind) and then
  communicate the failure reason back to the caller.
  That requires updates to the D-Bus interface etc.
 
 Sorry, no. I'm not familiar with D-Bus at all. I'll just create wrapper
 script for systemctl hibernate that does the check, and I assume that's
 not quite suitable for inclusion in the Debian package...
It'd be easier and more worthwhile I think, to work from the other and
and instruct the packaging scripts to also take a hibernation inhibitor.
This would have the advantage that other tools which already know how
to deal with inhibitor locks would be able to notify the user etc.
Something like 'systemd-inhibit --what sleep sleep 1', but smarter.
It seems that the inhibitor api is not flexible enough here, though,
so some upstream discussion would be required.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: init system discussion status

[Zbigniew Jędrzejewski-Szmek]

On Sun, Jan 05, 2014 at 12:16:58AM +0100, Zbigniew Jędrzejewski-Szmek wrote:
 git notes are used to mark backport-worthy commits. See
 http://lists.freedesktop.org/archives/systemd-devel/2013-September/013188.html.
 We currently mark patches as bugfixes (which includes fixes for bugs
 present in the latest release, but not those introduced later),
 or documentation and performance improvements.
 
  Fedora 19 appears to be packaging patches from v204-stable branch
  which I can't find anywhere public.
 It's my private branch I generate Fedora packages from [1]. It's the
 same content as in Fedora git repos [2], but in a more convenient form
 for me. I talked with other systemd maintainers in Fedora about making
 it more official and public, but we haven't found the time to do it
 yet. If it was to be useful to other people, it can certainly be done.
 
 [1] http://kawka.in.waw.pl/git/systemd/
 [2] http://cgit.freedesktop.org/systemd/systemd/
Hi,

stable branches have a new official home:
http://cgit.freedesktop.org/systemd/systemd-stable/

The policy wrt. to bugfixes and stable branches is described in
http://www.freedesktop.org/wiki/Software/systemd/Backports/.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#738316: [Pkg-systemd-maintainers] Bug#738316: Bug#738316: Bug#738316: systemd-inhibit(1) links to wrong man page

[Zbigniew Jędrzejewski-Szmek]

On Sun, Feb 09, 2014 at 11:08:41AM +0100, Michael Stapelberg wrote:
 control: tag -1 + pending
 
 Hi Zbigniew,
 
 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl writes:
  Thanks, fixed in 
  http://cgit.freedesktop.org/systemd/systemd/commit/?id=07b4b9b.
 You missed one instance of “systemd-logind.conf” (line 161) :).
Ooops, fixed now.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#738316: [Pkg-systemd-maintainers] Bug#738316: systemd-inhibit(1) links to wrong man page

[Zbigniew Jędrzejewski-Szmek]

Thanks, fixed in http://cgit.freedesktop.org/systemd/systemd/commit/?id=07b4b9b.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#738207: [Pkg-systemd-maintainers] Bug#738207: Bug#738207: systemd-sysv: sockets are not removed after being stopped

[Zbigniew Jędrzejewski-Szmek]

See also https://bugzilla.redhat.com/show_bug.cgi?id=802748.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#737264: /usr/share/man/man7/systemd.directives.7.gz: systemd.directives(7) is empty

[Zbigniew Jędrzejewski-Szmek]

On Mon, Feb 03, 2014 at 02:55:41PM +0100, Michael Biebl wrote:
 Am 03.02.2014 12:42, schrieb Michael Biebl:
  Am 02.02.2014 23:23, schrieb Sam Morris:
  
  Hm, I think we disagree on the desired results. :) I don't mean that the
  man page is absolutely empty... I'm referring to the actaul list of each
  directive along with the man page in which it is documented. Sorry for
  being unclear.
 
  I'm attaching the man page from version 204-5 so you can see what I
  mean.
  
  Ok, I understand now what you mean.
  
  Not quite sure yet. Seems to be a problem of using out-of-tree builds.
  At least I can *not* reproduce the issue with an in-tree build.
  
  When doing the oot build I'm getting a lot of those warnings:
  
  
  make[2]: Circular man/systemd.directives.xml - man/bootup.xml
  dependency dropped.
  make[2]: Circular man/systemd.directives.xml - man/daemon.xml
  dependency dropped.
  make[2]: Circular man/systemd.directives.xml - man/halt.xml dependency
  dropped.
  ...
  
I've seen this too, but only on Debian.

 Zbyszek, in case you want to reproduce the issue:
 - git clone the systemd repo
 - cleanup any pregenerated files: git -xdf,
 - ./autogen.sh
 - use out-of-tree build:
  mkdir build  cd build  ../configure  make
I do out-of-tree build most of the time, so and in general everything
works well. I just run those instruction on Fedora and the issue is
not there (automake-1.13.4-5.fc20.noarch, make-3.82-19.fc20.x86_64,
autoconf-2.69-14.fc20.noarch). It would be good to look where exactly
this circular dependency comes from.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: multiple init systems - formal resolution proposal

[Zbigniew Jędrzejewski-Szmek]

On Sat, Feb 01, 2014 at 11:25:01AM -0500, Steve Langasek wrote:
 Josselin has asserted not only that he considers systemd-as-init a hard
 dependency for GNOME in jessie, but that he expects the release team to side
 with him over the Technical Committee if the TC does not agree with him.
 This is unconscionable, given that there are two very straightforward and
 obvious technical solutions to this problem:
 
  - split the systemd package (as previously discussed) into separate
binaries, for the init system vs. the dbus interfaces, and have GNOME
depend on the latter
I the light of what I know about this issue from the systemd side,
splitting the package into multiple independent components is much
harder than it looks. For one, logind used to call into PID1 for
shutdown/suspend/... but not it'll also attempt to start the user
manager and tell PID1 to manage resource limits. I don't suppose
systemd-shim is ready for that. Second, logind uses journald for
logging. This actually is also an issue for gnome: gdm now logs to the
journal, which makes debugging gdm initialization issues so much
easier. Recently patches which redirect X logs to the journal have
been accepted (even if not merged yet afaik) [1]. The hypothethical
systemd-logind package would not only use a different provider of the
most crucial services, but would also lose existing logging
mechanism. Apart from that, loginctl communicates with PID1 to show
cgroup information... Put in a lot of work to build a more brittle
system and lose functionality? Even if it might have been easier for
old systemd versions, the components are now fairly tightly coupled.
Even if such a split could be done with enough man-hours thrown at it,
I think it's obvious why Joss and other people who use systemd aren't
thrilled about the prospect, especially with #727708 undecided.

  - have the systemd package declare one or more virtual packages via
Provides:, which GNOME packages depend on and one or more alternative
implementations may also provide.
As argued elsewhere, since there are no alternative providers, this
would amount to a hard dependency on systemd, which gnome is not allowed
to do.

Josselin's stance, even if strongly expressed, seems accurate.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=722889


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: call for votes on default Linux init system for jessie

[Zbigniew Jędrzejewski-Szmek]

On Tue, Jan 28, 2014 at 11:23:11AM -0800, Don Armstrong wrote:
 On Tue, 28 Jan 2014, Neil McGovern wrote:
  On Tue, Jan 28, 2014 at 11:05:18AM -0800, Don Armstrong wrote:
 Where feasible, software should interoperate with non-default init
 systems; maintainers are encouraged to accept technically sound
 patches to enable interoperation, even if it results in degraded
 operation.
   
  
  Did you mean degraded operation while running under the non-default
  init. or degraded operation while running under the default init.?
 
 The former. So :
 
Where feasible, software should interoperate with non-default init
systems; maintainers are encouraged to accept technically sound
patches to enable interoperation, even if it results in degraded
operation while running under the non-default init.
Maybe I'm dense...

Scenario: Let's say that OpenRC is the new default init and in the
meanwhile, Gnome has gained a dependency on systemd. A patch to
support Upstart in Gnome is posted that partially breaks the
functionality under systemd.

By your wording, maintainers are encouraged to accept the patch.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#719945: [Pkg-systemd-maintainers] Bug#719945: Bug#719945: systemd: Hangs during shutdown (likely NFS-related)

[Zbigniew Jędrzejewski-Szmek]

On Sun, Jan 26, 2014 at 07:44:29PM +0100, Zbigniew Jędrzejewski-Szmek wrote:
 On Tue, Aug 20, 2013 at 11:30:25AM +0200, Michael Biebl wrote:
  Am 20.08.2013 01:14, schrieb Andreas Kloeckner:
   No idea, up to you. Specifying _netdev for an NFS mount feels a bit
   redundant. It's also a regression of sorts, since sysvinit appears to
   handle this ok.
  
  sysvinit (or rather /etc/init.d/mountnfs.sh) contains a hard-coded list
  of network file systems. I'm not sure if this list is complete and
  would fail for more exotic ones, too.
  I'm not sure, if I'd like to maintain such a hard-coded list within
  systemd as this approach looks like a hack to me.
 Hack or no hack, we maintain such a list :P
 
 bool fstype_is_network(const char *fstype) {
 static const char table[] =
 cifs\0
 smbfs\0
 ncpfs\0
 ncp\0
 nfs\0
 nfs4\0
 gfs\0
 gfs2\0;
 
 return nulstr_contains(table, fstype);
 }
 
 _netdev can be used to tell systemd about fs types in addition to that
 lits. This actually doesn't change all that often, so it's not much of
 a problem.
I see that this was already pointed out in the other subthread.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#719945: [Pkg-systemd-maintainers] Bug#719945: Bug#719945: systemd: Hangs during shutdown (likely NFS-related)

[Zbigniew Jędrzejewski-Szmek]

On Tue, Aug 20, 2013 at 11:30:25AM +0200, Michael Biebl wrote:
 Am 20.08.2013 01:14, schrieb Andreas Kloeckner:
  No idea, up to you. Specifying _netdev for an NFS mount feels a bit
  redundant. It's also a regression of sorts, since sysvinit appears to
  handle this ok.
 
 sysvinit (or rather /etc/init.d/mountnfs.sh) contains a hard-coded list
 of network file systems. I'm not sure if this list is complete and
 would fail for more exotic ones, too.
 I'm not sure, if I'd like to maintain such a hard-coded list within
 systemd as this approach looks like a hack to me.
Hack or no hack, we maintain such a list :P

bool fstype_is_network(const char *fstype) {
static const char table[] =
cifs\0
smbfs\0
ncpfs\0
ncp\0
nfs\0
nfs4\0
gfs\0
gfs2\0;

return nulstr_contains(table, fstype);
}

_netdev can be used to tell systemd about fs types in addition to that
lits. This actually doesn't change all that often, so it's not much of
a problem.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#719945: [Pkg-systemd-maintainers] Bug#719945: systemd: Hangs during shutdown (likely NFS-related)

[Zbigniew Jędrzejewski-Szmek]

On Wed, Jan 22, 2014 at 11:59:27AM +, Sam Morris wrote:
 Package: systemd
 Version: 204-6
 Followup-For: Bug #719945
 
 I'm seeing this bug. Adding _netdev to the fstab entry doesn't help.
 I'm able to fiddle with configuration if you need my help to debug this
 further.
 
 /etc/fstab:
 
   gaia:/home /home nfs rw,nosuid,nodev,nfsvers=3,hard,intr,sloppy,_netdev 
 0 0
 
 Generator output after adding _netdev:
 
   diff --git i/home.mount w/home.mount
   index a3741bf..3d42691 100644
   --- i/home.mount
   +++ w/home.mount
   @@ -16,4 +16,4 @@ What=gaia:/home
Where=/home
Type=nfs
FsckPassNo=0
   -Options=rw,nosuid,nodev,nfsvers=3,hard,intr,sloppy
   +Options=rw,nosuid,nodev,nfsvers=3,hard,intr,sloppy,_netdev
 
 
 $ systemctl show home.mount
   Id=home.mount
   Names=home.mount
   Requires=-.mount
   Wants=network-online.target
   Conflicts=umount.target
   Before=local-fs.target umount.target remote-fs.target
This is wrong. The dependency for local-fs.target should not be there.
It'll cause the fs to be kept around until after the network is gone.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: init system discussion status

[Zbigniew Jędrzejewski-Szmek]

On Sat, Jan 04, 2014 at 06:59:46PM +, Dimitri John Ledkov wrote:
 Also it is sad that systemd upstream is actively promoting for
 everyone to execute runtime checks of is systemd-init pid1,...
This is done public systemd libraries to become NOPs if not running on
or not compiled for systemd, to make it easier to integrate
systemd-related functionality in programs portable to other
environments. Full original functionality can be always trivially
restored.

Built-in systemd components do no such checks, and generally are
written with the assumption that they are using the same systemd
versions. (What I'm saying is quite inprecise, but because it's not
the main point, I don't want to go into details.)

 what's systemd version number,
I'm not aware of any such checks.

 granted Martin Pitt did identify this
 problem and fixed majority of opensource projects to check for the
 requested/required functionality, instead of arbitrary transitive
 checks of pid1. This in part enables to run systemd-logind without
 systemd-init as pid 1.
 
 Also which upstream are staying with? systemd upstream git history[4]
 has only one branch, which is linear with linear version number
 increments, without any stable release branches or other indications
 of which patches are stable (or possibly security) bugfixes.
git notes are used to mark backport-worthy commits. See
http://lists.freedesktop.org/archives/systemd-devel/2013-September/013188.html.
We currently mark patches as bugfixes (which includes fixes for bugs
present in the latest release, but not those introduced later),
or documentation and performance improvements.

 Fedora 19 appears to be packaging patches from v204-stable branch
 which I can't find anywhere public.
It's my private branch I generate Fedora packages from [1]. It's the
same content as in Fedora git repos [2], but in a more convenient form
for me. I talked with other systemd maintainers in Fedora about making
it more official and public, but we haven't found the time to do it
yet. If it was to be useful to other people, it can certainly be done.

[1] http://kawka.in.waw.pl/git/systemd/
[2] http://cgit.freedesktop.org/systemd/systemd/

 Thankfully it's not a single giant patch as it's
 done by RedHat for their kernels, but actually git am formatted series
 of 116 patches[5].

 The diffstat between:
 - debian package and git v204 checkout is: 44 files, 246 +, 566 -
 - fedora 19-204 and v204 tarball: 102 files, 11368 +, 1366 -
 - rhel 7-beta-src-iso and v207 tarball: 133 files changed, 2364
 insertions(+), 1686 deletions(-)
 
 Which is a significant chunk of fixes. Looking at some of them, they
 are true gems like don't truncate and loose multiline syslog
 messages which is not fixed in Debian at the moment. Can we please
 not have journald by default in jessie, cause I don't want my syslog
 truncated?!
 
 In my opinion it is unwise to ship something into stable, ahead of
 upstream doing so for their support customers (RedHat Enterprise
 Linux).
I think you're overestimating the (direct) influence of Redhat on
system upstream bug fixes. There are patches (don't truncate and
loose multiline... being one of them) done as a result of a bug
reported in RHEL, but their number is insignificant compared to the
number of bugs reported and fixed in Fedora, the upstream bugtracker
and other distro's bugtrackers. Actually Debian is suffering here
becuase of the large version gap to current upstream. It makes it
much harder to reproduce bugs, and if fixes are done, additional
work is required in backporting. After various codebase cleanups
and the the post-208 rewrite to use libsystemd-bus in systemd
components, any patch which touch dbus codepaths has to be rewritten
rather than cherry-picked to such old versions.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: init system discussion status

[Zbigniew Jędrzejewski-Szmek]

On Sun, Jan 05, 2014 at 12:54:04AM +, Dimitri John Ledkov wrote:
 So components inside systemd-source tree do not follow what's advised
 to all other projects: e.g. link or statically include sd_* helper
 files, and perform runtime checks?
The advice for other projects assumes that systemd is
optional. systemd components assume that they will be installed as
part of systemd, so such compile time checks would make little sense.
Various configure swithes can be used to disable components, but not
the main systemd part.

 How much functionality / api is exported by libraries/dbus of systemd
 to move components out of the systemd tree and into separate projects.
systemd components share the same codebase, and they all make
extensive use of shared functionality (src/shared part in the source
tree, but not only there) which has does not have any stable external
API. So it's not possible to move out any of them in any practical
sense. This is a bit like with the kernel.

 Or, what's more interesting, projects external to systemd integrating
 on the same level. E.g. in upstart, all common code is factored into
 stable-api/abi libnih, which is then free to use by anyone (e.g. event
 bridges, mount helpers, etc.) and all communication to/from upstart is
 exported via dbus IPC (on private socket or systemd dbus). If
 code-sharing is only happening by being part of the systemd codebase,
 that also increases barrier of entry. 
In systemd most of the D-Bus interface, share-library interface,
and various other APIs are stable, but the internal C API is not. It
simply changes too fast and it is not possible to make it public.

 E.g. it would be benefitial for systemd to support hallyn's
 cgroupmanager (which can be used standalone or not).
I'm not really an expert in this area, so please don't take my words
as authoritiative, but... No, it wouldn't. systemd (the binary running
as PID 1) uses cgroup functionality extensively. If it was to become
external (in a seperate process) some sort of of synchronous protocol
would have to be used, complicating things immensly, for no gain. If
it was to be used as a library, that is in theory possible, but we
would replace working code with something external and not even
working at this point. Also cgmanager is supposed to follow a
different philosophy.

  Also which upstream are staying with? systemd upstream git history[4]
  has only one branch, which is linear with linear version number
  increments, without any stable release branches or other indications
  of which patches are stable (or possibly security) bugfixes.
  git notes are used to mark backport-worthy commits. See
  http://lists.freedesktop.org/archives/systemd-devel/2013-September/013188.html.
  We currently mark patches as bugfixes (which includes fixes for bugs
  present in the latest release, but not those introduced later),
  or documentation and performance improvements.
 
 
 Thank you for this link, I was not aware of this policy. This is the
 first time i see git-notes used for tracking bugs needed for
 stable/security/documentation, and it was hard for me to find.
 Does also mean such stable maintenance only started in September 2013?
 Or some other scheme used before that? Current debian version of
 systemd is v204, tagged in May 2013.
In this form, yes.

  Fedora 19 appears to be packaging patches from v204-stable branch
  which I can't find anywhere public.
  It's my private branch I generate Fedora packages from [1]. It's the
  same content as in Fedora git repos [2], but in a more convenient form
  for me. I talked with other systemd maintainers in Fedora about making
  it more official and public, but we haven't found the time to do it
  yet. If it was to be useful to other people, it can certainly be done.
 
 
 Thank you for pointing out where it is. Maybe add a URL in the spec
 file pointing to where it is? Cause I've search hard and didn't manage
 to find where it's maintained.
 (or push a mirror to github systemd projects or some other place if
 you require team commit access)
Either that, or fedorahosted, or the upstream... We'll have to pick something,
yes.

 I guess that also makes the RHEL patch series based on yours/fedora
 releases. Are there changes that are in RHEL7 and not in fedora, your
 fork, and upstream?
AFAIK, access to RHEL source code would require a subscription. I don't
know what RHEL packages include, sorry.

 Debian SytemD maintainers, has the ~116 fedora's stable fixes patch
 series been reviewed and decided to not be applied? What about stable
 branches from other distributions, have those been investigated?
Normally maintainers from various distros (including Fedora and Debian)
upstream all bugfix patches. So pulling from upstream is enough.

  In my opinion it is unwise to ship something into stable, ahead of
  upstream doing so for their support customers (RedHat Enterprise
  Linux).
  I think you're overestimating the (direct) influence of Redhat on
  system upstream 

Bug#727708: init system discussion status

[Zbigniew Jędrzejewski-Szmek]

On Sun, Jan 05, 2014 at 02:11:41AM +, Dimitri John Ledkov wrote:
  Were you unable to find
  http://pkgs.fedoraproject.org/cgit/systemd.git/log/?h=f19 ?  It's where
  Fedora has all of their packaging..

 As explained by Zbigniew Jędrzejewski-Szmek, that is not actually
 where that happens.
Hm, I was trying to explain that it *is* where it happens.
The only difference is that in the Fedora package those commits are
serialized as diffs, and my git tree has them as a, well, git tree.

That git repo is undiclosed only because it isn't particarly important.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: requirement of non-forking startup protocol

[Zbigniew Jędrzejewski-Szmek]

| 8. Policy rules for support for init systems must:
| 
|(a) Specify the use of a non-forking startup protocol (for
|upstart and systemd),
I'm not sure about upstart, but systemd is perfectly happy with
daemons which double fork (Type=forking in systemd parlance).
 It is mildly discouraged, because:

1. it is hard to get right
2. it is more code than the other options
3. it is easier to start the program manually if non-forking protocol is used

For new code, other protocols are certainly better. But for existing
daemons which work correctly, points 1 and 2 don't matter, and 3 is not
important enough.

This requirement might force mantainers to modify some hairy
internals in the startup code of daemons to avoid double forking. This
seems pointless, as in most cases it wouldn't result in any noticable
difference in speed or behaviour or correctness.

I think this should be changed to:

| 8. Policy rules for support for init systems must:
| 
|(a) Encourage the use of a non-forking startup protocol (for
|upstart and systemd),

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: requirement of non-forking startup protocol

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 02, 2014 at 10:04:12PM +, Ian Jackson wrote:
 Zbigniew Jędrzejewski-Szmek writes (Bug#727708: requirement of non-forking 
 startup protocol):
  | 8. Policy rules for support for init systems must:
  | 
  |(a) Specify the use of a non-forking startup protocol (for
  |upstart and systemd),
 
 [ Replying to this thread after a large glass of wine is probalby a
 bad idea, but this one seems OK I hope.  Please forgive me if I'm
 incoherent or rude, although of osurse I'll try not to be. ]
In vino veritas :) Anyway, your mail seems fine :)

  I'm not sure about upstart, but systemd is perfectly happy with
  daemons which double fork (Type=forking in systemd parlance).
   It is mildly discouraged, because:
  
  1. it is hard to get right
  2. it is more code than the other options
  3. it is easier to start the program manually if non-forking protocol is 
  used
 
 Am I right in thingking that this is what is described as guess main
 pid in the systemd documentation ?  In which case it is indeed
 discouraged.
Not always. If PIDFile= is specified, than GuessMainPID is not necessary.
Also, if there just one process after initialization has finished (which
is normally true with double forking), guessing the main PID is trivial,
so GuessMainPID is also fine. So it's mainly GuessMainPID=true with a dameon
consisting of multiple processes that is discouraged.

  For new code, other protocols are certainly better. But for existing
  daemons which work correctly, points 1 and 2 don't matter, and 3 is not
  important enough.
 
 I think if we're going to the trouble of converting all of the init
 systems, we should do so once and have them use the best arrangements.

  This requirement might force mantainers to modify some hairy
  internals in the startup code of daemons to avoid double forking. This
  seems pointless, as in most cases it wouldn't result in any noticable
  difference in speed or behaviour or correctness.
 
 Does this actualy arise as a problem in practice ?  I find it
 difficult to think of a case where it would but perhaps you know of
 one.
I don't have any examples. But as you yourself argued, changing both
the build system and the code and init system wrapper to add this
functionality *is* a bit of work. It's not too much work to do if
there's a reason for it, but in case of correctly working
double-forking daemons is see no reason. Multiplying it by all packages
in the archive to be converted I'd much rather see maintainers doing
things which have visible impact.

  I think this should be changed to:
  
  | 8. Policy rules for support for init systems must:
  | 
  |(a) Encourage the use of a non-forking startup protocol (for
  |upstart and systemd),
 
 In the case of upstart the -forking startup protocols are difficult to
 implement portably so in that case I think we should definitely retain
 a prohibition.
 
 I'm less sure about the systemd version of the resolution.
Ah, yes. So for upstart the trade-offs are different and it must stay
as you drafted it originally.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: Info received (Bug#727708: requirement of non-forking startup protocol)

[Zbigniew Jędrzejewski-Szmek]

 Yeah, most daemons that write external PID files have issues with external
 PID files left from other running instances of the same daemon.  (I assume
 that's what you're talking about.)  It's probably possible to avoid that
 with judicious use of file locking, but that's not common and is more
 complex.

systemd will unlink the pidfile after the daemon is stopped (and during
restart, etc.). Stale files shouldn't be an issue.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: requires in systemd

[Zbigniew Jędrzejewski-Szmek]

 As I understand, a systemd unit with Requires=jobTwo will not start
 without jobTwo running.
If you request the start of jobOne, without jobTwo running,
systemd will start jobTwo in addition to jobOne.

There's also a Requisite= dependency, where if jobTwo isn't runnning,
starting jobOne will fail.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: socket activation

[Zbigniew Jędrzejewski-Szmek]

Steve Langasek vor...@debian.org wrote:
 However, I think this gets to the heart of why upstart upstream has avoided
 ever recommending the use of socket-based activation.  There are some fairly
 fundamental problems that basically halted development of socket-based
 activation in upstart, and a look at systemd usage on
 Fedora led me to believe that systemd had not overcome these problems at
 all.
Your e-mail raises serious doubts about socket activation, so I'll
respond to try to clear up some issues even though most or all points
have been rebutted in other replies in this thread.

Before I do that though, let me say that your mail doesn't show the
level of dilligence that I'd expect from a seasoned Debian developer
and a member of the Technical Committee.

 If I'm not mistaken (no references to hand - sorry), systemd upstream has
 claimed in the course of discussions on debian-devel that lazy activation is
 not the purpose of socket-based activation, and that using socket-based
 activation does not require you to pay the service startup penalty at the
 time of first connection.
You get the *possiblity* of lazy activation, it is one of the
reasons. [1, 2, 3] list many good reasons (rootless access to ports  1024,
simplification of daemons, no explicit dependencies between services,
upgrades and restarts while keeping the socket open, and also lazy activation).
They are all non-conflicting. [3] shows great things that can be done
with lazy activation.

[1] http://0pointer.de/blog/projects/socket-activation.html
[2] http://0pointer.de/blog/projects/systemd.html
[3] http://0pointer.de/blog/projects/socket-activated-containers.html

There are also further plans (at the implemented-but-not-merged stage)
for further functionality: spawning multiple workers using SO_REUSEPORT [4].

[4] http://thread.gmane.org/gmane.comp.sysutils.systemd.devel/15364/focus=15598

 However, this is not borne out by my experiments
 with systemd on Fedora (which I would presume to be the go-to source for
 best practices of systemd service activation).
Fedora has something like 800 unit files. I'm sure that quite a few
could be improved. I don't think that proves anything.

 On Fedora 20, after enabling the sshd and rsync service+socket units (both
 installed but disabled by default on Fedora per their policies on running
 services out-of-the-box) and rebooting, I find that both port 22 and port
 873 are bound by pid 1.  Only upon connecting to the socket does systemd
 actually spawn the server (in the case of sshd, it spawns it as 'sshd
 -i', so has to start up the server anew on each connection;
You can switch to non-lazy single-instance mode by 'systemctl disable
sshd.socket; systemctl enable sshd.service', and back by 'systemctl
disable sshd.service; systemctl enable sshd.socket'.

It'd have been much better to simply ask How do I do (lazy
activation|inetd-style activation| non-lazy activation)? without
drawing all those premature conclusions. I added some more
documentation do systemd.socket(5) [5]. I'm also happy to take any
further documentation rfe's.

[5] http://cgit.freedesktop.org/systemd/systemd/commit/?id=3cf148f

 in the case of
 rsyncd, the .service definition is completely incompatible with socket-based
 activation and any attempt to connect results in the rsyncd.socket unit
 landing in a 'failed' state).
Like Russ, Uoti, and Josselin said, it a bug. Actually rsyncd.service
is fine, but rsyncd.socket is broken. This has been fixed for F19 [6],
but apparently not for F20. F20 came out a week ago, so probably
nobody has noticed so far.  Bug filed [7].

[6] https://bugzilla.redhat.com/show_bug.cgi?id=1018520
[7] https://bugzilla.redhat.com/show_bug.cgi?id=1047236

 If what one is trying to accomplish here is to provide a replacement for
 inetd, then this is perfectly sufficient.  But if one is trying to use
 socket-based activation for the claimed purpose of ensuring service startup
 ordering without the need to declare explicit dependencies, then you must
 accept the penalty of lazy activation
No, this is a complete misunderstanding.

 - which is almost never acceptable in
 a server context (where the purpose of the machine is to run the services
 that you have configured, and they should therefore not be started lazily),
Complete non sequitur. For many services the initialization time
(usually in miliseconds) is an acceptable delay on the first connection.
Probably it is even unnoticable compared to network latency. Please remember
that this is not for sysvinit scripts which do sleep in a loop. [8] is a
nice example of lazy socket activation of services on a massive scale.

[8] https://www.getpantheon.com/blog/pantheon-running-over-50-systemd-units

 and suboptimal even in a client context (since not starting services that
 are on the critical path for boot until the client requests them will
 potentially lead to a significant boot-time slowdown, if all the services
 are doing this).
No. If you have all services 

Bug#731387: [systemd-ui] Bug#731387: workaround for systemadm crashes

[Zbigniew Jędrzejewski-Szmek]

Can you instead try systemd-ui-3, recompiled with a recent vala?
With vala 0.22 it doesn't seem to crash (or at least not that much,
I haven't given it much testing). Also valgrind doesn't complain,
except about un-freed memory during exit.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: [PATCH] systemctl: allow globbing in commands which take multiple unit names

[Zbigniew Jędrzejewski-Szmek]

As discussed on IRC, here's a patch which takes the simple approach of
allowing globbing on loaded unit names in various (almost all :)) systemctl
commands. Comments?

(There were some preperatory patches which I'm not posting here.
Please fetch from http://kawka.in.waw.pl/git/systemd/commit/16a4169b
if you need something that'll apply to current git.)

Zbyszek

---
 man/systemctl.xml | 108 ++-
 src/core/device.c |   2 +-
 src/journal/journalctl.c  |   4 +-
 src/run/run.c |   6 +-
 src/shared/unit-name.c|  23 ++--
 src/shared/unit-name.h|   4 +-
 src/shared/util.c |   7 +
 src/shared/util.h |   1 +
 src/systemctl/systemctl.c | 336 +++---
 src/test/test-unit-name.c |   4 +-
 src/udev/udev-rules.c |   2 +-
 11 files changed, 304 insertions(+), 193 deletions(-)

diff --git a/man/systemctl.xml b/man/systemctl.xml
index 7e0216e..13a 100644
--- a/man/systemctl.xml
+++ b/man/systemctl.xml
@@ -580,15 +580,24 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
 /varlistentry
 
 varlistentry
-  termcommandstart 
replaceableNAME/replaceable.../command/term
+  termcommandstart 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraStart (activate) one or more units specified on the
 command line./para
+
+paraNote that glob patterns operate on a list of currently
+loaded units. Units which are not active and are not in a
+failed state usually are not loaded, and would not be
+matched by any pattern. In addition, in case of
+instantiated units, systemd is often unaware of the
+instance name until the instance has been started. Therefore
+using glob patterns with commandstart/command
+has limited usefulness./para
   /listitem
 /varlistentry
 varlistentry
-  termcommandstop 
replaceableNAME/replaceable.../command/term
+  termcommandstop 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraStop (deactivate) one or more units specified on the
@@ -596,7 +605,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
   /listitem
 /varlistentry
 varlistentry
-  termcommandreload 
replaceableNAME/replaceable.../command/term
+  termcommandreload 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraAsks all units listed on the command line to reload
@@ -617,7 +626,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
 
 /varlistentry
 varlistentry
-  termcommandrestart 
replaceableNAME/replaceable.../command/term
+  termcommandrestart 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraRestart one or more units specified on the command
@@ -626,7 +635,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
   /listitem
 /varlistentry
 varlistentry
-  termcommandtry-restart 
replaceableNAME/replaceable.../command/term
+  termcommandtry-restart 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraRestart one or more units specified on the command
@@ -637,7 +646,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
   /listitem
 /varlistentry
 varlistentry
-  termcommandreload-or-restart 
replaceableNAME/replaceable.../command/term
+  termcommandreload-or-restart 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraReload one or more units if they support it. If not,
@@ -646,7 +655,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
   /listitem
 /varlistentry
 varlistentry
-  termcommandreload-or-try-restart 
replaceableNAME/replaceable.../command/term
+  termcommandreload-or-try-restart 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraReload one or more units if they support it. If not,
@@ -676,7 +685,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
   /listitem
 /varlistentry
 varlistentry
-  termcommandkill 
replaceableNAME/replaceable.../command/term
+  termcommandkill 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 paraSend a signal to one or more processes of the
@@ -687,7 +696,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket 
systemd-udevd.service
   /listitem
 /varlistentry
 varlistentry
-  termcommandis-active 
replaceableNAME/replaceable.../command/term
+  termcommandis-active 
replaceablePATTERN/replaceable.../command/term
 
   listitem
 

Bug#727708: [PATCH] systemctl: allow globbing in commands which take multiple unit names

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 26, 2013 at 09:41:42PM +0100, Lennart Poettering wrote:
 On Thu, 26.12.13 10:09, Russ Allbery (r...@debian.org) wrote:
  What I've switched to instead is using tiny enums for this purpose.  So:
  
  enum mangle_type {
  MANGLE_NOGLOB = 0,
  MANGLE_GLOB   = 1
  };
  
  and then at the call site:
  
  n = unit_name_mangle(e, MANGLE_NOGLOB);
  
  which makes the meaning of that argument immediately obvious.
 
 As long as this is just one boolean arg on functions, or the functions
 are internally used I think booleans are fine to use.
 
 I don't think flag fields are necessarily the best choice for APIs in
 general. For APIs that are built around a context object seperate
 boolean setter calls are the better choice (i.e. foobar_set_waldo() to
 set som boolean called waldo on a context object foobar). 
I went ahead and added MANGLE_GLOB/NOGLOB as Russ suggested. I think
that it make the code in systemctl (which is pretty convoluted) easier
to read. It is a separate commit, so it's easy to revert if you think
the change isn't worth it.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: [PATCH] systemctl: allow globbing in commands which take multiple unit names

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 26, 2013 at 06:49:31PM +0100, Lennart Poettering wrote:
 Looks good. But maybe an additional warning should be printed if people
 use globs on the systemctl start command line? Some shorter version of
 the blurb you added to the man page? i am pretty sure people will run
 into this problem, and we should tell them what is going on...

I'm afraid it's hard to distinguish what is on purpose, and what is not.
One option might be to warn on start if no units were matched...
I left it out for now.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: init multiple instances of a daemon

[Zbigniew Jędrzejewski-Szmek]

On Sun, Dec 22, 2013 at 03:56:04PM -0800, Russ Allbery wrote:
 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl writes:
 
  But this should be safe for instance units, so I'd like to see
  'systemctl stop/status/... server@*.service' implemented.
 
 That would be very useful for distribution packagers.  If, for example,
 you support a multiple-instance Tomcat configuration (which is quite nice
 to do, since it isolates Java applications that may require different
 override JARs), you want to restart all of them when you upgrade the
 packaged Tomcat.

This bug can be considered fixed upstream:
http://cgit.freedesktop.org/systemd/systemd/commit/?id=e3e0314b56012f7

I doubt that anyone would want to port it to systemd = 208, because
in the porting to libsystemd-bus has been completely rewritten from
earlier versions. But it'll be part of systemd 209.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: init multiple instances of a daemon

[Zbigniew Jędrzejewski-Szmek]

On Sun, Dec 22, 2013 at 12:48:47PM -0800, Russ Allbery wrote:
 upstart calls this instances and systemd calls this unit
 templates.
We too call them instances: an instance is created from a template.

 Daniel Pocock dan...@pocock.com.au writes:
 
  Just to clarify: does this mean systemd and upstart can refer to the
  instances collectively or individually as required?  E.g. you can tell
  it restart all instances of httpd (on dpkg upgrade) or just restart one
  specific instance (after a config change)?
 
 It looks like both upstart and systemd don't provide direct mechanisms to
 manage all instances.
That's true (I'm only speaking about systemd). There have been requests for
such functionality before, but nothing was implemented. But I think we should
revisit this topic... I recently added globbing to a bunch of systemctl verbs
for listing stuff (list-units, list-sockets, etc.), and it was actually
trivial. I felt that allowing globing for verbs that influence state (start,
stop, enable, disable, ...) was too dangerous. But this should be safe for
instance units, so I'd like to see 'systemctl stop/status/... server@*.service'
implemented.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#732798: [Pkg-systemd-maintainers] Bug#732798: document --full as an option for status

[Zbigniew Jędrzejewski-Szmek]

On Sat, Dec 21, 2013 at 10:32:46PM +0100, Michael Stapelberg wrote:
 Russ Allbery r...@debian.org writes:
  systemctl status service truncates log messages from the journal to
  80 columns by default.  Some experimentation revealed that --full will
  prevent this and show the full log message, but this isn't documented
  in the systemctl man page.  --full only mentions a couple of other
  commands, and status doesn't reference --full.
 
  Could you add a cross-reference or two?
 I’ll let Zbigniew comment on that, given that he already pushed the last
 documentation request upstream directly. Thanks!
http://cgit.freedesktop.org/systemd/systemd/commit/?id=e213d1a3c3
http://cgit.freedesktop.org/systemd/systemd/commit/?id=69d918b

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#732603: ITP: python-daemonize -- Python module for writing system daemons

[Zbigniew Jędrzejewski-Szmek]

On Thu, Dec 19, 2013 at 10:19:55AM +, Neil Williams wrote:
 On Thu, 19 Dec 2013 10:46:02 +0100
 Mike Gabriel mike.gabr...@das-netzwerkteam.de wrote:
 
  Package: wnpp
  Severity: wishlist
  Owner: Mike Gabriel mike.gabr...@das-netzwerkteam.de
  
  * Package name: python-daemonize
Version : 2.2.1
Upstream Author : Ilya Otyutskiy ilya.otyuts...@icloud.com
  * URL : http://pypi.python.org/pypi/daemonize
  * License : Expat
Programming Lang: Python
Description : Python module for writing system daemons
 
 How does this compare to python-daemon which already exists in Debian
 and which does the same job?
Hi,
this probably should wait until [1] is resolved.

Nevertheless, looking at http://daemonize.sourceforge.net/daemonize.txt,
this package is racy: the parent exits before child is ready (or
even executed). This means that it cannot be used to reliably start a
dameon. A nice write-up of what is necessary is in [2].

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708
[2] http://www.freedesktop.org/software/systemd/man/daemon.html


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#732157: [Pkg-systemd-maintainers] Bug#732157: Notification in non-C programs

[Zbigniew Jędrzejewski-Szmek]

On Tue, Dec 17, 2013 at 06:51:19PM -0800, Nikolaus Rath wrote:
 A nice thing about the SIGSTOP mechanism is that it can trivially be
 used even in programs not written in C. Any Perl or Python script (and
 there are plenty of daemons written in these languages) can send itself
 SIGSTOP. Using sd_notify requires to re-implement the protocol or to add
 C adaptor code just for this...
The adaptor is there for some languages [1]. I have only used Python,
where it is trivially easy
   from systemd import daemon
   daemon.notify('READY=1')

Zbyszek

[1] http://www.freedesktop.org/wiki/Software/systemd/#relatedpackages


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#732156: [Pkg-systemd-maintainers] Bug#732156: Bug#732156: Precise documentation for ExecStart command syntax

[Zbigniew Jędrzejewski-Szmek]

On Sat, Dec 14, 2013 at 11:18:55PM +0100, Michael Stapelberg wrote:
 Hi Ian,
 
 Ian Jackson ijack...@chiark.greenend.org.uk writes:
  Reading systemd.service(5), I'm confused about the exact syntax and
  quoting for the ExecStart= directive.
I've now pushed a patch which expands this section and add a few examples:
http://cgit.freedesktop.org/systemd/systemd/commit/?id=ec6039b.

 I presume changing the wording of systemd.service(5) to read “Commands
 with their (whitespace-separated) arguments […]” would clarify. Do you
 agree? If so, I’ll send a patch upstream to change this.
We seem to have many question about this, so I think it is good
to be very explicit here.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#725422: [Pkg-systemd-maintainers] Bug#725422: Bug#725422: Bug#725422: systemd set kernel.sysrq=16

[Zbigniew Jędrzejewski-Szmek]

Hi Goffredo,

On Thu, Nov 28, 2013 at 10:19:41PM +0100, Goffredo Baroncelli wrote:
 Hi Michael,
 
 On 2013-11-28 21:51, Michael Stapelberg wrote:
  Hi Goffredo,
  
  Goffredo Baroncelli kreij...@inwind.it writes:
  Systemd has as main target Fedora. Debian is different from Fedora, so
  Nope.
  
  systemd is not targeting one distribution as its main target. It is
  running on many distributions, with no particular focus on any single
  one of them.
 
 Let me rephrase this sentence: systemd is not sensible to the debian
 requirements; otherwise there would not be the problems to support the
 other debian kernels (like hurd and bsd).
Even if this were true, this bug report is not about hurd or bsd.
Since this is a bug report, let me ask again: is there anything to fix
with the sysctl stuff?

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#725422: [Pkg-systemd-maintainers] Bug#725422: Bug#725422: systemd set kernel.sysrq=16

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 28, 2013 at 08:50:03AM +0100, Goffredo Baroncelli wrote:
 [1] I repeat I am not arguing about the values, I am arguing that a
 package which aim to replace the init must change a setting which aren't
 related to the package functionality.
Hi Goffredo,

upstream value for the sysrq setting was chosen as a safe default.
If you thing that the default should be changed, it is definitely
reasonable to discuss it upstream on systemd-de...@l.fd.o.

The mechanism to set the default is unlikely to change though.
Modulo the bugs with ordering that Michael pointed out, this
mechanism is nice and clean, and provides a safe way for the
administrator to locally override settings.

Also, this is a basic system setting, where it makes sense to change
the kernel default by default, so it is natural that systemd
does that. The goal of the systemd project is to take care of the
basic system setup.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#725422: [Pkg-systemd-maintainers] Bug#725422: Bug#725422: systemd set kernel.sysrq=16

[Zbigniew Jędrzejewski-Szmek]

So we're in agreement, except for this last point...

On Thu, Nov 28, 2013 at 07:27:34PM +0100, Goffredo Baroncelli wrote:
 This is the point where we have a totally different opinion:
 - is job of systemd to set the sysctl values ? Yes.
 - is job of systemd decide *which values* have to be set ? IMHO no. This
 is a distribution responsibility. And I think that a distribution should
 not accept the systemd defaults, but should provide the own ones.
Why would Debian's defaults differ from any other general purpose
distro? If you think Debian should do something different than the
upstream default of '16', then most probably upstream should do so too.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: tech-ctte: Decide which init system to default to in Debian.

[Zbigniew Jędrzejewski-Szmek]

On Wed, Oct 30, 2013 at 03:10:16PM +0100, Helmut Grohne wrote:
 The interfaces of all init systems (except sysvinit, but are we really
 considering that one?) still are somewhat in flux, so this is the point
 where we can still influence and shape them.
 
 dream
 Imagine a world where upstart and systemd would use the same syntax
 (structure) but implement different and somewhat overlapping aspects.
 Maybe 50% of the daemons would work with either of them without needing
 changes. wakeup/ But now we call it exec here and ExecStart there,
 export here and Environment there, and chdir here and
 WorkingDirectory there. Bummer.
 /dream
Hi Helmut,
exec vs. ExecStart= and export vs. Environment= is easy.
Anyone can whip up a sed script to convert between those. The question
is how to deal with more advanced options. Let's say that I have a
systemd unit with

CapabilityBoundingSet=CAP_SYS_TIME# limit the capability bounding set to 
CAP_SYS_TIME
PrivateTmp=yes# run with unshared /tmp
InaccessibleDirectories=/home # run without access to /home
WatchdogSec=60# consider the service dead if it hasn't 
pinged the
  # manager for in the last minute
Restart=on-failure# restart service on watchdog failure or 
unclean exit

This isn't a question of syntax, it's a question of significant
functionality in the manager. I think that sharing service
descriptions between disparate init managers is infeasible, unless we
forbid the use of anything but the basic features.

Another thing that is turning into a significant gap is socket
activation.  In systemd, socket activation allows the service to
receive multiple open sockets (e.g. ports 80 and 443 for an httpd
server). Socket activation of daemons is cool:
- it is very easy to write such a daemon, it must just do accept(), read() and 
write()
- resource efficent because of activation on demand
- great for running unpriviledged, since the daemon only does accept(), read() 
and write()
- we get rid of a lot of inter-daemon ordering dependencies.
With the recent addition of (experimental) systemd-socket-proxyd[1],
even daemons like apache which do not support socket activation
internally can be launched on demand by wrapping them with a
helper. Socket activation is a great technique, bound to be used more.
Achieving the same functionality with init scripts is very
hard, and AFAIK, upstart doesn't support socket activation with
more than one socket.

[1] http://www.freedesktop.org/software/systemd/man/systemd-socket-proxyd.html

 Oh wait, I am talking to one of the guys who can actually fix this. :)

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#727708: tech-ctte: Decide which init system to default to in Debian.

[Zbigniew Jędrzejewski-Szmek]

On Wed, Oct 30, 2013 at 09:50:53PM -0400, Theodore Ts'o wrote:
 On Wed, Oct 30, 2013 at 06:18:29PM -0700, Russ Allbery wrote:
  I suspect you and I have a root disagreement over the utility of exposing
  some of those degrees of freedom to every init script author, but if you
  have some more specific examples of policy that you wanted to change but
  couldn't, I'd be interested in examples.
 
 It's not necessarily the init script author who might want the degrees
 of freedom, but the local system administrator.
 
 The most basic is the idea that whether you can control (via shell
 scrpit fragments) whether or not a service should start at all, and
 what options or environments should be enabled by pasing some file.
 The fact that we can put that sort of thing in configuration files
 such as /etc/default/*, for example.
Both systemd and upstart support this well enough.

With systemd you can say EnvironmentFile=/etc/default/whatever
and use the variables in ExecStart=/path/to/prog $VAR1 $VAR2.
This is usually discouraged, not because it doesn't work, but
because it uses two files to provide very little value. In majority
of cases it turns out that the settings in /etc/default either
can't be changed in an useful way, or are better read by the daemon
itself from a different configuration file. But if you want this, then
it's there.

If the settings are to complicated for the declarative style, you
can always use an helper shell script to launch the daemon.
Again, discouraged, but certainly supported.

 Yes, yes, you can do this via if you use system V init scripts scripts
 in backwards compatibility mode, but you've argued that we should be
 moving briskly away from that.  In which case system administrators
 will need to hand-edit the services files by hand, which will no doubt
 increase the chances of conflicts at package upgrade time, compared to
 if the configuration options were isolated away in files such as
 /etc/default/rsync (for example).
Actually this doesn't happen that often. Typical systemd service file
has 1-3 lines of documentation, 1+ lines of ExecStart stuff, and a few
additional settings. They are mostly orthogonal, so you just override
the one you need. Probably most common case is to override ExecStart=,
done by adding the file like /etc/systemd/system/myservice.d/custom-start.conf
which only overrides ExecStart. Nice and simple, no conflicts on upgrade.

  I realize that
  the local administrator may have other goals, and they should have ways of
  achieving them, but both systemd and upstart support running SysV init
  scripts for those cases.
 
 If the package does not ship a SysV init script (which is your ideal
 long-term outcome), that may not be very practical option for a system
 adminsitrator who may need to recreate a SysV init script, especially
 if the service file is rather complicated, or is using some of the
 more advanced feature of systemd/upstart.
Why would you recreate the whole script? Let the init manager take
care of starting and stopping and supervising, and only do the custom
stuff in the script. Then it should be just a few lines.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#725815: [Pkg-systemd-maintainers] Bug#725815: [systemd-sysv] CIFS-mounts in fstab with comment=x-systemd and automount delay shutdown

[Zbigniew Jędrzejewski-Szmek]

On Tue, Oct 08, 2013 at 11:49:48PM +0200, Michael Biebl wrote:
 Am 08.10.2013 22:23, schrieb Silvério Santos:
  Is it possible that the network is torn down before the unmount happens?
Fixed in http://cgit.freedesktop.org/systemd/systemd/commit/?id=77009452cf.
http://cgit.freedesktop.org/systemd/systemd/commit/?id=fc676b00a7 is alos 
relevant.


signature.asc
Description: Digital signature

Bug#718592: [Pkg-systemd-maintainers] Bug#718592: Handle shipping .socket files

[Zbigniew Jędrzejewski-Szmek]

On Fri, Aug 02, 2013 at 07:04:49PM +0200, Michael Stapelberg wrote:
 Package: dh-systemd
 Version: 1.7
 Severity: normal
 
 Hi,
 
 as discussed at
 http://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2013-August/000328.html,
 it would be good for dh-systemd to be able to ship .socket files without
 the maintainer having to manually mkdir/cp files around.
 
 My inclination is to use debian/package.socket and
 debian/package.service, but install debian/package.service as
 package@.service (note the @ sign).
What about services which have sockets with Accept=no? Will
dh-systemd know not to add the @ sign then?

Zbyszek
-- 
they are not broken. they are refucktored
   -- alxchk


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#717491: [Pkg-systemd-maintainers] Bug#717491: documentation: file references all point to /usr/lib

[Zbigniew Jędrzejewski-Szmek]

On Sun, Jul 21, 2013 at 02:59:14PM +0200, Michael Biebl wrote:
 But since we already do XSL transformations, we can just use xslt to do
 the processing for us.
There's a list of replacements called $(substitutions) in Makefile.am,
which can be used to insert configure variables into the .c sources
and also into .xml sources. The substitutions are then inserted using
ENTITY; syntax. I think that currently the whole syntax is not being
used, because Lennart removed the substitutions from systemd.unit.xml.
But separate /usr is a supported configuration, so I think we should
make the man pages follow configured paths.

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#716369: fixed upstream

[Zbigniew Jędrzejewski-Szmek]

http://cgit.freedesktop.org/systemd/systemd-ui/commit/?id=7245ad7

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#714202: [Pkg-systemd-maintainers] Bug#714202: puppet: [PATCH] improve systemd packaging, use dh-systemd

[Zbigniew Jędrzejewski-Szmek]

On Wed, Jun 26, 2013 at 08:59:38PM +0200, Michael Stapelberg wrote:
 diff --git a/debian/puppet.service b/debian/puppet.service
 index a168128..1607745 100644
 --- a/debian/puppet.service
 +++ b/debian/puppet.service
 @@ -6,7 +6,6 @@ After=basic.target
  [Service]
  Type=forking
  PIDFile=/run/puppet/agent.pid
 -ExecStartPre=/usr/bin/install -d -o puppet -m 755 /run/puppet
  ExecStart=/usr/bin/puppet agent
  
  [Install]
 diff --git a/debian/puppet.tmpfile b/debian/puppet.tmpfile
 new file mode 100644
 index 000..867cb08
 --- /dev/null
 +++ b/debian/puppet.tmpfile
 @@ -0,0 +1 @@
 +d /var/run/puppet 0755 puppet root - -

Hm, why the backwards move to /var/run?

 diff --git a/debian/puppet.service b/debian/puppet.service
 index 1607745..f239c4d 100644
 --- a/debian/puppet.service
 +++ b/debian/puppet.service
 @@ -5,7 +5,7 @@ After=basic.target
  
  [Service]
  Type=forking
 -PIDFile=/run/puppet/agent.pid
 +PIDFile=/var/run/puppet/agent.pid
  ExecStart=/usr/bin/puppet agent

Zbyszek


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org