Re: two build system issues
Quoting Samuel Thibault (2014-04-05 23:42:23) Justus Winter, le Sat 05 Apr 2014 20:16:12 +0200, a écrit : buildbot@pluto:/home/pluto/hurd/build/build-deb$ make -C libdiskfs || echo $? 2 Yes, that's most helpful. Not. I've seen that too. It happens when the dep files have a non-existing entry. That's what I said :p I usually use make -d to see what is missing. Really? I use make --debug, -d is to verbose to me for being useful at all. Justus
[PATCH] libports: fix notify_port_t receiver lookups
* libports/Makefile (MIGSFLAGS): Include mig-mutate.h.
* libports/mig-decls.h: New file.
* libports/mig-mutate.h: Likewise.
* libports/notify-dead-name.c: Fix receiver lookups.
* libports/notify-no-senders.c: Likewise.
* libports/notify-msg-accepted.c: Adjust function declaration.
* libports/notify-port-deleted.c: Likewise.
* libports/notify-port-destroyed.c: Likewise.
* libports/notify-send-once.c: Likewise.
* libports/ports.h: Likewise.
* proc/Makefile (MIGSFLAGS): Include mig-mutate.h, move PROCESS mutators...
* proc/mig-mutate.h: ... into a new file, add NOTIFY mutators.
* proc/notify.c: Fix receiver lookups, adjust function declarations.
* term/devio.c (ports_do_mach_notify_send_once): Adjust accordingly.
---
libports/Makefile| 1 +
libports/mig-decls.h | 40
libports/mig-mutate.h| 25 +
libports/notify-dead-name.c | 5 ++---
libports/notify-msg-accepted.c | 3 ++-
libports/notify-no-senders.c | 5 ++---
libports/notify-port-deleted.c | 3 ++-
libports/notify-port-destroyed.c | 3 ++-
libports/notify-send-once.c | 2 +-
libports/ports.h | 20 ++--
proc/Makefile| 4 +---
proc/mig-mutate.h| 33 +
proc/notify.c| 24
term/devio.c | 6 +++---
14 files changed, 140 insertions(+), 34 deletions(-)
create mode 100644 libports/mig-decls.h
create mode 100644 libports/mig-mutate.h
create mode 100644 proc/mig-mutate.h
diff --git a/libports/Makefile b/libports/Makefile
index 767ee73..30da1c1 100644
--- a/libports/Makefile
+++ b/libports/Makefile
@@ -45,5 +45,6 @@ LDLIBS += -lpthread
OBJS = $(SRCS:.c=.o) notifyServer.o interruptServer.o
MIGCOMSFLAGS = -prefix ports_
+MIGSFLAGS = -imacros $(srcdir)/mig-mutate.h
include ../Makeconf
diff --git a/libports/mig-decls.h b/libports/mig-decls.h
new file mode 100644
index 000..f8c4f15
--- /dev/null
+++ b/libports/mig-decls.h
@@ -0,0 +1,40 @@
+/*
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ Written by Justus Winter.
+
+ This file is part of the GNU Hurd.
+
+ The GNU Hurd is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2, or (at
+ your option) any later version.
+
+ The GNU Hurd is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with the GNU Hurd. If not, see http://www.gnu.org/licenses/. */
+
+#ifndef __LIBPORTS_MIG_DECLS_H__
+#define __LIBPORTS_MIG_DECLS_H__
+
+#include ports.h
+
+/* Called by server stub functions. */
+
+static inline struct port_info * __attribute__ ((unused))
+begin_using_port_info_port (mach_port_t port)
+{
+ return ports_lookup_port (0, port, 0);
+}
+
+static inline void __attribute__ ((unused))
+end_using_port_info (struct port_info *p)
+{
+ if (p)
+ports_port_deref (p);
+}
+
+#endif /* __LIBPORTS_MIG_DECLS_H__ */
diff --git a/libports/mig-mutate.h b/libports/mig-mutate.h
new file mode 100644
index 000..2eca719
--- /dev/null
+++ b/libports/mig-mutate.h
@@ -0,0 +1,25 @@
+/*
+ Copyright (C) 2014 Free Software Foundation, Inc.
+ Written by Justus Winter.
+
+ This file is part of the GNU Hurd.
+
+ The GNU Hurd is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2, or (at
+ your option) any later version.
+
+ The GNU Hurd is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with the GNU Hurd. If not, see http://www.gnu.org/licenses/. */
+
+#define NOTIFY_INTRAN \
+ port_info_t begin_using_port_info_port (mach_port_t)
+#define NOTIFY_DESTRUCTOR \
+ end_using_port_info (port_info_t)
+#define NOTIFY_IMPORTS \
+ import mig-decls.h;
diff --git a/libports/notify-dead-name.c b/libports/notify-dead-name.c
index c67145d..f974e33 100644
--- a/libports/notify-dead-name.c
+++ b/libports/notify-dead-name.c
@@ -22,13 +22,12 @@
#include notify_S.h
error_t
-ports_do_mach_notify_dead_name (mach_port_t notify, mach_port_t dead_name)
+ports_do_mach_notify_dead_name (struct port_info *pi,
+ mach_port_t
innocent-looking patch that breaks oot-builds
Hi, here is the innocent-looking patch that breaks out of tree builds. Any thoughts how to fix this? Justus
Re: innocent-looking patch that breaks oot-builds
Justus Winter 4win...@informatik.uni-hamburg.de writes: Hi, here is the innocent-looking patch that breaks out of tree builds. Specifically? Could you please provide some excerpts of the build logs, highlighting the failure? Any thoughts how to fix this? For a random guess, the -imacros MiG option’s implementation may have to be checked. (An issue with relative filenames handling, perhaps?) -- FSF associate member #7257
Re: two build system issues
Justus Winter 4win...@informatik.uni-hamburg.de writes: The second issue is due to me doing an out-of-tree build. That it currently works is more or less by chance I fear. I just came up with a (legit in my opinion) patch that breaks oot builds. So what happens here is that demuxer.c includes ../libports/notify_S.h, which is a mig-generated file: build/build-deb/libdiskfs$ ls ../libports/notify_S.h ../libports/notify_S.h And this file includes mig-decls.h (that's the change I made that breaks the build), because that file is in the source tree: build/build-deb/libdiskfs$ ls ../libports/mig-decls.h ls: cannot access ../libports/mig-decls.h: No such file or directory build/build-deb/libdiskfs$ ls ../../libports/mig-decls.h ../../libports/mig-decls.h :/ Any help? Well, shouldn’t CPPFLAGS (or rather AM_CPPFLAGS, when Automake is used) include -I$(srcdir) whenever any header files from the source directory are to be included? -- FSF associate member #7257
Re: two build system issues
Quoting Ivan Shmakov (2014-04-06 11:15:06) Justus Winter 4win...@informatik.uni-hamburg.de writes: The second issue is due to me doing an out-of-tree build. That it currently works is more or less by chance I fear. I just came up with a (legit in my opinion) patch that breaks oot builds. So what happens here is that demuxer.c includes ../libports/notify_S.h, which is a mig-generated file: build/build-deb/libdiskfs$ ls ../libports/notify_S.h ../libports/notify_S.h And this file includes mig-decls.h (that's the change I made that breaks the build), because that file is in the source tree: build/build-deb/libdiskfs$ ls ../libports/mig-decls.h ls: cannot access ../libports/mig-decls.h: No such file or directory build/build-deb/libdiskfs$ ls ../../libports/mig-decls.h ../../libports/mig-decls.h :/ Any help? Well, shouldn’t CPPFLAGS (or rather AM_CPPFLAGS, when Automake is used) include -I$(srcdir) whenever any header files from the source directory are to be included? Yes, you are right, it should and it does :) So the solution is in fact to qualify the include with the subdirectory. So instead of doing #define NOTIFY_IMPORTS \ import mig-decls.h; it should read #define NOTIFY_IMPORTS \ import libports/mig-decls.h; Sweet, thanks :) Justus
RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
Hi, I'd like to see more of the debian/patches merged, especially the exec_filename_* series. These patches cover a lot of files, and they often break and I have to amend them manually. More often than not the fix is trivial, but it is still a nuisance. I'd also like to see the dde stuff from the incubator repository merged upstream. 1/ Code that does not live in the Hurd repository receives less attention. I've been doing a lot of cleanups, static analysis and performance improvements lately. The dde stuff for example has not received any of those. 2/ The nice Guix folks surely also want all those goodies, so they either have to copy those patches, or they miss out. In either way, I feel like we (Hurd upstream) would have failed them. 3/ I feel that this situation creates an unnecessary drag for the development. I'm working on protected payloads to improve the object lookups in the Hurd servers. I've just came up with a patch that fixes some object lookups in libports. This requires changing the API of libports. I fixed all the code in the Hurd repository accordingly, unfortunately the dde stuff breaks if it isn't fixed as well. If it was living in the same repository, I could fix it up in one commit, making sure that the change is atomic. So, is the dde stuff a first class citizen? If so, I believe it should be merged. If not, may I change the API of, say, libports freely? May I merge that patch that breaks the dde servers? If not, why not, and what is the best way to proceed? Cheers, Justus
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
Hello, Justus Winter, le Sun 06 Apr 2014 19:58:45 +0200, a écrit : I'd like to see more of the debian/patches merged, especially the exec_filename_* series. The discussion about it with Roland was unfortunately not finished. I'd also like to see the dde stuff from the incubator repository merged upstream. For this one, I'd like to avoid having to let userland processes disable interrupts, since this brings IRQ sharing issues. may I change the API of, say, libports freely? Well, there is not only dde, but all incubator projects, as well as hurdextras, etc. what is the best way to proceed? You can commit the dde-related changes in the incubator at the same time, both will be pulled at the same time in Debian updates, for instance. Samuel
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
Samuel Thibault, le Sun 06 Apr 2014 21:25:42 +0200, a écrit : I'd also like to see the dde stuff from the incubator repository merged upstream. For this one, I'd like to avoid having to let userland processes disable interrupts, since this brings IRQ sharing issues. (and the stack needs to be polished, for now it's quite drafty in the corners). Samuel
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
As I said in the irc it would be really helpfull if the debian patches got merged with the upstream. Right now whenever I get an error, I am searching in them, handpicking the part that solves it. Actually some of the debian patches are necessary to built glibc and libpthread. I will report exactly which parts I needed from which patches, as soon as possible, so I can help Justus merge them. Manolis
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
Hi, Quoting Samuel Thibault (2014-04-06 21:25:42) Justus Winter, le Sun 06 Apr 2014 19:58:45 +0200, a écrit : I'd like to see more of the debian/patches merged, especially the exec_filename_* series. The discussion about it with Roland was unfortunately not finished. Please enlighten me. I wasn't around when this was discussed, and I tried to find the discussion but failed. Meanwhile, I improved the exec_filename_* series and fixed the fakeroot translator. The fakeroot translator is somewhat less useful w/o that patch series, so I'd like to see this merged. I'd also like to see the dde stuff from the incubator repository merged upstream. For this one, I'd like to avoid having to let userland processes disable interrupts, since this brings IRQ sharing issues. Why does that prevent the merge? We have plenty of stuff in the repo that doesn't work, bitrots or is otherwise in a bad shape. If someone doesn't like the idea of letting userspace processes disable interrupts, let her just not use the dde drivers. may I change the API of, say, libports freely? Well, there is not only dde, but all incubator projects, as well as hurdextras, etc. what is the best way to proceed? You can commit the dde-related changes in the incubator at the same time, both will be pulled at the same time in Debian updates, for instance. How exactly is it that it is pulled in the Debian updates? Manually by you, I presume? There's one thing that really bugs me. I thought I stated this before, or even in the original mail, but maybe it didn't came across clearly. For me, building the Hurd is like driving rusty nails into ones knee, only less fun. It has caused me so much pain, that I created a continuous integration solution that builds Debian packages. ci is nice and stuff, but ultimately I just wanted to relieve the pain. Why is it painful? Because Debian/Hurd is patched beyond recognition. Clone the Hurd repository, build some component, say one of the core servers, put it into your Debian installation and voila, your system hangs at boot time. So you clone the Debian/Hurd packaging repo, which is a weird frankenstein-repo. The upstream repo is mixed with parts of the dde branch from the incubator, on top of that lives a long list of debian/patches, some of them also carry dde stuff in them. You patch some stuff, try to build it, and surely enough you need to refresh some debian/patches. See here for the difference of the Debian packaging repo and the hurd upstream repo: http://darnassus.sceen.net/gitweb/teythoon/packaging/hurd.git/tree So I got myself a solution for this problem. I automated every step that can possibly be automated, but I still have to amend the patches in there often, like this: http://darnassus.sceen.net/gitweb/teythoon/packaging/hurd.git/commitdiff/fe9c72e60a59109bff7558224871ac5e80b9f38d Building half a Hurd package takes ~30 minutes using my solution. My workflow goes roughly like this: 1. come up with a patch against hurd upstream 2. add it to the patch series in my debian packaging repo 3. start a build 4. see my patch break b/c some other debian/patch also touches some file that my new patch touches 5. amend either mine or the other patch 6. goto 3 I do this for every patch series I propose. Once the package is built, it is uploaded to my deb repo and my test vm automatically installs the new packages and verifies that the system still works. The amount of time I still spend on just building debian packages is mind-boggling. If I wasn't the masochist I am, I would have stopped doing that a long time ago. Also, the way it is now it will never be possible to fully-automatically build Debian packages from the Hurd repository. Today it got worse. The package built fine, but surely enough all the dde components broke apart at runtime b/c I changed the API of libports. The dde components that either are in the debian package repo or in the debian/patches in the debian package repo, I don't even know for sure. Sure, I can fix those components up in the incubator repo. It was mostly an awareness problem. But then what? How exactly are they then pulled into my build solution? How am I supposed to test that I didn't break anything? If I figure out how you create the patches from the incubator repo, I could do that too. In the end, we still end up duplicating the packaging effort. By the way, you'll need fe9c72e60a59 too for the next Hurd package. This has caused me so much pain. And I imagine it is even worse for new developers. Justus
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
On 07/04/14 00:26, Justus Winter wrote: Hi, Quoting Samuel Thibault (2014-04-06 21:25:42) Justus Winter, le Sun 06 Apr 2014 19:58:45 +0200, a écrit : I'd like to see more of the debian/patches merged, especially the exec_filename_* series. The discussion about it with Roland was unfortunately not finished. Please enlighten me. I wasn't around when this was discussed, and I tried to find the discussion but failed. Meanwhile, I improved the exec_filename_* series and fixed the fakeroot translator. The fakeroot translator is somewhat less useful w/o that patch series, so I'd like to see this merged. See http://marc.info/?l=hurd-bugm=127543467330197 and the rest of that thread. Regards, Emilio
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
Quoting Emilio Pozuelo Monfort (2014-04-07 00:38:28) On 07/04/14 00:26, Justus Winter wrote: Hi, Quoting Samuel Thibault (2014-04-06 21:25:42) Justus Winter, le Sun 06 Apr 2014 19:58:45 +0200, a écrit : I'd like to see more of the debian/patches merged, especially the exec_filename_* series. The discussion about it with Roland was unfortunately not finished. Please enlighten me. I wasn't around when this was discussed, and I tried to find the discussion but failed. Meanwhile, I improved the exec_filename_* series and fixed the fakeroot translator. The fakeroot translator is somewhat less useful w/o that patch series, so I'd like to see this merged. See http://marc.info/?l=hurd-bugm=127543467330197 and the rest of that thread. Seriously? Emilio, Jeremy, and Samuel think it's a worthwile addition, but Roland just isn't convinced it's worth doing? That has stalled the inclusion of this for three years? Well, if there is a reason to prolong my pain, this must be it. Justus
Upstreaming patches [Was: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff]
Justus Winter, le Mon 07 Apr 2014 00:26:29 +0200, a écrit : The discussion about it with Roland was unfortunately not finished. Please enlighten me. I wasn't around when this was discussed, Well, I wasn't really into the discussion either actually. Pochu did the work. See his mails called like Add a file_exec_file_name RPC around Wed, 26 May 2010. I'd also like to see the dde stuff from the incubator repository merged upstream. For this one, I'd like to avoid having to let userland processes disable interrupts, since this brings IRQ sharing issues. Why does that prevent the merge? Because that's a quite important incompatible detail in the behavior of the introduced gnumach IRQ RPC. We could perhaps have a look at having a backward-compatibility way, so we can just commit what we have, and change it later on, it's a matter of having a look at it. You can commit the dde-related changes in the incubator at the same time, both will be pulled at the same time in Debian updates, for instance. How exactly is it that it is pulled in the Debian updates? Manually by you, I presume? Just the same way as pulling the main branch. There's a README file in the README branch of the repository. This has caused me so much pain. And I imagine it is even worse for new developers. I fully understand all that, it's not fun for me either, and I'd really love to find the time to fix all that. But for now apparently only I have taken the time to dive into Zheng Da's work on DDE, so it doesn't help. We could as well just commit what is there since it's relatively split appart in the source, so it at least doesn't put ugly hacks into other directories. But it really needs some polishing in the end. As for the two other stuff in the Debian hurd package (random, procfs), yes it is a ugly hack, and I'd rather avoid it. It was just a way to get working /dev/random and /proc so people can ran a debian system at all. Of course we should think about just merging them into the main repository, or build them another way, etc. I for myself have never found the time to even just think about it. I end up spending my time mostly fixing pushing more-or-less-baked patches to Debian packages, so people at least get to try them, but the polishing+submitting work is much more work, and if nobody gives help there, well things can only go slowly for sure. There's a question of priority. For instance, should have I spent my time the other day on DDE / exec_filename / pushing some glibc patches upstream, or on fixing the pthread_condattr_setclock() so glib2.0 can continue working at all? Maybe I should just not do this kind of ugly pushing, so Debian doesn't get them, and thus people would have to really push their changes in properly, and not be happy with just seeing them applied to Debian. In the end, I'd essentially say Patch Thankfully Welcome, like they do so often on the cygwin mailing list. Of course, that means baked patches, which is quite some work, but the Hurd project is no exception to this requirement. If some discussion died just because a patch was not reviewed, then it's a matter of reviving it. I happen to have dozens of mails in my hurd inbox, I've just never found the time to revive their discussion. If nobody else helps on that, it'll continue getting dust. Put another way, please people just dive into patches, check why they are not upstream (just ask if it's not written in there), and have a look at how to fix that. I don't think there is anything strongly blocking anywhere, it's just a matter of doing things, so that things get done... Samuel
Re: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff
Justus Winter, le Mon 07 Apr 2014 00:53:40 +0200, a écrit : Quoting Emilio Pozuelo Monfort (2014-04-07 00:38:28) On 07/04/14 00:26, Justus Winter wrote: Hi, Quoting Samuel Thibault (2014-04-06 21:25:42) Justus Winter, le Sun 06 Apr 2014 19:58:45 +0200, a écrit : I'd like to see more of the debian/patches merged, especially the exec_filename_* series. The discussion about it with Roland was unfortunately not finished. Please enlighten me. I wasn't around when this was discussed, and I tried to find the discussion but failed. Meanwhile, I improved the exec_filename_* series and fixed the fakeroot translator. The fakeroot translator is somewhat less useful w/o that patch series, so I'd like to see this merged. See http://marc.info/?l=hurd-bugm=127543467330197 and the rest of that thread. Seriously? Emilio, Jeremy, and Samuel think it's a worthwile addition, but Roland just isn't convinced it's worth doing? That has stalled the inclusion of this for three years? Nobody took the time to discuss with Roland, indeed. Samuel
Re: Upstreaming patches [Was: RFC: upstreaming debian/patches/exec_filename_* and the dde stuff]
Samuel Thibault, le Mon 07 Apr 2014 01:02:56 +0200, a écrit : Put another way, please people just dive into patches, Put yet another way, if something is eating time in your workflow for no apparently good reason, there was probably no good reason except lack of time for doing things better, so just spend time on saving that time, and everybody will thank you. Samuel
