Re: [PATCH] build: Distribute tarball compressed with xz instead of bzip2
Hi! On Tue, 6 Nov 2018 02:47:53 +0100, Guillem Jover wrote: > * Makefile (dist): Change bz2 to xz. > (%.xz): Add target. > (%.bz2): Remove target. Curious: why a) use "xz" instead of what we got before, b) move from "bz2" to "xz" instead of offering both (like we continue to do for "gz")? Grüße Thomas > --- > Makefile | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/Makefile b/Makefile > index c0aa59a0e..6288a1573 100644 > --- a/Makefile > +++ b/Makefile > @@ -83,7 +83,7 @@ dist-version := $(shell cd $(top_srcdir)/ && > $(git_describe)) > > .PHONY: dist > ifdef configured > -dist: $(foreach Z,bz2 gz,$(dist-version).tar.$(Z)) > +dist: $(foreach Z,xz gz,$(dist-version).tar.$(Z)) > else > dist: > @echo >&2 'Cannot build a distribution from an unconfigured tree.' > @@ -238,8 +238,8 @@ install-headers: $(addsuffix > -install-headers,$(lib-subdirs) \ > TAGS: $(addsuffix -TAGS,$(working-prog-subdirs) $(lib-subdirs)) > etags -o $@ $(patsubst %-TAGS,-i %/TAGS,$^) > > -%.bz2: % > - bzip2 -9 < $< > $@ > +%.xz: % > + xz < $< > $@ > > %.gz: % > gzip -9n < $< > $@ > -- > 2.19.1.1182.g4ecb1133ce
Re: proxy-defpager for all users?
On Mon, Nov 05, 2018 at 11:02:47PM +0100, Samuel Thibault wrote: > proxy-defpager is typically set on /servers/default-pager, but its > permissions are by default 644, which makes it unusable by normal users, > it'd need to be 755 (see the x check in the defpager source). > > Apart from allowing users to eat memory, which they currently already > can do anyway, is there any downside to making this 755 so people can > mount their own tmpfs? I personally wondered why it wasn't the case from the start. That being said, I'll use this as an opportunity to restate a core problem of Mach memory management, as I couldn't find it on the wiki. This problem may or may not be even more triggered by using unprivileged tmpfs instances. Thu Dec 29 2016 : < braunr> i've identified a fundamental flaw with the default pager < braunr> and actually, with mach in general i suppose < braunr> i assumed that it was necessary to trust the server only < braunr> that a server didn't need to trust its client < braunr> but mach messages carry memory that is potentially mapped from unprivileged pagers < braunr> which means faulting on that memory effectively makes the faulting process a client to the unprivileged pager < braunr> and that's something that can happen to the default pager during heavy memory pressure < braunr> in which case it deadlocks on itself because the copyout hangs on a fault, waiting for the unprivileged pager to provide the data < braunr> (which it can't because of heavy memory pressure and because it's unprivileged, it's blocked, waiting until allocations resume) < braunr> the pageout daemon will keep paging out to the default pager in the hope those pages get freed < braunr> but sending to the default pager is now impossible because its map is locked on the never-ending fault -- Richard Braun
Re: proxy-defpager for all users?
Richard Braun, le mar. 06 nov. 2018 11:52:45 +0100, a ecrit:
> On Mon, Nov 05, 2018 at 11:02:47PM +0100, Samuel Thibault wrote:
> > proxy-defpager is typically set on /servers/default-pager, but its
> > permissions are by default 644, which makes it unusable by normal users,
> > it'd need to be 755 (see the x check in the defpager source).
> >
> > Apart from allowing users to eat memory, which they currently already
> > can do anyway, is there any downside to making this 755 so people can
> > mount their own tmpfs?
>
> I personally wondered why it wasn't the case from the start.
>
> That being said, I'll use this as an opportunity to restate a core
> problem of Mach memory management, as I couldn't find it on the wiki.
> This problem may or may not be even more triggered by using unprivileged
> tmpfs instances.
I actually encountered such an issue with php7.3's shm testing, which
triggers an ext2fs crash due to a tmpfs issue. I'll probably add the
attached patch to the debian package for now, but it seems that the
server side of RPCs needs to be more careful about receiving data when
it's passed out of line.
Samuel
Yes, the pointer provided by the caller, coming from the RPC buffer, may not
actually be safe to dereference. Try this with /run/shm as tmpfs with the crash server configured to dump cores:
#include
#include
#include
#include
#include
#define name "/run/shm/test.txt"
int main(void) {
int fd = open(name, O_RDWR|O_CREAT, 0777);
if (ftruncate(fd, 4096))
perror("fruncate");
char *c = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
if (c == MAP_FAILED)
perror("mmap");
if (close(fd))
perror("close");
if (unlink(name))
perror("unlink");
memset(c, 0, 4096);
if (munmap(c, 4096))
perror("munmap");
return 0;
}
It will make *ext2fs* crash, because
- removing a file from tmpfs make its memory object go away, thus making *c
unwritable (it's not the bug at stake, the program here is meant to crash)
- the crash server uses vm_read to read the process memory to write the
core. GNU Mach achieves it by playing with virtual memory.
- the crash server uses vm_write to write this to the FS. GNU Mach passes the
RPC data out of line by playing with virtual memory.
- ext2fs eventually tries to copy from the RPC data, assumed to be safe, to the
memory object, here backed by the pager. But the data is actually not safe.
That probably needs to be fixed at the mig layer, to make sure incoming
out-of-line data is accessible before handing it to the routine?
Index: hurd-debian/libpager/pager-memcpy.c
===
--- hurd-debian.orig/libpager/pager-memcpy.c
+++ hurd-debian/libpager/pager-memcpy.c
@@ -124,11 +124,14 @@ pager_memcpy (struct pager *pager, memor
__sync_synchronize();
if (prot == VM_PROT_READ)
- memcpy (other, (const void *) window + pageoff, copy_count);
+ err = hurd_safe_copyout (other, (const void *) window + pageoff, copy_count);
else
- memcpy ((void *) window + pageoff, other, copy_count);
+ err = hurd_safe_copyin ((void *) window + pageoff, other, copy_count);
vm_deallocate (mach_task_self (), window, window_size);
+ if (err)
+ return err;
+
offset += copy_count;
other += copy_count;
to_copy -= copy_count;
Re: [PATCH] build: Distribute tarball compressed with xz instead of bzip2
Hi! On Tue, 2018-11-06 at 10:47:02 +0100, Thomas Schwinge wrote: > On Tue, 6 Nov 2018 02:47:53 +0100, Guillem Jover wrote: > > * Makefile (dist): Change bz2 to xz. > > (%.xz): Add target. > > (%.bz2): Remove target. > > Curious: why a) use "xz" instead of what we got before, b) move from > "bz2" to "xz" instead of offering both (like we continue to do for "gz")? Sorry, should have added some rationale, thought it was obvious! :) xz compresses much better than gzip and bzip2. gzip is very universal in the availability sense, it's also pretty fast, and even though takes more space it uses less memory. bzip2 OTOH is not widely available, it's slow and takes more space, so it seems superfluous, and providing three compressed tarballs, seems like a waste of space on the server. :) Thanks, Guillem
Re: proxy-defpager for all users?
Richard Braun writes: > On Mon, Nov 05, 2018 at 11:02:47PM +0100, Samuel Thibault wrote: >> proxy-defpager is typically set on /servers/default-pager, but its >> permissions are by default 644, which makes it unusable by normal users, >> it'd need to be 755 (see the x check in the defpager source). >> >> Apart from allowing users to eat memory, which they currently already >> can do anyway, is there any downside to making this 755 so people can >> mount their own tmpfs? > > I personally wondered why it wasn't the case from the start. > > That being said, I'll use this as an opportunity to restate a core > problem of Mach memory management, as I couldn't find it on the wiki. > This problem may or may not be even more triggered by using unprivileged > tmpfs instances. I'll add this information to the wiki in a few days. > > Thu Dec 29 2016 : > < braunr> i've identified a fundamental flaw with the default pager > < braunr> and actually, with mach in general i suppose > < braunr> i assumed that it was necessary to trust the server only > < braunr> that a server didn't need to trust its client > < braunr> but mach messages carry memory that is potentially mapped from > unprivileged pagers > < braunr> which means faulting on that memory effectively makes the faulting > process a client to the unprivileged pager > < braunr> and that's something that can happen to the default pager during > heavy memory pressure > < braunr> in which case it deadlocks on itself because the copyout hangs on a > fault, waiting for the unprivileged pager to provide the data > < braunr> (which it can't because of heavy memory pressure and because it's > unprivileged, it's blocked, waiting until allocations resume) > < braunr> the pageout daemon will keep paging out to the default pager in the > hope those pages get freed > < braunr> but sending to the default pager is now impossible because its map > is locked on the never-ending fault
Re: proxy-defpager for all users?
Richard Braun, le mar. 06 nov. 2018 11:52:45 +0100, a ecrit: > On Mon, Nov 05, 2018 at 11:02:47PM +0100, Samuel Thibault wrote: > > proxy-defpager is typically set on /servers/default-pager, but its > > permissions are by default 644, which makes it unusable by normal users, > > it'd need to be 755 (see the x check in the defpager source). > > > > Apart from allowing users to eat memory, which they currently already > > can do anyway, is there any downside to making this 755 so people can > > mount their own tmpfs? > > I personally wondered why it wasn't the case from the start. Most probably because we didn't realize we had to make it +x to make it usable by users. Samuel
