Re: [gentoo-user] Re: distfiles contains extra files?
On 13/08/2014 23:48, James wrote: > Alan McKinnon gmail.com> writes: > > > > maybe I need to return to cleaning up distfiles/ by hand? >> eclean distfiles > > Yea, I very familiar with cleaning up distfiles, tools and manually > The tools never seems to be 100% effective before. They are very effective. There are only three main choices: 1. delete everything in distfiles that is no longer referenced in the tree 2. delete everything in distfiles referenced by entire packages no longer installed 3. delete everything in distfiles referenced by package versions no longer installed. There's a 4th one too: rm -rf and a few tweak options regarding size of files so you can delete all small one and keep all big ones etc etc -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Re: distfiles contains extra files?
On 13/08/2014 23:35, James wrote: > Neil Bothwick digimed.co.uk> writes: > >>> Now that I'm looking, it looks like a policy decision for the devs >>> to formally evaluate. /distfiles/ should not be a dir for garbage, >>> one-off-files and other such nonsense. > >> That's exactly what it is for. One-off files that the ebuild downloads, >> uses and then no longer needs. Nothing in $DISTDIR is needed by a running >> system. > > > Ok so anything needed for a build of a particular package goes into > /usr/portage/distfiles? yes > > I thought ebuilds use /var/tmp/portage for that. > If in needs to hang around longer (than a /tmp file typically > hangs out for, when not put it under the another logical place. no, /var/tmp/portage is the BUILDDIR, not the FETCHDIR. /var/distfiles is permanent so repeated emerges do not cause repeated fetches. The fetched sources are unpacked into /var/tmp/portage and built there, then the whole lot deleted after a successful merge > > Like I said I thought /distfiles/ contains compressed sources > and other file needed, all rolled into a common format, like*.bz2. > > It we start (continue) strowing files into /distfiles/ where does it end? > > (folks, it's a philosophical discussion so no need to denegrate into > crudedness), imho. > > > James > > > > > -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Anybody running gnome3 in a virtualbox gentoo guest?
On Wed, Aug 13, 2014 at 8:32 PM, walt wrote: > Because I'm paranoid, I update my virtualbox ~amd64-gentoo-guest machine every > day *before* I update my (real hardware) "production" machine. (The definition > of "production" is left as an exercise for the reader.) > > Not long ago I noticed that the vbox guest machine failed to start a gnome > session because "3D hardware support" is not available on the guest machine. > > To make a very long story short, I downgraded the xorg-server on the gentoo > guest from 1.16.0 to 1.15.1, and now gnome3 is happy again. > > Techie detail for nerds only: /usr/libexec/gnome-session-check-accelerated > returns an error message with xorg-server-1.16.0, but works normally with > xorg-server-1.15.1. > > Has anyone else seen the same behavior? I cannot recommend on VirtualBox (although, as some kernel developers have said, their modules are "crap"[1]), but I've succeeded in running GNOME 3 with Qemu using the qxl virtual video card. It's slow, but it works. Regards. [1] http://www.phoronix.com/scan.php?page=news_item&px=OTk5Mw -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] Anybody running gnome3 in a virtualbox gentoo guest?
On 08/14/2014 09:32 AM, walt wrote: > Because I'm paranoid, I update my virtualbox ~amd64-gentoo-guest machine every > day *before* I update my (real hardware) "production" machine. (The definition > of "production" is left as an exercise for the reader.) > > Not long ago I noticed that the vbox guest machine failed to start a gnome > session because "3D hardware support" is not available on the guest machine. > > To make a very long story short, I downgraded the xorg-server on the gentoo > guest from 1.16.0 to 1.15.1, and now gnome3 is happy again. > > Techie detail for nerds only: /usr/libexec/gnome-session-check-accelerated > returns an error message with xorg-server-1.16.0, but works normally with > xorg-server-1.15.1. > > Has anyone else seen the same behavior? > > Thx > > I saw similar problem running debian in virtualbox. Virtualbox guest addition is needed for 3D support, but the additions won't compile with the latest xorg-server.
[gentoo-user] Intermittent USB device failures
I've recently taken an old Windows XP system and rebuilt it to run Gentoo. Since then, I've been having issues using any type of USB input device (which is particularly bad, since it has no PS/2 input ports). After some indeterminate period of time, the input device simply stops responding. Typically, I can use the console for a few days at a time before the keyboard dies, but if I load up a GUI and start using the mouse it takes less than a few hours. I'm fairly sure this problem is USB-related, since I believe I've eliminated everything downstream of that: I've tried using both evdev and legacy mouse and keyboard drivers in both the kernel and X and all of them work the same way. evtest shows no activity from the device once it breaks, nor do the legacy /dev driver files. Most notably, removing and re-plugging the device doesn't register as a device attachment. If, for example, I take a working USB mouse and swap USB ports, I get this: [ 1219.418050] usb 2-8: USB disconnect, device number 3 [ 1225.258011] usb 2-7: new low-speed USB device number 4 using ohci-pci [ 1225.449627] usb 2-7: New USB device found, idVendor=046d, idProduct=c044 [ 1225.449946] usb 2-7: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 1225.450240] usb 2-7: Product: USB-PS/2 Optical Mouse [ 1225.450740] usb 2-7: Manufacturer: Logitech [ 1225.464165] input: Logitech USB-PS/2 Optical Mouse as /devices/pci:00/:00:0b.0/usb2/2-7/2-7:1.0/input/input6 [ 1225.464666] hid-generic 0003:046D:C044.0003: input,hidraw1: USB HID v1.10 Mouse [Logitech USB-PS/2 Optical Mouse] on usb-:00:0b.0-7/input0 If I do the same thing after the device has stopped functioning, I get the disconnect message but that's it: [199729.451060] i2c i2c-3: sendbytes: NAK bailout. [199733.215303] usb 2-7: USB disconnect, device number 4 [199814.495204] type=1006 audit(1394381639.494:3): pid=5861 uid=0 old auid=4294967295 new auid=1000 old ses=4294967295 new ses=2 res=1 (the i2c errors, I believe, are unrelated; they seem to be caused by the nouveau driver I'm using). I've gone through at least 6 different kernel versions and they all exhibit this same behavior. At this point I'm not even sure what else I can do to troubleshoot this problem, so any advice would be appreciated! Thanks, --Mike
[gentoo-user] Anybody running gnome3 in a virtualbox gentoo guest?
Because I'm paranoid, I update my virtualbox ~amd64-gentoo-guest machine every day *before* I update my (real hardware) "production" machine. (The definition of "production" is left as an exercise for the reader.) Not long ago I noticed that the vbox guest machine failed to start a gnome session because "3D hardware support" is not available on the guest machine. To make a very long story short, I downgraded the xorg-server on the gentoo guest from 1.16.0 to 1.15.1, and now gnome3 is happy again. Techie detail for nerds only: /usr/libexec/gnome-session-check-accelerated returns an error message with xorg-server-1.16.0, but works normally with xorg-server-1.15.1. Has anyone else seen the same behavior? Thx
[gentoo-user] Re: Kernel 3.14.14 build failure at DEPMOD stage
On 08/13/2014 03:51 PM, Walter Dnes wrote: > Going from 3.12.13 to 3.14.14 using "make old config" and then the > standard build (64 bit). At the DEPMOD stage near the very end I get... > > DEPMOD 3.14.14-gentoo > /usr/src/linux-3.14.14-gentoo/scripts/depmod.sh: line 57: 27721 > Segmentation fault "$DEPMOD" "$@" "$KERNELRELEASE" $SYMBOL_PREFIX > make: *** [_modinst_post] Error 139 I *hate* when that happens! Just a few random ideas that may help the old bulb light up :) Can you re-emerge 3.12.13 without any errors? The kernel build system uses perl (not sure about python), so I always wonder if perl-cleaner may help fix strange new errors. I'm guessing you're running gentoo-stable on that machine? Has gcc been updated recently? glibc? linux-headers? perl? python? kmod?
[gentoo-user] Kernel 3.14.14 build failure at DEPMOD stage
Going from 3.12.13 to 3.14.14 using "make old config" and then the standard build (64 bit). At the DEPMOD stage near the very end I get... DEPMOD 3.14.14-gentoo /usr/src/linux-3.14.14-gentoo/scripts/depmod.sh: line 57: 27721 Segmentation fault "$DEPMOD" "$@" "$KERNELRELEASE" $SYMBOL_PREFIX make: *** [_modinst_post] Error 139 I've backed out and eselected 3.12.13 again. Any ideas what gives? I can't find anything relevant on Google. Google search turns up stuff like so... 1) cross-compiling and using wrong host; I'm not cross-compiling here 2) out of disk space; nope, I've checked and have plenty of space 3) out of memory; on a Lenovo Thinkpad with 6 gigs of RAM? And yes, /usr/src/linux-3.14.14-gentoo/scripts/depmod.sh line 57 is... "$DEPMOD" "$@" "$KERNELRELEASE" $SYMBOL_PREFIX -- Walter Dnes I don't run "desktop environments"; I run useful applications
Re: [gentoo-user] Re: distfiles contains extra files?
On Wed, 13 Aug 2014 21:35:47 + (UTC), James wrote: > > That's exactly what it is for. One-off files that the ebuild > > downloads, uses and then no longer needs. Nothing in $DISTDIR is > > needed by a running system. > Ok so anything needed for a build of a particular package goes into > /usr/portage/distfiles? anything that needs to be downloaded, yes. > I thought ebuilds use /var/tmp/portage for that. No. PORTAGE_TMPDIR is the temporary directory use for building the software to install. > If in needs to hang around longer (than a /tmp file typically > hangs out for, when not put it under the another logical place. $DISTDIR is the logical place. It is where portage puts the files that projects use the distribute their work, hence the name. > Like I said I thought /distfiles/ contains compressed sources > and other file needed, all rolled into a common format, like*.bz2. It contains source files in whatever format they are supplied, which may or may not be compressed (think ebuilds that download from a VCS). source in this case means the source of the software, not necessarily source code as some ebuilds are for binary distributions. > It we start (continue) strowing files into /distfiles/ where does it > end? With a full hard drive, unless you maintain it, this is Gento after all. But other distros do a similar thing, Debian downloads all the .deb. files for an operation before it starts installing them, and keeps them in a cache. I really don't see the problem here. Portage has to download files, it has to save them somewhere, and we users may want to keep them for later re-use. Portage does all of that in a sane manner. The only thing that stops you from seeing this is the invalid assumption that $DISTDIR is only for compressed tarballs of source code. man make.conf describes what $DISTDIR, $PORTAGE_TMPDIR etc. are for, you cannot redefine that on a whim and then say portage is doing it wrong. -- Neil Bothwick To the optimist, the glass is half full. To the pessimist, the glass is half empty. To the engineer, the glass is twice as big as it needs to be. signature.asc Description: PGP signature
[gentoo-user] Re: distfiles contains extra files?
Alan McKinnon gmail.com> writes: > >>> maybe I need to return to cleaning up distfiles/ by hand? > eclean distfiles Yea, I very familiar with cleaning up distfiles, tools and manually The tools never seems to be 100% effective before. > > Maybe we can get systemd to clean this up? It's the new "daddy" > > for all sorts of poorly wrtten codes, so why not add cleaning up > > /distfiles/ to it's new fiefdom? > > just teasing(not really?).ha ha ha ha ah ha ha ha ha. > > Lennart can you help a bro out? > Lets not give Lennart extra ideas now, OK? It's humor, OK? Some folks don't like the munging of cgroups as they have functioned historically, so there is quite a bit if bitching. I do not think it is out of bounds to add a wee bit of humor into the entire spectacle surrounding cgroups. imho. thanks for the responses /end of thread. James
[gentoo-user] Re: distfiles contains extra files?
Neil Bothwick digimed.co.uk> writes: > > Now that I'm looking, it looks like a policy decision for the devs > > to formally evaluate. /distfiles/ should not be a dir for garbage, > > one-off-files and other such nonsense. > That's exactly what it is for. One-off files that the ebuild downloads, > uses and then no longer needs. Nothing in $DISTDIR is needed by a running > system. Ok so anything needed for a build of a particular package goes into /usr/portage/distfiles? I thought ebuilds use /var/tmp/portage for that. If in needs to hang around longer (than a /tmp file typically hangs out for, when not put it under the another logical place. Like I said I thought /distfiles/ contains compressed sources and other file needed, all rolled into a common format, like*.bz2. It we start (continue) strowing files into /distfiles/ where does it end? (folks, it's a philosophical discussion so no need to denegrate into crudedness), imho. James
[gentoo-user] Re: distfiles contains extra files?
Volker Armin Hemmann googlemail.com> writes: > Any other idiotic ideas? Hey, JUST GO FUCK OFF! If you cannot respondd in a cilized form, then PLEASE Just ingnore the post ASSHOLE! James
Re: [gentoo-user] Re: OT: pthreads condition variable/mutex question
On Wed 13 Aug 2014 04:19:19 PM EDT, Grant Edwards wrote: > On 2014-08-13, Alec Ten Harmsel wrote: > >>> I may have to stick with sockets when I want to block until some event >>> happens. >> >> To be clear, do you want to block or sleep/yield until an event >> happens? > > I don't see the difference -- isn't that what a blocking call does: > sleep/yield until some event happens? My bad, got a little confused. Trying to think a little too hard ;) Also, like Mr.McKinnon mentioned above, I'm a little out of my depth as well, although I have written some fancy multi-threaded computer vision code before. >> I'm sorry for not being too helpful. Just one last question: Can you >> describe what exactly your code is supposed to do, or is it something >> that you can't talk about because it's a work thing? I don't care >> either way, but I'm just curious because it seems you need to >> optimize quite a bit. > > One process implements a communications protocol that is maintaining > communications links with a handful of slave devices connected to > serial ports running at baud rates up to 230400 baud (38400 is the > most common). There are typically one (or maybe two) hundred messages > per second being exchanged with each slave device. I'll call that > process the server. Alright, following you so far. > There are other client processes that want to access the slaves and > the inforamation being received from them. Some of the clients just > want to do low-frequency transactions for configuration/diagnostic > purposes, and Unix domain sockets work fine for that. Seems legit. > Other clients may want to wake up every time a certain high frequency > event happens. That's where I'm trying to use condition variables. I think you should step away from the fancy decoupling and process the high frequency events in a separate thread in the server; if passing them over sockets is too much overhead, I don't see any other way to do this. I don't want to be extremely critical, but if the whole point of the server process is *only* the message passing, what does that gain you? Why bother only passing messages in the server? Might as well do some processing as well. > Other clients will periodically (typically once every 5-50 ms) want to > see the most recent copy of a particular received message. I'm > thinking about using shared memory and rwlocks for that, but I haven't > figured out how to deal with the case where a process aborts while > holding a lock. Assuming there's not too many different message types, you could cache the most recent of each type in a hash map in the server process, making retrieving the most recent copy just a query to the server and no problems with locking on the client side. In summation, I think you should process high-frequency messages in the server process, cache most recent messages in the server, and you can keep a separate client for checking the most recent messages and a separate client for the occasional debugging/configuration tasks. IIRC, I'm pretty sure that's what made nginx so much more performant than Apache initially; instead of launching a new thread for every request (Apache did/does this) or doing some sort of fancy message passing, nginx pushes requests into a queue in shared memory and a thread pool processes the queue. Alec
Re: [gentoo-user] Re: distfiles contains extra files?
On 13/08/2014 18:55, James wrote:
> Peter Humphrey prh.myzen.co.uk> writes:
>
>
>> On Wednesday 13 August 2014 15:07:19 James wrote:
>
>>> maybe I need to return to cleaning up distfiles/ by hand?
>
>> Yes, I see I have other things than .tar.bz2 too, now you mention it.
>
>> One of my boxes runs http-replicator to serve distfiles to the network. The
>> clean-up script I run after upgrades includes this line:
>> find /var/cache/http-replicator -ctime +182 -exec rm '-v' {} +
>
>> Also, have you tried eclean-dist, from app-portage/gentoolkit?
>
>
> I'm not sure why, I figured I did not have to manually clean up
> /distfiles/ any more. for a period of time I could sware that
> it was a clean repository for compressed/tar source files only.
>
> Now that I'm looking, it looks like a policy decision for the devs
> to formally evaluate. /distfiles/ should not be a dir for garbage,
> one-off-files and other such nonsense. It was (circa 2004 for me)
> a repository for compressed sources.
>
>
> Maybe we can get systemd to clean this up? It's the new "daddy"
> for all sorts of poorly wrtten codes, so why not add cleaning up
> /distfiles/ to it's new fiefdom?
eclean distfiles
Lets not give Lennart extra ideas now, OK?
>
> just teasing(not really?).ha ha ha ha ah ha ha ha ha.
> Lennart can you help a bro out?
>
>
> ;-)
> James
>
>
>
>
>
>
>
--
Alan McKinnon
alan.mckin...@gmail.com
Re: [gentoo-user] Re: OT: pthreads condition variable/mutex question
On 13/08/2014 22:28, Grant Edwards wrote: >> Have you considered a really simple solution like dbus? > I don't know if I would call dbus "really simple". :) > > My current implementation uses Unix domain sockets (which is what dbus > usually uses, isn't it?), and I'm trying to figure out how to reduce > overhead and latency. dbus would add even more overhead (it has code > to deal with byte ordering, serial/cookies, and various other features > and abstractions). I'm not sure it's really practical for > high-frequency events (e.g 100-200 events per second). To be honest, I'm somewhat out of my depth in this area. Sure, I can discuss it but you want details, and that I can't really provide. I mentioned dbus really just as a way to encourage you to think out of the box a little. Maybe it fits your needs, maybe not. But at least you would have given it some thought :-) Reading through the rest of the thread, perhaps dbus isn't suitable for this. You have lots of messages and they don't seem to be text-based. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Re: distfiles contains extra files?
On 13/08/2014 18:46, James wrote: > Neil Bothwick digimed.co.uk> writes: > > >>> I previously thought that /usr/portage/distfiles >>> only contains tar files. I have not clean up the >>> system, as I'm moving (dupicating some files for my >>> /usr/local/ needs. > >> It contains everything downloaded by ebuilds. > > I guess our ebuilds are getting creative on what they allow. ebuilds can do whatever they feel like doing, and there's no sane exact policy on how to deal with the various sources they use (the word sources used very loosely here). The general idea is "small files go in the tree, bigger files are downloaded to distfiles when the ebuild is emerged". Obviouslym, there's no policy on what constitutes small and large > >>> I thought all patch files where kept in the subdirs where the >>> ebuilds are located (files dir)? It is still where most *.patch >>> files are located. True, most patches are small. But there are 36,000 or so packages, what if every one of them had 3 patches? That's 100,000 inodes right there and every gentoo user must have all of them all of the time and they must all be checked somehow with every --sync, and most user will only use a small fraction of them. It makes more sense to download larger patches when the ebuild is emerged. Yes, it makes fetch() take a few seconds longer, but many other things are gained >> ISTR the policy is to only include smaller patch files in the tree, large >> patches are downloaded to avoid everyone having to sync them. > > OK, but wny not put them in a subdir under the specific ebuild > requiring those smaller patches? Seems to be a dumb move to me. See above. > >>> *.exe (like verdan32.exe webdin32.exe ) >> These are the installers for the MS corefonts. > > OK, why they need to be in /distfiles/? seriously .exe > files in a repository for sources? ARE YOU KIDDING ME? > Let's just ignor the /bin/sbin and name it /sbinge > as devs must be 'binge drinking' if this is our standard? Consider what those .exe files are - they are the "sources" for whatever the ebuild needs to do, and they do not have to be .tar.gz files either. Most .exes in distfiles are things like microsoft corefonts. Now these are redistibutable, but only if the sources are not modified. the sources are what microsoft published - a .exe file. The ebuild downloads them, unpacks them, finds the .ttfs and installs them. The fact that the packaging method used in this case is a .exe is completely irrelevant - that is how upstream provides the sources and ebuilds *always* store files fetch()ed to distfiles > > /bin/ms/? /usr/local/bin/ > >>> I guess I need "enlightenment" ? >> You can use whichever DE you prefer :) > > U very funny! I use 'enlightenment' compliments to the > Bhagavad Gita. I actually has to read the English version > of the entire book and write a paper on it, in only 3 days. > Academically speaking I got 'enlightenment' from that > literarture class...ymmv. > >>> cleaning up distfiles/ by hand? > >> You can use eclean to remove obsolete distfiles and packages. But don't >> use eclean on a shared $DISTDIR. > > Yes, agreed. But the point is this sort of stuff belongs in the file > sub-tree under the ebuild that requires it, not as part of /distfiles/ > imho. From my perspective strowing random files into /distfiles/ > puts it in the category of "cruft". Nope. Considering the size and extent of the tree: $ find /var/portage -type f | wc 153047 153047 8404011 $ find /var/portage -type f -size -1024c | wc 75350 75350 4304282 $ find /var/portage -type d | wc 25920 25920 928148 Holy smoke Batman! One hundred and fifty three thousand files! Half of them are smaller than one kilobyte! I doubt you have more than 5000 or so patch files total for everything installed, and that's being generous. Why download and sync all of that with every sync? Makes more sense to just fetch them when you need them and shove them in distfiles. Remember the huge efforts a few years back to deal with the unique characteristics of the tree? We were trying out reiserfs with and without tail packing, comparing ext3 to ext2 and wondering if a journal was needed, and much much more. These days ext4 deals with it so much better but the fact still remains that the tree is a unique construct and efforts to reduce it's size for things most poeple don;t use is always a good thing > > Do you agree or disagree? > > James > > > > > > -- Alan McKinnon alan.mckin...@gmail.com
[gentoo-user] Re: OT: pthreads condition variable/mutex question
On 2014-08-13, Alan McKinnon wrote: > On 13/08/2014 19:21, Grant Edwards wrote: >> This is not Gentoo specific, and while I'm doing my prototyping and >> development on a Gentoo system, the eventual target is not going to >> be running Gentoo -- so feel free to ignore this thread or throw >> things at me. >> >> I'm trying to figure out how to synchronize threads which may be in >> different processes. Basically, I want thread A to be able to wake up >> any number of other threads B, C, D, ... who are all blocking until A >> says "go" (and who may or may not be in other processes). > > Sounds like you a "one talker - many listeners" model. For that particular case, yes. > Have you considered a really simple solution like dbus? I don't know if I would call dbus "really simple". :) My current implementation uses Unix domain sockets (which is what dbus usually uses, isn't it?), and I'm trying to figure out how to reduce overhead and latency. dbus would add even more overhead (it has code to deal with byte ordering, serial/cookies, and various other features and abstractions). I'm not sure it's really practical for high-frequency events (e.g 100-200 events per second). -- Grant Edwards grant.b.edwardsYow! I've got a COUSIN at who works in the GARMENT gmail.comDISTRICT ...
[gentoo-user] Re: OT: pthreads condition variable/mutex question
On 2014-08-13, Alec Ten Harmsel wrote: >> I may have to stick with sockets when I want to block until some event >> happens. > > To be clear, do you want to block or sleep/yield until an event > happens? I don't see the difference -- isn't that what a blocking call does: sleep/yield until some event happens? > I'm sorry for not being too helpful. Just one last question: Can you > describe what exactly your code is supposed to do, or is it something > that you can't talk about because it's a work thing? I don't care > either way, but I'm just curious because it seems you need to > optimize quite a bit. One process implements a communications protocol that is maintaining communications links with a handful of slave devices connected to serial ports running at baud rates up to 230400 baud (38400 is the most common). There are typically one (or maybe two) hundred messages per second being exchanged with each slave device. I'll call that process the server. There are other client processes that want to access the slaves and the inforamation being received from them. Some of the clients just want to do low-frequency transactions for configuration/diagnostic purposes, and Unix domain sockets work fine for that. Other clients may want to wake up every time a certain high frequency event happens. That's where I'm trying to use condition variables. Other clients will periodically (typically once every 5-50 ms) want to see the most recent copy of a particular received message. I'm thinking about using shared memory and rwlocks for that, but I haven't figured out how to deal with the case where a process aborts while holding a lock. -- Grant Edwards grant.b.edwardsYow! Kids, don't gross me at off ... "Adventures with gmail.comMENTAL HYGIENE" can be carried too FAR!
Re: [gentoo-user] OT: pthreads condition variable/mutex question
On 13/08/2014 19:21, Grant Edwards wrote: > This is not Gentoo specific, and while I'm doing my prototyping and > development on a Gentoo system, the eventual target is not going to be > running Gentoo -- so feel free to ignore this thread or throw things > at me. > > I'm trying to figure out how to synchronize threads which may be in > different processes. Basically, I want thread A to be able to wake up > any number of other threads B, C, D, ... who are all blocking until A > says "go" (and who may or may not be in other processes). Sounds like you a "one talker - many listeners" model. Have you considered a really simple solution like dbus? > > Other (mostly embedded) OSes I've used had some sort of "event flag" > API that did exactly what I'm looking for, but I can't seem to find > such a thing in pthreads. > > A condition variable in shared memory is the closest thing I have > found, and my test applications are working OK (so far). But, I'm > unclear on the purpose of the mutex whose address you pass to > pthread_cond_wait(). > > Is it to prevent race conditions when manipulating the condition > variable's internal state? I don't see how that can be the case, since > the signal/broadcast call would have to be aware of it and it isn't. > > The mutex appears to be there to serialize access to some user-defined > variable(s) (outside the condition variable itself) which I don't > have. So all the mutex locking/unlocking and resultant blocking of B, > C, D is just wasted overhead and pointless latency. > > pthread_cond_wait(3) says > >When using condition variables there is always a Boolean predicate >involving shared variables associated with each condition wait that >is true if the thread should proceed. Spurious wakeups from the >pthread_cond_timedwait() or pthread_cond_wait() functions may >occur. Since the return from pthread_cond_timedwait() or >pthread_cond_wait() does not imply anything about the value of this >predicate, the predicate should be re-evaluated upon such return. > > I have no "Boolean predicate" (which presumably comprises the > "user-defined variables outside the condition variable" I mentioned > above), and I don't want spurious wakeups, so a pthreads condition > variable would appear to be the wrong thing to use. > > Is there something like an "event flag" similar to a condition > variable without spurious wakeup problem and without the extra > overhead of the mutex and "Boolean predicate". > > Or am I expected to build my own "event flag" using the aforesaid > "boolean predicate" just to avoid the spurious wakeup problem? [I'm > guessing this is the case...] > -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Re: distfiles contains extra files?
On Wed, 13 Aug 2014 16:55:47 + (UTC), James wrote: > Now that I'm looking, it looks like a policy decision for the devs > to formally evaluate. /distfiles/ should not be a dir for garbage, > one-off-files and other such nonsense. That's exactly what it is for. One-off files that the ebuild downloads, uses and then no longer needs. Nothing in $DISTDIR is needed by a running system. -- Neil Bothwick Press button to test: release to detonate. signature.asc Description: PGP signature
Re: [gentoo-user] Re: distfiles contains extra files?
On Wed, 13 Aug 2014 16:46:19 + (UTC), James wrote: > > > I previously thought that /usr/portage/distfiles > > > only contains tar files. I have not clean up the > > > system, as I'm moving (dupicating some files for my > > > /usr/local/ needs. > > > It contains everything downloaded by ebuilds. > > I guess our ebuilds are getting creative on what they allow. It's not what they allow, it's what they need. > > > I thought all patch files where kept in the subdirs where the > > > ebuilds are located (files dir)? It is still where most *.patch > > > files are located. > > > ISTR the policy is to only include smaller patch files in the tree, > > large patches are downloaded to avoid everyone having to sync them. > > OK, but wny not put them in a subdir under the specific ebuild > requiring those smaller patches? Seems to be a dumb move to me. In the portage tree? That idea sucks for several reasons. If you put them in the tree, the next sync will delete them. If you have a shared $DISTDIR, you will still have to download multiple times. > > > *.exe (like verdan32.exe webdin32.exe ) > > These are the installers for the MS corefonts. > > OK, why they need to be in /distfiles/? seriously .exe > files in a repository for sources? ARE YOU KIDDING ME? Where else would distribution files for software go but in distfiles? The clue is in the name. It's distfiles, not srcfiles, it contains the raw files needed to install the software, be that source or binary data. > Let's just ignor the /bin/sbin and name it /sbinge > as devs must be 'binge drinking' if this is our standard? Now you're just being silly, these are fonts, not executables. They have a .exe extension because they are self-extracting zip file for windows, that doesn't mean we run them. > > You can use eclean to remove obsolete distfiles and packages. But > > don't use eclean on a shared $DISTDIR. > > Yes, agreed. But the point is this sort of stuff belongs in the file > sub-tree under the ebuild that requires it, not as part of /distfiles/ > imho. From my perspective strowing random files into /distfiles/ > puts it in the category of "cruft". > > Do you agree or disagree? Disagree, but you've probably already worked that out. Anything an ebuild needs to download, be in source tarballs, binary archives or svn checkouts, goes in $DISTDIR. -- Neil Bothwick Scrotum is a small planet near Uranus. True/False? signature.asc Description: PGP signature
Re: [gentoo-user] Re: OT: pthreads condition variable/mutex question
On Wed 13 Aug 2014 03:23:21 PM EDT, Grant Edwards wrote: > On 2014-08-13, Alec Ten Harmsel wrote: >> 2014-08-13 12:21 GMT-05:00 Grant Edwards : > >> Without knowing what you're doing, this sounds like a bad idea; if >> you *need* to synchronize threads, why aren't they running in the >> same process? > > I'm trying to decouple different portions of a system as much as > possible. Currently, the different parts communicate via Unix domain > sockets. That works OK, but for a few of the high-frequence > operations I'm trying to find a way to eliminate the overhead that's > involved in sockets (system calls, context switches, copying data from > userspace to kernel space and back to user space). Decoupling == great. Is it possible that you could do something like this: Thread 'a' in process 'a': 1. Event in thread 'a' is generated or whatever 2. if low-frequency event, send a message over a socket to a different thread/process 3. if high-frequency event, push event into shared memory thread-safe queue Thread 'b' in process 'a': 1. infinite loop reading from shared memory queue and processing events Ideally, the high-frequency events should be handled in the same thread, then in the same process, and only lastly in a different process. While decoupling is great, it seems like you're losing the benefits of it by tightly coupling all of your "decoupled" threads and processes. > I may have to stick with sockets when I want to block until some event > happens. To be clear, do you want to block or sleep/yield until an event happens? I'm sorry for not being too helpful. Just one last question: Can you describe what exactly your code is supposed to do, or is it something that you can't talk about because it's a work thing? I don't care either way, but I'm just curious because it seems you need to optimize quite a bit. Alec
Re: [gentoo-user] Re: distfiles contains extra files?
On Wed, Aug 13, 2014 at 04:55:47PM +, James wrote > Now that I'm looking, it looks like a policy decision for the devs > to formally evaluate. /distfiles/ should not be a dir for garbage, > one-off-files and other such nonsense. It was (circa 2004 for me) > a repository for compressed sources. A question for you. A security problem is discovered in linux kernel 3.a.b.c.d. The problem is solved by changing 10 lines in the source code. Which option would you prefer... a) downloading the 10-line patchfile, and having the ebuild apply the patch against the existing kernel 3.a.b.c.d source? b) downloading a new 75 megabyte 3.a.b.c.d-r1 linux kernel Ditto for 120 megabytes of firefox sources, 83 megabytes of gcc sources, etc, etc. That would suck for anybody who has an download cap and/or a slow connection not to mention increasing bandwidth costs for the Gentoo servers and mirrors. And if you *REALLY* wanted, I suppose that foo.exe could be "compressed" to foo.exe.xz with a possible savings of a few bytes, or possibly being even slightly larger than the original. If the files are already compressed, there's no point in trying to compress them again. It provides zero benefit and creates extra work. -- Walter Dnes I don't run "desktop environments"; I run useful applications
[gentoo-user] Re: OT: pthreads condition variable/mutex question
On 2014-08-13, Andrés Becerra Sandoval wrote: > In short: > > Withouth the use of the lock, the condition variable and a shared > variable in concert you can get in trouble! That is often true, but 1) I don't have a shared variable that I want to associate with the condition variable. There is some shared data, but I'm going to use means other than a mutex to control access to it. 2) In most of the examples presented in that article the shared variable is an int that is set to 0 or 1 by one thread and is tested for non-zero value by the other. I'm not aware of any CPU on which Linux runs where those operations can result in a race condition thus requiring a mutex. [Yes I know that isn't guaranteed by the C language definiton, but one can verify that it's true.] If I create a shared mutex just to make pthread_cond_wait happy, and one of the "clients" (the ones calling pthread_cond_wait) dies while holding the mutex, then it deadlocks all of the other clients. To be safe it seems one should create a private mutex for each thread that's going to call pthread_cond_wait(). Private mutexes should be very little overhead in the uncontended case, so that's an option. -- Grant Edwards grant.b.edwardsYow! I wonder if I should at put myself in ESCROW!! gmail.com
[gentoo-user] Re: OT: pthreads condition variable/mutex question
On 2014-08-13, Alec Ten Harmsel wrote: > 2014-08-13 12:21 GMT-05:00 Grant Edwards : > Without knowing what you're doing, this sounds like a bad idea; if > you *need* to synchronize threads, why aren't they running in the > same process? I'm trying to decouple different portions of a system as much as possible. Currently, the different parts communicate via Unix domain sockets. That works OK, but for a few of the high-frequence operations I'm trying to find a way to eliminate the overhead that's involved in sockets (system calls, context switches, copying data from userspace to kernel space and back to user space). > If it's going to be running on Linux, you can send signals through > the kernel's signal API; specifically HUP, USR1, and USR2 might be of > interest to you. I may have to look into that. Unfortunately signals are a real pain to use in a multi-threaded application: all signals get delivered to a single thread, when then has to "distribute" them somehow to the thread that really cares. >> A condition variable in shared memory is the closest thing I have >> found, and my test applications are working OK (so far). But, I'm >> unclear on the purpose of the mutex whose address you pass to >> pthread_cond_wait(). > > I'm too much of a rookie to know how to do this; how are you sharing > memory between processes? The standard Posix way: shm_open() et al. >> The mutex appears to be there to serialize access to some >> user-defined variable(s) (outside the condition variable itself) >> which I don't have. So all the mutex locking/unlocking and resultant >> blocking of B, C, D is just wasted overhead and pointless latency. > > This is definitely not a task for mutexes, a boolean or signaling > would work much better. I'm not sure how one would use a boolean to wake up a thread. I may have to stick with sockets when I want to block until some event happens. -- Grant Edwards grant.b.edwardsYow! It's a hole all the at way to downtown Burbank! gmail.com
Re: [gentoo-user] OT: pthreads condition variable/mutex question
2014-08-13 12:21 GMT-05:00 Grant Edwards : > This is not Gentoo specific, and while I'm doing my prototyping and > development on a Gentoo system, the eventual target is not going to be > running Gentoo -- so feel free to ignore this thread or throw things > at me. You're close enough ;) I'll try to answer to the best of my ability, which is admittedly not much. > I'm trying to figure out how to synchronize threads which may be in > different processes. Basically, I want thread A to be able to wake up > any number of other threads B, C, D, ... who are all blocking until A > says "go" (and who may or may not be in other processes). Without knowing what you're doing, this sounds like a bad idea; if you *need* to synchronize threads, why aren't they running in the same process? If it's going to be running on Linux, you can send signals through the kernel's signal API; specifically HUP, USR1, and USR2 might be of interest to you. > A condition variable in shared memory is the closest thing I have > found, and my test applications are working OK (so far). But, I'm > unclear on the purpose of the mutex whose address you pass to > pthread_cond_wait(). I'm too much of a rookie to know how to do this; how are you sharing memory between processes? > The mutex appears to be there to serialize access to some user-defined > variable(s) (outside the condition variable itself) which I don't > have. So all the mutex locking/unlocking and resultant blocking of B, > C, D is just wasted overhead and pointless latency. This is definitely not a task for mutexes, a boolean or signaling would work much better. I'm not sure what exactly you're trying to do, but I hope this helps. If you can be more specific about the relationships between processes, I might be able to help more. Alec
Re: [gentoo-user] OT: pthreads condition variable/mutex question
2014-08-13 12:36 GMT-05:00 Andrés Becerra Sandoval : > 2014-08-13 12:21 GMT-05:00 Grant Edwards : >> This is not Gentoo specific, and while I'm doing my prototyping and >> development on a Gentoo system, the eventual target is not going to be >> running Gentoo -- so feel free to ignore this thread or throw things >> at me. >> >> I'm trying to figure out how to synchronize threads which may be in >> different processes. Basically, I want thread A to be able to wake up >> any number of other threads B, C, D, ... who are all blocking until A >> says "go" (and who may or may not be in other processes). >> >> Other (mostly embedded) OSes I've used had some sort of "event flag" >> API that did exactly what I'm looking for, but I can't seem to find >> such a thing in pthreads. >> >> A condition variable in shared memory is the closest thing I have >> found, and my test applications are working OK (so far). But, I'm >> unclear on the purpose of the mutex whose address you pass to >> pthread_cond_wait(). >> >> Is it to prevent race conditions when manipulating the condition >> variable's internal state? I don't see how that can be the case, since >> the signal/broadcast call would have to be aware of it and it isn't. >> >> The mutex appears to be there to serialize access to some user-defined >> variable(s) (outside the condition variable itself) which I don't >> have. So all the mutex locking/unlocking and resultant blocking of B, >> C, D is just wasted overhead and pointless latency. >> >> pthread_cond_wait(3) says >> >>When using condition variables there is always a Boolean predicate >>involving shared variables associated with each condition wait that >>is true if the thread should proceed. Spurious wakeups from the >>pthread_cond_timedwait() or pthread_cond_wait() functions may >>occur. Since the return from pthread_cond_timedwait() or >>pthread_cond_wait() does not imply anything about the value of this >>predicate, the predicate should be re-evaluated upon such return. >> >> I have no "Boolean predicate" (which presumably comprises the >> "user-defined variables outside the condition variable" I mentioned >> above), and I don't want spurious wakeups, so a pthreads condition >> variable would appear to be the wrong thing to use. >> >> Is there something like an "event flag" similar to a condition >> variable without spurious wakeup problem and without the extra >> overhead of the mutex and "Boolean predicate". >> >> Or am I expected to build my own "event flag" using the aforesaid >> "boolean predicate" just to avoid the spurious wakeup problem? [I'm >> guessing this is the case...] >> >> -- >> Grant Edwards grant.b.edwardsYow! I'm DESPONDENT ... I >> at hope there's something >> gmail.comDEEP-FRIED under this >>miniature DOMED STADIUM >> ... >> >> > > Hi Grant, > > The best explanation I have read is this chapter: > http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf > > from the book: > > http://pages.cs.wisc.edu/~remzi/OSTEP/ > > I know its 17 pages, but it is worth it! > > > -- > Andrés Becerra Sandoval In short: Withouth the use of the lock, the condition variable and a shared variable in concert you can get in trouble! -- Andrés Becerra Sandoval
Re: [gentoo-user] OT: pthreads condition variable/mutex question
2014-08-13 12:21 GMT-05:00 Grant Edwards : > This is not Gentoo specific, and while I'm doing my prototyping and > development on a Gentoo system, the eventual target is not going to be > running Gentoo -- so feel free to ignore this thread or throw things > at me. > > I'm trying to figure out how to synchronize threads which may be in > different processes. Basically, I want thread A to be able to wake up > any number of other threads B, C, D, ... who are all blocking until A > says "go" (and who may or may not be in other processes). > > Other (mostly embedded) OSes I've used had some sort of "event flag" > API that did exactly what I'm looking for, but I can't seem to find > such a thing in pthreads. > > A condition variable in shared memory is the closest thing I have > found, and my test applications are working OK (so far). But, I'm > unclear on the purpose of the mutex whose address you pass to > pthread_cond_wait(). > > Is it to prevent race conditions when manipulating the condition > variable's internal state? I don't see how that can be the case, since > the signal/broadcast call would have to be aware of it and it isn't. > > The mutex appears to be there to serialize access to some user-defined > variable(s) (outside the condition variable itself) which I don't > have. So all the mutex locking/unlocking and resultant blocking of B, > C, D is just wasted overhead and pointless latency. > > pthread_cond_wait(3) says > >When using condition variables there is always a Boolean predicate >involving shared variables associated with each condition wait that >is true if the thread should proceed. Spurious wakeups from the >pthread_cond_timedwait() or pthread_cond_wait() functions may >occur. Since the return from pthread_cond_timedwait() or >pthread_cond_wait() does not imply anything about the value of this >predicate, the predicate should be re-evaluated upon such return. > > I have no "Boolean predicate" (which presumably comprises the > "user-defined variables outside the condition variable" I mentioned > above), and I don't want spurious wakeups, so a pthreads condition > variable would appear to be the wrong thing to use. > > Is there something like an "event flag" similar to a condition > variable without spurious wakeup problem and without the extra > overhead of the mutex and "Boolean predicate". > > Or am I expected to build my own "event flag" using the aforesaid > "boolean predicate" just to avoid the spurious wakeup problem? [I'm > guessing this is the case...] > > -- > Grant Edwards grant.b.edwardsYow! I'm DESPONDENT ... I > at hope there's something > gmail.comDEEP-FRIED under this >miniature DOMED STADIUM ... > > Hi Grant, The best explanation I have read is this chapter: http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf from the book: http://pages.cs.wisc.edu/~remzi/OSTEP/ I know its 17 pages, but it is worth it! -- Andrés Becerra Sandoval
[gentoo-user] OT: pthreads condition variable/mutex question
This is not Gentoo specific, and while I'm doing my prototyping and development on a Gentoo system, the eventual target is not going to be running Gentoo -- so feel free to ignore this thread or throw things at me. I'm trying to figure out how to synchronize threads which may be in different processes. Basically, I want thread A to be able to wake up any number of other threads B, C, D, ... who are all blocking until A says "go" (and who may or may not be in other processes). Other (mostly embedded) OSes I've used had some sort of "event flag" API that did exactly what I'm looking for, but I can't seem to find such a thing in pthreads. A condition variable in shared memory is the closest thing I have found, and my test applications are working OK (so far). But, I'm unclear on the purpose of the mutex whose address you pass to pthread_cond_wait(). Is it to prevent race conditions when manipulating the condition variable's internal state? I don't see how that can be the case, since the signal/broadcast call would have to be aware of it and it isn't. The mutex appears to be there to serialize access to some user-defined variable(s) (outside the condition variable itself) which I don't have. So all the mutex locking/unlocking and resultant blocking of B, C, D is just wasted overhead and pointless latency. pthread_cond_wait(3) says When using condition variables there is always a Boolean predicate involving shared variables associated with each condition wait that is true if the thread should proceed. Spurious wakeups from the pthread_cond_timedwait() or pthread_cond_wait() functions may occur. Since the return from pthread_cond_timedwait() or pthread_cond_wait() does not imply anything about the value of this predicate, the predicate should be re-evaluated upon such return. I have no "Boolean predicate" (which presumably comprises the "user-defined variables outside the condition variable" I mentioned above), and I don't want spurious wakeups, so a pthreads condition variable would appear to be the wrong thing to use. Is there something like an "event flag" similar to a condition variable without spurious wakeup problem and without the extra overhead of the mutex and "Boolean predicate". Or am I expected to build my own "event flag" using the aforesaid "boolean predicate" just to avoid the spurious wakeup problem? [I'm guessing this is the case...] -- Grant Edwards grant.b.edwardsYow! I'm DESPONDENT ... I at hope there's something gmail.comDEEP-FRIED under this miniature DOMED STADIUM ...
Re: [gentoo-user] Re: distfiles contains extra files?
Am 13.08.2014 um 18:55 schrieb James:
> Peter Humphrey prh.myzen.co.uk> writes:
>
>
>> On Wednesday 13 August 2014 15:07:19 James wrote:
>>> maybe I need to return to cleaning up distfiles/ by hand?
>> Yes, I see I have other things than .tar.bz2 too, now you mention it.
>> One of my boxes runs http-replicator to serve distfiles to the network. The
>> clean-up script I run after upgrades includes this line:
>> find /var/cache/http-replicator -ctime +182 -exec rm '-v' {} +
>> Also, have you tried eclean-dist, from app-portage/gentoolkit?
>
> I'm not sure why, I figured I did not have to manually clean up
> /distfiles/ any more. for a period of time I could sware that
> it was a clean repository for compressed/tar source files only.
never was. Never will be.
>
> Now that I'm looking, it looks like a policy decision for the devs
> to formally evaluate. /distfiles/ should not be a dir for garbage,
> one-off-files and other such nonsense. It was (circa 2004 for me)
> a repository for compressed sources.
you are wrong. There is no 'garbage' in $DISTDIR, only stuff needed to
install a certain package. Nothing there is random or without reason.
Maybe you should educate yourself a bit. Not everything is distributed
as compressed tar. Not everything needed is included in archives. Not
everything is done because you imagine it that way.-
And it never was a repository for compressed sources - ever. At least
not since Gentoo 1.0
>
>
> Maybe we can get systemd
more idiotic ideas? Why not ask the kernel to clean it up. Or your
mother. Or maybe someone else who has nothing to do with package
managment...?
> to clean this up? It's the new "daddy"
> for all sorts of poorly wrtten codes, so why not add cleaning up
> /distfiles/ to it's new fiefdom?
because to be able to do that, it would have to know about portage?
Re: [gentoo-user] Re: distfiles contains extra files?
Am 13.08.2014 um 18:46 schrieb James: > Neil Bothwick digimed.co.uk> writes: > > >>> I previously thought that /usr/portage/distfiles >>> only contains tar files. I have not clean up the >>> system, as I'm moving (dupicating some files for my >>> /usr/local/ needs. >> It contains everything downloaded by ebuilds. > I guess our ebuilds are getting creative on what they allow. no. They just download what is needed. If a package needs a seperate patch. The patch is downloaded. If a needed package is only supplied as a self extracting zip - aka exe. Then that is what portage will download. Seriously, either get a life or have a look at the world at large. Just because you 'think' something is doing something does not mean it is true - or the correct way. >>> I thought all patch files where kept in the subdirs where the >>> ebuilds are located (files dir)? It is still where most *.patch >>> files are located. >> ISTR the policy is to only include smaller patch files in the tree, large >> patches are downloaded to avoid everyone having to sync them. > OK, but wny not put them in a subdir under the specific ebuild > requiring those smaller patches? Seems to be a dumb move to me. ARGH. WHY? why put a unneeded subdir into distfiles that only makes things harder? Or did you just ask to put $BIGPATCH into the tree? >>> *.exe (like verdan32.exe webdin32.exe ) >> These are the installers for the MS corefonts. > OK, why they need to be in /distfiles/? because they are the way the files needed are distributed? > seriously .exe > files in a repository for sources? ARE YOU KIDDING ME? are you an idiot? Seriously? distfiles is not for sources. IT IS FOR NEEDED FILES. Binary, source. Does not matter. > Let's just ignor the /bin/sbin and name it /sbinge > as devs must be 'binge drinking' if this is our standard? > > /bin/ms/? /usr/local/bin/ yep, idiot. not able to think about something, but critizing those who spent some times to come up with it... You can use eclean to remove obsolete distfiles and packages. But don't use eclean on a shared $DISTDIR. > Yes, agreed. But the point is this sort of stuff belongs in the file > sub-tree under the ebuild that requires it, not as part of /distfiles/ the day, emerge --sync downloads 50mb binary packages because of some fonts I never need, instead of downloading them when they ebuild is installed and putting them into DISTDIR where they belong. Is the day I will strangle someone. Any other idiotic ideas? >imho. From my perspective strowing random files into /distfiles/ puts it in the category of "cruft". Do you agree or disagree? James 'cruft' because they are in a central, easily cleaned resporitory, and only downloaded when needed versus scattered all over the tree and pushed on everybody you are wrong. No way to agree with you. Nope. Btw, these are not random files. As you should have understood as you read Neil's and the others explanations. Or the handbook. Or manpages.
[gentoo-user] Re: distfiles contains extra files?
Peter Humphrey prh.myzen.co.uk> writes:
> On Wednesday 13 August 2014 15:07:19 James wrote:
> > maybe I need to return to cleaning up distfiles/ by hand?
> Yes, I see I have other things than .tar.bz2 too, now you mention it.
> One of my boxes runs http-replicator to serve distfiles to the network. The
> clean-up script I run after upgrades includes this line:
> find /var/cache/http-replicator -ctime +182 -exec rm '-v' {} +
> Also, have you tried eclean-dist, from app-portage/gentoolkit?
I'm not sure why, I figured I did not have to manually clean up
/distfiles/ any more. for a period of time I could sware that
it was a clean repository for compressed/tar source files only.
Now that I'm looking, it looks like a policy decision for the devs
to formally evaluate. /distfiles/ should not be a dir for garbage,
one-off-files and other such nonsense. It was (circa 2004 for me)
a repository for compressed sources.
Maybe we can get systemd to clean this up? It's the new "daddy"
for all sorts of poorly wrtten codes, so why not add cleaning up
/distfiles/ to it's new fiefdom?
just teasing(not really?).ha ha ha ha ah ha ha ha ha.
Lennart can you help a bro out?
;-)
James
[gentoo-user] Re: distfiles contains extra files?
Neil Bothwick digimed.co.uk> writes: > > I previously thought that /usr/portage/distfiles > > only contains tar files. I have not clean up the > > system, as I'm moving (dupicating some files for my > > /usr/local/ needs. > It contains everything downloaded by ebuilds. I guess our ebuilds are getting creative on what they allow. > > I thought all patch files where kept in the subdirs where the > > ebuilds are located (files dir)? It is still where most *.patch > > files are located. > ISTR the policy is to only include smaller patch files in the tree, large > patches are downloaded to avoid everyone having to sync them. OK, but wny not put them in a subdir under the specific ebuild requiring those smaller patches? Seems to be a dumb move to me. > > *.exe (like verdan32.exe webdin32.exe ) > These are the installers for the MS corefonts. OK, why they need to be in /distfiles/? seriously .exe files in a repository for sources? ARE YOU KIDDING ME? Let's just ignor the /bin/sbin and name it /sbinge as devs must be 'binge drinking' if this is our standard? /bin/ms/? /usr/local/bin/ > > I guess I need "enlightenment" ? > You can use whichever DE you prefer :) U very funny! I use 'enlightenment' compliments to the Bhagavad Gita. I actually has to read the English version of the entire book and write a paper on it, in only 3 days. Academically speaking I got 'enlightenment' from that literarture class...ymmv. >> cleaning up distfiles/ by hand? > You can use eclean to remove obsolete distfiles and packages. But don't > use eclean on a shared $DISTDIR. Yes, agreed. But the point is this sort of stuff belongs in the file sub-tree under the ebuild that requires it, not as part of /distfiles/ imho. From my perspective strowing random files into /distfiles/ puts it in the category of "cruft". Do you agree or disagree? James
[gentoo-user] app-forensics/aide
Howdy, It purports to be a better file integrity checker than tripwire; it even supports using postgresql for very large needs. There is a scant list of files suggested in the aide docs to generate the initial md5 records of these (critically) monitored files. [1] # Next decide what directories/files you want in the database /etc p+i+u+g #check only permissions, inode, user and group for etc /bin MyRule # apply the custom rule to the files in bin /sbin MyRule # apply the same custom rule to the files in sbin /var MyRule !/var/log/.* # ignore the log dir it changes too often !/var/spool/.* # ignore spool dirs as they change too often !/var/adm/utmp$ # ignore the file /var/adm/utmp I'd be curious if anyone has a more, gentoo-specific list tailored to royjrt gentoo servers or workstations, to generate the initial md5 records for a (newly installed) gentoo system. [1] http://aide.sourceforge.net/stable/manual.html TIA, James
Re: [gentoo-user] distfiles contains extra files?
On Wednesday 13 August 2014 15:07:19 James wrote:
> I previously thought that /usr/portage/distfiles
> only contains tar files. I have not clean up the
> system, as I'm moving (dupicating some files for my
> /usr/local/ needs.
>
> But I do not remember ever seeing any types of files
> other than compressed tar files in /distfiles/
>
> Now I have (sampling):
>
> *.pathch
>
> I thought all patch files where kept in the subdirs where the
> ebuilds are located (files dir)? It is still where most *.patch
> files are located.
>
> *.exe (like verdan32.exe webdin32.exe )
>
> and many others (here's a sample from /usr/portage/distfiles):
>
> scons-2.3.0-user.html
> scons-2.3.0-user.pdf
> scrollkeeper-omf.dtd
> seamonkey-2.26.1-en-GB.xpi
> rdoc-4.0.1.gem
> readline62-004
> rfc2307bis.schema-20140524
> nmap-logo-64.png
> gobject-introspection-1.40.0.tar.xz._checksum_failure_.146ofr
>
> I guess I need "enlightenment" ?
> Also, in the past I have manually aged/pruned or size/prunded
> the disfiles dir, but I stopped doing that in the age
> of 2T+ sized drivesmaybe I need to return to cleaning up
> distfiles/ by hand?
Yes, I see I have other things than .tar.bz2 too, now you mention it.
One of my boxes runs http-replicator to serve distfiles to the network. The
clean-up script I run after upgrades includes this line:
find /var/cache/http-replicator -ctime +182 -exec rm '-v' {} +
Also, have you tried eclean-dist, from app-portage/gentoolkit?
--
Regards
Peter
Re: [gentoo-user] distfiles contains extra files?
On Wed, 13 Aug 2014 15:07:19 + (UTC), James wrote: > I previously thought that /usr/portage/distfiles > only contains tar files. I have not clean up the > system, as I'm moving (dupicating some files for my > /usr/local/ needs. It contains everything downloaded by ebuilds. > But I do not remember ever seeing any types of files > other than compressed tar files in /distfiles/ > > Now I have (sampling): > > *.pathch > > I thought all patch files where kept in the subdirs where the > ebuilds are located (files dir)? It is still where most *.patch > files are located. ISTR the policy is to only include smaller patch files in the tree, large patches are downloaded to avoid everyone having to sync them. > > *.exe (like verdan32.exe webdin32.exe ) These are the installers for the MS corefonts. > I guess I need "enlightenment" ? You can use whichever DE you prefer :) > Also, in the past I have manually aged/pruned or size/prunded > the disfiles dir, but I stopped doing that in the age > of 2T+ sized drivesmaybe I need to return to cleaning up > distfiles/ by hand? You can use eclean to remove obsolete distfiles and packages. But don't use eclean on a shared $DISTDIR. -- Neil Bothwick God is real, unless specifically declared integer. signature.asc Description: PGP signature
[gentoo-user] distfiles contains extra files?
I previously thought that /usr/portage/distfiles only contains tar files. I have not clean up the system, as I'm moving (dupicating some files for my /usr/local/ needs. But I do not remember ever seeing any types of files other than compressed tar files in /distfiles/ Now I have (sampling): *.pathch I thought all patch files where kept in the subdirs where the ebuilds are located (files dir)? It is still where most *.patch files are located. *.exe (like verdan32.exe webdin32.exe ) and many others (here's a sample from /usr/portage/distfiles): scons-2.3.0-user.html scons-2.3.0-user.pdf scrollkeeper-omf.dtd seamonkey-2.26.1-en-GB.xpi rdoc-4.0.1.gem readline62-004 rfc2307bis.schema-20140524 nmap-logo-64.png gobject-introspection-1.40.0.tar.xz._checksum_failure_.146ofr I guess I need "enlightenment" ? Also, in the past I have manually aged/pruned or size/prunded the disfiles dir, but I stopped doing that in the age of 2T+ sized drivesmaybe I need to return to cleaning up distfiles/ by hand? curiously, James
Re: [gentoo-user] distfiles contains extra files?
On Wed, Aug 13, 2014 at 8:37 PM, James wrote: > I previously thought that /usr/portage/distfiles > only contains tar files. I have not clean up the > system, as I'm moving (dupicating some files for my > /usr/local/ needs. > > But I do not remember ever seeing any types of files > other than compressed tar files in /distfiles/ > > Now I have (sampling): > > *.pathch > > I thought all patch files where kept in the subdirs where the > ebuilds are located (files dir)? It is still where most *.patch > files are located. > > *.exe (like verdan32.exe webdin32.exe ) > > and many others (here's a sample from /usr/portage/distfiles): > > scons-2.3.0-user.html > scons-2.3.0-user.pdf > scrollkeeper-omf.dtd > seamonkey-2.26.1-en-GB.xpi > rdoc-4.0.1.gem > readline62-004 > rfc2307bis.schema-20140524 > nmap-logo-64.png > gobject-introspection-1.40.0.tar.xz._checksum_failure_.146ofr > > I guess I need "enlightenment" ? > Also, in the past I have manually aged/pruned or size/prunded > the disfiles dir, but I stopped doing that in the age > of 2T+ sized drivesmaybe I need to return to cleaning up > distfiles/ by hand? > > > curiously, > James > > What distfiles are present there depend on the ebuilds. I use geek-sources from init6 overlay, it stores VCS repositories there.
[gentoo-user] Re: Do I really need kernel support for ppp?
On 2014-08-13, gottl...@nyu.edu wrote: > (I use gnome, hence systemd. Oh my. You have my condolances on both counts. ;) > I have no modem, except the cable modem.) > > network-manager Ouch. Another thorn... > requires ppp and the latest version of ppp (just installed by today's > update world) wants me to add ppp support to my kernel. > > I can certainly do so; is that considered advisable? If so must I do > it right away? Personally, I'd recommend ditching network-manager. I would think anything that assumes everybody must use PPP had been abandoned years (if not decades) ago and shouldn't be used. It's also got the always reliable mark of uselessness: a-long-name-with-a-hyphen-or-two in it. [If it ended in '-tool' you could be 200% sure it should be towed out to sea and scuttled in order to form an artificial reef.] -- Grant Edwards grant.b.edwardsYow! Well, O.K. at I'll compromise with my gmail.comprinciples because of EXISTENTIAL DESPAIR!
Re: [gentoo-user] Do I really need kernel support for ppp?
On Wed, Aug 13 2014, Alan McKinnon wrote: > On 13/08/2014 14:42, gottl...@nyu.edu wrote: >> (I use gnome, hence systemd. I have no modem, except the cable modem.) >> >> network-manager requires ppp and the latest version of ppp (just >> installed by today's update world) wants me to add ppp support to my >> kernel. >> >> I can certainly do so; is that considered advisable? If so must I do it >> right away? > > > Network Manager by design has 4 tabs for connections, ppp is one of > them. Makes sense for a portable workstation - you don't know in advance > what you might use to connect to a network. > > If the ppp ebuild wants ppp in-kernel support (entirely logical), then > do so. Nothing says you *have* to load the module. If it were dangerous, > then every Linux pc supporting ppp would be vulnerable, not so? SO > worrying about it seems very pointless. > > Having said that: > > USE="-ppp" I see. Thanks. This shamed me into reading the networkmanager ebuild and then (once again) reading man 5 ebuild to understand REQUIRED_USE. I now see that I could try USE="-ppp -modemmanager". However, my goal is simplicity, not just minimality, so I will follow your (and the ebuild's) advice and add the kernel support. thanks you and joost again, allan
Re: [gentoo-user] Do I really need kernel support for ppp?
On 13/08/2014 14:42, gottl...@nyu.edu wrote: > (I use gnome, hence systemd. I have no modem, except the cable modem.) > > network-manager requires ppp and the latest version of ppp (just > installed by today's update world) wants me to add ppp support to my > kernel. > > I can certainly do so; is that considered advisable? If so must I do it > right away? Network Manager by design has 4 tabs for connections, ppp is one of them. Makes sense for a portable workstation - you don't know in advance what you might use to connect to a network. If the ppp ebuild wants ppp in-kernel support (entirely logical), then do so. Nothing says you *have* to load the module. If it were dangerous, then every Linux pc supporting ppp would be vulnerable, not so? SO worrying about it seems very pointless. Having said that: USE="-ppp" -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Do I really need kernel support for ppp?
On Wednesday, August 13, 2014 08:42:57 AM gottl...@nyu.edu wrote: > (I use gnome, hence systemd. I have no modem, except the cable modem.) > > network-manager requires ppp and the latest version of ppp (just > installed by today's update world) wants me to add ppp support to my > kernel. > > I can certainly do so; is that considered advisable? If so must I do it > right away? You could put it as a module and forget about it? But isn't there an option to build network-manager without modem support? -- Joost
[gentoo-user] Do I really need kernel support for ppp?
(I use gnome, hence systemd. I have no modem, except the cable modem.) network-manager requires ppp and the latest version of ppp (just installed by today's update world) wants me to add ppp support to my kernel. I can certainly do so; is that considered advisable? If so must I do it right away? thanks, allan
Re: [gentoo-user] akonadi ... don't you just love it?
On 13/08/2014 14:18, Alec Ten Harmsel wrote: > On Wed 13 Aug 2014 03:10:22 AM EDT, Alan McKinnon wrote: >> Bash is also like that, omfg; bash seems to have been designed to make >> decent code fundamentally impossible to write... > > I mean, if you're trying to write anything more than a screenful in Bash > you should definitely use Ruby or Python IMNHO. The RVM (Ruby Version > Manager) project recently was raising money to pay a developer to port > ~20,000 lines of Bash to Ruby, which is just embarrassing. :-) > >> I will refrain from commenting on perl > > I wrote 250 lines of perl and forgot how it worked a day later ;) Write > it, ship it, forget about it. I had the pleasure of maintaining an exceptionally well-written bespoke perl app of 20,000+ lines. Written by an exceptional mathematician with 10+ years of sysadmin experience, that code base was a dream. It all just made so much sense, perl notwithstanding. At the same time I also had to maintain another giant 5,000 line monolithic beast that brought any strong developer to their knees. Oh, the pain, the pain. So with perl it can go both ways -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] akonadi ... don't you just love it?
On Wed 13 Aug 2014 03:10:22 AM EDT, Alan McKinnon wrote: > Bash is also like that, omfg; bash seems to have been designed to make > decent code fundamentally impossible to write... I mean, if you're trying to write anything more than a screenful in Bash you should definitely use Ruby or Python IMNHO. The RVM (Ruby Version Manager) project recently was raising money to pay a developer to port ~20,000 lines of Bash to Ruby, which is just embarrassing. > I will refrain from commenting on perl I wrote 250 lines of perl and forgot how it worked a day later ;) Write it, ship it, forget about it. Alec
Re: [gentoo-user] XFCE4: How cab I disable Restart and Shut down buttons?
On Tue, 12 Aug 2014 18:33:21 -0500
Canek Peláez Valdés wrote:
> Assuming you want to disable the functionality for one user, this
> disables both the functionality *AND* the buttons (they show up grayed
> out):
>
> $ cat /etc/polkit-1/rules.d/10-no-restart-shutdown.rules
> polkit.addRule(function(action, subject) {
> if (subject.user == "myuser") {
> if (action.id.match("org.freedesktop.login1.power-off") ||
> action.id.match("org.freedesktop.login1.reboot")) {
> return polkit.Result.NO;
> }
> }
> });
>
> Of course, change "myuser" for the user you want to disable this. You
> can also use groups (subject.isInGroup("group")), or use "suspend" or
> "hibernate" instead of "reboot".
>
That worked. Thank you very much. Now I have disabled (greyed out) the
buttons for certain users. That's what I have looked for.
Bye
Johannes
--
--//--
//Johannes R. Geiss Mac mini server and
\\ //OpenPandora user
--\X/-
Re: [gentoo-user] akonadi ... don't you just love it?
On 13/08/2014 08:38, J. Roeleveld wrote: >> The good thing about php is that everyone and their dog can knock out >> > running code. >> > The bad thing about php is that they do. > Not PHP's fault, lazy developers' fault. > I disagree. It starts with the language. If a language makes it trivially simple to write atrocious code, and makes the developer jump through hoops to write decent code, then the language is faulty. PHP is like that. Bash is also like that, omfg; bash seems to have been designed to make decent code fundamentally impossible to write... -- Alan McKinnon alan.mckin...@gmail.com
