Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:56:21AM -0500, Justin M. Forbes wrote: > > I'm really against splitting the modules up into more subpackages, > > regardless of how many it is. I will not spend any time looking at how > > to do that. I won't spend time discussing further plans to do something > > I don't feel is maintainable. If you find someone willing to, great. > > Post patches to the kernel list and we'll review them. But I'm telling > > you now that it will be an uphill battle. > > > Just to be clear on this, if someone feels it is worthwhile and wants to > step up and do the work, it can't just be someone who will send patches to > change the current kernel and leave. It has to be someone willing to sign > up to maintain the solution in the rawhide kernel. Modules change, get added > or removed, and/or have deps change pretty much every single release. 3.7 > just moved the nat modules. This is why we are saying that it is much > easier to do a different build of another flavor than to split out the > modules into more subpackages. Head nods to both of you. I'm happy to live in reality rather than utopia and this is helpful feedback. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 11:34:00AM -0400, Josh Boyer wrote: > On Thu, Oct 18, 2012 at 11:15 AM, Matthew Miller > wrote: > > On Thu, Oct 18, 2012 at 10:44:58AM -0400, Josh Boyer wrote: > > >> At the moment though, all of this is just talk anyway. If something > >> like this is to happen, someone actually has to do work. > > > > Start with an idea, discuss it, come up with a plan, find resources for that > > plan, and then implement. Sometimes things happen the other way around, but > > only when we happen to be lucky, and it often has consequences like extra > > ongoing work with no support. So, just talk is an important place to start. > > OK. So here's a more blunt response. > > I'm really against splitting the modules up into more subpackages, > regardless of how many it is. I will not spend any time looking at how > to do that. I won't spend time discussing further plans to do something > I don't feel is maintainable. If you find someone willing to, great. > Post patches to the kernel list and we'll review them. But I'm telling > you now that it will be an uphill battle. > Just to be clear on this, if someone feels it is worthwhile and wants to step up and do the work, it can't just be someone who will send patches to change the current kernel and leave. It has to be someone willing to sign up to maintain the solution in the rawhide kernel. Modules change, get added or removed, and/or have deps change pretty much every single release. 3.7 just moved the nat modules. This is why we are saying that it is much easier to do a different build of another flavor than to split out the modules into more subpackages. Justin -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 11:15 AM, Matthew Miller wrote: > On Thu, Oct 18, 2012 at 10:44:58AM -0400, Josh Boyer wrote: >> > I'm open to this idea, but I think it's nicer if one can go from the >> > reduced >> > selection to the full just by adding in the right package, not changing or >> > removing things. Unlike PAE or etc., I don't think we'd actually build >> > anything differently (would we?). >> Of course we would. The entire point is to reduce the size, and the >> only way to reduce the size is to build it with different config >> options. And we're not talking about going from kernel-virtguest to >> kernel by installing kernel-everythingnotinvirtguest. That's still >> going down the "split the kernel up into a bunch of subpackages" route >> which just creates more work for the maintainers. > > We already have kernel-modules-extra. I think the idea would be to add > kernel-modules-virt and kernel-modules-normal to that, at most. (Or, put the > virt modules in kernel, and just add one more subpackage, > kernel-modules-normal.) There's already code in the spec file for dealing > with modules-extra, so it's mostly a matter of extending that slightly -- > not doing something entirely new, *and* not going down the alarmist slope of > a horde of subpackages. I'm aware of what the idea is. I'm saying it's not as simple as it sounds. modules-extra is a pain. The only reason it still exists is because it is filled with modules that hardly anyone uses. Extending the idea to deal with modules that a large number of people use is a headache because then you have lists to maintain and move around even more than we already do. Automating it isn't as simple as it sounds because the module dependencies change from kernel to kernel and you wind up getting things shifted because some module you had in one package depends on another going into a different package, etc. modules-extra started as a middle ground between leaving little used drivers in the main package and not building them at all. If I had to do it over, I'd just not build them at all. Now another kernel flavor, like kernel-virtguest, with different config options is _not_ new. It's much easier to add, much easier to maintain and is how we've done things pretty much forever. I realize it isn't necessarily the utopia you're looking for, but we live in reality not utopia. It's achievable and fairly maintainable. >> At the moment though, all of this is just talk anyway. If something >> like this is to happen, someone actually has to do work. > > Start with an idea, discuss it, come up with a plan, find resources for that > plan, and then implement. Sometimes things happen the other way around, but > only when we happen to be lucky, and it often has consequences like extra > ongoing work with no support. So, just talk is an important place to start. OK. So here's a more blunt response. I'm really against splitting the modules up into more subpackages, regardless of how many it is. I will not spend any time looking at how to do that. I won't spend time discussing further plans to do something I don't feel is maintainable. If you find someone willing to, great. Post patches to the kernel list and we'll review them. But I'm telling you now that it will be an uphill battle. This is not me being stubborn, or cranky, or an a-hole. This is me trying to save time for all involved, both in the planning phase and in the long term maintenance of any solution. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:44:58AM -0400, Josh Boyer wrote: > > I'm open to this idea, but I think it's nicer if one can go from the reduced > > selection to the full just by adding in the right package, not changing or > > removing things. Unlike PAE or etc., I don't think we'd actually build > > anything differently (would we?). > Of course we would. The entire point is to reduce the size, and the > only way to reduce the size is to build it with different config > options. And we're not talking about going from kernel-virtguest to > kernel by installing kernel-everythingnotinvirtguest. That's still > going down the "split the kernel up into a bunch of subpackages" route > which just creates more work for the maintainers. We already have kernel-modules-extra. I think the idea would be to add kernel-modules-virt and kernel-modules-normal to that, at most. (Or, put the virt modules in kernel, and just add one more subpackage, kernel-modules-normal.) There's already code in the spec file for dealing with modules-extra, so it's mostly a matter of extending that slightly -- not doing something entirely new, *and* not going down the alarmist slope of a horde of subpackages. > At the moment though, all of this is just talk anyway. If something > like this is to happen, someone actually has to do work. Start with an idea, discuss it, come up with a plan, find resources for that plan, and then implement. Sometimes things happen the other way around, but only when we happen to be lucky, and it often has consequences like extra ongoing work with no support. So, just talk is an important place to start. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:39 AM, Matthew Miller wrote: > On Thu, Oct 18, 2012 at 10:33:27AM -0400, Bill Nottingham wrote: >> > All of this can probably already be done with a new 'flavor' in the >> > existing kernel.spec. I really wouldn't do the common/minimal split >> > though. It just makes it more complicated for not a whole lot of gain. >> > >> > The idea that Dave, Justin, and Kevin all had simlutaneously about >> > doing a 'kernel-virtguest' might be worthwhile if someone wants to >> > spend time poking at a config, etc. >> >> That also works with the normal paradigm where all the variants provide >> 'kernel' for RPM dependency purposes; if you try to have a kernel-minimal >> that >> provides 'kernel' while also having a 'kernel' package that requires >> 'kernel-minimal', things get a bit more strange. > > I'm open to this idea, but I think it's nicer if one can go from the reduced > selection to the full just by adding in the right package, not changing or > removing things. Unlike PAE or etc., I don't think we'd actually build > anything differently (would we?). Of course we would. The entire point is to reduce the size, and the only way to reduce the size is to build it with different config options. And we're not talking about going from kernel-virtguest to kernel by installing kernel-everythingnotinvirtguest. That's still going down the "split the kernel up into a bunch of subpackages" route which just creates more work for the maintainers. At the moment though, all of this is just talk anyway. If something like this is to happen, someone actually has to do work. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:33:27AM -0400, Bill Nottingham wrote: > > All of this can probably already be done with a new 'flavor' in the > > existing kernel.spec. I really wouldn't do the common/minimal split > > though. It just makes it more complicated for not a whole lot of gain. > > > > The idea that Dave, Justin, and Kevin all had simlutaneously about > > doing a 'kernel-virtguest' might be worthwhile if someone wants to > > spend time poking at a config, etc. > > That also works with the normal paradigm where all the variants provide > 'kernel' for RPM dependency purposes; if you try to have a kernel-minimal that > provides 'kernel' while also having a 'kernel' package that requires > 'kernel-minimal', things get a bit more strange. I'm open to this idea, but I think it's nicer if one can go from the reduced selection to the full just by adding in the right package, not changing or removing things. Unlike PAE or etc., I don't think we'd actually build anything differently (would we?). -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Josh Boyer (jwbo...@gmail.com) said: > > You'd want to do it something like that. > > > > kernel-minimal as you say but with a Provides: kernel, kernel-common as you > > say. > > > > > > I'd introduce a third metapackage just "kernel" that requires both of those > > and implicitly Provides: kernel. Most people would just get the "kernel" > > metapackage when a transaction asks for something to provide "kernel", but > > if you explicitly ask for kernel-minimal you'd get just the minimal. > > > > This would all be done from one kernel spec and built out at the same time. > > We've got a lot of new infrastructure coming for kernel builds and we don't > > want to make things even more complicated by having to do multiple rpm build > > runs. > > All of this can probably already be done with a new 'flavor' in the > existing kernel.spec. I really wouldn't do the common/minimal split > though. It just makes it more complicated for not a whole lot of gain. > > The idea that Dave, Justin, and Kevin all had simlutaneously about > doing a 'kernel-virtguest' might be worthwhile if someone wants to > spend time poking at a config, etc. That also works with the normal paradigm where all the variants provide 'kernel' for RPM dependency purposes; if you try to have a kernel-minimal that provides 'kernel' while also having a 'kernel' package that requires 'kernel-minimal', things get a bit more strange. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 3:43 PM, Reindl Harald wrote: > Am 17.10.2012 18:52, schrieb Dave Jones: >> With virtualised environments supporting pci/usb passthrough, where do you >> draw the line on what hardware to support in a hypothetical kernel-cloud >> package ? > > with vmxnet3, vmw_pvscsi, vmw_balloon to support vSphere > (all included in the upstream kernel) and the drivers > for basic KVM guests + all the iptabales (nat, conntrack, > rectent, multiport..) you are sipporting a really wide range > of minimized-size and full functional setups And xen and whatever it needs. I mean, if you're targeting virt and cloud, then you can't ignore EC2 and that is all xen. And hyper-v. Can't exclude them either. And whatever Rusty's virt thing is. And on and on. About the only "virt/cloud" thing you can exclude is e.g. virtualbox and that's only because they're not in the upstream kernel. Maybe one day they'll realize the error of their ways and then you have to include that too. > 99 out of 100 doe snot passthru physical hardware Maybe today, but it's an ever increasing target. I don't think it can be entirely dismissed. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 2:38 PM, Jesse Keating wrote: > On 10/17/2012 11:32 AM, Chris Adams wrote: >> >> I would think the only "sane" way would be to just change the packaing, >> not actually build multiple kernels (or even multiple packages with >> kernels). We already build multiple kernels. All from the same source, but still multiple kernels with different config options. E.g. PAE, debug, etc. >> For example, a "kernel-minimal" that has the kernel and the "core" >> modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, >> network support like ipv6 and iptables, and virtio-type drivers), a >> "kernel-common" that has the rest of the current contents of "kernel" >> (and probably obsoletes "kernel"), and then the current >> "kernel-modules-extras". >> >> There will always be requests to move modules from -common to -minimal, >> and it shouldn't be a big fight (I would bet most requests would be >> pretty obvious). That already exists some for -modules-extras. > > > You'd want to do it something like that. > > kernel-minimal as you say but with a Provides: kernel, kernel-common as you > say. > > > I'd introduce a third metapackage just "kernel" that requires both of those > and implicitly Provides: kernel. Most people would just get the "kernel" > metapackage when a transaction asks for something to provide "kernel", but > if you explicitly ask for kernel-minimal you'd get just the minimal. > > This would all be done from one kernel spec and built out at the same time. > We've got a lot of new infrastructure coming for kernel builds and we don't > want to make things even more complicated by having to do multiple rpm build > runs. All of this can probably already be done with a new 'flavor' in the existing kernel.spec. I really wouldn't do the common/minimal split though. It just makes it more complicated for not a whole lot of gain. The idea that Dave, Justin, and Kevin all had simlutaneously about doing a 'kernel-virtguest' might be worthwhile if someone wants to spend time poking at a config, etc. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On 10/17/2012 01:46 PM, David Malcolm wrote: Random worry about this: would this work OK with yum's "keep the last 3 kernels around" functionality? That's obviously something that would have to be tested if this is attempted. I'm not signing up for this work, I was just making a suggestion on how to structure the rpms that fell out of the kernel spec. -- Jesse Keating Fedora -- Freedom² is a feature! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, 2012-10-17 at 11:38 -0700, Jesse Keating wrote: > On 10/17/2012 11:32 AM, Chris Adams wrote: > > I would think the only "sane" way would be to just change the packaing, > > not actually build multiple kernels (or even multiple packages with > > kernels). > > > > For example, a "kernel-minimal" that has the kernel and the "core" > > modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, > > network support like ipv6 and iptables, and virtio-type drivers), a > > "kernel-common" that has the rest of the current contents of "kernel" > > (and probably obsoletes "kernel"), and then the current > > "kernel-modules-extras". > > > > There will always be requests to move modules from -common to -minimal, > > and it shouldn't be a big fight (I would bet most requests would be > > pretty obvious). That already exists some for -modules-extras. > > You'd want to do it something like that. > > kernel-minimal as you say but with a Provides: kernel, kernel-common as > you say. > > > I'd introduce a third metapackage just "kernel" that requires both of > those and implicitly Provides: kernel. Most people would just get the > "kernel" metapackage when a transaction asks for something to provide > "kernel", but if you explicitly ask for kernel-minimal you'd get just > the minimal. > > This would all be done from one kernel spec and built out at the same > time. We've got a lot of new infrastructure coming for kernel builds > and we don't want to make things even more complicated by having to do > multiple rpm build runs. Random worry about this: would this work OK with yum's "keep the last 3 kernels around" functionality? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Am 17.10.2012 18:52, schrieb Dave Jones: > With virtualised environments supporting pci/usb passthrough, where do you > draw the line on what hardware to support in a hypothetical kernel-cloud > package ? with vmxnet3, vmw_pvscsi, vmw_balloon to support vSphere (all included in the upstream kernel) and the drivers for basic KVM guests + all the iptabales (nat, conntrack, rectent, multiport..) you are sipporting a really wide range of minimized-size and full functional setups 99 out of 100 doe snot passthru physical hardware signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, 17 Oct 2012 14:40:39 -0400 Matthew Miller wrote: > On Wed, Oct 17, 2012 at 11:38:13AM -0700, Jesse Keating wrote: > > I'd introduce a third metapackage just "kernel" that requires both > > of those and implicitly Provides: kernel. Most people would just > > get the "kernel" metapackage when a transaction asks for something > > to provide "kernel", but if you explicitly ask for kernel-minimal > > you'd get just the minimal. > > +1 to this I think we should really avoid calling this "minimal" as we have seen that means different things to different folks. if someone wanted to explore this route, I would say call it 'kernel-virt-guest' or something similar. Establish exactly what virt env's are targeted by it, do some test runs to show that it helps anyone at all, and then sell it to the kernel maintainers. ;) Sounds like it could be a nice f19 feature if there are folks interested enough to work on it. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 11:38:13AM -0700, Jesse Keating wrote: > I'd introduce a third metapackage just "kernel" that requires both > of those and implicitly Provides: kernel. Most people would just > get the "kernel" metapackage when a transaction asks for something > to provide "kernel", but if you explicitly ask for kernel-minimal > you'd get just the minimal. +1 to this -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 01:32:23PM -0500, Chris Adams wrote: > There will always be requests to move modules from -common to -minimal, > and it shouldn't be a big fight (I would bet most requests would be > pretty obvious). That already exists some for -modules-extras. That's why I suggest defining the specific targets (for example, EC2, KVM) for the core package. The requirements may grow and change slightly, but in general it provides a simple test, with no fighting needed. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On 10/17/2012 11:32 AM, Chris Adams wrote: I would think the only "sane" way would be to just change the packaing, not actually build multiple kernels (or even multiple packages with kernels). For example, a "kernel-minimal" that has the kernel and the "core" modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, network support like ipv6 and iptables, and virtio-type drivers), a "kernel-common" that has the rest of the current contents of "kernel" (and probably obsoletes "kernel"), and then the current "kernel-modules-extras". There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. You'd want to do it something like that. kernel-minimal as you say but with a Provides: kernel, kernel-common as you say. I'd introduce a third metapackage just "kernel" that requires both of those and implicitly Provides: kernel. Most people would just get the "kernel" metapackage when a transaction asks for something to provide "kernel", but if you explicitly ask for kernel-minimal you'd get just the minimal. This would all be done from one kernel spec and built out at the same time. We've got a lot of new infrastructure coming for kernel builds and we don't want to make things even more complicated by having to do multiple rpm build runs. -- Jesse Keating Fedora -- Freedom² is a feature! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Once upon a time, Richard W.M. Jones said: > It really depends on what 'kernel-minimal' is. If it's the > same kernel (identical vmlinuz) with groups of modules, then I'm > assuming this is the same as what everyone else is proposing. I would think the only "sane" way would be to just change the packaing, not actually build multiple kernels (or even multiple packages with kernels). For example, a "kernel-minimal" that has the kernel and the "core" modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, network support like ipv6 and iptables, and virtio-type drivers), a "kernel-common" that has the rest of the current contents of "kernel" (and probably obsoletes "kernel"), and then the current "kernel-modules-extras". There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. -- Chris Adams Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 07:34:22PM +0200, drago01 wrote: > If it is all about using kernel-minimal (or whatever it is called) > instead of kernel there is no extra work for the ones that build > minimal images at all. It really depends on what 'kernel-minimal' is. If it's the same kernel (identical vmlinuz) with groups of modules, then I'm assuming this is the same as what everyone else is proposing. But more dangerous if this is a recompile of the kernel (maybe same source, different configs). Then inevitably we're testing different codepaths. [...] > > Unfortunately containers don't work for every application out there. > > Obviously *if* you can use a container, then you should, and you > > probably are already. > > That might be true but I can't think of such applications right now. > Couldn't the applications you have in mind be modified / fixed to work > in such an environment? LXC isn't a secure alternative to full virtualization -- try poking random /sys files on your container -- and even now OpenVZ isn't quite upstream. Also clouds are existing ecosystems. For example I can't call up Amazon and ask that they reimplement their service on top of LXC instead of Xen. > In that case you should place the image on your internal network. Well, no. I don't colocate with everyone who might want to download a minimal image. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 6:34 PM, drago01 wrote:
> On Wed, Oct 17, 2012 at 6:58 PM, Richard W.M. Jones wrote:
>> On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote:
>>> On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller
>>> wrote:
>>> > On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote:
>>> >> > Basically: it's hard,
>>> >> it is a mess.
>>> >> > but the only way we're going to get to a
>>> >> > reasonably-small minimal image,
>>> >> not true.
>>> >
>>> > Given that the kernel is currently a full quarter of the current image, I
>>> > think it has to be.
>>>
>>> No you could also use a different kernel image;
>>> build your own kernel;
>>
>> [I'll treat these two the same, because they amount to the same thing]
>
> No the one means we ship a kernel-foo (like kernel-minimal or whatever
> we call it) and the other is you build your own kernel.
>
>> It's a considerable amount of work for everyone if people building
>> minimal images have to use a different kernel.
>
> If it is all about using kernel-minimal (or whatever it is called)
> instead of kernel there is no extra work for the ones that build
> minimal images at all.
>
>> By using the same kernel as everyone else, it means that bug reports
>> against the normal kernel package are relevant, and it means that the
>> regular kernel gets more testing.
>
> They can be built from the same srpm (same version, same patches
> applied just some modules stripped).
>
>> Also it's a lot of work to compile another kernel, when we've already
>> got a team of (apparently 3) people doing this.
>
> I'd argue it is less work then building a subpackage for every kernel module.
>
>>> use a compressed filesystem,
>>
>> Every minimal image I've ever seen has been compressed to the max.
>
> The image itself might be the installed system isn't ... which is the
> think I was talking about (i.e this would allow you to save disk space
> after installation).
> The smaller image would just save bandwidth for the initial download
> and/or space on mirrors.
>
>>> don't use a kernel at all and some
>>> container based approach instead of full virt for your cloud instances
>>
>> Unfortunately containers don't work for every application out there.
>> Obviously *if* you can use a container, then you should, and you
>> probably are already.
>
> That might be true but I can't think of such applications right now.
> Couldn't the applications you have in mind be modified / fixed to work
> in such an environment?
>
>>> Outside of the cloud use case the disk space added by modules and
>>> firmware does not matter a bit (so I am ignoring this cases).
>>
>> It's partly disk space, it's more often the time taken to copy these
>> images over the network (eg. when users download minimal images, or
>> when they are PXE-booted).
>
> In that case you should place the image on your internal network. I
> doubt anyone uses a cloud infrastructure where you download the images
> over the internet using a 56k modem.
>
>>> So there are lots of other ways to achieve what you want without
>>> splitting the kernel into hundreds of sub packages.
>>
>> I don't think we're talking "hundreds" of sub packages. Most people
>> seem to be discussing a split into between 2 and 5 packages.
>
> People where talking about splitting each module ("driver") into its
> own package. This are far more then 5.
Not necessarily, I was the person wanting it split down and only to
groups of packages like "usb" for usb devices, "arm" for arm only
firmware, "storage" for enterprise storage, "wifi" etc.
Peter
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 6:58 PM, Richard W.M. Jones wrote:
> On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote:
>> On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller
>> wrote:
>> > On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote:
>> >> > Basically: it's hard,
>> >> it is a mess.
>> >> > but the only way we're going to get to a
>> >> > reasonably-small minimal image,
>> >> not true.
>> >
>> > Given that the kernel is currently a full quarter of the current image, I
>> > think it has to be.
>>
>> No you could also use a different kernel image;
>> build your own kernel;
>
> [I'll treat these two the same, because they amount to the same thing]
No the one means we ship a kernel-foo (like kernel-minimal or whatever
we call it) and the other is you build your own kernel.
> It's a considerable amount of work for everyone if people building
> minimal images have to use a different kernel.
If it is all about using kernel-minimal (or whatever it is called)
instead of kernel there is no extra work for the ones that build
minimal images at all.
> By using the same kernel as everyone else, it means that bug reports
> against the normal kernel package are relevant, and it means that the
> regular kernel gets more testing.
They can be built from the same srpm (same version, same patches
applied just some modules stripped).
> Also it's a lot of work to compile another kernel, when we've already
> got a team of (apparently 3) people doing this.
I'd argue it is less work then building a subpackage for every kernel module.
>> use a compressed filesystem,
>
> Every minimal image I've ever seen has been compressed to the max.
The image itself might be the installed system isn't ... which is the
think I was talking about (i.e this would allow you to save disk space
after installation).
The smaller image would just save bandwidth for the initial download
and/or space on mirrors.
>> don't use a kernel at all and some
>> container based approach instead of full virt for your cloud instances
>
> Unfortunately containers don't work for every application out there.
> Obviously *if* you can use a container, then you should, and you
> probably are already.
That might be true but I can't think of such applications right now.
Couldn't the applications you have in mind be modified / fixed to work
in such an environment?
>> Outside of the cloud use case the disk space added by modules and
>> firmware does not matter a bit (so I am ignoring this cases).
>
> It's partly disk space, it's more often the time taken to copy these
> images over the network (eg. when users download minimal images, or
> when they are PXE-booted).
In that case you should place the image on your internal network. I
doubt anyone uses a cloud infrastructure where you download the images
over the internet using a 56k modem.
>> So there are lots of other ways to achieve what you want without
>> splitting the kernel into hundreds of sub packages.
>
> I don't think we're talking "hundreds" of sub packages. Most people
> seem to be discussing a split into between 2 and 5 packages.
People where talking about splitting each module ("driver") into its
own package. This are far more then 5.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: > On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller > wrote: > > On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: > >> > Basically: it's hard, > >> it is a mess. > >> > but the only way we're going to get to a > >> > reasonably-small minimal image, > >> not true. > > > > Given that the kernel is currently a full quarter of the current image, I > > think it has to be. > > No you could also use a different kernel image; > build your own kernel; [I'll treat these two the same, because they amount to the same thing] It's a considerable amount of work for everyone if people building minimal images have to use a different kernel. By using the same kernel as everyone else, it means that bug reports against the normal kernel package are relevant, and it means that the regular kernel gets more testing. Also it's a lot of work to compile another kernel, when we've already got a team of (apparently 3) people doing this. > use a compressed filesystem, Every minimal image I've ever seen has been compressed to the max. > don't use a kernel at all and some > container based approach instead of full virt for your cloud instances Unfortunately containers don't work for every application out there. Obviously *if* you can use a container, then you should, and you probably are already. > Outside of the cloud use case the disk space added by modules and > firmware does not matter a bit (so I am ignoring this cases). It's partly disk space, it's more often the time taken to copy these images over the network (eg. when users download minimal images, or when they are PXE-booted). > So there are lots of other ways to achieve what you want without > splitting the kernel into hundreds of sub packages. I don't think we're talking "hundreds" of sub packages. Most people seem to be discussing a split into between 2 and 5 packages. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, 17 Oct 2012, Dave Jones wrote: On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: > > Given that the kernel is currently a full quarter of the current image, I > > think it has to be. > > No you could also use a different kernel image; build your own kernel; > use a compressed filesystem, don't use a kernel at all and some > container based approach instead of full virt for your cloud instances > (you could even base them on a btrfs subvolume and save more space > that way). > > Outside of the cloud use case the disk space added by modules and > firmware does not matter a bit (so I am ignoring this cases). > > So there are lots of other ways to achieve what you want without > splitting the kernel into hundreds of sub packages. So while it is a > way it is not "the only way". As reluctant as I am to introduce new kernel packages, an ultra-minimal kernel package for use in cloud environments may make more sense than splitting up the one-size-fits-all packages into hundreds of sub-packages. But even this option is a lot of work, and isn't a panacea. With virtualised environments supporting pci/usb passthrough, where do you draw the line on what hardware to support in a hypothetical kernel-cloud package ? You don't. More importantly I think the goal of shrinking the minimal install is a bad one. We're chasing a size that is better suited by custom spins or specific cloud installs. It should not be a goal of a distribution provider to crreate something custom tailored for a specific environment. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: > > Given that the kernel is currently a full quarter of the current image, I > > think it has to be. > > No you could also use a different kernel image; build your own kernel; > use a compressed filesystem, don't use a kernel at all and some > container based approach instead of full virt for your cloud instances > (you could even base them on a btrfs subvolume and save more space > that way). > > Outside of the cloud use case the disk space added by modules and > firmware does not matter a bit (so I am ignoring this cases). > > So there are lots of other ways to achieve what you want without > splitting the kernel into hundreds of sub packages. So while it is a > way it is not "the only way". As reluctant as I am to introduce new kernel packages, an ultra-minimal kernel package for use in cloud environments may make more sense than splitting up the one-size-fits-all packages into hundreds of sub-packages. But even this option is a lot of work, and isn't a panacea. With virtualised environments supporting pci/usb passthrough, where do you draw the line on what hardware to support in a hypothetical kernel-cloud package ? Dave -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller wrote: > On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: >> > Basically: it's hard, >> it is a mess. >> > but the only way we're going to get to a >> > reasonably-small minimal image, >> not true. > > Given that the kernel is currently a full quarter of the current image, I > think it has to be. No you could also use a different kernel image; build your own kernel; use a compressed filesystem, don't use a kernel at all and some container based approach instead of full virt for your cloud instances (you could even base them on a btrfs subvolume and save more space that way). Outside of the cloud use case the disk space added by modules and firmware does not matter a bit (so I am ignoring this cases). So there are lots of other ways to achieve what you want without splitting the kernel into hundreds of sub packages. So while it is a way it is not "the only way". -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 11:37:29AM -0400, Josh Boyer wrote: > What the hell did you drink today, Bill? Basically what you're > suggesting is that Fedora move to a kmod model for everything. Which > means you'd have to install all of them by default anyway or the kernel > team would be swamped with bugs like "I installed Fedora and my random > USB device doesn't work!" The default desktop case would pull in everything, I think. Splitting it up would be not just for fun but in order to target smaller cloud and virt images. Because we can reasonably define the targets there, it doesn't need to be completely all-out "kmod model for everything" approach at all. > Seriously, no. There are 3 kernel people. There's no way we have time > to sit around closing bugs with stuff like "Install kernel-module-foo". Yeah; it is reasonable to say that we can't do this without more resources. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: > > Basically: it's hard, > it is a mess. > > but the only way we're going to get to a > > reasonably-small minimal image, > not true. Given that the kernel is currently a full quarter of the current image, I think it has to be. > > so if that's important (and I believe it > > is), > I believe it isn't. And that's fine. We'll need to come to a group decision eventually but we don't need to right now. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 4:51 PM, Matthew Miller wrote: > On Wed, Oct 17, 2012 at 10:47:34AM -0400, Bill Nottingham wrote: >>> If you're suggesting 1, I'd be really really opposed to that. It would >>> make packaging in kernel.spec even more of a nightmare than it already >>> is. > [...] >> Both - if people want firmware packages split out of linux-firmware, it >> doesn't make sense unless the drivers (similar in size) are also split, >> and if you were to do so, you'd need to start adding the driver -> firmware >> package dependency mapping. > > Basically: it's hard, it is a mess. > but the only way we're going to get to a > reasonably-small minimal image, not true. > so if that's important (and I believe it > is), I believe it isn't. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 10:51 AM, Matthew Miller wrote: > On Wed, Oct 17, 2012 at 10:47:34AM -0400, Bill Nottingham wrote: >>> If you're suggesting 1, I'd be really really opposed to that. It would >>> make packaging in kernel.spec even more of a nightmare than it already >>> is. > [...] >> Both - if people want firmware packages split out of linux-firmware, it >> doesn't make sense unless the drivers (similar in size) are also split, >> and if you were to do so, you'd need to start adding the driver -> firmware >> package dependency mapping. What the hell did you drink today, Bill? Basically what you're suggesting is that Fedora move to a kmod model for everything. Which means you'd have to install all of them by default anyway or the kernel team would be swamped with bugs like "I installed Fedora and my random USB device doesn't work!" Seriously, no. There are 3 kernel people. There's no way we have time to sit around closing bugs with stuff like "Install kernel-module-foo". > Basically: it's hard, but the only way we're going to get to a > reasonably-small minimal image, so if that's important (and I believe it > is), it's something Fedora should invest resources in. No. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 10:47:34AM -0400, Bill Nottingham wrote: >> If you're suggesting 1, I'd be really really opposed to that. It would >> make packaging in kernel.spec even more of a nightmare than it already >> is. [...] > Both - if people want firmware packages split out of linux-firmware, it > doesn't make sense unless the drivers (similar in size) are also split, > and if you were to do so, you'd need to start adding the driver -> firmware > package dependency mapping. Basically: it's hard, but the only way we're going to get to a reasonably-small minimal image, so if that's important (and I believe it is), it's something Fedora should invest resources in. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Josh Boyer (jwbo...@gmail.com) said: > > However, if you go down that route, the kernel should be the same way, > > the firmware should be separate subpackages, and requires should be done at > > the module -> firmware level by generating it from the MODULE_FIRMWARE tags. > > (Unless you're relying on packagekit to install your firmware, which if > > you're going that minimal seems to have missed the forest for the trees > > somewhere.) > > I'm not understanding what you're proposing. Are you suggesting: > > 1) We have further split up module sub-packages that carry their own > firmware requires (e.g. kernel-module-iwlwifi requires iwlwifi-firmware) > > or > > 2) Even more firmware subpackages split out of linux-firmware. > > If you're suggesting 1, I'd be really really opposed to that. It would > make packaging in kernel.spec even more of a nightmare than it already > is. > > If you're suggesting 2, I don't see the point. The kernel will install > and even with per-module dependencies generated (somehow), it'll still > install all of the various -firmware packages because the modules will > be getting installed. Both - if people want firmware packages split out of linux-firmware, it doesn't make sense unless the drivers (similar in size) are also split, and if you were to do so, you'd need to start adding the driver -> firmware package dependency mapping. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 17, 2012 at 02:13:35PM +0200, Lennart Poettering wrote: > This is implemented now, but I called it --since= and --until=. I'll > push this into F18 as well, sicne it's actually a minor change only, and > just too useful. Thanks Lennart. This is great stuff. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Tue, 09.10.12 23:24, Lennart Poettering (mzerq...@0pointer.de) wrote: > I am not generally against adding time-based rotation, but really, this > is much less of a "necessity" than other things the journal provides, > which syslog does not: for example per-service rate limits, and > unfakable meta-data for log messages. I mean, really, how can we ship > a syslog where every random user can fake messages, say they are from a > privileged process and offer no way how to detect that? To settle this discussion as well I've now implemented time-based rotation for the journal as well, and this will also hit F18 soonishly. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Tue, Oct 16, 2012 at 9:07 AM, Bill Nottingham wrote: > Peter Robinson (pbrobin...@gmail.com) said: >> > I wonder... could we make linux-firmware optional? >> > >> > I would expect many virt env's don't need any firmware to work... >> > (but of course I could be wrong). >> >> It use to be optional, I know on the olpc xo-1 it use to be optional >> and there should be no firmware needed for an average VM. I'd also >> love to see it broken down for various profiles because most desktops >> don't need enterprise storage controllers, most servers don't need >> wifi and most ARM platforms don't need most of the stuff in there but >> do need a few ARM only firmware packages. > > However, if you go down that route, the kernel should be the same way, > the firmware should be separate subpackages, and requires should be done at > the module -> firmware level by generating it from the MODULE_FIRMWARE tags. > (Unless you're relying on packagekit to install your firmware, which if > you're going that minimal seems to have missed the forest for the trees > somewhere.) I'm not understanding what you're proposing. Are you suggesting: 1) We have further split up module sub-packages that carry their own firmware requires (e.g. kernel-module-iwlwifi requires iwlwifi-firmware) or 2) Even more firmware subpackages split out of linux-firmware. If you're suggesting 1, I'd be really really opposed to that. It would make packaging in kernel.spec even more of a nightmare than it already is. If you're suggesting 2, I don't see the point. The kernel will install and even with per-module dependencies generated (somehow), it'll still install all of the various -firmware packages because the modules will be getting installed. Or maybe you mean something else, and my jet-lagged and coffee deprived brain just isn't following. I actually hope this is the case, because neither of the above 2 options sound that great to me... josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Thu, 11.10.12 01:48, Lennart Poettering (mzerq...@0pointer.de) wrote: > On Wed, 10.10.12 16:50, Kevin Fenzi (ke...@scrye.com) wrote: > > > "My laptop started acting up last tuesday, I should see whats in the > > logs from then" > > > > "I'd like to run a daily report on my logs" > > These two are much better implemented via explicit time seeks. The > journal APIs support that just fine, journalctl currently > doesn't. However it's trivial to add that based on the lower level APIs, > the only thing that stopped me from doing that so far is that for that > we'd have to come up with a nice way to parse calendar timestamps, and I > want to be careful about that. that said the idea is to have two command > line args to journalctl where you can pass things such as: > > $ journalctl --start-time=2012-10-01 > ... > $ journalctl --start-time=-5d > ... > $ journalctl --start-time=2012-01-01 --end-time=2012-05-02 > ... A quick update: This is implemented now, but I called it --since= and --until=. I'll push this into F18 as well, sicne it's actually a minor change only, and just too useful. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
On 10/16/2012 01:39 PM, Peter Robinson wrote: one may say "disk storage is nothing these days" iw ould say: mulitply it with 20, 50, 100 virtual machines on really expensive SAN-storage where "disk space is cheap" is not true And I would say : get an entreprisey deduping san for production under load not really a good decision even if, save ressources is always a good idea over the long That depends. The process of actually de-duping is some what intensive depending on the process but if you clone off a template it's not and you get other advantages like improved use of cache and less I/O on the disk array so it's very dependent. I generally see more improvement than loss. Moreover, I was under the impression that BTRFS is going to do de-duping internally and transparently, because it keeps block checksums anyway so detecting duplicate data comes cheap. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
>>> one may say "disk storage is nothing these days" >>> iw ould say: mulitply it with 20, 50, 100 virtual machines >>> on really expensive SAN-storage where "disk space is cheap" >>> is not true >> >> And I would say : get an entreprisey deduping san > > for production under load not really a good decision > even if, save ressources is always a good idea over > the long That depends. The process of actually de-duping is some what intensive depending on the process but if you clone off a template it's not and you get other advantages like improved use of cache and less I/O on the disk array so it's very dependent. I generally see more improvement than loss. But that is getting off topic as if there's less OS it uses less space whether it's de-duped or not and that is not a bad thing. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
Am 16.10.2012 09:19, schrieb Nicolas Mailhot: > > >> one may say "disk storage is nothing these days" >> iw ould say: mulitply it with 20, 50, 100 virtual machines >> on really expensive SAN-storage where "disk space is cheap" >> is not true > > And I would say : get an entreprisey deduping san for production under load not really a good decision even if, save ressources is always a good idea over the long signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Tue, Oct 16, 2012 at 09:07:56AM -0400, Bill Nottingham wrote: > > > I wonder... could we make linux-firmware optional? > However, if you go down that route, the kernel should be the same way, > the firmware should be separate subpackages, and requires should be done at > the module -> firmware level by generating it from the MODULE_FIRMWARE tags. [...] > I wouldn't be against that, but I also don't have the time to look at that > right now. I think this would be an excellent candidate for the Features process (and maybe F19 or F20). -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Peter Robinson (pbrobin...@gmail.com) said: > > I wonder... could we make linux-firmware optional? > > > > I would expect many virt env's don't need any firmware to work... > > (but of course I could be wrong). > > It use to be optional, I know on the olpc xo-1 it use to be optional > and there should be no firmware needed for an average VM. I'd also > love to see it broken down for various profiles because most desktops > don't need enterprise storage controllers, most servers don't need > wifi and most ARM platforms don't need most of the stuff in there but > do need a few ARM only firmware packages. However, if you go down that route, the kernel should be the same way, the firmware should be separate subpackages, and requires should be done at the module -> firmware level by generating it from the MODULE_FIRMWARE tags. (Unless you're relying on packagekit to install your firmware, which if you're going that minimal seems to have missed the forest for the trees somewhere.) I wouldn't be against that, but I also don't have the time to look at that right now. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
>> Matthew Miller (mat...@fedoraproject.org) said: >> > On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: >> > > But hey, I don't need to install packages or want python! >> > > systemd+ util-linux + bash + initscripts + passwd: >> > > Install 6 Packages (+108 Dependent packages) >> > > Total download size: 94 M >> > > Installed size: 401 M >> > >> > Of which one quarter is the kernel and the other quarter is glibc >> > locale support, right? >> >> Or more: >> >> 122659574 kernel >> 117821428 glibc-common >> 35623360linux-firmware >> 14233540coreutils >> 13845828glibc > > I wonder... could we make linux-firmware optional? > > I would expect many virt env's don't need any firmware to work... > (but of course I could be wrong). It use to be optional, I know on the olpc xo-1 it use to be optional and there should be no firmware needed for an average VM. I'd also love to see it broken down for various profiles because most desktops don't need enterprise storage controllers, most servers don't need wifi and most ARM platforms don't need most of the stuff in there but do need a few ARM only firmware packages. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 02:36:20PM -0400, john.flor...@dart.biz wrote: > > From: Bill Nottingham > > > > Jesse Keating (jkeat...@j2solutions.net) said: > > > Well, we do currently have the "minimal" environment, which boils > > > down to @core + the couple things anaconda forces (authconfig, > > > system-config-firewall-base, kernel, bootloader). You can get to > > > that via kickstart with just: > > > > > > %packages > > > @core > > > %end > > > > > > But it's not close to what some of these people want out of a > > > "minimal" install. > > > > For reference: > > > > @core + kernel: > > Install 38 Packages (+157 Dependent packages) > > Total download size: 128 M > > Installed size: 506 M > > > > But hey, I just want something smaller! > > > > systemd + util-linux + bash + initscripts + passwd + yum: > > Install 7 Packages (+132 Dependent packages) > > Total download size: 106 M > > Installed size: 446 M > > > > But hey, I don't need to install packages or want python! > > > > systemd+ util-linux + bash + initscripts + passwd: > > > > Install 6 Packages (+108 Dependent packages) > > Total download size: 94 M > > Installed size: 401 M > > Bill, thanks for that excellent report! It shows me that even if you > strip away some of the "conveniences", you really don't save that much > over our normal "minimal" install. Very enlightening. If you remove files that are also duplicated on the host (assuption: host distro + version = guest distro + version) then you can make things MUCH smaller :-) $ febootstrap --names bash systemd yum $ ll -h * -rw-rw-r--. 1 rjones rjones 4.6M Oct 16 08:44 base.img -rw-rw-r--. 1 rjones rjones 630K Oct 16 08:44 hostfiles # To reconstruct the image before boot: $ febootstrap-supermin-helper -f cpio base.img hostfiles x86_64 kernel.out initrd.out $ ll -h kernel.out initrd.out -rw-r--r--. 1 rjones rjones 293M Oct 16 08:46 initrd.out lrwxrwxrwx. 1 rjones rjones 33 Oct 16 08:46 kernel.out -> /boot/vmlinuz-3.6.1-1.fc18.x86_64 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
> one may say "disk storage is nothing these days" > iw ould say: mulitply it with 20, 50, 100 virtual machines > on really expensive SAN-storage where "disk space is cheap" > is not true And I would say : get an entreprisey deduping san -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Am 16.10.2012 01:50, schrieb Kevin Fenzi: > On Mon, 15 Oct 2012 19:11:19 -0400 > Matthew Miller wrote: > >> On Mon, Oct 15, 2012 at 03:38:36PM -0700, Adam Williamson wrote: >> I wonder... could we make linux-firmware optional? >> [...] >>> I'd agree with Harald here. A hard dep seems excessive, just >>> including the package in the @core group (so people who want >>> something *really* minimal can at least take it out) would seem >>> better. >> >> I think this is already the case, actually. I just removed it from my >> F17 test vm and there were no deps and nothing immediately broke. > > Yeah, I guess I didn't look closely... it's actually a > Requires(pre): linux-firmware >= blahblahversion > > So, once the kernel has been installed, you can safely remove it. > (If you like) correct me but this leads to re-install it on each kernel-update signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, 15 Oct 2012 19:11:19 -0400 Matthew Miller wrote: > On Mon, Oct 15, 2012 at 03:38:36PM -0700, Adam Williamson wrote: > > > >> I wonder... could we make linux-firmware optional? > [...] > > I'd agree with Harald here. A hard dep seems excessive, just > > including the package in the @core group (so people who want > > something *really* minimal can at least take it out) would seem > > better. > > I think this is already the case, actually. I just removed it from my > F17 test vm and there were no deps and nothing immediately broke. Yeah, I guess I didn't look closely... it's actually a Requires(pre): linux-firmware >= blahblahversion So, once the kernel has been installed, you can safely remove it. (If you like) kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 03:38:36PM -0700, Adam Williamson wrote: > > >> I wonder... could we make linux-firmware optional? [...] > I'd agree with Harald here. A hard dep seems excessive, just including > the package in the @core group (so people who want something *really* > minimal can at least take it out) would seem better. I think this is already the case, actually. I just removed it from my F17 test vm and there were no deps and nothing immediately broke. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, 2012-10-15 at 21:48 +0200, Reindl Harald wrote: > > Am 15.10.2012 21:43, schrieb Bill Nottingham: > > Kevin Fenzi (ke...@scrye.com) said: > >>> 122659574 kernel > >>> 117821428 glibc-common > >>> 35623360linux-firmware > >>> 14233540coreutils > >>> 13845828glibc > >> > >> I wonder... could we make linux-firmware optional? > >> > >> I would expect many virt env's don't need any firmware to work... > >> (but of course I could be wrong). > > > > "It depends". It includes firmware for wired NICs as well as other things, > > so it depends on what hardware your virtual environment is deciding to > > emulate > > surely, it depends > > but the hard dependency is a little too much I haven't seen all the arguments pro and contra, but on the face of it I'd agree with Harald here. A hard dep seems excessive, just including the package in the @core group (so people who want something *really* minimal can at least take it out) would seem better. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
Am 15.10.2012 22:02, schrieb Chris Murphy: > On Oct 15, 2012, at 1:47 PM, Matthew Miller wrote: >> With this, the three versions of "minimal" you give come down to about: >> >> @core + kernel: 300MB >> systemd [...] yum: 240MB 20% savings >> systemd + not yum: 195M 35% savings >> >> Chew away at the dependencies and at the size of some of the other packages >> (python 2to3, I'm looking at you), and we could get the middle option down >> below 200MB. > > In my opinion, Minimal should include yum. Without yum, Minimal is suddenly > not Fedora. It's like the brim is totally missing. +1 yum/rpm is the core of a linux-distribution all other dependencies in my opinion should be as less as possible - if you decide "minimal" you should be aware if some packages are missing, in the worst case the firmware of your NIC but even this is solveable easy by put it on a CD/USB or mount a ISO in a VM to get network access - on the other side if you have no firmwares, apckages you are not active using you are able to make really tight systems having 20 of them affects how large the roofs has to be for your needs x count of instalaltions in a virtual environment after the setup it affects the amount of downloads for updates * mirror load * time for the updates * possible temporary dependency-problems * space fpr full vm-images (backups) * time for restore of the backups * time for cloning test-VMs cunclusion: even in days of really large and cheap disks there are many reasons fro as small as possible system footprints and in days of more and more VM/cloud setups this becomes meaningful in many cases signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
On Oct 15, 2012, at 1:47 PM, Matthew Miller wrote: > On Mon, Oct 15, 2012 at 03:24:09PM -0400, Bill Nottingham wrote: Total download size: 94 M Installed size: 401 M >>> Of which one quarter is the kernel and the other quarter is glibc locale >>> support, right? >> Or more: >> 122659574 kernel >> 117821428 glibc-common >> 35623360linux-firmware >> 14233540coreutils >> 13845828glibc > > So, if a minimal image is a very high priority, this could be shrunk. > Handwaving aside the difficulty for a moment, if the default kernel split > out some of the drivers, maybe that could get down to 60MB. Leave out > linux-firmware And glibc-common is almost _entirely_ locale and i18n. > Because we still want to be _Fedora_, not a tiny-linux distro, let's leave > coreutils and glibc as-is. Still, though, we've shaved off 200+ MB. > > With this, the three versions of "minimal" you give come down to about: > > @core + kernel: 300MB > systemd [...] yum: 240MB 20% savings > systemd + not yum: 195M 35% savings > > Chew away at the dependencies and at the size of some of the other packages > (python 2to3, I'm looking at you), and we could get the middle option down > below 200MB. In my opinion, Minimal should include yum. Without yum, Minimal is suddenly not Fedora. It's like the brim is totally missing. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
Am 15.10.2012 21:47, schrieb Matthew Miller: > On Mon, Oct 15, 2012 at 03:24:09PM -0400, Bill Nottingham wrote: Total download size: 94 M Installed size: 401 M >>> Of which one quarter is the kernel and the other quarter is glibc locale >>> support, right? >> Or more: >> 122659574 kernel >> 117821428 glibc-common >> 35623360linux-firmware >> 14233540coreutils >> 13845828glibc > > So, if a minimal image is a very high priority, this could be shrunk. > Handwaving aside the difficulty for a moment, if the default kernel split > out some of the drivers, maybe that could get down to 60MB. Leave out > linux-firmware And glibc-common is almost _entirely_ locale and i18n. > Because we still want to be _Fedora_, not a tiny-linux distro but what speaks against sub-packages and a mtea-packge installed as default which pulls all the packages in to have the same state as now but leave the option to remove meta-package and unneeded locale to the administrator? one may say "disk storage is nothing these days" iw ould say: mulitply it with 20, 50, 100 virtual machines on really expensive SAN-storage where "disk space is cheap" is not true signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Am 15.10.2012 21:43, schrieb Bill Nottingham: > Kevin Fenzi (ke...@scrye.com) said: >>> 122659574 kernel >>> 117821428 glibc-common >>> 35623360linux-firmware >>> 14233540coreutils >>> 13845828glibc >> >> I wonder... could we make linux-firmware optional? >> >> I would expect many virt env's don't need any firmware to work... >> (but of course I could be wrong). > > "It depends". It includes firmware for wired NICs as well as other things, > so it depends on what hardware your virtual environment is deciding to > emulate surely, it depends but the hard dependency is a little too much install it as default at fedora setup is fine so such wired NICs are supported but as example on a VMware platfrom these days you are using vmxnet3 and vmw_pvscsi what means that all virtual hardware is supported from the upstream kernel signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 03:43:11PM -0400, Bill Nottingham wrote: > "It depends". It includes firmware for wired NICs as well as other things, > so it depends on what hardware your virtual environment is deciding to > emulate. Whatever hardware support is needed to run out-of-box in KVM, Xen, VirtualBox, and, sigh, VMware. If that doesn't also get us EC2 and Rackspace, make it lightweight to add whatever is needed there. That'd just be for _minimal_, of course. Default install would include everything still. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
minimal install [was Re: systemd requires HTTP server and serves QR codes]
On Mon, Oct 15, 2012 at 03:24:09PM -0400, Bill Nottingham wrote: > > > Total download size: 94 M > > > Installed size: 401 M > > Of which one quarter is the kernel and the other quarter is glibc locale > > support, right? > Or more: > 122659574 kernel > 117821428 glibc-common > 35623360linux-firmware > 14233540coreutils > 13845828glibc So, if a minimal image is a very high priority, this could be shrunk. Handwaving aside the difficulty for a moment, if the default kernel split out some of the drivers, maybe that could get down to 60MB. Leave out linux-firmware And glibc-common is almost _entirely_ locale and i18n. Because we still want to be _Fedora_, not a tiny-linux distro, let's leave coreutils and glibc as-is. Still, though, we've shaved off 200+ MB. With this, the three versions of "minimal" you give come down to about: @core + kernel: 300MB systemd [...] yum: 240MB 20% savings systemd + not yum: 195M 35% savings Chew away at the dependencies and at the size of some of the other packages (python 2to3, I'm looking at you), and we could get the middle option down below 200MB. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Kevin Fenzi (ke...@scrye.com) said: > > 122659574 kernel > > 117821428 glibc-common > > 35623360linux-firmware > > 14233540coreutils > > 13845828glibc > > I wonder... could we make linux-firmware optional? > > I would expect many virt env's don't need any firmware to work... > (but of course I could be wrong). "It depends". It includes firmware for wired NICs as well as other things, so it depends on what hardware your virtual environment is deciding to emulate. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Am 15.10.2012 21:34, schrieb Kevin Fenzi:
> On Mon, 15 Oct 2012 15:24:09 -0400
> Bill Nottingham wrote:
>
>> Matthew Miller (mat...@fedoraproject.org) said:
>>> On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote:
But hey, I don't need to install packages or want python!
systemd+ util-linux + bash + initscripts + passwd:
Install 6 Packages (+108 Dependent packages)
Total download size: 94 M
Installed size: 401 M
>>>
>>> Of which one quarter is the kernel and the other quarter is glibc
>>> locale support, right?
>>
>> Or more:
>>
>> 122659574 kernel
>> 117821428 glibc-common
>> 35623360linux-firmware
>> 14233540coreutils
>> 13845828glibc
>
> I wonder... could we make linux-firmware optional?
>
> I would expect many virt env's don't need any firmware to work...
> (but of course I could be wrong)
you are right
the dependency was introduced not so long ago
a bugreport was closed with "WONT FIX"
i went the road below
[root@buildserver:~]$ cat /rpmbuild/SPECS/linux-firmware-dummy.spec
%global checkout 06c8f81
Summary: metapackage to satisfy kernel-dependencies on vmware-servers
Name: linux-firmware-dummy
Version: 20120206
Release: 0.1.git%{checkout}%{?dist}
BuildArch: noarch
Group: System Environment/Libraries
URL: http://www.thelounge.net/
License: GPL
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
Provides: linux-firmware = %{version}
Provides: kernel-firmware = %{version}
%description
metapackage to satisfy kernel-dependencies on vmware-servers
%install
rm -rf ${RPM_BUILD_ROOT}
%files
%clean
rm -rf ${RPM_BUILD_ROOT}
%changelog
* Wed Apr 11 2012 Reindl Harald
- initial build
signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, 15 Oct 2012 15:24:09 -0400 Bill Nottingham wrote: > Matthew Miller (mat...@fedoraproject.org) said: > > On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: > > > But hey, I don't need to install packages or want python! > > > systemd+ util-linux + bash + initscripts + passwd: > > > Install 6 Packages (+108 Dependent packages) > > > Total download size: 94 M > > > Installed size: 401 M > > > > Of which one quarter is the kernel and the other quarter is glibc > > locale support, right? > > Or more: > > 122659574 kernel > 117821428 glibc-common > 35623360linux-firmware > 14233540coreutils > 13845828glibc I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Matthew Miller (mat...@fedoraproject.org) said: > On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: > > But hey, I don't need to install packages or want python! > > systemd+ util-linux + bash + initscripts + passwd: > > Install 6 Packages (+108 Dependent packages) > > Total download size: 94 M > > Installed size: 401 M > > Of which one quarter is the kernel and the other quarter is glibc locale > support, right? Or more: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc ... Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
> From: Bill Nottingham > > Jesse Keating (jkeat...@j2solutions.net) said: > > Well, we do currently have the "minimal" environment, which boils > > down to @core + the couple things anaconda forces (authconfig, > > system-config-firewall-base, kernel, bootloader). You can get to > > that via kickstart with just: > > > > %packages > > @core > > %end > > > > But it's not close to what some of these people want out of a > > "minimal" install. > > For reference: > > @core + kernel: > Install 38 Packages (+157 Dependent packages) > Total download size: 128 M > Installed size: 506 M > > But hey, I just want something smaller! > > systemd + util-linux + bash + initscripts + passwd + yum: > Install 7 Packages (+132 Dependent packages) > Total download size: 106 M > Installed size: 446 M > > But hey, I don't need to install packages or want python! > > systemd+ util-linux + bash + initscripts + passwd: > > Install 6 Packages (+108 Dependent packages) > Total download size: 94 M > Installed size: 401 M Bill, thanks for that excellent report! It shows me that even if you strip away some of the "conveniences", you really don't save that much over our normal "minimal" install. Very enlightening. -- John Florian -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: > But hey, I don't need to install packages or want python! > systemd+ util-linux + bash + initscripts + passwd: > Install 6 Packages (+108 Dependent packages) > Total download size: 94 M > Installed size: 401 M Of which one quarter is the kernel and the other quarter is glibc locale support, right? -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Jesse Keating (jkeat...@j2solutions.net) said: > Well, we do currently have the "minimal" environment, which boils > down to @core + the couple things anaconda forces (authconfig, > system-config-firewall-base, kernel, bootloader). You can get to > that via kickstart with just: > > %packages > @core > %end > > But it's not close to what some of these people want out of a > "minimal" install. For reference: @core + kernel: Install 38 Packages (+157 Dependent packages) Total download size: 128 M Installed size: 506 M But hey, I just want something smaller! systemd + util-linux + bash + initscripts + passwd + yum: Install 7 Packages (+132 Dependent packages) Total download size: 106 M Installed size: 446 M But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Oct 9, 2012, at 7:14 PM, Jesse Keating wrote: > > Anaconda isn't going to do that unless there is rpm support to re-docify > yourself. To accomplish this right now, every package would have to split > out a -docs subpackage with all the docs in it. Anaconda /might/ do what you > want in the future, by way of kickstart commands, but that's not something > we're going to expose in the UI. Infrastructure Server option comes pretty close, I think. 800MB vs 1.1G, no-GUI, but does appear to have docs. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Sat, Oct 13, 2012 at 1:36 AM, Lennart Poettering wrote: > On Fri, 12.10.12 15:29, Bill Nottingham (nott...@redhat.com) wrote: >> And we've got a lot of technology going around. journald - that's >> technology. rsyslog - that's technology. libumberlog & ceelog - that's >> technology. > > THis really makes me wonder where CEE actually belongs in this. Is > anybody using this currently? What area is this supposed to cover that > is not already covered by the journal or rsyslog? Is there really room > for another format besides BSD syslog and journal records? Given that the (udp AND tcp) syslog is the primary multi-platform log transfer protocol in the UNIX world, we need to be able to take Linux log data, including data originally generated by applications using the journal API, and transport it using the syslog protocol. To be really useful, the syslog representation needs not to loose data (e.g. only including the MESSAGE field is not good enough). So, we need a structured representation compatible with the syslog protocol in any case, and Lumberjack/CEE provide one. (And as soon as there is a structured representation compatible with syslog, it is something non-systemd platforms, like Debian or other UNIXes, can use as well.) "old rsyslog" (pre-Lumberjack/CEE) doesn't cover the structured representation requirement. journal format and protocol don't cover the syslog protocol compatibility requirement. >> If people want CEE format logs, or plain text logs, maybe journald should >> grow those as output formats. > > To me it appears that CEE isn't widely accepted so far (heck, not even > properly defined as multiple different vocabularies for fields are > floating around), and I am bit unsure where it really fits in the big > picture. I am a bit conservative in adding output formatting for CEE if > it isn't clear that there is a need for CEE, that it's going to stick > around for long and we actually have people using this. The larger vision of CEE is to have a multi-platform event dictionary and using the same event format to represent the same kind of event (so that e.g. and 'user log" can be identified and parsed without knowing what specific piece of code generated it). I'm personally not sure how much that is achievable, or how much of the problem space it can cover.[1]. In any case, complying with this would require either modification of applications, or writing a CEE-specific log message translator; it's not something we can magically get by establishing a representation or protocol, or by only converting the structure of the data that currently arrives in the journal without looking at the content. Using the Lumberjack/CEE representation natively would probably make the application modification/translator implementation simpler (e.g. the current proposals rely on nesting in the structure and other syntax that is prohibited in the journal). But as you say, these specifications are not finalized yet. Mirek [1] http://carolina.mff.cuni.cz/~trmac/blog/2011/structured-logging/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Fri, Oct 12, 2012 at 9:29 PM, Bill Nottingham wrote:
> Konstantin Ryabitsev (i...@fedoraproject.org) said:
>> So, in other words, all our existing log analysis tools have to be
>> modified if they are to be of any use in Fedora 18?
>
> Right, you'll have to port them to understand CEE from updated rsyslog. HTH,
> HAND. <- note: THIS IS A JOKE.
FWIW - the current plan for Lumberjack/CEE is to keep
/var/log/{messages,secure etc} unmodified, and store the full data in
a separate file. We can easily change this if the logging users don't
think this is the right thing to do, and users who require maximum
logging performance an obviously use a customized information - but
keeping the files and not requiring a flag day for everyone to convert
their tools immediately sounds like a good default.
Mirek
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Fri, 12.10.12 15:29, Bill Nottingham (nott...@redhat.com) wrote: Heya, > And we've got a lot of technology going around. journald - that's > technology. rsyslog - that's technology. libumberlog & ceelog - that's > technology. THis really makes me wonder where CEE actually belongs in this. Is anybody using this currently? What area is this supposed to cover that is not already covered by the journal or rsyslog? Is there really room for another format besides BSD syslog and journal records? So, what's our story here with CEE? > If people want CEE format logs, or plain text logs, maybe journald should > grow those as output formats. To me it appears that CEE isn't widely accepted so far (heck, not even properly defined as multiple different vocabularies for fields are floating around), and I am bit unsure where it really fits in the big picture. I am a bit conservative in adding output formatting for CEE if it isn't clear that there is a need for CEE, that it's going to stick around for long and we actually have people using this. > Or maybe rsyslog should produce those formats. Maybe rsyslog should > grow a journald plugin, so instead of duplicating some of journald's > code for associating entries with pid/exec/etc., it can read the > already annotated journal stream and add its own metadata & spit out > whatever formats it wants. (Maybe it already does this!) Yes, this would certainly be useful. If rsyslog wants access to the full data stream systemd generates then using our C APIs is a good choice, it will get all meta data, and can process them the way they want. > Maybe rsyslog or journald should take over audit logging in some way. Since the audit logs contain a lot of useful data we definitely want to acquire auditing as another input for the journal. In fact, Eric has been working on kernel support to allow the journal to get a copy of the audit stream without interfering with auditd. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
Konstantin Ryabitsev (i...@fedoraproject.org) said: > > Not sure I can parse this, but IIUC you are wondering whether logwatch > > is compatible with the journal. Not to my knowledge, no. But adding this > > should be fairly easy as the output of "journalctl" is a pixel-perfect > > copy of the original format, so where it works on /var/log/messages it > > should simply work on the output of journalctl and all should be good. > > > > Note however that with the capabilities of the journal it might be > > interesting to add journal support to logwatch that goes beyond mere > > compatibility. For example, tests such as "look for messages which are > > claimed to come from PID xyz but actually came from uvw" and suchlike > > would be really interesting to have. That information is not available > > in the /var/log/messages format however... > > So, in other words, all our existing log analysis tools have to be > modified if they are to be of any use in Fedora 18? Right, you'll have to port them to understand CEE from updated rsyslog. HTH, HAND. <- note: THIS IS A JOKE. MORE SERIOUSLY There are a lot of usage cases that people want from their logging. 1) Administrators want their plain-text logs that they know and love (or at least know and have gotten accustomed to) that they can use their normal unix tools and their homegrown custom shell/awk/perl/python/whatever scripts for parsing. (For the purposes of this discussion, consider logwatch one of those homegrown things, as it basically is that writ large.) 2) System management authors would love to have a mechanism where they can subscribe to particular alerts as they come in, without having to subscribe to all messages, or try and parse the unstructured text of syslog 3) Application developers might want to be able to express stuff they log in a more structured fashion rather than just: "(function:line) bad juju happened here in frobnitz" 4) Administrators want to be able to do things like 'show me everything sshd did/logged about', or 'show me what happened last Thursday, because I can never get the hang of them.' 5) Standards People want to have messages in the new CEE format, so they can use their new CEE tools on them and merge some of their homegrown tools. 6) Meanwhile, you've got the poor audit logger over there on the side doing its own thing, and there are users who Really Like those audit logs. And we've got a lot of technology going around. journald - that's technology. rsyslog - that's technology. libumberlog & ceelog - that's technology. What we've got to do is take the usage cases we have, and the technology we have, and get a coherent solution that covers them. And it's certainly not clear at this point what that solution would be. If people want CEE format logs, or plain text logs, maybe journald should grow those as output formats. Or maybe rsyslog should produce those formats. Maybe rsyslog should grow a journald plugin, so instead of duplicating some of journald's code for associating entries with pid/exec/etc., it can read the already annotated journal stream and add its own metadata & spit out whatever formats it wants. (Maybe it already does this!) Maybe rsyslog or journald should take over audit logging in some way. But the point is, there's a lot of work in this space going on on all sides (take ceelog, liumberlog, and journald - all relatively new bits of technology touching portions of this space). And at this point I'd say it's way too early to say that Fedora Shall Be XYZ, or to conversly say that Fedora Shall Not Be XYZ. A full plan for hitting all the usage cases we might want just isn't known. (Although it would be a lot easier to get there if y'all would stop shouting AT & PAST each other.) So no, you don't need to change anything for Fedora 18. rsyslog is there by default, journald is there too if you want to look at that. And until we actually have a Plan, rather than just Technology, I'm not sure why you'd say that Fedora will do XYZ in F-19 either. Well, you can probably say that Fedora 19 won't ship with sysklogd by default; that should be safe. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]
On 10/9/12 12:34 PM, Adam Jackson wrote: On 10/9/12 9:18 AM, Lennart Poettering wrote: From the list of packages this minimal set still installs, that I'd really like to see gone: chkconfig gamin info systemd-sysv chkconfig seems like it could have the 'alternatives' bit split off. I've not investigated this in detail. Took a look this morning, it appears alternatives is almost completely orthogonal to the rest of the package. Naturally the bit that ruins everything is translations since the locale data is all mushed together among alternatives chkconfig and ntsysv, but that's straightforward to detangle. (Not that I've done so.) That'd drop the size on disk from ~700k for chkconfig to probably around 400k for just alternatives. Not a huge win, but not terrible either, and certainly Correct for a systemd world. Bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=865496 Better, though, is looking at what's pulling it in, a Requires(post) in openssh-server (actually for %triggerun and %triggerpostun, but rpm doesn't know how to Requires for those), and in particular for upgrading from versions of openssh-server older than what shipped in F16 Gold. And, the scriptlets are properly immunized against /sbin/chkconfig not existing, which means strictly we could drop the Requires(post) for it already. Bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=865498 - ajax -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 08:16:49AM -0500, Michael Cronenworth wrote: > Alexander Larsson wrote: > > Honestly, we should be building glib2 with --disable-fam, since glib > > will prefer the inotify notification module anyway (it has prio 20 and > > fam prio 10). > > It looks[1] like Matthias was watching this thread. Yay! > > [1] > http://pkgs.fedoraproject.org/cgit/glib2.git/commit/?id=3d798c3fdcbab4255323c570fcfc0090bd2980df Let's celebrate one less use of gamin ! Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ dan...@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 08:43:22AM -0400, Matthew Miller wrote: > On Wed, Oct 10, 2012 at 07:17:58PM +0800, Daniel Veillard wrote: > > > libxml2 takes up 5.2M, of which 3.8M is docs > > It really should go in -devel, I agree ! > > Check it out -- we've accomplished something with this thread. :) Done, fixed in rawhide, base package install size is now down to 1.6 MB, the docs were already in the -devel package so I kept them there. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ dan...@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Thu, 11 Oct 2012 01:48:07 +0200 Lennart Poettering wrote: > These two are much better implemented via explicit time seeks. The > journal APIs support that just fine, journalctl currently > doesn't. However it's trivial to add that based on the lower level > APIs, the only thing that stopped me from doing that so far is that > for that we'd have to come up with a nice way to parse calendar > timestamps, and I want to be careful about that. that said the idea > is to have two command line args to journalctl where you can pass > things such as: > > $ journalctl --start-time=2012-10-01 > ... > $ journalctl --start-time=-5d > ... > $ journalctl --start-time=2012-01-01 --end-time=2012-05-02 > ... > > And this would do the right things. Since the journal will coalesce > the current journal and the rotated ones into one this will simply > show you everything that matches. Sounds great. > Of course the time expressions for this need to be powerful enough so > that people can trivially express things like "everything from today", > or "everything since two weeks ago" and suchlike. Yeah, I am reminded (pardon the pun) of the 'remind' program that did this very well. > > "This thing might have messed up when I last booted... uptime shows > > 16 days" > > For this we already have "journalctl -b" which only shows messages > from the current boot. We'll probably extend that later so that you > can pass "journalctl -b4" or so which would show you the messages > from 4 boots earlier only. Excellent. > The takeaway here is that rotation is not a feature for finding > things. There are much better ways to find things and we should make > them available, and we can, because the backend allows that. Right, which is why I was trying to move to use cases over just asking for time rotation. ;) If these use cases can be solved better ways, thats just fine with me. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 2012-10-10 at 14:37 -0400, Konstantin Ryabitsev wrote: > On Wed, Oct 10, 2012 at 2:32 PM, Lennart Poettering > wrote: > >> Can journalctl send the logs via logwatch? > > > > Not sure I can parse this, but IIUC you are wondering whether logwatch > > is compatible with the journal. Not to my knowledge, no. But adding this > > should be fairly easy as the output of "journalctl" is a pixel-perfect > > copy of the original format, so where it works on /var/log/messages it > > should simply work on the output of journalctl and all should be good. > > > > Note however that with the capabilities of the journal it might be > > interesting to add journal support to logwatch that goes beyond mere > > compatibility. For example, tests such as "look for messages which are > > claimed to come from PID xyz but actually came from uvw" and suchlike > > would be really interesting to have. That information is not available > > in the /var/log/messages format however... > > So, in other words, all our existing log analysis tools have to be > modified if they are to be of any use in Fedora 18? The signal seems to have been lost somewhere along the path, so apart from anything else, no, because rsyslog is still installed by default in F18, and the systemd journal doesn't do permanent logging by default (/var/log/journal does not exist). rsyslog is still the primary system logging mechanism in F18 and that is not going to change (he said meaningfully, targeting the QA orbital laser on Lennart's home address.) There is a proposal in this thread to stop installing rsyslog by default and enable permanent logging by journal in F19, but that's just a proposal so far, and does not affect F18. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Thu, Oct 11, 2012 at 01:48:07AM +0200, Lennart Poettering wrote: > > "My laptop started acting up last tuesday, I should see whats in the > > logs from then" > > "I'd like to run a daily report on my logs" > These two are much better implemented via explicit time seeks. The > journal APIs support that just fine, journalctl currently > doesn't. However it's trivial to add that based on the lower level APIs, > the only thing that stopped me from doing that so far is that for that > we'd have to come up with a nice way to parse calendar timestamps, and I > want to be careful about that. that said the idea is to have two command > line args to journalctl where you can pass things such as: Not coincidentially, I filed an RFE bug for this yesterday: https://bugzilla.redhat.com/show_bug.cgi?id=864672 > Of course the time expressions for this need to be powerful enough so > that people can trivially express things like "everything from today", > or "everything since two weeks ago" and suchlike. +1 awesome. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 16:50, Kevin Fenzi (ke...@scrye.com) wrote: > "My laptop started acting up last tuesday, I should see whats in the > logs from then" > > "I'd like to run a daily report on my logs" These two are much better implemented via explicit time seeks. The journal APIs support that just fine, journalctl currently doesn't. However it's trivial to add that based on the lower level APIs, the only thing that stopped me from doing that so far is that for that we'd have to come up with a nice way to parse calendar timestamps, and I want to be careful about that. that said the idea is to have two command line args to journalctl where you can pass things such as: $ journalctl --start-time=2012-10-01 ... $ journalctl --start-time=-5d ... $ journalctl --start-time=2012-01-01 --end-time=2012-05-02 ... And this would do the right things. Since the journal will coalesce the current journal and the rotated ones into one this will simply show you everything that matches. Of course the time expressions for this need to be powerful enough so that people can trivially express things like "everything from today", or "everything since two weeks ago" and suchlike. > "This thing might have messed up when I last booted... uptime shows 16 > days" For this we already have "journalctl -b" which only shows messages from the current boot. We'll probably extend that later so that you can pass "journalctl -b4" or so which would show you the messages from 4 boots earlier only. The takeaway here is that rotation is not a feature for finding things. There are much better ways to find things and we should make them available, and we can, because the backend allows that. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On 10/09/2012 07:25 PM, Simo Sorce wrote: Can't you just you reinstall a package without the nodocs switch/conf in place to get the docs land on disk ? You probably also have to skip the scripts, which can have some unintended consequences. Also it means downloading the entire package set, not just the ones with docs. And it means hoping all the packages you've installed are still available in whatever source you installed them from. Anyway, it's just not something I'd feel comfortable exposing in the anaconda UI. -- Jesse Keating Fedora -- Freedom² is a feature! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
Once upon a time, Matthew Miller said: > On Wed, Oct 10, 2012 at 02:44:53PM -0400, Konstantin Ryabitsev wrote: > > Well, hang on, Kay. My understanding was that we're trying to make > > syslog an optional install in Fedora 18 (or is it 19?). If that is the > > The suggestion was to propose this as a feature for F19. I think there's > some additional basic functionality we really need in place before that > would be ready. One additional thing related to log analysis: I have some logs that are owned by different groups, and analysis tools that run under user accounts (for example CGIs scanning for certain types of errors). How does that work with journald? -- Chris Adams Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10 Oct 2012 22:02:26 +0200 Kay Sievers wrote: > On Wed, Oct 10, 2012 at 9:49 PM, Simo Sorce wrote: ...snip... > > So make it really better and support time-based rotation. You don't > > need to make time-based rotation the default, but you'll make a lot > > of people happy to have the option. > > I really don't mind someone implementing a "maximum retention policy" > for the journal, surely sounds useful for some setups, but I'm > personally not really interested in implementing it. Note there are more use cases than a "retention policy" type thing in having time based log rotation. "My laptop started acting up last tuesday, I should see whats in the logs from then" a) search each rotated journal file until you find last tuesday. or b) run journalctl on last tuesdays log since it was rotated daily and you can clearly see what one is tuesdays. "I'd like to run a daily report on my logs" a) journalctl out the journal, figure out when the last day started, cut things before that out. or b) journalctl after the daily rotate on the previous days journal. "This thing might have messed up when I last booted... uptime shows 16 days" a) Figure out what journal was from 16 days ago by hunting around. or b) journalctl out the one from 16 days ago kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Oct 10, 2012, at 2:54 PM, Lennart Poettering wrote: > On Wed, 10.10.12 14:39, Chris Murphy (li...@colorremedies.com) wrote: > >> How is rsyslog properly disabled? >> >> sockets.target syslog.target rsyslog.service all seem related. > > "systemctl disable rsyslog.service" should suffice. I did that and now syslog.socket is angry, or at least its status is failed. I'm not sure if it's related, and if it's merely cosmetic. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 14:39, Chris Murphy (li...@colorremedies.com) wrote: > > On Oct 10, 2012, at 2:02 PM, Kay Sievers wrote: > > > Syslog is by fact today already an "add-on", and not a > > required component, it is just installed by default today. I don't use > > or run syslog on any of my boxes since quite a while. > > How is rsyslog properly disabled? > > sockets.target syslog.target rsyslog.service all seem related. "systemctl disable rsyslog.service" should suffice. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 22:19, Tomasz Torcz (to...@pipebreaker.pl) wrote: > On Wed, Oct 10, 2012 at 03:49:11PM -0400, Simo Sorce wrote: > > So make it really better and support time-based rotation. You don't need > > to make time-based rotation the default, but you'll make a lot of people > > happy to have the option. > > Journald will rotate logs when signalled with SIGUSR2. So you need > something > like “systemctl kill --signal=USR2 systemd-journald.service” executed by cron > or from .timer unit. Note that this will not really implement something that would be useful for data retention policy enforcement. Sending USR2 will cause journald to rotate the files, but not delete more than necessary to fulfill the disk space limits. To enforce data retention policy enforcement we need to bump this logic up to delete all journal files which contain entries older than a specific time. Implementing this is actually not hard... happy to take patches. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, 10.10.12 21:06, Richard W.M. Jones (rjo...@redhat.com) wrote: > On Tue, Oct 09, 2012 at 10:58:45AM +, "Jóhann B. Guðmundsson" wrote: > > Like to me rsyslog since the journal is an integrated part of systemd. > > Leaving aside the merits or otherwise of the journal, why does it need > to be part of systemd? Why not have it as a separate project? They are tightly integrated. The journal can run very early at boot, where very little else runs and integrates closely with systemd for that. Also, systemd itself generates a lot of events for it, and queries it too when necessary, for example in the "systemctl status" output. The journal is implicitly augmented with systemd-specific data, for example with fields declaring which systemd service is logging. Journald also is responsible for handling stdout/stderr of all running services, and for that is a dependency of the systemd process spawning logic. Hence journald depends on systemd, and systemd on journald, and separating makes no sense. In other words: systemd and journald are separate enough from each other to make them two processes, but way too integrated to separate them from each other entirely and allow one running without the other. > The init system and logging have always been two different things. Yes, but this is hardly a reason to keep it that way. We have good technical reasons to integrate them. I believe logging is an absolutely essential part of service supervision. As such I believe no service supervisor could ever be complete without tight integration with the logging framework. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Oct 10, 2012, at 2:02 PM, Kay Sievers wrote: > Syslog is by fact today already an "add-on", and not a > required component, it is just installed by default today. I don't use > or run syslog on any of my boxes since quite a while. How is rsyslog properly disabled? sockets.target syslog.target rsyslog.service all seem related. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 10:06 PM, Richard W.M. Jones wrote: > On Tue, Oct 09, 2012 at 10:58:45AM +, "Jóhann B. Guðmundsson" wrote: >> Like to me rsyslog since the journal is an integrated part of systemd. > > Leaving aside the merits or otherwise of the journal, why does it need > to be part of systemd? Why not have it as a separate project? > (Perhaps requiring systemd, or as a plugin of systemd if it's really > needed.) > > The init system and logging have always been two different things. And init had never any idea what a service was doing after it double-forked, it could never monitor it, could not tell much about its current state, had no history about the behaviour, could not even safely shut it down. Systemd is a real service babysitter, and tracks everything across the entire life time of all services; the logs are just integral part of systemd's job. It also provides the kernel and userspace early boot logging support, and the out-of-the-box service stdout/stderr logging support; that's why the journal daemon is mandatory and not an add-on. Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 09:06:56PM +0100, Richard W.M. Jones wrote: > On Tue, Oct 09, 2012 at 10:58:45AM +, "Jóhann B. Guðmundsson" wrote: > > Like to me rsyslog since the journal is an integrated part of systemd. > > Leaving aside the merits or otherwise of the journal, why does it need > to be part of systemd? Why not have it as a separate project? > (Perhaps requiring systemd, or as a plugin of systemd if it's really > needed.) > > The init system and logging have always been two different things. The init system, sure. But what about platform-defining (and Linux-defining) central daemon? We did not have such thing before. -- Tomasz Torcz Morality must always be based on practicality. xmpp: zdzich...@chrome.pl-- Baron Vladimir Harkonnen -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 03:49:11PM -0400, Simo Sorce wrote: > So make it really better and support time-based rotation. You don't need > to make time-based rotation the default, but you'll make a lot of people > happy to have the option. Journald will rotate logs when signalled with SIGUSR2. So you need something like “systemctl kill --signal=USR2 systemd-journald.service” executed by cron or from .timer unit. BTW, .timer units will grow calendar scheduling in future, so cron will go after rsyslog, too. (Johann, I've stolen your idea ;) -- Tomasz Torcz Morality must always be based on practicality. xmpp: zdzich...@chrome.pl-- Baron Vladimir Harkonnen -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 3:44 PM, Kay Sievers wrote: >> I think you overestimate how much a sysadmin cares about fake >> messages. The thing that's really important to a sysadmin is to make >> sure that none of the REAL messages are lost. If someone fakes root >> login entries by using something as trivial as "logger", I can easily >> establish they are fake by looking at auditd logs. And then I would >> *really* make that user regret their actions by using blunt >> cryptanalysis tools. >> >> So, it's not accurate to say that we don't currently have ways to detect >> that. > > That works only for very very few of the logged messages, and it is a > good example how things should really not be designed or work today. Yeah, I wasn't saying it's a stellar system, but it is well-understood by sysadmins -- syslog messages are "discretionary logging" vs. auditd messages, which are "compulsory syscall logging." I monitor the former, since it's my first-line alert system for something strange going on, but I certainly don't rely solely on syslog for forensics. > We need one source of system log and not a bunch of daemons with all > overlap but still have only parts of the picture, store their own > stuff all over the place. Well, the counter-argument is that we also don't want to put all our proverbial eggs in one basket. I was kinda fond of not mixing discretionary free-for-all "I-think-I-just-burped" random junk that ends up in syslog from hard auditd data. My favourite was always seeing syslog entries in other languages if workstation user happened to select something other than "English" for their desktop. > Manual matching between the different data sources can sometimes be > used to find out what was really going on, but that's really not good > enough today. It is nearly always inevitable, especially in large heterogeneous environments. I've done quite a few forensic analyses in the past and you always have to correlate logs from multiple sources. You'll have Apache log files, PHP error log files, database log files, FTP log files, etc. I'm not even sure I want to put it all into journal -- and a lot of it can't go into journal for various reasons. Apache can either log to syslog or to a file, unless you do some horrible magic with piping it to tee and logger. Not saying that the situation won't be improved with journal, but it will have less of an impact on "real" people for whom log analysis and correlation is bread-and-butter. > The journal daemon uses similar close-to-the-kernel properties to > establish trust in logged messages, and in the future it is planned > that it will also rad all audit messages directly. The audit daemon > will then mostly be a policy execution engine for (rather exotic) > requirements like "crash the box if the message does not go to disk". I'm not sure anyone actually cares to join the two, honestly. Ausearch and aureport are well understood and cherished by (admittedly few) people that know what they do. Best, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Tue, Oct 09, 2012 at 10:58:45AM +, "Jóhann B. Guðmundsson" wrote: > Like to me rsyslog since the journal is an integrated part of systemd. Leaving aside the merits or otherwise of the journal, why does it need to be part of systemd? Why not have it as a separate project? (Perhaps requiring systemd, or as a plugin of systemd if it's really needed.) The init system and logging have always been two different things. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 9:49 PM, Simo Sorce wrote: > On Wed, 2012-10-10 at 21:44 +0200, Kay Sievers wrote: >> On Wed, Oct 10, 2012 at 9:31 PM, Konstantin Ryabitsev >> wrote: >> > On Tue, Oct 9, 2012 at 5:24 PM, Lennart Poettering >> > wrote: >> >> I am not generally against adding time-based rotation, but really, this >> >> is much less of a "necessity" than other things the journal provides, >> >> which syslog does not: for example per-service rate limits, and >> >> unfakable meta-data for log messages. I mean, really, how can we ship >> >> a syslog where every random user can fake messages, say they are from a >> >> privileged process and offer no way how to detect that? >> > >> > I think you overestimate how much a sysadmin cares about fake >> > messages. The thing that's really important to a sysadmin is to make >> > sure that none of the REAL messages are lost. If someone fakes root >> > login entries by using something as trivial as "logger", I can easily >> > establish they are fake by looking at auditd logs. And then I would >> > *really* make that user regret their actions by using blunt >> > cryptanalysis tools. >> > >> > So, it's not accurate to say that we don't currently have ways to detect >> > that. >> >> That works only for very very few of the logged messages, and it is a >> good example how things should really not be designed or work today. >> >> We need one source of system log and not a bunch of daemons with all >> overlap but still have only parts of the picture, store their own >> stuff all over the place. >> >> Manual matching between the different data sources can sometimes be >> used to find out what was really going on, but that's really not good >> enough today. >> >> The journal daemon uses similar close-to-the-kernel properties to >> establish trust in logged messages, and in the future it is planned >> that it will also rad all audit messages directly. The audit daemon >> will then mostly be a policy execution engine for (rather exotic) >> requirements like "crash the box if the message does not go to disk". > > It seem your intention is to make the journal so much better that it > will be the preferred choice (and indeed the default). The journal is nothing really to choose, it's a mandatory core part of the operating system, systemd needs it itself, and it always runs. A running syslog daemon always gets its data forwarded only from the journal daemon. Syslog is by fact today already an "add-on", and not a required component, it is just installed by default today. I don't use or run syslog on any of my boxes since quite a while. > So make it really better and support time-based rotation. You don't need > to make time-based rotation the default, but you'll make a lot of people > happy to have the option. I really don't mind someone implementing a "maximum retention policy" for the journal, surely sounds useful for some setups, but I'm personally not really interested in implementing it. Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 2012-10-10 at 21:44 +0200, Kay Sievers wrote: > On Wed, Oct 10, 2012 at 9:31 PM, Konstantin Ryabitsev > wrote: > > On Tue, Oct 9, 2012 at 5:24 PM, Lennart Poettering > > wrote: > >> I am not generally against adding time-based rotation, but really, this > >> is much less of a "necessity" than other things the journal provides, > >> which syslog does not: for example per-service rate limits, and > >> unfakable meta-data for log messages. I mean, really, how can we ship > >> a syslog where every random user can fake messages, say they are from a > >> privileged process and offer no way how to detect that? > > > > I think you overestimate how much a sysadmin cares about fake > > messages. The thing that's really important to a sysadmin is to make > > sure that none of the REAL messages are lost. If someone fakes root > > login entries by using something as trivial as "logger", I can easily > > establish they are fake by looking at auditd logs. And then I would > > *really* make that user regret their actions by using blunt > > cryptanalysis tools. > > > > So, it's not accurate to say that we don't currently have ways to detect > > that. > > That works only for very very few of the logged messages, and it is a > good example how things should really not be designed or work today. > > We need one source of system log and not a bunch of daemons with all > overlap but still have only parts of the picture, store their own > stuff all over the place. > > Manual matching between the different data sources can sometimes be > used to find out what was really going on, but that's really not good > enough today. > > The journal daemon uses similar close-to-the-kernel properties to > establish trust in logged messages, and in the future it is planned > that it will also rad all audit messages directly. The audit daemon > will then mostly be a policy execution engine for (rather exotic) > requirements like "crash the box if the message does not go to disk". It seem your intention is to make the journal so much better that it will be the preferred choice (and indeed the default). So make it really better and support time-based rotation. You don't need to make time-based rotation the default, but you'll make a lot of people happy to have the option. Simo. -- Simo Sorce * Red Hat, Inc * New York -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 9:31 PM, Konstantin Ryabitsev wrote: > On Tue, Oct 9, 2012 at 5:24 PM, Lennart Poettering > wrote: >> I am not generally against adding time-based rotation, but really, this >> is much less of a "necessity" than other things the journal provides, >> which syslog does not: for example per-service rate limits, and >> unfakable meta-data for log messages. I mean, really, how can we ship >> a syslog where every random user can fake messages, say they are from a >> privileged process and offer no way how to detect that? > > I think you overestimate how much a sysadmin cares about fake > messages. The thing that's really important to a sysadmin is to make > sure that none of the REAL messages are lost. If someone fakes root > login entries by using something as trivial as "logger", I can easily > establish they are fake by looking at auditd logs. And then I would > *really* make that user regret their actions by using blunt > cryptanalysis tools. > > So, it's not accurate to say that we don't currently have ways to detect that. That works only for very very few of the logged messages, and it is a good example how things should really not be designed or work today. We need one source of system log and not a bunch of daemons with all overlap but still have only parts of the picture, store their own stuff all over the place. Manual matching between the different data sources can sometimes be used to find out what was really going on, but that's really not good enough today. The journal daemon uses similar close-to-the-kernel properties to establish trust in logged messages, and in the future it is planned that it will also rad all audit messages directly. The audit daemon will then mostly be a policy execution engine for (rather exotic) requirements like "crash the box if the message does not go to disk". Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Tue, Oct 9, 2012 at 5:24 PM, Lennart Poettering wrote: > I am not generally against adding time-based rotation, but really, this > is much less of a "necessity" than other things the journal provides, > which syslog does not: for example per-service rate limits, and > unfakable meta-data for log messages. I mean, really, how can we ship > a syslog where every random user can fake messages, say they are from a > privileged process and offer no way how to detect that? I think you overestimate how much a sysadmin cares about fake messages. The thing that's really important to a sysadmin is to make sure that none of the REAL messages are lost. If someone fakes root login entries by using something as trivial as "logger", I can easily establish they are fake by looking at auditd logs. And then I would *really* make that user regret their actions by using blunt cryptanalysis tools. So, it's not accurate to say that we don't currently have ways to detect that. Regards, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 9:01 PM, Matthew Miller wrote: > Additionally, it _would_ be cool for log monitoring and analysis tools to > gain journald support, so that users of those tools can take advantage of > all the features Lennart lists. If we could have some of those in place > along with the proposed feature, that would be a win. Along with the ability to retrieve data from the journal, tools should probably start at the same time to support real message ids. They will allow us reliable recognition without weird regex matches in human readable syslog lines, allow catalogization of messages, documentation, metadata handling, or even localization. What we have in systemd so far is: http://cgit.freedesktop.org/systemd/systemd/tree/src/systemd/sd-messages.h We also have proper identifiers for devices/hardware in the kernel logs now. The journal reads them already and connects them to the current udev supplied data. These identifiers should also be used to identify a device instead of the unreliable guessing of strings in human readable syslog messages: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 2012-10-10 at 15:01 -0400, Matthew Miller wrote: > On Wed, Oct 10, 2012 at 02:44:53PM -0400, Konstantin Ryabitsev wrote: > > case, then even if I require rsyslog for a package, that won't work > > unless rsyslog is started and running. So, sysadmin's experience > > changes: > > Was: Install logwatch. > > Becomes: Install logwatch. Make sure you install and enable rsyslog. > > I just want to make sure people are aware of the change. > > Well, we've got: http://fedoraproject.org/wiki/Features/PackagePresets and > it seems like we could probably come up with a preset selection for > non-desktop system use. I'd say "server-presets", except it goes beyond > server, of course. But yeah, we'd need to make that easy -- a list of "now Then call it unix-presets perhaps? > you get to jump through these hoops because we've made things better!" won't > make anyone happy with us. We are just dropping another part of the UNIX API - this time the system logs. Who cares? (I do.) -- Tomas Mraz No matter how far down the wrong road you've gone, turn back. Turkish proverb -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 3:01 PM, Matthew Miller wrote: > Additionally, it _would_ be cool for log monitoring and analysis tools to > gain journald support, so that users of those tools can take advantage of > all the features Lennart lists. If we could have some of those in place > along with the proposed feature, that would be a win. Hint-hint, nudge-nudge? :) I'm not sure I can swing that, unfortunately. :( But I certainly am interested in seeing where journal is headed, as it improves a lot of aspects of log management that log analysis tools have to work around. Regards, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 02:44:53PM -0400, Konstantin Ryabitsev wrote: > Well, hang on, Kay. My understanding was that we're trying to make > syslog an optional install in Fedora 18 (or is it 19?). If that is the The suggestion was to propose this as a feature for F19. I think there's some additional basic functionality we really need in place before that would be ready. > case, then even if I require rsyslog for a package, that won't work > unless rsyslog is started and running. So, sysadmin's experience > changes: > Was: Install logwatch. > Becomes: Install logwatch. Make sure you install and enable rsyslog. > I just want to make sure people are aware of the change. Well, we've got: http://fedoraproject.org/wiki/Features/PackagePresets and it seems like we could probably come up with a preset selection for non-desktop system use. I'd say "server-presets", except it goes beyond server, of course. But yeah, we'd need to make that easy -- a list of "now you get to jump through these hoops because we've made things better!" won't make anyone happy with us. Additionally, it _would_ be cool for log monitoring and analysis tools to gain journald support, so that users of those tools can take advantage of all the features Lennart lists. If we could have some of those in place along with the proposed feature, that would be a win. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 8:44 PM, Konstantin Ryabitsev wrote: > On Wed, Oct 10, 2012 at 2:39 PM, Kay Sievers wrote: >>> So, in other words, all our existing log analysis tools have to be >>> modified if they are to be of any use in Fedora 18? >> >> What part of "Run the syslog daemon like you always did, if you need >> syslog files." did you not understand? > > Well, hang on, Kay. My understanding was that we're trying to make > syslog an optional install in Fedora 18 (or is it 19?). Surely not f18, and there is not even a feature for f19 as of now. > If that is the > case, then even if I require rsyslog for a package, that won't work > unless rsyslog is started and running. Services can pull-in service dependencies to start stuff they depend on, it's unreleated RPM dependencies. > So, sysadmin's experience > changes: > > Was: Install logwatch. > Becomes: Install logwatch. Make sure you install and enable rsyslog. > > I just want to make sure people are aware of the change. Ah, sorry that I was just unable to translate: "all our existing log analysis tools have to be modified if they are to be of any use in Fedora" to "just want to make sure ... you install and enable rsyslog". :) Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 2:39 PM, Kay Sievers wrote: >> So, in other words, all our existing log analysis tools have to be >> modified if they are to be of any use in Fedora 18? > > What part of "Run the syslog daemon like you always did, if you need > syslog files." did you not understand? Well, hang on, Kay. My understanding was that we're trying to make syslog an optional install in Fedora 18 (or is it 19?). If that is the case, then even if I require rsyslog for a package, that won't work unless rsyslog is started and running. So, sysadmin's experience changes: Was: Install logwatch. Becomes: Install logwatch. Make sure you install and enable rsyslog. I just want to make sure people are aware of the change. Best, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10 Oct 2012, Kay Sievers wrote: So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? What part of "Run the syslog daemon like you always did, if you need syslog files." did you not understand? Kay, This is not an acceptable tone. There is no need for this sort of sarcasm or snark. Please amend this in the future. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 02:37:05PM -0400, Konstantin Ryabitsev wrote: > So, in other words, all our existing log analysis tools have to be > modified if they are to be of any use in Fedora 18? No, not in the even slightest. I don't think that's even up for discussion. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 8:37 PM, Konstantin Ryabitsev wrote: > On Wed, Oct 10, 2012 at 2:32 PM, Lennart Poettering > wrote: >>> Can journalctl send the logs via logwatch? >> >> Not sure I can parse this, but IIUC you are wondering whether logwatch >> is compatible with the journal. Not to my knowledge, no. But adding this >> should be fairly easy as the output of "journalctl" is a pixel-perfect >> copy of the original format, so where it works on /var/log/messages it >> should simply work on the output of journalctl and all should be good. >> >> Note however that with the capabilities of the journal it might be >> interesting to add journal support to logwatch that goes beyond mere >> compatibility. For example, tests such as "look for messages which are >> claimed to come from PID xyz but actually came from uvw" and suchlike >> would be really interesting to have. That information is not available >> in the /var/log/messages format however... > > So, in other words, all our existing log analysis tools have to be > modified if they are to be of any use in Fedora 18? What part of "Run the syslog daemon like you always did, if you need syslog files." did you not understand? Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 2:32 PM, Lennart Poettering wrote: >> Can journalctl send the logs via logwatch? > > Not sure I can parse this, but IIUC you are wondering whether logwatch > is compatible with the journal. Not to my knowledge, no. But adding this > should be fairly easy as the output of "journalctl" is a pixel-perfect > copy of the original format, so where it works on /var/log/messages it > should simply work on the output of journalctl and all should be good. > > Note however that with the capabilities of the journal it might be > interesting to add journal support to logwatch that goes beyond mere > compatibility. For example, tests such as "look for messages which are > claimed to come from PID xyz but actually came from uvw" and suchlike > would be really interesting to have. That information is not available > in the /var/log/messages format however... So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? Best, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 10:12, Richard W.M. Jones (rjo...@redhat.com) wrote: > On Wed, Oct 10, 2012 at 09:54:28AM +0100, Richard W.M. Jones wrote: > > On Wed, Oct 10, 2012 at 09:50:43AM +0200, Björn Persson wrote: > > > Lennart Poettering wrote: > > > > On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: > > > > > How do you read this log when the system is not running (e.g. > > > > > mounting filesystems of a drive on another system, running from a > > > > > rescue image, etc.)? > > > > > > > > journalctl -D > > > > > > So the rescue system (which might not always be Fedora) must have > > > journalctl installed. Is the file format stable, or can it break if the > > > rescue system has a different version of journalctl? Is the format > > > perchance even documented so that other tools for reading logs could be > > > written? > > > > This would be essential for libguestfs tools to parse logs out of > > guests (we do it now by reading /var/log/messages etc which has all of > > the properties you state). > > I checked out the code, and it does seem as if the format is intended > to be backwards compatible. It uses a set of filesystem-like > "compatible" and "incompatible" flags, so presumably a sufficiently > recent journalctl would be able to read any previous version of the > binary file format. > > It would be nice to have this confirmed, and indeed enshrined in the > policy of the journal, because it is IMHO essential that the binary > log files will always be readable. Yes, the compatible and incompatible flag bit fields are precisely to provide good compatibility as the format evolves. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 09:54, Richard W.M. Jones (rjo...@redhat.com) wrote: > On Wed, Oct 10, 2012 at 09:50:43AM +0200, Björn Persson wrote: > > Lennart Poettering wrote: > > > On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: > > > > How do you read this log when the system is not running (e.g. > > > > mounting filesystems of a drive on another system, running from a > > > > rescue image, etc.)? > > > > > > journalctl -D > > > > So the rescue system (which might not always be Fedora) must have > > journalctl installed. Is the file format stable, or can it break if the > > rescue system has a different version of journalctl? Is the format > > perchance even documented so that other tools for reading logs could be > > written? > > This would be essential for libguestfs tools to parse logs out of > guests (we do it now by reading /var/log/messages etc which has all of > the properties you state). I'd recommend simply using our C API for this. For details see: http://www.freedesktop.org/software/systemd/man/ Look for the various APIs with the sd_journal_ prefix. With those you get full access to the journal. Here you find an example how to do this: http://www.freedesktop.org/software/systemd/man/sd_journal_next.html Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
