Re: [osol-discuss] How to make syslogger write to a fifo?
I believe you can create a named pipe (using mkfifo) somewhere in the filesystem (I'd suggest somewhere persistent, not /tmp or /var/run, etc...). I've seen it created as /var/log/logpipe or similar. Then simply name the file in syslog.conf without the '|' symbol. However, note that if there is nothing reading from the pipe when syslogd starts (or when it receives a HUP) then it will ignore it, so you need to make sure something is reading the pipe before syslog starts. SMF is probably the best way to do this - you can insert a dependancy for the svc:/system/system-log service to make sure your pipe reader starts first. Also, if there is nothing reading from the pipe when syslogd tries to write to it, it will close it and ignore it until the next restart/refresh. However, the smf "restart on" property of the dependancy can refresh system-log if your reader dies and is restarted. Hope that helps, Brian Harry Putnam wrote: Can anyone tell me how to make the system logger write to a fifo? I've tried this in /etc/syslog.conf *.debug |/var/adm/flt Where `flt' is a fifo and the two elements are separated by tabs. After a sighup to systemlogger I'm not seeing any data with `cat flt' Made sure data is been sent by system-log but nothing appears at the fifo. Doing something like (in two different xterms): xterm 1 cat flt xterm 2 cat messages > flt And I see the data so I know the fifo is working. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org -- Brian Ruthven Solaris Revenue Product Engineering Sun Microsystems UK Sparc House, Guillemont Park, Camberley, GU17 9QG ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
Brian Ruthven - Solaris Network Sustaining - Sun UK writes: > I believe you can create a named pipe (using mkfifo) somewhere in the > filesystem (I'd suggest somewhere persistent, not /tmp or /var/run, > etc...). I've seen it created as /var/log/logpipe or similar. Then > simply name the file in syslog.conf without the '|' symbol. Haa... ok, that was my error... my linux background.. showing... on linux syslog you need the pipe there. > However, note that if there is nothing reading from the pipe when > syslogd starts (or when it receives a HUP) then it will ignore it, so > you need to make sure something is reading the pipe before syslog > starts. SMF is probably the best way to do this - you can insert a > dependancy for the svc:/system/system-log service to make sure your pipe > reader starts first. > Also, if there is nothing reading from the pipe when syslogd tries to > write to it, it will close it and ignore it until the next > restart/refresh. However, the smf "restart on" property of the > dependancy can refresh system-log if your reader dies and is restarted. Thanks, all good info... but one thing... How do you keep the reader running? I mean a script using the fifo for input like `script myfifo' quits as soon as the first line of syslog output comes thru.. Whereas something like `tail -f myfifo' keeps reading. What I'm after is attaching a perl script to the fifo that is capable of searching and sorting on any regex or any 2 regex actually, that you feed the script on startup. But unlike linux... where the pipe symbol goes into /etc/syslog.conf on opensolaris my script just quits on the first line of output. Maybe I can just add some trickiness to the script to make it act like tail -f... but is that really necessary? Am I missing something obvious here that would make any script continue to read from the fifo? man syslog.conf has no hits on either `fifo' or `pipe' so I'm guessing there is no help there... I haven't pored through every line but nothing jumps out as helpful. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
>Can anyone tell me how to make the system logger write to a fifo? > >I've tried this in /etc/syslog.conf > > *.debug |/var/adm/flt > >Where `flt' is a fifo and the two elements are separated by tabs. I'm assuming that "flt" is a named pipe; doesn't it work when you use /var/adm/flt? I just tested this and it does seems to work. But syslog will close it when nothing is reading from the pipe. Casper ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
[ Please note - this is off the top of my head, so nothing is tested.
There may^H^H^H will be errors in here :-) ]
Personally, I would use an SMF service to wrap my reader, and place a
dependent entry in the manifest something like this:
Then, syslog will wait until your service is running (i.e. reading from
the pipe) before starting syslog. Moreover, I believe the restart_on
property should restart syslogd if your service dies, is refreshed or is
stopped. A value of 'error' may be sufficient for your service...
Then, to answer your second part, the program must open and continuously
read from the file until EPIPE is received (or some other appropriate
FIFO error). You can do this in something like C or perl explicitly
(like "tail -f" does), but as a quick workaround, try:
tail -f mypipe | script
(and make 'script' read from stdin). Yes tail may insert a delay, but at
least you can quickly workaround the quitting issue. It's not pretty,
but shouldn't require much modification to your script.
If you're using perl, then this works with syslog:
#!/usr/bin/perl -w
use strict;
if (open(PIPE, "< /var/tmp/mypipe")) {
while () {
print "Read $_";
# Do processing stuff here...
}
}
Hope that helps,
Brian
Harry Putnam wrote:
Brian Ruthven - Solaris Network Sustaining - Sun UK
writes:
I believe you can create a named pipe (using mkfifo) somewhere in the
filesystem (I'd suggest somewhere persistent, not /tmp or /var/run,
etc...). I've seen it created as /var/log/logpipe or similar. Then
simply name the file in syslog.conf without the '|' symbol.
Haa... ok, that was my error... my linux background.. showing... on
linux syslog you need the pipe there.
However, note that if there is nothing reading from the pipe when
syslogd starts (or when it receives a HUP) then it will ignore it, so
you need to make sure something is reading the pipe before syslog
starts. SMF is probably the best way to do this - you can insert a
dependancy for the svc:/system/system-log service to make sure your pipe
reader starts first.
Also, if there is nothing reading from the pipe when syslogd tries to
write to it, it will close it and ignore it until the next
restart/refresh. However, the smf "restart on" property of the
dependancy can refresh system-log if your reader dies and is restarted.
Thanks, all good info... but one thing... How do you keep the reader
running?
I mean a script using the fifo for input like `script myfifo' quits as
soon as the first line of syslog output comes thru..
Whereas something like `tail -f myfifo' keeps reading.
What I'm after is attaching a perl script to the fifo that is capable
of searching and sorting on any regex or any 2 regex actually, that you
feed the script on startup.
But unlike linux... where the pipe symbol goes into /etc/syslog.conf
on opensolaris my script just quits on the first line of output.
Maybe I can just add some trickiness to the script to make it act like
tail -f... but is that really necessary?
Am I missing something obvious here that would make any script
continue to read from the fifo?
man syslog.conf has no hits on either `fifo' or `pipe' so I'm guessing
there is no help there... I haven't pored through every line but nothing
jumps out as helpful.
___
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
--
Brian Ruthven
Solaris Revenue Product Engineering
Sun Microsystems UK
Sparc House, Guillemont Park, Camberley, GU17 9QG
___
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
Brian Ruthven - Sun UK writes: > Personally, I would use an SMF service to wrap my reader, and place a > dependent entry in the manifest something like this: Brian.. can you post some URLs about using SMF. I'm completely ignorant about it. I realize I can google it but hoped URLs you provided might save lots of time wasted on baloney hits. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
casper@sun.com writes:
>>Can anyone tell me how to make the system logger write to a fifo?
>>
>>I've tried this in /etc/syslog.conf
>>
>> *.debug |/var/adm/flt
>>
>>Where `flt' is a fifo and the two elements are separated by tabs.
>
>
> I'm assuming that "flt" is a named pipe; doesn't it work when
> you use /var/adm/flt?
>
> I just tested this and it does seems to work. But syslog will close it
> when nothing is reading from the pipe.
Yes it does... but like you said... it leaves a problem for the
script. My coding skills are quite slim. I see a command like
`tail -f named-pipe'
Will keep reading so any ouput from system logger is displayed... but
a regular kind of command like:
cat named-pipe
cat will close after the first line is read.
Similar for something like awk '/REGEX/{print}' named-pipe > some.log
awk will close after the first line of input.
I guess I need to write a script that behaves like `tail -f' and I'm
sure I can in perl, but is that really necessary?
Isn't there some way to get a standard filtering script to get all
lines the system logger puts out, without trickery inside the script
to keep it from closing after the first line.
I guess the linux way of using a pipe `|' in the actual syslog.conf is
one way to get it done... but that does not work on opensol.
___
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
>Will keep reading so any ouput from system logger is displayed... but
>a regular kind of command like:
>
> cat named-pipe
>
>cat will close after the first line is read.
Not my experience; "cat" works.
>Similar for something like awk '/REGEX/{print}' named-pipe > some.log
>
>awk will close after the first line of input.
>
>I guess I need to write a script that behaves like `tail -f' and I'm
>sure I can in perl, but is that really necessary?
>Isn't there some way to get a standard filtering script to get all
>lines the system logger puts out, without trickery inside the script
>to keep it from closing after the first line.
>
>I guess the linux way of using a pipe `|' in the actual syslog.conf is
>one way to get it done... but that does not work on opensol.
Well, you have the source (though I understand that your code skill
are light)
Casper
___
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
casper@sun.com writes: >>Will keep reading so any ouput from system logger is displayed... but >>a regular kind of command like: >> >> cat named-pipe >> >>cat will close after the first line is read. > > Not my experience; "cat" works. Sorry, yes you are right... cat fifo restart syslog. Then from a users terminal `ssh r...@localhost' produces output at cat... but unlike `tail -f' another restart of syslogger at this point and cat closes (tail -f does not) ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
Hi Harry, I've got nothing canned which I can quickly pass on, however, anything I gather will be from google ;-) The top three hits searching for "writing smf manifest" look useful: http://www.sun.com/bigadmin/content/selfheal/sdev_intro.jsp http://wikis.sun.com/display/BigAdmin/SMF+Short+Cuts http://blogs.warwick.ac.uk/chrismay/entry/solaris_smf_manifest/ I've not looked at them in depth, but they seem to contain a mixture of background and contain examples (or fragments). I'd suggest copying /var/svc/manifest/system/utmp.xml as a starting point for modification (obviously don't modify utmp.xml itself!). IIRC, it implements a simple, single-daemon 'child' type of service, which is restarted if it dies. Hopefully that is enough to get you started. I'd suggest copying the manifest from a simple service such as utmp.xml and customise it to your needs. If your service needs a startup script, then you should include /lib/svc/share/smf_include.sh so you can use the correct exit codes to signal the right things to the framework. Hope that helps, Brian Harry Putnam wrote: Brian Ruthven - Sun UK writes: Personally, I would use an SMF service to wrap my reader, and place a dependent entry in the manifest something like this: Brian.. can you post some URLs about using SMF. I'm completely ignorant about it. I realize I can google it but hoped URLs you provided might save lots of time wasted on baloney hits. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org -- Brian Ruthven Solaris Revenue Product Engineering Sun Microsystems UK Sparc House, Guillemont Park, Camberley, GU17 9QG -- Brian Ruthven Solaris Revenue Product Engineering Sun Microsystems UK Sparc House, Guillemont Park, Camberley, GU17 9QG ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
* Brian Ruthven - Solaris Network Sustaining - Sun UK
[2009-10-27 10:39]:
> http://www.sun.com/bigadmin/content/selfheal/sdev_intro.jsp
> http://wikis.sun.com/display/BigAdmin/SMF+Short+Cuts
> http://blogs.warwick.ac.uk/chrismay/entry/solaris_smf_manifest/
A relatively recent (Mar 2009) tool is Chris Miles's "manifold",
hosted at
http://code.google.com/p/manifold/
which will guide you through a series of questions and produce a
manifest. ("pfexec easy_install Manifold" should get it installed on
your system.)
Cheers
Stephen
___
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
Brian Ruthven - Solaris Network Sustaining - Sun UK
writes:
> Hi Harry,
>
> I've got nothing canned which I can quickly pass on, however, anything
> I gather will be from google ;-)
[...]
Stephen Hahn writes:
[...]
> A relatively recent (Mar 2009) tool is Chris Miles's "manifold",
> hosted at
>
> http://code.google.com/p/manifold/
>
> which will guide you through a series of questions and produce a
> manifest. ("pfexec easy_install Manifold" should get it installed on
> your system.)
Both aproaches look very interesting... haven't got to them yet
though.
Thanks to both of you for your time.
___
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
Brian Ruthven - Solaris Network Sustaining - Sun UK writes: > Hi Harry, > > I've got nothing canned which I can quickly pass on, however, anything > I gather will be from google ;-) > > The top three hits searching for "writing smf manifest" look useful: > > http://www.sun.com/bigadmin/content/selfheal/sdev_intro.jsp > http://wikis.sun.com/display/BigAdmin/SMF+Short+Cuts > http://blogs.warwick.ac.uk/chrismay/entry/solaris_smf_manifest/ [...] Jesus... after looking at those URL a bit I feel like just start boohooing and go home. This SMF stuff seems horribly complicated to me. They use terms like `JBoss' with no explanation.. And the XML itself just seem vastly overdone for something that should be fairly simple. > Hopefully that is enough to get you started. I'd suggest copying the > manifest from a simple service such as utmp.xml and customise it to > your needs. If your service needs a startup script, then you should > include /lib/svc/share/smf_include.sh so you can use the correct exit > codes to signal the right things to the framework. I guess it will be a start... but man I don't understand hardly any of it. To attach a script to an existing service and make it restart when that service restarts is not really something that should require yards and yards of code, several documents, and god only knows what else. Its tempting to just write a perl script, that looks for the service to be running, and starts up if it is. Is that a really bad approach for this? `this' in case it has gotten away in the thread is to run a script that reads from a named-pipe.. that the syslogger writes everything to. The purpose of the script is to have finely grained control over writing various things to logs... using regular expressions. And after starting on the script, I realized I might want to change the regular expressions as the script runs. So far, I've figured out a way to do that I think, by making the script read a secondary file every five minutes. I might write a regular expression and matching log file in the secondary file and the script as it runs will start looking for those. And writing hits to the new log. So far I plan just to use matching pairs in the secondary file like this. REGEX LOG.log What I haven't really got into yet is the best way to have this script running in the background... checking for syslog to be running. That is, the script would never stop running even if the syslogger shut down. Maybe even some kind of `trap' in the script where it would send me a message in the event it was killed. (At least for some kinds of KILL) That's where it starts to look like it might be better to insinuate this script in there through SMF. I'd really like to see a simplified example of how that might be done. Maybe there is an example like that in the URLS you posted. I haven't gotten very far looking into them. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
Hi Harry, It's not actually that difficult once you get used to it. Think of it like learning a new programming language - you're not going to write an entire OS from scratch on your first day. Here's an example that took about 30 mins to write (although the bulk of this was looking into the restart_on property for the tag): "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> /lib/svc/method/svc-myservice could contain: #!/bin/sh . /lib/svc/share/smf_include.sh FIFO=/tmp/myfifo # Something here to grep $FIFO /etc/syslog.conf and add it if not there [ ! -p $FIFO ] && mkfifo -m 644 $FIFO /usr/local/bin/pipe_reader.pl $FIFO & exit $SMF_EXIT_OK You could even use a property within the SMF service (defined in the manifest) to store the location of the fifo, but we're getting more advanced there... It does avoid having to modify the script each time you want to change the fifo location. Do bear in mind that the SMF framework is much more than simply a "if not running, start" type of monitor for services. It allows interlinking with dependencies, multiple instances of the same service, grouping of faults into boundaries (the "contract" subsystem), running in "degraded" mode (although I've not seen this used yet), handling of multiple zones, etc... One of the most useful places I've interacted with SMF is the rpcbind program. If rpcbind dies and is restarted on Solaris 8/9, then all your services stop working unless you restart each by hand. With Solaris 10+ and SMF, if rpcbind dies, SMF will restart it. Moreover, it will also restart any services which have declared a dependency upon rpc/bind and with a restart_on property which is != 'none'. The example manifest above will actually restart system-log (i.e. syslogd in the current zone) if "myservice" dies. Also, the "optional_all" dependency in the "dependent" section states that system-log will not be prevented from running if myservice is disabled. i.e. syslog won't break if you disable myservice. All in all, very useful for automated restarting of a service *and its dependents*. Hope that helps, Brian Harry Putnam wrote: Brian Ruthven - Solaris Network Sustaining - Sun UK writes: Hi Harry, I've got nothing canned which I can quickly pass on, however, anything I gather will be from google ;-) The top three hits searching for "writing smf manifest" look useful: http://www.sun.com/bigadmin/content/selfheal/sdev_intro.jsp http://wikis.sun.com/display/BigAdmin/SMF+Short+Cuts http://blogs.warwick.ac.uk/chrismay/entry/solaris_smf_manifest/ [...] Jesus... after looking at those URL a bit I feel like just start boohooing and go home. This SMF stuff seems horribly complicated to me. They use terms like `JBoss' with no explanation.. And the XML itself just seem vastly overdone for something that should be fairly simple. Hopefully that is enough to get you started. I'd suggest copying the manifest from a simple service such as utmp.xml and customise it to your needs. If your service needs a startup script, then you should include /lib/svc/share/smf_include.sh so you can use the correct exit codes to signal the right things to the framework. I guess it will be a start... but man I don't understand hardly any of it. To attach a script to an existing service and make it restart when that service restarts is not really something that should require yards and yards of code, several documents, and god only knows what else. Its tempting to just write a perl script, that looks for the service to be running, and starts up if it is. Is that a really bad approach for this? `this' in case it has gotten away in the thread is to run a script that reads from a named-pipe.. that the syslogger writes everything to. The purpose of the script is to have finely grained control over writing various things to logs... using regular expressions. And after starting on the script, I realized I might want to change the regular expressions as the script runs. So far, I've figured out a way to do that I think, by making the script read a secondary file every five minutes. I might write a regular expression and matching log file in the secondary file and the script as it runs will start looking for those. And writing hits to the new log. So far I plan just to use matching pairs in the secondary file like this. REGEX LOG.log What I haven't really got into yet is the best way to have this script running in the background... checking for syslog to be running. That is, the script would never stop running even if the syslogger shut down. Maybe even some kind of `trap' in the script where it would send me a message in the event it was killed. (At least for some kinds of KILL) That's where it starts to look like it might be better to insinuate this script in there through SMF. I'd really like to see a
Re: [osol-discuss] How to make syslogger write to a fifo?
Brian Ruthven - Sun UK writes: > Hi Harry, > > It's not actually that difficult once you get used to it. Think of it > like learning a new programming language - you're not going to write > an entire OS from scratch on your first day. Here's an example that > took about 30 mins to write (although the bulk of this was looking > into the restart_on property for the tag): > [...] > Hope that helps, Wow... I'll say. That was really beyond the call of duty and seriously appreciated on this end... thanks ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
I like what you're doing... will keep tuned. Will Splunk do what you're looking for, or have you reviewed it? Sorry if you already have. Wow... learning a *lot* about Unix in here. =) On Fri, Oct 30, 2009 at 13:53, Harry Putnam wrote: > Brian Ruthven - Sun UK > writes: > >> Hi Harry, >> >> It's not actually that difficult once you get used to it. Think of it >> like learning a new programming language - you're not going to write >> an entire OS from scratch on your first day. Here's an example that >> took about 30 mins to write (although the bulk of this was looking >> into the restart_on property for the tag): >> > > [...] > >> Hope that helps, > > Wow... I'll say. That was really beyond the call of duty and > seriously appreciated on this end... thanks > > ___ > opensolaris-discuss mailing list > opensolaris-discuss@opensolaris.org > -- " ' With the first link, the chain is forged. The first speech censured, the first thought forbidden, the first freedom denied, chains us all irrevocably.' Those words were uttered by Judge Aaron Satie as wisdom and warning... The first time any man's freedom is trodden on we’re all damaged." - Jean-Luc Picard, quoting Judge Aaron Satie, Star Trek: TNG episode "The Drumhead" - Alex Smith (K4RNT) - Murfreesboro/Nashville, Tennessee USA ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] How to make syslogger write to a fifo?
"Alex Smith (K4RNT)" writes: > I like what you're doing... will keep tuned. > > Will Splunk do what you're looking for, or have you reviewed it? Sorry > if you already have. Splunk is aimed at much broader level than my scripting. I'm not sure if the same logging input would occur either. I've never used splunk but from what I've read here and there it tries to gather all logs, even across a network for a collection of hosts to uncover problems by observing patterns that kind of scope would allow. I suppose it could be setup for on a smaller scale but I believe its intent is the large scope. What I meant about whether it would gather the same log data, is that the fifo I use is being written to by a rule like *.*, in /etc/syslog.conf which would normally only be called for when debugging operations. I hoped to use it for a finer grain splitting of log messages than syslog.conf or even syslog-ng can do readily. What got me started was seeing certain messages in the logs I don't understand and wanting to collect them right from the system logger as the occurred. I still haven't solved that problem... hehe... too many other things going on right now.. But I started thinking that the fifo reader might be very handy at times, so decided to go ahead and get a good working perl script together. I'm still working on that too, in between several other things I'm involved with. One of the biggest time drains for me is hanging out with my Great, Great grand children. A pair of twin boys that are really a joy to be around. They are just entering there eleventh mnth One is walking pretty well already and the other right behind him. They both have very recognizable personalities already. I don't know who gets the most out of our time together... but I suspect its me. Those two little fellas have captured my heart and then some. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
