Re: [CentOS] Random files in homedir gets deleted
On Friday 04 January 2008 17:18:25 Radu Radutiu wrote: > Hi you can try to use the kernel audit facility: > 1) enable the auditd daemon: > service auditd start > > 2) enable audit for the home directory (only audit write operations to > the directory inode); the command is not recursive and you cannot use > wildcards > > auditctl -w /home/user -pw > > 3) after a file disapears use ausearch to find who removed it (and > what command was used to remove it); suppose file "test" was removed > > ausearch -f /home/user/test Thanks Radu for the directions. I google for more information and found this very nice article: http://www.cyberciti.biz/tips/linux-audit-files-to-see-who-made-changes-to-a-file.html But it seems that there's no man page for the /etc/audit.rules? -- Fajar Priyanto | Reg'd Linux User #327841 | Linux tutorial http://linux2.arinet.org 17:04:31 up 2:35, 2.6.22-14-generic GNU/Linux Let's use OpenOffice. http://www.openoffice.org The real challenge of teaching is getting your students motivated to learn. -- Fajar Priyanto | Reg'd Linux User #327841 | Linux tutorial http://linux2.arinet.org 18:43:16 up 19 min, 2.6.22-14-generic GNU/Linux Let's use OpenOffice. http://www.openoffice.org The real challenge of teaching is getting your students motivated to learn. signature.asc Description: This is a digitally signed message part. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Random files in homedir gets deleted
Do what I do when I need to setup a new Linux facility. Google "linux audit" I remember getting a good hit near the top with that. There are cli tools for adding files/folders/mounts to the audit system and you can tailor which type of activity to audit. It's no where as difficult to do as it sounds. -Ross -Original Message- From: [EMAIL PROTECTED] <[EMAIL PROTECTED]> To: CentOS mailing list Sent: Fri Jan 04 04:25:17 2008 Subject: RE: [CentOS] Random files in homedir gets deleted > You can enable auditing to determine if the files are disappearing due to human/machine intervention (audit file system deletes) or if it is due to file system corruption (files disappear and no delete audits recorded). > > It may just be an errant rsync script. > > -Ross How do I enable auditing of the home dir? /Christopher ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos __ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Random files in homedir gets deleted
Hi you can try to use the kernel audit facility: 1) enable the auditd daemon: service auditd start 2) enable audit for the home directory (only audit write operations to the directory inode); the command is not recursive and you cannot use wildcards auditctl -w /home/user -pw 3) after a file disapears use ausearch to find who removed it (and what command was used to remove it); suppose file "test" was removed ausearch -f /home/user/test Radu On Jan 4, 2008 11:25 AM, Christopher Thorjussen <[EMAIL PROTECTED]> wrote: > > > You can enable auditing to determine if the files are disappearing due > to human/machine intervention (audit file system deletes) or if it is > due to file system corruption (files disappear and no delete audits > recorded). > > > > It may just be an errant rsync script. > > > > -Ross > > How do I enable auditing of the home dir? > > /Christopher > > ___ > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos > ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Random files in homedir gets deleted
> You can enable auditing to determine if the files are disappearing due to human/machine intervention (audit file system deletes) or if it is due to file system corruption (files disappear and no delete audits recorded). > > It may just be an errant rsync script. > > -Ross How do I enable auditing of the home dir? /Christopher ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Random files in homedir gets deleted
> On Thursday 03 January 2008 19:09:11 Christopher Thorjussen wrote:
> > On one of my systems I seem to loose a file or two from time to
time.
> > Last night, one of my files (/home/online/sh/NattjobbPrivat.sh) was
> > deleted/removed/vanished. Another time it was /home/online/sh/daemon
> > that was deleted.
> >
> > But I can't seem to find anything strange in the logs or in the
history,
> > nor would any of my scripts running in crontab mess with those
files.
> >
> > Where can I look for clues? And how do I enable audit for file
> > operations in my home folder?
>
> Hi, this really sounds weird. In order to audit it, the following
> checklist
> might help:
> 1. If the system was administered by an admin other than you and he
got
> fired/dismissed with hard feeling on him, he might put a crontab that
> would
> do nasty thing randomly. Audit all the files in:
> /var/spool/cron
> /var/spool/at
> Also all the script in /etc/cron.{d,daily,weekly,monthly},
/etc/crontab
No admin or anyone else with access have quit or been fired. The files
and folders looks fine.
> 2, Audit all RPM files installed using:
> rpm -Va, looks for a difference in md5sum for binary files such
> as /bin/ls,/bin/ps, etc. You might want to use cracker detection
script
> such as rkhunter.
The files look fine. Some files are marked as MD5 mismatch but it's
mostly config files I've changed. The only files I'm not sure of is:
SM5T/usr/share/rhn/rhn_applet/rhn_applet.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_animation.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_apt.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_dialogs.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_model.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_protocols.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_rpc.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_rpm.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_version.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_applet_yum.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_sources.pyc
SM5T/usr/share/rhn/rhn_applet/rhn_utils.pyc
But I'm not running X so the applet isn't running.
>
> 3. Looks for the word "error" in log files:
> grep -r error /var/log
> See for related error such as filesystem corruption, etc
[EMAIL PROTECTED] tmp]# grep -r error /var/log
/var/log/Xorg.0.log:(WW) warning, (EE) error, (NI) not implemented,
(??) unknown.
/var/log/anaconda.log:* getting rpm error class
/var/log/prelink.log:/usr/lib64/libgpg-error.so.0.1.3
003c50e0-003c50f02878
/var/log/rpmpkgs.4:libgpg-error-1.0-1.x86_64.rpm
/var/log/rpmpkgs.1:libgpg-error-1.0-1.x86_64.rpm
/var/log/messages.2:Dec 17 08:13:10 ora01 kernel: daemon[1562]: segfault
at 007fc000 rip 002a957af4b2 rsp 007fbfffe730 error 6
/var/log/scrollkeeper.log:I/O error : Attempt to load network entity
http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeep
er-omf.dtd
/var/log/scrollkeeper.log:I/O error : Attempt to load network entity
http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeep
er-omf.dtd
/var/log/scrollkeeper.log:I/O error : Attempt to load network entity
http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeep
er-omf.dtd
/var/log/scrollkeeper.log:I/O error : Attempt to load network entity
http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeep
er-omf.dtd
/var/log/scrollkeeper.log:I/O error : Attempt to load network entity
http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeep
er-omf.dtd
/var/log/scrollkeeper.log:I/O error : Attempt to load network entity
http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeep
er-omf.dtd
/var/log/rpmpkgs.2:libgpg-error-1.0-1.x86_64.rpm
/var/log/Xorg.0.log.old:(WW) warning, (EE) error, (NI) not
implemented, (??) unknown.
/var/log/rpmpkgs.3:libgpg-error-1.0-1.x86_64.rpm
/var/log/rpmpkgs:libgpg-error-1.0-1.x86_64.rpm
/var/log/anaconda.xlog: (WW) warning, (EE) error, (NI) not implemented,
(??) unknown.
/var/log/anaconda.xlog:error opening security policy file
/etc/X11/xserver/SecurityPolicy
> 4. It's a long shot, but could be a misconfigured rsync script?
Rsync is not running/used, but some custom scripts are running cleaning
up some folders. I'm trying to battle through them to see if somethings
wrong in them, but so far I've found nothing.
> HTH, pls let us know the result.
Will do.
/Christopher
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Random files in homedir gets deleted
>> Where can I look for clues? And how do I enable audit for >> file operations in my home folder? >> > > If your system is capable, use the SMART tools to check your drive out > (as CM suggests), something like this: > > smartctl -a /dev/sda > > See how your 'error count log' is doing. If there are errors, then you > might want to run that command a few times and see if the error count is > still rising. It's a Dell PowerEdge 2950 running in raid 1+0 on the Perc 5/I with SCSI drives. [EMAIL PROTECTED] ~]# smartctl -a /dev/sda7 smartctl version 5.33 [x86_64-redhat-linux-gnu] Copyright (C) 2002-4 Bruce Allen Home page is http://smartmontools.sourceforge.net/ Device: DELL PERC 5/i Version: 1.03 Serial number: 008f71137876e77c0e00b4fdc230c201 Device type: disk Local Time is: Fri Jan 4 09:43:37 2008 CET Device does not support SMART Error Counter logging not supported Error Events logging not supported Device does not support Self Test logging > Is it everything in the /home/online/sh/ directory that is getting > deleted, or can you see any pattern at all? (it sounds like it is random > from what you said...but hard to think of why files would be deleted > randomlyas you know!) No pattern so far. Yeah I know it sounds strange for files to be randomly deleted. /Christopher ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Random files in homedir gets deleted
>On Thu, 3 Jan 2008 13:09:11 +0100 >"Christopher Thorjussen" <[EMAIL PROTECTED]> wrote: > >> On one of my systems I seem to loose a file or two from time to time. >> >> Where can I look for clues? >Is your system visible to the internet? Maybe it's running some kind of >Apache with homedirs loosely enabled and one unsecure php script + one >little h4x0r could do the trick? Is your file system sane? Is your hard >drive(s) SMART-wise OK? >CM The system is visible only for a few defined IP addresses on the internet Apache is not installed. It runs Oracle 10.2g /Christopher ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Random files in homedir gets deleted
On Thursday 03 January 2008 19:09:11 Christopher Thorjussen wrote:
> On one of my systems I seem to loose a file or two from time to time.
> Last night, one of my files (/home/online/sh/NattjobbPrivat.sh) was
> deleted/removed/vanished. Another time it was /home/online/sh/daemon
> that was deleted.
>
> But I can't seem to find anything strange in the logs or in the history,
> nor would any of my scripts running in crontab mess with those files.
>
> Where can I look for clues? And how do I enable audit for file
> operations in my home folder?
Hi, this really sounds weird. In order to audit it, the following checklist
might help:
1. If the system was administered by an admin other than you and he got
fired/dismissed with hard feeling on him, he might put a crontab that would
do nasty thing randomly. Audit all the files in:
/var/spool/cron
/var/spool/at
Also all the script in /etc/cron.{d,daily,weekly,monthly}, /etc/crontab
2, Audit all RPM files installed using:
rpm -Va, looks for a difference in md5sum for binary files such
as /bin/ls,/bin/ps, etc. You might want to use cracker detection script such
as rkhunter.
3. Looks for the word "error" in log files:
grep -r error /var/log
See for related error such as filesystem corruption, etc
4. It's a long shot, but could be a misconfigured rsync script?
HTH, pls let us know the result.
--
Fajar Priyanto | Reg'd Linux User #327841 | Linux tutorial
http://linux2.arinet.org
21:09:01 up 1:02, 2.6.22-14-generic GNU/Linux
Let's use OpenOffice. http://www.openoffice.org
The real challenge of teaching is getting your students motivated to learn.
signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] Random files in homedir gets deleted
> > On one of my systems I seem to loose a file or two from time > to time. Last night, one of my files > (/home/online/sh/NattjobbPrivat.sh) was > deleted/removed/vanished. Another time it was > /home/online/sh/daemon that was deleted. > > But I can't seem to find anything strange in the logs or in > the history, nor would any of my scripts running in crontab > mess with those files. > > Where can I look for clues? And how do I enable audit for > file operations in my home folder? > If your system is capable, use the SMART tools to check your drive out (as CM suggests), something like this: smartctl -a /dev/sda See how your 'error count log' is doing. If there are errors, then you might want to run that command a few times and see if the error count is still rising. Is it everything in the /home/online/sh/ directory that is getting deleted, or can you see any pattern at all? (it sounds like it is random from what you said...but hard to think of why files would be deleted randomlyas you know!) ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Random files in homedir gets deleted
You can enable auditing to determine if the files are disappearing due to human/machine intervention (audit file system deletes) or if it is due to file system corruption (files disappear and no delete audits recorded). It may just be an errant rsync script. -Ross -Original Message- From: [EMAIL PROTECTED] <[EMAIL PROTECTED]> To: centos@centos.org Sent: Thu Jan 03 07:09:11 2008 Subject: [CentOS] Random files in homedir gets deleted On one of my systems I seem to loose a file or two from time to time. Last night, one of my files (/home/online/sh/NattjobbPrivat.sh) was deleted/removed/vanished. Another time it was /home/online/sh/daemon that was deleted. But I can't seem to find anything strange in the logs or in the history, nor would any of my scripts running in crontab mess with those files. Where can I look for clues? And how do I enable audit for file operations in my home folder? /Christopher Thorjussen ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos __ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Random files in homedir gets deleted
On Thu, 2008-01-03 at 13:09 +0100, Christopher Thorjussen wrote: > On one of my systems I seem to loose a file or two from time to time. > Last night, one of my files (/home/online/sh/NattjobbPrivat.sh) was > deleted/removed/vanished. Another time it was /home/online/sh/daemon > that was deleted. > > But I can't seem to find anything strange in the logs or in the history, > nor would any of my scripts running in crontab mess with those files. > > Where can I look for clues? And how do I enable audit for file > operations in my home folder? Although less common than "In The Day", if file system corruption was repaired by *fsck during boot or other times, the "lost+found" directory of the affected FS may contain some files with weird names, having mostly numbers. These numbers are i-node numbers associated with the "chunks" that appear in the lost+found directory. If you see these, it indicates that corruption occurred and was repaired. If the files are plain text, there is a possibility to reconstruct the original files via visual inspection and concatenation. Often they are mixed or binary and more difficult to reconstruct. If it's not corruption, finding the cause may not be so easy. Long ago, before the we had the plethora of tools now available, I found the cause of such a mystery with a multi-pronged approach. 1. Fire up the system accounting package (sar). 2. Start a "daemon" (really just a "nohup &" that checked for the existence of the file every so many seconds. This if really nothing more than a if [ ! -r ] ; then ; fi The "-r" can be replaced/compounded with other flags as desired. See "man bash", "CONDITIONAL EXPRESSIONS. When the file went missing, a "log" entry was generated. 3. Generate the sar reports and examine them to see what was running around that time. 4. As I did, go viciously beat the offending user that made a flawed script about the ears with a rather rigid clue bat. 5. Problem solved. > > > /Christopher Thorjussen > HTH -- Bill ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Random files in homedir gets deleted
On Thu, 3 Jan 2008 13:09:11 +0100 "Christopher Thorjussen" <[EMAIL PROTECTED]> wrote: > On one of my systems I seem to loose a file or two from time to time. > > Where can I look for clues? Is your system visible to the internet? Maybe it's running some kind of Apache with homedirs loosely enabled and one unsecure php script + one little h4x0r could do the trick? Is your file system sane? Is your hard drive(s) SMART-wise OK? CM -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
