Hi folks.
Here's a revised version of my requirements doc. It's now
"feature complete" but still needs another round of review
before it can be widely published.
In particular...
* Magosanyi, please review sections referencing [Arpad],
which include statements based on your previous
comments to me.
* Mark R., similarly for [Roth].
* Balazs, similarly for [Scheidler] and also the section
on syslog-ng.
* Chris L., please review how I have referenced your own
draft through [Lonvick]. Also please offer me any
guidance on "next steps" in the Internet Draft process
(as your new position as the grand and exalted chairman
of the IETF Syslog WG).
* Darren, please review my coverage of Nsyslogd.
* Herve, similarly for my coverage of ULM. I also
apologize for not having the accent mark over the e,
but such are the IETF's requirements for ASCII text.
Security Issues in Network Event C. Calabrese
Logging (syslog) August 2000
INTERNET DRAFT
Expires: XXXX XX, 200X
<draft-XXX-YYY-01.txt>
Requirements for a System Event Logging Protocol
Status of this Memo
This document is an Internet-Draft and is in full
conformance with all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working
groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum
of six months and may be updated, replaced, or obsoleted
by other documents at any time. It is inappropriate to
use Internet- Drafts as reference material or to cite
them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be
accessed at http://www.ietf.org/shadow.html
To learn the current status of any Internet-Draft, please
check the "1id-abstracts.txt" listing contained in the
Internet-Drafts Shadow Directories on ftp.is.co.za
(Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific
Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US
West Coast).
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL',
'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED',
'MAY', and 'OPTIONAL' in this document are to be
interpreted as described in RFC 2119[1].
Copyright Notice
Copyright (C) The Internet Society (2000). All Rights
Reserved.
This document and translations of it may be copied and
furnished to others, and derivative works that comment on
or otherwise explain it or assist in its implementation
may be prepared, copied, published and distributed, in
whole or in part, without restriction of any kind,
provided that the above copyright notice and this
INTERNET DRAFT Event Logging Requirements August 2000
paragraph are included on all such copies and derivative
works. However, this document itself may not be modified
in any way, such as by removing the copyright notice or
references to the Internet Society or other Internet
organizations, except as needed for the purpose of
developing Internet standards in which case the
procedures for copyrights defined in the Internet
Standards process must be followed, or as required to
translate it into languages other than English.
The limited permissions granted above are perpetual and
will not be revoked by the Internet Society or its
successors or assigns.
This document and the information contained herein is
provided on an "AS IS" basis and THE INTERNET SOCIETY AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN
WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Abstract
This document presents requirements for sending system
event/audit logs over the Internet. In developing these
requirements, it considers the need for log management,
security, high-performance, and compatibility with
existing practice.
Table of Contents
1 Introduction ???
2 Log Reporting Considerations ???
3 Security Considerations ???
4 Resource Utilization ???
5 Existing Practice ???
6 Conclusions ???
7 References ???
8 Author's Addresses ???
1. Introduction
Computer systems often need to "log" or record
information about important events. In many systems,
these event logs are recorded locally, in persistent
storage within the system where the event occurs. These
messages may also be transmitted to a remote system where
persistent storage takes place. Reasons for such remote
transmission include event logging on systems with no
local persistent storage and centralization of log record
management.
INTERNET DRAFT Event Logging Requirements August 2000
This memo concerns itself with the requirements for
transmitting log messages over an Internet Protocol
network, including requirements for log management, event
correlation, security, performance, and compatibility
with existing practice. This is only a requirements
document. The architecture and specifications for the
delivery options will be detailed in subsequent RFCs.
2. Log Reporting Considerations
2.1 Introduction and Rational
The ultimate reason for transmitting log messages over
the network is so that these messages can be examined,
whether by humans or by computer programs, to find
anomolous events, statistics on past performance,
indications of future trends, or other information of
interest to the examiner. Although the transmission
of the messages itself is not directly related to this
examination, the systems related to transmission
should aid the process of examination to the largest
extent practical.
2.2 Identification of Event Threads
Since logging system tend to generate and transmit
discrete event messages, one of the most important
reporting requirements for a logging system is to be
able to figure out how those event messages fit
together to form a thread of messages about the same
incident.
Messages in a thread are typically identified by the
originating machine the program or sub-system on the
originating machine, and the instance of that program
or sub-system. Additionally, some kind of sequence
information must be present to determine the order of
event messages in the thread.
In most cases, the originating machine can be
determined by the source IP address of log messages
and the sequence determined by the order of arriving
packets (UDP) or relative ordering in the transmission
stream (TCP). However, the IP address can be forged
and is therefore unreliable. Similarly, an attacker
could reorder packets in a UDP stream. Furthermore,
there is no easy method of determining the
program/sub-system and its instance on the originating
machine without including such identifying information
directly in the message. Therefore, it seems plain
that the logging messages must directly contain all
these pieces of information in order to facilitate the
INTERNET DRAFT Event Logging Requirements August 2000
identification of event threads.
This issue is also discussed in the Security section
below.
2.3 Event Correlation
In addition to identifying event messages within the
same incident thread from the same source, it is also
desirable to correlate messages regarding the same
incident from multiple sources. This implies a
standard mechanism for identifying incidents based on
various criteria such as the time the event occurred,
IP addresses of network entities involved, etc.
2.4 Event Filters
2.4.1 Introduction and Rational
Selecting which events to transmit reduces network
load. Selecting which events to accept when
transmitted reduces the amount of data that must be
processed by higher-level software and which must
be persistently stored. The log transmission
protocols can help these efforts by making it easy
to select events for filtering.
2.4.2 Filtering By Source
The most obvious attribute to filter data on in an
Internet Protocol network is the source IP address
of the data.
2.4.3 Filtering By Priority
Another obvious filter would be by message priority
so that high priority messages can be posted as
alarms, very low priority messages ignored, etc.
2.4.4 Filtering By Facility
Not all high priority messages should necessarily
be handled the same way. Aside from message
priority, the primary method of filtering in the
traditional syslog system[2], is by something
called the message facility (LOG_AUTH, LOG_DAEMON,
LOG_MAIL, etc.).
2.4.5 Filtering By Other Attributes
Source address, priority and facility are by no
means the only pieces of information log messages
will need to be filtered by. Indeed, just about
any attribute of a log message is fair game, and
the potential attributes too numerous to enumerate
here. Some have argued that log messages should be
structured to make such filtering easier (and some
structure is already needed for thread
INTERNET DRAFT Event Logging Requirements August 2000
identification, event correlation, and other
reasons); however, others have argued that rigid
structures are not needed if filters are based on
regular expressions[3].
3. Security Considerations
3.1 Introduction and Rational
According to the Department of Defense Trusted
Computer System Evaluation Criteria[4], the
fundamental requirements for computer security are:
1. An explicit and well-defined security policy
enforced by the system.
2. Access control labels associated with objects.
3. Identification of individual subjects.
4. Selective storage and protection of audit
records so that actions affecting security can
be traced to the responsible party.
5. Assurance that the computer system enforces
security requirements.
6. Trusted mechanisms that enforce these basic
requirements that are continuously protected
against tampering and/or unauthorized changes.
In the case of system event logs, the logs themselves
may contain the aforementioned audit records.
3.2 Implications
3.2.1 Policies
The questions of what appropriate policies are for
system logging and how to enforce them are beyond
the scope of this memo. However, any protocol for
transmitting log messages must take into account
the fact that different systems may have different
security policies. Therefore, such protocols
should not require adherence to any particular
policy.
3.2.2 Access Control Labels
Access control labels fall into two categories.
Discretionary access controls (DAC's - think *nix
file permissions) and mandatory access controls
(MAC's - secret, top-secret, etc.).
INTERNET DRAFT Event Logging Requirements August 2000
In the case of system log messages, DAC labels map
neatly into the facility tags present in the
existing syslog system[2]. Note that this implies
that facility tags must be present in all
messages[5].
On the surface, MAC labels would seem to require
some kind of hierarchical identification scheme.
However, by including support for DAC, we can
reduce this from a hierarchical scheme to a simple
ordered scheme. Additionally, if we recognize that
logs operating at lower priority also tend to be
much more verbose, then we can allow an inverse
relationship between MAC labels and log priorities
as used in the existing syslog system (LOG_DEBUG,
LOG_INFO, LOG_NOTICE, LOG_WARNING, etc.)[2]. In
other words, higher-priority messages may be more
widely viewed and less widely created. Conversely,
lower-priority messages may be less widely viewed
but more widely created. Note that this implies
that message priority and facility tags must be
present in all messages[5].
Also note that a strictly TSEC environment would
require the protocol and the components operating
on it to either act as a multilevel channel or use
distinct single-level channels for each set of MAC
labels. A true multilevel channel would require
much closer mapping to the native MAC labels than
the simple mapping described above. Furthermore,
the any non-MAC to MAC transitions must be done in
a single-level channel only. [4] [5]
3.2.3 Identification
Ideally, we want to be able to show by strong
identification mechanisms (i.e., digital
signatures) that a certain log message was
generated by a certain program running with certain
privileges/user-id on a certain machine at a
certain time. This not only allows the exact
subject originating the message to be identified,
but also allows non-repudiation of the messages and
easy correlation of events between machines for
purposes such as intrusion detection and forensics.
The most obvious way of implementing such
identification is to issue a digital signature key
to each program/user/machine combination and to
require all messages to be thusly signed. However,
this runs into two major problems. The first is
one of key management (i.e., there are too many
keys to manage) and the second one of performance
(i.e., not all log messages require such
INTERNET DRAFT Event Logging Requirements August 2000
identification capabilities and not all machines
have the horsepower to supply them). However, if
the logging mechanism on the originating machine is
part of the system's trusted computing base (TCB),
then it is sufficient for TCB to sign the identity
of the log messages on the originating program's
behalf when directed to do so.
Note that it is not necessary for each message to
be digitally signed in its entirity. A common
practice is to negotiate and sign a sessoin key and
then use that key to encrypt a message digest or
message authentication code (MAC) of the individual
messages[6].
3.2.4 Selective Storage
Essentially this boils down to the requirement that
the originator of a log message have the capability
to detect that a particular message has or has not
reached its destination. This does not necessarily
imply that only one copy of the message may reach
its destination or that the sender must block
waiting for delivery notification, which leaves
some room to balance this requirement against
issues such as performance[7]. On the other hand,
the Department of Defense Trusted Computer System
Evaluation Criteria[4] requires security sensitive
applications to shut down if they can't log
security-related events, so the capability to block
waiting for delivery notification must be present
in such environments.
3.2.5 Assurance
Issues such as protocol and implementation
evaluation against security policies and
requirements are beyond the scope of this memo.
However, we can address this issue at a
requirements level by observing that all security
policies center around confidentially, integrity,
and availability.
3.2.5.1 Confidentiality
It is generally acknowledged that the only
practical way to achieve confidentiality of
messages traveling over an open network is
through the use of cryptography.
3.2.5.2 Integrity
Integrity of log messages boils down to
identification of the sender, immutability of
messages without detection, and assurance that
logs have reached their destination.
Identification and immutability can be addressed
INTERNET DRAFT Event Logging Requirements August 2000
through digital signatures as discussed in the
Identification section above. Assurance of
delivery, is discussed in the Selective Storage
section above.
3.2.5.3 Availability
The following seem reasonable requirements for
high-availability:
1. Logging mechanisms should be able to
transmit their log messages to multiple
destinations. Therefore increasing the
probability that at least one will be
available.
2. Logging protocols should make it easy to
filter unwanted messages received.
Therefore reducing the possibility of a
denial of service attack on the recipient
through the generation of bogus messages.
3. Logging protocols should not require log
senders to wait synchronously for positive
acknowledgement of messages sent before
sending any additional messages.
Therefore making denial of service attacks
on log senders more difficult through the
generation of excessive traffic on a
particular network link. Note that not
requiring applications to wait for postive
acknowledgement is not the same thing as
precluding them from waiting[7]. The
Department of Defense Trusted Computer
System Evaluation Criteria[4] requires
security sensitive applications to shut
down if they can't log security-related
events, so the capability to block waiting
for delivery notification must be present
in such environments.
4. Logging protocols should provide the
capability to retransmit unacknowledged
messages. Therefore also making denial of
service attacks on log senders more
difficult through the generation of
excessive traffic on a particular network
link. Note that this facility may be
provided by lower level protocols such as
by the data integrity features of TCP[6].
This implies degraded capabilities when
running over other lower level protocols,
which may be a reasonable trade-off on
many circumstances.
INTERNET DRAFT Event Logging Requirements August 2000
High availability protocol and implementation
issues such as high-availability clustering,
resistance to buffer overrun attacks, etc. are
beyond the scope of this memo.
3.2.6 Trusted Mechanisms
This area is beyond the scope of this memo.
3.3 Protocol Level for Cryptographic Functions
One open issue from the above discussion is whether
cryptographic functions for identification signatures
and confidentiality should be performed at the network
transport layer or at higher-level application-
specific protocols.
Performing these functions at the network transport
layer allows the use of standard transport layer
security mechanisms (TLS, IPsec), making
implementation easier and possibly leading to higher
performance due to hardware assist for standard
mechanisms, smaller memory footprint on devices
already supporting such mechanisms, etc. It also
allows filtering of garbage messages at a lower level,
making denial of service attacks more difficult.
On the other hand, having identification signatures
operate at the network transport level means rejecting
all unsigned or improperly signed messages since
signing information will be lost to the higher
protocol layers. Since not all devices generating log
messages will have sufficient computing abilities to
sign all messages and not all messages will contain
information requiring signature, can be problematic.
Furthermore, signing and encrypting individual
messages facilitates being able to show the
confidentiality and integrity of messages without
needing to show full chain-of-custody or prove that
all links in the chain were appropriately trusted.
Therefore, network logging systems should allow
cryptographic identification functions to be performed
both at the network transport layer and also at higher
layers. Cryptographic confidentiality functions, on
the other hand, are likely to be more appropriate at
the network transport level.
4. Resource Utilization Considerations
From a logging protocol standpoint, the following
resources may be considered:
INTERNET DRAFT Event Logging Requirements August 2000
1. CPU/memory utilization on the sending system to
marshal internal log messages into the on-the-wire
protocol, including any encryption, siging, and/or
compression.
2. On-the-wire bandwidth utilization, including
amenability for the protocol to be compressed by
hardware compressors.
3. CPU/memory utilization on the receiving system
decompress, filter, verify the integrity of, and/or
store incoming messages.
Most of these considerations argue for a simple text
based protocol that is easy to marshal and compress where
application-level compression, encryption, and signatures
are a configurable option. However, the need to filter
messages argues for a more structured protocol to make
filtering easier. Futhermore, there may be interactions
between the encryption mechanism and the ability to
compress the message stream.
5. Existing Practice
5.1 Existing syslog
The syslog system is probably the most widely deployed
mechanism for storing, filtering, and transferring log
message over the network. It is ubiquitous in *nix
environments, being bundled with all *nix flavors. It
is also heavily used by routers, switches, and other
network applicances where its ability to transmit logs
over the network, its simplicity, and the availability
of free implementations make it a natural choice. It
has even crept into usage in the Microsoft Windows
world, where the native logging system does not work
well in large networked environments nor can it
interoperate with the logging mechanisms of other
operating systems.
Unfortunately, syslog also has some very serious and
widely recognized weaknesses.[8]
1. Message authenticity is based on easy-to-spoof
IP addresses.
2. There is no mechanism to assure message
confidentiality or integrity.
3. The protocol does not guarantee delivery of
messages, or ordered delivery of messages.
INTERNET DRAFT Event Logging Requirements August 2000
4. The relatively unstructure nature of syslog
messages makes them difficult to filter and/or
analyze.
Several systems have been devised to improve upon
syslog, and the rest of this section will address
these.
5.2 Schneier and Kelsey
Bruce Schneier and John Kelsey of Counterpane Internet
Security presented a paper at the the Seventh USENIX
Security Symposium on cryptographic mecahanisms for
preserving message integrity and confidentiality when
storing log messages on an untrusted system.[9]. They
also presented a follow-up paper on accessing those
logs over a network at the Second International
Workshop on the Recent Advances in Intrusion Detection
(RAID '99).[10] To the best of my knowledge, no
practical implementations of this system has been
built.
5.3 Nsyslogd
Nsyslogd is a drop-in syslog replacement developed by
Darren Reed of The Australian National University that
sports improved filtering based on regular expressions
and guaranteed message delivery and ordering through
the uses of TCP as a transport mechanism.[11]
5.4 Secure Syslog
Secure Syslog is another drop-in syslog replacement,
this one developed by Core-SDI. It offers message
integrity and confidentiality guarantees through the
use of Core-SDI's PEO-1 cryptographic protocol.[12]
5.5 ULM
ULM is a toolset developed by Herve Schauer
Consultants to query syslog based logs that are stored
in a structured format. The original version of this
tool used a format described in a now-expired Internet
Draft by Abela and Debeaupuis[13]. A newer version
uses an XML schema developed by this author and based
on the original ULM format.[14]
5.6 syslog-ng
Syslog-ng is another drop-in syslog replacement, this
one developed by Balazs Scheidler of BaliBit
Computing. Like nsyslogd, it offers improved
filtering and guaranteed message delivery and
INTERNET DRAFT Event Logging Requirements August 2000
ordering. However, it goes one step further by also
offering over-the-wire confidentiality through the use
of TLS, and message integrity through the use of
digital signatures.[15]
6. Conclusions
Although several systems have been developed to improve
upon the ubiquitous syslog system, none adequately
addresses all the requirements identified here. It is
hoped that this excercise of identifying these
requirements will provide the designers of these and
other logging systems with the input they need to address
all these issues. In particular, this document and the
process behind its development are playing an important
role in the development of system logging protocols being
carried out by the IETF Security Issues in Network Event
Logging Working Group.
In particular, this process has identified the following
as areas that need to be addressed:
o Log message format structures to support
Identification of Even Threads, Event Correlation,
and Event Filters.
o Security mechanisms to support log message Access
Control Labels, Message Identification, Selective
Storage, Message Confidentiality, Message Integrity,
and System Availability.
7. References
[1] S. Bradner, "Key words for use in RFCs to
Indicate Requirement Levels." The Internet
Society. March 1997. Available from
ftp.ietf.org/rfc/rfc2119.txt.
[2] CSRG, "Unix Programmer's Manual, 4.3 Berkeley
Software Distribution, Virtual VAX-11 Version,"
Six Volumes and an Index Volume. University of
California Computer Systems Research Group, April
1986.
[3] Messages on the [EMAIL PROTECTED] mailing
list dated October, November and December 1999
from Alex Brown ([EMAIL PROTECTED]), Chris
Calabrese ([EMAIL PROTECTED]), Chris M.
Lonvick ([EMAIL PROTECTED]), Darren Reed
([EMAIL PROTECTED]), and Mark Roth
([EMAIL PROTECTED]). Mailing list archive available
from www.mail-archive.com/syslog-
INTERNET DRAFT Event Logging Requirements August 2000
sec%40employees.org.
[4] U.S. Department of Defense, Assistant Secretary
of Defense for Command, Control, Communications,
and Intelligence, "Department of Defense Trusted
Computer System Evaluation Criteria" (DoD
5200.28-STD). December 1985. Available from
www.radium.ncsc.mil/tpep/library/rainbow/5200.28-STD.html.
[5] Private correspondence with Magosanyi Arpad
([EMAIL PROTECTED]), March 2000.
[6] Private correspondence with Balazs Scheidler
([EMAIL PROTECTED]), March 2000.
[7] Private correspondence with Mark D. Roth
([EMAIL PROTECTED]), March 2000.
[8] Private correspondence with Chris M. Lonvick
([EMAIL PROTECTED]) relating to an IETF Internet
Draft being written by him on the syslog
protocol, July 2000.
[9] B. Schneier and J. Kelsey, "Cryptographic Support
for Secure Logs on Untrusted Machines," The
Seventh USENIX Security Symposium Proceedings,
USENIX Press, January 1998, pp. 53-62.
Available from www.counterpane.com/secure-
logs.html.
[10] B. Schneier and J. Kelsey, "Minimizing Bandwidth
for Remote Access to Cryptographically Protected
Audit Logs," Second International Workshop on the
Recent Advances in Intrusion Detection (RAID
'99), September 1999. Available from
www.counterpane.com/auditlog2.html.
[11] Darren Reed, Nsyslogd home web page,
coombs.anu.edu.au/~avalon/nsyslog.html.
[12] Core-SDI, README file for the Secure Syslog
package, 1998. Available from www.core-
sdi.com/english/slogging/ssyslog-dl.html.
[13] J. Abela and T. Debeaupuis, "Universal Format for
Logger Messages," offered as an Internet Draft,
May 1999.
[14] Private correspondance with Nicolas Jombart of
Herve Schauer Consultants
([EMAIL PROTECTED]), May 2000.
[15] BaliBit Computing, Syslog-ng home web page,
www.balabit.hu/products/syslog-ng.
8. Author's Address
C. Calabrese
26 Wellesley Road
Montclair, NJ, USA 07043
Phone: (201) 703-7218
EMail: [EMAIL PROTECTED]
Send comments regarding this memo to
INTERNET DRAFT Event Logging Requirements August 2000
[EMAIL PROTECTED]
Title: Requirements for a System Event Logging Protocol
begin:vcard n:Calabrese;Chris tel;work:201-703-7218 x-mozilla-html:TRUE org:Merck-Medco Managed Care, L.L.C.;Internet Infrastructure and Security adr:;;1900 Pollitt Drive;Fair Lawn;NJ;07410;USA version:2.1 email;internet:[EMAIL PROTECTED] title:Internet Security Administrator fn:Chris Calabrese end:vcard
