Central syslog service
From EgeeWiki
Contents |
Central syslog over LBTE
Logging and Bookkeeping transport (LBTE) layer is used to transmit messages from local syslog daemon to the central syslog server. This transport is robust, reliable and secure. Clients are authenticated with the server and vice versa. LBTE ensures that non message is lost during transport from client to the server.
LBTE transport is available on AFS /software/lb-transport/
Building the client and the server
- Build gLite with enhancements for Kerberos
- Get the source code cvs -d :gserver:lindir.ics.muni.cz:/cvs/meta co syslog-lb
- Edit Makefile and set path to the gLite lib (default is /software/glite-lb-2.0/lib)
- Run make
- Two binaries will be built: lbte-serv and lbte-clnt
Building gLite
The system is almost solely based on current development release of LB as available from HEAD of the LB CVS. The adaptations necessary to build with Kerberos reside on the mechglue_branch of the org.glite.security.gss module that must be used instead of the HEAD version. Following modules are needed to build the syslog-lb system (built as either RPM or tar.gz packages):
- glite-jobid-api-c
- glite-lb-client
- glite-lb-common
- glite-lb-logger
- glite-lbjp-common-server-bones
- glite-lbjp-common-trio
- glite-security-gss
you will also need the cares library, which is available from the etics repository.
Installation and configuration of the Syslog LBTE Server
Server receives messages from the client and pass them to the UNIX socket where Syslog daemon listens.
Prerequisites:
- Syslog-NG
- GSSAPI library
- LBTE Transport
Installation of the LBTE Transport
Configuration of the Syslog-ng server
- edit /etc/syslog-ng/syslog-ng.conf and add next lines, after it restart the syslog-ng daemon
# Template for output from LBTE
template t_sysloglb {
template("$DATE>$MSG\n"); template_escape(no);
};
# Specify source UNIX socket of LBTE messages
source s_sysloglb {
unix-stream("/var/lib/lbte.server.sock" perm(0600));
};
# Specify destination file for LBTE messages
destination df_sysloglb {
file("/mnt/export/log/sysloglb/log" template(t_sysloglb));
};
# Log rule for LBTE
log {
source(s_sysloglb);
destination(df_sysloglb);
};
Run the LBTE Server
- Use /software/lb-transport/init.d-scripts/lbtsd and copy it to the local init.d directory
- Edit lbtsd file, each option is explained there
- Start server /etc/inid.d/lbtsd start
Installation and configuration of the Syslog LBTE Client
Client listens on the pipe or on the UNIX socket and sends all the messages to the LBTE server. Pipe is prefered. Client consists of three processes:
- lbte-clnt - listen on the syslog socket or pipe
- glite-lb-logd
- glite-lb-interl
Prerequisites:
- Syslog-NG or syslog
- GSSAPI library
- LBTE Transport
Configuration of the Syslog-ng client (pipe)
- edit /etc/syslog-ng/syslog-ng.conf and add lines below
- restart the syslog-ng daemon
# Specify target pipe
destination dp_lbte {
pipe("/var/lib/lbte.pipe");
};
# All messages are sent to the pipe
log {
source(s_all);
destination(dp_lbte);
};
Configuration of the Syslog-ng client (UNIX socket)
- edit /etc/syslog-ng/syslog-ng.conf and add lines below
- restart the syslog-ng daemon
# Specify target socket
destination dp_lbte {
unix-stream("/var/lib/lbte.sock");
};
# All messages are sent to the pipe
log {
source(s_all);
destination(dp_lbte);
};
Configuration of the Syslog server
- edit /etc/syslog.conf and add lines below and restart the syslog daemon
*.* |/var/lib/lbte.pipe
Run the LBTE Client
- Use /software/lb-transport/init.d-scripts/lbtcd and copy it to the local init.d directory
- Edit lbtcd file, each option is explained there
- Start client /etc/inid.d/lbtcd start
Notice: Option --ignore-lb-syslog-msg [msg] causes that LBTE client won't send messages containing [msg] to the central syslog server, [msg] has to be "glite-lb" in order to prevent loops in logging.
Central Syslog over Stunnel
Installation of stunnel
In case the stunnel from the distribution does not create stunnel user
/usr/sbin/useradd -c stunnel -d /var/lib/stunnel -s /bin/false -m -u 122 stunnel
Get latest version of stunnel from your distribution (version > 4.0 is recommended )
Server configuration: Edit /etc/stunnel/stunnel.conf
# Make symbolic links in /var/lib/stunnel/etc/host[key|cert].pem to /etc/grid-security/host[key|cert].pem cert = /etc/hostcert.pem key = /etc/hostkey.pem # Make symbolic link in /var/lib/stunnel/etc/certs to /etc/grid-security/certificates CApath = /etc/certs chroot = /var/lib/stunnel pid = /run/stunnel.pid setuid = stunnel setgid = nogroup verify = 2 debug = 0 [1980] accept = 147.251.3.70:5140 connect = 127.0.0.1:514
Client configuration: For stunnel > 0.4 Edit /etc/stunnel/stunnel.conf
client = yes # Make symbolic links in /var/lib/stunnel/etc/host[key|cert].pem to /etc/grid-security/host[key|cert].pem cert = /etc/hostcert.pem key = /etc/hostkey.pem # Make symbolic link in /var/lib/stunnel/etc/certs to /etc/grid-security/certificates CApath = /etc/certs chroot = /var/lib/stunnel pid = /run/stunnel.pid setuid = stunnel setgid = nogroup verify = 2 debug = 0 [syslog-ng] accept = 127.0.0.1:514 connect = 147.251.3.70:5140
For stunnel < 0.4 (hostcert.pem must contain both certificate and private key)
stunnel -c -D 0 -v 2 -a /etc/certs -d localhost:514 \ -r 147.251.3.70:5140 -s stunnel -g nogroup -p /etc/hostcert.pem \ -P /var/run/stunnel.pid
Startup script for Debian like systems
#! /bin/sh
### BEGIN INIT INFO
# Provides: stunnel
# Required-Start: $local_fs $network $time
# Required-Stop: $local_fs $network $time
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Starting stunnel daemon
# Description: Starting stunnel
### END INIT INFO#
#
# skeleton example file to build /etc/init.d/ scripts.
# This file should be used to construct scripts for /etc/init.d.
#
# Written by Miquel van Smoorenburg <miquels@cistron.nl>.
# Modified for Debian GNU/Linux
# by Ian Murdock <imurdock@gnu.ai.mit.edu>.
#
# Version: @(#)skeleton 1.8 03-Mar-1998 miquels@cistron.nl
# This file was customized by SZALAY Attila <sasa@debian.org>
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
test -f /usr/local/sbin/stunnel || exit 0
CONF="/etc/stunnel/stunnel.conf"
PIDFILE="/usr/local/var/run/stunnel/stunnel.pid"
if [ ! -f $CONF ]; then
echo "There is no configuration file $CONF"
exit 0
fi
STUNNEL="/usr/local/sbin/stunnel"
NAME="stunnel"
stunnel_start() {
echo -n "Starting stunnel: $NAME"
start-stop-daemon --start --quiet --exec "$STUNNEL" --pidfile "$PIDFILE" || { echo " start failed."; return 1; }
echo "."
return 0
}
stunnel_stop() {
echo -n "Stopping stunnel: $NAME"
echo $PIDFILE
start-stop-daemon --stop --quiet --name "$NAME" --retry 3 --pidfile "$PIDFILE" || { echo " not running."; return 1; }
echo "."
rm -f "$PIDFILE"
return 0
}
stunnel_reload() {
echo -n "Reload stunnel: $NAME"
if /sbin/syslog-ng -s
then
start-stop-daemon --stop --signal 1 --quiet --exec "$STUNNEL" --pidfile "$PIDFILE" || return 1
echo "."
return 0
else
echo " Failed"
return 1
fi
}
case "$1" in
start)
stunnel_start || exit 1
;;
stop)
stunnel_stop || exit 1
;;
reload|force-reload)
stunnel_reload || exit 1
;;
restart)
stunnel_stop
stunnel_start || exit 1
;;
*)
echo "Usage: /etc/init.d/$NAME {start|stop|restart|reload|force-reload}" >&2
exit 1
;;
esac
exit 0
Installation of Syslog-ng
Install syslog-ng, it is present in all main distributions.
Change syslog-ng startup script:
start)
...
if [ -f /usr/local/etc/stunnel/stunnel.conf \
-a -x /usr/local/sbin/stunnel ]; then
echo "Starting stunnel"
mkdir -p /var/run/stunnel/run
chown stunnel:stunnel /var/run/stunnel/run
/usr/local/sbin/stunnel
echo "Starting syslog-ng"
/usr/local/sbin/syslog-ng
fi
...
stop)
...
if [ -f /var/run/stunnel/run/stunnel.pid ]; then
syspid=`/usr/bin/cat /var/run/stunnel/run/stunnel.pid`
[ "$syspid" -gt 0 ] && kill -15 $syspid && \
echo "Killed stunnel"
fi
...
Configure syslog-ng.conf for the server:
options {
...
#Specifies whether to trust hostname as it is included in the log message.
keep_hostname(yes);
...
};
source s_net {
tcp(ip("127.0.0.1")
port(1981)
max-connections(1));
};
log {
source(s_net);
filter(f_auth);
destination(dn_auth);
};
log {
source(s_net);
filter(f_syslog);
destination(dn_syslog);
};
log {
source(s_net);
filter(f_daemon);
destination(dn_daemon);
};
log {
source(s_net);
filter(f_kern);
destination(dn_kern);
};
log {
source(s_net);
filter(f_user);
destination(dn_user);
};
log {
source(s_net);
filter(f_messages);
destination(dn_messages);
};
Configure logrotate: Add next content into the file and save it in /etc/logrotate.d
/mnt/export/log/*.log
{
daily
missingok
compress
create 600 root root
rotate 90
sharedscripts
postrotate
if [ -f /var/run/syslog-ng.pid ]; then \
kill -HUP `cat /var/run/syslog-ng.pid`; \
fi;
endscript
}
Configure syslog-ng.conf for the client: Add following lines into the syslog-ng.conf
options {
...
#Add Fully Qualified Domain Name instead of short hostname
use_fqdn(yes);
#Disable statistic log messages.
stats_freq(0);
...
};
# Remote syslog-ng server over stunnel
destination dn_net { tcp("127.0.0.1" port(1981)); };
# Add destination(dn_net) into log where are filters: f_auth, f_syslog, f_daemon, f_kern, f_messages
log {
...
destination(dn_net);
...
};
