Pages

Wednesday, July 10, 2013

Syslog-ng : log Consolidation -Tailing Files

Consider a case where we need to use the syslog-ng for files that dont use any log4j configuration. Consider if you want the garbage logs to be sent to the syslog server , but these logs are being updated by the Java JVM , so there is no way for using any sort of log4j configuration for this. In this case we can use the syslog-ng in this way ,

1.Configure the /syslog-ng/etc/syslog-ng/syslog-ng.conf with

#----------------------------------------------------------------------
# Sources
#----------------------------------------------------------------------
source s_gc { file ( "/logs/jboss/ews/domains/abc/hello" follow_freq(1) flags(no-parse) ); };

#In the above case we are going to tail the file named “hello” for any content change and send the updated content to the syslog server

#----------------------------------------------------------------------
# Destinations
#----------------------------------------------------------------------
destination d_gc { udp ( "198.12.34.22" port(59503) ); };

#configure the Destination like

#----------------------------------------------------------------------
# Logging
#----------------------------------------------------------------------
log { source ( s_gc ); destination ( d_gc ); };

#configure the Log Source

The above configuration is done on the Sender side since we dont have any log4j sort of configuration here.

Once the configuration is done start the process using
/syslog-ng/sbin/syslog-ng -f /syslog-ng/etc/syslog-ng/syslog-ng.conf

We are starting the process here because , since we want to tail logs and send content to the syslog server . We don't have any sort of log4j configuration here to automatically send the content to the syslog server. In this case we start the process which will tail the logs continuously for the content change and push the modified content to the syslog server

Make sure the process is Up and running.

2.On the Receiver Side configure the syslog like

source s_jboss_abc { udp(ip(0.0.0.0) port(59503)); };

destination d_abc-GCserver { file("/logs/chipper/WEBINF/dev/jas/ews/jas/hello.log.$DAY"); };

filter f_abcCGCserver { ( match("hello") ); };
# we are using the filter names hello since the content will be coming from the file hello from Sender

log { source(s_jboss_abc); filter(f_abcCGCserver); destination(d_abc-GCserver); flags(final); };

Once the above configuration is done the receiver side , run the syslog process.

3.We can test this using
echo “some thing” >> hello

we are sending some content to the hello file which inturn checks by the syslog process running in sender side and will send that to the receiver.

The receiver will collect the log content and save them in the log file that we specified.