Real Time Log Monitoring in Web Browser with Papertrail

In this article I will explain how you can do real time log monitoring in web browser. Papertrail allow you free sing-up ( no credit/debit card required ) to monitor your log. It is hosted log management system for your virtual and dedicated servers. Real time tail is included in Papertrail. It will also provide search, alerts for your application and log platform.

Real Time Log Monitoring in Web Browser with Papertrail

Follow these steps to monitor your logs in real time in web browser with Papertail

Create Papertrail Acco0unt

First you will need to create an account on papertrail free sign-up page “https://papertrailapp.com/ “.

After  creating account , you can add your first system. Click on “Add System “ button which is located on right corner of papertrail home page.

Once clicking on “Add System” button it will re-directed to the papertrail installation instruction on your system. You will need to find out what logger system is installed on your system. In my case I am using “rsyslog.conf” for configuration.

In papertrail home page you can see one line “Your systems & apps will log to logs2.papertrailapp.com:13626. “. You will need this port later in this tutorial.

Follow the below command to find the logger system on your system.

Find logger on your running system

# ls –ld /etc/*syslog*

You will get back some output like below.

-rw-r--r-- 1 root root 2936 Nov 22 23:20 /etc/rsyslog.conf

drwxr-xr-x 2 root root 4096 Dec 17  2014 /etc/rsyslog.d

Above you can see my logger damen is  rsylog .

Logon as root and edit “/etc/rsyslog.conf “ with your favourite text editor like vim and nano. Paste below line at the end.

*.*          @logs2.papertrailapp.com:

In my case papertrail port is “13626”

# vim  /etc/rsyslog.conf

# /etc/rsyslog.conf Configuration file for rsyslog.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
# Default logging rules can be found in /etc/rsyslog.d/50-default.conf
#################
#### MODULES ####
#################$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog # provides kernel logging support
#$ModLoad immark # provides --MARK-- message capability# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
###########################
#### GLOBAL DIRECTIVES ####
############################
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat# Filter duplicated messages
$RepeatedMsgReduction on#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf*.* @logs2.papertrailapp.com:13626

Once edited your file with your papertrail port save it and exit.

Now restart and reload your rsyslog to take effect.

# service rsyslog restart
Or 
# service rsyslog reload

Download remote_syslog2

Now download “remote_syslog2” and execute below command.

# mkdir /root/download
# cd /root/download
# wget https://github.com/papertrail/remote_syslog2/releases/download/v0.19/remote_syslog_linux_i386.tar.gz\

Un-tar gz file.

# tar –zxvf remote_syslog_linux_i386.tar.gz
# cd remote_syslog
# cp remote_syslog /bin/

After downloaded remote_syslog run below command.

#remote_syslog   -p 13626   -d logs2.papertrailapp.com   --pid-file=/var/run/remote_syslog.pid   /var/log/httpd/example.com_access

Where:

• -p : Papertrail port
• -d : Destination
• –pid-file : remote_rsyslog pid file location
• /var/log/httpd/example.com_access : It is log my Apache log access log file you can change it with your log file.

Check remote_rsyslog is running or not.

#ps aux | grep remote_syslog

root     27960  0.0  0.2 796496  4596 ?        Sl   00:03   0:00 remote_syslog -p 13626 -d logs2.papertrailapp.com --pid-file=/var/run/remote_syslog.pid /var/log/httpd/example.com_access

We can see remote_rsyslog is running with 27960 process id.

After executed above command successfully now go to your papertrail home page and login again if logged out and reload the page.

Here you can see your system has been add to in system list. In my case my system host-name is m01.exampele.com.

Now click on the host-name to view real-time monitoring of your log.

I know you don’t want to run this command every-time when your system is rebooted. So don’t worry I am going to make init script to make it easy and execute remote_rsyslog command automatically when system rebooted.

init script for remote_rsyslog

Here I am going to guide, how we can make a init shell scripts for remote_rsyslog. you will need to create a remote_rsyslog file in /etc/init.d/ location and set the executable permission on it.

#cd /etc/init.d/

#vim remote_rsyslog
#!/bin/bash

### BEGIN INIT INFO
# Provides: remote_syslog
# Required-Start: $network $remote_fs $syslog
# Required-Stop: $network $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start and Stop
# Description: Runs remote_syslog
### END INIT INFO

# /etc/init.d/remote_syslog
#
# Starts the remote_syslog daemon
#
# chkconfig: 345 90 5
# description: Runs remote_syslog
#
# processname: remote_syslog

prog="remote_syslog"
port="13626"
dest="logs2.papertrailapp.com"
pid_dir="/var/run"
pid_file="$pid_dir/$prog.pid"

log_path="/var/log/httpd/example.com_access"

PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$PATH

RETVAL=0

is_running(){
[ -e $pid_file ]
}

start(){
echo -n $"Starting $prog: "

unset HOME MAIL USER USERNAME
$prog -p $port -d $dest --pid-file=$pid_file $log_path
RETVAL=$?
echo
return $RETVAL
}

stop(){
echo -n $"Stopping $prog: "
if (is_running); then
kill -9 `cat $pid_file`
rm -rvf /var/run/remote_syslog.pid >/dev/null
RETVAL=$?
echo
return $RETVAL
else
echo "$pid_file Stopped"
fi
}

status(){
echo -n $"Checking for $pid_file: "

if (is_running); then
echo "Running"
else
echo "Stopped"
fi
}

reload(){
restart
}

restart(){
stop
start
}

condrestart(){
is_running && restart
return 0
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
restart
;;
reload)
reload
;;
condrestart)
condrestart
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|reload}"
RETVAL=1
esac

exit $RETVAL

Save and exit.

Set permission

#chmod +x remote_rsyslog

init script is ready to work. We can check it with following command.

#service remote_rsyslog start

Starting remote_syslog:

# service remote_rsyslog stop

Stopping remote_syslog:

# service remote_rsyslog restart

Stopping remote_syslog: /var/run/remote_syslog.pid Stopped
Starting remote_syslog:

#service remote_rsyslog status

Checking for /var/run/remote_syslog.pid: Running

After checking remote_syslog we will use chkconfig command to enable service at boot time.

#chkconfig --add remote_rsyslog

#chkconfig remote_rsyslog on

#chkconfig --list | grep remote_rsyslog

remote_rsyslog  0:off   1:off   2:on    3:on    4:on    5:on    6:off

Your remote_rsyslog init script is setup-ed to start service automatically during system reboot.

I hope this article will help to monitor your log in real time  using web browser. If you have any queries and problem please comment in comment section.

Thanks:)