If you are facing difficulty while systematically reading log files in Linux. Then you are in the right place. Simply doing cat to read a log file is a kinda basic way to inspect a log file. But when you want to filter data in an organized way based on time, then we use awk, grep, and sed commands.
The awk is a popular command-line tool used to manipulate data in files and generate a report based on a given pattern in the Linux system. While grep is a simple pattern tool that searches for patterns using a regular expression.
Today, we will filter /var/log/syslog file based on date range using both awk and grep command-line tools.
Filter Data Using the Awk Command
First, we filter the Syslog file based on the time range using the awk command as shown.
$ sudo awk -v start=07:00 -v stop=19:00 'start <= $2 && $2 < stop' /var/log/syslog
Sample Output
Dec 11 19:39:19 LINUX dbus-daemon[2742]: [session uid=1000 pid=2742] Activating via systemd: service name='org.freedesktop.Tracker1' unit='tracker-store.service' requested by ':1.3' (uid=1000 pid=2740 comm="/usr/libexec/tracker-miner-fs " label="unconfined") Dec 11 19:39:19 LINUX systemd[2727]: Starting Tracker metadata database store and lookup manager... Dec 11 19:39:19 LINUX systemd[1]: session-23.scope: Succeeded. Dec 11 19:39:19 LINUX dbus-daemon[2742]: [session uid=1000 pid=2742] Successfully activated service 'org.freedesktop.Tracker1'
Replace start=HH:MM with starting hour and minute, stop=HH:MM with final hour and minute.
If you want to check the log more easily and in a simple way, then you can follow the grep command utility.
Filter Data Using the Grep Command
I’ll show a few examples using grep like I want to check the log for Today, then I’ll pass the below command.
$ sudo grep -E " Dec 11" /var/log/syslog
Sample Output
Dec 11 12:07:35 LINUX /usr/libexec/gdm-x-session[1231]: (==) Log file: "/var/lib/gdm3/.local/share/xorg/Xorg.0.log", Time: Fri Dec 11 12:07:35 2020 Dec 11 12:08:39 LINUX /usr/libexec/gdm-x-session[2812]: (==) Log file: "/home/shen/.local/share/xorg/Xorg.1.log", Time: Fri Dec 11 12:08:39 2020 Dec 11 12:09:04 LINUX gnome-shell[3019]: GNOME Shell started at Fri Dec 11 2020 12:08:55 GMT+0530 (IST) Dec 11 14:24:41 LINUX /usr/libexec/gdm-x-session[1208]: (==) Log file: "/var/lib/gdm3/.local/share/xorg/Xorg.0.log", Time: Fri Dec 11 14:24:41 2020 Dec 11 14:25:13 LINUX /usr/libexec/gdm-x-session[2783]: (==) Log file: "/home/shen/.local/share/xorg/Xorg.1.log", Time: Fri Dec 11 14:25:13 2020 Dec 11 14:25:40 LINUX gnome-shell[3049]: GNOME Shell started at Fri Dec 11 2020 14:25:31 GMT+0530 (IST)
Now we will explain to you, how this command works “grep”. A grep is used to search regular patterns in the provided file.
The Parameters -E is called extended-regex that is used to set patterns and lastly, you need to provide a log file or log directory.
To get data between a range of date examples from 01 Dec to 10 Dec, use the below command.
$ sudo grep -E "Dec 1|Dec 10" syslog
Sample Output
Dec 11 19:48:04 LINUX systemd[1]: Starting Hostname Service... Dec 11 19:48:04 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1' Dec 11 19:48:04 LINUX systemd[1]: Started Hostname Service. Dec 11 19:48:34 LINUX systemd[1]: systemd-hostnamed.service: Succeeded. Dec 11 19:50:01 LINUX CRON[39178]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1) Dec 11 19:51:58 LINUX dbus-daemon[856]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.320' (uid=1000 pid=38994 comm="/usr/bin/gnome-screenshot --gapplication-service " label="unconfined") Dec 11 19:51:58 LINUX systemd[1]: Starting Hostname Service... Dec 11 19:51:58 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1' Dec 11 19:51:58 LINUX systemd[1]: Started Hostname Service.
Filter Data Using the Sed Command
Now we will show you how to filter data using the sed command line. To get data between a range of dates with certain time examples from 11 Dec 18:45:06 to 11 Dec 19:00:01, use the below command.
$ sudo sed -n '/Dec 11 18:45:06 /,/Dec 11 19:00:01/p' /var/log/syslog
Sample Output
Dec 11 18:45:06 LINUX dbus-daemon[856]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.301' (uid=1000 pid=35830 comm="/usr/bin/gnome-screenshot --gapplication-service " label="unconfined") Dec 11 18:45:06 LINUX systemd[1]: Starting Hostname Service... Dec 11 18:45:06 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1' Dec 11 18:45:06 LINUX systemd[1]: Started Hostname Service. Dec 11 18:45:36 LINUX systemd[1]: systemd-hostnamed.service: Succeeded. Dec 11 18:47:22 LINUX dbus-daemon[856]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.303' (uid=1000 pid=35830 comm="/usr/bin/gnome-screenshot --gapplication-service " label="unconfined") Dec 11 18:47:22 LINUX systemd[1]: Starting Hostname Service... Dec 11 18:47:22 LINUX dbus-daemon[856]: [system] Successfully activated service 'org.freedesktop.hostname1' Dec 11 18:47:22 LINUX systemd[1]: Started Hostname Service. Dec 11 18:47:52 LINUX systemd[1]: systemd-hostnamed.service: Succeeded. Dec 11 18:50:01 LINUX CRON[36057]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1) Dec 11 18:55:01 LINUX CRON[36206]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1) Dec 11 19:00:01 LINUX CRON[36473]: (shen) CMD (~/duckdns/duck.sh >/dev/null 2>&1)
The -n parameter with sed will not print the output of each line. Now the /pattern1/,/pattern2/ will be the expression. In the last -p parameters suggest sed verify the next expression according to a given pattern. After all this, you need to provide the log file or log file location.
If you have any query and issue related to the Date filter. Feel free to ask your query in the comment section.
