Project Idea 02 – journalctl Parser

This post published on my blog before

Hi everyone. Today I'm going to tell you about my project idea. Before this post, I published another one.

What's this journalctl?

Let's dig into journalctl man page using the below command;

man journalctl

We will see an output like that

journalctl may be used to query the contents of the systemd(1) 
journal as written by systemd-journald.service(8)

So, it's a command to get systemd logs and it uses systemd-journald.service

What is systemd-journal service

systemd-journald is a system service that collects and stores logging data. It creates and maintains structured, indexed journals based on logging information that is received from a variety of sources. These are can be message types;

• Kernel Messages
• Simple Log Messages
• Audit records

There are a lot of messages you can find.

Let's See Some journalctl Commands

If you're using journalctl without any parameters it will show full output;

journalctl

The output

-- Logs begin at Sat 2020-01-18 21:00:40 +03, end at Sat 2020-05-09 10:47:50 +03
Jan 18 21:00:40 opcode kernel: microcode: microcode updated early to revision 0x
Jan 18 21:00:40 opcode kernel: Linux version 5.3.0-26-generic (buildd@lgw01-amd6
Jan 18 21:00:40 opcode kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.3.0-26-g
Jan 18 21:00:40 opcode kernel: KERNEL supported cpus:
Jan 18 21:00:40 opcode kernel:   Intel GenuineIntel

You can get json output in pretty format

journalctl -o json-pretty

The output

{"__CURSOR":"s=a62023d453d2404c949ce66c81b4b97a;i=1;b=5bf547fda1f147129
        "__REALTIME_TIMESTAMP" : "1579370440473152",
        "__MONOTONIC_TIMESTAMP" : "5696941",
        "_BOOT_ID" : "5bf547fda1f147129ac28544e4d1b35f",
        "_SOURCE_MONOTONIC_TIMESTAMP" : "0",
        "_TRANSPORT" : "kernel",
        "PRIORITY" : "6",
        "SYSLOG_FACILITY" : "0",
        "SYSLOG_IDENTIFIER" : "kernel",
        "MESSAGE" : "microcode:microcodeupdatedearlytorevision0x27,date="_MACHINE_ID":"35bb650aeefb48379f3b1920848e2a5a","_HOSTNAME":"opcode"}//morepageshere

You can also get specific outputs for instance chrome's logs;

journalctl _COMM=chrome

The output

-- Logs begin at Sat 2020-01-18 21:00:40 +03, end at Sat 2020-05-09 10:47:50 +03
Jan 20 20:49:09 opcode chrome[8566]: Failed to load module "canberra-gtk-module"
Jan 20 20:49:09 opcode chrome[8566]: Failed to load module "canberra-gtk-module"
Jan 20 20:49:10 opcode audit[8804]: AVC apparmor="DENIED" operation="sendmsg" pr
Jan 20 20:50:24 opcode chromium_chromium.desktop[8566]: [9131:1:0120/205024.3276
Jan 20 20:50:24 opcode chromium_chromium.desktop[8566]: [9131:1:0120/205024.3

As you see, these are the oldest messages. What about the current boot's log or specific boot's messages?

To get a list of boots, use this command;

journalctl --list-boots

The output

-92 5bf547fda1f147129ac28544e4d1b35f Sat 2020-01-18 21:00:40 +03—Sat 2020-01-18 
-91 f6a4dc011a8847bb94572a02de1c8401 Sat 2020-01-18 21:25:32 +03—Sun 2020-01-19
// more than this

To see boot 91's message, use this command;

journalctl -b 91

There are many commands you should know.

What Will We Do?

As we see journalctl useful command to understand system or application logs. But it's also hard to understand. You have to use a terminal, you have to know all commands. (In this idea you have to) but the end-user may don't want to know all commands.

We can write a parser in our best programming language. It can be a web project or another terminal project or GUI application.

Users can filter logs between two dates

To do this idea, use this command;

journalctl -S"2020-01-01 00:00:00"-U"2020-01-02 00:00:00"

• -S: since
• -U: until

And search about this command

Users can filter logs by specific services

For example, you want to see logs for apache2 use this command;

journalctl -u apache2.service

• -u: unit

Users can filter logs by specific binary

For example, you want to see logs for chrome use this command;

journalctl _COMM=chrome

• _COMM: match for the script name is added to the query

Users can see all boots

I'm an end-user who wants to see all boots. But it's really hard to see for me. Use this command;

journalctl --list-boots

Users can see logs from different boots

For instance, we want to see the logs for boot 35, we should use this command;

journalctl -b 35

Users can filter logs by priority

To filter logs by priority use this command;

journalctl -p 0

You can specify the number or level key.

journalctl -p crit

• -p: priority

These are log levels;

• 0: emerg
• 1: alert
• 2: crit
• 3: err
• 4: warning
• 5: notice
• 6: info"
• 7: debug"

Technologies

You can use various technologies to achieve this idea. For example, golang really good programming language. I believe you can do that in Python easily. I'll choose NodeJS to do that.

EOL

Actually these are my thoughts. You can extend them. Your project will have better features than my project's features.

Sorry for the grammar mistakes.

Thanks for reading ^_^ and if there is something wrong, tell me.

Resources

These resources helped a lot while thinking of this idea. I learned many new things. Remember that you can learn new things while thinking about something.

• https://www.howtogeek.com/499623/how-to-use-journalctl-to-read-linux-system-logs/
• https://www.linode.com/docs/quick-answers/linux/how-to-use-journalctl/
• https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs
• https://stackabuse.com/executing-shell-commands-with-node-js/
• https://unix.stackexchange.com/questions/225401/how-to-see-full-log-from-systemctl-status-service
• https://www.thegeekdiary.com/beginners-guide-to-journalctl-how-to-use-journalctl-to-view-and-manipulate-systemd-logs/
• https://coreos.com/os/docs/latest/reading-the-system-log.html
• https://net2.com/how-to-analyze-linux-systemd-logs-using-journalctl-advanced-filtering-options/
• https://www.maketecheasier.com/use-journalctl-read-linux-system-logs/
• https://www.shellhacks.com/journalctl-tail-service-logs-systemd-journal/
• https://www.freedesktop.org/software/systemd/man/journalctl.html
• https://www.golinuxcloud.com/view-logs-using-journalctl-filter-journald/
• https://www.tecmint.com/manage-systemd-logs-using-journalctl/