Log Functions

At present, although there are many excellent open source log packages for us to choose from, in a large system, these open source log packages may not be able to meet our customized needs. At this time, we need to develop our own log packages.

These log packages may be based on one or several open source log packages, or they may be newly developed log packages. So when developing a log package, what functions do we need to implement and how to implement them? Next, let’s talk about it in detail.

According to the importance of the functions, I divide the functions that the log package needs to implement into basic functions, advanced functions and optional functions.

Table of contents

Youtube Video

Basic Function

Basic functions are essential functions for excellent log packages. They can meet most usage scenarios and are suitable for some small and medium-sized projects. A log package should have the following 4 basic functions.

1. Support basic log information

The log package needs to support basic log information, including timestamp, file name, line number, log level, and log information.

Timestamps can record the time when logs occur. When locating problems, we often need to restore the request process based on timestamps and check the context under the same timestamp to locate the problem.

The file name and line number allow us to locate the print log location more quickly and find the problem code.

The log level can be used to determine the error type of the log. The most common usage is to directly filter out logs at the Error level so that the error point can be directly located, and then the cause of the error can be located by combining other logs.

Through the log information, we can know the specific cause of the error.

2. Support different log levels

Different log levels represent different log types. For example, an Error-level log indicates that the log is an error type. When troubleshooting, the error-level log will be checked first. A Warn-level log indicates that an exception has occurred, but it does not affect the program operation. If the result of the program execution does not meet expectations, you can refer to the Warn-level log to locate the exception. An Info-level log can assist us in debugging and record some useful information for later analysis.

Usually a logging package should implement at least 6 levels. I provide you with a table, arranged from low to high priority as follows:

When printing logs, a log call actually has two properties:

• Output level: When printing logs, we expect the output level of the logs. For example, if we call glog.Info("This is info message") to print a log message, the output log level is Info.
• Switch level: When starting the application, what output level of logs are expected to be printed. For example, when using glog, -v=4 means that only logs with a log level higher than 4 will be printed.

If the switch level is set to L, logs will be printed only when the output level >= L

3. Support custom configuration

Different operating environments require different log output configurations. For example, in the development and testing environment, the log level needs to be set to the Debug level for easy debugging; in the live network environment, the log level needs to be set to the Info level to improve the performance of the application. For another example, the live network environment usually outputs logs in JSON format for the convenience of log collection; the development and testing environment outputs logs in TEXT format for the convenience of viewing logs.

Therefore, our log package needs to be configurable, and different configurations should be used in different environments. Through configuration, the logging behavior can be changed without recompiling the code.

4. Supports output to standard output and files

Logs are always read, either output to standard output for developers to read in real time, or saved to files for developers to view later. Output to standard output and saving to files are the most basic functions of a logging package.

Advanced Features

1. Support multiple log formats

The log format is also a point we need to consider. A good log format is not only convenient for viewing logs, but also makes it easier for some log collection components to collect logs and connect to log search engines such as Elasticsearch.

A log package needs to provide at least the following two formats:

• TEXT format: Logs in TEXT format are easy to read and can be viewed during development and debugging.
• JSON format: Logs in JSON format can record more detailed information. The logs contain some common or custom fields for future query and analysis. They can also be easily collected and reported by log collection tools such as filebeat and logstash.

2. Ability to output by level

In order to quickly locate the required logs, a better approach is to output the logs by level. At least the error-level logs can be output to separate files.

3. Support structured logging

Structured Logging uses JSON or other encoding methods to structure logs, which makes it easier to use various tools such as Filebeat and Logstash Shipper to collect, filter, analyze, and search logs.

4. Support log rotation

In a large project, dozens of GB of logs may be generated in a day. In order to prevent the logs from filling up the disk space and causing server or program abnormalities, it is necessary to ensure that the logs are cut, compressed, and transferred when the log size reaches a certain level.

How to split? You can split by log size or by date. The log splitting, compression and transfer functions can be encapsulated based on some excellent open source packages on GitHub. For example, lumberjack can support archiving logs by size and date, and file-rotatelogs supports log cutting by hours.

As for the log rotation function, I don’t recommend adding it to the log package, because it will increase the complexity of the log package. I recommend using other tools to achieve log rotation. For example, in Linux systems, you can use Logrotate to rotate logs. Logrotate is powerful and is a professional log rotation tool.

5. Hook-capable

The Hook capability allows us to customize the log content. For example, when a certain level of log is generated, send an email or call the alarm interface to issue an alarm. Many excellent open source log packages provide the Hook capability, such as logrus and zap.

In a large system, log alerts are very important functions, but a better way to implement them is to make the alert capability a bypass function. The bypass function can ensure that the log package function is focused and concise. For example, logs can be collected to Elasticsearch and log alerts can be performed through ElastAlert.

Optional Features

1. Support color output

When developing and testing, you can turn on color output. Different levels of logs will be marked with different colors, so that we can easily find some Error and Warn level logs, which is convenient for development and debugging. When publishing to the production environment, you can turn off color output to improve performance.

2. Compatible with the standard library log package

Some early Go projects used the standard library log package extensively. If our log library is compatible with the standard library log package, we can easily replace the standard library log package. For example, logrus is compatible with the standard library log package.

Here, let’s look at a code that uses the standard library log package:

package main

import (
    "log"
)

func main() {
    log.Print("call Print: line1")
    log.Println("call Println: line2")
}

Simply replace “log” with “github.com/sirupsen/logrus” to switch to the standard library log package:

package main

import (
    log "github.com/sirupsen/logrus"
)

func main() {
    log.Print("call Print: line1")
    log.Println("call Println: line2")
}

3. Supports output to different locations

In a distributed system, a service will be deployed on multiple machines. If we want to view the logs, we need to log in to different machines separately, which is very troublesome. We would rather send the logs to Elasticsearch and view them on Elasticsearch.

We may also need to analyze the number of calls to a certain interface, the number of requests from a certain user, and other information from the logs, which requires us to process the logs. The general approach is to deliver the logs to Kafka, and the data processing service consumes the logs saved in Kafka to analyze information such as the number of calls.

In the above two scenarios, logs need to be delivered to components such as Elasticsearch and Kafka respectively. If our log package supports delivering logs to different destinations, it will be a very exciting feature, Of course, if the logs do not support delivery to different downstream components, such as Elasticsearch, Kafka, Fluentd, Logstash, etc., you can also use Filebeat to collect log files on the disk and then deliver them to downstream components.