By Philip Knerr

Introduction

By default, Amazon Linux systems on Elastic Compute Cloud (EC2) instances at Amazon Web Services (AWS) will automatically delete system log files after several weeks.

When hardening a server, you can improve security by disabling this automatic deletion process. This is useful because:

• In some environments, regulations require logs to be retained for a minimum period of time. (Please note that these instructions are not guaranteed to be sufficient to comply with any particular regulation.)
• If all else fails, system log files are useful for investigating security incidents. Suppose an intruder tampered with your database three months ago without being immediately detected. With the default log retention, evidence in the system log files which would have been helpful for investigating this breach is long gone.

The instructions below will help you avoid these issues by retaining system log files for at least a century. This effectively disables deletion of rotated log files, assuming you are not still using the EC2 instance a century from now!

Note that other flavors of the Linux operating system typically also rotate system log files. These instructions will likely work on other flavors of Linux, although this has not been tested.

The context and caveats in the Foreword apply to this article.

This information is current as of June 29, 2016.

Requirements

This article assumes you have already instantiated an Amazon EC2 instance which you would like to harden (e.g., make more secure).

Backing up log files

Having system logs dating back to when Gerald Ford was President on your EC2 instance doesn’t help you if an intruder deletes your log files to cover their tracks. Therefore, your system log files should be included in your backup processes.

Doing so also allows you to manually delete any system logs which you have confirmed to be included in your backups. This is desirable because retaining logs covering long periods of time will take up significant amounts of disk space. In fact, if you’re not careful, retained logs can completely fill up the partition where the logs are stored. This partition is usually the system partition. Filling all available space on this partition breaks many operations which expect this partition to have free space. Ironically, this includes logging itself.

Extending log retention

Edit the main log rotation configuration file, /etc/logrotate.conf, as the superuser. For example, to do so using the nano text editor, execute this command:

sudo nano /etc/logrotate.conf

Near the top of the file, there is a line which reads, rotate 4. This line controls how many rotated log files are retained if a more specific configuration entry does not apply. Change this line to read, rotate 5300. This causes most log files to be retained for 5300 weeks. You may optionally change the comment above this line accordingly.

Near the bottom of the file, there are two lines which read, rotate 1. These lines control how many rotated log files related to authentication are retained. Change these lines to read, rotate 1200. This causes these log files to be retained for 1200 months.

Then, edit the accounting log rotation configuration file, /etc/logrotate.d/psacct, as the superuser. For example, to do so using the nano text editor, execute this command:

sudo nano /etc/logrotate.d/psacct

Near the middle of the file, there is a line which reads, rotate 31. Change this line to read, rotate 36525. This causes the accounting log files to be retained for 36525 days.

Finally, edit the audit daemon configuration file, /etc/audit/auditd.conf, as the superuser. For example, to do so using the nano text editor, execute this command:

sudo nano /etc/audit/auditd.conf

A few lines from the top of the file, there is a line which reads, num_logs = 5. This line controls how many rotated log files related to the audit daemon are retained. Change this line to read, num_logs = 99. This causes up to 99 rotated audit daemon log files to be retained.

Note that the audit daemon log files are rotated when they reach a maximum size (and not necessarily after a specific time interval). Also, 99 is the maximum value allowed for this setting. As a result, it is not possible to specify that the audit daemon log files should be retained forever. Therefore, to retain these log files, they should be backed up offline (which ideally should happen anyway).

Applying a specific retention period

The instructions above can be adapted to apply a specific retention period to log files. To do so, adjust all rotate directives listed above to retain the desired number of files. The number after the word, rotate, specifies the number of rotated log files to be retained. The resulting time period is generally this number, multiplied by the frequency with which the log files are rotated. For example:

• If a log file is rotated daily, rotate 366 causes 366 days (or at least a year) of logs to be retained.
• If a log file is rotated weekly, rotate 53 causes 53 weeks (or just over a year) of logs to be retained.
• If a log file is rotated monthly, rotate 12 causes 12 months (or exactly a year) of logs to be retained.

Note that this all becomes slightly more complicated if a size directive is applicable. However, such a directive can only delay rotation of the current log file. As a result, it can only increase the retention period.

If applying a specific retention period, other files in the /etc/logrotate.d directory should be reviewed. This directory contains configuration files for the log rotation daemon. If a configuration file specifies a rotation period other than the default of weekly, you may need to add a rotate directive specifying a number of rotated log files which, in conjunction with the rotation period, results in the desired retention period.

Unfortunately, for the reasons noted above, it is difficult to enforce a specific retention period for the audit daemon log files.

Also, if you are backing up log files offline and then deleting them on the server, the real issue becomes retaining the backups during the retention period.

Conclusion

Your system log files will now be retained for a longer period of time. As a result, your system is now more secure. Congratulations!

However, to truly harden your system, extending log retention is just one of many necessary steps. The other necessary steps depend on your unique environment and the specific threats you need to counteract. Moreover, system hardening is an ongoing process, as new vulnerabilities and exploits are constantly being discovered.

–Phil Knerr