Difference between revisions of "Limit the size of .log files & the journal"

Limit the size of .log files & the journal (view source)

Revision as of 03:45, 14 August 2013

4,227 bytes added , 10 years ago

→‎Handling /var/log/*.log files

Anonymous user

imported>Handy

@@ Line 41: / Line 41: @@
 <br clear="all"/>
-= Handling /var/log/*.log files =
+= Handling /var/log/* files =
 == Introducing Logrotate & friends ==
-'''/etc/logrotate.conf''' holds general settings for the '''/usr/bin/logrotate''' command that are applied to ''some'' of the /var/log/ files.
+What is this Logrotate? [http://linux.die.net/man/8/logrotate] logrotate is a powerful tool used to manage the log files created by system processes. It can be instructed to automatically compress, rename in a variety of ways, remove logs, to do all of this & more in a way that maximizes the convenience of logs & conserves your system's resources. An enormous amount of control is available to users including running scripts on your rotated files.
-'''logrotate''' is called however often it is set to be called. Meaning, it can be called from any of the following directories that reside in '''/etc/''' ''cron.hourly'', ''cron.daily'', ''cron.weekly'', ''cron.monthly'', ''cron.yearly'', by placing a script that calls it into any of these previously mentioned directories. (All of which bar the cron.yearly exist by default in Manjaro.)
+A problem I face in trying to make this article about logrotate as simple as possible is that logrotate can be called in so many ways, & these ways are not mutually exclusive.
-Logrotate can also be called to run at any time via '''crontab''' .
+For example, logrotate can be called to run on a file, or multiple files in any combination or multiple of '''hourly, daily, weekly, monthly & yearly''', via scripts placed in the /etc/ in the already existing directories '''cron.d daily weekly monthly''' the '''yearly''' directory can be added if required. '''crontab''' [http://www.adminschoice.com/crontab-quick-reference/] can be used to run logrotate or scripts as complex as a person needs. logrotate can be combined with other tools in anyway that a user can come up with to process these rotated files at any time & frequency.
-The logrotate.conf , as well as whatever commands it contains, also runs any scripts that are located in the '''/etc/logrotate.d''' which it does via the following command:
+<br clear="all"/>
+==== The scope of this article ====
+That said, much of the power of logrotate is for the benefit of those administering servers & will not be dealt with in the following. Though what we will deal with can be used on more than just our log files. We can use logrotate to backup any other files that we choose. I will expand on this at a later date.
+<br clear="all"/>
+== /etc/logrotate.conf & /etc/logrotate.d ==
+The logrotate.conf configuration file largely dictates logrotate's behaviour, it holds global settings, but most of the work that logrotate does is via script files stored in the '''/etc/logrotate.d''' directory, which take precedence over the global settings held in logrotate.conf.
+Applications such as Apache, MySQL, Cups & others, put scripts into the /etc/logrotate.d directory to manage their log files.
+If you manually run the command '''sudo logrotate
+''' , you will be presented with its usage template. logrotate needs you to specify the path to the script that you want it to use, including the logrotate.conf file which one may think due to its name would be automatically read, it is not.
+To run logrotate & the logrotate.conf file you use the following command line:
+ logrotate /etc/logrotate.conf
+<br clear="all"/>
+=== Can I store & run my script files elsewhere? ===
+A line exists in logrotate.conf that tells logrotate to run all of the scripts that exist in /etc/logrotate.d
   include /etc/logrotate.d
-Some installed applications drop scripts into /etc/logrotate.d to aid in their own maintenance one way or another. We can use another location of our choosing for these or other scripts if we want, we just have to call its path the same way, as in the following is an example:
+We can use the '''include''' command in logrotate.conf to add other directories or use another directory instead of logrotate.d if we have reason to. Be careful what you do as there are files placed into the logrotate.d directory by other programs.
+<br clear="all"/>
+=== My settings in logrotate.conf don't effect all of the .log files? ===
+Script files that are called via the logrotate.conf file take precedence over the global settings in logrotate.conf . So if we have a script that is set to work on all *.log files, the few that don't get rotated may need to have scripts written that specify them individually & are stored in the /etc/logrotate.d directory.
+<br clear="all"/>
+=== Can I store my scripts where I want? ===
+Some applications such as Apache cups, drop scripts into /etc/logrotate.d to aid in their own self maintenance. We can use a location of our choosing for these or other scripts if we want. We just have to call its path in the /etc/logrotate.conf file, the same way, as shown in the following example:
   include /home/handy/.config/mylogrotate
-Apart from adding our own scripts to /etc/logrotate.d (or any other path that we have included, we can also add scripts into the previously mentioned /etc/ cron.hourly, cron.daily, cron.weekly, cron.monthly, folders.
+Apart from adding our own scripts to /etc/logrotate.d (or any other path that we have chosen to include), we can also add scripts into any of the previously mentioned '''/etc/ cron.hourly cron.daily cron.weekly cron.monthly''' folders. OR we can add a script into any of these folders that suit our needs that runs the logrotate /etc/logrotate.conf command which will have the logrotate.conf file, direct logrotate to the default /etc/logrotate.d directory where we have our script(s). OR to another directory where we have our script & have included the path in logrotate.conf . whew!
-So you can see there are multiple ways to call logrotate.
+So you can see there are a variety of ways to call logrotate (let alone use it).
 <br clear="all"/>
 == Some uses for Logrotate ==
-For example, if we wanted to rotate all of the .log files in our /var/log/ directory (some .log files will not be affected by this command) on a daily basis; keep the rotated files for 7 days before they are deleted (meaning there would be 7 separate log files for the week); compress the files but do it on the next cycle; do it in a fashion where the original contents of the file is copied before the original file's contents is deleted; do nothing if the file is empty; if the file is missing don't give an error; we should be able to do that with the following:
+For example, script block below does the following, listed line by line:
+* '''/var/log/*.log {''' specifies the file or the files as this example uses a wild card that says all files ending in .log , the '''{''' starts the list of commands that will be used on the file(s) just specified.
+* '''daily''' Here we are saying cycle these commands daily, we can also say weekly, monthly, yearly (or specify other times with crontab)[http://www.adminschoice.com/crontab-quick-reference/].
+* '''size''' is where we can place a size limit that will cause a file to be rotated. I placed a '''1M''' one megabyte size limit in the example.
+* '''dateext''' this puts the date of the rotation on the new copy, so it would use this format: '''<file.name>.log-20130815'''
+* '''rotate 7''' means keep 7 of our daily (in this script) backups, delete the oldest when it would become the 8th.
+* '''compress''' is obvious, it uses gzip by default & adds a .gz extension to your file, which will make it look like this: <file.name>.log.1.gz you can choose other compression methods, I'm not going into that here.
+* '''delaycompress''' tells logrotate to compress the newly rotated file in the next cycle. This has advantages in ease of access & also if the file is still being written to by a process after it has been rotated.
+* '''copytruncate''' this is a great option, as it copies the contents of the file to a new new file <file.name>.log.1 & then deletes the contents of the original file. You can have no permission problems crop up when you do it this way.
+* '''notifempty''' do nothing if the file is empty, which makes good logical sense.
+* '''missingok''' if the file does not exist, give no error.
+* '''}''' this curly bracket closes the block of commands.
   /var/log/*.log {
    daily
+  size 1M
+  dateext
    rotate 7
    compress
@@ Line 79: / Line 134: @@
     }
+The above script can be used as is, it does not need to be made executable, it just needs to be put somewhere that logrotate will see (in this example) every day.
-Now if we put the previous into a script by itself, or with other similar blocks of code that have been tailored to suit different files contained in /var/log/ , into the /etc/logrotate.d/ directory & we have /etc/logrotate.conf being called daily, then logrotate.conf will run any scripts that we have in logrotate.d everyday for us thus our log files will be rotated as we have instructed.
+We can use the above script block as a template, easily removing parts & modifying its relatively simple settings. It can duplicated in a script with each script block specifying custom settings tailored for individual files.
 <br clear="all"/>