Difference between revisions of "Limit the size of .log files & the journal"

Marked this version for translation
m (added languages and translate tags)
(Marked this version for translation)
Line 2: Line 2:
__TOC__
__TOC__
<translate>
<translate>
= Introduction =
= Introduction = <!--T:1-->


  Note: 18-Feb-17: Updated the Journal section
  <!--T:2-->
Note: 18-Feb-17: Updated the Journal section
       & created a Support page in the forum.
       & created a Support page in the forum.
       12-Feb-14: Improved Journal section.  
       12-Feb-14: Improved Journal section.  
Line 10: Line 11:
       added content to the next section.
       added content to the next section.


<!--T:3-->
Log files & the systemd journal do the same thing in different ways. They keep a record of everything that happens on your computer system. This makes it possible to understand what is going right & what is going wrong. As an example, if your system had been infiltrated by an ssh attack, this could be verified in the log/journal. So these log files are good for more than tracking troublesome hardware, or driver problems, badly written network manager code or the plethora of other problems that the complex & dynamic GNU/Linux system has to deal with.  
Log files & the systemd journal do the same thing in different ways. They keep a record of everything that happens on your computer system. This makes it possible to understand what is going right & what is going wrong. As an example, if your system had been infiltrated by an ssh attack, this could be verified in the log/journal. So these log files are good for more than tracking troublesome hardware, or driver problems, badly written network manager code or the plethora of other problems that the complex & dynamic GNU/Linux system has to deal with.  


<!--T:4-->
These logs are an absolute blessing, as not all systems have them, & those server administrators who do have them sould be very grateful, as they can be the bread & butter of what they do.  
These logs are an absolute blessing, as not all systems have them, & those server administrators who do have them sould be very grateful, as they can be the bread & butter of what they do.  


<!--T:5-->
Generally, only server administrators have use for logs that go back any length of time. Few users who run distros on their desktop, notebook, netbook and such machines, have need to keep such huge log files or histories going back for many months or even years on their system. They are a waste of space & also makes viewing your log files more cumbersome.  
Generally, only server administrators have use for logs that go back any length of time. Few users who run distros on their desktop, notebook, netbook and such machines, have need to keep such huge log files or histories going back for many months or even years on their system. They are a waste of space & also makes viewing your log files more cumbersome.  


<!--T:6-->
<br clear="all"/>
<br clear="all"/>


==== The first topic on this page will briefly cover the '''systemd journal''' ====
==== The first topic on this page will briefly cover the '''systemd journal''' ==== <!--T:7-->


<!--T:8-->
The systemd journal has taken the place of log files though it will happily run in parallel with the standard type log files. These are still created & maintained by default in Arch & Manjaro. If you delete syslog-ng & all of the /var/log/*log files on reboot you will find that some log files will be automatically created again. On my machine after having deleted syslog-ng & all of the /var/log/*.log files (except for the Xorg.0.log files), my machine now has the following (wtmp & btmp are created on boot by the /etc/logrotate.conf file) contents in my /var/log/ :
The systemd journal has taken the place of log files though it will happily run in parallel with the standard type log files. These are still created & maintained by default in Arch & Manjaro. If you delete syslog-ng & all of the /var/log/*log files on reboot you will find that some log files will be automatically created again. On my machine after having deleted syslog-ng & all of the /var/log/*.log files (except for the Xorg.0.log files), my machine now has the following (wtmp & btmp are created on boot by the /etc/logrotate.conf file) contents in my /var/log/ :


  journal/
  <!--T:9-->
journal/
  squid/
  squid/
  Xorg.0.log
  Xorg.0.log
Line 32: Line 39:
  wtmp
  wtmp


<!--T:10-->
<br clear="all"/>
<br clear="all"/>


==== The second topic will cover handling log files ====
==== The second topic will cover handling log files ==== <!--T:11-->


<!--T:12-->
This topic will go into far more depth, covering the use of the '''logrotate''' command, '''logrotate.conf''', the '''/etc/cron.daily cron.weekly cron.monthly cron.yearly''', and some ways to run created scripts, plus a mention of the '''crontab''' method of running a script also. I'll try to make this section accessible to as many people as possible, which means this will be a long page.
This topic will go into far more depth, covering the use of the '''logrotate''' command, '''logrotate.conf''', the '''/etc/cron.daily cron.weekly cron.monthly cron.yearly''', and some ways to run created scripts, plus a mention of the '''crontab''' method of running a script also. I'll try to make this section accessible to as many people as possible, which means this will be a long page.


<!--T:13-->
<br clear="all"/>
<br clear="all"/>


= The journal & the logs duplicate the same information =
= The journal & the logs duplicate the same information = <!--T:14-->


<!--T:15-->
You can read the text of the log files in a text editor, or using the '''cat''', '''more''', '''less''' & such commands as you would on any other text file. The journal on the other hand requires the '''journalctl''' command to be able to access its contents. The following is a good way to read the journal:
You can read the text of the log files in a text editor, or using the '''cat''', '''more''', '''less''' & such commands as you would on any other text file. The journal on the other hand requires the '''journalctl''' command to be able to access its contents. The following is a good way to read the journal:


  sudo journalctl
  <!--T:16-->
sudo journalctl


<!--T:17-->
<br clear="all"/>
<br clear="all"/>


=Read this its important=
=Read this its important= <!--T:18-->


  Note: '''etc/systemd/journald.conf.d/*.conf'''  
  <!--T:19-->
Note: '''etc/systemd/journald.conf.d/*.conf'''  
  overrides the file '''journald.conf'''
  overrides the file '''journald.conf'''


<!--T:20-->
'''man journald.conf'''
'''man journald.conf'''


<!--T:21-->
When packages need to customize the configuration, they can install configuration snippets in
When packages need to customize the configuration, they can install configuration snippets in
/usr/lib/systemd/*.conf.d/. Files in /etc/ are reserved for the local administrator, who may use
/usr/lib/systemd/*.conf.d/. Files in /etc/ are reserved for the local administrator, who may use
Line 66: Line 82:
ordering of the files.
ordering of the files.


<!--T:22-->
What that means is that you can create configuration files '''.conf''' in the '''/etc/systemd/journald.conf.d/''' directory, with suitable names of your choice. The content of these files take precedence over any other settings or configurations in systemd. Please bear that in mind when you read the following? In my cumbersome way I've tried to make it all too obvious...
What that means is that you can create configuration files '''.conf''' in the '''/etc/systemd/journald.conf.d/''' directory, with suitable names of your choice. The content of these files take precedence over any other settings or configurations in systemd. Please bear that in mind when you read the following? In my cumbersome way I've tried to make it all too obvious...


<!--T:23-->
<br clear="all"/>
<br clear="all"/>


== How to set a maximum size limit for the journal ==
== How to set a maximum size limit for the journal == <!--T:24-->


<!--T:25-->
There is usually no need to interfere with the maximum size of the journal, as it has been built to monitor the amount of free space on the partition where it exists & will shrink itself by deleting the oldest entries when a shortage of space demands it.
There is usually no need to interfere with the maximum size of the journal, as it has been built to monitor the amount of free space on the partition where it exists & will shrink itself by deleting the oldest entries when a shortage of space demands it.


<!--T:26-->
Use your favorite text editor with root privileges, (starting it with '''sudo''' will do the job).
Use your favorite text editor with root privileges, (starting it with '''sudo''' will do the job).


  Note: the name '''size'''.conf is user created.
  <!--T:27-->
Note: the name '''size'''.conf is user created.


<!--T:28-->
With your text editor create a file called '''size.conf''' in the following location '''/etc/systemd/journald.conf.d/size.conf''' The following sets the maximum file size to a 50 MB limit for the '''/var/log/journal''' . ''(The SystemMaxUse=250M is for use with logrotate, which is looked at in the 2nd section of this page. Talk to papajoke on the forum if you need help with logrotate & systemd.)''
With your text editor create a file called '''size.conf''' in the following location '''/etc/systemd/journald.conf.d/size.conf''' The following sets the maximum file size to a 50 MB limit for the '''/var/log/journal''' . ''(The SystemMaxUse=250M is for use with logrotate, which is looked at in the 2nd section of this page. Talk to papajoke on the forum if you need help with logrotate & systemd.)''


  [Journal]
  <!--T:29-->
[Journal]
  SystemMaxUse=250M
  SystemMaxUse=250M
  SystemMaxFileSize=50M
  SystemMaxFileSize=50M


<!--T:30-->
You can also limit the content of the journal by specifying the level of the data to be added to the journal. This is done by creating & editing the file '''/etc/systemd/journald.conf.d/level.conf'''
You can also limit the content of the journal by specifying the level of the data to be added to the journal. This is done by creating & editing the file '''/etc/systemd/journald.conf.d/level.conf'''


  Note: the name '''level'''.conf is user created.
  <!--T:31-->
Note: the name '''level'''.conf is user created.


  [Journal]
  <!--T:32-->
[Journal]
  # not save all levels but only 0 to 4
  # not save all levels but only 0 to 4
  MaxLevelStore=warning
  MaxLevelStore=warning


<!--T:33-->
<br clear="all"/>
<br clear="all"/>


=The Journalctl command - a quick reference [http://www.freedesktop.org/software/systemd/man/journalctl.html]=
=The Journalctl command - a quick reference [http://www.freedesktop.org/software/systemd/man/journalctl.html]= <!--T:34-->


  Note: Following are few pointers on things
  <!--T:35-->
Note: Following are few pointers on things
  you can do to make using journalctl quicker,  
  you can do to make using journalctl quicker,  
  easier & more effective, on your system.
  easier & more effective, on your system.


==You don't have to use "sudo" with journalctl==
==You don't have to use "sudo" with journalctl== <!--T:36-->


<!--T:37-->
Add your '''user''' to '''adm''' group. This gives your <user> full use of the '''journalctl''' command. No more need to use sudo.  
Add your '''user''' to '''adm''' group. This gives your <user> full use of the '''journalctl''' command. No more need to use sudo.  
Swap "handy" for your username in the following:
Swap "handy" for your username in the following:
Line 107: Line 136:
  # usermod -a -G adm handy
  # usermod -a -G adm handy


==See the whole line of the journalctl output text==
==See the whole line of the journalctl output text== <!--T:38-->


<!--T:39-->
You can pipe the output of journalctl to a file or to a text display tool like "More" or "Less", as follows:
You can pipe the output of journalctl to a file or to a text display tool like "More" or "Less", as follows:
    
    
Line 115: Line 145:
Doing so, gives you a means of avoiding the truncation of output which some system displays configurations may experience.
Doing so, gives you a means of avoiding the truncation of output which some system displays configurations may experience.


===Use a ~/.bashrc alias to make this easy===
===Use a ~/.bashrc alias to make this easy=== <!--T:40-->


<!--T:41-->
I use the following ~/.bashrc alias:
I use the following ~/.bashrc alias:
   
   
Line 123: Line 154:
On entering '''errors''' in the Terminal, all errors or worse since the last boot are sent to (piped) to the Terminal based text display tool called '''Less''' which wraps the text output of the journalctl command. Apart from anything else, it makes the errors more useful for anyone reading them in the forum!
On entering '''errors''' in the Terminal, all errors or worse since the last boot are sent to (piped) to the Terminal based text display tool called '''Less''' which wraps the text output of the journalctl command. Apart from anything else, it makes the errors more useful for anyone reading them in the forum!


<!--T:42-->
Type '''Q''' (upper or lower case) to close "Less".
Type '''Q''' (upper or lower case) to close "Less".




=== Access to full journal containing info from the system & users: ===
=== Access to full journal containing info from the system & users: === <!--T:43-->


  $ journalctl
  <!--T:44-->
$ journalctl


=== Live view, shows the last 10 lines of the journal & all content as it happens: ===
=== Live view, shows the last 10 lines of the journal & all content as it happens: === <!--T:45-->


  $ journalctl -f
  <!--T:46-->
$ journalctl -f


== Basic filtering: ==
== Basic filtering: == <!--T:47-->


=== Shows all output to the journal since the last boot:===
=== Shows all output to the journal since the last boot:=== <!--T:48-->


  $ journalctl -b
  <!--T:49-->
$ journalctl -b


=== Shows all output with priority level ERROR & worse, since last boot: ===
=== Shows all output with priority level ERROR & worse, since last boot: === <!--T:50-->


  $ journalctl -b -p err
  <!--T:51-->
$ journalctl -b -p err


<!--T:52-->
Following is the above command with its output sent to a file called '''-ERRORS''' in your ''/home/<user>'' directory. Having the '''-''' at the beginning of the name should cause the file to be shown at the top of the list of files when viewing the contents of your '''~/''' (''/home/<user>'') directory. This command makes it easy to copy the contents of the -ERRORS file, & then paste it to the forum. Doing so allows us to display ALL of the command's output instead of only being able to cut & paste the truncated lines from our terminal:
Following is the above command with its output sent to a file called '''-ERRORS''' in your ''/home/<user>'' directory. Having the '''-''' at the beginning of the name should cause the file to be shown at the top of the list of files when viewing the contents of your '''~/''' (''/home/<user>'') directory. This command makes it easy to copy the contents of the -ERRORS file, & then paste it to the forum. Doing so allows us to display ALL of the command's output instead of only being able to cut & paste the truncated lines from our terminal:


  $ Journalctl -b -p err > -ERRORS
  <!--T:53-->
$ Journalctl -b -p err > -ERRORS


== Filtering based on time: ==
== Filtering based on time: == <!--T:54-->


=== Since yesterday: ===
=== Since yesterday: === <!--T:55-->


  $ journalctl --since=yesterday
  <!--T:56-->
$ journalctl --since=yesterday


=== Give a specific time period: ===
=== Give a specific time period: === <!--T:57-->


  $ journalctl --since=2012-10-15 --until="2011-10-16 23:59:59"
  <!--T:58-->
$ journalctl --since=2012-10-15 --until="2011-10-16 23:59:59"


=== Pick a specific service & time period: ===
=== Pick a specific service & time period: === <!--T:59-->


  $ journalctl -u httpd --since=00:00 --until=9:30
  <!--T:60-->
$ journalctl -u httpd --since=00:00 --until=9:30


== Point journalctl at specific devices, services, binaries ==
== Point journalctl at specific devices, services, binaries == <!--T:61-->


=== Look at a specific device: ===
=== Look at a specific device: === <!--T:62-->


  $ journalctl /dev/sdc
  <!--T:63-->
$ journalctl /dev/sdc


=== Check on a binary: ===
=== Check on a binary: === <!--T:64-->


  $ journalctl /usr/sbin/vpnc
  <!--T:65-->
$ journalctl /usr/sbin/vpnc


=== Check on the interlieved output from two specifics: ===
=== Check on the interlieved output from two specifics: === <!--T:66-->


  $ journalctl /usr/sbin/vpnc /usr/sbin/dhclient
  <!--T:67-->
$ journalctl /usr/sbin/vpnc /usr/sbin/dhclient


=== Show all systemd units that have been started in your journal: ===
=== Show all systemd units that have been started in your journal: === <!--T:68-->


  $ journalctl -F _SYSTEMD_UNIT
  <!--T:69-->
$ journalctl -F _SYSTEMD_UNIT


<!--T:70-->
You can then interrogate the journal specifying any of those units.
You can then interrogate the journal specifying any of those units.


== A summation ==
== A summation == <!--T:71-->


<!--T:72-->
The Systemd Journal is capable of advanced functions beyond what has been mentioned here. The above is very good food for thought for people that are wondering if they need to be running '''syslog-ng''' or the like that creates most of the '''/var/log/*log''' files on their systems.
The Systemd Journal is capable of advanced functions beyond what has been mentioned here. The above is very good food for thought for people that are wondering if they need to be running '''syslog-ng''' or the like that creates most of the '''/var/log/*log''' files on their systems.


<!--T:73-->
By experimenting with the above commands one can make an informed decision for themselves, though as mentioned at the beginning of the Journal section, Arch & therefore Manjaro still run both the new systemd journal & the old style log file system in parallel. So if you find the /var/log/*log files to be redundant & you want to be rid of them, various methods would be effective.
By experimenting with the above commands one can make an informed decision for themselves, though as mentioned at the beginning of the Journal section, Arch & therefore Manjaro still run both the new systemd journal & the old style log file system in parallel. So if you find the /var/log/*log files to be redundant & you want to be rid of them, various methods would be effective.


<!--T:74-->
As of this writing I'm running my system with '''syslog-ng''' (& its dependency) deleted. I deleted all of the log files from the /var/log directory (leaving any that are in their own sub-directories), except for Xorg.0.log , Xorg.0.old , lastlog , btmp & wtmp, (pacman.log turned up when pacman was used, depending on what you have installed on your system, you may have applications that create their own logs - which can be turned off - too). (Note: These days I'm systemd free as I've been very happily using the OpenRC init system instead.)  
As of this writing I'm running my system with '''syslog-ng''' (& its dependency) deleted. I deleted all of the log files from the /var/log directory (leaving any that are in their own sub-directories), except for Xorg.0.log , Xorg.0.old , lastlog , btmp & wtmp, (pacman.log turned up when pacman was used, depending on what you have installed on your system, you may have applications that create their own logs - which can be turned off - too). (Note: These days I'm systemd free as I've been very happily using the OpenRC init system instead.)  


<!--T:75-->
<br clear="all"/>
<br clear="all"/>


= Managing /var/log/* files =
= Managing /var/log/* files = <!--T:76-->


== Introducing Logrotate & friends ==
== Introducing Logrotate & friends == <!--T:77-->


<!--T:78-->
What is this Logrotate? [http://linux.die.net/man/8/logrotate] logrotate is a powerful tool used to manage the log files created by system processes. It can be instructed to automatically compress, rename in a variety of ways, remove logs, to do all of this & more in a way that maximizes the convenience of logs & conserves your system's resources. An enormous amount of control is available to users including running scripts on your rotated files.  
What is this Logrotate? [http://linux.die.net/man/8/logrotate] logrotate is a powerful tool used to manage the log files created by system processes. It can be instructed to automatically compress, rename in a variety of ways, remove logs, to do all of this & more in a way that maximizes the convenience of logs & conserves your system's resources. An enormous amount of control is available to users including running scripts on your rotated files.  


<!--T:79-->
A problem I face in trying to make this article about logrotate as simple as possible is that logrotate can be called in so many ways, & these ways are not mutually exclusive.  
A problem I face in trying to make this article about logrotate as simple as possible is that logrotate can be called in so many ways, & these ways are not mutually exclusive.  


<!--T:80-->
For example, logrotate can be called to run on a file, or multiple files in any combination or multiple of '''hourly, daily, weekly, monthly & yearly''', via scripts placed in the /etc/ in the already existing directories '''hourly daily weekly monthly''' the '''yearly''' directory can be added if required. '''crontab''' [http://www.adminschoice.com/crontab-quick-reference/] can be used to run logrotate or scripts as complex as a person needs. logrotate can be combined with other tools in anyway that a user can come up with to process these rotated files at any time & frequency.
For example, logrotate can be called to run on a file, or multiple files in any combination or multiple of '''hourly, daily, weekly, monthly & yearly''', via scripts placed in the /etc/ in the already existing directories '''hourly daily weekly monthly''' the '''yearly''' directory can be added if required. '''crontab''' [http://www.adminschoice.com/crontab-quick-reference/] can be used to run logrotate or scripts as complex as a person needs. logrotate can be combined with other tools in anyway that a user can come up with to process these rotated files at any time & frequency.


<br clear="all"/>
<br clear="all"/>
==== The scope of this article ====
==== The scope of this article ==== <!--T:81-->


<!--T:82-->
That said, much of the power of logrotate is for the benefit of those administering servers & will not be dealt with in the following. Though what we will deal with can be used on more than just our log files. We can use logrotate to backup any other files that we choose. I will expand on this at a later date.
That said, much of the power of logrotate is for the benefit of those administering servers & will not be dealt with in the following. Though what we will deal with can be used on more than just our log files. We can use logrotate to backup any other files that we choose. I will expand on this at a later date.


<!--T:83-->
<br clear="all"/>
<br clear="all"/>


== /etc/logrotate.conf & /etc/logrotate.d ==
== /etc/logrotate.conf & /etc/logrotate.d == <!--T:84-->


<!--T:85-->
The logrotate.conf configuration file largely dictates logrotate's behaviour, it holds global settings, but most of the work that logrotate does is via script files stored in the '''/etc/logrotate.d''' directory, which take precedence over the global settings held in logrotate.conf.
The logrotate.conf configuration file largely dictates logrotate's behaviour, it holds global settings, but most of the work that logrotate does is via script files stored in the '''/etc/logrotate.d''' directory, which take precedence over the global settings held in logrotate.conf.


<!--T:86-->
Applications such as Apache, MySQL, Cups & others, put scripts into the /etc/logrotate.d directory to manage their log files.
Applications such as Apache, MySQL, Cups & others, put scripts into the /etc/logrotate.d directory to manage their log files.


<!--T:87-->
If you manually run the command '''sudo logrotate''', you will be presented with its usage template. logrotate needs you to specify the path to the script that you want it to use, including the logrotate.conf file which one may think due to its name would be automatically read, it is not.
If you manually run the command '''sudo logrotate''', you will be presented with its usage template. logrotate needs you to specify the path to the script that you want it to use, including the logrotate.conf file which one may think due to its name would be automatically read, it is not.


<!--T:88-->
To run logrotate & the logrotate.conf file you use the following command line:
To run logrotate & the logrotate.conf file you use the following command line:


  logrotate /etc/logrotate.conf
  <!--T:89-->
logrotate /etc/logrotate.conf


<br clear="all"/>
<br clear="all"/>
=== Can I store & run my script files elsewhere? ===
=== Can I store & run my script files elsewhere? === <!--T:90-->
   
   
A line exists in logrotate.conf that tells logrotate to run all of the scripts that exist in /etc/logrotate.d  
A line exists in logrotate.conf that tells logrotate to run all of the scripts that exist in /etc/logrotate.d  


  include /etc/logrotate.d
  <!--T:91-->
include /etc/logrotate.d


<!--T:92-->
We can use the '''include''' command in logrotate.conf to add other directories or use another directory instead of logrotate.d if we have reason to. Be careful what you do as there are files placed into the logrotate.d directory by other programs.
We can use the '''include''' command in logrotate.conf to add other directories or use another directory instead of logrotate.d if we have reason to. Be careful what you do as there are files placed into the logrotate.d directory by other programs.


<br clear="all"/>
<br clear="all"/>
=== My settings in logrotate.conf don't effect all of the .log files? ===
=== My settings in logrotate.conf don't effect all of the .log files? === <!--T:93-->


<!--T:94-->
Script files that are called via the logrotate.conf file take precedence over the global settings in logrotate.conf . That means that if you call a script from logrotate.conf that is located in the /etc/logrotate.d directory, then that script is more powerful than any of the global setting in logrotate.conf .
Script files that are called via the logrotate.conf file take precedence over the global settings in logrotate.conf . That means that if you call a script from logrotate.conf that is located in the /etc/logrotate.d directory, then that script is more powerful than any of the global setting in logrotate.conf .


<!--T:95-->
I use a script '''/etc.logrotate.d/rotate.logs''' that is set to work on all *.log files, & it does. The two that don't get rotated are called '''faillog''' & '''lastlog''' , apart from not having the '''.log''' file extension, these two files are not normal log files, they are accessed via terminal commands of the same name.
I use a script '''/etc.logrotate.d/rotate.logs''' that is set to work on all *.log files, & it does. The two that don't get rotated are called '''faillog''' & '''lastlog''' , apart from not having the '''.log''' file extension, these two files are not normal log files, they are accessed via terminal commands of the same name.


<!--T:96-->
<br clear="all"/>
<br clear="all"/>


=== Can I store my scripts where I want? ===
=== Can I store my scripts where I want? === <!--T:97-->


<!--T:98-->
Some applications such as Apache cups, drop scripts into /etc/logrotate.d to aid in their own self maintenance. We can use a location of our choosing for these or other scripts if we want. We just have to call its path in the /etc/logrotate.conf file, the same way, as shown in the following example:
Some applications such as Apache cups, drop scripts into /etc/logrotate.d to aid in their own self maintenance. We can use a location of our choosing for these or other scripts if we want. We just have to call its path in the /etc/logrotate.conf file, the same way, as shown in the following example:


  include /home/handy/.config/mylogrotate
  <!--T:99-->
include /home/handy/.config/mylogrotate


<!--T:100-->
Apart from adding our own scripts to /etc/logrotate.d (or any other path that we have chosen to include), we can also add scripts into any of the previously mentioned '''/etc/ cron.hourly cron.daily cron.weekly cron.monthly''' folders. OR we can add a script into any of these folders that suit our needs that runs the logrotate /etc/logrotate.conf command which will have the logrotate.conf file, direct logrotate to the default /etc/logrotate.d directory where we have our script(s). OR to another directory where we have our script & have included the path in logrotate.conf . whew!
Apart from adding our own scripts to /etc/logrotate.d (or any other path that we have chosen to include), we can also add scripts into any of the previously mentioned '''/etc/ cron.hourly cron.daily cron.weekly cron.monthly''' folders. OR we can add a script into any of these folders that suit our needs that runs the logrotate /etc/logrotate.conf command which will have the logrotate.conf file, direct logrotate to the default /etc/logrotate.d directory where we have our script(s). OR to another directory where we have our script & have included the path in logrotate.conf . whew!


<!--T:101-->
So you can see there are a variety of ways to call logrotate (let alone use it).
So you can see there are a variety of ways to call logrotate (let alone use it).


<!--T:102-->
<br clear="all"/>
<br clear="all"/>


== Some uses for Logrotate ==
== Some uses for Logrotate == <!--T:103-->


<!--T:104-->
For example, script block below does the following, listed line by line:
For example, script block below does the following, listed line by line:


<!--T:105-->
* '''/var/log/*.log {''' specifies the file or the files as this example uses a wild card that says all files ending in .log , the '''{''' starts the list of commands that will be used on the file(s) just specified.
* '''/var/log/*.log {''' specifies the file or the files as this example uses a wild card that says all files ending in .log , the '''{''' starts the list of commands that will be used on the file(s) just specified.


<!--T:106-->
* '''daily''' Here we are saying cycle these commands daily, we can also say weekly, monthly, yearly (or specify other times with crontab)[http://www.adminschoice.com/crontab-quick-reference/].
* '''daily''' Here we are saying cycle these commands daily, we can also say weekly, monthly, yearly (or specify other times with crontab)[http://www.adminschoice.com/crontab-quick-reference/].


<!--T:107-->
* '''size''' is where we can place a size limit that will cause a file to be rotated. I placed a '''1M''' one megabyte size limit in the example.
* '''size''' is where we can place a size limit that will cause a file to be rotated. I placed a '''1M''' one megabyte size limit in the example.


<!--T:108-->
* '''dateext''' this puts the date of the rotation on the new copy, so it would use this format: '''<file.name>.log-20130815'''
* '''dateext''' this puts the date of the rotation on the new copy, so it would use this format: '''<file.name>.log-20130815'''


<!--T:109-->
* '''rotate 7''' means keep 7 of our daily (in this script) backups, delete the oldest when it would become the 8th.
* '''rotate 7''' means keep 7 of our daily (in this script) backups, delete the oldest when it would become the 8th.


<!--T:110-->
* '''compress''' is obvious, it uses gzip by default & adds a .gz extension to your file, which will make it look like this: <file.name>.log.1.gz you can choose other compression methods, I'm not going into that here.
* '''compress''' is obvious, it uses gzip by default & adds a .gz extension to your file, which will make it look like this: <file.name>.log.1.gz you can choose other compression methods, I'm not going into that here.


<!--T:111-->
* '''delaycompress''' tells logrotate to compress the newly rotated file in the next cycle. This has advantages in ease of access & also if the file is still being written to by a process after it has been rotated.
* '''delaycompress''' tells logrotate to compress the newly rotated file in the next cycle. This has advantages in ease of access & also if the file is still being written to by a process after it has been rotated.


<!--T:112-->
* '''copytruncate''' this is a great option, as it copies the contents of the file to a new new file <file.name>.log.1 & then deletes the contents of the original file. You can have no permission problems crop up when you do it this way.
* '''copytruncate''' this is a great option, as it copies the contents of the file to a new new file <file.name>.log.1 & then deletes the contents of the original file. You can have no permission problems crop up when you do it this way.


<!--T:113-->
* '''notifempty''' do nothing if the file is empty, which makes good logical sense.
* '''notifempty''' do nothing if the file is empty, which makes good logical sense.


<!--T:114-->
* '''missingok''' if the file does not exist, give no error.
* '''missingok''' if the file does not exist, give no error.


<!--T:115-->
* '''}''' this curly bracket closes the block of commands.  
* '''}''' this curly bracket closes the block of commands.  


  /var/log/*.log {
  <!--T:116-->
/var/log/*.log {
   daily
   daily
   size 1M
   size 1M
Line 289: Line 372:
   }
   }


<!--T:117-->
The above script can be used as is, it does not need to be made executable, it just needs to be put somewhere that logrotate will see (in this example) every day.
The above script can be used as is, it does not need to be made executable, it just needs to be put somewhere that logrotate will see (in this example) every day.


<!--T:118-->
We can use the above script block as a template, easily removing parts & modifying its relatively simple settings. It can be duplicated in a script with each script block specifying custom settings tailored for individual files.
We can use the above script block as a template, easily removing parts & modifying its relatively simple settings. It can be duplicated in a script with each script block specifying custom settings tailored for individual files.


<!--T:119-->
<br clear="all"/>
<br clear="all"/>


== An Example that you can modify to suit ==
== An Example that you can modify to suit == <!--T:120-->


<!--T:121-->
I'll show how I have my system set (at the time of this writing), you can use the information already given on this page & other available on the web to fine tune your set up to suit your needs (if you have the need anyway).
I'll show how I have my system set (at the time of this writing), you can use the information already given on this page & other available on the web to fine tune your set up to suit your needs (if you have the need anyway).


=== Firstly - Be sure this file is here /etc/cron.daily/logrotate ===
=== Firstly - Be sure this file is here /etc/cron.daily/logrotate ===   <!--T:122-->
   
   
  #!/bin/sh
  #!/bin/sh
Line 327: Line 414:


<br clear="all"/>
<br clear="all"/>
=== Secondly - Create /etc/logrotate.d/rotate.logs using the following ===
=== Secondly - Create /etc/logrotate.d/rotate.logs using the following === <!--T:123-->


  ## rotate all /var/log files with names ending in log
  <!--T:124-->
## rotate all /var/log files with names ending in log
  /var/log/*log {
  /var/log/*log {
  ## cycle through these commands once per day
  ## cycle through these commands once per day
Line 349: Line 437:
   }  
   }  


<!--T:125-->
<br clear="all"/>
<br clear="all"/>


=== A Summary of the above example thus far ===
=== A Summary of the above example thus far === <!--T:126-->


<!--T:127-->
The First step puts a file into '''/etc/cron.daily''' which is an easy way to add the script to a daily cron job. Which means that script will be run everyday.
The First step puts a file into '''/etc/cron.daily''' which is an easy way to add the script to a daily cron job. Which means that script will be run everyday.


<!--T:128-->
It basically runs this command:
It basically runs this command:


  logrotate /etc/logrotate.conf
  <!--T:129-->
logrotate /etc/logrotate.conf


<!--T:130-->
As logrotate.conf goes through its list of commands it calls this one:
As logrotate.conf goes through its list of commands it calls this one:


  include /etc/logrotate.d
  <!--T:131-->
include /etc/logrotate.d


<!--T:132-->
Which means that any scripts that are inside of '''/etc/logrotate.d''' are also run.
Which means that any scripts that are inside of '''/etc/logrotate.d''' are also run.


<!--T:133-->
This brings us to the second step (above), where we created '''/etc/logrotate.d/rotate.logs''' . This script will be run everyday. The comments I added to the rotate.logs file above give a general idea of what it does. You can delete, modify & add to that script, but do it carefully.
This brings us to the second step (above), where we created '''/etc/logrotate.d/rotate.logs''' . This script will be run everyday. The comments I added to the rotate.logs file above give a general idea of what it does. You can delete, modify & add to that script, but do it carefully.


=== The effect of running /etc/logrotate.d/rotate.logs everyday ===
=== The effect of running /etc/logrotate.d/rotate.logs everyday === <!--T:134-->


<!--T:135-->
Is that any file in /var/log that had '''log''' at the end of its name will be processed by the commands in the '''rotate.logs''' script. This will back up these files to a new file '''<name>.log.1''' & empty the original file to size 0. Any previous copies with '''<name>.log.<number>''' will have their numbers bumped up one, until the day when they would have been given an 8, that is the day that they are deleted.
Is that any file in /var/log that had '''log''' at the end of its name will be processed by the commands in the '''rotate.logs''' script. This will back up these files to a new file '''<name>.log.1''' & empty the original file to size 0. Any previous copies with '''<name>.log.<number>''' will have their numbers bumped up one, until the day when they would have been given an 8, that is the day that they are deleted.


<!--T:136-->
As well as this rotating (copying) & renaming of files, all files will be compressed in gzip format on the next rotation. Which means that you always have the current file & yesterdays file in /var/log in uncompressed format.  
As well as this rotating (copying) & renaming of files, all files will be compressed in gzip format on the next rotation. Which means that you always have the current file & yesterdays file in /var/log in uncompressed format.  


<!--T:137-->
No files that are empty will be processed, & a file being missing will throw no errors.
No files that are empty will be processed, & a file being missing will throw no errors.


<!--T:138-->
<br clear="all"/>
<br clear="all"/>


=Support=
=Support= <!--T:139-->


<!--T:140-->
Following is a link to this page's forum counterpart where you can post any related feedback: [https://forum.manjaro.org/t/wiki-limit-the-size-of-log-files-the-journal/17875]
Following is a link to this page's forum counterpart where you can post any related feedback: [https://forum.manjaro.org/t/wiki-limit-the-size-of-log-files-the-journal/17875]
</translate>
</translate>


[[Category:Contents Page{{#translation:}}]]
[[Category:Contents Page{{#translation:}}]]