Manjaro Difference between pages "Using Manjaro for Beginners" and "Limit the size of .log files & the journal"

Difference between pages "Using Manjaro for Beginners" and "Limit the size of .log files & the journal"

From Manjaro
(Difference between pages)
imported>Kaimahi
 
imported>Handy
 
Line 1: Line 1:
= Welcome to Manjaro! =
= Introduction =


This guide provides a brief overview of some key knowledge points necessary to get the most out of using the Manjaro operating system.
NOTE: This page is still a work in progress.


= XFCE: Software Updates and Management =


[[File:pamac-updater-088.png|thumb|left|200px]]
Log files & the systemd journal do the same thing in different ways. They keep a record of everything that happens on your computer system. This makes it possible to understand what is going right & what is going wrong. As an example, if your system had been infiltrated by an ssh attack, this could be verified in the log/journal. So these log files are good for more than tracking troublesome hardware, or driver problems, badly written network manager code or the plethora of other problems that the complex & dynamic GNU/Linux system has to deal with.  


These logs are an absolute blessing, as not all systems have them, & any server administrators who does have them would be very grateful as they can be the bread & butter of what they do.


The '''Pamac Updater''' will automatically check and notify you of any available system updates.
Generally only server administrators have use for logs that go back any length of time. Few users who run distros on their desktop, Notebook, Netbook... machines, need to keep such huge log files, histories going back for many months or even years, on their system, are a waste of space & also makes viewing your log files more cumbersome.  


To install updates when listed, simply click the '''Apply''' button.
<br clear="all" />


==== The first topic on this page will briefly cover the '''systemd journal''' ====


[[File:pamac-manager-088.png|thumb|left|375px]]
The systemd journal has taken the place of log files though it will happily run in parallel with the standard type log files. These are still created & maintained by default in Arch & Manjaro, which means that even if you delete syslog-ng & all of the /var/log/*log files on reboot you will find the log files have been automatically created again, this will change in the future.


==== The second topic will cover handling log files ====


{{Tip| Pamac can install Packages from the AUR as well}}
This topic will go into far more depth, it will covering the use of the '''logrotate''' command, '''logrotate.conf''', the '''/etc/cron.daily cron.weekly cron.monthly cron.yearly''', some ways to run created scripts, & a mention of the '''crontab''' method of running a script also. I'll try to make this section accessible to as many people as possible, which means this will be a long page.


The '''Pamac Manager''' is a very simple yet powerful tool to add and remove software packages (applications) from your system. This can be found in the '''System''' menu category, or in Openbox, via Synapse or the desktop menu.
<br clear="all"/>


Upon launching, it will automatically check the official Manjaro Repositories for new and updated software. Once complete, simply enter the name or short description of what you want to install or remove, and click the '''Find''' button.
= The journal & the logs duplicate the same information =


All installed and available software matching your search will then be displayed on the right. Click the name of any result to see more information about it immediately below.
You can read the text of the log files in a text editor, or using the '''cat''', '''more''', '''less''' & such commands as you would on any other text file. The journal on the other hand requires the '''journalctl''' command to be able to access its contents. The following is a good way to read the journal:
<br clear="all" />


[[File:pamac-manager2-088.png|thumb|left|375px]]
sudo journalctl


<br clear="all"/>


A checkbox next to the package name will indicate whether it is installed or not.
== How to set a maximum size limit for the journal ==


* '''To install a package''', double-click the adjacent checkbox to mark it.
You need to use your text editor with root priviliges, starting it with '''sudo''' will do the job.


* '''To remove an installed package''', double-click the adjacent checkbox to clear it.
With a simple edit of the '''/etc/systemd/journald.conf''' you can set the maximum size limit of the '''/var/log/journal''' .


Once package boxes have been marked and/or cleared, you may undertake more searches before clicking the Apply (check mark) button to conform your choice(s). It really is that simple!
* Uncomment the following line & add the size limit that you want. I set mine to 50MB like this:
<br clear="all" />


= Openbox: Software Updates and Management =
SystemMaxUse=50M


[[File:octopi-note-088.png|thumb|left|375px]]
After a reboot, you should find that your journal is now somewhat less than 50MB (or whatever limit you set), as it needs to have some working space. Mine went to 38M from 391M.


See the troubleshooting section if the above did not work for you.


Openbox uses an application called '''Octopi''' for software updates and management. Notifcations will automatically appear on the desktop when new updates become available.
<br clear="all"/>
<br clear="all" />
== The Journalctl command - a quick reference [http://www.freedesktop.org/software/systemd/man/journalctl.html]==


=== Add user to adm group, gives <user> full use of the journalctl command: ===


[[File:octopi-088.png|thumb|left|375px]]
# usermod -a -G adm handy


{{Tip| Octopi can install Packages from the AUR (via yaourt) as well}}
===== Access to full journal containing info from the system & users: =====


* '''To install available updates''' click the menu icon at the bottom of the application window (in this illustration, next to the red star), and then select '''Install'''.
$ journalctl


* '''To install a package''' after searching, right-click the listed package (ins, and then select '''Install'''. Once done, then click the green tick icon at the top to confirm.
=== Live view, shows the last 10 lines of the journal & all content as it happens: ===


* '''To remove a package''' after searching right-click the listed package (installed packages will have a green icon with a tick next to them), and then select '''Remove'''. Once done, then click the green tick icon at the top to confirm.
$ journalctl -f
<br clear="all" />


== Basic filtering: ==


=== Shows all output to the journal since the last boot:===


$ journalctl -b


On occasion, particularly when installing new software applications, several other software packages will also be automatically installed as well.  These are known as dependencies, as they are necessary for whatever is being installed to work properly. In other words, the software package being downloaded is dependent upon them.
=== Shows all output with priority level ERROR & worse, since last boot: ===


Pamac will list the dependencies required for software packages in the information pane. For example, as illustrated, if the VLC Media Player were to be installed, then several other software packages --such as to allow it to play different media formats-- would also be automatically be downloaded if not already installed on the system. Without them, the Media Player would not be able to play certain media formats, or perhaps not be able to play anything at all!
$ journalctl -b -p err


{{Tip| You won't need to worry about dependencies yourself, as they will be automatically identified and downloaded for you when necessary.}}
== Filtering based on time: ==


=== Since yesterday: ===


$ journalctl --since=yesterday


= Manjaro Settings Manager =
=== Give a specific time period: ===


[[File:Msm.jpg|thumb|left|375px]]
$ journalctl --since=2012-10-15 --until="2011-10-16 23:59:59"


The [[Manjaro Settings Manager]] offers you GUI settings, which are either enabled by Manjaro (i.e. installation of multiple kernels and easy installation of non-proprietary and proprietary graphics drivers) or are missing from at least some of the popular [[Desktop Environments|Desktop Environments and Window Managers]].
=== Pick a specific service & time period: ===


Manjaro Settings Manager is under development and might offer even more settings in the future than illustrated on the left.
$ journalctl -u httpd --since=00:00 --until=9:30


<br clear="all" />
== Point journalctl at specific devices, services, binaries ==


=== Look at a specific device: ===


$ journalctl /dev/sdc


= The Arch User Repository (AUR) =
=== Check on a binary: ===


[[File:yaourtsearch.png|thumb|left|375px]]
$ journalctl /usr/sbin/vpnc


=== Check on the interlieved output from two specifics: ===


Although Manjaro is compatible to Arch --being based on Arch itself-- it is not possible to access the official repositories of the Arch System to download software. Manjaro instead uses its own official repositories in order to ensure that any software packages provided (e.g., system updates and applications) have been fully tested and are completely stable before release.
$ journalctl /usr/sbin/vpnc /usr/sbin/dhclient


However, it is still possible to access additional software packages from the Arch User Repository (AUR), which is managed by the Arch community of users themselves. Although this repository is unofficial, software packages first placed here are known to make their way into Arch's official repositories if they become popular enough. A '''[[Arch_User_Repository|guide on how to Access the AUR]]''' has been provided.
=== Show all systemd units that have been started in your journal: ===
<br clear="all" />


= Using the Terminal =
$ journalctl -F _SYSTEMD_UNIT


[[file:terminal-88.png|thumb|left|375px]]
You can then interrogate the journal specifying any of those units.


== A summation ==


In simple terms, a terminal (or console) is an interface that allows for text commands to be entered and displayed. As it is an exceptionally powerful and versatile tool to use, Arch and other Arch-based systems are notable for relying far more heavily on their use than other (user-friendly) distributions such as ''Ubuntu'' or ''Mint'', which have placed a greater focus on the use of Graphical User Interfaces (GUI).
The Systemd Journal is capable of advanced functions beyond what has been mentioned here. The above is very good food for thought for people that are wondering if they need to be running '''syslog-ng''' or the like that creates most of the '''/var/log/*log''' files on their systems.


Although GUIs to undertake many common tasks have been provided with Manjaro, particularly to assist new users just as with other user-friendly distributions, it may on occasion be necessary to use the terminal to get something done. '''Most of the guides contained in the wiki rely on using the terminal, particularly where solving problems and tweaking the system is concerned'''. For those users who wish to learn more about how Manjaro works, and for those who wish to take full advantage of its versatility, it is highly recommended to learn how to use the terminal. Don't worry: it's not that hard or scary, and this wiki can be used to guide you every step of the way.
By experimenting with the above commands one can make an informed decision for themselves, though as mentioned at the beginning of the Journal section, Arch & therefore Manjaro still run both the new systemd journal & the old style log file system in parallel. So if you find the /var/log/*log files to be redundant & you want to be rid of them, various methods would be effective. I'm currently using /etc/cron.hourly & a script in the /etc/logrotate.d directory. I'll write further on this later.
<br clear="all" />


= Using Multiple Kernels =
= Managing /var/log/* files =


[[file:kernel_select.png|thumb|left|375px]]
== Introducing Logrotate & friends ==


What is this Logrotate? [http://linux.die.net/man/8/logrotate] logrotate is a powerful tool used to manage the log files created by system processes. It can be instructed to automatically compress, rename in a variety of ways, remove logs, to do all of this & more in a way that maximizes the convenience of logs & conserves your system's resources. An enormous amount of control is available to users including running scripts on your rotated files.


The Linux kernel is the core of a Linux operating system. It acts as an interface between your computer's hardware and the applications that run on it. Manjaro not only supports the use of multiple kernels (selectable from the boot screen), but allows easy access to the very latest, bleeding edge kernels as well. All available kernels installed on your system will be presented upon booting up, including backup copies of each kernel version installed. If sub-menus are in effect, you will need to press Enter on the second item of a set.  
A problem I face in trying to make this article about logrotate as simple as possible is that logrotate can be called in so many ways, & these ways are not mutually exclusive.  


A '''[[Manjaro_Kernels|guide on how to manage kernels]]''' has been provided.
For example, logrotate can be called to run on a file, or multiple files in any combination or multiple of '''hourly, daily, weekly, monthly & yearly''', via scripts placed in the /etc/ in the already existing directories '''hourly daily weekly monthly''' the '''yearly''' directory can be added if required. '''crontab''' [http://www.adminschoice.com/crontab-quick-reference/] can be used to run logrotate or scripts as complex as a person needs. logrotate can be combined with other tools in anyway that a user can come up with to process these rotated files at any time & frequency.
<br clear="all" />


= Adding Printing Capabilities =
<br clear="all"/>
==== The scope of this article ====


That said, much of the power of logrotate is for the benefit of those administering servers & will not be dealt with in the following. Though what we will deal with can be used on more than just our log files. We can use logrotate to backup any other files that we choose. I will expand on this at a later date.


[[File:CUPS7.png|thumb|left|375px]]
<br clear="all"/>


== /etc/logrotate.conf & /etc/logrotate.d ==


While optional, adding printing capabilities is highly recommended.  
The logrotate.conf configuration file largely dictates logrotate's behaviour, it holds global settings, but most of the work that logrotate does is via script files stored in the '''/etc/logrotate.d''' directory, which take precedence over the global settings held in logrotate.conf.


Full instructions on installing the necessary software are provided in the '''[[Printing Enablement]]''' guide.  
Applications such as Apache, MySQL, Cups & others, put scripts into the /etc/logrotate.d directory to manage their log files.


Once the necessary software has been installed and printing capabilities have been enabled, printers can be added and configured using the '''[[Printer Configuration]]''' guide. Links have been provided in both guides to return here.
If you manually run the command '''sudo logrotate''', you will be presented with its usage template. logrotate needs you to specify the path to the script that you want it to use, including the logrotate.conf file which one may think due to its name would be automatically read, it is not.
<br clear="all" />
 
To run logrotate & the logrotate.conf file you use the following command line:
 
logrotate /etc/logrotate.conf
 
<br clear="all"/>
=== Can I store & run my script files elsewhere? ===
A line exists in logrotate.conf that tells logrotate to run all of the scripts that exist in /etc/logrotate.d
 
include /etc/logrotate.d
 
We can use the '''include''' command in logrotate.conf to add other directories or use another directory instead of logrotate.d if we have reason to. Be careful what you do as there are files placed into the logrotate.d directory by other programs.
 
<br clear="all"/>
=== My settings in logrotate.conf don't effect all of the .log files? ===
 
Script files that are called via the logrotate.conf file take precedence over the global settings in logrotate.conf . That means that if you call a script from logrotate.conf that is located in the /etc/logrotate.d directory, then that script is more powerful than any of the global setting in logrotate.conf .
 
I use a script '''/etc.logrotate.d/rotate.logs''' that is set to work on all *.log files, & it does. The two that don't get rotated are called '''faillog''' & '''lastlog''' , apart from not having the '''.log''' file extension, these two files are not normal log files, they are accessed via terminal commands of the same name.
 
<br clear="all"/>
 
=== Can I store my scripts where I want? ===
 
Some applications such as Apache cups, drop scripts into /etc/logrotate.d to aid in their own self maintenance. We can use a location of our choosing for these or other scripts if we want. We just have to call its path in the /etc/logrotate.conf file, the same way, as shown in the following example:
 
include /home/handy/.config/mylogrotate
 
Apart from adding our own scripts to /etc/logrotate.d (or any other path that we have chosen to include), we can also add scripts into any of the previously mentioned '''/etc/ cron.hourly cron.daily cron.weekly cron.monthly''' folders. OR we can add a script into any of these folders that suit our needs that runs the logrotate /etc/logrotate.conf command which will have the logrotate.conf file, direct logrotate to the default /etc/logrotate.d directory where we have our script(s). OR to another directory where we have our script & have included the path in logrotate.conf . whew!
 
So you can see there are a variety of ways to call logrotate (let alone use it).
 
<br clear="all"/>
 
== Some uses for Logrotate ==
 
For example, script block below does the following, listed line by line:
 
* '''/var/log/*.log {''' specifies the file or the files as this example uses a wild card that says all files ending in .log , the '''{''' starts the list of commands that will be used on the file(s) just specified.
 
* '''daily''' Here we are saying cycle these commands daily, we can also say weekly, monthly, yearly (or specify other times with crontab)[http://www.adminschoice.com/crontab-quick-reference/].
 
* '''size''' is where we can place a size limit that will cause a file to be rotated. I placed a '''1M''' one megabyte size limit in the example.
 
* '''dateext''' this puts the date of the rotation on the new copy, so it would use this format: '''<file.name>.log-20130815'''
 
* '''rotate 7''' means keep 7 of our daily (in this script) backups, delete the oldest when it would become the 8th.
 
* '''compress''' is obvious, it uses gzip by default & adds a .gz extension to your file, which will make it look like this: <file.name>.log.1.gz you can choose other compression methods, I'm not going into that here.
 
* '''delaycompress''' tells logrotate to compress the newly rotated file in the next cycle. This has advantages in ease of access & also if the file is still being written to by a process after it has been rotated.
 
* '''copytruncate''' this is a great option, as it copies the contents of the file to a new new file <file.name>.log.1 & then deletes the contents of the original file. You can have no permission problems crop up when you do it this way.
 
* '''notifempty''' do nothing if the file is empty, which makes good logical sense.
 
* '''missingok''' if the file does not exist, give no error.
 
* '''}''' this curly bracket closes the block of commands.
 
/var/log/*.log {
  daily
  size 1M
  dateext
  rotate 7
  compress
  delaycompress
  copytruncate
  notifempty
  missingok
  }
 
The above script can be used as is, it does not need to be made executable, it just needs to be put somewhere that logrotate will see (in this example) every day.
 
We can use the above script block as a template, easily removing parts & modifying its relatively simple settings. It can duplicated in a script with each script block specifying custom settings tailored for individual files.
 
<br clear="all"/>
== An Example that you can modify to suit ==
 
I'll show how I have my system set, you can use the information already given on this page & other available on the web to fine tune your set up to suit your needs (if you have the need anyway).
 
=== Firstly - Be sure this file is here /etc/cron.daily/logrotate === 
#!/bin/sh
# nicenesses range from -20 (most favorable scheduling) to 19 (least favorable)
NICE=19
# 0 for none, 1 for real time, 2 for best-effort, 3 for idle
IONICE_CLASS=2
# 0-7 (for IONICE_CLASS 1 and 2 only), 0=highest, 7=lowest
IONICE_PRIORITY=7
CMD_LOGROTATE="/usr/bin/logrotate /etc/logrotate.conf"
if [ -x /usr/bin/nice ]; then
  CMD_LOGROTATE="/usr/bin/nice -n ${NICE:-19} ${CMD_LOGROTATE}"
fi
if [ -x /usr/bin/ionice ]; then
  CMD_LOGROTATE="/usr/bin/ionice -c ${IONICE_CLASS:-2} -n ${IONICE_PRIORITY:-7} ${CMD_LOGROTATE}"
fi
${CMD_LOGROTATE}
exit 0
 
<br clear="all"/>
=== Secondly - Create /etc/logrotate.d/rotate.logs using the following ===
 
## rotate all /var/log files with names ending in log
/var/log/*log {
## cycle through these commands once per day
  daily
## keep the results of 7 cycles
  rotate 7
## use gzip to compress each rotated (copied) log file
  compress
## compress the file on the next cycle
  delaycompress
## copy the contents of the log file to a new file <name>.log.1
## & then delete the contents of the original log file
  copytruncate
## do nothing to empty files
  notifempty
## create no errors if a file is missing
  missingok
## after the files have been rotated run the following command
  }
 
<br clear="all"/>
 
=== A Summary of the above example thus far ===
 
The First step puts a file into '''/etc/cron.daily''' which is an easy way to add the script to a daily cron job. Which means that script will be run everyday.
 
It basically runs this command:
 
logrotate /etc/logrotate.conf
 
As logrotate.conf goes through its list of commands it calls this one:
 
include /etc/logrotate.d
 
Which means that any scripts that are inside of '''/etc/logrotate.d''' are also run.
 
This brings us to the second step (above), where we created '''/etc/logrotate.d/rotate.logs''' . This script will be run everyday. The comments I added to the rotate.logs file above give a general idea of what it does. You can delete, modify & add to that script, but do it carefully.
 
=== The effect of running /etc/logrotate.d/rotate.logs everyday ===
 
Is that any file in /var/log that had '''log''' at the end of its name will be processed by the commands in the '''rotate.logs''' script. This will back up these files to a new file '''<name>.log.1''' & empty the original file to size 0. Any previous copies of with '''<name>.log.<number>''' will have their numbers bumped up one, until the day when they would have been given an 8, that is the day that they are deleted.
 
As well as this rotating (copying) & renaming of files, all files will be compressed in gzip format on the next rotation. Which means that you always have the current file & yesterdays file in /var/log in uncompressed format.
 
No files that are empty will be processed, & a file being missing will throw no errors.
 
<br clear="all"/>
 
=Support=
 
Following is a link to this page's forum counterpart where you can post any related feedback: [http://forum.manjaro.org/index.php?topic=6050.0]




[[Category:Contents Page]]
[[Category:Contents Page]]

Revision as of 02:35, 25 August 2013

Introduction

NOTE: This page is still a work in progress.


Log files & the systemd journal do the same thing in different ways. They keep a record of everything that happens on your computer system. This makes it possible to understand what is going right & what is going wrong. As an example, if your system had been infiltrated by an ssh attack, this could be verified in the log/journal. So these log files are good for more than tracking troublesome hardware, or driver problems, badly written network manager code or the plethora of other problems that the complex & dynamic GNU/Linux system has to deal with.

These logs are an absolute blessing, as not all systems have them, & any server administrators who does have them would be very grateful as they can be the bread & butter of what they do.

Generally only server administrators have use for logs that go back any length of time. Few users who run distros on their desktop, Notebook, Netbook... machines, need to keep such huge log files, histories going back for many months or even years, on their system, are a waste of space & also makes viewing your log files more cumbersome.


The first topic on this page will briefly cover the systemd journal

The systemd journal has taken the place of log files though it will happily run in parallel with the standard type log files. These are still created & maintained by default in Arch & Manjaro, which means that even if you delete syslog-ng & all of the /var/log/*log files on reboot you will find the log files have been automatically created again, this will change in the future.

The second topic will cover handling log files

This topic will go into far more depth, it will covering the use of the logrotate command, logrotate.conf, the /etc/cron.daily cron.weekly cron.monthly cron.yearly, some ways to run created scripts, & a mention of the crontab method of running a script also. I'll try to make this section accessible to as many people as possible, which means this will be a long page.


The journal & the logs duplicate the same information

You can read the text of the log files in a text editor, or using the cat, more, less & such commands as you would on any other text file. The journal on the other hand requires the journalctl command to be able to access its contents. The following is a good way to read the journal:

sudo journalctl


How to set a maximum size limit for the journal

You need to use your text editor with root priviliges, starting it with sudo will do the job.

With a simple edit of the /etc/systemd/journald.conf you can set the maximum size limit of the /var/log/journal .

  • Uncomment the following line & add the size limit that you want. I set mine to 50MB like this:
SystemMaxUse=50M

After a reboot, you should find that your journal is now somewhat less than 50MB (or whatever limit you set), as it needs to have some working space. Mine went to 38M from 391M.

See the troubleshooting section if the above did not work for you.


The Journalctl command - a quick reference [1]

Add user to adm group, gives <user> full use of the journalctl command:

# usermod -a -G adm handy
Access to full journal containing info from the system & users:
$ journalctl

Live view, shows the last 10 lines of the journal & all content as it happens:

$ journalctl -f

Basic filtering:

Shows all output to the journal since the last boot:

$ journalctl -b

Shows all output with priority level ERROR & worse, since last boot:

$ journalctl -b -p err

Filtering based on time:

Since yesterday:

$ journalctl --since=yesterday

Give a specific time period:

$ journalctl --since=2012-10-15 --until="2011-10-16 23:59:59"

Pick a specific service & time period:

$ journalctl -u httpd --since=00:00 --until=9:30

Point journalctl at specific devices, services, binaries

Look at a specific device:

$ journalctl /dev/sdc

Check on a binary:

$ journalctl /usr/sbin/vpnc

Check on the interlieved output from two specifics:

$ journalctl /usr/sbin/vpnc /usr/sbin/dhclient

Show all systemd units that have been started in your journal:

$ journalctl -F _SYSTEMD_UNIT

You can then interrogate the journal specifying any of those units.

A summation

The Systemd Journal is capable of advanced functions beyond what has been mentioned here. The above is very good food for thought for people that are wondering if they need to be running syslog-ng or the like that creates most of the /var/log/*log files on their systems.

By experimenting with the above commands one can make an informed decision for themselves, though as mentioned at the beginning of the Journal section, Arch & therefore Manjaro still run both the new systemd journal & the old style log file system in parallel. So if you find the /var/log/*log files to be redundant & you want to be rid of them, various methods would be effective. I'm currently using /etc/cron.hourly & a script in the /etc/logrotate.d directory. I'll write further on this later.

Managing /var/log/* files

Introducing Logrotate & friends

What is this Logrotate? [2] logrotate is a powerful tool used to manage the log files created by system processes. It can be instructed to automatically compress, rename in a variety of ways, remove logs, to do all of this & more in a way that maximizes the convenience of logs & conserves your system's resources. An enormous amount of control is available to users including running scripts on your rotated files.

A problem I face in trying to make this article about logrotate as simple as possible is that logrotate can be called in so many ways, & these ways are not mutually exclusive.

For example, logrotate can be called to run on a file, or multiple files in any combination or multiple of hourly, daily, weekly, monthly & yearly, via scripts placed in the /etc/ in the already existing directories hourly daily weekly monthly the yearly directory can be added if required. crontab [3] can be used to run logrotate or scripts as complex as a person needs. logrotate can be combined with other tools in anyway that a user can come up with to process these rotated files at any time & frequency.


The scope of this article

That said, much of the power of logrotate is for the benefit of those administering servers & will not be dealt with in the following. Though what we will deal with can be used on more than just our log files. We can use logrotate to backup any other files that we choose. I will expand on this at a later date.


/etc/logrotate.conf & /etc/logrotate.d

The logrotate.conf configuration file largely dictates logrotate's behaviour, it holds global settings, but most of the work that logrotate does is via script files stored in the /etc/logrotate.d directory, which take precedence over the global settings held in logrotate.conf.

Applications such as Apache, MySQL, Cups & others, put scripts into the /etc/logrotate.d directory to manage their log files.

If you manually run the command sudo logrotate, you will be presented with its usage template. logrotate needs you to specify the path to the script that you want it to use, including the logrotate.conf file which one may think due to its name would be automatically read, it is not.

To run logrotate & the logrotate.conf file you use the following command line:

logrotate /etc/logrotate.conf


Can I store & run my script files elsewhere?

A line exists in logrotate.conf that tells logrotate to run all of the scripts that exist in /etc/logrotate.d

include /etc/logrotate.d

We can use the include command in logrotate.conf to add other directories or use another directory instead of logrotate.d if we have reason to. Be careful what you do as there are files placed into the logrotate.d directory by other programs.


My settings in logrotate.conf don't effect all of the .log files?

Script files that are called via the logrotate.conf file take precedence over the global settings in logrotate.conf . That means that if you call a script from logrotate.conf that is located in the /etc/logrotate.d directory, then that script is more powerful than any of the global setting in logrotate.conf .

I use a script /etc.logrotate.d/rotate.logs that is set to work on all *.log files, & it does. The two that don't get rotated are called faillog & lastlog , apart from not having the .log file extension, these two files are not normal log files, they are accessed via terminal commands of the same name.


Can I store my scripts where I want?

Some applications such as Apache cups, drop scripts into /etc/logrotate.d to aid in their own self maintenance. We can use a location of our choosing for these or other scripts if we want. We just have to call its path in the /etc/logrotate.conf file, the same way, as shown in the following example:

include /home/handy/.config/mylogrotate

Apart from adding our own scripts to /etc/logrotate.d (or any other path that we have chosen to include), we can also add scripts into any of the previously mentioned /etc/ cron.hourly cron.daily cron.weekly cron.monthly folders. OR we can add a script into any of these folders that suit our needs that runs the logrotate /etc/logrotate.conf command which will have the logrotate.conf file, direct logrotate to the default /etc/logrotate.d directory where we have our script(s). OR to another directory where we have our script & have included the path in logrotate.conf . whew!

So you can see there are a variety of ways to call logrotate (let alone use it).


Some uses for Logrotate

For example, script block below does the following, listed line by line:

  • /var/log/*.log { specifies the file or the files as this example uses a wild card that says all files ending in .log , the { starts the list of commands that will be used on the file(s) just specified.
  • daily Here we are saying cycle these commands daily, we can also say weekly, monthly, yearly (or specify other times with crontab)[4].
  • size is where we can place a size limit that will cause a file to be rotated. I placed a 1M one megabyte size limit in the example.
  • dateext this puts the date of the rotation on the new copy, so it would use this format: <file.name>.log-20130815
  • rotate 7 means keep 7 of our daily (in this script) backups, delete the oldest when it would become the 8th.
  • compress is obvious, it uses gzip by default & adds a .gz extension to your file, which will make it look like this: <file.name>.log.1.gz you can choose other compression methods, I'm not going into that here.
  • delaycompress tells logrotate to compress the newly rotated file in the next cycle. This has advantages in ease of access & also if the file is still being written to by a process after it has been rotated.
  • copytruncate this is a great option, as it copies the contents of the file to a new new file <file.name>.log.1 & then deletes the contents of the original file. You can have no permission problems crop up when you do it this way.
  • notifempty do nothing if the file is empty, which makes good logical sense.
  • missingok if the file does not exist, give no error.
  • } this curly bracket closes the block of commands.
/var/log/*.log {
 daily
 size 1M
 dateext
 rotate 7
 compress
 delaycompress
 copytruncate
 notifempty
 missingok
  }

The above script can be used as is, it does not need to be made executable, it just needs to be put somewhere that logrotate will see (in this example) every day.

We can use the above script block as a template, easily removing parts & modifying its relatively simple settings. It can duplicated in a script with each script block specifying custom settings tailored for individual files.


An Example that you can modify to suit

I'll show how I have my system set, you can use the information already given on this page & other available on the web to fine tune your set up to suit your needs (if you have the need anyway).

Firstly - Be sure this file is here /etc/cron.daily/logrotate

#!/bin/sh

# nicenesses range from -20 (most favorable scheduling) to 19 (least favorable)
NICE=19

# 0 for none, 1 for real time, 2 for best-effort, 3 for idle
IONICE_CLASS=2

# 0-7 (for IONICE_CLASS 1 and 2 only), 0=highest, 7=lowest
IONICE_PRIORITY=7

CMD_LOGROTATE="/usr/bin/logrotate /etc/logrotate.conf"

if [ -x /usr/bin/nice ]; then
  CMD_LOGROTATE="/usr/bin/nice -n ${NICE:-19} ${CMD_LOGROTATE}"
fi

if [ -x /usr/bin/ionice ]; then
  CMD_LOGROTATE="/usr/bin/ionice -c ${IONICE_CLASS:-2} -n ${IONICE_PRIORITY:-7} ${CMD_LOGROTATE}"
fi

${CMD_LOGROTATE}

exit 0


Secondly - Create /etc/logrotate.d/rotate.logs using the following

## rotate all /var/log files with names ending in log
/var/log/*log {
## cycle through these commands once per day
 daily
## keep the results of 7 cycles
 rotate 7
## use gzip to compress each rotated (copied) log file
 compress
## compress the file on the next cycle
 delaycompress
## copy the contents of the log file to a new file <name>.log.1
## & then delete the contents of the original log file
 copytruncate
## do nothing to empty files
 notifempty
## create no errors if a file is missing
 missingok
## after the files have been rotated run the following command
  } 


A Summary of the above example thus far

The First step puts a file into /etc/cron.daily which is an easy way to add the script to a daily cron job. Which means that script will be run everyday.

It basically runs this command:

logrotate /etc/logrotate.conf

As logrotate.conf goes through its list of commands it calls this one:

include /etc/logrotate.d

Which means that any scripts that are inside of /etc/logrotate.d are also run.

This brings us to the second step (above), where we created /etc/logrotate.d/rotate.logs . This script will be run everyday. The comments I added to the rotate.logs file above give a general idea of what it does. You can delete, modify & add to that script, but do it carefully.

The effect of running /etc/logrotate.d/rotate.logs everyday

Is that any file in /var/log that had log at the end of its name will be processed by the commands in the rotate.logs script. This will back up these files to a new file <name>.log.1 & empty the original file to size 0. Any previous copies of with <name>.log.<number> will have their numbers bumped up one, until the day when they would have been given an 8, that is the day that they are deleted.

As well as this rotating (copying) & renaming of files, all files will be compressed in gzip format on the next rotation. Which means that you always have the current file & yesterdays file in /var/log in uncompressed format.

No files that are empty will be processed, & a file being missing will throw no errors.


Support

Following is a link to this page's forum counterpart where you can post any related feedback: [5]

Cookies help us deliver our services. By using our services, you agree to our use of cookies.