December 2014 - docs-commits - Fedora Mailing-Lists

[install-guide/f21-branch] Silly typo in Boot Options

by pbokoc

commit 3213dcd70081c6747324c8c37552e8f0a6f94b0b Author: Petr Bokoc <pbokoc(a)redhat.com> Date: Mon Dec 15 19:55:46 2014 +0100 Silly typo in Boot Options en-US/Boot_Options.xml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/en-US/Boot_Options.xml b/en-US/Boot_Options.xml index fa561f6..e727801 100644 --- a/en-US/Boot_Options.xml +++ b/en-US/Boot_Options.xml @@ -217,7 +217,7 @@ </listitem> </varlistentry> <varlistentry> - <term><option>isnt.askmethod</option></term> + <term><option>inst.askmethod</option></term> <listitem> <para> Prevents the installer from trying to automatically configure an installation source.

9 years, 5 months

1
0
0 / 0

[install-guide] (3 commits) ...BZ#1045205 - Updated information about inst.askmethod in Boot Options and SourceSpoke

by pbokoc

Summary of changes: 0dcda8b... BZ#1076963 - updating Recommended partitioning scheme and A (*) ff2b5e0... Moving inst.headless into deprecated boot options (*) 760f614... BZ#1045205 - Updated information about inst.askmethod in Bo (*) (*) This commit already existed in another branch; no separate mail sent

9 years, 5 months

1
0
0 / 0

[install-guide/f21-branch] BZ#1045205 - Updated information about inst.askmethod in Boot Options and SourceSpoke

by pbokoc

commit 760f614b5f12585fdb2c8cd1f3b10cf2a043cd0d Author: Petr Bokoc <pbokoc(a)redhat.com> Date: Mon Dec 15 16:35:03 2014 +0100 BZ#1045205 - Updated information about inst.askmethod in Boot Options and SourceSpoke en-US/Boot_Options.xml | 18 ++++++++++++++++-- en-US/SourceSpoke.xml | 7 ++++++- 2 files changed, 22 insertions(+), 3 deletions(-) --- diff --git a/en-US/Boot_Options.xml b/en-US/Boot_Options.xml index 5747e10..fa561f6 100644 --- a/en-US/Boot_Options.xml +++ b/en-US/Boot_Options.xml @@ -216,6 +216,20 @@ </important> </listitem> </varlistentry> + <varlistentry> + <term><option>isnt.askmethod</option></term> + <listitem> + <para> + Prevents the installer from trying to automatically configure an installation source. + </para> + <para> + Normally, when you first enter the <guilabel>Installation Summary</guilabel> screen, the installer will attempt to configure an installation source based on the type of media you used to boot. The full &PRODUCT; Server DVD will configure the source as local media, the netinst ISO image will configure the closest network mirror, etc. This process takes time; therefore, if you plan to use a different source than the default, this option will allow you to enter the <guilabel>Installation Source</guilabel> screen immediately without having to wait for the initial configuration to finish. + </para> + <para> + Using this option will leave the installation source configuration in an error state - you must manually specify a source in the <guilabel>Installation Source</guilabel> screen to proceed with the installation. Alternatively, you can use the <option>inst.repo=</option> option described above to configure a specific source directly in the boot menu. + </para> + </listitem> + </varlistentry> </variablelist> </section> @@ -1026,10 +1040,10 @@ Password: </para> <variablelist> <varlistentry> - <term><option>askmethod</option>, <option>asknetwork</option></term> + <term><option>asknetwork</option></term> <listitem> <para> - The installation program's <systemitem>initramfs</systemitem> is now completely non-interactive, which means that these options are not available anymore. Instead, use the <option>inst.repo=</option> to specify the installation method and <option>ip=</option> to configure network settings. + The installation program's <systemitem>initramfs</systemitem> is now completely non-interactive, which means that this option are not available anymore. Use <option>ip=</option> to configure network settings. </para> </listitem> </varlistentry> diff --git a/en-US/SourceSpoke.xml b/en-US/SourceSpoke.xml index 2e211e1..28d7c49 100644 --- a/en-US/SourceSpoke.xml +++ b/en-US/SourceSpoke.xml @@ -8,6 +8,11 @@ <para> The <guilabel>Installation Source</guilabel> screen allows you to specify a location (local or on the network) from which packages will be downloaded and installed on your system. This screen will be configured automatically in most cases, but you can change your settings or add additional sources. </para> + <note> + <para> + Normally, when you first enter the <guilabel>Installation Summary</guilabel> screen, the installer will attempt to configure an installation source based on the type of media you used to boot. The full &PRODUCT; Server DVD will configure the source as local media, the netinst ISO image will configure the closest network mirror, etc. This process takes some time, especially if the default source is a network mirror. If you plan to use a custom installation source, use the <option>inst.askmethod</option> boot option to skip the initial configuration; this will allow you to enter this screen immediately. See <xref linkend="sect-boot-options-sources" /> for information about boot options. + </para> + </note> <figure> <title>Installation Source</title> <mediaobject> @@ -16,7 +21,7 @@ </imageobject> <textobject> <para> - The Installation Source screen, configured to download packages to be installed from the closest network mirror. An options to use a locally available ISO file is displayed above. + The Installation Source screen, configured to download packages to be installed from the closest network mirror. An option to use a locally available ISO file is displayed above. </para> </textobject> </mediaobject>

9 years, 5 months

1
0
0 / 0

[install-guide/f21-branch] Moving inst.headless into deprecated boot options

by pbokoc

commit ff2b5e000b18336767f5b7cd08ba89be363519cc Author: Petr Bokoc <pbokoc(a)redhat.com> Date: Mon Dec 15 16:26:10 2014 +0100 Moving inst.headless into deprecated boot options en-US/Boot_Options.xml | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) --- diff --git a/en-US/Boot_Options.xml b/en-US/Boot_Options.xml index e3f3070..5747e10 100644 --- a/en-US/Boot_Options.xml +++ b/en-US/Boot_Options.xml @@ -416,14 +416,6 @@ </listitem> </varlistentry> <varlistentry> - <term><option>inst.headless</option></term> - <listitem> - <para> - Specifies that the machine being installed onto does not have any display hardware. In other words, this options prevents the installation program from trying to detect a screen. - </para> - </listitem> - </varlistentry> - <varlistentry> <term><option>inst.xdriver=</option></term> <listitem> <para> @@ -882,6 +874,14 @@ Password: </para> </note> <variablelist> + <varlistentry> + <term><option>inst.headless</option></term> + <listitem> + <para> + Used to specify that the machine being installed onto does not have any display hardware, and to prevent the installer from trying to detect a display. + </para> + </listitem> + </varlistentry> <varlistentry> <term><option>method=</option></term> <listitem>

9 years, 5 months

1
0
0 / 0

[install-guide/f21-branch] BZ#1076963 - updating Recommended partitioning scheme and Advice on partitions

by pbokoc

commit 0dcda8b0ecbdf110989243077dbc51fee8988470 Author: Petr Bokoc <pbokoc(a)redhat.com> Date: Mon Dec 15 16:12:24 2014 +0100 BZ#1076963 - updating Recommended partitioning scheme and Advice on partitions en-US/CustomSpoke_PartitioningAdvice.xml | 5 +++++ en-US/CustomSpoke_RecommendedScheme.xml | 13 ++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-) --- diff --git a/en-US/CustomSpoke_PartitioningAdvice.xml b/en-US/CustomSpoke_PartitioningAdvice.xml index 4e8c6ca..f49c322 100644 --- a/en-US/CustomSpoke_PartitioningAdvice.xml +++ b/en-US/CustomSpoke_PartitioningAdvice.xml @@ -41,6 +41,11 @@ </listitem> <listitem> <para> + The contents of the <filename class="directory">/var</filename> directory usually change very often. This may cause problems with older solid state drives (SSDs), as they can handle a lower number of read/write cycles before becoming unusable. If your system root is on an SSD, consider creating a separate mount point for <filename class="directory">/var</filename> on a classic (platter) HDD. + </para> + </listitem> + <listitem> + <para> The <filename class="directory">/usr</filename> directory holds the majority of software on a typical &PRODUCT; installation. The partition or volume containing this directory should therefore be at least 5 GB for minimal installations, and at least 10 GB for installations with a graphical environment. </para> </listitem> diff --git a/en-US/CustomSpoke_RecommendedScheme.xml b/en-US/CustomSpoke_RecommendedScheme.xml index ca5796c..b5afeca 100644 --- a/en-US/CustomSpoke_RecommendedScheme.xml +++ b/en-US/CustomSpoke_RecommendedScheme.xml @@ -6,7 +6,7 @@ <section id="sect-installation-gui-manual-partitioning-recommended"> <title>Recommended Partitioning Scheme</title> <para> - In most cases, at least the following partitions should always be created: + In most cases, at least the following mount points should always be created: </para> <variablelist> <varlistentry> @@ -32,20 +32,23 @@ This is where the root directory is located. The root directory is the top level of the directory structure. By default, all files are written to this partition unless a different partition is mounted in the path being written to (for example, <filename class="directory">/boot</filename> or <filename class="directory">/home</filename>). If you follow the recommended scheme described in this section, this will be the partition where most software packages will be installed. </para> <para> - For a minimal installation, a 5 GB root partition will be sufficient. However, for most common installations which include extra packages and a graphical user interface, the root partition should be at least 10 GB. + For a minimal installation, a 5 GB root partition will be sufficient. However, for most common installations which include extra packages and a graphical user interface, the root partition should be at least 10 GB; with 20 GB being sufficient for most common use cases. </para> <note> <para> - Do not confuse the <filename class="directory">/</filename> directory with the <filename class="directory">/root</filename> directory. The latter is the home directory of the <systemitem class="username">root</systemitem> user, and is sometimes referred to as "<emphasis>slash root</emphasis>" to distinguish it from <filename class="directory">/</filename>. + The <filename class="directory">/</filename> mount point is the top of the Linux Filesystem Hierarchy, and is referred to as the <firstterm>root file system</firstterm>, or root. The <filename class="directory">/root</filename> directory, sometimes pronounced "<firstterm>slash-root</firstterm>", is the home directory for the <systemitem class="username">root</systemitem> user. </para> </note> </listitem> </varlistentry> <varlistentry> - <term><filename class="directory">/home</filename> - at least 1 GB</term> + <term><filename class="directory">/home</filename> - at least 10 GB</term> <listitem> <para> - To store user data separately from system data, create a dedicated partition within a volume group for the /home directory. This partition should be sized based on the amount of data that will be stored locally, number of users, and so on. This will allow you to upgrade or reinstall &PRODUCT; without erasing user data files. During the installation, a separate <filename class="directory">/home</filename> partition will be created if there are 50 GB or more free space for your &PRODUCT; installation. + To store user data separately from system data, create a dedicated mount point for the /home directory. This partition should be sized based on the amount of data that will be stored locally, number of users, and so on. This will allow you to upgrade or reinstall &PRODUCT; without erasing user data files. During the installation, a separate <filename class="directory">/home</filename> partition will be created if there are 50 GB or more free space for your &PRODUCT; installation. + </para> + <para> + When using &PRODUCT; as a workstation for normal use with a graphical environment, this mount point should have the most disk space assigned to it, as it will likely hold the most data (user settings, images, videos, etc). </para> </listitem> </varlistentry>

9 years, 5 months

1
0
0 / 0

[system-administrators-guide] kernel update RTC if chrony updates sys clock

by stephenw

commit 4c555796d5b449efe561282029b580ecda9c24f4 Author: Stephen Wadeley <swadeley(a)redhat.com> Date: Sun Dec 14 19:30:14 2014 +0100 kernel update RTC if chrony updates sys clock en-US/Configuring_NTP_Using_ntpd.xml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/en-US/Configuring_NTP_Using_ntpd.xml b/en-US/Configuring_NTP_Using_ntpd.xml index 0346f79..471c46e 100644 --- a/en-US/Configuring_NTP_Using_ntpd.xml +++ b/en-US/Configuring_NTP_Using_ntpd.xml @@ -769,7 +769,7 @@ manycastclient 239.255.254.254 key 30</screen> <screen>~]# <command>hwclock --systohc</command></screen> </para> <para> - When the system clock is being synchronized by <systemitem class="service">ntpd</systemitem>, the kernel will in turn update the RTC every 11 minutes automatically. + When the system clock is being synchronized by <systemitem class="service">ntpd</systemitem> or <systemitem class="service">chronyd</systemitem>, the kernel will in turn update the RTC every 11 minutes automatically. </para> </section>

9 years, 5 months

1
0
0 / 0

[system-administrators-guide] Viewing and Managing Log Files

by stephenw

commit 0949dc5622bd1684fb77a8f1e97e9bff1e54ac1d Author: Stephen Wadeley <swadeley(a)redhat.com> Date: Sun Dec 14 17:13:39 2014 +0100 Viewing and Managing Log Files Applying improvements from upstream version, after proofreading and SME feedback en-US/Viewing_and_Managing_Log_Files.xml | 2534 ++++++++++++++++++++------ en-US/images/redhat-filter-enable.png | Bin 0 -> 967679 bytes en-US/images/redhat-filter-sample.png | Bin 0 -> 358405 bytes en-US/images/redhat-filters.png | Bin 0 -> 309252 bytes en-US/images/redhat-logviewer-add.png | Bin 0 -> 965881 bytes en-US/images/redhat-logviewer-monitoring.png | Bin 0 -> 967679 bytes en-US/images/redhat-logviewer.png | Bin 0 -> 965270 bytes en-US/images/rsyslog_message_flow.png | Bin 0 -> 73315 bytes 8 files changed, 1948 insertions(+), 586 deletions(-) --- diff --git a/en-US/Viewing_and_Managing_Log_Files.xml b/en-US/Viewing_and_Managing_Log_Files.xml index 0264b99..7f2b5a3 100644 --- a/en-US/Viewing_and_Managing_Log_Files.xml +++ b/en-US/Viewing_and_Managing_Log_Files.xml @@ -3,139 +3,70 @@ ]> <chapter id="ch-Viewing_and_Managing_Log_Files"> <title>Viewing and Managing Log Files</title> - <indexterm significance="normal"> + <indexterm> <primary>log files</primary> - <seealso> - <application>Log Viewer</application> - </seealso> + <seealso><application>System Log</application></seealso> </indexterm> - <indexterm significance="normal"> + <indexterm> <primary>log files</primary> <secondary>description</secondary> </indexterm> - <para> - <firstterm>Log files</firstterm> are files that contain messages about the system, including the kernel, services, and applications running on it. There are different log files for different information. For example, there is a default system log file, a log file just for security messages, and a log file for cron tasks.</para> - <para>Log files can be very useful when trying to troubleshoot a problem with the system such as trying to load a kernel driver or when looking for unauthorized login attempts to the system. This chapter discusses where to find log files, how to view log files, and what to look for in log files.</para> - <indexterm significance="normal"> + <para> + <firstterm>Log files</firstterm> are files that contain messages about the system, including the kernel, services, and applications running on it. There are several types of log files for storing various information. For example, there is a default system log file, a log file just for security messages, and a log file for cron tasks. Log files can be very useful in many situations, for instance to troubleshoot a problem with the system, when trying to load a kernel driver, or when looking for unauthorized login attempts to the system. + </para> + <indexterm> <primary>log files</primary> - <secondary> - <command>rsyslogd daemon</command> - </secondary> + <secondary><command>rsyslogd daemon</command></secondary> </indexterm> - <indexterm significance="normal"> - <primary> - <application>rsyslog</application> - </primary> + <indexterm> + <primary><application>rsyslog</application></primary> </indexterm> - <para>Some log files are controlled by a daemon called <systemitem class="daemon">rsyslogd</systemitem>. A list of log files maintained by <systemitem class="daemon">rsyslogd</systemitem> can be found in the <filename>/etc/rsyslog.conf</filename> configuration file.</para> + <para> + Some log files are controlled by a daemon called <systemitem class="daemon">rsyslogd</systemitem>. The <systemitem class="daemon">rsyslogd</systemitem> daemon is an enhanced replacement for <application>syslogd</application>, and provides extended filtering, various configuration options, input and output modules, support for transportation via the <systemitem class="protocol">TCP</systemitem> or <systemitem class="protocol">UDP</systemitem> protocols. A list of log files maintained by <systemitem class="daemon">rsyslogd</systemitem> can be found in the <filename>/etc/rsyslog.conf</filename> configuration file. Most log files are located in the <filename>/var/log/</filename> directory. + </para> <para> - <application>rsyslog</application> is an enhanced, multi-threaded syslog daemon which replaced the <command>sysklogd</command> daemon. <application>rsyslog</application> supports the same functionality as <application>sysklogd</application> and extends it with enhanced filtering, encryption protected relaying of messages, various configuration options, or support for transportation via the <systemitem class="protocol">TCP</systemitem> or <systemitem class="protocol">UDP</systemitem> protocols. Note that <application>rsyslog</application> is compatible with <application>sysklogd</application>. + Log files can also be managed by the <systemitem class="daemon">journald</systemitem> daemon – a component of <systemitem class="daemon">systemd</systemitem>. The <systemitem class="daemon">journald</systemitem> daemon captures Syslog messages, kernel log messages, initial RAM disk and early boot messages as well as messages written to standard output and standard error output of all services, indexes them and makes this available to the user. The native journal file format, which is a structured and indexed binary file, improves searching and provides faster operation, and it also stores meta data information like time stamps or user IDs. Log files produced by <systemitem class="daemon">journald</systemitem> are by default not persistent, log files are stored only in memory or a small ring-buffer in the <filename class="directory">/run/log/journal/</filename> directory. The amount of logged data depends on free memory, when you reach the capacity limit, the oldes t entries are deleted. However, this setting can be altered – see <xref linkend="s2-Enabling_Persistent_Storage"/>. For more information on Journal see <xref linkend="s1-Using_the_Journal"/>. </para> - <section id="s1-configuring-rsyslog"> - <title>Configuring rsyslog</title> - <para> - The main configuration file for <application>rsyslog</application> is <filename>/etc/rsyslog.conf</filename>. It consists of <firstterm>global directives</firstterm>, <firstterm>rules</firstterm> or comments (any empty lines or any text following a hash sign (<literal>#</literal>)). Both, global directives and rules are extensively described in the sections below. + <para> + By default, these two logging tools coexist on your system. The <systemitem class="daemon">journald</systemitem> daemon is the primary tool for troubleshooting. It also provides additional data necessary for creating structured log messages. Data acquired by <systemitem class="daemon">journald</systemitem> is forwarded into the <systemitem>/run/systemd/journal/syslog</systemitem> socket that may be used by <systemitem class="daemon">rsyslogd</systemitem> to process the data further. However, <application>rsyslog</application> does the actual integration by default via the <systemitem>imjournal</systemitem> input module, thus avoiding the aforementioned socket. You can also transfer data in the opposite direction, from <systemitem class="daemon">rsyslogd</systemitem> to <systemitem class="daemon">journald</systemitem> with use of <systemitem>omjournal</systemitem> module. See <xref linkend="s1-interaction_of_rsyslog_and_journal" /> for further information. The integration enables maintaining text-based logs in a consistent format to ensure compatibility with possible applications or configurations dependent on <systemitem class="daemon">rsyslogd</systemitem>. Also, you can maintain rsyslog messages in a structured format (see <xref linkend="s1-structured_logging_with_rsyslog"/>). </para> - <section id="s2-global-directives"> - <title>Global Directives</title> - <para> - Global directives specify configuration options that apply to the <systemitem class="daemon">rsyslogd</systemitem> daemon. They usually specify a value for a specific pre-defined variable that affects the behavior of the <systemitem class="daemon">rsyslogd</systemitem> daemon or a rule that follows. All of the global directives must start with a dollar sign (<literal>$</literal>). Only one directive can be specified per line. The following is an example of a global directive that specifies the maximum size of the syslog message queue:</para> - <screen> -$MainMsgQueueSize 50000 - </screen> - <para> - The default size defined for this directive (<constant>10,000</constant> messages) can be overridden by specifying a different value (as shown in the example above). - </para> - <para> - You may define multiple directives in your <filename>/etc/rsyslog.conf</filename> configuration file. A directive affects the behavior of all configuration options until another occurrence of that same directive is detected. + <section id="s1-logfiles-locating"> + <title>Locating Log Files</title> + <indexterm> + <primary>log files</primary> + <secondary>locating</secondary> + </indexterm> + <para>Most log files are located in the <filename>/var/log/</filename> directory. Some applications such as <command>httpd</command> and <command>samba</command> have a directory within <filename>/var/log/</filename> for their log files.</para> + <indexterm> + <primary>log files</primary> + <secondary>rotating</secondary> + </indexterm> + <indexterm> + <primary><command>logrotate</command></primary> + </indexterm> + <para>You may notice multiple files in the <filename>/var/log/</filename> directory with numbers after them (for example, <filename>cron-20100906</filename>). These numbers represent a time stamp that has been added to a rotated log file. Log files are rotated so their file sizes do not become too large. The <filename>logrotate</filename> package contains a cron task that automatically rotates log files according to the <filename>/etc/logrotate.conf</filename> configuration file and the configuration files in the <filename class="directory">/etc/logrotate.d/</filename> directory.</para> + </section> + <section id="s1-basic_configuration_of_rsyslog"> + <title>Basic Configuration of Rsyslog</title> + <para> + The main configuration file for <application>rsyslog</application> is <filename>/etc/rsyslog.conf</filename>. Here, you can specify <firstterm>global directives</firstterm>, <firstterm>modules</firstterm>, and <firstterm>rules</firstterm> that consist of <firstterm>filter</firstterm> and <firstterm>action</firstterm> parts. Also, you can add comments in the form of text following a hash sign (<literal>#</literal>). </para> - <para> - A comprehensive list of all available configuration directives and their detailed description can be found in <filename>/usr/share/doc/rsyslog/rsyslog_conf_global.html</filename>. - </para> - </section> - - <section id="s2-modules"> - <title>Modules</title> - <para> - Due to its modular design, <application>rsyslog</application> offers a variety of <firstterm>modules</firstterm> which provide dynamic functionality. Note that modules can be written by third parties. Most modules provide additional inputs (see <emphasis>Input Modules</emphasis> below) or outputs (see <emphasis>Output Modules</emphasis> below). Other modules provide special functionality specific to each module. The modules may provide additional configuration directives that become available after a module is loaded. To load a module, use the following syntax: - </para> - <screen> -$ModLoad <replaceable><MODULE></replaceable> - </screen> - <para> - where <option>$ModLoad</option> is the global directive that loads the specified module and <replaceable><MODULE></replaceable> represents your desired module. For example, if you want to load the <literal>Text File Input Module</literal> (<command>imfile</command> — enables <application>rsyslog</application> to convert any standard text files into syslog messages), specify the following line in your <filename>/etc/rsyslog.conf</filename> configuration file: - </para> - <screen> -$ModLoad imfile - </screen> - <para> - <application>rsyslog</application> offers a number of modules which are split into these main categories: - </para> - <itemizedlist> - <listitem> - <para> - Input Modules — Input modules gather messages from various sources. The name of an input module always starts with the <literal>im</literal> prefix, such as <command>imfile</command>, <command>imrelp</command>, etc. - </para> - </listitem> - <listitem> - <para> - Output Modules — Output modules provide a facility to store messages into various targets such as sending them across network, storing them in a database or encrypting them. The name of an output module always starts with the <literal>om</literal> prefix, such as <command>omsnmp</command>, <command>omrelp</command>, etc. - </para> - </listitem> - <listitem> - <para> - Filter Modules — Filter modules provide the ability to filter messages according to specified rules. The name of a filter module always starts with the <literal>fm</literal> prefix. - </para> - </listitem> - <listitem> - <para> - Parser Modules — Parser modules use the message parsers to parse message content of any received messages. The name of a parser module always starts with the <literal>pm</literal> prefix, such as <command>pmrfc5424</command>, <command>pmrfc3164</command>, etc. - </para> - </listitem> - <listitem> - <para> - Message Modification Modules — Message modification modules change the content of a syslog message. The message modification modules only differ in their implementation from the output and filter modules but share the same interface. - </para> - </listitem> - <listitem> - <para> - String Generator Modules — String generator modules generate strings based on the message content and strongly cooperate with the template feature provided by <application>rsyslog</application>. For more information on templates, refer to <xref linkend="s3-templates"/>. The name of a string generator module always starts with the <literal>sm</literal> prefix, such as <command>smfile</command>, <command>smtradfile</command>, etc. - </para> - </listitem> - <listitem> - <para> - Library Modules — Library modules generally provide functionality for other loadable modules. These modules are loaded automatically by <application>rsyslog</application> when needed and cannot be configured by the user. - </para> - </listitem> - </itemizedlist> - <para> - A comprehensive list of all available modules and their detailed description can be found at <ulink url="http://www.rsyslog.com/doc/rsyslog_conf_modules.html/">http://www.rsyslog.com/doc/rsyslog_conf_modules.html</ulink> - </para> - <warning> - <title>Make sure you use trustworthy modules only</title> + <section id="s2-Filters"> + <title>Filters</title> <para> - Note that when <application>rsyslog</application> loads any modules, it provides them with access to some of its functions and data. This poses a possible security threat. To minimize security risks, use trustworthy modules only. - </para> - </warning> - </section> - <section id="s2-rules"> - <title>Rules</title> - <para> - A rule is specified by a <firstterm>filter</firstterm> part, which selects a subset of syslog messages, and an <firstterm>action</firstterm> part, which specifies what to do with the selected messages. To define a rule in your <filename>/etc/rsyslog.conf</filename> configuration file, define both, a filter and an action, on one line and separate them with one or more spaces or tabs. For more information on filters, refer to <xref linkend="s3-filter-conditions"/> and for information on actions, refer to <xref linkend="s3-actions"/>. + A rule is specified by a <emphasis>filter</emphasis> part, which selects a subset of syslog messages, and an <emphasis>action</emphasis> part, which specifies what to do with the selected messages. To define a rule in your <filename>/etc/rsyslog.conf</filename> configuration file, define both, a filter and an action, on one line and separate them with one or more spaces or tabs. + </para> + <para> + <application>rsyslog</application> offers various ways to filter syslog messages according to selected properties. The available filtering methods can be divided into <emphasis>Facility/Priority-based</emphasis>, <emphasis>Property-based</emphasis>, and <emphasis>Expression-based</emphasis> filters. </para> - <section id="s3-filter-conditions"> - <title>Filter Conditions</title> - <para> - <application>rsyslog</application> offers various ways how to filter syslog messages according to various properties. This sections sums up the most used filter conditions. - </para> <variablelist> <varlistentry> <term>Facility/Priority-based filters</term> <listitem> - <para>The most used and well-known way to filter syslog messages is to use the facility/priority-based filters which filter syslog messages based on two conditions: <firstterm>facility</firstterm> and <firstterm>priority</firstterm>. To create a selector, use the following syntax: + <para>The most used and well-known way to filter syslog messages is to use the facility/priority-based filters which filter syslog messages based on two conditions: <firstterm>facility</firstterm> and <firstterm>priority</firstterm> separated by a dot. To create a selector, use the following syntax: </para> <screen> -<replaceable><FACILITY></replaceable>.<replaceable><PRIORITY></replaceable> +<replaceable>FACILITY</replaceable>.<replaceable>PRIORITY</replaceable> </screen> <para> where: @@ -143,44 +74,82 @@ $ModLoad imfile <itemizedlist> <listitem> <para> - <replaceable><FACILITY></replaceable> specifies the subsystem that produces a specific syslog message. For example, the <command>mail</command> subsystem handles all mail related syslog messages. <replaceable><FACILITY></replaceable> can be represented by one of these keywords: <command>auth</command>, <command>authpriv</command>, <command>cron</command>, <command>daemon</command>, <command>kern</command>, <command>lpr</command>, <command>mail</command>, <command>news</command>, <command>syslog</command>, <command>user</command>, <command>uucp</command>, and <command>local0</command> through <command>local7</command>. + <replaceable>FACILITY</replaceable> specifies the subsystem that produces a specific syslog message. For example, the <command>mail</command> subsystem handles all mail-related syslog messages. <replaceable>FACILITY</replaceable> can be represented by one of the following keywords (or by a numerical code): <command>kern</command> (0), <command>user</command> (1), <command>mail</command> (2), <command>daemon</command> (3), <command>auth</command> (4), <command>syslog</command> (5), <command>lpr</command> (6), <command>news</command> (7), <command>uucp</command> (8), <command>cron</command> (9), <command>authpriv</command> (10), <command>ftp</command> (11), <command>ntp</command> (12), <command>logaudit</command> (13), <command>logalert</command> (14), <command>clock</command> (15), and <command>local0</command> through <command>local7</command> (16 - 23). </para> </listitem> <listitem> <para> - <replaceable><PRIORITY></replaceable> specifies a priority of a syslog message. <replaceable><PRIORITY></replaceable> can be represented by one of these keywords (listed in an ascending order): <command>debug</command>, <command>info</command>, <command>notice</command>, <command>warning</command>, <command>err</command>, <command>crit</command>, <command>alert</command>, and <command>emerg</command>. + <replaceable>PRIORITY</replaceable> specifies a priority of a syslog message. <replaceable>PRIORITY</replaceable> can be represented by one of the following keywords (or by a number): <command>debug</command> (7), <command>info</command> (6), <command>notice</command> (5), <command>warning</command> (4), <command>err</command> (3), <command>crit</command> (2), <command>alert</command> (1), and <command>emerg</command> (0). </para> <para> - By preceding any priority with an equal sign (<literal>=</literal>), you specify that only syslog messages with that priority will be selected. All other priorities will be ignored. Conversely, preceding a priority with an exclamation mark (<literal>!</literal>) selects all syslog messages but those with the defined priority. By not using either of these two extensions, you specify a selection of syslog messages with the defined or higher priority. + The aforementioned syntax selects syslog messages with the defined or <emphasis>higher</emphasis> priority. By preceding any priority keyword with an equal sign (<literal>=</literal>), you specify that only syslog messages with the specified priority will be selected. All other priorities will be ignored. Conversely, preceding a priority keyword with an exclamation mark (<literal>!</literal>) selects all syslog messages except those with the defined priority. </para> </listitem> </itemizedlist> <para> - In addition to the keywords specified above, you may also use an asterisk (<literal>*</literal>) to define all facilities or priorities (depending on where you place the asterisk, before or after the dot). Specifying the keyword <literal>none</literal> serves for facilities with no given priorities. - </para> - <para> - To define multiple facilities and priorities, simply separate them with a comma (<literal>,</literal>). To define multiple filters on one line, separate them with a semi-colon (<literal>;</literal>). + In addition to the keywords specified above, you may also use an asterisk (<literal>*</literal>) to define all facilities or priorities (depending on where you place the asterisk, before or after the comma). Specifying the priority keyword <literal>none</literal> serves for facilities with no given priorities. Both facility and priority conditions are case-insensitive. </para> <para> - The following are a few examples of simple facility/priority-based filters: + To define multiple facilities and priorities, separate them with a comma (<literal>,</literal>). To define multiple selectors on one line, separate them with a semi-colon (<literal>;</literal>). Note that each selector in the selector field is capable of overwriting the preceding ones, which can exclude some priorities from the pattern. </para> - <screen> -kern.* # Selects all kernel syslog messages with any priority - </screen> - <screen> -mail.crit # Selects all mail syslog messages with priority <command>crit</command> and higher. - </screen> - <screen> -cron.!info,!debug # Selects all cron syslog messages except those with the <command>info</command> or <command>debug</command> priority. - </screen> + <example id="ex-facility_priority-based_filters"> + <title>Facility/Priority-based Filters</title> + <para> + The following are a few examples of simple facility/priority-based filters that can be specified in <filename>/etc/rsyslog.conf</filename>. To select all kernel syslog messages with any priority, add the following text into the configuration file: + </para> + <screen> +kern.* + </screen> + <para> + To select all mail syslog messages with priority <command>crit</command> and higher, use this form: + </para> + <screen> +mail.crit + </screen> + <para> + To select all cron syslog messages except those with the <command>info</command> or <command>debug</command> priority, set the configuration in the following form: + </para> + <screen> +cron.!info,!debug + </screen> + </example> </listitem> </varlistentry> <varlistentry> <term>Property-based filters</term> <listitem> <para> - Property-based filters let you filter syslog messages by any property, such as <parameter>timegenerated</parameter> or <parameter>syslogtag</parameter>. For more information on properties, refer to <xref linkend="s4-properties"/>. Each of the properties specified in the filters lets you compare it to a specific value using one of the compare-operations listed in <xref linkend="table-compare-operations"/>. + Property-based filters let you filter syslog messages by any property, such as <parameter>timegenerated</parameter> or <parameter>syslogtag</parameter>. For more information on properties, see <xref linkend="brid-properties"/>. You can compare each of the specified properties to a particular value using one of the compare-operations listed in <xref linkend="table-compare-operations"/>. Both property names and compare operations are case-sensitive. + </para> + <para> + Property-based filter must start with a colon (<literal>:</literal>). To define the filter, use the following syntax: </para> + <screen>:<replaceable>PROPERTY</replaceable>, [!]<replaceable>COMPARE_OPERATION</replaceable>, "<replaceable>STRING</replaceable>"</screen> + <para> + where: + </para> + <itemizedlist> + <listitem> + <para> + The <replaceable>PROPERTY</replaceable> attribute specifies the desired property. + </para> + </listitem> + <listitem> + <para> + The optional exclamation point (<literal>!</literal>) negates the output of the compare-operation. Other Boolean operators are currently not supported in property-based filters. + </para> + </listitem> + <listitem> + <para> + The <replaceable>COMPARE_OPERATION</replaceable> attribute specifies one of the compare-operations listed in <xref linkend="table-compare-operations"/>. + </para> + </listitem> + <listitem> + <para> + The <replaceable>STRING</replaceable> attribute specifies the value that the text provided by the property is compared to. This value must be enclosed in quotation marks. To escape certain character inside the string (for example a quotation mark (<literal>"</literal>)), use the backslash character (<literal>\</literal>). + </para> + </listitem> + </itemizedlist> <table id="table-compare-operations"> <title>Property-based compare-operations</title> <tgroup cols="2"> @@ -202,7 +171,7 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c <parameter>contains</parameter> </entry> <entry> - Checks whether the provided string matches any part of the text provided by the property. + Checks whether the provided string matches any part of the text provided by the property. To perform case-insensitive comparisons, use <parameter>contains_i</parameter>. </entry> </row> <row> @@ -210,7 +179,7 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c <parameter>isequal</parameter> </entry> <entry> - Compares the provided string against all of the text provided by the property. + Compares the provided string against all of the text provided by the property. These two values must be exactly equal to match. </entry> </row> <row> @@ -218,7 +187,7 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c <parameter>startswith</parameter> </entry> <entry> - Checks whether the provided string matches a prefix of the text provided by the property. + Checks whether the provided string is found exactly at the beginning of the text provided by the property. To perform case-insensitive comparisons, use <parameter>startswith_i</parameter>. </entry> </row> <row> @@ -226,8 +195,8 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c <parameter>regex</parameter> </entry> <entry> - Compares the provided POSIX BRE (Basic Regular Expression) regular expression against the text provided by the property. - </entry> + Compares the provided POSIX BRE (Basic Regular Expression) against the text provided by the property. + </entry> </row> <row> <entry> @@ -237,124 +206,90 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c Compares the provided POSIX ERE (Extended Regular Expression) regular expression against the text provided by the property. </entry> </row> + <row> + <entry> + <parameter>isempty</parameter> + </entry> + <entry> + Checks if the property is empty. The value is discarded. This is especially useful when working with normalized data, where some fields may be populated based on normalization result. + </entry> + </row> </tbody> </tgroup> </table> - <para> - To define a property-based filter, use the following syntax: - </para> - <screen>:<replaceable><PROPERTY></replaceable>, [!]<replaceable><COMPARE_OPERATION></replaceable>, "<replaceable><STRING></replaceable>"</screen> - <para> - where: - </para> - <itemizedlist> - <listitem> - <para> - The <replaceable><PROPERTY></replaceable> attribute specifies the desired property (for example, <parameter>timegenerated</parameter>, <parameter>hostname</parameter>, etc.). - </para> - </listitem> - <listitem> - <para> - The optional exclamation point (<literal>!</literal>) negates the output of the compare-operation (if prefixing the compare-operation). - </para> - </listitem> - <listitem> - <para> - The <replaceable><COMPARE_OPERATION></replaceable> attribute specifies one of the compare-operations listed in <xref linkend="table-compare-operations"/>. - </para> - </listitem> - <listitem> - <para> - The <replaceable><STRING></replaceable> attribute specifies the value that the text provided by the property is compared to. To escape certain character (for example a quotation mark (<literal>"</literal>)), use the backslash character (<literal>\</literal>). - </para> - </listitem> - </itemizedlist> - <para> - The following are few examples of property-based filters: - </para> - <itemizedlist> - <listitem> + <example id="ex-property-based_filters"> + <title>Property-based Filters</title> <para> - The following filter selects syslog messages which contain the string <literal>error</literal> in their message text: - </para> + The following are a few examples of property-based filters that can be specified in <filename>/etc/rsyslog.conf</filename>. To select syslog messages which contain the string <literal>error</literal> in their message text, use: + </para> <screen>:msg, contains, "error"</screen> - </listitem> - <listitem> <para> - The following filter selects syslog messages received from the hostname <literal>host1</literal>: - </para> + The following filter selects syslog messages received from the host name <literal>host1</literal>: + </para> <screen>:hostname, isequal, "host1"</screen> - </listitem> - <listitem> <para> - The following filter selects syslog messages which do not contain any mention of the words <literal>fatal</literal> and <literal>error</literal> with any or no text between them (for example, <literal>fatal lib error</literal>): - </para> + To select syslog messages which do not contain any mention of the words <literal>fatal</literal> and <literal>error</literal> with any or no text between them (for example, <literal>fatal lib error</literal>), type: + </para> <screen>:msg, !regex, "fatal .* error"</screen> - </listitem> - </itemizedlist> + </example> </listitem> </varlistentry> <varlistentry> <term>Expression-based filters</term> <listitem> <para> - Expression-based filters select syslog messages according to defined arithmetic, boolean or string operations. Expression-based filters use <application>rsyslog</application>'s own scripting language. The syntax of this language is defined in <filename>/usr/share/doc/rsyslog/rscript_abnf.html</filename> along with examples of various expression-based filters. - </para> - <para> - To define an expression-based filter, use the following syntax: - </para> - <screen>if <replaceable><EXPRESSION></replaceable> then <replaceable><ACTION></replaceable> - </screen> + Expression-based filters select syslog messages according to defined arithmetic, Boolean or string operations. Expression-based filters use <application>rsyslog</application>'s own scripting language called <emphasis>RainerScript</emphasis> to build complex filters. See <xref linkend="brid-Log_Files-Resources-Online"/> for the syntax definition of this script along with examples of various expression-based filters. Also RainerScript is a basis for <application>rsyslog</application>'s new configuration format, see <xref linkend="s2-using_the_new_configuration_format"/> + </para> + <para> + The basic syntax of expression-based filter looks as follows: + </para> + <screen> +if <replaceable>EXPRESSION</replaceable> then <replaceable>ACTION</replaceable> else <replaceable>ACTION</replaceable> + </screen> <para> where: </para> <itemizedlist> <listitem> <para> - The <replaceable><EXPRESSION></replaceable> attribute represents an expression to be evaluated, for example: <literal>$msg startswith 'DEVNAME'</literal> or <literal>$syslogfacility-text == 'local0'</literal>. - </para> + The <replaceable>EXPRESSION</replaceable> attribute represents an expression to be evaluated, for example: <literal>$msg startswith 'DEVNAME'</literal> or <literal>$syslogfacility-text == 'local0'</literal>. You can specify more than one expression in a single filter by using <function>and</function> and <function>or</function> operators. + </para> </listitem> <listitem> <para> - The <replaceable><ACTION></replaceable> attribute represents an action to be performed if the expression returns the value <literal>true</literal>. - </para> + The <replaceable>action</replaceable> attribute represents an action to be performed if the expression returns the value <literal>true</literal>. this can be a single action, or an arbitrary complex script enclosed in curly braces. + </para> + </listitem> + <listitem> + <para> + Expression-based filters are indicated by the keyword <emphasis>if</emphasis> at the start of a new line. the <emphasis>then</emphasis> keyword separates the <replaceable>expression</replaceable> from the <replaceable>action</replaceable>. optionally, you can employ the <emphasis>else</emphasis> keyword to specify what action is to be performed in case the condition is not met. + </para> </listitem> </itemizedlist> - <note> - <title>Define an expression-based filter on a single line</title> - <para> - When defining an expression-based filter, it must be defined on a single line. - </para> - </note> - <note> - <title>Do not use regular expressions</title> - <para> - Regular expressions are currently not supported in expression-based filters. + <para> + With expression-based filters, you can nest the conditions by using a script enclosed in curly braces as in <xref linkend="ex-expression-based_filters"/>. the script allows you to use <emphasis>facility/priority-based</emphasis> filters inside the expression. on the other hand, <emphasis>property-based</emphasis> filters are not recommended here. RainerScript supports regular expressions with specialized functions <function>re_match()</function> and <function>re_extract()</function>. </para> - </note> - </listitem> - </varlistentry> - <varlistentry> - <term>BSD-style blocks</term> - <listitem> + <example id="ex-expression-based_filters"> + <title>Expression-based Filters</title> <para> - <application>rsyslog</application> supports BSD-style blocks inside the <filename>/etc/rsyslog.conf</filename> configuration file. Each block consists of rules which are preceded with a program or hostname label. Use the '!<replaceable><PROGRAM></replaceable>' or '-<replaceable><PROGRAM></replaceable>' labels to include or exclude programs, respectively. Use the '+<replaceable><HOSTNAME></replaceable> ' or '-<replaceable><HOSTNAME></replaceable> ' labels include or exclude hostnames, respectively. + The following expression contains two nested conditions. The log files created by a program called <emphasis>prog1</emphasis> are split into two files based on the presence of the "test" string in the message. </para> - <para><xref linkend="example-bsd-block1"/> shows a BSD-style block that saves all messages generated by <application>yum</application> to a file.</para> - <example id="example-bsd-block1"> - <title>BSD-style block</title> - <screen> -!yum -*.* /var/log/named.log - </screen> - </example> + <screen> +if $programname == 'prog1' then { + action(type="omfile" file="/var/log/prog1.log") + if $msg contains 'test' then + action(type="omfile" file="/var/log/prog1test.log") + else + action(type="omfile" file="/var/log/prog1notest.log") +} + </screen> + </example> </listitem> </varlistentry> </variablelist> - </section> - - <section id="s3-actions"> - <title>Actions</title> + </section> + <section id="s2-Actions"> + <title>Actions</title> <para> Actions specify what is to be done with the messages filtered out by an already-defined selector. The following are some of the actions you can define in your rule: </para> @@ -363,18 +298,32 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c <term>Saving syslog messages to log files</term> <listitem> <para> - The majority of actions specify to which log file a syslog message is saved. This is done by specifying a file path after your already-defined selector. The following is a rule comprised of a selector that selects all <application>cron</application> syslog messages and an action that saves them into the <filename>/var/log/cron.log</filename> log file: + The majority of actions specify to which log file a syslog message is saved. This is done by specifying a file path after your already-defined selector: + </para> + <synopsis><replaceable>FILTER</replaceable> <replaceable>PATH</replaceable></synopsis> + <para> + where <replaceable>FILTER</replaceable> stands for user-specified selector and <replaceable>PATH</replaceable> is a path of a target file. + </para> + <para> + For instance, the following rule is comprised of a selector that selects all <application>cron</application> syslog messages and an action that saves them into the <filename>/var/log/cron.log</filename> log file: + </para> + <synopsis>cron.* /var/log/cron.log</synopsis> + <para> + By default, the log file is synchronized every time a syslog message is generated. Use a dash mark (<literal>-</literal>) as a prefix of the file path you specified to omit syncing: + </para> + <synopsis><replaceable>FILTER</replaceable> -<replaceable>PATH</replaceable></synopsis> + <para> + Note that you might lose information if the system terminates right after a write attempt. However, this setting can improve performance, especially if you run programs that produce very verbose log messages. </para> - <screen>cron.* /var/log/cron.log - </screen> <para> - Use a dash mark (<literal>-</literal>) as a prefix of the file path you specified if you want to omit syncing the desired log file after every syslog message is generated. + Your specified file path can be either <emphasis>static</emphasis> or <emphasis>dynamic</emphasis>. Static files are represented by a fixed file path as shown in the example above. Dynamic file paths can differ according to the received message. Dynamic file paths are represented by a template and a question mark (<literal>?</literal>) prefix: </para> + <synopsis><replaceable>FILTER</replaceable> ?<replaceable>DynamicFile</replaceable></synopsis> <para> - Your specified file path can be either static or dynamic. Static files are represented by a simple file path as was shown in the example above. Dynamic files are represented by a template and a question mark (<literal>?</literal>) prefix. For more information on templates, refer to <xref linkend="s4-generating-dynamic-fnames"/>. + where <replaceable>DynamicFile</replaceable> is a name of a predefined template that modifies output paths. You can use the dash prefix (<literal>-</literal>) to disable syncing, also you can use multiple templates separated by a colon (<literal>;</literal>). For more information on templates, see <xref linkend="brid-generating-dynamic-fnames"/>. </para> <para> - If the file you specified is an existing <application>tty</application> or <filename>/dev/console</filename> device, syslog messages are sent to standard output (using special <application>tty</application>-handling) or your console (using special <filename>/dev/console</filename>-handling) when using the X Window System, respectively. + If the file you specified is an existing <application>terminal</application> or <filename>/dev/console</filename> device, syslog messages are sent to standard output (using special <application>terminal</application>-handling) or your console (using special <filename>/dev/console</filename>-handling) when using the X Window System, respectively. </para> </listitem> </varlistentry> @@ -382,9 +331,9 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c <term>Sending syslog messages over the network</term> <listitem> <para> - <application>rsyslog</application> allows you to send and receive syslog messages over the network. This feature allows to administer syslog messages of multiple hosts on one machine. To forward syslog messages to a remote machine, use the following syntax: + <application>rsyslog</application> allows you to send and receive syslog messages over the network. This feature allows you to administer syslog messages of multiple hosts on one machine. To forward syslog messages to a remote machine, use the following syntax: </para> - <screen>@<optional>(<replaceable><OPTION></replaceable>)</optional><replaceable><HOST></replaceable>:<optional><replaceable><PORT></replaceable></optional> + <screen>@<optional>(<command>z<replaceable>NUMBER</replaceable></command>)</optional><replaceable>HOST</replaceable>:<optional><replaceable>PORT</replaceable></optional> </screen> <para> where: @@ -397,91 +346,107 @@ cron.!info,!debug # Selects all cron syslog messages except those with the <c </listitem> <listitem> <para> - The <replaceable><OPTION></replaceable> attribute can be replaced with an option such as <command>z<replaceable><NUMBER></replaceable> - </command>. This option enables <application>zlib</application> compression for syslog messages; the <replaceable><NUMBER></replaceable> attribute specifies the level of compression. To define multiple options, simply separate each one of them with a comma (<literal>,</literal>). + The optional <command>z<replaceable>NUMBER</replaceable></command> setting enables <application>zlib</application> compression for syslog messages. The <replaceable>NUMBER</replaceable> attribute specifies the level of compression (from 1 – lowest to 9 – maximum). Compression gain is automatically checked by <systemitem class="daemon">rsyslogd</systemitem>, messages are compressed only if there is any compression gain and messages below 60 bytes are never compressed. </para> </listitem> <listitem> <para> - The <replaceable><HOST></replaceable> attribute specifies the host which receives the selected syslog messages. + The <replaceable>HOST</replaceable> attribute specifies the host which receives the selected syslog messages. </para> </listitem> <listitem> <para> - The <replaceable><PORT></replaceable> attribute specifies the host machine's port. + The <replaceable>PORT</replaceable> attribute specifies the host machine's port. </para> </listitem> </itemizedlist> <para> When specifying an <systemitem class="protocol">IPv6</systemitem> address as the host, enclose the address in square brackets (<literal>[</literal>, <literal>]</literal>). </para> - <para> - The following are some examples of actions that forward syslog messages over the network (note that all actions are preceded with a selector that selects all messages with any priority): - </para> - <screen>*.* @192.168.0.1 # Forwards messages to 192.168.0.1 via the <systemitem class="protocol">UDP</systemitem> protocol</screen> - <screen>*.* @@example.com:18 # Forwards messages to "example.com" using port 18 and the <systemitem class="protocol">TCP</systemitem> protocol</screen> - <screen> -*.* @(z9)[2001::1] # Compresses messages with <application>zlib</application> (level 9 compression) - # and forwards them to 2001::1 using the <systemitem class="protocol">UDP</systemitem> protocol</screen> + <example id="ex-logging_over_the_network"> + <title>Sending syslog Messages over the Network</title> + <para> + The following are some examples of actions that forward syslog messages over the network (note that all actions are preceded with a selector that selects all messages with any priority). To forward messages to <systemitem class="ipaddress">192.168.0.1</systemitem> via the <systemitem class="protocol">UDP</systemitem> protocol, type: + </para> + <screen> +*.* @192.168.0.1 + </screen> + <para> + To forward messages to "example.com" using port 18 and the <systemitem class="protocol">TCP</systemitem> protocol, use: + </para> + <screen> +*.* @@example.com:18 + </screen> + <para> + The following compresses messages with <application>zlib</application> (level 9 compression) and forwards them to <systemitem class="ipaddress">2001:db8::1</systemitem> using the <systemitem class="protocol">UDP</systemitem> protocol + </para> + <screen> +*.* @(z9)[2001:db8::1] + </screen> + </example> </listitem> </varlistentry> <varlistentry> <term>Output channels</term> <listitem> <para> - Output channels are primarily used for log file rotation (for more info on log file rotation, refer to <xref linkend="configuring-logrotate"/>), that is, to specify the maximum size a log file can grow to. To define an output channel, use the following syntax: + Output channels are primarily used to specify the maximum size a log file can grow to. This is very useful for log file rotation (for more information see <xref linkend="s2-log_rotation"/>). An output channel is basically a collection of information about the output action. Output channels are defined by the <literal>$outchannel</literal> directive. To define an output channel in <filename>/etc/rsyslog.conf</filename>, use the following syntax: </para> - <screen>$outchannel <replaceable><NAME></replaceable>, <replaceable><FILE_NAME></replaceable>, <replaceable><MAX_SIZE></replaceable>, <replaceable><ACTION></replaceable> </screen> + <screen>$outchannel <replaceable>NAME</replaceable>, <replaceable>FILE_NAME</replaceable>, <replaceable>MAX_SIZE</replaceable>, <replaceable>ACTION</replaceable> </screen> <para> where: </para> <itemizedlist> <listitem> <para> - The <replaceable><NAME></replaceable> attribute specifies the name of the output channel. + The <replaceable>NAME</replaceable> attribute specifies the name of the output channel. </para> </listitem> <listitem> <para> - The <replaceable><FILE_NAME></replaceable> attribute specifies the name of the output file. + The <replaceable>FILE_NAME</replaceable> attribute specifies the name of the output file. Output channels can write only into files, not pipes, terminal, or other kind of output. </para> </listitem> <listitem> <para> - The <replaceable><MAX_SIZE></replaceable> attribute represents the maximum size the specified file (in <replaceable><FILE_NAME></replaceable>) can grow to. This value is specified in <emphasis>bytes</emphasis>. + The <replaceable>MAX_SIZE</replaceable> attribute represents the maximum size the specified file (in <replaceable>FILE_NAME</replaceable>) can grow to. This value is specified in <emphasis>bytes</emphasis>. </para> </listitem> <listitem> <para> - The <replaceable><ACTION></replaceable> attribute specifies the action that is taken when the maximum size, defined in <replaceable><MAX_SIZE></replaceable>, is hit. + The <replaceable>ACTION</replaceable> attribute specifies the action that is taken when the maximum size, defined in <replaceable>MAX_SIZE</replaceable>, is hit. </para> </listitem> </itemizedlist> <para> - <xref linkend="example-log-rotation"/> shows a simple log rotation through the use of an output channel. First, the output channel is defined via the <parameter>$outchannel</parameter> directive and then used in a rule which selects every syslog message with any priority and executes the previously-defined output channel on the acquired syslog messages. Once the limit (in the example <constant>100 MB</constant>) is hit, the <filename>/home/joe/log_rotation_script</filename> is executed. This script can contain anything from moving the file into a different folder, editing specific content out of it, or simply removing it. - </para> + To use the defined output channel as an action inside a rule, type: + </para> + <synopsis><replaceable>FILTER</replaceable> :omfile:$<replaceable>NAME</replaceable></synopsis> <example id="example-log-rotation"> <title>Output channel log rotation</title> + <para> + The following output shows a simple log rotation through the use of an output channel. First, the output channel is defined via the <parameter>$outchannel</parameter> directive: + </para> + <screen> + $outchannel log_rotation, /var/log/test_log.log, 104857600, /home/joe/log_rotation_script + </screen> + <para> + and then it is used in a rule that selects every syslog message with any priority and executes the previously-defined output channel on the acquired syslog messages: + </para> <screen> -$outchannel log_rotation,/var/log/test_log.log, 104857600, /home/joe/log_rotation_script - -*.* $log_rotation - </screen> - </example> - <note> - <title>Support for output channels is to be removed in the future</title> +*.* :omfile:$log_rotation + </screen> <para> - Output channels are currently supported by <application>rsyslog</application>, however, they are planned to be removed in the nearby future. - </para> - </note> + Once the limit (in the example <constant>100 MB</constant>) is hit, the <filename>/home/joe/log_rotation_script</filename> is executed. This script can contain anything from moving the file into a different folder, editing specific content out of it, or simply removing it. + </para> + </example> </listitem> </varlistentry> - <varlistentry> <term>Sending syslog messages to specific users</term> <listitem> <para> - <application>rsyslog</application> can send syslog messages to specific users by simply specifying a username of the user you wish to send the messages to. To specify more than one user, separate each username with a comma (<literal>,</literal>). To send messages to every user that is currently logged on, use an asterisk (<literal>*</literal>). + <application>rsyslog</application> can send syslog messages to specific users by specifying a user name of the user you want to send the messages to (as in <xref linkend="ex-multiple_actions"/>). To specify more than one user, separate each user name with a comma (<literal>,</literal>). To send messages to every user that is currently logged on, use an asterisk (<literal>*</literal>). </para> </listitem> </varlistentry> @@ -489,73 +454,83 @@ $outchannel log_rotation,/var/log/test_log.log, 104857600, /home/joe/log_rotatio <term>Executing a program</term> <listitem> <para> - <application>rsyslog</application> lets you execute a program for selected syslog messages and uses the <systemitem>system()</systemitem> call to execute the program in shell. To specify a program to be executed, prefix it with a caret character (<literal>^</literal>). Consequently, specify a template that formats the received message and passes it to the specified executable as a one line parameter (for more information on templates, refer to <xref linkend="s3-templates"/>). In the following example, any syslog message with any priority is selected, formatted with the <parameter>template</parameter> template and passed as a parameter to the <application>test-program</application> program, which is then executed with the provided parameter: + <application>rsyslog</application> lets you execute a program for selected syslog messages and uses the <systemitem>system()</systemitem> call to execute the program in shell. To specify a program to be executed, prefix it with a caret character (<literal>^</literal>). Consequently, specify a template that formats the received message and passes it to the specified executable as a one line parameter (for more information on templates, see <xref linkend="s2-Templates"/>). + </para> + <synopsis><replaceable>FILTER</replaceable> ^<replaceable>EXECUTABLE</replaceable>; <replaceable>TEMPLATE</replaceable></synopsis> + <para> + Here an output of the <replaceable>FILTER</replaceable> condition is processed by a program represented by <replaceable>EXECUTABLE</replaceable>. This program can be any valid executable. Replace <replaceable>TEMPLATE</replaceable> with the name of the formatting template. + </para> + <example id="ex-esecuting_a_program"> + <title>Executing a Program</title> + <para> + In the following example, any syslog message with any priority is selected, formatted with the <parameter>template</parameter> template and passed as a parameter to the <application>test-program</application> program, which is then executed with the provided parameter: </para> <screen>*.* ^test-program;template</screen> + </example> <warning> <title>Be careful when using the shell execute action</title> <para> - When accepting messages from any host, and using the shell execute action, you may be vulnerable to command injection. An attacker may try to inject and execute commands specified by the attacker in the program you specified (in your action) to be executed. To avoid any possible security threats, thoroughly consider the use of the shell execute action. + When accepting messages from any host, and using the shell execute action, you may be vulnerable to command injection. An attacker may try to inject and execute commands in the program you specified to be executed in your action. To avoid any possible security threats, thoroughly consider the use of the shell execute action. </para> </warning> </listitem> </varlistentry> <varlistentry> - <term>Inputting syslog messages in a database</term> + <term>Storing syslog messages in a database</term> <listitem> <para> Selected syslog messages can be directly written into a database table using the <emphasis>database writer</emphasis> action. The database writer uses the following syntax: </para> - <screen>:<replaceable><PLUGIN></replaceable>:<replaceable><DB_HOST></replaceable>,<replaceable><DB_NAME></replaceable>,<replaceable><DB_USER></replaceable>,<replaceable><DB_PASSWORD></replaceable>;<optional><replaceable><TEMPLATE></replaceable></optional></screen> + <screen>:<replaceable>PLUGIN</replaceable>:<replaceable>DB_HOST</replaceable>,<replaceable>DB_NAME</replaceable>,<replaceable>DB_USER</replaceable>,<replaceable>DB_PASSWORD</replaceable>;<optional><replaceable>TEMPLATE</replaceable></optional></screen> <para> where: </para> <itemizedlist> <listitem> <para> - The <replaceable><PLUGIN></replaceable> calls the specified plug-in that handles the database writing (for example, the <systemitem>ommysql</systemitem> plug-in). + The <replaceable>PLUGIN</replaceable> calls the specified plug-in that handles the database writing (for example, the <systemitem>ommysql</systemitem> plug-in). </para> </listitem> <listitem> <para> - The <replaceable><DB_HOST></replaceable> attribute specifies the database hostname. + The <replaceable>DB_HOST</replaceable> attribute specifies the database host name. </para> </listitem> <listitem> <para> - The <replaceable><DB_NAME></replaceable> attribute specifies the name of the database. + The <replaceable>DB_NAME</replaceable> attribute specifies the name of the database. </para> </listitem> <listitem> <para> - The <replaceable><DB_USER></replaceable> attribute specifies the database user. + The <replaceable>DB_USER</replaceable> attribute specifies the database user. </para> </listitem> <listitem> <para> - The <replaceable><DB_PASSWORD></replaceable> attribute specifies the password used with the aforementioned database user. + The <replaceable>DB_PASSWORD</replaceable> attribute specifies the password used with the aforementioned database user. </para> </listitem> <listitem> <para> - The <replaceable><TEMPLATE></replaceable> attribute specifies an optional use of a template that modifies the syslog message. For more information on templates, refer to <xref linkend="s3-templates"/>. + The <replaceable>TEMPLATE</replaceable> attribute specifies an optional use of a template that modifies the syslog message. For more information on templates, see <xref linkend="s2-Templates"/>. </para> </listitem> </itemizedlist> <important> <title>Using MySQL and PostgreSQL</title> <para> - Currently, <application>rsyslog</application> provides support for <systemitem class="systemname">MySQL</systemitem> (for more information, refer to <filename>/usr/share/doc/rsyslog/rsyslog_mysql.html</filename>) and <systemitem class="systemname">PostgreSQL</systemitem> databases only. - In order to use the <systemitem class="systemname">MySQL</systemitem> and <systemitem class="systemname">PostgreSQL</systemitem> database writer functionality, install the <package>rsyslog-mysql</package> and <package>rsyslog-pgsql</package> packages installed, respectively. Also, make sure you load the appropriate modules in your <filename>/etc/rsyslog.conf</filename> configuration file: + Currently, <application>rsyslog</application> provides support for <systemitem class="systemname">MySQL</systemitem> and <systemitem class="systemname">PostgreSQL</systemitem> databases only. + In order to use the <systemitem class="systemname">MySQL</systemitem> and <systemitem class="systemname">PostgreSQL</systemitem> database writer functionality, install the <package>rsyslog-mysql</package> and <package>rsyslog-pgsql</package> packages, respectively. Also, make sure you load the appropriate modules in your <filename>/etc/rsyslog.conf</filename> configuration file: </para> <screen> $ModLoad ommysql # Output module for MySQL support $ModLoad ompgsql # Output module for PostgreSQL support </screen> - <para>For more information on <application>rsyslog</application> modules, refer to <xref linkend="s2-modules"/>. + <para>For more information on <application>rsyslog</application> modules, see <xref linkend="s1-using_rsyslog_modules"/>. </para> <para> - Alternatively, you may use a generic database interface provided by the <systemitem>omlibdb</systemitem> module. However, this module is currently not compiled. + Alternatively, you may use a generic database interface provided by the <systemitem>omlibdb</systemitem> module (supports: Firebird/Interbase, MS SQL, Sybase, SQLLite, Ingres, Oracle, mSQL). </para> </important> </listitem> @@ -564,45 +539,58 @@ $ModLoad ompgsql # Output module for PostgreSQL support <term>Discarding syslog messages</term> <listitem> <para> - To discard your selected messages, use the tilde character (<literal>~</literal>). The following rule discards any cron syslog messages: + To discard your selected messages, use the tilde character (<literal>~</literal>). + </para> + <synopsis><replaceable>FILTER</replaceable> ~</synopsis> + <para> + The discard action is mostly used to filter out messages before carrying on any further processing. It can be effective if you want to omit some repeating messages that would otherwise fill the log files. The results of discard action depend on where in the configuration file it is specified, for the best results place these actions on top of the actions list. Please note that once a message has been discarded there is no way to retrieve it in later configuration file lines. + </para> + <para> + For instance, the following rule discards any cron syslog messages: </para> <screen>cron.* ~</screen> </listitem> </varlistentry> </variablelist> + <bridgehead>Specifying Multiple Actions</bridgehead> <para> - For each selector, you are allowed to specify multiple actions. To specify multiple actions for one selector, write each action on a separate line and precede it with an ampersand character (&). Only the first action is allowed to have a selector specified on its line. The following is an example of a rule with multiple actions: + For each selector, you are allowed to specify multiple actions. To specify multiple actions for one selector, write each action on a separate line and precede it with an ampersand character (&): </para> - <screen> -kern.=crit joe + <screen> +<replaceable>FILTER</replaceable> <replaceable>ACTION</replaceable> +& <replaceable>ACTION</replaceable> +& <replaceable>ACTION</replaceable> + </screen> + <para> + Specifying multiple actions improves the overall performance of the desired outcome since the specified selector has to be evaluated only once. + </para> + <example id="ex-multiple_actions"> + <title>Specifying Multiple Actions</title> + <para> + In the following example, all kernel syslog messages with the critical priority (<parameter>crit</parameter>) are sent to user <systemitem class="username">user1</systemitem>, processed by the template <parameter>temp</parameter> and passed on to the <parameter>test-program</parameter> executable, and forwarded to <systemitem class="ipaddress">192.168.0.1</systemitem> via the <systemitem class="protocol">UDP</systemitem> protocol. + </para> + <screen> +kern.=crit user1 & ^test-program;temp & @192.168.0.1 - </screen> - <para> - In the example above, all kernel syslog messages with the critical priority (<parameter>crit</parameter>) are send to user <systemitem class="username">joe</systemitem>, processed by the template <parameter>temp</parameter> and passed on to the <parameter>test-program</parameter> executable, and forwarded to <systemitem class="ipaddress">192.168.0.1</systemitem> via the <systemitem class="protocol">UDP</systemitem> protocol. - </para> - <para> - Specifying multiple actions improves the overall performance of the desired outcome since the specified selector has to be evaluated only once. - </para> + </screen> + </example> <para> - Note that any action can be followed by a template that formats the message. To specify a template, suffix an action with a semicolon (<literal>;</literal>) and specify the name of the template. + Any action can be followed by a template that formats the message. To specify a template, suffix an action with a semicolon (<literal>;</literal>) and specify the name of the template. For more information on templates, see <xref linkend="s2-Templates"/>. </para> <warning> <title>Using templates</title> <para> - A template must be defined before it is used in an action, otherwise, it is ignored. + A template must be defined before it is used in an action, otherwise it is ignored. In other words, template definitions should always precede rule definitions in <filename>/etc/rsyslog.conf</filename>. </para> </warning> + </section> + <section id="s2-Templates"> + <title>Templates</title> <para> - For more information on templates, refer to <xref linkend="s3-templates"/>. - </para> - </section> - <section id="s3-templates"> - <title>Templates</title> - <para> - Any output that is generated by <application>rsyslog</application> can be modified and formatted according to your needs through the use of templates. To create a template use the following syntax: + Any output that is generated by <application>rsyslog</application> can be modified and formatted according to your needs with the use of <emphasis>templates</emphasis>. To create a template use the following syntax in <filename>/etc/rsyslog.conf</filename>: </para> - <screen>$template <replaceable><TEMPLATE_NAME></replaceable>,"<replaceable>text %<PROPERTY>% more text</replaceable>", <optional><replaceable><OPTION></replaceable></optional></screen> + <screen>$template <replaceable>TEMPLATE_NAME</replaceable>,"<replaceable>text %PROPERTY% more text</replaceable>", <optional><replaceable>OPTION</replaceable></optional></screen> <para> where: </para> @@ -614,63 +602,68 @@ kern.=crit joe </listitem> <listitem> <para> - <parameter><TEMPLATE_NAME></parameter> is the name of the template. Use this name to refer to the template. + <parameter>TEMPLATE_NAME</parameter> is the name of the template. Use this name to refer to the template. </para> </listitem> <listitem> <para> - Anything between the two quotation marks (<literal>"</literal>…<literal>"</literal>) is the actual template text. Within this text, you are allowed to escape characters in order to use their functionality, such as <literal>\n</literal> for new line or <literal>\r</literal> for carriage return. Other characters, such as <literal>%</literal> or <literal>"</literal>, have to be escaped in case you want to those characters literally. - </para> + Anything between the two quotation marks (<literal>"</literal>…<literal>"</literal>) is the actual template text. Within this text, special characters, such as <literal>\n</literal> for new line or <literal>\r</literal> for carriage return, can be used. Other characters, such as <literal>%</literal> or <literal>"</literal>, have to be escaped if you want to use those characters literally. + </para> + </listitem> + <listitem> <para> - The text specified within two percent signs (<literal>%</literal>) specifies a <firstterm>property</firstterm> that is consequently replaced with the property's actual value. For more information on properties, refer to <xref linkend="s4-properties"/> + The text specified between two percent signs (<literal>%</literal>) specifies a <firstterm>property</firstterm> that allows you to access specific contents of a syslog message. For more information on properties, see <xref linkend="brid-properties"/>. </para> </listitem> <listitem> <para> - The <parameter><OPTION></parameter> attribute specifies any options that modify the template functionality. Do not mistake these for property options, which are defined inside the template text (between <literal>"</literal>…<literal>"</literal>). The currently supported template options are <parameter>sql</parameter> and <parameter>stdsql</parameter> used for formatting the text as an SQL query. - </para> + The <parameter>OPTION</parameter> attribute specifies any options that modify the template functionality. The currently supported template options are <parameter>sql</parameter> and <parameter>stdsql</parameter>, which are used for formatting the text as an SQL query. + </para> <note> <title>The sql and stdsql options</title> <para> - Note that the database writer (for more information, refer to section <citetitle>Inputting syslog messages in a database</citetitle> in <xref linkend="s3-actions"/>) checks whether the <parameter>sql</parameter> and <parameter>stdsql</parameter> options are specified in the template. If they are not, the database writer does not perform any action. This is to prevent any possible security threats, such as SQL injection. + Note that the database writer checks whether the <parameter>sql</parameter> or <parameter>stdsql</parameter> options are specified in the template. If they are not, the database writer does not perform any action. This is to prevent any possible security threats, such as SQL injection. + </para> + <para> + See section <citetitle>Storing syslog messages in a database</citetitle> in <xref linkend="s2-Actions"/> for more information. </para> </note> </listitem> </itemizedlist> - <section id="s4-generating-dynamic-fnames"> - <title>Generating dynamic file names</title> + <bridgehead id="brid-generating-dynamic-fnames">Generating Dynamic File Names</bridgehead> <para> - Templates can be used to generate dynamic file names. By specifying a property as a part of the file path, a new file will be created for each unique property. For example, use the <parameter>timegenerated</parameter> property to generate a unique file name for each syslog message: + Templates can be used to generate dynamic file names. By specifying a property as a part of the file path, a new file will be created for each unique property, which is a convenient way to classify syslog messages. + </para> + <para> + For example, use the <parameter>timegenerated</parameter> property, which extracts a time stamp from the message, to generate a unique file name for each syslog message: </para> <screen>$template DynamicFile,"/var/log/test_logs/%timegenerated%-test.log"</screen> <para> - Keep in mind that the <parameter>$template</parameter> directive only specifies the template. You must use it inside a rule for it to take effect: + Keep in mind that the <parameter>$template</parameter> directive only specifies the template. You must use it inside a rule for it to take effect. In <filename>/etc/rsyslog.conf</filename>, use the question mark (<literal>?</literal>) in an action definition to mark the dynamic file name template: </para> <screen>*.* ?DynamicFile</screen> - </section> - <section id="s4-properties"> - <title>Properties</title> + <bridgehead id="brid-properties">Properties</bridgehead> <para> - Properties defined inside a template (within two percent signs (<literal>%</literal>)) allow you to access various contents of a syslog message through the use of a <firstterm>property replacer</firstterm>. To define a property inside a template (between the two quotation marks (<literal>"</literal>…<literal>"</literal>)), use the following syntax: - </para> - <screen>%<replaceable><PROPERTY_NAME></replaceable><optional>:<replaceable><FROM_CHAR></replaceable>:<replaceable><TO_CHAR></replaceable>:<replaceable><OPTION></replaceable></optional>%</screen> + Properties defined inside a template (between two percent signs (<literal>%</literal>)) enable access various contents of a syslog message through the use of a <firstterm>property replacer</firstterm>. To define a property inside a template (between the two quotation marks (<literal>"</literal>…<literal>"</literal>)), use the following syntax: + </para> + <screen>%<replaceable>PROPERTY_NAME</replaceable><optional>:<replaceable>FROM_CHAR</replaceable>:<replaceable>TO_CHAR</replaceable>:<replaceable>OPTION</replaceable></optional>%</screen> <para> where: </para> <itemizedlist> <listitem> <para> - The <replaceable><PROPERTY_NAME></replaceable> attribute specifies the name of a property. A comprehensible list of all available properties and their detailed description can be found in <filename>/usr/share/doc/rsyslog/property_replacer.html</filename> under the section <emphasis>Available Properties</emphasis>. + The <replaceable>PROPERTY_NAME</replaceable> attribute specifies the name of a property. A list of all available properties and their detailed description can be found in the <filename>rsyslog.conf(5)</filename> manual page under the section <emphasis>Available Properties</emphasis>. </para> </listitem> <listitem> <para> - <replaceable><FROM_CHAR></replaceable> and <replaceable><TO_CHAR></replaceable> attributes denote a range of characters that the specified property will act upon. Alternatively, regular expressions can be used to specify a range of characters. To do so, specify the letter <literal>R</literal> as the <replaceable><FROM_CHAR></replaceable> attribute and specify your desired regular expression as the <replaceable><TO_CHAR></replaceable> attribute. + <replaceable>FROM_CHAR</replaceable> and <replaceable>TO_CHAR</replaceable> attributes denote a range of characters that the specified property will act upon. Alternatively, regular expressions can be used to specify a range of characters. To do so, set the letter <literal>R</literal> as the <replaceable>FROM_CHAR</replaceable> attribute and specify your desired regular expression as the <replaceable>TO_CHAR</replaceable> attribute. </para> </listitem> <listitem> <para> - The <replaceable><OPTION></replaceable> attribute specifies any property options. A comprehensible list of all available properties and their detailed description can be found in <filename>/usr/share/doc/rsyslog/property_replacer.html</filename> under the section <emphasis>Property Options</emphasis>. + The <replaceable>OPTION</replaceable> attribute specifies any property options, such as the <option>lowercase</option> option to convert the input to lowercase. A list of all available property options and their detailed description can be found in the <filename>rsyslog.conf(5)</filename> manual page under the section <emphasis>Property Options</emphasis>. </para> </listitem> </itemizedlist> @@ -680,7 +673,7 @@ kern.=crit joe <itemizedlist> <listitem> <para> - The following property simply obtains the whole message text of a syslog message: + The following property obtains the whole message text of a syslog message: </para> <screen>%msg%</screen> </listitem> @@ -698,130 +691,114 @@ kern.=crit joe </listitem> <listitem> <para> - The following property obtains the first 10 characters of the timestamp that is generated when the syslog message is received and formats it according to the RFC 3999 date standard. + The following property obtains the first 10 characters of the time stamp that is generated when the syslog message is received and formats it according to the <ulink url="http://www.rfc-editor.org/info/rfc3999"><citetitle pubwork="webpage">RFC 3999</citetitle></ulink> date standard. </para> <screen>%timegenerated:1:10:date-rfc3339%</screen> </listitem> </itemizedlist> - </section> - <section id="s4-template-examples"> - <title>Template Examples</title> - <para> - This section presents few examples of <application>rsyslog</application> templates. - </para> - <para> - <xref linkend="example-temp1"/> shows a template that formats a syslog message so that it outputs the message's severity, facility, the timestamp of when the message was received, the hostname, the message tag, the message text, and ends with a new line. - </para> + <bridgehead id="brid-template-examples">Template Examples</bridgehead> + <para> + This section presents a few examples of <application>rsyslog</application> templates. + </para> + <para> + <xref linkend="example-temp1"/> shows a template that formats a syslog message so that it outputs the message's severity, facility, the time stamp of when the message was received, the host name, the message tag, the message text, and ends with a new line. + </para> <example id="example-temp1"> <title>A verbose syslog message template</title> - <screen>$template verbose,"%syslogseverity%,%syslogfacility%,%timegenerated%,%HOSTNAME%,%syslogtag%,%msg%\n"</screen> + <screen>$template verbose, "%syslogseverity%, %syslogfacility%, %timegenerated%, %HOSTNAME%, %syslogtag%, %msg%\n"</screen> </example> <para> - <xref linkend="example-temp2"/> shows a template that resembles a traditional wall message (a message that is send to every user that is logged in and has their <parameter>mesg(1)</parameter> permission set to <parameter>yes</parameter>). This template outputs the message text, along with a hostname, message tag and a timestamp, on a new line (using <literal>\r</literal> and <literal>\n</literal>) and rings the bell (using <literal>\7</literal>). - </para> + <xref linkend="example-temp2"/> shows a template that resembles a traditional wall message (a message that is send to every user that is logged in and has their <parameter>mesg(1)</parameter> permission set to <parameter>yes</parameter>). This template outputs the message text, along with a host name, message tag and a time stamp, on a new line (using <literal>\r</literal> and <literal>\n</literal>) and rings the bell (using <literal>\7</literal>). + </para> <example id="example-temp2"> <title>A wall message template</title> <screen>$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag% %msg%\n\r"</screen> </example> <para> <xref linkend="example-temp3"/> shows a template that formats a syslog message so that it can be used as a database query. Notice the use of the <parameter>sql</parameter> option at the end of the template specified as the template option. It tells the database writer to format the message as an MySQL <systemitem class="systemname">SQL</systemitem> query. - </para> + </para> <example id="example-temp3"> <title>A database formatted message template</title> - <screen>$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql</screen> + <screen>$template dbFormat,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')", sql</screen> </example> <para> - <application>rsyslog</application> also contains a set of predefined templates identified by the <literal>RSYSLOG_</literal> prefix. It is advisable to not create a template using this prefix to avoid any conflicts. The following list shows these predefined templates along with their definitions. + <application>rsyslog</application> also contains a set of predefined templates identified by the <literal>RSYSLOG_</literal> prefix. These are reserved for the syslog's use and it is advisable to not create a template using this prefix to avoid conflicts. The following list shows these predefined templates along with their definitions. </para> <variablelist> <varlistentry> <term><parameter>RSYSLOG_DebugFormat</parameter></term> <listitem> + <para> + A special format used for troubleshooting property problems. + </para> <screen>"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n\"</screen> </listitem> </varlistentry> <varlistentry> <term><parameter>RSYSLOG_SyslogProtocol23Format</parameter></term> <listitem> - <screen>"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\"</screen> + <para> + The format specified in IETF's internet-draft ietf-syslog-protocol-23, which is assumed to become the new syslog standard RFC. + </para> + <screen>"%PRI%1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\"</screen> </listitem> </varlistentry> <varlistentry> <term><parameter>RSYSLOG_FileFormat</parameter></term> <listitem> + <para> + A modern-style logfile format similar to TraditionalFileFormat, but with high-precision time stamps and time zone information. + </para> <screen>"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n\"</screen> </listitem> </varlistentry> <varlistentry> <term><parameter>RSYSLOG_TraditionalFileFormat</parameter></term> <listitem> + <para> + The older default log file format with low-precision time stamps. + </para> <screen>"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n\"</screen> </listitem> </varlistentry> <varlistentry> <term><parameter>RSYSLOG_ForwardFormat</parameter></term> <listitem> - <screen>"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%\"</screen> + <para> + A forwarding format with high-precision time stamps and time zone information. + </para> + <screen>"%PRI%%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%\"</screen> </listitem> </varlistentry> <varlistentry> <term><parameter>RSYSLOG_TraditionalForwardFormat</parameter></term> <listitem> - <screen>"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%\"</screen> - </listitem> + <para> + The traditional forwarding format with low-precision time stamps. + </para> + <screen>"%PRI%%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%\"</screen> + </listitem> </varlistentry> </variablelist> - </section> - </section> </section> - <section id="s2-rsyslog-cmd-options"> - <title><application>rsyslog</application> Command Line Configuration</title> - <para> - Some of <application>rsyslog</application>'s functionality can be configured through the command line options, as <application>sysklogd</application>'s can. Note that as of version 3 of <application>rsyslog</application>, this method was deprecated. To enable some of these option, you must specify the compatibility mode <application>rsyslog</application> should run in. However, configuring <application>rsyslog</application> through the command line options should be avoided. - </para> - <para> - To specify the compatibility mode <application>rsyslog</application> should run in, use the <option>-c</option> option. When no parameter is specified, <application>rsyslog</application> tries to be compatible with <application>sysklogd</application>. This is partially achieved by activating configuration directives that modify your configuration accordingly. Therefore, it is advisable to supply this option with a number that matches the major version of <application>rsyslog</application> that is in use and update your <filename>/etc/rsyslog.conf</filename> configuration file accordingly. If you want to, for example, use <application>sysklogd</application> options (which were deprecated in version 3 of <application>rsyslog</application>), you can specify so by executing the following command: - </para> - <screen>~]# rsyslogd -c 2</screen> - <para> - Options that are passed to the <systemitem class="daemon">rsyslogd</systemitem> daemon, including the backward compatibility mode, can be specified in the <filename>/etc/sysconfig/rsyslog</filename> configuration file. - </para> - <para> - For more information on various <command>rsyslogd</command> options, refer to <command>man rsyslogd</command>. - </para> + <section id="s2-global_directives"> + <title>Global Directives</title> + <para> + Global directives are configuration options that apply to the <systemitem class="daemon">rsyslogd</systemitem> daemon. They usually specify a value for a specific predefined variable that affects the behavior of the <systemitem class="daemon">rsyslogd</systemitem> daemon or a rule that follows. All of the global directives must start with a dollar sign (<literal>$</literal>). Only one directive can be specified per line. The following is an example of a global directive that specifies the maximum size of the syslog message queue: + </para> + <screen> +$MainMsgQueueSize 50000 + </screen> + <para> + The default size defined for this directive (<constant>10,000</constant> messages) can be overridden by specifying a different value (as shown in the example above). + </para> + <para> + You may define multiple directives in your <filename>/etc/rsyslog.conf</filename> configuration file. A directive affects the behavior of all configuration options until another occurrence of that same directive is detected. Global directives can be used to configure actions, queues and for debugging. A comprehensive list of all available configuration directives can be found in <xref linkend="brid-Log_Files-Resources-Online"/>. Currently, a new configuration format has been developed that replaces the $-based syntax (see <xref linkend="s2-using_the_new_configuration_format"/>). However, classic global directives remain supported as a legacy format. + </para> </section> - </section> - -  - - <section id="s1-logfiles-locating"> - <title>Locating Log Files</title> - <indexterm significance="normal"> - <primary>log files</primary> - <secondary>locating</secondary> - </indexterm> - <para>Most log files are located in the <filename>/var/log/</filename> directory. Some applications such as <command>httpd</command> and <command>samba</command> have a directory within <filename>/var/log/</filename> for their log files.</para> - <indexterm significance="normal"> - <primary>log files</primary> - <secondary>rotating</secondary> - </indexterm> - <indexterm significance="normal"> - <primary> - <command>logrotate</command> - </primary> - </indexterm> - <para>You may notice multiple files in the <filename>/var/log/</filename> directory with numbers after them (for example, <filename>cron-20100906</filename>). These numbers represent a timestamp that has been added to a rotated log file. Log files are rotated so their file sizes do not become too large. The <filename>logrotate</filename> package contains a cron task that automatically rotates log files according to the <filename>/etc/logrotate.conf</filename> configuration file and the configuration files in the <filename>/etc/logrotate.d/</filename> directory.</para> - <section id="configuring-logrotate"> - <title>Configuring logrotate</title> - <para> + <section id="s2-log_rotation"> + <title>Log Rotation</title> + <para> The following is a sample <filename>/etc/logrotate.conf</filename> configuration file: </para> <screen> @@ -833,12 +810,12 @@ rotate 4 compress </screen> <para> - All of the lines in the sample configuration file define global options that apply to every log file. In our example, log files are rotated weekly, rotated log files are kept for the duration of 4 weeks, and all rotated log files are compressed by <application>gzip</application> into the <literal>.gz</literal> format. Any lines that begin with a hash sign (#) are comments and are not processed</para> + All of the lines in the sample configuration file define global options that apply to every log file. In our example, log files are rotated weekly, rotated log files are kept for four weeks, and all rotated log files are compressed by <application>gzip</application> into the <literal>.gz</literal> format. Any lines that begin with a hash sign (#) are comments and are not processed.</para> <para> - You may define configuration options for a specific log file and place it under the global options. However, it is advisable to create a separate configuration file for any specific log file in the <filename>/etc/logrotate.d/</filename> directory and define any configuration options there. + You may define configuration options for a specific log file and place it under the global options. However, it is advisable to create a separate configuration file for any specific log file in the <filename class="directory">/etc/logrotate.d/</filename> directory and define any configuration options there. </para> <para> - The following is an example of a configuration file placed in the <filename>/etc/logrotate.d/</filename> directory: + The following is an example of a configuration file placed in the <filename class="directory">/etc/logrotate.d/</filename> directory: </para> <screen> /var/log/messages { @@ -850,193 +827,1547 @@ compress } </screen> <para> - The configuration options in this file are specific for the <filename>/var/log/messages</filename> log file only. The settings specified here override the global settings where possible. Thus the rotated <filename>/var/log/messages</filename> log file will be kept for five weeks instead of four weeks as was defined in the global options. + The configuration options in this file are specific for the <filename>/var/log/messages</filename> log file only. The settings specified here override the global settings where possible. Thus the rotated <filename>/var/log/messages</filename> log file will be kept for five weeks instead of four weeks as was defined in the global options. + </para> + <para> + The following is a list of some of the directives you can specify in your <application>logrotate</application> configuration file: + </para> + <itemizedlist> + <listitem> + <para> + <parameter>weekly</parameter> — Specifies the rotation of log files to be done weekly. Similar directives include: + <itemizedlist> + <listitem> + <para> + <parameter>daily</parameter> + </para> + </listitem> + <listitem> + <para> + <parameter>monthly</parameter> + </para> + </listitem> + <listitem> + <para> + <parameter>yearly</parameter> + </para> + </listitem> + </itemizedlist> + </para> + </listitem> + <listitem> + <para> + <parameter>compress</parameter> — Enables compression of rotated log files. Similar directives include: + <itemizedlist> + <listitem> + <para> + <parameter>nocompress</parameter> + </para> + </listitem> + <listitem> + <para> + <parameter>compresscmd</parameter> — Specifies the command to be used for compressing. + </para> + </listitem> + <listitem> + <para> + <parameter>uncompresscmd</parameter> + </para> + </listitem> + <listitem> + <para> + <parameter>compressext</parameter> — Specifies what extension is to be used for compressing. + </para> + </listitem> + <listitem> + <para> + <parameter>compressoptions</parameter> — Lets you specify any options that may be passed to the used compression program. + </para> + </listitem> + <listitem> + <para> + <parameter>delaycompress</parameter> — Postpones the compression of log files to the next rotation of log files. + </para> + </listitem> + </itemizedlist> + </para> + </listitem> + <listitem> + <para> + <parameter>rotate <replaceable>INTEGER</replaceable> + </parameter> — Specifies the number of rotations a log file undergoes before it is removed or mailed to a specific address. If the value <constant>0</constant> is specified, old log files are removed instead of rotated. + </para> + </listitem> + <listitem> + <para> + <parameter>mail <replaceable>ADDRESS</replaceable> + </parameter> — This option enables mailing of log files that have been rotated as many times as is defined by the <parameter>rotate</parameter> directive to the specified address. Similar directives include: + </para> + <itemizedlist> + <listitem> + <para> + <parameter>nomail</parameter> + </para> + </listitem> + <listitem> + <para> + <parameter>mailfirst</parameter> — Specifies that the just-rotated log files are to be mailed, instead of the about-to-expire log files. + </para> + </listitem> + <listitem> + <para> + <parameter>maillast</parameter> — Specifies that the about-to-expire log files are to be mailed, instead of the just-rotated log files. This is the default option when <parameter>mail</parameter> is enabled. + </para> + </listitem> + </itemizedlist> + </listitem> + </itemizedlist> + <para> + For the full list of directives and various configuration options, see the <filename>logrotate(5)</filename> manual page. + </para> + </section> + <section id="s2-using_the_new_configuration_format"> + <title>Using the New Configuration Format</title> + <para> + In <application>rsyslog</application> version 6, a new configuration syntax has been introduced. This new configuration format aims to be more powerful, more intuitive, and to prevent common mistakes by not permitting certain invalid constructs. The syntax enhancement is enabled by the new configuration processor that relies on RainerScript. The legacy format is still fully supported and it is used by default in the <filename>/etc/rsyslog.conf</filename> configuration file. + </para> + <para> + RainerScript is a scripting language designed for processing network events and configuring event processors such as <application>rsyslog</application>. RainerScript was primarily used to define expression-based filters, see <xref linkend="ex-expression-based_filters"/>. The newest version of RainerScript implements the <function>input()</function> and <function>ruleset()</function> statements, which permit the <filename>/etc/rsyslog.conf</filename> configuration file to be written in the new style only. + </para> + <para> + In the following examples you can compare the configuration written with legacy-style parameters: + </para> + <screen> +$InputFileName /tmp/inputfile +$InputFileTag tag1: +$InputFileStateFile inputfile-state +$InputRunFileMonitor + </screen> + <para> + and the same configuration with use of the new format statement: + </para> + <screen> +input(type="imfile" file="/tmp/inputfile" tag="tag1:" statefile="inputfile-state") + </screen> + <para> + This significantly reduces the number of parameters used in configuration, improves readability, and also provides higher execution speed. For more information on RainerScript statements and parameters see <xref linkend="brid-Log_Files-Resources-Online"/>. + </para> + </section> + <section id="s2-Rulesets"> + <title>Rulesets</title> + <para> + Leaving special directives aside, <application>rsyslog</application> handles messages as defined by <emphasis>rules</emphasis> that consist of a filter condition and an action to be performed if the condition is true. With a traditionally written <filename>/etc/rsyslog.conf</filename> file, all rules are evaluated in order of appearance for every input message. This process starts with the first rule and continues until all rules have been processed or until the message is discarded by one of the rules. + </para> + <para> + However, rules can be grouped into sequences called <firstterm>rulesets</firstterm>. With rulesets, you can limit the effect of certain rules only to selected inputs or enhance the performance of <application>rsyslog</application> by defining a distinct set of actions bound to a specific input. In other words, filter conditions that will be inevitably evaluated as false for certain types of messages can be skipped. With the new configuration format, the <function>input()</function> and <function>ruleset()</function> statements are reserved for this operation. The ruleset definition in <filename>/etc/rsyslog.conf</filename> can look as follows: + </para> + <screen> +ruleset(name="<replaceable>rulesetname</replaceable>") { + <replaceable>rule</replaceable> + <replaceable>rule2</replaceable> + call <replaceable>rulesetname2</replaceable> + … +} + </screen> +  + <para> + Replace <replaceable>rulesetname</replaceable> with an identifier for your ruleset. The ruleset name cannot start with <literal>RSYSLOG_</literal> since this namespace is reserved for use by <application>rsyslog</application>. <literal>RSYSLOG_DefaultRuleset</literal> then defines the default set of rules to be performed if the message has no other ruleset assigned. With <replaceable>rule</replaceable> and <replaceable>rule2</replaceable> you can define rules in filter-action format mentioned above. With the <parameter>call</parameter> parameter, you can nest rulesets by calling them from inside other ruleset blocks. + </para> + <para> + After creating a ruleset, you need to specify what input it will apply to: + </para> + <synopsis>input(type="<replaceable>input_type</replaceable>" port="<replaceable>port_num</replaceable>" ruleset="<replaceable>rulesetname</replaceable>");</synopsis> + <para> + Here you can identify an input message by <replaceable>input_type</replaceable>, which is an input module that gathered the message, or by <replaceable>port_num</replaceable> – the port number. Other parameters such as <emphasis>file</emphasis> or <emphasis>tag</emphasis> can be specified for <function>input()</function>. Replace <replaceable>rulesetname</replaceable> with a name of the ruleset to be evaluated against the message. In case an input message is not explicitly bound to a ruleset, the default ruleset is triggered. + </para> + <para> + You can also use the legacy format to define rulesets, for more information see <xref linkend="brid-Log_Files-Resources-Online"/>. + </para> + <example id="ex-using_rulesets"> + <title>Using rulesets</title> + <para> + The following rulesets ensure different handling of remote messages coming from different ports. Add the following into <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +ruleset(name="remote-10514") { + action(type="omfile" file="/var/log/remote-10514") +} + +ruleset(name="remote-10515") { + cron.* action(type="omfile" file="/var/log/remote-10515-cron") + mail.* action(type="omfile" file="/var/log/remote-10515-mail") +} + +input(type="imtcp" port="10514" ruleset="remote-10514"); +input(type="imtcp" port="10515" ruleset="remote-10515"); + </screen> + <para> + Rulesets shown in the above example define log destinations for the remote input from two ports, in case of 10515, messages are sorted according to the facility. Then, the TCP input is enabled and bound to rulesets. Note that you must load the required modules (imtcp) for this configuration to work. + </para> + </example> + </section> + <section id="s2-compatibility_with_syslogd"> + <title>Compatibility with syslogd</title> + <para> + From <application>rsyslog</application> version 6, compatibility mode specified via the <option>-c</option> option has been removed. Also, the syslogd-style command line options are deprecated and configuring <application>rsyslog</application> through these command line options should be avoided. However, you can use several templates and directives to configure <systemitem class="daemon">rsyslogd</systemitem> to emulate syslogd-like behavior. + </para> + <para> + For more information on various <systemitem class="daemon">rsyslogd</systemitem> options, see the <filename>rsyslogd(8)</filename>manual page. + </para> + </section> + </section> + <section id="s1-working_with_queues_in_rsyslog"> + <title>Working with Queues in Rsyslog</title> + <para> + Queues are used to pass content, mostly syslog messages, between components of <application>rsyslog</application>. With queues, rsyslog is capable of processing multiple messages simultaneously and to apply several actions to a single message at once. The data flow inside <application>rsyslog</application> can be illustrated as follows: + </para> + <figure id="fig-message_flow_in_rsyslog"> + <title>Message Flow in Rsyslog</title> + <mediaobject> + <imageobject> + <imagedata fileref="images/rsyslog_message_flow.png" format="PNG"/> + </imageobject> + <textobject> + <para>Message Flow in Rsyslog</para> + </textobject> + </mediaobject> + </figure> + <para> + Whenever <application>rsyslog</application> receives a message, it passes this message to the preprocessor and then places it into the <firstterm>main message queue</firstterm>. Messages wait there to be dequeued and passed to the <firstterm>rule processor</firstterm>. + </para> + <para> + The <emphasis>rule processor</emphasis> is a parsing and filtering engine. Here, the rules defined in <filename>/etc/rsyslog.conf</filename> are applied. Based on these rules, the rule processor evaluates which actions are to be performed. Each action has its own action queue. Messages are passed through this queue to the respective action processor which creates the final output. Note that at this point, several actions can run simultaneously on one message. For this purpose, a message is duplicated and passed to multiple action processors. + </para> + <para> + Only one queue per action is possible. Depending on configuration, the messages can be sent right to the action processor without action queuing. This is the behavior of <emphasis>direct queues</emphasis> (see below). In case the output action fails, the action processor notifies the action queue, which then takes an unprocessed element back and after some time interval, the action is attempted again. + </para> + <para> + To sum up, there are two positions where queues stand in <application>rsyslog</application>: either in front of the rule processor as a single <emphasis>main message queue</emphasis> or in front of various types of output actions as <emphasis>action queues</emphasis>. Queues provide two main advantages that both lead to increased performance of message processing: + </para> + <itemizedlist> + <listitem> + <para> + they serve as buffers that <emphasis>decouple</emphasis> producers and consumers in the structure of <application>rsyslog</application> + </para> + </listitem> + <listitem> + <para> + they allow for <emphasis>parallelization</emphasis> of actions performed on messages + </para> + </listitem> + </itemizedlist> + <para> + Apart from this, queues can be configured with several directives to provide optimal performance for your system. These configuration options are covered in the following sections. + </para> + <section id="s2-defining_queues"> + <title>Defining Queues</title> + <para> + Based on where the messages are stored, there are several types of queues: <emphasis>direct</emphasis>, <emphasis>in-memory</emphasis>, <emphasis>disk</emphasis>, and <emphasis>disk-assisted in-memory</emphasis> queues that are most widely used. You can choose one of these types for the main message queue and also for action queues. Add the following into <filename>/etc/rsyslog.conf</filename>: + </para> + <synopsis>$<replaceable>object</replaceable>QueueType <replaceable>queue_type</replaceable></synopsis> + <para> + Here, you can apply the setting for the main message queue (replace <replaceable>object</replaceable> with <option>MainMsg</option>) or for an action queue (replace <replaceable>object</replaceable> with <option>Action</option>). Replace <replaceable>queue_type</replaceable> with one of <literal>direct</literal>, <literal>linkedlist</literal> or <literal>fixedarray</literal> (which are in-memory queues), or <literal>disk</literal>. + + </para> + <para> + The default setting for a main message queue is the FixedArray queue with a limit of 10,000 messages. Action queues are by default set as Direct queues. + </para> +  + <bridgehead id="br-direct_queues">Direct Queues</bridgehead> + <para> + For many simple operations, such as when writing output to a local file, building a queue in front of an action is not needed. To avoid queuing, use: + </para> + <synopsis>$<replaceable>object</replaceable>QueueType Direct</synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option> to use this option to the main message queue or for an action queue respectively. With direct queue, messages are passed directly and immediately from the producer to the consumer. + </para> + <bridgehead id="br-disk_queues">Disk Queues</bridgehead> + <para> + Disk queues store messages strictly on a hard drive, which makes them highly reliable but also the slowest of all possible queuing modes. This mode can be used to prevent the loss of highly important log data. However, disk queues are not recommended in most use cases. To set a disk queue, type the following into <filename>/etc/rsyslog.conf</filename>: + </para> + <synopsis>$<replaceable>object</replaceable>QueueType Disk</synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option> to use this option to the main message queue or for an action queue respectively. Disk queues are written in parts, with a default size 10 Mb. This default size can be modified with the following configuration directive: + </para> + <synopsis>$<replaceable>object</replaceable>QueueMaxFileSize <replaceable>size</replaceable></synopsis> + <para> + where <replaceable>size</replaceable> represents the specified size of disk queue part. The defined size limit is not restrictive, <application>rsyslog</application> always writes one complete queue entry, even if it violates the size limit. Each part of a disk queue matches with an individual file. The naming directive for these files looks as follows: + </para> + <synopsis>$<replaceable>object</replaceable>QueueFilename <replaceable>name</replaceable></synopsis> + <para> + This sets a <replaceable>name</replaceable> prefix for the file followed by a 7-digit number starting at one and incremented for each file. + </para> + <bridgehead id="br-in-memory_queues">In-memory Queues</bridgehead> + <para> + With in-memory queue, the enqueued messages are held in memory which makes the process very fast. The queued data is lost if the computer is power cycled or shut down. However, you can use the <option>$ActionQueueSaveOnShutdown</option> setting to save the data before shutdown. There are two types of in-memory queues: + </para> + <itemizedlist> + <listitem> + <para> + <emphasis>FixedArray</emphasis> queue — the default mode for the main message queue, with a limit of 10,000 elements. This type of queue uses a fixed, pre-allocated array that holds pointers to queue elements. Due to these pointers, even if the queue is empty a certain amount of memory is consumed. However, FixedArray offers the best run time performance and is optimal when you expect a relatively low number of queued messages and high performance. + </para> + </listitem> + <listitem> + <para> + <emphasis>LinkedList</emphasis> queue — here, all structures are dynamically allocated in a linked list, thus the memory is allocated only when needed. LinkedList queues handle occasional message bursts very well. + </para> + </listitem> + </itemizedlist> + <para> + In general, use LinkedList queues when in doubt. Compared to FixedArray, it consumes less memory and lowers the processing overhead. + </para> + <para> + Use the following syntax to configure in-memory queues: + </para> + <synopsis>$<replaceable>object</replaceable>QueueType LinkedList</synopsis> + <synopsis>$<replaceable>object</replaceable>QueueType FixedArray</synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option> to use this option to the main message queue or for an action queue respectively. + </para> + <bridgehead id="br-disk-assisted_in-memory_queues">Disk-Assisted In-memory Queues</bridgehead> + <para> + Both disk and in-memory queues have their advantages and <application>rsyslog</application> lets you combine them in <emphasis>disk-assisted in-memory queues</emphasis>. To do so, configure a normal in-memory queue and then add the <option>$objectQueueFileName</option> directive to define a file name for disk assistance. This queue then becomes <emphasis>disk-assisted</emphasis>, which means it couples an in-memory queue with a disk queue to work in tandem. + </para> + <para> + The disk queue is activated if the in-memory queue is full or needs to persist after shutdown. With a disk-assisted queue, you can set both disk-specific and in-memory specific configuration parameters. This type of queue is probably the most commonly used, it is especially useful for potentially long-running and unreliable actions. + </para> + <para> + To specify the functioning of a disk-assisted in-memory queue, use the so-called <emphasis>watermarks</emphasis>: + </para> + <synopsis>$<replaceable>object</replaceable>QueueHighWatermark <replaceable>number</replaceable></synopsis> + <synopsis>$<replaceable>object</replaceable>QueueLowWatermark <replaceable>number</replaceable></synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option> to use this option to the main message queue or for an action queue respectively. Replace <replaceable>number</replaceable> with a number of enqueued messages. When an in-memory queue reaches the number defined by the high watermark, it starts writing messages to disk and continues until the in-memory queue size drops to the number defined with the low watermark. Correctly set watermarks minimize unnecessary disk writes, but also leave memory space for message bursts since writing to disk files is rather lengthy. Therefore, the high watermark must be lower than the whole queue capacity set with <emphasis>$objectQueueSize</emphasis>. The difference between the high watermark and the overall queue size is a spare memory buffer reserved for message bursts. On the other hand, setting the high watermark too low will turn on disk assistance unnecessarily ofte n. + </para> + <example id="ex-net_forwarding_with_queue"> + <title>Reliable Forwarding of Log Messages to a Server</title> + <para> + Rsyslog is often used to maintain a centralized logging system, where log messages are forwarded to a server over the network. To avoid message loss when the server is not available, it is advisable to configure an action queue for the forwarding action. This way, messages that failed to be sent are stored locally until the server is reachable again. Note that such queues are not configurable for connections using the <systemitem class="protocol">UDP</systemitem> protocol. To establish a fully reliable connection, for example when your logging server is outside of your private network, consider using the RELP protocol described in <xref linkend="s2-using_RELP" />. + </para> + <procedure id="proc-net_forwarding_with_queue"> + <title>Forwarding To a Single Server</title> + <para> + Suppose the task is to forward log messages from the system to a server with host name <emphasis>example.com</emphasis>, and to configure an action queue to buffer the messages in case of a server outage. To do so, perform the following steps: + </para> + <step> + <para> + Create a working directory to store the queue files. For example: + </para> + <synopsis>~]# <command>mkdir</command> <filename class="directory">/rsyslog/work/</filename></synopsis> + </step> + <step> + <para> + Use the following configuration in <filename>/etc/rsyslog.conf</filename> or create a file with the following content in the <filename class="directory">/etc/rsyslog.d/</filename> directory: + </para> + <screen>$WorkDirectory /rsyslog/work + +$ActionQueueType LinkedList +$ActionQueueFileName example_fwd +$ActionResumeRetryCount -1 +$ActionQueueSaveOnShutdown on +*.* @@example.com:18</screen> + <itemizedlist> + <para> + Where: + </para> + <listitem> + <para> + the <filename class="directory">/rsyslog/work/</filename> directory created in the previous step is marked as a working directory, + </para> + </listitem> + <listitem> + <para> + <option>$ActionQueueType</option> enables a LinkedList in-memory queue, + </para> + </listitem> + <listitem> + <para> + <option>$ActionFileName</option> defines a disk storage, in this case the backup files are created in the <filename class="directory">/rsyslog/work/</filename> directory with the <emphasis>example_fwd</emphasis> prefix, + </para> + </listitem> + <listitem> + <para> + the <option>$ActionResumeRetryCount -1</option> setting prevents rsyslog form dropping messages when retrying to connect if server is not responding, + </para> + </listitem> + <listitem> + <para> + enabled <option>$ActionQueueSaveOnShutdown</option> saves in-memory data if rsyslog shuts down, + </para> + </listitem> + <listitem> + <para> + the last line forwards all received messages to the logging server, port specification is optional. + </para> + </listitem> + </itemizedlist> + <para> + With the above configuration, rsyslog keeps messages in memory if the remote server is not reachable. A file on disk is created only if rsyslog runs out of the configured memory queue space or needs to shut down, which benefits the system performance.</para> + </step> + </procedure> + <procedure id="proc-net_forwarding_two_servers"> + <title>Forwarding To Multiple Servers</title> + <para> + The process of forwarding log messages to multiple servers is similar to the previous procedure: + </para> + <step> + <para> + Create a working directory for rsyslog to store the queue files. For example: + </para> + <synopsis>~]# <command>mkdir</command> <filename class="directory">/rsyslog/work/</filename></synopsis> + </step> + <step> + <para> + Each destination server requires a separate forwarding rule, action queue specification, and backup file on disk. For example, use the following configuration in <filename>/etc/rsyslog.conf</filename> or create a file with the following content in the <filename class="directory">/etc/rsyslog.d/</filename> directory: + </para> + <screen>$WorkDirectory /rsyslog/work + +$ActionQueueType LinkedList +$ActionQueueFileName example_fwd1 +$ActionResumeRetryCount -1 +$ActionQueueSaveOnShutdown on +*.* @(a)example1.com + +$ActionQueueType LinkedList +$ActionQueueFileName example_fwd2 +$ActionResumeRetryCount -1 +$ActionQueueSaveOnShutdown on +*.* @(a)example2.com</screen> + </step> + </procedure> + </example> + </section> + <section id="s2-managing_queues"> + <title>Managing Queues</title> + <para> + All types of queues can be further configured to match your requirements. You can use several directives to modify both action queues and the main message queue. Currently, there are more than 20 queue parameters available, see <xref linkend="brid-Log_Files-Resources-Online"/>. Some of these settings are used commonly, others, such as worker thread management, provide closer control over the queue behavior and are reserved for advanced users. With advanced settings, you can optimize <application>rsyslog</application>'s performance, schedule queuing, or modify the behavior of queue on system shutdown. + </para> + <bridgehead id="br-limiting_queue_size">Limiting Queue Size</bridgehead> + <para> + You can limit the number of messages that queue can contain with the following setting: + </para> + <synopsis>$<replaceable>object</replaceable>QueueHighWatermark <replaceable>number</replaceable></synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option> to use this option to the main message queue or for an action queue respectively. Replace <replaceable>number</replaceable> with a number of enqueued messages. You can set the queue size only as the number of messages, not as their actual memory size. The default queue size is 10,000 messages for the main message queue and ruleset queues, and 1000 for action queues. + </para> + <para> + Disk assisted queues are unlimited by default and can not be restricted with this directive, but you can reserve them physical disk space in bytes with the following settings: + </para> + <synopsis>$<replaceable>object</replaceable>QueueMaxDiscSpace <replaceable>number</replaceable></synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option>. When the size limit specified by <replaceable>number</replaceable> is hit, messages are discarded until sufficient amount of space is freed by dequeued messages. + </para> + <bridgehead id="br-discarding_messages">Discarding Messages</bridgehead> + <para> + When a queue reaches a certain number of messages, you can discard less important messages in order to save space in the queue for entries of higher priority. The threshold that launches the discarding process can be set with the so-called <emphasis>discard mark</emphasis>: + </para> + <synopsis>$<replaceable>object</replaceable>QueueDiscardMark <replaceable>number</replaceable></synopsis> + <para> + Replace <replaceable>object</replaceable> with <option>MainMsg</option> or with <option>Action</option> to use this option to the main message queue or for an action queue respectively. Here, <replaceable>number</replaceable> stands for a number of messages that have to be in the queue to start the discarding process. To define which messages to discard, use: + </para> + <synopsis>$<replaceable>object</replaceable>QueueDiscardSeverity <replaceable>priority</replaceable></synopsis> + <para> + Replace <replaceable>priority</replaceable> with one of the following keywords (or with a number): <command>debug</command> (7), <command>info</command> (6), <command>notice</command> (5), <command>warning</command> (4), <command>err</command> (3), <command>crit</command> (2), <command>alert</command> (1), and <command>emerg</command> (0). With this setting, both newly incoming and already queued messages with lower than defined priority are erased from the queue immediately after the discard mark is reached. + </para> + <bridgehead id="br-using_timeframes">Using Timeframes</bridgehead> + <para> + You can configure <application>rsyslog</application> to process queues during a specific time period. With this option you can, for example, transfer some processing into off-peak hours. To define a time frame, use the following syntax: + </para> + <synopsis>$<replaceable>object</replaceable>QueueDequeueTimeBegin <replaceable>hour</replaceable></synopsis> + <synopsis>$<replaceable>object</replaceable>QueueDequeueTimeEnd <replaceable>hour</replaceable></synopsis> + <para> + With <replaceable>hour</replaceable> you can specify hours that bound your time frame. Use the 24-hour format without minutes. + </para> + <bridgehead id="br-configuring_worker_threads">Configuring Worker Threads</bridgehead> + <para> + A <firstterm>worker thread</firstterm> performs a specified action on the enqueued message. For example, in the main message queue, a worker task is to apply filter logic to each incoming message and enqueue them to the relevant action queues. When a message arrives, a worker thread is started automatically. When the number of messages reaches a certain number, another worker thread is turned on. To specify this number, use: + </para> + <synopsis>$<replaceable>object</replaceable>QueueWorkerThreadMinimumMessages <replaceable>number</replaceable></synopsis> + <para> + Replace <replaceable>number</replaceable> with a number of messages that will trigger a supplemental worker thread. For example, with <replaceable>number</replaceable> set to 100, a new worker thread is started when more than 100 messages arrive. When more than 200 messages arrive, the third worker thread starts and so on. However, too many working threads running in parallel becomes ineffective, so you can limit the maximum number of them by using: + </para> + <synopsis>$<replaceable>object</replaceable>QueueWorkerThreads <replaceable>number</replaceable></synopsis> + <para> + where <replaceable>number</replaceable> stands for a maximum number of working threads that can run in parallel. For the main message queue, the default limit is 1 thread. Once a working thread has been started, it keeps running until an inactivity timeout appears. To set the length of timeout, type: + </para> + <synopsis>$<replaceable>object</replaceable>QueueWorkerTimeoutThreadShutdown <replaceable>time</replaceable></synopsis> + <para> + Replace <replaceable>time</replaceable> with the duration set in milliseconds. Without this setting, a zero timeout is applied and a worker thread is terminated immediately when it runs out of messages. If you specify <replaceable>time</replaceable> as <literal>-1</literal>, no thread will be closed. + </para> + <bridgehead id="br-batch_dequeuing">Batch Dequeuing</bridgehead> + <para> + To increase performance, you can configure <application>rsyslog</application> to dequeue multiple messages at once. To set the upper limit for such dequeueing, use: + </para> + <synopsis>$<replaceable>object</replaceable>QueueDequeueBatchSize <replaceable>number</replaceable></synopsis> + <para> + Replace <replaceable>number</replaceable> with the maximum number of messages that can be dequeued at once. Note that a higher setting combined with a higher number of permitted working threads results in greater memory consumption. + </para> + <bridgehead id="br-terminating_queues">Terminating Queues</bridgehead> + <para> + When terminating a queue that still contains messages, you can try to minimize the data loss by specifying a time interval for worker threads to finish the queue processing: + </para> + <synopsis>$<replaceable>object</replaceable>QueueTimeoutShutdown <replaceable>time</replaceable></synopsis> + <para> + Specify <replaceable>time</replaceable> in milliseconds. If after that period there are still some enqueued messages, workers finish the current data element and then terminate. Unprocessed messages are therefore lost. Another time interval can be set for workers to finish the final element: + </para> + <synopsis>$<replaceable>object</replaceable>QueueTimeoutActionCompletion <replaceable>time</replaceable></synopsis> + <para> + In case this timeout expires, any remaining workers are shut down. To save data at shutdown, use: + </para> + <synopsis>$<replaceable>object</replaceable>QueueTimeoutSaveOnShutdown <replaceable>time</replaceable></synopsis> + <para> + If set, all queue elements are saved to disk before <application>rsyslog</application> terminates. + </para> + </section>  + </section> + <section id="s1-using_rsyslog_modules"> + <title>Using Rsyslog Modules</title> + <para> + Due to its modular design, <application>rsyslog</application> offers a variety of <firstterm>modules</firstterm> which provide additional functionality. Note that modules can be written by third parties. Most modules provide additional inputs (see <emphasis>Input Modules</emphasis> below) or outputs (see <emphasis>Output Modules</emphasis> below). Other modules provide special functionality specific to each module. The modules may provide additional configuration directives that become available after a module is loaded. To load a module, use the following syntax: + </para> + <screen> +$ModLoad <replaceable>MODULE</replaceable> + </screen> + <para> + where <option>$ModLoad</option> is the global directive that loads the specified module and <replaceable>MODULE</replaceable> represents your desired module. For example, if you want to load the Text File Input Module (<command>imfile</command>) that enables <application>rsyslog</application> to convert any standard text files into syslog messages, specify the following line in the <filename>/etc/rsyslog.conf</filename> configuration file: + </para> + <screen> +$ModLoad imfile + </screen> + <para> + <application>rsyslog</application> offers a number of modules which are split into the following main categories: + </para> + <itemizedlist> + <listitem> + <para> + Input Modules — Input modules gather messages from various sources. The name of an input module always starts with the <literal>im</literal> prefix, such as <command>imfile</command>, <command>imjournal</command>, etc. + </para> + </listitem> + <listitem> + <para> + Output Modules — Output modules provide a facility to issue message to various targets such as sending across a network, storing in a database, or encrypting. The name of an output module always starts with the <literal>om</literal> prefix, such as <command>omsnmp</command>, <command>omrelp</command>, etc. + </para> + </listitem> + <listitem> + <para> + Parser Modules — These modules are useful in creating custom parsing rules or to parse malformed messages. With moderate knowledge of the C programming language, you can create your own message parser. The name of a parser module always starts with the <literal>pm</literal> prefix, such as <command>pmrfc5424</command>, <command>pmrfc3164</command>, and so on. + </para> + </listitem> + <listitem> + <para> + Message Modification Modules — Message modification modules change content of syslog messages. Names of these modules start with the <literal>mm</literal> prefix. Message Modification Modules such as <command>mmanon</command>, <command>mmnormalize</command>, or <command>mmjsonparse</command> are used for anonymization or normalization of messages. + </para> + </listitem> + <listitem> + <para> + String Generator Modules — String generator modules generate strings based on the message content and strongly cooperate with the template feature provided by <application>rsyslog</application>. For more information on templates, see <xref linkend="s2-Templates"/>. The name of a string generator module always starts with the <literal>sm</literal> prefix, such as <command>smfile</command> or <command>smtradfile</command>. + </para> + </listitem> + <listitem> + <para> + Library Modules — Library modules provide functionality for other loadable modules. These modules are loaded automatically by <application>rsyslog</application> when needed and cannot be configured by the user. + </para> + </listitem> + </itemizedlist> + <para> + A comprehensive list of all available modules and their detailed description can be found at <ulink url="http://www.rsyslog.com/doc/rsyslog_conf_modules.html/">http://www.rsyslog.com/doc/rsyslog_conf_modules.html</ulink>. + </para> + <warning> + <para> + Note that when <application>rsyslog</application> loads any modules, it provides them with access to some of its functions and data. This poses a possible security threat. To minimize security risks, use trustworthy modules only. + </para> + </warning> + <section id="s2-importing_text_files"> + <title>Importing Text Files</title> + <para> + The Text File Input Module, abbreviated as <command>imfile</command>, enables <application>rsyslog</application> to convert any text file into a stream of syslog messages. You can use <command>imfile</command> to import log messages from applications that create their own text file logs. To load <command>imfile</command>, add the following into <filename>etc/rsyslog.conf</filename>: + </para> + <screen> +$ModLoad imfile +$InputFilePollInterval <replaceable>int</replaceable> + </screen> + <para> + It is sufficient to load <command>imfile</command> once, even when importing multiple files. The <emphasis>$InputFilePollInterval</emphasis> global directive specifies how often <application>rsyslog</application> checks for changes in connected text files. The default interval is 10 seconds, to change it, replace <replaceable>int</replaceable> with a time interval specified in seconds. + </para> + <para> + To identify the text files to import, use the following syntax in <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +# File 1 +$InputFileName <replaceable>path_to_file</replaceable> +$InputFileTag <replaceable>tag:</replaceable> +$InputFileStateFile <replaceable>state_file_name</replaceable> +$InputFileSeverity <replaceable>severity</replaceable> +$InputFileFacility <replaceable>facility</replaceable> +$InputRunFileMonitor + +# File 2 +$InputFileName <replaceable>path_to_file2</replaceable> +... + </screen> + <para> + Four settings are required to specify an input text file: + </para> + <itemizedlist > + <listitem> + <para> + replace <replaceable>path_to_file</replaceable> with a path to the text file. + </para> + </listitem> + <listitem> + <para> + replace <replaceable>tag:</replaceable> with a tag name for this message. + </para> + </listitem> + <listitem> + <para> + replace <replaceable>state_file_name</replaceable> with a unique name for the <emphasis>state file</emphasis>. <emphasis>State files</emphasis>, which are stored in the rsyslog working directory, keep cursors for the monitored files, marking what partition has already been processed. If you delete them, whole files will be read in again. Make sure that you specify a name that does not already exist. + </para> + </listitem> + <listitem> + <para> + add the <emphasis>$InputRunFileMonitor</emphasis> directive that enables the file monitoring. Without this setting, the text file will be ignored. + </para> + </listitem> + </itemizedlist> + <para> + Apart from the required directives, there are several other settings that can be applied on the text input. Set the severity of imported messages by replacing <replaceable>severity</replaceable> with an appropriate keyword. Replace <replaceable>facility</replaceable> with a keyword to define the subsystem that produced the message. The keywords for severity and facility are the same as those used in facility/priority-based filters, see <xref linkend="s2-Filters" />. + </para> + <example id="ex-importing_text_files"> + <title>Importing Text Files</title> + <para> + The Apache HTTP server creates log files in text format. To apply the processing capabilities of <application>rsyslog</application> to apache error messages, first use the <command>imfile</command> module to import the messages. Add the following into <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +$ModLoad imfile + +$InputFileName /var/log/httpd/error_log +$InputFileTag apache-error: +$InputFileStateFile state-apache-error +$InputRunFileMonitor + </screen> + </example> + </section> + <section id="s2-exporting_messages_to_a_database"> + <title>Exporting Messages to a Database</title> + <para> + Processing of log data can be faster and more convenient when performed in a database rather than with text files. Based on the type of DBMS used, choose from various output modules such as <command>ommysql</command>, <command>ompgsql</command>, <command>omoracle</command>, or <command>ommongodb</command>. As an alternative, use the generic <command>omlibdbi</command> output module that relies on the <systemitem class="library">libdbi</systemitem> library. The <command>omlibdbi</command> module supports database systems Firebird/Interbase, MS SQL, Sybase, SQLite, Ingres, Oracle, mSQL, MySQL, and PostgreSQL. + </para>  + <example id="ex-exporting_rsyslog_messages_to_a_database"> + <title>Exporting Rsyslog Messages to a Database</title> + <para> + To store the rsyslog messages in a MySQL database, add the following into <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +$ModLoad ommysql + +$ActionOmmysqlServerPort 1234 +*.* :ommysql:database-server,database-name,database-userid,database-password + </screen> + <para> + First, the output module is loaded, then the communication port is specified. Additional information, such as name of the server and the database, and authentication data, is specified on the last line of the above example. + </para> +  + </example> + </section> + <section id="s2-enabling_encrypted_transport"> + <title>Enabling Encrypted Transport</title> + <para> + Confidentiality and integrity in network transmissions can be provided by either the <emphasis>TLS</emphasis> or <emphasis>GSSAPI</emphasis> encryption protocol. + </para> + <para> + <emphasis>Transport Layer Security</emphasis> (TLS) is a cryptographic protocol designed to provide communication security over the network. When using TLS, rsyslog messages are encrypted before sending, and mutual authentication exists between the sender and receiver. + </para> + <para> + <emphasis>Generic Security Service API</emphasis> (GSSAPI) is an application programming interface for programs to access security services. To use it in connection with <application>rsyslog</application> you must have a functioning <application>Kerberos</application> environment. + </para> +  + </section> + <section id="s2-using_RELP"> + <title>Using RELP</title> + <para> + <emphasis>Reliable Event Logging Protocol</emphasis> (RELP) is a networking protocol for data logging in computer networks. It is designed to provide reliable delivery of event messages, which makes it useful in environments where message loss is not acceptable. + </para> +  + </section> + </section> + <section id="s1-interaction_of_rsyslog_and_journal"> + <title>Interaction of Rsyslog and Journal</title> + <para> + As mentioned above, <application>Rsyslog</application> and <application>Journal</application>, the two logging applications present on your system, have several distinctive features that make them suitable for specific use cases. In many situations it is useful to combine their capabilities, for example to create structured messages and store them in a file database (see <xref linkend="s1-structured_logging_with_rsyslog" />). A communication interface needed for this cooperation is provided by input and output modules on the side of <application>Rsyslog</application> and by the <application>Journal</application>'s communication socket. + </para> + <para> + By default, <systemitem class="daemon">rsyslogd</systemitem> uses the <systemitem>imjournal</systemitem> module as a default input mode for journal files. With this module, you import not only the messages but also the structured data provided by <systemitem class="daemon">journald</systemitem>. Also, older data can be imported from <systemitem class="daemon">journald</systemitem> (unless forbidden with the <option>$ImjournalIgnorePreviousMessages</option> directive). See <xref linkend="s2-importing_data_from_journal" /> for basic configuration of <systemitem>imjournal</systemitem>. + </para> + <para> + As an alternative, configure <systemitem class="daemon">rsyslogd</systemitem> to read from the socket provided by <systemitem class="daemon">journal</systemitem> as an output for syslog-based applications. The path to the socket is <filename>/run/systemd/journal/syslog</filename>. Use this option when you want to maintain plain rsyslog messages. Compared to <systemitem>imjournal</systemitem> the socket input currently offers more features, such as ruleset binding or filtering. To import <application>Journal</application> data trough the socket, use the following configuration in <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +$ModLoad imuxsock +$OmitLocalLogging off + </screen> + <para> + The above syntax loads the <systemitem>imuxsock</systemitem> module and turns off the <option>$OmitLocalLogging</option> directive, which enables the import trough the system socket. The path to this socket is specified separately in <filename>/etc/rsyslog.d/listen.conf</filename> as follows: + </para> + <screen> +$SystemLogSocketName /run/systemd/journal/syslog + </screen> + <para> + You can also output messages from <application>Rsyslog</application> to <application>Journal</application> with the <systemitem>omjournal</systemitem> module. Configure the output in <filename>/etc/rsyslog.conf</filename> as follows: + </para> + <screen> +$ModLoad omjournal + +*.* :omjournal: + </screen> + <para> + For instance, the following configuration forwards all received messages on tcp port 10514 to the Journal: + </para> + <screen> +$ModLoad imtcp +$ModLoad omjournal + +$RuleSet remote +*.* :omjournal: + +$InputTCPServerBindRuleset remote +$InputTCPServerRun 10514 + </screen> + </section> + <section id="s1-structured_logging_with_rsyslog"> + <title>Structured Logging with Rsyslog</title> + <para> + On systems that produce large amounts of log data, it can be convenient to maintain log messages in a <emphasis>structured format</emphasis>. With structured messages, it is easier to search for particular information, to produce statistics and to cope with changes and inconsistencies in message structure. <application>Rsyslog</application> uses the <emphasis>JSON</emphasis> (JavaScript Object Notation) format to provide structure for log messages. + </para> + <para> + Compare the following unstructured log message: + </para> + <synopsis>Oct 25 10:20:37 localhost anacron[1395]: Jobs will be executed sequentially</synopsis> + <para> + with a structured one: + </para> + <synopsis>{"timestamp":"2013-10-25T10:20:37", "host":"localhost", "program":"anacron", "pid":"1395", "msg":"Jobs will be executed sequentially"}</synopsis> + <para> + Searching structured data with use of key-value pairs is faster and more precise than searching text files with regular expressions. The structure also lets you to search for the same entry in messages produced by various applications. Also, JSON files can be stored in a document database such as <database class="name">MongoDB</database>, which provides additional performance and analysis capabilities. On the other hand, a structured message requires more disk space than the unstructured one. + </para> + <para> + In <application>rsyslog</application>, log messages with meta data are pulled from <application>Journal</application> with use of the <systemitem>imjournal</systemitem> module. With the <systemitem>mmjsonparse</systemitem> module, you can parse data imported from <application>Journal</application> and from other sources and process them further, for example as a database output. For parsing to be successful, <systemitem>mmjsonparse</systemitem> requires input messages to be structured in a way that is defined by the <emphasis role="bold">Lumberjack</emphasis> project. + </para> + <para> + The <emphasis role="bold">Lumberjack</emphasis> project aims to add structured logging to <application>rsyslog</application> in a backward-compatible way. To identify a structured message, <emphasis role="bold">Lumberjack</emphasis> specifies the <emphasis role="bold">@cee:</emphasis> string that prepends the actual JSON structure. Also, <emphasis role="bold">Lumberjack</emphasis> defines the list of standard field names that should be used for entities in the JSON string. For more information on <emphasis role="bold">Lumberjack</emphasis>, see <xref linkend="brid-Log_Files-Resources-Online"/>. + </para> + <para> + The following is an example of a lumberjack-formatted message: + </para> + <synopsis> + @cee: {"pid":17055, "uid":1000, "gid":1000, "appname":"logger", "msg":"Message text."} + </synopsis> + <para> + To build this structure inside <application>Rsyslog</application>, a template is used, see <xref linkend="s2-filtering_structured_messages" />. Applications and servers can employ the <systemitem class="library">libumberlog</systemitem> library to generate messages in the lumberjack-compliant form. For more information on <systemitem class="library">libumberlog</systemitem>, see <xref linkend="brid-Log_Files-Resources-Online"/>. + </para> + <section id="s2-importing_data_from_journal"> + <title>Importing Data from Journal</title> + <para> + The <command>imjournal</command> module is <application>Rsyslog</application>'s input module to natively read the journal files (see <xref linkend="s1-interaction_of_rsyslog_and_journal" />). Journal messages are then logged in text format as other rsyslog messages. However, with further processing, it is possible to translate meta data provided by <application>Journal</application> into a structured message. + </para> + <para> + To import data from <application>Journal</application> to <application>Rsyslog</application>, use the following configuration in <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +$ModLoad imjournal + +$imjournalPersistStateInterval <replaceable>number_of_messages</replaceable> +$imjournalStateFile <replaceable>path</replaceable> +$imjournalRatelimitInterval <replaceable>seconds</replaceable> +$imjournalRatelimitBurst <replaceable>burst_number</replaceable> +$ImjournalIgnorePreviousMessages <replaceable>off/on</replaceable> + </screen> + <itemizedlist> + <listitem> + <para> + With <replaceable>number_of_messages</replaceable>, you can specify how often the journal data must be saved. This will happen each time the specified number of messages is reached. + </para> + </listitem> + <listitem> + <para> + Replace <replaceable>path</replaceable> with a path to the state file. This file tracks the journal entry that was the last one processed. + </para> + </listitem> + <listitem> + <para> + With <replaceable>seconds</replaceable>, you set the length of the rate limit interval. The number of messages processed during this interval can not exceed the value specified in <replaceable>burst_number</replaceable>. The default setting is 20,000 messages per 600 seconds. Rsyslog discards messages that come after the maximum burst within the time frame specified. + </para> + </listitem> + <listitem> + <para> + With <option>$ImjournalIgnorePreviousMessages</option> you can ignore messages that are currently in Journal and import only new messages, which is used when there is no state file specified. The default setting is <literal>off</literal>. Please note that if this setting is off and there is no state file, all messages in the Journal are processed, even if they were already processed in a previous rsyslog session. + </para> + </listitem> + </itemizedlist> + <note> + <para> + You can use <systemitem>imjournal</systemitem> simultaneously with <systemitem>imuxsock</systemitem> module that is the traditional system log input. However, to avoid message duplication, you must prevent <systemitem>imuxsock</systemitem> from reading the Journal's system socket. To do so, use the <option>$OmitLocalLogging</option> directive: + </para> + <screen> +$ModLoad imuxsock +$ModLoad imjournal + +$OmitLocalLogging on +$AddUnixListenSocket /run/systemd/journal/syslog + </screen> + </note> + <para> + You can translate all data and meta data stored by <application>Journal</application> into structured messages. Some of these meta data entries are listed in <xref linkend="ex-verbose_journalctl_output" />, for a complete list of journal fields see the <filename>systemd.journal-fields(7)</filename> manual page. For example, it is possible to focus on <emphasis>kernel journal fields</emphasis>, that are used by messages originating in the kernel. + </para> + </section> + <section id="s2-filtering_structured_messages"> + <title>Filtering Structured Messages</title> + <para> + To create a lumberjack-formatted message that is required by <application>rsyslog</application>'s parsing module, use the following template: + </para> + <screen> +template(name="CEETemplate" type="string" string="%TIMESTAMP% %HOSTNAME% %syslogtag% @cee: %$!all-json%\n") + </screen> + <para> + This template prepends the <literal>@cee:</literal> string to the JSON string and can be applied, for example, when creating an output file with <systemitem>omfile</systemitem> module. To access JSON field names, use the <emphasis role="bold">$!</emphasis> prefix. For example, the following filter condition searches for messages with specific <emphasis>hostname</emphasis> and <emphasis>UID</emphasis> : + </para> + <screen> +($!hostname == "<replaceable>hostname</replaceable>" && $!UID== "<replaceable>UID</replaceable>") + </screen> + </section> + <section id="s2-parsing_JSON"> + <title>Parsing JSON</title> + <para> + The <systemitem>mmjsonparse</systemitem> module is used for parsing structured messages. + + These messages can come from <application>Journal</application> or from other input sources, and must be formatted in a way defined by the <emphasis role="bold">Lumberjack</emphasis> project. These messages are identified by the presence of the <emphasis role="bold">@cee:</emphasis> string. Then, <systemitem>mmjsonparse</systemitem> checks if the JSON structure is valid and then the message is parsed. + </para> + <para> + To parse lumberjack-formatted JSON messages with <systemitem>mmjsonparse</systemitem>, use the following configuration in the <filename>/etc/rsyslog.conf</filename>: + </para> + <screen> +$ModLoad mmjsonparse + +*.* :mmjsonparse: + </screen> + <para> + In this example, the <systemitem>mmjsonparse</systemitem> module is loaded on the first line, then all messages are forwarded to it. Currently, there are no configuration parameters available for <systemitem>mmjsonparse</systemitem>. + </para> + </section> + <section id="s2-storing_messages_in_the_mongodb"> + <title>Storing Messages in the MongoDB</title> + <para> + <application>Rsyslog</application> supports storing JSON logs in the <database class="name">MongoDB</database> document database through the <emphasis>ommongodb</emphasis> output module. + </para> + <para> + To forward log messages into <database class="name">MongoDB</database>, use the following syntax in the <filename>/etc/rsyslog.conf</filename> (configuration parameters for <emphasis>ommongodb</emphasis> are available only in the new configuration format; see <xref linkend="s2-using_the_new_configuration_format" />): </para> + <screen> +$ModLoad ommongodb + +*.* action(type="ommongodb" server="<replaceable>DB_server</replaceable>" serverport="<replaceable>port</replaceable>" db="<replaceable>DB_name</replaceable>" collection="<replaceable>collection_name</replaceable>" uid="<replaceable>UID</replaceable>" pwd="<replaceable>password</replaceable>") + </screen> + <itemizedlist> + <listitem> + <para> + Replace <replaceable>DB_server</replaceable> with the name or address of the MongoDB server. Specify <replaceable>port</replaceable> to select a non-standard port from the MongoDB server. The default <replaceable>port</replaceable> value is <literal>0</literal> and usually there is no need to change this parameter. + </para> + </listitem> + <listitem> + <para> + With <replaceable>DB_name</replaceable>, you identify to which database on the MongoDB server you want to direct the output. Replace <replaceable>collection_name</replaceable> with the name of a collection in this database. In <database class="name">MongoDB</database>, collection is a group of documents, the equivalent of an RDBMS table. + </para> + </listitem> + <listitem> + <para> + You can set your login details by replacing <replaceable>UID</replaceable> and <replaceable>password</replaceable>. + </para> + </listitem> + </itemizedlist> <para> - The following is a list of some of the directives you can specify in your <application>logrotate</application> configuration file: + You can shape the form of the final database output with use of templates. By default, <application>ryslog</application> uses a template based on standard <application>lumberjack</application> field names. </para> - <itemizedlist> - <listitem> + </section> + </section> + <section id="s1-debugging_rsyslog"> + <title>Debugging Rsyslog</title> + <para> + To run <systemitem class="daemon">rsyslogd</systemitem> in debugging mode, use the following command: + </para> + <synopsis><systemitem class="daemon">rsyslogd</systemitem> <option>-dn</option></synopsis> + <para> + With this command, <systemitem class="daemon">rsyslogd</systemitem> produces debugging information and prints it to the standard output. The <option>-n</option> stands for "no fork". You can modify debugging with environmental variables, for example, you can store the debug output in a log file. Before starting <systemitem class="daemon">rsyslogd</systemitem>, type the following on the command line: + </para> + <screen> +export RSYSLOG_DEBUGLOG="<replaceable>path</replaceable>" +export RSYSLOG_DEBUG="Debug" + </screen> <para> - <parameter>weekly</parameter> — Specifies the rotation of log files on a weekly basis. Similar directives include: - <itemizedlist> - <listitem> - <para> - <parameter>daily</parameter> - </para> - </listitem> - <listitem> - <para> - <parameter>monthly</parameter> - </para> - </listitem> - <listitem> - <para> - <parameter>yearly</parameter> - </para> - </listitem> - </itemizedlist> + Replace <replaceable>path</replaceable> with a desired location for the file where the debugging information will be logged. For a complete list of options available for the RSYSLOG_DEBUG variable, see the related section in the <filename>rsyslogd(8)</filename> manual page. </para> - </listitem> - <listitem> <para> - <parameter>compress</parameter> — Enables compression of rotated log files. Similar directives include: - <itemizedlist> - <listitem> - <para> - <parameter>nocompress</parameter> - </para> - </listitem> - <listitem> - <para> - <parameter>compresscmd</parameter> — Specifies the command to be used for compressing. - </para> - </listitem> - <listitem> + To check if syntax used in the <filename>etc/rsyslog.conf</filename> file is valid use: + </para> + <synopsis><systemitem class="daemon">rsyslogd</systemitem> <option>-N</option> <literal>1</literal></synopsis> + <para> + Where <literal>1</literal> represents level of verbosity of the output message. This is a forward compatibility option because currently, only one level is provided. However, you must add this argument to run the validation. + </para> + </section> + <section id="s1-Using_the_Journal"> + <title>Using the Journal</title> + <para> + The Journal is a component of <application>systemd</application> that is responsible for viewing and management of log files. It can be used in parallel, or in place of a traditional syslog daemon, such as <systemitem class="daemon">rsyslogd</systemitem>. The Journal was developed to address problems connected with traditional logging. It is closely integrated with the rest of the system, supports various logging technologies and access management for the log files. + </para> + <para> + Logging data is collected, stored, and processed by the Journal's <systemitem class="daemon">journald</systemitem> service. It creates and maintains binary files called <emphasis>journals</emphasis> based on logging information that is received from the kernel, from user processes, from standard output, and standard error output of system services or via its native API. These journals are structured and indexed, which provides relatively fast seek times. Journal entries can carry a unique identifier. The <systemitem class="daemon">journald</systemitem> service collects numerous meta data fields for each log message. The actual journal files are secured, and therefore cannot be manually edited. + </para> + <section id="s2-Viewing_Log_Files"> + <title>Viewing Log Files</title> + <para> + To access the journal logs, use the <application>journalctl</application> tool. For a basic view of the logs type as <systemitem class="username">root</systemitem>: + </para> + <synopsis><command>journalctl</command></synopsis> + <para> + An output of this command is a list of all log files generated on the system including messages generated by system components and by users. The structure of this output is similar to one used in <filename>/var/log/messages/</filename> but with certain improvements: + </para> + <itemizedlist> + <listitem> <para> - <parameter>uncompresscmd</parameter> + the priority of entries is marked visually. Lines of error priority and higher are highlighted with red color and a bold font is used for lines with notice and warning priority </para> - </listitem> - <listitem> + </listitem> + <listitem> <para> - <parameter>compressext</parameter> — Specifies what extension is to be used for compressing. + the time stamps are converted for the local time zone of your system </para> - </listitem> - <listitem> + </listitem> + <listitem> <para> - <parameter>compressoptions</parameter> — Lets you specify any options that may be passed to the used compression program. + all logged data is shown, including rotated logs </para> - </listitem> - <listitem> + </listitem> + <listitem> <para> - <parameter>delaycompress</parameter> — Postpones the compression of log files to the next rotation of log files. + the beginning of a boot is tagged with a special line </para> - </listitem> - </itemizedlist> - </para> - </listitem> - <listitem> + </listitem> + </itemizedlist> + <example id="ex-Example_Output_of_journalctl"> + <title>Example Output of journalctl</title> + <para> + The following is an example output provided by the <application>journalctl</application> tool. When called without parameters, the listed entries begin with a time stamp, then the host name and application that performed the operation is mentioned followed by the actual message. This example shows the first three entries in the journal log: + </para> + <screen> +# <command>journalctl</command> +-- Logs begin at Thu 2013-08-01 15:42:12 CEST, end at Thu 2013-08-01 15:48:48 CEST. -- +Aug 01 15:42:12 localhost systemd-journal[54]: Allowing runtime journal files to grow to 49.7M. +Aug 01 15:42:12 localhost kernel: Initializing cgroup subsys cpuset +Aug 01 15:42:12 localhost kernel: Initializing cgroup subsys cpu + +[...] + </screen> + </example> + <para> + In many cases, only the latest entries in the journal log are relevant. The simplest way to reduce <command>journalctl</command> output is to use the <option>-n</option> option that lists only the specified number of most recent log entries: + </para> + <synopsis><command>journalctl</command> <option>-n</option> <replaceable>Number</replaceable></synopsis> + <para> + Replace <replaceable>Number</replaceable> with the number of lines to be shown. When no number is specified, <command>journalctl</command> displays the ten most recent entries. + </para> + <para> + The <command>journalctl</command> command allows controlling the form of the output with the following syntax: + </para> + <synopsis><command>journalctl</command> <option>-o</option> <replaceable>form</replaceable></synopsis> + <para> + Replace <replaceable>form</replaceable> with a keyword specifying a desired form of output. There are several options, such as <option>verbose</option>, which returns full-structured entry items with all fields, <option>export</option>, which creates a binary stream suitable for backups and network transfer, and <option>json</option>, which formats entries as JSON data structures. For the full list of keywords, see the <filename>journalctl(1)</filename> manual page. + </para> + <example id="ex-verbose_journalctl_output"> + <title>Verbose journalctl Output</title> + <para> + To view full meta data about all entries, type: + </para> + <screen> +# <command>journalctl</command> <option>-o verbose</option> +[...] + +Fri 2013-08-02 14:41:22 CEST [s=e1021ca1b81e4fc688fad6a3ea21d35b;i=55c;b=78c81449c920439da57da7bd5c56a770;m=27cc + _BOOT_ID=78c81449c920439da57da7bd5c56a770 + PRIORITY=5 + SYSLOG_FACILITY=3 + _TRANSPORT=syslog + _MACHINE_ID=69d27b356a94476da859461d3a3bc6fd + _HOSTNAME=localhost.localdomain + _PID=562 + _COMM=dbus-daemon + _EXE=/usr/bin/dbus-daemon + _CMDLINE=/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation + _SYSTEMD_CGROUP=/system/dbus.service + _SYSTEMD_UNIT=dbus.service + SYSLOG_IDENTIFIER=dbus + SYSLOG_PID=562 + _UID=81 + _GID=81 + _SELINUX_CONTEXT=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 + MESSAGE=[system] Successfully activated service 'net.reactivated.Fprint' + _SOURCE_REALTIME_TIMESTAMP=1375447282839181 + +[...] + </screen> + <para> + This example lists fields that identify a single log entry. These meta data can be used for message filtering as shown in <xref linkend="advanced_filtering"/>. For a complete description of all possible fields see the <filename>systemd.journal-fields(7)</filename> manual page. + </para> + </example> + </section> + <section id="s2-Access_Control"> + <title>Access Control</title> + <para> + By default, <application>Journal</application> users without <systemitem class="username">root</systemitem> privileges can only see log files generated by them. The system administrator can add selected users to the <emphasis>adm</emphasis> group, which grants them access to complete log files. To do so, type as <systemitem class="username">root</systemitem>: + </para> + <synopsis><command>usermod</command> <option>-a</option> <option>-G</option> <emphasis>adm</emphasis> <replaceable>username</replaceable></synopsis> + <para> + Here, replace <replaceable>username</replaceable> with a name of the user to be added to the <emphasis>adm</emphasis> group. This user then receives the same output of the <command>journalctl</command> command as the root user. Note that access control only works when persistent storage is enabled for <application>Journal</application>. + </para> + </section> + <section id="s2-Using_The_Live_View"> + <title>Using The Live View</title> + <para> + When called without parameters, <command>journalctl</command> shows the full list of entries, starting with the oldest entry collected. With the live view, you can supervise the log messages in real time as new entries are continuously printed as they appear. To start <application>journalctl</application> in live view mode, type: + </para> + <synopsis><command>journalctl</command> <option>-f</option></synopsis> + <para> + This command returns a list of the ten most current log lines. The <application>journalctl</application> utility then stays running and waits for new changes to show them immediately. + </para> + </section> + <section id="s2-Filtering_Messages"> + <title>Filtering Messages</title> + <para> + The output of the <command>journalctl</command> command executed without parameters is often extensive, therefore you can use various filtering methods to extract information to meet your needs. + </para> + <bridgehead id="filtering_by_priority">Filtering by Priority</bridgehead> + <para> + Log messages are often used to track erroneous behavior on the system. To view only entries with a selected or higher priority, use the following syntax: + </para> + <synopsis><command>journalctl</command> <option>-p</option> <replaceable>priority</replaceable></synopsis> + <para> + Here, replace <replaceable>priority</replaceable> with one of the following keywords (or with a number): <command>debug</command> (7), <command>info</command> (6), <command>notice</command> (5), <command>warning</command> (4), <command>err</command> (3), <command>crit</command> (2), <command>alert</command> (1), and <command>emerg</command> (0). + </para> + <example id="ex-filtering_by_priority"> + <title>Filtering by Priority</title> <para> - <parameter>rotate <replaceable><INTEGER></replaceable> - </parameter> — Specifies the number of rotations a log file undergoes before it is removed or mailed to a specific address. If the value <constant>0</constant> is specified, old log files are removed instead of rotated. + To view only entries with <emphasis>error</emphasis> or higher priority, use: </para> - </listitem> - <listitem> + <synopsis><command>journalctl</command> <option>-p err</option></synopsis> + </example> + <bridgehead id="filtering_by_time">Filtering by Time</bridgehead> + <para> + To view log entries only from the current boot, type: + </para> + <synopsis><command>journalctl</command> <option>-b</option></synopsis> + <para> + If you reboot your system just occasionally, the <option>-b</option> will not significantly reduce the output of <command>journalctl</command>. In such cases, time-based filtering is more helpful: + </para> + <synopsis><command>journalctl</command> <option>--since</option>=<replaceable>value</replaceable> <option>--until</option>=<replaceable>value</replaceable></synopsis> <para> - <parameter>mail <replaceable><ADDRESS></replaceable> - </parameter> — This option enables mailing of log files that have been rotated as many times as is defined by the <parameter>rotate</parameter> directive to the specified address. Similar directives include: + With <option>--since</option> and <option>--until</option>, you can view only log messages created within a specified time range. You can pass <replaceable>values</replaceable> to these options in form of date or time or both as shown in the following example. </para> - <itemizedlist> - <listitem> - <para> - <parameter>nomail</parameter> - </para> - </listitem> - <listitem> - <para> - <parameter>mailfirst</parameter> — Specifies that the just-rotated log files are to be mailed, instead of the about-to-expire log files. - </para> - </listitem> - <listitem> - <para> - <parameter>maillast</parameter> — Specifies that the just-rotated log files are to be mailed, instead of the about-to-expire log files. This is the default option when <parameter>mail</parameter> is enabled. - </para> - </listitem> - </itemizedlist> - </listitem> - </itemizedlist> + <example id="ex-filtering_by_time_and_priority"> + <title>Filtering by Time and Priority</title> + <para> + Filtering options can be combined to reduce the set of results according to specific requests. For example, to view the <emphasis>warning</emphasis> or higher priority messages from a certain point in time, type: + </para> + <synopsis><command>journalctl</command> <option>-p warning</option> <option>--since="2013-3-16 23:59:59"</option></synopsis> + </example> + <bridgehead id="advanced_filtering">Advanced Filtering</bridgehead> + <para> + <xref linkend="ex-verbose_journalctl_output"/> lists a set of fields that specify a log entry and can all be used for filtering. For a complete description of meta data that <systemitem>systemd</systemitem> can store, see the <filename>systemd.journal-fields(7)</filename> manual page. This meta data is collected for each log message, without user intervention. Values are usually text-based, but can take binary and large values; fields can have multiple values assigned though it is not very common. + </para> + <para> + To view a list of unique values that occur in a specified field, use the following syntax: + </para> + <synopsis><command>journalctl</command> <option>-F</option> <replaceable>fieldname</replaceable></synopsis> + <para> + Replace <replaceable>fieldname</replaceable> with a name of a field you are interested in. + </para> + <para> + To show only log entries that fit a specific condition, use the following syntax: + </para> + <synopsis><command>journalctl</command> <replaceable>fieldname</replaceable>=<replaceable>value</replaceable></synopsis> + <para> + Replace <replaceable>fieldname</replaceable> with a name of a field and <replaceable>value</replaceable> with a specific value contained in that field. As a result, only lines that match this condition are returned. + </para> + <note> + <title><keycap>Tab</keycap> Completion on Field Names</title> + <para> + As the number of meta data fields stored by <systemitem>systemd</systemitem> is quite large, it is easy to forget the exact name of the field of interest. When unsure, type: + </para> + <synopsis><command>journalctl</command></synopsis> + <para> + and press the <keycap>Tab</keycap> key two times. This shows a list of available field names. <keycap>Tab</keycap> completion based on context works on field names, so you can type a distinctive set of letters from a field name and then press <keycap>Tab</keycap> to complete the name automatically. Similarly, you can list unique values from a field. Type: + </para> + <synopsis><command>journalctl</command> <replaceable>fieldname</replaceable>=</synopsis> + <para> + and press <keycap>Tab</keycap> two times. This serves as an alternative to <command>journalctl</command> <option>-F</option> <replaceable>fieldname</replaceable>. + </para> + </note> + <para> + You can specify multiple values for one field: + </para> + <synopsis><command>journalctl</command> <replaceable>fieldname</replaceable>=<replaceable>value1</replaceable> <replaceable>fieldname</replaceable>=<replaceable>value2</replaceable> ...</synopsis> + <para> + Specifying two matches for the same field results in a logical <function>OR</function> combination of the matches. Entries matching <replaceable>value1</replaceable> or <replaceable>value2</replaceable> are displayed. + </para> + <para> + Also, you can specify multiple field-value pairs to further reduce the output set: + </para> + <synopsis><command>journalctl</command> <replaceable>fieldname1</replaceable>=<replaceable>value</replaceable> <replaceable>fieldname2</replaceable>=<replaceable>value</replaceable> ...</synopsis> + <para> + If two matches for different field names are specified, they will be combined with a logical <function>AND</function>. Entries have to match both conditions to be shown. + </para> + <para> + With use of the <emphasis>+</emphasis> symbol, you can set a logical <function>OR</function> combination of matches for multiple fields: + </para> + <synopsis><command>journalctl</command> <replaceable>fieldname1</replaceable>=<replaceable>value</replaceable> + <replaceable>fieldname2</replaceable>=<replaceable>value</replaceable> ...</synopsis> + <para> + This command returns entries that match at least one of the conditions, not only those that match both of them. + </para> + <example id="ex-advanced_filtering"> + <title>Advanced filtering</title> + <para> + To display entries created by <systemitem class="daemon">avahi-daemon.service</systemitem> or <systemitem class="daemon">crond.service</systemitem> under user with UID 70, use the following command: + </para> + <synopsis><command>journalctl</command> <literal>_UID=70</literal> <literal>_SYSTEMD_UNIT=avahi-daemon.service</literal> <literal>_SYSTEMD_UNIT=crond.service</literal></synopsis> + <para> + Since there are two values set for the <literal>_SYSTEMD_UNIT</literal> field, both results will be displayed, but only when matching the <literal>_UID=70</literal> condition. This can be expressed simply as: (UID=70 and (avahi or cron)). + </para> + </example> + <para> + You can apply the aforementioned filtering also in the live-view mode to keep track of the latest changes in the selected group of log entries: + </para> + <synopsis><command>journalctl</command> <option>-f</option> <replaceable>fieldname</replaceable>=<replaceable>value</replaceable> ...</synopsis> +  + </section> + <section id="s2-Enabling_Persistent_Storage"> + <title>Enabling Persistent Storage</title> + <para>By default, <application>Journal</application> stores log files only in memory or a small ring-buffer in the <filename class="directory">/run/log/journal/</filename> directory. This is sufficient to show recent log history with <command>journalctl</command>. This directory is volatile, log data is not saved permanently. With the default configuration, syslog reads the journal logs and stores them in the <filename class="directory">/var/log/</filename> directory. With persistent logging enabled, journal files are stored in <filename>/var/log/journal</filename> which means they persist after reboot. Journal can then replace <application>rsyslog</application> for some users (but see the chapter introduction). + </para> + <para> + Enabled persistent storage has the following advantages + </para> + <itemizedlist> + <listitem> + <para> + Richer data is recorded for troubleshooting in a longer period of time + </para> + </listitem> + <listitem> + <para> + For immediate troubleshooting, richer data is available after a reboot + </para> + </listitem> + <listitem> + <para> + Server console currently reads data from journal, not log files + </para> + </listitem> + </itemizedlist> + <para> + Persistent storage has also certain disadvantages: + </para> + <itemizedlist > + <listitem> + <para> + Even with persistent storage the amount of data stored depends on free memory, there is no guarantee to cover a specific time span + </para> + </listitem> + <listitem> + <para> + More disk space is needed for logs + </para> + </listitem> + </itemizedlist> + <para> + To enable persistent storage for Journal, create the journal directory manually as shown in the following example. As <systemitem class="username">root</systemitem> type: + </para> + <synopsis><command>mkdir</command> <option>-p</option> <filename class="directory">/var/log/journal</filename></synopsis> + <para> + Then, restart <systemitem class="daemon">journald</systemitem> to apply the change: + </para> + <synopsis><command>systemctl</command> <option>restart</option> <systemitem class="daemon">systemd-journald</systemitem></synopsis> + </section> +  </section> - </section> - <section id="s1-logfiles-viewing"> + <section id="s1-managing_log_files_in_a_graphical_environment"> + <title>Managing Log Files in a Graphical Environment</title> + <para> + As an alternative to the aforementioned command-line utilities, Red Hat Enterprise Linux 7 provides an accessible GUI for managing log messages. + </para> + <section id="s2-logfiles-viewing"> <title>Viewing Log Files</title> - <indexterm significance="normal"> + <indexterm> <primary>log files</primary> <secondary>viewing</secondary> </indexterm> - <para>Most log files are in plain text format. You can view them with any text editor such as <command>Vi</command> or <application>Emacs</application>. Some log files are readable by all users on the system; however, <systemitem class="username">root</systemitem> privileges are required to read most log files.</para> - <indexterm significance="normal"> - <primary> - <command>gnome-system-log</command> - </primary> - <see> - <application>Log File Viewer</application> - </see> + <para>Most log files are stored in plain text format. You can view them with any text editor such as <command>Vi</command> or <application>Emacs</application>. Some log files are readable by all users on the system; however, root privileges are required to read most log files.</para> + <indexterm> + <primary><command>gnome-system-log</command></primary> + <see><application>System Log</application></see> </indexterm> - <para>To view system log files in an interactive, real-time application, use the <application>Log File Viewer</application>. + <para>To view system log files in an interactive, real-time application, use the <application>System Log</application>. </para> <note> <title>Installing the gnome-system-log package</title> - <para>In order to use the <application>Log File Viewer</application>, first ensure the <package>gnome-system-log</package> package is installed on your system by running, as <systemitem class="username">root</systemitem>:</para> - <screen><command>yum install gnome-system-log</command></screen> - <para>For more information on installing packages with Yum, refer to <xref linkend="sec-Installing"/>.</para> + <para>In order to use the <application>System Log</application>, first ensure the <package>gnome-system-log</package> package is installed on your system by running, as <systemitem class="username">root</systemitem>:</para> + <screen>~]# <command>yum install gnome-system-log</command> + </screen> + <para>For more information on installing packages with Yum, see <xref linkend="sec-Installing"/>.</para> </note> - <para>After you have installed the <package>gnome-system-log</package> package, you can open the <application>Log File Viewer</application> by selecting <menuchoice><guimenu>Applications</guimenu><guisubmenu>System Tools</guisubmenu><guimenuitem>Log File Viewer</guimenuitem></menuchoice> from the <guimenu>Activities</guimenu> menu, or type the following command at a shell prompt:</para> - <screen><command>gnome-system-log</command></screen> + <para>After you have installed the <package>gnome-system-log</package> package, open the <application>System Log</application> by clicking <menuchoice><guimenu>Applications</guimenu><guimenuitem>System Tools</guimenuitem><guimenuitem>System Log</guimenuitem></menuchoice>, or type the following command at a shell prompt:</para> + <screen>~]$ <command>gnome-system-log</command> + </screen> <para>The application only displays log files that exist; thus, the list might differ from the one shown in <xref linkend="fig-redhat-logviewer"/>.</para> - <indexterm significance="normal"> - <primary> - <application>Log File Viewer</application> - </primary> + <indexterm> + <primary><application>System Log</application></primary> <secondary>searching</secondary> </indexterm> - <indexterm significance="normal"> - <primary> - <application>Log File Viewer</application> - </primary> + <indexterm> + <primary><application>System Log</application></primary> <secondary>filtering</secondary> </indexterm> <figure id="fig-redhat-logviewer"> - <title> - Log File Viewer - </title> + <title>System Log</title> <mediaobject> <imageobject> - <imagedata fileref="images/system-log-viewer.png" format="PNG" scalefit="0" /> + <imagedata fileref="images/redhat-logviewer.png" format="PNG"/> </imageobject> <textobject> - <para>Log File Viewer</para> + <para>System Log</para> </textobject> </mediaobject> </figure> - <indexterm significance="normal"> - <primary> - <application>Log Viewer</application> - </primary> + <indexterm> + <primary><application>System Log</application></primary> <secondary>refresh rate</secondary> </indexterm> - <para>The <application>Log File Viewer</application> application lets you filter any existing log file. Click on <guimenuitem>Filters</guimenuitem> from the menu and select <guimenuitem>Manage Filters</guimenuitem> to define or edit your desired filter.</para> + <para>The <application>System Log</application> application lets you filter any existing log file. Click on the button marked with the gear symbol to view the menu, select <menuchoice><guimenuitem>Filters</guimenuitem> <guimenuitem>Manage Filters</guimenuitem></menuchoice> to define or edit the desired filter.</para> <figure id="fig-redhat-filters"> - <title> - Log File Viewer — filters - </title> + <title>System Log - Filters</title> <mediaobject> <imageobject> - <imagedata fileref="images/system-log-viewer-filters-manage.png" format="PNG" scalefit="0" /> + <imagedata fileref="images/redhat-filters.png" format="PNG"/> </imageobject> <textobject> - <para>Log File Viewer — filters</para> + <para>System Log - Filters</para> </textobject> </mediaobject> </figure> <para>Adding or editing a filter lets you define its parameters as is shown in <xref linkend="fig-redhat-filter-sample"/>.</para> <figure id="fig-redhat-filter-sample"> - <title> - Log File Viewer — defining a filter - </title> + <title>System Log - defining a filter</title> <mediaobject> <imageobject> - <imagedata fileref="images/system-log-viewer-filters-define.png" format="PNG" scalefit="0" /> + <imagedata fileref="images/redhat-filter-sample.png" format="PNG" /> </imageobject> <textobject> - <para>Log File Viewer — defining a filter</para> + <para>System Log - Defining a Filter</para> </textobject> </mediaobject> </figure> <para> - When defining a filter, you can edit the following parameters: + When defining a filter, the following parameters can be edited: </para> <itemizedlist> <listitem> @@ -1068,38 +2399,36 @@ compress </listitem> </itemizedlist> <para> - When you have at least one filter defined, you may select it from the <guilabel>Filters</guilabel> menu and it will automatically search for the strings you have defined in the filter and highlight/hide every successful match in the log file you are currently viewing. + When you have at least one filter defined, it can be selected from the <guilabel>Filters</guilabel> menu and it will automatically search for the strings you have defined in the filter and highlight or hide every successful match in the log file you are currently viewing. </para> <figure id="fig-redhat-filter-enable"> - <title> - Log File Viewer — enabling a filter - </title> + <title>System Log - enabled filter</title> <mediaobject> <imageobject> - <imagedata fileref="images/system-log-viewer-filters-enable.png" format="PNG" scalefit="0" /> + <imagedata fileref="images/redhat-filter-enable.png" format="PNG"/> </imageobject> <textobject> - <para>Log File Viewer — enabling a filter</para> + <para>System Log - Enabling a Filter</para> </textobject> </mediaobject> </figure> <para> - When you check the <guilabel>Show matches only</guilabel> option, only the matched strings will be shown in the log file you are currently viewing. + When you select the <guilabel>Show matches only</guilabel> option, only the matched strings will be shown in the log file you are currently viewing. </para> </section> - <section id="s1-logfiles-adding"> + <section id="s2-logfiles-adding"> <title>Adding a Log File</title> - <para>To add a log file you wish to view in the list, select <menuchoice><guimenu>File</guimenu> + <para>To add a log file you want to view in the list, select <menuchoice><guimenu>File</guimenu> <guimenuitem>Open</guimenuitem> - </menuchoice>. This will display the <guilabel>Open Log</guilabel> window where you can select the directory and file name of the log file you wish to view.<xref linkend="fig-redhat-logviewer-add"/> illustrates the <guimenu>Open Log</guimenu> window.</para> + </menuchoice>. This will display the <guilabel>Open Log</guilabel> window where you can select the directory and file name of the log file you want to view. <xref linkend="fig-redhat-logviewer-add"/> illustrates the <guimenu>Open Log</guimenu> window.</para> <figure id="fig-redhat-logviewer-add"> - <title>Log File Viewer — adding a log file</title> + <title>System Log - adding a log file</title> <mediaobject> <imageobject> - <imagedata fileref="images/system-log-viewer-add.png" format="PNG" scalefit="0" /> + <imagedata fileref="images/redhat-logviewer-add.png" format="PNG" /> </imageobject> <textobject> - <para>Log File Viewer — adding a log file</para> + <para>System Log - Adding a Log File</para> </textobject> </mediaobject> </figure> @@ -1107,86 +2436,119 @@ compress <note> <title>Reading zipped log files</title> <para> - The <application>Log File Viewer</application> also allows you to open log files zipped in the <literal>.gz</literal> format. + The <application>System Log</application> also allows you to open log files zipped in the <literal>.gz</literal> format. </para> </note> </section> - <section id="s1-logfiles-examining"> + <section id="s2-logfiles-examining"> <title>Monitoring Log Files</title> - <indexterm significance="normal"> + <indexterm> <primary>log files</primary> <secondary>monitoring</secondary> </indexterm> - <indexterm significance="normal"> - <primary> - <application>Log File Viewer</application> - </primary> + <indexterm> + <primary><application>System Log</application></primary> <secondary>monitoring</secondary> </indexterm> <para> - <application>Log File Viewer</application> monitors all opened logs by default. If a new line is added to a monitored log file, the log name appears in bold in the log list. If the log file is selected or displayed, the new lines appear in bold at the bottom of the log file. <xref linkend="fig-redhat-logviewer-monitoring"/> illustrates a new alert in the <guilabel>yum.log</guilabel> log file and in the <guilabel>messages</guilabel> log file. Clicking on the <guimenuitem>messages</guimenuitem> log file displays the logs in the file with the new lines in bold.</para> + <application>System Log</application> monitors all opened logs by default. If a new line is added to a monitored log file, the log name appears in bold in the log list. If the log file is selected or displayed, the new lines appear in bold at the bottom of the log file. <xref linkend="fig-redhat-logviewer-monitoring"/> illustrates a new alert in the <guilabel>cron</guilabel> log file and in the <guilabel>messages</guilabel> log file. Clicking on the <guimenuitem>messages</guimenuitem> log file displays the logs in the file with the new lines in bold.</para> <figure id="fig-redhat-logviewer-monitoring"> - <title>Log File Viewer — new log alert</title> + <title>System Log - new log alert</title> <mediaobject> <imageobject> - <imagedata fileref="images/system-log-viewer-alerts.png" format="PNG" scalefit="0" /> + <imagedata fileref="images/redhat-logviewer-monitoring.png" format="PNG" /> </imageobject> <textobject> - <para>Log File Viewer — new log alert</para> + <para>System Log - New Log Alert</para> </textobject> </mediaobject> </figure> + </section> </section> <section id="s1-log-files-additional-resources"> <title>Additional Resources</title> - <para>To learn more about <application>rsyslog</application>, <application>logrotate</application>, and log files in general, refer to the following resources.</para> - <section id="s2-log-files-installed-docs"> - <title>Installed Documentation</title> - <indexterm significance="normal"> - <primary>log files</primary> - <secondary>additional resources</secondary> - <tertiary>installed documentation</tertiary> - </indexterm> - <itemizedlist> - <listitem> - <para> - <command>rsyslogd</command> manual page — Type <command>man rsyslogd</command> to learn more about <command>rsyslogd</command> and its many options.</para> - </listitem> - <listitem> - <para> - <command>rsyslog.conf</command> manual page — Type <command>man rsyslog.conf</command> to learn more about the <command>/etc/rsyslog.conf</command> configuration file and its many options.</para> - </listitem> - <listitem> - <para> - <filename>/usr/share/doc/rsyslog/ - </filename> — After installing the <package>rsyslog</package> package, this directory contains extensive documentation in the <systemitem class="protocol">html</systemitem> format.</para> - </listitem> - <listitem> - <para> - <command>logrotate</command> manual page — Type <command>man logrotate</command> to learn more about <command>logrotate</command> and its many options.</para> - </listitem> - </itemizedlist> - </section> - <section id="s2-log-files-useful-websites"> - <title>Useful Websites</title> - <indexterm significance="normal"> - <primary>log files</primary> - <secondary>additional resources</secondary> - <tertiary>useful websites</tertiary> - </indexterm> - <itemizedlist> - <listitem> - <para> - <ulink url="http://www.rsyslog.com/">http://www.rsyslog.com/</ulink> — Offers a thorough technical breakdown of <package>rsyslog</package> features, documentation, configuration examples, and video tutorials.</para> - </listitem> - <listitem> + <para> + For more information on how to configure the <systemitem class="service">rsyslog</systemitem> daemon and how to locate, view, and monitor log files, see the resources listed below. + </para> + <bridgehead id="brid-Log_Files-Resources-Installed">Installed Documentation</bridgehead> + <itemizedlist> + <listitem> + <para> + <systemitem>rsyslogd</systemitem>(8) — The manual page for the <systemitem class="service">rsyslogd</systemitem> daemon documents its usage. + </para> + </listitem> + <listitem> + <para> + <systemitem>rsyslog.conf</systemitem>(5) — The manual page named <systemitem>rsyslog.conf</systemitem> documents available configuration options. + </para> + </listitem> + <listitem> + <para> + <systemitem>logrotate</systemitem>(8) — The manual page for the <application>logrotate</application> utility explains in greater detail how to configure and use it. + </para> + </listitem> + <listitem> + <para> + <systemitem>journalctl</systemitem>(1) — The manual page for the <command>journalctl</command> daemon documents its usage. + </para> + </listitem> + <listitem> + <para> + <systemitem>journald.conf</systemitem>(5) — This manual page documents available configuration options. + </para> + </listitem> + <listitem> + <para> + <systemitem>systemd.journal-fields</systemitem>(7) — This manual page lists special <application>Journal</application> fields. + </para> + </listitem> + </itemizedlist> + <bridgehead id="brid-Log_Files-Resources-Online">Online Documentation</bridgehead> + <itemizedlist> + <listitem> + <para> + <ulink url="http://www.rsyslog.com/">rsyslog Home Page</ulink> — The <application>rsyslog</application> home page offers a thorough technical breakdown of its features, documentation, configuration examples, and video tutorials. + </para> + </listitem> + <listitem> + <para> + <ulink url="http://www.rsyslog.com/doc/rainerscript.html"><emphasis>RainerScript</emphasis> documentation on the rsyslog Home Page</ulink> — Commented summary of data types, expressions, and functions available in <emphasis>RainerScript</emphasis>. + </para> + </listitem> + <listitem> <para> - <ulink url="http://wiki.rsyslog.com/index.php/Main_Page">http://wiki.rsyslog.com/index.php/Main_Page</ulink> — Contains useful <filename>/etc/rsyslog.conf</filename> configuration examples.</para> - </listitem> - </itemizedlist> + <ulink url="http://www.rsyslog.com/doc/queues.html">Description of <emphasis>queues</emphasis> on the rsyslog Home Page</ulink> — General information on various types of message queues and their usage. + </para> + </listitem> + <listitem> + <para> + <ulink url="http://wiki.rsyslog.com/index.php/Main_Page">rsyslog Wiki</ulink> — <citetitle pubwork="webpage">The rsyslog Wiki</citetitle> contains useful configuration examples. + </para> + </listitem> +  + + </itemizedlist> +  </section> - </section> </chapter> - - - diff --git a/en-US/images/redhat-filter-enable.png b/en-US/images/redhat-filter-enable.png new file mode 100644 index 0000000..e0e4051 Binary files /dev/null and b/en-US/images/redhat-filter-enable.png differ diff --git a/en-US/images/redhat-filter-sample.png b/en-US/images/redhat-filter-sample.png new file mode 100644 index 0000000..66ce188 Binary files /dev/null and b/en-US/images/redhat-filter-sample.png differ diff --git a/en-US/images/redhat-filters.png b/en-US/images/redhat-filters.png new file mode 100644 index 0000000..40dd5f8 Binary files /dev/null and b/en-US/images/redhat-filters.png differ diff --git a/en-US/images/redhat-logviewer-add.png b/en-US/images/redhat-logviewer-add.png new file mode 100644 index 0000000..f2cf04d Binary files /dev/null and b/en-US/images/redhat-logviewer-add.png differ diff --git a/en-US/images/redhat-logviewer-monitoring.png b/en-US/images/redhat-logviewer-monitoring.png new file mode 100644 index 0000000..5f66789 Binary files /dev/null and b/en-US/images/redhat-logviewer-monitoring.png differ diff --git a/en-US/images/redhat-logviewer.png b/en-US/images/redhat-logviewer.png new file mode 100644 index 0000000..365d52c Binary files /dev/null and b/en-US/images/redhat-logviewer.png differ diff --git a/en-US/images/rsyslog_message_flow.png b/en-US/images/rsyslog_message_flow.png new file mode 100644 index 0000000..7503f4a Binary files /dev/null and b/en-US/images/rsyslog_message_flow.png differ

9 years, 5 months

1
0
0 / 0

[system-administrators-guide] Applying improvements to RPM

by stephenw

commit 6beb6918027fd0bb7bcb3dbfd04c7b7da1130e28 Author: Stephen Wadeley <swadeley(a)redhat.com> Date: Sun Dec 14 10:49:05 2014 +0100 Applying improvements to RPM from upstream version, after proofreading updated most package names to suit Fed21 en-US/RPM.xml | 1105 +++++++++++++++++++++++++++++---------------------------- 1 files changed, 563 insertions(+), 542 deletions(-) --- diff --git a/en-US/RPM.xml b/en-US/RPM.xml index 482f897..35fb303 100644 --- a/en-US/RPM.xml +++ b/en-US/RPM.xml @@ -11,33 +11,45 @@ <indexterm> <primary>RPM</primary> </indexterm> - <para>The <firstterm>RPM Package Manager</firstterm> (RPM) is an open packaging system<indexterm><primary>packages</primary> - <secondary>RPM</secondary> - </indexterm>, which runs on &MAJOROS; as well as other Linux and UNIX systems. Red Hat, Inc. and the Fedora Project encourage other vendors to use RPM for their own products. RPM is distributed under the terms of the <firstterm>GPL</firstterm> (<firstterm>GNU General Public License</firstterm>).</para> - <para>The RPM Package Manager only works with packages built to work with the <emphasis>RPM format</emphasis>. RPM is itself provided as a pre-installed <package>rpm</package> package. For the end user, RPM makes system updates easy. Installing, uninstalling and upgrading RPM packages can be accomplished with short commands. RPM maintains a database of installed packages and their files, so you can invoke powerful queries and verifications on your system.</para> - <para>The RPM package format has been improved for &MAJOROSVER;. RPM packages are now compressed using the XZ lossless data compression format, which has the benefit of greater compression and less CPU usage during decompression, and support multiple strong hash algorithms, such as SHA-256, for package signing and verification.</para> - <warning - id="warning-Use_Yum_Instead_of_RPM_Whenever_Possible"> + <indexterm> + <primary>packages</primary> + <secondary>RPM</secondary> + </indexterm> + <para> + The <firstterm>RPM Package Manager</firstterm> (<application>RPM</application>) is an open packaging system that runs on &MAJOROS; as well as other Linux and UNIX systems. Red Hat and the Fedora Project encourage other vendors to use <application>RPM</application> for their own products. <application>RPM</application> is distributed under the terms of the <firstterm>GPL</firstterm> (<firstterm>GNU General Public License</firstterm>). + </para> + <para> + The <application>RPM Package Manager</application> only works with packages built in the <emphasis>RPM format</emphasis>. <application>RPM</application> itself is provided as the pre-installed <package>rpm</package> package. For the end user, <application>RPM</application> makes system updates easy. Installing, uninstalling, and upgrading <application>RPM</application> packages can be accomplished with short commands. <application>RPM</application> maintains a database of installed packages and their files, so you can make queries and verify installed files on your system. There are several applications, such as <application>Yum</application> or <application>PackageKit</application>, that can make working with packages in the <application>RPM</application> format even easier. + </para> + <warning id="warning-Use_Yum_Instead_of_RPM_Whenever_Possible"> <title>Use Yum Instead of RPM Whenever Possible</title> - <para>For most package management tasks, the <application>Yum</application> package manager offers equal and often greater capabilities and utility than RPM<indexterm><primary>packages</primary> - <secondary>Yum instead of RPM</secondary> - </indexterm>. <application>Yum</application> also performs and tracks complicated system dependency resolution, and will complain and force system integrity checks if you use RPM as well to install and remove packages. For these reasons, it is highly recommended that you use <application>Yum</application> instead of RPM whenever possible to perform package management tasks. See <xref - linkend="ch-yum"/>.</para> - <para>If you prefer a graphical interface, you can use the <application>PackageKit</application> GUI application, which uses <application>Yum</application> as its back end, to manage your system's packages.</para> + <indexterm> + <primary>packages</primary> + <secondary>Yum instead of RPM</secondary> + </indexterm> + <para> + For most package-management tasks, the <application>Yum</application> package manager offers equal and often greater capabilities and utility than <application>RPM</application>. <application>Yum</application> also performs and tracks complicated system-dependency resolutions. <application>Yum</application> maintains the system integrity and forces a system integrity check if packages are installed or removed using another application, such as <application>RPM</application>, instead of <application>Yum</application>. For these reasons, it is highly recommended that you use <application>Yum</application> instead of <application>RPM</application> whenever possible to perform package-management tasks. See <xref linkend="ch-yum"/>. + </para> + <para> + If you prefer a graphical interface, you can use the <application>PackageKit</application> GUI application, which uses <application>Yum</application> as its back end, to manage your system's packages. + </para> </warning> - <important> - <title>Install RPM packages with the correct architecture!</title> - <para>When installing a package, ensure it is compatible with your operating system and processor architecture. This can usually be determined by checking the package name. Many of the following examples show RPM packages compiled for the AMD64/Intel 64 computer architectures; thus, the RPM file name ends in <filename>x86_64.rpm</filename>.</para> - </important> - <para>During upgrades, RPM handles configuration files carefully, so that you never lose your customizations—something that you cannot accomplish with regular <filename>.tar.gz</filename> files.</para> - <para>For the developer, RPM allows you to take software source code and package it into source and binary packages for end users.<indexterm><primary>packages</primary> - <secondary>RPM</secondary> - <tertiary>source and - binary packages</tertiary> - </indexterm> This process is quite simple and is driven from a single file and optional patches that you create. This clear delineation between <firstterm>pristine</firstterm> sources and your patches along with build instructions eases the maintenance of the package as new versions of the software are released.</para> - <note> - <title>Running rpm commands must be performed as root</title> - <para>Because RPM makes changes to your system, you must be logged in as root to install, remove, or upgrade an RPM package.</para> + <para> + During upgrades, <application>RPM</application> handles configuration files carefully, so that you never lose your customizations — something that you cannot accomplish with regular <filename>.tar.gz</filename> files. + </para> + <indexterm> + <primary>packages</primary> + <secondary>RPM</secondary> + <tertiary>source and binary packages</tertiary> + </indexterm> + <para> + For the developer, <application>RPM</application> enables software source code to be packaged into source and binary packages for end users. This process is quite simple and is driven from a single file and optional patches that you create. This clear delineation between pristine sources and your patches along with build instructions eases the maintenance of the package as new versions of the software are released. + </para> + <note id="note-Root_Permissions"> + <title>Note</title> + <para> + Because <application>RPM</application> can make changes to the system itself, performing operations like installing, upgrading, downgrading, and uninstalling binary packages system-wide requires <systemitem class="username">root</systemitem> privileges in most cases. + </para> </note> <section id="s1-rpm-design"> @@ -46,47 +58,71 @@ <primary>RPM</primary> <secondary>design goals</secondary> </indexterm> - <para>To understand how to use RPM, it can be helpful to understand the design goals of RPM:</para> + <para> + To understand how to use <application>RPM</application>, it is helpful to understand the design goals of <application>RPM</application>: + </para> <variablelist> <varlistentry> - <term>Upgradability<indexterm><primary>RPM</primary> + <term> + <indexterm> + <primary>RPM</primary> <secondary>design goals</secondary> <tertiary>upgradability</tertiary> </indexterm> + Upgradability </term> <listitem> - <para>With RPM, you can upgrade individual components of your system without completely reinstalling. When you get a new release of an operating system based on RPM, such as &MAJOROS;, you do not need to reinstall a fresh copy of the operating system your machine (as you might need to with operating systems based on other packaging systems). RPM allows intelligent, fully-automated, in-place upgrades of your system. In addition, configuration files in packages are preserved across upgrades, so you do not lose your customizations. There are no special upgrade files needed to upgrade a package because the same RPM file is used to both install and upgrade the package on your system.</para> + <para> + With <application>RPM</application>, you can upgrade individual components of your system without a complete reinstallation. When you get a new release of an operating system based on <application>RPM</application>, such as &MAJOROS;, you do not need to reinstall a fresh copy of the operating system on your machine (as you might need to with operating systems based on other packaging systems). <application>RPM</application> allows for intelligent, fully-automated, in-place upgrades of your system. In addition, configuration files in packages are preserved across upgrades, so you do not lose your customizations. There are no special upgrade files needed to upgrade a package because the same <application>RPM</application> file is used to both install and upgrade the package on the system. + </para> </listitem> </varlistentry> <varlistentry> - <term>Powerful Querying<indexterm><primary>RPM</primary> + <term> + <indexterm> + <primary>RPM</primary> <secondary>design goals</secondary> <tertiary>powerful querying</tertiary> </indexterm> + Powerful Querying </term> <listitem> - <para>RPM is designed to provide powerful querying options. You can perform searches on your entire database for packages or even just certain files. You can also easily find out what package a file belongs to and from where the package came. The files an RPM package contains are in a compressed archive, with a custom binary header containing useful information about the package and its contents, allowing you to query individual packages quickly and easily.</para> + <para> + <application>RPM</application> is designed to provide powerful querying options. You can perform searches on your copy of the database for packages or even just certain files. You can also easily find out what package a file belongs to and where the package came from. The files an <application>RPM</application> package contains are in a compressed archive, with a custom binary header containing useful information about the package and its contents, allowing you to query individual packages quickly and easily. + </para> </listitem> </varlistentry> <varlistentry> - <term> System Verification<indexterm><primary>RPM</primary> + <term> + <indexterm> + <primary>RPM</primary> <secondary>design goals</secondary> <tertiary>system verification</tertiary> </indexterm> + System Verification </term> <listitem> - <para>Another powerful RPM feature is the ability to verify packages. If you are worried that you deleted an important file for some package, you can verify the package. You are then notified of anomalies, if any—at which point you can reinstall the package, if necessary. Any configuration files that you modified are preserved during reinstallation.</para> + <para> + Another powerful <application>RPM</application> feature is the ability to verify packages. It allows you to verify that the files installed on the system are the same as the ones supplied by a given package. If an inconsistency is detected, <application>RPM</application> notifies you, and you can reinstall the package if necessary. Any configuration files that you modified are preserved during reinstallation. + </para> </listitem> </varlistentry> <varlistentry> - <term>Pristine Sources <indexterm><primary>packages</primary> + <term> + <indexterm> + <primary>packages</primary> <secondary>RPM</secondary> <tertiary>pristine sources</tertiary> </indexterm> + Pristine Sources </term> <listitem> - <para>A crucial design goal was to allow the use of <emphasis>pristine </emphasis> software sources, as distributed by the original authors of the software. With RPM, you have the pristine sources along with any patches that were used, plus complete build instructions. This is an important advantage for several reasons. For instance, if a new version of a program is released, you do not necessarily have to start from scratch to get it to compile. You can look at the patch to see what you <emphasis>might</emphasis> need to do. All the compiled-in defaults, and all of the changes that were made to get the software to build properly, are easily visible using this technique.</para> - <para>The goal of keeping sources pristine may seem important only for developers, but it results in higher quality software for end users, too.</para> + <para> + A crucial design goal was to allow the use of <emphasis>pristine</emphasis> software sources, as distributed by the original authors of the software. With <application>RPM</application>, you have the pristine sources along with any patches that were used, plus complete build instructions. This is an important advantage for several reasons. For instance, if a new version of a program is released, you do not necessarily have to start from scratch to get it to compile. You can look at the patch to see what you <emphasis>might</emphasis> need to do. All the compiled-in defaults, and all of the changes that were made to get the software to build properly, are easily visible using this technique. + </para> + <para> + The goal of keeping sources pristine may seem important only for developers, but it results in higher quality software for end users. + </para> </listitem> </varlistentry> </variablelist> @@ -94,58 +130,13 @@ <section id="s1-rpm-using"> <title>Using RPM</title> - <para>RPM has five basic modes of operation<indexterm><primary>RPM</primary> + <para><application>RPM</application> has five basic modes of operation<indexterm><primary>RPM</primary> <secondary>basic modes</secondary> - </indexterm> (not counting package building): installing, uninstalling, upgrading, querying, and verifying. This section contains an overview of each mode. For complete details and options, try <command>rpm --help</command> or <command>man rpm</command>. You can also refer to <xref - linkend="s1-rpm-additional-resources"/> for more information on RPM.</para> - <section - id="s2-rpm-finding"> - <title>Finding RPM Packages</title> - <indexterm> - <primary>packages</primary> - <secondary>finding RPM packages</secondary> - </indexterm> - <indexterm> - <primary>RPM</primary> - <secondary>finding RPM packages</secondary> - </indexterm> - <para>Before using any RPM packages, you must know where to find them. An Internet search returns many RPM repositories, but if you are looking for &MAJOROS; RPM packages, they can be found at the following locations:</para> - <itemizedlist> - <listitem> - <para>The &MAJOROS; installation media <indexterm><primary>&MAJOROS; installation media</primary> - <secondary>installable packages</secondary> - </indexterm> - <indexterm> - <primary> packages</primary> - <secondary>&MAJOROS; installation media</secondary> - </indexterm> contain many installable RPMs.</para> - </listitem> - <listitem> - <para>The initial RPM repositories provided with the YUM package manager<indexterm><primary>initial RPM repositories</primary> - <secondary>installable packages</secondary> - </indexterm> - <indexterm> - <primary> packages</primary> - <secondary>initial RPM repositories</secondary> - </indexterm>. See <xref - linkend="ch-yum"/> for details on how to use the official &MAJOROS; package repositories.</para> - </listitem> - <listitem> - <para>The active &MAJOROS; mirrors contains many installable RPMs: <ulink - url="http://mirrors.fedoraproject.org/publiclist/"/>.</para> - </listitem> - <listitem> - <para>Unofficial, third-party repositories not affiliated with &OSORG; also provide RPM packages.</para> - <important> - <title>Third-party repositories and package compatibility</title> - <para>When considering third-party repositories for use with your &MAJOROS; system, pay close attention to the repository's web site with regard to package compatibility before adding the repository as a package source. Alternate package repositories may offer different, incompatible versions of the same software, including packages already included in the &MAJOROS; repositories.</para> - </important> - </listitem> - </itemizedlist> - </section> - <section - id="sec-Installing_and_Upgrading"> - <title>Installing and Upgrading</title> + </indexterm> (not counting package building): installing, uninstalling, upgrading, querying, and verifying. This section contains an overview of each mode. For complete details and options, try <command>rpm --help</command> or see <citerefentry><refentrytitle>rpm</refentrytitle><manvolnum>8</manvolnum></citerefentry>. Also, see <xref + linkend="s1-rpm-additional-resources"/> for more information on <application>RPM</application>. + </para> + <section id="sec-Installing_and_Upgrading"> + <title>Installing and Upgrading Packages</title> <indexterm> <primary>RPM</primary> <secondary>installing</secondary> @@ -162,100 +153,121 @@ <primary>packages</primary> <secondary>upgrading RPM</secondary> </indexterm> - <para>RPM packages typically have file names <indexterm><primary>RPM</primary> - <secondary>file name</secondary> - </indexterm> - like <filename>tree-1.5.3-2.&PKGOS;.x86_64.rpm</filename>. The file name includes the package name (<filename>tree</filename>), version (<filename>1.5.3</filename>), release (<filename>2</filename>), operating system major version (<filename>&PKGOS;</filename>) and CPU architecture (<filename>x86_64</filename>).</para> - <para>You can use <command>rpm</command>'s <option>-U</option> option to:</para> + <indexterm> + <primary>RPM</primary> + <secondary>file name</secondary> + </indexterm> + <para> +<application>RPM</application> packages typically have file names in the following form: +</para> + <synopsis>package_name-version-release-operating_system-CPU_architecture.rpm</synopsis> + <para> + For example the <filename>tree-1.7.0-3.&PKGOS;.x86_64.rpm</filename> file name includes the package name (<filename>tree</filename>), version (<filename>1.7.0</filename>), release (<filename>3</filename>), operating system major version (<filename>&PKGOS;</filename>) and <acronym>CPU</acronym> architecture (<filename>x86_64</filename>). + </para> + <important> + <title>Important</title> + <para> + When installing a package, ensure it is compatible with your operating system and processor architecture. This can usually be determined by checking the package name. For example, the file name of an <application>RPM</application> package compiled for the AMD64/Intel 64 computer architectures ends with <filename>x86_64.rpm</filename>. + </para> + </important> + <para> + The <option>-U</option> (or <option>--upgrade</option>) option has two functions, it can be used to: + </para> <itemizedlist> <listitem> - <para>upgrade an existing but older package on the system to a newer version, or</para> + <para> + upgrade an existing package on the system to a newer version, or + </para> </listitem> <listitem> - <para>install the package even if an older version is not already installed.</para> + <para> + install a package if an older version is not already installed. + </para> </listitem> </itemizedlist> - <para>That is, <command>rpm -U <replaceable><rpm_file></replaceable> - </command> is able to perform the function of either <emphasis>upgrading</emphasis> or <emphasis>installing</emphasis> as is appropriate for the package.</para> - <para>Assuming the <filename>tree-1.5.3-2.&PKGOS;.x86_64.rpm</filename> package is in the current directory, log in as root and type the following command at a shell prompt to either upgrade or install the <package>tree</package> package as determined by <command>rpm</command>:</para> -  - <screen> -<command>rpm -Uvh tree-1.5.3-2.&PKGOS;.x86_64.rpm</command> - </screen> + <para> + The <command>rpm -U <replaceable>package.rpm</replaceable></command> command is therefore able to either <emphasis>upgrade</emphasis> or <emphasis>install</emphasis>, depending on the presence of an older version of <replaceable>package.rpm</replaceable> on the system. + </para> + <para>Assuming the <filename>tree-1.7.0-3.&PKGOS;.x86_64.rpm</filename> package is in the current directory, log in as <systemitem class="username">root</systemitem> and type the following command at a shell prompt to either upgrade or install the <package>tree</package> package:</para> + <screen>~]# <command>rpm -Uvh tree-1.7.0-3.&PKGOS;.x86_64.rpm</command></screen> <note id="note-Use_-Uvh_for_nicely-formatted_RPM_installs"> <title>Use -Uvh for nicely-formatted RPM installs</title> <para>The <option>-v</option> and <option>-h</option> options (which are combined with <option>-U</option>) cause <application>rpm</application> to print more verbose output and display a progress meter using hash signs.</para> </note> - <para>If the upgrade/installation is successful, the following output is displayed:</para> + <para>If the upgrade or installation is successful, the following output is displayed:</para> <screen>Preparing... ########################################### [100%] 1:tree ########################################### [100%]</screen> <warning id="warning-Always_use_the_-i_install_option_to_install_new_kernel_packages"> <title>Always use the -i (install) option to install new kernel packages!</title> <para> - <command>rpm</command> provides two different options for installing packages: the aforementioned <option>-U</option> option (which historically stands for <emphasis>upgrade</emphasis>), and the <option>-i</option> option, historically standing for <emphasis>install</emphasis>. Because the <option>-U</option> option subsumes both install and upgrade functions, we recommend to use <command>rpm -Uvh</command> with all packages <emphasis>except <package>kernel</package> packages</emphasis>.</para> - <para>You should always use the <option>-i</option> option to simply <emphasis>install</emphasis> a new kernel package instead of upgrading it. This is because using the <option>-U</option> option to upgrade a kernel package removes the previous (older) kernel package, which could render the system unable to boot if there is a problem with the new kernel. Therefore, use the <command>rpm -i <replaceable><kernel_package></replaceable> - </command> command to install a new kernel <emphasis>without replacing any older <package>kernel</package> packages</emphasis>. For more information on installing <package>kernel</package> packages, refer to <xref + <command>rpm</command> provides two different options for installing packages: the aforementioned <option>-U</option> option (which historically stands for <emphasis>upgrade</emphasis>), and the <option>-i</option> option (which historically stands for <emphasis>install</emphasis>). Because the <option>-U</option> option includes both install and upgrade functions, the use of <command>rpm -Uvh</command> with all packages, <emphasis>except <package>kernel</package> packages</emphasis>, is recommended.</para> + <para>You should always use the <option>-i</option> option to <emphasis>install</emphasis> a new kernel package instead of upgrading it. This is because using the <option>-U</option> option to upgrade a kernel package removes the previous (older) kernel package, which could render the system unable to boot if there is a problem with the new kernel. Therefore, use the <command>rpm -i <replaceable>kernel_package</replaceable> + </command> command to install a new kernel <emphasis>without replacing any older <package>kernel</package> packages</emphasis>. For more information on installing <package>kernel</package> packages, see <xref linkend="ch-Manually_Upgrading_the_Kernel"/>.</para> </warning> - <para>The signature of a package is checked automatically when installing or upgrading a package. The signature confirms that the package was signed by an authorized party. For example, if the verification of the signature fails, an error message such as the following is displayed:</para> - <screen>error: tree-1.5.2.2-4.&PKGOS;.x86_64.rpm: Header V3 RSA/SHA256 signature: BAD, key ID -d22e77f2</screen> - - <para>If it is a new, header-only, signature, an error message such as the following is displayed:</para> - <screen>error: tree-1.5.2.2-4.&PKGOS;.x86_64.rpm: Header V3 RSA/SHA256 signature: BAD, -key ID d22e77f2</screen> - <para>If you do not have the appropriate key installed to verify the signature, the message contains the word <computeroutput>NOKEY</computeroutput>:</para> - <screen>warning: tree-1.5.2.2-4.&PKGOS;.x86_64.rpm: Header V3 RSA/SHA1 signature: NOKEY, key ID 57bbccba</screen> - <para>See <xref - linkend="s1-check-rpm-sig"/> for more information on checking a package's signature.</para> - <section - id="s3-rpm-errors"> - <title>Package Already Installed</title> - <para>If a package of the same name and version is already installed<indexterm><primary>RPM</primary> - <secondary>already installed</secondary> - </indexterm> - <indexterm> - <primary>packages</primary> - <secondary>RPM</secondary> - <tertiary>already installed</tertiary> - </indexterm>, the following output is displayed:</para> + <para> + The signature of a package is checked automatically when installing or upgrading a package. The signature confirms that the package was signed by an authorized party. If the verification of the signature fails, an error message is displayed. + </para> + <para> + If you do not have the appropriate key installed to verify the signature, the message contains the word <computeroutput>NOKEY</computeroutput>: + </para> + <screen>warning: tree-1.7.0-3.&PKGOS;.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 431d51: NOKEY</screen> + <para> + See <xref linkend="s1-check-rpm-sig"/> for more information on checking package signatures. + </para> + <section id="s3-rpm-errors"> + <title>Replacing Already-Installed Packages</title> + <indexterm> + <primary>RPM</primary> + <secondary>already installed</secondary> + </indexterm> + <indexterm> + <primary>packages</primary> + <secondary>RPM</secondary> + <tertiary>already installed</tertiary> + </indexterm> + <para> + If a package of the same name and version is already installed, the following output is displayed: + </para> <screen>Preparing... ########################################### [100%] - package tree-1.5.3-2.&PKGOS;.x86_64 is already installed</screen> - <para>However, if you want to install the package anyway, you can use the <command>--replacepkgs</command> option, which tells RPM to ignore the error:</para> - <screen> -<command>rpm -Uvh --replacepkgs tree-1.5.3-2.&PKGOS;.x86_64.rpm</command> - </screen> - <para>This option is helpful if files installed from the RPM were deleted or if you want the original configuration files from the RPM to be installed.</para> + package tree-1.7.0-3.&PKGOS;.x86_64 is already installed</screen> + <para> + To install the package anyway, use the <option>--replacepkgs</option> option, which tells <application>RPM</application> to ignore the error: + </para> + <screen>~]# <command>rpm -Uvh --replacepkgs tree-1.7.0-3.&PKGOS;.x86_64.rpm</command></screen> + <para> + This option is helpful if files installed from the package were deleted or if you want the original configuration files to be installed. + </para> + <para> + If you attempt an upgrade to an <emphasis>older</emphasis> version of a package (that is, if a newer version of the package is already installed), <application>RPM</application> informs you that a newer version is already installed. To force <application>RPM</application> to perform the downgrade, use the <command>--oldpackage</command> option: + </para> + <synopsis><command>rpm -Uvh --oldpackage <replaceable>older_package.rpm</replaceable></command></synopsis> </section> - <section - id="s3-rpm-conflicting-files"> - <title>Conflicting Files</title> + <section id="s3-rpm-conflicting-files"> + <title>Resolving File Conflicts</title> <indexterm> <primary>RPM</primary> <secondary>file conflicts</secondary> <tertiary>resolving</tertiary> </indexterm> - <para>If you attempt to install a package that contains a file which has already been installed by another package<indexterm><primary>RPM</primary> - <secondary>conflicts</secondary> - </indexterm> - <indexterm> - <primary>packages</primary> - <secondary>RPM</secondary> - <tertiary>conflict</tertiary> - </indexterm>, the following is displayed:</para> - <screen>Preparing... ################################################## - file /usr/bin/foobar from install of foo-1.0-1.&PKGOS;.x86_64 conflicts -with file from package bar-3.1.1.&PKGOS;.x86_64</screen> - <para>To make RPM ignore this error, use the <command>--replacefiles</command> option:</para> - <screen> -<command>rpm -Uvh --replacefiles foo-1.0-1.&PKGOS;.x86_64.rpm</command> - </screen> + <indexterm> + <primary>RPM</primary> + <secondary>conflicts</secondary> + </indexterm> + <indexterm> + <primary>packages</primary> + <secondary>RPM</secondary> + <tertiary>conflict</tertiary> + </indexterm> + <para> + If you attempt to install a package that contains a file that has already been installed by another package, a conflict message is displayed. To make <application>RPM</application> ignore this error, use the <command>--replacefiles</command> option: + </para> + <synopsis><command>rpm -Uvh --replacefiles <replaceable>package.rpm</replaceable></command></synopsis> </section> - <section - id="s3-rpm-unresolved-dependency"> - <title>Unresolved Dependency</title> + <section id="s3-rpm-unresolved-dependency"> + <title>Satisfying Unresolved Dependencies</title> <indexterm> <primary>RPM</primary> <secondary>dependencies</secondary> @@ -264,69 +276,61 @@ with file from package bar-3.1.1.&PKGOS;.x86_64</screen> <primary>packages</primary> <secondary>dependencies</secondary> </indexterm> - <para>RPM packages may sometimes depend on other packages<indexterm><primary>RPM</primary> - <secondary>failed dependencies</secondary> - </indexterm> - <indexterm> - <primary>packages</primary> - <secondary>RPM</secondary> - <tertiary>failed dependencies</tertiary> - </indexterm>, which means that they require other packages to be installed to run properly. If you try to install a package which has an unresolved dependency, output similar to the following is displayed:</para> -  - <screen>error: Failed dependencies: - bar.so.3()(64bit) is needed by foo-1.0-1.&PKGOS;.x86_64</screen> - <para>If you are installing a package from the &MAJOROS; installation media, such as from a CD-ROM or DVD, the dependencies may be available. Find the suggested package(s) on the &MAJOROS; installation media or on one of the active &MAJOROS; mirrors and add it to the command:</para> - <screen> -<command>rpm -Uvh foo-1.0-1.&PKGOS;.x86_64.rpm    bar-3.1.1.&PKGOS;.x86_64.rpm</command> - </screen> - <para>If installation of both packages is successful, output similar to the following is displayed:</para> - <screen>Preparing... ########################################### [100%] - 1:foo ########################################### [ 50%] - 2:bar ########################################### [100%]</screen> - <para>You can try the <option>--whatprovides</option> option to determine which package contains the required file.</para> - <screen> -<command>rpm -q --whatprovides "bar.so.3"</command> - </screen> - <para>If the package that contains <filename>bar.so.3</filename> is in the RPM database, the name of the package is displayed:</para> - <screen>bar-3.1.1.&PKGOS;.i586.rpm</screen> - <warning - id="warning-install-Warning-Forcing_Package_Installation"> - <title>Warning: Forcing Package Installation</title> - <para>Although we can <emphasis>force</emphasis> - <command>rpm</command> to install a package that gives us a <computeroutput>Failed dependencies</computeroutput> error (using the <option>--nodeps</option> option), this is <emphasis>not</emphasis> recommended, and will usually result in the installed package failing to run. Installing or removing packages with <command>rpm --nodeps</command> can cause applications to misbehave and/or crash, and can cause serious package management problems or, possibly, system failure. For these reasons, it is best to heed such warnings; the package manager—whether <application>RPM</application>, <application>Yum</application> or <application>PackageKit</application>—shows us these warnings and suggests possible fixes because accounting for dependencies is critical. The <application>Yum</application> package manager can perform dependency resolution and fetch dependencies from online repositories, making it safer, easier and smarter than forcing <command>rpm</command> to car ry out actions without regard to resolving dependencies.</para> + <indexterm> + <primary>RPM</primary> + <secondary>failed dependencies</secondary> + </indexterm> + <indexterm> + <primary>packages</primary> + <secondary>RPM</secondary> + <tertiary>failed dependencies</tertiary> + </indexterm> + <para> + <application>RPM</application> packages sometimes depend on other packages, which means that they require other packages to be installed to run properly. If you try to install a package that has an unresolved dependency, a message about a failed dependency is displayed. + </para> + <para> + Find the suggested package(s) on the &MAJOROS; installation media or on one of the active &MAJOROS; mirrors and add it to the installation command. To determine which package contains the required file, use the <option>--whatprovides</option> option: + </para> + <synopsis><command>rpm -q --whatprovides "<replaceable>required_file</replaceable>"</command></synopsis> + <para> + If the package that contains <replaceable>required_file</replaceable> is in the <application>RPM</application> database, the name of the package is displayed. + </para> + <warning id="warning-Forcing_Package_Installation"> + <title>Warning</title> + <para> + Although you can <emphasis>force</emphasis> <command>rpm</command> to install a package that has an unresolved dependency (using the <option>--nodeps</option> option), this is <emphasis>not</emphasis> recommended and will usually result in the installed software failing to run. Installing packages with <option>--nodeps</option> can cause applications to misbehave or terminate unexpectedly. It can also cause serious package-management problems or system failure. For these reasons, heed the warnings about missing dependencies. The <application>Yum</application> package manager performs automatic dependency resolution and fetches dependencies from on-line repositories. + </para> </warning> </section> - </section> - <section - id="sec-Configuration_File_Changes"> - <title>Configuration File Changes</title> - <indexterm> - <primary>RPM</primary> - <secondary>configuration file changes</secondary> - </indexterm> - <indexterm> - <primary>packages</primary> - <secondary>RPM</secondary> - <tertiary>configuration file changes</tertiary> - </indexterm> - <para>Because RPM performs intelligent upgrading of packages with configuration files<indexterm><primary>RPM</primary> + <section id="sec-Configuration_File_Changes"> + <title>Preserving Changes in Configuration Files</title> + <indexterm> + <primary>RPM</primary> + <secondary>configuration file changes</secondary> + </indexterm> + <indexterm> + <primary>packages</primary> + <secondary>RPM</secondary> + <tertiary>configuration file changes</tertiary> + </indexterm> + <indexterm><primary>RPM</primary> <secondary>configuration file changes</secondary> <tertiary>conf.rpmsave</tertiary> - </indexterm>, you may see one or the other of the following messages:</para> - <screen>saving /etc/foo.conf as /etc/foo.conf.rpmsave</screen> - <para>This message means that changes you made to the configuration file may not be <emphasis>forward-compatible</emphasis> with the new configuration file in the package, so RPM saved your original file and installed a new one. You should investigate the differences between the two configuration files and resolve them as soon as possible, to ensure that your system continues to function properly.</para> - <para>Alternatively, RPM may save the package's <emphasis>new</emphasis> configuration file as, for example, <filename>foo.conf.rpmnew</filename>, and leave the configuration file you modified untouched. You should still resolve any conflicts between your modified configuration file and the new one, usually by merging changes from the old one to the new one with a <command>diff</command> program.</para> - <para>If you attempt to upgrade to a package with an <emphasis>older</emphasis> version number (that is, if a higher version of the package is already installed), the output is similar to the following:</para> - <screen>package foo-2.0-1.&PKGOS;.x86_64.rpm (which is newer than foo-1.0-1) is already installed</screen> - <para>To force RPM to upgrade anyway, use the <command>--oldpackage</command> option:</para> - <screen> -<command>rpm -Uvh --oldpackage foo-1.0-1.&PKGOS;.x86_64.rpm</command> - </screen> + </indexterm> + <para> + Because <application>RPM</application> performs intelligent upgrading of packages with configuration files, you may see the following message: + </para> + <screen>saving <replaceable>/etc/configuration_file.conf</replaceable> as <replaceable>/etc/configuration_file.conf</replaceable>.rpmsave</screen> + <para> + This message means that the changes you made to the configuration file may not be <emphasis>forward-compatible</emphasis> with the new configuration file in the package, so <application>RPM</application> saved your original file and installed a new one. You should investigate the differences between the two configuration files and resolve them as soon as possible to ensure that your system continues to function properly. + </para> + <para> + Alternatively, <application>RPM</application> may save the package's <emphasis>new</emphasis> configuration file as, for example, <filename><replaceable>configuration_file.conf</replaceable>.rpmnew</filename> and leave the configuration file you modified untouched. You should still resolve any conflicts between your modified configuration file and the new one, usually by merging changes from the old one to the new one, for example using the <command>diff</command> program. + </para> + </section> </section> - <section - id="s2-rpm-uninstalling"> - <title>Uninstalling</title> + <section id="s2-rpm-uninstalling"> + <title>Uninstalling Packages</title> <indexterm> <primary>RPM</primary> <secondary>uninstalling</secondary> @@ -345,36 +349,33 @@ with file from package bar-3.1.1.&PKGOS;.x86_64</screen> <secondary>RPM</secondary> <tertiary>removing</tertiary> </indexterm> - <para>Uninstalling a package is just as simple as installing one. Type the following command at a shell prompt:</para> - <screen>rpm -e foo</screen> + <para> + Uninstalling a package is just as simple as installing one. Type the following command at a shell prompt as <systemitem class="username">root</systemitem>: + </para> + <synopsis><command>rpm -e <replaceable>package</replaceable></command></synopsis> <note> <title>rpm -e and package name errors</title> - <para>Notice that we used the package <emphasis>name</emphasis> - <filename>foo</filename>, not the name of the original package <emphasis>file</emphasis>, <filename>foo-1.0-1.&PKGOS;.x86_64</filename>. If you attempt to uninstall a package using the <command>rpm -e</command> command and the original full file name, you will receive a package name error.</para> + <para> + Note that the command expects only the package <emphasis>name</emphasis>, not the name of the original package <emphasis>file</emphasis>. If you attempt to uninstall a package using the <command>rpm -e</command> command and provide the original full file name, you receive a package-name error. + </para> </note> <para>You can encounter dependency errors when uninstalling a package if another installed package depends on the one you are trying to remove. For example:</para> - <screen> -<command>rpm -e ghostscript</command> + <screen>~]# <command>rpm -e ghostscript</command> error: Failed dependencies: - libgs.so.8()(64bit) is needed by (installed) libspectre-0.2.2-3.&PKGOS;.x86_64 - libgs.so.8()(64bit) is needed by (installed) foomatic-4.0.3-1.&PKGOS;.x86_64 - libijs-0.35.so()(64bit) is needed by (installed) gutenprint-5.2.4-5.&PKGOS;.x86_64 - ghostscript is needed by (installed) printer-filters-1.1-4.&PKGOS;.noarch</screen> - <para>Similar to how we searched for a shared object library (i.e. a <filename><replaceable><library_name></replaceable>.so.<replaceable><number></replaceable> - </filename> file) in <xref - linkend="s3-rpm-unresolved-dependency"/>, we can search for a 64-bit shared object library using this exact syntax (and making sure to quote the file name):</para> - <screen>~]# <command>rpm -q --whatprovides "libgs.so.8()(64bit)"</command> -ghostscript-8.70-1.&PKGOS;.x86_64</screen> - <warning - id="warning-uninstall-Warning-Forcing_Package_Installation"> + ghostscript is needed by (installed) ghostscript-cups-9.07-16.&PKGOS;.x86_64 + ghostscript is needed by (installed) foomatic-4.0.9-6.&PKGOS;.x86_64 + libgs.so.9()(64bit) is needed by (installed) libspectre-0.2.7-4.&PKGOS;.x86_64 + libijs-0.35.so()(64bit) is needed by (installed) gutenprint-5.2.9-15.&PKGOS;.x86_64 + libijs-0.35.so()(64bit) is needed by (installed) cups-filters-1.0.35-15.&PKGOS;.x86_64</screen> + <warning id="warning-uninstall-Warning-Forcing_Package_Installation"> <title>Warning: Forcing Package Installation</title> - <para>Although we can <emphasis>force</emphasis> - <command>rpm</command> to remove a package that gives us a <computeroutput>Failed dependencies</computeroutput> error (using the <option>--nodeps</option> option), this is <emphasis>not</emphasis> recommended, and may cause harm to other installed applications. Installing or removing packages with <command>rpm --nodeps</command> can cause applications to misbehave and/or crash, and can cause serious package management problems or, possibly, system failure. For these reasons, it is best to heed such warnings; the package manager—whether <application>RPM</application>, <application>Yum</application> or <application>PackageKit</application>—shows us these warnings and suggests possible fixes because accounting for dependencies is critical. The <application>Yum</application> package manager can perform dependency resolution and fetch dependencies from online repositories, making it safer, easier and smarter than forcing <command>rpm</command> to carry out actions w ithout regard to resolving dependencies.</para> + <para> + Although you can <emphasis>force</emphasis> <command>rpm</command> to uninstall a package that has unresolved dependencies (using the <option>--nodeps</option> option), this is <emphasis>not</emphasis> recommended. Removing packages with <option>--nodeps</option> can cause applications from the packages whose dependencies are removed to misbehave or terminate unexpectedly. It can also cause serious package-management problems or system failure. For these reasons, heed the warnings about failed dependencies. + </para> </warning> </section> - <section - id="s2-rpm-freshening"> - <title>Freshening</title> + <section id="s2-rpm-freshening"> + <title>Freshening Packages</title> <indexterm> <primary>RPM</primary> <secondary>freshening</secondary> @@ -384,21 +385,23 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <secondary>RPM</secondary> <tertiary>freshening</tertiary> </indexterm> - <para>Freshening is similar to upgrading, except that only existent packages are upgraded. Type the following command at a shell prompt:</para> - <screen> -<command>rpm -Fvh foo-2.0-1.&PKGOS;.x86_64.rpm</command> - </screen> - <para>RPM's freshen option checks the versions of the packages specified on the command line against the versions of packages that have already been installed on your system. When a newer version of an already-installed package is processed by RPM's freshen option, it is upgraded to the newer version. However, RPM's freshen option does not install a package if no previously-installed package of the same name exists. This differs from RPM's upgrade option, as an upgrade <emphasis>does</emphasis> install packages whether or not an older version of the package was already installed.</para> - <para>Freshening works for single packages or package groups. If you have just downloaded a large number of different packages, and you only want to upgrade those packages that are already installed on your system, freshening does the job. Thus, you do not have to delete any unwanted packages from the group that you downloaded before using RPM.</para> - <para>In this case, issue the following with the <filename>*.rpm</filename> glob:</para> - <screen> -<command>rpm -Fvh *.rpm</command> - </screen> - <para>RPM then automatically upgrades only those packages that are already installed.</para> + <para> + Freshening is similar to upgrading, except that only installed packages are upgraded. Type the following command at a shell prompt as <systemitem class="username">root</systemitem>: + </para> + <synopsis><command>rpm -Fvh <replaceable>package.rpm</replaceable></command></synopsis> + <para> + The <option>-F</option> (or <option>--freshen</option>) option compares the versions of the packages specified on the command line with the versions of packages that are already installed on the system. When a newer version of an already-installed package is processed by the <option>--freshen</option> option, it is upgraded to the newer version. However, the <option>--freshen</option> option does not install a package if no previously-installed package of the same name exists. This differs from regular upgrading, as an upgrade installs all specified packages regardless of whether or not older versions of the packages are already installed. + </para> + <para> + Freshening works for single packages or package groups. For example, freshening can help if you download a large number of different packages, and you only want to upgrade those packages that are already installed on the system. In this case, issue the following command with the <filename>*.rpm</filename> global expression: + </para> + <screen>~]# <command>rpm -Fvh *.rpm</command></screen> + <para> + <application>RPM</application> then automatically upgrades only those packages that are already installed. + </para> </section> - <section - id="s2-rpm-querying"> - <title>Querying</title> + <section id="s2-rpm-querying"> + <title>Querying Packages</title> <indexterm> <primary>RPM</primary> <secondary>querying</secondary> @@ -408,60 +411,21 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <secondary>RPM</secondary> <tertiary>querying</tertiary> </indexterm> - <para>The RPM database stores information about all RPM packages installed in your system. It is stored in the directory <filename>/var/lib/rpm/</filename>, and is used to query what packages are installed, what versions each package is, and to calculate any changes to any files in the package since installation, among other use cases.</para> - <para>To query this database, use the <command>-q</command> option. The <command>rpm -q <replaceable>package name</replaceable> - </command> command displays the package name, version, and release number of the installed package <replaceable><package_name></replaceable>. For example, using <command>rpm -q tree</command> to query installed package <filename>tree</filename> might generate the following output:</para> - <screen>tree-1.5.2.2-4.&PKGOS;.x86_64</screen> - <para>You can also use the following <emphasis>Package Selection Options</emphasis> (which is a subheading in the RPM man page: see <command>man rpm</command> for details) to further refine or qualify your query:</para> - <itemizedlist> - <listitem> - <para> - <command>-a</command> — queries all currently installed packages.</para> - </listitem> - <listitem> - <para> - <command>-f <filename><replaceable><file_name></replaceable> - </filename> - </command> — queries the RPM database for which package owns <filename><replaceable><file_name></replaceable> - </filename>. Specify the absolute path of the file (for example, <command>rpm -qf <filename>/bin/ls</filename> - </command> instead of <command>rpm -qf ls</command>).</para> - </listitem> - <listitem> - <para> - <command>-p <filename><replaceable><package_file></replaceable> - </filename> - </command> — queries the uninstalled package <filename><replaceable><package_file></replaceable> - </filename>.</para> - </listitem> - </itemizedlist> - <para>There are a number of ways to specify what information to display about queried packages. The following options are used to select the type of information for which you are searching. These are called the <emphasis>Package Query Options</emphasis>.</para> - <itemizedlist> - <listitem> - <para> - <command>-i</command> displays package information including name, description, release, size, build date, install date, vendor, and other miscellaneous information.</para> - </listitem> - <listitem> - <para> - <command>-l</command> displays the list of files that the package contains.</para> - </listitem> - <listitem> - <para> - <command>-s</command> displays the state of all the files in the package.</para> - </listitem> - <listitem> - <para> - <command>-d</command> displays a list of files marked as documentation (man pages, info pages, READMEs, etc.) in the package.</para> - </listitem> - <listitem> - <para> - <command>-c</command> displays a list of files marked as configuration files. These are the files you edit after installation to adapt and customize the package to your system (for example, <filename>sendmail.cf</filename>, <filename>passwd</filename>, <filename>inittab</filename>, etc.).</para> - </listitem> - </itemizedlist> - <para>For options that display lists of files, add <command>-v</command> to the command to display the lists in a familiar <command>ls -l</command> format.</para> + <para> + The <application>RPM</application> database stores information about all <application>RPM</application> packages installed on the system. It is stored in the <filename>/var/lib/rpm/</filename> directory and is used for many things, including querying what packages are installed, what version each package is, and for calculating changes to files in packages since their installation. To query this database, use the <command>rpm</command> command with the <option>-q</option> (or <option>--query</option>) option: + </para> + <synopsis><command>rpm -q <replaceable>package_name</replaceable></command></synopsis> + <para> + This command displays the package name, version, and release number of the installed package <replaceable>package_name</replaceable>. For example: + </para> + <screen>~]$ <command>rpm -q tree</command> +tree-1.7.0-3.&PKGOS;.x86_64</screen> + <para> + See the <computeroutput>Package Selection Options</computeroutput> subheading in the <citerefentry><refentrytitle>rpm</refentrytitle><manvolnum>8</manvolnum></citerefentry> manual page for a list of options that can be used to further refine or qualify your query. Use options listed below the <computeroutput>Package Query Options</computeroutput> subheading to specify what information to display about the queried packages. + </para> </section> - <section - id="s2-rpm-verifying"> - <title>Verifying</title> + <section id="s2-rpm-verifying"> + <title>Verifying Packages</title> <indexterm> <primary>RPM</primary> <secondary>verifying</secondary> @@ -471,123 +435,240 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <secondary>RPM</secondary> <tertiary>verifying</tertiary> </indexterm> - <para>Verifying a package compares information about files installed from a package with the same information from the original package. Among other things, verifying compares the file size, MD5 sum, permissions, type, owner, and group of each file.</para> - <para>The command <command>rpm -V</command> verifies a package. You can use any of the <emphasis>Verify Options</emphasis> listed for querying to specify the packages you wish to verify. A simple use of verifying is <command>rpm -V tree</command>, which verifies that all the files in the <command>tree</command> package are as they were when they were originally installed. For example:</para> - <itemizedlist - mark="bullet"> - <listitem> - <para>To verify a package containing a particular file:</para> - <screen> -<command>rpm -Vf /usr/bin/tree</command> - </screen> - <para>In this example, <filename>/usr/bin/tree</filename> is the absolute path to the file used to query a package.</para> - </listitem> - <listitem> - <para>To verify ALL installed packages throughout the system (which will take some time):</para> - <screen> -<command>rpm -Va</command> - </screen> - </listitem> - <listitem> - <para>To verify an installed package against an RPM package file:</para> - <screen> -<command>rpm -Vp tree-1.5.2.2-4.&PKGOS;.x86_64.rpm</command> - </screen> - <para>This command can be useful if you suspect that your RPM database is corrupt.</para> - </listitem> - </itemizedlist> - <para>If everything verified properly, there is no output. If there are any discrepancies, they are displayed. The format of the output is a string of eight characters (a "<computeroutput>c</computeroutput>" denotes a configuration file) and then the file name. Each of the eight characters denotes the result of a comparison of one attribute of the file to the value of that attribute recorded in the RPM database. A single period (<computeroutput>.</computeroutput>) means the test passed. The following characters denote specific discrepancies:</para> + <para> + Verifying a package is comparing information about files on the system installed from a package with the same information from the original package. Among other parameters, verifying compares the file size, MD5 sum, permissions, type, owner, and the group of each file. + </para> + <para> + Use the <command>rpm</command> command with the <option>-V</option> (or <option>--verify</option>) option to verify packages. For example: + </para> + <screen>~]$ <command>rpm -V tree</command></screen> + <para> + See the <computeroutput>Package Selection Options</computeroutput> subheading in the <citerefentry><refentrytitle>rpm</refentrytitle><manvolnum>8</manvolnum></citerefentry> manual page for a list of options that can be used to further refine or qualify your query. Use options listed below the <computeroutput>Verify Options</computeroutput> subheading to specify what characteristics to verify in the queried packages. + </para> + <para> + If everything verifies properly, there is no output. If there are any discrepancies, they are displayed. The output consists of lines similar to these: + </para> + <screen>~]# <command>rpm -V abrt</command> +S.5....T. c /etc/abrt/abrt.conf +.M....... /var/spool/abrt-upload</screen> + <para> + The format of the output is a string of nine characters followed by an optional attribute marker and the name of the processed file. + </para> + <para> + The first nine characters are the results of tests performed on the file. Each test is the comparison of one attribute of the file to the value of that attribute as recorded in the <application>RPM</application> database. A single period (<computeroutput>.</computeroutput>) means the test passed, and the question-mark character (<computeroutput>?</computeroutput>) signifies that the test could not be performed. The following table lists symbols that denote specific discrepancies: + </para> + <table id="tab-rpm-verification-symbols"> + <title>RPM Verification Symbols</title> + <indexterm> + <primary>RPM</primary> + <secondary>verification</secondary> + </indexterm> + <tgroup cols="2"> + <colspec colname="symbol" colnum="1" colwidth="20*"/> + <colspec colname="description" colnum="2" colwidth="80*"/> + <thead> + <row> + <entry>Symbol</entry> + <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry><computeroutput>S</computeroutput></entry> + <entry>file size differs</entry> + </row> + <row> + <entry><computeroutput>M</computeroutput></entry> + <entry>mode differs (includes permissions and file type)</entry> + </row> + <row> + <entry><computeroutput>5</computeroutput></entry> + <entry>digest (formerly MD5 sum) differs</entry> + </row> + <row> + <entry><computeroutput>D</computeroutput></entry> + <entry>device major/minor number mismatch</entry> + </row> + <row> + <entry><computeroutput>L</computeroutput></entry> + <entry><citerefentry><refentrytitle>readLink</refentrytitle><manvolnum>2</manvolnum></citerefentry> path mismatch</entry> + </row> + <row> + <entry><computeroutput>U</computeroutput></entry> + <entry>user ownership differs</entry> + </row> + <row> + <entry><computeroutput>G</computeroutput></entry> + <entry>group ownership differs</entry> + </row> + <row> + <entry><computeroutput>T</computeroutput></entry> + <entry>mtime differs</entry> + </row> + <row> + <entry><computeroutput>P</computeroutput></entry> + <entry>capabilities differ</entry> + </row> + </tbody> + </tgroup> + </table> + <para> + The attribute marker, if present, describes the purpose of the given file. The following table lists the available attribute markers: + </para> + <table id="tab-rpm-verification-markers"> + <title>RPM Verification Symbols</title> + <indexterm> + <primary>RPM</primary> + <secondary>verification</secondary> + </indexterm> + <tgroup cols="2"> + <colspec colname="marker" colnum="1" colwidth="20*"/> + <colspec colname="description" colnum="2" colwidth="80*"/> + <thead> + <row> + <entry>Marker</entry> + <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry><computeroutput>c</computeroutput></entry> + <entry>configuration file</entry> + </row> + <row> + <entry><computeroutput>d</computeroutput></entry> + <entry>documentation file</entry> + </row> + <row> + <entry><computeroutput>l</computeroutput></entry> + <entry>license file</entry> + </row> + <row> + <entry><computeroutput>r</computeroutput></entry> + <entry>readme file</entry> + </row> + </tbody> + </tgroup> + </table> + <para> + If you see any output, use your best judgment to determine if you should remove the package, reinstall it, or fix the problem in another way. + </para> + </section> + </section> + <section id="s1-find-verify-rpm"> + <title>Finding and Verifying RPM Packages</title> + <indexterm> + <primary>RPM</primary> + <secondary>finding and verifying RPM packages</secondary> + </indexterm> + <para> + Before using any <application>RPM</application> packages, you must know where to find them and be able to verify if you can trust them. + </para> + <section id="s2-rpm-finding"> + <title>Finding RPM Packages</title> + <indexterm> + <primary>packages</primary> + <secondary>finding Fedora RPM packages</secondary> + </indexterm> + <indexterm> + <primary>RPM</primary> + <secondary>finding Fedora RPM packages</secondary> + </indexterm> + <para> + Although there are many <application>RPM</application> repositories on the Internet, for security and compatibility reasons, you should consider installing only official Fedora-provided RPM packages. The following is a list of sources for <application>RPM</application> packages: + </para> <itemizedlist> <listitem> <para> - <computeroutput>5</computeroutput> — MD5 checksum</para> - </listitem> - <listitem> - <para> - <computeroutput>S</computeroutput> — file size</para> - </listitem> - <listitem> - <para> - <computeroutput>L</computeroutput> — symbolic link</para> - </listitem> - <listitem> - <para> - <computeroutput>T</computeroutput> — file modification time</para> - </listitem> - <listitem> - <para> - <computeroutput>D</computeroutput> — device</para> - </listitem> - <listitem> - <para> - <computeroutput>U</computeroutput> — user</para> - </listitem> - <listitem> - <para> - <computeroutput>G</computeroutput> — group</para> + <indexterm> + <primary>&MAJOROS; installation media</primary> + <secondary>installable packages</secondary> + </indexterm> + <indexterm> + <primary>packages</primary> + <secondary>&MAJOROS; installation media</secondary> + </indexterm> + Official &MAJOROS; installation media. + </para> </listitem> <listitem> <para> - <computeroutput>M</computeroutput> — mode (includes permissions and file type)</para> + <indexterm> + <primary>initial RPM repositories</primary> + <secondary>installable packages</secondary> + </indexterm> + <indexterm> + <primary>packages</primary> + <secondary>initial RPM repositories</secondary> + </indexterm> + Official <application>RPM</application> repositories provided with the <application>Yum</application> package manager. See <xref linkend="ch-yum"/> for details on how to use the official &MAJOROS; package repositories. + </para> </listitem> + + + <listitem> <para> - <computeroutput>?</computeroutput> — unreadable file (file permission errors, for example)</para> + Unofficial, third-party repositories not affiliated with &OSORG; also provide RPM packages. + </para> + <important> + <title>Important</title> + <para> + When considering third-party repositories for use with your &MAJOROS; system, pay close attention to the repository's web site with regard to package compatibility before adding the repository as a package source. Alternate package repositories may offer different, incompatible versions of the same software, including packages already included in the &MAJOROS; repositories. + </para> + </important> </listitem> </itemizedlist> - <para>If you see any output, use your best judgment to determine if you should remove the package, reinstall it, or fix the problem in another way.</para> - </section> - </section> - <section - id="s1-check-rpm-sig"> - <title>Checking a Package's Signature</title> - <indexterm> - <primary>RPM</primary> - <secondary>md5sum</secondary> - </indexterm> - <para>To verify that a package has not been corrupted or tampered with, examine the checksum by typing the following command at a shell prompt (where <replaceable><rpm_file></replaceable> is the file name of the RPM package):</para> - <screen><command>rpm -K --nosignature <replaceable><rpm_file></replaceable></command></screen> - <para>If the message <computeroutput><replaceable><rpm_file></replaceable>: sha1 md5 OK</computeroutput> (specifically the <computeroutput>OK</computeroutput> part of it) is displayed, the file was not corrupted during download. To see a more verbose message, replace <option>-K</option> with <option>-Kvv</option> in the command.</para> - <para>On the other hand, how trustworthy is the developer who created the package? If the package is <firstterm>signed</firstterm> with the developer's GnuPG <firstterm>key</firstterm>, you know that the developer really is who they say they are.</para> - <indexterm> - <primary>RPM</primary> - <secondary>GnuPG</secondary> - </indexterm> - <indexterm> - <primary>RPM</primary> - <secondary>checking package signatures</secondary> - </indexterm> - <indexterm> - <primary>GnuPG</primary> - <secondary>checking RPM package signatures</secondary> - </indexterm> - <para>An RPM package can be signed using <firstterm>GNU Privacy Guard</firstterm> (or GnuPG), to help you make certain your downloaded package is trustworthy.</para> - <para>GnuPG is a tool for secure communication; it is a complete and free replacement for the encryption technology of PGP, an electronic privacy program. With GnuPG, you can authenticate the validity of documents and encrypt/decrypt data to and from other recipients. GnuPG is capable of decrypting and verifying PGP 5.<replaceable>x</replaceable> files as well.</para> - <para>During installation, GnuPG is installed by default, which enables you to immediately start using it to verify any packages that you download from the Fedora Project. Before doing so, you first need to import the correct Fedora key.</para> - <section - id="s2-keys-importing"> - <title>Importing Keys</title> - <para>Fedora GnuPG keys are located in the <filename>/etc/pki/rpm-gpg/</filename> directory. To verify a Fedora Project package, first import the correct key based on your processor architecture:</para> - <screen><command>rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-<replaceable>x86_64</replaceable></command></screen> - <para>To display a list of all keys installed for RPM verification, execute the command:</para> - <screen><command>rpm -qa gpg-pubkey*</command></screen> - <para>For the Fedora Project key, the output states:</para> - <screen>gpg-pubkey-57bbccba-4a6f97af</screen> - <para>To display details about a specific key, use <command>rpm -qi</command> followed by the output from the previous command:</para> - <screen><command>rpm -qi gpg-pubkey-57bbccba-4a6f97af</command></screen> </section> - <section - id="s2-keys-checking"> - <title>Verifying Signature of Packages</title> - <para>To check the GnuPG signature of an RPM file after importing the builder's GnuPG key, use the following command (replace <replaceable><rpm_file></replaceable> with the file name of the RPM package):</para> - <screen><command>rpm -K <replaceable><rpm_file></replaceable></command></screen> - <para>If all goes well, the following message is displayed: <computeroutput>rsa sha1 (md5) pgp md5 OK</computeroutput>. This means that the signature of the package has been verified, that it is not corrupt, and is therefore safe to install and use.</para> - <para>For more information, including a list of currently-used Fedora Project keys and their fingerprints, refer to <ulink url="http://fedoraproject.org/en/keys"/>.</para> + <section id="s1-check-rpm-sig"> + <title>Checking Package Signatures</title> + <indexterm> + <primary>RPM</primary> + <secondary>GnuPG</secondary> + </indexterm> + <indexterm> + <primary>RPM</primary> + <secondary>checking package signatures</secondary> + </indexterm> + <indexterm> + <primary>GnuPG</primary> + <secondary>checking RPM package signatures</secondary> + </indexterm> + <para> + <application>RPM</application> packages can be signed using <application>GNU Privacy Guard</application> (or <application>GPG</application>), which helps you make certain that downloaded packages are trustworthy. <application>GPG</application> is a tool for secure communication. With <application>GPG</application>, you can authenticate the validity of documents and encrypt or decrypt data. + </para> + <para> + To verify that a package has not been corrupted or tampered with, check its <application>GPG</application> signature by using the <command>rpmkeys</command> command with the <option>-K</option> (or <option>--checksig</option>) option: + </para> + <synopsis><command>rpmkeys -K <replaceable>package.rpm</replaceable></command></synopsis> + <para> + Note that the <application>Yum</application> package manager performs automatic checking of <application>GPG</application> signatures during installations and upgrades. + </para> + <para> + <application>GPG</application> is installed by default, as well as a set of Red Hat keys for verifying packages. To import additional keys for use with <application>RPM</application>, see <xref linkend="s2-keys-importing"/>. + </para> + <section id="s2-keys-importing"> + <title>Importing GPG Keys</title> + <para> + To verify Red Hat packages, a Red Hat <application>GPG</application> key needs to be installed. A set of basic keys is installed by default. To view a list of installed keys, execute the following command at a shell prompt: + </para> + <screen>~]$ <command>rpm -qa gpg-pubkey*</command></screen> + <para> + To display details about a specific key, use <command>rpm -qi</command> followed by the output from the previous command. For example: + </para> + <screen>~]$ <command>rpm -qi gpg-pubkey-fd431d51-4ae0493b</command></screen> + <para> + Use the <command>rpmkeys</command> command with the <option>--import</option> option to install a new key for use with <application>RPM</application>. The default location for storing <application>RPM</application> <acronym>GPG</acronym> keys is the <filename class="directory">/etc/pki/rpm-gpg/</filename> directory. To import new keys, use a command like the following as <systemitem class="username">root</systemitem>: + </para> + <screen>~]# <command>rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release</command></screen> + <para> + See the <ulink url="https://access.redhat.com/security/team/key/">Product Signing (GPG) Keys</ulink> article on the Red Hat Customer Portal for additional information about Red Hat package-signing practices. + </para> + </section> </section> </section> - <section - id="s1-rpm-impressing"> - <title>Practical and Common Examples of RPM Usage</title> + <section id="s1-rpm-usage-examples"> + <title>Common Examples of RPM Usage</title> <indexterm> <primary>RPM</primary> <secondary>tips</secondary> @@ -597,10 +678,14 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <secondary>RPM</secondary> <tertiary>tips</tertiary> </indexterm> - <para>RPM is a useful tool for both managing your system and diagnosing and fixing problems. The best way to make sense of all its options is to look at some examples.</para> + <para> + <application>RPM</application> is a useful tool for both managing your system and diagnosing and fixing problems. See the following examples for an overview of some of the most-used options. + </para> <itemizedlist> <listitem> - <para>Perhaps you have deleted some files by accident, but you are not sure what you deleted. To verify your entire system and see what might be missing, you could try the following command:</para> + <para> + To verify your entire system and see what files are missing, issue the following command as <systemitem class="username">root</systemitem>: + </para> <indexterm> <primary>RPM</primary> <secondary>finding deleted files with</secondary> @@ -609,13 +694,15 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <primary>packages</primary> <secondary>finding deleted files from</secondary> </indexterm> - <screen> -<command>rpm -Va</command> - </screen> - <para>If some files are missing or appear to have been corrupted, you should probably either re-install the package or uninstall and then re-install the package.</para> + <synopsis><command>rpm -Va</command></synopsis> + <para> + If some files are missing or appear corrupted, consider reinstalling relevant packages. + </para> </listitem> <listitem> - <para>At some point, you might see a file that you do not recognize. To find out which package owns it, enter:</para> + <para> + To determine which package owns a file, enter: + </para> <indexterm> <primary>RPM</primary> <secondary>determining file ownership with</secondary> @@ -624,21 +711,18 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <primary>packages</primary> <secondary>determining file ownership with</secondary> </indexterm> - <screen> -<command>rpm -qf /usr/bin/ghostscript</command> - </screen> - <para>The output would look like the following:</para> - <screen>ghostscript-8.70-1.&PKGOS;.x86_64</screen> + <synopsis><command>rpm -qf <replaceable>file</replaceable></command></synopsis> </listitem> <listitem> - <para>We can combine the above two examples in the following scenario. Say you are having problems with <filename>/usr/bin/paste</filename>. You would like to verify the package that owns that program, but you do not know which package owns <command>paste</command>. Enter the following command,</para> - <screen> -<command>rpm -Vf /usr/bin/paste</command> - </screen> - <para>and the appropriate package is verified.</para> + <para> + To verify the package that owns a particular file, enter as <systemitem class="username">root</systemitem>: + </para> + <synopsis><command>rpm -Vf <replaceable>file</replaceable></command></synopsis> </listitem> <listitem> - <para>Do you want to find out more information about a particular program? You can try the following command to locate the documentation which came with the package that owns that program:</para> + <para> + To locate documentation files that are a part of a package to which a file belongs, enter: + </para> <indexterm> <primary>RPM</primary> <secondary>documentation with</secondary> @@ -651,34 +735,12 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <primary>documentation</primary> <secondary>finding installed</secondary> </indexterm> - <screen> -<command>rpm -qdf /usr/bin/free</command> - </screen> - <para>The output would be similar to the following:</para> - <screen>/usr/share/doc/procps-ng/BUGS -/usr/share/doc/procps-ng/FAQ -/usr/share/doc/procps-ng/NEWS -/usr/share/doc/procps-ng/TODO -/usr/share/man/man1/free.1.gz -/usr/share/man/man1/pgrep.1.gz -/usr/share/man/man1/pkill.1.gz -/usr/share/man/man1/pmap.1.gz -/usr/share/man/man1/ps.1.gz -/usr/share/man/man1/pwdx.1.gz -/usr/share/man/man1/skill.1.gz -/usr/share/man/man1/slabtop.1.gz -/usr/share/man/man1/snice.1.gz -/usr/share/man/man1/tload.1.gz -/usr/share/man/man1/top.1.gz -/usr/share/man/man1/uptime.1.gz -/usr/share/man/man1/w.1.gz -/usr/share/man/man1/watch.1.gz -/usr/share/man/man5/sysctl.conf.5.gz -/usr/share/man/man8/sysctl.8.gz -/usr/share/man/man8/vmstat.8.gz</screen> + <synopsis><command>rpm -qdf <replaceable>file</replaceable></command></synopsis> </listitem> <listitem> - <para>You may find a new RPM, but you do not know what it does. To find information about it, use the following command:</para> + <para> + To find information about a (non-installed) package file, use the following command: + </para> <indexterm> <primary>RPM</primary> <secondary>querying uninstalled packages</secondary> @@ -687,26 +749,12 @@ ghostscript-8.70-1.&PKGOS;.x86_64</screen> <primary>packages</primary> <secondary>querying uninstalled</secondary> </indexterm> - <screen> -<command>rpm -qip crontabs-1.10-31.&PKGOS;.noarch.rpm</command> - </screen> - <para>The output would be similar to the following:</para> - <screen>Name : crontabs Relocations: (not relocatable) -Size : 2486 License: Public Domain and GPLv2 -Signature : RSA/SHA1, Tue 11 Aug 2009 01:11:19 PM CEST, Key ID 9d1cc34857bbccba -Packager : Fedora Project -Summary : Root crontab files used to schedule the execution of programs -Description : -The crontabs package contains root crontab files and directories. -You will need to install cron daemon to run the jobs from the crontabs. -The cron daemon such as cronie or fcron checks the crontab files to -see when particular commands are scheduled to be executed. If commands -are scheduled, it executes them. -Crontabs handles a basic system function, so it should be installed on -your system.</screen> + <synopsis><command>rpm -qip <replaceable>package.rpm</replaceable></command></synopsis> </listitem> <listitem> - <para>Perhaps you now want to see what files the <filename>crontabs</filename> RPM package installs. You would enter the following:</para> + <para> + To list files contained in a package, use: + </para> <indexterm> <primary>RPM</primary> <secondary>querying for file list</secondary> @@ -715,95 +763,68 @@ your system.</screen> <primary>packages</primary> <secondary>obtaining list of files</secondary> </indexterm> - <screen> -<command>rpm -qlp crontabs-1.10-31.&PKGOS;.noarch.rpm</command> - </screen> - <para>The output is similar to the following:</para> - <screen>/etc/cron.daily -/etc/cron.hourly -/etc/cron.monthly -/etc/cron.weekly -/etc/crontab -/usr/bin/run-parts -/usr/share/man/man4/crontabs.4.gz</screen> + <synopsis><command>rpm -qlp <replaceable>package.rpm</replaceable></command></synopsis> </listitem> </itemizedlist> - <para>These are just a few examples. As you use RPM, you may find more uses for it.</para> + <para> + See the <citerefentry><refentrytitle>rpm</refentrytitle><manvolnum>8</manvolnum></citerefentry> manual page for more options. + </para> </section> - <section - id="s1-rpm-additional-resources"> + <section id="s1-rpm-additional-resources"> <title>Additional Resources</title> <indexterm> <primary>RPM</primary> <secondary>additional resources</secondary> </indexterm> - <para>RPM is an extremely complex utility with many options and methods for querying, installing, upgrading, and removing packages. See the following resources to learn more about RPM.</para> - <section - id="s2-rpm-installed-docs"> - <title>Installed Documentation</title> - <itemizedlist> - <listitem> - <para> - <command>rpm --help</command> — This command displays a quick reference of RPM parameters.</para> - </listitem> - <listitem> - <para> - <command>man rpm</command> — The RPM man page gives more detail about RPM parameters than the <command>rpm --help</command> command.</para> - </listitem> - </itemizedlist> - </section> - <section - id="s2-rpm-useful-websites"> - <title>Useful Websites</title> - <itemizedlist> - <indexterm> - <primary>RPM</primary> - <secondary>website</secondary> - </indexterm> - <listitem> - <para>The RPM website — <ulink - url="http://www.rpm.org/">http://www.rpm.org/</ulink> - </para> - </listitem> - <listitem> - <para>The RPM mailing list can be subscribed to, and its archives read from, here — <ulink - url="http://www.redhat.com/mailman/listinfo/rpm-list/">https://lists.rpm.org/mailman/listinfo/rpm-list</ulink> - </para> - </listitem> - </itemizedlist> - </section> - <section - id="s2-rpm-related-books"> - <title>Related Books</title> + <para> + <application>RPM</application> is a complex utility with many options and methods for querying, installing, upgrading, and removing packages. See the following resources to learn more about <application>RPM</application>. + </para> + <bridgehead id="brid-rpm-resources-installed" renderas="sect2">Installed Documentation</bridgehead> + <itemizedlist> + <listitem> + <para> + <command>rpm --help</command> — This command displays a quick reference of <application>RPM</application> parameters. + </para> + </listitem> + <listitem> + <para> + <citerefentry><refentrytitle>rpm</refentrytitle><manvolnum>8</manvolnum></citerefentry> — The <application>RPM</application> manual page offers an overview of all available <application>RPM</application> parameters. + </para> + </listitem> + </itemizedlist> + <bridgehead id="brid-rpm-resources-online" renderas="sect2">Online Documentation</bridgehead> + <itemizedlist> <indexterm> <primary>RPM</primary> - <secondary>book about</secondary> + <secondary>website</secondary> </indexterm> <indexterm> - <primary> - <citetitle>Fedora Hat RPM Guide</citetitle> - </primary> + <primary>RPM</primary> + <secondary>online documentation</secondary> </indexterm> - <variablelist> - <varlistentry> - <term> - <citetitle>Maximum RPM</citetitle> — <ulink - url="http://www.rpm.org/max-rpm/"/> - </term> - <listitem> - <para>The <citetitle>Maximum RPM</citetitle> book, which you can read online, covers everything from general RPM usage to building your own RPMs to programming with rpmlib.</para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <citetitle>Fedora RPM Guide</citetitle> — <ulink - url="http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/R..."/> - </term> - <listitem> - <para>The <citetitle>Fedora RPM Guide</citetitle> by Eric Foster-Johnson is an excellent resource on all details of the RPM package format and the RPM package management utility.</para> - </listitem> - </varlistentry> - </variablelist> - </section> + + <listitem> + <para> + The <application>RPM</application> website — <ulink url="http://www.rpm.org/" /> + </para> + </listitem> + <listitem> + <para> + The <application>RPM</application> mailing list — <ulink url="http://lists.rpm.org/mailman/listinfo/rpm-list" /> + </para> + </listitem> + </itemizedlist> + <bridgehead id="brid-rpm-resources-also" renderas="sect2">See Also</bridgehead> + <itemizedlist> + <indexterm> + <primary>RPM</primary> + <secondary>see also</secondary> + </indexterm> + <listitem> + <para> + <xref linkend="ch-yum" /> describes how to use the <application>Yum</application> package manager to search, install, update, and uninstall packages on the command line. + </para> + </listitem> + </itemizedlist> </section> </appendix>

9 years, 5 months

1
0
0 / 0

[firewall-guide] master: Brought in firewalld text from Fedora Security Guide as a baseline upon which to build. (10b49f3)

by Simon Clark

Repository : http://git.fedorahosted.org/cgit/docs/firewall-guide.git On branch : master >--------------------------------------------------------------- commit 10b49f38956d1404e9fe7f9856becb93ac241c55 Author: Simon Clark <simon.richard.clark(a)gmail.com> Date: Sat Dec 13 16:05:05 2014 +0000 Brought in firewalld text from Fedora Security Guide as a baseline upon which to build. >--------------------------------------------------------------- Diff suppressed because of size. To see it, use: git diff --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol ^10b49f38956d1404e9fe7f9856becb93ac241c55~1 10b49f38956d1404e9fe7f9856becb93ac241c55

9 years, 5 months

1
0
0 / 0

[system-administrators-guide] Applying improvements to OProfile

by stephenw

commit 29a16eb6e18cac53164a85cbd99f75bdd3aae2b6 Author: Stephen Wadeley <swadeley(a)redhat.com> Date: Sat Dec 13 14:58:01 2014 +0100 Applying improvements to OProfile from upstream version, after proofreading and SME feedback en-US/OProfile.xml | 1082 +++++++++++++++++++++++++++++++++------------------- 1 files changed, 694 insertions(+), 388 deletions(-) --- diff --git a/en-US/OProfile.xml b/en-US/OProfile.xml index dd9f724..e5e2ffa 100644 --- a/en-US/OProfile.xml +++ b/en-US/OProfile.xml @@ -4,20 +4,18 @@ <chapter id="ch-OProfile"> <title>OProfile</title> - <indexterm - significance="normal"> + <indexterm> <primary>system analysis</primary> <secondary>OProfile</secondary> <see>OProfile</see> </indexterm> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> </indexterm> <para>OProfile is a low overhead, system-wide performance monitoring tool. It uses the performance monitoring hardware on the processor to retrieve information about the kernel and executables on the system, such as when memory is referenced, the number of L2 cache requests, and the number of hardware interrupts received. On a &MAJOROS; system, the <filename>oprofile</filename> package must be installed to use this tool.</para> - <para>Many processors include dedicated performance monitoring hardware. This hardware makes it possible to detect when certain events happen (such as the requested data not being in cache). The hardware normally takes the form of one or more <firstterm>counters</firstterm> that are incremented each time an event takes place. When the counter value, essentially rolls over, an interrupt is generated, making it possible to control the amount of detail (and therefore, overhead) produced by performance monitoring.</para> + <para>Many processors include dedicated performance monitoring hardware. This hardware makes it possible to detect when certain events happen (such as the requested data not being in cache). The hardware normally takes the form of one or more <firstterm>counters</firstterm> that are incremented each time an event takes place. When the counter value increments, an interrupt is generated, making it possible to control the amount of detail (and therefore, overhead) produced by performance monitoring.</para> <para>OProfile uses this hardware (or a timer-based substitute in cases where performance monitoring hardware is not present) to collect <firstterm>samples</firstterm> of performance-related data each time a counter generates an interrupt. These samples are periodically written out to disk; later, the data contained in these samples can then be used to generate reports on system-level and application-level performance.</para> - <para>OProfile is a useful tool, but be aware of some limitations when using it:</para> + <para>Be aware of the following limitations when using OProfile:</para> <itemizedlist> <listitem> <para> @@ -38,7 +36,7 @@ </listitem> <listitem> <para> - <emphasis>Hardware performance counters do not work on guest virtual machines</emphasis> — Because the hardware performance counters are not available on virtual systems, you need to use the <computeroutput>timer</computeroutput> mode. Run the command <command>opcontrol --deinit</command>, and then execute <command>modprobe oprofile timer=1</command> to enable the <computeroutput>timer</computeroutput> mode.</para> + <emphasis>Hardware performance counters do not work on guest virtual machines</emphasis> — Because the hardware performance counters are not available on virtual systems, you need to use the <computeroutput>timer</computeroutput> mode. Enter the command <command>opcontrol --deinit</command>, and then execute <command>modprobe oprofile timer=1</command> to enable the <computeroutput>timer</computeroutput> mode.</para> </listitem> <listitem> <para> @@ -48,14 +46,13 @@ <section id="s1-oprofile-overview-tools"> <title>Overview of Tools</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>overview of tools</secondary> </indexterm> <para> <xref - linkend="tb-oprofile-tools"/> provides a brief overview of the tools provided with the <filename>oprofile</filename> package.</para> + linkend="tb-oprofile-tools"/> provides a brief overview of the most commonly used tools provided with the <filename>oprofile</filename> package.</para> <table id="tb-oprofile-tools"> <title>OProfile Commands</title> @@ -116,6 +113,14 @@ </row> <row> <entry> + <command>operf</command> + </entry> + <entry> + <para>Recommended tool to be used in place of <command>opcontrol</command> for profiling. See <xref linkend="s1-using-operf"/> for details.</para> For differences between <command>operf</command> and <command>opcontrol</command> see <xref linkend="s2-operf_vs_opcontrol"/>. + </entry> + </row> + <row> + <entry> <command>opreport</command> </entry> <entry> @@ -134,59 +139,203 @@ </tbody> </tgroup> </table> + <section id="s2-operf_vs_opcontrol"> + <title>operf vs. opcontrol</title> + <para> + There are two mutually exclusive methods for collecting profiling data with OProfile. You can either use the newer and preferred <command>operf</command> or the <command>opcontrol</command> tool. + </para> + <bridgehead>operf</bridgehead> + <para> + This is the recommended mode for profiling. The <command>operf</command> tool uses the Linux Performance Events Subsystem, and therefore does not require the <emphasis>oprofile</emphasis> kernel driver. The <command>operf</command> tool allows you to target your profiling more precisely, as a single process or system-wide, and also allows OProfile to co-exist better with other tools using the performance monitoring hardware on your system. Unlike <command>opcontrol</command>, it can be used without the <systemitem class="username">root</systemitem> privileges. However, <command>operf</command> is also capable of system-wide operations with use of the <option>--system-wide</option> option, where root authority is required. + </para> + <para> + With <command>operf</command>, there is no initial setup needed. You can invoke <command>operf</command> with command-line options to specify your profiling settings. After that, you can run the OProfile post-processing tools described in <xref linkend="s1-oprofile-analyzing-data"/>. See <xref linkend="s1-using-operf"/> for further information. + </para> + <bridgehead>opcontrol</bridgehead> + <para> + This mode consists of the <command>opcontrol</command> shell script, the <systemitem class="daemon">oprofiled</systemitem> daemon, and several post-processing tools. The <command>opcontrol</command> command is used for configuring, starting, and stopping a profiling session. An OProfile kernel driver, usually built as a kernel module, is used for collecting samples, which are then recorded into sample files by <systemitem class="daemon">oprofiled</systemitem>. You can use legacy mode only if you have <systemitem class="username">root</systemitem> privileges. In certain cases, such as when you need to sample areas with disabled interrupt request (IRQ), this is a better alternative. + </para> + <para> + Before OProfile can be run in legacy mode, it must be configured as shown in <xref linkend="s1-oprofile-configuring"/>. These settings are then applied when starting OProfile (<xref linkend="s1-oprofile-starting"/>). + </para> + </section> + </section> + <section id="s1-using-operf"> + <title>Using operf</title> + <para> + <command>operf</command> is the recommended profiling mode that does not require initial setup before starting. All settings are specified as command-line options and there is no separate command to start the profiling process. To stop <command>operf</command>, press Ctrl+C. The typical <command>operf</command> command syntax looks as follows: + </para> + <synopsis><command>operf</command> <replaceable>options</replaceable> <replaceable>range</replaceable> <replaceable>command</replaceable> <replaceable>args</replaceable></synopsis> + <para> + Replace <replaceable>options</replaceable> with the desired command-line options to specify your profiling settings. Full set of options is described in <systemitem>operf(1)</systemitem> manual page. Replace <replaceable>range</replaceable> with one of the following: + </para> + <para> + <option>--system-wide</option> - this setting allows for global profiling, see <xref linkend="using_operf_system-wide"/> + </para> + <para> + <option>--pid=<replaceable>PID</replaceable></option> - this is to profile a running application, where <replaceable>PID</replaceable> is the process ID of the process you want to profile. + </para> + <para> + With <replaceable>command</replaceable> and <replaceable>args</replaceable>, you can define a specific command or application to be profiled, and also the input arguments that this command or application requires. Either <replaceable>command</replaceable>, <option>--pid</option> or <option>--system-wide</option> is required, but these cannot be used simultaneously. + </para> + <para> + When you invoke <command>operf</command> on a command line without setting the <replaceable>range</replaceable> option, data will be collected for the children processes. + </para> + <note id="using_operf_system-wide"> + <title>Using <command>operf</command> in System-wide Mode</title> + <para> + To run <command>operf</command> <option>--system-wide</option>, you need <systemitem class="username">root</systemitem> authority. When finished profiling, you can stop <command>operf</command> with <literal>Ctrl+C</literal>. + </para> + <para> + If you run <command>operf</command> <option>--system-wide</option> as a background process (with <literal>&</literal>), stop it in a controlled manner in order to process the collected profile data. For this purpose, use: + </para> + <synopsis><command>kill -SIGINT operf-PID </command></synopsis> + <para> + When running <command>operf</command> <option>--system-wide</option>, it is recommended that your current working directory is <filename class="directory">/root</filename> or a subdirectory of <filename class="directory">/root</filename> so that sample data files are not stored in locations accessible by regular users. + </para> + </note> + <section + id="s2-operf-kernel"> + <title>Specifying the Kernel</title> + <para> + To monitor the kernel, execute the following command: + </para> + <synopsis><command>operf</command> <option>--vmlinux</option>=<replaceable>vmlinux_path</replaceable></synopsis> + <para> + With this option, you can specify a path to a vmlinux file that matches the running kernel. Kernel samples will be attributed to this binary, allowing post-processing tools to attribute samples to the appropriate kernel symbols. If this option is not specified, all kernel samples will be attributed to a pseudo binary named "no-vmlinux". + </para> + </section> + <section id="s2-operf-events"> + <title>Setting Events to Monitor</title> + <para> + Most processors contain counters, which are used by OProfile to monitor specific events. As shown in <xref linkend="tb-oprofile-processors"/>, the number of counters available depends on the processor. + </para> + <para> + The events for each counter can be configured via the command line or with a graphical interface. For more information on the graphical interface, see <xref linkend="s1-oprofile-gui"/>. If the counter cannot be set to a specific event, an error message is displayed. + </para> + <note> + <title>Older Processors and operf</title> + <para> + Some older processor models are not supported by the underlying Linux Performance Events Subsystem kernel and therefore are not supported by <command>operf</command>. If you receive this message: + <screen> +Your kernel's Performance Events Subsystem does not support your processor type + </screen> + when attempting to use <command>operf</command>, try profiling with <command>opcontrol</command> to see if your processor type may be supported by OProfile's legacy mode. + </para> + </note> + <note> + <title>Using operf on Virtual Systems</title> + <para> + Since hardware performance counters are not available on guest virtual machines, you have to enable <emphasis>timer</emphasis> mode to use <application>operf</application> on virtual systems. To do so, type as <systemitem class="username">root</systemitem>: + </para> + <synopsis><command>opcontrol</command> <option>--deinit</option></synopsis> + <synopsis><command>modprobe</command> <command>oprofile</command> <option>timer=1</option></synopsis> + </note> + + <para> + To set the event for each configurable counter via the command line, use: + </para> + <synopsis><command>operf</command> <option>--events</option>=<replaceable>event1</replaceable>,<replaceable>event2</replaceable>…</synopsis> + <para> + Here, pass a comma-separated list of event specifications for profiling. Each event specification is a colon-separated list of attributes in the following form: + </para> + <synopsis><replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:<replaceable>kernel</replaceable>:<replaceable>user</replaceable></synopsis> + <para> + <xref linkend="tab_event_specifications"/> summarizes these options. The last three values are optional, if you omit them, they will be set to their default values. Note that certain events do require a unit mask. + </para> + <table id="tab_event_specifications"> + <title>Event Specifications</title> + <tgroup cols='2' align='left'> + <thead> + <row> + <entry>Specification</entry> + <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry><replaceable>event-name</replaceable></entry> + <entry> The exact symbolic event name taken from <command>ophelp</command></entry> + </row> + <row> + <entry><replaceable>sample-rate</replaceable></entry> + <entry>The number of events to wait before sampling again. The smaller the count, the more frequent the samples. For events that do not happen frequently, a lower count may be needed to capture a statistically significant number of event instances. On the other hand, sampling too frequently can overload the system. By default, OProfile uses a time-based event set, which creates a sample every 100,000 clock cycles per processor. </entry> + </row> + <row> + <entry><replaceable>unit-mask</replaceable></entry> + <entry> + Unit masks, which further define the event, are listed in <command>ophelp</command>. + You can insert either a hexadecimal value, beginning with "0x", or a string that matches the first word of the unit mask description in <command>ophelp</command>. Definition by name is valid only for unit masks having "extra:" parameters, as shown by the output of <command>ophelp</command>. This type of unit mask cannot be defined with a hexadecimal value. Note that on certain architectures, there can be multiple unit masks with the same hexadecimal value. In that case they have to be specified by their names only. + </entry> + </row> + <row> + <entry><replaceable>kernel</replaceable></entry> + <entry>Specifies whether to profile kernel code (insert <literal>0</literal> or <literal>1</literal>(default))</entry> + </row> + <row> + <entry><replaceable>user</replaceable></entry> + <entry>Specifies whether to profile user-space code (insert <literal>0</literal> or <literal>1</literal> (default))</entry> + </row> + </tbody> + </tgroup> + </table> + <para> + The events available vary depending on the processor type. When no event specification is given, the default event for the running processor type will be used for profiling. See <xref linkend="tb-oprofile-default-events"/> for a list of these default events. To determine the events available for profiling, use the <command>ophelp</command> command. + </para> + <synopsis><command>ophelp</command></synopsis> + </section> + <section id="s2-operf-categorization"> + <title>Categorization of Samples</title> + <para> + The <option>--separate-thread</option> option categorizes samples by thread group ID (tgid) and thread ID (tid). This is useful for seeing per-thread samples in multi-threaded applications. When used in conjunction with the <option>--system-wide</option> option, <option>--separate-thread</option> is also useful for seeing per-process (i.e., per-thread group) samples for the case where multiple processes are executing the same program during a profiling run. + </para> + <para> + The <option>--separate-cpu</option> option categorizes samples by CPU. + </para> + </section> </section> <section id="s1-oprofile-configuring"> - <title>Configuring OProfile</title> - <indexterm - significance="normal"> + <title>Configuring OProfile Using Legacy Mode</title> + <indexterm> <primary>OProfile</primary> <secondary>configuring</secondary> </indexterm> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> - <secondary> - <command>opcontrol</command> - </secondary> + <secondary><command>opcontrol</command></secondary> </indexterm> - <indexterm - significance="normal"> - <primary> - <command>opcontrol</command> - </primary> + <indexterm> + <primary><command>opcontrol</command></primary> <see>OProfile</see> </indexterm> - <para>Before OProfile can be run, it must be configured. At a minimum, selecting to monitor the kernel (or selecting not to monitor the kernel) is required. The following sections describe how to use the <command>opcontrol</command> utility to configure OProfile. As the <command>opcontrol</command> commands are executed, the setup options are saved to the <filename>/root/.oprofile/daemonrc</filename> file.</para> + <para> + Before OProfile can be run in legacy mode, it must be configured. At a minimum, selecting to monitor the kernel (or selecting not to monitor the kernel) is required. The following sections describe how to use the <command>opcontrol</command> utility to configure OProfile. As the <command>opcontrol</command> commands are executed, the setup options are saved to the <filename>/root/.oprofile/daemonrc</filename> file. + </para> <section id="s2-oprofile-kernel"> <title>Specifying the Kernel</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>monitoring the kernel</secondary> </indexterm> <para>First, configure whether OProfile should monitor the kernel. This is the only configuration option that is required before starting OProfile. All others are optional.</para> - <para>To monitor the kernel, execute the following command as root:</para> - <indexterm - significance="normal"> + <para>To monitor the kernel, execute the following command as <systemitem class="username">root</systemitem>: + </para> + <indexterm> <primary>OProfile</primary> - <secondary> - <command>opcontrol</command> - </secondary> - <tertiary> - <option>--vmlinux=</option> - </tertiary> + <secondary><command>opcontrol</command></secondary> + <tertiary><option>--vmlinux=</option></tertiary> </indexterm>  <screen>~]# <command>opcontrol --setup --vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux</command></screen> - <note> + <important> <title>Install the debuginfo package</title> - <para>The <package>debuginfo</package> package for the kernel must be installed (which contains the uncompressed kernel) in order to monitor the kernel.</para> - </note> - <para>To configure OProfile not to monitor the kernel, execute the following command as root:</para> - <indexterm - significance="normal"> + <para>In order to monitor the kernel, the <package>debuginfo</package> package which contains the uncompressed kernel must be installed. + </para> + </important> + <para>To configure OProfile not to monitor the kernel, execute the following command as <systemitem class="username">root</systemitem>:</para> + <indexterm> <primary>OProfile</primary> <secondary> <command>opcontrol</command> @@ -198,14 +347,13 @@ <screen>~]# <command>opcontrol --setup --no-vmlinux</command></screen> <para>This command also loads the <computeroutput>oprofile</computeroutput> kernel module, if it is not already loaded, and creates the <filename>/dev/oprofile/</filename> directory, if it does not already exist. See <xref linkend="s1-oprofile-dev-oprofile"/> for details about this directory.</para> - <para>Setting whether samples should be collected within the kernel only changes what data is collected, not how or where the collected data is stored. To generate different sample files for the kernel and application libraries, refer to <xref + <para>Setting whether samples should be collected within the kernel only changes what data is collected, not how or where the collected data is stored. To generate different sample files for the kernel and application libraries, see <xref linkend="s2-oprofile-starting-separate"/>.</para> </section> <section id="s2-oprofile-events"> <title>Setting Events to Monitor</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>events</secondary> <tertiary>setting</tertiary> @@ -232,222 +380,297 @@ <thead> <row> <entry> - Processor - </entry> + Processor + </entry> <entry> <command>cpu_type</command> </entry> <entry> - Number of Counters - </entry> + Number of Counters + </entry> </row> </thead> <tbody> <row> <entry> - AMD64 - </entry> + AMD64 + </entry> <entry> - x86-64/hammer - </entry> + x86-64/hammer + </entry> <entry> - 4 - </entry> + 4 + </entry> </row> <row> <entry> - AMD Athlon - </entry> + AMD Family 10h + </entry> <entry> - i386/athlon - </entry> + x86-64/family10 + </entry> <entry> - 4 - </entry> + 4 + </entry> </row> <row> <entry> - AMD Family 10h - </entry> + AMD Family 11h + </entry> <entry> - x86-64/family10 - </entry> + x86-64/family11 + </entry> <entry> - 4 - </entry> + 4 + </entry> </row> <row> <entry> - AMD Family 11h - </entry> + AMD Family 12h + </entry> <entry> - x86-64/family11 - </entry> + x86-64/family12 + </entry> <entry> - 4 - </entry> + 4 + </entry> </row> <row> <entry> - AMD Family 12h - </entry> + AMD Family 14h + </entry> <entry> - x86-64/family12 - </entry> + x86-64/family14 + </entry> <entry> - 4 - </entry> + 4 + </entry> </row> <row> <entry> - AMD Family 14h - </entry> + AMD Family 15h + </entry> <entry> - x86-64/family14 - </entry> + x86-64/family15 + </entry> <entry> - 4 - </entry> + 6 + </entry> </row> <row> <entry> - AMD Family 15h - </entry> + Applied Micro X-Gene + </entry> <entry> - x86-64/family15 - </entry> + arm/armv8-xgene + </entry> <entry> - 6 - </entry> + 4 + </entry> </row> <row> <entry> - IBM eServer System i and IBM eServer System p - </entry> + ARM Cortex A53 + </entry> <entry> - timer - </entry> + arm/armv8-ca53 + </entry> <entry> - 1 - </entry> + 6 + </entry> </row> <row> <entry> - IBM POWER4 - </entry> + ARM Cortex A57 + </entry> <entry> - ppc64/power4 - </entry> + arm/armv8-ca57 + </entry> <entry> - 8 - </entry> + 6 + </entry> </row> <row> <entry> - IBM POWER5 - </entry> + IBM eServer System i and IBM eServer System p + </entry> <entry> - ppc64/power5 - </entry> + timer + </entry> <entry> - 6 - </entry> + 1 + </entry> </row> <row> <entry> - IBM PowerPC 970 - </entry> + IBM POWER4 + </entry> <entry> - ppc64/970 - </entry> + ppc64/power4 + </entry> <entry> - 8 - </entry> + 8 + </entry> </row> <row> <entry> - IBM S/390 and IBM System z - </entry> + IBM POWER5 + </entry> <entry> - timer - </entry> + ppc64/power5 + </entry> <entry> - 1 - </entry> + 6 + </entry> </row> <row> <entry> - Intel Core i7 - </entry> + IBM PowerPC 970 + </entry> <entry> - i386/core_i7 - </entry> + ppc64/970 + </entry> <entry> - 4 - </entry> + 8 + </entry> + </row> + <row> + <entry>IBM PowerPC 970MP</entry> + <entry> ppc64/970MP</entry> + <entry>8</entry> + </row> + <row> + <entry> IBM POWER5+</entry> + <entry>ppc64/power5+</entry> + <entry>6</entry> + </row> + <row> + <entry> IBM POWER5++</entry> + <entry>ppc64/power5++</entry> + <entry>6</entry> + </row> + <row> + <entry> IBM POWER56</entry> + <entry>ppc64/power6</entry> + <entry>6</entry> + </row> + <row> + <entry> IBM POWER7</entry> + <entry>ppc64/power7</entry> + <entry>6</entry> + </row> + <row> + <entry> IBM POWER8</entry> + <entry>ppc64/power7</entry> + <entry>8</entry> </row> <row> <entry> - Intel Nehalem microarchitecture - </entry> + IBM S/390 and IBM System z + </entry> <entry> - i386/nehalem - </entry> + timer + </entry> <entry> - 4 - </entry> + 1 + </entry> </row> <row> <entry> - Intel Pentium 4 (non-hyper-threaded) - </entry> + Intel Core i7 + </entry> <entry> - i386/p4 - </entry> + i386/core_i7 + </entry> <entry> - 8 - </entry> + 4 + </entry> </row> <row> <entry> - Intel Pentium 4 (hyper-threaded) - </entry> + Intel Nehalem microarchitecture + </entry> <entry> - i386/p4-ht - </entry> + i386/nehalem + </entry> <entry> - 4 - </entry> + 4 + </entry> </row> <row> <entry> - Intel Westmere microarchitecture - </entry> + Intel Westmere microarchitecture + </entry> <entry> - i386/westmere - </entry> + i386/westmere + </entry> <entry> - 4 - </entry> + 4 + </entry> + </row> + <row> + <entry>Intel Haswell microarchitecture (non-hyper-threaded)</entry> + <entry>i386/haswell</entry> + <entry>8</entry> + </row> + <row> + <entry>Intel Haswell microarchitecture (hyper-threaded)</entry> + <entry>i386/haswell-ht</entry> + <entry>4</entry> + </row> + <row> + <entry>Intel Ivy Bridge microarchitecture (non-hyper-threaded)</entry> + <entry>i386/ivybridge</entry> + <entry>8</entry> + </row> + <row> + <entry>Intel Ivy Bridge microarchitecture (hyper-threaded)</entry> + <entry>i386/ivybridge-ht</entry> + <entry>4</entry> + </row> + <row> + <entry>Intel Sandy Bridge microarchitecture (non-hyper-threaded)</entry> + <entry>i386/sandybridge</entry> + <entry>8</entry> + </row> + <row> + <entry>Intel Sandy Bridge microarchitecture</entry> + <entry>i386/sandybridge-ht</entry> + <entry>4</entry> + </row> + <row> + <entry>Intel Broadwell microarchitecture (non-hyper-threaded)</entry> + <entry>i386/broadwell</entry> + <entry>8</entry> + </row> + <row> + <entry>Intel Broadwell microarchitecture (hyper-threaded)</entry> + <entry>i386/broadwell-ht</entry> + <entry>4</entry> + </row> + <row> + <entry>Intel Silvermont microarchitecture</entry> + <entry>i386/silvermont</entry> + <entry>2</entry> </row> <row> <entry> - TIMER_INT - </entry> + TIMER_INT + </entry> <entry> - timer - </entry> + timer + </entry> <entry> - 1 - </entry> + 1 + </entry> </row> </tbody> </tgroup> </table> <para>Use <xref - linkend="tb-oprofile-processors"/> to verify that the correct processor type was detected and to determine the number of events that can be monitored simultaneously. <computeroutput>timer</computeroutput> is used as the processor type if the processor does not have supported performance monitoring hardware.</para> + linkend="tb-oprofile-processors"/> to determine the number of events that can be monitored simultaneously for your CPU type. If the processor does not have supported performance monitoring hardware, the <computeroutput>timer</computeroutput> is used as the processor type.</para> <para>If <computeroutput>timer</computeroutput> is used, events cannot be set for any processor because the hardware does not have support for hardware performance counters. Instead, the timer interrupt is used for profiling.</para> - <para>If <computeroutput>timer</computeroutput> is not used as the processor type, the events monitored can be changed, and counter 0 for the processor is set to a time-based event by default. If more than one counter exists on the processor, the counters other than counter 0 are not set to an event by default. The default events monitored are shown in <xref + <para>If <computeroutput>timer</computeroutput> is not used as the processor type, the events monitored can be changed, and counter 0 for the processor is set to a time-based event by default. If more than one counter exists on the processor, the counters other than 0 are not set to an event by default. The default events monitored are shown in <xref linkend="tb-oprofile-default-events"/>.</para> <table id="tb-oprofile-default-events"> @@ -469,141 +692,203 @@ <thead> <row> <entry> - Processor - </entry> + Processor + </entry> <entry> - Default Event for Counter - </entry> + Default Event for Counter + </entry> <entry> - Description - </entry> + Description + </entry> </row> </thead> <tbody> <row> <entry> - AMD Athlon and AMD64 - </entry> + AMD Athlon and AMD64 + </entry> <entry> - CPU_CLK_UNHALTED - </entry> + CPU_CLK_UNHALTED + </entry> <entry> - The processor's clock is not halted - </entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - AMD Family 10h, AMD Family 11h, AMD Family 12h - </entry> + AMD Family 10h, AMD Family 11h, AMD Family 12h + </entry> <entry> - CPU_CLK_UNHALTED - </entry> + CPU_CLK_UNHALTED + </entry> <entry> - The processor's clock is not halted - </entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - AMD Family 14h, AMD Family 15h - </entry> + AMD Family 14h, AMD Family 15h + </entry> <entry> - CPU_CLK_UNHALTED - </entry> + CPU_CLK_UNHALTED + </entry> <entry> - The processor's clock is not halted - </entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - IBM POWER4 - </entry> + Applied Micro X-Gene + </entry> <entry> - CYCLES - </entry> + CPU_CYCLES + </entry> <entry> - Processor Cycles - </entry> + Processor Cycles + </entry> </row> <row> <entry> - IBM POWER5 - </entry> + ARM Cortex A53 + </entry> <entry> - CYCLES - </entry> + CPU_CYCLES + </entry> <entry> - Processor Cycles - </entry> + Processor Cycles + </entry> </row> <row> <entry> - IBM PowerPC 970 - </entry> + ARM Cortex A57 + </entry> <entry> - CYCLES - </entry> + CPU_CYCLES + </entry> <entry> - Processor Cycles - </entry> + Processor Cycles + </entry> </row> - - - - <row> <entry> - Intel Core i7 - </entry> + IBM POWER4 + </entry> <entry> - CPU_CLK_UNHALTED - </entry> + CYCLES + </entry> <entry> - The processor's clock is not halted - </entry> + Processor Cycles + </entry> + </row> + <row> + <entry> + IBM POWER5 + </entry> + <entry> + CYCLES + </entry> + <entry> + Processor Cycles + </entry> + </row> + <row> + <entry> + IBM POWER8 + </entry> + <entry> + CYCLES + </entry> + <entry> + Processor Cycles + </entry> + </row> + <row> + <entry> + IBM PowerPC 970 + </entry> + <entry> + CYCLES + </entry> + <entry> + Processor Cycles + </entry> + </row> + <row> + <entry> + Intel Core i7 + </entry> + <entry> + CPU_CLK_UNHALTED + </entry> + <entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - Intel Nehalem microarchitecture - </entry> + Intel Nehalem microarchitecture + </entry> <entry> - CPU_CLK_UNHALTED - </entry> + CPU_CLK_UNHALTED + </entry> <entry> - The processor's clock is not halted - </entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - Intel Pentium 4 (hyper-threaded and non-hyper-threaded) - </entry> + Intel Pentium 4 (hyper-threaded and non-hyper-threaded) + </entry> + <entry> + GLOBAL_POWER_EVENTS + </entry> + <entry> + The time during which the processor is not stopped + </entry> + </row> + <row> + <entry> + Intel Westmere microarchitecture + </entry> + <entry> + CPU_CLK_UNHALTED + </entry> + <entry> + The processor's clock is not halted + </entry> + </row> + <row> + <entry> + Intel Broadwell microarchitecture + </entry> <entry> - GLOBAL_POWER_EVENTS - </entry> + CPU_CLK_UNHALTED + </entry> <entry> - The time during which the processor is not stopped - </entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - Intel Westmere microarchitecture - </entry> + Intel Silvermont microarchitecture + </entry> <entry> - CPU_CLK_UNHALTED - </entry> + CPU_CLK_UNHALTED + </entry> <entry> - The processor's clock is not halted - </entry> + The processor's clock is not halted + </entry> </row> <row> <entry> - TIMER_INT - </entry> + TIMER_INT + </entry> <entry> - (none) - </entry> + (none) + </entry> <entry> - Sample for each timer interrupt - </entry> + Sample for each timer interrupt + </entry> </row> </tbody> </tgroup> @@ -611,21 +896,30 @@ <para>The number of events that can be monitored at one time is determined by the number of counters for the processor. However, it is not a one-to-one correlation; on some processors, certain events must be mapped to specific counters. To determine the number of counters available, execute the following command:</para> <screen>~]# <command>ls -d /dev/oprofile/[0-9]*</command></screen> <para>The events available vary depending on the processor type. To determine the events available for profiling, execute the following command as root (the list is specific to the system's processor type):</para> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary> <command>ophelp</command> </secondary> </indexterm> - <indexterm - significance="normal"> - <primary> - <command>ophelp</command> - </primary> + <indexterm> + <primary><command>ophelp</command></primary> </indexterm> <screen>~]# <command>ophelp</command></screen> - <para>The events for each counter can be configured via the command line or with a graphical interface. For more information on the graphical interface, refer to <xref + <note> + <title>Make sure that OProfile is configured</title> + <para> + Unless OProfile is properly configured, <command>ophelp</command> fails with the following error message: + </para> + <screen>Unable to open cpu_type file for reading +Make sure you have done opcontrol --init +cpu_type 'unset' is not valid +you should upgrade oprofile or force the use of timer mode</screen> + <para> + To configure OProfile, follow the instructions in <xref linkend="s1-oprofile-configuring" />. + </para> + </note> + <para>The events for each counter can be configured via the command line or with a graphical interface. For more information on the graphical interface, see <xref linkend="s1-oprofile-gui"/>. If the counter cannot be set to a specific event, an error message is displayed.</para> <para>To set the event for each configurable counter via the command line, use <command>opcontrol</command>:</para> <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable></command></screen> @@ -633,39 +927,39 @@ <section id="s3-oprofile-events-sampling"> <title>Sampling Rate</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>events</secondary> <tertiary>sampling rate</tertiary> </indexterm> - <para>By default, a time-based event set is selected. It creates a sample every 100,000 clock cycles per processor. If the timer interrupt is used, the timer is set to whatever the jiffy rate is and is not user-settable. If the <computeroutput>cpu_type</computeroutput> is not <computeroutput>timer</computeroutput>, each event can have a <firstterm>sampling rate</firstterm> set for it. The sampling rate is the number of events between each sample snapshot.</para> + <para>By default, a time-based event set is selected. It creates a sample every 100,000 clock cycles per processor. If the timer interrupt is used, the timer is set to the respective rate and is not user-settable. If the <computeroutput>cpu_type</computeroutput> is not <computeroutput>timer</computeroutput>, each event can have a <firstterm>sampling rate</firstterm> set for it. The sampling rate is the number of events between each sample snapshot.</para> <para>When setting the event for the counter, a sample rate can also be specified:</para> <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable></command></screen> <para>Replace <replaceable>sample-rate</replaceable> with the number of events to wait before sampling again. The smaller the count, the more frequent the samples. For events that do not happen frequently, a lower count may be needed to capture the event instances.</para> <warning> <title>Sampling too frequently can overload the system</title> - <para>Be extremely careful when setting sampling rates. Sampling too frequently can overload the system, causing the system to appear as if it is frozen or causing the system to actually freeze.</para> + <para>Be extremely careful when setting sampling rates. Sampling too frequently can overload the system, causing the system to appear frozen or causing the system to actually freeze.</para> </warning> </section> <section id="s3-oprofile-events-unit-masks"> <title>Unit Masks</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>unit mask</secondary> </indexterm> <para>Some user performance monitoring events may also require unit masks to further define the event.</para> <para>Unit masks for each event are listed with the <command>ophelp</command> command. The values for each unit mask are listed in hexadecimal format. To specify more than one unit mask, the hexadecimal values must be combined using a bitwise <firstterm>or</firstterm> operation.</para> <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable></command></screen> + <para> + Note that on certain architectures, there can be multiple unit masks with the same hexadecimal value. In that case they have to be specified by their names only. + </para> </section> </section> <section id="s2-oprofile-starting-separate"> <title>Separating Kernel and User-space Profiles</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>configuring</secondary> <tertiary>separating profiles</tertiary> @@ -675,29 +969,33 @@ <para>Execute the following command to start profiling kernel mode for the counter again:</para> <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:1</command></screen> <para>To configure OProfile to ignore events in user mode for a specific counter, execute the following command:</para> - <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:<replaceable>kernel</replaceable>:0</command></screen> + <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:1:0</command></screen> <para>Execute the following command to start profiling user mode for the counter again:</para> - <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:<replaceable>kernel</replaceable>:1</command></screen> + <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:1:1</command></screen> <para>When the OProfile daemon writes the profile data to sample files, it can separate the kernel and library profile data into separate sample files. To configure how the daemon writes to sample files, execute the following command as root:</para> <screen>~]# <command>opcontrol --separate=<replaceable>choice</replaceable></command></screen> <para> - <replaceable>choice</replaceable> can be one of the following:</para> + The <replaceable>choice</replaceable> argument can be one of the following:</para> <itemizedlist> <listitem> <para> - <computeroutput>none</computeroutput> — do not separate the profiles (default)</para> + <computeroutput>none</computeroutput> — Do not separate the profiles (default). + </para> </listitem> <listitem> <para> - <command>library</command> — generate per-application profiles for libraries</para> + <command>library</command> — Generate per-application profiles for libraries. + </para> </listitem> <listitem> <para> - <command>kernel</command> — generate per-application profiles for the kernel and kernel modules</para> + <command>kernel</command> — Generate per-application profiles for the kernel and kernel modules. + </para> </listitem> <listitem> <para> - <command>all</command> — generate per-application profiles for libraries and per-application profiles for the kernel and kernel modules</para> + <command>all</command> — Generate per-application profiles for libraries and per-application profiles for the kernel and kernel modules. + </para> </listitem> </itemizedlist> <para>If <option>--separate=library</option> is used, the sample file name includes the name of the executable as well as the name of the library.</para> @@ -709,15 +1007,13 @@ </section> <section id="s1-oprofile-starting"> - <title>Starting and Stopping OProfile</title> - <indexterm - significance="normal"> + <title>Starting and Stopping OProfile Using Legacy Mode</title> + <indexterm> <primary>OProfile</primary> <secondary>starting</secondary> </indexterm> <para>To start monitoring the system with OProfile, execute the following command as root:</para> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary> <command>opcontrol</command> @@ -730,62 +1026,65 @@ <para>Output similar to the following is displayed:</para> <screen>Using log file /var/lib/oprofile/oprofiled.log Daemon started. Profiler running.</screen> <para>The settings in <filename>/root/.oprofile/daemonrc</filename> are used.</para> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary> <command>oprofiled</command> </secondary> </indexterm> - <indexterm - significance="normal"> - <primary> - <command>oprofiled</command> - </primary> + <indexterm> + <primary><command>oprofiled</command></primary> <see>OProfile</see> </indexterm> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary> <command>oprofiled</command> </secondary> <tertiary>log file</tertiary> </indexterm> - <para>The OProfile daemon, <command>oprofiled</command>, is started; it periodically writes the sample data to the <filename>/var/lib/oprofile/samples/</filename> directory. The log file for the daemon is located at <filename>/var/lib/oprofile/oprofiled.log</filename>.</para> + <para>The OProfile daemon, <command>oprofiled</command>, is started; it periodically writes the sample data to the <filename class="directory">/var/lib/oprofile/samples/</filename> directory. The log file for the daemon is located at <filename>/var/lib/oprofile/oprofiled.log</filename>.</para> <important> <title>Disable the nmi_watchdog registers</title> <para>On a &MAJOROSVER; system, the <computeroutput>nmi_watchdog</computeroutput> registers with the <computeroutput>perf</computeroutput> subsystem. Due to this, the <computeroutput>perf</computeroutput> subsystem grabs control of the performance counter registers at boot time, blocking OProfile from working.</para> - <para>To resolve this, either boot with the <command>nmi_watchdog=0</command> kernel parameter set, or run the following command to disable <computeroutput>nmi_watchdog</computeroutput> at run time:</para> - <screen>~]# <command>echo 0 > /proc/sys/kernel/nmi_watchdog</command></screen> - <para>To re-enable <computeroutput>nmi_watchdog</computeroutput>, use the following command:</para> - <screen>~]# <command>echo 1 > /proc/sys/kernel/nmi_watchdog</command></screen> + <para>To resolve this, either boot with the <command>nmi_watchdog=0</command> kernel parameter set, or run the following command as <systemitem class="username">root</systemitem> to disable <computeroutput>nmi_watchdog</computeroutput> at run time:</para> + <screen>~]# <command>echo 0 > /proc/sys/kernel/nmi_watchdog</command></screen> + <para>To re-enable <computeroutput>nmi_watchdog</computeroutput>, use the following command as <systemitem class="username">root</systemitem>:</para> + <screen>~]# <command>echo 1 > /proc/sys/kernel/nmi_watchdog</command></screen> </important> <para>To stop the profiler, execute the following command as root:</para> <screen>~]# <command>opcontrol --shutdown</command></screen> </section> <section id="s1-oprofile-saving-data"> - <title>Saving Data</title> - <indexterm - significance="normal"> + <title>Saving Data in Legacy Mode</title> + <indexterm> <primary>OProfile</primary> <secondary>saving data</secondary> </indexterm> <para>Sometimes it is useful to save samples at a specific time. For example, when profiling an executable, it may be useful to gather different samples based on different input data sets. If the number of events to be monitored exceeds the number of counters available for the processor, multiple runs of OProfile can be used to collect data, saving the sample data to different files each time.</para> - <para>To save the current set of sample files, execute the following command, replacing <replaceable>name</replaceable> with a unique descriptive name for the current session.</para> + <para>To save the current set of sample files, execute the following command, replacing <replaceable>name</replaceable> with a unique descriptive name for the current session:</para> <screen>~]# <command>opcontrol --save=<replaceable>name</replaceable></command></screen> - <para>The directory <filename>/var/lib/oprofile/samples/<replaceable>name</replaceable>/</filename> is created and the current sample files are copied to it.</para> + <para>The command creates the directory <filename>/var/lib/oprofile/samples/<replaceable>name</replaceable>/</filename> and the current sample files are copied to it.</para> + <para> + To specify the session directory to hold the sample data, use the <option>--session-dir</option> option. If not specified, the data is saved in the <filename class="directory">oprofile_data/</filename> directory on the current path. + </para> </section> <section id="s1-oprofile-analyzing-data"> <title>Analyzing the Data</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>reading data</secondary> </indexterm> - <para>Periodically, the OProfile daemon, <command>oprofiled</command>, collects the samples and writes them to the <filename>/var/lib/oprofile/samples/</filename> directory. Before reading the data, make sure all data has been written to this directory by executing the following command as root:</para> + <para> + The same OProfile post-processing tools are used whether you collect your profile with <command>operf</command> or <command>opcontrol</command> in legacy mode. + </para> + <para> + By default, <command>operf</command> stores the profiling data in the <filename class="directory"><replaceable>current_dir</replaceable>/oprofile_data/</filename> directory. You can change to a different location with the <option>--session-dir</option> option. The usual post-profiling analysis tools such as <command>opreport</command> and <command>opannotate</command> can be used to generate profile reports. These tools search for samples in <filename class="directory"><replaceable>current_dir</replaceable>/oprofile_data/</filename> first. If this directory does not exist, the analysis tools use the standard session directory of <filename class="directory">/var/lib/oprofile/</filename>. Statistics, such as total samples received and lost samples, are written to the <filename><replaceable>session_dir</replaceable>/samples/operf.log</filename> file. + </para> + <para> + When using legacy mode, the OProfile daemon, <command>oprofiled</command>, periodically collects the samples and writes them to the <filename>/var/lib/oprofile/samples/</filename> directory. Before reading the data, make sure all data has been written to this directory by executing the following command as root:</para> <screen>~]# <command>opcontrol --dump</command></screen> <para>Each sample file name is based on the name of the executable. For example, the samples for the default event on a Pentium III processor for <command>/bin/bash</command> becomes:</para> <screen>\{root\}/bin/bash/\{dep\}/\{root\}/bin/bash/CPU_CLK_UNHALTED.100000</screen> @@ -805,31 +1104,26 @@ <para>Use these tools, along with the binaries profiled, to generate reports that can be further analyzed.</para> <warning> <title>Back up the executable and the sample files</title> - <para>The executable being profiled must be used with these tools to analyze the data. If it must change after the data is collected, back up the executable used to create the samples as well as the sample files. Please note that the sample file and the binary have to agree. Making a backup is not going to work if they do not match. <command>oparchive</command> can be used to address this problem.</para> + <para>The executable being profiled must be used with these tools to analyze the data. If it must change after the data is collected, back up the executable used to create the samples as well as the sample files. Note that the names of the sample file and the binary have to agree. You cannot make a backup if these names do not match. As an alternative, <command>oparchive</command> can be used to address this problem.</para> </warning> - <para>Samples for each executable are written to a single sample file. Samples from each dynamically linked library are also written to a single sample file. While OProfile is running, if the executable being monitored changes and a sample file for the executable exists, the existing sample file is automatically deleted. Thus, if the existing sample file is needed, it must be backed up, along with the executable used to create it before replacing the executable with a new version. The OProfile analysis tools use the executable file that created the samples during analysis. If the executable changes the analysis tools will be unable to analyze the associated samples. See <xref + <para>Samples for each executable are written to a single sample file. Samples from each dynamically linked library are also written to a single sample file. While OProfile is running, if the executable being monitored changes and a sample file for the executable exists, the existing sample file is automatically deleted. Thus, if the existing sample file is needed, it must be backed up, along with the executable used to create it before replacing the executable with a new version. The OProfile analysis tools use the executable file that created the samples during analysis. If the executable changes, the analysis tools will be unable to analyze the associated samples. See <xref linkend="s1-oprofile-saving-data"/> for details on how to back up the sample file.</para> <section id="s2-oprofile-reading-opreport"> - <title>Using <command>opreport</command> - </title> - <indexterm - significance="normal"> + <title>Using <command>opreport</command></title> + <indexterm> <primary>OProfile</primary> <secondary> <command>opreport</command> </secondary> </indexterm> - <indexterm - significance="normal"> - <primary> - <command>opreport</command> - </primary> + <indexterm> + <primary><command>opreport</command></primary> <see>OProfile</see> </indexterm> - <para>The <command>opreport</command> tool provides an overview of all the executables being profiled.</para> - <para>The following is part of a sample output:</para> - <screen>Profiling through timer interrupt + <para>The <command>opreport</command> tool provides an overview of all the executables being profiled. The following is part of a sample output from the <command>opreport</command> command:</para> + <screen>~]$ <command>opreport</command> +Profiling through timer interrupt TIMER:0| samples| %| ------------------ @@ -855,39 +1149,42 @@ samples| %| 1 0.0038 libX11.so.6.2 1 0.0038 libgthread-2.0.so.0.400.7 1 0.0038 libwnck-1.so.4.9.0</screen> - <para>Each executable is listed on its own line. The first column is the number of samples recorded for the executable. The second column is the percentage of samples relative to the total number of samples. The third column is the name of the executable.</para> - <para>See the <command>opreport</command> man page for a list of available command line options, such as the <option>-r</option> option used to sort the output from the executable with the smallest number of samples to the one with the largest number of samples.</para> + <para> + Each executable is listed on its own line. The first column is the number of samples recorded for the executable. The second column is the percentage of samples relative to the total number of samples. The third column is the name of the executable. + </para> + <para> + See the <systemitem>opreport(1)</systemitem> manual page for a list of available command-line options, such as the <option>-r</option> option used to sort the output from the executable with the smallest number of samples to the one with the largest number of samples. You can also use the <option>-t</option> or <option>--threshold</option> option to trim the output of <command>opcontrol</command>. + </para> </section> <section id="s2-oprofile-reading-opreport-single"> <title>Using opreport on a Single Executable</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary> <command>opreport</command> </secondary> <tertiary>on a single executable</tertiary> </indexterm> - <indexterm - significance="normal"> - <primary> - <command>opreport</command> - </primary> + <indexterm> + <primary><command>opreport</command></primary> <see>OProfile</see> </indexterm> - <para>To retrieve more detailed profiled information about a specific executable, use <command>opreport</command>:</para> + <para>To retrieve more detailed profiled information about a specific executable, use the <command>opreport</command> command:</para> <screen>~]# <command>opreport <replaceable>mode</replaceable> <replaceable>executable</replaceable></command></screen> <para> - <replaceable>executable</replaceable> must be the full path to the executable to be analyzed. <replaceable>mode</replaceable> must be one of the following:</para> + Replace <replaceable>executable</replaceable> with the full path to the executable to be analyzed. <replaceable>mode</replaceable> stands for one of the following options:</para> <variablelist> <varlistentry> <term> <option>-l</option> </term> <listitem> - <para>List sample data by symbols. For example, the following is part of the output from running the command <command>opreport -l /lib/tls/libc-<replaceable>version</replaceable>.so</command>:</para> + <para>This option is used to list sample data by symbols. For example, running this command: + <synopsis>~]# <command>opreport -l /lib/tls/libc-<replaceable>version</replaceable>.so</command></synopsis> + </para> + <para> produces the following output:</para> <screen>samples % symbol name 12 21.4286 __gconv_transform_utf8_internal 5 8.9286 _int_malloc 4 7.1429 malloc @@ -924,7 +1221,12 @@ samples| %| </option> </term> <listitem> - <para>List sample data specific to a symbol name. For example, the following output is from the command <command>opreport -l -i __gconv_transform_utf8_internal /lib/tls/libc-<replaceable>version</replaceable>.so</command>:</para> + <para>List sample data specific to a symbol name. For example, running: + <synopsis>~]# <command>opreport -l -i __gconv_transform_utf8_internal /lib/tls/libc-<replaceable>version</replaceable>.so</command></synopsis> + </para> + <para> + returns the following output: + </para> <screen>samples % symbol name 12 100.000 __gconv_transform_utf8_internal</screen> <para>The first line is a summary for the symbol/executable combination.</para> @@ -936,7 +1238,12 @@ samples| %| <option>-d</option> </term> <listitem> - <para>List sample data by symbols with more detail than <option>-l</option>. For example, the following output is from the command <command>opreport -l -d __gconv_transform_utf8_internal /lib/tls/libc-<replaceable>version</replaceable>.so</command>:</para> + <para>This option lists sample data by symbols with more detail than the <option>-l</option> option. For example, with the following command: + <synopsis>~]# <command>opreport -d -i __gconv_transform_utf8_internal /lib/tls/libc-<replaceable>version</replaceable>.so</command></synopsis> + </para> + <para> + this output is returned: + </para> <screen>vma samples % symbol name 00a98640 12 100.000 __gconv_transform_utf8_internal 00a98640 1 8.3333 @@ -955,11 +1262,11 @@ samples| %| </varlistentry> <varlistentry> <term> - <option>-x</option> - <replaceable>symbol-name</replaceable> + <option>-e</option> + <replaceable>symbol-name</replaceable>… </term> <listitem> - <para>Exclude the comma-separated list of symbols from the output.</para> + <para>With this option, you can exclude some symbols from the output. Replace <replaceable>symbol-name</replaceable> with the comma-separated list of symbols you want to exclude.</para> </listitem> </varlistentry> <varlistentry> @@ -967,27 +1274,25 @@ samples| %| <option>session</option>:<replaceable>name</replaceable> </term> <listitem> - <para>Specify the full path to the session or a directory relative to the <filename>/var/lib/oprofile/samples/</filename> directory.</para> + <para>Here, you can specify the full path to the session, a directory relative to the <filename class="directory">/var/lib/oprofile/samples/</filename> directory, or if you are using <command>operf</command>, a directory relative to <filename class="directory">./oprofile_data/samples/</filename>.</para> </listitem> </varlistentry> </variablelist> </section> <section id="s2-oprofile-module-output"> - <title>Getting more detailed output on the modules</title> - <indexterm - significance="normal"> + <title>Getting More Detailed Output on the Modules</title> + <indexterm> <primary>OProfile</primary> <secondary> <command>opreport</command> </secondary> </indexterm> - <indexterm - significance="normal"> + <indexterm> <primary/> <see>OProfile</see> </indexterm> - <para>OProfile collects data on a system-wide basis for kernel- and user-space code running on the machine. However, once a module is loaded into the kernel, the information about the origin of the kernel module is lost. The module could have come from the <filename>initrd</filename> file on boot up, the directory with the various kernel modules, or a locally created kernel module. As a result, when OProfile records sample for a module, it just lists the samples for the modules for an executable in the root directory, but this is unlikely to be the place with the actual code for the module. You will need to take some steps to make sure that analysis tools get the executable.</para> + <para>OProfile collects data on a system-wide basis for kernel- and user-space code running on the machine. However, once a module is loaded into the kernel, the information about the origin of the kernel module is lost. The module could come from the <filename>initrd</filename> file on boot up, the directory with the various kernel modules, or a locally created kernel module. As a result, when OProfile records samples for a module, it just lists the samples for the modules for an executable in the root directory, but this is unlikely to be the place with the actual code for the module. You will need to take some steps to make sure that analysis tools get the proper executable.</para> <para>To get a more detailed view of the actions of the module, you will need to either have the module "unstripped" (that is installed from a custom build) or have the <package>debuginfo</package> package installed for the kernel.</para> <para>Find out which kernel is running with the <command>uname -a</command> command, obtain the appropriate <package>debuginfo</package> package and install it on the machine.</para> <para>Then proceed with clearing out the samples from previous runs with the following command:</para> @@ -1015,49 +1320,40 @@ samples % symbol name </section> <section id="s2-oprofile-reading-opannotate"> - <title>Using <command>opannotate</command> - </title> - <indexterm - significance="normal"> + <title>Using <command>opannotate</command></title> + <indexterm> <primary>OProfile</primary> <secondary> <command>opannotate</command> </secondary> </indexterm> - <indexterm - significance="normal"> - <primary> - <command>opannotate</command> - </primary> + <indexterm> + <primary><command>opannotate</command></primary> <see>OProfile</see> </indexterm> - <para>The <command>opannotate</command> tool tries to match the samples for particular instructions to the corresponding lines in the source code. The resulting files generated should have the samples for the lines at the left. It also puts in a comment at the beginning of each function listing the total samples for the function.</para> - <para>For this utility to work, the appropriate <package>debuginfo</package> package for the executable must be installed on the system. By default, &MAJOROS; <package>debuginfo</package> packages are not installed together with their corresponding packages, which contain the executable, so that you have to obtain and install the <package>debuginfo</package> packages separately.</para> + <para>The <command>opannotate</command> tool tries to match the samples for particular instructions to the corresponding lines in the source code. The resulting generated files should have the samples for the lines at the left. It also puts in a comment at the beginning of each function listing the total samples for the function.</para> + <para>For this utility to work, the appropriate <package>debuginfo</package> package for the executable must be installed on the system. On &MAJOROS;, the <package>debuginfo</package> packages are not automatically installed with the corresponding packages that contain the executable. You have to obtain and install them separately.</para> <para>The general syntax for <command>opannotate</command> is as follows:</para> <screen>~]# <command>opannotate --search-dirs <replaceable>src-dir</replaceable> --source <replaceable>executable</replaceable></command></screen> - <para>The directory containing the source code and the executable to be analyzed must be specified. See the <command>opannotate</command> man page for a list of additional command line options.</para> + <para>These command-line options are mandatory. Replace <replaceable>src-dir</replaceable> with a path to the directory containing the source code and specify the executable to be analyzed. See the <systemitem>opannotate(1)</systemitem> manual page for a list of additional command line options.</para> </section> </section> <section id="s1-oprofile-dev-oprofile"> - <title>Understanding <filename>/dev/oprofile/</filename> - </title> - <indexterm - significance="normal"> + <title>Understanding the /dev/oprofile/ directory +</title> + <indexterm> <primary>OProfile</primary> - <secondary> - <filename>/dev/oprofile/</filename> - </secondary> + <secondary><filename class="directory">/dev/oprofile/</filename></secondary> </indexterm> - <indexterm - significance="normal"> - <primary> - <filename>/dev/oprofile/</filename> - </primary> + <indexterm> + <primary><filename class="directory">/dev/oprofile/</filename></primary> </indexterm> - <para>The <filename>/dev/oprofile/</filename> directory contains the file system for OProfile. Use the <command>cat</command> command to display the values of the virtual files in this file system. For example, the following command displays the type of processor OProfile detected:</para> - <screen>~]# <command>cat /dev/oprofile/cpu_type</command></screen> - <para>A directory exists in <filename>/dev/oprofile/</filename> for each counter. For example, if there are 2 counters, the directories <filename>/dev/oprofile/0/</filename> and <filename>dev/oprofile/1/</filename> exist.</para> + <para> + When using OProfile in legacy mode, the <filename class="directory">/dev/oprofile/</filename> directory is used to store the file system for OProfile. On the other hand, <command>operf</command> does not require <filename class="directory">/dev/oprofile/</filename>. Use the <command>cat</command> command to display the values of the virtual files in this file system. For example, the following command displays the type of processor OProfile detected: + </para> + <screen>~]# <command>cat /dev/oprofile/cpu_type</command></screen> + <para>A directory exists in <filename class="directory">/dev/oprofile/</filename> for each counter. For example, if there are 2 counters, the directories <filename class="directory">/dev/oprofile/0/</filename> and <filename class="directory">/dev/oprofile/1/</filename> exist.</para> <para>Each directory for a counter contains the following files:</para> <itemizedlist> <listitem> @@ -1099,7 +1395,7 @@ samples % symbol name <itemizedlist> <listitem> <para> - <emphasis>Determine which applications and services are used the most on a system</emphasis> — <command>opreport</command> can be used to determine how much processor time an application or service uses. If the system is used for multiple services but is under performing, the services consuming the most processor time can be moved to dedicated systems.</para> + <emphasis>Determine which applications and services are used the most on a system</emphasis> — <command>opreport</command> can be used to determine how much processor time an application or service uses. If the system is used for multiple services but is underperforming, the services consuming the most processor time can be moved to dedicated systems.</para> </listitem> <listitem> <para> @@ -1110,35 +1406,44 @@ samples % symbol name <section id="s1-oprofile-java-support"> <title>OProfile Support for Java</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>Java</secondary> </indexterm> - <para>OProfile allows you to profile dynamically compiled code (also known as "just-in-time" or JIT code) of the Java Virtual Machine (JVM). OProfile in &MAJOROSVER; includes build-in support for the JVM Tools Interface (JVMTI) agent library, which supports Java 1.5 and higher.</para> + <para>OProfile allows you to profile dynamically compiled code (also known as "just-in-time" or JIT code) of the Java Virtual Machine (JVM). OProfile in &MAJOROSVER; includes built-in support for the JVM Tools Interface (JVMTI) agent library, which supports Java 1.5 and higher.</para> <section id="s1-oprofile-java-profiling"> <title>Profiling Java Code</title> <para>To profile JIT code from the Java Virtual Machine with the JVMTI agent, add the following to the JVM startup parameters:</para> - <screen><option>-agentlib:jvmti_oprofile</option></screen> + <screen><option>-agentlib:</option><replaceable>jvmti_oprofile</replaceable></screen> + <para> + Where <replaceable>jvmti_oprofile</replaceable> is a path to the OProfile agent. For 64-bit JVM, the path looks as follows: + </para> + <screen><option>-agentlib:/usr/lib64/oprofile/libjvmti_oprofile.so</option></screen> + <para> + Currently, you can add one command-line option: <option>--debug</option>, which enables debugging mode. + </para> <note> <title>Install the oprofile-jit package</title> - <para>The <package>oprofile-jit</package> package must be installed on the system in order to profile JIT code with OProfile.</para> + <para>The <package>oprofile-jit</package> package must be installed on the system in order to profile JIT code with OProfile. With this package, you gain the capability to show method-level information.</para> </note> - <para>To learn more about Java support in OProfile, refer to the OProfile Manual, which is linked from <xref - linkend="s1-oprofile-additional-resources"/>.</para> + <para> + Depending on the JVM that you are using, you may have to install the <emphasis>debuginfo</emphasis> package for the JVM. For OpenJDK, this package is required, there is no debuginfo package for Oracle JDK. To keep the debug information packages synchronized with their respective non-debug packages, you also need to install the <emphasis>yum-plugin-auto-update-debug-info</emphasis> plug-in. This plug-in searches the debug information repository for corresponding updates. + </para> + <para> + After successful setup, you can apply the standard profiling and analyzing tools described in previous sections + </para> + <para>To learn more about Java support in OProfile, see the OProfile Manual, which is linked from <xref + linkend="s1-oprofile_additional_resources"/>.</para> </section> </section> <section id="s1-oprofile-gui"> <title>Graphical Interface</title> - <indexterm - significance="normal"> - <primary> - <command>oprof_start</command> - </primary> + <indexterm> + <primary><command>oprof_start</command></primary> </indexterm> - <para>Some OProfile preferences can be set with a graphical interface. To start it, execute the <command>oprof_start</command> command as root at a shell prompt. To use the graphical interface, you will need to have the <filename>oprofile-gui</filename> package installed.</para> + <para>Some OProfile preferences can be set with a graphical interface. Make sure you have the <filename>oprofile-gui</filename> package that provides the OProfile GUI installed on your system. To start the interface, execute the <command>oprof_start</command> command as root at a shell prompt.</para> <para>After changing any of the options, save them by clicking the <guibutton>Save and quit</guibutton> button. The preferences are written to <filename>/root/.oprofile/daemonrc</filename>, and the application exits. Exiting the application does not stop OProfile from sampling.</para> <para>On the <guilabel>Setup</guilabel> tab, to set events for the processor counters as discussed in <xref linkend="s2-oprofile-events"/>, select the counter from the pulldown menu and select the event from the list. A brief description of the event appears in the text box below the list. Only events available for the specific counter and the specific architecture are displayed. The interface also displays whether the profiler is running and some brief statistics about it.</para> @@ -1160,13 +1465,13 @@ samples % symbol name </mediaobject> </figure> <para>On the right side of the tab, select the <guilabel>Profile kernel</guilabel> option to count events in kernel mode for the currently selected event, as discussed in <xref - linkend="s2-oprofile-starting-separate"/>. If this option is unselected, no samples are collected for the kernel.</para> + linkend="s2-oprofile-starting-separate"/>. If this option is not selected, no samples are collected for the kernel.</para> <para>Select the <guilabel>Profile user binaries</guilabel> option to count events in user mode for the currently selected event, as discussed in <xref - linkend="s2-oprofile-starting-separate"/>. If this option is unselected, no samples are collected for user applications.</para> + linkend="s2-oprofile-starting-separate"/>. If this option is not selected, no samples are collected for user applications.</para> <para>Use the <guilabel>Count</guilabel> text field to set the sampling rate for the currently selected event as discussed in <xref linkend="s3-oprofile-events-sampling"/>.</para> <para>If any unit masks are available for the currently selected event, as discussed in <xref - linkend="s3-oprofile-events-unit-masks"/>, they are displayed in the <guilabel>Unit Masks</guilabel> area on the right side of the <guilabel>Setup</guilabel> tab. Select the checkbox beside the unit mask to enable it for the event.</para> + linkend="s3-oprofile-events-unit-masks"/>, they are displayed in the <guilabel>Unit Masks</guilabel> area on the right side of the <guilabel>Setup</guilabel> tab. Select the check box beside the unit mask to enable it for the event.</para> <para>On the <guilabel>Configuration</guilabel> tab, to profile the kernel, enter the name and location of the <filename>vmlinux</filename> file for the kernel to monitor in the <guilabel>Kernel image file</guilabel> text field. To configure OProfile not to monitor the kernel, select <guilabel>No kernel image</guilabel>.</para> <figure float="0" @@ -1184,7 +1489,7 @@ samples % symbol name </textobject> </mediaobject> </figure> - <para>If the <guilabel>Verbose</guilabel> option is selected, the <command>oprofiled</command> daemon log includes more information.</para> + <para>If the <guilabel>Verbose</guilabel> option is selected, the <command>oprofiled</command> daemon log includes more detailed information.</para> <para>If <guilabel>Per-application profiles</guilabel> is selected, OProfile generates per-application profiles for libraries. This is equivalent to the <command>opcontrol --separate=library</command> command. If <guilabel>Per-application profiles, including kernel</guilabel> is selected, OProfile generates per-application profiles for the kernel and kernel modules as discussed in <xref linkend="s2-oprofile-starting-separate"/>. This is equivalent to the <command>opcontrol --separate=kernel</command> command.</para> <para>To force data to be written to samples files as discussed in <xref @@ -1194,29 +1499,25 @@ samples % symbol name <section id="s1-oprofile-and-systemtap"> <title>OProfile and SystemTap</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>SystemTap</secondary> </indexterm> - <para>SystemTap is a tracing and probing tool that allows users to study and monitor the activities of the operating system in fine detail. It provides information similar to the output of tools like <command>netstat</command>, <command>ps</command>, <command>top</command>, and <command>iostat</command>; however, SystemTap is designed to provide more filtering and analysis options for collected information.</para> + <para>SystemTap is a tracing and probing tool that allows users to study and monitor the activities of the operating system in fine detail. It provides information similar to the output of tools like <command>netstat</command>, <command>ps</command>, <command>top</command>, and <command>iostat</command>; however, SystemTap is designed to provide more filtering and analysis options for the collected information.</para> <para>While using OProfile is suggested in cases of collecting data on where and why the processor spends time in a particular area of code, it is less usable when finding out why the processor stays idle.</para> - <para>You might want to use SystemTap when instrumenting specific places in code. Because SystemTap allows you to run the code instrumentation without having to stop and restart the instrumentation, it is particularly useful for instrumenting the kernel and daemons.</para> - <para>For more information on SystemTap, refer to <xref - linkend="s2-oprofile-useful-websites"/> for the relevant SystemTap documentation.</para> + <para>You might want to use SystemTap when instrumenting specific places in code. Because SystemTap allows you to run the code instrumentation without having to stop and restart the instrumented code, it is particularly useful for instrumenting the kernel and daemons.</para> + <para>For more information on SystemTap, see <xref + linkend="br-oprofile_online_documentation"/> for the relevant SystemTap documentation.</para> </section> <section - id="s1-oprofile-additional-resources"> + id="s1-oprofile_additional_resources"> <title>Additional Resources</title> - <indexterm - significance="normal"> + <indexterm> <primary>OProfile</primary> <secondary>additional resources</secondary> </indexterm> - <para>This chapter only highlights OProfile and how to configure and use it. To learn more, refer to the following resources.</para> - <section - id="s1-oprofile-installed-docs"> - <title>Installed Docs</title> + <para>To learn more about OProfile and how to configure it, see the following resources.</para> + <bridgehead id="br-oprofile_installed_documentation">Installed Documentation</bridgehead> <itemizedlist> <listitem> <para> @@ -1225,26 +1526,31 @@ samples % symbol name </listitem> <listitem> <para> - <command>oprofile</command> man page — Discusses <command>opcontrol</command>, <command>opreport</command>, <command>opannotate</command>, and <command>ophelp</command> + <filename>oprofile(1)</filename> manual page — Discusses <command>opcontrol</command>, <command>opreport</command>, <command>opannotate</command>, and <command>ophelp</command> </para> </listitem> + <listitem> + <para> + <filename>operf(1)</filename> manual page + </para> + </listitem> </itemizedlist> - </section> - <section - id="s2-oprofile-useful-websites"> - <title>Useful Websites</title> + <bridgehead id="br-oprofile_online_documentation">Online Documentation</bridgehead> <itemizedlist> <listitem> <para> <ulink - url="http://oprofile.sourceforge.net/">http://oprofile.sourceforge.net/</ulink> — Contains the latest documentation, mailing lists, IRC channels, and more.</para> + url="http://oprofile.sourceforge.net/">http://oprofile.sourceforge.net/</ulink> — Contains the latest upstream documentation, mailing lists, IRC channels, and more.</para> </listitem> + </itemizedlist> + <bridgehead id="br-oprofile_see_also">See Also</bridgehead> + <itemizedlist> <listitem> <para> <ulink - url="http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/SystemT...">SystemTap Beginners Guide</ulink> — Provides basic instructions on how to use SystemTap to monitor different subsystems of &MAJOROS; in finer detail.</para> + url="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...">SystemTap Beginners Guide</ulink> — Provides basic instructions on how to use SystemTap to monitor different subsystems of &MAJOROS; in finer detail.</para> + </listitem> </itemizedlist> - </section> </section> </chapter>

9 years, 5 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

docs-commits December 2014