Manual¶

NAME¶

BeakerLib - a shell-level integration testing library

DESCRIPTION¶

BeakerLib is a shell-level integration testing library, providing convenience functions which simplify writing, running and analysis of integration and blackbox tests.

The essential features include:

Journal - uniform logging mechanism (logs & results saved in flexible XML format, easy to compare results & generate reports)
Phases - logical grouping of test actions, clear separation of setup / test / cleanup (preventing false fails)
Asserts - common checks affecting the overall results of the individual phases (checking for exit codes, file existence & content…)
Helpers - convenience functions for common operations such as managing services, backup & restore

The main script sets the BEAKERLIB variable and sources other scripts where the actual functions are defined. You should source it at the beginning of your test with:

. /usr/share/beakerlib/beakerlib.sh

See the EXAMPLES section for quick start inspiration.

See the BKRDOC section for more information about Automated documentation generator for BeakerLib tests.

Result fingerprint¶

Beakerlib provides two types of fingerprint of the test run. The fingerprints are counted at the phase end and journal end.

The first one is a phases fingerprint which represents a list of the results of all the executed phases in the order of the execution.

The second one is a asserts fingerprint which represents a list of the results of all the executed asserts in the order of the execution.

The lists are then hashed resulting in a 8 letters string.

Such fingerprint may be used as a reference result of the respective test run in a particular environment.

FUNCTIONS¶

Journalling¶

rlJournalStart¶

Initialize the journal file.

rlJournalStart

Run on the very beginning of your script to initialize journalling functionality.

rlJournalEnd¶

Summarize the test run and upload the journal file.

rlJournalEnd

Run on the very end of your script to print summary of the whole test run, generate OUTPUTFILE and include journal in Beaker logs.

rlJournalPrint¶

Print the content of the journal in pretty xml format.

rlJournalPrint [type]

type

Can be either ‘raw’ or ‘pretty’, with the latter as a default. Raw: xml is in raw form, no indentation etc Pretty: xml is pretty printed, indented, with one record per line

Example:

<?xml version="1.0"?>
<BEAKER_TEST>
  <test_id>debugging</test_id>
  <package>setup</package>
  <pkgdetails>setup-2.8.9-1.fc12.noarch</pkgdetails>
  <starttime>2010-02-08 15:17:47</starttime>
  <endtime>2010-02-08 15:17:47</endtime>
  <testname>/examples/beakerlib/Sanity/simple</testname>
  <release>Fedora release 12 (Constantine)</release>
  <hostname>localhost</hostname>
  <arch>i686</arch>
  <purpose>PURPOSE of /examples/beakerlib/Sanity/simple
    Description: Minimal BeakerLib sanity test
    Author: Petr Splichal &lt;psplicha@redhat.com&gt;

    This is a minimal sanity test for BeakerLib. It contains a single
    phase with a couple of asserts. We Just check that the "setup"
    package is installed and that there is a sane /etc/passwd file.
  </purpose>
  <log>
    <phase endtime="2010-02-08 15:17:47" name="Test" result="PASS"
            score="0" starttime="2010-02-08 15:17:47" type="FAIL">
      <test message="Checking for the presence of setup rpm">PASS</test>
      <test message="File /etc/passwd should exist">PASS</test>
      <test message="File '/etc/passwd' should contain 'root'">PASS</test>
    </phase>
  </log>
</BEAKER_TEST>

rlJournalPrintText¶

Print the content of the journal in pretty text format.

rlJournalPrintText [--full-journal]

–full-journal

The option is now deprecated, has no effect and will be removed in a future version.

Example:

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: TEST PROTOCOL
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   LOG    ] :: Test run ID   : debugging
:: [   LOG    ] :: Package       : debugging
:: [   LOG    ] :: Test started  : 2010-02-08 14:45:57
:: [   LOG    ] :: Test finished : 2010-02-08 14:45:58
:: [   LOG    ] :: Test name     :
:: [   LOG    ] :: Distro:       : Fedora release 12 (Constantine)
:: [   LOG    ] :: Hostname      : localhost
:: [   LOG    ] :: Architecture  : i686

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Test description
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

PURPOSE of /examples/beakerlib/Sanity/simple
Description: Minimal BeakerLib sanity test
Author: Petr Splichal <psplicha@redhat.com>

This is a minimal sanity test for BeakerLib. It contains a single
phase with a couple of asserts. We Just check that the "setup"
package is installed and that there is a sane /etc/passwd file.


::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Test
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   PASS   ] :: Checking for the presence of setup rpm
:: [   PASS   ] :: File /etc/passwd should exist
:: [   PASS   ] :: File '/etc/passwd' should contain 'root'
:: [   LOG    ] :: Duration: 1s
:: [   LOG    ] :: Assertions: 3 good, 0 bad
:: [   PASS   ] :: RESULT: Test

rlGetTestState¶

Returns number of failed asserts so far, or 255 if there are more than 255 failures. The variable ECODE is set to the precise number.

rlGetTestState

rlGetPhaseState¶

Returns number of failed asserts in the current phase so far, or 255 if there are more than 255 failures. The variable ECODE is set to the precise number.

rlGetPhaseState

Logging¶

rlLog¶

rlLogDebug¶

rlLogInfo¶

rlLogWarning¶

rlLogError¶

rlLogFatal¶

Create a time/priority-labelled message in the log. There is a bunch of aliases which can create messages formated as DEBUG/INFO/WARNING/ERROR or FATAL (but you would probably want to use rlDie instead of the last one).

rlLog message [logfile] [priority] [label]

message

Message you want to show (use quotes when invoking).
logfile

Log file. If not supplied, OUTPUTFILE is assumed.
priority

Priority of the log.
label

Print this text instead of time in log label.

rlDie¶

Create a time-labelled message in the log, report test result, upload logs, close unfinished phase and terminate the test.

rlDie message [file...]

message

Message you want to show (use quotes when invoking) - this option is mandatory.
file

Files (logs) you want to upload as well. rlBundleLogs will be used for it. Files which are not readable will be excluded before calling rlBundleLogs, so it is safe to call even with possibly nonexistent logs and it will succeed.

rlBundleLogs¶

Create a tarball of files (e.g. logs) and attach them to the test result.

rlBundleLogs package file [file...]

package

Name of the package. Will be used as part of the tarball name.
file

File(s) to be packed and submitted.

Returns result of submitting the tarball.

rlFileSubmit¶

Resolves absolute path to the file, replaces / with - and uploads this renamed file using rhts-submit-log. It also allows you to specify your custom name for the uploaded file.

rlFileSubmit [-s sep] path_to_file [required_name]

-s sep

Sets separator (i.e. the replacement of the /) to sep.
path_to_file

Either absolute or relative path to file. Relative path is converted to absolute.
required_name

Default behavior renames file to full_path_to_file with / replaced for -, if this does not suit your needs, you can specify the name using this option.

Examples:

rlFileSubmit logfile.txt -> logfile.txt cd /etc; rlFileSubmit ./passwd -> etc-passwd rlFileSubmit /etc/passwd -> etc-passwd rlFileSubmit /etc/passwd my-top-secret_file -> my-top-secret-file rlFileSubmit -s ‘_’ /etc/passwd -> etc_passwd

Info¶

rlShowPackageVersion¶

Shows a message about version of packages.

rlShowPackageVersion package [package...]

package

Name of a package(s) you want to log.

rlGetArch¶

Sanitize architecture for simplier matching.

Return base arch for the current system (good when you need base arch on a multilib system).

rlGetArch

On any 32-bit Intel (i386, i486, i586, …) system you will get i386.

rlGetPrimaryArch¶

Return primary arch for the current system (good when you need base arch on a multilib system).

rlGetPrimaryArch

On non-RHEL systems if the primary/secondary sedicision fails a fallback to rlGetArch is done.

rlGetSecondaryArch¶

Return base arch for the current system (good when you need base arch on a multilib system).

rlGetSecondaryArch

rlGetDistroRelease¶

rlGetDistroVariant¶

Return release or variant of the distribution on the system.

rlGetDistroRelease
rlGetDistroVariant

For example on the RHEL-4-AS you will get release 4 and variant AS, on the RHEL-5-Client you will get release 5 and variant Client.

rlShowRunningKernel¶

Log a message with version of the currently running kernel.

rlShowRunningKernel

Phases¶

rlPhaseStart¶

Starts a phase of a specific type. The final phase result is based on all asserts included in the phase. Do not forget to end phase with rlPhaseEnd when you are done.

rlPhaseStart type [name]

type

Type of the phase, one of the following:
- FAIL
  
  When assert fails here, phase will report a FAIL.
- WARN
  
  When assert fails here, phase will report a WARN.
name

Optional name of the phase (if not provided, one will be generated).

If all asserts included in the phase pass, phase reports PASS.

rlPhaseEnd¶

End current phase, summarize asserts included and report phase result.

rlPhaseEnd

Final phase result is based on included asserts and phase type.

rlPhaseStartSetup¶

rlPhaseStartTest¶

rlPhaseStartCleanup¶

Start a phase of the specified type: Setup -> WARN, Test -> FAIL, Cleanup -> WARN.

rlPhaseStartSetup [name]
rlPhaseStartTest [name]
rlPhaseStartCleanup [name]

name

Optional name of the phase. If not specified, default Setup/Test/Cleanup are used.

If you do not want these shortcuts, use plain rlPhaseStart function.

Reboot¶

Controlled reboot must not be executed inside of an open phase. Otherwise it is possible the Overall result of the test will be reported incorrectly.

Metric¶

rlLogMetricLow¶

Log a metric, which should be as low as possible to the journal. (Example: memory consumption, run time)

rlLogMetricLow name value [tolerance]

name

Name of the metric. It has to be unique in a phase.
value

Value of the metric.
tolerance

It is used when comparing via rcw. It means how larger can the second value be to not trigger a FAIL. Default is 0.2

When comparing FIRST, SECOND, then:

FIRST >= SECOND means PASS
FIRST+FIRST*tolerance >= SECOND means WARN
FIRST+FIRST*tolerance < SECOND means FAIL

Example: Simple benchmark is compared via this metric type in rcw. It has a tolerance of 0.2. First run had 1 second. So:

For PASS, second run has to be better or equal to first.
        So any value of second or less is a PASS.
For WARN, second run can be a little worse than first.
        Tolerance is 0.2, so anything lower than 1.2 means WARN.
For FAIL, anything worse than 1.2 means FAIL.

rlLogMetricHigh¶

Log a metric, which should be as high as possible to the journal. (Example: number of executions per second)

rlLogMetricHigh name value [tolerance]

name

Name of the metric. It has to be unique in a phase.
value

Value of the metric.
tolerance

It is used when comparing via rcw. It means how lower can the second value be to not trigger a FAIL. Default is 0.2

When comparing FIRST, SECOND, then:

FIRST <= SECOND means PASS
FIRST+FIRST*tolerance <= SECOND means WARN
FIRST+FIRST*tolerance > SECOND means FAIL

Manual Asserts¶

rlPass¶

Manual assertion, asserts and logs PASS.

rlPass comment

comment

Short test summary.

Returns 0 and asserts PASS.

rlFail¶

Manual assertion, asserts and logs FAIL.

rlFail comment

comment

Short test summary.

Returns 1 and asserts FAIL.

Arithmetic Asserts¶

rlAssert0¶

Assertion checking for the equality of parameter to zero.

rlAssert0 comment value

comment

Short test summary, e.g. “Test if compilation ended successfully”.
value

Integer value (usually return code of a command).

Returns 0 and asserts PASS when value == 0.

rlAssertEquals¶

Assertion checking for the equality of two parameters.

rlAssertEquals comment value1 value2

comment

Short test summary, e.g. “Test if all 3 packages have been downloaded”.
value1

First parameter to compare, can be a number or a string
value2

Second parameter to compare, can be a number or a string

Returns 0 and asserts PASS when value1 == value2.

rlAssertNotEquals¶

Assertion checking for the non-equality of two parameters.

rlAssertNotEquals comment value1 value2

comment

Short test summary, e.g. “Test if return code is not 139”.
value1

First parameter to compare, can be a number or a string
value2

Second parameter to compare, can be a number or a string

Returns 0 and asserts PASS when value1 != value2.

rlAssertGreater¶

Assertion checking whether first parameter is greater than the second one.

rlAssertGreater comment value1 value2

comment

Short test summary, e.g. “Test whether there are running more instances of program.”
value1

Integer value.
value2

Integer value.

Returns 0 and asserts PASS when value1 > value2.

rlAssertGreaterOrEqual¶

Assertion checking whether first parameter is greater or equal to the second one.

rlAssertGreaterOrEqual comment value1 value2

comment

Short test summary (e.g. “There should present at least one…”)
value1

Integer value.
value2

Integer value.

Returns 0 and asserts PASS when value1 ≥= value2.

rlAssertLesser¶

Assertion checking whether first parameter is lesser than the second one.

rlAssertLesser comment value1 value2

comment

Short test summary, e.g. “Test whether there are running more instances of program.”
value1

Integer value.
value2

Integer value.

Returns 0 and asserts PASS when value1 < value2.

rlAssertLesserOrEqual¶

Assertion checking whether first parameter is lesser or equal to the second one.

rlAssertLesserOrEqual comment value1 value2

comment

Short test summary (e.g. “There should present at least one…”)
value1

Integer value.
value2

Integer value.

Returns 0 and asserts PASS when value1 ≤= value2.

File Asserts¶

rlAssertExists¶

Assertion checking for the existence of a file or a directory.

rlAssertExists file|directory

file|directory

Path to the file or directory.

Returns 0 and asserts PASS when file exists.

rlAssertNotExists¶

Assertion checking for the non-existence of a file or a directory.

rlAssertNotExists file|directory

file|directory

Path to the file or directory.

Returns 0 and asserts PASS when file does not exist.

rlAssertGrep¶

Assertion checking if the file contains a pattern.

rlAssertGrep pattern file [options]

pattern

Regular expression to be searched for.
file

Path to the file.
options

Optional parameters to be passed to grep, default is -q. Can be used to perform case insensitive matches (-i), or using extended (-E) or perl (-P) regular expressions.

Returns 0 and asserts PASS when file exists and contains given pattern.

rlAssertNotGrep¶

Assertion checking that the file does not contain a pattern.

rlAssertNotGrep pattern file [options]

pattern

Regular expression to be searched for.
file

Path to the file.
options

Optional parameters to be passed to grep, default is -q. Can be used to perform case insensitive matches (-i), or using extended (-E) or perl (-P) regular expressions.

Returns 0 and asserts PASS when file exists and does not contain given pattern.

rlAssertDiffer¶

Assertion checking that two files differ (are not identical).

rlAssertDiffer file1 file2

file1

Path to first file1
file2

Path to second file

Returns 0 and asserts PASS when file1 and file2 differs.

rlAssertNotDiffer¶

Assertion checking that two files do not differ (are identical).

rlAssertNotDiffer file1 file2

file1

Path to first file1
file2

Path to second file

Returns 0 and asserts PASS when file1 and file2 do not differ.

Run, Watch, Report¶

rlRun¶

Run command with optional comment and make sure its exit code matches expectations.

rlRun [-t] [-l] [-c] [-s] command [status[,status...] [comment]]

-t

If specified, stdout and stderr of the command output will be tagged with strings ‘STDOUT: ‘ and ‘STDERR: ‘.
-l

If specified, output of the command (tagged, if -t was specified) is logged using rlLog function. This is intended for short outputs, and therefore only last 50 lines are logged this way. Longer outputs should be analysed separately, or uploaded via rlFileSubmit or rlBundleLogs.
-c

Same as -l, but only log the command output if it failed.
-s

Store stdout and stderr to a file (mixed together, as the user would see it on a terminal) and set $rlRun_LOG variable to name of the file. $rlRun_LOG is now actually an array where the first index holds the last reference to the file. Thus its behavior is not changed if used without an index. The array is consumed by the rlJournalEnd function to remove the leftover files, so no manual files removal is needed anymore. Note that if you need to use such a file after calling the rlJournalEnd function you need to create your own copy of the respective file. When -t option is used, the content of the file becomes tagged too.

If the -s option is not used, $rlRun_LOG is not modified and keeps its content from the last “rlRun -s”.
command

Command to run.
status

Expected exit code(s). Optional, default 0. If you expect more exit codes, separate them with comma (e.g. “0,1” when both 0 and 1 are OK and expected), or use from-to notation (i.e. “2-5” for “2,3,4,5”), or combine them (e.g. “2-4,26” for “2,3,4,26”).
comment

Short summary describing the action (optional, but recommended - explain what you are doing here).

Returns the exit code of the command run. Asserts PASS when command's exit status is in the list of expected exit codes.

Note:

The output of rlRun is buffered when using -t, -l or -s option (they use unix pipes, which are buffered by nature). If you need an unbuffered output just make sure that expect package is installed on your system (its “unbuffer” tool will automatically be used to produce unbuffered output).
Be aware that there are some variables which can collide with your code executed within rlRun. You should avoid using __INTERNAL_rlRun_* variables.
When any of -t -l, -c, or -s option is used, special file descriptors 111 and 112 are used to avoid the issue with incomplete log file, bz1361246. As there might be an indefinite loop, there’s a timeout of two minutes implemented as a fix for bz1416796. Also an error message is issued to signal the possibility of running subprocess which keeps the file descriptors open.

Do not use these options if you expect process forking and continuouse run. Try your own apropriate solution instead.

Warning: using unbuffer tool is now disabled because of bug 547686.

rlWatchdog¶

Run command. If it does not finish in specified time, then kill it using signal. Note that the command is executed in a sub-shell so modified environment (e.g. set variables) is not relfected in the test environment.

rlWatchdog command timeout [signal] [callback]

command

Command to run.
timeout

Timeout to wait, in seconds.
signal

Signal to use (optional, default KILL).
callback

Callback function to be called before the signal is send (optional, none by default). The callback function will have one argument available – PGID of the process group.

Returns 0 if the command ends normally, without need to be killed.

rlReport¶

Report test result to the test harness. The command to be used for reporting is set by the respective plugin. You can also use the $BEAKERLIB_COMMAND_REPORT_RESULT variable to use your custom command.

rlReport name result [score] [log]

name

Name of the test result.
result

Result (one of PASS, WARN, FAIL, SKIP). If called with something else, will use WARN.
score

Test score (optional).
log

Optional log file to be submitted instead of default OUTPUTFILE.

rlCmpVersion¶

Compare two given versions composed by numbers and letters divided by dot (.), underscore (_), or dash (-).

rlCmpVersion ver1 ver2

If ver1 = ver2, sign ‘=’ is printed to stdout and 0 is returned. If ver1 > ver2, sign ‘>’ is printed to stdout and 1 is returned. If ver1 < ver2, sign ‘<’ is printed to stdout and 2 is returned.

rlTestVersion¶

Test releation between two given versions based on given operator.

rlTestVersion ver1 op ver2

op

Operator defining the logical expression. It can be ‘=’, ‘==’, ‘!=’, ‘<’, ‘<=’, ‘=<’, ‘>’, ‘>=’, or ‘=>’.

Returns 0 if the expresison ver1 op ver2 is true; 1 if the expression is false and 2 if something went wrong.

Release Info¶

rlIsRHEL¶

rlIsRHEL [VERSION_SPEC]...

VERSION_SPEC

Argument specifies version of particular RHEL distribution, eg. 8, 8.4 “>8.4”.

For more details see description of “rlIsOSVersion”.

Returns 0 when we’re running on RHEL. With given version specification in arguments returns 0 if the particular RHEL version is running.

Prototype:

  rlIsRHEL

Returns 0 if we are running on RHEL.

  rlIsRHEL 7 '>=8.5'

Returns 0 if we are running any RHEL 7 or RHEL 8.5 and higher.

rlIsFedora¶

rlIsFedora [VERSION_SPEC]...

VERSION_SPEC

Argument specifies version of particular Fedora distribution, eg. 30, “>30”.

For more details see description of “rlIsOSVersion”.

Returns 0 when we’re running on Fedora. With given version specification in arguments returns 0 if the particular Fedora version is running.

Prototype:

  rlIsFedora

Returns 0 if we are running on Fedora.

  rlIsFedora 30 32

Returns 0 if we are running Fedora 30 or 32.

rlIsCentOS¶

rlIsCentOS [VERSION_SPEC]...

VERSION_SPEC

Argument specifies version of particular CentOS distribution, eg. 7, “>7”.

For more details see description of “rlIsOSVersion”.

Returns 0 when we’re running on CentOS. With given version specification in arguments returns 0 if the particular CentOS version is running.

Prototype:

  rlIsCentOS

Returns 0 if we are running on CentOS.

  rlIsCentOS 7.1 6

Returns 0 if we are running CentOS 7.1 or any CentOS 6.

rlIsOS¶

rlIsOS ID

ID

Argument is based on contents of /etc/os-release. Possible options of ID are e.g. fedora, rhel, centos, ol.

Returns 0 when we’re running on system with respective ID. Returns 1 when parameter does not match with ID in os-release. Returns 2 when there is no ID defined. Returns 3 when no argument is given.

Prototype:

  rlIsOS rhel

Returns 0 if we are running on RHEL.

rlIsOSLike¶

rlIsOSLike ID_LIKE

ID_LIKE

Argument is based on contents of /etc/os-release. Possible options of ID_LIKE are e.g. fedora, rhel.

Returns 0 when we’re running on system with requested ID_LIKE. Returns 1 when parameter does not match with ID nor ID_LIKE in os-release. Returns 2 when there is no ID or ID_LIKE defined. Returns 3 when no argument is given.

Prototype:

  rlIsOSLike rhel

Returns 0 if we are running on RHEL, CentOS, Rocky Linux, etc..

  rlIsOSLike fedora

Returns 0 if we are running on Fedora, RHEL, CentOS, Oracle Linux, etc..

rlIsOSVersion¶

rlIsOsVersion VERSION_SPEC...

VERSION_SPEC

Parameter is based on VERSION_ID in /etc/os-release. It consists of either major or major.minor refering to a particular release.

It accepts multiple arguments separated by space (8.1 8.2 8.3 9).

To specify a group of distributions optional operator can be passed in argument together with the version number as one string (‘>8.5’). Operator can be ‘<’, ‘<=’, ‘=<’, ‘=’, ‘>’, ‘>=’ matching whether the currently installed version is lesser, lesser or equal, equal, equal or greater, greater than the version number supplied as second half of the argument.

Note that when using >/< operators you have to either put the argument in quotes or escape the operators to avoid them being interpreted as bash redirection operator.

Returns 0 when we’re running distribution of the particular version requested by the argument.

It usually follows after rlIsOS and rlIsOSLike.

Be cautious when using together with rlIsOSLike as different distributions may use different versioning schema.

Prototype:

 rlIsOSVersion 6 7 9

Returns 0 if we are running distribution with VERSION_ID 6, 7 or 9.

  rlIsOSVersion 7.8 8

Returns 0 if we are running distribution 7.8 or any 8.

  rlIsOSVersion ">=7.5" or rlIsOSVersion \>=7.5

Returns 0 if we are running distribution 7.5 or higher (both minors or majors).

Note:

  rlIsOSVersion '<7.5' || rlIsOSVersion '<8.5'

would also cover 7.9 as it is less than 8.5, which is not what you want.
So if you want to construct a condition for a distribution <7.5 within the major 7 or
a distribution <8.5 within the major 8 you actually need to use following construct:

  rlIsOSVersion 7 && rlIsOSVersion '<7.5' || rlIsOSVersion 8 && rlIsOSVersion '<8.5'

This returns 0 when running distribution less than 7.5 and less then 8.5, but not 7.9 (nor 6.9).

rlIsRHELLike¶

rlIsRHELLike [VERSION_SPEC]...

VERSION_SPEC

Argument specifies version of particular RHEL distribution, eg. 8, 8.4 “>8.4”.

For more details see description of “rlIsOSVersion”.

Returns 0 when we’re running on RHEL-like distribution. These are considered to be RHEL, CentOS, Rocky Linux, etc.. With given version specification in arguments returns 0 if the particular version of RHEL-like distribution is running.

Prototype:

  rlIsRHELLike

Returns 0 if we are running on RHEL-like system.

  rlIsRHELLike ">=6"

Returns 0 if we are running on RHEL-like distribution of version 6.0 or higher.

Rpm Handling¶

rlCheckRpm¶

Check whether a package is installed.

rlCheckRpm name [version] [release] [arch]

name

Package name like kernel
version

Package version like 2.6.25.6
release

Package release like 55.fc9
arch

Package architucture like i386

Returns 0 if the specified package is installed.

rlAssertRpm¶

Assertion making sure that a package is installed.

rlAssertRpm name [version] [release] [arch]>
rlAssertRpm --all

name

Package name like kernel
version

Package version like 2.6.25.6
release

Package release like 55.fc9
arch

Package architucture like i386
–all

Assert all packages listed in the $PACKAGES $REQUIRES and $COLLECTIONS environment variables.

Returns 0 and asserts PASS if the specified package is installed.

rlAssertNotRpm¶

Assertion making sure that a package is not installed. This is just inverse of rlAssertRpm.

rlAssertNotRpm name [version] [release] [arch]>

name

Package name like kernel
version

Package version like 2.6.25.6
release

Package release like 55.fc9
arch

Package architucture like i386

Returns 0 and asserts PASS if the specified package is not installed.

Example¶

Function rlAssertRpm is useful especially in prepare phase where it causes abort if a package is missing, while rlCheckRpm is handy when doing something like:

if ! rlCheckRpm package; then
     yum install package
     rlAssertRpm package
fi

rlAssertBinaryOrigin¶

Assertion making sure that given binary is owned by (one of) the given package(s).

rlAssertBinaryOrigin binary package [package2 [...]]
PACKAGES=... rlAssertBinaryOrigin binary

binary

Binary name like ksh or /bin/ksh
package

List of packages like ksh mksh. The parameter is optional. If missing, contents of environment variable $PACKAGES are taken into account.

Returns 0 and asserts PASS if the specified binary belongs to (one of) the given package(s). Returns 1 and asserts FAIL if the specified binary does not belong to (any of) the given package(s). Returns 2 and asserts FAIL if the specified binary is not found. Returns 3 and asserts FAIL if no packages are given. Returns 100 and asserts FAIL if invoked with no parameters.

Example¶

Function rlAssertBinaryOrigin is useful especially in prepare phase where it causes abort if a binary is missing or is owned by different package:

PACKAGES=mksh rlAssertBinaryOrigin ksh
or
rlAssertBinaryOrigin ksh mksh

Returns true if ksh is owned by the mksh package (in this case: /bin/ksh is a symlink pointing to /bin/mksh).

rlGetMakefileRequires¶

Prints space separated list of requirements defined in Makefile using ‘Requires’ attribute.

Return 0 if success.

rlGetYAMLdeps¶

Prints space separated list of requirements defined in metadata.yaml using ‘require’ and / or ‘recommend’ attribute.

Full fmf ids and library references are to be ignored.

rlGetYAMLdeps DEP_TYPE [array_var_name]

DEP_TYPE

Dependency type defined as a regexp. Expected values are ‘require’, ‘recommend’, or ‘require|recommend’. The matching attribute values are then processed. Defaults to ‘require’.
array_var_name

Name of the variable to put the output to in a form of an array. This can hold also dependencies with specific version. If used, the output to stdout is suppressed.

Return 0 if success.

rlCheckRequirements¶

Check that all given requirements are covered eigther by installed package or by binary available in PATHs or by some package’s provides.

rlRun "rlCheckRequirements REQ..."

REQ

Requirement to be checked. It can be package name, provides string or binary name. Moreover, the requirement can be written the same way as the Require in the spec file, including the version specification, e.g. “bash >= 4.4”. The comparsion operator may be any of supported by rlTestVersion(), see its manual.

Returns number of unsatisfied requirements.

rlCheckMakefileRequires¶

Check presence of required packages / binaries defined in metadata.yaml provided by tmt or Makefile of the test.

Also check presence of recommended packages / binaries defined in metadata.yaml provided by tmt. These however do not participate on the return code, these are just informative.

Example¶

rlRun "rlCheckMakefileRequires"

Return 255 if requirements could not be retrieved, 0 if all requirements are satisfied or number of unsatisfied requirements.

rlGetRequired¶

=head3 rlGetRecommended

Get a list of required or recommended packages / binaries defined in metadata.yaml provided by tmt or in a Makefile of the test.

rlGetRequired [ARRAY_VAR_NAME]
rlGetRecommended [ARRAY_VAR_NAME]

ARRAY_VAR_NAME

If provided the variable is populated as an array with the respective dependencies. Otherwise the dependencies are printed out to the STDOUT.

Return 255 if requirements could not be retrieved, 0 otherwise.

rlCheckRequired¶

=head3 rlCheckRecommended =head3 rlCheckDependencies

Check presence of required, recommended or both packages / binaries defined in metadata.yaml provided by tmt or in a Makefile of the test.

Example¶

rlRun "rlCheckRequired"
rlRun "rlCheckRecommended"
rlRun "rlCheckDependencies"

Return 255 if requirements could not be retrieved, 0 if all requirements are satisfied or number of unsatisfied requirements.

rlAssertRequired¶

Ensures that all required packages provided in either metadata.yaml or Makefile are installed.

rlAssertRequired

Prints out a verbose list of installed/missing packages during operation.

Returns 0 if all required packages are installed, 1 if one or more packages are missing or if no Makefile is present.

Getting RPMs¶

Download methods¶

Functions handling rpm downloading/installing can use more methods for actual download of the rpm.

Currently there are two download methonds available:

direct

Use use direct download from build system (brew).
yum

Use yumdownloader or dnf download.

The methods and their order are defined by BEAKERLIB_RPM_DOWNLOAD_METHODS variable as space separated list. By default it is ‘direct yum’. This can be overridden by user. There may be also additions or changes done to the original value, e.g. BEAKERLIB_RPM_DOWNLOAD_METHODS=’yum ${BEAKERLIB_RPM_DOWNLOAD_METHODS/yum/}’

¶

Beakerlib is prepared for more Koji-based sources of packages while usigng direct download method. By default packages are fetched from Koji, specifically from https://kojipkgs.fedoraproject.org/packages.

rlRpmInstall¶

Try to install specified package from local Red Hat sources.

rlRpmInstall [--quiet] package version release arch

–quiet

Make the download and the install process be quiet.
package

Package name like kernel
version

Package version like 2.6.25.6
release

Package release like 55.fc9
arch

Package arch like i386

Returns 0 if specified package was installed successfully.

rlRpmDownload¶

Try to download specified package.

rlRpmDownload [--quiet] {package version release arch|N-V-R.A}
rlRpmDownload [--quiet] --source {package version release|N-V-R}

–quiet

Make the download process be quiet.
package

Package name like kernel
version

Package version like 2.6.25.6
release

Package release like 55.fc9
arch

Package arch like i386

Returns 0 if specified package was downloaded successfully.

rlFetchSrcForInstalled¶

Tries various ways to download source rpm for specified installed rpm.

rlFetchSrcForInstalled [--quiet] package

–quiet

Make the download process be quiet.
package

Installed package name like kernel. It accepts in-direct names. E.g. for the package name krb5-libs the function will download the krb5 source rpm.

Returns 0 if the source package was successfully downloaded.

Mounting¶

rlMount¶

Create mount point (if neccessary) and mount a NFS share.

rlMount [-o MOUNT_OPTS] server share mountpoint

server

NFS server hostname.
share

Shared directory name.
mountpoint

Local mount point.
MOUNT_OPTS

Mount options.

Returns 0 if mounting the share was successful.

rlCheckMount¶

Check either if a directory is a mountpoint, if it is a mountpoint to a specific server, or if it is a mountpoint to a specific export on a specific server and if it uses specific mount options.

rlCheckMount [-o MOUNT_OPTS] mountpoint
rlCheckMount [-o MOUNT_OPTS] server mountpoint
rlCheckMount [-o MOUNT_OPTS] server share mountpoint

mountpoint

Local mount point.
server

NFS server hostname
share

Shared directory name
MOUNT_OPTS

Mount options to check (comma separated list)

With one parameter, returns 0 when specified directory exists and is a mountpoint. With two parameters, returns 0 when specific directory exists, is a mountpoint and an export from specific server is mounted there. With three parameters, returns 0 if a specific shared directory is mounted on a given server on a given mountpoint

If the -o option is provided, returns 0 if the mountpoint uses all the given options, 2 otherwise.

rlAssertMount¶

Assertion making sure that given directory is a mount point, if it is a mountpoint to a specific server, or if it is a mountpoint to a specific export on a specific server and if it uses specific mount options.

rlAssertMount [-o MOUNT_OPTS]  mountpoint
rlAssertMount [-o MOUNT_OPTS]  server mountpoint
rlAssertMount [-o MOUNT_OPTS]  server share mountpoint

mountpoint

Local mount point.
server

NFS server hostname
share

Shared directory name
MOUNT_OPTS

Mount options to check (comma separated list)

If the -o option is provided, asserts PASS if the above conditions are met and the mountpoint uses all the given options.

rlHash, rlUnhash¶

Hashes/Unhashes given string and prints the result to stdout.

rlHash [--decode] [--algorithm HASH_ALG] --stdin|STRING
rlUnhash [--algorithm HASH_ALG] --stdin|STRING

–decode

Unhash given string.
–algorithm

Use given hash algorithm. Currently supported algorithms: base64 base64_ - this is standard base64 where ‘=’ is replaced by ‘_’ hex

Defaults to hex. Default algorithm can be override using global variable rlHashAlgorithm.
–stdin

Get the string from stdin.
STRING

String to be hashed/unhashed.

Returns 0 if success.

Backup and Restore¶

rlFileBackup¶

Create a backup of files or directories (recursive). Can be used multiple times to add more files to backup. Backing up an already backed up file overwrites the original backup.

rlFileBackup [--clean] [--namespace name] [--missing-ok|--no-missing-ok] file [file...]

You can use rlRun for asserting the result but keep in mind meaning of exit codes, especialy exit code 8, if using without –clean option.

–clean

If this option is provided (has to be the first option of the command), then file/dir backed up using this command (provided in next options) will be (recursively) removed before we restore it. This option implies –missing-ok, which can be overridden by –no-missing-ok.
–namespace name

Specifies the namespace to use. Namespaces can be used to separate backups and their restoration.
–missing-ok

Do not raise an error in case of missing file to backup.
–no-missing-ok

Do raise an error in case of missing file to backup. This is useful with –clean. This behaviour can be globally set by global variable BEAKERLIB_FILEBACKUP_MISSING_OK=false.
file

Files and/or directories to be backed up.

Returns 0 if the backup was successful. Returns 1 if parsing of parameters was not successful. Returns 2 if no files specification was provided. Returns 3 if BEAKERLIB_DIR variable is not set, e.g. rlJournalStart was not executed. Returns 4 if creating of main backup destination directories was not successful. Returns 5 if creating of file specific backup destination directories was not successful. Returns 6 if the copy of backed up files was not successful. Returns 7 if attributes of backed up files were not successfuly copied. Returns 8 if backed up files do not exist. This can be suppressed based on other options.

Example with –clean:¶

touch cleandir/aaa
rlRun "rlFileBackup --clean cleandir/"
touch cleandir/bbb
ls cleandir/
aaa   bbb
rlRun "rlFileRestore"
ls cleandir/
aaa

rlFileRestore¶

Restore backed up files to their original location. rlFileRestore does not remove new files appearing after backup has been made. If you don’t want to leave anything behind just remove the whole original tree before running rlFileRestore, or see --clean option of rlFileBackup.

rlFileRestore [--namespace name]

You can use rlRun for asserting the result.

–namespace name

Specifies the namespace to use. Namespaces can be used to separate backups and their restoration.

Returns 0 if backup dir is found and files are restored successfully.

Return code bits meaning:

XXXXX
|||||
||||\_ error parsing parameters
|||\__ could not find backup directory
||\___ files cleanup failed
|\____ files restore failed
\_____ no files were restored nor cleaned

Services¶

Following routines implement comfortable way how to start/stop system services with the possibility to restore them to their original state after testing.

rlServiceStart¶

Make sure the given service is running with fresh configuration. (Start it if it is stopped and restart if it is already running.) In addition, when called for the first time, the current state is saved so that the service can be restored to its original state when testing is finished, see rlServiceRestore.

rlServiceStart service [service...]

service

Name of the service(s) to start.

Returns number of services which failed to start/restart; thus zero is returned when everything is OK.

rlServiceStop¶

Make sure the given service is stopped. Stop it if it is running and do nothing when it is already stopped. In addition, when called for the first time, the current state is saved so that the service can be restored to its original state when testing is finished, see rlServiceRestore.

rlServiceStop service [service...]

service

Name of the service(s) to stop.

Returns number of services which failed to become stopped; thus zero is returned when everything is OK.

rlServiceRestore¶

Restore given service into its original state (before the first rlServiceStart or rlServiceStop was called). If rlServiceStart was called on already running service, rlServiceRestore will restart the service when restoring its state.

rlServiceRestore [service...]

service

Name of the service(s) to restore to original state. All services will be restored in reverse order if no service name is given.

Returns number of services which failed to get back to their original state; thus zero is returned when everything is OK.

rlServiceStatus¶

Print status (output of `service SERVICE status` or `systemctl status SERVICE`) of given SERVICE.

rlServiceStatus SERVICE [SERVICE...]

SERVICE

The service to get the status of.

Returns service status return code of the last provided SERVICE.

rlServiceEnable¶

Enables selected services if needed, or does nothing if already enabled. In addition, when called for the first time, the current state is saved so that the service can be restored to its original state when testing is finished, see rlServiceRestore.

rlServiceEnable service [service...]

service

Name of the service(s) to enable.

Returns number of services which failed enablement; thus zero is returned when everything is OK.

rlServiceDisable¶

Disables selected services if needed, or does nothing if already disabled. In addition, when called for the first time, the current state is saved so that the service can be restored to its original state when testing is finished, see rlServiceRestore.

rlServiceDisable service [service...]

service

Name of the service(s) to disable.

Returns number of services which failed disablement; thus zero is returned when everything is OK.

Sockets¶

Following routines implement comfortable way how to start/stop system sockets with the possibility to restore them to their original state after testing.

rlSocketStart¶

Make sure the given socket is running. (Start it if stopped, leave it if already running.) In addition, when called for the first time, the current state is saved so that the socket can be restored to its original state when testing is finished, see rlSocketRestore.

rlSocketStart socket [socket...]

socket

Name of the socket(s) to start.

Returns number of sockets which failed to start/restart; thus zero is returned when everything is OK.

rlSocketStop¶

Make sure the given socket is stopped. Stop it if it is running and do nothing when it is already stopped. In addition, when called for the first time, the current state is saved so that the socket can be restored to its original state when testing is finished, see rlSocketRestore.

rlSocketStop socket [socket...]

socket

Name of the socket(s) to stop.

Returns number of sockets which failed to become stopped; thus zero is returned when everything is OK.

rlSocketRestore¶

Restore given socket into its original state (before the first rlSocketStart or rlSocketStop was called).

Warning !!! Xinetd process [even though it might have been used] is NOT restored. It is recommended to call rlServiceRestore xinetd as well.

rlSocketRestore socket [socket...]

socket

Name of the socket(s) to restore to original state.

Returns number of sockets which failed to get back to their original state; thus zero is returned when everything is OK.

Cleanup management¶

Cleanup management works with a so-called cleanup buffer, which is a temporary representation of what should be run at cleanup time, and a final cleanup script (executable), which is generated from this buffer and wraps it using BeakerLib essentials (journal initialization, cleanup phase, …). The cleanup script must always be updated on an atomic basis (filesystem-wise) to allow an asynchronous execution by a third party (ie. test watcher).

The test watcher usage is mandatory for the cleanup management system to work properly as it is the test watcher that executes the actual cleanup script. Limited, catastrophe-avoiding mechanism is in place even when the test is not run in test watcher, but that should be seen as a backup and such situation is to be avoided whenever possible.

The cleanup script shares all environment (variables, exported or not, and functions) with the test itself - the cleanup append/prepend functions “sample” or “snapshot” the environment at the time of their call, IOW any changes to the test environment are synchronized to the cleanup script only upon calling append/prepend. When the append/prepend functions are called within a function which has local variables, these will appear as global in the cleanup.

While the cleanup script receives $PWD from the test, its working dir is set to the initial test execution dir even if $PWD contains something else. It is impossible to use relative paths inside cleanup reliably - certain parts of the cleanup might have been added under different current directories (CWDs). Therefore always use absolute paths in append/prepend cleanup or make sure you never ‘cd’ elsewhere (ie. to a TmpDir).

rlCleanupAppend¶

Appends a string to the cleanup buffer and recreates the cleanup script.

rlCleanupAppend string

Returns 0 if the operation was successful, 1 otherwise.

rlCleanupPrepend¶

Prepends a string to the cleanup buffer and recreates the cleanup script.

rlCleanupPrepend string

Returns 0 if the operation was successful, 1 otherwise.

Time Performance¶

rlPerfTime_RunsInTime¶

Measures, how many runs of some commands can be done in specified time. This approach is suitable for short-time running tasks (up to few seconds), where averaging few runs is not precise. This is done several times, and the final result is the average of all runs. It prints the number on stdout, so it has to be captured.

rlPerfTime_RunsInTime command [time] [runs]

command

Command to run.
time

Time in seconds (optional, default=30).
runs

Number of averaged runs (optional, default=3).

rlPerfTime_AvgFromRuns¶

Measures the average time of running some task. This approach is suitable for long-time running tasks (tens of seconds and more), where it is precise enough. Measured runs can be preceded by dry run, which is not measured and it’s purpose is to warm up various caches. It prints the number on stdout, so it has to be captured. Or, result is then stored in special rl_retval variable.

rlPerfTime_AvgFromRuns command [count] [warmup]

command

Command to run.
count

Times to run (optional, default=3).
warmup

Warm-up run, run if this option is not “warmup” (optional, default=”warmup”)

Analyze¶

rlDejaSum¶

TODO description

rlDejaSum par1 par2

par1

TODO description
par2

TODO description

Return 0 if… TODO

rlImport¶

Imports code provided by one or more libraries into the test namespace. The library search mechanism is based on Beaker test hierarchy system, i.e.:

/component/type/test-name/test-file

When test-file calls rlImport with ‘foo/bar’ parameter, the libraries are searched in following locations: these are the possible path prefixes

- colon-separated paths from $BEAKERLIB_LIBRARY_PATH
- /mnt/tests
- /usr/share/beakerlib-libraries

the next component of the path is one of the following:

- /foo/Library/bar
- /foo/bar
- /libs/foo/bar
- /*/foo/Library/bar
- /*/foo/bar
- /libs/*/foo/Library/bar
- /libs/*/foo/bar

the directory path is then constructed as prefix/path/lib.sh If the library is still not found an upwards directory traversal is used, and a check for presence of the library in the above mentioned possible paths is to be performed. This means this function needs to be called from the test hierarchy, not e.g. the /tmp directory.

Once library is found, it is sourced and a verifier function is called. The verifier function is cunstructed by composing the library prefix and LibraryLoaded. Library prefix must be defined in the library itself. It should be part of lib.sh header in format: ‘# library-prefix = ’. If the verifier passes the library is ready to use. Also variable LibraryDir is created and it points to the library folder.

Usage:

rlImport --all
rlImport LIBRARY [LIBRARY2...]

–all

Read $BEAKERLIB_DIR/metadata.yaml or ./Makefile, pickup the library requirements and import them all.
LIBRARY

Must have ‘[component/]path’ or ‘.’ format. Identifies the library to import. The dot (.) is a special case where the lib.sh from the current directory is used.

Returns 0 if the import of all libraries was successful. Returns non-zero if one or more library failed to import.

Process Synchronisation¶

rlWaitForCmd¶

Pauses script execution until command exit status is the expeced value. Logs a WARNING and returns 1 if the command didn’t exit successfully before timeout elapsed or a maximum number of invocations has been reached.

rlWaitForCmd command [-p PID] [-t time] [-m count] [-d delay] [-r retval]

command

Command that will be executed until its return code is equal 0 or value speciefied as option to ‘-r’.
-t time

Timeout in seconds, default=120. If the command doesn’t return 0 before time elapses, the command will be killed.
-p PID

PID of the process to check before running command. If the process exits before the socket is opened, the command will log a WARNING.
-m count

Maximum number of ‘command’ executions before continuing anyway. Default is infite. Returns 1 if the maximum was reached.
-d delay

Delay between ‘command’ invocations. Default 1.
-r retval

Expected return value of command. Default 0.

rlWaitForFile¶

Pauses script execution until specified file or directory starts existing. Returns 0 if file started existing, 1 if timeout was reached or PID exited. Return code is greater than 1 in case of error.

rlWaitForFile path [-p PID] [-t time] [-d delay]

path

Path to file that should start existing.
-t time

Timeout in seconds (optional, default=120). If the file isn’t opened before the time elapses the command returns 1.
-p PID

PID of the process that should also be running. If the process exits before the file is created, the command returns with status code of 1.
-d delay

Delay between subsequent checks for existence of file. Default 1.

rlWaitForSocket¶

Pauses script execution until local socket starts listening. Returns 0 if socket started listening, 1 if timeout was reached or PID exited. Return code is greater than 1 in case of error.

rlWaitForSocket {port|path} [-p PID] [-t time] [-d delay] [--close] [--remote]

port|path

Network port to wait for opening or a path to UNIX socket. Regular expressions are also supported.
-t time

Timeout in seconds (optional, default=120). If the socket isn’t opened before the time elapses the command returns 1.
-p PID

PID of the process that should also be running. If the process exits before the socket is opened, the command returns with status code of 1.
-d delay

Delay between subsequent checks for availability of socket. Default 1.
–close

Wait for the socket to stop listening.
–remote

Wait for the remote socket to start listening instead of local one.

rlWait¶

Wrapper around bash builtin ‘wait’ command. See bash_builtins(1) man page. Kills the process and all its children if the timeout elapses.

rlWait [n ...] [-s SIGNAL] [-t time]

n

List of PIDs to wait for. They need to be background tasks of current shell. See bash_builtins(1) section for ‘wait’ command.
-t time

Timeout in seconds (optional, default=30). If the wait isn’t successful before the time elapses then all specified tasks are killed.
-s SIGNAL

Signal used to kill the process, optional SIGTERM by default.

Virtual X Server¶

Functions providing simple way how to start and stop virtual X server.

rlVirtualXStart¶

Start a virtual X server on a first free display. Tries only first N displays, so you can run out of them.

rlVirtualXStart name [N]

name

String identifying the X server.
N

Maximum number of displays to try. Defaults to 3.

Returns 0 when the server is started successfully.

rlVirtualXGetDisplay¶

Get the DISPLAY variable for specified virtual X server.

rlVirtualXGetDisplay name

name

String identifying the X server.

Displays the number of display where specified virtual X server is running to standard output. Returns 0 on success.

rlVirtualXStop¶

Kill the specified X server.

rlVirtualXStop name

name

String identifying the X server.

Returns 0 when the server is stopped successfully.

Example¶

Below is a simple example of usage. Note that a lot of usefull debugging information is reported on the DEBUG level, so you can run your test with DEBUG=1 make run to get them.

rlVirtualXStart $TEST
rlAssert0 "Virtual X server started" $?
export DISPLAY="$( rlVirtualXGetDisplay $TEST )"
# ...your test which needs X...
rlVirtualXStop $TEST
rlAssert0 "Virtual X server killed" $?

These are “Requires” lines for your scripts - note different package names for different RHEL versions:

@echo "Requires: xorg-x11-server-Xvfb" >> $(METADATA) # RHEL-5
@echo "Requires: xorg-x11-Xvfb"        >> $(METADATA) # RHEL-4
@echo "Requires: XFree86-Xvfb"         >> $(METADATA) # RHEL-3

rlYash_parse¶

Parse yaml data to the associative array.

rlYash_parse VAR_NAME YAML_DATA

VAR_NAME

Name of the variable to which the yaml structure will be saved.

Note that the variable needs to be predeclared as an associative array.
YAML_DATA

The actual yaml data.

Beakerlib Profiling¶

Enable profiling¶

Set environment variable BEAKERLIB_PROFILING=1

BEAKERLIB_PROFILING=1 make run

A file /dev/shm/beakerlib_profile will be created for later processing.

Process the profile¶

/usr/share/beakerlib/profiling.sh process > profile.csv

OUTPUT FILES¶

Location of test results related output files can be configured by setting BEAKERLIB_DIR variable before running the test. If it is not set, temporary directory is created.

journal.txt¶

Journal in human readable form.

journal.xml¶

Journal in XML format, requires python. This dependency can be avoided if the test is run with variable BEAKERLIB_JOURNAL set to 0 in which case journal.xml is not created.

XSLT¶

XML journal can be transformed through XSLT template. Which template is used is configurable by setting BEAKERLIB_JOURNAL variable. Value can be either filename in which case beakerlib will try to use $INSTALL_DIR/xslt-template/$filename (e.g.: /usr/share/beakerlib/xstl-templates/xunit.xsl) or it can be path to a template anywhere on the system.

TestResults¶

Overall results of the test in a ‘sourceable’ form. Each line contains a pair VAR=VALUE. All variable names have ‘TESTRESULT_’ prefix.

List of variables:¶

TESTRESULT_STATE

Current state of the test run; possible values: started, incomplete and complete. - ‘started’ is set after a Journal is opened. - ‘incomplete’ is set after a Phase is closed. - ‘complete’ is set after a Journal is closed.
TESTRESULT_RESULT_STRING

Result of the test in a string, e.g.: PASS, FAIL, WARN.
TESTRESULT_RESULT_ECODE

Result of the test as an integer, 0 equals to PASS.
TESTRESULT_PHASES_PASSED

Number of phases that ended with PASS.
TESTRESULT_PHASES_FAILED

Number of phases that ended with non-PASS result.
TESTRESULT_PHASES_SKIPPED

Number of skipped phases.
TESTRESULT_ASSERTS_FAILED

Number of asserts that ended with non-PASS result in the whole test.
TESTRESULT_STARTTIME

Time when test started in seconds since epoch.
TESTRESULT_ENDTIME

Time when test ended in seconds since epoch.
TESTRESULT_DURATION

Duration of the test run in seconds.
TESTRESULT_BEAKERLIB_DIR

Directory with test results files.
TESTRESULT_PHASES_FINGERPRINT

A hash of all phases results which may serve as a reference result in a specific environment.
TESTRESULT_ASSERTS_FINGERPRINT

A hash of all asserts results which may serve as a reference result in a specific environment.

EXAMPLES¶

Simple¶

A minimal BeakerLib test can look like this:

. /usr/share/beakerlib/beakerlib.sh

rlJournalStart
    rlPhaseStartTest
        rlAssertRpm "setup"
        rlAssertExists "/etc/passwd"
        rlAssertGrep "root" "/etc/passwd"
    rlPhaseEnd
rlJournalEnd

Phases¶

Here comes a bit more interesting example of a test which sets all the recommended variables and makes use of the phases:

# Include the BeakerLib environment
. /usr/share/beakerlib/beakerlib.sh

# Set the full test name
TEST="/examples/beakerlib/Sanity/phases"

# Package being tested
PACKAGE="coreutils"

rlJournalStart
    # Setup phase: Prepare test directory
    rlPhaseStartSetup
        rlAssertRpm $PACKAGE
        rlRun 'TmpDir=$(mktemp -d)' 0 'Creating tmp directory' # no-reboot
        rlRun "pushd $TmpDir"
    rlPhaseEnd

    # Test phase: Testing touch, ls and rm commands
    rlPhaseStartTest
        rlRun "touch foo" 0 "Creating the foo test file"
        rlAssertExists "foo"
        rlRun "ls -l foo" 0 "Listing the foo test file"
        rlRun "rm foo" 0 "Removing the foo test file"
        rlAssertNotExists "foo"
        rlRun "ls -l foo" 2 "Listing foo should now report an error"
    rlPhaseEnd

    # Cleanup phase: Remove test directory
    rlPhaseStartCleanup
        rlRun "popd"
        rlRun "rm -r $TmpDir" 0 "Removing tmp directory"
    rlPhaseEnd
rlJournalEnd

# Print the test report
rlJournalPrintText

The ouput of the rlJournalPrintText command would produce an output similar to the following:

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   TEST PROTOCOL
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

    Package       : coreutils
    Installed     : coreutils-8.27-19.fc27.x86_64
    beakerlib RPM : beakerlib-1.17-6.fc27.noarch
    Test started  : 2018-02-08 15:43:03 CET
    Test finished : 2018-02-08 15:43:04 CET
    Test duration : 1 seconds
    Test name     : /examples/beakerlib/Sanity/phases
    Distro        : Fedora release 27 (Twenty Seven)
    Hostname      : localhost
    Architecture  : x86_64
    CPUs          : 4 x Intel Core Processor (Skylake)
    RAM size      : 1992 MB
    HDD size      : 17.55 GB

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Test description
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

PURPOSE of /examples/beakerlib/Sanity/phases
Description: Testing BeakerLib phases
Author: Petr Splichal <psplicha@redhat.com>

This example shows how the phases work in the BeakerLib on a
trivial smoke test for the "touch", "ls" and "rm" commands from
the coreutils package.



::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Setup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 15:43:03 ] :: [   PASS   ] :: Checking for the presence of coreutils rpm
:: [ 15:43:03 ] :: [   LOG    ] :: Package versions:
:: [ 15:43:03 ] :: [   LOG    ] ::   coreutils-8.27-19.fc27.x86_64
:: [ 15:43:03 ] :: [   PASS   ] :: Creating tmp directory (Expected 0, got 0)
:: [ 15:43:03 ] :: [   PASS   ] :: Command 'pushd /tmp/tmp.MMQf7dj9QV' (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 1s
::   Assertions: 3 good, 0 bad
::   RESULT: PASS


::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Test
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 15:43:04 ] :: [   PASS   ] :: Creating the foo test file (Expected 0, got 0)
:: [ 15:43:04 ] :: [   PASS   ] :: File foo should exist
:: [ 15:43:04 ] :: [   PASS   ] :: Listing the foo test file (Expected 0, got 0)
:: [ 15:43:04 ] :: [   PASS   ] :: Removing the foo test file (Expected 0, got 0)
:: [ 15:43:04 ] :: [   PASS   ] :: File foo should not exist
:: [ 15:43:04 ] :: [   PASS   ] :: Listing foo should now report an error (Expected 2, got 2)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 6 good, 0 bad
::   RESULT: PASS


::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Cleanup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 15:43:04 ] :: [   PASS   ] :: Command 'popd' (Expected 0, got 0)
:: [ 15:43:04 ] :: [   PASS   ] :: Removing tmp directory (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 2 good, 0 bad
::   RESULT: PASS


::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   /examples/beakerlib/Sanity/phases
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 15:43:04 ] :: [   LOG    ] :: Phases fingerprint:  ZmU1NmJ
:: [ 15:43:04 ] :: [   LOG    ] :: Asserts fingerprint: YTBiZDE
:: [ 15:43:04 ] :: [   LOG    ] :: JOURNAL XML: /var/tmp/beakerlib-pRiJfWE/journal.xml
:: [ 15:43:04 ] :: [   LOG    ] :: JOURNAL TXT: /var/tmp/beakerlib-pRiJfWE/journal.txt
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 1s
::   Phases: 3 good, 0 bad
::   OVERALL RESULT: PASS

Note that the detailed test description is read from a separate file PURPOSE placed in the same directory as the test itself.

A lot of useful debugging information is reported on the DEBUG level. You can run your test with DEBUG=1 make run to get them.

Verbosity of the test logging may be altered also by setting the LOG_LEVEL variable. Possible values are “ERROR”, “WARNING”, “INFO” and “DEBUG”. You will see messages like the ones below.

:: [ WARNING  ] :: rlGetArch: Update test to use rlGetPrimaryArch/rlGetSecondaryArch
:: [  DEBUG   ] :: rlGetArch: This is architecture 'x86_64'
:: [  ERROR   ] :: this function was dropped as its development is completely moved to the beaker library
:: [   INFO   ] :: if you realy on this function and you really need to have it present in core beakerlib, file a RFE, please

Setting LOG_LEVEL=”DEBUG” is equivalent to DEBUG=1.

Set DEBUG_TO_CONSOLE_ONLY=1 in conjuction with DEBUG=1 to avoid storing debug messages in journal. This also speeds up the execution in debug mode as those messages are not fully processed than.

BKRDOC¶

Description

Bkrdoc is a documentation generator from tests written using BeakerLib library. This generator makes documentation from test code with and also without any documentation markup.
What it’s good for

For fast, brief and reliable documentation creation. It`s good for quick start with unknown BeakerLib test. Created documentations provides information about the documentation credibility. Also created documentations shows environmental variables and helps reader to run test script from which was documentation created.
Bkrdoc project page

https://github.com/rh-lab-q/bkrdoc

LINKS¶

Project Page

https://github.com/beakerlib/beakerlib
Manual

https://github.com/beakerlib/beakerlib/wiki/man
Issues list

https://github.com/beakerlib/beakerlib/issues
Reporting issues

https://github.com/beakerlib/beakerlib/issues/new

AUTHORS¶

Petr Muller pmuller@redhat.com
Ondrej Hudlicky ohudlick@redhat.com
Jan Hutar jhutar@redhat.com
Petr Splichal psplicha@redhat.com
Ales Zelinka azelinka@redhat.com
Dalibor Pospisil dapospis@redhat.com
Martin Kyral mkyral@redhat.com
Jakub Prokes jprokes@redhat.com
Jakub Heger jheger@redhat.com