X11::GUITest - Provides GUI testing/interaction routines.
0.29
Updates are made available at the following sites:
http\://sourceforge.net/projects/x11guitest
http\://www.cpan.orgPlease consult 'Changes' for the list of changes between module revisions.
This Perl package is intended to facilitate the testing of GUI applications by means of user emulation. It can be used to test/interact with GUI applications; which have been built upon the X library or toolkits (i.e., GTK+, Xt, Qt, Motif, etc.) that "wrap" the X library's functionality.
A basic recorder (x11guirecord) is also available, and can be found in the source code repository.
An X server with the XTest extensions enabled. This seems to be the norm. If it is not enabled, it usually can be by modifying the X server configuration (i.e., XF86Config).
This module talks to the X server via the standard DISPLAY environment variable, exactly like any other X11 client. By default DISPLAY is usually ":0.0" (the local machine). To target a different X server -- for example, one on a remote host -- set DISPLAY before running your script:
export DISPLAY=<hostname-or-ipaddress>:<display>.<screen>Under most circumstances, xhost will also need to be run on the remote host to permit the connection.
DISPLAY only helps if there is an actual X11 server to point it at. This module requires a real X11 server -- either a native X11 session, or XWayland running as an X11 compatibility layer inside a Wayland session. It cannot function under a pure Wayland session with no XWayland present. See WAYLAND NOTES below for how to diagnose and work around this on a Wayland desktop.
There is a known incompatibility between the XTest and Xinerama extensions, which causes the XTestFakeMotionEvent() function to misbehave when the Xinerama (X server) extension is active. MoveMouseAbs() has an alternate, XWarpPointer()-based implementation for this case, but it is not enabled by default -- it requires rebuilding with the X11_GUITEST_USING_XINERAMA preprocessor macro defined *in addition to* this module's other defines (a DEFINE= override on the command line replaces the whole value, it does not add to it):
perl Makefile.PL DEFINE="-DNDEBUG -DX11_GUITEST_ALT_L_FALLBACK_META_L -DX11_GUITEST_USING_XINERAMA"
make
make installOnly do this if Xinerama is actually active on the target X server; otherwise leave the module at its default build.
The optional x11guirecord recorder utility (see 'recorder/' and INSTALLATION below) additionally requires the X server's RECORD extension. This is separate from XTest, and while common, isn't guaranteed to be present/enabled on every X server -- particularly minimal or older commercial X implementations.
perl Makefile.PL
make
make test
make install
# If "make install" fails with a permissions error, see INSTALLATION
# NOTES below.
# If the build has errors, you may need to install the following dependencies:
# libxt-dev, libxtst-dev
# If you'd like to install the recorder, use these steps:
cd recorder
./autogen.sh
./configure
make
make install
x11guirecord --help
# See INSTALLATION NOTES below if this "make install" fails with a
# permissions error too.
# The recorder has its own build-time dependencies, separate from the
# above:
# autoconf, automake (needed to run autogen.sh)
# popt development files (e.g. popt-devel, libpopt-dev)
# libX11, libXtst development files, if not already installed
# for the main module above
#
# GNU gettext (libintl.h) is optional -- if it isn't found, the recorder
# builds without translation support rather than failing. This is
# relevant mainly on AIX and HP-UX, which don't provide GNU gettext
# natively.make install writes into your Perl's installation directories (e.g. site_perl under /usr/lib/perl5 or /usr/local). If that location isn't writable by your user -- the common case for a system Perl -- rerun the last step as sudo make install. Perl setups that install to a user-writable location instead (local::lib, perlbrew, plenv, etc.) normally don't need sudo. If you're not sure which applies, try plain make install first and only add sudo if it fails with a permissions error.
The same applies to the recorder's make install step.
For additional examples, please look under the 'eg/' sub-directory from the installation folder.
use X11::GUITest qw/
StartApp
WaitWindowViewable
SendKeys
/;
# Start gedit application
# Note: The interface for this example application keeps changing.
StartApp('gedit');
# Wait for application window to come up and become viewable.
my ($GEditWinId) = WaitWindowViewable('Untitled Document 1');
if (!$GEditWinId) {
die("Couldn't find gedit window in time!");
}
# Send text to it
SendKeys("Hello, how are you?\n");
# Close Application (Alt-f, q).
SendKeys('%(f)q');
# Handle gedit's Question window if it comes up when closing. Wait
# at most 5 seconds for it.
if (WaitWindowViewable('Question', undef, 5)) {
# Don't Save (Alt-n)
SendKeys('%(n)');
}Even under XWayland, be aware that Wayland's per-application isolation model prevents synthetic input (sent via the XTest extension, which this module relies on) from reaching native Wayland clients -- only X11/XWayland-rooted windows will respond to it. This is a deliberate security property of the Wayland protocol, not a bug, and there is no compile-time or runtime workaround. See https://www.linuxteck.com/x11-vs-wayland/ for background on the X11 vs. Wayland input model.
On a Wayland session (check with 'echo $XDG_SESSION_TYPE'), most applications default to running as native Wayland clients, invisible to this module no matter what DISPLAY is set to. Forcing a specific application through XWayland instead is usually enough to fix that. Which environment variable to set depends on the GUI toolkit the target application is built with:
Toolkit Variable to set Example
------- --------------- -------
GTK (GNOME, etc.) GDK_BACKEND=x11 GDK_BACKEND=x11 gedit &
Qt / KDE Plasma QT_QPA_PLATFORM=xcb QT_QPA_PLATFORM=xcb dolphin &Two gotchas that can make this appear not to work, regardless of toolkit:
Many apps use single-instance activation (GTK's GApplication or KDE's KDBusService, both implemented over D-Bus). If the application is already running, a new invocation with the variable set just signals the existing (Wayland) process to open a window -- it does not start a new, X11-backed process. Fully quit the application first (i.e., confirm no process is running), then relaunch it.
Flatpak-packaged applications (increasingly the default for some apps on Fedora) do not automatically inherit environment variables from the host shell. Use 'flatpak run --env=GDK_BACKEND=x11 <app-id>' (substitute QT_QPA_PLATFORM=xcb for Qt/KDE apps) instead of GDK_BACKEND=x11 <app> &, and use 'flatpak list --app' to find the app id if it's not obvious from the command name.
If neither works, or you need this reliably for many applications, the more robust fix is a session-level one: log out and choose an X11 session at the login screen, if your desktop still offers one -- e.g. "GNOME on Xorg" or "Plasma (X11)" (some recent GNOME/Fedora releases have deprecated or removed that option).
Parameters enclosed within [] are optional.
If there are multiple optional parameters available for a function and you would like to specify the last one, for example, you can utilize undef for those parameters you don't specify.
REGEX in the documentation below denotes an item that is treated as a regular expression. For example, the regex "^OK$" would look for an exact match for the word OK.
Finds the window Ids of the windows matching the specified title regex. Optionally one can specify the window to start under; which would allow one to constrain the search to child windows of that window.
An array of window Ids is returned for the matches found. An empty array is returned if no matches were found.
my @WindowIds = FindWindowLike('gedit');
# Only worry about first window found
my ($WindowId) = FindWindowLike('gedit');Waits for a window to come up that matches the specified title regex. Optionally one can specify the window to start under; which would allow one to constrain the search to child windows of that window.
One can optionally specify an alternative wait amount in seconds. A window will keep being looked for that matches the specified title regex until this amount of time has been reached. The default amount is defined in the DEF_WAIT constant available through the \:CONST export tag.
If a window is going to be manipulated by input, WaitWindowViewable is the more robust solution to utilize.
An array of window Ids is returned for the matches found. An empty array is returned if no matches were found.
my @WindowIds = WaitWindowLike('gedit');
# Only worry about first window found
my ($WindowId) = WaitWindowLike('gedit');
WaitWindowLike('gedit') or die("gedit window not found!");Similar to WaitWindow, but only recognizes windows that are viewable. When GUI applications are started, their window isn't necessarily viewable yet, let alone available for input, so this function is very useful.
Likewise, this function will only return an array of the matching window Ids for those windows that are viewable. An empty array is returned if no matches were found.
Waits for the specified window to close.
One can optionally specify an alternative wait amount in seconds. The window will keep being checked to see if it has closed until this amount of time has been reached. The default amount is defined in the DEF_WAIT constant available through the \:CONST export tag.
zero is returned if window is not gone, non-zero if it is gone.
Pauses execution for the specified amount of seconds.
WaitSeconds(0.5); # Wait 1/2 second
WaitSeconds(3); # Wait 3 secondsClicks on the specified window with the mouse.
Optionally one can specify the X offset and Y offset. By default, the top left corner of the window is clicked on, with these two parameters one can specify a different position to be clicked on.
One can also specify an alternative button. The default button is M_LEFT, but M_MIDDLE and M_RIGHT may be specified too. Also, you could use the logical Id for the button: M_BTN1, M_BTN2, M_BTN3, M_BTN4, M_BTN5. These are all available through the \:CONST export tag.
zero is returned on failure, non-zero for success
Returns a list of window ids discovered for the specified process id (pid).
undef is returned on error.
Returns the window that is at the specified point. If no screen is given, it is taken from the value given when opening the X display.
zero is returned if there are no matches (i.e., off screen).
Determines if the specified window is a child of the specified parent.
zero is returned for false, non-zero for true.
Quotes {} characters in the specified string that would be interpreted as having special meaning if sent to SendKeys directly. This function would be useful if you had a text file in which you wanted to use each line of the file as input to the SendKeys function, but didn't want any special interpretation of the characters in the file.
Returns the quoted string, undef is returned on error.
# Quote ~, %, etc. as {~}, {%}, etc for literal use in SendKeys.
SendKeys( QuoteStringForSendKeys('Hello: ~%^(){}+#') );
SendKeys( QSfSK('#+#') );The international AltGr key - modifier character (&) is not escaped by this function. Escape this character manually ("{&}"), if used/needed.
Starts a program directly using exec. This function returns after checking that the child process remains running. Useful for starting GUI /applications and then going on to work with them.
zero is returned on failure, non-zero for success
StartApp('gedit');Environment variable assignments (e.g. GDK_BACKEND=x11, may be needed to force a GTK application through XWayland instead of native Wayland -- see DEPENDENCIES above) can be prefixed onto COMMANDLINE, provided the whole thing is passed as a single string:
StartApp('GDK_BACKEND=x11 gedit'); # Works.
StartApp('GDK_BACKEND=x11', 'gedit'); # Does NOT work.Uses the shell to execute a program until its completion.
Return value will be application specific, however -1 is returned to indicate a failure in starting the program.
RunApp('/work/myapp');Clicks the specified mouse button. Available mouse buttons are: M_LEFT, M_MIDDLE, M_RIGHT, M_DOWN, M_UP. Also, you could use the logical Id for the button: M_BTN1, M_BTN2, M_BTN3, M_BTN4, M_BTN5. These are all available through the \:CONST export tag.
zero is returned on failure, non-zero for success.
Returns the screen number specified in the X display value used to open the display.
Leverages the Xlib macro of the same name.
Returns the number of screens in the X display specified when opening it.
Leverages the Xlib macro of the same name.
Sets the milliseconds of delay between events being sent to the X display. It is usually not a good idea to set this to 0.
Please note that this delay will also affect SendKeys.
Returns the old delay amount in milliseconds.
Returns the current event sending delay amount in milliseconds.
Sets the milliseconds of delay between keystrokes.
Returns the old delay amount in milliseconds.
Returns the current keystroke sending delay amount in milliseconds.
Returns the window name for the specified window Id. undef is returned if name could not be obtained.
# Return the name of the window that has the input focus.
my $WinName = GetWindowName(GetInputFocus());Returns the process id (pid) associated with the specified window. 0 is returned if the pid is not available.
# Return the pid of the window that has the input focus.
my $pid = GetWindowPid(GetInputFocus());Sets the window name for the specified window Id.
zero is returned on failure, non-zero for success.
Returns the Id of the root window of the screen. This is the top/root level window that all other windows are under. If no screen is given, it is taken from the value given when opening the X display.
Returns an array of the child windows for the specified window Id. If it detects that the window hierarchy is in transition, it will wait half a second and try again.
Moves the mouse cursor to the specified absolute position in the optionally given screen. If no screen is given, it is taken from the value given when opening the X display.
Zero is returned on failure, non-zero for success.
Returns an array containing the position and the screen (number) of the mouse cursor.
my ($x, $y, $scr_num) = GetMousePos();Presses the specified mouse button. Available mouse buttons are: M_LEFT, M_MIDDLE, M_RIGHT, M_DOWN, M_UP. Also, you could use the logical Id for the button: M_BTN1, M_BTN2, M_BTN3, M_BTN4, M_BTN5. These are all available through the \:CONST export tag.
zero is returned on failure, non-zero for success.
Releases the specified mouse button. Available mouse buttons are: M_LEFT, M_MIDDLE, M_RIGHT, M_DOWN, M_UP. Also, you could use the logical Id for the button: M_BTN1, M_BTN2, M_BTN3, M_BTN4, M_BTN5. These are all available through the \:CONST export tag.
zero is returned on failure, non-zero for success.
Sends keystrokes to the window that has the input focus.
The keystrokes to send are those specified in KEYS parameter. Some characters have special meaning, they are:
Modifier Keys:
^ CTRL
% ALT
+ SHIFT
# META
& ALTGR
Other Keys:
~ ENTER
\n ENTER
\t TAB
( and ) MODIFIER GROUPING
{ and } QUOTE / ESCAPE CHARACTERSSimply, one can send a text string like so:
SendKeys('Hello, how are you today?');Parenthesis allow a modifier to work on one or more characters. For example:
SendKeys('%(f)q'); # Alt-f, then press q
SendKeys('%(fa)^(m)'); # Alt-f, Alt-a, Ctrl-m
SendKeys('+(abc)'); # Uppercase ABC using shift modifier
SendKeys('^(+(l))'); # Ctrl-Shift-l
SendKeys('+'); # Press shiftBraces are used to quote special characters, for utilizing aliased key names, or for special functionality. Multiple characters can be specified in a brace by space delimiting the entries. Characters can be repeated using a number that is space delimited after the preceding key.
Quote Special Characters
SendKeys('{{}'); # {
SendKeys('{+}'); # +
SendKeys('{#}'); # #
You can also use QuoteStringForSendKeys (QSfSK) to perform quoting.Aliased Key Names
SendKeys('{BAC}'); # Backspace
SendKeys('{F1 F2 F3}'); # F1, F2, F3
SendKeys('{TAB 3}'); # Press TAB 3 times
SendKeys('{SPC 3 a b c}'); # Space 3 times, a, b, cSpecial Functionality
# Pause execution for 500 milliseconds
SendKeys('{PAUSE 500}');Combinations
SendKeys('abc+(abc){TAB PAUSE 500}'); # a, b, c, A, B, C, Tab, Pause 500
SendKeys('+({a b c})'); # A, B, CThe following abbreviated key names are currently recognized within a brace set. If you don't see the desired key, you can still use the unabbreviated name for the key. If you are unsure of this name, utilize the xev (X event view) tool, press the key you want and look at the tools output for the name of that key. Names that are in the list below can be utilized regardless of case. Ones that aren't in this list are going to be case sensitive and also not abbreviated. For example, using 'xev' you will find that the name of the backspace key is BackSpace, so you could use {BackSpace} in place of {bac} if you really wanted to.
Name Action
-------------------
BAC BackSpace
BS BackSpace
BKS BackSpace
BRE Break
CAN Cancel
CAP Caps_Lock
DEL Delete
DOWN Down
END End
ENT Return
ESC Escape
F1 F1
... ...
F12 F12
HEL Help
HOM Home
INS Insert
LAL Alt_L
LMA Meta_L
LCT Control_L
LEF Left
LSH Shift_L
LSK Super_L
MNU Menu
NUM Num_Lock
PGD Page_Down
PGU Page_Up
PRT Print
RAL Alt_R
RMA Meta_R
RCT Control_R
RIG Right
RSH Shift_R
RSK Super_R
SCR Scroll_Lock
SPA Space
SPC Space
TAB Tab
UP Upzero is returned on failure, non-zero for success. For configurations (Xvfb) that don't support Alt_Left, Meta_Left is automatically used in its place.
Presses the specified key.
One can utilize the abbreviated key names from the table listed above as outlined in the following example:
# Alt-n
PressKey('LAL'); # Left Alt
PressKey('n');
ReleaseKey('n');
ReleaseKey('LAL');
# Uppercase a
PressKey('LSH'); # Left Shift
PressKey('a');
ReleaseKey('a');
ReleaseKey('LSH');The ReleaseKey calls in the above example are there to set both key states back.
zero is returned on failure, non-zero for success.
Releases the specified key. Normally follows a PressKey call.
One can utilize the abbreviated key names from the table listed above.
ReleaseKey('n');zero is returned on failure, non-zero for success.
Presses and releases the specified key.
One can utilize the abbreviated key names from the table listed above.
PressReleaseKey('n');This function is affected by the key send delay.
zero is returned on failure, non-zero for success.
Determines if the specified key is currently being pressed.
You can specify such things as 'bac' or the unabbreviated form 'BackSpace' as covered in the SendKeys information. Brace forms such as '{bac}' are unsupported. A '{' is taken literally and letters are case sensitive.
if (IsKeyPressed('esc')) { # Is Escape pressed?
if (IsKeyPressed('a')) { # Is a pressed?
if (IsKeyPressed('A')) { # Is A pressed?Returns non-zero for true, zero for false.
Determines if the specified mouse button is currently being pressed.
Available mouse buttons are: M_LEFT, M_MIDDLE, M_RIGHT. Also, you could use the logical Id for the button: M_BTN1, M_BTN2, M_BTN3, M_BTN4, M_BTN5. These are all available through the \:CONST export tag.
if (IsMouseButtonPressed(M_LEFT)) { # Is left button pressed?Returns non-zero for true, zero for false.
zero is returned if the specified window Id is not for something that can be recognized as a window. non-zero is returned if it looks like a window.
zero is returned if the specified window Id is for a window that isn't viewable. non-zero is returned if the window is viewable.
Determines if the specified window has the specified cursor.
zero is returned for false, non-zero for true.
The following cursors are available through the \:CONST export tag.
Name
-------------------
XC_NUM_GLYPHS
XC_X_CURSOR
XC_ARROW
XC_BASED_ARROW_DOWN
XC_BASED_ARROW_UP
XC_BOAT
XC_BOGOSITY
XC_BOTTOM_LEFT_CORNER
XC_BOTTOM_RIGHT_CORNER
XC_BOTTOM_SIDE
XC_BOTTOM_TEE
XC_BOX_SPIRAL
XC_CENTER_PTR
XC_CIRCLE
XC_CLOCK
XC_COFFEE_MUG
XC_CROSS
XC_CROSS_REVERSE
XC_CROSSHAIR
XC_DIAMOND_CROSS
XC_DOT
XC_DOTBOX
XC_DOUBLE_ARROW
XC_DRAFT_LARGE
XC_DRAFT_SMALL
XC_DRAPED_BOX
XC_EXCHANGE
XC_FLEUR
XC_GOBBLER
XC_GUMBY
XC_HAND1
XC_HAND2
XC_HEART
XC_ICON
XC_IRON_CROSS
XC_LEFT_PTR
XC_LEFT_SIDE
XC_LEFT_TEE
XC_LEFTBUTTON
XC_LL_ANGLE
XC_LR_ANGLE
XC_MAN
XC_MIDDLEBUTTON
XC_MOUSE
XC_PENCIL
XC_PIRATE
XC_PLUS
XC_QUESTION_ARROW
XC_RIGHT_PTR
XC_RIGHT_SIDE
XC_RIGHT_TEE
XC_RIGHTBUTTON
XC_RTL_LOGO
XC_SAILBOAT
XC_SB_DOWN_ARROW
XC_SB_H_DOUBLE_ARROW
XC_SB_LEFT_ARROW
XC_SB_RIGHT_ARROW
XC_SB_UP_ARROW
XC_SB_V_DOUBLE_ARROW
XC_SHUTTLE
XC_SIZING
XC_SPIDER
XC_SPRAYCAN
XC_STAR
XC_TARGET
XC_TCROSS
XC_TOP_LEFT_ARROW
XC_TOP_LEFT_CORNER
XC_TOP_RIGHT_CORNER
XC_TOP_SIDE
XC_TOP_TEE
XC_TREK
XC_UL_ANGLE
XC_UMBRELLA
XC_UR_ANGLE
XC_WATCH
XC_XTERMMoves the window to the specified location.
zero is returned on failure, non-zero for success.
Resizes the window to the specified size.
zero is returned on failure, non-zero for success.
Minimizes (Iconifies) the specified window.
zero is returned on failure, non-zero for success.
Unminimizes (UnIconifies) the specified window.
zero is returned on failure, non-zero for success.
Raises the specified window to the top of the stack, so that no other windows cover it.
zero is returned on failure, non-zero for success.
Lowers the specified window to the bottom of the stack, so other existing windows will cover it.
zero is returned on failure, non-zero for success.
Returns the window that currently has the input focus.
Sets the specified window to be the one that has the input focus.
zero is returned on failure, non-zero for success.
Returns an array containing the position information for the specified window. It also returns size information (including border width) and the number of the screen where the window resides.
my ($x, $y, $width, $height, $borderWidth, $screen) =
GetWindowPos(GetRootWindow());Returns the parent of the specified window.
zero is returned if parent couldn't be determined (i.e., root window).
Returns the color depth for the screen. If no screen is specified, it is taken from the value given when opening the X display. If the screen (number) is invalid, -1 will be returned.
Value is represented as bits, i.e. 16.
my $depth = GetScreenDepth();Returns the screen resolution. If no screen is specified, it is taken from the value given when opening the X display. If the screen (number) is invalid, the returned list will be empty.
my ($x, $y) = GetScreenRes();
Module Changes
Coding-Style Guidelines
ToDo List
Copy of the GPL License
Copyright(c) 2003-2026 Dennis K. Paulsen, All Rights Reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License.
Dennis K. Paulsen <ctrondlp@cpan.org> (Iowa USA)
Paulo E. Castro <pauloedgarcastro tata gmail.com>
Thanks to everyone; including those specifically mentioned below for patches, suggestions, etc.:
David Dick
Alexey Tourbin
Richard Clamp
Gustav Larsson
Nelson D. Caro