Text Recognition Support

Text recognition methods enable you to conveniently interact with test applications that contain highly customized controls, which cannot be identified using object recognition. You can use text clicks instead of coordinate-based clicks to click on a specified text string within a control.

For example, you can simulate selecting the first cell in the second row of the following table:

Specifying the text of the cell results in the following code:

table.textClick("Brian Miller");

Text recognition methods are supported for the following technology domains:

Win32.
WPF.
Windows Forms.
Java SWT and Eclipse.
Java AWT/Swing.
Note: For Java Applets, and for Swing applications with Java versions prior to version 1.6.10, text recognition is supported out-of-the-box. For Swing applications with Java version 1.6.10 or later, which do not support Direct3D, you have to add the following command-line element when starting the application:
```
-Dsun.java2d.d3d=false
```
For example:
```
javaw.exe -Dsun.java2d.d3d=false -jar mySwingApplication.jar
```
Text recognition is not supported for Java Applets and Swing applications that support Direct3D.
Internet Explorer.
WebDriver-based browsers.

Note: Text recognition does not work with controls that are not visible on the screen. For example, you cannot use text recognition for a text that is scrolled out of view.

Note: Text recognition might not work if the font that is used in the target text is not installed on the machine on which the test is executed.

WebDriver-based browsers

The text recognition methods can be applied to BrowserWindow and DomElement objects.

Note: Text recognition does not work for text that is drawn in <canvas> elements.

Note: Text recognition does not work for content added by CSS pseudo-elements like ::before and ::after.

Text recognition methods

Silk4J offers the following methods to drive testing through interacting with the text that the AUT renders on the screen:

TextCapture: Returns the text that is within a control. Also returns text from child controls.
TextClick: Clicks on a specified text within a control. Waits until the text is found or the Object resolve timeout, which you can define in the synchronization options, is over.
TextRectangle: Returns the rectangle of a certain text within a control or a region of a control.
TextExists: Determines whether a given text exists within a control or a region of a control.

The text recognition methods prefer whole word matches over partially matched words. Silk4J recognizes occurrences of whole words previously than partially matched words, even if the partially matched words are displayed before the whole word matches on the screen. If there is no whole word found, the partly matched words will be used in the order in which they are displayed on the screen.

The methods TextClick, TextRectangle, and TextExists internally use TextCapture to grab the visible text from the application and allow for further processing of that text. The underlying TextCapture method is implemented in two different ways. Silk4J decides which implementation to use depending on the type of the application under test.

For native windows applications, including WPF, WinForms, and Java applications, but also Internet Explorer, Silk4J hooks into the text rendering functions of the Windows API to extract the text that the application draws on the screen.
For Google Chrome, Mozilla Firefox, Microsoft Edge, and Apple Safari, Silk4J uses a JavaScript-based approach to retrieve the text after it was rendered by the browser.

Note: Because of the different nature of these two implementations, Silk4J might return different text for the same web application, depending on which browser is used.

Example

The user interface displays the text the hostname is the name of the host. The following code clicks on host instead of hostname, although hostname is displayed before host on the screen:

control.textClick("host");

The following code clicks on the substring host in the word hostname by specifying the second occurrence:

control.textClick("host", 2);