Surface

Surface plugin has been designed with image recognition capabilities to capture images/regions on a screen. The captured image can be used for image search, mouse Actions on the image, image labeling and optical character recognition (OCR).

Surface Automation is particularly useful on remote machines where we do not have any handle to the content and everything is an image or pattern. It can be useful to work on remote environments such as Citrix.

Prerequisite:

• For surface plugin steps, Microsoft Visual C++ version 2015-2019 Redistributable (x64) and Microsoft Visual C++ version 2015-2019 Redistributable (x86) should be installed.

  • This is available at the following URL,

  • https://support.microsoft.com/en-in/help/2977003/the-latest-supported-visual-c- downloads

• Download the following two executable:

  • x86: vc_redist.x86.exe

  • x64: vc_redist.x64.exe

• Surface spy is required to build workflows with Surface plugin steps. For a description on how to use Surface Spy refer the following,

Additional Prerequisite for Windows Server OS:

Surface Plugins use some of Windows Media Service DLL's (ex.mfplat.dll) for image recognition. You need to enable "Desktop Experience" windows feature to get the required DLL's. Following are the steps to install the required Windows Media Service - DLL's

1. On Windows Start menu search for 'Turn windows features on or off'

2. Skip the roles screen and directly go to Feature screen

3. Select "Desktop Experience" under "User Interfaces and Infrastructure"

4. Restart the system.

Limitation: Any type of UI Automation with Surface plugin steps is not supported in Linux based OS.

References:

For a description on how to use Surface Spy refer to Appendix 4: Surface Spy Tutorial.

tip

-If the field appears as a widget, then it can handle both Text (static value or environment variable) and Combo (drop down containing values from previous steps).

OR

If the field is a text box, then you can provide static values only.
OR
If is available, then support is provided for selecting environment variables or static values.
OR

If dropdown is available, it supports available static values only.