How to Test with Visual Regression Testing Tool- Screenster?

Introduction to Visual Testing- Screenster

Screenster.io  is a cloud-based automation tool that can automate visual regression testing of any web-based application. It offers a record and playback option just like other automation tools have. But unlike Microfocus UFT and like ZapTest it offers codeless automation. Apart from codeless automation, this tool offers several other features.

Screenster was introduced to the market in 2014. But As per their official Facebook page, it was introduced in 2016. Slogan- 10-12 * faster UI automation.

Why virtual regression testing is important?

Layout issues, bad or broken CSS, font issues are not possible to capture easily via traditional automation where these tools look for id, XPath, or use some unique identifier to find a single element. So, traditional tools leave to check the rich UIs and if they are broken in UIs.

How Screenster is different from other tools?

• Screenster is best for stakeholders ( functional experts, project managers and business owners) who need early testing using the shift-left approach. This will help the team to quickly deliver test script without deploying many testers.
• Popular IDEs were developed to the rapid creation of tests without understanding the UAT’s underlying technology. Like (UFT, OATs, Test complete etc).
• Most IDEs converts the user’s interaction into some kind of coding/ scripting language automatically. Now the issue with this approach to update or maintain company has to take employees with that background who can understand the language and update whenever necessary. Sometimes, due to lack of resources, existing employees need to learn the language and troubleshoot the issues.
• The other factors that a test engineer need to take care are the messy auto-generated code, hardcoded values, making non-reusable plain code to reusable functions locators are very simple during recording but fails during execution of test script.
• Overall, the in sprint automation never happen, automation is always pushed to sprint +1 or sprint+2. It is delayed and laid back. Automation cannot come up with a shift left strategy.
• Screenster can identify broken style sheet or even if the style sheet has been changed. It can detect a wrong image, test (content) or if a button is not visible to testers.
• When the majority of tools only care about the UI and functionality via code, Screenster cares about visual testing by smart screenshot capturing. It compares the actual image with the stored image pixel by pixel and generates the report.
• Interestingly when we test using screenshot comparison the tolerance becomes 0%. If an extra line gets added or screen resolution gets changed due to antialiasing effect tests will fail. Screenster manages these falls alarms very well by combining DOM screenshots. So Screenster can identify and hands shifted content, elements and dynamic areas.

Features of Screenster

1. Fast, reliable and codeless UI automation.
2. Works on very minimal setup (initial).
3. It does not require any browser plugin.
4. It is not required to install the tool in the local desktop.
5. Very less learning curve.
6. Provides a hierarchical view of test assets.
7. Supports parameterized DataDriven testing.
8. Supports test script review and update on the fly.
9. Supports copy and move test components.
10. Supports requirement driven testing.
11. Supports end to end regression functional testing.
12. Supports core browser testing, parallel execution, self-healing testing.
13. Screenster can ignore certain portion (dynamic region). This feature can be used to ignore a certain portion of the web page during record and playback.
14. Screenster supports import and export projects as zip format. However, it creates test/project backup in the cloud storage.
15. No technical non-coders (functional testers) can be more productive.

Pricing details:

Screenster offers a license approx $100.00 per user per month. It is charged regardless of Servers. Read here for details about pricing. It offers free trial too.

Major Customer of Screenster

Screenster upon launching drew attraction to the agile-based development teams. The major customer is Samsung.

System requirements for screenster

How Screenster works?

Screenster provides three basic functionality (Record, Run, Review). So Screenster works on the 3R principle.

How Screnster works?

Project and Test suite in Screenster

As soon as we login to the screenster, we see all projects (we are part of). Once we click on the project (to enter into the project), we get the project dashboard. Hence we get all the tests (Each test can be executed or edited).

Once we navigate the suites (Separate tab). We can create a suite by providing suite Name and Description. We can drag and drop tests to create the suite (This is reusability feature).

Record details of Screenstrer

Record- As soon as we interact with page Screenstere records all our actions and stores them inside. We do not require any coding. No selenium like a strategy to find the correct element. No UFT like checkpoint addition and selenium like assertions.

Recording mode- When users record a test script screenster captures UI screenshots and DOM screenshots. These snapshots are used to compare with actuals during execution. If it can not find an exact match it points out the anomalies.

Interestingly of the new GUI is the latest and instead of creating a bug we want to update the existing script with the latest snapshots, we can baseline the latest with a single click.

Screenster renders the UAT page as a snapshot (image) and stores as a baseline. Practically when we complete recording of a test, navigation manually, the automation script is completed at the time only.

How to record a test in screenster?

In order to record a test in screenster, we need to perform the following steps –

1. Launch screenster URL.
2. Login to screenster with your credentials.
3. Select/create a new project.
4. To create a new project, click on create a project button and provide a valid name and description > click save.
5. Once the project is opened click on create test button.
6. Select recording mode – (Record my Action)
7. Select Browser as Chrome
8. Select screen size laptop
9. Click on Record
10. Once a new blank page opens, provide your valid URL and navigate.
11. Start recording your page action.
12. Once you are done with recording click Save Test button to stop and save the test.

What are the different recording modes available in Screenster?

Different recording modes are –

1. Create manually
2. Record my action
3. Code with selenium

What are the different browsers supported by Screenster?

Different Browser modes-

1. Chrome
2. Firefox
3. IE

What are the resolutions supported by Screenster?

Different screen size – (Resolution)

1. Laptop
2. Desktop
3. iPhone
4. iPad
5. Samsung Galaxy SS
6. Samsung Galaxy Tab
7. Any custom size

How to insert Random text/dynamic test into your test?

Screenster provides some random options to enter into the test. They are as follows –

1. ‘${system.time}’ > current time.
2. ‘${system.date}’ > current date.
3. ‘${system random}’ > random value.

Execution details of Screenstrer

Run- Runs the action automatically after optimizing the code internally. It offers no manual intervention, automatic execution and self-healing of tests.

How to provide parameters to Screenster test using code?

Upon successful test recording, screenster can automatically extract the URL, editable bases to provide parameters. Then it will ask us to extract the parameters from the test. Screenster will open a grid-like view for its parameterization.

Syntax to add the parameter –

The system to feed parameters are as follows –

• URL
• input#page form_Control
• textareaform_Control.

How to pass/extract parameter from a recorded test?

To extract a parameter from a recorded test, we need to open the test > click on the Gear icon > click on the Parameters.

How to provide parameters to Screenster test Manually?

Screen star provides an option to insert parameters manually too!

The steps are as follows –

1. Open the recorded project.
2. Click the gear icon button.
3. A dropdown will open.
4. Select parameters from it.

However, note that, from the project level, it is possible to access and update the test parameter too! Manual addition of parameter is also possible.

1. Open the recorded project.
2. Click on the fourth tab called PARAMETERS.
3. List of parameters opened in the below in a grid-like manner.
4. Every parameter will have an Edit option (Pencil icon) to edit/update.

How to execute Screenster tests?

Running tests in screenster –

The test run in screenster can be customized as per tester’s need.

The browsers on which the tester wants to execute -(Need to select one)

1. Chrome
2. Firefox
3. IE

Extra settings for IE for Screenster

In order to ensure that IE will run smoothly on your local machine please do the following after installing Screenster:

• Open IE, go to Settings > Internet Options > General and make sure the “Delete browser history on exit” option is checked.
• Switch to the tab Settings > Internet Options > Security and make sure that the “Enable Protected Mode” option is NOT checked for each zone (“Internet”, “Local intranet”, “Trusted sites”, “Restricted sites”)
• Open Display Settings on your desktop and make sure that you have the “100% (Recommended)” option selected in the Size dropdown.

Next, the screenster expects is the type of verification the tester wants. The verifications are –

1. Content
2. Visual
3. Navigation only
4. Use the Data set.

In an advanced setting, the tester can provide custom test URL. Once all configuration is done, testers need to click on the Run button.

Advanced Settings for url

Custom URL can be your local environment like localhost screenster also lets us test a test case against data set. Datasets are configured in the data set grid page. The data grid can be set from the drop-down (select test).

Review details of Screenstrer

Review- It allows user to select and approve intended changes and baseline the UI script. User can exclude one or several other UI elements for testing.

How to smartly insert a step?

Instead of manual edit, screenster provides a smart option to continue recording even on a saved test.

1. Open project in Edit mode.
2. Navigate the step where we need to edit.
3. If we mouse over a step you can see “continue recording before this step” option.
4. Click on this option to enter/ operand new steps.
5. This will act as a breakpoint.
6. After placing the breakpoint click on continue recording button to insert steps.

How to use drag and drop facility in Screenster?

Screenster also supports drag and drop option along with mouse clicks, keyboard inputs and scrolling. To drag and drop as UI element, in the browser, just select the component and drag to the portion you want to add. As soon as you release the button the UI component will be uploaded into.

What are the verifications options available in Screenster?

Let us understand different verification options –

1. Content verification – It is the default verification method to detect possible bugs in UI. Screenster checks the DOMS of stored (Expected) and the UI (actual) to test the mismatches of content and elements.
2. Visual comparison – This is a very sophisticated comparison of the screen. The Actual and Expected screens are compared pixel by pixel by a visual comparison algorithm. Visual comparison can detect very minor shifts in layout. But this technique is slower due to pixel comparison. Not recommended for normal testing.
3. Navigation only – Mostly this verification is used to check the smoke test. After fixing some minor issues, this verification can be run.

Screenster verifies Action vs Expected along with navigation, mouse actions, keyboard entries. However, Screenster does not take any new screenshot until there is a failure.

Reporting in Screenster

As this is a hierarchy-based test tool, every test will have several steps. Each step becomes a verification point for screenster. Verification points are another way known as checkpoints.

Screenster provides four below status for a verification point.

• Passed – (green) – The step is passed and no changes detected between actual and expected.
• Differences (orange) – The execution of a step is successfully completed but there may be differences in contents or elements.
• Error (red) – The execution of a step is failed as the expected and actual is mismatching due to a technical issue.
• Draft (grey) – This colour code signifies that the test step is new or the step is not attached to a test or not executed due to an error. Greyed out test step cannot contain a screenshot.

The test status is also available in project level –

The checkpoints in project-level are-

1. Passed
2. Differences
3. Error
4. Recorded

How Screenster helps in Agile?

Screenster allows collaboration of developers, testers, the manual function expects to test an application effortlessly.

How screenster detect UI changes?

In screenster, the baseline version is the stable version that will be continuously checked with other versions.

The UI changes happen due to the following –

1. Changes in HTML, CSS, and content.
2. How UI responds for user action (mouse click, keyboard clicks)

   In case screenster finds out differences, it allows us to baseline the updated content by using smart approval.

   • Screenster allows us to ignore dynamic regions automatically. However, if testers find any dynamic region in the mismatched content, they can ignore the region by simply clicking ignore this element.
   • Screenster allows us to update components (elements/contents) if major changes are introduced in the application to update components we can use “use this element as step target” to update the component.

   Once all the changes are incorporated finally, we can approve the incorporated changes as baseline lay clicking on the Approve (✔) button.

   The ignore UI regions is having zoom functionality to enlarge the element. Screenster also allows us to manually create, edit and delete rule to ignore UI regions.

   In the rule creation popup, we need to define –

   1. Name – The complete path of the element.
   2. Visible at dropdown.
   3. Allow Screenster to apply filter.
   4. Any specific CSS locator.

   How to perform Data-Driven testing in Screenster?

   Screenster allows us to perform Data-Driven testing. So, we can use external test data to execute out test cases over iterations.

   To perform data-driven testing, we need to perform the following tests –

   1. Record a test.
   2. Extract parameters.
   3. Create a data set for the test.
   4. Attach the data set to the test.
   5. Execute the test.

   Screenster provides two distinct ways to provide data set to a project.

   1. Attach CSV file.
   2. Create a data set in Screenster.

   Attaching CSV file to a test

   The CSV files need to be prepared in the following way –

   1. The header row should have parameters name.
   2. The last header must be ‘expected result’.
   3. Now the value for the parameters will go.

   To upload the CSV file and attach to a test we must perform the following steps –

   1. Open project.
   2. Navigate to tests view.
   3. Click on Data sets.
   4. Click on upload data set.
   5. Select the desired CSV file and upload.
   6. Screenster will render the file and open in a grid view to edit.

   Create a data set in Screenster

   To create data set in screenster we need to perform the following steps –

   1. Open project.
   2. Navigate to tests view.
   3. Click on Data set.
   4. Click on create button.
   5. Provide the data set name.
   6. Start adding parameters by clicking on the ‘Add Parameter’ button.
   7. For each parameter provide the name. The name must be unique (should not contain white space dollar $ sign).
   8. Click on the Ok button.
   9. The parameter will be added to a grid-like interface.
   10. Each parameter will have edit button to edit and delete button to delete.
   11. Provide value against each parameter.

   During the initiation of Run, we can associate data set in the Run window by selecting the correct data set from the drop-down.

   How to Add/Delete Steps in Screenster?

   Add steps

   In the project, inside the test, we can see all the steps. In order to add a step, we can hover over the three dots (⋮), a new popup will open. We need to select a new step.

   To add a step, we need to provide the following –

   1. Name
   2. Type
   3. Time out
   4. wait full-time out
   5. CSS selector

   Delete a step

   To delete a step, just hover over three dots on the step and select delete step.

   Screenster also supports copy-paste elements in edit mode.  Multiple steps can be copied. Screenster allows us to change the order of the test case by simply drag and drop.

   The tester can check the elements selector by selecting the element and selecting “show this elements selector” option. The element selection can be accomplished with mouse hover over the element.

   How to schedule a Test in screenster?

   In the suite page, the clock symbol signifies the scheduling. Once clicked, it opens a scheduling configuration. Scheduling can be-

   • None
   • Hourly
   • Daily
   • Weekly.

   During scheduling execution, the verification can be-

   • Content
   • Visual
   • Navigation only.

   It is possible to switch between verification on to verification off by switching to Navigation only execution setup. We can also select the target browser-

   • Chrome
   • Firefox
   • IE

   Checking on save will save the scheduler settings. In case we want to disable scheduling, we need to select None option and click on save.

   How to invoke another test from a test?

   Screenster allows us to call to an existing in a current test (another form of reusability). To add an already existing test, we need to click on the new step of Test Commands menu.

   Once the new step pop up opens-

   1. Select type on- Invoke with parameters
   2. Script- The name of the already existing script.
   3. Optionally we can provide custom will of the script. Click on save to attack the existing script in the new script.

   This Test addition is read-only. That means we cannot edit the added script (Equivalent to call to action in UFT).

   How to extend a Script?

   Extending a script is equivalent to copy to Action in UFT. Here we can extend another script. That will allows us to copy and edit the steps of another test.

   To extend a script. We need to perform the following steps-

   1. Navigate to the history page.
   2. In the test history grid, all tests will be shown.
   3. Click on ‘Extend Test’ in the main menu bar.
   4. This will open the Extend Test window.
   5. Provide Name of the script, Screen size, and browser
   6. A new cloned test will open.
   7. Click on SAVE button to finalize the script cloning.

   How to handle timeouts in screenster?

   Timeouts can be handled in step pop up. This field specifies how many seconds screenster will wait before executing the next step.

   Timeouts are really helpful when the page loads slowly. The element or contents appears on the screen slowly. The Ajax may have not returned anything when screenster perform the search operation. The browser renders the page slowly.

   Screenster smartly can calculate the page load time during recording. We can update those values as per our need.

   Test Infrastructure Setup for Screenster

   Screenster provides two option to configure test setup –

   1. Cloud-based server – Screenster will set up the necessary infrastructure.
   2. Local server setup – The local team is responsible to setup.

   Deployment types in Screenster

   • Installed – Mac, Windows
   • Web-Based, Cloud, SaaS
   • Mobile-iPhone / iPad, Android

   How to restore the backup for a new screenster installation?

   Sreenster backs up projects daily (It is a full file backup). It will be available under your installation directory (typically c:/ Program Files /screenster) and in the backups folder.

   • Before re-installing a fresh screenster, we need to copy the existing backup (name starts with “results -yyyy -mm -dd….” zip to another folder.
   • Stop the screenster service running on the system by navigating to Windows Task Manager > Services > Find Name as screenster then stop.
   • Now unpack the new installation files and install/override the existing installation. Once the new installation is over, put back the backup file. We are done now!
   • Start the screenster service now.

   Can Screenster support Mobile testing?

   Yes, currently out of the box, screenster is supporting iPhone, iPad, Samsung Galaxy SS and Samsung Galaxy Tab. Further phones, we need to understand the screen size and use the customization option.

   Where do I find installation-related logs for Screenster?

   In the installation folder of Screenster, (mostly c:/ Program Files/ Screenster) we can find a folder called – logs. Inside logs, Screenster keeps all sorts of logs you can choose the one you need. Screenshot typically captures five daily logs. This log file, name starts with ” tail – Catalina” talks about actions.

   How to handle User Access rights in Screenster?

   Screenster installation requires admin rights. It is just like any other tool.So if you installed it on your machine, there should be no problem running it. If somebody else performed the installation, make sure you have the admin rights to launch the server under your current account.

   In rare cases, changes to your local security settings may influence your access rights. If your access rights have been modified, reconfigure them by following these steps:

   1. Find the Screenster service in your Local Services folder
   2. Stop the service if it is running
   3. Right-click on it and select Properties
   4. Go to the Log On tab and select the This account option
   5. Click on Browse to add your account (unless the fields are already pre-populated)
   6. Find a necessary account name by browsing locations or by checking available names after typing some hint
   7. Re-check the password and save your changes
   8. Restart the Screenster service
   9. Try to open localhost:7070 – it might require a couple of minutes for the service to start. After the page has opened, you can log in with your Screenster credentials

   How to share projects in Screenster?

   Screenster projects hosted locally or cloud supports multiple subscriptions hence valid users can log in and configure their tests. However, screenster also provide sharing option between different stakeholders.

   Steps to share a project –

   1. Open screenster
   2. Create/open a project
   3. Go to project option (triple dots) and click on it
   4. Select share access
   5. It will open up a new popup
   6. Provide the username
   7. Select role
   8. Click on the share button.

   Visual verification tolerance in Screenster

   Visual verification tolerance is an important aspect when we test using screenster. Moving pixels is a common issue during shift left approach. The quick development and release may end up shifting pixel here and there.

   Screenster provides an option to tolerate the limit. This setting is available on the project page or during project creation. We need to put a tolerance limit for a project in percentage.

   Can Screenster test localization testing?

   Yes, but it is a little tricky. Localization testing help-https://screenster.io/documentation/running-tests-verification-modes/

   How I can use File Upload with screenster.io?

   If you’re using “<input type=”file”/>” tag for file upload on your website just click the “Browse” button and you will get a file upload dialogue.

   Can Screenster support the recording of scrolling?

   No, but the purpose of scrolling is to compare test and other elements. For content comparison, it only checks the visible part of the page. It is due to performance and scalability. But in order to check the content of the whole page. Visual verification can be used.

   Frequently faced issues in IE and how to solve

   Not able to record in ie?

   1. Go to the folder where Screenster is installed (it’s C:\Program Files (x86)\Screenster by default)
   2. Find EnableIESupport.bat script and run it with administrator rights
   3. Provide existing user name and password that will be used to run IE
   4. Try to record/run a test with IE

   Another solution

   IE may have different default security settings on different Windows versions. Could you please check that “Enable Protected Mode” in IE security settings is disabled for each zone (“Internet”, “Local intranet”, “Trusted sites”, “Restricted sites”) as it mentioned here https://screenster.io/documentation/local-installation/

   Please disable it if it’s enabled in any zone. Then do the following steps:

   • Stop Screenster service
   • Run “DisableIESupport.bat” script in the Screenster folder (it’s C:\Program Files (x86)\Screenster by default) with administrator rights
   • Run “EnableIESupport.bat” script with administrator rights
   • Provide existing user name and password that will be used to run IE
   • Start Screenster and try to record/run a test with IE (it may take a few minutes for IE to start when you run it for the first time)

   How to download a specific test run for bug reporting purpose?

   If there is a specific need to download a test run along with the Screenshots, DOM, web pages (HTML format), you need to –

   1. Open the specific run from screenster page.
   2. Click on the clear icon.
   3. Set up download this run option.

   This test will be downloaded to your local machine.

   Training Available in Screenster

   various modes of training available for Screenster learning. They are as follows:

   • Documentation
   • Online groups and community
   • Webinars
   • In-Person classroom-based
   • Live Online

   What kind of support do you get from Screenster?

   Screenster supports the following modes of supports:

   • In-person during Business Hours
   • Online during non Business Hours

   Advantages of Screenster

   • No installation- This tool is cloud-based and purely web-based hence test engineers need not install into local. Also for recording the test. Testers do not need any plugins. For an on-premise test, testers get a single click executable file for quick and easy setup and perming any maintenance activity.
   • Shared portal for quick collaboration- The test suite is shared across interested stakeholders. This feature helps the stakeholders collaborate early and test jointly.
   • Smart record playback- Codeless scripting using a selected browser recorder.
   • Selenium support- Screenster supports selenium commands along with custom javascript code to empower test script.
   • Easy and user-friendly scripting- Users are allowed to edit, delete, add or alter test scripts without touching the code.
   • Auto-update locaters- Incase UI changes, screenster can auto-update the elements and locaters. This Autocorrect smart locater strategy is best for speeding up automation.
   • Excellent timeout handling- While dealing with complex javascript based (AJAX) page, screenster can determine and perform optional time out (wait for syntax).
   • No assert or verification- Screenster can automatically detect any visual changes by its algorithms and captures the result by painting out the new, deleted and changed UI and content.
   • Smart verification- Screenster can verify content comparison across multiple resolutions.
   • Execute locally or on the cloud- Runs on the cloud, locally or even in offline mode.
   • Smart Execution support- Screenster supports parallel execution, a data-driven and Test scheduler.
   • Smart dashboard- Screenster provides a smart dashboard which is multi-level for different stakeholders and very much user friendly.
   • Reusable test script- Screenster supports reusable test scripts which can be plugged in other test scripts very easily.
   • Smart versioning- Screenster test scripts are exported to a lightweight JSON format. So these files can be put to any version control like git, subversion etc.
   • Supports CI CD pipeline- Screenster script can be integrated with Jenkins, Bamboo, TeamCity etc.
   • Jira integration- Screenster can be integrated with Jira to create Jira tickets from the run page.

   Disadvantages of Screenster

   • In the browser, record and playback are slow.
   • Supports web application s only.
   • Currently supports Windows platform only.
   • Does not support Live Web Testing.
   • No effective bug tracking mechanism.
   • It is affected with Access Control issues.
   • Currently supports chrome, firefox and IE.
   • For each test case creation, a new browser is needed to be launched.
   • Users have very limited control over recorded scripts for modifications.
   • No support for Geo-Location Testing.

   Popular Alternative to Screenstrer

   1. Test office
   2. Meveryx
   3. Rapise
   4. Z-automate
   5. Ranorex Studio
   6. Witbe
   7. Trudon
   8. accIQ
   9. Degust

   Other Useful links:

   • https://screenster.io/documentation/instruction-how-to-check-that-a-file-was-downloaded/
   • https://screenster.io/documentation/creating-visual-tests-manually/
   • https://screenster.io/documentation/screenster-executing-custom-selenium-code/
   • https://screenster.io/documentation/importing-and-exporting-tests/
   • https://screenster.io/documentation/hand-coded-tests/

   Conclusion

   Screenster certainly becoming one of the major visual regression tools. It can slowly kill UFT and the need for Selenium or any other IDE based tool. The scoreless behaviour of the Screenster is certainly a plus for non-technical or non-coders.