Before we go ahead and discuss more about automation framework, let us understand few concepts about automation testing.
Lets us talk about few myth first about automation testing.
- Test Automation is very easy and anybody can implement test automation strategy. Anybody can develop a script.Well,this myth is developed by the sales person of the company to make sure their tool gets sold out.While demoing their tool,they always follow the record playback strategy. just record a simple flow , enhance by using test data concept,run the script shows the result.With these sales technique we started feeling that any body can script.But reality is completely different. Automation is not all about record and playback.It needs to be designed,then developed and tested. Definitely test engineer needs to be from some programming language background or they need to learn languages,at least the basics.Automation development is not as complex as application development but these source codes must be saved as application source code.
- Tool design is better than evaluate tools.Mainly this concept came from the expensive tools available in market. Test engineers or organization try to avoid the license cost of such tool. They prefer to build their own tool. Or try to use free scripting or languages like VBA,macro ,perl,Ruby to design test.These thing can be considered as lean approach .But not Automation.See the below links for lean approach.
What is lean approach,All Lean Tools,Thinking Lean, Muda
But the fact is when you use a proper tool to automate the productivity will be higher which is significantly less than the licence cost. Using other technique can be considered as research and development but not proper automation.
With a proper test tool with licence will give automation test engineer more command over the QA arena. Let us see how?
- Most advanced tools will have a proper encapsulation of programming. So less advanced coding required here.
- More the mature tool more great customer care.But opensource tool will have less or no customer support.
- Advanced tools will get constant update about technologies.
- Huge support base including different forums.
- Readily available framework and integration with other technologies.
- Advance tool will have more functionalities in different technologies covered.
In general Framework is an integrated part of automation testing.This is an structured combination of ..
a.Testing best approaches
before going for a automation testing…nono….forget about automation testing prior to any testing, client and service provider need to sit together and determine the approaches…
let us understand the tool based testing…
Simply purchasing or leasing a tool does not guarantee success with that tool. Each type of tool may require additional effort to achieve real and lasting benefits. There are potential benefits and opportunities with the use of tools in testing, but there are also risks.
- Repetitive work is reduced (e.g. running regression tests, re-entering the same test data, and checking against coding standards).
- Greater consistency and repeatability (e.g. tests executed by a tool, and tests derived from requirements).
- Objective assessment (e.g. static measures, coverage).
- Ease of access to information about tests or testing (e.g. statistics and graphs about test progress, incident rates and performance).Risks of using tools include:
- Unrealistic expectations for the tool (including functionality and ease of use).
- Underestimating the time, cost and effort for the initial introduction of a tool (including training and external expertise).
- Underestimating the time and effort needed to achieve significant and continuing benefits from the tool (including the need for changes in the testing process and continuous improvement of the way the tool is used).
- Underestimating the effort required to maintain the test assets generated by the tool.
- Over-reliance on the tool (replacement for test design or where manual testing would be better).
A study report of US NIST says —
This shows in 2004 the automation coverage was 5% and in 2006 it was 20% but today it is almost 50%
Software product based companies looses $21.2 billion /year due to insufficient testing.
$931 million was invested during 1999 and in 2004 it went up to $2.6 billion.
The solution may be….
1.The scope and need of testing.They must not goahead and do a testing activity for the buzzword testing–QTP etc.We need to think what will be the scope of testing.Will it be Product Oriented? Will it be Company Oriented? or Project Oriented?
Second scoping should be what kind of testing we are going to perform? Functional Testing or Load Testing or Web Service Testing or API Testing?
2.The requirement gathering and identification of proper automation tool is one of the basic criteria before going to testing viz automation testing.We need to understand the nature of requirement , priority of such requirements and priority of them There are several cases where requirement gets changed in every alternate day(Agile Testing).
Now second step of this section is to evaluate the testing tool.The Major search about testing tool in a search engine as as follows:
for tool selection there are several charts are available..
Over 300 tools are readily available in market.
Java test tool-48
other Web Testing-76
So it would be a very tough choice of tools.We need to prepare a checklist for evaluation of such tool.Identify required resources in company.Now do prototype for few sample flows.Check for different actions,Validation and reporting feature.
3.Again if it passes the above stages then business people and technical persons are required to sit together to identify the scenarios to automate and most critical prior to automation testing.100% requirements can not be covered and is not required for any automation.
Here is a article that gives more insight of tool evaluation-https://www.techtravelhub.com/2016/10/how-to-pilot-test-automation-project.html
Some basic Tool evaluation criteria are:
- Easy coding and develop scripts
- Easy maintenance
- Easy and huge support base
- Less cost
- Available training doc or training materials
- Technologies support
- Execution in distributed mode.