Choosing the Right Test Cases for Automation

Pradeep G, Cognizant Technology Solutions

With hopes of reducing testing cost and effort, companies often look to test automation as the cure-all for their problems. However, without clear and practical objectives, a test automation project is bound to fail. One key factor in setting automation objectives is to identify which test cases should be automated and which should remain manual processes. Pradeep G describes a practical methodology to identify the best test cases as candidates for automation. His nine-point decision tree process for selecting test cases examines technical feasibility, execution frequency, component reusability, criticality, effort required for automation, total resource requirements, test case complexity, portability, and execution time. Discover how to achieve significant return on your automation investments by creating test scripts that are amenable to easy execution, reuse, and more.

As more companies move to agile software delivery approaches, new challenges and dynamics are impacting their testing practices. Organizations face many issues when implementing automation, including selecting tools that are usable and flexible, encouraging non-technical and non-testing staff to contribute tests, enabling open-source integration, and promoting test-driven development. Dietmar Strasser shares his experiences tackling these challenges as many organizations shift from traditional test automation to agile. Learn about the increased importance of testing in the agile development environment; the role that process and tools play  in supporting the agile team; the differences between traditional and agile test automation; how to develop fast, automated test scripts; the use of agile and traditional testing methods side-by-side; and how to deal with test automation in a distributed development environment.

In the coming years, testers will be placed under ever increasing pressure. Joachim Herschmann describes key future trends including the increasing alignment of development and test with business needs, the integration of traditionally separate disciplines, a shared responsibility for quality, and the increased use of testing technology. Joachim describes the experiences of Borland's Linz development lab as a framework for a broader discussion about these kinds of changes and their cultural impact on the organization. He describes the journey from a waterfall-based methodology to an iterative, sprint-based development approach and the integration of developers and testers into a single team of engineers. They found that agile development provided new levels of productivity and value—and posed new challenges of shortened test cycles and a need for new test skills and tools. You’ll leave with a deeper insight into the resulting culture shift caused by the transformation, and the test process changes that resulted.

Test environments are an important part of our testing portfolio, yet often we seem to spend very little time planning, creating, and maintaining them. Julie Gardiner explains the reasons we fail to build test environments that are realistic, reliable, representative, and have integrity. As a result, they become the weakest link in our testing process. Julie provides examples of environments—good, bad, and sometimes ugly—and shares why the ugly are often a symptom of the organization's disregard for testing. She offers practical advice for transforming your current test environment from the weakest into the strongest link of your testing. Julie’s specific advice includes identifying early signs that the environment will cause problems, convincing management that extra resources are required, and tool support to assist you in creating an excellent environment, providing data preparation, test design, oracles, and test execution tools.

Experience has shown that many organizations attempt to automate their testing processes without effective vision, planning, and follow through. As a result, within a year or two, test automation efforts are declared worthless and the tools are moved to the shelf. By creating a centralized team with domain expertise and identifying specific test automation needs, Intuit is able to build, deploy, and test products using a common set of tools, processes, and methodologies they call Autolab. Shoba Raj describes how the Small Business Group at Intuit maximizes its return on investment by utilizing the Autolab. She explores the benefits of time savings, capital cost savings, quality improvements, product health checks, and tool license fee aggregation. Learn how to build a centralized testing team and create your own Autolab that can leverage your services with standard tools, test environments, and processes.

In these times of economic crisis, cost reduction is usually the number one motive for test process improvement. Although improvement models such as TMMi® and TPI® are very popular, they require formal assessments, process change working groups, extensive implementation programs, and new organizational structures. Instead, you can quickly implement measures that improve your testing process incrementally within your day-to-day activities. Martin Pol presents a low-budget way to select and implement a set of measures that can rapidly improve testing’s contribution to your project’s success—simple risk analysis, proactive test design, coverage targeting, and novel ways to reuse tools, environments, expertise, and existing testware. Learn how low-budget test process improvement can become a natural behavior for your testing staff. Achieve quick wins by working more closely with development and using what you have instead of buying or creating new tools.

TMMi® is the registered trademark of the TMMi Foundation.
TPI® is a registered trademark of Sogeti USA LLC.

Although software testing can be an exciting challenge, testers often are bogged down in voluminous, manual testing and re-testing with relatively shallow requirements-based test cases. This old approach costs projects time and money while stealing resources away from more creative testing. Organizations look at two ways to reduce these repetitive testing costs—automation and offshoring. Combining these two approaches has the promise of even more savings to the organization. However, the reality of off-shored test automation can be disappointing and even lead to a "double backlash" instead of a "double benefit" because both automation and offshoring are complex operations in and of themselves. Hans Buwalda, a test automation pioneer, presents his key elements for success—clear direction, good methods, appropriate tools, and effective supervision— and explores the most common pitfalls to avoid.

You already have taken some basic cost-cutting steps and saved your organization money. Now, you are asked to dig even deeper into your testing budget. Where should you start? You may be looking right at the areas to address and not know what you are seeing or what to do about them. Paul Trompeter explains how to take a fresh look at your existing hardware components, re-examine reliability and availability requirements, and prepare for a future scalable environment. Paul discusses how to get regulatory affairs in order, defuse the ticking storage overload bomb, and streamline testing of complex software systems. For each budget killer, you’ll learn innovative ways to overcome budget challenges while maintaining an effective test organization. Discover how to slow the spending of your testing budget while increasing the return on your testing investment and, at the same time, keeping your sanity and sense of humor.

When you’re given a product to test, a variety of clients to satisfy, and a short deadline to meet, how do you decide specifically what to test and how to test it? One way is to identify the things that might matter to some clients and not to others. In this interactive session, Michael Bolton describes the skill of factoring (not to be confused with refactoring)—ways to identify dimensions of interest relevant to testing. Through individual and group exercises, you’ll practice the skills of factoring, learn guideword heuristics that can help identify important factors, and develop guidewords of your own. Experience the ways in which small changes in context can dramatically expand or contract your understanding of what's important and unimportant. Learn a framework for identifying factors that matter to your clients so you can respond rapidly, confidently, and expertly to any testing mission.

With model-based test design, you first create a high-level functional model of the system to be tested. The model is the input to an automated test generation tool that creates the test designs and associated test scripts. Recently available commercial, automated test generation tools are making automated test generation a practical and powerful alternative to manual test design. Antti Huima discusses the advances in modeling methods, test generation tools, and the implications for test design productivity, quality, and overall test process efficiency. He explains the differences between “modeling for implementation” and “modeling for test” that make model-based test design applicable to both new and existing systems. Antti uses real-world examples to demonstrate automated test design tools and presents the results achieved deploying these tools. Find out how your test organization can take advantage of automated, model-based test design—a paradigm shift whose time has finally come.

Has this happened to you? You try to implement a new software quality improvement program and it ends up failing. And, to make matters worse, you can't figure out why. It may be that your great idea didn't mesh well with your organization’s current culture. Jennifer Bonine shares a toolkit to help you determine which ideas will—and will not—work within your organization. The toolkit includes five rules for change management, a checklist to help you analyze the type of change process needed in your organization, a set of questions you can ask to better understand your company’s goals, techniques for overcoming resistance to change, and the formal roles necessary to enable successful change. This toolkit—together with an awareness of your organization’s core culture—allows you to identify the changes you can successfully implement and those that should be left for the future.

While today’s distributed software architectures such as SOA, Software-as-a-Service, and Cloud Computing offer organizations increased agility and new reuse opportunities, they also increase the risk of costly software problems due to constant change and increasing complexity. System testing is constrained by dependency on critical services, systems, and data that may be unavailable or too costly to use. A new approach, Service Virtualization (SV), allows test teams to create “virtual worlds” that emulate software behavior and model functional, data, and performance characteristics in the target production environment. Testers can capture, model, and configure target environments while decreasing their need to access production systems and costly “pay as you go” services. With Service Virtualization, testing teams become decoupled from constraints in the lifecycle, with drastically reduced support, data maintenance, and access costs.