Friday, May 20, 2005, 10:00 AM  Go to 11:15 AM

F1	Test Management

In many industries, we must comply with state or federal statutes, government regulations, and other legal standards. The Sarbanes-Oxley Act (SOX) has brought a new awareness to these issues within testing. So, how do you incorporate legal compliance into your QA and test efforts, and how do you get the information you need to do the job well? Elle Ringham, who deals with these important issues every day at Fidelity National Financial, shares her knowledge and experiences. First, she helps you understand the types of legal compliance bodies that can affect applications and offers research methods for industry specific areas of legal compliance, including internal sources, Web sites, and search strategies. Then, she discusses the artifacts and metrics needed to be maintained for compliance audit purposes. Specifically, she offers the Sarbanes-Oxley testing matrix used in her organization and, more generally, ways to add compliance mapping and testing into your planning.

• Legal counsel, finance, and external resources for compliance research
• Audit artifacts and metrics for legal compliance
• Legal requirements as a part of architecture and development processes

F2	Test Techniques

Finding non-functional, system level problems before a system is delivered—and early enough to allow for low cost corrections—is a perennial testing issue. You need a systematic approach to system testing that keeps key system test objectives visible and assigns the correct priority among other test activities. Based on his team’s experiences at IBM, Jerry Cobry describes how there needs to be a special mindset for system testing and describes some of the challenges they faced. Then, using the mnemonic STRIVE FOR Quality, Jerry describes key factors for successful system testing: Stress, Time, Resources, Interaction, Volume, Environment, Focus, Operational profiles, Randomness, and Quality. Go beyond simply finding system problems and become a product assessment engineer. Question your old ways of doing things, get out of your comfort zone, and strive for quality in conducting your own system tests.

• A heuristic model to guide system testing
• How to make systems tests more dynamic and more productive
• The systems tester’s mindset

F3	Exploratory Testing

Exploratory testing is more than just thinking of clever test ideas and executing them on a whim. It's a craft, requiring practice of several classic scientific skills—one of those skills is careful documentation of observations and conjectures. But as much as testers are scientists, they are also explorers. They must document their actions and observations during testing in such a way that stakeholders can easily understand the important problems and issues that are being discovered. In this track talk, expert exploratory tester Jon Bach compares exploratory test notes from several software projects to the journals of historical adventurers, showing how a tester's journey through unchartered software can reveal similar risks and riches. Jon will discuss three common note-taking styles, as well as good, bad, and ugly notes he has seen (and produced!) in his ten years of testing.

• Three common note-taking styles for exploratory testing
• Test ''chartering'' techniques and skills for exploratory testing
• How much documentation is enough?

F4	Metrics

In addition to the efficiency improvements you expect from automated testing tools, you can—and should—expect them to provide valuable metrics to help manage your testing effort. By exploiting the programmability of automation tools, you can support the measurement and reporting aspects of your department. Learn how Jack Frank employs these tools with minimal effort to create test execution status reports, coverage metrics, and other key management reports. Learn what measurement data your automation tool needs to log for later reporting. See examples of the operational reports his automation tools generate, including run/re-run/not run, pass/fail, percent complete, and percent of overall system tested. Take with you examples of senior management reports, including Jack’s favorite, “My Bosses’ Boss Test Status Report”—names will be changed to hide the guilty. Regardless of the automation tool used in your development department, you, too, can simplify your reporting and more quickly get the decision-making information you need.

• Key metrics produced from an automated testing tool
• A test automation architecture to capture the needed data for effective metrics
• Generate test management reports in your automation tool or MS Office

F5	Special Topics

Good project managers speak the language of risk. Their understanding of risk guides important decisions. Testers can contribute to an organization's decision-making ability by speaking that same language. During this session you will learn how to evaluate risk in both quantitative and qualitative ways. Identifying risk is important but managing risk is vital. Julie will discuss how to deal with the misunderstandings some managers have about risk-based testing including testing is always ''risk-based,'' risk-based testing is nothing more than prioritizing tests, risk-based testing is a one-time-only activity, risk-based testing is a waste of time, and risk-based testing will delay the project.

• Five different but complementary approaches to risk evaluation
• Vital areas to consider when choosing your risk-based approach
• Misconceptions of management regarding risk-based testing

Friday, May 20, 2005, 11:15 AM  Go to 10:00 AM

F6	Test Management

Used successfully in hundreds of agile and iterative development projects, SCRUM is a software project management approach that employs fixed cycle time “sprints” and daily “scrums.” SCRUM emphasizes self-directing teams and the role of a “Scrum Master” to remove obstacles to the team’s success. Because testing often succumbs to delivery pressures and becomes chaotic, especially during the endgame of projects, SCRUM can be an extremely helpful management framework for test groups. By implementing SCRUM practices, you increase the focus of test efforts on the most important actions, empower the test team, improve morale, and heighten your influence on the overall success of the project. Learn how Robert Galen instituted SCRUM in a test department, overlaying this new framework on existing processes with a surprisingly positive impact on both the people and the product.

• An overview of the SCRUM methodology and its application to test teams
• Manage testing as a product backlog activity prioritized with and by your customers
• Implementing daily scrums, sprints, and sprint reviews

F7	Test Techniques

For more complete testing, you need to find and simulate possible error conditions in a system. Many methods throw exceptions when an error occurs. And although the application’s code catches many of these exceptions, an “unhandled” error condition could lead to unpredictable events and big problems for customers. Rather than using only intuition to guide your error testing, join Christopher Shelley for a strategy to identify specific error conditions in your systems. Sharing sample code, he offers tips and hints for finding these often hidden defects in software. Then, he explains ways to devise specific error injection tests that expose these problems to the developers. Learn the skills you need to find unhandled exceptions within the source code and make sure that your code is exercised through all decision trees and error traps.

• Expose hidden error condition defects in real-world scenarios
• A specific methodology to devise error injection tests
• Make the application more resistant to possible errors

F8	Exploratory Testing

Exploratory testing and error guessing are valuable functional testing techniques. Like all other methods, though, they have limitations partly because they are based on the knowledge, experience, and intuition of the test engineer. If you primarily use unstructured approaches for testing, you risk wasting effort on redundant testing, testing in non-critical areas, and under-testing critical areas— all of which can lead to missed bugs or finding defects later in the cycle. BJ Rollison uses two case studies to demonstrate the limits of unstructured testing and how fundamental test design techniques and gray box test design can improve the effectiveness of your tests. You’ll correctly prioritize critical areas and uncover serious issues earlier and with less effort. The first case tests Weinberg’s triangle algorithm from the design requirements and a C# implementation. The second tests the “Save As” function of Notepad on Windows XP using the ANSI 1252 character set.

• Effective and ineffective uses of limited exploratory testing and error guessing
• A gray box approach to test design with improved efficiency and effectiveness
• Test smarter, not harder

F9	Metrics

Because people can easily relate to a familiar paradigm, analogies are an excellent way to communicate complex data. Rob Sabourin uses baseball as an analogy to set up a series of status reports to manage test projects, share results with stakeholders, and measure test effectiveness. For test status, different audiences—test engineers, test leads and managers, development managers, customers, and senior management—need different information, different levels of detail, and different ways of looking at data. So, what “stats” would you put on the back of Testing Bubble Gum cards? Testing Sports Page Standing tables? Game Score sheet? And up on the giant Scoreboard that is visible to everyone? In this entertaining and informative session, Rob shows real-world examples of baseball reports he has developed and discusses ways to use the information for better decisionmaking. See if your current test reports would get on base, hit a home run, or strike out!

• Present test status reports and information in a way stakeholders can understand
• Produce test reports that support timely and informed decisions
• Customize test report format and detail for the audience

F10	Special Topics

Traditionally, test engineers have had some type of a visual user interface for testing client/server and Web applications. Web services, on the other hand, are completely without a user interface, providing only an application program interface (API). A Web service does not display a visual output for testing. Although this fact makes manual testing very difficult, Web services are ideal candidates for automated testing. As a result, some programming skills are almost certainly needed for testers who test Web services. What about testers with less technical skills? Learn about the challenges Papa Acquah faced with Web services testing: WSDL validation, unit testing, functional testing, client side testing, and server testing. Find out how he used a test harness as well as existing commercial testing tools to accomplish his testing needs.

• What is a Web service and how can you test it
• Tools available to test Web services
• Testing without a visual user interface

Software Quality Engineering Home       Conference Home       To Exhibit       Get a Brochure       Register for STAREAST 2005

feedback  © 2005