Wednesday, May 18, 2005, 11:30 AM  Go to 1:45 PM   Go to 3:00 PM

W1	Test Management

Some blind men encountered an elephant. “What is this thing?” they cried. “It’s a snake,” said Tom. “No, it’s a wall,” said Joe. Harry laughed, “You fools! It’s plain to see—it’s a fan moving from side to side.” For many organizations, software quality is an elephant found by blind men who think they can see. Based on their roles and interests, people within an organization hold different opinions about quality. If we testers focus on one definition (our own), we will not communicate well with others in the organization, and our work ultimately will not meet their expectations. However, if we examine our customers’ definition of excellence, we will find better ways to measure quality and set priorities. Join Lee Copeland for this enlightening discussion of the differing viewpoints of quality and how to work with your customers and users to clearly see their quality elephant as well as your own and integrate their vision into your testing activities.

• The customer’s view of software quality and governance
• Corporate excellence frameworks to establish testing measures of success
• Test improvements that align with your customer’s goals and strategies

W2	Test Techniques

Structuring test designs and prioritizing your test effort for large and complex software systems are daunting tasks, ones that have beaten many, very good test engineers. If you add concurrency issues and a distributed system architecture to the mix, some would simply throw up their hands. At Microsoft, where Keith Stobie plies his trade, that is not an option. Keith and others have reengineered their testing, employing dependency analysis for test design, model property static checking, “all pairs” configuration testing, robust unit testing, and more. They employ coverage to successfully help select and prioritize tests and make effective use of random testing including fuzz testing security. Finally, models of their systems help them generate good stochastic tests and act as test oracles for automation.

• Test checklists for large, complex, distributed systems
• Appropriate use of code coverage with increased coverage for less effort
• Dependency analysis for improved test designs

W3	Test Automation

Even though most everyone recognizes that automation is a key element in improving test efficiency, many automation efforts unfortunately fall far short of achieving the desired results. One tool for keeping progress visible is an Automation Dashboard—a one-page report that tells the automation story clearly and simply with charts and gauges. This report becomes a tool to improve your organization’s understanding, communication, and use of good automation practices. At the same time it helps keep the focus on costs, benefits, purposes, and related automation issues that are often overlooked. Kelly Whitmill explains how the dashboard provides a quick measurement of the automation and allows results to be compared to expectations and other test efforts. Measurement and visibility alone promote improvement by increasing awareness of your automation goals. Additionally, the simplicity of having a single, relatively easy-to-fill-in page makes it achievable and easier to share with others.

• A better understanding and visibility of good automation practices
• Simplify automation strategy measurement and reporting
• The fundamental principles of successful test automation

W4	Agile Testing

Has it ever happened to you? Due to an immovable delivery schedule, time for testing is squeezed and long hours are the norm at the end of a project. Well, let’s move the testing forward in the development process. In fact, teams are learning that, rather than relegating tests to near the end of a project, there are huge benefits to adopting a test-driven process—a more maintainable and robust design, testable code, fewer defects, and a cross-functional highperformance team. By combining unit test-driven and acceptance test-driven processes in a more agile setting, you can deliver on shorter schedules without sacrificing quality. Mike Cohn shares his experiences with test-driven development, including the flexibility it offers for dealing with schedule pressure, the role of documentation, and the steps for getting started. He also describes a test “pyramid” and how to separate test specifications from test writing.

• A model for test-driven development with unit and acceptance testing
• Split large features into small, but valuable, pieces that can be written in a test-driven manner
• The critical role of test automation in test-driven development

W5	Security Testing

Most everyone now realizes that we cannot solve security vulnerabilities with firewalls, virus scanners, and other tactics that build an electronic “moat” around systems. According to Julian Harty, security is not an operational issue, not a developer issue, and not a testing issue. It is a systems issue that you must focus on throughout the software’s life. From a QA/testing perspective, we need to look early in the development process for adequate security requirements. Then, we should assess the designs for vulnerabilities and participate in security code reviews. When specialized, security tests find bugs that get past our early prevention efforts, causal analysis helps prevent the recurring security defects. Dig into system security issues with Julian and learn about manual techniques, commercial software, and home-brew automation tools to help you find security vulnerabilities—before the bad guys do.

• Attack security problems at their source and in multiple ways
• Security anomaly testing techniques
• The latest automation support for security checking

Wednesday, May 18, 2005, 1:45 PM  Go to 11:30 AM   Go to 3:00 PM

W6	Test Management

When creating an entirely new QA organization, where do you start? Although establishing QA processes and standard practices is important, you must communicate with and obtain buy-in from QA staff, developers, customers, and business analysts. Walk in Karl Shearer’s footsteps as he describes the initial stages of QA's formation at Erie Insurance, the challenges and roadblocks he faced, and, ultimately, the successes he has enjoyed. From organizational issues, the mission statement, and job descriptions to selecting and hiring QA staff, you’ll take away a checklist for starting up a test group or improving your test organization. Employing a risk-based approach to define the levels of testing and a multi-pass system test strategy, Karl established basic test processes. Then, he added defect tracking, automation, and requirements management to the mix and designed templates for all QA deliverables. If you just are starting out, growing your test organization, or simply looking for new ideas, join Karl for a discussion of the issues and challenges he faced and, most importantly, the actions he took to create a new QA group.

• Basic steps that need to be taken to set up the QA infrastructure
• Lessons learned in staffing a new QA department (e.g., organization structure, job descriptions)
• Sample QA deliverables and metrics that allow you to add value early

W7	Test Techniques

Many have tried to implement software inspections in commercial development shops . . . and failed. Common reasons include “not enough resources,” “takes too long,” and “makes people uncomfortable.” Ron Yun demonstrates how ADP has implemented both “full” and “lite” inspection practices that reduce resources, automate many aspects of the process, and help take the personality issues out of the process. With their practices, ADP completes inspection meetings quickly and has eliminated all manual forms. They have merged key inspection information with data from their defect tracking system to determine progress toward the goal of finding defects as early as possible in development. Their automation tools track the identified issues, provide a permanent audit trail of the inspection, and automatically generate metrics to report status and gauge effectiveness. Take back templates that you can use with MS-Excel to implement these inspection practices and automation tools in your shop.

• Examples and benefits of inspections in commercial development
• Tools you can create to automate key aspects of the inspection process
• Conduct any inspection meeting in 60-minutes or less

W8	Test Automation

You have the automation tool. You have the right technical skills. You have the application experts at your disposal. It’s time to jump in and start coding! Or is it? Many well-intentioned test automation efforts fail due to a lack of planning. Steve Walters describes his practical approach for developing an overall test automation strategy. Learn how to plan for automation success, select the right tests to automate, and prioritize them for a faster return on investment. By quickly eliminating poor automation candidates and using Steve’s scorecard to assess the value of automating a test, you will be on the right track to achieving your automation goals. Take away a quantitative approach for deciding what to automate—and what not to automate—and the steps to develop a realistic plan and timeline for getting the job done.

• A written plan to set or reset expectations about the automation effort
• The challenges of successfully implementing automated testing
• An automation scorecard to prioritize automation candidates

W9	Agile Testing

Use Cases provide a user-focused sequence of events that also can serve as a template for functional testing activity. In an agile environment, use case-based testing brings testing earlier into the process and helps teams more quickly deliver higher levels of functionality to their customers. Testing with use cases also provides early peer review for the logic of intended functions. Involving testers in use case development and harvesting use case scenarios for test case development has shown great practical benefit in numerous application development projects. Dean Leffingwell describes the use case-based testing method and highlights practical tips for applying the technique. Learn to employ use cases to express functional requirements and how use case scenarios can serve as templates for testing activities.

• An overview of use case development practices
• Accelerating test development with use cases
• Tips and pitfalls in applying use case-driven testing in agile development

Double-Track Session!

W10  Security Testing Web Testing Circus: An Expert-Led Search for Security Defects Mike Andrews, Foundstone Professional Services Step right up! Come see the sights! Join in the fun! The circus is in town and admission is free with your STAREAST badge. Right before your very eyes, our security testing ringmaster Mike Andrews demonstrates for you the wonders of Web security testing. Behold death-defying feats of SQL injection. Stare open-mouthed as he hacks a site using the cross-site scripting attack. Watch him hijack a Web session before your very eyes. Find out how and why Web servers are the most attacked resource on the Internet and what you can do to protect yours. Learn the history of some successful and insidious Web hacks and the freak-show of hackers that perpetrate them. Bring your laptop with wireless access, and join in the attack! We will set up a wireless network and Web site with known vulnerabilities on an intranet in the session room so you can have a go at finding security bugs.

Wednesday, May 18, 2005, 3:00 PM  Go to 11:30 AM   Go to 1:45 PM

W11	Test Management

With billions of dollars changing hands every day, financial trading systems demand extremely high accuracy and reliability. So, how do you improve test process performance in the areas of time to market and efficiency and at the same time reduce failures? Over the last three years, using process and project measurement data as a guide, SIAC has focused on doing exactly that. Steve Boycan highlights the key elements of the process changes that have led to SIAC’s current performance: the use of a rigorous requirements engineering process; controlled parallel and iterative work flows; changes to the level of abstraction in test documentation; emphasis on test planning, analysis, and design; causal analysis; and improving the test team’s skills. Throughout the improvement program, SIAC has used quantitative information to guide the way, monitor performance, and plan each test project on one of the most critical NYSE floor trading systems.

• Test process changes that can significantly improve performance
• Key test process, planning, and tracking metrics
• Examples of improvements in test cycle time and defect containment

W12	Test Techniques

Testers sometimes focus only on product functionality; however, the best testers also consider the system implementation details: programming language, runtime environment, operating system, resource usage, and more. To improve your testing and find more issues that affect customers, you first need to understand what is really happening inside your product. Second, you need to test the interactions between the product and the system implementation (but you don’t need to re-test everything). Third, you need to ignore what you have learned—customers will always blame the product even if the system implementation caused something bad to happen. Join John Lambert to see examples of implementation details that cause problems in GUI-based applications, APIs, and Web systems. Find out how you can develop skills that will help you test your current product and will transfer to entirely different products in the future.

• Examples of implementation details that affect products
• How to test the interactions with the underlying implementation
• New test techniques for automated, exploratory, and penetration testing

W13	Test Automation

In many companies, there is a strange phenomenon—treating test automation specialists and the tests they develop . . . like crazy Uncle George. You know Uncle George. He’s not invited to the holiday parties or summer picnics when we can avoid it. He’s different. We don’t understand him completely. Like Uncle George, the automation team is out there working on . . . something. We are pretty sure they are. They run their tests . . . regularly. They seem to be contributing. But why should automated tests be different from other tests? Automation is simply a way of running test cases without (hopefully) so much human interaction. To get the most out of your automation effort, join Jamie Mitchell and learn ways to integrate automated testing into your other test and development processes. Decide who should own an automated test, who owns the failures, and ways to make test results available to everyone, including developers. From test results, develop and maintain meaningful metrics to help you assess the quality and value of automated testing in your organization.

• How automated tests are different from—and the same as—manual tests
• Artifacts from the automation process as valuable assets
• Metrics to assess the value of automated testing

W14	Agile Testing

Flexible, reusable, and restorable test data for both the unit and system level is an absolute necessity for testers working on agile development projects. For teams following a more traditional development process, agile approaches for handling test data can enhance testing efforts as well. Discussing agile testing approaches, such the ObjectMother pattern, Peter Schuh explains how testers design tests and test frameworks to survive the ever-changing data structures found in agile projects. Learn how an agile process allows testers to get much more mileage out of test data sets during and after the initial development project. Leave with a set of practices and techniques to apply directly to agile development projects or modify for their current development environment.

• The many dimensions of application test data
• Produce and maintain better tests and test data with agile approaches
• Agile techniques to create and maintain test data on agile and non-agile projects

Software Quality Engineering Home       Conference Home       To Exhibit       Get a Brochure       Register for STAREAST 2005

feedback  © 2005