The ROI of Testing
Shaun Bradshaw, Questcon Technologies

In today’s competitive business environment, corporations need and demand a good return on investment (ROI) for everything they do—and testing is no exception. Although executive managers are requesting meaningful metrics more often than ever, many test managers are struggling to justify the cost versus benefit of their departments’ work. Often these test managers are unsure how to calculate investment costs versus dollars saved when using solid QA and testing methodologies. Test managers need the business tools and techniques to provide this business critical information, not only to satisfy upper management but also to ensure their departments are indeed making a positive contribution. Shaun Bradshaw demonstrates the tangible benefits of testing on the software development lifecycle by defining ROI in the context of software testing and defect prevention. In addition, Shaun provides several key metrics you can use to demonstrate the quantitative benefits of testing.

• Prevention (QA) costs versus detection (testing) costs
• Key metrics for calculating testing’s value to the business
• Testing ROI calculation examples

Monty Python's Flying Test Lab
Robert Sabourin, AmiBug.com

And now for something completely different . . . Monty Python's Flying Circus revolutionized comedy and brought zany British humor to a world-wide audience. However, buried deep in the hilarity and camouflaged in its twisted wit, lie many important testing lessons—tips and techniques you can apply to real world problems to deal with turbulent projects, changing requirements, and stubborn project stakeholders. Rob Sabourin examines some of the most famous Python bits—“The Spanish Inquisition” telling us to expect the unexpected, “The Dead Parrot” asking if we should really deliver this product to the customer, “The Argument” teaching us about bug advocacy, “Self Defense Against Fresh Fruit” demonstrating the need to pick the right testing tool, and a host of other goofy gags, each one with a lesson for testers.

• How to test effectively with persistence
• Make your point with effective communication
• Keys ways to clarify project goals and requirements

For Success, Build Record/Playback into Your Application
Gerard Meszaros, ClearStream Consulting

Stories about failed attempts to automate functional testing are very easy to find and have given the record/playback style test automation a black eye. Is this approach fundamentally flawed or can the business benefits of automated testing be realized through recorded tests? The flaw with most commercial record/playback tools is that they are intended for use with existing applications that have not been designed for testability. Therefore, the tools can only interact with the application through the user interface, making execution slow and prone to flakiness because user interfaces make terrible machine interfaces. Gerard Meszaros introduces the concept of designing testability and test recording capabilities directly into the application. This approach allows automated tests to interact with the application via a programming API to make them much more robust. Case studies illustrate how to design record/playback test automation into different application technologies and how this technique impacts the application design. 

• Why general purpose record/ playback tools have fatal flaws
• How recording can work well when built onto the program
• Designing testability into software design to improve your application

Performance Testing in Enterprise Application Environments
David Chadwick, IBM

As systems become more complex—serving the enterprise and implemented on the Web and across the Internet—performance testing is becoming more important and more difficult. David Chadwick suggests that the starting point is to design tests that reflect real user activity, including independent arrivals of transactions and varying input data to prevent “cache only” results. David explains how to break down the end-to-end system response time into the distributed components involved in processing the transactions. Learn to use resource-monitoring data to discover bottlenecks on individual systems. By examining the frequency and time spent in various processes, performance testers can determine where resources are being consumed and how to tune a system for better performance. Gain a new understanding of specific bottlenecks: repeated calls for individual items that could be returned as a set, requesting unfiltered data and filtering it repeatedly, inefficient structures and algorithms for sorting and searching, and lack of proper local caching.

• Recording and generating Web testing sessions
• Challenges of testing modern Web systems
• Using response time data to find software inefficiencies

Lightning Talks: A Potpourri of 5-Minute Presentations
Facilitated by Dawn Haynes

Lightning Talks are nine five-minute talks in one conference session. Lightning Talks represent a much smaller investment of time than track speaking and offer the chance to try conference speaking without the heavy commitment. Lightning Talks are an opportunity to present your single biggest bang-for-the-buck idea quickly. Use this as an opportunity to give a first time talk or to present a new topic for the first time. Maybe you just want to ask a question, invite people to help you with your project, boast about something you did, or tell a short cautionary story. These things are all interesting and worth talking about, but there might not be enough to say about them to fill up a full conference session.

Seven Habits of Highly Effective Automation Testers
Krishna Iyer & Mukesh Mulchandani, ZenTEST Labs

In many organizations, test automation is becoming a specialized career path. Mukesh Mulchandani identifies seven habits of highly effective automation specialists and compares them with Stephen Covey’s classic Seven Habits of Highly Effective People. Mukesh not only describes behavior patterns of effective automation testers but he also discusses how to internalize these patterns so that you use them instinctively. Drawing on his experience of managing large test automation projects for financial applications, Mukesh describes obvious habits such as saving and reusing tests. He then describes the uncommon but essential habits of strategizing, seeking, selling, and communicating. Learn how to avoid the bad habits that automation test novices—and even experts—may unconsciously adopt.

• Keys to successful test automation  
• Leadership skills needed by test automation specialists
• List of automation problem behaviors

Avoid Performance Testing Data Deception
Ben Simo, Standard and Poors

Don't be fooled by your performance test results. Performance testing can easily generate an unwieldy amount of data—some relevant and some not. Testers and their tools often use statistical methods to make sense of the data, but using statistics requires sacrificing accuracy and thoroughness. The good news is that we do not need to understand all the details to make good use of test results. The challenge is to determine what information really matters and how to present it in a useful manner. Join Ben Simo as he addresses  common performance test statistical problems including built-in bias, agreeable averages, invisible inadequacies, gargantuan groupings, stingy sets, mountainous molehills, creative charting, alien alliances, and more. Find out how statistical reporting can deceive rather than inform—often unintentionally—and recognize what the numbers do not say. By learning how to tell the truth with performance test results, you can give your stakeholders the information they really need to make good decisions. 

• What is takes to understand performance data
• How to protect yourself from misinformation
• Provide useful and understandable performance information to stakeholders

Optimize Your Testing with Virtual Test Lab Automation
Brad Johnson, Borland Software

The complex nature of software development often requires testing on multiple hardware platforms, operating systems, Web and application servers, and databases. Add to it the many different builds, patches, and regionalized versions that development delivers and you understand the immense challenge faced by test engineers trying to provide adequate test coverage. Adding virtual lab automation to your testing process can help your organization overcome these challenges and may dramatically improve the way you test—at a fraction of the cost of traditional multi-system approaches. Brad Johnson explores ways to seamlessly integrate virtual environments into the software testing process and explains how virtual test labs enable a test team to test more efficiently, across a wider range of environments, and with greater coverage of critical requirements. Join Brad to see if virtual test lab automation can improve software quality across your organization. 

• Leverage virtualization technologies to create a more efficient test environment
• Seamlessly integrate virtual environments into testing
• Test across a wider range of environments for increased coverage

Test Strategies for the Modern Distributed World
Dan Koloski, Empirix

Enterprise application development is quickly evolving with SOA and Web 2.0 taking center stage. Organizational structures are changing, with growing numbers of testing teams employing offshore resources. What do these changes mean to you, and what should you do to prepare? Most testing groups were created based on traditional development processes, traditional application architectures, and traditional organizational structures. As agile enters the mainstream, more change is on the way. Outsourcing, offshore development, and acquisitions continuously change the organizational landscape. Dan Koloski discusses proven and practical practices for adapting to today’s new technologies, new structures, and the new modern distributed world. He will discuss how to effectively communicate across virtual and physical silos as well as ways to adapt your test strategies and execution to component-based applications. Finally, you will learn how your organization can leverage Open Source applications and the Software-as-a-Service (SaaS) delivery model.

• Why traditional test groups are organized the way they are
• New technologies and new organizations for new testing approaches
• Ways to leverage new technologies to your advantage

Telling Your Exploratory Story
Jon Bach, Quardev

What do you say when your manager asks you, “How did it go today?” If you have a pile of test cases on your desk, it may be acceptable for you to say, “I ran x% of these tests today,” or “I’ll be finished with this stack in y days at the rate I’m going.”  However, if you’re using exploratory testing as your approach, it may be downright terrifying to try to give a status report, especially if project stakeholders think exploratory testing is irresponsible and downright reckless compared to pre-scripted test cases. So how, then, can you retain the freedom and power of exploration, and yet give a report that earns you credibility, respect, and perhaps more autonomy? Jon Bach suggests ways to describe and explain the critical and creative thinking that drives your exploratory testing so that others have a better chance of understanding and appreciating your value as an exploratory tester.

• Sixteen skills and tactics for exploratory testing
• Strategies for taking useful notes during exploration
• Preparing for stakeholder scrutiny of exploratory testing