Evidence based software testing

This work-in-progress paper will present our experiences in developing a new Software Testing and Quality Assurance QA course that integrates evidence from research and new developments in software testing as well as engineering education.

The specific goals are: 1. To incorporate empirical studies in software engineering to supplement instruction in testing of all aspects, including safety, security, reliability, and performance.

To increase focus on particular topics of high relevance such as formal testing of safety-critical systems and software inspection through targeted pedagogical interventions.

To leverage existing instructional materials from the software engineering education community to create and explore blended learning models such as a flipped classroom. To integrate and promote inclusive and reflective teaching practices in computer engineering courses, in general. We present detailed courseware and instructional modalities, including implementation details of daily in-class active learning activities, out-of-class assignments, and project resources, along with supporting materials from the literature and student feedback results.

Subbian, V. ASEE holds the copyright on this document. It may be read by the public free of charge. Other scholars may excerpt or quote from these materials with the same citation. But how does it relate to evidence-based managing of software? Gunther : Scrum is indeed a framework for complex product development, and brings empirical process control to software development.

However, the empirical foundation of Scrum can be similarly applied in the way the product development organization is being managed. Empirical management, like empirical product development thrives on inspection and adaptation, feedback loops, iterative-incremental progress and validated learning. Evidence-based management is a core practice of empirical management. Evidence of value is the primary source for inspection, bringing transparency over the value of software.

Managerial decisions over the creation of the software, the selected process and practices, can then be made against the assumption of improving the value. Assumptions obviously need to be frequently validated, hence the need for regular updates of evidence, the detection of patterns, and the correlation with the adoption of process and practices as reflected in secondary evidence.

Empirical management offers a way out of predictive management. Empirical management thrives not on predictions, and long-term detailed plans, but on frequent inspection and adaptation. It allows organizations a validated way of continuous adaptation, business agility.

InfoQ: Scaling agile seems hotter than ever before. How would you relate empirical management with the desire of companies to scale agile? Gunther : Many organizations are indeed under gigantic pressure, either through the market, either by their shareholders, to deliver more software, faster.

That in turn becomes the main driver for scaling efforts or programs. Empirical management is a splendid way to guide such efforts. It provides a better purpose to scaling efforts, i. But it also provides the means to measure and to direct the scaling efforts. Gunther : Many organizations have adopted Scrum to be agile in their software development, but fail to use Scrum to grow Scrum. The pressure is so high that entire development organizations are turned around toward agile development with Scrum at a pace that is not feasible.

The challenge of most organizations that want to become agile and subsequently scale agile is to understand agile first, and subsequently become agile by being agile. An important aspect of agile is emergent, concurrent development, performed in short iterations.

A scaling effort towards increased agility should thrive on those principles. Agility cannot be planned, nor dictated. And agility has no formal end-state. Agility is achieved through continuous adaptation. InfoQ: You stated that professional Scrum is the only foundation to scale. Can you explain this? Gunther : Ha. How can you improve your implementation of Scrum so it transcends the initial mechanical way of doing Scrum?

Every organization, every team starts by implementing the rules and roles of Scrum. They have the roles in place, organize the meetings, and follow the process. Yet, the creation of releasable software in a Sprint of 1 or a couple of weeks remains extremely difficult.

If one Scrum Team is unable to produce releasable software in a Sprint of 30 days, or less, let alone on a daily base, how would multiple Scrum Teams working on the same product be able to do so? InfoQ: Which advice do you want to give to enterprises that want to adopt Scrum? Facilitate existing teams with better and more means to create better software. The means to maximize Scrum first are countless. They relate to how the roles are implemented, how product management is involved, which engineering practices and infrastructure is in place, the stability of teams and many other aspects of working in teams.

Use the intelligence, knowledge and creativity of your teams. Facilitate bottom-up knowledge creation with guidance, vision, objectives, and evidence. Invest in professional Scrum, the Scrum framework implemented from its values and principles, and with a strong emphasis on technical excellence.

If you then need to add teams, you can do so on a solid foundation. It allows increasing the benefits created through Scrum through horizontal scaling, without adding complexity, without the need to add roles, or re-introduce phases, hand-overs and layers. And even if your complete development organization is being transformed toward Scrum, or has transformed, look for those teams that are Professional Scrum, or are most advanced. Allow them to laterally seed other teams with their professionalism.

And if you guide the scaling effort with evidence, the value of the software is maximized, which works a lot better than blindly increasing the volume of software produced. Watch Now! Join a community of over , senior developers. View an example. You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.