Root Cause Analysis of Defects

Here is a list of potential (common) defect Causes which can be used to divide the root cause of a defect list:

• Unintended Side Effect – The bug is due to an unanticipated side effect from a change somewhere else in the code.  This category can be used when the programmer would say to himself/herself “I had no idea a change in that area would affect something over in this area”. 
  
• Logic / Simple Err – The bug is due to a simple programming error.  This could include bad logic in an if statement, a typo, an uninitialized variable, a missing break statement, not checking for a nill pointer, etc. 
  
• Overlooked Use Case – The bug is due to the software being used in some way that the programmer didn’t expect. 
  
• Code Path Not Done – The bug is due to a code path not being completely implemented.  This could be the result of incomplete work being checked in and tested.  It could be the result of forgetting to come back to fill in some code.  This cause should always be selected if an “Unimplemented” assert was found in the code. 
  
• Misunderstood Rqmt – The bug is due to a misunderstanding of the feature requirements. 
  
• Requirements Changed – The bug is due to a change in the feature requirements at some point while the software was being written.  One example could be a changed UI. 
  
• External Dependency Chg – The bug is due to a change in an external subsystem or application.  This could include API changes or semantic changes in a library. 
  
• Design Flaw – The bug is due to a design flaw in the original implementation. 
  
• Code Rot or Hack – The bug is due to code that has become convoluted in logic or implementation because it has had so many changes done to it.  Another example is a bug due to a partial fix for some other problem that was put in due to lack of time. 
  
• Code Misunderstanding – The bug is due to the programmer not fully understanding the true functionality of the code.  This could be due to the code being overly complex.  Another possibility is that the programmer didn’t fully understand a new architecture that they were helping to implement.  Or perhaps they “cut and pasted” several lines of code without understanding what it did.
  

Collection of Best Practices of Software Development

Best practice is an idea that asserts that there is a technique, method, process, activity, incentive or reward that is more effective at delivering a particular outcome than any other technique, method, process, etc. The idea is that with proper processes, checks, and testing, a desired outcome can be delivered with fewer problems and unforeseen complications. Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people. - Wikipedia

I am writing this post to document the learnings & best practices that have either worked for me in my past or I strongly believe that these would work in practical circumstances and help replicate demonstrated success.

Design and Development:

1. Good, detailed requirements (Functional and Non Functional) help writing better deisgn, code and better test cases.

2. Keep the Design Simple.

3. Perform peer reviews.

• Provide training to ensure that your team members share a common understanding of the review process. Adopt some written procedures, checklists and forms for documentation of formal reviews. 
• Keep time for code reviews in the project plan
• Only people who would contribute to the review should participate. Big review teams are often not as productive as a small team. Keep it 3 -7
• Give ample time to the reviwers to review individually before coming for the meeting
• Reviewers Critique the Product, not the Producer
  
• Focus on the substance not the style of the code being reviewed
  
• Review meetings should not drift into problem solving

4. Keeping a threshold for open bug assigned to all individual (developers) help curb the hassles of the End game of a product release.

5. Test the code before Check-in.

Configuration Management:

1. Continuous commit and Integration by all developers to the mainline with Self Testing code i.e. Unit Test cases. I have not worked on Test Driven Development(TDD) but having a strong unit test case framework is the least that is required for continuous integration. A continuous integration server can also be used which acts a monitor to the repository. Every time a commit against the repository finishes the server automatically checks out the sources onto the integration machine, initiates a build, and notifies the committer of the result of the build. Many organizations do regular builds on a timed schedule, such as every night. This is not the same thing as a continuous build and isn't enough for continuous integration. The whole point of continuous integration is to find problems as soon as you can. Nightly builds mean that bugs lie undetected for a whole day before anyone discovers them. Once they are in the system that long, it takes a long time to find and remove them.

TDD is a design technique that drives the development process through testing. In essence you follow three simple steps repeatedly: Write a test for the next bit of functionality you want to add; Write the functional code until the test passes; Refactor both new and old code to make it well structure.

2.  Maintain a single source repository for all developers (CVS, VSS, Subversion, Perforce etc.) and keep the branches to a minimum.

3. Automate the build process. Keeping it manual can be a a breeding ground for mistakes. A common mistake is not to include everything in the automated build. You can use a smart tool to check the dates of the source and object files and only compile the module if the source date is later.

4. Daily Build and Smoke Test helps keeping the build stable.

5. A strong BVT (Build Verification Test) before build release to QE, can help save a lot of effort later. Usage of Virtualized machines can reduce the cost of your test labs drastically.
If you have build problems quite common, put a calendar on the wall that shows a full year with a small square for each day. Every day the QA group would put a green sticker on the day if they had received one stable build that passed the commit tests, otherwise a red square.

6. Keep a standard place/server location to keep the build for usage. It makes it easy to find and backup.

Testing:

1. Use functional requirements to write test cases and test design at the same time that the developer is performing the low level design and coding. This will help in refinement of the requirements and hence the implementation before the build is provided for testing.

2. Have formal Entry and Exit Criteria for SDLC phases and milestones.

3. Understand and implement in your test design how to write variations (different input, expected output) of a test to gain code coverage.

4. Create a sound plan for multiple platform testing by periodic rotation of the platforms being tested.

5. Plan Internal Beta testing leveraging on existing resources of your company. Dependent on the product, you may use the product within your organization at a production level.

6. Plan External Beta or Prerelease programs.

7. Develop User Scenarios where exhaustive testing is not possible. It also helps in reducing the complexities of the test cases.

8. Pay attention on usalibity, it was be the first factor for a user to define Quality. See what is the latest happening in Human Computer Interface.

9.   Teaming testers with Developers. Form the right team where the skills of various team members compliment each other.

10. Use Code coverage to see the efficiency of the test cases and minimizing regression tests.

11.  Automated Environment Generator: Go in for automated deployment and rollback of the build to the QA machines or servers.

12. Instrument the code for MTTF: You can implement a reporting back of failures and failure data which can help in diagnosing the problem.

13. Get the test plans reviewed by peers before getting it reviewd by the Product Management and Development teams.

14. Do not automate without optimizing the test cases eg. for functional and code coverage. Automating Junk helps creating junk faster.

15. It is important to analyze the results (predominately the Failures) of an automation run as soon as the results are provided.

16. Replication of user environment would help in replication of non-reproducible defects in house.

17. Usage of Virtualized machines can reduce the cost of your test labs drastically.

 

Project Management:

1. Do not create reports that are not required.

2. Perform post mortems and document the learnings while they are fresh.

3. A Plan is only as good as its implementation.

4. If you fail to plan, you plan to fail.

Further Reading:

http://www.ibm.com/developerworks/websphere/library/techarticles/0306_perks/perks2.html

http://www.construx.com/

http://www.sdbestpractices.com/

http://www.martinfowler.com/articles/continuousIntegration.html (ThoughtWorks)

http://www.processimpact.com/articles/revu_sins.html

http://www.belizenorth.com/articles/TIMEBOX.htm

http://www.spmn.com/lessons.html

http://www.ddj.com/architect/184415753

http://www.perforce.com/perforce/bestpractices.html

http://testing.com/

http://www.chillarege.com/authwork/TestingBestPractice.pdf

2 min Refresher for SCRUM - An agile development process

Please see the link http://www.controlchaos.com/about for a more detailed insight.

Scrum is a wrapper for existing engineering practices:

• Product Backlog > Series of Sprints > Sprint Planning meeting > Sprint Goals > Sprint Backlog > Daily Scrum meeting > Sprint Review meeting
  
• Roles: Product Owner, Scrum Master, Scrum Team
  
• Product Owner: Prioritize Product Backlog, provides and clarifies requirements
• Scrum Master: Responsible for the success of the scrum, establishes Scrum rules, shields the team from obstacles
• Scrum Team (PIGS - committed): Is cross functional (7±2). Can add to the product backlog, expands and self organize sprint goal and backlog to sprint tasks, can add or remove tasks, update estimates
• Observers (Chickens): People who are involved but not dedicated to the project 
• Sprint (normally 30 days) produces a potentially shippable set of functionality. No changes in sprint goal allowed in a sprint, unless the sprint goal is not making any sense. Stabalization sprints can be used for brining the quality to the required level or nearing the release. 
• Daily Scrum 15 minutes of scychronization not for problem solving - What have you done since the last Scrum? What will you do between now and the next Scrum? What got in you way of doing work? - Gives a clear status to all, creates a peer pressure for delivering on time.
• Sprint Review: The team (each team member) presents to management, customers, users and the Product Owner, the product increment built during the Sprint. Powerpoint presentations are forbidden!
• Scrums of Scrums is called Meta-Scrum

Difference between Iterative and Incremental Development:

Remember, you build a house iteratively and a colony incrementaly.

Alistair Cockburn Link

Sticky Minds Link