Even the most type-safe and well-tested code can have some unhandled edge cases or unexpected behavior. Whether you're using an open source project or some commercial software, it's important to know how to write a good bug report. A good bug report can be the difference between a bug that is easily fixed, and a bug that is simply closed with the label WONTFIX.

Here are some of the most important qualities of a good bug report:

1. Bug Reports Should Be Unique

If you find a bug in an open source library, the first thing you should do is search the project's GitHub Issues for any existing bug reports. It's also important to search both closed and open issues, as well as any pull requests. When you visit a GitHub issues page, you should remove the default is:issue is:open text in the search box before you type in your search query.

When reporting a bug in some commercial software, you might want to search Google or any community forums for known issues.

2. Bug Reports Should Use the Latest Version

Once you've confirmed that the bug hasn't been reported yet, the next thing you need to do is to confirm that the bug is still happening in the latest version of your software. The bug might not be reported, but the maintainers might have already noticed it and fixed the issue.

This has happened to me more than a few times in the past:

  • I find a bug in an open source project.
  • I figure out how to reproduce the bug, and get half-way through writing up a bug report.
  • I finally decide to search GitHub issues or test the latest version, and then I realize that the bug has already been fixed.

Hopefully I've learned my lesson from these experiences! You should always search GitHub issues first, and then make sure you're using the latest version.

3. Bug Reports Should Be Reproducible

Here are the steps I generally follow whenever I fix a bug in DocSpring:

  • Reproduce the bug while running the application on my local machine
  • Read through the code and figure out why the bug is happening
  • Write a failing test case
  • Run the test case, and ensure that the test is failing for the expected reason
  • Fix the bug in my code
  • Run the test again, and confirm that the test is now passing
  • Confirm that the bug is fixed on my local machine
  • Run any other relevant test cases on my local machine
  • Commit and push the changes
  • Confirm that all tests are passing in the CI build
  • Deploy the changes to my staging servers
  • Confirm that the bug is fixed on the staging servers
  • Deploy changes to production
  • Confirm that the bug is fixed on production
  • Notify any affected customers that the bug is now fixed!

When you're reading through this checklist, what sounds like the most difficult part? Is it fixing the bug, figuring out why the bug is happening, or maybe writing a test case that reproduces the bug?

These steps can all be tricky, but it might be surprising to learn that the very first step in this process usually takes up the vast majority of my time and effort. Once I'm able to reproduce a bug consistently, everything else usually falls into place. Then I'm usually able to run through the rest of the checklist with very little effort. Reproducing a bug can be similar to solving a very difficult step in a Sudoku puzzle; sometimes you just need to find one more number, and this number completely unwinds the rest of the puzzle with very little effort. On the other hand, an unreproducible bug is usually an unfixable bug.

Being able to consistently reproduce a bug can be the most difficult and frustrating step, but the good news is that the end-user of some software will often have the opportunity to perform most of this work. You can take a huge burden off the maintainers or customer support team if you spend an extra 10-20 minutes trying to find some step-by-step instructions that consistently reproduce a bug. Your bug is also much more likely to be fixed in a short amount of time.

4. Bug Reports Should Include a Minimal Example

You should always try to provide a minimal example in your bug report. This means that you provide the smallest possible example that consistently reproduces a bug. An example that only involves a single method call is far more helpful than just giving access to your entire project, or pasting in the contents of 10 different files with 100 method calls. A small example means that the maintainer doesn't have to spend any time wading through lots of irrelevant code. They don't have to spend any time installing dependencies and trying to get your project running on their own computer.

However, here's the most important reason for a minimal example: It shows that the bug is actually caused by a specific library. Maintainers don't have the time or energy to wade through a whole bunch of code and finally reproduce a bug, only to find out that the bug isn't even related to their code. Or even worse: it's just a typo in your own code.

There have been many occasions where I've started working on a minimal example, only to find out that the bug was caused by something completely unexpected. Then I can delete my half-written GitHub issue instead of wasting everyone's time.

Summary

These are some of the most important qualities in a good bug report:

  1. Bug Reports Should Be Unique
  2. Bug Reports Should Use the Latest Version
  3. Bug Reports Should Be Reproducible
  4. Bug Reports Should Include a Minimal Example

Whether you're writing a bug report for an open source project or some commercial software, please try to include some step-by-step instructions that can consistently reproduce the bug. A few minutes of your time can sometimes save hours of frustration for open source maintainers, and a well-written bug report is far more likely to result in a fixed bug. And of course, a well tested Pull Request is even better!