Finding false positives and false negatives can be tricky. Learn how to identify them.
August 29, 2018

What Are False Positives and False Negatives?

Static Analysis

Static analysis tools are notorious. They have a reputation for revealing a ‘wall of bugs’ every time you check your code.

All of these issues will need to be reviewed. And you’ll need to decide what to do about them. After all, a static analysis tool checks code against rules you’ve set up. But no tool is perfect. There will always be some false positives.

What Is a False Positive?

A false positive is an issue that doesn’t actually exist in the code. It doesn’t need to be fixed. This happens when no rule violation exists, but a diagnostic is generated.

Meanwhile, a true positive is an issue that needs to be fixed. It violates a rule and is, in fact, a real problem.

But sifting the true positives from the false ones can be tricky. And false negatives can be even trickier.

What Is a False Negative?

A false negative is an issue that goes undetected. This happens when a rule violation exists, but no diagnostic is created.

Meanwhile, a true negative means you don’t have an issue. There is no rule violation.

So, finding false negatives is really tricky. How will you know if there’s a bug you’ve missed?

What Causes False Positives and False Negatives?

There are two primary causes of false diagnostics.

Tools Make Mistakes

Tools aren’t perfect. They make mistakes. And false positives and negatives are inevitable.

That’s why it’s critical to have a human looking over your code — and any violations detected by the tool.

For instance, you may have a rule that there can be no Divide By Zero (DBZ) issues. The tool may then flag a section of code with a DBZ issue. So, you take a closer look at it and realize that there isn’t actually an issue here. You just had a false positive.

Undecidable Rules

You might have coding rules that can’t be decided — they’re undecidable. And that means it can’t be enforced with 100 percent accuracy.

How Does Undecidability Happen?

Undecidability can happen when you lack visibility.

If you had perfect visibility into everything in your program, you’d be able to decide whether a rule was violated or not. You could review diagnostics from a static analyzer and know “That’s a false positive!”

But, you don’t know everything that’s gone into your program. Other programmers wrote code for other parts of the program that you don’t have access to (e.g., firmware). Input came in from elsewhere. So, without clear visibility into everything, you can’t tell if there’s a real problem.

How to Diagnose False Positives and False Negatives

There are some false positives and false negatives that are no-brainers. They’re clearly black or white.

But there’s always a grey area.

Deciding False Positives and Negatives

Deciding diagnostics is subjective. It depends on the industry you’re working in. And it depends on the coding rules you’re working with.

False Positives Vary

A false positive for one company might not be a false positive for another.

Here's a false positive example. You might be developing software that will go in a car. Lives could be at risk if there are issues in the software. So, if you have a rule that there can be no DBZ issues — and you get a diagnostic that there are — you’ll need to carefully evaluate each violation.

But, you might be developing software to go in an entertainment system. So, you’d want to dismiss false positives quickly. You only want to look at true positives.

False Negatives Vary, Too

Likewise, a false negative for one company might not be false for another.

Here's a false negative example. You might use CERT or MISRA coding rules if you need to be really defensive about your program. A rule would be a false negative if it didn’t catch the possibility of something happening.

But, for another company, it would only be a false negative if it didn’t catch something that will absolutely happen.

As you expand your visibility, what you would consider a false positive or false negative gets refined.

Proving False Positives and Negatives

How much work you need to do to prove false positives and negatives varies. If you’re in a high-risk, safety-critical industry, you’ll need to prove it false. If you’re in a lower-risk industry, you might be able to review the diagnostic, dismiss it as false, and move on.

Examples of False Positives and False Negatives

Different developers have different interpretations of diagnostics. This has to do with both the industry they are working in — and their experience.

Here's how three types of developers interpret diagnostics. 

Different developers can think of the same diagnotics differently

How to Reduce False Positives and False Negatives

False positives and false negatives are inevitable.

False positives cost additional review time. And they may cause real issues to be hastily dismissed.

False negatives are a key concern for mission-critical software developers. For these developers, false positives are better than false negatives.

Not All Code Checkers Are the Same…

Not all code checkers — e.g., MISRA checkers — are the same. Some are more accurate than others. And some will give you more false positives and false negatives in your diagnostics.


How to Compare MISRA Checkers >>


Choose the Best Code Checker

Choosing the right code analyzer gives you better diagnostics. 

When you get the right diagnostics, you can reduce false positives and false negatives. So, you’ll have safe and secure code. Consistent style. And an easier-to-maintain codebase.

Find out why Helix QAC is the best code analyzer for C/C++.

Your future self will thank you. Even if you’re a Rockstar.

Why Helix QAC?