How should I verify ”overlapping” results from Code Prover and Bug Finder?

3 views (last 30 days)
At my company, we use Bug Finder in tandem with Code Prover to check for potential bugs/violations of coding standards and some specific run-time errors.
Normally, I skip through some of the Bug Finder check results (namely zero-division, integers overflow, out-of-bound-access pointers/arrays/loops, etecera) if there's already "overlapping" checkers from Code Prover to save time. For example, if I had confirmed the validity for all Code Prover's overflow checks against a piece of code (which also produced another set of overflow check results in Bug Finder as well), then regardless of whether the Bug Finder results are duplicates or different checks from Code Prover, I'd automatically assume those are not defects and ignore them totally.
My question are as following:
  1. What is your approach when verifying results of categories that are checked in both tools?
  2. Is it safe to justify only the result from Coder Prover and totally ignore the "less" exhaustive check results from Bug Finder in such cases where both tools offer the same check? If not, then what are potential risks if I only rely on Code Prover results for run-time errors detection? (As stated in the offical documents, Bug Finder's main usage is to quickly detect and rectify bugs and it might not detect all the possible causes for run-time errors, which is one of Coder Prover's strong points so I think it's would be fine to just solely rely on Code Prover)
Thanks in advance for reading and it would be great if someone can give some pointers!

Accepted Answer

Anirban on 6 Feb 2023
Edited: Anirban on 6 Feb 2023
  1. The recommended approach is to use Bug Finder first (and more frequently), and fix the simpler bugs found. Then, use Code Prover to perform the more exhaustive checks. If you have Polyspace as You Code, it can be run even prior to running Bug Finder.
  2. You can skip the checkers of Bug Finder if you review the results of the equivalent checkers in Code Prover. In general, if a Bug Finder checker can find an issue, the equivalent checker in Code Prover can also find the issue (as long as the same options are used for the Bug Finder and Code Prover analysis - for instance, if the tools are run on the same project in the UI or using the same options file). Of course, as you know, Bug Finder can also find many, many other types of issues that Code Prover does not even look for (because finding those issues do not require mathematical proof). But I am assuming that in this question, you were only asking about those checkers of Bug Finder that overlap with Code Prover.
ドゥック・ミン on 21 Feb 2023
Hi Anirban,
1. We deliberately check for Code Prover results preceeding Bug Finder's, that is due to the requirement from our client. In our case, checking Bug Finder results is requested later and this is supposed to be more of a supplementary check. Furthermore, we tend to check Code Prover results often since each result provides a detailed line and column, which is sometimes missing from Bug Finder check.
Regardless, we have never encountered an instance of Bug Finder results negating the equivalent Code Prover results so it is probably not a problem for our codes but I will make sure to keep this in mind.
2. I meant "checking conditions", but you got it right so no issue here,
"For Code Prover to find the same things as Bug Finder (and then some more), the analysis options used must be the same. More precisely, the options in the categories Target & Compiler, Macros, Environment Settings, Inputs & Stubbing must be the same. One easy way to make sure the options are same is to run Bug Finder and Code Prover on the same project (or using the same options file)."
→ I am not well-informed on these settings one-by-one since we had our team develop in-house tool to automate the Code Prover/Bug Finder analysis not on the Polyspace interface but rather from inside MATLAB interface. However, I do know that we use some options for our tool so I will make sure to forward this part of your reply to our team for further investigation.
"For the subset of checkers that are common to Bug Finder and Code Prover, essentially the same analysis algorithms are used, except that in Bug Finder, certain heuristics are used to minimize false positives and minimize the analysis time. These heuristics prevent certain things from being shown in Bug Finder while they appear as orange checks in Code Prover (the orange cahecks might be real issues, especially the critical orange checks, or false positives). So yes, for the equivalent checkers, the issues found by Code Prover should be a superset of the ones found by Bug Finder. "
→ Except for orange checks related to floating-point overflow and some other obscure checks, we do review all remaining orange check results very carefully, and lastly, Bug Finder results. If the checking conditions found in Code Prover for a particular category is a superset of those found in Bug Finder, then I think my current checking routine should not be affected in anyway. As you pointed out, this part of your answer would be of tremendously helpful to form our baseline for reviewing codes and inquiry sessions with our client.
For the time being, all of my questions were answered and I don't have any further follow-up question so I will mark your answer as the solution.
Thank you again for taking your time to answer! I wish you a good day and take care!

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by