Static analysis of code in Visual Studio, Rule sets, etc

Published Dec 14, 2018

Intro

Visual Studio has a very interesting feature called Rule Sets. You basically create a file where you declare which warnings from analyzers will be ignored, displayed as info, warning or error. With the built in code analysis, but also with the help of a plethora of extensions and NuGet packages, this can be a very powerful tool. I am using VS2017 Professional for this post.

Create a rule set

Let's start with creating a new project (a .NET Framework console app) called Rulesets, which will create the standard Program.cs file and a solution for the project. Next, right click the solution in Solution Explorer and go to Add → New Item, go to the General category and select Code Analysis Rule Set. Name it whatever you want to name it, I will call it default.ruleset, then save it.

At this point you should be in a rule set editor, showing you a list of rules grouped by code analyzer id. Press F4 or go to the little wrench icon so that the Properties window is open, then give your ruleset a name (Default Rule Set) and save the file (ctrl-S).

You can create as many of these files and they will be saved in the Solution Items or as a file in a project (I recommend the former) and associate them to any number of projects. Let's assign the new rule set to our project: Go to the solution properties, select Common Properties → Code Analysis Settings, then select as many projects as you want in the list in the right. Then click on the little dropdown arrow an you should be prompted with a list of possible rules, including Default Rule Set. Select it.

Obviously, you can just choose a Microsoft included rule set instead, but those do not take into account your own extensions/packages.

Note: in order to start an incremental process, let's say starting with the minimum recommended settings from Microsoft and then adding stuff to it, use the Include element in a .ruleset file. The files coming by default with Visual Studio can be found at %ProgramFiles(x86)%/Microsoft Visual Studio/2017/Professional/Team Tools/Static Analysis Tools/Rule Sets. Example:

<Include Path="minimumrecommendedrules.ruleset" Action="Default" />

From the Visual Studio GUI you can click on the folder icon in the rule set editor top bar to include other sets and the wrench icon to open the settings.

This helps a lot with having small variations between your projects. For example a tests project might have different settings. Or the data access layer project might have auto generated files that don't respect your coding standards. You can just create a new ruleset that includes the default, then disables some of the rules, like mandatory class documentation.

Note that the rule set editor is not perfect. It will only show the rules as defined in the current file, ignoring the included sets. That is why if the default for a rule is None and you set it to Warning in your base set which you then include in another set where you set it back to None, it will not be saved correctly. Some manual checks are required to ensure correctness.

A list of analyzers

Now, I've noticed a list of possible extensions for Visual Studio that use this system. Here is a list of the ones I thought were good enough, free and useful.
Visual Studio extensions:

Microsoft Code Analysis 2017 - Live code analysis rules and code fixes addressing API design, performance, security, and best practices for C# and Visual Basic.
Security Code Scan - Detects various security vulnerability patterns: SQL Injection, Cross-Site Scripting (XSS), Cross-Site Request Forgery (CSRF), XML eXternal Entity Injection (XXE), etc.
MetricsAnalyzer - analyzer extension to check if you code follow metrics rules
Moq.Analyzers - Visual Studio extension that helps to write unit tests using Moq mocking library by highlighting typical errors and suggesting quick fixes
Code Cracker for C# - analyzer library for C# that uses Roslyn to produce refactorings, code analysis, and other niceties
Visual Studio Intellicode - this is interesting in the sense that it uses AI to improve your code and intellisense
Roslynator 2017 - A collection of 500+ analyzers, refactorings and fixes for C#, powered by Roslyn.
SonarLint for Visual Studio 2017 - Roslyn based static code analysis: Find and instantly fix nasty bugs and code smells in C#, VB.Net, C, C++ and JS.
clean-code-net - Set of C# Roslyn analyzers to improve code correctness
CommentCop - Analyzes (mostly) xml comments and provides code fixes. Uses Roslyn C# code analyser.

NuGet packages:

StyleCop.Analyzers - there is also a StyleCop extension, but weirdly it does not use the Rule Set system and you just run it manually and gives you warnings that you can control only through the extension configuration
Public API Analyzer - An analyzer for packages with public APIs.
UnityEngineAnalyzer - Roslyn Analyzer for Unity3D
AsyncAwaitAnalyzer - A set of Roslyn Diagnostic Analyzers and Code Fixes for Async/Await Programming in C#.
ef-perf-analyzer - EntityFramework Performance Analyzer
Asyncify-CSharp - an analyzer and codefix that allows you to quickly update your code to use the Task Asynchronous Programming model.

The packages above are NuGets you install in your project. Just check out the list from NuGet: NuGet packages containing Analyzer. Many extensions also have a NuGet counterpart. It's your choice if you want to not bloat your Visual Studio and choose to install analyzers on a per project basis. There is one advantage more in using project packages: the analysis will pop up at build time and you can thus enforce not being able to compile without following the rules in the set.

Curating your rule sets

So you've learned how to choose a rule set for your projects, how to create your own, but what do you use them for? My suggestion is to work on a complete rule set (or sets) for your entire company and then you can just enforce a coding style without having to manually code review everything.

If you are like me, then you probably installed everything in that list above and then tried it on your project... and you got tens of thousands of warnings and errors. How do you curate a rule set without having to go through every single message? I see several ways of doing this.

The perfect project

Some people/companies have a flagship technical project that they are very proud of. It uses all the latest technologies, it is perfectly written, thoroughly code reviewed by all the members of the team. It can do no wrong. If you have such a project, just enable all possible rules, then disable all that make suggestions for change. In the end you will have a rule set enforcing your coding style, for better or worse.

Start from scratch

The other solution is to start with a new project, then review every message until you get none. Then start coding. An iterative process, for sure, one you will never finish, but it will be good enough after a while and it will also engage your team in technical discussions on how to improve their code, which can't possibly hurt.

Add analyzers one by one

Start with a rule set where all rules are disabled, then start reviewing them one by one, as you either chose to disable them or refactor your code. This solution may be the worse, because it gives excuses to just stop the process midway, but it might be the only one available. Anyway, try to install extensions and packages one by one, too.

Start from the coding standards

Perhaps you have a document describing the coding standards in your team. You might start from it, then look for the rules that enforce it. I think that this will only make you see how woefully inadequate your coding standards are, but it might work.

Other notes

I've had the situation where I created derived rulesets from the default one (using Include) and somehow they ended up with an absolute path for the default ruleset file. It might be an issue with how the editor saves the file.

By default, rules in the ruleset editor are grouped by analyzer ID, but multiple analyzers might manage the same rules, so always manage rules individually, else you will see that enabling or disabling an entire group will change other groups as well and you won't know where you started from.

The ruleset editor is not perfect. One very annoying issue is with ruleset inheritance (doesn't load the parent rules). One example is that you want to have a general ruleset that does NOT include an analyzer (let's say the XUnit one) and then you want something inheriting the base ruleset to DO include the XUnit analyzer. While you can do it by hand, the ruleset editor will not allow you to make these changes, as the original ruleset will have every XUnit rule disabled, but the editor for the unit test one will not know this. There are two solutions for this:

Have a very inclusive basic rule set, then remove rules from the others. This is not perfect, as there could be circular needs (one set has one rule and not the other, while the other has them the other way around)
Have a lot of basic rule sets, split on topic. Then manage every rule set that you need as just includes. This works in every situation, but requires you edit the used rulesets by hand. In the case above you could have a special DoNotUseXunitAnalyzers.ruleset, for example.
A third solution that I do not recommend is to never include anything, instead just copy paste the content of the basic ruleset in every inheriting one. While this works and allows the editor and whatever engine behind to work well, it would be a nightmare to maintain.

Conclusion

I've discussed how to define and control static code analysis in your Visual Studio projects. If nothing of what is available is up to your standards, Roslyn now allows making your own code analyzers in a very simple way. Using code analyzers (and refactorings) can improve productivity, engage the team in technical analysis of standards, enforce some coding standards and help you find hard to detect errors in your code.

Hope it helps.