PostSharp internals: Handling C# 8.0 nullable reference types

In PostSharp 6.4, we added full support for C# 8.0 and that includes nullable reference types. In this article, I discuss what we needed to do to implement this feature.

PostSharp as a code weaver

PostSharp is primarily a code weaver: it modifies the assembly produced by the C# compiler. As such, it works on your own code, and you may already be using nullable reference types. Did PostSharp need to do anything to keep working well in such scenarios?

It turns out that no, not really. Everything just works. Of course, C# doesn't have an up-to-date specification anymore, so it's hard to say for certain. But based on our tests and the information we collected from the web (especially the nullable-metadata.md file and the description of the new nullable-related attributes), PostSharp would continue working fine even had we done nothing. And we did a lot of testing, in fact, most of the time I spent implementing C# 8.0 support was spent on manual and automated testing.

The reason everything sort-of just works is that at the IL level, at which PostSharp operates, nullability annotations are represented as hidden attributes.

The hidden nullability attributes of C# 8

When you type string? myField in C#, compile it and then decompile with a pre-C# 8 decompiler, you will get [System.Runtime.CompilerServices.NullableAttribute(2)] string myField;. The NullableAttribute is one of two new "hidden" attributes. Its parameter determines the nullable state of the type:

• 0 means null-oblivious (pre-C# 8 or outside a nullable annotation context),
• 1 means non-nullable (e.g. "string"), and
• 2 means "may be null" (e.g. "string?").

The use of attributes in this way means that old code created before C# 8 can use C# 8 code. For us, it also meant a lot less work. After all, PostSharp is all about attributes so we're well equipped to handle those.

That said, there were a couple of corner cases that we wanted to solve. They may seem a little convoluted. Certainly most of our users will never encounter them. But they existed and we never want to ship a product with known defects. We've been burned by that before when it forced us to make backwards-incompatible changes later to fix the defects.

So here's one such edge case:

Edge case 1: Nullability of methods introduced with [IntroduceMember]

PostSharp has an attribute called [IntroduceMember] which you can use to insert a property or a method into another class, like this:

[SelfDescribing]
public class Creature
{
  public string? Description { get; set; }
  public int? MaximumAge { get; set; }
} 
[Serializable]
public class SelfDescribingAttribute : InstanceLevelAspect
{  
  [ImportMember(nameof(Creature.Description))]
  public string ImportedDescription;
  
  [IntroduceMember]
  public string DescribeSelf()
  {
      return "I am " + ImportedDescription;
  }
}

In the example above, the SelfDescribingAttribute adds the method DescribeSelf to the target class Creature.

Now, PostSharp modifies the binary, not the source code, so you won't actually be able to use the method in this project or in projects in the same solution (because project references refer to source code, not the binary). That is why this feature is used mostly to add methods expected by frameworks (the most notable case being XAML/PropertyChanged).

However, if somebody else imports your projects as a DLL library (either by referencing the .dll file itself, or as a NuGet package), they will see the introduced method. From their perspective, the class would look like this:

public class Creature
{
  public string? Description { get; set; }
  public int? MaximumAge { get; set; }
  public string DescribeSelf();
} 

But what then is the nullability of the return type of the method DescribeSelf — is it non-nullable (string) or nullable (string?). By the principle of least surprise, we felt the correct answer is "the same as in the template method", which here means non-nullable, so that's what we do — we make sure the metadata on the introduced member reflects that.

But if we did nothing (by not copying any attributes from the template method onto the target method), then in this case, the answer would be nullable. Why? Because the C# compiler doesn't just use NullableAttribute to mark which values are nullable, it also saves up on the assembly size by compacting several NullableAttributes into a single NullableContextAttribute.

The exact algorithm is described in the nullable-metadata.md file but it's along the lines of "if a class has more nullable members than non-nullable members, annotate only the non-nullable members with NullableAttribute and mark the class itself as nullable using NullableContextAttribute". The class would look like this: 

[Nullable(0)] // Class itself doesn’t have a nullability
[NullableContext(2)] // Members are nullable.
public class Creature
{
  public string Description { get; set; } // Inherits nullability from class
  public int MaximumAge { get; set; } // Inherits nullability from class
  public string DescribeSelf(); // Oops, this was supposed to be non-nullable.
} 

Now you may already see the problem. The target class was considered entirely nullable, but now we're introducing a method with a non-nullable return type to it. Therefore, we must take care to copy (or create, if necessary) proper attributes on any introduced methods (and the methods' parameters and return values) and on any introduced properties and events.

That's why the final class, as modified by PostSharp, would look like this if decompiled:

[Nullable(0)]
[NullableContext(2)]
public class Creature
{
  public string Description { get; set; }
  public int MaximumAge { get; set; }
  [NullableContext(1)] // The method’s return value and any parameters are non-nullable
  public string DescribeSelf();
} 

Edge case 2: New attributes on fields

Let's move to another edge case, one that occurs when you add one of the new nullability custom attributes (AllowNull, DisallowNull, MaybeNull, …) to a field or property.

These attributes allow you to express the nullability semantics of properties and methods that are more complex than "always may be null" or "is never null". For example, you may put the new [AllowNull] attribute on a property which is otherwise non-nullable. That is a note to the compiler that "null is allowed to be put into this property, but, since this is a non-nullable property, it will never return null".

Here's how you might use such an attribute:

[AllowNull, // <- a C# 8 attribute
UseRandomNameIfNull] // <- an example LocationInterceptionAspect, explained further down
public string Name { get; set; }

The AllowNullAttribute combined with the fact that the property type (string) is non-nullable in C# 8, means that this property's nullable status can be explained in English like this: "I never return null, but feel free to assign null to me. I'll handle it."

In vanilla C#, you could handle it by implementing the property's getter to return something instead of null, or by implementing its setter to assign something to a backing field. With PostSharp, you can do the same thing with a LocationInterceptionAspect:

[Serializable]
public class UseRandomNameIfNullAttribute : LocationInterceptionAspect
{
  private string name;
  public override void OnGetValue(LocationInterceptionArgs args) 
  {
    args.ProceedGetValue();
    if (args.Value == null) {
      if (name == null) {
        name = R.GenerateRandomString();
      }
      args.Value = name;
    } 
  }
}

But with PostSharp, you can also apply location interception aspects to fields, like this:

[AllowNull, UseRandomNameIfNull]
public string Name;

When I did that while testing PostSharp with C# 8, everything seemed fine — until I loaded the PostSharp-modified assembly in another project, at which point Visual Studio started complaining that I'm assigning a null to a non-nullable property. But why? Did I not use the AllowNull attribute properly?

Well, the way PostSharp makes location interception aspects work for fields is by transforming them into auto-implemented properties. That means that the IL code, decompiled, would look more like this:

[AllowNull, UseRandomNameIfNull]
public string Name { get; set; }

That seems just like the previous declaration, which works without a hitch. The problem is that the C# 8 compiler, when it sees [AllowNull] or [DisallowNull] on a property, silently moves those attributes to the property setter's parameter (the keyword "value"). A similar thing happens for [MaybeNull] and [NotNull]: those attributes get moved by the C# compiler onto the getter's return value.

This was a surprise to me. There being no specification, I looked in GitHub history but found little. I do remember that I found a single comment about this on some related issue (I lost it) and I know that this wasn't yet decided in May 2019, but that's it. Either way, emulating Roslyn helped, so PostSharp now does what the C# compiler does and moves these attributes onto their appropriate places if you declare them on a field.

Conclusion

As you can see, implementing support for C# 8.0 was not completely obvious, but we managed to address the corner cases anyway. But wow, do I miss the days that C# had a complete specification :). I completely empathize with the commenter at issue 64.

And there's still work to be done. PostSharp is not only a code weaver, but also a collection of libraries (such as PostSharp Logging or PostSharp Caching). As a library producer, we will eventually want to annotate our public API with question marks and the new attributes. I'm actually looking forward to this: going through all public types and methods and investigating their nullability sounds fun. Until then, even though our API remains null-oblivious, our code weaver handles code with nullable reference types well.