Skip to content

Source generated type shape provider loads all assemblies for shaped types when only one is requestedΒ #252

Closed

Description

@AArnott

In adopting this in Visual Studio, we noticed unexpected assembly loads on startup when migrating to PolyType. Here is what I found as the blame when I wrote an assembly load test:

System.Exception : Assembly System.Numerics was loaded unexpectedly by the test with this stack trace: 
   at PolyType.SourceGenerator.ShapeProvider_Nerdbank_MessagePack_Tests.GetShape(Type type)
   at PolyType.Abstractions.TypeShapeProvider.Resolve(ITypeShapeProvider shapeProvider, Type type)
   at PolyType.Utilities.TypeCache.GetOrAdd(Type type)
   at Nerdbank.MessagePack.PolyfillExtensions.GetOrAddOrThrow(MultiProviderTypeCache cache, Type type, ITypeShapeProvider provider) in D:\source\repos\Nerdbank.MessagePack\src\Nerdbank.MessagePack\PolyfillExtensions.cs:line 47
   at Nerdbank.MessagePack.ConverterCache.GetOrAddConverter[T](ITypeShapeProvider provider) in D:\source\repos\Nerdbank.MessagePack\src\Nerdbank.MessagePack\ConverterCache.cs:line 130
   at Nerdbank.MessagePack.MessagePackSerializer.Serialize[T](MessagePackWriter& writer, T& value, ITypeShapeProvider provider, CancellationToken cancellationToken) in D:\source\repos\Nerdbank.MessagePack\src\Nerdbank.MessagePack\MessagePackSerializer.cs:line 256
   at Nerdbank.MessagePack.MessagePackSerializer.Serialize[T](T& value, ITypeShapeProvider provider, CancellationToken cancellationToken) in D:\source\repos\Nerdbank.MessagePack\src\Nerdbank.MessagePack\MessagePackSerializer.FriendlyOverloads.cs:line 124
   at MessagePackSerializerPolyfill.Serialize[T](MessagePackSerializer serializer, T& value, CancellationToken cancellationToken) in D:\source\repos\Nerdbank.MessagePack\test\Nerdbank.MessagePack.Tests\MessagePackSerializerPolyfill.cs:line 11
   at AssemblyLoadTests.AppDomainTestDriver.SerializeSomethingSimple() in D:\source\repos\Nerdbank.MessagePack\test\Nerdbank.MessagePack.Tests\AssemblyLoadTests.cs:line 81

It appears the source generated type shape provider has one massive GetShape(Type) method that references every single type for which we have a source generated shape. Very large methods take a long time to JIT, but more importantly every type reference in a given method must be resolved (and assembly loaded) before the method can execute, regardless of the paths that method's execution might take.

I took steps to avoid an assembly load problem like this in my source generated GetAssociatedTypeShape method by converting the input Type to a string and then implementing the method as a switch statement over the string, calling into other methods for each type shape 'factory' based on that string, so that the JIT doesn't need to load any unrelated assemblies. We could potentially apply that here. One complication with the string approach is whether to include the assembly name of the type, since including it can cause mismatches in the case of type forwarding and excluding it can cause type collisions where there are none. But I suspect we could work something out (e.g. disambiguate on type name collisions by loading colliding assemblies at that point).

Another option might be to use a ITypeShapeProvider.Resolve<T> method, which is the strategy that Nerdbank.MessagePack itself uses to avoid loading converters and types that are not necessary. This works because the JIT can tell at invocation time how if checks will evaluate and only loads the types necessary. We might test this more thoroughly to make sure it's true in all cases though (e.g. across ref types rather than just value types).

Activity

  1. AArnott commented on Sep 6, 2025

    @AArnott
    ContributorAuthor

    Upon further testing, the generic method approach isn't working. I'll have to test more to understand why it seems to work in other places. But I have confirmed that this works:

    internal object? NonGenericByString(Type t)
    {
    	string typeName = t.FullName;
    	if (typeName == "System.Numerics.BigInteger") return LoadBigInteger();
    	if (typeName == typeof(Stream).FullName) return Stream.Null;
    	if (typeName == typeof(int).FullName) return 0x42;
    	if (typeName == typeof(double).FullName) return 43.0;
    
    	[MethodImpl(MethodImplOptions.NoInlining)]
    	object LoadBigInteger() => new BigInteger(42);
    
    	return null;
    }

    Note we have to compare by string in order to avoid loading type references in the switching method. Also note that the actual factory is in a (local) function that has an attribute to prohibit inlining. Then the assembly (anti-)load test passes when verifying that System.Numerics did not load, when asking for something other than BigInteger.

  2. AArnott commented on Sep 6, 2025

    @AArnott
    ContributorAuthor

    Constructing the proper string is non-trivial, but already implemented in PolyType's SourceGenTypeShape<T>.ConstructStableName method.
    That method has only been used for associated types so far. If we're going to retrieve all type shapes based on this, we'd want to review it for correctness and perf. Constructing a string from a type on every invocation is probably a non-starter, but perhaps if we construct one for each type and store it in a ConcurrentDictionary it'll be ok?

  3. AArnott commented on Sep 6, 2025

    @AArnott
    ContributorAuthor

    Or maybe if #215 decorates a [GeneratesShape] type with an attribute, we can obtain the type shape directly without going through ITypeShapeProvider.GetShape(Type), thereby avoiding the massive switch statement and the assembly loads that follow.
    And maybe we can even avoid all calls to GetShape and thus make the whole graph trimmable.

  4. AArnott commented on Sep 6, 2025

    @AArnott
    ContributorAuthor

    Go figure. The string-based approach defeats trimming, and the Type-based approach defeats assembly lazy load. πŸ˜΅β€πŸ’«

  5. AArnott commented on Sep 6, 2025

    @AArnott
    ContributorAuthor

    Turns out the winning combination is... a combination.

    Start with string checks to avoid assembly loads, then also do a type check so that trimming works.
    And all this has to happen in generic methods or trimming doesn't work.

  6. eiriktsarpalis commented on Sep 7, 2025

    @eiriktsarpalis
    Owner

    I presume this issue is prompted by APIs that accept ITypeShapeProvider parameters, right? If so, I think this issue is closely related to #215. As it happens, I've been working on a solution that should make shape discovery for netstandard2.0 simpler, without needing to load the universe or compromising substantially on AOT support and trimmability.

  7. AArnott commented on Sep 7, 2025

    @AArnott
    ContributorAuthor

    Yes. It's impacting VS startup.

    The solution to this would ideally work for external types that require a witness type, such as serializing an int as the top level type.

  8. AArnott commented on Sep 7, 2025

    @AArnott
    ContributorAuthor

    What if we used a source generating interceptor to recognize calls to apis that take ITypeShapeProvider, and if it passes in a Witness class, replace it with a call to an ITypeShape pulled directly from the distinct property on the source generated provider?
    I guess that would have to be done at the level of my library since my library has the apis. And it would only help people who directly call into the serializer where the types are known to compile time. It wouldn't help the RPC scenario.

  9. eiriktsarpalis commented on Sep 7, 2025

    @eiriktsarpalis
    Owner

    TL;DR the prototype involves introducing a TypeShapeProviderAttribute that accepts a Type parameter implementing an ITypeShapeProvider. In netstandard2.0/netfx targets, the source generator will emit a local ITypeShapeProvider implementation inside every type annotated with [GenerateShape] or [GenerateShapeFor] that proxies the global, assembly-wide provider only for top-level types present in the local annotations. In an ns2.0 compatible project, the annotations:

    [GenerateShape]
    [GenerateShapeFor(typeof(Bar)]
    [GenerateShapeFor(typeof(Baz)]
    partial class Foo;

    Will result in the following codegen:

    [global::PolyType.Abstractions.TypeShapeProvider(typeof(__LocalTypeShapeProvider__))]
    partial class Foo
    {
        private sealed class __LocalTypeShapeProvider__ : ITypeShapePerovider
        {
             ITypeShape? GetShape(Type type)
             {
                  if (type == typeof(Foo))
                  {
                       return global::PolyType.SourceGenerator.ShapeProvider_PolyType_TestCases.Default.Foo;
                  }
    
                  if (type == typeof(Bar))
                  {
                       return global::PolyType.SourceGenerator.ShapeProvider_PolyType_TestCases.Default.Bar;
                  }
    
                  if (type == typeof(Bar))
                  {
                       return global::PolyType.SourceGenerator.ShapeProvider_PolyType_TestCases.Default.Baz;
                  }
    
                  return null;
             }
        }
    }

    This attribute is then consumed by a new method, TypeShapeProvider.ResolveDynamic<T, TProvider>() alongside the existing TypeShapeProvider.Resolve<T, TProvider>() where TProvider : IShapeable<T> method group that uses AOT-safe reflection to read the annotation and return the resultant type shape. I've been rolling out use of this ResolveDynamic method to my example libraries and I think it somewhat bridges the usability gap between modern .NET and framework.

    As I side-effect, I think this approach should serve to alleviate the immediate performance issue you are facing with VS. From a trimmability perspective, the trade-off is that each witness type is always linked to its provided types via the new attribute, however that should only impact netstandard2.0 targets.

  10. AArnott commented on Sep 7, 2025

    @AArnott
    ContributorAuthor

    I confirmed that this is an effective workaround when you know the type at compile-time:

    ITypeShape<bool> typeShape = PolyType.SourceGenerator.ShapeProvider_Nerdbank_MessagePack_Tests.Default.Boolean;
    byte[] buffer = serializer.Serialize(value, typeShape);
    return serializer.Deserialize(buffer, typeShape);

    How do you feel about users hard-coding knowledge of where you emit the source generated type provider like this?

  11. AArnott commented on Sep 7, 2025

    @AArnott
    ContributorAuthor

    I haven't gotten my head around your solution to #215. It looks like it will not automatically fix the assembly load issue, but it will allow isolating the unwanted assembly loads to sets defined by individual Witness types, so that if a particular serialization job doesn't want to load System.Numerics (for example) they can define an independent Witness type from what other code in the assembly uses.

  12. eiriktsarpalis commented on Sep 7, 2025

    @eiriktsarpalis
    Owner

    How do you feel about users hard-coding knowledge of where you emit the source generated type provider like this?

    I think that's fine, the location should remain stable as long as the project name doesn't change, but even if it does it should manifest as an easily fixable source breaking change.

    but it will allow isolating the unwanted assembly loads to sets defined by individual Witness types

    Not only that, it also substantially reduces the number of types by virtue of it only returning the top-level types present on attribute annotations. This is unlike the global provider which returns the full transitive closure produced by the source generator. That is, unless your local witness type explicitly includes a type from System.Numerics, it shouldn't be loaded at this point in time.

  13. AArnott commented on Sep 7, 2025

    @AArnott
    ContributorAuthor

    Another benefit of your design is it will reduce freeloaders where the Witness type only works because of another witness type defined in the assembly.

  14. AArnott commented on Sep 8, 2025

    @AArnott
    ContributorAuthor

    @eiriktsarpalis How do you feel about re-opening this issue given we still have some need and ideas for delivering on this? Particularly for the RPC scenario, serialization is based on an ITypeShapeProvider, and the only source generated one we can use today is the assembly-wide one which loads too many assemblies.

    Some ideas for fixes we've talked about offline:

    1. Expose Witness types' local type shape providerΒ #255
    2. Rewrite the source generated provider's GetShape method to start with a switch(type.Name) statement, which ultimately calls into [MethodImpl(MethodImplOptions.NoInlining)] local functions to return a shape for a type only when we know the type is almost certain to match, thereby avoiding unrelated assembly loads.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions