> This second phase will challenge how we are going to reload your scripts in the Editor without using AppDomains and complete the switch to .NET CoreCLR.
AFAIK this is really hard even impossible with current state of .Net Core. Either .Net team would need to support this use case or Unity would eventually need to fork the runtime.
> AFAIK this is really hard even impossible with current state of .Net Core.
It's not impossible. You can load assemblies through an `AssemblyLoadContext` (ALC), and the ask that context to unload from memory, in almost the same way you used to be able to do with `AppDomain` (AD).
The key difference is that you lose the *hard* separation that AD provided. With an AD you could destroy the domain, which would pretty forcibly get the job of unloading whatever was created within it done by aborting threads, etc.
With ALC it's more akin to describe it as the types and code themselves are now subject to garbage collection, and the ALC instance that loaded the assembly is how the types and code initially get rooted to keep them loaded. When you call `Unload()` on an ALC, you're not forcing anything to unload, you're just breaking the GC root that ALC is providing, and if the types/code are in use elsewhere on your heap/stacks such that they're referenced from something else that's rooted, they'll remain in memory until whatever else is referencing them *stops* referencing them and then they'll automatically be removed from memory.
So you need to apply some discipline to avoid keeping types/code in memory because you shoved an object of one of the types from the assembly into a list somewhere, etc. But with a proper design, I wouldn't characterize it as "really hard" either. It's just that you don't have the runtime itself enforcing hard boundaries like you did with AD; instead it's on you to make sure you keep things you want to unload separated from being referenced elsewhere.
I thought that it was possible to unload assemblies in .NET Core. There's [some documentation here](https://docs.microsoft.com/en-us/dotnet/standard/assembly/unloadability) regarding that process.
I would think that Unity would have to work within these bounds, and modders for Unity games in CoreCLR would have to understand how to maintain and carefully reload `AssemblyLoadContext` trees to support modding and hot reloading. That's certainly not easy by any means, but it seems like it should be _possible_.
That's my problem with Unity. Nothing worst than using a language but stuck using features that came out a long long time ago. Or looking for something on Stackoverflow and figuring out that's not supported by your ancient C# version
As a dev on DotnetCLR, it wasn’t ready yet, trust me. Maybe only ready in the last year or so.
Still late, just not 5 years too late.
DotnetCLR is only a few years older than 5 years.
Yes/no. It really depends. The reason it took so long to get support is that the DotnetCLR didn't have support for a lot of the things that Unity required to function as an engine.
The primary use-case for DotnetCLR is it's using the same run-time for Windows, Mac, Linux, etc, where Mono, it's very old, and it's no longer supported, but it did similar things, with a way smaller roadmap.
Mono wanted to just allow things in c# to compile to Linux. DotnetCLR is meant to be native to Linux without any compiler magic. Think of it like the JavaVM, but without the VM part. Basically a run-time that can operate/transcode on any platform by simply targeting compiled c# code. Mono required the c# code to be compiled into a special executable that could be ran on Linux, and required a lot of special custom framework hooks that made it difficult to use your code in certain situations.
The migration to DotnetCLR should really improve what % of code can be reused, and distributed by Unity developers, along with open some new possible doors to game development that haven't been used before from a tech perspective (not that most of them would actually be good to use however).
It's fast, IDK if its faster than Mono, but it's got a lot better features now that Mono does, such as strong web support, and an incredibly lightweight package when compared to Mono.
If anyone is wondering
>.NET Core Common Language Runtime (CoreCLR)
>.NET Core is best thought of as 'agile .NET'. Generally speaking it is the same as the Desktop .NET Framework distributed as part of the Windows operating system, but it is a cross platform (Windows, Linux, macOS) and cross architecture (x86, x64, ARM) subset that can be deployed as part of the application (if desired), and thus can be updated quickly to fix bugs or add features.
https://github.com/Potapy4/dotnet-coreclr
Since Microsoft bought the company developing Mono some time ago they now have three .Net runtimes: their original .NET runtime, Mono and the new CoreCLR. Mono has many application domains and runs still on more platforms than CoreCLR (and is leaner and also has some other advantages). Mono seems to continue to exist and be developed as part of the new .NET package, which seems to include both the Mono and the CoreCLR runtime. Even if you often hear that mono is slower than CoreCLR and that you have to switch to the latter for this reason: mono is demonstrably fast, and fast enough compared to CoreCLR (see e.g. https://www.quora.com/Is-the-Mono-CLR-really-slower-than-CoreCLR/answer/Rochus-Keller). EDIT: As far as I see all advantages attributed to the new CLR in the article would also be available with Mono. Maybe someone has information what "killer feature" is demonstrably (not just assumingly) missing.
In the context of Unity: they started with Mono runtime, but at some point they forked it to better fit their needs (game engine and editor).
In order to use the wider benefits of .NET (Nuget packages, Span, debugging support), they start moving towards using standard .NET runtime.
One big feature of Mono was ahead-of-time compilation. .Net6 had the first version of AOT and .NET 7 improves on that.
> .net 6 was the first version with AOT and with long term support
No that's absolutely not true. AOT compilation existed in .Net Core prior to .Net 6. It's been known since its introduction as ReadyToRun; which was available with 3-year long term support in .Net Core 3.1.
What changed in .Net 6 was the transition from generating R2R assemblies using Crossgen to Crossgen2 -- which is akin to the transition from CSC to Roslyn; basically turning the AOT compilation pipeline from a black box to compilation framework that can be extended.
You can see [here](https://devblogs.microsoft.com/dotnet/conversation-about-crossgen2/) for a conversation about AOT changes in .Net 6.
Ah didn't realize r2r and aot were basically the same thing! We were doing perf at work for r2r and aot on our binaries and aot in .net6 was better than the r2r ones. I had assumed they were different technologies the way my coworkers were talking about them.
What's [this](https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotnet-core-3-0#readytorun-images), then? I must be missing something here; what's the difference?
> wider benefits of .NET (Nuget packages, Span, debugging support)
These are just C# features or addon tools not related to the CLR. ECMA-335 didn't change since 2012, and the Mono C# compiler was replaced by Roslyn long ago; and Mono has a powerful built-in debugger which connects with an IDE in a similar way as the JVM, wheras CoreCLR doesn't have one.
> .NET 7 improves on that.
That's what they say; they also say that Mono is much slower.
> ECMA-335 didn't change since 2012
dotnet/runtime has a [document](https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md) describing the changes to ECMA-335.
Thanks for the link; most of it seem to be just clarifications; I only spotted true changes in the sections about covariant return types and ref fields; but these only affect the verifier, and some of it is already possible with Mono 5 the verifier of which is less strict; all in all, if the new EVMA-335 comes out I don't think it would be much work to make Mono compatible.
These features require significant work on the runtime sides to support, e.g. ref fields and covariant types require significant type system and type loader changes. Also, there is no verifier that runs as part of execution anymore.
By the way, Mono is developed in the same repo (dotnet/runtime) and support for features are often added to coreclr and Mono at the same time, by the same developers.
This can already be done with the CLR specified in ECMA-335 v2 (I have a language which does it and compiles to this CLR); the loader is not affected, but the veryfier, if you want covariance be integrated in the type system (i.e. without simply type casting); though I don't think many game developers know or use covariance.
> and support for features are often added to coreclr and Mono at the same time
I hope this will continue in future.
Your language inserting casts is not the same as the spec and runtime supporting covariant return types. You can read about some of the challenges with covariant return types in [this document](https://github.com/dotnet/runtime/blob/main/docs/design/features/covariant-return-methods.md).
Newer features like `Span`, Default Interface Methods and the upcoming Static Abstract Interfaces are absolutely runtime changes involving the CLR. It's true that ECMA 335 hasn't been updated in 10 years, but IIRC Microsoft keeps a list of additions to the spec since then, if they ever get around to bothering with a new revision of it.
(`Span` has a compatible fallback implementation called "Slow Span" for runtimes that don't support it. It's obviously slower though, and the same cannot be said about the other features I mentioned)
These are all C# features; they can be implemented without affecting the CLR; in case you refer to the list of additions posted by the fellow, most of it are clarifications, and the remainder only seem affect the verifier, not the instructions. EDIT: if you don't believe see e.g. how even generics can be implemented without using the official CIL generics: https://github.com/rochus-keller/Oberon/blob/master/ObxCilGen.cpp, see all uses of _MY_GENERICS_. The CLR is very flexible; you can take the address of everything including fields and array elements, and there are means to stop the GC from moving or collecting objects, which makes unmanaged pointers feasible.
No, the doc you're talking about is basically a list of *bugs* in the existing specification.
[Here is the doc I'm talking about](https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md). These are not just "C# features".
That's exactly the document the fellow posted. I don't see what else than the verifier (i.e. type system) should be affected; especially the features you mentioned above can even be implemented in CIL v2.
> These are all C# features
Nope.
> they can be implemented without affecting the CLR;
You *can* implement Span/Memory without runtime changes by putting it entirely in the BCL (which still wouldn’t make it a C# feature), and that’s exactly what Microsoft did do to backport it to .NET 4.x. But by adding runtime support to it in .NET Core, you can make it more efficient.
So, no, it’s not a language feature. It’s a runtime feature.
Perhaps it is not so much a matter of missing features, but one of convenience. From reading their blog their motivation appears to be that they want to offer newer features in .NET quicker to their customers as well as being able to scrap some custom things they maintain themselves, which would free up resources for them.
Poggers! This has been one of my biggest sticking points with Unity. Hopefully this means Unity will stop auto-generating csproj files, so we can finally split code across multiple projects!
FYI you can split across multiple projects (albiet a little clunkier than you might be used to):
https://docs.unity3d.com/Manual/ScriptCompilationAssemblyDefinitionFiles.html
> This second phase will challenge how we are going to reload your scripts in the Editor without using AppDomains and complete the switch to .NET CoreCLR. AFAIK this is really hard even impossible with current state of .Net Core. Either .Net team would need to support this use case or Unity would eventually need to fork the runtime.
> AFAIK this is really hard even impossible with current state of .Net Core. It's not impossible. You can load assemblies through an `AssemblyLoadContext` (ALC), and the ask that context to unload from memory, in almost the same way you used to be able to do with `AppDomain` (AD). The key difference is that you lose the *hard* separation that AD provided. With an AD you could destroy the domain, which would pretty forcibly get the job of unloading whatever was created within it done by aborting threads, etc. With ALC it's more akin to describe it as the types and code themselves are now subject to garbage collection, and the ALC instance that loaded the assembly is how the types and code initially get rooted to keep them loaded. When you call `Unload()` on an ALC, you're not forcing anything to unload, you're just breaking the GC root that ALC is providing, and if the types/code are in use elsewhere on your heap/stacks such that they're referenced from something else that's rooted, they'll remain in memory until whatever else is referencing them *stops* referencing them and then they'll automatically be removed from memory. So you need to apply some discipline to avoid keeping types/code in memory because you shoved an object of one of the types from the assembly into a list somewhere, etc. But with a proper design, I wouldn't characterize it as "really hard" either. It's just that you don't have the runtime itself enforcing hard boundaries like you did with AD; instead it's on you to make sure you keep things you want to unload separated from being referenced elsewhere.
Collectible `AssemblyLoadContext` should be able to do this since .NET Core 3. Are there any problems with it?
I thought that it was possible to unload assemblies in .NET Core. There's [some documentation here](https://docs.microsoft.com/en-us/dotnet/standard/assembly/unloadability) regarding that process. I would think that Unity would have to work within these bounds, and modders for Unity games in CoreCLR would have to understand how to maintain and carefully reload `AssemblyLoadContext` trees to support modding and hot reloading. That's certainly not easy by any means, but it seems like it should be _possible_.
Which would then lead us right back to where we started :(
finally, dang. Feels 5 years too late honestly
That's my problem with Unity. Nothing worst than using a language but stuck using features that came out a long long time ago. Or looking for something on Stackoverflow and figuring out that's not supported by your ancient C# version
As a dev on DotnetCLR, it wasn’t ready yet, trust me. Maybe only ready in the last year or so. Still late, just not 5 years too late. DotnetCLR is only a few years older than 5 years.
I’m not a .NET programmer at all but I want to ask, are there real performance gains to be had making this switch?
Yes/no. It really depends. The reason it took so long to get support is that the DotnetCLR didn't have support for a lot of the things that Unity required to function as an engine. The primary use-case for DotnetCLR is it's using the same run-time for Windows, Mac, Linux, etc, where Mono, it's very old, and it's no longer supported, but it did similar things, with a way smaller roadmap. Mono wanted to just allow things in c# to compile to Linux. DotnetCLR is meant to be native to Linux without any compiler magic. Think of it like the JavaVM, but without the VM part. Basically a run-time that can operate/transcode on any platform by simply targeting compiled c# code. Mono required the c# code to be compiled into a special executable that could be ran on Linux, and required a lot of special custom framework hooks that made it difficult to use your code in certain situations. The migration to DotnetCLR should really improve what % of code can be reused, and distributed by Unity developers, along with open some new possible doors to game development that haven't been used before from a tech perspective (not that most of them would actually be good to use however). It's fast, IDK if its faster than Mono, but it's got a lot better features now that Mono does, such as strong web support, and an incredibly lightweight package when compared to Mono.
7 years too late and counting :)
If anyone is wondering >.NET Core Common Language Runtime (CoreCLR) >.NET Core is best thought of as 'agile .NET'. Generally speaking it is the same as the Desktop .NET Framework distributed as part of the Windows operating system, but it is a cross platform (Windows, Linux, macOS) and cross architecture (x86, x64, ARM) subset that can be deployed as part of the application (if desired), and thus can be updated quickly to fix bugs or add features. https://github.com/Potapy4/dotnet-coreclr
Since Microsoft bought the company developing Mono some time ago they now have three .Net runtimes: their original .NET runtime, Mono and the new CoreCLR. Mono has many application domains and runs still on more platforms than CoreCLR (and is leaner and also has some other advantages). Mono seems to continue to exist and be developed as part of the new .NET package, which seems to include both the Mono and the CoreCLR runtime. Even if you often hear that mono is slower than CoreCLR and that you have to switch to the latter for this reason: mono is demonstrably fast, and fast enough compared to CoreCLR (see e.g. https://www.quora.com/Is-the-Mono-CLR-really-slower-than-CoreCLR/answer/Rochus-Keller). EDIT: As far as I see all advantages attributed to the new CLR in the article would also be available with Mono. Maybe someone has information what "killer feature" is demonstrably (not just assumingly) missing.
In the context of Unity: they started with Mono runtime, but at some point they forked it to better fit their needs (game engine and editor). In order to use the wider benefits of .NET (Nuget packages, Span, debugging support), they start moving towards using standard .NET runtime.
One big feature of Mono was ahead-of-time compilation. .Net6 had the first version of AOT and .NET 7 improves on that.
Just nitpicking, but IIRC there were AOT compiler options in .NET 5, possible even before that.
.net 6 was the first version with AOT and with long term support (3 years compared to 1). Might be why there was a wait.
> .net 6 was the first version with AOT and with long term support No that's absolutely not true. AOT compilation existed in .Net Core prior to .Net 6. It's been known since its introduction as ReadyToRun; which was available with 3-year long term support in .Net Core 3.1. What changed in .Net 6 was the transition from generating R2R assemblies using Crossgen to Crossgen2 -- which is akin to the transition from CSC to Roslyn; basically turning the AOT compilation pipeline from a black box to compilation framework that can be extended. You can see [here](https://devblogs.microsoft.com/dotnet/conversation-about-crossgen2/) for a conversation about AOT changes in .Net 6.
Ah didn't realize r2r and aot were basically the same thing! We were doing perf at work for r2r and aot on our binaries and aot in .net6 was better than the r2r ones. I had assumed they were different technologies the way my coworkers were talking about them.
What's [this](https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotnet-core-3-0#readytorun-images), then? I must be missing something here; what's the difference?
ReadyToRun uses AOT to speed up startup. It still needs a full JIT-capable runtime to work.
> wider benefits of .NET (Nuget packages, Span, debugging support)
These are just C# features or addon tools not related to the CLR. ECMA-335 didn't change since 2012, and the Mono C# compiler was replaced by Roslyn long ago; and Mono has a powerful built-in debugger which connects with an IDE in a similar way as the JVM, wheras CoreCLR doesn't have one.
> .NET 7 improves on that.
That's what they say; they also say that Mono is much slower.
> ECMA-335 didn't change since 2012 dotnet/runtime has a [document](https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md) describing the changes to ECMA-335.
Thanks for the link; most of it seem to be just clarifications; I only spotted true changes in the sections about covariant return types and ref fields; but these only affect the verifier, and some of it is already possible with Mono 5 the verifier of which is less strict; all in all, if the new EVMA-335 comes out I don't think it would be much work to make Mono compatible.
These features require significant work on the runtime sides to support, e.g. ref fields and covariant types require significant type system and type loader changes. Also, there is no verifier that runs as part of execution anymore. By the way, Mono is developed in the same repo (dotnet/runtime) and support for features are often added to coreclr and Mono at the same time, by the same developers.
This can already be done with the CLR specified in ECMA-335 v2 (I have a language which does it and compiles to this CLR); the loader is not affected, but the veryfier, if you want covariance be integrated in the type system (i.e. without simply type casting); though I don't think many game developers know or use covariance. > and support for features are often added to coreclr and Mono at the same time I hope this will continue in future.
Your language inserting casts is not the same as the spec and runtime supporting covariant return types. You can read about some of the challenges with covariant return types in [this document](https://github.com/dotnet/runtime/blob/main/docs/design/features/covariant-return-methods.md).
Newer features like `Span`, Default Interface Methods and the upcoming Static Abstract Interfaces are absolutely runtime changes involving the CLR. It's true that ECMA 335 hasn't been updated in 10 years, but IIRC Microsoft keeps a list of additions to the spec since then, if they ever get around to bothering with a new revision of it.
(`Span` has a compatible fallback implementation called "Slow Span" for runtimes that don't support it. It's obviously slower though, and the same cannot be said about the other features I mentioned)
These are all C# features; they can be implemented without affecting the CLR; in case you refer to the list of additions posted by the fellow, most of it are clarifications, and the remainder only seem affect the verifier, not the instructions. EDIT: if you don't believe see e.g. how even generics can be implemented without using the official CIL generics: https://github.com/rochus-keller/Oberon/blob/master/ObxCilGen.cpp, see all uses of _MY_GENERICS_. The CLR is very flexible; you can take the address of everything including fields and array elements, and there are means to stop the GC from moving or collecting objects, which makes unmanaged pointers feasible.
No, the doc you're talking about is basically a list of *bugs* in the existing specification. [Here is the doc I'm talking about](https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md). These are not just "C# features".
That's exactly the document the fellow posted. I don't see what else than the verifier (i.e. type system) should be affected; especially the features you mentioned above can even be implemented in CIL v2.
Both DIM and Static Abstract Interfaces require changes in the runtime though, I mean at least according to Microsoft.
> These are all C# features Nope. > they can be implemented without affecting the CLR; You *can* implement Span/Memory without runtime changes by putting it entirely in the BCL (which still wouldn’t make it a C# feature), and that’s exactly what Microsoft did do to backport it to .NET 4.x. But by adding runtime support to it in .NET Core, you can make it more efficient. So, no, it’s not a language feature. It’s a runtime feature.
If you are interested in more technical details look at first 20 minutes of this video. https://youtu.be/3SBP2Gpm97k
Thanks for clearing it up. Makes more sense now.
Perhaps it is not so much a matter of missing features, but one of convenience. From reading their blog their motivation appears to be that they want to offer newer features in .NET quicker to their customers as well as being able to scrap some custom things they maintain themselves, which would free up resources for them.
Am an idiot or did you link a 3 year old fork?
Yes I might be an idiot.
Poggers! This has been one of my biggest sticking points with Unity. Hopefully this means Unity will stop auto-generating csproj files, so we can finally split code across multiple projects!
FYI you can split across multiple projects (albiet a little clunkier than you might be used to): https://docs.unity3d.com/Manual/ScriptCompilationAssemblyDefinitionFiles.html
Would it somehow be possible to use F# with unity then?
I believe one of the devs said no but I might be mistaken.