Miguel de Icaza releases

After almost 12 years, we are upgrading the guts of HttpWebRequest, the engine that powers the basic HTTP client stack in Mono.

.NET offers two sets of HTTP client APIs, the original HttpWebRequest that offers a comprehensive and configurable API that can communicate with HTTP 1.x servers using the original .NET 1.0 programming model, and a modern, pluggable and more limited API in the form of HttpClient which can also talk to HTTP 2.0 servers.

HttpClient is an interesting API as it has been designed to allow for different providers to be used and was also designed with async programming support in mind.

On each platform, HttpClient tries to use the best available transport:

Host/Runtime Backend
Windows/.NET Framework HttpWebRequest
Windows/Mono HttpWebRequest
Windows/UWP Windows native WinHttpHandler (HTTP 2.0 capable)
Windows/.NET Core Windows native WinHttpHandler (HTTP 2.0 capable)
Android/Xamarin Default to Android’s HTTP transport
Can be configured to be HttpWebRequest.
iOS, tvOS, watchOS/Xamarin Default to NSUrlSession (HTTP 2.0 capable)
Can be configured to use HttpWebRequest.
macOS/Xamarin Default to NSUrlSession (HTTP 2.0 capable)
Can be configured to use HttpWebRequest
macOS/Mono HttpWebRequest
macOS/.NET Core libcurl-based HTTP transport (HTTP 2.0 capable)
Linux/Mono HttpWebRequest
Linux/.NET Core libcurl-based HTTP transport (HTTP 2.0 capable)

As you can see, while HttpWebRequest is not the default across the board and lacks HTTP 2.0 capabilities, it is still available in various configurations in Mono-powered stacks (Mono and Xamarin).

Mono’s original HttpWebRequest stack was written in 2004 and was built using the asynchronous APIs that were available in .NET 1.0, that is the BeginInvoke / EndInvoke patterns, a pattern that was heavily based on queuing work, waiting for a result and resuming execution at a later point.

While the code grew in capabilities, features and reliability over the years, all of this was built on top of these 2004-era programming idioms. The code is difficult to read, to follow and understand. Simple bugs can take a long time to fix, and subtle problems can burn the most passionate developer. It has been a source of frustration for both us, and our users when a rare bug comes up in this stack.

At one point we tried to use the .NET Reference Source implementation, but that version was just too difficult to extract as it relies on many internals that do not exist in Mono, and in turn relies on Windows capabilities that we would have needed to support [1]. But this code also used those old idioms and while it might have fewer bugs and we might increase the compatibility of the stacks, the extraction was too complicated.

Over the years we have embraced expedience in the form of band-aids for this code, over the right fix, given just how complex the code had gotten. We are facing a couple of corner cases that are hard to fix and we wanted to fix the code for good, not apply another band aid.

So we decided to refactor the code from the old idioms from 2004 into 2017 idioms, using everything that is available to us in C# 7.

Chief among those capabilities is the use of async programming that removes both the complexity from the code and makes the code a pleasure to read and understand. It has taken us a few weeks to do the port, and we are currently validating every known scenario and test suite against it, but things are looking very promising.

You can track the state of this effort on Martin’s GitHub pull request.

[1] At one point we did write such a layer, the effort still lives dormant on a branch, for anyone that might be interested in resuming that work.

Jonathan Peppers releases

Today we are announcing the first preview release of Embeddinator-4000 for Android available on NuGet. The package also includes the previously released iOS support.

nuget install Embeddinator-4000

This will place Embeddinator-4000.exe into the packages/Embeddinator-4000/tools directory.

This release supports the following scenarios:

  • Call C# code from Java or C
  • Package a Xamarin.Android library project into an AAR package for use in Android Studio
    • Include Android layouts, drawables, assets, etc. from C# projects
    • Default linker behavior of SdkOnly
    • Generate AAR files on either Windows or OS X
  • Use Xamarin.Forms’ new embedding API to use Xamarin.Forms from Java
  • Package a .NET library project into a JAR file for use from Java on desktop platforms
  • Package a .NET library project into a dylib, so, or dll as appropriate per platform for use from C

For general usage, run Embeddinator on a compiled Android library assembly as follows:

mono Embeddinator-4000.exe -gen=Java -platform=Android -c YourAndroidLibrary.dll

This will output YourAndroidLibrary.aar for use in Android Studio.

In Android Studio, create a new module and choose Import .JAR/.AAR package: Import AAR Package

Adding a dependency to the new module, will give Java access to your .NET code: Add Module Dependency

For detailed instructions on using Embeddinator-4000 for Android see our Getting Started guide.

For a complete walkthrough, check out this video embedding Charles Petzold’s FingerPaint demo in an Android Studio project here:

Embeddinator-4000 for Android

For iOS support, you can use objcgen.exe included in the NuGet package. Additionally, you can follow the iOS Getting Started guide.

Rodrigo Kumpera, Miguel de Icaza runtime

Concurrent GC

In Mono 5.0 we are shipping a new operation mode for our Garbage Collector: Concurrent Garbage Collection.

Traditionally, when Mono’s memory manager determined that it should perform a garbage collection, the collector had to pause all Mono running threads, perform the garbage collection, and once it was done, it resumed the execution of those threads.

With concurrent garbage collection, we are able to perform collections on the old generation (what we call major collections) mostly concurrently with your application - it happens at the same time as your program is running. When the major collection is completed, the collector only needs to pause the Mono threads for a very brief period of time at the end.

This was an important feature for both our users of desktop workloads (like running Xamarin Studio, or Visual Studio for Mac) as well as game and mobile developers that did not want their application to exhibit large pauses when they had very large memory heaps to be collected.

The concurrent garbage collector is now enabled on Mono deployments by default, and we are making it available as an experimental option to all Xamarin platforms (Xamarin.Android, Xamarin.iOS, Xamarin.Mac, Xamarin.tvOS, Xamarin.watchOS).

We are rolling it out as the default on desktop first since it has more friendly hardware and it has received the most tuning. We are interested in hearing your experiences with the collector with your mobile applications to help us further tune the heuristics in the collector.

To get a sense of how this affect real world applications, let us examine Xamarin Studio.

Long GC latency can cause typing delays that are noticeable and lead to a bad typing experience. Pauses over 100-150ms are noticeable by most of us [1].

To give you a taste of how the concurrent collector improves things, I used it with Xamarin Studio to open itself, and build itself. This is one of the biggest solutions that I had around. As I open and build it, Xamarin Studio will be frantically allocating objects which, in turn, trigger the garbage collector - those GC pauses can be felt as hiccups when typing.

The experiment is to run this with and without the concurrent mode enabled. We measured the pause time of each GC run and grouped them in buckets by duration. For example, the 20ms bucket has all collections that paused between 10ms and 20ms. One thing to keep in mind is that the Y axis is in logarithmic scale, as the number of short pauses is significantly larger.

GC Pause Times

The result is that the long pauses, 200ms, 500ms and higher are now completely gone and even the worst case for concurrent GC shows that there are fewer 150ms pauses.

Even if those numbers are a significant quality of life improvement, there’s plenty of work left for us to get to no visible delays due to the garbage collector. We have already a solid pipeline of improvements coming on our next releases.

The concurrent collector work opens the door for many more improvements. This is just the beginning. What we plan on doing next is provide some defaults that are suitable for different workloads. On one end of the spectrum we have games that have very constrained timing budgets, and on the other hand we have applications that do not care about pauses, but would like to get more throughput.

Our focus for the next year is to leverage this work to satisfy the needs of both scenarios.

In the immediate roadmap, you can expect:

  • Mono 5.2 will introduce an optional parallel major collection mode to speed up the finishing pause for large heaps. This will help with some of the outliers we saw above.

  • Mono 5.4 will introduce and optional parallel minor collection mode that will help workloads with high activity on old objects. Roslyn falls into this category.

Using the Concurrent GC

On the desktop you enable the concurrent GC using the MONO_GC_PARAMS env var. Set it to major=marksweep-conc. Starting with mono 5.0 the concurrent collector is the default on desktop environments so you don’t need to use it.

On Xamarin.Android and Xamarin.iOS to enable it, go to project settings, under build options enable the “Use the experimental concurrent garbage collector” option.

Tuning the Nursery Size

The graph above shows the defaults for Mono.

In Mono, the Nursery is the name that we give to the youngest heap generation. When we talk about performing a minor collection, we talk about performing a collection that is limited to the nursery.

For games and other interactive applications that might create lots of small objects during their rendering cycle that are not long-lived, you might be able to take advantage of making the nursery size larger.

This reduces the chances that you will hit the limit that would trigger a major collection, and you can manually trigger a complete GC at a convenient time with GC.Collect() .

We hope to make this tuning a thing of the past when we introduce some of our new game and interactive features for this new mode.

[1] Typing with Pleasure

Alexander Köplinger releases

Mono 5.0 is out in the stable channel !

Highlights include shipping the Roslyn C# compiler and msbuild, enabling concurrent SGen garbage collection by default and continued integration of code shared with .NET.

Check out our release notes for more details about what is new on Mono 5.0.

This release was made up of over 2000 commits since Mono 4.8 and is the result of many months of work by the Mono team and contributors!

New Linux package repository structure

Linux users: we switched our package repositories to a new structure which means we’re now using a separate repository for each supported Linux distribution instead of one for all. This means you need to update e.g. your apt feed lists, please head over to the download page for updated instructions.

Read this blog post for more background info on the change.

Jo Shields linux, plans

Up to now, Linux packages on mono-project.com have come in two flavours – RPM built for CentOS 7 (and RHEL 7), and .deb built for Debian 7. Universal packages that work on the named distributions, and anything newer.

Except that’s not entirely true.

Firstly, there have been “compatibility repositories” users need to add, to deal with ABI changes in libtiff, libjpeg, and Apache, since Debian 7. Then there’s the packages for ARM64 and PPC64el – neither of those architectures is available in Debian 7, so they’re published in the Debian 7 repo but actually built on Debian 8.

A large reason for this is difficulty in our package publishing pipeline – apt only allows one version-architecture mix in the repository at once, so I can’t have, say, built on AMD64 on both Debian 7 and Ubuntu 16.04.

We’ve been working hard on a new package build/publish pipeline, which can properly support multiple distributions, based on Jenkins Pipeline. This new packaging system also resolves longstanding issues such as “can’t really build anything except Mono” and “Architecture: All packages still get built on Jo’s laptop, with no public build logs”

Mono repo pipeline

So, here’s the old build matrix:

Distribution Architectures
Debian 7 ARM hard float, ARM soft float, ARM64 (actually Debian 8), AMD64, i386, PPC64el (actually Debian 8)
CentOS 7 AMD64

And here’s the new one:

Distribution Architectures
Debian 7 ARM hard float (v7), ARM soft float, AMD64, i386
Debian 8 ARM hard float (v7), ARM soft float, ARM64, AMD64, i386, PPC64el
Raspbian 8 ARM hard float (v6)
Ubuntu 12.04 (*) ARM hard float (v7), AMD64, i386
Ubuntu 14.04 ARM hard float (v7), ARM64, AMD64, i386, PPC64el
Ubuntu 16.04 ARM hard float (v7), ARM64, AMD64, i386, PPC64el
CentOS 6 AMD64, i386
CentOS 7 AMD64

The compatibility repositories will no longer be needed on recent Ubuntu or Debian – just use the right repository for your system. If your distribution isn’t listed… sorry, but we need to draw a line somewhere on support, and the distributions listed here are based on heavy analysis of our web server logs and bug requests.

You’ll want to change your package manager repositories to reflect your system more accurately, once Mono 5.0 is published (Update: Mono 5.0 is now live, please check out the new instructions on the download page).

We’re debating some kind of automated handling of this, but I’m loathe to touch users’ sources.list without their knowledge. CentOS builds are going to be late – I’ve been doing all my prototyping against the Debian builds, as I have better command of the tooling. Hopefully no worse than a week or two.

(*) this is mainly for Travis CI support which still uses Ubuntu 12.04 for now (despite it being EOL)