Implement a mono debug server feature

[aldelaro5]

This brings the debugging capability of the bootstrap to be on par with doorstop.

First off, I want to clarify something I found while making this: while melonloader USED TO support dnspy debugging, it hasn't had the feature for a while. This is odd because this page specifically implies it's supported, but the logic simply wasn't there: https://melonwiki.xyz/#/modders/debugging

Luckily, this pr makes addressing this relatively easily. Since the guide also mentioned about this being tied to the debug_mode setting, I decided to reuse it for this as the enable toggle. I was debating if I should add a setting, but it would add confusion with launchdebugger which is specific to windows and il2cpp so I decided to keep it simple and reimplement what the guide mentions which now becomes accurate with this pr.

Why do it like this?

It is by far the most powerful and versatile debug method out there. It's not specific to dnspy, it doesn't require any patched mono or to convert the build into a dev one and it is in general VERY close to the debugging experience a game dev would have if they were to debug a player build. The only thing that's lacking is support for the unity editor's profiling where if you need that, you still need to convert the build to a dev one, but as far as just regular debugging things in vs, rider or dnspy, this supports everything. In some situations, it's even possible to debug the game itself by recompiling their Assembly-CSharp.dll (ilspy to decompile, then recompile that with symbols) and debug the whole game for research purposes.

Due to this leveraging a built in mono feature, it works in as far as I can tell, even ANCIENT games. I had this work on a 4.6.6p3 game to give you an idea while also getting it to work on a 2018.4.12f1 game in new mono (I also tested one with a 2019 version). It supports attachment after launch or to let the debug server stall the process until a debugger attach so you can really get a debugger early in the boot process. It even works over network: you could set the address to 0.0.0.0 and debug from another device in your LAN. It's really powerful!

On dnspy, you can select Unity and do as the guide I linked above says and it will work, but you now can use Unity (connect) with the save ip address and port set in the loader config. For Visual Studio, you need to have VSTU installed, go into debug -> Attack Unity debugger, "input IP" and same deal from there. For Rider, it's built in and it's in Debug -> Attach Unity Process, "enter manually" and same thing. I only briefly tested vscode and while I could connect, I think their client is broken because it doesn't let me do much once it connects. Still, it could now in theory be possible to use vscode if it's fixed.

How does it work?

There's only 2 things you need to have any mono be hijacked to start a debug server even on game builds that aren't development builds:

• Either hijack or call mono_parse_jit_options with some debugging arguments (the whole thing that starts with --debugger-agent and stuff)
• Call mono_debug_init

At a high level, that's exactly what this PR is doing, but the above has to be done before calling mono_jit_init_version. So I install a hook for mono_parse_jit_options in case it gets called before (this can happen sometimes on newer monos) which adds an arg with the params we want. Whenever our mono_jit_init_version hook kicks in, I force a call to the mono_parse_jit_options detour which makes sure that it was called by this point (double calling here is fine and didn't resulted in crashes, but doing it this way makes sure that by this point, the args were hijacked). Once that's done, mono_debug_init is called, but since it's technically possible that we are dealing with a development game build, there's protection in place to not call it if mono itself did because that can lead to crashes (also happens if mono_debug_enabled is available AND returns true, not all mono has that symbol exported).

and...that's really all there is to ti! There are some details to keep in mind:

• The net35 mono uses a different syntax to specify you don't want the suspend feature. This is why it's handled differently
• There is no reasons to ever call mono_debug_domain_create. This was figured out on doorstop a couple weeks ago by 6pak and I and I explained the whole reasoning why here: Improve the mono init debug logic to be simplified and works on Unity 5.4 and older NeighTools/UnityDoorstop#70 Luckily, the bootstrap was smart enough to handle it in case it wasn't present so no crashes happened, but I could still get symbols to work without and considering not all mono has it, it's not really a good thing to call it when we know mono will call it for us anyway
• DnSpy config is interesting: this server works as is if you attach, but for STARTING using DnSpy, there's an env var that gives you the debug args directly. In that case, it has to disregard the loader config because dnspy will select a random port and we have to honor it by passing them as is.
• The default port is simply because it's also the default on dnspy when you select Unity (connect). It was 10000 for doorstop, but it's really fine as long as the port isn't reserved or in use so I opted to use 55555 as the default here.

My testing

I tested this on the following unity versions built games:

• unity 2018.4.12f1. new mono
• unity 2018.4.12f1, old mono
• 2019.4.24
• 5.5.4
• 4.6.6p3

So I feel confident that this supports a wide range of unity games.

About the settings

I would normally rethink what "debug_mode" really means, but I feel like for this pr, it's better to not worry too much about it and have the universality branch rethink how we approach that setting because I think it's a bit weird the setting was supposed to both start dnspy debugging AND have debug log channels when these 2 things should be separate, but considering a guide mentions to use it for debugging, I think it makes sense for alpha-dev for now and maybe we can recheck in universality.

Limitations with symbols

One last thing and it's about symbols handling. The mono version tells you what kind of symbols are supported. If it's old mono, it's basically just mdb and to get these, you have to generate FULL (not portable) pdbs and run the pdb2mdb utility (it also might be possible to do the conversion with cecil, but I am not sure). For early net45 monos, portable pdb works, but not embedded (it wasn't yet supported until a couple versions later). This means that you can't currently see any melonloader source info in stacktraces on these monos because the symbols won't be loaded by mono. I was thinking to address that in a separate pr because I also want to suggest other related symbols changes. Only more recent monos supports embedded symbols. Interestingly btw, it seems Rider supports full pdb symbols loading, but it's not something mono by itself can do so it's Rider doing it here.

Hopefully that covers everything!

[aldelaro5]

Forgot to mention, it closes #463

[slxdy]

Minor oversight, but the default value defined in the code is 55555. @HerpDerpinstine maybe do a quick commit fixing this?

[aldelaro5]

oh, whoops, yeah I only decided later to use 55555 and forgot to change it, here #865