This all started with a "simple" premise, can you use the .net runtime as a just-in-time compiler for any language. 2 months later and now I have a fully working code base that can compile most 6502 functions into MSIL and execute them on demand.
It achieves this by instantiating a memory bus with any memory mapped I/O devices you may have the need for, and which memory regions they map to. For the NES, this includes the CPU ram (and its mirrored regions), the PPU device, the cartridge space, etc...
Then the JIT compiler is told to run the function at a specific address. The JIT compiler then:
1. Traces out the function boundaries of the function using the passed in address as the entry point.
2. After all instructions and their ordering is determined, the instructions are disassembled.
3. The 6502 disassembled instructions are converted to one or more intermediary representation instructions
4. A JitCustomization process is run that allows different emulators/hardware setups to modify how the IR instructions are set up. This also allows for analysis and optimization passes.
5. The final set of IR instructions are passed one by one into a MSIL generation class, and the MSIL is written to the `ILGenerator`
6. This IL is then added into an assembly builder and compiled on the fly, providing a static .net method containing that function's code.
7. The JIT compiler then turns that function into an Executable method delegate and executes it
8. The function runs until a cancellation token gets a cancellation signal, or the function hits a return statement with a new address of a function to call. The JIT compiler then repeats this process, but now with the function at the address returned.
This allows the above video, where NES games are running inside the .net runtime via MSIL. Since it is just-in-time compilation, in theory even arbitrary code execution exploits would be executable. The main bugs visible in SMB are due to my inaccurate PPU emulation and not about the 6502 code itself.
# Why An Intermediary Representation?
Creating MSIL at runtime is pretty error prone and is hard to debug. If you have one simple mistake (such as passing a byte into a `ldc_i4` emit call) you get a generic "This is not a valid program" exception with no debugging. So limiting how much MSIL I had to generate ended up pretty beneficial.
One significant benefit though is simplicity. The 6502 has 56 official instructions, each with some significant side effects. Creating MSIL for each of these with all the different memory addressing modes they could contain would spiral out.
However, it turns out you can create any 6502 instruction by composing about 12 smaller instructions together. This made it much simpler to write the MSIL for each IR instruction, and much easier test coverage to ensure they actually compile and work.
# Assembly Creation
There are code paths (disabled) that can actually create real dll files for each function generated. In theory this means that if you run an application for a sufficient amount of time, you could collect all the dlls and piece them together for a MSIL precompiled build.
# NES Limitations
The NES emulator side isn't complete. It can run games as long as they are up to 32k ROMs with 16K character data. This is just because I didn't bother adding support for all the different bank/memory switchers that cartridges implement.
# What's Next?
Not sure. I'm tempted to add some other 6502 emulations. Atari 2600 would work but may not be interesting. Using this to fully JIT the commodore 64 is something that is interesting, though I'm not totally sure how much of a rabbit hole emulating the video and other I/O devices would be.