Setting up an OS development environment

You want to start working on your own operating system, but don't know where to start? This article describes the tools and settings you need to configure to prepare your system for operating system development.

A long time ago, writing an operating system was done by writing assembly code *on paper*, going through the CPU manual and translating every assembly instruction into binary by hand, and then punching tape using a special typewriter (citation needed). Luckily for us, things have gotten a lot better and we have much better tools available. We have smart editors to help us write code easier, advanced compilers and assemblers, and really good debugging tools.

can use our computers running pretty much any general purpose operating system.

However, not all existing OSs are well suited for this task; some are too restrictive, and the available tools leave a lot to be desired.

For the best experience, the general recommendation is to use a Unix-like system such as Linux, MacOS or a BSD variant.

Windows is the most popular operating system for the desktop. There are multiple ways of setting it up for OS development:

• using WSL to create a Linux environment. Most OS developers choose this path, and is the one used in the tutorials on this site.
• using a port of the Linux environment, such as Cygwin or MSYS2. The main disadvantages are the performance of the tools (like the compiler) which means longer compile times, and the difficulty of creating disk images on Windows
• using native Windows and Microsoft tools.

WSL is really easy to set up. All you have to do is open an elevated command prompt (i.e. run as administrator), and run:

wsl --install

This will install all the prerequisites and will set up an Ubuntu distribution. After installation, you only need to search for “Ubuntu” in the start menu to open the WSL instance. Many tools, such as Visual Studio Code and Windows Terminal, will automatically detect any installed distribution and make it very easy to interact with.

For more information, as well as instructions on how to install different distributions, check the official documentation.

Caveat: There are 2 versions of WSL. WSL 1 is built as a translation layer that translate linux system calls into Windows system calls. This has some limitations, the most notable is the lack of support for mounting devices. There are other limitations too that you will encounter when running certain Linux programs. WSL 2 works by virtualizing the Linux kernel and providing a “transparency” layer that has features like the integrated file system, a built-in X server for running GUI applications and audio pass-through. If you encounter problems running certain commands, you might be using WSL 1, in which case you will have to migrate your distribution to WSL 2.

If you are using Linux, you are already set up, all you need is to install the tools in the chapters below.

While MacOS is a Unix-like operating system, it doesn't come with all tools out of the box, so you will need to install homebrew.

There are some small differences between Linux and MacOS, such as the commands used setting up loopback devices, or the command used to format a disk.

• BSD: you are already set up, you just need to install the tools below.
• Android: even though Android runs Linux under the hood, it is heavily locked down, making it a very poor choice for OS development. While most required tools can be found in Termux, there are many limitations such as not being able to mount disk images unless the device is rooted. Another issue is that most Android devices are built on the ARM architecture. If you're working on an x86 operating system, you will need to emulate x86 which is going to be very slow.
• ChromeOS: similar issues to Android. The best option is to create a virtual machine running a Linux distribution, and doing the development in that virtual machine.
• iOS: the operating system is too locked down to allow any kind of software development on the device.