Building Web Services from Scratch: Part 1 - Intro
What and why on earth would this be useful or a good idea?
I have this terrible idea to try to build a selection of web services from scratch. I have a few reasons for this:
- So much of modern web development is build on top of abstractions that we don't need to fully understand in order to use them. I want to look under the hood.
- I want to understand how the these services work at a lower level and from first principles.
- I think it will be a fun challenge (I will more than likely regret this).
- I want to improve my Rust skills.
- I throughly enjoy banging my head against a wall.
I plan to document my progress with this mini series of posts. Feel free to follow along and laugh at my mistakes or learn from them. The code is available on GitHub.
What services am I going to build?
- DNS server, client, and resolver with support for A records and possibly MX records: here
- HTTP API: here
- Load balancer with support for round-robin and path based routing: here
- Persistent row oriented data store: here
- Key-value in-memory cache store: here
- Message queue: here
- Microserivce architecture with path based routing: here
- Object storage with support for GET and PUT: here
- Conclusion: here
Rules of engagement
- I want to try to keep this as simple as possible.
- I plan to use Rust or Typescript for almost everything (there may be some bash scripting and Docker involved).
- I will try to avoid using any libraries that do the heavy lifting for me however I want to focus on the core functionality of the services so I may cheat a little.
- I won't be explaining much theory, these services are all pretty self-explanatory and there are plenty of resources online that explain how they work. I will focus on the implementation.
Right, let's get started. I am going to start with the DNS set up. I have pretty much no idea how DNS actually works (something about port 53) so this should be fun...
Wait
This is an edit. This gets out of hand real quick without some sort of container management. So I am going to use Docker and docker-compose.
The Dockerfiles for each binary are very simple as I don't really care about container size or building the binary for production. I just want to be able to run the services in a container. There is also a .dockerignore file to ignore the target directory when building the image. Finally, there is also a docker-compose.yml file in the root of the project that will build and run all the services.
Simply run docker-compose up to build and run all the services. I make this clear in each subsequent post but I thought I would mention it here too.
Okay, now let's get started.