Solve the “But it works on my machine!” problem with cloud-based development environments
This article is distilled from my recent talk at the WeAreDeveloper Conference in Berlin. If you wish to watch the full talk please click here.
In this article, I will show you how to set up a development environment that once configured can be instantly replicated by anyone and with that how you will almost never configure one ever again.
Furthermore, you will learn how to leverage this knowledge to create a seamless onboarding experience which is fast, fully secure and, most importantly, will work on any machine.
Why should you even care about a seamless onboarding process?
Research by the Brandon Hall Group shows that companies, with a great onboarding experience, see an increase in employee retention by a staggering 82%. I believe that this is a compelling enough reason to evaluate your approach and apply some of the things you will learn in this article.
The “Usual”
I’ll start by taking a look at the classic onboarding process I like to call The “Usual”.
Step 1: Clone the project. Nothing exciting there.
Step 2: Open the README file. Even more boring.
Step 3: Start executing the installation steps that will, hopefully, set up everything you need to get started.
Step 4: The script might fail for any number of reasons:
- For instance, your OS is not updated and doesn’t support the newest tooling that your project uses.
- Your language runtime is not in sync with the project. For example, you could have .NET 6 installed, but the project runs on version 5.
- Or simply, the script is just outdated. I know we all sometimes forget what we install locally while working on new features.
Step 5: You try to fix it:
- You finally click that update button that you’ve been snoozing for months.
- You manage to install the right language runtime.
- You realise somehow what’s missing from the script and add it yourself.
BUT IT STILL DOESN’T WORK!!
Step 6: A cry for help to your colleagues because you did everything in your power to try to make it work but just can’t.
Step 7: Your colleague tries to help you out but with no success and starts to share your frustrations. They’ve been held up long enough and have to get back to their work so they say the feared phrase: “But it works on my machine!” and leave you to fight the battle on your own.
All you can do now is flip your table, go back home and hope you have more luck tomorrow.
(╯°□°)╯︵ ┻━┻
A Few Days Pass…
and the setup somehow succeeds and you finally start coding.
You can see how this process can leave you frustrated and I felt bored even talking about it. But not only is it frustrating, you and your company are wasting resources because of it. You are not bringing value to the company while you're wasting time installing everything, and your colleagues who are helping you out are not focused on their work.
This is not a one time thing
Depending on your company, you might be switching projects on a weekly or monthly basis.
Having to experience this kind of onboarding so frequently really has an impact on your productivity and on your satisfaction in the company.
The “New”
Now let’s see what a great onboarding experience should look like in one I like to call The “New”.
Step 1: Click a button.
Step 2: Start coding.
You can see below that setting up an environment like this will take less time than just TALKING ABOUT The “Usual”.
As you will see, I click a button or a link in Github in this example.
The environment starts building.
It’s created and I’m instantly loading into the IDE.
The IDE loads, I’m directed to the main file of the project and can start coding RIGHT AWAY.
It should truly be as simple as that.
Let’s dive into how to make this setup possible.
Configuration
To understand how the configuration works, we must first understand the concept of infrastructure as code.
IaC is the managing and provisioning infrastructure resources through code rather than manually through a web interface for example.
The major benefit of this approach is that all your configuration can become a part of your source code and thus be version controlled.
Suddenly, you will always know how your infrastructure is defined and when and why it changed.
IaC is not new.
Using tools such as Vagrant developers can provision development environments inside virtual machines with all dependencies preinstalled.
Although this is a great step towards solving the "it works on my machine" problem, there are still drawbacks.
Creating and configuring VMs requires time and this is why developers tend to reuse the same VM for multiple tasks.
Not to mention that VMs are extremely resource intensive.
Container Technology
This is where container technology, like Docker, comes into play.
Containers are not VMs and since there is no emulation layer they run at native speed and all you need is a configuration file in your project.
This technology allows you to create consistent environments that are tailored to your project.
They can work on any machine and can be recreated as many times as you need.
Docker-compose
Environment Example
Let’s say I have a dev environment running locally.
I’m pretty satisfied because I set up the configuration so it works on everyone’s machine.
I’ve been using that environment for a couple of weeks.
I noticed a warning in the console saying I should update the mysql version.
I do so by just executing a few commands in a terminal.
I also add some data and modify a few tables.
Suddenly, my colleague can’t seem to start the project because of a database error.
They rebuild the environment but it still doesn’t work.
I realise that my environment had changed quite a bit since I started it for the first time but I didn’t notice that because I just kept reusing the same container every time I worked on the project.
Ephemeral Development Environments
To solve this, I’ll introduce the concept of ephemeral development environments. Ephemeral literally means something that lasts for a very short time.
They are quite common in Devops workflows. For example, when a separate environment is created for each PR that’s used to test your code.
Ephemeral environments are not just a fancy new concept.
They represent a new approach to development.
The idea of this new approach is to spawn a fresh environment whenever we start a contribution to a project, like opening a pull request.
After we finish with the contribution, we simply toss the environment and release its resources.
We can repeat this any number of times and the environment is guaranteed to work every time.
A Concrete Environment
Now that we know the theory, we’ll take a look at how to create a simple configuration for an environment with a language runtime like *rust*, *mysql* database and a *redis* cache store.
On the right you can see what the configuration file would look like for the Codeanywhere platform.
You’ll notice that it is very similar to the docker compose file I showed earlier.
That’s because it is. We wanted to create a schema that will be familiar to developers while giving us the freedom to expand it with some features like tasks.
As with docker-compose, this file will define everything our projects needs to be able to run on any machine.
And the most important part of the configuration are services.
Services are basically container configurations that have a name, image, resource limits, environment variables and optionally volume mounts.
This here is a simple service definition for the mysql database. I defined it to use 1 CPU core and 4 gigs of RAM. Additionally, I included an environment variable that sets the root password.
Similarly, you can define the rust and redis services.
With this simple file configuration, we’ve defined our whole development environment. This environment can now be spawned any number of times and it is guaranteed that it will be the same every time.
This configuration file is exactly what Infrastructure as Code is all about. The entirety of your development environment infrastructure, defined as code and fully source controlled.
I think this is just awesome!
Okay, so we’ve covered the configuration of the environments.
But… If you run this environment locally with all the resources in use, you’ll notice that your laptop might start burning up.
This environment will run you 4 CPU cores and 14 gigs of RAM. That’s quite a load which I know my laptop can’t handle. I have a running joke in the office that I don’t need a stove to cook my eggs in the morning. I can just run our dev environment and drop some eggs on my overheating laptop.
Companies might remedy this by giving employees a strict list of laptops they can buy with absurd specs and, of course, a steep price tag.
Well I think that’s not the answer.
Companies should want employees to be comfortable with their machines and actually give them a choice to work on anything. If they’re attached to their 2015 8GB of RAM Macbook Air, don’t take that away from them, but enable them to work without compromise.
This is where the cloud comes into play and where I’ll show you how to make these environments cloud-based.
Ephemeral Cloud-Based Development Environments
If we move our environments to the cloud, we can utilize the basically unlimited resources it offers while still keeping that local feel.
And to do that, you can use any tool that you are comfortable with, like Kubernetes or any Container as a service solutions.
These tools will tie our whole environment together and allow us to easily deploy it in the cloud. This ephemeral cloud-based development environment approach, relies heavily on tooling like this to make it fast, scalable and secure.
Scaling
But what about scaling?
After all, it’s inevitable that projects keep growing, and so do the resources needed to run them.
Soon enough, we can run out of RAM and start optimizing which apps to keep running while developing. If we wanted to upgrade our machine, there’s nothing we can do but toss our old one and buy an upgrade. This of course takes time and it’s definitely not environment friendly.
Laptops nowadays aren’t meant for easy upgrading.
On the other hand, let’s say that our database container needs 4 more gigs of RAM. We can just toss the environment. And get a new one running in seconds.
This is where the beauty of Infrastructure as Code comes into play.
All this takes is one simple value change in our configuration.
Just like that, we scaled our dev environment.
Security
Because these cloud-based environments will run on your cluster, you are in full control of its security. You control the inputs and outputs of all the data streams of the cluster.
Moreover, each of the environments are isolated from each other. That means that you can have multiple teams working inside the same cluster without worrying about code leakage.
Lastly, the most important security feature would be that all the code stays only contained on that cluster. There’s no laptop security management or wiping hard drives. If you want to revoke someone’s access, just block him from the network and the code stays safe.
How to code inside cloud-based environments?
You can connect with your local IDE like VSCode or any Jetbrains IDE like Intellij. This will keep you in your favorite editor while using cloud resources to run the project.
Or you can install a browser-based IDE in your environment and code in the browser.
Codeanywhere installs the Theia IDE into all the environments, so you can enjoy the VSCode experience in a fully customizable IDE that we tuned up with additional features like powerful real time collaboration, app previewing, and so on.
See it in action
To see the entire flow in action, head over here.
As you will see in the recording, to create an environment for this simple rust app, I just need to click a button in the README file. The environment is loaded near-instantly and I enter the IDE with our main file already opened.
To run the project, I do what I would do locally, that is open the terminal and enter the run command. You’ll notice that an error popped up saying that the app can’t connect to a database.
I remember that I forgot to include it in the configuration file.
So I just close my tab, return to Github, and add the environment variable for my database to the configuration. I commit and click on the exact same link as before.
As you can see, the app now runs successfully and not only that, all of my colleagues can now run the code in the exact same way, and most importantly, on any machine they want!!
But this doesn’t just fix the “But it works on my machine problem”.
In my opinion, this is another step in the evolution of software development.
The Next Step in the Software Development Evolution
Companies like Microsoft, with their Github Codespaces are moving to this next step.
While the platform engineering principles of compmanies such as Daytona enable developers to efficiently define and replicate their ideal working environments, reducing unnecessary dev commutes and providing them with the necessary tools and resources. Developers can focus on what they do best by seamlessly integrating an efficient developer commute into platform engineering.
In this next step, you will never install anything locally again!
But what’s even better, developers can just be developers and focus only on coding!
So just remember…
Step 1: Click a button.
Step 2: Start coding.
Thank you for reading,
Toma