Your Model Isn't Done Until Someone Else Can Call It

To give you context, I got curious about what goes into building and deploying machine learning models. So, rather than learn theory. As always, I decided to learn by building, because I still believe this is the best way to learn. The first project I chose to tackle was a churn predictor api.It's a small FastAPI app I built to predict whether a customer is about to churn. How it works is that, you feed it a customer's data, it hands back a probability, a prediction, and a risk level. In my previous article, I trained the model, wrapped it in an endpoint, and tested it in Swagger UI on my own laptop. It worked and every request came back with a real answer. You can read the article here. At this point, I thought I was done with it.Not even close. It turns out that the model I built isn’t useful in any way. It's just another toy only I can play around with. It couldn’t survive my laptop going to sleep, let alone being reachable by an actual stranger typing in a URL.So I’m going to be addressing this issue in this article. I'll be going through the unglamorous work of making it reachable. So, I’ll be walking you through how I containerized it, and moved it to a real server. By the way, I also ran into three separate failures I didn't see coming, which I’ll also point out in this article.Quick recap of where Part A left offThis project implements a FastAPI app churn-api. It provides a simple /predict endpoint that uses a pre-trained scikit-learn pipeline consisting of a scaler and a classifier. This is used to predict the likelihood of a given customer churning. If you are new to the project, I recommend starting with Part A, as this article continues from there.Why DockerSo here is the problem. Sure the project works perfectly on my computer, the issue is that it has a specific python version, specific package versions, specific file paths which someone else's computer on which it would be deployed on or an AWS server might not have.Docker solves this. Here is how: Instead of delivering just code which hopefully works after the machine is set correctly (which is an issue for another day), the deliverable now is a self-sufficient image that runs on any computer and on whose environment (python version, various libraries required, folder structure) no longer has an effect. This is because all of this ships with the image itself so now nothing is needed to be set by the receiver.Writing the Dockerfile and what brokeBefore I even started writing the Dockerfile, I wanted to make sure I knew exactly what was installed in my local environment. The goal was to use the same package versions inside the container, rather than letting pip install whatever happened to be the latest version at the time.At first, I tried using conda list, but it wasn't giving me the clean list of package versions I needed. So I switched to pip freeze, which gave me the exact versions of the packages I was actually working with:And there was one more detail that turned out to matter: I was running Python 3.14.6 locally, which is pretty new. So I couldn't just reach for the usual python:3.11-slim base image out of habit. If I wanted the container to match my local environment as closely as possible, the Dockerfile needed to use Python 3.14 as well.The build itself succeeded and I got no errors. Although it took about 166 seconds, likely because some packages didn't have prebuilt wheels yet for this new a Python version and had to compile from source.Then I ran it, and got this:This was one of those problems I only realised existed after I'd begun running the container. During the build nothing had suggested that such an error would arise.When I had run the app locally, I'd always found myself in the app/ folder, so running uvicorn main:app --reload wasn't ever a problem: schemas.py was nearby, next to the main.py , easily found by Python.But the Dockerfile was subtly different. There my CMD was running uvicorn app.main:app in /code which led python to believe that the app was a package and that it should be being treated as such. In other words, it had a different working directory and that was enough to break it.The fix was simple. Instead of altering to the project structure, much simpler to just tell Uvicorn to use and load the application using the app folder as the root. This would mimic how I'd previously used it before.After running this, I rebuilt the image and span the container, and this time, I got no issues.Armed with this; just as in Part A, I then sent off a dummy request to my FastAPI for the sole purpose of checking against my Part A numbers. This test came back with exactly what I'd hoped: same churn probability probability, same prediction, same risk level.I therefore knew my container hadn't changed or altered how the application behaved. It ran the same way as it ran on my machine. So first checkpoint: done.Choosing AWS and standing up EC2​I picked EC2 again, just as with the rss-pipeline deployment. This is useful in maintaining consistency in my ideas about the architecture of projects; this project is a t3.micro instance using Ubuntu; this instance type is eligible for the free tier and is ideal for deploying a FastAPI application running on a single model.​The setup process involved mostly navigating the options on the AWS console. I launched an instance and configured it with Ubuntu; I chose the t3.micro instance type and generated a key pair for the SSH connection. The configuration of the security group should be done more carefully: I chose two inbound rules. The first applied for SSH on port 22 and worked for traffic from my IP, while the second involved a custom TCP rule on port 8000. It applied to everyone, as API access was desired to be permitted on this port.​One last thing: I followed the habit of using an IAM user, not the root AWS account, when I wrote this. When an operation is carried out from the root AWS account, access to this account is not constrained by any boundaries. Running work like setting up an additional instance makes use of this account. However, this is of greater risk because even the slightest chance of an unanticipated situation will cause a large amount of destruction. Using the IAM user with permissions will perform as it would when the root account performs the job and does so with a lower radius if anything happens, as it should not.​Getting Docker onto EC2 and shipping the imageAfter SSHing into the instance, installed Docker on the server:I decided to rebuild the Docker image on the instance directly with the Dockerfile, than by adding it to Amazon ECR. With a small-scale project like this, copying over local test data seemed easier- which saved me some AWS-learnings.I then used scp to get the project files onto the instance, I then ran into a wall:My first thought, inevitably, is that I've done something wrong on the instance. However, this is, in fact, not what has gone wrong. Instead, it was my security group firewall rule that is to blame, which had restricted SSH access to my own IP address. As a result, when I next tried SCP, my IP had since moved. After updating, I was in again and the files successfully transferred.A small point maybe, but a common hiccup worth pointing out.Once that was sorted, the copy went through, and building the image on the instance itself was the same command as local:It ran much slower than how it normally ran on my laptop, and that's because t3.micro has less CPU to throw at compiling packages. But the build ran successfully and that's all that matters to me.Running the container in productionThe -d here matters here. What it does is that it keeps the container running in the background and detached, so it keeps running even after I close the SSH session. Without it, the container dies the moment I disconnect, which defeats the entire point.Then, from my own laptop, not the server, I opened a browser and went to:http://:8000/docsThe same Swagger UI loaded. I then sent the same test customer through /predict one more time, and got back the same values I'd gotten locally and in the local container. Now, someone else, anywhere, with nothing but that IP address, could now call my model and get a real answer.A stable address: the Elastic IPI wanted to ignore this phase initially, but I felt like it's worth keeping in to make this project production ready. You see, EC2's default public IP isn't guaranteed to stay the same, especially if the instance stops and starts. Attaching an elastic ip fixes this by ensuring the IP address stays the same and doesn't change whenever I reload my server. It didn't take long to setup. What's still fragile​This works, but it's not production-grade and I'd prefer to be candid about the remaining issues than gloss over them:There's no HTTPS. Currently, everything communicates via plain HTTP on port 8000, which is perfectly acceptable for a learning project, but unsustainable when dealing with actual customer data.The endpoint lacks authentication. Anyone with the IP can invoke /predict. No API key, no rate limiting, nothing to deter misuse exists.If the instance reboots, the container won't automatically restart. If AWS restarts the underlying hardware, my API will be silently deactivated until I notice and restart it manually.​These issues aren't difficult to address, however, they haven't been resolved yet and I'd like to point them out than have the article create a misleading impression of completion.What's next​The model has evolved from a notebook, then to a local API, later to a containerized API and finally to a state where it is accessible by anyone on the internet. This progression encapsulates the primary objective of this series and stands out as the most significant differentiator from conventional code writing; in fact, the majority of issues encountered weren't due to logical errors within my code but resulted from environmental mismatches or subtle alterations in infrastructure resources.​The subsequent stage on the journey is the construction and deployment of a fully functional ML application to the cloud. This will entail deploying the same insights gained about working with different environments and changing infrastructure and applying them to a model larger than a simple one-endpoint service.​