Fun with python #2

In the previous episode I have concentrated on building a simple python application. That was nice and refreshing. Then I thought why not up the ante and move all of the code to an API running in docker. So I did.

An application programming interface (API) is a computing interface that defines interactions between multiple software or mixed hardware-software intermediaries. It defines the kinds of calls or requests that can be made, how to make them, the data formats that should be used, the conventions to follow, etc. It can also provide extension mechanisms so that users can extend existing functionality in various ways and to varying degrees.

Docker provides a local virtualization service. In other words if you’d like to run a software-based virtual computer instance with a straightforward config, rather than play with another server for your tests, you can use docker. They are also a good way to encapsulating operating system requirements into a container. In other words, once you built a docker image, you can technically be sure your application will work anywhere it will eventually get deployed.

Both provide a good way of separating your own system and modules from other people’s systems and modules. They also make local testing of your new modules much easier. It is worth noting that they also introduce a lot of hidden complexity. No engineer should forget about it, but we can back to it some other time.

The experiment repo is here.

---

For the API code I have written, I am using a python module called FastAPI. Lets see what are some of the comments from people who use it.

[…] I’m using FastAPI a ton these days. […] I’m actually planning to use it for all of my team’s ML services at Microsoft. Some of them are getting integrated into the core Windows product and some Office products.

Kabir Khan – Microsoft (ref) (source: https://github.com/tiangolo/fastapi)

There are much more references for this API engine in the original GitHub repository (including ones from Uber and Netflix). My methods of the API are listed in this module. This is how one API method definition looks like (pretty neat):

@router.post(
    "/lesson",
    status_code=status.HTTP_200_OK,
    response_model=LessonResponseText,
    responses={
        status.HTTP_200_OK: {
            "model": LessonResponseText,
            "description": "return the text to the caller",
        }
    },
    response_class=JSONResponse,
)
async def return_lesson(lesson_number: LessonRequest):
    """returns lessons"""
    return {"response_text": select_lesson(lesson_number.lesson_number)}

For my docker build, I am using python:slim image from dockerhub. A docker image is nothing else but a snapshot of a working operating system image with all python 3.8 tools installed. You can browse for more here. The main logic on what needs to be built is in Dockerfile and docker-compose.yml. I am running my image as a deamon process. In short, it runs in background, allowing me to query the API from my ssh session. This is the command used to start it:

docker-compose -f ./docker-config/docker-compose.yml up -d --build --force python-playground

---

I still can see quite a few things that feel dirty in the codebase. Especially the below method selector on the back of the api request.

from src.service.lessons.lesson_one import run_lesson_one

lessons_dict = {
    1: run_lesson_one,
}


def select_lesson(lesson_number: int):
    return lessons_dict.get(lesson_number, default)()


# If user enters invalid option then this method will be called
def default():
    return "ERROR: This lesson does not exist yet."

I am pretty sure this can be done much better and will try to optimize this for the next time.

I should also probably write some integration and proper smoke tests. I have not added pytest for nothing.

One other learning I thought was worth sharing is the way to wait for a service to come up before running the current smoke tests. cURL is a very powerful tool.

timeout 300 bash -c 'while [[ "$(curl -s -o /dev/null -w ''%{http_code}'' localhost:8000/health)" != "200" ]]; do sleep 5; done' || false

---

This is all I had to share for today. Thank you and see you next week.