Setting A Dockerized Python Atmosphere — The Arduous Manner | by Rami Krispin | Feb, 2024


Thank you for reading this post, don't forget to subscribe!

Rami Krispin

Towards Data Science

This put up will evaluate totally different strategies to run a dockerized Python atmosphere from the command line (CLI). Am I recommending you run your Python atmosphere from the CLI? Completely not!

There are higher methods to arrange a Python growth atmosphere, resembling utilizing VScode and the Dev Containers extension. We’ll use the “exhausting method” to arrange a Python atmosphere from the CLI for studying functions. We’ll evaluate totally different approaches to launching a container with the run command and see find out how to customise a built-in picture utilizing the Dockerfile.

A Pixal Python Snake (created by the writer with Midjourney)

To observe together with this tutorial, you will want the next:

  • Docker Desktop (or equal) if you’re utilizing a macOS or Home windows OS machine, or Docker put in if you’re utilizing a Linux OS
  • Docker Hub account to tug the picture from.

All through this tutorial, we are going to use the official Python picture — python:3.1o.

Let’s get began by pulling the official Python 3.10 picture from Docker Hub. First, let’s log in to Docker Hub by utilizing the docker logincommand:

docker login docker.io                                                                                                                                                                      okay
Authenticating with current credentials...
Login Succeeded

Subsequent, we are going to use the docker pull command from the terminal:

docker pull python:3.10                                                                                                                                                                     okay

If that is the primary time you pull the picture, you must anticipate the next output:

3.10: Pulling from library/python
66932e2b787d: Pull full
4afa7e263db1: Pull full
c812910e5e62: Pull full
f4e4299bb649: Pull full
5213cc2f9120: Pull full
4a3b5b2f0e66: Pull full
c214ceb1cabf: Pull full
f5336038b15c: Pull full
Digest: sha256:f94601bb6374b0b63835a70c9e5c3ba1b19bc009133900a9473229a406018e46
Standing: Downloaded newer picture for python:3.10
docker.io/library/python:3.10

You’ll be able to evaluate the picture particulars with using the docker photos command:

docker photos                                                                                                                                                                          okay  11s
REPOSITORY TAG IMAGE ID CREATED SIZE
python 3.10 f7537c504c9a 7 days in the past 1.01GB

Earlier than operating the container, let’s evaluate the picture metadata with the docker examine command:

docker examine python:3.10

This can return the under JSON output:

[

"Id": "sha256:f7537c504c9a91a22c9a255ee02048e7079cacdee583290e8238c605d17f9600",
"RepoTags": [
"python:3.10"
],
"RepoDigests": [
"python@sha256:f94601bb6374b0b63835a70c9e5c3ba1b19bc009133900a9473229a406018e46"
],
"Mother or father": "",
"Remark": "buildkit.dockerfile.v0",
"Created": "2024-02-03T10:49:13Z",
"Container": "",
"ContainerConfig":
"Hostname": "",
"Domainname": "",
"Person": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": null,
"Picture": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
,
"DockerVersion": "",
"Writer": "",
"Config":
"Hostname": "",
"Domainname": "",
"Person": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8",
"GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D",
"PYTHON_VERSION=3.10.13",
"PYTHON_PIP_VERSION=23.0.1",
"PYTHON_SETUPTOOLS_VERSION=65.5.1",
"PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py",
"PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9"
],
"Cmd": [
"python3"
],
"ArgsEscaped": true,
"Picture": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
,
"Structure": "arm64",
"Variant": "v8",
"Os": "linux",
"Dimension": 1005570383,
"VirtualSize": 1005570383,
"GraphDriver":
"Information":
"LowerDir": "/var/lib/docker/overlay2/d2fd76e7396796018a959209b51fe8311a188c8eae8e339e9e556de0889ca0bd/diff:/var/lib/docker/overlay2/bbedb25c5aa6ec3f2fc632e62a53989a329b907143fac165f899209293627a69/diff:/var/lib/docker/overlay2/ed6a4bf49214e6b496b7716443b8de380481cd9416bc4a378f29183c9129786f/diff:/var/lib/docker/overlay2/ac9543f44a835c203fb0b0b28958d94df72d206c9060c9d83307b39f50355102/diff:/var/lib/docker/overlay2/94a9f92c36ea6251feda52be8e76ec4da4a9c66b744a29472e1ccfdf34a6f69d/diff:/var/lib/docker/overlay2/6ee22c274256834a64008022856d365d91373bb490ae9f2f1723cb524b246a29/diff:/var/lib/docker/overlay2/2fa272376e0ce68f4f34f18e5ecb1ddd58a32fb20a82e5a417514047f8e684a3/diff",
"MergedDir": "/var/lib/docker/overlay2/f2d64d1affbe99afb05251435f7705cb97e2efa4f8febb494b4cbaa21e7f742a/merged",
"UpperDir": "/var/lib/docker/overlay2/f2d64d1affbe99afb05251435f7705cb97e2efa4f8febb494b4cbaa21e7f742a/diff",
"WorkDir": "/var/lib/docker/overlay2/f2d64d1affbe99afb05251435f7705cb97e2efa4f8febb494b4cbaa21e7f742a/work"
,
"Identify": "overlay2"
,
"RootFS":
"Sort": "layers",
"Layers": [
"sha256:9f8c60461a42fd9f275c56f4ec8fea8a8ea2d938493e316e830994a3814cf0aa",
"sha256:927a28cdbf6c1705342b2cba0069457313815058dcebe1996e46cade38a09370",
"sha256:e85139d96aee18e99fc964d02546be48cc6a4d4dfd9f465a91f172b7c535e55f",
"sha256:f3168ba6a8d2ec30e12002ad5b7b497cf7409f3e19cc8d8f447f6cf4231a2482",
"sha256:acbc6c8127209b09fa336e354037fdc630d3594e15174f2bc1bdbf31d5591376",
"sha256:06c4da96c7dd2fbbbb541e000bd0ea4cfbf7c80b24f098a9d67f677ef5e6c63e",
"sha256:1cdf208dc10679cf5db6d4f0e17ff6d5bfe754b4195ddd3b153b6d1ff51ce909",
"sha256:5d6f554f67c7da9d34763916bac632a450902c6e6fdbf9244f888f110fd37740"
]
,
"Metadata":
"LastTagTime": "0001-01-01T00:00:00Z"


]

The ispect command offers lots of helpful details about the picture, such because the layers info, the picture dimension, the {hardware} structure, and so on. As we need to run the picture, essentially the most attention-grabbing element is the CMD setting. The CMDcommand within the Dockerfile defines what command to execute in the course of the container launch time. We are able to parse from the above output the CMD info with the jq bundle:

docker examine python:3.10 | jq '.[] | .Config | .Cmd'                                                                                                                                      okay
[
"python3"
]

As you’ll be able to see, the picture is about to execute the python3 command in the course of the container run time. Let’s now launch the container utilizing the run command:

docker run python:3.10

And… nothing occurs. The explanation for that, briefly, is that we have to give Docker entry to the terminal. We’ll use the interactive and TTY arguments to run the picture in an interactive mode:

docker run --interactive --tty python:3.10

This can connect the terminal to the container and open Python contained in the container:

Working the Python picture in interactive mode (screenshot by the writer)

Whereas we have been capable of launch Python inside a container, it isn’t as helpful:

  • We can’t create, edit, and run scripts contained in the Python interpreter
  • By default, the Python picture comes with a restricted variety of libraries. On this mode, you can’t add further ones
  • Final however not least, the container is ephemeral. When you cease it, all of the work is misplaced

For instance, if we are going to attempt to load pandas, we are going to get the next error:

Attempting to load the Pandas library (screenshot by the writer)

Within the following sections, we are going to tackle these points by making a Dockerfile and customizing the bottom picture performance. This consists of including the next options:

  • Set a digital atmosphere and set up packages with a necessities file. For simplicity,
  • Set up a vim editor to edit information
  • Change the CMD command to open a shell terminal upon launch (versus the Python interpreter). This can allow us to create new scripts, edit, and execute from the terminal

To customise the Python atmosphere and make the above adjustments, we are going to create a Dockerfile with the next performance:

  • Import the Python picture — python:3.10
  • Set a digital atmosphere
  • Set up required libraries
  • Set up vim editor
  • Expose a bash terminal

Setting a Python Digital Atmosphere

To arrange a Python digital atmosphere, we are going to use the next two helper information:

necessities.txt

wheel==0.40.0
pandas==2.0.3

This file defines the checklist of Python libraries to put in within the digital atmosphere. For instance, on this case, we are going to set up the Pandas library, model 2.0.3. Usually, I additionally set up the wheels library, which handles C dependencies.

The subsequent helper file is the under bash script, which units the digital atmosphere and installs the required libraries from the necessities.txt file.

set_python_env.sh

#!/usr/bin/env bash

PYTHON_ENV=$1

python3 -m venv /decide/$PYTHON_ENV
&& export PATH=/decide/$PYTHON_ENV/bin:$PATH
&& echo "supply /decide/$PYTHON_ENV/bin/activate" >> ~/.bashrc

supply /decide/$PYTHON_ENV/bin/activate

pip3 set up -r ./necessities/necessities.txt

Notice: We use a variable (marked as $1) to outline the atmosphere title, which might be assigned to the PYTHON_ENV variable. Utilizing variables in the course of the construct is an efficient apply, because it allows us to switch among the picture traits with out modifying the code. We’ll assign the variable by way of the Dockerfile.

Let’s clarify the next concatenate code from the above bash script that units the digital atmosphere:

python3 -m venv /decide/$PYTHON_ENV  
&& export PATH=/decide/$PYTHON_ENV/bin:$PATH
&& echo "supply /decide/$PYTHON_ENV/bin/activate" >> ~/.bashrc

The above three strains of code embody three concatenate expressions:

  • First, the python3 -m venv /decide/$PYTHON_ENV set a digital atmosphere with the venv command
  • Second, add the digital atmosphere path to the PATH variable
  • Third, add to the .bashrc file the activate command of the atmosphere. This can be certain that at any time when we launch the terminal, it should activate this digital atmosphere by default (in any other case, you’ll have to do it manually upon the launch of the atmosphere)

As soon as the atmosphere is about, we use the supply command to activate the atmosphere, and the pip3 command to put in the libraries contained in the atmosphere.

Making a Dockerfile

After we evaluate the helper information, let’s have a look at how they’re included contained in the under Dockerfile.

Dockerfile

FROM python:3.10

ARG PYTHON_ENV=my_env
ENV PYTHON_ENV=$PYTHON_ENV

RUN mkdir necessities

COPY necessities.txt set_python_env.sh /necessities/

RUN bash ./necessities/set_python_env.sh $PYTHON_ENV

RUN apt-get replace &&
apt-get set up -y
vim
&& apt replace

CMD ["/bin/sh", "-c", "bash"]

As you’ll be able to see, we’re utilizing the identical picture — python:3.10as our base picture.

Subsequent, we set an argument named PYTHON_ENV with the ARG command to outline the digital atmosphere title. We setmy_env because the default worth, which may be modified in the course of the construct time utilizing the arg argument. We use the PYTHON_ENV argument to set an atmosphere variable as nicely.

Earlier than setting the digital atmosphere, we are going to create contained in the picture a brand new library beneath the foundation folder named necessities and use the COPY command to repeat the above helper information — necessities.txt and set_my_python.sh to the necessities folder.

Subsequent, we name the bash script — set_my_python.sh , which units the digital atmosphere and installs the required libraries. As talked about above, we use the PYTHON_ENV variable as an argument with the set_my_python.sh file to set the digital atmosphere title dynamically.

We use the apt command to put in vim — a CLI editor. This can allow us to edit code on by way of the container CLI.

Final however not least, use the CMD command to launch a shell terminal utilizing bash:

CMD ["/bin/sh", "-c", "bash"]

At this level, we have now the under information within the native folder:

.
├── Dockerfile
├── necessities.txt
└── set_python_env.sh

Let’s now go forward and construct the picture with the docker construct command:

docker construct .-f Dockerfile -t my_python_env:3.10
[+] Constructing 47.3s (10/10) FINISHED
=> [internal] load construct definition from Dockerfile 0.2s
=> => transferring dockerfile: 389B 0.0s
=> [internal] load .dockerignore 0.2s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.10 0.0s
=> [1/5] FROM docker.io/library/python:3.10 1.3s
=> [internal] load construct context 0.4s
=> => transferring context: 460B 0.0s
=> [2/5] RUN mkdir necessities 0.5s
=> [3/5] COPY necessities.txt set_python_env.sh /necessities/ 0.2s
=> [4/5] RUN bash ./necessities/set_python_env.sh my_env 26.8s
=> [5/5] RUN apt-get replace && apt-get set up -y vim && apt replace 17.4s
=> exporting to picture 0.7s
=> => exporting layers 0.6s
=> => writing picture sha256:391879baceea6154c191692d4bcb9ec9690de6dc4d5edd5b2ed13f6c579dd05c 0.0s
=> => naming to docker.io/library/my_python_env:3.10

Let’s run once more the docker photos command to evaluate the present photos:

docker photos                                                  
REPOSITORY TAG IMAGE ID CREATED SIZE
my_python_env 3.10 391879baceea 7 minutes in the past 1.23GB
python 3.10 f7537c504c9a 8 days in the past 1.01GB

As you’ll be able to notice, including the digital atmosphere and putting in the packages added about 250 Mb to the picture dimension.

Working the Python Atmosphere

After we constructed the picture, let’s launch the picture with the docker run command and examine if the above properties are outlined as anticipated:

docker run --interactive --tty my_python_env:3.10

This launches the picture in interactive mode, and opens a bash terminal as anticipated:

Launching the container by way of the shell terminal (screenshot by the writer)

As you’ll be able to discover within the above screenshot, it launched the container contained in the bash terminal, and the digital atmosphere is about as anticipated as my_env. The pandas library was put in and may be loaded, and we are able to now edit information from the terminal.

One problem to care for is that the container continues to be ephemeral. Subsequently, any code we create contained in the picture shouldn’t be exportable and might be misplaced after we cease the container from operating.

A easy answer is to mount a quantity with the quantity argument. For simplicity, we are going to go forward and mount the native folder, the place we preserve the Dockerfile and the helper information, to a brand new folder contained in the container named my_scripts:

docker run -v .:/my_scripts  --interactive --tty my_python_env:3.10

And right here is the output:

Mounting an area quantity to the container (screenshot by the writer)

As soon as the folder is mounted, any file that’s created, modified, or deleted from the mounted folder contained in the container might be mirrored to the native folder. This allows you to keep your code when stopping the container.

On this tutorial, we reviewed find out how to set a dockerized Python atmosphere utilizing the command line. Whereas that is neither a sensible nor really helpful strategy to develop with Python, it’s a nice studying expertise of Docker core instructions and primary functionalities. We present how we are able to simply take a built-in picture and customise it in accordance with our wants. Final however not least, we noticed find out how to mount an area folder to the container with the amount argument to switch the container from an ephemeral mode to a persistent mode.



Leave a Reply

Your email address will not be published. Required fields are marked *