Understanding Docker Connectivity, Networking and Configurations.

When working with Docker to build scalable applications, networking and service configuration are critical components. Over the course of our exploration, we’ve touched on a variety of topics, starting with creating a docker-compose.yaml template, understanding network modes like bridge and host, and configuring services like MySQL.

This article captures the complete journey to demystify Docker’s networking capabilities and service interconnectivity.

Creating a Docker Compose Template A docker-compose.yaml file is the backbone of a multi-container Docker application. It allows you to define and orchestrate services in one place. Here is an example of comprehensive docker template:

services:
  app:
    image: ${APP_IMAGE}
    container_name: ${APP_CONTAINER_NAME}
    restart: ${RESTART_POLICY}
    environment:
      - APP_ENV=${APP_ENV}
      - APP_PORT=${APP_PORT}
      - DATABASE_HOST=${DATABASE_HOST}
      - DATABASE_PORT=${DATABASE_PORT}
      - DATABASE_USER=${DATABASE_USER}
      - DATABASE_PASSWORD=${DATABASE_PASSWORD}
      - DATABASE_NAME=${DATABASE_NAME}
    ports:
      - "${HOST_PORT}:${APP_PORT}"
    volumes:
      - ${APP_VOLUME_SOURCE}:${APP_VOLUME_TARGET}
    networks:
      - ${NETWORK_NAME}
  db:
    image: ${DB_IMAGE}
    container_name: ${DB_CONTAINER_NAME}
    restart: ${RESTART_POLICY}
    environment:
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}
    ports:
      - "${DB_HOST_PORT}:${DB_PORT}"
    volumes:
      - ${DB_VOLUME_SOURCE}:${DB_VOLUME_TARGET}
    networks:
      - ${NETWORK_NAME}
  redis:
    image: ${REDIS_IMAGE}
    container_name: ${REDIS_CONTAINER_NAME}
    restart: ${RESTART_POLICY}
    ports:
      - "${REDIS_HOST_PORT}:${REDIS_PORT}"
    volumes:
      - ${REDIS_VOLUME_SOURCE}:${REDIS_VOLUME_TARGET}
    networks:
      - ${NETWORK_NAME}
volumes:
  ${APP_VOLUME_SOURCE}:
  ${DB_VOLUME_SOURCE}:
  ${REDIS_VOLUME_SOURCE}:
networks:
  ${NETWORK_NAME}:
    driver: ${NETWORK_DRIVER}
Code language: JavaScript (javascript)

This template is designed to be modular, using environment variables for maximum configurability. Define these variables in a .env file.

Understanding Docker Network Modes Docker provides different network modes to connect containers and external resources.

Two commonly used modes are bridge and host.

Bridge Network

  • Default Network: Containers get isolated private IPs.
  • Use Case: Containers communicate with each other via their private IPs, and the host can access them through exposed ports.

Host Network

  • Shared Network: Containers share the host’s network stack.
  • Use Case: High-performance networking or when direct access to the host’s network is required.

Scenario: Mixing Network Modes You can mix network modes for services. For example:

  • App Service: Uses bridge mode for port mapping.
  • Database and Redis Services: Use host mode for direct access to resources on the host or other external hosts.

Example:

services:
  app:
    image: my-app-image
    container_name: my-app
    network_mode: "bridge"
    ports:
      - "8080:80"
    environment:
      - DATABASE_HOST=192.168.11.21
      - DATABASE_PORT=3306
      - REDIS_HOST=192.168.11.22
      - REDIS_PORT=6379
Code language: JavaScript (javascript)

Why Doesn’t localhost Work Inside a Container? Inside a Docker container, localhost refers to the container itself. If MySQL is running on the Docker host or another external machine, using localhost won’t work.

Instead, you can:

  1. Use Host IP Address: mysql -h 192.168.11.11 -P 3306 -u username -p
  2. Use Docker Gateway IP (172.17.0.1): mysql -h 172.17.0.1 -P 3306 -u username -p
  3. Use host.docker.internal (Modern Docker): mysql -h host.docker.internal -P 3306 -u username -p

Configuring MySQL for Remote Access To allow connections to MySQL from a container:

  1. Update MySQL Configuration: bind-address = 0.0.0.0
  2. Grant Access: GRANT ALL PRIVILEGES ON *.* TO 'username'@'%' IDENTIFIED BY 'password'; FLUSH PRIVILEGES;
  3. Check Firewall Rules: sudo ufw allow from <docker-host-ip> to any port 3306

Conclusion, Docker networking can seem complex at first, but understanding modes like bridge and host, and configuring services like MySQL correctly, can help you achieve seamless connectivity.

Use this guide to troubleshoot and optimize your Docker setups, ensuring containers can interact with hosts and external resources effortlessly.

Let us know if you have additional tips or challenges in the comments below!

Where Docker Stores Images on Debian 12 and How to Customize the Storage Directory

Continuing from our previous article on setting up Docker and running Docker Compose on Debian 12 Bookworm, let’s dive deeper into understanding where Docker stores its image files and how to customize the storage directory for better management.

Default Docker Image Storage Location on Debian 12

By default, Docker stores all its data, including images, containers, volumes, and other artifacts, under the following path:

/var/lib/docker
Code language: JavaScript (javascript)

This directory contains subdirectories managed by the storage driver Docker uses. For instance, if the overlay2 storage driver is used (the default for most Linux distributions, including Debian 12), images and their layers will be stored in:

/var/lib/docker/overlay2
Code language: JavaScript (javascript)

You can verify the current Docker root directory by running:

docker info | grep "Docker Root Dir"
Code language: JavaScript (javascript)

This command outputs the current path where Docker stores its data:

Docker Root Dir: /var/lib/docker
Code language: JavaScript (javascript)

While the default location works for most setups, there are scenarios where customizing the storage path is beneficial, such as:

  • Low disk space on the default partition.
  • Organizing Docker data on a dedicated storage drive or partition.
  • Simplifying backups and maintenance.

Customizing Docker’s Storage Directory

To customize where Docker stores its images and other data, follow these steps:

Step 1, Create a New Directory for Docker Data

Choose a new location for Docker’s data, such as /mnt/docker-data. Create the directory and ensure appropriate permissions:

sudo mkdir -p /mnt/docker-data
sudo chown -R root:root /mnt/docker-data
sudo chmod -R 700 /mnt/docker-data
Step 2, Update Docker Configuration

Docker’s storage directory can be updated by modifying its daemon configuration file:

  1. Open the Docker daemon configuration file: sudo nano /etc/docker/daemon.json
  2. Add or update the data-root field to point to the new directory: { "data-root": "/mnt/docker-data" }

If the file doesn’t exist, create it and add the above content.

Step 3, Stop Docker and Move Existing Data

Before restarting Docker, stop the service and move existing data to the new directory:

sudo systemctl stop docker
sudo rsync -a /var/lib/docker/ /mnt/docker-data/
Code language: JavaScript (javascript)

This command ensures that all existing data is safely copied to the new location. Once completed, rename or backup the old directory:

sudo mv /var/lib/docker /var/lib/docker.bak
Code language: JavaScript (javascript)
Step 4, Restart Docker

Restart the Docker service to apply the changes:

sudo systemctl start docker

Verify that Docker is now using the new storage directory:

docker info | grep "Docker Root Dir"
Code language: JavaScript (javascript)

The output should now display the updated path, e.g.:

Docker Root Dir: /mnt/docker-data
Code language: JavaScript (javascript)
Step 5, Test the Setup

Pull an image and verify it is stored in the new directory:

docker pull hello-world
ls /mnt/docker-data

You should see the newly pulled image data in the customized path.

Reverting to Default Settings

If you encounter issues and need to revert to the default storage location, simply:

  1. Stop Docker: sudo systemctl stop docker
  2. Restore the original directory: sudo mv /var/lib/docker.bak /var/lib/docker
  3. Remove the data-root entry from /etc/docker/daemon.json or set it back to /var/lib/docker.
  4. Restart Docker: sudo systemctl start docker

Final Words …

Customizing Docker’s storage directory on Debian 12 can help optimize disk usage and improve data management.

By following the steps above, you can easily configure Docker to use a different location for storing images and containers.

As always, test changes thoroughly in your environment and ensure backups are in place before making significant modifications.

How to setup Docker and run Docker Compose for Managing multi-container on Debian 12 Bookworm

Part 1, Install Docker and Docker Compose

Here’s a step-by-step guide to install Docker and Docker Compose on Debian 12 Bookworm

Step 1: Update System Packages

Ensure your system packages are up to date before starting the installation.

sudo apt update && sudo apt upgrade -y

Step 2: Install Required Dependencies

Install packages that allow apt to use HTTPS and required tools for adding Docker’s official repository.

sudo apt install apt-transport-https ca-certificates curl software-properties-common gnupg lsb-release -y

Step 3: Add Docker’s Official GPG Key

You need to add Docker’s GPG key to verify the packages.

curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Step 4: Set Up Docker Repository

Add Docker’s official repository to your system.

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Step 5: Install Docker Engine

Update the package database and install Docker.

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io -y

Step 6: Verify Docker Installation

Check if Docker is installed and running.

sudo systemctl status docker

(Optional) Add your user to the docker group to avoid needing sudo for Docker commands:

sudo usermod -aG docker $USERCode language: PHP (php)

You’ll need to log out and log back in for this change to take effect, or apply

newgrp docker

Step 7: Install Docker Compose

Now install Docker Compose, which is a tool to define and run multi-container Docker applications.

sudo curl -L "https://github.com/docker/compose/releases/download/v2.20.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Make the binary executable:

sudo chmod +x /usr/local/bin/docker-compose

Verify the installation:

docker-compose --version

Step 8: Enable Docker to Start on Boot

Enable the Docker service to automatically start on system boot.

sudo systemctl enable docker

Step 9: Test Docker and Docker Compose

You can run a simple Docker container to verify everything is set up properly:

docker run hello-world

For Docker Compose, create a simple YAML file (docker-compose.yml) and run it:

version: '3'
services:
  hello-world:
    image: hello-worldCode language: JavaScript (javascript)

Run the following command:

docker-compose up

Part 1, Summary

Now, Docker and Docker Compose are installed on your Debian 12 system, ready to manage containerized applications!

This guide will fit well into your blog post as a detailed tutorial for preparing Debian 12 to run Docker containers.

Part 2, Running Containers with Docker

Running Docker containers is straightforward once Docker and Docker Compose are installed. Here’s a basic guide on how to run containers.

Step 1: Pull a Docker Image

Docker images are pre-configured applications that you can run in containers. You can search for and pull images from Docker Hub.

For example, to pull an official Nginx image, run:

docker pull nginx

Step 2: Run a Docker Container

Once the image is pulled, you can start a container. For example, to run an Nginx container:

docker run --name my-nginx -p 80:80 -d nginxCode language: CSS (css)
  • --name my-nginx: Assigns a name to the container.
  • -p 80:80: Maps port 80 on the host to port 80 on the container.
  • -d: Runs the container in detached mode (in the background).
  • nginx: The name of the image.

You can now visit http://localhost in your browser, and you should see the Nginx welcome page.

Step 3: Check Running Containers

To see all running containers:

docker ps

To see all containers, including stopped ones:

docker ps -a

Step 4: Stop and Remove Containers

To stop a running container:

docker stop my-nginx

To remove a stopped container:

docker rm my-nginx

You can also stop and remove containers in one command:

docker stop my-nginx && docker rm my-nginx

Running Containers with Docker Compose

Docker Compose is useful for managing multi-container environments with a single configuration file.

Step 1: Create a docker-compose.yml File

Here’s an example for running WordPress and MySQL:

version: '3'
services:
  db:
    image: mysql:5.7
    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: example

  wordpress:
    image: wordpress:latest
    ports:
      - "8000:80"
    restart: always
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_PASSWORD: example
    depends_on:
      - db

volumes:
  db_data:Code language: JavaScript (javascript)

Step 2: Start the Containers

Navigate to the directory containing your docker-compose.yml file and run:

docker-compose up -d

This command will pull the required images, create containers, and start them in detached mode.

Step 3: Check Running Containers with Docker Compose

To see running containers managed by Docker Compose:

docker-compose ps

Step 4: Stop and Remove Containers

To stop the running containers:

docker-compose down

This command stops and removes all containers defined in the docker-compose.yml file.

Step 5: View Logs

To view logs of your services:

docker-compose logs

You can specify a service to view only its logs, for example:

docker-compose logs wordpress

Part 2, Summary

With Docker, you can easily run single containers, while Docker Compose allows for managing multi-container applications. This should give you the foundation to start running containers on Debian 12 and add more advanced setups to your blog post later.