This readme file explains how you can build and publish the artifacts on our nebulastream docker hub repository. Currently, we have three images in our repository: a build image used by GitHub-CI, a dev-image for using with IDE, and an executable-image used for running a standalone-cluster within the docker image.

(Note: Please make sure you have the right to publish images in the DockerHub nebulastream organization.)

We have three docker files in our source repository, one for each docker image, listing build instructions for preparing docker images. In case you want to add or modify any dependencies in the docker image please edit the corresponding docker file.

This is the docker image for our CI builds. There is no way to connect to a running container of this image, aside from docker attach. The build-image can be found at NES-Build-Image.

The ENTRYPOINT is located at entrypoint-nes-build.sh. The entrypoint script checks if the source code is mounted correctly within the docker container and executes a set of CMake commands for building and testing the source code.

This image is based on NES-Build-image. Additionally, we have added a set of connectivity and interactivity tools to this image. It has everything installed that we need for development, including ssh connectivity. Its purpose is to start in the background while our IDEs connect to it for remote debugging purposes. It can be found at NES-Dev-Image.

Currently, there is no need for an ENTRYPOINT for this image, tt stars sshd and we can be keep it in the background. For more info, check Docker's official docs here.

The Executable image is based on the Build image. There is no way to connect to a running container of this image, aside from docker attach. Its purpose is to start a standalone-cluster of NebulaStream (NES) with pre-configured parameters. Each time we update the master branch of our source code, we install the latest version of NES using a deb package inside the docker image and a new version is released. All versions of the executable-image can be found at NES-Executable-Image.

The ENTRYPOINT is located in entrypoint-nes-executable.sh.

If you want to change the startup behavior of the docker images, please change the corresponding entrypoint script.

First, you need to install docker. You further find an introduction to the docker concepts and useful tutorials on the linked webpage.

In general, it does not matter from which directory you are entering the following commands. Depending on how your docker is set up, you might need to run docker commands with sudo

(1) This command pulls the latest nes-executable-image from the registry.

docker pull nebulastream/nes-executable-image:latest

Note: You need to delete this image before the next pull, else it is not updated: sudo docker rmi nebulastream/nes-executable-image:latest

(2) This command runs the nebulastream image downloaded from the registry

docker run -d nebulastream/nes-executable-image:latest

or

this additionally mounts the Nebulastream codebase into the container, accessible via /nebulastream, in the Development image:

docker run -p 2222:22 -d –rm -v <path_to_project>/:/nebulastream nebulastream/nes-dev-image

Where <path_to_project> is the root directory of the nebulastream repository downloaded to the host machine. This runs a docker container in the background, with all dependencies installed. To connect, use ssh with username and pass: “nes:password” on localhost, port 2222.

(3) This command shows the running containers with their images and names.

docker ps

(4) This command shows and updates the logs of the running container <container_name>.

docker logs -f <container_name>

(5) This command will get you into the terminal of the container, the rest is set for work.

docker exec -it <container_name> /bin/bash

(6) This command stops the container <container_name>.

docker stop <container_name>

(7) This command removes the container <container_name>.

docker rm -f <container_name>

Please execute the following command for building the build image locally:

docker build . -f /path/to/Dockerfile-NES-XXX -t nebulastream/nes-XXX-image:YYY. Where XXX represents one image type out of:

• devel
• build
• executable

and YYY as one tag out of:

• latest
• testing

The image types have been explained. For tag, we use latest for latest stable and testing for development purposes. If no tag is specified at build time, then latest is the default.

This section describes, how to publish the image to docker repository. First, please login into your docker account locally by executing :

docker login

(Note: it is important for you to have access to nebulastream docker hub organization for further steps)

Afterwards, one can execute

docker push nebulastream/nes-XXX-image:YYY

Where XXX represents one image type out of:

• devel
• build
• executable

and YYY as one tag out of:

• latest
• testing

The image types have been explained. For tag, we use latest for latest stable and testing for development purposes. If no tag is specified at build time, then latest is the default.