Running COMPSs Applications on G5K (Docker deployment)

This section is intended to show how to execute COMPSs applications on Grid’5000. This example uses the official Docker image of COMPSs. This image comes with COMPSs installed and ready to use.

In this example you will learn how to (see Figure 1: COMPSs deployment):

  • Define the experimental environment:

    • Layers and Services + Monitoring + Define the logic of your COMPSs Service;

    • Network constraints;

    • Workflow (tasks: prepare, launch, and finalize).

  • Deploy a Docker COMPSs Cluster: 1 Master + 3 Workers

  • Run COMPSs applications

../_images/compss-deployment.png

Fig. 16 Figure 1: COMPSs deployment

Experiment Artifacts

Download the experimental setup for this COMPSs Docker experiment:

$ git clone https://gitlab.inria.fr/E2Clab/examples/compss
$ cd compss/

The structure of the experimental setup looks like this:

compss/
├── artifacts/  # ARTIFACTS_DIR
│   ├── generate_project_xml_file.py     # Python scripts to generate the COMPSs configuration files
│   └── generate_resources_xml_file.py
├── docker/     # SCENARIO_DIR
│   ├── layers_services.yaml
│   ├── network.yaml
│   └── workflow.yaml
└── standalone/

Defining the Experimental Environment

Layers & Services Configuration

This configuration file presents the Layers and Services that compose this example. The COMPSs Service (a cluster of four nodes, quantity: 4) is composed of one Master and three Workers (please, see the COMPSs.py Service). The name of the COMPSs Service - name: COMPSs must be the same as the COMPSs.py file.

Note that we also added a TIG (Telegraf, InfluxDB, and Grafana) Monitoring stack monitoring. Adding roles: [monitoring] in the COMPSs Service we request monitoring of all machines.

 1---
 2environment:
 3  job_name: compss
 4  walltime: "00:59:00"
 5  g5k:
 6    job_type: ["deploy"]
 7    env_name: "debian11-x64-big"
 8    cluster: paradoxe
 9    # key_name: "your_g5k_ssh_key"
10monitoring:
11  type: tig
12  provider: g5k
13  cluster: paradoxe
14  network: shared
15  ipv: 6
16layers:
17- name: cloud
18  services:
19  - name: COMPSs
20    quantity: 4
21    roles: [monitoring]
22    docker_tag: 3.3.1

Defining the logic of your COMPSs Service

User-Defined Service: add the following service to e2clab :

$ cd ~
$ git clone https://gitlab.inria.fr/E2Clab/user-defined-services.git
Here, we define the logic of our COMPSs Service. It consists mainly in:

  • installing Docker and pulling the official image of COMPSs;

  • assigning machines for the Master and Workers;

  • creating an overlay network for standalone containers;

  • adding information about the Workers to the Master (see extra=extra_compss_master in COMPSs.py) to generate the resources.xml and project.xml files;

  • registering the Master and Worker as a subservice of the COMPSs Service.

Please, read the comments in the code for more details.

Run this command to register this service to e2clab

$ e2clab services add ~/user-defined-services/COMPSs.py --link

Network Configuration

The file below presents the network configuration between machines in the COMPSs cluster. In this example, we defined a constraint between the Master and all Workers.

1networks:
2- src: cloud.compss.1.master.1
3  dst: cloud.compss.1.worker.*
4  delay: "2ms"
5  rate: "10gbit"
6  loss: 10%

Workflow Configuration

This configuration file presents the application workflow configuration, they are:

  • Regarding just the COMPSs Master cloud.compss.*.master.*:

    • in prepare we are copying the Python scripts to genetrate the COMPSs configuration files and then we generate such files (resources.xml and project.xml). Note that we used --workers {{ _self.workers }} since we added this information in the COMPSs.py Service. Finally, we copy both files to the container.

    • in launch we run the COMPSs application.

  • Regarding all Workers in the COMPSs cluster cloud.compss.*.worker.*, in prepare we add the COMPSs applications.

 1- hosts: cloud.compss.*.master.*
 2  prepare:
 3    - debug:
 4        msg: "[{{ lookup('pipe','date +%Y-%m-%d-%H-%M-%S') }}] Adding applications to the COMPSs Master. My workers are: {{ _self.workers }}"
 5    - copy:
 6        src: "{{ working_dir }}/generate_project_xml_file.py"
 7        dest: "/tmp/generate_project_xml_file.py"
 8    - copy:
 9        src: "{{ working_dir }}/generate_resources_xml_file.py"
10        dest: "/tmp/generate_resources_xml_file.py"
11    - shell: cd /tmp/ && git clone https://github.com/bsc-wdc/tutorial_apps.git tutorial_apps
12    - shell: docker cp /tmp/tutorial_apps/ {{ _self.container_name }}:/root/
13    - shell: cd /tmp/ && python generate_project_xml_file.py --workers {{ _self.workers }} --install_dir /opt/COMPSs/ --working_dir /tmp/COMPSsWorker --user root --app_dir /root/ --path_to_new_file /tmp/default_project.xml
14    - shell: cd /tmp/ && python generate_resources_xml_file.py --workers {{ _self.workers }} --computing_units 24 --memory_size 125 --min_port_nio 43001 --max_port_nio 43002 --path_to_new_file /tmp/default_resources.xml
15    - shell: docker cp /tmp/default_project.xml {{ _self.container_name }}:/opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
16    - shell: docker cp /tmp/default_resources.xml {{ _self.container_name }}:/opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
17  launch:
18    - debug:
19        msg: "Running COMPSs application: simple.py"
20    - shell: docker exec -it {{ _self.container_name }} bash -c '/opt/COMPSs/Runtime/scripts/user/runcompss --project="/opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml" --resources="/opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml" -d /root/tutorial_apps/python/simple/src/simple.py 2 2>&1'
21- hosts: cloud.compss.*.worker.*
22  prepare:
23    - debug:
24        msg: "[{{ lookup('pipe','date +%Y-%m-%d-%H-%M-%S') }}] Adding applications to the COMPSs Workers"
25    - shell: cd /tmp/ && git clone https://github.com/bsc-wdc/tutorial_apps.git tutorial_apps
26    - shell: docker cp /tmp/tutorial_apps/ {{ _self.container_name }}:/root/

Note

Besides prepare and launch, you could also use finalize to backup some data (e.g., experiment results). E2Clab first runs on all machines the prepare tasks. Then, the launch tasks on all machines, and finally the finalize tasks. Regarding the hosts order, it is top to down as defined by the users in the workflow.yaml file.

Running & Verifying Experiment Execution

Find below the command to deploy the COMPSs cluster on G5K and run COMPSs applications.

Before starting:

  • make sure that your COMPSs.py file is located in: e2clab/e2clab/services/.

  • in the command bellow, compss/docker/ is the scenario directory (where the files layers_services.yaml, network.yaml, and workflow.yaml must be placed and where the results will be saved).

  • in the command bellow, compss/artifacts/ is the artifacts directory (where the Python scripts to generate the COMPSs configuration files must be placed).

$ e2clab deploy compss/docker/ compss/artifacts/

During application runtime, you may want to access the Grafana Web interface to visualize the moniotring data of all machines that compose the COMPSs cluster (please, check the compss/docker/layers_services-validate.yaml file to get the instructions to access Grafana).

After the application execution, you can check the log files as follows:

$ docker exec -it compss_master bash
$ cat /root/.COMPSs/simple.py_01/runtime.log
$ cat /root/.COMPSs/simple.py_01/jobs/job1_NEW.out

Deployment Validation & Experiment Results

Find below the files generated after the execution of the experiment. It consists of validation files (layers_services-validate.yaml, workflow-validate.out, and network-validate/) and monitoring data influxdb-data.tar.gz.

$ ls compss/docker/<yyyymmdd-hhmmss>/

layers_services-validate.yaml   # Mapping between layers and services with physical machines
workflow-validate.out           # Commands used to deploy the application (prepare, launch, and finalize)
network-validate/               # Network configuration for each physical machine
influxdb-data.tar.gz            # Monitoring data

Note

Providing a systematic methodology to define the experimental environment and providing access to the methodology artifacts (layers_services.yaml, network.yaml, workflow.yaml, and the COMPSs.py Service) leverages the experiment Repeatability, Replicability, and Reproducibility, see ACM Digital Library Terminology.