Running COMPSs Applications on G5K (standalone deployment)

This section is intended to show how to execute COMPSs applications on Grid’5000. This example assumes you’ve built a custom environment including all COMPSs requirements (please, see How to create a custom environment on G5K)

In this example you will learn how to (see Figure 1: COMPSs deployment):

  • Define the experimental environment:
    • Layers and Services + Define the logic of your COMPSs Service;

    • Network;

    • Workflow (prepare, launch, finalize).

  • Deploy a standalone COMPSs Cluster: 1 Master + 3 Workers

  • Run COMPSs applications

../_images/compss-standalone-deployment.png

Fig. 15 Figure 1: COMPSs deployment

Experiment Artifacts

$ git clone https://gitlab.inria.fr/E2Clab/examples/compss
$ cd compss/

The structure of the experimental setup looks like this:

compss/
├── artifacts/          # ARTIFACTS_DIR
│   │                   # Python scripts to generate the COMPSs configuration files
│   ├── generate_project_xml_file.py
│   └── generate_resources_xml_file.py
├── standalone/         # SCENARIO_DIR
│   ├── layers_services.yaml
│   ├── network.yaml
│   └── workflow.yaml
└── docker/

Defining the Experimental Environment

Defining the logic of your COMPSs Service

This example relies on a user-defined-service called COMPSs_std which you need to import.

Clone the user-defined-services repository:

$ git clone https://gitlab.inria.fr/E2Clab/user-defined-services.git

Run this command to register this service to e2clab

$ e2clab services add ~/user-defined-services/COMPSs_std.py --link
Here we exmplain what the COMPSs_std service does:
  • assigning machines for the Master and Workers;

  • adding information about the Workers to the Master (see extra=extra_compss_master in COMPSs.py) to generate the resources.xml and project.xml files;

  • registering the Master and Worker as a subservice of the COMPSs Service.

Layers & Services Configuration

This configuration file presents the Layers and Services that compose this example. The COMPSs_std Service (a cluster of four nodes, quantity: 4) is composed of one Master and three Workers (please, see the COMPSs_std.py service definition).

We also define a env_name that points to a custom grid5000 kadeploy3 environment provided by one of the maintainers of e2clab that already contains an working COMPSs 3.3.1 distribution (installed via pip). You can also use your own custom grid5000 environments by looking at the tutorial. This option also requires a "deploy job type.

Note

- name: COMPSs_std explicitely refers to the COMPSs_std.py service.

 1---
 2environment:
 3  job_name: compss
 4  walltime: "01:59:00"
 5  g5k:
 6    job_type: ["deploy"]
 7    env_name: https://api.grid5000.fr/sid/sites/rennes/public/tbadts/pycompss.yaml
 8    cluster: nova
 9    key_name: path/to/your/grid5000_ssh_key.pub  # defaults to ~/.ssh/id_rsa.pub
10layers:
11- name: cloud
12  services:
13  - name: COMPSs_std
14    quantity: 4

Network Configuration

The file below presents the network configuration between machines in the COMPSs cluster. In this example, we defined a constraint between the Master and all Workers.

1networks:
2- src: cloud.compss_std.1.master.1
3  dst: cloud.compss_std.1.worker.*
4  delay: "2ms"
5  rate: "10gbit"
6  loss: "10%"

Workflow Configuration

This configuration file presents the application workflow configuration, they are:

  • Regarding all machines in the COMPSs cluster cloud.compss.*, in prepare we compile the COMPSs application code and generate the .jar file.

  • Regarding just the COMPSs Master cloud.compss.*.master.*, in prepare we are copying the Python scripts to genetrate the COMPSs configuration files and then we generate such files (resources.xml and project.xml). Note that we used --workers {{ _self.workers }} since we added this information in the COMPSs.py Service. Then, in launch we run the COMPSs application.

 1# Git clone COMPSs tutorial apps git on all hosts
 2- hosts: cloud.compss_std.*
 3  prepare:
 4    - debug:
 5        msg: "[{{ lookup('pipe','date +%Y-%m-%d-%H-%M-%S') }}] Preparing application on the COMPSs Master and Workers."
 6    - shell: rm -rf tutorial_apps && git clone https://github.com/bsc-wdc/tutorial_apps.git tutorial_apps
 7    - shell: cd tutorial_apps/java/hello/ && mvn clean package 
 8# Run on master host
 9- hosts: cloud.compss_std.*.master.*
10  prepare:
11    - debug:
12        msg: "[{{ lookup('pipe','date +%Y-%m-%d-%H-%M-%S') }}] Generating project.xml and resources.xml files. My workers are: {{ _self.workers }}."
13    #  Copy python scripts
14    - copy:
15        src: "{{ working_dir }}/generate_project_xml_file.py"
16        dest: "/tmp/generate_project_xml_file.py"
17    - copy:
18        src: "{{ working_dir }}/generate_resources_xml_file.py"
19        dest: "/tmp/generate_resources_xml_file.py"
20    # Run python scrips to generate 'project.xml' and 'resources.xml'
21    - shell: source ~/.bashrc && /bin/bash /etc/profile.d/compss.sh && . /etc/profile && . ~/.bashrc && cd /tmp/ && python3 generate_project_xml_file.py --workers {{ _self.workers }} --install_dir $COMPSS_HOME --working_dir /tmp/COMPSsWorker --user root --app_dir /root/ --path_to_new_file /tmp/project.xml
22      args:
23        executable: /bin/bash
24    - shell: cd /tmp/ && python3 generate_resources_xml_file.py --workers {{ _self.workers }} --computing_units 24 --memory_size 125 --min_port_nio 43001 --max_port_nio 43002 --path_to_new_file /tmp/resources.xml
25  # Run COMPSs
26  # We make sure to source the COMPSs environment variables
27  launch:
28    - debug:
29        msg: "Running COMPSs application"
30    - shell: source ~/.bashrc && source /etc/profile.d/compss.sh && runcompss --project="/tmp/project.xml" --resources="/tmp/resources.xml" -d --classpath=/root/tutorial_apps/java/hello/jar/hello.jar hello.Hello
31      args:
32        executable: /bin/bash
33
34# You can check the outputs here on the master host
35# /root/.COMPSs/hello.Hello_01/runtime.log
36# /root/.COMPSs/hello.Hello_01/jobs/job1_NEW.out
37# /root/.COMPSs/hello.Hello_01/jobs/job1_NEW.err

Note

Besides prepare and launch, you could also use finalize to backup some data (e.g., experiment results). E2Clab first runs on all machines the prepare tasks. Then, the launch tasks on all machines, and finally the finalize tasks. Regarding the hosts order, it is top to down as defined by the users in the workflow.yaml file.

Running & Verifying Experiment Execution

Find below the command to deploy the COMPSs cluster on G5K and run COMPSs applications.

Before starting:

  • make sure that your COMPSs_std service is correctly imported by e2clab by runing the following command: e2clab services list.

  • make sure that you have correctly setup your shh key_name variable as we will need to do calls to the grid5000 API to use a custom environment.

  • in the command bellow, compss/standalone/ is the scenario directory (where the files layers_services.yaml, network.yaml, and workflow.yaml must be placed and where the results will be saved).

  • in the command bellow, compss/artifacts/ is the artifacts directory (where the Python scripts to generate the COMPSs configuration files must be placed).

$ e2clab deploy compss/standalone/ compss/artifacts/

Next, you can check the log files after the application execution.

# After you've ssh'd into the master host, you can find the address in 'layers_services-validate.yaml' file.
$ cat /root/.COMPSs/hello.Hello_01/runtime.log
$ cat /root/.COMPSs/hello.Hello_01/jobs/job1_NEW.out

Deployment Validation & Experiment Results

Find below the files generated after the execution of the experiment. It consists of validation files layers_services-validate.yaml and workflow-validate.out.

$ ls compss/standalone/<your_experiment_dir>/

e2clab.err                      # E2Clab logs
e2clab.log                      # E2Clab error logs
network-validate/               # Network validation files
layers_services-validate.yaml   # Mapping between layers and services with physical machines
workflow-validate.out           # Commands used to deploy the application (prepare, launch, and finalize)

Note

Providing a systematic methodology to define the experimental environment and providing access to the methodology artifacts (layers_services.yaml, network.yaml, workflow.yaml, and the COMPSs.py Service) leverages the experiment Repeatability, Replicability, and Reproducibility, see ACM Digital Library Terminology.