Running COMPSs Applications on G5K (standalone deployment)
This section is intended to show how to execute COMPSs applications on Grid’5000. This example assumes you’ve built a custom environment including all COMPSs requirements (please, see How to create a custom environment on G5K)
In this example you will learn how to (see Figure 1: COMPSs deployment):
- Define the experimental environment:
Layers and Services + Define the logic of your COMPSs Service;
Network;
Workflow (prepare, launch, finalize).
Deploy a standalone COMPSs Cluster: 1 Master + 3 Workers
Run COMPSs applications
Fig. 15 Figure 1: COMPSs deployment
Experiment Artifacts
$ git clone https://gitlab.inria.fr/E2Clab/examples/compss
$ cd compss/
The structure of the experimental setup looks like this:
compss/
├── artifacts/ # ARTIFACTS_DIR
│ │ # Python scripts to generate the COMPSs configuration files
│ ├── generate_project_xml_file.py
│ └── generate_resources_xml_file.py
├── standalone/ # SCENARIO_DIR
│ ├── layers_services.yaml
│ ├── network.yaml
│ └── workflow.yaml
└── docker/
Defining the Experimental Environment
Defining the logic of your COMPSs Service
This example relies on a user-defined-service called COMPSs_std which you need to import.
Clone the user-defined-services repository:
$ git clone https://gitlab.inria.fr/E2Clab/user-defined-services.git
Run this command to register this service to e2clab
$ e2clab services add ~/user-defined-services/COMPSs_std.py --link
- Here we exmplain what the COMPSs_std service does:
assigning machines for the Master and Workers;
adding information about the Workers to the Master (see
extra=extra_compss_masterin COMPSs.py) to generate the resources.xml and project.xml files;registering the Master and Worker as a subservice of the COMPSs Service.
Layers & Services Configuration
This configuration file presents the Layers and Services that compose this example. The COMPSs_std Service (a cluster of four nodes,
quantity: 4) is composed of one Master and three Workers (please, see the COMPSs_std.py service definition).
We also define a env_name that points to a custom grid5000 kadeploy3 environment provided by one of the maintainers of e2clab that already contains an working COMPSs 3.3.1 distribution (installed via pip). You can also use your own custom grid5000 environments by looking at the tutorial. This option also requires a "deploy job type.
Note
- name: COMPSs_std explicitely refers to the COMPSs_std.py service.
1---
2environment:
3 job_name: compss
4 walltime: "01:59:00"
5 g5k:
6 job_type: ["deploy"]
7 env_name: https://api.grid5000.fr/sid/sites/rennes/public/tbadts/pycompss.yaml
8 cluster: nova
9 key_name: path/to/your/grid5000_ssh_key.pub # defaults to ~/.ssh/id_rsa.pub
10layers:
11- name: cloud
12 services:
13 - name: COMPSs_std
14 quantity: 4
Network Configuration
The file below presents the network configuration between machines in the COMPSs cluster. In this example, we defined a constraint between the Master and all Workers.
1networks:
2- src: cloud.compss_std.1.master.1
3 dst: cloud.compss_std.1.worker.*
4 delay: "2ms"
5 rate: "10gbit"
6 loss: "10%"
Workflow Configuration
This configuration file presents the application workflow configuration, they are:
Regarding all machines in the COMPSs cluster
cloud.compss.*, inpreparewe compile the COMPSs application code and generate the .jar file.Regarding just the COMPSs Master
cloud.compss.*.master.*, inpreparewe are copying the Python scripts to genetrate the COMPSs configuration files and then we generate such files (resources.xmlandproject.xml). Note that we used--workers {{ _self.workers }}since we added this information in the COMPSs.py Service. Then, inlaunchwe run the COMPSs application.
1# Git clone COMPSs tutorial apps git on all hosts
2- hosts: cloud.compss_std.*
3 prepare:
4 - debug:
5 msg: "[{{ lookup('pipe','date +%Y-%m-%d-%H-%M-%S') }}] Preparing application on the COMPSs Master and Workers."
6 - shell: rm -rf tutorial_apps && git clone https://github.com/bsc-wdc/tutorial_apps.git tutorial_apps
7 - shell: cd tutorial_apps/java/hello/ && mvn clean package
8# Run on master host
9- hosts: cloud.compss_std.*.master.*
10 prepare:
11 - debug:
12 msg: "[{{ lookup('pipe','date +%Y-%m-%d-%H-%M-%S') }}] Generating project.xml and resources.xml files. My workers are: {{ _self.workers }}."
13 # Copy python scripts
14 - copy:
15 src: "{{ working_dir }}/generate_project_xml_file.py"
16 dest: "/tmp/generate_project_xml_file.py"
17 - copy:
18 src: "{{ working_dir }}/generate_resources_xml_file.py"
19 dest: "/tmp/generate_resources_xml_file.py"
20 # Run python scrips to generate 'project.xml' and 'resources.xml'
21 - shell: source ~/.bashrc && /bin/bash /etc/profile.d/compss.sh && . /etc/profile && . ~/.bashrc && cd /tmp/ && python3 generate_project_xml_file.py --workers {{ _self.workers }} --install_dir $COMPSS_HOME --working_dir /tmp/COMPSsWorker --user root --app_dir /root/ --path_to_new_file /tmp/project.xml
22 args:
23 executable: /bin/bash
24 - shell: cd /tmp/ && python3 generate_resources_xml_file.py --workers {{ _self.workers }} --computing_units 24 --memory_size 125 --min_port_nio 43001 --max_port_nio 43002 --path_to_new_file /tmp/resources.xml
25 # Run COMPSs
26 # We make sure to source the COMPSs environment variables
27 launch:
28 - debug:
29 msg: "Running COMPSs application"
30 - shell: source ~/.bashrc && source /etc/profile.d/compss.sh && runcompss --project="/tmp/project.xml" --resources="/tmp/resources.xml" -d --classpath=/root/tutorial_apps/java/hello/jar/hello.jar hello.Hello
31 args:
32 executable: /bin/bash
33
34# You can check the outputs here on the master host
35# /root/.COMPSs/hello.Hello_01/runtime.log
36# /root/.COMPSs/hello.Hello_01/jobs/job1_NEW.out
37# /root/.COMPSs/hello.Hello_01/jobs/job1_NEW.err
Note
Besides prepare and launch, you could also use finalize to backup some data (e.g., experiment results).
E2Clab first runs on all machines the prepare tasks. Then, the launch tasks on all machines, and finally the
finalize tasks. Regarding the hosts order, it is top to down as defined by the users in the workflow.yaml
file.
Running & Verifying Experiment Execution
Find below the command to deploy the COMPSs cluster on G5K and run COMPSs applications.
Before starting:
make sure that your COMPSs_std service is correctly imported by e2clab by runing the following command:
e2clab services list.make sure that you have correctly setup your shh
key_namevariable as we will need to do calls to the grid5000 API to use a custom environment.in the command bellow,
compss/standalone/is the scenario directory (where the fileslayers_services.yaml,network.yaml, andworkflow.yamlmust be placed and where the results will be saved).in the command bellow,
compss/artifacts/is the artifacts directory (where the Python scripts to generate the COMPSs configuration files must be placed).
$ e2clab deploy compss/standalone/ compss/artifacts/
Next, you can check the log files after the application execution.
# After you've ssh'd into the master host, you can find the address in 'layers_services-validate.yaml' file.
$ cat /root/.COMPSs/hello.Hello_01/runtime.log
$ cat /root/.COMPSs/hello.Hello_01/jobs/job1_NEW.out
Deployment Validation & Experiment Results
Find below the files generated after the execution of the experiment. It consists of validation files
layers_services-validate.yaml and workflow-validate.out.
$ ls compss/standalone/<your_experiment_dir>/
e2clab.err # E2Clab logs
e2clab.log # E2Clab error logs
network-validate/ # Network validation files
layers_services-validate.yaml # Mapping between layers and services with physical machines
workflow-validate.out # Commands used to deploy the application (prepare, launch, and finalize)
Note
Providing a systematic methodology to define the experimental environment and providing access to the methodology artifacts (layers_services.yaml, network.yaml, workflow.yaml, and the COMPSs.py Service) leverages the experiment Repeatability, Replicability, and Reproducibility, see ACM Digital Library Terminology.