Experimental workflow

Once your experiment setup is defined (see documentation) you are ready to run your experiment.

Enforcing your Layers & Services

Enforcing your experiment’s Layers & Services is possible using a single command that will pass your layers_services.yaml file to e2clab’s Layers and Services manager.

E2clab’s Layers and Services manager will interpret your layers_services.yaml file to reserve the physical resources on the defined testbeds and to install, configure and launch all services defined by the user.

Run the following command to initiate your experiment and enforce your layers and services :

e2clab layers-services /path/to/SCENARIO_DIR /path/to/ARTIFACTS_DIR

Once the command is done, e2clab will have created a env binary file in your SCENARIO_DIR that is used to store the current state of your experiment.

Note

That means that you now must always pass the same SCENARIO_DIR parameter to the following command or else e2clab won’t be able to retreive your experiment’s status and it will result in an error.

If you want to run an another experiment, you will need to re-run this command passing the SCENARIO_DIR pointing to your new experiment’s status.

Experiment folder

The command will also initiate your experiment’s result folder inside your SCENARIO_DIR with a name like YYYYmmdd-hhmmss where e2clab will output verification file and where you can output your experiment’s result using the working_dir variable in your workflow.yaml (doc). Your SCENARIO_DIR should look something like this

SCENARIO_DIR/
├─layers_services.yaml
├─network.yaml
├─workflow.yaml
├─env                   # experiment status
└─"YYYYmmdd-hhmmss"     # result folder
  └─...

layers_services-validate

A layers_services-validate.yaml file will be created in your result folder, detailing the deployment of your services (hosts tags and addresses, monitoring dashboard…).

Example layers_services.yaml file

---
environment:
  job_name: monitoring-tpg-g5k-iotlab
  walltime: "00:59:00"
  g5k:
    cluster: paradoxe
    job_type: ["allow_classic_ssh"]
  iotlab:
    cluster: grenoble
monitoring:
  type: tpg
  provider: g5k
  cluster: paradoxe
  network: shared
  ipv: 6
layers:
- name: cloud
  services:
  - name: MyServer
    environment: g5k
    roles: [monitoring]
    quantity: 1
- name: edge
  services:
  - name: MyClientA
    environment: iotlab
    cluster: grenoble
    archi: a8:at86rf231
    roles: [monitoring]
    quantity: 1
  - name: MyClientB
    environment: iotlab
    cluster: grenoble
    archi: rpi3:at86rf233
    roles: [monitoring]
    quantity: 1

Example layers_services-validate.yaml file generated from the deployment of the previous file:

# '* * * * * * * Monitoring Service (started during workflow ''launch'' step)'
# 'Available at: http://localhost:3000'
# 'Access from your local machine: ssh -NL 3000:localhost:3000 paradoxe-2.rennes.grid5000.fr'
# 'username: admin / password: admin'
# '* * * * * * * Configure network.yaml and workflow.yaml using the information below!'
cloud:
- cloud.myserver.1.1:
  - paradoxe-7.rennes.grid5000.fr
edge:
- edge.myclienta.1.1:
  - node-a8-103.grenoble.iot-lab.info
- edge.myclientb.2.1:
  - node-rpi3-1.grenoble.iot-lab.info

Your SCENARIO_DIR should look something like this:

SCENARIO_DIR/
├─layers_services.yaml
├─network.yaml
├─workflow.yaml
├─env
├─...
└─"YYYYmmdd-hhmmss"
  ├─layers_services-validate.yaml
  └─...

CLI reference

Enforcing your Network

Once your have initiated your experiment and enforced your layers and services, you can enforce your network emulation defined in network.yaml with the following command :

e2clab network /path/to/SCENARIO_DIR

Once the command is done, you should find network validating files inside the network-validate folder located in your experiment’s result folder.

SCENARIO_DIR/
├─layers_services.yaml
├─network.yaml
├─workflow.yaml
├─env
├─...
└─"YYYYmmdd-hhmmss"
  ├─...
  └─network-validate
    └─...

CLI reference

Enforcing your Workflow

Once your experiment’s Layers & Services and Network have been enforced, you are ready to launch your experiment’s Workflow.

In your workflow.yaml file, you should have defined up to three tasks for your experiment: prepare, launch and finlaize.

You can run each of these steps indpendently with the following commands:

prepare

To run the prepare task of your workflow (e.g. install dependencies, import configuration files):

e2clab workflow /path/to/SCENARIO_DIR prepare

launch

To run the launch task of your workflow (e.g. run a bash script):

e2clab workflow /path/to/SCENARIO_DIR launch

Note

When your launch task is done if you use the --finalize option, e2clab will automatically run the finalize task of your workflow where you should have defined the steps to back-up your experiment’s data. This allows you to run your launch task multiple times without having to worry about backing-up your data after each run.

e2clab workflow /path/to/SCENARIO_DIR launch --finalize

finalize

To run the finalize task of your workflow (e.g. backup your data):

e2clab workflow /path/to/SCENARIO_DIR finalize

workflow_validate

A workflow-validate.out file will be generated in your experiment’s result folder exposing ansible’s logging for each task defined in your workflow.yaml file.

Your SCENARIO_DIR should look something like this:

SCENARIO_DIR/
├─layers_services.yaml
├─network.yaml
├─workflow.yaml
├─env
├─...
└─"YYYYmmdd-hhmmss"
  ├─workflow-validate.out
  └─...

CLI reference

Finalizing your experiment

The finalize command is a useful utility to back-up all your experiment’s information.

The Layers and Services manager will backup all monitoring and provenance data in your experiment folder.
The Workflow manager will run your workflow’s finalize tasks.

e2clab finalize /path/to/SCENARIO_DIR

CLI reference

Freeing your experiment’s resources

Once you are done with your experiment and your reservations our your testbeds have not reached the defined walltime you might want to free all the resources (e.g. stop your OAR job on grid5000). You can run the following command:

e2clab destroy /path/to/SCENARIO_DIR

CLI reference

Automating your whole experiment

Running or automating all those previous commands can be cumbursome. The deploy command runs the Layers and Services and Network managers, and then enforces the prepare and launch tasks from your workflow, then waits for the provided --duration beforme finalizing the experiment (i.e. backing-up data and running the finalize task). deploy automatically runs your whole experiment.

You can repeat you experiment automatically using the --repeat option.

# Running 3 experiments with a duration of 2 minutes after 'launch'
e2clab deploy /path/to/SCENARIO_DIR /path/to/ARTIFACTS_DIR --repeat 3 -duration 120

CLI reference

Automating workflow configurations

app_conf parameter

You might find the way that the Workflow manager deals with the workflow.yaml configuration file too rigid to test different application flavors or deploying multiple experiments to test different parameters too long and redundant.

For example, you want to run a stress command on your hosts but you want to change the stress type between ‘cpu’ or ‘io’ when launching your workflow.

You don’t want to have to modify your workflow.yaml between e2clab workfow launch commands.
You don’t want to have to configure several experiments and have to re-deploy the infrastructure.

The e2clab deploy and e2clab workflow commands have a –app_conf arguments that allows the user to dynamically pass application configurations to the Workflow manager in the workflow.yaml in the {{ app_conf }} variable.

The following command and workflow.yaml file allows you to chain one experiment stressing the hosts cpus and ios:

# here we finalize (backup) the experiment between each launch with --finalize
e2clab workflow /path/to/SCENARIO_DIR/ launch --app_conf cpu,io --finalize

Also possibile using the deploy command:

e2clab deploy /path/to/SCENARIO_DIR /path/to/ARTIFACTS_DIR --app_conf cpu,io

# FLINK
- hosts: cloud.*
  depends_on:
    - service_selector: fog.*
      grouping: "round_robin"
      prefix: "_fog"
  vars:
    stress_name: "stress"
  prepare:
    - debug:
        msg: "Installing {{ stress_name }}"
    - shell: "sudo apt-get update && sudo apt-get install -y stress"
  launch:
    - debug:
        # {{ app_conf }} will first take the value 'cpu' during the first launch
        # and 'io' during the second
        msg: "Starting stress {{ app_conf }}"
    - shell: "stress --{{ app_conf }} 4 --timeout 60s > /tmp/exp.txt"
  finalize:
    - fetch:
        src: "/tmp/exp.txt"
        dest: "{{ working_dir }}/experiment_results/"
        validate_checksum: no
- hosts: fog.*
  prepare:
    - debug:
        msg: "Installing stress"
    - shell: "sudo apt-get update && sudo apt-get install -y stress"
  launch:
    - debug:
        msg: "Starting {{ app_conf }} stress"
    - shell: "stress --{{ app_conf }} 4 --timeout 60s"

workflow_env.yaml and application configuration

You might also find the app_conf too limiting and want to dynamically set multiple variables in your workflow.yaml.

The values passed to app_conf can be used as keys to fetch parameters in an extra configuration file called workflow_env.yaml that will be available with the env_ prefix.

For example you want to run a stress command on your hosts but you want to change the number of stress “workers” deployed in our experiment and the “timeout” of the command.

Our experiment setup might look something like this:

SCENARIO_DIR/
├─layers_services.yaml
├─network.yaml
├─workflow.yaml
├─workflow_env.yaml
└─...

workflow_env.yaml:

# app_conf parameters must match either 'cpu' or 'io'
cpu:
  workers: 4    # {{ env_workers }} in 'workflow.yaml'
  timeout: 60s  # {{ env_timeout }} in 'workflow.yaml'

io:
  workers: 2    # {{ env_workers }} in 'workflow.yaml'
  timeout: 120s # {{ env_timeout }} in 'workflow.yaml'

workflow.yaml:

# FLINK
- hosts: cloud.*
  depends_on:
    - service_selector: fog.*
      grouping: "round_robin"
      prefix: "_fog"
  vars:
    stress_name: "stress"
  prepare:
    - debug:
        msg: "Installing {{ stress_name }}"
    - shell: "sudo apt-get update && sudo apt-get install -y stress"
  launch:
    - debug:
        # {{ app_conf }} will first take the value 'cpu' during the first launch
        # and 'io' during the second
        msg: "Starting stress {{ app_conf }}"
    - shell: "stress --{{ app_conf }} {{ env_workers }} --timeout {{ env_timeout }} > /tmp/exp.txt"
  finalize:
    - fetch:
        src: "/tmp/exp.txt"
        dest: "{{ working_dir }}/experiment_results/"
        validate_checksum: no
- hosts: fog.*
  prepare:
    - debug:
        msg: "Installing stress"
    - shell: "sudo apt-get update && sudo apt-get install -y stress"
  launch:
    - debug:
        msg: "Starting {{ app_conf }} stress"
    - shell: "stress --{{ app_conf }} {{ env_workers }} --timeout {{ env_timeout }}"

Experimental workflow

Enforcing your Layers & Services

Experiment folder

layers_services-validate

CLI reference

e2clab layers-services

Enforcing your Network

CLI reference

e2clab network

Enforcing your Workflow

prepare

launch

finalize

workflow_validate

CLI reference

e2clab workflow

Finalizing your experiment

CLI reference

e2clab finalize

Freeing your experiment’s resources

CLI reference

e2clab destroy

Automating your whole experiment

CLI reference

e2clab deploy

Automating workflow configurations

app_conf parameter

workflow_env.yaml and application configuration