***************
Getting Started
***************

.. contents::
   :depth: 2


Installation
============

Setting up a virtual environment
--------------------------------

.. code-block:: bash

    # In your workstation
    $ cd git/
    $ virtualenv -p python3 venv
    $ source venv/bin/activate
    $ git clone https://gitlab.inria.fr/E2Clab/e2clab.git
    $ cd e2clab
    $ pip install -U -e .

.. note::

  It's a good practice to use a virtualenv or python version manager like `pyenv`_.


Create your ssh keys
--------------------

Skip this step if your keys already exist in ~/.ssh/.

.. code-block:: bash

    $ ssh-keygen -q -t rsa -f ~/.ssh/id_rsa -N ""

Setting up testbeds
===================

Please, refer to the tutorials below to setup connectivity to the different testbeds:

- `Grid'5000 <https://e2clab.gitlabpages.inria.fr/e2clab/environments/getting-started-g5k.html>`_
- `Chameleon Cloud and Edge <https://e2clab.gitlabpages.inria.fr/e2clab/environments/getting-started-chameleon.html>`_
- `FIT IoT LAB <https://e2clab.gitlabpages.inria.fr/e2clab/environments/getting-started-iotlab.html>`_

Checking Connectivity to testbeds
---------------------------------

This command gives you a view of:

- the installed providers (dependency check)
- the connectivity to the various testbeds (connectivity check)

.. code-block:: bash

    $ e2clab check-testbeds

E2Clab Command Line Interface
=============================

The commands allowed in E2Clab are presented below. Note that you can use just a single
command to deploy your application ``deploy`` or you can deploy it separately using the
commands ``layers-services``, ``network``, ``workflow`` (``prepare``, ``launch``, or
``finalize``), and ``finalize``, respectively.

.. code-block:: text

    $ e2clab --help
    Usage: e2clab [OPTIONS] COMMAND [ARGS]...

    Options:
      --help  Show this message and exit.

    Commands:
      check-testbeds     Checks the connectivity to the various testbeds.
      deploy             Manage experiment deployment, execution, and backup.
        layers-services  Enforce layers and services in experiment environment.
        network          Enforce communication rules in experiment environment.
        workflow         Enforce workflow in experiment environment.
        finalize         Backup experiment data.
      optimize           Optimize application workflow.



Running Multiple Experiments
----------------------------

.. code-block:: text

    $ e2clab deploy --help
    Usage: e2clab deploy [OPTIONS] SCENARIO_DIR ARTIFACTS_DIR

      Manage experiment deployment, execution, and backup.

    Options:
      --scenarios_name TEXT  Scenario names separated by comma.
      --app_conf TEXT        Application configurations separated by comma.
      --repeat INTEGER       Number of times to repeat the experiment.
      --duration INTEGER     Duration of each experiment in seconds.
      --logging_level TEXT   Logging level.
      --help                 Show this message and exit.



- **Example**: the following experiment will consist of four iterations ``--repeat 3``, each one with a duration of four minutes ``--duration 240``.


.. code-block:: bash

    $ e2clab deploy --repeat 3 --duration 240 /path/to/scenario/ /path/to/artifacts/



Incremental Experiment Configuration
------------------------------------

- **Layers & Services**: Enforce layers and services in experiment environment.

.. code-block:: bash

    $ e2clab layers-services /path/to/scenario/ /path/to/artifacts/


- **Network**: Enforce network communication rules in experiment environment.

.. code-block:: bash

    $ e2clab network /path/to/scenario/


- **Workflow**: Enforce workflow in experiment environment ``prepare``, ``launch``, or ``finalize``.

.. code-block:: bash

    $ e2clab workflow /path/to/scenario/ prepare   # prepare workflow (i.e. copy dataset and libraries, create a Kafka topic, etc.)
    $ e2clab workflow /path/to/scenario/ launch    # execute workflow (i.e. start data producers, submit Flink job, etc.)
    $ e2clab workflow /path/to/scenario/ finalize  # backup workflow data (i.e. data generated from processes that compose the workflow)

.. note::

  E2Clab first runs on all machines the ``prepare`` tasks. Then, the ``launch`` tasks on
  all machines, and finally the ``finalize`` tasks. Regarding the ``hosts`` order, it is
  top to down as defined by the users in the ``workflow.yaml`` file.


- **Finalize**: Backup all experiment data, such as log files, monitoring data, etc.

.. code-block:: bash

    $ e2clab finalize /path/to/scenario/



.. _pyenv: https://github.com/pyenv/pyenv