********************* User-Defined Services ********************* .. toctree:: :maxdepth: 2 :caption: Contents: This section is intended to explain **all you need to know about User-Defined Services**. That means, how to implement, deploy, and manage your Services. What is a User-Defined Service? =============================== In E2Clab, a service represents an agent that provides a specific functionality or action in the scenario workflow. If you plan to deploy your application using E2Clab you have to implement a User-Defined Service. A User-Defined Service is an **E2Clab Service** implemented by users according to their needs. Users may use the `EnOSlib `_ library to implement their services. `Find in our repository `_ a list of more than 10 Services already implemented. You can reuse them or build your own service inspired by the existing ones. These examples include: - master-worker based services - using Docker container images - services with subservices Feel free to share your services in this repository, so others can reuse them! How to implement a Service? =========================== The steps to implement a service are: - Each service must be implemented in a ".py" file, e.g.: ``MyService.py``. - The class name should be the same as the file and must inherit the `Service class `_. - Implement the logic of your service in ``deploy()``. - Register your service using ``register_service()``. Next, we explain how to: - use the **Service class** to implement your service; - assign service's hosts to subservices (e.g., master and workers); - run commands on hosts and access the service metadata; - assign extra information to services; - register your service; The Service class ----------------- From the **Service class**, users may get access of the **services's metadata** defined by users in the ``layers_services.yaml`` file. Furthermore, users may generate **service's extra information** that may be accessed by other services in the ``workflow.yaml`` file. Please, see :ref:`service-dataflow`. .. _service-dataflow: .. figure:: service-dataflow.png :width: 100% :align: center Figure 1: Service dataflow The **Service class** provides two methods to implement a service, they are: - ``deploy()``: where you should implement the logic of your service; and - ``register_service()``: to register your service. It also provides the following attributes: - ``self.hosts``: all hosts (`EnOSlib Host `_) assigned to the Service. - ``self.layer_name``: the **Layer** name that the Service belongs to (in ``layers_services.yaml``). - ``self.roles``: `EnOSlib Roles `_ grouping all hosts. - ``self.service_metadata``: the **Service** metadata defined by users in ``layers_services.yaml`` file. Assigning service's hosts to subservices ---------------------------------------- First host for the **Master** and the remaining ones for the **Workers**. .. code-block:: python :linenos: roles_master = Roles({"master": [self.hosts[0]]}) roles_worker = Roles({"worker": self.hosts[1:len(self.hosts)]}) Running commands on hosts & accessing the service metadata ---------------------------------------------------------- EnOSlib uses `Ansible `_ to run commands on hosts. In the example below, ``lines 4 and 6`` use `run() `_ to run commands on the hosts. While, ``lines 12 and 13`` use Ansible modules such as `shell `_ and `docker_container `_, respectively, to run commands on the host. Line ``15`` accesses the **service metadata** defined by the user in the ``layers_services.yaml`` file. .. code-block:: python :linenos: import enoslib as en # Running a command in the Master host my_command_output = en.run("my command", roles=roles_master)[0].stdout # Using the output of the previous command as a command in the Worker hosts en.run(f"{my_command_output}", roles=roles_worker) # Installing a package and then starting a container on the Master host. # The image name is obtained from the service metadata defined in # 'layers_services.yaml' file. with en.actions(roles=roles_master) as a: a.shell("pip3 install ...") a.docker_container( name="master", image=self.service_metadata["data_2"]["sub_data_1"], ... ) Please, find below other methods provided by EnOSlib to run commands on hosts: - `run `_: to run commands; - `run_command `_: to run shell commands with `Ansible patterns `_; - `run_play `_: to run Ansible tasks; - `run_ansible `_: to run Ansible playbooks. Assigning extra information to your service ------------------------------------------- .. code-block:: python :linenos: workers = ['wk_id_1', 'wk_id_2', 'wk_id_3'] # adding in the Master information about Workers extra_master = [{'container_name': "master", 'workers': workers}] # adding information to the Workers. Each element in the list refers to a Worker. extra_workers = [ {'key1': "value1", ..., 'keyN': "valueN"}, # 1st Worker {'key1': "value1", ..., 'keyN': "valueN"}, # 2nd Worker {'key1': "value1", ..., 'keyN': "valueN"}, # 3rd Worker ] Registering your service ------------------------ .. code-block:: python :linenos: # Register the Service (this example consider a service with subservices) # 'Master' is a subservice of the 'MyService' service self.register_service(_roles=roles_master, sub_service="master", service_port=8888, extra=extra_master) # 'Worker' is a subservice of the 'MyService' service self.register_service(_roles=roles_worker, sub_service="worker", extra=extra_workers) # service_extra_info: extra attributes assigned to the service. They can be accessed # in the "workflow.yaml" file. # service_roles: the registered services. return self.service_extra_info, self.service_roles .. note:: Besides the extra information defined by users, such as: - self.register_service(..., extra=extra_master) - self.register_service(..., extra=extra_workers) E2Clab adds to all hosts in a service the following extra information: - ``_id``: refers to the final Service ID: **LayerID_ServiceID_MachineID** - ``__address__``: refers to the **hostname** - ``url``: **hostname:port** or just **hostname** (if port number is not defined) - ``__address4__``: refers to the **ipv4 address** of the host - ``url4``: **ipv4:port** or just **ipv4** (if port number is not defined) - ``__address6__``: (if applied) refers to the **ipv6 address** of the host - ``url6``: (if applied) **ipv6:port** or just **ipv6** (if port number is not defined) Therefore, after the Service registration, the whole extra information, of the ``master service``, will be: | { | ``_id``: '1_1_1', | ``__address__``: '10.52.0.9', | ``url``: '10.52.0.9:8888', | ``__address4__``: '10.52.0.9', | ``url4``: '10.52.0.9:8888', | ``container_name``: "master", | ``workers``: ['wk_id_1', 'wk_id_2', 'wk_id_3'] | } Once registered, E2Clab generates tags for each Service. The tags are generated as follows: - Service without subservices: ``layer_name.service_name.service_id.host_id``. - Service with subservices: ``layer_name.service_name.service_id.subservice_name.host_id``. .. note:: ``service_id`` starts from **1 (one)** and is **incremented per layer**. ``host_id`` starts from **1 (one)** and is **incremented with the number of hosts**. Please, see examples below: Considering a **one Service** named ``MyService`` composed by four hosts deployed on the Cloud **Layer**, the tag will be generated as follows: .. code-block:: python :linenos: cloud.myservice.1.1 cloud.myservice.1.2 cloud.myservice.1.3 cloud.myservice.1.4 Considering **one Service** named ``MyService`` with the **subservices** 1 Master and 3 Workers deployed on the Cloud **Layer**, the tag will be generated as follows: .. code-block:: python :linenos: cloud.myservice.1.master.1 cloud.myservice.1.worker.1 cloud.myservice.1.worker.2 cloud.myservice.1.worker.3 Considering **two Services** named ``Client`` and ``Server`` composed by two hosts each and deployed on the Cloud **Layer**, the tag will be generated as follows: .. code-block:: python :linenos: cloud.client.1.1 cloud.client.1.2 cloud.server.2.1 cloud.server.2.2 .. note:: You can find in ``scenario_dir/layers_services-validate.yaml`` file the **Services tags** generated by E2Clab and their respective hosts. The ``layers_services-validate.yaml`` file is generated after running one of the following commands: ``e2clab layers-service scenario_dir/ artifacts_dir/`` or ``e2clab deploy scenario_dir/ artifacts_dir/``. How to deploy a Service? ======================== In order to deploy you service, you have to: - define it in the ``layers_services.yaml`` file. The ``MyService.py`` file name must be exactly the same (case sensitive) name of the service defined in ``layers_services.yaml`` file. - place your ``MyService.py`` file in `e2clab/e2clab/services/ `_ directory, so E2Clab can find it. .. literalinclude:: layers_services.yaml :language: yaml :linenos: How can services access information from each other? ==================================================== Next, we explain: - How to access services' hosts in the ``workflow.yaml`` file. - How to access information of the service itself. - How services access information from each other. Accessing services' hosts in the workflow.yaml file --------------------------------------------------- Find below how to access services' hosts in the ``workflow.yaml file`` from a ``coarse-grained`` to a ``fine-grained`` selection (based on `Ansible patterns `_): - ``hosts: cloud.*`` **all services' hosts** (multiple services) deployed in the **Cloud layer**. - ``hosts: cloud.myservice.*`` **all hosts** that compose **one or more MyService service(s)**. - ``hosts: cloud.myservice.1.*`` **all hosts (Master & Workers)** that compose the **MyService** service. - ``hosts: cloud.myservice.1.worker.*``: **all Worker hosts** that compose the **MyService** service. - ``hosts: cloud.myservice.1.worker.3``: **a single Worker host** (3rd worker) in **MyService** service. Accessing information of the service itself ------------------------------------------- Users may use ``_self.`` to access **information of the service itself**. In the example below, we are accessing the container name ``{{ _self.container_name }}`` and the workers ``{{ _self.workers }}``. Remember that this **extra information** was defined previously in the ``MyService.py`` service as follows: ``extra_master = [{'container_name': "master", 'workers': workers}]``. .. code-block:: yaml :linenos: - hosts: cloud.myservice.1.master.1 prepare: - shell: docker exec -it -d {{ _self.container_name }} bash -c 'python example.py --workers {{ _self.workers }}' Accessing information from another service(s) --------------------------------------------- Please, refer to `Services Relationships <../services_dependencies/index.html>`_ to know how to define dependencies and relationships between services.