User-Defined Services

This section is intended to explain all you need to know about User-Defined Services. That means, how to implement, deploy, and manage your Services.

What is a User-Defined Service?

In E2Clab, a service represents an agent that provides a specific functionality or action in the scenario workflow. If you plan to deploy your application using E2Clab you have to implement a User-Defined Service. A User-Defined Service is an E2Clab Service implemented by users according to their needs. Users may use the EnOSlib library to implement their services.

Find in our repository a list of more than 10 Services already implemented. You can reuse them or build your own service inspired by the existing ones. These examples include:

  • master-worker based services

  • using Docker container images

  • services with subservices

Feel free to share your services in this repository, so others can reuse them!

How to implement a Service?

The steps to implement a service are:

  • Each service must be implemented in a “.py” file, e.g.: MyService.py.

  • The class name should be the same as the file and must inherit the Service class.

  • Implement the logic of your service in deploy().

  • Register your service using register_service().

Next, we explain how to:

  • use the Service class to implement your service;

  • assign service’s hosts to subservices (e.g., master and workers);

  • run commands on hosts and access the service metadata;

  • assign extra information to services;

  • register your service;

The Service class

From the Service class, users may get access of the services’s metadata defined by users in the layers_services.yaml file. Furthermore, users may generate service’s extra information that may be accessed by other services in the workflow.yaml file. Please, see Figure 1: Service dataflow.

../_images/service-dataflow.png

Figure 1: Service dataflow

The Service class provides two methods to implement a service, they are:

  • deploy(): where you should implement the logic of your service; and

  • register_service(): to register your service.

It also provides the following attributes:

  • self.hosts: all hosts (EnOSlib Host) assigned to the Service.

  • self.layer_name: the Layer name that the Service belongs to (in layers_services.yaml).

  • self.roles: EnOSlib Roles grouping all hosts.

  • self.service_metadata: the Service metadata defined by users in layers_services.yaml file.

Assigning service’s hosts to subservices

First host for the Master and the remaining ones for the Workers.

1roles_master = Roles({"master": [self.hosts[0]]})
2roles_worker = Roles({"worker": self.hosts[1:len(self.hosts)]})

Running commands on hosts & accessing the service metadata

EnOSlib uses Ansible to run commands on hosts. In the example below, lines 4 and 6 use run() to run commands on the hosts. While, lines 12 and 13 use Ansible modules such as shell and docker_container, respectively, to run commands on the host. Line 15 accesses the service metadata defined by the user in the layers_services.yaml file.

 1 import enoslib as en
 2
 3 # Running a command in the Master host
 4 my_command_output = en.run("my command", roles=roles_master)[0].stdout
 5 # Using the output of the previous command as a command in the Worker hosts
 6 en.run(f"{my_command_output}", roles=roles_worker)
 7
 8 # Installing a package and then starting a container on the Master host.
 9 # The image name is obtained from the service metadata defined in
10 # 'layers_services.yaml' file.
11 with en.actions(roles=roles_master) as a:
12     a.shell("pip3 install ...")
13     a.docker_container(
14         name="master",
15         image=self.service_metadata["data_2"]["sub_data_1"],
16         ...
17     )

Please, find below other methods provided by EnOSlib to run commands on hosts:

Assigning extra information to your service

1 workers = ['wk_id_1', 'wk_id_2', 'wk_id_3']
2 # adding in the Master information about Workers
3 extra_master = [{'container_name': "master", 'workers': workers}]
4 # adding information to the Workers. Each element in the list refers to a Worker.
5 extra_workers = [
6     {'key1': "value1", ..., 'keyN': "valueN"},  # 1st Worker
7     {'key1': "value1", ..., 'keyN': "valueN"},  # 2nd Worker
8     {'key1': "value1", ..., 'keyN': "valueN"},  # 3rd Worker
9 ]

Registering your service

 1 # Register the Service (this example consider a service with subservices)
 2 # 'Master' is a subservice of the 'MyService' service
 3 self.register_service(_roles=roles_master, sub_service="master",
 4                       service_port=8888, extra=extra_master)
 5
 6 # 'Worker' is a subservice of the 'MyService' service
 7 self.register_service(_roles=roles_worker, sub_service="worker", extra=extra_workers)
 8
 9 # service_extra_info: extra attributes assigned to the service. They can be accessed
10 # in the "workflow.yaml" file.
11 # service_roles: the registered services.
12 return self.service_extra_info, self.service_roles

Note

Besides the extra information defined by users, such as:

  • self.register_service(…, extra=extra_master)

  • self.register_service(…, extra=extra_workers)

E2Clab adds to all hosts in a service the following extra information:

  • _id: refers to the final Service ID: LayerID_ServiceID_MachineID

  • __address__: refers to the hostname

  • url: hostname:port or just hostname (if port number is not defined)

  • __address4__: refers to the ipv4 address of the host

  • url4: ipv4:port or just ipv4 (if port number is not defined)

  • __address6__: (if applied) refers to the ipv6 address of the host

  • url6: (if applied) ipv6:port or just ipv6 (if port number is not defined)

Therefore, after the Service registration, the whole extra information, of the master service, will be:

{
_id: ‘1_1_1’,
__address__: ‘10.52.0.9’,
url: ‘10.52.0.9:8888’,
__address4__: ‘10.52.0.9’,
url4: ‘10.52.0.9:8888’,
container_name: “master”,
workers: [‘wk_id_1’, ‘wk_id_2’, ‘wk_id_3’]
}

Once registered, E2Clab generates tags for each Service. The tags are generated as follows:

  • Service without subservices: layer_name.service_name.service_id.host_id.

  • Service with subservices: layer_name.service_name.service_id.subservice_name.host_id.

Note

service_id starts from 1 (one) and is incremented per layer. host_id starts from 1 (one) and is incremented with the number of hosts.

Please, see examples below:

Considering a one Service named MyService composed by four hosts deployed on the Cloud Layer, the tag will be generated as follows:

1cloud.myservice.1.1
2cloud.myservice.1.2
3cloud.myservice.1.3
4cloud.myservice.1.4

Considering one Service named MyService with the subservices 1 Master and 3 Workers deployed on the Cloud Layer, the tag will be generated as follows:

1cloud.myservice.1.master.1
2cloud.myservice.1.worker.1
3cloud.myservice.1.worker.2
4cloud.myservice.1.worker.3

Considering two Services named Client and Server composed by two hosts each and deployed on the Cloud Layer, the tag will be generated as follows:

1cloud.client.1.1
2cloud.client.1.2
3cloud.server.2.1
4cloud.server.2.2

Note

You can find in scenario_dir/layers_services-validate.yaml file the Services tags generated by E2Clab and their respective hosts. The layers_services-validate.yaml file is generated after running one of the following commands: e2clab layers-service scenario_dir/ artifacts_dir/ or e2clab deploy scenario_dir/ artifacts_dir/.

How to deploy a Service?

In order to deploy you service, you have to:

  • define it in the layers_services.yaml file. The MyService.py file name must be exactly the same (case sensitive) name of the service defined in layers_services.yaml file.

  • place your MyService.py file in e2clab/e2clab/services/ directory, so E2Clab can find it.

 1environment: ...
 2monitoring: ...
 3layers:
 4- name: cloud
 5  services:
 6  - name: MyService
 7    quantity: 4
 8    roles: [monitoring]
 9    # service metadata added by the user
10    data_1: value
11    data_2:
12      sub_data_1: value
13      sub_data_n: value
14    data_n: value

How can services access information from each other?

Next, we explain:

  • How to access services’ hosts in the workflow.yaml file.

  • How to access information of the service itself.

  • How services access information from each other.

Accessing services’ hosts in the workflow.yaml file

Find below how to access services’ hosts in the workflow.yaml file from a coarse-grained to a fine-grained selection (based on Ansible patterns):

  • hosts: cloud.* all services’ hosts (multiple services) deployed in the Cloud layer.

  • hosts: cloud.myservice.* all hosts that compose one or more MyService service(s).

  • hosts: cloud.myservice.1.* all hosts (Master & Workers) that compose the MyService service.

  • hosts: cloud.myservice.1.worker.*: all Worker hosts that compose the MyService service.

  • hosts: cloud.myservice.1.worker.3: a single Worker host (3rd worker) in MyService service.

Accessing information of the service itself

Users may use _self. to access information of the service itself.

In the example below, we are accessing the container name {{ _self.container_name }} and the workers {{ _self.workers }}. Remember that this extra information was defined previously in the MyService.py service as follows: extra_master = [{'container_name': "master", 'workers': workers}].

1- hosts: cloud.myservice.1.master.1
2  prepare:
3    - shell: docker exec -it -d {{ _self.container_name }}
4                          bash -c 'python example.py --workers {{ _self.workers }}'

Accessing information from another service(s)

Please, refer to Services Relationships to know how to define dependencies and relationships between services.