Savanna Edge-to-Cloud: Grid’5000 + FIT IoT LAB

../_images/g5k%2Bfitiot-logo.png

In this tutorial, we show how to analyze the performance of a real-life Edge-to-Cloud application deployed in the African savanna (illustrated in Figure 1: Edge-to-Cloud application).

This application is composed of distributed Edge devices monitoring animal migration in the Serengeti region. Devices at the Edge collect and compress wildlife images, then the image is sent to the Cloud where the animal classification happens using a pre-trained Neural Network model. Finally, classified data helps conservationists to learn what management strategies work best to protect species.

The goals of these experiments are to understand the impact on performance of Cloud-centric and Hybrid (Edge+Cloud) processing approaches.

In this example you will learn how to:

  • Configure a cloud server on Grid’5000 testbed (animal classification using Deep Learning)

  • Configure an edge device on FIT IoT LAB testbed (collect and compress wildlife images, then send images to the cloud server)

  • Configure the network between the edge device and cloud server.

  • Execute experiments and analyze results

../_images/savanna-setup.png

Fig. 17 Figure 1: Edge-to-Cloud application

Experiment Artifacts

$ git clone https://gitlab.inria.fr/E2Clab/examples/savanna
$ cd savanna/

The structure of the experimental setup looks like this:

savanna/
├── artifacts/    # ARTIFACTS_DIR
├── g5k-fitiot/   # SCENARIO_DIR
│   ├── layers_services.yaml
│   ├── network.yaml
│   └── workflow.yaml
└── chameleon/

Defining the Experimental Environment

Layers & Services Configuration

This configuration file presents the layers and services that compose this example. The Cloud Server (one machine quantity: 1 in Grid’5000 environment: g5k). The Edge Client (one Raspberry Pi 3 quantity: 1 in FIT IoT LAB environment: iotlab).

 1---
 2environment:
 3  job_name: savanna-edge-cloud
 4  walltime: "00:59:00"
 5  g5k:
 6    cluster: paradoxe
 7    job_type: []
 8    # key_name: your_g5k_id.pub
 9    ipv6: true
10    firewall_rules:
11      - services: ["Server"]
12        ports: [1883]
13  iotlab:
14    cluster: grenoble
15layers:
16- name: cloud
17  services:
18  - name: Server
19    environment: g5k
20    quantity: 1
21- name: edge
22  services:
23  - name: Client
24    environment: iotlab
25    cluster: grenoble
26    archi: rpi3:at86rf233
27    # You can also specify the nodes by 
28    # uncommenting 'servers' and commenenting 'cluster' and 'archi'
29    # if some node does not boot correctly
30    # servers: ["rpi3-2.grenoble.iot-lab.info"]
31    quantity: 1

Note

We create a firewall rule on Grid’5000 to allow the Cloud Server to receive data from the Edge Client (FIT IoT LAB) in port 1883 MQTT (Message Queuing Telemetry Transport) protocol.

Network Configuration

The file below presents the network configuration between the cloud and edge infrastructures delay: 30ms, loss: 0.1%, rate: 512kbit.

1networks:
2- src: cloud
3  dst: edge
4  delay: 30ms
5  rate: 512kbit
6  loss: 0.1%

Workflow Configuration

This configuration file presents the application workflow configuration.

  • The Cloud Server cloud.server.*:

prepare copies from the local machine to the remote machine the artifacts

launch executes the Python application (animal classification using Deep Learning)

finalize after experiment ends, copies the results from the remote to the local machine

  • The Edge Client edge.client.*:

prepare copies from the local machine to the remote machine the artifacts

launch executes the client application (collect and compress wildlife images, then send images to the cloud server) asynchronously.

 1# SERVER
 2- hosts: cloud.server.*
 3  prepare:
 4    - copy:
 5        src: "{{ working_dir }}/artifacts_cloud/"
 6        dest: /tmp
 7  launch:
 8    - debug:
 9        msg: "Running the g5k_ipv6 server"
10    - shell: cd /tmp && bash cloud_worker.sh > cloud-log.log
11    - shell: python3 /tmp/predict_loop.py
12      async: 3120
13      poll: 0
14  finalize:
15    - debug:
16        msg: "Saving results"
17    - fetch:
18        src: /tmp/predict.log
19        dest: "{{ working_dir }}/experiment-results/"
20        validate_checksum: no
21# CLIENT
22- hosts: edge.client.*
23  depends_on:
24    - service_selector: cloud.server.*
25      grouping: "round_robin"
26      prefix: g5k_ipv6
27  prepare:
28    - copy:
29        src: "{{ working_dir }}/artifacts_edge/"
30        dest: /tmp
31  launch:
32    - debug:
33        msg: "Running the client"
34    - shell: "bash edge_worker.sh edge_data 100 {{ g5k_ipv6.__address6__ }} True > edge-log.log"
35      args: 
36        chdir: "/tmp/"

Understanding the parameters of edge_worker.sh:

  • edge_data is the topic name

  • 100 is the number of times the edge device will send images to the Cloud (every 30 seconds)

  • True means Edge+Cloud processing approach (False means Cloud-only processing approach)

Note

Using depends_on on the Edge Client edge.client.* we can access the IPv6 address of the Cloud Server as follows {{ server.__address6__ }}.

Running & Verifying Experiment Execution

Find below the commands to deploy this application and check its execution.

$ e2clab deploy ./g5k-fitiot/ ./artifacts/

Note

For the first deployment, a good practice is to do it incrementally, as explained here.

Firewall rule

ssh to the Cloud server

While the deployment is enforcing the launch step of the workflow, you can ssh to the server and check the logs of the prediction.

# Select the cloud.server.1.1 machine
$ e2clab ssh ./g5k-fitiot
root@paradoxe-4:~$ tail -f /tmp/predict.log

You may also check the mosquitto topic

You can use the Python script at ~/git/savanna/artifacts/utils/mosquitto_sub_img.py to download images received. Images will be downloaded in the directory you run the script. In the example below, 2001:660:4406:700:1::28 is the IPv6 address of the cloud server.

Note

You can find the public ipv6 address of the server by connecting to it (with the same e2clab ssh command for example) and running the following command:

$ ip -6 addr show scope global

$ python mosquitto_sub_img.py --topic edge_data --mqtt_broker 2001:660:4406:700:1::28

Deployment Validation & Experiment Results

Find below the files generated after the execution of each experiment. It consists of:

  • validation files: layers_services-validate.yaml, network-validate/, and workflow-validate.out

  • experiment resutls: for each experiment a new directory is generated 20230623-113827/.

$ ls ~/git/savanna/g5k-fitiot/20230623-113827/

layers_services-validate.yaml   # Mapping between layers and services with physical machines
network-validate/               # Network configuration for each physical machine
workflow-validate.out           # Commands used to deploy application (prepare, launch, and finalize)
experiment-results/             # Experiment results
../_images/plot-g5k-fit.png

Fig. 18 Figure 2: Cloud-centric vs Edge+Cloud processing