Using the multitasking capabilities of the ESP32 / FreeRTOS
One thing that I wanted to learn more about is the multitasking capabilities of the ESP32. I digged into the ESP-IDF manuals and found helpful information about the FreeRTOS SMP and how it works.
What I found then is that if you program the ESP32 using the Arduino core for ESP32 as your framework, then it is quite simple. Then loop() is running as a task already in parallel to others that handle WiFi, BLE, …
To split your app tasks into several tasks, there are several functions to start a task available. The two most useful I found are
- xTaskCreate()
- xTaskCreatePinnedToCore()
According to the documentation the difference is that xTaskCreate() adds a new task to the existing list of task, but leaves it to the task scheduler to decide on which CPU core the task will run. And (as the function call already suggests) xTaskCreatePinnedToCore() does the same, but tells the task scheduler which CPU core should run the task.
Why is it important on which CPU core the tasks will run on? According to FreeRTOS SMP :
However the ESP32 is dual core containing a Protocol CPU (known as CPU 0 or PRO_CPU) and an Application CPU (known as CPU 1 or APP_CPU).
CPU 0 might be very busy if you have a lot of WiFi communication or BLE activities, so it might be better to pin them to CPU 1 to make sure they get enough runtime to do what they should do.
Looking on my application I then tried to decide which “tasks”, that were handled in the main loop should be moved into separate tasks. The candidates I found where
- MQTT client
- Light measurement
- Temperature and humidity measurement
- Getting weather information from Weather Underground
Why move this 4 “tasks” in “real” FreeRTOS tasks and not handle them in the loop(), as we are all used to from old Arduino programming times?
The MQTT client I want to have in a separate task, because sending status messages to the MQTT server might be time consuming and depends on the quality and availability of the internet connection. I didn’t want the loop() to hang while a MQTT message is sent (and might fail to send because the server is down).
The light and temperature measurements are called in a fixed frequency. For this I could have just used a simple time-elapsed check in the loop(), but hey, we want to try out the features of ESP32 and FreeRTOS, so a time-elapsed–now-do-something approach seems to be inappropriate.
And getting the weather information from the internet can have the same problems as the MQTT client. The connection could be slow, the server could be down.
IMHO, when programming under FreeRTOS, the loop() should basically be very empty and the activities should be split into time or event triggered tasks.
A few things to keep in mind.
- Using the Arduino framework, at the time we can create tasks in setup() the task scheduler is already active. So the task that is created will start to run immediately. This might be not the best thing, as the initialization might not be finished and the task might fail to perform what it should do.
- If using OTA updates, it is crucial to stop all created tasks to make sure they do not interrupt or slow down the OTA data transfer.
For the delayed start of the tasks, I use a global boolean flag tasksEnabled. This flag is false at startup. When a task it started, before doing anything, it checks if this flag is true. If not, the tasks sends herself immediately into sleep.
To stop the tasks once an OTA process has started, I use another global boolean flag otaRunning. This flag is set true when the OTA process started. Each of the tasks checks this flag on each run. If the tasks find this flag true, the task suspends herself.
Create and control the time triggered tasks.
As an example for the time triggered task I take the one that reads values from the light sensor.
It is usually necessary to do some initialization. initLight() is called from setup() after the ESP32 finished booting and FreeRTOS is running.
First we setup the analog in port for the LDR sensor. You would expect that I initialize the TSL2561 sensor here as well. But while writing and testing the app, I found out that the initialization always failed (maybe because both CPU’s are very busy at this early stage). So I moved that into the task itself.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
/** * initLight * Setup port for LDR and TSL2561 sensor * Starts task and timer for repeated measurement * @return byte * 0 if light sensor was found and update task and timer are inialized * 1 if update task and update timer could not be started */ byte initLight() { byte resultValue = 0; // Initialize analog port for LDR pinMode(ldrPin,INPUT); adcAttachPin(ldrPin); analogReadResolution(11); analogSetAttenuation(ADC_6db); |
Next the task that does actually read the light values is defined and added to the scheduler
|
1 2 3 4 5 6 7 8 9 |
// Start task for light value readings xTaskCreatePinnedToCore( lightTask, /* Function to implement the task */ "LightMeasure ", /* Name of the task */ 4000, /* Stack size in words */ NULL, /* Task input parameter */ 5, /* Priority of the task */ &lightTaskHandle, /* Task handle. */ 1); /* Core where the task should run */ |
If the task is successfully added to the scheduler, we can start the timer that will activate the task.
|
1 2 3 4 5 6 7 8 9 |
if (lightTaskHandle == NULL) { resultValue = 2; } else { // Start update of light values data every 10 seconds lightTicker.attach(10, triggerGetLight); } return resultValue; } |
The ISR that is called every 10 seconds from the timer, does not much. It just checks if the task exists and wakes it up.
|
1 2 3 4 5 6 7 8 |
/** * Start task to reads TSL2561 light sensor and LDR analog value */ void triggerGetLight() { if (lightTaskHandle != NULL) { xTaskResumeFromISR(lightTaskHandle); } } |
The task itself must be implemented as an endless loop. It is not allowed to return. Finishing the task is in the responsibility of the task scheduler after a vTaskDelete() has been called.
Within the task, the first thing is to check if an OTA update has started. If this is the case, we tell the scheduler to finish this task by calling vTaskDelete.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
/** * Task to read data from TSL2561 light sensor and LDR sensor * @param pvParameters * pointer to task parameters */ void lightTask(void *pvParameters) { // Serial.println("lightTask loop started"); while (1) // lightTask loop { if (otaRunning) { vTaskDelete(NULL); } |
Next is a check if the setup() function has been finished and if the app is ready to get light values.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
if (tasksEnabled) { // Ready to run the task? if (hasTSLSensor) { // Is the sensor initialized? // Read TSL2561 light sensor long collLight = readLux(); if (collLight != 65536) { newTSLValue = collLight; } else { // newTSLValue = 0; Serial.println(errorLabel + digitalTimeDisplaySec() + " Failed to read from TSL2561"); hasTSLSensor = false; } esp32Wire.reset(); } else { // Try to initialize sensor again // Initialize light sensor with SDA,SCL,frequency esp32Wire.reset(); esp32Wire.begin(21,22,100000); if (tsl.begin(&esp32Wire)) { hasTSLSensor = true; configureSensor(); } else { hasTSLSensor = false; } } // Read analog value of LDR newLDRValue = analogRead(ldrPin); |
When this code is run the first time the TSL2561 sensor has not been initialized yet, so the sensor will be initialized first. Otherwise the TSL 2561 sensor values are read (with some error management). And then the LDR analog value is read.
After that the task sends herself into sleep by calling vTaskSuspend(NULL);
|
1 2 3 4 |
} vTaskSuspend(NULL); } } |
The task handling the measurement of temperature and humidity is build up similar. The code for this can be found in my Github repo.
Event based activation of the MQTT client task
In this application the MQTT client is only used to send messages to the MQTT server. These messages can be sent from any of the running tasks at any time and there might be overlaps in the call to send these messages because of task switches.
To get control over that, I created a buffer, that can queue up to 15 messages. Each time a task or function wants to send a MQTT message, it calls addMqttMsg(). addMqttMsg then adds the message into the queue and wakes up the task that is actually sending the message to the MQTT server. The way I created the queue is not the most efficient one, but it works and is easy to implement. A FiFo buffer might be more efficient here, maybe I implement it at a later time.
How does this look in code?
First we define the message buffer structure and the buffer itself. For the messages I send, I need only four elements. The topic and the payload define the message. retained defines if the message should be sent as a retained message or not. And waiting is simply a flag if this buffer entry is waiting to be sent or can be overwritten.
|
1 2 3 4 5 6 7 8 9 10 |
/** Structure for MQTT message Buffer */ struct mqttMsgStruct { String topic = ""; String payload = ""; bool waiting = false; bool retained = false; }; /** Queued messages for MQTT */ mqttMsgStruct mqttMsg[msgBufferSize]; |
Next thing is to initialize the buffer, connect to the MQTT server and start the task to send messages.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
/** * initMqtt * Initialize Meeo connection */ void initMqtt() { // Clear message structure for (int index = 0; index < msgBufferSize; index++) { mqttMsg[index].waiting = false; } // Connect to MQTT broker if (!connectMQTT()) { Serial.println("Can't connect to MQTT broker"); return; } // Start task for MEEO publishing xTaskCreatePinnedToCore( mqttTask, /* Function to implement the task */ "MQTTPublisher ", /* Name of the task */ 4000, /* Stack size in words */ NULL, /* Task input parameter */ 5, /* Priority of the task */ &mqttTaskHandle, /* Task handle. */ 1); /* Core where the task should run */ // Tell MQTT broker we are alive mqttClient.publish((char *)"/DEV/ESP32",(char *)"esp32",5,true,0); } |
The function to add messages into the queue is scanning through the queue to find an empty slot. If it finds one, it saves the message into this slot. Then it wakes up the task to send out pending messages from the queue. The task is woken up even if there is no more space to queue messages, this should help to empty the queue in case something got stuck in the communication.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
/** * addMqttMsg * Adds a message to the buffer to be processed by meeoTask() * * @param topic * String with the topic * @param payload * String with the payload * @return bool * true if message is added to the buffer * false if buffer was full */ bool addMqttMsg (String topic, String payload, bool retained) { bool queueResults = false; for (byte index = 0; index < msgBufferSize; index ++) { if (!mqttMsg[index].waiting) { // found an empty slot? mqttMsg[index].topic = topic; mqttMsg[index].payload = payload; mqttMsg[index].waiting = true; mqttMsg[index].retained = retained; queueResults = true; break; } } if (tasksEnabled) { vTaskResume(mqttTaskHandle); } return queueResults; } |
The task itself loops through the message buffer and searches for unsent messages. If it finds one it tries to send the message to the MQTT server. The task stays active until all messages in the queue are sent. Then it sends itself to sleep again.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
/** * Task to send data from meeoMsg buffer to Meeo.IO * @param pvParameters * pointer to task parameters */ void mqttTask(void *pvParameters) { // Serial.println("mqttTask loop started"); while (1) // mqttTask loop { if (otaRunning) { vTaskDelete(NULL); } for (byte index = 0; index < msgBufferSize; index ++) { if (mqttMsg[index].waiting) { if (mqttMsg[index].topic == "WEI") { // Broadcast weather status over UDP IPAddress multiIP (192, 168, 0, 255); udpSendMessage(multiIP, mqttMsg[index].payload, 9997); mqttMsg[index].waiting = false; } else if (mqttClient.publish("/"+mqttMsg[index].topic,mqttMsg[index].payload,mqttMsg[index].retained,0)) { mqttMsg[index].waiting = false; } else { // Publishing error. Maybe we lost connection ??? // Serial.println("Sending to MQTT broker failed"); if (connectMQTT()) { // Try to reconnect and resend the message if (mqttClient.publish("/"+mqttMsg[index].topic,mqttMsg[index].payload,mqttMsg[index].retained,0)) { mqttMsg[index].waiting = false; mqttClient.publish((char *)"/DEV/ESP32",(char *)"esp32",5,true,0); } } } } } bool queueIsEmpty = true; for (byte index = 0; index < msgBufferSize; index ++) { if (mqttMsg[index].waiting) { queueIsEmpty = false; } } if (queueIsEmpty) { vTaskSuspend(NULL); } } } |
Ok, that’s how I use the multi tasking capabilities of the ESP32. Hope you can use some of it in your own application.
09:45
Is the full code available on Github?
20:32
Yes it is available on Github. Link is the post, but here again: Task source code