ESPAsyncWebserver - Send chunked response from PROGMEM with template processor
October 27, 2024 @ 01:56
In case someone needs it, I had to search and dig around for quite a while to come up with this solution.
ESPAsyncWebserver is pretty good documented in my opinion. The documentation of the fork from github user mathieucarbou I use for my CanGrow project gets even more into detail.
I had a problem with corrupted responses when the webserver had to send a very big html file processed with some templates in it. So I thought it might help to use chunked responses.
Update: I think I understood the problem and what chunking is and does not correct. So my problem with corrupted pages when using the template processor persists even when using chunked response.
The documentation for this says just
/* Chunked Response containing templates
*
* Used when content length is unknown. Works best if the client supports HTTP/1.1
*/
String processor(const String& var)
{
if(var == "HELLO_FROM_TEMPLATE")
return F("Hello world!");
return String();
}
// ...
AsyncWebServerResponse *response = request->beginChunkedResponse("text/plain", [](uint8_t *buffer, size_t maxLen, size_t index) -> size_t {
//Write up to "maxLen" bytes into "buffer" and return the amount written.
//index equals the amount of bytes that have been already sent
//You will be asked for more data until 0 is returned
//Keep in mind that you can not delay or yield waiting for more data!
return mySource.read(buffer, maxLen);
}, processor);
response->addHeader("Server","ESP Async Web Server");
request->send(response);
With my very limited knowledge about C programming (and programming at all) this wasnt very helpful. Using return mySource.read(buffer, maxLen); just gave me an error, beacuse I am not using a String. My page is served from PROGMEM which is stored in the flash memory of the esp chips to save some memory.
I finally came up with the solution, and it took me some hours to get there. So I want to share my findings with you and hopefully save you some time, in case you are looking for the same topic.
Here is my example code:
#include <Arduino.h>
#ifdef ESP32
#include <AsyncTCP.h>
#include <WiFi.h>
#elif defined(ESP8266)
#include <ESP8266WiFi.h>
#include <ESPAsyncTCP.h>
#endif
#include <ESPAsyncWebServer.h>
const char* ssid = "WIFI";
const char* password = "PASSWORD";
AsyncWebServer server(80);
const char* Big_HTML_in_Flash PROGMEM = R"(<html><body>
My fancy content with some %REPLACEMENT%
</body></html>)";
String Template_Processor(const String& var) {
if(var == "REPLACEMENT") {
return String("nice replacement");
} else {
return String();
}
}
auto Chunk_len(uint8_t *buffer, size_t maxlen, size_t index) -> size_t {
// https://github.com/mathieucarbou/ESPAsyncWebServer?tab=readme-ov-file#chunked-response-containing-templates
//Write up to "maxLen" bytes into "buffer" and return the amount written.
//index equals the amount of bytes that have been already sent
//You will be asked for more data until 0 is returned
//Keep in mind that you can not delay or yield waiting for more data!
// https://forum.arduino.cc/t/espasyncwebserver-replay-page-size/1049106/19
// https://forum.arduino.cc/t/strlen-and-progmem/629376/3 - get len of PROGMEM
size_t len_html = strlen_P(reinterpret_cast<const char *>(Big_HTML_in_Flash));
size_t len = min(maxlen, len_html - index);
memcpy(buffer, Big_HTML_in_Flash + index, len);
Serial.printf("Sending len %u bytes, maxlen %u, index %u, len_html %un", len, maxlen, index, len_html);
return len;
}
void Chunked_response_page(AsyncWebServerRequest *request) {
AsyncWebServerResponse *response = request->beginChunkedResponse("text/html", Chunk_len, Template_Processor);
request->send(response);
}
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
Serial.print("Connecting to ");
Serial.print(ssid);
int i = 0;
while (WiFi.status() != WL_CONNECTED) {
delay(1000);
Serial.print(++i); Serial.print('.');
}
Serial.println("CONNECTED!");
Serial.print("IP:t");
Serial.println(WiFi.localIP());
server.on("/", HTTP_GET, Chunked_response_page);
server.begin();
}
void loop() { }