ESP32 MQTT connecting and publishing to ThingSpeak but channel is not updating

I followed this example to use MQTT from my ESP32:

https://uk.mathworks.com/help/thingspeak/use-arduino-client-to-publish-to-a-channel.html

Lets make some fake variables for discussion:

long channelID = 1234567;
char mqttPass[] = "ABCDEFGHIJKLM"; // MQTT API KEY (generated from user profile)
char apiKey[] = "APIKEY"; //Channel Write API Key (generated from channel API keys tab)

I can connect with the example code (sometimes it fails, but it reconnects and then holds the connection well).

After the postingInterval has passed, the publish function is called:

void mqttPublishFeed(float field1, float field2, float field3, float field4, float field5, float field6, float field7, float field8) {
    
  // Create data string to send to ThingSpeak.
   String data = "field1=";
    data += String(field1, DEC);
    data += "&field2=";
    data += String(field2, DEC);
    data += "&field3=";
    data += String(field3, DEC);
    data += "&field4=";
    data += String(field4, DEC);
    data += "&field5=";
    data += String(field5, DEC);
    data += "&field6=";
    data += String(field6, DEC);
    data += "&field7=";
    data += String(field7, DEC);
    data += "&field8=";
    data += String(field8, DEC);

  
  int length = data.length();
  const char *msgBuffer;
  msgBuffer=data.c_str();
  Serial.println(msgBuffer);
  
  // Create a topic string and publish data to ThingSpeak channel feed. 
  String topicString = "channels/" + String( channelID ) + "/publish/"+String(apiKey);
  Serial.println(topicString);
  length = topicString.length();
  const char *topicBuffer;
  topicBuffer = topicString.c_str();
  mqttClient.publish( topicBuffer, msgBuffer );
  lastConnectionTime = millis();
}

Here is an example of the topicString and msgBuffer variables that are printed to the serial (using the fake variables state above)

msgBuffer:
field1=829.0000000000&field2=477.0000000000&field3=28.0499992371&field4=35.0566406250&field5=23.0000000000&field6=50.9949913025&field7=0.0000000000&field8=995.9716186523

topicString:
channels/1234567/publish/APIKEY

Problem:

I have confirmed that (on my serial) the channelID and the channel API write key are correct and data seems to publish without error (though it is QOS 0 so its really hard to tell). Despite this, my channel does not update. I have previously used HTTPS and had no issues with API keys or channel updating so I know they aren't the issue.

[Graham Coulby]

OK, I found that I was unable to send the same request to HiveMQ's public broker. After a little testing I found that any message where (topic+payload).length > 121 the message didn't send I know MQTT treats messages <127 bytes differently from larger messages.

My question now is, are there any limits from ThingSpeak, that prevent me from sending >127byte messages over MQTT?

Answer 1

[Graham Coulby]

OK, as I suspected (in my comment). There was a limit being imposed on the payload. At first I thought it was related to the Free Account. However, after seeing similar limitations in HiveMQ, I realised it had to be a code related issue.

For others experiencing the same issues:

The PubSubClient library used in many of the documentation examples imposes as 256byte limit (which I imagine includes all of the headers, QOS details and timestamp etc. either way, if you don't specify your buffer size, it will go over and then not get sent. Since ThingSpeak requires QOS 0, you aren't going to get alerted to these issues, but they are there:

For the Adruino PubSubClient you must specify the packet size after the include:

#include <PubSubClient.h>
#define MQTT_MAX_PACKET_SIZE 2048

This allowed me to send 8 fields of decimal values.

[Christopher Stapels]

Thanks for sharing. This has caught me in the past as well.

By the way,its very lucky that you happened to get channel 1234567. Some of my examples have used your channel.