ReplyingKafkaTemplate: ERROR — No pending reply: ConsumerRecord

The story will tell you about the timeout or the configuration we need to know about implementing the ReplyingKafkaTemplate in your project.

ReplyingKafkaTemplate: KafkaTemplate that implements request/reply semantics. for more please visit https://shuaibabdulla40.medium.com/apache-kafka-request-reply-semantics-implementation-replyingkafkatemplate-5bf64958268c

When you have a replay topic is shared with multiple consumers you should be careful during the configuration of group id and sharedreplaytopic configuration.

As you could see above diagram, if your application is deployed in, for example, two-node, and was given the same group id, for example, group-node for both node1 and node 2 then the below cases may occur.

  • The request was initiated from Node1 with replaykafkatemplate and posted the message successfully in the Request Topic.
  • The initiated thread in the Node1 will be waiting to get the response back in the replay topic with a mentioned timeout.
  • The request will be processed by some microservice and will send the response to the Response Topic.
  • Now as both Node1 and Node2 are subscribed to the response topic with the same groupid, the Kafka topic will send a message to either of the nodes, The request initiated Node1 or Node2.
  • If the message is subscribed to Node1 the request-replay semantic will work, else you will get a timeout because the response message will be sent to only Node2.
  • If you are getting the below exception
  • No pending reply: ConsumerRecord(topic = request-reply-topic, partition = 8, offset = 1, CreateTime = 1544653843269, serialized key size = -1, serialized value size = 1609, headers = RecordHeaders(headers = [RecordHeader(key = kafka_correlationId, value = [-14, 65, 21, -118, 70, -94, 72, 87, -113, -91, 92, 72, -124, -110, -64, -94])], isReadOnly = false), key = null, with correlationId: [-18271255759235816475365319231847350110], perhaps timed out, or using a shared reply topic

You get the log if a message is received with a correlation id that does not match the entries currently in this. futures (pending replies). This can only occur under the following circumstances:

  1. The request timed out (in which case there will be a corresponding WARN log).
  2. The template is stopped ()ped (in which case this.futures is cleared).
  3. An already processed reply is redelivered for some reason (shouldn’t happen).
  4. The reply is received before the key is added to this.futures (can’t happen since it’s inserted before send()ing the record).
  5. The server side sends 2 or more replies for the same request.
  6. Some other application is sending data to the same reply topic. If you can reproduce it with DEBUG logging, it would help because then we log the correlation key on the send as well.

Solution

Use different groupid for different nodes so will guarantee each node will get one message at least from the topic. See the documentation.

Java 8 | Spring | React | Angular | CI/CD | Microservices | Monitoring |Apache Kafka. https://www.linkedin.com/in/shuaib-kunhabdulla-83780188

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store