Skip to content

Message internal format

Hermes appends internal metadata to every published message. They are not visible to neither publisher nor subscriber, but are present on Kafka. Thus if you use tools to read data directly from Kafka, you will see events along with their metadata. Metadata format depends on data format (Avro, JSON) - please see format documentation for details.

Currently metadata holds two fields:

  • messageId - string containing Hermes-Message-Id as UUID v4
  • timestamp - Unix epoch timestamp in milliseconds holding date of reception

There is no guarantee that this metadata won't be extended.

JSON

Messages with metadata have following format:

{
    "metadata": {
        "messageId": "b0838e5f-c3d1-47b3-8949-abe7a1f65202",
        "timestamp": 1439129947000
    },
    "message": {
        // contents of your message goes here
    }
}

Avro

Hermes metadata is stored in __metadata field, which is specified as map of optional elements. It will always contain at least messageId and timestamp.

{
  "type": "record",
  "name": "SomeTopic",
  "namespace": "pl.allegro.hermes",
  "doc": "Schema of some event",
  "fields": [
    {
      "name": "__metadata",
      "type": [
        "null",
        {
          "type": "map",
          "values": "string"
        }
      ],
      "doc": "Field used to propagate metadata like messageId and timestamp",
      "default": null
    },
    // other event fields
  ]
}

Custom reading internal messages

Hermes allows to provide custom implementation of reading Kafka records, for example for reading metadata from Kafka headers.

To do this, implement the interfaces MessageContentReader and MessageContentReaderFactory:

class CustomMessageContentReader implements MessageContentReader {
    @Override
    public UnwrappedMessageContent read(ConsumerRecord<byte[], byte[]> message, ContentType contentType) {
        // custom implementation of reading consumer record
    }
}

class CustomMessageContentReaderFactory implements MessageContentReaderFactory {
    @Override
    public MessageContentReader provide(Topic topic) {
        return new CustomMessageContentReader();
    }
}

and register the implementation of MessageContentReaderFactory as a bean:

@Configuration
public class CustomHermesConsumersConfiguration {

    @Bean
    @Primary
    public MessageContentReaderFactory customMessageContentReaderFactory() {
        return new CustomMessageContentReaderFactory();
    }
}