Frontend module is crucial for the client-perceived performance and stability of Hermes. There are many configuration options that can be changed to fine tune the Undertow HTTP server and communication with Kafka.
Hermes Frontend can be configured to send the response after no longer than specific timeout. This prevents long-running requests in case of Kafka downtime and helps to reason about service SLA. This is related to response codes: depending on whether message has been persisted in Kafka before the timeout, publisher receives 201 or 202 response code.
You can change the timeout values separately for messages with different acknowledgment levels. To do so, use
frontend.idle.timeout for leader ACKfrontend.long.idle.timeout for all ACKThose timeouts are counted from the time the request has been parsed till response is sent. This means, that interaction with Kafka needs to take place during this period. If timeout is reached when event is in sending to Kafka state, 202 Accepted response is returned, since message is already in sending buffer and Hermes guarantees it will end up in Kafka. Otherwise normal request timed out message is sent.
| Option | Description | Default value |
|---|---|---|
| frontend.port | port to listen on | 8080 |
| frontend.read.timeout | maximum time to wait for first portion of data to appear on socket | 2000ms |
| frontend.request.parse.timeout | maximum time to wait before full request is received | 5000ms |
| frontend.request.chunk.size | chunk size | 1024B |
| frontend.io.threads.count | number of Undertow IO threads | 2 * cores |
| frontend.worker.threads.count | number of Undertow worker threads | 200 |
| frontend.graceful.shutdown.initial.wait.ms | time between setting health endpoint to return DOWN and actually stopping the server | 10 000ms |
Default timeout settings make Frontend safe against Slowloris attack.
Hermes uses Kafka in-memory send buffers to tolerate any downtime or hiccups, as described in
publishing guide. Use kafka.producer.buffer.memory option to change the size of buffer.
By default it is set to 256 MB. Changing the size might extend the period for which Hermes is able to receive
messages in case of Kafka downtime. Since internally there are two Kafka producers spawned, one for ACK-leader and one
for ACK-all, there are also two buffers - keep this in mind when deciding on heap size.
Kafka producer properties map 1:1 to Kafka producer configuration options. See Kafka documentation if you have any doubts or need an extended description.
| Option | Kafka config | Description | Default value |
|---|---|---|---|
| kafka.producer.metadata.max.age | METADATA_MAX_AGE_CONFIG | how old can topic metadata be | 30000 ms |
| kafka.proudcer.compression.codec | COMPRESSION_TYPE_CONFIG | compression algorithm | none |
| kafka.producer.retires | RETRIES_CONFIG | how many times should we retry sending | Integer.MAX_VALUE |
| kafka.producer.retry.backoff.ms | RETRY_BACKOFF_MS_CONFIG | backoff between retries | 256 ms |
| kafka.producer.batch.size | BATCH_SIZE_CONFIG | size of sent message batch in bytes | 16 kB |
| kafka.producer.tcp.send.buffer | SEND_BUFFER_CONFIG | size of TCP buffer | 128 kB |
Processing time for first event arriving on each topic may be longer than expected from Hermes frontend. This is because topic metadata has to be fetched from Kafka. What is more, Avro topics need schema in order to validate incoming messages.
To get rid of this issue and reduce event processing latency, all the required data can be fetched during Hermes frontend's startup, just before it's healthcheck goes green. Note, enabling startup data loading will make frontend boot a little longer.
| Option | Description | Default value |
|---|---|---|
| frontend.startup.topic.metadata.loading.enabled | should the startup topic metadata loading be enabled | false |
| frontend.startup.topic.metadata.loading.retry.interval | retry interval between retry loops | 1s |
| frontend.startup.topic.metadata.loading.retry.count | number of retries between topic metadata fetch loops | 5 |
| frontend.startup.topic.metadata.loading.thread.pool.size | number of worker threads loading metadata concurrently | 16 |
| frontend.startup.topic.schema.loading.enabled | should the startup topic schema loading be enabled | false |
| frontend.startup.topic.schema.loading.retry.count | number of retries between topic schema fetch loops | 5 |
| frontend.startup.topic.schema.loading.thread.pool.size | number of worker threads loading schemas concurrently | 16 |