​
Infrastructure prerequisites

Before proceeding with the setup, ensure that the following components have been set up:
  • Redis - version 6.0 or higher
  • Kafka - version 2.8 or higher

​
Dependencies

  • Kafka - used for event communication
  • Redis - used for caching

​
Configuration

​
Configuring Kafka

Set the following Kafka-related configurations using environment variables:
  • SPRING_KAFKA_BOOTSTRAP_SERVERS - the address of the Kafka server, it should be in the format “host:port”

​
Groupd IDs

The configuration parameters “KAFKA_CONSUMER_GROUP_ID_*” are used to set the consumer group name for Kafka consumers that consume messages from topics. Consumer groups in Kafka allow for parallel message processing by distributing the workload among multiple consumer instances. By configuring the consumer group ID, you can specify the logical grouping of consumers that work together to process messages from the same topic, enabling scalable and fault-tolerant message consumption in your Kafka application.
Configuration ParameterDefault valueDescription
KAFKA_CONSUMER_GROUP_ID_PROCESS_ENGINE_COMMANDS_MESSAGEengine-commands-messageConsumer group ID for processing engine commands messages
KAFKA_CONSUMER_GROUP_ID_PROCESS_ENGINE_COMMANDS_DISCONNECTengine-commands-disconnectConsumer group ID for processing engine commands disconnect messages
KAFKA_CONSUMER_GROUP_ID_PROCESS_ENGINE_COMMANDS_CONNECTengine-commands-connectConsumer group ID for processing engine commands connect messages
KAFKA_CONSUMER_GROUP_ID_PROCESS_TASK_COMMANDStask-commands-messageConsumer group ID for processing task commands

​
Threads

The configuration parameters “KAFKA_CONSUMER_THREADS_*” are utilized to specify the number of threads assigned to Kafka consumers for processing messages from topics. These parameters allow you to fine-tune the concurrency and parallelism of your Kafka consumer application, enabling efficient and scalable message consumption from Kafka topics.
Configuration ParameterDefault valueDescription
KAFKA_CONSUMER_THREADS_PROCESS_ENGINE_COMMANDS_MESSAGE10Number of threads for processing engine commands messages
KAFKA_CONSUMER_THREADS_PROCESS_ENGINE_COMMANDS_DISCONNECT5Number of threads for processing engine commands disconnect messages
KAFKA_CONSUMER_THREADS_PROCESS_ENGINE_COMMANDS_CONNECT5Number of threads for processing engine commands connect messages
KAFKA_CONSUMER_THREADS_TASK_COMMANDS10Number of threads for task commands
KAFKA_AUTH_EXCEPTION_RETRY_INTERVAL10Interval between retries after an AuthorizationException is thrown by the Kafka consumer
Configuration ParameterDefault value
KAFKA_TOPIC_EVENTS_GATEWAY_PROCESS_INSTANCE_IN_MESSAGEai.flowx.dev.eventsgateway.engine.commands.message.v1
KAFKA_TOPIC_EVENTS_GATEWAY_PROCESS_INSTANCE_IN_DISCONNECTai.flowx.dev.eventsgateway.engine.commands.disconnect.v1
KAFKA_TOPIC_EVENTS_GATEWAY_PROCESS_INSTANCE_IN_CONNECTai.flowx.dev.eventsgateway.engine.commands.connect.v1
Configuration ParameterDefault value
KAFKA_TOPIC_EVENTS_GATEWAY_TASK_IN_MESSAGEai.flowx.eventsgateway.task.commands.message.v1

​
Configuring authorization & access roles

Set the following environment variables to connect to the identity management platform:
Configuration ParameterDescription
SECURITY_OAUTH2_BASE_SERVER_URLBase URL of the OAuth2 server
SECURITY_OAUTH2_CLIENT_CLIENT_IDClient ID for OAuth2 authentication
SECURITY_OAUTH2_CLIENT_CLIENT_SECRETClient secret for OAuth2 authentication
SECURITY_OAUTH2_REALMRealm for OAuth2 authentication

​
Redis

The process engine sends the messages to the events-gateway, which is responsible for sending them to Redis.
Configuration ParameterDescription
SPRING_REDIS_HOSTHostname of the Redis server
SPRING_REDIS_PASSWORDPassword for Redis server
SPRING_REDIS_TTLTime-to-live for Redis keys (default value:5000000 # milliseconds)

​
Master replica

The events-gateway can be configured to communicate with Redis using the MASTER_REPLICA replication mode by configuring the following property: spring.redis.sentinel.nodes: replica1, replica2, replica3, etc…
Example
spring.redis.sentinel.nodes=host1:26379,host2:26379,host3:26379
In the above example, the Spring Boot application will connect to three Redis Sentinel nodes: host1:26379, host2:26379, and host3:26379. The property value should be a comma-separated list of host:port pairs, where each pair represents the hostname or IP address and the port number of a Redis Sentinel node.
By default, Redis is standalone, so the configuration with redis-replicas is optional for high load use cases.
In the context of Spring Boot and Redis Sentinel integration, the spring.redis.sentinel.nodes property is used to specify the list of Redis Sentinel nodes that the Spring application should connect to. These nodes are responsible for monitoring and managing Redis instances.

​
Events

This configuration helps manage how event data is stored and accessed in Redis.
Configuration ParameterDefaultDescription
EVENTS_REDIS_FREQUENCYMILLIS200Time interval (in milliseconds) between Redis queries by the events gateway to check for new messages
EVENTS_REDIS_TTLHOURS4Sets the time-to-live for events in Redis to 4 hours

​
Configuring logging

The following environment variables could be set in order to control log levels:
Configuration ParameterDescription
LOGGING_LEVEL_ROOTLogging level for the root Spring Boot microservice logs
LOGGING_LEVEL_APPLogging level for the application-level logs