Infrastructure prerequisites
Before proceeding with the setup, ensure that the following components have been set up:
Configuration
Configuring Kafka
Set the following Kafka-related configurations using environment variables:
SPRING_KAFKA_BOOTSTRAPSERVERS
- the address of the Kafka server, it should be in the format βhost:portβ
Groupd IDs
The configuration parameters βKAFKA_CONSUMER_GROUPID_*β are used to set the consumer group name for Kafka consumers that consume messages from topics. Consumer groups in Kafka allow for parallel message processing by distributing the workload among multiple consumer instances. By configuring the consumer group ID, you can specify the logical grouping of consumers that work together to process messages from the same topic, enabling scalable and fault-tolerant message consumption in your Kafka application.
Configuration Parameter | Default value | Description |
---|
KAFKA_CONSUMER_GROUPID_PROCESSENGINECOMMANDS_MESSAGE | engine-commands-message | Consumer group ID for processing engine commands messages |
KAFKA_CONSUMER_GROUPID_PROCESSENGINECOMMANDS_DISCONNECT | engine-commands-disconnect | Consumer group ID for processing engine commands disconnect messages |
KAFKA_CONSUMER_GROUPID_PROCESSENGINECOMMANDS_CONNECT | engine-commands-connect | Consumer group ID for processing engine commands connect messages |
KAFKA_CONSUMER_GROUPID_PROCESS_TASKCOMMANDS_MESSAGE | task-commands-message | Consumer group ID for processing task commands |
KAFKA_CONSUMER_GROUPID_PROCESSVERSIONCOMMANDS_MESSAGE | process-version-commands-message | Consumer group ID for processing process version commands messages |
KAFKA_CONSUMER_GROUPID_GENERICCOMMANDS | generic-commands-message | Consumer group ID for processing generic commands messages |
KAFKA_CONSUMER_GROUPID_USERBROADCASTCOMMANDS | user-broadcast-commands-message | Consumer group ID for processing user broadcast commands messages |
Threads
The configuration parameters βKAFKA_CONSUMER_THREADS_*β are utilized to specify the number of threads assigned to Kafka consumers for processing messages from topics. These parameters allow you to fine-tune the concurrency and parallelism of your Kafka consumer application, enabling efficient and scalable message consumption from Kafka topics.
Configuration Parameter | Default value | Description |
---|
KAFKA_CONSUMER_THREADS_PROCESSENGINECOMMANDS_MESSAGE | 10 | Number of threads for processing engine commands messages |
KAFKA_CONSUMER_THREADS_PROCESSENGINECOMMANDS_DISCONNECT | 5 | Number of threads for processing engine commands disconnect messages |
KAFKA_CONSUMER_THREADS_PROCESSENGINECOMMANDS_CONNECT | 5 | Number of threads for processing engine commands connect messages |
KAFKA_CONSUMER_THREADS_TASKCOMMANDS | 10 | Number of threads for task commands |
KAFKA_CONSUMER_THREADS_PROCESSVERSIONCOMMANDS | 10 | Number of threads for processing process version commands messages |
KAFKA_CONSUMER_THREADS_GENERICCOMMANDS | 10 | Number of threads for processing generic commands messages |
KAFKA_CONSUMER_THREADS_USERBROADCASTCOMMANDS | 10 | Number of threads for processing user broadcast commands messages |
Configuration Parameter | Default value |
---|
KAFKA_TOPIC_EVENTSGATEWAY_PROCESSINSTANCE_IN_MESSAGE | ai.flowx.dev.eventsgateway.engine.commands.message.v1 |
KAFKA_TOPIC_EVENTSGATEWAY_PROCESSINSTANCE_IN_DISCONNECT | ai.flowx.dev.eventsgateway.engine.commands.disconnect.v1 |
KAFKA_TOPIC_EVENTSGATEWAY_PROCESSINSTANCE_IN_CONNECT | ai.flowx.dev.eventsgateway.engine.commands.connect.v1 |
Configuration Parameter | Default value |
---|
KAFKA_TOPIC_EVENTSGATEWAY_PROCESSVERSION_IN_MESSAGE | ai.flowx.dev.eventsgateway.process-version-commands.message.v1 |
Configuration Parameter | Default value |
---|
KAFKA_TOPIC_EVENTSGATEWAY_USERMESSAGES_IN_MESSAGE | ai.flowx.dev.core.designer.notification.user.v1 |
Configuring authorization & access roles
Set the following environment variables to connect to the identity management platform:
Configuration Parameter | Description |
---|
SECURITY_OAUTH2_BASESERVERURL | Base URL of the OAuth2 server |
SECURITY_OAUTH2_CLIENT_CLIENTID | Client ID for OAuth2 authentication |
SECURITY_OAUTH2_CLIENT_CLIENTSECRET | Client secret for OAuth2 authentication |
SECURITY_OAUTH2_REALM | Realm for OAuth2 authentication |
Redis configuration
FlowX Events Gateway supports three Redis deployment modes. The process engine sends messages to the events-gateway, which is responsible for sending them to Redis.
Standalone mode (default)
For single Redis instance deployments:
Configuration Parameter | Description |
---|
SPRING_DATA_REDIS_HOST | Hostname of the Redis server |
SPRING_DATA_REDIS_PASSWORD | Password for Redis server |
SPRING_DATA_REDIS_PORT | Connect to the Redis server |
Sentinel mode
For high-availability Redis with automatic failover:
Configuration Parameter | Description | Example Value |
---|
SPRING_DATA_REDIS_SENTINEL_MASTER | Name of the Redis master instance | mymaster |
SPRING_DATA_REDIS_SENTINEL_NODES | Comma-separated list of Sentinel nodes | host1:26379,host2:26379,host3:26379 |
SPRING_DATA_REDIS_PASSWORD | Password for Redis authentication | |
Cluster mode
For Redis cluster deployments with data sharding:
Configuration Parameter | Description | Example Value |
---|
SPRING_DATA_REDIS_CLUSTER_NODES | Comma-separated list of cluster nodes | host1:6379,host2:6379,host3:6379 |
SPRING_DATA_REDIS_PASSWORD | Password for Redis authentication | yourpassword |
Starting with FlowX 5.x, Master-Replica mode is no longer supported. Use Sentinel mode for high availability with automatic failover, or Cluster mode for horizontal scaling.
Redis Deployment Modes:
- Standalone: Single Redis instance (suitable for development and low-traffic environments)
- Sentinel: High availability with automatic failover (recommended for production)
- Cluster: Horizontal scaling with data sharding (for high-throughput applications)
Choose the appropriate mode based on your availability and scalability requirements.
This configuration helps manage how event data is stored and accessed in Redis.
Configuration Parameter | Default | Description |
---|
EVENTS_REDIS_FREQUENCYMILLIS | 200 | Time interval (in milliseconds) between Redis queries by the events gateway to check for new messages |
EVENTS_REDIS_TTLHOURS | 4 | Sets the time-to-live for events in Redis to 4 hours |
Configuring logging
The following environment variables could be set in order to control log levels:
Configuration Parameter | Description |
---|
LOGGING_LEVEL_ROOT | Logging level for the root Spring Boot microservice logs |
LOGGING_LEVEL_APP | Logging level for the application-level logs |