Scheduler setup

Infrastructure prerequisites

MongoDB
Kafka
OpenID Connect Settings

Dependencies

MongoDB database
Ability to connect to a Kafka instance used by the FlowX Engine
Scheduler service account - required for using Start Timer event node. For more information, see the scheduler service account configuration.

The service comes with most of the required configuration properties pre-filled. However, certain custom environment variables need to be set up.

This service needs to connect to a Mongo database that has replicas to work correctly.

Scheduler configuration

Scheduler

scheduler:
  thread-count: 30  # Configure the number of threads to be used for sending expired messages.
  callbacks-thread-count: 60 # Configure the number of threads for handling Kafka responses, whether the message was successfully sent or not
  cronExpression: "*/10 * * * * *" #every 10 seconds
  retry: # new retry mechanism
    max-attempts: 3
    seconds: 1
    thread-count: 3
    cronExpression: "*/10 * * * * *" #every 10 seconds
  cleanup:
    cronExpression: "*/25 * * * * *" #every 25 seconds

SCHEDULER_THREAD_COUNT: Used to configure the number of threads to be used for sending expired.
SCHEDULER_CALLBACKS_THREAD_COUNT: Used to configure the number of threads for handling Kafka responses, whether the message was successfully sent or not.

The “scheduler.cleanup.cronExpression” is valid for both scheduler and timer event scheduler.

Retry mechanism

SCHEDULER_RETRY_THREAD_COUNT: Specify the number of threads to use for resending messages that need to be retried.
SCHEDULER_RETRY_MAX_ATTEMPTS: This configuration parameter sets the number of retry attempts. For instance, if it’s set to 3, it means that the system will make a maximum of three retry attempts for message resending.
SCHEDULER_RETRY_SECONDS: This configuration parameter defines the time interval, in seconds, for retry attempts. For example, when set to 1, it indicates that the system will retry the operation after a one-second delay.

Cleanup

SCHEDULER_CLEANUP_CRONEXPRESSION: It specifies how often, in seconds, events that have already been processed should be cleaned up from the database.

Recovery mechanism

flowx:
  timer-calculator:
    delay-max-repetitions: 1000000

You have a “next execution” set for 10:25, and the cycle step is 10 minutes. If the instance goes down for 2 hours, the next execution time should be 12:25, not 10:35. To calculate this, you add 10 minutes repeatedly to 10:25 until you reach the current time. So, it would be 10:25 + 10 min + 10 min + 10 min, until you reach the current time of 12:25. This ensures that the next execution time is adjusted correctly after the downtime.

FLOWX_TIMER_CALCULATOR_DELAY_MAX_REPETITIONS: This means that, for example, if our cycle step is set to one second and the system experiences a downtime of two weeks, which is equivalent to 1,209,600 seconds, and we have the “max repetitions” set to 1,000,000, it will attempt to calculate the next schedule. However, when it reaches the maximum repetitions, an exception is thrown, making it impossible to calculate the next schedule. As a result, the entry remains locked and needs to be rescheduled. This scenario represents a critical case where the system experiences extended downtime, and the cycle step is very short (e.g., 1 second), leading to the inability to determine the next scheduled event.

Timer event scheduler

Configuration for Timer Event scheduler designed to manage timer events. Similar configuration to scheduler.

timer-event-scheduler:
  thread-count: 30
  callbacks-thread-count: 60
  cronExpression: "*/1 * * * * *" #every 1 seconds
  retry:
    max-attempts: 3
    seconds: 1
    thread-count: 3
    cronExpression: "*/5 * * * * *" #every 5 seconds

OpenID connect settings

Environment variable	Description	Default value
`SECURITY_TYPE`	Token validation mechanism (JWT public key validation)	`jwt-public-key`
`SECURITY_OAUTH2_BASESERVERURL`	Base URL of the Keycloak server
`SECURITY_OAUTH2_SAREALM`	Service-accounts realm ID	`00000002-0002-4002-8002-000000000002`
`FLOWX_LIB_SECURITY_SERVICES_ORGANIZATIONMANAGER_BASEURL`	URL of the organization-manager service, used by the security library	`http://organization-manager:80`
`SECURITY_PATHAUTHORIZATIONS_0_PATH`	Security path pattern	`/api/**`
`SECURITY_PATHAUTHORIZATIONS_0_ROLESALLOWED`	Roles allowed for path access	`ANY_AUTHENTICATED_USER`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID`	Service account client ID	`flowx-scheduler-core-sa`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET`	Service account client secret (Keycloak-issued)
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTID`	Anonymous service account client ID, used for anonymous runtime access	`flowx-anonymous-sa`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTSECRET`	Anonymous service account client secret
`SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKENURI`	Provider token URI, resolved against the service-accounts realm	`${SECURITY_OAUTH2_BASESERVERURL}/realms/${SECURITY_OAUTH2_SAREALM}/protocol/openid-connect/token`

Upgrading from 5.1.x? Remove the legacy opaque-token env vars: SECURITY_OAUTH2_REALM, SECURITY_OAUTH2_CLIENT_CLIENTID, SECURITY_OAUTH2_CLIENT_CLIENTSECRET, and SECURITY_OAUTH2_SERVICEACCOUNT_ADMIN_*. These belong to the removed introspection model and prevent the service from starting on 5.9.x. See the authentication and IAM migration guide for the full list.

When deploying with the FlowX Helm chart, SECURITY_OAUTH2_BASESERVERURL is supplied through the chart value flowx.keycloak.baseServerUrl and the service-account client secrets are injected from the chart-managed Keycloak secret. The remaining values ship as image defaults.

The service account is essential for enabling the Start Timer event node. More details about the necessary service account, here: Scheduler service account

Configuring datasoruce (MongoDB)

The MongoDB database is used to persist scheduled messages until they are sent back. The following configurations need to be set using environment variables:

SPRING_DATA_MONGODB_URI: The URI for the MongoDB database.

Configuring Kafka

Core Kafka settings

Environment Variable	Description	Default Value
`KAFKA_BOOTSTRAP_SERVERS`	Kafka broker addresses (fallback: `SPRING_KAFKA_BOOTSTRAP_SERVERS`)	`localhost:9092`
`KAFKA_SECURITY_PROTOCOL`	Security protocol for Kafka connections (fallback: `SPRING_KAFKA_SECURITY_PROTOCOL`)	`PLAINTEXT`
`SPRING_KAFKA_CONSUMER_GROUPID`	Consumer group identifier	`scheduler-consumer`
`KAFKA_MESSAGE_MAX_BYTES`	Maximum message size (bytes)	`52428800` (50 MB)
`KAFKA_AUTHEXCEPTIONRETRYINTERVAL`	Retry interval after authorization exceptions (seconds)	`10`

Consumer configuration

Environment Variable	Description	Default Value
`KAFKA_CONSUMER_THREADS`	Number of Kafka consumer threads	`1`
`KAFKA_CONSUMER_SCHEDULEDTIMEREVENTS_THREADS`	Number of threads for starting Timer Events	`1`
`KAFKA_CONSUMER_SCHEDULEDTIMEREVENTS_GROUPID`	Consumer group for starting timer events	`scheduled-timer-events`
`KAFKA_CONSUMER_STOPSCHEDULEDTIMEREVENTS_THREADS`	Number of threads for stopping Timer Events	`1`
`KAFKA_CONSUMER_STOPSCHEDULEDTIMEREVENTS_GROUPID`	Consumer group for stopping timer events	`stop-scheduled-timer-events`

OAuth authentication (when using SASL_PLAINTEXT)

Environment Variable	Description	Default Value
`KAFKA_OAUTH_CLIENT_ID`	OAuth client ID	`kafka`
`KAFKA_OAUTH_CLIENT_SECRET`	OAuth client secret	`kafka-secret`
`KAFKA_OAUTH_TOKEN_ENDPOINT_URI`	OAuth token endpoint	`kafka.auth.localhost`

When using the kafka-auth profile, the security protocol will automatically be set to SASL_PLAINTEXT and the SASL mechanism will be set to OAUTHBEARER.

Topic naming configuration

Environment Variable	Description	Default Value
`KAFKA_TOPIC_NAMING_PACKAGE`	Package prefix for topic names	`ai.flowx.`
`KAFKA_TOPIC_NAMING_ENVIRONMENT`	Environment segment for topic names
`KAFKA_TOPIC_NAMING_VERSION`	Version suffix for topic names	`.v1`
`KAFKA_TOPIC_NAMING_SEPARATOR`	Primary separator for topic names	`.`
`KAFKA_TOPIC_NAMING_SEPARATOR2`	Secondary separator for topic names	`-`

Kafka topics

Schedule topics

Environment Variable	Description	Default Value
`KAFKA_TOPIC_SCHEDULE_IN_SET`	Receives scheduled message setting requests	`ai.flowx.core.trigger.set.schedule.v1`
`KAFKA_TOPIC_SCHEDULE_IN_STOP`	Handles requests to terminate scheduled messages	`ai.flowx.core.trigger.stop.schedule.v1`

Timer events topics

Environment Variable	Description	Default Value
`KAFKA_TOPIC_SCHEDULEDTIMEREVENTS_IN_SET`	Topic for setting timer events	`ai.flowx.core.trigger.set.timer-event-schedule.v1`
`KAFKA_TOPIC_SCHEDULEDTIMEREVENTS_IN_STOP`	Topic for stopping timer events	`ai.flowx.core.trigger.stop.timer-event-schedule.v1`

Make sure the topics configured for this service don’t follow the engine pattern.

Configuring logging

The following environment variables can be set to control log levels:

LOGGING_LEVEL_ROOT: Root Spring Boot microservice logs (Default: INFO)
LOGGING_LEVEL_APP: App level logs (Default: INFO)

Troubleshooting

Common issues

Scheduled processes not triggering

Symptoms: Scheduled processes do not start at the expected times.Solutions:

Verify the cron expressions in SCHEDULER_CLEANUP_CRONEXPRESSION and the scheduler cronExpression settings are correct
Check that Kafka topics (KAFKA_TOPIC_SCHEDULE_IN_SET, KAFKA_TOPIC_SCHEDULE_IN_STOP) are correctly configured and accessible
Confirm the scheduler service is healthy by checking /actuator/health
Review scheduler logs at DEBUG level for missed or skipped executions

Duplicate schedule executions

Symptoms: Scheduled messages are sent multiple times for the same event.Solutions:

Check the replica count — running multiple replicas without proper leader election can cause duplicates
Verify that the MongoDB replica set is healthy, as the scheduler relies on it for distributed locking
Review SCHEDULER_THREAD_COUNT and SCHEDULER_CALLBACKS_THREAD_COUNT to ensure they are not excessively high for your workload

Scheduler fails to start

Symptoms: The scheduler service crashes or fails during startup.Solutions:

Verify MongoDB connectivity and ensure the database has replicas enabled (required for the scheduler to work correctly)
Check that Kafka bootstrap servers are reachable (KAFKA_BOOTSTRAP_SERVERS)
Confirm the service account credentials are valid — verify SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID and SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET
Review startup logs for specific connection error messages

Timer events not firing

Symptoms: Timer event nodes in processes do not trigger at the configured times.Solutions:

Verify the timer event configuration in the process definition (Start Timer, Intermediate Timer)
Check that the timer event Kafka topics are correctly set (KAFKA_TOPIC_SCHEDULEDTIMEREVENTS_IN_SET, KAFKA_TOPIC_SCHEDULEDTIMEREVENTS_IN_STOP)
Ensure the scheduler service account has the necessary permissions — see the Scheduler service account configuration
Review the FLOWX_TIMER_CALCULATOR_DELAY_MAX_REPETITIONS setting if the system experienced recent downtime

Scheduled Processes

Configure and manage scheduled process executions

Timer Events

Timer event node types and configuration options

Redis Configuration

Complete Redis setup including Sentinel and Cluster modes

Microservices

AI Platform

Plugins

Observability

Access management

Scheduler setup

Infrastructure prerequisites

Dependencies

Scheduler configuration

Scheduler

Retry mechanism

Cleanup

Recovery mechanism

Timer event scheduler

OpenID connect settings

Configuring datasoruce (MongoDB)

Configuring Kafka

Core Kafka settings

Consumer configuration

OAuth authentication (when using SASL_PLAINTEXT)

Topic naming configuration

Kafka topics

Schedule topics

Timer events topics

Configuring logging

Troubleshooting

Common issues

Scheduled Processes

Timer Events

Redis Configuration

​Infrastructure prerequisites

​Dependencies

​Scheduler configuration

​Scheduler

​Retry mechanism

​Cleanup

​Recovery mechanism

​Timer event scheduler

​OpenID connect settings

​Configuring datasoruce (MongoDB)

​Configuring Kafka

​Core Kafka settings

​Consumer configuration

​OAuth authentication (when using SASL_PLAINTEXT)

​Topic naming configuration

​Kafka topics

​Schedule topics

​Timer events topics

​Configuring logging

​Troubleshooting

​Common issues

​Related resources

Scheduled Processes

Timer Events

Redis Configuration

Infrastructure prerequisites

Dependencies

Scheduler configuration

Scheduler

Retry mechanism

Cleanup

Recovery mechanism

Timer event scheduler

OpenID connect settings

Configuring datasoruce (MongoDB)

Configuring Kafka

Core Kafka settings

Consumer configuration

OAuth authentication (when using SASL_PLAINTEXT)

Topic naming configuration

Kafka topics

Schedule topics

Timer events topics

Configuring logging

Troubleshooting

Common issues

Related resources