FlowX Engine setup

​

Infrastructure prerequisites

Before installing the FlowX.AI Engine, verify that the following infrastructure components are installed and configured:

• Kafka: Version 2.8 or higher
• Elasticsearch: Version 7.11.0 or higher
• PostgreSQL: Version 13 or higher for storing application data
• MongoDB: Version 4.4 or higher for managing runtime builds

​

Dependencies

The FlowX Engine requires the following components:

• Database: Primary storage for the engine
• Redis Server: Used for caching. See Redis Configuration
• Kafka: Handles messaging and event-driven architecture. See Configuring Kafka

For a microservices architecture, services typically manage their data via dedicated databases.

​

Required external services

• Redis Cluster: Caches process definitions, compiled scripts, and Kafka responses
• Kafka Cluster: Enables communication with external plugins and integrations

​

Configuration setup

FlowX.AI Engine uses environment variables for configuration. This section covers key configuration areas:

• Database configuration
• Script engine configuration
• Authorization and access roles
• Kafka configuration
• Advancing controller configuration
• Cleanup mechanism configuration
• File upload configuration
• Application management
• RBAC configuration

​

Database configuration

​

PostgreSQL

​

MongoDB configuration

Configure connection to the Runtime MongoDB instance:

​

Configuring script engine

FlowX.AI Engine supports multiple scripting languages for business rules and actions.

​

Python runtime configuration

​

GraalVM cache configuration

When using GraalVM (FLOWX_SCRIPTENGINE_USEGRAALVM=true), ensure the engine has proper access to a cache directory within the container. By default, this is configured in the /tmp directory.

There are two methods to configure the GraalVM cache location:

​

Option 1: Using Java options (Preferred)

Add the following Java option to your deployment configuration:

-Dpolyglot.engine.userResourceCache=/tmp

This option is set by default in the standard Docker image but might need to be included if you’re overriding the Java options.

​

Option 2: Using environment variables

Alternatively, set the following environment variable:

XDG_CACHE_HOME=/tmp

For more details about supported scripting languages and their capabilities, see the Supported scripts documentation.

​

Authorization & access roles

This section covers OAuth2 configuration settings for securing the Spring application.

​

Resource server settings (OAuth2 configuration)

​

Service account settings

Configure the process engine service account:

For more information about the necessary service account, see Process Engine Service Account.

​

Security configuration

​

Configuring Kafka

Kafka handles all communication between the FlowX.AI Engine, external plugins, and integrations. It also notifies running process instances when certain events occur.

​

Kafka connection settings

​

Message routing configuration

​

Kafka consumer retry settings

​

Consumer groups & consumer threads configuration

Both a producer and a consumer must be configured:

​

About consumer groups and threads

A consumer group is a set of consumers that jointly consume messages from one or more Kafka topics. Each consumer group has a unique identifier (group ID) that Kafka uses to manage message distribution.

Thread numbers refer to the number of threads a consumer application uses to process messages. Increasing thread count can improve parallelism and efficiency, especially with high message volumes.

​

Consumer group configuration

​

Consumer thread configuration

All events that start with a configured pattern will be consumed by the Engine. This enables you to create new integrations and connect them to the engine without changing the configuration.

​

Configuring Kafka topics

topic:
  naming:
    package: "ai.flowx."
    environment: "dev."
    version: ".v1"
    prefix: ${kafka.topic.naming.package}${kafka.topic.naming.environment}
    suffix: ${kafka.topic.naming.version}
    engineReceivePattern: engine.receive.
    integrationReceivePattern: integration.receive.

  pattern: ${kafka.topic.naming.prefix}${kafka.topic.naming.engineReceivePattern}*
  integrationPattern: ${kafka.topic.naming.prefix}${kafka.topic.naming.integrationReceivePattern}*

​

Core engine topics

​

Topics related to the Task Management plugin

OPERATIONS_IN request example

{
  "operationType": "UNASSIGN", //type of operation performed in Task Management plugin
  "taskId": "some task id",
  "processInstanceUuid": "1cff0b7d-966b-4b35-9e9b-63b1d6757ec6",
  "swimlaneName": "Default",
  "swimlaneId": "51ec1241-fe06-4576-9c84-31598c05c527",
  "owner": {
    "firstName": null,
    "lastName": null,
    "username": "service-account-flowx-process-engine-account",
    "enabled": false
  },
  "author": "admin@flowx.ai"
}

BULK_IN request example

{
  "operations": [
    {
      "operationType": "HOLD",
      "taskId": "some task id",
      "processInstanceUuid": "d3aabfd8-d041-4c62-892f-22d17923b223", // the id of the process instance
      "swimlaneName": "Default", //name of the swimlane
      "owner": null,
      "author": "john.doe@flowx.ai"
    },
    {
      "operationType": "HOLD",
      "taskId": "some task id",
      "processInstanceUuid": "d3aabfd8-d041-4c62-892f-22d17923b223",
      "swimlaneName": "Default", //name of the swimlane
      "owner": null,
      "author": "john.doe@flowx.ai"
    }
  ]
}      

{"processInstanceId": ${processInstanceId}, "callbackTopic": "test.operations.out", "requestID":"1234567890"}

​

Topics related to the scheduler extension

​

Topics related to Timer Events

​

Topics related to the Search Data service

​

Topics related to the Audit service

​

Topics related to ES indexing

​

Processes that can be started by sending messages to a Kafka topic

​

Topics related to Message Events

​

Topics related to Events-gateway microservice

​

Topics related to platform components

​

Inter-service topic coordination

When configuring FlowX services, ensure the following:

1. The Engine’s pattern must match the pattern used by services sending messages to the Engine
2. The integrationPattern must match the pattern used by the Integration Designer
3. Output topics from one service must match the expected input topics of another service

For example:

• Services send to topics matching ai.flowx.dev.engine.receive.* → Engine listens
• Engine sends to topics matching ai.flowx.dev.integration.receive.* → Integration Designer listens

​

Kafka message size configuration

This setting affects:

• Producer message max bytes
• Producer max request size
• Consumer max partition fetch bytes

​

Kafka authentication

For secure environments, you can enable OAuth authentication with the following configuration:

spring.config.activate.on-profile: kafka-auth

spring:
  kafka:
    security.protocol: "SASL_PLAINTEXT"
    properties:
      sasl:
        mechanism: "OAUTHBEARER"
        jaas.config: "org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required oauth.client.id=\"${KAFKA_OAUTH_CLIENT_ID:kafka}\" oauth.client.secret=\"${KAFKA_OAUTH_CLIENT_SECRET:kafka-secret}\"  oauth.token.endpoint.uri=\"${KAFKA_OAUTH_TOKEN_ENDPOINT_URI:kafka.auth.localhost}\" ;"
        login.callback.handler.class: io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler

You need to set the following environment variables:

• KAFKA_OAUTH_CLIENT_ID
• KAFKA_OAUTH_CLIENT_SECRET
• KAFKA_OAUTH_TOKEN_ENDPOINT_URI

​

Configuring file upload size

​

Connecting the Advancing controller

To use the advancing controller, configure the following variables:

​

Configuring the Advancing controller

​

Configuring cleanup mechanism

​

Managing subprocesses expiration

​

Configuring application management

​

RBAC configuration

Process Engine requires specific RBAC permissions for proper access to Kubernetes resources:

rbac:
  create: true
  rules:
    - apiGroups:
        - ""
      resources:
        - secrets
        - configmaps
        - pods
      verbs:
        - get
        - list
        - watch