​
Prerequisites

Before deployment, ensure you have the following components available:

ComponentVersionPurpose
Kubernetes1.19+Container orchestration
PostgreSQL13+Advancing data source
MongoDB4.4+Integration configurations
Kafka2.8+Event-driven communication
OAuth2 Server-Authentication (Keycloak recommended)

​
Core configuration environment variables

​
Basic service configuration

Environment VariableDescriptionExample Value
CONFIG_PROFILESpring configuration profilesk8stemplate_v2,kafka-auth
LOGGING_LEVEL_APPApplication logging levelINFO

​
Database configuration

PostgreSQL environment variables
Environment VariableDescriptionExample Value
ADVANCING_DATASOURCE_URLPostgreSQL JDBC URLjdbc:postgresql://postgresql:5432/advancing
ADVANCING_DATASOURCE_USERNAMEDatabase usernameflowx
ADVANCING_DATASOURCE_PASSWORDDatabase passwordsecurePassword
ADVANCING_DATASOURCE_DRIVER_CLASS_NAMEJDBC driverorg.postgresql.Driver
MongoDB environment variables

| Environment Variable | Description | Example Value | | MONGODB_URI | Integration Designer MongoDB URI | mongodb://mongodb-0.mongodb-headless:27017/integration-designer | | MONGODB_USERNAME | MongoDB username | integration-designer | | MONGODB_PASSWORD | MongoDB password | secureMongoPass | | MONGODB_RUNTIME_URI | Runtime MongoDB URI | mongodb://mongodb-0.mongodb-headless:27017/app-runtime | | MONGODB_RUNTIME_USERNAME | Runtime MongoDB username | app-runtime | | MONGODB_RUNTIME_PASSWORD | Runtime MongoDB password | secureRuntimePass |

​
Kafka configuration environment variables

​
Kafka connection and security variables

Environment VariableDescriptionExample Value
SPRING_KAFKA_BOOTSTRAP_SERVERSKafka broker addresseslocalhost:9092
SPRING_KAFKA_SECURITY_PROTOCOLSecurity protocolPLAINTEXT or SASL_PLAINTEXT

​
Kafka producer configuration

Environment VariableDescriptionExample Value
KAFKA_MESSAGE_MAX_BYTESMaximum message size52428800 (50MB)
SPRING_KAFKA_PRODUCER_KEY_SERIALIZERKey serializer classorg.apache.kafka.common.serialization.StringSerializer
SPRING_KAFKA_PRODUCER_VALUE_SERIALIZERValue serializer classorg.springframework.kafka.support.serializer.JsonSerializer

​
Kafka consumer configuration

Environment VariableDescriptionExample Value
KAFKA_CONSUMER_GROUP_ID_START_WORKFLOWSStart workflows consumer groupstart-workflows-group
KAFKA_CONSUMER_GROUP_ID_RES_ELEM_USAGE_VALIDATIONResource usage validation consumer groupintegration-designer-res-elem-usage-validation-group
KAFKA_CONSUMER_THREADS_START_WORKFLOWSStart workflows consumer threads3
KAFKA_CONSUMER_THREADS_RES_ELEM_USAGE_VALIDATIONResource usage validation consumer threads3

​
Kafka topic naming variables

Environment VariableDescriptionExample Value
KAFKA_TOPIC_NAMING_PACKAGEBase package for topicsai.flowx.
KAFKA_TOPIC_NAMING_ENVIRONMENTEnvironment identifierdev.
KAFKA_TOPIC_NAMING_VERSIONTopic version.v1
KAFKA_TOPIC_NAMING_SEPARATORTopic name separator.
KAFKA_TOPIC_NAMING_SEPARATOR2Alternative separator-

​
OAuth authentication variables (when using SASL_PLAINTEXT)

Environment VariableDescriptionExample Value
KAFKA_OAUTH_CLIENT_IDOAuth client IDkafka
KAFKA_OAUTH_CLIENT_SECRETOAuth client secretkafka-secret
KAFKA_OAUTH_TOKEN_ENDPOINT_URIOAuth token endpointkafka.auth.localhost

​
Predefined Kafka topics

Environment VariableDescriptionExample Value
KAFKA_TOPIC_AUDIT_OUTAudit topicai.flowx.dev.core.trigger.save.audit.v1
KAFKA_TOPIC_EVENTS_GATEWAY_OUT_MESSAGEEvents gateway workflow instances topicai.flowx.dev.eventsgateway.receive.workflowinstances.v1
KAFKA_TOPIC_APPLICATION_IN_RES_ELEM_USAGE_VALIDATIONResource usage validation topicai.flowx.dev.application-version.resources-usages.sub-res-validation.request-integration.v1
KAFKA_TOPIC_RESOURCES_USAGES_REFRESHResources usage refresh topicai.flowx.dev.application-version.resources-usages.refresh.v1

​
Authentication configuration environment variables

Environment VariableDescriptionExample Value
SECURITY_OAUTH2_BASE_SERVER_URLOAuth2 server base URLhttps://keycloak.example.com/auth
SECURITY_OAUTH2_REALMAuthentication realmflowx
SECURITY_OAUTH2_CLIENT_IDClient IDflowx-platform-authorize
SECURITY_OAUTH2_CLIENT_SECRETClient secretplatformClientSecret
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_IDAdmin service account IDintegration-designer-admin
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRETAdmin service account secretadminServiceSecret

​
Monitoring and health check environment variables

Environment VariableDescriptionExample Value
MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDEExposed actuator endpointshealth,info,metrics,prometheus
MANAGEMENT_HEALTH_KUBERNETES_ENABLEDKubernetes health checktrue
MANAGEMENT_METRICS_EXPORT_PROMETHEUS_ENABLEDPrometheus metricstrue

​
Resource usage monitoring

Environment VariableDescriptionExample Value
LIB_RESOURCES_USAGES_ENABLEDEnable resource usage trackingtrue
LIB_RESOURCES_USAGES_REFRESH_LISTENER_ENABLEDEnable refresh listenertrue
LIB_RESOURCES_USAGES_COLLECTOR_THREAD_COUNTCollector thread count5
LIB_RESOURCES_USAGES_COLLECTOR_MAX_BATCH_SIZEMax batch size1000

​
Kafka topics configuration

​
Topic naming convention

Integration Designer uses a structured topic naming convention:

ai.flowx.{environment}.{service}.{action}.{detail}.v1

Key components:

  • ai.flowx.: Base package identifier
  • {environment}: Deployment environment (e.g., dev, prod)
  • {service}: Specific service (e.g., core, eventsgateway)
  • {action}: Operation type (e.g., receive, trigger)
  • {detail}: Specific topic detail
  • v1: Version identifier

​
Important Kafka topics

  1. Workflow Instances Topic

    • Pattern: ai.flowx.{env}.eventsgateway.receive.workflowinstances.v1
    • Purpose: Handles workflow instance messages
    • Recommended configuration:
      • Retention: 7 days
      • Partition count: Minimum 3

  2. Engine Receive Topics

    • Pattern: ai.flowx.{env}.engine.receive.*
    • Purpose: Engine service communication
    • Recommended configuration:
      • Retention: 3 days
      • Partition count: Minimum 3

  3. Integration Receive Topics

    • Pattern: ai.flowx.{env}.integration.receive.*
    • Purpose: Integration service communication
    • Recommended configuration:
      • Retention: 3 days
      • Partition count: Minimum 3

  4. Resource Usage Topics

    • Validation Request: ai.flowx.{env}.application-version.resources-usages.sub-res-validation.request-integration.v1
    • Refresh: ai.flowx.{env}.application-version.resources-usages.refresh.v1
    • Purpose: Resource tracking and validation
    • Recommended configuration:
      • Retention: 1 day
      • Partition count: 1-2

​
Large message handling for workflow instances topic

Updating max.message.bytes configuration

The workflow instances topic requires special configuration to handle large messages. By default, Kafka has message size limitations that may prevent Integration Designer from processing large workflow payloads.

Recommended max.message.bytes value: 10485760 (10 MB)

Method 1: Update using AKHQ (Recommended)

  1. Access AKHQ

    • Open the AKHQ web interface
    • Log in if authentication is required

  2. Navigate to Topic

    • Go to the “Topics” section
    • Find the topic: ai.flowx.dev.eventsgateway.receive.workflowinstances.v1

  3. Edit Configuration

    • Click on the topic name
    • Go to the “Configuration” tab
    • Locate or add max.message.bytes
    • Set the value to 10485760
    • Save changes

​
Additional Resources

Was this page helpful?

Suggest editsRaise issue
Runtime manager setupFlowX Advancing Controller setup