This guide provides step-by-step instructions to set up and configure the Integration Designer service, including database, Kafka, and OAuth2 authentication settings, to ensure integration and data flow management
CONFIG_PROFILE
- This environment variable must be set explicitly and exactly to ensure that no unintended profiles are loaded by mistake. The value of this variable should represent a minimal configuration state, relying only on defaults specified in the application.properties
file of the application.ADVANCING_DATASOURCE_URL
- Database URL for the advancing data source in PostgreSQLADVANCING_DATASOURCE_USERNAME
- Username for the advancing data source in PostgreSQLADVANCING_DATASOURCE_PASSWORD
- The password for the advancing data source in PostgreSQL.
ADVANCING_DATASOURCE_DRIVER_CLASS_NAME
- The database driver class name for the advancing data source
org.postgresql.Driver
oracle.jdbc.OracleDriver
integration-designer
database is dedicated to Integration Designer, while the shared app-runtime
database supports multiple services.
integration-designer
): Stores data specific to Integration Designer, such as integration configurations, metadata, and other operational data.app-runtime
): Shared across multiple services, this database manages runtime data essential for integration and data flow execution.SPRING_DATA_MONGODB_URI
- URI for connecting to the Integration Designer MongoDB instance
mongodb://${DB_USERNAME}:${DB_PASSWORD}@<host1>,<host2>,<arbiter-host>:<port>/${DB_NAME}?retryWrites=false
DB_USERNAME
: integration-designer
DB_NAME
: integration-designer
DB_PASSWORD
: DB password.SPRING_DATA_MONGODB_STORAGE
- Specifies the storage type used for the Runtime MongoDB instance (Azure environments only)
mongodb
, cosmosdb
mongodb
SPRING_DATA_MONGODB_RUNTIME_URI
- URI for connecting to MongoDB for Runtime MongoDB (app-runtime
)
SPRING_DATA_MONGODB_RUNTIME_URI
: mongodb://${RUNTIME_DB_USERNAME}:${RUNTIME_DB_PASSWORD}@<host1>,<host2>,<arbiter-host>:<port>/${RUNTIME_DB_NAME}?retryWrites=false
RUNTIME_DB_USERNAME
: app-runtime
RUNTIME_DB_NAME
: app-runtime
RUNTIME_DB_PASSWORD
: DB password.Environment variable | Description | Default value | Possible values |
---|---|---|---|
FLOWX_CONFIG_PARAMS_VARS_PROVIDER_TYPE | Provider type for environment variables | k8s | k8s |
FLOWX_CONFIG_PARAMS_SECRETS_PROVIDER_TYPE | Provider type for secrets | k8s | k8s |
FLOWX_CONFIG_PARAMS_K8S_SECRETS_0 | First Kubernetes secret name | flowx-rt | Any valid Kubernetes secret name |
FLOWX_CONFIG_PARAMS_K8S_CONFIGMAPS_0 | First Kubernetes ConfigMap name | flowx-rt | Any valid Kubernetes ConfigMap name |
FLOWX_CONFIG_PARAMS_K8S_SECRETS_1
, FLOWX_CONFIG_PARAMS_K8S_CONFIGMAPS_1
). Values are overridden based on the order in which the maps are defined.FLOWX_CONFIG_PARAMS_K8S_SECRETS_0
(default: flowx-rt
)FLOWX_CONFIG_PARAMS_K8S_CONFIGMAPS_0
(default: flowx-rt
)SPRING_KAFKA_BOOTSTRAP_SERVERS
- Address of the Kafka server in the format host:port
KAFKA_TOPIC_NAMING_ENVIRONMENT
- Environment-specific suffix for Kafka topicsFLOWX_WORKFLOW_CREATETOPICS
- To automatically create kafka topics for development environments
KAFKA_CONSUMER_GROUP_ID_START_WORKFLOWS
- Consumer group ID for starting workflows
start-workflows-group
KAFKA_CONSUMER_THREADS_START_WORKFLOWS
- Number of Kafka consumer threads for starting workflows
3
KAFKA_AUTH_EXCEPTION_RETRY_INTERVAL
- Interval (in seconds) between retries after an AuthorizationException
10
Component | Description | Default Value |
---|---|---|
package | Package identifier for namespace | ai.flowx. |
environment | Environment identifier | dev. |
version | Version identifier for topic compatibility | .v1 |
separator | Primary separator for components | . |
separator2 | Secondary separator for additional distinction | - |
prefix | Combines package and environment as a topic prefix | ${kafka.topic.naming.package}${kafka.topic.naming.environment} |
suffix | Appends version to the end of the topic name | ${kafka.topic.naming.version} |
kafka.topic.naming.engineReceivePattern
engine${dot}receive${dot}
ai.flowx.dev.engine.receive.
kafka.topic.naming.integrationReceivePattern
integration${dot}receive${dot}
ai.flowx.dev.integration.receive.
${kafka.topic.naming.prefix}eventsgateway${dot}receive${dot}workflowinstances${kafka.topic.naming.suffix}
ai.flowx.dev.eventsgateway.receive.workflowinstances.v1
max.message.bytes
configuration should be updated for this topic to ensure compatibility with Integration Designer message sizes.How to Update `max.message.bytes` Using AKHQ
ai.flowx.dev.eventsgateway.receive.workflowinstances.v1
.max.message.bytes
setting. If it’s not present, add it.max.message.bytes
(e.g., 10485760
for 10 MB).${kafka.topic.naming.prefix}${kafka.topic.naming.engineReceivePattern}
ai.flowx.dev.engine.receive.*
${kafka.topic.naming.prefix}${kafka.topic.naming.integrationReceivePattern}*
ai.flowx.dev.integration.receive.*
FLOWX_WEBCLIENT_BUFFERSIZE
- Specifies the buffer size (in bytes) for the FlowX WebClient. Default value 1048576 (1MB)
FLOWX_WEBCLIENT_BUFFERSIZE=10485760
. This ensures smooth handling of large API responses, particularly for legacy APIs without pagination support.SECURITY_OAUTH2_BASE_SERVER_URL
- Base URL for the OAuth 2.0 Authorization ServerSECURITY_OAUTH2_CLIENT_CLIENT_ID
- Unique identifier for the client application registered with the OAuth 2.0 serverSECURITY_OAUTH2_CLIENT_CLIENT_SECRET
- Secret key for authenticating requests made by the authorization clientSECURITY_OAUTH2_REALM
- The realm name for OAuth2 authenticationSECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_ID
- Client ID for the integration designer service accountSECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRET
- Client Secret for the integration designer service accountSECURITY_OAUTH2_BASE_SERVER_URL
- Base URL for the OAuth2 authorization serverSECURITY_OAUTH2_REALM
- Realm for OAuth2 authenticationSECURITY_OAUTH2_CLIENT_CLIENT_ID
- Client ID for the Integration Designer OAuth2 clientSECURITY_OAUTH2_CLIENT_CLIENT_SECRET
- Client Secret for the Integration Designer OAuth2 clientSECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_ID
- Client ID for the Keycloak admin service accountSECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRET
- Client Secret for the Keycloak admin service accountLOGGING_LEVEL_ROOT
- The log level for root Spring Boot microservice logsLOGGING_LEVEL_APP
- The log level for application-level logstrue
to enable the admin ingress route.MANAGEMENT_PROMETHEUS_METRICS_EXPORT_ENABLED
- Enables or disables Prometheus metrics export (default: false).rbac.create
: Set to true to create RBAC resources.rbac.rules
: Define custom RBAC rules as follows:get
, list
, watch
) to ConfigMaps, Secrets, and Pods, which is essential for retrieving application settings and credentials required by Integration Designer.