The Application Manager is a backend microservice in FlowX.AI that:

✅ Manages FlowX applications, versions, manifests, and configurations.
✅ Acts as a proxy for front-end resource requests.

The Application Manager and Runtime Manager share the same container image and Helm chart. Check the Deployment Guidelines in the release notes for version compatibility.

Infrastructure prerequisites

Before you start setting up the Application Manager service, ensure the following infrastructure components are in place:

ComponentVersionPurpose
PostgreSQL13+Storing application data
MongoDB4.4+Managing runtime builds
Redis6.0+Caching needs
Kafka2.8+Event-driven communication
OAuth2 Server-Authentication (Keycloak recommended)

Ensure that the database for storing application data is properly set up and configured before starting the service.

Dependencies

The Application Manager relies on other FlowX services and components to function properly:

Core configuration environment variables

Basic service configuration

Environment VariableDescriptionExample Value
CONFIG_PROFILESpring configuration profilesk8stemplate_v2,kafka-auth
MULTIPART_MAX_FILE_SIZEMaximum file upload size25MB
MULTIPART_MAX_REQUEST_SIZEMaximum request size25MB
LOGGING_CONFIG_FILELogging configuration filelogback-spring.xml

Database configuration

PostgreSQL configuration

Environment VariableDescriptionExample Value
SPRING_DATASOURCE_URLPostgreSQL JDBC URLjdbc:postgresql://postgresql:5432/app_manager
SPRING_DATASOURCE_USERNAMEDatabase usernameflowx
SPRING_DATASOURCE_PASSWORDDatabase passwordpassword
SPRING_DATASOURCE_DRIVER_CLASS_NAMEJDBC driver classorg.postgresql.Driver

MongoDB configuration

The Application Manager requires MongoDB to store runtime build information. Use the following environment variables for configuration:

Environment VariableDescriptionExample Value
SPRING_DATA_MONGODB_URIMongoDB connection URImongodb://${DB_USERNAME}:${DB_PASSWORD}@mongodb-0.mongodb-headless:27017/app-runtime?retryWrites=false
DB_USERNAMEMongoDB usernameapp-runtime
DB_PASSWORDMongoDB passwordpassword
SPRING_DATA_MONGODB_STORAGEStorage type (Azure environments only)mongodb (alternative: cosmosdb)

Redis configuration

If caching is required, configure Redis using the following environment variables:

Environment VariableDescriptionExample Value
SPRING_REDIS_HOSTRedis server hostnameredis-master
SPRING_REDIS_PASSWORDRedis passwordpassword
SPRING_REDIS_PORTRedis server port6379

Kafka configuration

Kafka connection and security variables

Environment VariableDescriptionExample Value
SPRING_KAFKA_BOOTSTRAP_SERVERSKafka broker addresseskafka-flowx-kafka-bootstrap:9092
SPRING_KAFKA_SECURITY_PROTOCOLSecurity protocolPLAINTEXT
KAFKA_TOPIC_NAMING_ENVIRONMENTEnvironment prefixdev.
KAFKA_MESSAGE_MAX_BYTESMaximum message size52428800 (50MB)
FLOWX_KAFKA_PAYLOAD_SIZE_LIMITPayload size limit512000 (500KB)

Kafka producer configuration

Environment VariableDescriptionExample Value
SPRING_KAFKA_PRODUCER_KEY_SERIALIZERKey serializer classorg.apache.kafka.common.serialization.StringSerializer
SPRING_KAFKA_PRODUCER_MAX_REQUEST_SIZEMaximum request size52428800 (50MB)

OAuth authentication variables (when using SASL_PLAINTEXT)

Environment VariableDescriptionExample Value
KAFKA_OAUTH_CLIENT_IDOAuth client IDflowx-service-client
KAFKA_OAUTH_CLIENT_SECRETOAuth client secretflowx-service-client-secret
KAFKA_OAUTH_TOKEN_ENDPOINT_URIOAuth token endpointhttps://auth-pr1180.dev3.flowxai.dev/auth/realms/kafka-authz/protocol/openid-connect/token

Kafka consumer configuration

Environment VariableDescriptionDefault Value
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_EXPORTApplication export consumer groupappResourceExportGroup
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_IMPORTApplication import consumer groupappResourceImportGroup
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_USAGESResource usages consumer groupappResourceUsagesGroup
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_ELEM_VALIDResource element validation groupappResElemUsageValidationResp
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_COPYResource copy consumer groupappResourceCopyGroup
KAFKA_CONSUMER_GROUP_ID_APPLICATION_MERGEApplication merge consumer groupappMergeItemGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_CREATEBuild create consumer groupbuildCreateGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_UPDATEBuild update consumer groupbuildUpdateGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_RESOURCE_EXPORTBuild export consumer groupbuildResourceExportGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_RESOURCE_IMPORTBuild import consumer groupbuildResourceImportGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_START_TIMER_EVENTS_UPDATESBuild timer events updates consumerbuildStartTimerEventsUpdatesGroup
KAFKA_CONSUMER_GROUP_ID_PROCESS_STARTProcess start consumer groupprocessStartGroup
KAFKA_AUTH_EXCEPTION_RETRY_INTERVALAuth exception retry interval (seconds)10

Kafka consumer threads configuration

Environment VariableDescriptionDefault Value
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_EXPORTApplication export consumer threads3
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_IMPORTApplication import consumer threads3
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_USAGESResource usages consumer threads3
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_ELEM_VALID_RESPResource validation response threads3
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_COPYResource copy consumer threads3
KAFKA_CONSUMER_THREADS_APPLICATION_MERGEApplication merge consumer threads3
KAFKA_CONSUMER_THREADS_BUILD_CREATEBuild create consumer threads2
KAFKA_CONSUMER_THREADS_BUILD_UPDATEBuild update consumer threads4
KAFKA_CONSUMER_THREADS_BUILD_RESOURCE_EXPORTBuild export consumer threads3
KAFKA_CONSUMER_THREADS_BUILD_RESOURCE_IMPORTBuild import consumer threads3
KAFKA_CONSUMER_THREADS_BUILD_START_TIMER_EVENTS_UPDATESBuild timer events updates consumer threads3

Topic naming convention and pattern creation

The Application Manager uses a sophisticated topic naming convention that follows a structured pattern. This ensures consistency across environments and makes topics easily identifiable.

Topic naming components

ComponentDefault ValueEnvironment VariableDescription
packageai.flowx.KAFKA_TOPIC_NAMING_PACKAGEBase package identifier
environmentdev.KAFKA_TOPIC_NAMING_ENVIRONMENTDeployment environment
version.v1KAFKA_TOPIC_NAMING_VERSIONTopic version
separator.KAFKA_TOPIC_NAMING_SEPARATORMain separator (referred to as dot)
separator2-KAFKA_TOPIC_NAMING_SEPARATOR2Secondary separator (as dash)
prefix--Combined package and environment
suffix--The version suffix

Topic pattern creation

Topics are constructed using the following pattern:

{prefix} + service + {separator/dot} + action + {separator/dot} + detail + {suffix}

For example, a typical topic might look like:

ai.flowx.dev.application-version.export.v1

Where:

  • ai.flowx.dev. is the prefix (package + environment)
  • application-version is the service
  • export is the action
  • .v1 is the suffix (version)

For more complex topics, additional components are added:

ai.flowx.dev.application-version.resources-usages.sub-res-validation.response.v1

Where:

  • resources-usages represents the resource type
  • sub-res-validation represents the operation type
  • response indicates it’s a response message

Kafka topic configuration

Application resource topics
Environment VariableDescriptionDefault Pattern
KAFKA_TOPIC_APPLICATION_RESOURCE_EXPORTApplication resource export topicai.flowx.dev.application-version.export.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_IMPORTApplication resource import topicai.flowx.dev.application-version.import.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_USAGES_INResource usages in topicai.flowx.dev.application-version.resources-usages.operations.bulk.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_USAGES_OUTResource usages out topicai.flowx.dev.application-version.resources-usages.operations.bulk.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_USAGES_REFRESHResource usages refresh topicai.flowx.dev.application-version.resources-usages.refresh.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_RESELEMUSAGEVALIDATION_RESPONSEResource element usage validation responseai.flowx.dev.application-version.resources-usages.sub-res-validation.response.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_RESELEMUSAGEVALIDATION_OUT_INTEGRATIONResource validation integration topicai.flowx.dev.application-version.resources-usages.sub-res-validation.request-integration.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_RESELEMUSAGEVALIDATION_OUT_CMSResource validation CMS topicai.flowx.dev.application-version.resources-usages.sub-res-validation.cms.v1
KAFKA_TOPIC_APPLICATION_RESOURCE_COPYResource copy topicai.flowx.dev.application-version.copy-resource.v1
KAFKA_TOPIC_APPLICATION_MERGEApplication merge topicai.flowx.dev.application-version.merge.v1
Build resource topics
Environment VariableDescriptionDefault Pattern
KAFKA_TOPIC_BUILD_UPDATEBuild update topicai.flowx.dev.build.update.v1
KAFKA_TOPIC_BUILD_CREATEBuild create topicai.flowx.dev.build.create.v1
KAFKA_TOPIC_BUILD_RESOURCE_EXPORTBuild export topicai.flowx.dev.build.export.v1
KAFKA_TOPIC_BUILD_RESOURCE_IMPORTBuild import topicai.flowx.dev.build.import.v1
KAFKA_TOPIC_BUILD_START_TIMER_EVENTS_IN_UPDATESTimer events updates topicai.flowx.dev.build.start-timer-events.updates.in.v1
Process topics
Environment VariableDescriptionDefault Pattern
KAFKA_TOPIC_PROCESS_START_FOR_EVENT_INProcess start for event topicai.flowx.dev.core.trigger.start-for-event.process.v1
KAFKA_TOPIC_PROCESS_START_BY_NAME_INProcess start by name topicai.flowx.dev.core.trigger.start-by-name.process.v1
KAFKA_TOPIC_PROCESS_START_BY_NAME_OUTProcess start by name out topicai.flowx.dev.core.trigger.start-by-name.process.out.v1
KAFKA_TOPIC_PROCESS_SCHEDULED_TIMER_EVENTS_OUT_SETSet timer schedule topicai.flowx.dev.core.trigger.set.timer-event-schedule.v1
KAFKA_TOPIC_PROCESS_SCHEDULED_TIMER_EVENTS_OUT_STOPStop timer schedule topicai.flowx.dev.core.trigger.stop.timer-event-schedule.v1
Other topics
Environment VariableDescriptionDefault Pattern
KAFKA_TOPIC_AUDIT_OUTAudit topicai.flowx.dev.core.trigger.save.audit.v1
KAFKA_TOPIC_EVENTS_GATEWAY_OUT_MESSAGEEvents gateway messages topicai.flowx.dev.eventsgateway.receive.copyresource.v1

These Kafka topics use predefined naming conventions for ease of use. Optional adjustments may be made if the desired topic name cannot be achieved with the standard structure.

Authentication configuration

Environment VariableDescriptionExample Value
SECURITY_OAUTH2_BASE_SERVER_URLOAuth2 server base URLhttps://auth-pr1180.dev3.flowxai.dev/auth
SECURITY_OAUTH2_CLIENT_CLIENT_IDClient IDflowx-platform-authorize
SECURITY_OAUTH2_CLIENT_CLIENT_SECRETClient secretsecret
SECURITY_OAUTH2_REALMAuthentication realmflowx
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRETAdmin service account secretsecret
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_IDAdmin service account IDflowx-runtime-manager-sa

Refer to the dedicated section for configuring user roles and access rights:

Access Management

File storage configuration

S3 is used in the Application Manager for:

  • Storing imported and exported resources
  • Storing application versions and builds that are imported or exported
Environment VariableDescriptionExample ValueDefault
APPLICATION_FILE_STORAGE_S3_SERVER_URLS3 server URLhttp://minio:9000None
APPLICATION_FILE_STORAGE_S3_ACCESS_KEYS3 access keyHa0wvtOE9gQ2NSzghEcsNone
APPLICATION_FILE_STORAGE_S3_SECRET_KEYS3 secret keyjY7nYLVtNh9JzMflliQKu3noPpjxD3prxIkliErXNone
APPLICATION_FILE_STORAGE_TYPEStorage types3s3
APPLICATION_FILE_STORAGE_DELETION_STRATEGYFile deletion strategydeletedelete
APPLICATION_FILE_STORAGE_S3_ENABLEDEnable S3 storagetruetrue
APPLICATION_FILE_STORAGE_S3_ENCRYPTION_ENABLEDEnable S3 encryptionfalsefalse
APPLICATION_FILE_STORAGE_S3_BUCKET_PREFIXS3 bucket name prefixapplications-bucketapplications-bucket

Monitoring and health check configuration

Environment VariableDescriptionExample ValueDefault
MANAGEMENT_METRICS_EXPORT_PROMETHEUS_ENABLEDPrometheus metricstruefalse
MANAGEMENT_HEALTH_KUBERNETES_ENABLEDKubernetes health checkfalsetrue
MANAGEMENT_HEALTH_REDIS_ENABLEDRedis health checkfalsetrue
MANAGEMENT_HEALTH_KAFKA_ENABLEDKafka health checktruetrue
MANAGEMENT_HEALTH_LIVENESSSTATE_ENABLEDLiveness statetruefalse
MANAGEMENT_HEALTH_READINESSSTATE_ENABLEDReadiness statetruefalse
MANAGEMENT_ENDPOINT_HEALTH_GROUP_LIVENESS_INCLUDELiveness probesping,diskSpace,accessInfo,buildInfo,db,mongo,kafkaClusterHealthCheckIndicatorping
MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDEReadiness probesping,diskSpace,accessInfo,buildInfoping

Resource proxy configuration

The Resource Proxy module forwards resource-related requests to appropriate services, handling CRUD operations on the manifest. It requires proper configuration of proxy endpoints:

Environment VariableDescriptionExample ValueDefault
RESOURCE_PROXY_MANIFEST_URLManifest URL for resource proxyURL valueNone
RESOURCE_PROXY_TARGET_URLTarget URL for resource forwardingURL valueNone
FLOWX_RESOURCE_PROXY_RETRY_GET_RESOURCE_TIMEOUT_MSResource retrieval timeout500500
FLOWX_RESOURCE_PROXY_RETRY_GET_RESOURCE_MAX_COUNTMaximum resource retrieval retries1010
FLOWX_RESOURCE_PROXY_WEB_CLIENT_RETRY_ATTEMPTSWeb client retry attempts22
FLOWX_RESOURCE_PROXY_WEB_CLIENT_RETRY_BACKOFFRetry backoff time (seconds)11
FLOWX_RESOURCE_PROXY_WEB_CLIENT_MAX_IN_MEMORY_SIZEMaximum in-memory size5MB5MB
FLOWX_RUNTIMEEXECUTIONPROXY_WEBCLIENT_MAXINMEMORYSIZEMaximum REST request size5242880 (5 MB)5242880

FLOWX_RUNTIMEEXECUTIONPROXY_WEBCLIENT_MAXINMEMORYSIZE - Specifies the maximum size (in bytes) of in-memory data for REST requests. This is particularly useful when dealing with large payloads to prevent excessive memory consumption.

  • Default Value: 5242880 (5 MB)
  • Usage Example: Set to 10485760 (10 MB) to allow larger in-memory request sizes.

Scheduler configuration

The Application Manager scheduler supports retrying failed deployments and master election for better coordination of tasks across instances:

Environment VariableDescriptionExample ValueDefault
FLOWX_SCHEDULER_RETRY_FAILED_DEPLOYMENTS_CRONFailed deployment retry cron0 * * * * *None
FLOWX_SCHEDULER_MASTER_ELECTION_ENABLEDEnable master electiontruefalse
FLOWX_SCHEDULER_MASTER_ELECTION_CRON_EXPRESSIONMaster election cron*/30 * * * * *None
FLOWX_SCHEDULER_MASTER_ELECTION_PROVIDERElection providerredisNone

Retry failed deployments

Configures a cron job to retry updating builds in the runtime database every minute when previous attempts have failed.

Master election

Enables master election for improved scheduling coordination when multiple instances of the Application Manager are running, ensuring that scheduled tasks are only executed once.

Configuring logging

To control the logging levels for the Application Manager, use the following environment variables:

Environment VariableDescriptionExample Value
LOGGING_LEVEL_ROOTRoot Spring Boot logs levelINFO
LOGGING_LEVEL_APPApplication-level logs levelINFO
LOGGING_LEVEL_DBDatabase interactions log levelINFO

Data model overview

The Application Manager stores application data using a relational database schema, with key entities such as application, application_version, and application_manifest. Below are descriptions of primary entities:

  • Application - Defines an application with its details like name, type, and metadata.
  • Application Branch - Represents branches for versioning within an application.
  • Application Version - Keeps track of each version of an application, including committed and WIP statuses.
  • Application Manifest - Contains the list of resources associated with a specific application version.

Ingress configuration

Configure ingress to control external access to Application Manager:

ingress:
  enabled: true
  public:
    enabled: false
  admin:
    enabled: true
    hostname: "{{ .Values.flowx.ingress.admin }}"
    path: /appmanager(/|$)(.*)
    annotations:
      nginx.ingress.kubernetes.io/rewrite-target: /$2
      nginx.ingress.kubernetes.io/cors-allow-headers: DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization,flowx-platform