The Application Manager is a backend microservice for managing FlowX applications, libraries, versions, manifests, configurations, and builds. This guide provides detailed instructions for setting up the service and configuring its components to manage application-related operations effectively.
The Application manager is microservice that manages applications and also acts as a proxy for front-end requests related to resources.
Before you start setting up the Application Manager service, ensure the following infrastructure components are in place:
Ensure that the database for storing application data is properly set up and configured before starting the service.
The Application Manager relies on other FlowX services and components to function properly:
APP_MANAGER_DB_URL
- Connection string for the relational databaseAPP_MANAGER_DB_USER
- Username for the databaseAPP_MANAGER_DB_PASSWORD
- Password for the databaseAPP_MANAGER_DB_NAME
- Database nameTo integrate the Application Manager with the identity management system for authorization, set the following environment variables:
SECURITY_OAUTH2_BASE_SERVER_URL
- Base URL for the OAuth 2.0 Authorization ServerSECURITY_OAUTH2_CLIENT_CLIENT_ID
- Unique identifier for the client application registered with the OAuth 2.0 serverSECURITY_OAUTH2_CLIENT_CLIENT_SECRET
- Secret key for authenticating requests made by the authorization clientSECURITY_OAUTH2_REALM
- The realm name for OAuth2 authenticationSECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_ID
- Client ID for the application manager service accountSECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRET
- Client Secret for the application manager service accountRefer to the dedicated section for configuring user roles and access rights:
Configure the data sources for PostgreSQL and MongoDB as follows:
SPRING_DATASOURCE_URL
- Database URL for PostgreSQLSPRING_DATASOURCE_USERNAME
- Username for PostgreSQLSPRING_DATASOURCE_PASSWORD
- Password for PostgreSQLSPRING_DATASOURCE_DRIVER_CLASS_NAME
- Driver class for PostgreSQLThe Application Manager requires MongoDB to store runtime build information. Use the following environment variables for configuration:
SPRING_DATA_MONGODB_URI
- URI for connecting to the MongoDB instance -> to connect to app-runtime
database
mongodb://${DB_USERNAME}:${DB_PASSWORD}@<host1>,<host2>,<arbiter-host>:<port>/${DB_NAME}?retryWrites=false
DB_USERNAME
: app-runtime
DB_NAME
: app-runtime
DB_PASSWORD
: DB password.SPRING_DATA_MONGODB_STORAGE
- Specifies the storage type used for the Runtime MongoDB instance (Azure environments only)
mongodb
, cosmosdb
mongodb
If caching is required, configure Redis using the following environment variables:
SPRING_DATA_REDIS_HOST
- Hostname or IP address of the Redis server.SPRING_DATA_REDIS_PASSWORD
- Password for authenticating with the Redis server.SPRING_DATA_REDIS_PORT
- Hostname of the Redis server. This specifies where the Redis server is running and should be accessible from the application.To control the size of REST requests handled by the Application Manager, use the following environment variable:
FLOWX_RUNTIMEEXECUTIONPROXY_WEBCLIENT_MAXINMEMORYSIZE
- Specifies the maximum size (in bytes) of in-memory data for REST requests. This is particularly useful when dealing with large payloads to prevent excessive memory consumption.
5242880
(5 MB)10485760
(10 MB) to allow larger in-memory request sizes.The Application Manager scheduler supports retrying failed deployments and master election:
FLOWX_SCHEDULER_RETRY_FAILED_DEPLOYMENTS_CRON: "0 * * * * *"
FLOWX_SCHEDULER_MASTER_ELECTION_ENABLED: true
FLOWX_SCHEDULER_MASTER_ELECTION_CRON_EXPRESSION: "*/30 * * * * *"
FLOWX_SCHEDULER_MASTER_ELECTION_PROVIDER: redis
The Application Manager uses Kafka for event-driven operations. Set up the Kafka configuration using the following environment variables:
SPRING_KAFKA_BOOTSTRAP_SERVERS
- Address of the Kafka server, formatted as “host”SPRING_KAFKA_CONSUMER_GROUP_ID
- Consumer group ID for Kafka topicsKAFKA_CONSUMER_THREADS
- Number of Kafka consumer threadsFLOWX_KAFKA_PAYLOAD_SIZE_LIMIT
- Maximum payload size for Kafka messages (default: 512000
).Configure Kafka consumer groups and threads for efficient handling of build-related topics:
Consumer Groups:
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_EXPORT: appResourceExportGroup
KAFKA_CONSUMER_GROUP_ID_APPLICATION_RESOURCE_IMPORT: appResourceImportGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_CREATE: buildCreateGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_UPDATE: buildUpdateGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_RESOURCE_EXPORT: buildResourceExportGroup
KAFKA_CONSUMER_GROUP_ID_BUILD_RESOURCE_IMPORT: buildResourceImportGroup
KAFKA_CONSUMER_GROUP_ID_PROCESS_START: processStartGroup
KAFKA_CONSUMER_GROUP_ID_APPLICATION_MERGE: appMergeItemGroup
Consumer Threads:
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_EXPORT: 3
KAFKA_CONSUMER_THREADS_APPLICATION_RESOURCE_IMPORT: 3
KAFKA_CONSUMER_THREADS_BUILD_CREATE: 2
KAFKA_CONSUMER_THREADS_BUILD_UPDATE: 4
KAFKA_CONSUMER_THREADS_BUILD_RESOURCE_EXPORT: 3
KAFKA_CONSUMER_THREADS_BUILD_RESOURCE_IMPORT: 3
KAFKA_CONSUMER_THREADS_APPLICATION_MERGE: 3
The Application Manager uses a structured naming convention for Kafka topics, designed to support flexibility through pre-defined components in the topic structure. These components, while not directly configurable, allow optional modifications when the desired version cannot be obtained through $package . $environment . $separator . $version
.
Each topic adheres to a consistent naming schema for streamlined communication across environments and versions.
Component | Description | Default Value |
---|---|---|
package | Package identifier for namespace | ai.flowx. |
environment | Environment identifier | dev. |
version | Version identifier for topic compatibility | .v1 |
separator | Primary separator for components | . |
separator2 | Secondary separator for additional distinction | - |
prefix | Combines package and environment as a topic prefix | ${kafka.topic.naming.package}${kafka.topic.naming.environment} |
suffix | Appends version to the end of the topic name | ${kafka.topic.naming.version} |
Resource Export
-**Environment variable: KAFKA_TOPIC_APPLICATION_RESOURCE_EXPORT
${kafka.topic.naming.prefix}application${separator2}version${separator}export${kafka.topic.naming.suffix}
ai.flowx.application-version.export.v1
Resource Import
KAFKA_TOPIC_APPLICATION_RESOURCE_IMPORT
${kafka.topic.naming.prefix}application${separator2}version${separator}import${kafka.topic.naming.suffix}
ai.flowx.application-version.import.v1
Build Export
KAFKA_TOPIC_BUILD_RESOURCE_EXPORT
${kafka.topic.naming.prefix}build${separator}export${kafka.topic.naming.suffix}
ai.flowx.build.export.v1
Build Import
KAFKA_TOPIC_BUILD_RESOURCE_IMPORT
${kafka.topic.naming.prefix}build${separator}import${kafka.topic.naming.suffix}
ai.flowx.build.import.v1
Resource Create:
KAFKA_TOPIC_BUILD_RESOURCE_CREATE
${kafka.topic.naming.prefix}build${dot}create${kafka.topic.naming.suffix}
ai.flowx.build.create.v1
Resource Update:
KAFKA_TOPIC_BUILD_RESOURCE_UPDATE
${kafka.topic.naming.prefix}build${dot}update${kafka.topic.naming.suffix}
ai.flowx.build.update.v1
KAFKA_TOPIC_START_TIMER_EVENTS_UPDATES
${kafka.topic.naming.prefix}build${dot}start${dash}timer${dash}events${dot}updates${dot}in${kafka.topic.naming.suffix}
ai.flowx.build.start-timer-events.updates.in.v1
Start for Event
KAFKA_TOPIC_PROCESS_START_FOR_EVENT
${kafka.topic.naming.prefix}core${separator}trigger${separator}start${separator2}for${separator2}event${separator}process${kafka.topic.naming.suffix}
ai.flowx.core.trigger.start-for-event.process.v1
Start by Name
KAFKA_TOPIC_PROCESS_START_BY_NAME
${kafka.topic.naming.prefix}core${separator}trigger${separator}start${separator2}by${separator2}name${separator}process${kafka.topic.naming.suffix}
ai.flowx.core.trigger.start-by-name.process.v1
KAFKA_TOPIC_PROCESS_START_BY_NAME_OUT
${kafka.topic.naming.prefix}core${separator}trigger${separator}start${separator2}by${separator2}name${separator}process${separator}out${kafka.topic.naming.suffix}
ai.flowx.core.trigger.start-by-name.process.out.v1
Scheduled Timer Events
KAFKA_TOPIC_PROCESS_SCHEDULED_TIMER_EVENTS_SET
${kafka.topic.naming.prefix}core${separator}trigger${separator}set${separator}timer${separator2}event${separator2}schedule${kafka.topic.naming.suffix}
ai.flowx.core.trigger.set.timer-event-schedule.v1
KAFKA_TOPIC_PROCESS_SCHEDULED_TIMER_EVENTS_STOP
${kafka.topic.naming.prefix}core${separator}trigger${separator}stop${separator}timer${separator2}event${separator2}schedule${kafka.topic.naming.suffix}
ai.flowx.core.trigger.stop.timer-event-schedule.v1
KAFKA_TOPIC_AUDIT_OUTPUT
${kafka.topic.naming.prefix}core${separator}trigger${separator}save${separator}audit${kafka.topic.naming.suffix}
ai.flowx.core.trigger.save.audit.v1
KAFKA_TOPIC_MERGE
${kafka.topic.naming.prefix}application${dash}version${dot}merge${kafka.topic.naming.suffix}
ai.flowx.application-version.merge.v1
These Kafka topics use predefined naming conventions for ease of use. Optional adjustments may be made if the desired topic name cannot be achieved with the $package . $environment . $separator . $version
structure.
The Resource Proxy module forwards resource-related requests to appropriate services, handling CRUD operations on the manifest. It requires proper configuration of proxy endpoints:
RESOURCE_PROXY_MANIFEST_URL
- URL for managing the application manifestRESOURCE_PROXY_TARGET_URL
- URL for forwarding resource-related requests to their respective servicesTo control the logging levels for the Application Manager, use the following environment variables:
LOGGING_LEVEL_ROOT
- Log level for the root service logsLOGGING_LEVEL_APP
- Log level for application-level logsLOGGING_LEVEL_DB
- Log level for database interactionsS3 is used in the Application Manager for:
If the Application Manager requires file storage for resources or builds, configure S3-compatible storage using the following environment variables:
APPLICATION_FILE_STORAGE_TYPE
: Type of storage to use. Set this to s3
to enable S3-compatible file storage.APPLICATION_FILE_STORAGE_DELETION_STRATEGY
: Strategy for handling file deletion. Options include:
delete
: Files are permanently deleted. (DEFAULT
)disabled
: File deletion is disabled.deleteBypassingGovernanceRetention
: Files are deleted, bypassing governance retention settings.APPLICATION_FILE_STORAGE_S3_ENABLED
: Set to true
to enable S3-compatible storage.APPLICATION_FILE_STORAGE_S3_SERVER_URL
: URL of the S3-compatible storage server.APPLICATION_FILE_STORAGE_S3_ENCRYPTION_ENABLED
: Set to true to enable server-side encryption for stored files. Default: false.APPLICATION_FILE_STORAGE_S3_ACCESS_KEY
: Access key for the S3 storage.APPLICATION_FILE_STORAGE_S3_SECRET_KEY
: Secret key for the S3 storage.APPLICATION_FILE_STORAGE_S3_BUCKET_PREFIX
: Prefix for S3 bucket names. Default: "applications-bucket"
.The Application Manager stores application data using a relational database schema, with key entities such as application, application_version, and application_manifest. Below are descriptions of primary entities:
To monitor the performance and health of the Application Manager, use tools like Prometheus or Grafana. Configure Prometheus metrics with the following environment variable:
MANAGEMENT_PROMETHEUS_METRICS_EXPORT_ENABLED
- Enables or disables Prometheus metrics export (default: false).Configure ingress to control external access to Application Manager:
Note: Replace placeholders with actual values for your environment before starting the service.