Data-sync job setup guide
This guide provides essential environment variables for configuring the Data-Sync Job. You should use these environment variables to set up and run the Job in your Kubernetes environment.
Overview
The Data-Sync Job is designed to synchronize and transfer data across multiple databases, ensuring data consistency and up-to-date information across all connected systems.
It operates by connecting to multiple databases, retrieves data, and synchronizes changes across them. The job logs its actions and can be scheduled to run regularly, keeping all databases in sync and up-to-date.
Data-sync job setup guide
This guide details the essential environment variables required to configure and run the Data-Sync Job for synchronizing data across various databases.
Required environment variables
Config profile
CONFIG_PROFILE
- This environment variable must be set explicitly and exactly to ensure that no unintended profiles are loaded by mistake. The value of this variable should represent a minimal configuration state, relying only on defaults specified in theapplication.properties
file of the application.
Example:
Skipping services for data-sync
If your deployment doesn’t have certain plugins or other microservices installed, you should configure data-sync to skip migrations for those:
FLOWX_SKIPPEDRESOURCESERVICES
- Comma-separated list of service names for which data-sync should skip running migrations.
Example:
Avoid blank spaces in the value of the FLOWX_SKIPPEDRESOURCESERVICES
variable.
This prevents data-sync from failing when trying to synchronize databases for services that aren’t installed in your environment.
Available services datasources are:
scheduler-core
: schedulercms-core
: cmstask-management-plugin
: task-managerdocument-plugin
: document-pluginnotification-plugin
: notification-pluginruntime-manager
: app-runtime,appmanagerintegration-designer
: integration-designeradmin
: admin,engineprocess-engine
: engineapplication-manager
: appmanager
MongoDB connections
CMS MongoDB
FLOWX_DATASOURCE_CMS_URI
- MongoDB URI for CMS database.- Example:
"mongodb://${CMS_MONGO_USERNAME}:${CMS_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${CMS_MONGO_DATABASE}"
- Example:
CMS_MONGO_USERNAME
- Username for MongoDB CMS database.CMS_MONGO_PASSWORD
- Password for the CMS MongoDB database.CMS_MONGO_DATABASE
- Database name for CMS.
Scheduler MongoDB
FLOWX_DATASOURCE_SCHEDULER_URI
- MongoDB URI for Scheduler database.- Example:
"mongodb://${SCHEDULER_MONGO_USERNAME}:${SCHEDULER_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${SCHEDULER_MONGO_DATABASE}"
- Example:
SCHEDULER_MONGO_USERNAME
- Username for MongoDB Scheduler database.SCHEDULER_MONGO_PASSWORD
- Password for the Scheduler MongoDB database.SCHEDULER_MONGO_DATABASE
- Database name for Scheduler.
Task Manager MongoDB
FLOWX_DATASOURCE_TASKMANAGER_URI
- MongoDB URI for Task Manager database.- Example:
"mongodb://${TASKMANAGER_MONGO_USERNAME}:${TASKMANAGER_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${TASKMANAGER_MONGO_DATABASE}"
- Example:
TASKMANAGER_MONGO_USERNAME
- Username for MongoDB Task Manager database.TASKMANAGER_MONGO_PASSWORD
- Password for the Task Manager MongoDB database.TASKMANAGER_MONGO_DATABASE
- Database name for Task Manager.
Document plugin MongoDB
FLOWX_DATASOURCE_DOCUMENTPLUGIN_URI
- MongoDB URI for Document Plugin database.- Example:
"mongodb://${DOCUMENTPLUGIN_MONGO_USERNAME}:${DOCUMENTPLUGIN_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${DOCUMENTPLUGIN_MONGO_DATABASE}"
- Example:
DOCUMENTPLUGIN_MONGO_DATABASE
- MongoDB database name for the Document Plugin.DOCUMENTPLUGIN_MONGO_USERNAME
- MongoDB username for the Document Plugin.DOCUMENTPLUGIN_MONGO_PASSWORD
- Password for the MongoDB Document Plugin database.
Notification plugin MongoDB
FLOWX_DATASOURCE_NOTIFICATIONPLUGIN_URI
- MongoDB URI for the Notification Plugin.- Example:
"mongodb://${NOTIFICATIONPLUGIN_MONGO_USERNAME}:${NOTIFICATIONPLUGIN_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${NOTIFICATIONPLUGIN_MONGO_DATABASE}"
- Example:
NOTIFICATIONPLUGIN_MONGO_USERNAME
- MongoDB username for the Notification Plugin.NOTIFICATIONPLUGIN_MONGO_PASSWORD
- Password for the MongoDB Notification Plugin database.NOTIFICATIONPLUGIN_MONGO_DATABASE
- MongoDB database name for the Notification Plugin.
App Runtime MongoDB
FLOWX_DATASOURCE_APPRUNTIME_URI
- MongoDB URI for App Runtime database.- Example:
"mongodb://${APPRUNTIME_MONGO_USERNAME}:${APPRUNTIME_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${APPRUNTIME_MONGO_DATABASE}"
- Example:
APPRUNTIME_MONGO_USERNAME
- Username for MongoDB App Runtime database.APPRUNTIME_MONGO_DATABASE
- Database name for App Runtime.
Integration Designer MongoDB
FLOWX_DATASOURCE_INTEGRATIONDESIGNER_URI
- MongoDB URI for Integration Designer database.- Example:
"mongodb://${INTEGRATIONDESIGNER_MONGO_USERNAME}:${INTEGRATIONDESIGNER_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${INTEGRATIONDESIGNER_MONGO_DATABASE}"
- Example:
INTEGRATIONDESIGNER_MONGO_USERNAME
- Username for MongoDB Integration Designer database.INTEGRATIONDESIGNER_MONGO_DATABASE
- Database name for Integration Designer.
Admin MongoDB
FLOWX_DATASOURCE_ADMIN_URI
- MongoDB URI for Admin database.- Example:
"mongodb://${ADMIN_MONGO_USERNAME}:${ADMIN_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${ADMIN_MONGO_DATABASE}"
- Example:
ADMIN_MONGO_USERNAME
- Username for MongoDB Admin database.ADMIN_MONGO_DATABASE
- Database name for Admin.
PostgreSQL connections
Process Engine database
FLOWX_DATASOURCE_ENGINE_URL
- PostgreSQL URL for Process Engine database.FLOWX_DATASOURCE_ENGINE_USERNAME
- Username for PostgreSQL Process Engine database.FLOWX_DATASOURCE_ENGINE_DRIVERCLASSNAME
- Driver class name for the Process Engine PostgreSQL database.
Application Manager database
FLOWX_DATASOURCE_APPMANAGER_URL
- PostgreSQL URL for Application Manager database.FLOWX_DATASOURCE_APPMANAGER_USERNAME
- Username for PostgreSQL Application Manager database.FLOWX_DATASOURCE_APPMANAGER_PASSWORD
- Password for the Application Manager PostgreSQL database.FLOWX_DATASOURCE_APPMANAGER_DRIVERCLASSNAME
- Details: Required to ensure proper database connectivity. This value can be overridden for other databases, such as Oracle.
- For PostgreSQL:
org.postgresql.Driver
- For Oracle:
oracle.jdbc.OracleDriver
- For PostgreSQL:
- Details: Required to ensure proper database connectivity. This value can be overridden for other databases, such as Oracle.
Spring JPA Variables
SPRING_JPA_DATABASE
- The database type for Spring JPA (e.g., oracle).
SPRING_JPA_PROPERTIES_HIBERNATE_DEFAULTSCHEMA
- The default schema for Hibernate in Spring JPA.
Logging
LOGGING_CONFIG_FILE
- Configuration file for logging.
Deployment
To deploy the Data-Sync Job, apply the YAML configuration with the required environment variables:
Monitor the Job status and logs as needed to ensure successful execution.
Was this page helpful?