​
Overview

The Data-Sync Job is designed to synchronize and transfer data across multiple databases, ensuring data consistency and up-to-date information across all connected systems. It operates by connecting to multiple databases, retrieves data, and synchronizes changes across them. The job logs its actions and can be scheduled to run regularly, keeping all databases in sync and up-to-date.

​
Data-sync job setup guide

This guide details the essential environment variables required to configure and run the Data-Sync Job for synchronizing data across various databases.

​
Required environment variables

​
Config profile

  • CONFIG_PROFILE - This environment variable must be set explicitly and exactly to ensure that no unintended profiles are loaded by mistake. The value of this variable should represent a minimal configuration state, relying only on defaults specified in the application.properties file of the application.
Example: 
- name: CONFIG_PROFILE
  value: k8stemplate_v2,kafka-auth

​
MongoDB connections

​
CMS MongoDB

  • FLOWX_DATASOURCE_CMS_URI - MongoDB URI for CMS database.
    • Example: "mongodb://${CMS_MONGO_USERNAME}:${CMS_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${CMS_MONGO_DATABASE}"
  • CMS_MONGO_USERNAME - Username for MongoDB CMS database.
  • CMS_MONGO_PASSWORD - Password for the CMS MongoDB database.
  • CMS_MONGO_DATABASE - Database name for CMS.

​
Scheduler MongoDB

  • FLOWX_DATASOURCE_SCHEDULER_URI - MongoDB URI for Scheduler database.
    • Example: "mongodb://${SCHEDULER_MONGO_USERNAME}:${SCHEDULER_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${SCHEDULER_MONGO_DATABASE}"
  • SCHEDULER_MONGO_USERNAME - Username for MongoDB Scheduler database.
  • SCHEDULER_MONGO_PASSWORD - Password for the Scheduler MongoDB database.
  • SCHEDULER_MONGO_DATABASE - Database name for Scheduler.

​
Task Manager MongoDB

  • FLOWX_DATASOURCE_TASKMANAGER_URI - MongoDB URI for Task Manager database.
    • Example: "mongodb://${TASKMANAGER_MONGO_USERNAME}:${TASKMANAGER_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${TASKMANAGER_MONGO_DATABASE}"
  • TASKMANAGER_MONGO_USERNAME - Username for MongoDB Task Manager database.
  • TASKMANAGER_MONGO_PASSWORD - Password for the Task Manager MongoDB database.
  • TASKMANAGER_MONGO_DATABASE - Database name for Task Manager.

​
Document plugin MongoDB

  • FLOWX_DATASOURCE_DOCUMENTPLUGIN_URI - MongoDB URI for Document Plugin database.
    • Example: "mongodb://${DOCUMENTPLUGIN_MONGO_USERNAME}:${DOCUMENTPLUGIN_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${DOCUMENTPLUGIN_MONGO_DATABASE}"
  • DOCUMENTPLUGIN_MONGO_DATABASE - MongoDB database name for the Document Plugin.
  • DOCUMENTPLUGIN_MONGO_USERNAME - MongoDB username for the Document Plugin.
  • DOCUMENTPLUGIN_MONGO_PASSWORD - Password for the MongoDB Document Plugin database.

​
Notification plugin MongoDB

  • FLOWX_DATASOURCE_NOTIFICATIONPLUGIN_URI - MongoDB URI for the Notification Plugin.
    • Example: "mongodb://${NOTIFICATIONPLUGIN_MONGO_USERNAME}:${NOTIFICATIONPLUGIN_MONGO_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/${NOTIFICATIONPLUGIN_MONGO_DATABASE}"
  • NOTIFICATIONPLUGIN_MONGO_USERNAME - MongoDB username for the Notification Plugin.
  • NOTIFICATIONPLUGIN_MONGO_PASSWORD - Password for the MongoDB Notification Plugin database.
  • NOTIFICATIONPLUGIN_MONGO_DATABASE - MongoDB database name for the Notification Plugin.

​
PostgreSQL connections

​
Process Engine database

  • FLOWX_DATASOURCE_ENGINE_URL - PostgreSQL URL for Process Engine database.
  • FLOWX_DATASOURCE_ENGINE_USERNAME - Username for PostgreSQL Process Engine database.
  • FLOWX_DATASOURCE_ENGINE_DRIVERCLASSNAME - Driver class name for the Process Engine PostgreSQL database.

​
Application Manager database

  • FLOWX_DATASOURCE_APPMANAGER_URL - PostgreSQL URL for Application Manager database.
  • FLOWX_DATASOURCE_APPMANAGER_USERNAME - Username for PostgreSQL Application Manager database.
  • FLOWX_DATASOURCE_APPMANAGER_PASSWORD - Password for the Application Manager PostgreSQL database.
  • FLOWX_DATASOURCE_APPMANAGER_DRIVERCLASSNAME
    • Details: Required to ensure proper database connectivity. This value can be overridden for other databases, such as Oracle.
      • For PostgreSQL: org.postgresql.Driver
      • For Oracle: oracle.jdbc.OracleDriver

​
Spring JPA Variables

SPRING_JPA_DATABASE - The database type for Spring JPA (e.g., oracle). SPRING_JPA_PROPERTIES_HIBERNATE_DEFAULTSCHEMA - The default schema for Hibernate in Spring JPA.

​
Logging

  • LOGGING_CONFIG_FILE - Configuration file for logging.

​
Deployment

To deploy the Data-Sync Job, apply the YAML configuration with the required environment variables: 
kubectl apply -f data-sync-job.yaml
Monitor the Job status and logs as needed to ensure successful execution.