Data architecture

Overview

FlowX.AI uses multiple specialized data stores, each chosen for specific workload characteristics. They fall into two categories:

Embedded data components — shipped with the platform, managed by FlowX.AI
Platform data services — third-party dependencies deployed by your team

Embedded data components

Embedded data components are delivered as part of the FlowX.AI platform (Docker images + Helm charts). They run on Kubernetes only, and their versioning is managed by FlowX.AI. These components are included in the standard deployment and cannot be replaced by alternative implementations.

Qdrant

Type: Vector databaseQdrant is used by the AI Platform RAG path (knowledgebase-rag, embedder, knowledgebase-indexer-v2) and by AI agents for semantic retrieval. It stores vector embeddings and serves similarity search over indexed knowledge.It holds separate collections for runtime knowledge bases (queried by business agents at run time), design-time knowledge bases (used by the config-time agents in FlowX Designer), and the AI assistant corpus. Qdrant runs embedded as a single node by default; cluster mode is recommended for production.

Platform data services

The following data services are required by the FlowX.AI platform but are third-party dependencies. You choose the deployment model — managed service or self-hosted, inside or outside the Kubernetes cluster — according to your enterprise standards. FlowX.AI connects to these services via configuration.For supported versions and compatibility details, see the Third-party components compatibility matrix.

Relational storage — PostgreSQL / Oracle

PostgreSQL is the primary relational database and system-of-record for most core services. Oracle Database is also supported as an alternative. They store:

Process definitions and metadata
Runtime instance state
Platform configuration
Administrative data

Most FlowX.AI microservices use PostgreSQL or Oracle as their authoritative storage layer.

Document storage — MongoDB

MongoDB provides document-oriented storage for unstructured and semi-structured content. It stores:

Runtime configuration
Runtime workflow state
Flexible data structures used by apps and integrations

MongoDB is also the storage layer for FlowX Database — a cross-instance, long-term storage feature that enables apps to persist and query data beyond a single process instance.

Caching — Redis

Redis is an in-memory cache used for performance optimization across the platform. It stores:

Cached process definitions
Compiled scripts
Transient session data
Distributed locks across microservices

Redis is used strictly for short-lived, performance-related data and is not a system-of-record. Long-term persistence is handled by PostgreSQL and MongoDB.

Message-start deduplication and pending external (received) messages are persisted in PostgreSQL, not Redis alone. Redis still backs the fast deduplication path, but the durable copy in PostgreSQL means in-flight message processing survives a Redis restart or data loss.

Event streaming — Kafka

Kafka is the event streaming backbone that enables asynchronous, event-driven communication between microservices. It handles:

Internal event propagation between platform services
External integration messaging
Decoupled processing workflows

Kafka is used for transient message propagation. Authoritative business data is persisted in PostgreSQL and MongoDB, not in Kafka.

Object storage — S3-compatible

S3-compatible object storage provides persistent file storage. It stores:

File attachments
Document outputs (generated PDFs, converted files)
Large binary assets

FlowX.AI accesses object storage via an S3-compatible interface.

For providers without native S3 support (such as Azure Blob Storage), an S3 proxy is required to expose an S3-compatible interface. If you use a native S3 service (like AWS S3), no proxy is needed.

Search and indexing — Elasticsearch

Elasticsearch is used primarily for audit logging and full-text search across workflow and runtime data. It stores:

Searchable representations of audit events
Platform activity logs
Indexed workflow data for fast retrieval

Elasticsearch is optimized for fast search and traceability rather than primary data storage.

AI Platform data stores

The AI Platform does not introduce a separate data stack. Its services reuse the same store families as the rest of the platform — PostgreSQL, MongoDB, Kafka, and object storage — and add Qdrant as the one AI-specific store for vector embeddings.

Data store	How the AI Platform uses it
Qdrant	Vector embeddings and similarity search for knowledge-base RAG and AI agents. Separate collections for runtime knowledge bases, design-time knowledge bases, and the AI assistant.
PostgreSQL	Service state for AI components — embedding-job tracking, document-enrichment artifacts, and conversation, thread, and message history for the AI gateway. Standalone services (document parser, speech-to-text) keep durable job state here.
Kafka	Asynchronous AI job topics (document parsing, speech-to-text, web crawling, evaluations, data privacy) and the internal knowledge-base pipeline that chains indexing, embedding, and enrichment.
Object storage (S3)	Knowledge-base enrichment offloads, parsed-document results, and agent-builder run artifacts.
MongoDB	Knowledge-base metadata — store and entry definitions, owned by Integration Designer. The vectors themselves live in Qdrant, so a knowledge base spans MongoDB (metadata) and Qdrant (embeddings).

All vector storage and similarity search across the AI Platform use Qdrant. PostgreSQL holds only service/job state for the AI components — not embeddings.

For exact databases, environment variables, Kafka topics, and provisioning details, see the AI Platform setup guide.

Quick reference

Data Store	Role	What it stores	Persistence
PostgreSQL / Oracle	System of record	Process definitions, config, metadata, instance state	Permanent
MongoDB	Document store	Runtime state, flexible data, FlowX Database	Permanent
Redis	Cache	Process definitions cache, sessions, locks	Transient
Kafka	Event streaming	Inter-service messages, integration events	Transient
Object Storage	File store	Attachments, documents, binary assets	Permanent
Elasticsearch	Search engine	Audit logs, indexed workflow data	Permanent
Qdrant	Vector database (AI)	Embeddings for RAG and semantic retrieval	Permanent

Service-level database mapping

The tables and diagrams below show which FlowX.AI services connect to which databases and what operations they perform. This is useful for infrastructure sizing, backup planning, and troubleshooting.

PostgreSQL databases

Each core service owns its own PostgreSQL database and manages its schema through Liquibase migrations at startup.

Service	Database	Operations
Admin	`flowxadmin`	Creates, reads, writes
Advancing Controller	`advancing`	Creates, reads, writes
Application Manager	`app_manager`	Creates, reads, writes
Authorization System	`auth_system`	Creates, reads, writes
Document Plugin	`document`	Creates, reads, writes
Email Gateway	`email_gateway`	Creates, reads, writes
License	`license`	Creates, reads, writes
Organization Manager	`org_manager`	Creates, reads, writes
Process Engine	`process_engine`	Creates, reads, writes
Task Management	`process_engine`	Reads, writes

Task Management shares the process_engine database with Process Engine. Data Sync acts as a centralized migration coordinator for app_manager, auth_system, and process_engine.

MongoDB databases

Each service manages its own MongoDB database. The app-runtime database is a shared read-only dependency for multiple services.

Service	Database	Operations
Admin	`data-model`	Creates, reads, writes
Application Manager	`app-runtime`	Creates, reads, writes
CMS Core	`cms-core`	Creates, reads, writes
Document Plugin	`document`	Creates, reads, writes
Document Plugin	`app-runtime`	Reads
Integration Designer	`integration-designer`	Creates, reads, writes
Integration Designer	`app-runtime`	Reads
Notification Plugin	`notification`	Creates, reads, writes
Notification Plugin	`app-runtime`	Reads
Scheduler	`notification`	Creates, reads, writes
Task Management	`task-management-plugin`	Creates, reads, writes
Task Management	`app-runtime`	Reads
Process Engine	`app-runtime`	Reads
Email Gateway	`app-runtime`	Reads
NoSQL DB Runner	`nosql-db-runner`	Creates, reads, writes

The app-runtime database is created by Application Manager and read by 6 other services. It contains runtime configuration, build data, and deployed app state. Availability of this database is critical for runtime operations.

Elasticsearch indexes

Service	Index	Operations
Audit Core	Audit index	Creates, reads, writes
Process Engine	Process instance index	Creates, reads, writes
Data Search	Process instance index	Reads

Redis

All core services use Redis for caching only via Spring Cache Manager. Events Gateway additionally uses Redis for pub/sub messaging. Services using Redis for caching: Admin, Application Manager, Authorization System, CMS Core, Document Plugin, Email Gateway, Integration Designer, License, Notification Plugin, Organization Manager, Process Engine, Task Management.

Service	Usage
Events Gateway	Cache, pub/sub

S3-compatible object storage buckets

Bucket names shown are defaults from the configuration and are configurable via environment variables (e.g., MINIO_BUCKET_PREFIX). Your deployment may use different names. In some deployments, multiple services may share the same bucket. Ensure cross-bucket read access is configured where needed.

Service	Default Bucket	Env Variable	Purpose
Application Manager	`applications-bucket`	—	App builds and exported/imported resources
CMS Core	`media-library-bucket`	—	Public media assets
CMS Core	`cms-private-storage-bucket`	—	Private theme and font files
Document Plugin	`flowx-dev-bucket`	`MINIO_BUCKET_PREFIX`	Document templates and generated files
Document Plugin	`temp-bucket`	`MINIO_TEMP_BUCKET`	Temporary processing files
Integration Designer	`workflows-bucket`	—	Workflow definitions and test files
Notification Plugin	`flowx-dev`	`MINIO_BUCKET_PREFIX`	Notification templates and attachments

Qdrant collections

The AI Platform stores vector embeddings in separate Qdrant collections. The embedder service writes vectors; retrieval and design-time services read them.

Collection	Written by	Read by	Content
Runtime knowledge bases	Embedder	Knowledgebase RAG, Planner	Embeddings for knowledge bases queried by business agents at run time
Design-time knowledge bases	Embedder	KB Enrichment, Planner, Knowledgebase RAG	Layout-aware embeddings for documents used by the config-time agents in FlowX Designer
AI assistant	Snapshot on startup	AI Assistant	Pre-indexed documentation corpus for the AI assistant

Collection names are versioned and encode the embedding models in use, so exact names vary by deployment and release. Treat the collection roles above as the stable reference.

Third-party components

Supported versions and compatibility matrix

FlowX Database

Cross-instance, long-term data storage using MongoDB

FlowX.AI architecture

Overall platform architecture and microservices overview

Redis configuration

Redis deployment modes and configuration

AI Platform setup

AI Platform data stores, environment variables, and Kafka topics

Getting Started

Concepts

FlowX.AI Designer

Building projects

Building blocks

Platform deep dive

Resources

API reference

Data architecture

Overview

Embedded data components

Qdrant

Platform data services

Relational storage — PostgreSQL / Oracle

Document storage — MongoDB

Caching — Redis

Event streaming — Kafka

Object storage — S3-compatible

Search and indexing — Elasticsearch

AI Platform data stores

Quick reference

Service-level database mapping

PostgreSQL databases

MongoDB databases

Elasticsearch indexes

Redis

S3-compatible object storage buckets

Qdrant collections

Third-party components

FlowX Database

FlowX.AI architecture

Redis configuration

AI Platform setup

​Overview

​Embedded data components

Qdrant

​Platform data services

​Relational storage — PostgreSQL / Oracle

​Document storage — MongoDB

​Caching — Redis

​Event streaming — Kafka

​Object storage — S3-compatible

​Search and indexing — Elasticsearch

​AI Platform data stores

​Quick reference

​Service-level database mapping

​PostgreSQL databases

​MongoDB databases

​Elasticsearch indexes

​Redis

​S3-compatible object storage buckets

​Qdrant collections

​Related resources

Third-party components

FlowX Database

FlowX.AI architecture

Redis configuration

AI Platform setup

Overview

Embedded data components

Platform data services

Relational storage — PostgreSQL / Oracle

Document storage — MongoDB

Caching — Redis

Event streaming — Kafka

Object storage — S3-compatible

Search and indexing — Elasticsearch

AI Platform data stores

Quick reference

Service-level database mapping

PostgreSQL databases

MongoDB databases

Elasticsearch indexes

Redis

S3-compatible object storage buckets

Qdrant collections

Related resources