FlowX Database
Store and access structured data between processes and applications
What is FlowX Database?
- Share data between different process instances
- Store and retrieve data across multiple processes
- Create a persistent data store within the FlowX ecosystem
- Access structured data without directly interacting with external systems
FlowX Database is powered by the nosql-db-runner microservice, which provides MongoDB functionality within the FlowX ecosystem. This service supports both native MongoDB and Azure Cosmos DB (MongoDB API), enabling you to choose the database backend that best fits your infrastructure needs while maintaining FlowX’s workflow integration capabilities.For more information about the nosql-db-runner service and CosmosDB considerations, see the nosql-db-runner documentation.
Key concepts
In the FlowX platform, Data Sources represent connections to different data repositories. With the introduction of FlowX Database, there are now two types of data sources available in the Integration Designer:
Each FlowX Database data source represents a collection in the underlying MongoDB database.
| Data Source Type | Description |
|---|---|
| RESTful System | Traditional integration with external REST APIs |
| FlowX Database | Connection to the internal FlowX Database for data persistence |
Benefits of FlowX Database
Data Sharing
Share data between different process instances, enabling more complex workflows and business scenarios
Independence
Store data structured according to your needs, independent of external systems
Extended Use Cases
Enable new business cases that require persistent data across multiple processes
Simplified Integration
Reduce the need for creating custom connectors to external systems for basic data persistence
Native Integration
Integrate with workflows through the existing workflow designer
Complete CRUD Operations
Perform all standard Create, Read, Update, and Delete operations on your data
Bulk Operations
Handle multiple documents efficiently with insertMany, updateMany, and deleteMany operations
Flexible Querying
Use MongoDB’s powerful query syntax to filter, sort, and project your data
How to use FlowX Database
Creating a collection
1
Access Data Sources
Navigate to the ‘Integrations’ section in the left sidebar of FlowX Designer, then click on ‘Data Sources’.
Accessing Data Sources in the navigation menu
2
Add a new Data Source
Click the ”+” button to add a new Data Source. In the “Add Data Source” dialog, select “FlowX Database” from the dropdown list.
Adding a new FlowX Database data source
3
Configure the collection
Provide a name and description for your collection. The name will be used to identify this database collection in your workflows.
Configuring the collection
4
Create the collection
Click “Create” to save the collection in FlowX Database. Your new collection will appear in the Data Sources list with the type “FlowX Database”.
Each FlowX Database data source represents a collection in the underlying MongoDB database. When you create a new data source with the FlowX Database type, you’re essentially creating a new collection where your documents will be stored.
Viewing collection data
1
Navigate to collection
Go to the collection you want to view in the ‘Data Sources’ section.
2
Access Documents tab
Click the ‘Documents’ section within the collection.
3
Browse documents
You’ll see a listing of currently saved documents in the collection.
4
Search and filter
Use the search functionality to find specific documents in the collection.
Creating database operations
1
Access operations list
Navigate to your FlowX Database collection in the Data Sources section.
2
Create new operation
Click the ”+” button to create a new operation.
Creating a new operation for a FlowX Database collection
3
Select operation type
Choose the operation type from the dropdown (find, findOne, insertOne, insertMany, etc.).
Selecting the operation type
4
Name the operation
Give your operation a meaningful name and description that indicates its purpose.
5
Define the operation parameters
Configure the parameters for your operation based on its type. For example, for a find operation:
Configuring operation parameters
- Filter: Define the criteria to match documents
- Sort: Specify the sorting order
- Projection: Select which fields to include in the results
- Skip: For find operations, specify the number of documents to skip
- Limit: Set the maximum number of documents to return
6
Test the operation
Use the test functionality to verify your operation works correctly.
Testing an operation
7
Save the operation
Click “Save” to add the operation to your collection. It will now be available for use in workflows.
Working with FlowX Database in workflows
Database node in a workflow
1
Add Data Source node
In your workflow, add a new node and select the ‘DATA_SOURCE’ type.
2
Configure data source
In the node properties panel:
Data Source node configuration
- Choose “FlowX Database” as the system type from the dropdown
- Select your collection from the available options
- Choose the operation you want to perform (find, findOne, insertOne, insertMany, updateOne, updateMany, deleteOne, deleteMany)
3
Configure operation parameters
Set the parameters for your selected operation. The available parameters will depend on the operation type:
You can use static JSON or reference process variables using the expression syntax.
- filter - For find operations, define the criteria to match documents
- sort - For find operations, specify the sorting order for results
- document - For insert operations, define the document to insert
- update - For update operations, define the document to update
- delete - For delete operations, define the criteria to match documents
- skip - For find operations, specify the number of documents to skip
- limit - For find operations, specify the maximum number of documents to return
- projection - For find operations, specify the fields to include or exclude in the results
Configuring operation parameters
4
Set response key
Specify where the operation result should be stored in the process instance. This makes the data available to subsequent nodes in your workflow.
Example: Using filter parameter in find operation
Example: Using filter parameter in find operation
For more information about MongoDB comparison operators like $gt, see Comparison Query Operators.
Example: Using process variables in parameters
Example: Using process variables in parameters
Learn more about MongoDB query syntax in the Query Documents Tutorial.
MongoDB operations supported
FlowX Database leverages MongoDB’s powerful query capabilities through the nosql-db-runner service, providing pure MongoDB functionality. For detailed information about MongoDB query operators, filters, and syntax, refer to the MongoDB Query and Projection Operators documentation. The following MongoDB operations are supported in FlowX Database. Each tab includes practical examples that you can use as a starting point for your own operations.Parameters:For complete documentation on find operations and query syntax, see MongoDB db.collection.find().
- filter (document, optional): Criteria to match documents. See Query Filter Documents
- sort (document, optional): Document specifying the sorting order. See Sort Query Results
- skip (integer, optional): Number of documents to skip. See Limit Query Results
- limit (integer, optional): Maximum number of documents to return. See Limit Query Results
- projection (document, optional): Fields to include or exclude. See Project Fields
Example: List active customers
Example: List active customers
This example finds all active customers, sorts them by registration date (newest first), limits to 10 results, and returns only specific fields.Operation name: listActiveCustomersThis operation will return up to 10 active customers, with the most recently registered customers first.
Example: Find products by category and price
Example: Find products by category and price
This example finds products in a specific category within a price range.Operation name: findAffordableElectronicsThis operation will return up to 20 electronics products with prices between 100, sorted from lowest to highest price.
Always use caution with update and delete operations, especially those that affect multiple documents (updateMany, deleteMany). Always include specific filter criteria to avoid unintended changes to your data.
Real-world example: Customer management system
This example demonstrates how a customer management system could use FlowX Database to persist and share customer data across different processes.Setup
Setup
1
Create Customer collection
Create a new FlowX Database data source named “Customers”.
2
Define operations
Create the following operations:
Operation Type: findOne
Parameters:This operation retrieves a single customer by their customerId, returning only selected fields.
Customer support process
Customer support process
Customer support workflow with FlowX DB
- Support agent enters the customer ID
- A Data Source node retrieves the customer details from the Customers collection
- Customer information is displayed to the agent
- After resolving the issue, the support agent updates the customer status
- Another Data Source node updates the customer record in the database
Benefits demonstrated
Benefits demonstrated
This example shows how FlowX Database:
- Enables data persistence across multiple processes
- Provides a single source of truth for customer data
- Simplifies data retrieval and updates
- Eliminates the need for external systems to store basic customer information
Limitations and considerations
Best practices
Design before implementation: Plan your collections and operations carefully before implementation.
Keep document structure simple: Avoid deeply nested structures for better performance and easier querying.
Use meaningful names: Choose clear names for collections and operations.
Limit document size: Keep individual documents under 16MB for optimal performance.
Test operations thoroughly: Use the testing functionality to ensure operations work as expected.
Consider indexing: For large collections, consider indexing frequently queried fields.
Provide default values: Ensure all data types have default values (e.g., empty string for null, 0 for integers).
Troubleshooting
Operation fails with filter error
Operation fails with filter error
1
Problem
Find operation fails with an error related to the filter syntax.
2
Solution
Check your filter syntax to ensure it follows MongoDB query syntax. Common issues:
- Missing quotes around string values
- Incorrect operator syntax (e.g., using > instead of $gt)
- Mismatched brackets or braces
Documents not appearing in results
Documents not appearing in results
1
Problem
Your query doesn’t return expected documents.
2
Solution
- Verify your filter conditions are correct
- Check if documents actually exist in the collection
- Try a more general query to see if documents are returned
- Check if field names in your filter match exactly with the document structure
Performance issues with large result sets
Performance issues with large result sets
1
Problem
Queries returning large amounts of data are slow.
2
Solution
- Add pagination using skip and limit parameters
- Add more specific filter conditions
- Use projection to return only needed fields
- Consider indexing frequently queried fields
Insert operation fails
Insert operation fails
1
Problem
Insert operation fails with an error.
2
Solution
- Check that your document JSON is valid
- Ensure document size is under 16MB
- Verify all required fields are present
- Check for any unique constraint violations
Related features
Data Search
Learn about the Data Search service for indexing and searching process data.
Integration Management
Understand how to manage integrations with external systems.
Workflow
See how workflows can interact with data sources.
Process Data Model
Learn about data models used within processes.
Summary
FlowX Database provides a powerful persistence layer that enables you to store and share data across different processes and projects. By leveraging MongoDB’s capabilities through the nosql-db-runner service, FlowX Database offers:- Flexible data storage for any structured data
- Data sharing between process instances
- Integration with workflows
- Independence from external systems for basic data persistence
MongoDB learning resources
To get the most out of FlowX Database, familiarize yourself with MongoDB concepts and operations:MongoDB Manual
Complete MongoDB documentation covering all operations and concepts
Query Tutorial
Learn how to construct effective MongoDB queries
Update Operations
Master document update operations and operators
Data Modeling
Best practices for structuring your data in MongoDB
FAQs
When should I use Data Search vs. FlowX Database?
When should I use Data Search vs. FlowX Database?
FlowX Database does not replace Data Search. Each serves different purposes in the platform.
- Acts as a persistent data store
- Optimized for CRUD operations
- Stores structured business data
- Designed for sharing data across processes
- Uses MongoDB as the underlying technology (via nosql-db-runner service)
What types of data should I store in FlowX Database?
What types of data should I store in FlowX Database?
FlowX Database is designed for operational data with temporary character, including:
- Dashboard data: Aggregated information for reporting and visualization
- Cached data: Frequently accessed information like daily exchange rates, product catalogs, or configuration data that changes periodically
- Shared process data: Information that needs to be accessed across multiple process instances - this solves the key pain point of data sharing between different process instances
- Temporary operational data: Working data that supports business processes but isn’t part of your permanent records
Important: FlowX Database should not become your primary book of records or system of record. Use it for operational data that supports your processes, not for critical business data that requires long-term retention and governance.
Can I share FlowX Database collections between different projects?
Can I share FlowX Database collections between different projects?
Why doesn't FlowX Database support automatic data migration?
Why doesn't FlowX Database support automatic data migration?
FlowX Database intentionally does not provide automatic migration capabilities for several important reasons:
- Data integrity: Many documents need to remain in their original format to maintain historical accuracy and compliance requirements
- Process dependencies: Existing processes may depend on specific data structures, and automatic changes could break functionality
- Business context: Data migration often requires business logic and context that automated systems cannot provide
- Risk management: Manual migration allows for proper testing and validation before changes are applied
Can I store different types of documents in the same collection?
Can I store different types of documents in the same collection?
No, this is strongly discouraged. Each collection should contain documents with the same structure and purpose.DON’T DO IT - Mixing different document types in one collection can lead to:
- Query complexity: Filtering becomes more complex when documents have different schemas
- Performance issues: Indexes become less effective with mixed document types
- Maintenance problems: Updates and changes become harder to manage
- Data consistency: Harder to ensure data quality and validation
How does FlowX Database solve cross-process data sharing?
How does FlowX Database solve cross-process data sharing?
FlowX Database solves the key pain point of accessing the same data from multiple process instances:
- Cross-instance access: Multiple process instances can read from and write to the same collections simultaneously
- Data persistence: Data remains available even after individual processes complete
- Concurrent access: The underlying MongoDB technology (via nosql-db-runner) handles concurrent read/write operations safely
- Shared state: Enables complex workflows where processes need to coordinate through shared data
What's the relationship between FlowX Database and the nosql-db-runner service?
What's the relationship between FlowX Database and the nosql-db-runner service?
FlowX Database is powered by the nosql-db-runner microservice, which provides:
- Pure MongoDB functionality: Direct access to MongoDB operations within the FlowX ecosystem
- Native integration: Connection between FlowX workflows and MongoDB operations
- Scalable architecture: A dedicated service that can be scaled independently based on database operation needs