> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flowx.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# NoSQL DB Runner

> Set up the NoSQL DB Runner microservice that powers FlowX Database with MongoDB and Azure Cosmos DB support.

> [!WARNING]
>
> **Behavior change in FlowX.AI 5.9.1.** Insert operations no longer accept a custom `_id` field.
>
> `_id` is reserved and assigned internally by MongoDB. Supplying your own `_id` on insert was never supported and could cause errors on later reads, so `Insert` and `Insert Many` now reject any payload that contains an `_id` field with `400 Bad Request` and the message `_id is a reserved keyword`. Remove `_id` from insert payloads; if you need to keep a custom identifier, store it under a different key (for example, `externalId`). Update and query operations are not affected.

## Configuration

The service comes with most of the needed configuration properties filled in, but there are several environment variables that need to be set up for your specific deployment.

### Database configuration

The NoSQL DB Runner connects to MongoDB for data persistence. Configure the following variables:

| Environment Variable | Description | Default Value |
| - | - | - |
| `DB_USERNAME` | MongoDB username for the service | `nosql-db-runner` |
| `DB_PASSWORD` | MongoDB password (should be stored as secret) | `secret(mongodb-generic)` |
| `SPRING_DATA_MONGODB_URI` | Complete MongoDB connection string | `mongodb://${DB_USERNAME}:${DB_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/nosql-db-runner` |

> [!NOTE]
>
> The MongoDB URI supports replica set configuration for high availability. The connection string includes multiple MongoDB instances and an arbiter for proper replica set functionality.

### Configuring authorization & access roles

The NoSQL DB Runner validates incoming tokens with the JWT public key mechanism and authenticates to other FlowX services with a dedicated service account (the `mainIdentity` client registration) in the service-accounts realm:

| Environment Variable | Description | Default Value |
| - | - | - |
| `SECURITY_TYPE` | Token validation mechanism (JWT public key validation) | `jwt-public-key` |
| `SECURITY_OAUTH2_BASESERVERURL` | Base URL of the Keycloak server | |
| `SECURITY_OAUTH2_SAREALM` | Service-accounts realm ID | `00000002-0002-4002-8002-000000000002` |
| `FLOWX_LIB_SECURITY_SERVICES_ORGANIZATIONMANAGER_BASEURL` | URL of the organization-manager service, used by the security library | `http://organization-manager:80` |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID` | Service account client ID | `flowx-nosql-db-runner-sa` |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET` | Service account client secret (Keycloak-issued) | |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTID` | Anonymous service account client ID, used for [anonymous runtime access](https://docs.flowx.ai/6.x/docs/platform-deep-dive/user-roles-management/anonymous-runtime-access.md) | `flowx-anonymous-sa` |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTSECRET` | Anonymous service account client secret | |
| `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKENURI` | Provider token URI, resolved against the service-accounts realm | `${SECURITY_OAUTH2_BASESERVERURL}/realms/${SECURITY_OAUTH2_SAREALM}/protocol/openid-connect/token` |

> [!WARNING]
>
> **Upgrading from 5.1.x?** Remove the legacy opaque-token env vars: `SECURITY_OAUTH2_REALM`, `SECURITY_OAUTH2_CLIENT_CLIENTID`, `SECURITY_OAUTH2_CLIENT_CLIENTSECRET`, and `SECURITY_OAUTH2_SERVICEACCOUNT_ADMIN_*`. These belong to the removed introspection model and prevent the service from starting on 5.9.x. See the [authentication and IAM migration guide](https://docs.flowx.ai/5.9/migrating-from-5.1-lts/authentication-iam.md) for the full list.

### FlowX operations document limit

This setting allows you to configure the maximum number of documents that can be retrieved from MongoDB in a single request.

| Environment Variable | Description | Default Value |
| - | - | - |
| `FLOWX_RUNOPERATIONS_DOCUMENTLIMIT` | Maximum number of documents to be processed in a single operation | `1000` |

### External MongoDB connection caching

| Environment Variable | Description | Default Value |
| - | - | - |
| `FLOWX_MONGO_CONNECTIONS_CACHE_MAX_ENTRIES` | Maximum number of cached external MongoDB connections | `100` |
| `FLOWX_MONGO_CONNECTIONS_CACHE_TTL` | Time-to-live for cached connections (duration format, e.g. `1d`, `12h`) | `1d` |

> [!TIP]
>
> If you connect to many distinct external MongoDB instances, increase `FLOWX_MONGO_CONNECTIONS_CACHE_MAX_ENTRIES` to avoid frequent reconnections. Lower the TTL if your MongoDB credentials rotate frequently.

### Relational database connection pooling

These defaults apply to all relational database data sources: Oracle, PostgreSQL, and IBM Db2.

| Environment Variable | Description | Default Value |
| - | - | - |
| `FLOWX_JDBC_CONNECTIONS_CACHE_MAX_ENTRIES` | Maximum cached JDBC connection pools | `100` |
| `FLOWX_JDBC_CONNECTIONS_CACHE_TTL` | Time-to-live for cached connection pools (duration format, e.g. `1d`, `12h`) | `1d` |
| `FLOWX_SQL_DEFAULTS_MAX_POOL_SIZE` | Maximum HikariCP pool size per data source | `5` |
| `FLOWX_SQL_DEFAULTS_CONNECTION_TIMEOUT_SECONDS` | JDBC connection timeout | `10` |
| `FLOWX_SQL_DEFAULTS_ROW_LIMIT` | Default row cap on `SELECT` responses | `1000` |
| `FLOWX_SQL_DEFAULTS_QUERY_TIMEOUT_SECONDS` | Query execution timeout | `30` |

> [!TIP]
>
> Increase `FLOWX_SQL_DEFAULTS_ROW_LIMIT` if your workflows need to read more than 1000 rows in a single query. Adjust `FLOWX_SQL_DEFAULTS_QUERY_TIMEOUT_SECONDS` for long-running analytical queries.

### Database host restrictions

> [!NOTE]
>
> **Available starting with FlowX.AI 5.9.5**

Relational data sources (Oracle, PostgreSQL, and IBM Db2) and Unmanaged MongoDB data sources may only connect to hosts that resolve to a public address. If **any** address the hostname resolves to is private, the connection is rejected with `400 Bad Request`, error key `error.sql.internalHostNotPermitted`, and the message `Connection to internal addresses is not permitted: <host>`.

For MongoDB, the NoSQL DB Runner checks the hosts in the connection URI before connecting, then every address the driver resolves later, including replica-set members and `mongodb+srv` targets. A private address found at that later stage shows up as a connection timeout, not as the `400` error. The platform's own MongoDB, which backs FlowX Database, is never checked.

| Environment Variable | Description | Default Value |
| - | - | - |
| `FLOWX_SSRF_PROTECTION_ENABLED` | Turns the host restriction on or off. Set to `false` to skip the check entirely. | `true` |
| `FLOWX_SSRF_ALLOWED_HOSTS` | Comma-separated hostnames, IP addresses, or CIDR blocks that are permitted even though they resolve to a private address, for example a database on a private network reached over a site-to-site VPN. Hostname entries must match the configured host exactly (case-insensitive, no wildcards). | |

> [!TIP]
>
> Prefer a hostname entry over a CIDR block. A hostname covers every address that name resolves to, so it keeps working if the DNS record later gains another address. A CIDR block is matched per address, so every private address the hostname returns must fall inside an allowed block, or the connection is still rejected.

> [!NOTE]
>
> Loopback, link-local, and wildcard addresses are never permitted, even when the host is listed in `FLOWX_SSRF_ALLOWED_HOSTS`.

### Configuring logging

Control log levels for different components of the service:

| Environment Variable | Description | Default Value |
| - | - | - |
| `LOGGING_CONFIG` | Path to the logging configuration file | `logback-spring.xml` |
| `LOGGING_LEVEL_ROOT` | Root logger level for the Spring Boot service | `INFO` |
| `LOGGING_LEVEL_APP` | Application-specific log level | `INFO` |

> [!WARNING]
>
> Using DEBUG log levels in production environments may impact performance and generate large log volumes. Consider using INFO or WARN levels for production deployments.

### Configuring application management

Configure management endpoints and monitoring capabilities:

| Environment Variable | Description | Default Value |
| - | - | - |
| `MANAGEMENT_SERVER_PORT` | Port for management endpoints | `8081` |
| `MANAGEMENT_ENDPOINT_HEALTH_GROUP_LIVENESS_INCLUDE` | Health checks for liveness probes | `ping,buildInfo` |
| `MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE` | Health checks for readiness probes | `ping,buildInfo,mongo` |
| `MANAGEMENT_HEALTH_KUBERNETES_ENABLED` | Enable Kubernetes health checks | `false` |

### Prometheus metrics configuration

Enable Prometheus metrics export for monitoring:

| Environment Variable | Description | Default Value |
| - | - | - |
| `MANAGEMENT_PROMETHEUS_METRICS_EXPORT_ENABLED` | Enable Prometheus metrics export | `false` |

> [!NOTE]
>
> The older form `MANAGEMENT_METRICS_EXPORT_PROMETHEUS_ENABLED` also works (both resolve to the same setting). The property path above is the preferred form.

***

## Troubleshooting

### Common issues

> [accordion] MongoDB connection failures
>
> **Symptoms:** Service fails to start with database connection errors.
>
> **Solutions:**
>
> 1. Verify MongoDB connection string format and credentials
> 2. Ensure MongoDB replica set is properly configured and accessible
> 3. Check network connectivity between service and MongoDB instances
> 4. Verify MongoDB user permissions and database access rights

> [accordion] Kafka connectivity issues
>
> **Symptoms:** Service cannot consume or produce Kafka messages.
>
> **Solutions:**
>
> 1. Verify `SPRING_KAFKA_BOOTSTRAP_SERVERS` points to the correct broker(s)
> 2. Check Kafka security protocol and authentication configuration
> 3. Ensure network policies allow traffic to the Kafka cluster
> 4. Verify topic names and consumer group assignments are correct

> [accordion] OAuth authentication errors
>
> **Symptoms:** 401/403 errors when communicating with other FlowX services.
>
> **Solutions:**
>
> 1. Verify OAuth client ID and client secret are correct
> 2. Check OAuth server URL and realm configuration
> 3. Ensure OAuth provider is accessible from the service
> 4. Validate token endpoint URI configuration

> [accordion] Insert fails with '\_id is a reserved keyword'
>
> **Symptoms:** An `Insert` or `Insert Many` operation returns `400 Bad Request` with the message `_id is a reserved keyword`.
>
> **Cause:** `_id` is reserved and assigned by MongoDB. Supplying it on insert was never supported and could break later reads, so starting with FlowX.AI 5.9.1 it is rejected.
>
> **Solution:** Remove the `_id` field from the insert payload. If you need to keep a specific identifier, store it under a different key (for example, `externalId`).

> [accordion] Connection fails with 'Connection to internal addresses is not permitted'
>
> **Symptoms:** An Oracle, PostgreSQL, IBM Db2, or Unmanaged MongoDB data source returns `400 Bad Request` with the message `Connection to internal addresses is not permitted: <host>` (error key `error.sql.internalHostNotPermitted`).
>
> **Cause:** The hostname resolves to at least one private address. Every address a hostname resolves to is checked, so a host with several DNS records is rejected if any one of them is private, even when the others are public.
>
> **Solution:** If the database is legitimately on a private network, for example reached over a site-to-site VPN, add its hostname to `FLOWX_SSRF_ALLOWED_HOSTS`. See [Database host restrictions](#database-host-restrictions).

> [accordion] Update or delete fails with 'requires a non-empty filter'
>
> **Symptoms:** An `UPDATE`, `UPDATE_MANY`, `DELETE`, or `DELETE_MANY` operation on a FlowX Database or Unmanaged MongoDB data source returns `400 Bad Request` with the message `<operation> requires a non-empty filter` (error key `error.collection.filterRequired`).
>
> **Cause:** These operations are rejected when the filter is missing or empty, so a forgotten filter can't change or delete every document in a collection.
>
> **Solution:** Add a filter to the operation. To match every document on purpose, use the filter `{"_id": {"$exists": true}}`.

***

## Related resources

> [card] [Redis Configuration](https://docs.flowx.ai/6.x/setup-guides/redis-configuration.md)
>
> Complete Redis setup including Sentinel and Cluster modes

> [card] [Kafka Authentication](https://docs.flowx.ai/6.x/setup-guides/kafka-authentication-config.md)
>
> Configure Kafka security and authentication

> [card] [IAM Configuration](https://docs.flowx.ai/6.x/setup-guides/access-management/configuring-an-iam-solution.md)
>
> Identity and access management setup


## Related topics

- [FlowX.AI architecture: services, layers, and data flow](https://docs.flowx.ai/6.x/docs/platform-overview/flowx-architecture.md)
- [Deployment guidelines v5.6.0](https://docs.flowx.ai/release-notes/v5.x/v5.6.0-march-2026/deployment-guidelines-v5.6.md)
- [Install new services](https://docs.flowx.ai/release-notes/v5.x/v5.1.x-lts/v5.1.0-october-2025/migrating-from-v4.7.x-to-5.1/install-services.md)
- [Db2 Database](https://docs.flowx.ai/6.x/docs/platform-deep-dive/integrations/db2-database.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.