> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flowx.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# File Gateway setup

> Set up the File Gateway microservice — ingests files from FTP, SFTP, S3, and Azure Blob storage into FlowX workflows, delivers generated files back, and can trigger processes when new files arrive.

> [!NOTE]
>
> The File Gateway lets workflows exchange files with external storage systems (FTP, SFTP, S3, Azure Blob) and start processes when new files land in a watched location.

## Dependencies

Before setting up the File Gateway, ensure you have the following dependencies in place:

* **MongoDB** for storing file-trigger definitions and poll state
* **Kafka** for publishing process start events
* **Keycloak** (or compatible OAuth2 provider) for authentication and authorization
* **SpiceDB** for fine-grained, workspace-level authorization
* **Document service** for storing ingested and generated files
* **File storage targets** — the FTP, SFTP, S3, or Azure Blob systems you connect to (credentials are tenant-supplied per connection, not set as service env vars)

## Infrastructure prerequisites

| Component | Description |
| - | - |
| MongoDB | Stores `file_triggers`, `seen_file_cursors`, `failed_files`, `poll_leases` collections (shares the runtime MongoDB instance) |
| Kafka | Message broker for triggering process starts |
| Keycloak | Identity provider for service authentication |
| SpiceDB | Authorization service for workspace validation |
| Document | Receives uploaded files via the document service internal API |

***

## Configuration

### Authorization configuration

The File Gateway validates incoming tokens with the JWT public key mechanism and authenticates to other FlowX services with a dedicated service account (the `mainIdentity` client registration) in the service-accounts realm:

| Environment Variable | Description | Default Value |
| - | - | - |
| `SECURITY_TYPE` | Token validation mechanism (JWT public key validation) | `jwt-public-key` |
| `SECURITY_OAUTH2_BASESERVERURL` | Base URL of the Keycloak server | |
| `SECURITY_OAUTH2_SAREALM` | Service-accounts realm ID | `00000002-0002-4002-8002-000000000002` |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID` | Service account client ID | `flowx-file-gateway-sa` |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET` | Service account client secret (Keycloak-issued) | |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTID` | Anonymous service account client ID, used for [anonymous runtime access](https://docs.flowx.ai/6.x/docs/platform-deep-dive/user-roles-management/anonymous-runtime-access.md) | `flowx-anonymous-sa` |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTSECRET` | Anonymous service account client secret | |
| `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKENURI` | Provider token URI, resolved against the service-accounts realm | `${SECURITY_OAUTH2_BASESERVERURL}/realms/${SECURITY_OAUTH2_SAREALM}/protocol/openid-connect/token` |

> [!WARNING]
>
> **Upgrading from 5.1.x?** Remove the legacy opaque-token env vars: `SECURITY_OAUTH2_REALM`, `SECURITY_OAUTH2_CLIENT_CLIENTID`, `SECURITY_OAUTH2_CLIENT_CLIENTSECRET`, and `SECURITY_OAUTH2_SERVICEACCOUNT_ADMIN_*`. These belong to the removed introspection model and prevent the service from starting on 5.9.x. See the [authentication and IAM migration guide](https://docs.flowx.ai/5.9/migrating-from-5.1-lts/authentication-iam.md) for the full list.

***

### MongoDB configuration

The File Gateway uses the runtime MongoDB instance (the same one used by the Email Gateway) to store file-trigger definitions and polling state.

| Environment Variable | Description | Default Value |
| - | - | - |
| `SPRING_DATA_MONGODB_URI` | MongoDB connection URI | - |

***

### Kafka configuration

The File Gateway publishes a process-start event when a watched location receives a new file (a file trigger).

#### Core Kafka settings

| Environment Variable | Description | Default Value |
| - | - | - |
| `SPRING_KAFKA_BOOTSTRAP_SERVERS` | Address of the Kafka server(s) | `localhost:9092` |
| `SPRING_KAFKA_SECURITY_PROTOCOL` | Security protocol for Kafka connections | `PLAINTEXT` |

#### OAuth authentication (when using SASL\_PLAINTEXT)

| Environment Variable | Description | Default Value |
| - | - | - |
| `KAFKA_OAUTH_CLIENT_ID` | OAuth client ID | `kafka` |
| `KAFKA_OAUTH_CLIENT_SECRET` | OAuth client secret | `kafka-secret` |
| `KAFKA_OAUTH_TOKEN_ENDPOINT_URI` | OAuth token endpoint | `kafka.auth.localhost` |

> [!NOTE]
>
> When using the `kafka-auth` profile, the security protocol will automatically be set to `SASL_PLAINTEXT` and the SASL mechanism will be set to `OAUTHBEARER`.

#### Kafka topics

| Environment Variable | Description | Default Value |
| - | - | - |
| `KAFKA_TOPIC_TRIGGER_STARTFOREVENT_PROCESS_OUT` | Topic for starting a process when a watched file arrives | `ai.flowx.core.trigger.start-for-event.process.v1` |

***

### CAS lib configuration (SpiceDB)

| Environment Variable | Description | Default Value |
| - | - | - |
| `FLOWX_SPICEDB_HOST` | SpiceDB hostname | `spicedb` |
| `FLOWX_SPICEDB_PORT` | SpiceDB gRPC port | `50051` |
| `FLOWX_SPICEDB_TOKEN` | SpiceDB authentication token | - |
| `FLOWX_LIB_CASCLIENT_RUNTIME_IMPLEMENTATION` | Runtime authorization backend used by the CAS client. Keep the default unless instructed by FlowX. | `CUSTOM` |

***

### Service communication

The File Gateway communicates with the following FlowX services:

| Environment Variable | Description | Default Value |
| - | - | - |
| `FLOWX_LIB_SECURITY_SERVICES_ORGANIZATIONMANAGER_BASEURL` | Organization Manager base URL | `http://organization-manager:80` |
| `FLOWX_LIB_CASCLIENT_SERVICES_AUTHORIZATIONSYSTEM_BASEURL` | Authorization System base URL | `http://authorization-system:80` |
| `FLOWX_DOCUMENTPLUGIN_BASEURL` | Document Plugin base URL used to store ingested files | `http://document-plugin:80` |

***

### Logging configuration

| Environment Variable | Description | Default Value |
| - | - | - |
| `LOGGING_LEVEL_ROOT` | Root logging level | `INFO` |
| `LOGGING_LEVEL_APP` | Application-specific log level | `INFO` |

***

## File storage connections

File storage targets (FTP, SFTP, S3, Azure Blob) are **not** configured through service environment variables. Each connection — including its host, path, and credentials — is defined per workspace in the FlowX Designer as a **File Storage** data source. Credentials are tenant-supplied and stored securely; the File Gateway never uses pod identity, IAM roles, or managed identities to reach a customer's storage.

Supported authentication methods:

| Protocol | Authentication |
| - | - |
| FTP / FTPS | Username + password |
| SFTP | Username + password, or private key |
| S3 | Access key + secret |
| Azure Blob | Shared key (access key) |

***

## Secrets management

| Secret Name | Description |
| - | - |
| `SPRING_DATA_MONGODB_URI` | MongoDB connection URI (contains credentials) |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET` | Keycloak service account secret |
| `FLOWX_SPICEDB_TOKEN` | SpiceDB authentication token |
| `KAFKA_OAUTH_CLIENT_SECRET` | Kafka OAuth client secret (if using OAuth) |

***

## Deployment

### Helm values example

```yaml theme={"dark"}
fullnameOverride: file-gateway

image:
  registry: <your-registry>
  repository: file-gateway

replicaCount: 1

env:
  SPRING_PROFILES_ACTIVE: production

  # Kafka
  SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092

  # OAuth2
  SECURITY_TYPE: oauth2
  SECURITY_OAUTH2_BASESERVERURL: https://keycloak.example.com/auth

  # SpiceDB
  FLOWX_SPICEDB_HOST: spicedb
  FLOWX_SPICEDB_PORT: 50051

extraEnvVarsMultipleSecretsCustomKeys:
  - name: mongodb-generic
    secrets:
      SPRING_DATA_MONGODB_URI: mongodb-uri
  - name: flowx-auth
    secrets:
      SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET: keycloakFileGatewayClientSecret
  - name: spicedb-generic
    secrets:
      FLOWX_SPICEDB_TOKEN: spicedb-token

rbac:
  create: true

podLabels:
  flowx.ai/network-log: "true"
  flowx.ai/egress-s-kafka: "true"
  flowx.ai/routing-name: "file-gateway"
  flowx.ai/prometheus-scrape: "file-gateway"
```

> [!NOTE]
>
> The `image.registry` and `image.repository` split applies to **5.9.4 and later**. To redirect every chart at once, use `global.imageRegistry`. See [Container images and registry](https://docs.flowx.ai/6.x/setup-guides/setup-guides-overview.md#container-images-and-registry).

> [!NOTE]
>
> The File Gateway makes outbound connections to customer-managed FTP/SFTP/S3/Azure Blob endpoints. Ensure egress network policies allow traffic to those destinations.

***

## Verify your setup

> [!TIP]
>
> The File Gateway pod is running and healthy: `kubectl get pods -l app=file-gateway`

> [!TIP]
>
> The health endpoint returns HTTP 200: `curl http://file-gateway:8080/actuator/health`

> [!TIP]
>
> MongoDB collections appear (`file_triggers`, `seen_file_cursors`, `failed_files`, `poll_leases`) with the expected indexes

> [!TIP]
>
> SpiceDB connection is established — check pod logs for successful CAS client initialization

***

## Troubleshooting

> [accordion] Files are not picked up from a watched location
>
> **Solutions:**
>
> 1. Confirm the File Storage data source and file trigger are configured and toggled ON in the Designer
> 2. Verify the connection credentials and path in the data source
> 3. Check egress network policies allow the File Gateway to reach the storage host
> 4. Review pod logs for poll or authentication errors

> [accordion] SpiceDB connection failures
>
> **Solutions:**
>
> 1. Verify SpiceDB is running and reachable at the configured host and port
> 2. Check that the SpiceDB token is correct
> 3. Ensure network policies allow gRPC traffic to SpiceDB on port `50051`

> [accordion] Processes not starting from file triggers
>
> **Solutions:**
>
> 1. Verify Kafka bootstrap servers are reachable
> 2. Check that the `ai.flowx.core.trigger.start-for-event.process.v1` topic exists
> 3. Confirm the bound Message Start Event and active build for the process

***

## Related resources

> [card] [SpiceDB Configuration](https://docs.flowx.ai/6.x/setup-guides/spicedb.md)
>
> Fine-grained authorization setup

> [card] [Kafka Authentication](https://docs.flowx.ai/6.x/setup-guides/kafka-authentication-config.md)
>
> Configure Kafka security and authentication

> [card] [Email Gateway setup](https://docs.flowx.ai/6.x/setup-guides/email-gateway-setup.md)
>
> The email-trigger gateway this service mirrors

> [card] [IAM Configuration](https://docs.flowx.ai/6.x/setup-guides/access-management/configuring-an-iam-solution.md)
>
> Identity and access management setup


## Related topics

- [File Storage](https://docs.flowx.ai/6.x/docs/platform-deep-dive/integrations/file-storage.md)
- [Email Gateway setup](https://docs.flowx.ai/6.x/setup-guides/email-gateway-setup.md)
- [Webhook Gateway setup](https://docs.flowx.ai/6.x/setup-guides/webhook-gateway-setup.md)
- [AI Gateway setup](https://docs.flowx.ai/6.x/setup-guides/ai-gateway-setup.md)
- [FlowX Events Gateway setup](https://docs.flowx.ai/6.x/setup-guides/events-gateway-setup.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.