> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flowx.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# File Gateway setup

> Set up the File Gateway microservice — ingests files from FTP, SFTP, S3, and Azure Blob storage into FlowX workflows, delivers generated files back, and can trigger processes when new files arrive.

<Info>
  The File Gateway lets workflows exchange files with external storage systems (FTP, SFTP, S3, Azure Blob) and start processes when new files land in a watched location.
</Info>

## Dependencies

Before setting up the File Gateway, ensure you have the following dependencies in place:

* **MongoDB** for storing file-trigger definitions and poll state
* **Kafka** for publishing process start events
* **Keycloak** (or compatible OAuth2 provider) for authentication and authorization
* **SpiceDB** for fine-grained, workspace-level authorization
* **Document service** for storing ingested and generated files
* **File storage targets** — the FTP, SFTP, S3, or Azure Blob systems you connect to (credentials are tenant-supplied per connection, not set as service env vars)

## Infrastructure prerequisites

| Component | Description                                                                                                                  |
| --------- | ---------------------------------------------------------------------------------------------------------------------------- |
| MongoDB   | Stores `file_triggers`, `seen_file_cursors`, `failed_files`, `poll_leases` collections (shares the runtime MongoDB instance) |
| Kafka     | Message broker for triggering process starts                                                                                 |
| Keycloak  | Identity provider for service authentication                                                                                 |
| SpiceDB   | Authorization service for workspace validation                                                                               |
| Document  | Receives uploaded files via the document service internal API                                                                |

***

## Configuration

### Authorization configuration

The File Gateway validates incoming tokens with the JWT public key mechanism and authenticates to other FlowX services with a dedicated service account (the `mainIdentity` client registration) in the service-accounts realm:

| Environment Variable                                                        | Description                                                                                                                                           | Default Value                                                                                      |
| --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `SECURITY_TYPE`                                                             | Token validation mechanism (JWT public key validation)                                                                                                | `jwt-public-key`                                                                                   |
| `SECURITY_OAUTH2_BASESERVERURL`                                             | Base URL of the Keycloak server                                                                                                                       |                                                                                                    |
| `SECURITY_OAUTH2_SAREALM`                                                   | Service-accounts realm ID                                                                                                                             | `00000002-0002-4002-8002-000000000002`                                                             |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID`          | Service account client ID                                                                                                                             | `flowx-file-gateway-sa`                                                                            |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET`      | Service account client secret (Keycloak-issued)                                                                                                       |                                                                                                    |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTID`     | Anonymous service account client ID, used for [anonymous runtime access](/5.9/docs/platform-deep-dive/user-roles-management/anonymous-runtime-access) | `flowx-anonymous-sa`                                                                               |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTSECRET` | Anonymous service account client secret                                                                                                               |                                                                                                    |
| `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKENURI`          | Provider token URI, resolved against the service-accounts realm                                                                                       | `${SECURITY_OAUTH2_BASESERVERURL}/realms/${SECURITY_OAUTH2_SAREALM}/protocol/openid-connect/token` |

<Warning>
  **Upgrading from 5.1.x?** Remove the legacy opaque-token env vars: `SECURITY_OAUTH2_REALM`, `SECURITY_OAUTH2_CLIENT_CLIENTID`, `SECURITY_OAUTH2_CLIENT_CLIENTSECRET`, and `SECURITY_OAUTH2_SERVICEACCOUNT_ADMIN_*`. These belong to the removed introspection model and prevent the service from starting on 5.9.x. See the [authentication and IAM migration guide](/5.9/migrating-from-5.1-lts/authentication-iam) for the full list.
</Warning>

***

### MongoDB configuration

The File Gateway uses the runtime MongoDB instance (the same one used by the Email Gateway) to store file-trigger definitions and polling state.

| Environment Variable      | Description            | Default Value |
| ------------------------- | ---------------------- | ------------- |
| `SPRING_DATA_MONGODB_URI` | MongoDB connection URI | -             |

***

### Kafka configuration

The File Gateway publishes a process-start event when a watched location receives a new file (a file trigger).

#### Core Kafka settings

| Environment Variable             | Description                             | Default Value    |
| -------------------------------- | --------------------------------------- | ---------------- |
| `SPRING_KAFKA_BOOTSTRAP_SERVERS` | Address of the Kafka server(s)          | `localhost:9092` |
| `SPRING_KAFKA_SECURITY_PROTOCOL` | Security protocol for Kafka connections | `PLAINTEXT`      |

#### OAuth authentication (when using SASL\_PLAINTEXT)

| Environment Variable             | Description          | Default Value          |
| -------------------------------- | -------------------- | ---------------------- |
| `KAFKA_OAUTH_CLIENT_ID`          | OAuth client ID      | `kafka`                |
| `KAFKA_OAUTH_CLIENT_SECRET`      | OAuth client secret  | `kafka-secret`         |
| `KAFKA_OAUTH_TOKEN_ENDPOINT_URI` | OAuth token endpoint | `kafka.auth.localhost` |

<Info>
  When using the `kafka-auth` profile, the security protocol will automatically be set to `SASL_PLAINTEXT` and the SASL mechanism will be set to `OAUTHBEARER`.
</Info>

#### Kafka topics

| Environment Variable            | Description                                              | Default Value                                      |
| ------------------------------- | -------------------------------------------------------- | -------------------------------------------------- |
| `KAFKA_TOPIC_PROCESS_START_OUT` | Topic for starting a process when a watched file arrives | `ai.flowx.core.trigger.start-for-event.process.v1` |

***

### CAS lib configuration (SpiceDB)

| Environment Variable                         | Description                                                                                        | Default Value |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------- | ------------- |
| `FLOWX_SPICEDB_HOST`                         | SpiceDB hostname                                                                                   | `spicedb`     |
| `FLOWX_SPICEDB_PORT`                         | SpiceDB gRPC port                                                                                  | `50051`       |
| `FLOWX_SPICEDB_TOKEN`                        | SpiceDB authentication token                                                                       | -             |
| `FLOWX_LIB_CASCLIENT_RUNTIME_IMPLEMENTATION` | Runtime authorization backend used by the CAS client. Keep the default unless instructed by FlowX. | `CUSTOM`      |

***

### Service communication

The File Gateway communicates with the following FlowX services:

| Environment Variable                                       | Description                                           | Default Value                    |
| ---------------------------------------------------------- | ----------------------------------------------------- | -------------------------------- |
| `FLOWX_LIB_SECURITY_SERVICES_ORGANIZATIONMANAGER_BASEURL`  | Organization Manager base URL                         | `http://organization-manager:80` |
| `FLOWX_LIB_CASCLIENT_SERVICES_AUTHORIZATIONSYSTEM_BASEURL` | Authorization System base URL                         | `http://authorization-system:80` |
| `FLOWX_DOCUMENTPLUGIN_BASEURL`                             | Document Plugin base URL used to store ingested files | `http://document-plugin:80`      |

***

### Logging configuration

| Environment Variable | Description                    | Default Value |
| -------------------- | ------------------------------ | ------------- |
| `LOGGING_LEVEL_ROOT` | Root logging level             | `INFO`        |
| `LOGGING_LEVEL_APP`  | Application-specific log level | `INFO`        |

***

## File storage connections

File storage targets (FTP, SFTP, S3, Azure Blob) are **not** configured through service environment variables. Each connection — including its host, path, and credentials — is defined per workspace in the FlowX Designer as a **File Storage** data source. Credentials are tenant-supplied and stored securely; the File Gateway never uses pod identity, IAM roles, or managed identities to reach a customer's storage.

Supported authentication methods:

| Protocol   | Authentication                      |
| ---------- | ----------------------------------- |
| FTP / FTPS | Username + password                 |
| SFTP       | Username + password, or private key |
| S3         | Access key + secret                 |
| Azure Blob | Shared key (access key)             |

***

## Secrets management

| Secret Name                                                            | Description                                   |
| ---------------------------------------------------------------------- | --------------------------------------------- |
| `SPRING_DATA_MONGODB_URI`                                              | MongoDB connection URI (contains credentials) |
| `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET` | Keycloak service account secret               |
| `FLOWX_SPICEDB_TOKEN`                                                  | SpiceDB authentication token                  |
| `KAFKA_OAUTH_CLIENT_SECRET`                                            | Kafka OAuth client secret (if using OAuth)    |

***

## Deployment

### Helm values example

```yaml theme={"system"}
fullnameOverride: file-gateway

image:
  repository: <your-registry>/file-gateway

replicaCount: 1

env:
  SPRING_PROFILES_ACTIVE: production

  # Kafka
  SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092

  # OAuth2
  SECURITY_TYPE: oauth2
  SECURITY_OAUTH2_BASESERVERURL: https://keycloak.example.com/auth

  # SpiceDB
  FLOWX_SPICEDB_HOST: spicedb
  FLOWX_SPICEDB_PORT: 50051

extraEnvVarsMultipleSecretsCustomKeys:
  - name: mongodb-generic
    secrets:
      SPRING_DATA_MONGODB_URI: mongodb-uri
  - name: flowx-auth
    secrets:
      SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENT_SECRET: keycloakFileGatewayClientSecret
  - name: spicedb-generic
    secrets:
      FLOWX_SPICEDB_TOKEN: spicedb-token

rbac:
  create: true

podLabels:
  flowx.ai/network-log: "true"
  flowx.ai/egress-s-kafka: "true"
  flowx.ai/routing-name: "file-gateway"
  flowx.ai/prometheus-scrape: "file-gateway"
```

<Note>
  The File Gateway makes outbound connections to customer-managed FTP/SFTP/S3/Azure Blob endpoints. Ensure egress network policies allow traffic to those destinations.
</Note>

***

## Verify your setup

<Check>
  The File Gateway pod is running and healthy: `kubectl get pods -l app=file-gateway`
</Check>

<Check>
  The health endpoint returns HTTP 200: `curl http://file-gateway:8080/actuator/health`
</Check>

<Check>
  MongoDB collections appear (`file_triggers`, `seen_file_cursors`, `failed_files`, `poll_leases`) with the expected indexes
</Check>

<Check>
  SpiceDB connection is established — check pod logs for successful CAS client initialization
</Check>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Files are not picked up from a watched location">
    **Solutions:**

    1. Confirm the File Storage data source and file trigger are configured and toggled ON in the Designer
    2. Verify the connection credentials and path in the data source
    3. Check egress network policies allow the File Gateway to reach the storage host
    4. Review pod logs for poll or authentication errors
  </Accordion>

  <Accordion title="SpiceDB connection failures">
    **Solutions:**

    1. Verify SpiceDB is running and reachable at the configured host and port
    2. Check that the SpiceDB token is correct
    3. Ensure network policies allow gRPC traffic to SpiceDB on port `50051`
  </Accordion>

  <Accordion title="Processes not starting from file triggers">
    **Solutions:**

    1. Verify Kafka bootstrap servers are reachable
    2. Check that the `ai.flowx.core.trigger.start-for-event.process.v1` topic exists
    3. Confirm the bound Message Start Event and active build for the process
  </Accordion>
</AccordionGroup>

***

## Related resources

<CardGroup cols={2}>
  <Card title="SpiceDB Configuration" icon="shield-halved" href="./spicedb">
    Fine-grained authorization setup
  </Card>

  <Card title="Kafka Authentication" icon="lock" href="./kafka-authentication-config">
    Configure Kafka security and authentication
  </Card>

  <Card title="Email Gateway setup" icon="envelope" href="./email-gateway-setup">
    The email-trigger gateway this service mirrors
  </Card>

  <Card title="IAM Configuration" icon="key" href="./access-management/configuring-an-iam-solution">
    Identity and access management setup
  </Card>
</CardGroup>