File Gateway setup

The File Gateway lets workflows exchange files with external storage systems (FTP, SFTP, S3, Azure Blob) and start processes when new files land in a watched location.

Dependencies

Before setting up the File Gateway, ensure you have the following dependencies in place:

MongoDB for storing file-trigger definitions and poll state
Kafka for publishing process start events
Keycloak (or compatible OAuth2 provider) for authentication and authorization
SpiceDB for fine-grained, workspace-level authorization
Document service for storing ingested and generated files
File storage targets — the FTP, SFTP, S3, or Azure Blob systems you connect to (credentials are tenant-supplied per connection, not set as service env vars)

Infrastructure prerequisites

Component	Description
MongoDB	Stores `file_triggers`, `seen_file_cursors`, `failed_files`, `poll_leases` collections (shares the runtime MongoDB instance)
Kafka	Message broker for triggering process starts
Keycloak	Identity provider for service authentication
SpiceDB	Authorization service for workspace validation
Document	Receives uploaded files via the document service internal API

Configuration

Authorization configuration

The File Gateway validates incoming tokens with the JWT public key mechanism and authenticates to other FlowX services with a dedicated service account (the mainIdentity client registration) in the service-accounts realm:

Environment Variable	Description	Default Value
`SECURITY_TYPE`	Token validation mechanism (JWT public key validation)	`jwt-public-key`
`SECURITY_OAUTH2_BASESERVERURL`	Base URL of the Keycloak server
`SECURITY_OAUTH2_SAREALM`	Service-accounts realm ID	`00000002-0002-4002-8002-000000000002`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID`	Service account client ID	`flowx-file-gateway-sa`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET`	Service account client secret (Keycloak-issued)
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTID`	Anonymous service account client ID, used for anonymous runtime access	`flowx-anonymous-sa`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTSECRET`	Anonymous service account client secret
`SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKENURI`	Provider token URI, resolved against the service-accounts realm	`${SECURITY_OAUTH2_BASESERVERURL}/realms/${SECURITY_OAUTH2_SAREALM}/protocol/openid-connect/token`

Upgrading from 5.1.x? Remove the legacy opaque-token env vars: SECURITY_OAUTH2_REALM, SECURITY_OAUTH2_CLIENT_CLIENTID, SECURITY_OAUTH2_CLIENT_CLIENTSECRET, and SECURITY_OAUTH2_SERVICEACCOUNT_ADMIN_*. These belong to the removed introspection model and prevent the service from starting on 5.9.x. See the authentication and IAM migration guide for the full list.

MongoDB configuration

The File Gateway uses the runtime MongoDB instance (the same one used by the Email Gateway) to store file-trigger definitions and polling state.

Environment Variable	Description	Default Value
`SPRING_DATA_MONGODB_URI`	MongoDB connection URI	-

Kafka configuration

The File Gateway publishes a process-start event when a watched location receives a new file (a file trigger).

Core Kafka settings

Environment Variable	Description	Default Value
`SPRING_KAFKA_BOOTSTRAP_SERVERS`	Address of the Kafka server(s)	`localhost:9092`
`SPRING_KAFKA_SECURITY_PROTOCOL`	Security protocol for Kafka connections	`PLAINTEXT`

OAuth authentication (when using SASL_PLAINTEXT)

Environment Variable	Description	Default Value
`KAFKA_OAUTH_CLIENT_ID`	OAuth client ID	`kafka`
`KAFKA_OAUTH_CLIENT_SECRET`	OAuth client secret	`kafka-secret`
`KAFKA_OAUTH_TOKEN_ENDPOINT_URI`	OAuth token endpoint	`kafka.auth.localhost`

When using the kafka-auth profile, the security protocol will automatically be set to SASL_PLAINTEXT and the SASL mechanism will be set to OAUTHBEARER.

Kafka topics

Environment Variable	Description	Default Value
`KAFKA_TOPIC_PROCESS_START_OUT`	Topic for starting a process when a watched file arrives	`ai.flowx.core.trigger.start-for-event.process.v1`

CAS lib configuration (SpiceDB)

Environment Variable	Description	Default Value
`FLOWX_SPICEDB_HOST`	SpiceDB hostname	`spicedb`
`FLOWX_SPICEDB_PORT`	SpiceDB gRPC port	`50051`
`FLOWX_SPICEDB_TOKEN`	SpiceDB authentication token	-
`FLOWX_LIB_CASCLIENT_RUNTIME_IMPLEMENTATION`	Runtime authorization backend used by the CAS client. Keep the default unless instructed by FlowX.	`CUSTOM`

Service communication

The File Gateway communicates with the following FlowX services:

Environment Variable	Description	Default Value
`FLOWX_LIB_SECURITY_SERVICES_ORGANIZATIONMANAGER_BASEURL`	Organization Manager base URL	`http://organization-manager:80`
`FLOWX_LIB_CASCLIENT_SERVICES_AUTHORIZATIONSYSTEM_BASEURL`	Authorization System base URL	`http://authorization-system:80`
`FLOWX_DOCUMENTPLUGIN_BASEURL`	Document Plugin base URL used to store ingested files	`http://document-plugin:80`

Logging configuration

Environment Variable	Description	Default Value
`LOGGING_LEVEL_ROOT`	Root logging level	`INFO`
`LOGGING_LEVEL_APP`	Application-specific log level	`INFO`

File storage connections

File storage targets (FTP, SFTP, S3, Azure Blob) are not configured through service environment variables. Each connection — including its host, path, and credentials — is defined per workspace in the FlowX Designer as a File Storage data source. Credentials are tenant-supplied and stored securely; the File Gateway never uses pod identity, IAM roles, or managed identities to reach a customer’s storage. Supported authentication methods:

Protocol	Authentication
FTP / FTPS	Username + password
SFTP	Username + password, or private key
S3	Access key + secret
Azure Blob	Shared key (access key)

Secrets management

Secret Name	Description
`SPRING_DATA_MONGODB_URI`	MongoDB connection URI (contains credentials)
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET`	Keycloak service account secret
`FLOWX_SPICEDB_TOKEN`	SpiceDB authentication token
`KAFKA_OAUTH_CLIENT_SECRET`	Kafka OAuth client secret (if using OAuth)

Deployment

Helm values example

fullnameOverride: file-gateway

image:
  repository: <your-registry>/file-gateway

replicaCount: 1

env:
  SPRING_PROFILES_ACTIVE: production

  # Kafka
  SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092

  # OAuth2
  SECURITY_TYPE: oauth2
  SECURITY_OAUTH2_BASESERVERURL: https://keycloak.example.com/auth

  # SpiceDB
  FLOWX_SPICEDB_HOST: spicedb
  FLOWX_SPICEDB_PORT: 50051

extraEnvVarsMultipleSecretsCustomKeys:
  - name: mongodb-generic
    secrets:
      SPRING_DATA_MONGODB_URI: mongodb-uri
  - name: flowx-auth
    secrets:
      SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENT_SECRET: keycloakFileGatewayClientSecret
  - name: spicedb-generic
    secrets:
      FLOWX_SPICEDB_TOKEN: spicedb-token

rbac:
  create: true

podLabels:
  flowx.ai/network-log: "true"
  flowx.ai/egress-s-kafka: "true"
  flowx.ai/routing-name: "file-gateway"
  flowx.ai/prometheus-scrape: "file-gateway"

The File Gateway makes outbound connections to customer-managed FTP/SFTP/S3/Azure Blob endpoints. Ensure egress network policies allow traffic to those destinations.

Verify your setup

The File Gateway pod is running and healthy: kubectl get pods -l app=file-gateway

The health endpoint returns HTTP 200: curl http://file-gateway:8080/actuator/health

MongoDB collections appear (file_triggers, seen_file_cursors, failed_files, poll_leases) with the expected indexes

SpiceDB connection is established — check pod logs for successful CAS client initialization

Troubleshooting

Files are not picked up from a watched location

Solutions:

Confirm the File Storage data source and file trigger are configured and toggled ON in the Designer
Verify the connection credentials and path in the data source
Check egress network policies allow the File Gateway to reach the storage host
Review pod logs for poll or authentication errors

SpiceDB connection failures

Solutions:

Verify SpiceDB is running and reachable at the configured host and port
Check that the SpiceDB token is correct
Ensure network policies allow gRPC traffic to SpiceDB on port 50051

Processes not starting from file triggers

Solutions:

Verify Kafka bootstrap servers are reachable
Check that the ai.flowx.core.trigger.start-for-event.process.v1 topic exists
Confirm the bound Message Start Event and active build for the process

SpiceDB Configuration

Fine-grained authorization setup

Kafka Authentication

Configure Kafka security and authentication

Email Gateway setup

The email-trigger gateway this service mirrors

IAM Configuration

Identity and access management setup

​Dependencies

​Infrastructure prerequisites

​Configuration

​Authorization configuration

​MongoDB configuration

​Kafka configuration

​Core Kafka settings

​OAuth authentication (when using SASL_PLAINTEXT)

​Kafka topics

​CAS lib configuration (SpiceDB)

​Service communication

​Logging configuration

​File storage connections

​Secrets management

​Deployment

​Helm values example

​Verify your setup

​Troubleshooting

​Related resources

SpiceDB Configuration

Kafka Authentication

Email Gateway setup

IAM Configuration

Dependencies

Infrastructure prerequisites

Configuration

Authorization configuration

MongoDB configuration

Kafka configuration

Core Kafka settings

OAuth authentication (when using SASL_PLAINTEXT)

Kafka topics

CAS lib configuration (SpiceDB)

Service communication

Logging configuration

File storage connections

Secrets management

Deployment

Helm values example

Verify your setup

Troubleshooting

Related resources