Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.flowx.ai/llms.txt

Use this file to discover all available pages before exploring further.

The File Gateway lets workflows exchange files with external storage systems (FTP, SFTP, S3, Azure Blob) and start processes when new files land in a watched location.

​
Dependencies

Before setting up the File Gateway, ensure you have the following dependencies in place:
  • MongoDB for storing file-trigger definitions and poll state
  • Kafka for publishing process start events
  • Keycloak (or compatible OAuth2 provider) for authentication and authorization
  • SpiceDB for fine-grained, workspace-level authorization
  • Document service for storing ingested and generated files
  • File storage targets — the FTP, SFTP, S3, or Azure Blob systems you connect to (credentials are tenant-supplied per connection, not set as service env vars)

​
Infrastructure prerequisites

ComponentDescription
MongoDBStores file_triggers, seen_file_cursors, failed_files, poll_leases collections (shares the runtime MongoDB instance)
KafkaMessage broker for triggering process starts
KeycloakIdentity provider for service authentication
SpiceDBAuthorization service for workspace validation
DocumentReceives uploaded files via the document service internal API

​
Configuration

​
Authorization configuration

Set the following environment variables to connect to your identity management platform:
Environment VariableDescriptionDefault Value
SECURITY_TYPESecurity typeoauth2
SECURITY_OAUTH2_BASE_SERVER_URLBase URL of the OAuth2/OIDC server
SECURITY_OAUTH2_REALMOAuth2 realm name
SECURITY_OAUTH2_CLIENT_CLIENT_IDClient ID for token introspection
SECURITY_OAUTH2_CLIENT_CLIENT_SECRETClient secret for token introspection
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_IDService account client IDflowx-file-gateway-sa
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRETService account client secret

​
MongoDB configuration

The File Gateway uses the runtime MongoDB instance (the same one used by the Email Gateway) to store file-trigger definitions and polling state.
Environment VariableDescriptionDefault Value
SPRING_DATA_MONGODB_URIMongoDB connection URI-

​
Kafka configuration

The File Gateway publishes a process-start event when a watched location receives a new file (a file trigger).

​
Core Kafka settings

Environment VariableDescriptionDefault Value
SPRING_KAFKA_BOOTSTRAP_SERVERSAddress of the Kafka server(s)localhost:9092
SPRING_KAFKA_SECURITY_PROTOCOLSecurity protocol for Kafka connectionsPLAINTEXT

​
OAuth authentication (when using SASL_PLAINTEXT)

Environment VariableDescriptionDefault Value
KAFKA_OAUTH_CLIENT_IDOAuth client IDkafka
KAFKA_OAUTH_CLIENT_SECRETOAuth client secretkafka-secret
KAFKA_OAUTH_TOKEN_ENDPOINT_URIOAuth token endpointkafka.auth.localhost
When using the kafka-auth profile, the security protocol will automatically be set to SASL_PLAINTEXT and the SASL mechanism will be set to OAUTHBEARER.

​
Kafka topics

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_PROCESS_START_OUTTopic for starting a process when a watched file arrivesai.flowx.core.trigger.start-for-event.process.v1

​
CAS lib configuration (SpiceDB)

Environment VariableDescriptionDefault Value
FLOWX_SPICEDB_HOSTSpiceDB hostnamespicedb
FLOWX_SPICEDB_PORTSpiceDB gRPC port50051
FLOWX_SPICEDB_TOKENSpiceDB authentication token-
FLOWX_LIB_CASCLIENT_RUNTIME_IMPLEMENTATIONRuntime authorization backend used by the CAS client. Keep the default unless instructed by FlowX.CUSTOM

​
Service communication

The File Gateway communicates with the following FlowX services:
Environment VariableDescriptionDefault Value
FLOWX_LIB_SECURITY_SERVICES_ORGANIZATIONMANAGER_BASEURLOrganization Manager base URLhttp://organization-manager:80
FLOWX_LIB_CASCLIENT_SERVICES_AUTHORIZATIONSYSTEM_BASEURLAuthorization System base URLhttp://authorization-system:80
FLOWX_DOCUMENTPLUGIN_BASEURLDocument Plugin base URL used to store ingested fileshttp://document-plugin:80

​
Logging configuration

Environment VariableDescriptionDefault Value
LOGGING_LEVEL_ROOTRoot logging levelINFO
LOGGING_LEVEL_APPApplication-specific log levelINFO

​
File storage connections

File storage targets (FTP, SFTP, S3, Azure Blob) are not configured through service environment variables. Each connection — including its host, path, and credentials — is defined per workspace in the FlowX Designer as a File Storage data source. Credentials are tenant-supplied and stored securely; the File Gateway never uses pod identity, IAM roles, or managed identities to reach a customer’s storage. Supported authentication methods:
ProtocolAuthentication
FTP / FTPSUsername + password
SFTPUsername + password, or private key
S3Access key + secret
Azure BlobShared key (access key)

​
Secrets management

Secret NameDescription
SPRING_DATA_MONGODB_URIMongoDB connection URI (contains credentials)
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRETKeycloak service account secret
FLOWX_SPICEDB_TOKENSpiceDB authentication token
KAFKA_OAUTH_CLIENT_SECRETKafka OAuth client secret (if using OAuth)

​
Deployment

​
Helm values example

fullnameOverride: file-gateway

image:
  repository: <your-registry>/file-gateway

replicaCount: 1

env:
  SPRING_PROFILES_ACTIVE: production

  # Kafka
  SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092

  # OAuth2
  SECURITY_TYPE: oauth2
  SECURITY_OAUTH2_BASE_SERVER_URL: https://keycloak.example.com/auth

  # SpiceDB
  FLOWX_SPICEDB_HOST: spicedb
  FLOWX_SPICEDB_PORT: 50051

extraEnvVarsMultipleSecretsCustomKeys:
  - name: mongodb-generic
    secrets:
      SPRING_DATA_MONGODB_URI: mongodb-uri
  - name: flowx-auth
    secrets:
      SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRET: keycloakFileGatewayClientSecret
  - name: spicedb-generic
    secrets:
      FLOWX_SPICEDB_TOKEN: spicedb-token

rbac:
  create: true

podLabels:
  flowx.ai/network-log: "true"
  flowx.ai/egress-s-kafka: "true"
  flowx.ai/routing-name: "file-gateway"
  flowx.ai/prometheus-scrape: "file-gateway"
The File Gateway makes outbound connections to customer-managed FTP/SFTP/S3/Azure Blob endpoints. Ensure egress network policies allow traffic to those destinations.

​
Verify your setup

The File Gateway pod is running and healthy: kubectl get pods -l app=file-gateway
The health endpoint returns HTTP 200: curl http://file-gateway:8080/actuator/health
MongoDB collections appear (file_triggers, seen_file_cursors, failed_files, poll_leases) with the expected indexes
SpiceDB connection is established — check pod logs for successful CAS client initialization

​
Troubleshooting

Solutions:
  1. Confirm the File Storage data source and file trigger are configured and toggled ON in the Designer
  2. Verify the connection credentials and path in the data source
  3. Check egress network policies allow the File Gateway to reach the storage host
  4. Review pod logs for poll or authentication errors
Solutions:
  1. Verify SpiceDB is running and reachable at the configured host and port
  2. Check that the SpiceDB token is correct
  3. Ensure network policies allow gRPC traffic to SpiceDB on port 50051
Solutions:
  1. Verify Kafka bootstrap servers are reachable
  2. Check that the ai.flowx.core.trigger.start-for-event.process.v1 topic exists
  3. Confirm the bound Message Start Event and active build for the process

SpiceDB Configuration

Fine-grained authorization setup

Kafka Authentication

Configure Kafka security and authentication

Email Gateway setup

The email-trigger gateway this service mirrors

IAM Configuration

Identity and access management setup
Last modified on June 2, 2026