Email Gateway setup

The Email Gateway is a FlowX.AI microservice that handles email communication workflows. It is available as a Docker image and requires specific infrastructure components to function properly.

Dependencies

Before setting up the Email Gateway, ensure you have the following dependencies in place:

PostgreSQL database for storing email gateway data
MongoDB connection to the app-runtime database for runtime data access
Kafka for event-driven communication with other FlowX.AI services
Redis for caching
Keycloak (or compatible OAuth2 provider) for authentication and authorization

The Email Gateway does not require an ingress configuration as it operates as an internal service communicating via Kafka.

Infrastructure prerequisites

Component	Description
PostgreSQL	Dedicated database named `email_gateway`
MongoDB	Shared connection to the `app-runtime` database
Kafka	Message broker for inter-service communication
Redis	Caching layer for improved performance
Keycloak	Identity provider for service authentication

Configuration

Authorization configuration

The Email Gateway validates incoming tokens with the JWT public key mechanism and authenticates to other FlowX services with a dedicated service account (the mainIdentity client registration) in the service-accounts realm:

Environment Variable	Description	Default Value
`SECURITY_TYPE`	Token validation mechanism (JWT public key validation)	`jwt-public-key`
`SECURITY_OAUTH2_BASESERVERURL`	Base URL of the Keycloak server
`SECURITY_OAUTH2_SAREALM`	Service-accounts realm ID	`00000002-0002-4002-8002-000000000002`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID`	Service account client ID	`flowx-email-gateway-sa`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET`	Service account client secret (Keycloak-issued)
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTID`	Anonymous service account client ID, used for anonymous runtime access	`flowx-anonymous-sa`
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_ANONYMOUSIDENTITY_CLIENTSECRET`	Anonymous service account client secret
`SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKENURI`	Provider token URI, resolved against the service-accounts realm	`${SECURITY_OAUTH2_BASESERVERURL}/realms/${SECURITY_OAUTH2_SAREALM}/protocol/openid-connect/token`

Upgrading from 5.1.x? Remove the legacy opaque-token env vars: SECURITY_OAUTH2_REALM, SECURITY_OAUTH2_CLIENT_CLIENTID, SECURITY_OAUTH2_CLIENT_CLIENTSECRET, and SECURITY_OAUTH2_SERVICEACCOUNT_ADMIN_*. These belong to the removed introspection model and prevent the service from starting on 5.9.x. See the authentication and IAM migration guide for the full list.

PostgreSQL configuration

The Email Gateway uses its own dedicated PostgreSQL database for storing email-related data.

Environment Variable	Description	Default Value
`SPRING_DATASOURCE_URL`	JDBC connection URL for PostgreSQL	`jdbc:postgresql://postgresql:5432/email_gateway`
`SPRING_DATASOURCE_USERNAME`	Database username	`flowx`
`SPRING_DATASOURCE_PASSWORD`	Database password	-

Ensure the email_gateway database is created before deploying the service. The Email Gateway will manage its own schema migrations.

MongoDB configuration (app-runtime)

The Email Gateway connects to the shared app-runtime MongoDB database for accessing runtime data.

Environment Variable	Description	Default Value
`SPRING_DATA_MONGODB_RUNTIME_URI`	MongoDB connection URI for app-runtime	`mongodb://${DB_USERNAME}:${DB_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/app-runtime?retryWrites=false`
`DB_USERNAME`	MongoDB username	`app-runtime`
`DB_PASSWORD`	MongoDB password	-

The retryWrites=false parameter is required for compatibility with MongoDB replica set configurations that include arbiters.

Redis configuration

Email Gateway uses Redis for caching. Configure Redis connection using the standard Redis environment variables. Quick reference:

Environment Variable	Description	Example Value	Status
`SPRING_DATA_REDIS_HOST`	Redis server hostname	`localhost`	Recommended
`SPRING_DATA_REDIS_PORT`	Redis server port	`6379`	Recommended
`SPRING_DATA_REDIS_PASSWORD`	Redis authentication password	-	Recommended
`REDIS_TTL`	Cache TTL in milliseconds	`5000000`	Optional

Both SPRING_DATA_REDIS_* and SPRING_REDIS_* variable prefixes are supported. The SPRING_DATA_REDIS_* prefix is the modern Spring Boot standard and is recommended for new deployments.

For advanced Redis deployment modes (Sentinel, Cluster) and SSL/TLS setup, see the Redis Configuration guide. Note that Sentinel and Cluster modes are only supported by the Events Gateway service.

Kafka configuration

Core Kafka settings

Environment Variable	Description	Default Value
`SPRING_KAFKA_BOOTSTRAP_SERVERS`	Address of the Kafka server(s)	`localhost:9092`
`SPRING_KAFKA_SECURITY_PROTOCOL`	Security protocol for Kafka connections	`PLAINTEXT`
`KAFKA_MESSAGE_MAX_BYTES`	Maximum message size (bytes)	`52428800` (50 MB)
`KAFKA_AUTHEXCEPTIONRETRYINTERVAL`	Retry interval after authorization exceptions (seconds)	`10`

OAuth authentication (when using SASL_PLAINTEXT)

Environment Variable	Description	Default Value
`KAFKA_OAUTH_CLIENT_ID`	OAuth client ID	`kafka`
`KAFKA_OAUTH_CLIENT_SECRET`	OAuth client secret	`kafka-secret`
`KAFKA_OAUTH_TOKEN_ENDPOINT_URI`	OAuth token endpoint	`kafka.auth.localhost`

When using the kafka-auth profile, the security protocol will automatically be set to SASL_PLAINTEXT and the SASL mechanism will be set to OAUTHBEARER.

Topic naming configuration

Environment Variable	Description	Default Value
`KAFKA_TOPIC_NAMING_PACKAGE`	Package prefix for topic names	`ai.flowx.`
`KAFKA_TOPIC_NAMING_ENVIRONMENT`	Environment segment for topic names
`KAFKA_TOPIC_NAMING_VERSION`	Version suffix for topic names	`.v1`
`KAFKA_TOPIC_NAMING_SEPARATOR`	Primary separator for topic names	`.`
`KAFKA_TOPIC_NAMING_SEPARATOR2`	Secondary separator for topic names	`-`

IMAP configuration

Configure the Email Gateway’s IMAP polling behavior for inbound email processing:

Environment Variable	Description	Default Value
`EMAIL_GATEWAY_IMAP_CONNECTION_TIMEOUT`	IMAP server connection timeout in milliseconds	`10000`
`EMAIL_GATEWAY_IMAP_MAX_CONCURRENT_POLLS`	Maximum number of concurrent IMAP polling operations	`30`
`EMAIL_GATEWAY_IMAP_MAX_MESSAGES_PER_POLL`	Maximum messages to fetch per polling cycle	`50`
`EMAIL_GATEWAY_IMAP_POLLING_INTERVAL`	Interval between IMAP polling cycles in seconds	`30`
`EMAIL_GATEWAY_IMAP_READ_TIMEOUT`	IMAP read timeout in milliseconds	`30000`

Microsoft Graph configuration

Environment Variable	Description	Default Value
`EMAIL_GATEWAY_MS_GRAPH_MAX_MESSAGES_PER_POLL`	Maximum messages to fetch per MS Graph polling cycle	`50`
`EMAIL_GATEWAY_MS_GRAPH_CLIENT_CACHE_TTL`	TTL for cached MS Graph client instances	`1d`

The client cache TTL controls how long authenticated MS Graph client instances are kept in memory before being refreshed. Increase this value in stable environments to reduce token refresh overhead.

Master election configuration

The Email Gateway uses a master election mechanism to coordinate work across multiple replicas:

Environment Variable	Description	Default Value
`EMAIL_GATEWAY_MASTER_ELECTION_CRON`	Cron expression for master election checks	`/5 * * * *`
`EMAIL_GATEWAY_MASTER_ELECTION_TTL`	Master election TTL in seconds	`15`

Rebalancing configuration

Configure how email accounts are distributed across service instances:

Environment Variable	Description	Default Value
`EMAIL_GATEWAY_MAX_EMAILS_PER_INSTANCE`	Maximum email accounts per service instance	`100`
`EMAIL_GATEWAY_REBALANCING_CRON`	Cron expression for email account rebalancing	`/30 * * * *`
`EMAIL_GATEWAY_STALE_TIMEOUT_SECONDS`	Timeout in seconds before marking an instance as stale	`90`

Kafka topics

The Email Gateway publishes to the following Kafka topics:

Environment Variable	Description	Default Value
`KAFKA_TOPIC_AUDIT_OUT`	Topic for audit event logging	`ai.flowx.core.trigger.save.audit.v1`
`KAFKA_TOPIC_PROCESS_START_OUT`	Topic for triggering process starts from email events	`ai.flowx.core.trigger.start-for-event.process.v1`

Logging configuration

Control logging levels for different components:

Environment Variable	Description	Default Value
`LOGGING_LEVEL_ROOT`	Root logging level	`INFO`
`LOGGING_LEVEL_APP`	Application-specific log level	`INFO`
`LOGGING_CONFIG_FILE`	Path to external logging config file	-

Secrets management

The Email Gateway requires several secrets to be configured. These should be stored securely and referenced via Kubernetes secrets or a secrets management solution.

Secret Name	Description
`SPRING_DATASOURCE_PASSWORD`	PostgreSQL database password
`DB_PASSWORD`	MongoDB password for app-runtime access
`SPRING_REDIS_PASSWORD`	Redis authentication password
`SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET`	Keycloak service account secret
`KAFKA_OAUTH_CLIENT_SECRET`	Kafka OAuth client secret (if using OAuth)

Deployment

Helm values example

Below is an example Helm values configuration for deploying the Email Gateway:

fullnameOverride: email-gateway

image:
  repository: <your-registry>/email-gateway

replicaCount: 1

env:
  SPRING_PROFILES_ACTIVE: production
  
  # PostgreSQL
  SPRING_DATASOURCE_URL: jdbc:postgresql://postgresql:5432/email_gateway
  SPRING_DATASOURCE_USERNAME: flowx
  
  # MongoDB (app-runtime)
  SPRING_DATA_MONGODB_RUNTIME_URI: "mongodb://${DB_USERNAME}:${DB_PASSWORD}@mongodb-0.mongodb-headless,mongodb-1.mongodb-headless,mongodb-arbiter-0.mongodb-arbiter-headless:27017/app-runtime?retryWrites=false"
  DB_USERNAME: app-runtime
  
  # Kafka
  SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092
  KAFKA_OAUTH_CLIENT_ID: kafka-client
  KAFKA_OAUTH_TOKEN_ENDPOINT_URI: https://keycloak.example.com/auth/realms/kafka-authz/protocol/openid-connect/token
  
  # OAuth2
  SECURITY_OAUTH2_BASESERVERURL: https://keycloak.example.com/auth
  SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTID: flowx-email-gateway-sa
  
  # Redis
  SPRING_REDIS_HOST: redis-master

# Secrets configuration
extraEnvVarsMultipleSecretsCustomKeys:
  - name: postgresql-generic
    secrets:
      SPRING_DATASOURCE_PASSWORD: postgresql-password-key
  - name: mongodb-generic
    secrets:
      DB_PASSWORD: mongodb-password
  - name: redis-generic
    secrets:
      SPRING_REDIS_PASSWORD: redis-password
  - name: flowx-auth
    secrets:
      SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_MAINIDENTITY_CLIENTSECRET: keycloakEmailGatewayClientSecret

rbac:
  create: true

# Email Gateway does not expose HTTP endpoints externally
ingress:
  enabled: false

podLabels:
  flowx.ai/network-log: "true"
  flowx.ai/egress-s-kafka: "true"
  flowx.ai/egress-s-postgresql: "true"
  flowx.ai/egress-s-mongodb: "true"
  flowx.ai/routing-name: "email-gateway"
  flowx.ai/prometheus-scrape: "email-gateway"

Network policies

The Email Gateway requires network access to the following services:

Service	Purpose	Pod Label
Kafka	Message broker communication	`flowx.ai/egress-s-kafka`
PostgreSQL	Primary data storage	`flowx.ai/egress-s-postgresql`
MongoDB	App-runtime data access	`flowx.ai/egress-s-mongodb`

Monitoring

The Email Gateway exposes Prometheus metrics for monitoring. Enable scraping by setting the pod label:

podLabels:
  flowx.ai/prometheus-scrape: "email-gateway"

Health endpoints

Endpoint	Description
`/actuator/health`	Health check endpoint
`/actuator/metrics`	Prometheus metrics endpoint
`/actuator/info`	Application info endpoint

Troubleshooting

Common issues

Database connection failures

Symptoms: Service fails to start with database connection errors.Solutions:

Verify the email_gateway database exists in PostgreSQL
Check that the database user has appropriate permissions
Ensure network connectivity between the pod and PostgreSQL service
Verify the JDBC URL format is correct

MongoDB connection issues

Symptoms: Errors accessing app-runtime data.Solutions:

Verify MongoDB replica set is healthy
Check that retryWrites=false is set in the connection URI
Ensure the MongoDB user has read access to the app-runtime database
Verify network policies allow MongoDB traffic

Kafka authentication failures

Symptoms: Kafka consumer/producer fails to connect.Solutions:

Verify OAuth token endpoint is accessible
Check Kafka OAuth client credentials
Ensure the kafka-authz realm exists in Keycloak
Verify network connectivity to both Kafka and Keycloak

Service account authentication errors

Symptoms: 401/403 errors when calling other FlowX services.Solutions:

Verify the Keycloak service account is properly configured
Check that client secrets match between configuration and Keycloak
Ensure the service account has required roles assigned

Email Trigger

Configure IMAP connections to trigger processes from incoming emails

Email Sender

Configure SMTP connections to send emails from processes

Microsoft Outlook

Connect to Outlook via MS Graph API for reading and sending emails

Redis Configuration

Complete Redis setup including Sentinel and Cluster modes

Kafka Authentication

Configure Kafka security and authentication

IAM Configuration

Identity and access management setup

Microservices

AI Platform

Plugins

Observability

Access management

Email Gateway setup

Dependencies

Infrastructure prerequisites

Configuration

Authorization configuration

PostgreSQL configuration

MongoDB configuration (app-runtime)

Redis configuration

Kafka configuration

Core Kafka settings

OAuth authentication (when using SASL_PLAINTEXT)

Topic naming configuration

IMAP configuration

Microsoft Graph configuration

Master election configuration

Rebalancing configuration

Kafka topics

Logging configuration

Secrets management

Deployment

Helm values example

Network policies

Monitoring

Health endpoints

Troubleshooting

Common issues

Email Trigger

Email Sender

Microsoft Outlook

Redis Configuration

Kafka Authentication

IAM Configuration

​Dependencies

​Infrastructure prerequisites

​Configuration

​Authorization configuration

​PostgreSQL configuration

​MongoDB configuration (app-runtime)

​Redis configuration

​Kafka configuration

​Core Kafka settings

​OAuth authentication (when using SASL_PLAINTEXT)

​Topic naming configuration

​IMAP configuration

​Microsoft Graph configuration

​Master election configuration

​Rebalancing configuration

​Kafka topics

​Logging configuration

​Secrets management

​Deployment

​Helm values example

​Network policies

​Monitoring

​Health endpoints

​Troubleshooting

​Common issues

​Related resources

Email Trigger

Email Sender

Microsoft Outlook

Redis Configuration

Kafka Authentication

IAM Configuration

Dependencies

Infrastructure prerequisites

Configuration

Authorization configuration

PostgreSQL configuration

MongoDB configuration (app-runtime)

Redis configuration

Kafka configuration

Core Kafka settings

OAuth authentication (when using SASL_PLAINTEXT)

Topic naming configuration

IMAP configuration

Microsoft Graph configuration

Master election configuration

Rebalancing configuration

Kafka topics

Logging configuration

Secrets management

Deployment

Helm values example

Network policies

Monitoring

Health endpoints

Troubleshooting

Common issues

Related resources