Integration Designer setup

​

Infrastructure prerequisites

The Integration Designer service requires the following components to be set up before it can be started:

​

Configuration

​

Core service configuration

​

WebClient configuration

Integration Designer interacts with various APIs, some of which return large responses. To handle such cases efficiently, the FlowX WebClient buffer size must be configured to accommodate larger payloads, especially when working with legacy APIs that do not support pagination.

​

Database configuration

​

PostgreSQL

​

MongoDB

Integration Designer requires two MongoDB databases for managing integration-specific data and runtime data:

• Integration Designer Database (integration-designer): Stores data specific to Integration Designer, such as integration configurations, metadata, and other operational data.
• Shared Runtime Database (app-runtime): Shared across multiple services, this database manages runtime data essential for integration and data flow execution.

​

Kafka configuration

​

Kafka connection and security variables

​

Message size configuration

This setting affects:

• Producer message max bytes
• Producer max request size

​

Producer configuration

​

Consumer configuration

​

Topic naming convention and pattern creation

The Integration Designer uses a structured topic naming convention that follows a standardized pattern, ensuring consistency across environments and making topics easily identifiable.

Topic naming components

Topics are constructed using the following pattern:

{prefix} + service + {separator/dot} + action + {separator/dot} + detail + {suffix}

For example, a typical topic might look like:

ai.flowx.dev.eventsgateway.receive.workflowinstances.v1

Where:

• ai.flowx.dev. is the prefix (package + environment)
• eventsgateway is the service
• receive is the action
• workflowinstances is the detail
• .v1 is the suffix (version)

​

Kafka topic configuration

Core topics

Events gateway topics

Engine and Integration communication topics

Application resource usage topics

​

OAuth authentication variables (when using SASL_PLAINTEXT)

​

Inter-Service topic coordination

When configuring Kafka topics in the FlowX ecosystem, ensure proper coordination between services:

1. Topic name matching: Output topics from one service must match the expected input topics of another service.

2. Pattern consistency: The pattern values must be consistent across services:

   • Process Engine listens to topics matching: ai.flowx.dev.engine.receive.*
   • Integration Designer listens to topics matching: ai.flowx.dev.integration.receive.*

3. Communication flow:

   • Other services write to topics matching the Engine’s pattern → Process Engine listens
   • Process Engine writes to topics matching the Integration Designer’s pattern → Integration Designer listens

​

Kafka topics best practices

​

Large message handling for workflow instances topic

The workflow instances topic requires special configuration to handle large messages. By default, Kafka has message size limitations that may prevent Integration Designer from processing large workflow payloads.

Recommended max.message.bytes value: 10485760 (10 MB)

​

Method: Update using AKHQ (Recommended)

1. Access AKHQ

   • Open the AKHQ web interface
   • Log in if authentication is required

2. Navigate to Topic

   • Go to the “Topics” section
   • Find the topic: ai.flowx.dev.eventsgateway.receive.workflowinstances.v1

3. Edit Configuration

   • Click on the topic name
   • Go to the “Configuration” tab
   • Locate or add max.message.bytes
   • Set the value to 10485760
   • Save changes

​

Configuring authentication and access roles

Integration Designer uses OAuth2 for secure access control. Set up OAuth2 configurations with these environment variables:

For detailed instructions on configuring user roles and access rights, refer to:

For configuring a service account, refer to:

​

Configuring logging

To control the log levels for Integration Designer, set the following environment variables:

​

Configuring admin ingress

Integration Designer provides an admin ingress route, which can be enabled and customized with additional annotations for SSL certificates or routing preferences.

ingress:
  enabled: true
  admin:
    enabled: true
    hostname: "{{ .Values.flowx.ingress.admin }}"
    path: /integration(/|$)(.*)
    annotations:
      nginx.ingress.kubernetes.io/rewrite-target: /$2
      nginx.ingress.kubernetes.io/cors-allow-headers: DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization,platform,Flowx-Platform

​

Monitoring and maintenance

To monitor the performance and health of the Integration Designer, use tools like Prometheus or Grafana. Configure Prometheus metrics with:

​

RBAC configuration

Integration Designer requires specific RBAC (Role-Based Access Control) permissions to access Kubernetes ConfigMaps and Secrets, which store necessary configurations and credentials. Set up these permissions by enabling RBAC and defining the required rules:

rbac:
  create: true
  rules:
    - apiGroups:
        - ""
      resources:
        - secrets
        - configmaps
        - pods
      verbs:
        - get
        - list
        - watch

This configuration grants read access (get, list, watch) to ConfigMaps, Secrets, and Pods, which is essential for retrieving application settings and credentials required by Integration Designer.

​

Additional resources

• Integration Designer Documentation