Infrastructure prerequisites

The Audit service requires the following components to be set up before it can be started:

  • Docker engine - version 17.06 or higher
  • Kafka - version 2.8 or higher
  • Elasticsearch - version 7.11.0 or higher

Dependencies

The Audit service is built as a Docker image and runs on top of Kafka and Elasticsearch. Therefore, these services must be set up and running before starting the Audit service.

Configuration

Configuring Kafka

To configure the Kafka server for the Audit service, set the following environment variables:

Connection settings

VariableDescriptionExample Value
SPRING_KAFKA_BOOTSTRAPSERVERSAddress of the Kafka server(s)localhost:9092
SPRING_KAFKA_SECURITYPROTOCOLProtocol used to communicate with brokersPLAINTEXT

Consumer configuration

VariableDescriptionExample Value
SPRING_KAFKA_CONSUMER_GROUPIDConsumer group ID for audit logsaudit-gid
SPRING_KAFKA_CONSUMER_PROPERTIES_MAX_PARTITION_FETCH_BYTESMaximum data size per partition${KAFKA_MESSAGE_MAX_BYTES:52428800}
KAFKA_CONSUMER_THREADSNumber of consumer threads1
KAFKA_AUTHEXCEPTIONRETRYINTERVALRetry interval after auth failure (seconds)10

Topic naming configuration

VariableDescriptionExample Value
DOTReference to primary separator${kafka.topic.naming.separator}
DASHReference to secondary separator${kafka.topic.naming.separator2}
KAFKA_TOPIC_NAMING_SEPARATORPrimary separator for topic names.
KAFKA_TOPIC_NAMING_SEPARATOR2Secondary separator for topic names-
KAFKA_TOPIC_NAMING_PACKAGEBase namespace for topicsai${dot}flowx${dot}
KAFKA_TOPIC_NAMING_ENVIRONMENTEnvironment indicatordev${dot}
KAFKA_TOPIC_NAMING_VERSIONVersion component${dot}v1
KAFKA_TOPIC_NAMING_PREFIXCombined prefix${kafka.topic.naming.package}${kafka.topic.naming.environment}
KAFKA_TOPIC_NAMING_SUFFIXCombined suffix${kafka.topic.naming.version}
KAFKA_TOPIC_AUDIT_INTopic for receiving audit logs${kafka.topic.naming.prefix}core${dot}trigger${dot}save${dot}audit${kafka.topic.naming.suffix}

With default settings, the KAFKA_TOPIC_AUDIT_IN resolves to: ai.flowx.dev.core.trigger.save.audit.v1

Configuring Elasticsearch

To configure Elasticsearch, set the following environment variables:

VariableDescriptionExample Value
SPRING_ELASTICSEARCH_REST_PROTOCOLProtocol for ES connectionhttps
SPRING_ELASTICSEARCH_REST_URISURL(s) of Elasticsearch nodeslocalhost:9200
SPRING_ELASTICSEARCH_REST_DISABLESSLDisable SSL for connectionsfalse
SPRING_ELASTICSEARCH_REST_USERNAMEUsername for authenticationelastic
SPRING_ELASTICSEARCH_REST_PASSWORDPassword for authenticationyour-secure-password
SPRING_ELASTICSEARCH_INDEX_SETTINGS_DATASTREAMName of the audit data streamaudit-logs
SPRING_ELASTICSEARCH_INDEX_SETTINGS_SHARDSNumber of primary shards2
SPRING_ELASTICSEARCH_INDEX_SETTINGS_REPLICASNumber of replica shards2

The Elasticsearch index settings determine how your audit data is distributed and replicated across the cluster. The number of shards affects search performance and indexing, while replicas provide redundancy.

Configuring logging

To control the log levels, set the following environment variables:

VariableDescriptionExample Value
LOGGING_LEVEL_ROOTLog level for root serviceINFO
LOGGING_LEVEL_APPLog level for applicationINFO