The plugin is available as a docker image.

​
Dependencies

Before setting up the plugin, ensure that you have the following dependencies installed and configured:
  • PostgreSQL Database: You will need a PostgreSQL database to store data related to document templates and documents.
  • MongoDB Database: MongoDB is required for the HTML templates feature of the plugin.
  • Kafka: Establish a connection to the Kafka instance used by the FLOWX.AI engine.
  • Redis: Set up a Redis instance for caching purposes.
  • S3-Compatible File Storage Solution: Deploy an S3-compatible file storage solution, such as Min.io, to store document files.

​
Configuration

The plugin comes with pre-filled configuration properties, but you need to set up a few custom environment variables to tailor it to your specific setup. Here are the key configuration steps:

​
Redis server

The plugin can utilize the Redis component already deployed for the FLOWX.AI engine. Make sure it is configured properly.

​
Authorization configuration

To connect to the identity management platform, set the following environment variables:
Environment VariableDescriptionDefault Value
SECURITY_OAUTH2_BASESERVERURLBase URL of the OAuth2/OIDC serverhttps://keycloak.example.com/auth
SECURITY_OAUTH2_CLIENT_CLIENTIDOAuth2 client ID for the Documents Plugindocument-plugin
SECURITY_OAUTH2_CLIENT_CLIENTSECRETOAuth2 client secretsecret
SECURITY_OAUTH2_REALMOAuth2 realm nameflowx

​
Document processing configuration

Environment VariableDescriptionDefault Value
FLOWX_DEFAULTCLIENTTYPEDefault client typePF
FLOWX_HTML_TEMPLATES_ENABLEDEnable HTML templates featurefalse
FLOWX_HTML_TEMPLATES_PDFFONTPATHSPaths to fonts for HTML templates/statics/fonts/Calibri.ttf, /statics/fonts/DejaVuSans.ttf
FLOWX_CONVERT_DPIDPI setting for PDF to image conversion150
Set the FLOWX_HTML_TEMPLATES_PDFFONTPATHS config to select the font used for generating documents based on PDF templates.
If you want to use specific fonts in your PDF templates, override the FLOWX_HTML_TEMPLATES_PDFFONTPATHS config. By default, Calibri and DejaVuSans are available fonts.
FLOWX_CONVERT_DPI: Sets the DPI (dots per inch) for PDF to JPEG conversion. Higher values result in higher resolution images. (Default value: 150).
After making these configurations, the fonts will be available for use within PDF templates.

​
Database configuration

​
SQL database (PostgreSQL)

The Documents Plugin uses a PostgreSQL database for relational data storage.

​
Primary MongoDB configuration

Environment VariableDescriptionExample Value
SPRING_DATA_MONGODB_URIMongoDB connection URImongodb://flowx:password@jx-document-mongodb:27017/document
MONGODB_USERNAMEMongoDB usernameflowx
MONGODB_PASSWORDMongoDB passwordpassword
SPRING_DATA_MONGODB_UUID_REPRESENTATIONUUID representation formatstandard
SPRING_DATA_MONGODB_STORAGEMongoDB storage typemongodb

​
Runtime MongoDB configuration

Environment VariableDescriptionExample Value
SPRING_DATA_MONGODB_RUNTIME_ENABLEDEnable runtime MongoDB connectiontrue
SPRING_DATA_MONGODB_RUNTIME_URIRuntime MongoDB connection URImongodb://${MONGODB_USERNAME:flowx}:${MONGODB_PASSWORD:password}@jx-document-mongodb:27017/document
MONGODB_USERNAMERuntime MongoDB usernamepaperflow
MONGODB_PASSWORDRuntime MongoDB passwordpassword
SPRING_DATA_MONGODB_RUNTIME_AUTOINDEXCREATIONEnable automatic index creationfalse
SPRING_DATA_MONGODB_RUNTIME_UUIDREPRESENTATIONUUID representation format for runtime connectionstandard

​
Redis configuration

Set the following values with the corresponding Redis-related values:
Environment VariableDescriptionDefault Value
SPRING_DATA_REDIS_HOSTHostname of the Redis serverlocalhost
SPRING_DATA_REDIS_PORTPort number of the Redis server6379
SPRING_DATA_REDIS_PASSWORDPassword for Redis authenticationdefaultpassword
REDIS_TTLTime-to-live (TTL) for Redis cache5000000 (miliseconds)

​
Multipart upload configuration

Environment VariableDescriptionDefault Value
SPRING_SERVLET_CONTEXTPATHServlet context path/
SPRING_SERVLET_MULTIPART_MAXFILESIZEMaximum file size for uploads50MB
SPRING_SERVLET_MULTIPART_MAXREQUESTSIZEMaximum request size for multipart uploads50MB

​
Basic Kafka configuration

Environment VariableDescriptionDefault Value
SPRING_KAFKA_BOOTSTRAPSERVERSAddress of Kafka server(s)localhost:9092
SPRING_KAFKA_SECURITY_PROTOCOLSecurity protocol for KafkaPLAINTEXT
SPRING_KAFKA_CONSUMER_GROUPIDConsumer group ID for the servicekafka-svc-document-consumer-local-test2
KAFKA_MESSAGE_MAX_BYTESMaximum message size in bytes52428800 (50MB)
KAFKA_AUTHEXCEPTIONRETRYINTERVALRetry interval for auth exceptions10 (seconds)

​
Topic naming configuration

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_NAMING_SEPARATORPrimary separator for topic names.
KAFKA_TOPIC_NAMING_SEPARATOR2Secondary separator for topic names-
KAFKA_TOPIC_NAMING_PACKAGEPackage prefix for topic namesai.flowx.
KAFKA_TOPIC_NAMING_ENVIRONMENTEnvironment segment for topic namesdev.
KAFKA_TOPIC_NAMING_VERSIONVersion suffix for topic names.v1

​
Thread configuration

Environment VariableDescriptionDefault Value
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_DOCUMENTGENERATEHTMLINThreads for HTML document generation5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_DOCUMENTPERSISTINThreads for document persistence5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_DOCUMENTSPLITTINThreads for document splitting5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_DOCUMENTGETURLSINThreads for document URL retrieval5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_OCRINThreads for OCR processing5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_FILEDELETEINThreads for file deletion5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_FILEUPDATEINThreads for file updates5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_FILECONVERTINThreads for file conversion5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_FILEPERSISTINThreads for file persistence5
KAFKA_CONSUMER_THREADPOOLS_THREADPOOLGENERIC_THREADCOUNTPERCONTAINER_FILECOMBINEINThreads for file combining5

​
Document topics

​
Generate HTML documents

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_DOCUMENT_GENERATE_HTML_INTopic for incoming HTML generation requestsai.flowx.dev.plugin.document.trigger.generate.html.v1
KAFKA_TOPIC_DOCUMENT_GENERATE_HTML_OUTTopic for HTML generation resultsai.flowx.dev.engine.receive.plugin.document.generate.html.results.v1

​
Document persistence

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_DOCUMENT_PERSIST_INTopic for incoming document persistence requestsai.flowx.dev.plugin.document.trigger.persist.document.v1
KAFKA_TOPIC_DOCUMENT_PERSIST_OUTTopic for document persistence resultsai.flowx.dev.engine.receive.plugin.document.persist.document.results.v1

​
Document splitting

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_DOCUMENT_SPLIT_INTopic for incoming document splitting requestsai.flowx.dev.plugin.document.trigger.split.document.v1
KAFKA_TOPIC_DOCUMENT_SPLIT_OUTTopic for document splitting resultsai.flowx.dev.engine.receive.plugin.document.split.document.results.v1

​
Document URL retrieval

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_DOCUMENT_GET_URLS_INTopic for incoming URL retrieval requestsai.flowx.dev.plugin.document.retrieve.urls.v1
KAFKA_TOPIC_DOCUMENT_GET_URLS_OUTTopic for URL retrieval resultsai.flowx.dev.engine.receive.plugin.document.urls.results.v1

​
OCR processing

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_OCR_INTopic for incoming OCR requestsai.flowx.dev.plugin.document.store.ocr.v1
KAFKA_TOPIC_OCR_OUTTopic for OCR processing resultsai.flowx.dev.engine.receive.plugin.document.ocr.results.v1

​
File operation topics

​
File deletion

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_FILE_DELETE_INTopic for incoming file deletion requestsai.flowx.dev.plugin.document.trigger.delete.file.v1
KAFKA_TOPIC_FILE_DELETE_OUTTopic for file deletion resultsai.flowx.dev.engine.receive.plugin.document.delete.file.results.v1

​
File update

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_FILE_UPDATE_INTopic for incoming file update requestsai.flowx.dev.plugin.document.trigger.update.file.v1
KAFKA_TOPIC_FILE_UPDATE_OUTTopic for file update resultsai.flowx.dev.engine.receive.plugin.document.update.file.results.v1

​
File conversion

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_FILE_CONVERT_OUTTopic for file conversion resultsai.flowx.dev.engine.receive.plugin.document.convert.file.results.v1

​
File persistence

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_FILE_PERSIST_INTopic for incoming file persistence requestsai.flowx.dev.plugin.document.trigger.persist.file.v1
KAFKA_TOPIC_FILE_PERSIST_INTopic for incoming file persistence requestsai.flowx.dev.plugin.document.trigger.persist.file.v1
KAFKA_TOPIC_FILE_PERSIST_OUTTopic for file persistence resultsai.flowx.dev.engine.receive.plugin.document.persist.file.results.v1

​
File combination

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_FILE_COMBINE_INTopic for incoming file combination requestsai.flowx.dev.plugin.document.trigger.combine.file.v1
KAFKA_TOPIC_FILE_COMBINE_OUTTopic for file combination resultsai.flowx.dev.engine.receive.plugin.document.combine.file.results.v1

​
Audit

Environment VariableDescriptionDefault Value
KAFKA_TOPIC_AUDIT_OUTTopic for sending audit logsai.flowx.dev.core.trigger.save.audit.v1

​
General storage configuration

Environment VariableDescriptionDefault Value
APPLICATION_DEFAULTLOCALEDefault locale for the applicationen
APPLICATION_SUPPORTEDLOCALESComma-separated list of supported localesen, ro
APPLICATION_FILESTORAGE_TYPEStorage type to use (s3 or fileSystem)s3
APPLICATION_FILESTORAGE_DISKDIRECTORYDirectory for file storage when using filesystemMS_SVC_DOCUMENT
APPLICATION_FILESTORAGE_PARTITIONSTRATEGYSet the partition strategy for file storage. Use NONE to save documents in minio/amazon-s3 as before, with a bucket for each process instance. Use PROCESS_DATE to save documents in a single bucket with a subfolder structure, for example: bucket/2022/2022-07-04/process-id-xxxx/customer-id/file.pdfNONE
APPLICATION_FILESTORAGE_DELETIONSTRATEGYStrategy for deleting files (delete, disabled, or deleteBypassingGovernanceRetention)delete
APPLICATION_FILESTORAGE_DELETIONSTRATEGY:
  • disabled: This will disable entirely the deletion of temporary files from the temporary bucket, and the responsibility to delete and clean up the bucket will move in the ownership of the admins of the implementing project.
  • deleteBypassingGovernanceRetention: This will still delete the temporary files and further more will add in the delete request the header: x-amz-bypass-governance-retention:true , to enable deletion of governed files, in case the s3 configured user for document-plugin, will have the s3:BypassGovernanceRetention permission.

​
S3-Compatible storage configuration

Environment VariableDescriptionDefault Value
APPLICATION_FILESTORAGE_S3_ENABLEDEnable S3-compatible storagetrue
APPLICATION_FILESTORAGE_S3_SERVERURLURL of MinIO or S3-compatible serverhttp://minio-service:9000
APPLICATION_FILESTORAGE_S3_ACCESSKEYAccess key for MinIO/S3minio
APPLICATION_FILESTORAGE_S3_SECRETKEYSecret key for MinIO/S3minio123
APPLICATION_FILESTORAGE_S3_BUCKETPREFIXPrefix for bucket namesqdevlocal-preview-paperflow
APPLICATION_FILESTORAGE_S3_TEMPBUCKETName of temporary bucket for initial file uploadstemp-bucket
APPLICATION_FILESTORAGE_S3_ENCRYPTIONENABLEDEnable server-side encryptionfalse
Make sure to follow the recommended bucket naming rules when choosing the bucket prefix name.

​
Logging configuration

Environment VariableDescriptionDefault Value
LOGGING_LEVEL_APPLog level for application logsDEBUG
LOGGING_LEVEL_MONGO_DRIVERLog level for MongoDB driverINFO
LOGGING_LEVEL_LIQUIBASELog level for Liquibase database migrationsINFO
LOGGING_LEVEL_REDISLog level for Redis/Lettuce clientOFF
LOGGING_LEVEL_ROOTRoot logging level for the application-