The FlowX.AI Reporting plugin helps you build and bootstrap custom reports using data from your BPMN processes. Moreover, it supports technical reports based on process instance data. Integrated with the FlowX.AI Engine, this plugin transforms raw data into actionable insights, enhancing decision-making and optimizing business processes.
Watch this quick walkthrough video to get started:
The plugin uses Superset as a free data exploration and visualization tool.
You can however use your own BI tool like Tableau, PowerBI etc. as an alternative to Superset.
Use the suggested query structure and logic to ensure accurate reporting and avoid duplicates, even during database updates. Do not just use select on single reporting tables.
Apache Superset is an open-source software application for data exploration and data visualization able to handle data at a large scale. It enables users to connect their company’s databases and perform data analysis, and easily build charts and assemble dashboards.
Superset is also an SQL IDE, so users can write SQL, join data create datasets and so on.
Here’s an overview of the reporting plugin architecture:
Establish reporting data model to be extracted from process
If you want to extract custom business parameters for a process, first you should set up the master reporting data model in the published version of that process. This is done in Designer, as explained below in the Reporting Data Model.
The master reporting data model that is applied to all historical versions is the one belonging to the version that is set as Published. If the currently published version is read-only, you might need to create a new version, create the data model in it, mark it for reporting and set it as published (as explained in the section below).
See Reporting Data Model section for more details.
Enable reporting on the process versions to be reported
This is accomplished by checking the “Use process in reporting” button in the settings / general page of each process version you want to report. Please note that you will not be able to change that setting on read-only versions.
It is essential that the currently Published version has this setting enabled, as it is the master version for reporting all historical data.
Only the process instances belonging to reporting-enabled versions will be extracted to the database, using the master data model from the currently Published version.
The reason why you should first set up the data model on the published (or to-be-published) version is that all changes in the data model of the published version that is also marked “use_in_reporting” are instantly sent to the reporting plugin, potentially causing all historical process instances to be re-computed multiple times before you are done setting up the data model.
To optimize operations, you should always aim to finish the modifications on the master data model on a version that is either not published, or not marked as “use_in_reporting”, before marking it as “use_in_reporting and ensuring it is published.
See Enabling Process Reporting section for more details.
You can only create or modify the reporting data model in non-committed versions, as commited versions are read-only.
There are three parameter structures that you can report:
Singleton or primitive, for which a single value is saved for each process instance. They can be of the types number, string or Boolean and will all be found in the reporting table named params_{process_alias}
.
Array of primitives, in which several rows are saved for each process instance, in a dedicated table (one value per row).
Array of objects, in which the system extracts one or more “leaves” from each object of an array of objects. Also saved as a dedicated table.
In the following example, there are 3 simple parameters that will be extracted: loan_approvalFee
, loan_currency
and loan_downPayment
.
They will be reported in the table params_{process_alias}
, one row per instance.
In the following example, the applicant can submit more than one email address, so each email will be extracted to a separate row in a dedicated table in the reporting database.
Extracting arrays is computationally intensive, do not use them if the only purpose is just to aggregate them afterward. Aggregation should be performed in the process, not in the reporting stage.
In this example, the applicant can have several real estate assets, so a subset of data items (currentValue
, mortgageBalance
, propertyType
) will be extracted for each one of them.
Extracting arrays of objects is even more is computationally intensive, do not use them if the only purpose is just to aggregate them afterward. Aggregation should be performed in the process, not in the reporting stage.
Enable reporting by checking the “Use process in reporting” button on the settings/general page of each process version.
You can also make data model changes on an already published version (if it is not set as read-only), but the effects of the changes are immediate and every periodic refresh will try to re-compute all historical instances to adapt to the changing data model, potentially taking a lot of computing.
The reporting refresh schedule is set for a fixed interval (currently 5 minutes):
Modifying the data model for the Published version causes all data for that process to be deleted from the database and to be rebuilt from scratch using the Data Model of the current Published version.
The same thing happens whenever the Published version changes, or if the process name is modified.
Adding Process Versions: Adds new rows to existing tables.
Removing Non-Published Process Versions from reporting (by unchecking their “Use in reporting” switch in Settings) simply deletes their corresponding rows from the table.
Removing the Currently Published Version from Reporting: Not advisable as it applies a random older data model and deletes all instances processed with this version.
Reporting Data Structure Obsolescence and Backwards Compatibility: The master data model is applied to all historical data from all reported versions. If only the name of a parameter changes, the plugin uses its UUID to map the old name to the new one. However, if the full path changes or the parameter does not exist in an older version, it returns a Null
value to the database.
Common fields for joins:
inst_id
: Unique identifier for each process instancequery_time
: Timestamp for when the plugin extracts dataUseful fields from instances table:
date_started/finished
timestampsprocess_alias
(process name with ”_” instead of spaces)The published version alias will be used for all the reported versions.
context_name
: Useful if the process is started by another processUseful fields from current_nodes table:
node_started/finished
timestamps for the nodesnode_name
, node_type
swimlane
of the current nodeprev_node_end
: For calculating when a token is waiting for another process branchUseful fields from token_history table:
Similar to current_nodes
but includes all nodes in the instance history.
Common fields, on which the joins are built:
inst_id
, unique identifier for each process instancequery_time
, recorded at the moment the plugin extracts data from the databaseFor the params_
table, there is a single row per process instance. For arrays and arrays of objects tables, there are multiple rows per process instance.
The Data tab represents the sources of all information:
Reporting plugin can be used with Superset by connecting it with a PostgreSQL DB.
Charts represent the output of the information. There are multiple visualization charts/ plugins available.
With the use of dashboards, you can share persuading flows, show how metrics change in various scenarios and match your company efforts with logical, evidence‐based visual indicators.
Contains all the information for extracting and processing data from the DB, including SQL queries, calculated metrics information, cache settings, etc. Datasets can also be exported / imported.
Before using Superset, ensure you have a PostgreSQL database installed and configured. Follow these guides for setup:
Read-only users should be used in production in the reporting-plugin cronjob.
To connect Superset to a database, follow the next steps:
The SQLAlchemy URI for reporting-db should be in this format: postgresql://postgres:XXXXXXXXXX@reporting-plugin-postgresql:{{port}}/reporting
.
There are multiple ways in which you can create/configure a chart.
To create a Chart using the first method, you must follow the next steps:
You need to have a dataset added to Superset first. From that particular dataset you can build a visualization.
To view all the existent visualization types, click View all charts, the charts’ gallery will open.
Using the Chart gallery is a useful method when you are not quite sure about which chart will be best for you.
To create a Chart using the second method, you must follow the next steps:
If you wish to explore all the chart types available, filter by All charts. The charts are also grouped by categories.
Configure the Query fields by dragging and dropping relevant columns to the matching fields. Note that time and query form attributes vary by chart type.
You can export and import charts to help you analyze your data and manipulate dashboards. To export/import a chart, follow the next steps:
Table example:
To create a dashboard follow the next steps:
For details on how to configure the FlowX.AI reporting plugin, check the following section: