> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flowx.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Reporting plugin

> Build custom and technical reports from BPMN process data and process instance data, integrated with the FlowX.AI Engine.

### Quick walkthrough video

Watch this quick walkthrough video to get started:

<video controls src="https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/release40/Reporting%20%281%29.mp4" />

### Data exploration and visualization

The plugin uses **Superset** as a free data exploration and visualization tool.

> [!TIP]
>
> You can however use your own BI tool like Tableau, PowerBI etc. as an alternative to Superset.

Use the suggested query structure and logic to ensure accurate reporting and avoid duplicates, even during database updates. Do not just use select on single reporting tables.

![Reporting plugin overview](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/reporting.png)

Apache Superset is an open-source software application for data exploration and data visualization able to handle data at a large scale. It enables users to connect their company's databases and perform data analysis, and build charts and assemble dashboards.

Superset is also an SQL IDE, so you can write SQL, join data, create datasets, and more.

> [card] [Superset documentation](https://superset.apache.org/docs/intro/)

### Plugin architecture

Here’s an overview of the reporting plugin architecture:

![Reporting plugin architecture diagram](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/release40/Reporting_diagram_small2.png)

### Setting up reporting for a process definition

**1. Establish reporting data model to be extracted from process**

If you want to extract custom business parameters for a process, first you should set up the master reporting data model in the published version of that process. This is done in Designer, as explained below in the [**Reporting Data Model**](#reporting-data-model).

The master reporting data model that is applied to all historical versions is the one belonging to the version that is set as Published. If the currently published version is read-only, you might need to create a new version, create the data model in it, mark it for reporting and set it as published (as explained in the section below).

See [**Reporting Data Model**](#reporting-data-model) section for more details.

**2. Enable reporting on the process versions to be reported**

This is accomplished by checking the “Use process in reporting” button in the settings / general page of each process version you want to report. Please note that you will not be able to change that setting on read-only versions.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/release40/use_proc_reporting.gif)

It is essential that the currently Published version has this setting enabled, as it is the master version for reporting all historical data.

Only the process instances belonging to reporting-enabled versions will be extracted to the database, using the master data model from the currently Published version.

> [!NOTE]
>
> The reason why you should first set up the data model on the published (or to-be-published) version is that all changes in the data model of the published version that is also marked “use\_in\_reporting” are instantly sent to the reporting plugin, potentially causing all historical process instances to be re-computed multiple times before you are done setting up the data model.

To optimize operations, you should always aim to finish the modifications on the master data model on a version that is either not published, or not marked as "use\_in\_reporting", before marking it as "use\_in\_reporting and ensuring it is published.

See [**Enabling Process Reporting**](#enabling-process-reporting) section for more details.

## Reporting data model

> [!NOTE]
>
> You can only create or modify the reporting data model in non-committed versions, as committed versions are read-only.

1. Open the branch view.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/branching_view.png)

2. If the current project version is committed, start a new version.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/start_new_version.png)

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/release40/rep6.png)

3. Make sure the new work-in-progress branch is selected, then go back to the process definition page.
4. Click on the Data Model icon and navigate in the object model to the target parameters.
5. Set up the business data structure to be reported by using the "Use in reporting" flag.

* Click "Show details".
* Click "Use in reporting".

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/data_to_be_reported.png)

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/flag_use_in_reporting.gif)

You can do this also for each parameter individually.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/parameter_type.png)

> [!NOTE]
>
> There are three parameter structures that you can report:
>
> **Singleton** or primitive, for which a single value is saved for each process instance. They can be of the types number, string, Boolean or date, and will all be found in the reporting table named `params_{process_alias}`. Date and zoned date-time attributes are reported as timestamp columns; any other type is reported as a string.
>
> **Array of primitives**, in which several rows are saved for each process instance, in a dedicated table (one value per row).
>
> **Array of objects**, in which the system extracts one or more “leaves” from each object of an array of objects. Also saved as a dedicated table.

### Primitive parameters reporting

In the following example, there are 3 simple parameters that will be extracted: `loan_approvalFee`, `loan_currency` and `loan_downPayment`.
They will be reported in the table `params_{process_alias}`, one row per instance.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/primitive_parameters_reporting.png)

### Arrays of primitives reporting

In the following example, the applicant can submit more than one email address, so each email will be extracted to a separate row in a dedicated table in the reporting database.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/arrays_of_primitives.png)

> [!WARNING]
>
> Extracting arrays is computationally intensive, do not use them if the only purpose is just to aggregate them afterward. Aggregation should be performed in the process, not in the reporting stage.

### Array of objects reporting

In this example, the applicant can have several real estate assets, so a subset of data items (`currentValue`, `mortgageBalance`, `propertyType`) will be extracted for each one of them.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/470/array_of_objects.png)

> [!WARNING]
>
> Extracting arrays of objects is even more is computationally intensive, do not use them if the only purpose is just to aggregate them afterward. Aggregation should be performed in the process, not in the reporting stage.

> [!WARNING]
>
> **Keep a reporting field's type stable across process versions.** A key mapped into the reporting data model is expected to hold the same type in every version. If you repurpose an existing reporting key to a different type between versions — for example, a key that held an array of objects is later used to store a plain string — reporting can stop for the affected versions, because the extracted value no longer matches the type already established for that field in the reporting database. If a field genuinely needs a different type, add a new key for it rather than reusing the existing one. If a key's type has already changed and reporting has stopped, removing that key from the reporting data model restores ingestion.

## Enabling process reporting

Enable reporting by checking the “Use process in reporting” button on the settings/general page of each process.

* Modifying the data model of a process version has no impact until the moment the version or build of the project is set in the "Active policy" tab and is also set to "use\_in\_reporting".

The reporting refresh schedule is set for a fixed interval (currently 5 minutes):

* No processing overlaps are allowed. If processing takes more than 5 minutes, the next processing is automatically postponed until the current one is finished.
* The reporting plugin chooses the number of Spark executors at each run, based on the volume of data to be processed: one executor for small deltas, up to the configured maximum for large ones. It does not use Spark dynamic allocation, and the `instances` value in the chart is only the driver's starting point.
* Rebuilding the whole history for a process (if the master data model, process name or the Published version change) typically takes more time. It is better to make these changes after the working hours.

## Reporting database

### Main tables

Common fields for joins:

* `inst_id`: Unique identifier for each process instance
* `query_time`: Timestamp for when the plugin extracts data
* `organization_id`, `workspace_id`, `build_app_version_id`: The organization, workspace and app version the instance belongs to

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/release40/rep20.png)

**Useful fields from instances table**:

* `date_started/finished` timestamps
* `process_alias` (process name with "\_" instead of spaces)

> [!NOTE]
>
> The published version alias will be used for all the reported versions.

* State of the process (started, finished, etc.)
* `context_name`: Useful if the process is started by another process

**Useful fields from current\_nodes table**:

* `node_started/finished` timestamps for the nodes
* `node_name`, `node_type`
* `swimlane` of the current node
* `prev_node_end`: For calculating when a token is waiting for another process branch

**Useful fields from token\_history table**:

Similar to `current_nodes` but includes all nodes in the instance history.

### Parameters and object tables

Common fields, on which the joins are built:

* `inst_id`, unique identifier for each process instance
* `query_time`, recorded at the moment the plugin extracts data from the database
* `organization_id`, `workspace_id`, `build_app_version_id`, carried on every parameter and array table

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/release40/reporting30.png)

For the `params_` table, there is a single row per process instance. For arrays and arrays of objects tables, there are multiple rows per process instance.

## Reporting for UI Flows and workflows

Besides process instances, the reporting plugin extracts **UI Flow sessions** and **integration workflow** runs into the same reporting database.

### Turning it on

For a UI Flow, open its **General settings** and toggle **Use UI flow in reporting** **ON**. For an integration workflow, open its **Settings** and toggle **Use workflow in reporting** **ON**.

As with processes, only resources that belong to the active build are extracted.

### How the columns are decided

Processes are reported attribute by attribute: you mark each data model attribute for reporting, and the plugin builds the columns from those definitions.

UI Flows and workflows have no per-attribute marking. The plugin infers the structure from the data itself on every run, so each variable present in a session or a token becomes a column, and every array is exploded into its own child table.

> [!NOTE]
>
> Because the structure is inferred rather than declared, the columns follow whatever the data contains. Renaming or restructuring variables changes the columns on the next run.

### UI Flow tables

| Table | Rows | Keys |
| - | - | - |
| `uiflow_sessions` | One per session | `session_id`, `uf_query_time` |
| `uiparams_{app_uiflow_id}` | One per session | `uf_session_id`, `uf_query_time` |
| `arr_{app_uiflow_id}_{branch}` | One per array element | `uf_session_id`, `uf_query_time` |

`uiflow_sessions` also carries `app_uiflow_id`, `resource_id`, `status`, `uf_started` and `uf_updated`.

> [!WARNING]
>
> The session identifier is named `session_id` in `uiflow_sessions` but `uf_session_id` in the parameter and array tables, so joins are `uiflow_sessions.session_id = uiparams_{app_uiflow_id}.uf_session_id`.

> [!NOTE]
>
> For a session that is still in progress, `uf_updated` falls back to the start date, because the engine only stamps the update date when a session finishes.

### Workflow tables

| Table | Rows | Keys |
| - | - | - |
| `wf_instances` | One per workflow instance | `wf_inst_uuid`, `wf_query_time` |
| `wf_tokens` | One per token | `token_uuid`, `wf_query_time` |
| `wfparams_{app_workflow_id}` | One per token | `wf_token_id`, `wf_query_time` |
| `arr_{app_workflow_id}_{branch}` | One per array element | `wf_token_id`, `wf_query_time` |

`wf_instances` also carries `app_workflow_id`, `workflow_resource_id`, `wf_status`, `wf_started` and `wf_finished`. `wf_tokens` adds `token_status` and `wf_updated`, and links to its instance through `wf_inst_uuid`.

The same naming difference applies here: the token identifier is `token_uuid` in `wf_tokens` and `wf_token_id` in the parameter and array tables.

Every table listed above also carries `organization_id`, `workspace_id` and `build_app_version_id`.

> [!NOTE]
>
> Array table names are truncated to 64 characters, so deeply nested branches with long names can produce shortened table names.

***

## Using Superset for data exploration

### Data sources

The **Data** tab represents the sources of all information:

* Databases
* CSV files
* Tables
* Excel files

Reporting plugin can be used with Superset by connecting it with a PostgreSQL DB.

### Charts

Charts represent the output of the information. There are multiple visualization charts/ plugins available.

### Dashboards

With the use of dashboards, you can share persuading flows, show how metrics change in various scenarios and match your company efforts with logical, evidence‐based visual indicators.

### Datasets

Contains all the information for extracting and processing data from the DB, including SQL queries, calculated metrics information, cache settings, etc. Datasets can also be exported / imported.

### Connecting to a database

Before using Superset, ensure you have a PostgreSQL database installed and configured. Follow these guides for setup:

[FlowX Engine DB configuration](https://docs.flowx.ai/6.x/setup-guides/flowx-engine-setup-guide/engine-setup.md)

[Reporting DB configuration](https://docs.flowx.ai/6.x/setup-guides/plugins-setup-guide/reporting-setup.md)

> [!NOTE]
>
> Read-only users should be used in production in the reporting-plugin cronjob.

To connect Superset to a database, follow the next steps:

1. From the Toolbar, hover your cursor over the **"+"** icon, then select **Data**, and then select **Connect Database**.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/connecting_to_db.png)

2. The **Connect a database** window appears. Select the appropriate **database card** (in this case - PostgreSQL).

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/connect_db_superset.png)

3. After you selected the DB, click **Connect this database with a SQLAlchemy URI string instead?**.
4. Fill in the **SQLALCHEMY URI** and then click **Connect**.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/superset_db_URI.png)

> [!NOTE]
>
> The **SQLAlchemy URI** for **reporting-db** should be in this format: `postgresql://postgres:XXXXXXXXXX@reporting-plugin-postgresql:{{port}}/reporting`.

### Creating and configuring charts

There are multiple ways in which you can create/configure a chart.

#### Creating charts using Datasets tab

To create a Chart using the first method, you must follow the next steps:

1. From the top toolbar, select **Data** and then **Datasets**.

> [!TIP]
>
> You need to have a dataset added to Superset first. From that particular dataset you can build a visualization.

2. Select the desired **dataset**.
3. On the **explore** page, choose the **visualization type** and click **Create chart**.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/superset_add_chart.gif)

4. When you select a dataset, by default **table** visualization type is selected.

> [!NOTE]
>
> To view all the existent visualization types, click **View all charts**, the charts' gallery will open.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/superset_visualization_type.png)

#### Creating charts using Chart gallery

Using the Chart gallery is a useful method when you are not quite sure about which chart will be best for you.

To create a Chart using the second method, you must follow the next steps:

1. Select the **"+"** icon from the top toolbar and choose **Chart**.
2. Choose the **dataset** and **chart type**.
3. Review the description and example graphics of the selected chart, then click **Create new chart**.

> [!TIP]
>
> If you wish to explore all the chart types available, filter by **All charts**. The charts are also grouped by categories.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/superset_add_chart_second.gif)

### Configuring a Chart

Configure the **Query** fields by dragging and dropping relevant columns to the matching fields. Note that time and query form attributes vary by chart type.

### Exporting/importing a Chart

You can export and import charts to help you analyze your data and manipulate dashboards. To export/import a chart, follow the next steps:

1. Open **Superset** and navigate to **Charts** from the top navigation bar.
2. Select the desired **chart** and click the **breadcrumbs** menu in the top-right corner.
3. Choose an export option: .CSV, .JSON, or Download as image.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/reporting_export_imp.png)

**Table example**:

* **Time** - time related form attributes
* **Query** - query attributes
  * **Dimensions** - one or many columns to group by
  * **Metrics** - metrics to display
  * **Percentage metrics** - metrics for which percentage of total are to be displayed, calculated from only data within the row limit
  * **Filters** - metric used for filtering
  * **Sort by** - metric used to define how the top series are sorted if a series or row limit is present
  * **Row limit** - limits the number of rows that get displayed

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/superset_query.png)

### Creating a dashboard

To create a dashboard follow the next steps:

1. Create a new chart and save it to a new dashboard.
2. To publish, click **Save and go to Dashboard**.

![](https://s3.eu-west-1.amazonaws.com/docx.flowx.ai/platform-deep-dive/save_dashboard.gif)

For details on how to configure the FlowX.AI reporting plugin, check the following section:

> [card] [Reporting setup guide](https://docs.flowx.ai/6.x/setup-guides/plugins-setup-guide/reporting-setup.md)


## Related topics

- [FlowX custom plugins](https://docs.flowx.ai/6.x/docs/platform-deep-dive/plugins/custom-plugins.md)
- [FlowX.AI architecture: services, layers, and data flow](https://docs.flowx.ai/6.x/docs/platform-overview/flowx-architecture.md)
- [Reporting setup](https://docs.flowx.ai/6.x/setup-guides/plugins-setup-guide/reporting-setup.md)
- [Process data model](https://docs.flowx.ai/6.x/docs/building-blocks/process/data-model.md)
- [Authorization & access roles](https://docs.flowx.ai/6.x/docs/platform-deep-dive/plugins/custom-plugins/reporting/access-and-authorization.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.