Create and Manage Data Flows

Prev Next

Plan availability options include Essentials, Professional, and Enterprise plans for users.

Data Flows provide a streamlined way to group Data Actions and trigger them using an Action button on a Board. This article explains how to create and configure Data Flows effectively.

⚠️ Important

You must have saved Metric-to-Metric, List-to-List, Integration or CSV import configurations in an Application before configuring a Data Flow.

Before you begin

Data Flows are sets of Data Actions grouped into a single configuration. Once created, they can be triggered from a Board using an Action button. This allows users to run multiple Data Actions with a single click.

Parameters can be used to make Data Actions reusable by prompting Members to select values when they run the Data Action. For more information on Parameters, see Customize Data Actions with Parameters.

⚠️ Important

You need the Configure Blocks permission to set up Data Flows.

To run a Data Flow from a Board, you must have permission to perform each individual Data Action in the flow.

Supported Data Actions

By default, Data Flows support Metric-to-Metric, List-to-List, Integration and CSV imports from Pigment.

ℹ️ Note

For CSV imports:

  • You can only upload one CSV file per Data Flow.

  • That one CSV file can be used to populate multiple Blocks. The Data Flow can contain multiple import actions from that file.

  • As with Metric-to-Metric, List-to-List and Integration imports, configuration files for CSV imports must already be set up on each recipient Metric or List.

Data Flow execution order

Data Flows supports three types of Execution Mode:

  • Parallel. This runs all Data Actions at the same time, independently of each other.

  • Sequential. This runs Data Actions one after another, in a predefined order. If a previous step of the Data Flow impacts the source of a Data Action (for instance through a formula), the Data Action waits for the Data source to be updated before starting.

  • Hybrid. This lets you mix sequential and parallel execution in a single flow. Steps run one after another and each step can hold several Data Actions that start together. It can mean shorter execution times than Sequential.

Create a Data Flow

To create a Data Flow:

  1. Select the three-dot menu and choose Data Actions & Flows, or go to All Settings then Data Actions & Flows.

  2. Select Create a Data Flow.

  3. In the Details section, add a name and description:

    1. Use the Name field to identify the Data Flow in the list.

    2. Use the Description field to explain what the Data Flow does and the business logic it follows.

  4. In the Execution section, select between Parallel, Sequential and Hybrid. If you select Sequential or Hybrid, you can specify the behavior in case of failure:

    1. Stop the Data Flow. If a Data Action fails, the remaining Actions are not executed. They are marked as Skipped in the Data Flow report.

    2. Continue with next Data Actions. If a Data Action fails, the remaining Data Actions are still executed.

  5. In the Data Actions section, define the actions the Data Flow should run:

    1. Select Add action to add one or more Data Actions. If you selected Hybrid, choose between Add sequential action and Add parallel action.

    2. Choose the Target Block and Configuration from the drop-down menus.

  6. (Optional) When the flow contains at least one CSV import, you can use the Instructions field to describe which file users should upload (for example, “Upload the Actuals CSV export from the shared Excel file.”). It is shown in the upload prompt when the flow is triggered, instead of a generic message.

  7. Select Save.

ℹ️ Note

If you are using Sequential or Hybrid Execution, you can reorder the steps of the Data Flow using drag and drop.

View or edit a Data Flow

All configurations are stored in the Data Flows overview, and each one is specific to an Application.

To view or edit a Data Flow:

  1. In your Application, open the three-dot menu and select Data Actions & Flows, or go to All Settings then Data Actions & Flows.

  2. From the list, you can:

    1. Select Show details to review a configuration.

    2. Select the Edit icon to make changes.

    3. Use the list item’s three-dot menu, then select Duplicate to create a copy of the configuration.

ℹ️ Note

Parallel, Sequential and Hybrid Data Flows are represented with different symbols.

Trigger a Data Flow from a Board

You can run a Data Flow from a Board using an Action button:

  1. Go to a Board.

  2. Select Edit, then + Add a widget.

  3. Choose Action.

  4. Select Trigger Data Flow, then select the configuration from the dropdown.

  5. Choose Card or Button for the widget format and add a Title, optional Description, or Button label.

  6. Select Add to board.

Once added, any Member with the required permissions can trigger the Data Flow directly from the Board.

If the Data Flow contains one or more CSV import actions, Pigment prompts Members selecting the button to upload files. If you wrote custom instructions (for more information, see Create a Data Flow), they appear in the CSV upload dialog.

Schedule a Data Flow

You can create and edit all scheduled Data Flows from the Scheduling page. For more information on how to create and manage a scheduled Data Flow, see Schedule imports in Pigment.

Data Flows containing a CSV import can’t be scheduled. If you subsequently add a CSV import to a scheduled Data Flow, the flow fails.

View Data Flow History

To view a history of executed Data Flows:

  1. Go to an Application.

  2. Select History from the sidebar.

  3. Under All Actions, select Data Flows from the dropdown to view execution history.