Leverage sparsity for optimal performance

Prev Next

​​​​​​​Metrics perform better in Pigment on sparse versus dense data. This page explains sparsity and how to optimize performance.

What is sparsity and why is Pigment’s engine sparse?

A sparse engine means that empty values are not filled by default with a 0 or a FALSE: your datasets can include blanks. This allows for much faster computation than in a data-dense engine. In Pigment, calculations run on as little data as possible.

While there are instances where you need to have values in every cell, there are other instances where you can use BLANK instead of zeros or FALSE. Therefore, the goal is to ensure your Blocks are as sparse as possible.

First, identify the most dense Metrics:

  1. Open the Blocks dropdown in the sidebar and select All Blocks.

  2. In the main pane, select Display top right.

  3. Toggle Maximum Size to On.

  4. Maximum Size is added as a column header. Select it to sort your Metrics by Maximum Size.
    This gives you an indication of the densest Metrics in your Application.

  5. Select the Performance Insights button top right. For more information, see Use Performance Insights to Optimize your Application.

  6. Toggle Density to On.
    This shows you whether these Metrics are at risk of becoming your densest Metrics.

  7. Select the Dimensions tab in the main pane and select Display top right.

  8. Toggle Number of items to On.
    Assess whether you can reduce the number of Dimensions in your Metrics or reduce their size.

Then make dense Metrics more sparse:

  • Review functions like IFBLANK,ISBLANK, ISNOTBLANK and see if they are needed, or if the output of the formula is being populated with values that can be replaced with BLANK instead.

  • Review any usage of zeros instead of BLANK and see if it is needed.

  • Review any usage of FALSE instead of BLANK and see if it is needed. In particular for Access Rights, if you are on the current version of Access Rights, use BLANK instead of FALSE. This results in the same behavior and produces a sparser Metric.