Optimizing Incremental Refreshes

Performing full refreshes of an extract may be time consuming, depending on the size of the extract. Alternatively, you can configure Tableau to incrementally refresh your extract by taking only rows that are new to the underlying data since the previous refresh was performed, and adding it to your extract. The way Tableau identifies new data depends on what you choose as an increment key. An increment key is a column in the underlying data source that is assigned non-decreasing values to each inserted row of data. To ensure that all new rows of data are captured in your extract, the increment key should be unique. Primary keys are unique and often an appropriate choice to use as an increment key. The following examples illustrate why it is important to use a unique key.

Note: Tableau recognizes integers (18 digits long), dates, and date/time as valid key options. To see the data type of a dimension that you are considering to use as the increment key, right click the dimension in the data window, and select Describe.

Suppose you have configured Tableau to incrementally refresh your extract by date column, everyday at 6 AM. You have the following new rows in the data source:

Because the extract refresh is configured to increment by date, the highlighted row is not included in the subsequent refresh because it was added to the underlying data on the same date the refresh occurred but after the incremental refresh was scheduled to perform. However, the last row is included in the subsequent incremental refresh because it was added to the underlying data source on the day but before the time of the subsequent incremental refresh. To avoid a scenario like this, ensure that your incremental refreshes are scheduled to occur after the last entry is written to the database for that day. Alternatively, use a unique key, such as"Transaction Number" in the example, on which to base your incremental refreshes.

Note: Enable scheduling on Tableau Server to create a refresh schedule for your extract. Refer to the"Scheduling" section in the Server Online Help and the"Publishing with Extracts" in the Tableau Server Administrator Guide for more information.

Suppose you have configured Tableau to incrementally refresh your extract by Customer ID column. You have the following new rows in the data source:

Because the extract refresh is configured to increment by Customer ID number, the highlighted row is not included in the subsequent refresh because the previous refresh already collected new information about Customer 38. However, the final row is included in the subsequent incremental refresh because Customer 40 is sequentially the next Customer ID after the last Customer ID in the previous refresh. To avoid this scenario, use a unique key, such as"Order Number", on which to base your incremental refresh.

Note: Updates to existing data and deletions are only included in full refreshes. To configure Tableau Desktop to perform incremental refreshes, refer to the "Refreshing Extracts" topic in the Online Help.