# Storing data between runs

> How APIANT automations keep data between runs with Lookup Tables (scopes, expiry, encryption) and Collector buckets, and when to use Variables instead.

Each run starts with no memory of earlier runs. To carry data from one run to the next, such as which records were already synced or the ID a record has in another app, an automation writes it to a **Lookup Table** or a **Collector bucket**. Both are built-in apps and need no connection.

## Choosing where to keep data

| Need | Use |
|---|---|
| A value only for the rest of this run, such as a running total inside a loop | The Variables app. Values last for one run. |
| A value later runs look up by key, such as a mapping from a CRM ID to a billing ID | Lookup Tables |
| A list of items that builds up over many runs and is processed together later, such as a daily digest | Collector |
| Tracking which records a polling trigger has seen | Nothing. The trigger already does this. |
| Storing OAuth tokens | Nothing. Connections store and refresh them. |

## Lookup Tables

A lookup table row is addressed by three names and holds a value:

| Part | Example |
|---|---|
| Key group | `crm_to_billing` |
| Key | `{{ trigger.body.contact_id }}` |
| Value name | `billing_id` |
| Value | `{{ steps.create_customer.output.id }}` |

A key group is only a name. Use one per purpose so rows do not collide.

### Actions

| Action | Does | Returns in `result` |
|---|---|---|
| Save value | Writes one value | Empty text. A failed save fails the step. |
| Save multiple values, Save multiple mapped values | Writes several values in one step (up to 20 pairs for mapped values) | Empty text |
| Get value | Reads the values under a key group, key and value name | A list of strings. An empty list means not found. |
| Get multiple values | Reads several value names at once | An object of value name to list of values |
| Get key | Finds the key that holds a value | A list of keys |
| Find records | Searches by any mix of key group, key, value name, exact value, or numeric range | A list of `{keygroup, key, value_name, value}` objects, one per stored value |
| Find value names | Lists the value names under a key | A list of strings |
| Delete single value, Delete value by value name, Delete values by key, Delete values by key group | Removes rows | Empty text. Deleting nothing still succeeds. |
| Step from starting number (account scope or automation scope) | Keeps a counter: the first run returns the starting number, each later run adds the step value | The new counter value |

Get value always returns a list, because one value name can hold several values. Read the first value with `{{ steps.get_billing_id.output.result[0] }}`, and test for "not found" by checking that the list is empty.

Save value's `data_replacement` input decides what an existing row is replaced by: the matching value name (default), the whole key, the whole key group, or nothing, which appends another value.

### Example: map IDs between two apps

Ask: "when a contact is created in the CRM, create the customer in billing unless we already did, and remember the billing ID".

1. **Get value** with key group `crm_to_billing`, key = the CRM contact ID, value name `billing_id`.
2. **Condition**: `len(steps.get_billing_id.output.result) == 0`.
3. **Then**: create the customer in billing, then **Save value** with the new billing ID and expiry `180 days`.
4. **Else**: use `{{ steps.get_billing_id.output.result[0] }}` as the existing billing ID.

### Scope

Lookup Tables actions take a `scope` input, except **Step from starting number**, whose scope is in its name. A read only sees rows written with the same scope.

| Scope | Rows are shared by |
|---|---|
| This automation (default) | Runs of this automation in your account |
| My account | Every automation in your account |
| Linked accounts | Your account's linked parent and its linked children. With no linked parent, the same as My account. |

Linked accounts are an Enterprise feature; see [Linked and associated accounts](/docs/account/linked-accounts).

### Expiry

Each save sets when the row is deleted, using the `expiry` input:

| Expiry | Use |
|---|---|
| 90 days (default) | Caches, duplicate markers, short-lived state |
| 180 days | ID mappings between apps. If a mapping expires, the next run looks the record up again and saves a new one. |
| 365 days | Yearly state |
| Never | Configuration that must persist indefinitely |

Saving the same row again restarts its expiry. Any other expiry text fails the step instead of falling back to a default. Expired rows are no longer returned and are removed by a regular cleanup task.

### Encryption

Saved values are encrypted at rest by default. Set `encrypted` to false on a save only when you need **Find records** to search that value by numeric range, which works on unencrypted number values only. Keys, key groups and value names are always stored unencrypted, so do not put secrets in them.

### Seeing what an automation stored

In the dashboard gear menu, **Processing > Inspect stored data** lists the rows the automation saved with **This automation** scope, with search on each column. **Processing > Dump stored data** downloads them as a CSV file.

## Collector buckets

A Collector bucket holds a list of items that grows across runs until something empties it. One automation collects, and another, usually on a schedule, dumps the bucket and processes everything at once.

| Operation | Type | Does |
|---|---|---|
| Collect items into a bucket | Action | Adds an item. A list adds each element as its own item. |
| Dump items from bucket | Action | Returns every item as a list, oldest first, and empties the bucket |
| Fetch all items from bucket in order added | Action | Returns every item without emptying the bucket |
| All new items in bucket | Trigger | Each time it polls and the bucket has items, runs the steps once with all items joined by an optional separator, and empties the bucket |
| Each new item in bucket | Trigger | Each time it polls and the bucket has items, runs the steps once per item, and empties the bucket |

Both triggers have **Emit just the item count?**, which passes only the number of items instead of the items. The bucket is still emptied.

### Example: a daily digest

Ask: "collect every new feed article during the day and email me a digest at 22:00".

1. **Collector automation**: a Feed trigger, then **Collect items into a bucket** with bucket `newsfeed_daily` and item `{{ trigger.titlePlaintext }}: {{ trigger.link }}`.
2. **Digest automation**: scheduled `0 22 * * *`, then **Dump items from bucket** for `newsfeed_daily`, then **Array to single value** (Transform Data) to join the list, then the email step.
3. Turn both automations on. A build leaves each one off.

Dump returns a list, and most email body fields expect text, so join the list before sending it.

### Bucket rules

- A bucket belongs to your account, not to one automation. Any automation in the account that uses the same bucket name reads and writes the same items.
- Blank bucket names become `default`. Prefix names by purpose, such as `errors_24h`, when several automations collect.
- Items are deleted 90 days after they were added, whether or not anything dumped them.
- Use one way of emptying a bucket. A scheduled dump and a Collector trigger on the same bucket compete and split the items between them.
- A dump removes the items as it reads them. If a later step in that run fails, those items are not put back.
- Buckets cannot be shared across accounts.

## Troubleshooting

| Symptom | Likely cause | What to do |
|---|---|---|
| Get value never finds a row that was saved | The save and the read use different scopes, key groups or value names | Ask Claude to compare the Save and Get settings |
| A mapping disappeared after a few months | The row expired | Save mappings with a longer expiry, or re-create them on a miss |
| The digest email is empty | The collecting automation is off, the bucket names differ, or a second automation emptied the bucket first | Check both automations are on and use exactly the same bucket name |
| Find records by number range returns nothing | The values were saved encrypted | Save those values with Encrypted turned off |

## Next steps

- [Conditions, loops and parallel branches](/docs/automations/control-flow)
- [Scheduled triggers](/docs/automations/triggers/schedules)
- [Two-way sync](/docs/automations/two-way-sync)
- [Actions and built-in apps](/docs/automations/actions)