> ## Documentation Index
> Fetch the complete documentation index at: https://getconvoy.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Retention Policy

Convoy can drop old Postgres partitions and export webhook data to cold storage.

> **Note:**
>
> * Retention policies are available only on a paid license.
> * Partition conversion is part of that license. There is no separate feature flag to turn partitioning on.
> * `CONVOY_RETENTION_PERIOD` sets the keep window for the 01:00 UTC partition drop. `CONVOY_RETENTION_ENABLED` controls whether that drop runs and defaults to `true`.
> * `CONVOY_WEBHOOK_ARCHIVING_ENABLED` controls cold-storage export only and defaults to `false`. It does not control partition drop.
> * Archiving needs a usable storage policy. Convoy skips export when storage is missing or invalid.

## Configuring Retention

### Archive vs partition drop

| Setting                                                          | What it controls                                               |
| ---------------------------------------------------------------- | -------------------------------------------------------------- |
| `CONVOY_RETENTION_PERIOD` / `retention.period`                   | Keep window for the 01:00 partition drop on licensed instances |
| `CONVOY_RETENTION_ENABLED` / `retention.enabled`                 | Whether the 01:00 partition drop runs (default `true`)         |
| `CONVOY_WEBHOOK_ARCHIVING_ENABLED` / `webhook_archiving.enabled` | Whether archive export runs (default `false`)                  |
| `CONVOY_BACKUP_INTERVAL` / `webhook_archiving.interval`          | How often scheduled archive export runs (default `1h`)         |
| `CONVOY_CDC_BACKUP_ENABLED` / `webhook_archiving.cdc_enabled`    | Stream changes to storage instead of using scheduled export    |
| `CONVOY_REPLICATION_DSN` / `webhook_archiving.replication_dsn`   | Postgres connection used for change data capture               |
| `storage_policy`                                                 | Where archive uploads go (required when export is enabled)     |

Legacy keys still load when the new keys are unset. `CONVOY_RETENTION_POLICY` and `retention_policy.policy` map to the new retention period. `CONVOY_RETENTION_POLICY_ENABLED` and `retention_policy.enabled` map to webhook archiving. Legacy backup interval and change data capture fields also move under `webhook_archiving`.

### JSON Configuration

You can configure cold storage by adding the `storage_policy` object to your configuration JSON.

| Key                | Description                                               |
| ------------------ | --------------------------------------------------------- |
| `type`             | Storage type, either "s3" or "on\_prem"                   |
| `s3.prefix`        | Prefix for S3 storage path                                |
| `s3.bucket`        | Name of the S3 bucket                                     |
| `s3.access_key`    | AWS access key for S3 authentication                      |
| `s3.secret_key`    | AWS secret key for S3 authentication                      |
| `s3.region`        | AWS region where the S3 bucket is located                 |
| `s3.session_token` | Temporary session token for AWS authentication (optional) |
| `s3.endpoint`      | Custom endpoint URL for S3-compatible storage             |
| `on_prem.path`     | File system path for on-premises storage location         |

```json using storage policy via JSON configuration theme={null}
{
	"storage_policy": {
		"type": "s3 | on_prem",
		"s3": {
			"prefix": "<insert-s3-prefix>",
			"bucket": "<insert-s3-bucket>",
			"access_key": "<insert-s3-access-key>",
			"secret_key": "<insert-s3-secret-key>",
			"region": "<insert-s3-region>",
			"session_token": "<insert-s3-session-token>",
			"endpoint": "<insert-s3-endpoint>"
		},
		"on_prem": {
			"path": "<insert-on-prem-path>"
		}
	}
}
```

### Environment Variables Configuration

Alternatively, you can supply environment variables to configure the storage policy.

| Environment Variable               | Description                                               |
| ---------------------------------- | --------------------------------------------------------- |
| `CONVOY_STORAGE_POLICY_TYPE`       | Storage type, either "s3" or "on\_prem"                   |
| `CONVOY_STORAGE_AWS_PREFIX`        | Prefix for S3 storage path                                |
| `CONVOY_STORAGE_AWS_BUCKET`        | Name of the S3 bucket                                     |
| `CONVOY_STORAGE_AWS_ACCESS_KEY`    | AWS access key for S3 authentication                      |
| `CONVOY_STORAGE_AWS_SECRET_KEY`    | AWS secret key for S3 authentication                      |
| `CONVOY_STORAGE_AWS_REGION`        | AWS region where the S3 bucket is located                 |
| `CONVOY_STORAGE_AWS_SESSION_TOKEN` | Temporary session token for AWS authentication (optional) |
| `CONVOY_STORAGE_AWS_ENDPOINT`      | Custom endpoint URL for S3-compatible storage             |
| `CONVOY_STORAGE_PREM_PATH`         | File system path for on-premises storage location         |

```shell using storage policy via environment variable configuration theme={null}
CONVOY_STORAGE_POLICY_TYPE=''
CONVOY_STORAGE_AWS_PREFIX=''
CONVOY_STORAGE_AWS_BUCKET=''
CONVOY_STORAGE_AWS_ACCESS_KEY=''
CONVOY_STORAGE_AWS_SECRET_KEY=''
CONVOY_STORAGE_AWS_REGION=''
CONVOY_STORAGE_AWS_SESSION_TOKEN=''
CONVOY_STORAGE_AWS_ENDPOINT=''
CONVOY_STORAGE_PREM_PATH=''
```

## Retention Using Postgres Partitions

Convoy retains webhook history with Postgres-native daily partitions
([go\_partman](https://github.com/jirevwe/go_partman)). Licensed instances can convert tables
from the dashboard or the CLI. Partitioning is a paid feature; it is not a feature flag.

Before `Convoy v25.1.1` the retention job both uploaded to S3 and ran `DELETE` on the same
tables. Timeouts and network errors left duplicates or skipped deletes, and large deletes
triggered autovacuum pressure. Partitions drop a day at a time instead.

### Tables

These tables can be partitioned:

* `delivery_attempts`
* `event_deliveries`
* `events`
* `events_search`

Retention only drops data after all four tables are partitioned. Convoy skips the drop when any of them is still unpartitioned.

Back up the database, or try the conversion on a copy, before you convert a large production table.

### Convert from the admin dashboard

On a licensed instance, open **Admin → Table partitions**. Pick a table and an operation. Convert one table at a time.

**Attach** (the usual conversion) keeps ingestion running. The existing table becomes one history partition that holds everything written up to the cutoff; new rows go into daily partitions after that. Retention expires that history partition in full once every row in it is older than your window, then expires later days one partition at a time.

**Detach** / **copy unpartition** convert back to a plain table. Copy unpartition should run in a maintenance window: rows written during the copy are not carried over, so pause ingestion first.

Progress for the in-flight run is shown on the same page.

### Convert from the CLI

```shell partition all tables theme={null}
convoy utils partition
```

Or one table at a time:

```shell partition tables one after the other theme={null}
convoy utils partition delivery_attempts
convoy utils partition event_deliveries
convoy utils partition events
convoy utils partition events_search
```

> If you are running Convoy in Docker:
>
> ```shell partitioning using docker exec theme={null}
> docker exec -it convoy-staging /cmd utils partition
> ```

Un-partition:

```shell unpartition theme={null}
convoy utils unpartition
```

```shell unpartition one table theme={null}
convoy utils unpartition delivery_attempts
convoy utils unpartition event_deliveries
convoy utils unpartition events
convoy utils unpartition events_search
```

### Breaking change in v26.7.2

Partitions created by `convoy utils partition` used to be ignored by retention because their names were folded to lower case. They are managed now. History that survived only because those partitions were skipped is dropped at `CONVOY_RETENTION_POLICY` on the next maintenance run. Confirm that duration before you upgrade.

### Indexes after conversion

An interrupted index build, or an upgrade that drops a large index so migrate can finish, leaves work on **Admin → Table indexes**. `convoy utils indexes` reports the same list.

From `v26.7.3` server and agent work through that list themselves, in the background after boot, one index at a time and unique indexes first. No rebuild holds a lock against traffic, but a large index takes hours. Use **Rebuild** on that page to retry one that failed.

### Breaking change in v26.7.3

Boot also drops any index Postgres left `INVALID` (a killed `CREATE INDEX CONCURRENTLY`) and queues it the same way, instead of leaving it until the next `migrate` with the planner ignoring it silently.

An invalid unique index still enforces its key. Dropping one gives that up until the rebuild finishes, so upgrade in a window where losing the constraint for a few hours is acceptable, and watch the page until every owed index is valid again.

A rebuild that meets duplicate rows is recorded as **blocked**, with the offending key, and skipped from then on. Resolve those duplicates by hand before you retry it.