Vaulthalla Logo

Sync

Configure and troubleshoot local, S3, and R2 vault sync behavior, remote indexes, dry-runs, inventory imports, event ingestion, and reconcile.

Sync

Vault sync keeps Vaulthalla's local state, metadata, and remote object storage aligned according to the vault type and sync policy. S3/R2 sync adds remote-index management, request budgeting, and price-budget checks.

Manual Sync

Trigger a sync:

1vh vault sync <vault>

Inspect policy and remote-index state:

1vh vault sync info <vault>

Change common policy fields:

1vh vault sync set <vault> --interval 15m2vh vault sync set <vault> --on-sync-conflict ask

S3/R2 policy updates:

1vh vault sync set archive --sync-strategy cache --on-sync-conflict keep_local2vh vault sync set archive --max-remote-index-age 24h

Strategies For S3/R2

StrategyUse whenImportant behavior
cacheYou want a local working cache over a bucketRemote objects can be indexed without downloading every body. Bodies are fetched when accessed or planned.
syncLocal and remote should both accept changesChanges on either side can be propagated according to conflict policy.
mirrorThe local vault should publish to the bucketRemote changes are not treated as local source changes.

Choose the strategy before connecting Vaulthalla to a bucket with existing data. Use dry-run and request budgets to understand planned work.

Conflict Policies

Local vault policies:

  • overwrite
  • keep_both
  • ask

S3/R2 vault policies:

  • keep_local
  • keep_remote
  • ask

Use ask when automatic conflict resolution would be risky and an operator should review conflicts.

Remote Index

S3/R2 vaults maintain a remote object index. The index lets Vaulthalla plan sync work without repeatedly scanning a large bucket.

The remote-index flow can use:

  • A Vaulthalla manifest under .vaulthalla/.
  • Local database index rows.
  • S3 Inventory imports.
  • S3 Event Notification ingestion.
  • Explicit reconcile scans.

vh vault sync info <vault> reports remote-index source, indexed time, manifest ETag, generated time, object count, and fresh or stale status.

If the local index is older than max_remote_index_age and the manifest cannot be refreshed, sync stalls instead of silently trusting stale data.

Dry-Run

Run dry-run before raising budgets or syncing a large bucket:

1vh vault sync dry-run <vault>

Default dry-run is local-index-only. It reads local state and the local remote index, builds a plan, and prints estimated request and traffic pressure. It does not invent a plan when no usable remote index exists.

Refresh the remote index only when you intentionally allow S3 work:

1vh vault sync dry-run <vault> --refresh-index2vh vault sync dry-run <vault> --refresh-remote-index

Pricing controls:

1vh vault sync dry-run <vault> --refresh-pricing2vh vault sync dry-run <vault> --no-pricing

Dry-run output is the safest way to compare request budgets, price budgets, and planned body downloads before real sync work starts.

Import S3 Inventory

For large existing buckets, import S3 Inventory instead of starting with a full bucket scan:

1vh vault sync inventory <vault> --file inventory.csv

For CSVs without headers, provide a schema:

1vh vault sync inventory <vault> \2  --file inventory.csv \3  --schema bucket,key,size,last_modified_date,etag,storage_class

Inventory import indexes object metadata and publishes the Vaulthalla manifest without downloading object bodies.

Ingest S3 Events

Use event ingestion to keep the index current after initial import:

1vh vault sync events <vault> --file s3-events.json

ObjectCreated events upsert remote-index rows. ObjectRemoved events delete rows. Sequencers prevent older events from overwriting newer index state when the provider supplies them. Manifest objects under .vaulthalla/ are ignored.

Reconcile

Reconcile performs an explicit ListObjectsV2 pass:

1vh vault sync reconcile <vault> --allow-list-scan

Without --allow-list-scan, reconcile requires a configured LIST request budget on the sync policy. This prevents accidental unbounded scans on large buckets.

When a prior index exists, reconcile prints a rough estimate of one LIST request per 1,000 indexed objects. Use that estimate to set an appropriate --s3-budget-list value.

Archive-Tier Objects

Archive-tier objects can be indexed without body download. If the provider requires restore before download, Vaulthalla skips planned body downloads and reports the skip in sync or dry-run output. Restore the object with the provider first, then run sync again.

Budget Stalls

When a request or downloaded-byte budget is exceeded, the sync event is marked stalled with a budget-related reason. This is expected protective behavior, not necessarily a service failure.

Start with:

1vh vault sync info <vault>2vh vault sync dry-run <vault>

Then raise the narrow exhausted budget field rather than switching to unlimited.