Getting Started with Export Pipelines

Widget Exports are deprecated. Existing widget exports keep running for now, but don't set up new ones: use Export Pipelines instead. To move an existing widget export, see Migrating from Legacy Exports to Export Pipelines.

Export Pipelines deliver automated, recurring data exports from Clarisights to your own cloud storage (Amazon S3 or Google Cloud Storage). You decide what data to export, where to send it, and when, and Clarisights handles the rest.

Key Concepts

An export pipeline is composed of four building blocks:

Concept	What it does
Pipeline	The top-level container. Defines a name and which data sources (channels) to include.
Manifest	Defines what data to export: one or more extracts, each with its own metrics, dimensions, and filters. Manifests are versioned, so you can change the shape of your data without breaking existing consumers.
Destination	Defines where to send the data: an S3 bucket or GCS bucket, with the access settings Clarisights needs to write to it.
Schedule	Defines when to export: a cron expression, timezone, date range, file name template, and compression. Links the pipeline to a destination.

Each time a schedule fires, it creates an Export Run: a record of the execution with its status, file links, and file sizes.

What's Supported

Destinations: Amazon S3, Google Cloud Storage
Format: CSV
Compression: gzip (default) or none
Scheduling: Any cron expression with a minimum interval of 1 hour, in the timezone of your choice
Date ranges: Presets like today, yesterday, last_7d, last_30d, moving ranges, or explicit start and end dates
Row limit: None. Each extract is delivered as a single file, however many rows it has.

Quick Walkthrough

Your Clarisights team sets up export pipelines with you. Here's the typical flow for your first pipeline:

Prepare a Destination: Create (or pick) an S3 or GCS bucket and grant Clarisights permission to write to it. See Configuring Export Pipeline Destinations & Permissions.
Choose your data sources: Decide which connected channels the pipeline should include (for example Google Ads, Facebook, TikTok).
Define your extracts: For each file you want to receive, list the metrics, dimensions, and any filters. The metrics and dimensions available depend on the data sources you selected.
Choose a schedule: Pick when the export should run (for example 0 12 * * * for daily at noon), the timezone, and the date range each run should cover.
Send the details to your Clarisights team through the messenger support on the platform. We create the pipeline, manifest, destination, and schedule, and activate it.
Check your bucket: After the first scheduled time, your files appear in the bucket. Each data file comes with a .meta.json file that describes its columns and date range.

Pipeline Lifecycle

Active: Schedules fire normally and create export runs.
Paused: No exports are triggered. Resume at any time by setting status back to active.
Deleted: Soft-deleted. The pipeline and its data are retained for 30 days before permanent removal.

You can pause and resume both individual schedules and entire pipelines independently.

Manifest Versioning

When a new manifest is created for a pipeline, the previous manifest enters a 30-day grace period. During this time, every scheduled run exports both the old and the new manifest, each into its own version folder (v1, v2, and so on). After 30 days, the old manifest expires and is no longer exported.

This means you can update your export shape (add or remove metrics, change filters) without breaking downstream consumers that still expect the old format.

What Gets Exported

Each export run produces, for every extract:

A CSV data file (gzip-compressed unless you chose no compression). The file has a header row and an extra _extracted_at column with the time of the export.
A metadata file (.meta.json) alongside the data file, containing the column mapping, date range, and compression info.

By default, files are organized in your bucket as:


{path_prefix}/{pipeline_name}/v{manifest_version}/{extract_id}/{epoch_time}_{export_start_date}_{export_end_date}.csv.gz

You can change this layout with the schedule's file name template. See Managing Export Schedules & Manifests.

Feature	Export Pipelines	Data Exporters (Legacy)	Widget Exports (Legacy)
Choose metrics & dimensions	Yes	Fixed per channel	Linked to widget config
Several channels in one file	Yes	No (one set of files per channel)	Linked to widget config
Filters	Yes (flexible filter syntax)	No	Linked to widget config
Custom schedule	Any cron (min 1hr)	Daily, plus an optional run for the last 3 days	A chosen hour, every 1 to 24 hours
Row limit	No limit (one file per extract)	No limit (may be split into parts)	100,000 rows
Manifest versioning	Yes (30-day grace)	No	No

Note: Data Exporters and Widget Exports are being retired, and new Data Exporters can no longer be set up. Existing legacy exports keep running for now, but we recommend moving them to Export Pipelines. See Migrating from Legacy Exports to Export Pipelines.

For detailed setup instructions, see: