How do Clarisights Data Exporters work?

Widget Exports are deprecated. Existing widget exports keep running for now, but don't set up new ones: use Export Pipelines instead. To move an existing widget export, see Migrating from Legacy Exports to Export Pipelines.

You may need to export data from Clarisights for reasons like top-management reporting, or cost reporting for finance teams. Clarisights has offered this through 2 legacy features: widget exports and data exports. This article describes data exports (Data Exporters) for customers who still receive them.

Data exports deliver all the data from a channel, in a partially de-normalized format, to data warehouses maintained by customers. They run once a day to the destination buckets specified by the customer, and can include a second daily run that re-exports only the last 3 days. For any new export, we recommend Export Pipelines.

Note: Amazon S3 and Google Cloud Storage are currently the only supported destinations.

Granting Permissions

Data exports need the following permissions on your bucket (widget exports need the same permissions). If you move an existing data export to a new bucket, grant these on the new bucket.

Google Cloud Storage

Grant the following permissions to the service account "101712083634-compute@developer.gserviceaccount.com":

• storage.objects.create

• storage.objects.delete

• storage.objects.get

• storage.objects.list

AWS S3

Grant the following permissions to the role "arn:aws:iam::747179778889:role/ClarisightsExternalAccessRole":

• GetObject

• PutObject

• PutObjectAcl

• DeleteObject

• ListBucket

Important: after granting these permissions, share the bucket name, the bucket region and the path with the Clarisights team.

The rest of this article describes the data sent through these exports.

Channels

Data exports are available for Facebook, Google Ads, Bing (Microsoft Advertising), Snapchat, TikTok, Twitter (X), Awin, POD channels, and Custom Advertising channels. Export Pipelines cover every connected data source.

File Naming Structure

The file structure for the outgoing exports is as follows (<path_prefix> is the path you shared with us, if any):

<path_prefix>/from_clarisights/<company_name>/<data_channel>/v<version>/core_<level>/<upload_date>_from_<start_date>_to_<end_date>.gz.<part>
<path_prefix>/from_clarisights/<company_name>/<data_channel>/v<version>/stats_<level>/<upload_date>_from_<start_date>_to_<end_date>.gz.<part>
<path_prefix>/from_clarisights/<company_name>/<data_channel>/v<version>/conversions_<level>/<upload_date>_from_<start_date>_to_<end_date>.gz.<part>
<path_prefix>/from_clarisights/<company_name>/<data_channel>/v<version>/breakdowns_<breakdown_type>/core_<level>/<upload_date>_from_<start_date>_to_<end_date>.gz.<part>
<path_prefix>/from_clarisights/<company_name>/<data_channel>/v<version>/breakdowns_<breakdown_type>/stats_<level>/<upload_date>_from_<start_date>_to_<end_date>.gz.<part>

File types:

• core_: these files contain the dimensions data for the channel

• stats_: these files contain the metrics data for the channel (except conversion metrics)

• conversions_: these files contain the conversion metrics data for the channel (optional, only when conversion metrics are part of your export)

• breakdowns_: these files are present in case any breakdown dimensions are being exported for the channel (optional)

Parts of the file name:

• company_name: the name of your company

• data_channel: the name of the channel for the data (like Facebook, Google Ads, or the name of the POD or Custom Advertising channel)

• version: the version of the data export (see below)

• level: the level of the data. All dimensions in Clarisights are stored at "levels" (Account, Campaign, Ad Group/Ad Set, Ad)

• upload_date: the date of upload for the data

• start_date: the start date of upload for the data

• end_date: the end date of upload for the data

• breakdown_type: the specific breakdown dimension being exported

• part: the part number of the file, starting at 1

Export Versions

The version number goes up whenever the set of exported columns changes (for example when a dimension, metric, level, or conversion is added or removed). Changing the date range or the destination does not change the version.

When a new version starts, the previous version keeps being exported alongside it for 30 days, so you have time to update your downstream processing. After that, only the new version is exported. If you match file paths in your pipeline, make sure it picks up the new v<version> folder.

Upload Structure

Each file is compressed with gzip, and its name ends with a part number (.gz.1). A large export can be split into several parts of up to 100k rows each (.gz.1, .gz.2, and so on). No data is dropped when a file is split, so always read every part of a file.

File Format

• Each CSV file is in RFC 4180 compatible format

• Each CSV file has an identifying header

• There is a special field created_at that identifies the date of CSV export

• Each CSV file is comma-separated

If you have any issues or questions, reach out to your Clarisights POC.