You would often need data to be exported from Clarisights for various reasons like top-management reporting, or build cost reporting for use by finance teams. We allow you to do just that through 2 functionalities on our platform: widget exports and data exports.

In data exports, all the data from a channel is exported in a partially de-normalized format and delivered to data warehouses maintained by customers. These exports are configured daily to the destination buckets specified by the customer. We recommend you to use data exports when the volume of data to be exported on your end is large, for other cases, we recommend using widget exports.

NOTE: Currently, we only offer Amazon S3 and Google Cloud Storage as destination data storage systems.

Granting Permissions

You will need to grant the required permissions to the Clarisights team for exporting data (the same set of permissions are required for our widget exporters as well).

For Google Cloud Storage:

Grant the following permissions to the service account "101712083634-compute@developer.gserviceaccount.com":

  • storage.objects.create

  • storage.objects.delete

  • storage.objects.get

  • storage.objects.list

For AWS S3:

Grant the following permissions to the role "arn:aws:iam::747179778889:role/ClarisightsExternalAccessRole":

  • GetObject

  • PutObject

  • PutObjectAcl

  • DeleteObject

  • ListBucket

Please share the bucket name and path with the Clarisights team after granting these permissions.

After completing the above steps, you can contact your Clarisights representative to set up these data exports for you.

The following part of the article explains the specifications of the data sent through these exports.

Channels

All native API-connection channels, POD channels, and Custom Advertising channels can be exported via this feature.

File Naming Structure

The file structure for the outgoing exports is as follows:

<company_name>/<data_channel>/<version>/core_<level>/<upload_date>_from_<start_date>_to_<end_date>.csv
<company_name>/<data_channel>/<version>/stats_<level>/<upload_date>_from_<start_date>_to_<end_date>.csv
<company_name>/<data_channel>/<version>/conversions_<level>/<upload_date>_from_<start_date>_to_<end_date>.csv
<company_name>/<data_channel>/<version>/breakdowns_<breakdown_type>/core_<level>/<upload_date>_from_<start_date>_to_<end_date>.csv
<company_name>/<data_channel>/<version>/breakdowns_<breakdown_type>/stats_<level>/<upload_date>_from_<start_date>_to_<end_date>.csv

These are the different types of files mentioned above:

  • core_: these files contain the dimensions data for the channel

  • stats_: these files contain the metrics data for the channel (except conversion metrics)

  • conversions_: these files contain the conversion metrics data for the channel (Optional)

  • breakdowns_: these files are present in case any breakdown dimensions are being exported for the channel (Optional)

Description of the filename structure:

  • company_name: the name of your company

  • data_channel: the name of the channel for the data (like Facebook, Google Ads, or the name of the POD or Custom Advertising channel)

  • version: the version of data export. There can be instances of changing the format of data exports which could have a potential impact on the data pipelines of your company, so we maintain versions on our end. Your Clarisights POC will inform you in advance in case of any breaking changes to our data exports along with deadline dates

  • level: the level of the data. All dimensions in Clarisights are stored at "levels" (Account, Campaign, Ad Group/Ad Set, Ad)

  • upload_date: the date of upload for the data

  • start_date: the start date of upload for the data

  • end_date: the date of upload for the data

  • breakdown_type: the specific breakdown dimension being exported (eg.: Age & Gender (Breakdown) dimension in Facebook Ads)

Upload Structure

Each CSV file (as detailed above) will be split into smaller chunks (each chunk contains a maximum of 100k rows) and each split will be compressed into gzip archives. So, a sample upload could look like this:

<company_name>/<adv_channel>/<version>/core_<level>/<upload_date>_from_<start_date>_to_<end_date>_1.gz
<company_name>/<adv_channel>/<version>/core_<level>/<upload_date>_from_<start_date>_to_<end_date>_2.gz
<company_name>/<adv_channel>/<version>/stats_<level>/<upload_date>_from_<start_date>_to_<end_date>_1.gz
<company_name>/<adv_channel>/<version>/stats_<level>/<upload_date>_from_<start_date>_to_<end_date>_2.gz

File Format

Below are the specifications of our file format:

  • Each CSV file is RFC 4180 compatible format

  • Each CSV file will have an identifying header that identifies the CSV column order

  • There is a special field created_at that identifies the date of CSV export

  • Every line after the header will contain the relevant data

  • Each CSV file is comma-separated

  • Every field is double-quoted and the presence of a quote in a field will be handled using another quote (eg.: the value Campaign "A" | City 1 will be processed and sent as "Campaign ""A"" | City 1")

Examples

Below are some example export filenames for some of our channels.

Snapchat

Core account CSV:

account_object_id, account_id, created_at
“1456”,  “1122334455”, “2018-01-01”

Stats account CSV:

date, account_object_id, spend, impressions, swipes ….. created_at
“2017-01-01”, “1456”, “200.32”, “10000”, “100”,... “2018-01-01”

Core campaign CSV:

campagin_object_id, account_object_id, campaign_id, campaign_name, city, created_at
“123456”, “1456”, “11223376563”, “Test Campaign”, “London”, “2018-01-01”

Stats campaign CSV:

date, campaign_object_id, spend, impressions, swipes ….. created_at
“2017-01-01”, “123456”, “200.32”, “10000”, “100”,... “2018-01-01”

Conversions campaign Stats CSV:

Date, campaign_object_id, conversion_name, view_28d_swipe_28d,… created_at
“2017-01-01”,”123456”,”app open”, “20”.... “2018-01-01”

Facebook

Core account CSV:

account_object_id, account_id, created_at
“1456”,  “act_1122334455”, “2018-01-01”

Stats account CSV:

date, account_object_id, spend, impressions, clicks ….. created_at
“2017-01-01”, “1456”, “200.32”, “10000”, “100”,... “2018-01-01”

Core campaign CSV:

campagin_object_id, account_object_id, campaign_id, campaign_name, city, created_at
“123456”, “1456”, “11223376563”, “Test Campaign”, “London”, “2018-01-01”

Stats campaign CSV:

date, campaign_object_id, spend, impressions, clicks ….. created_at
“2017-01-01”, “123456”, “200.32”, “10000”, “100”,... “2018-01-01”

City Breakdown Core campaign CSV:

breakdown_campaign_object_id, campaign_object_id, account_object_id, campaign id, country, created_at
“1248”, “123456”, ”1456”, “11223376563”, “UK”, “2018-01-01”

City Breakdown Stats campaign CSV:

date, breakdown_campaign_object_id, spend, impressions, clicks ….. created_at
“2017-01-01”, “1248”, “100.32”, “5000”, “50”,... “2018-01-01”

Twitter

Core account CSV:

account_object_id, account_id, created_at
“1456”,  “1122334455”, “2018-01-01”

Stats account CSV:

date, account_object_id, billed_charge, impressions, clicks ….. created_at
“2017-01-01”, “1456”, “200.32”, “10000”, “100”,... “2018-01-01”

Core campaign CSV:

campagin_object_id, account_object_id, campaign_id, campaign_name, city, created_at
“123456”, “1456”, “11223376563”, “Test Campaign”, “London”, “2018-01-01”

Stats campaign CSV:

date, campaign_object_id, billed_charge, impressions, clicks ….. created_at
“2017-01-01”,“123456”, “200.32”, “10000”, “100”,... “2018-01-01”

And that's it! We hope this information gives you an overview of how our data exports work. Please reach out to your Clarisights POC in case of any issues or questions. :)

Did this answer your question?