Web Video SDK Discrepancy Management
As part of the video integration certification, IAS requires producing and sending your daily aggregated impression counts to an IAS SFTP folder. Then, IAS calculates the daily discrepancy rate by comparing your count with IAS's count and puts the report in an outbound folder.
Overview
When using the web video SDK, some amount of discrepancy is expected, since messages from one system to another can be lost or dropped for any number of reasons (for example, temporary network or server outages). As part of the video integration certification, IAS requires producing and sending your daily aggregated impression counts to an IAS SFTP folder. Then, IAS calculates the daily discrepancy rate by comparing your count with IAS's count and puts the report in an outbound folder.
The IAS SFTP has 2 folders:
Folder | Description |
inbound | contains your impression counts file under the video_integration subfolder. You may need to create the video_integration folder. Additionally, this folder may include mapping.txt which defines select fields to include in the inbound files. |
outbound | contains the IAS discrepancy management report under the video_integration subfolder. |
Note: contact your IAS Technical Solutions representative for more information about the SFTP server.
Inbound files
IAS processes inbound files and creates a discrepancy management report in the outbound/video_integration folder. The inbound files include your impression counts stored in a comma separated variable (CSV) formatted file and an optional corresponding .md5 file.
On a given day, IAS processes an inbound file with the previous day's date in the filename. Note, IAS won't re-process a file, so you should contact IAS Technical Solutions representative to initiate re-processing of files if required.
The inbound file contains your data in a .csv file which must include a header, as defined in the section 'Feed CSV Fields'. The .csv file must be RFC 4180 compliant (see 'Common Format and MIME Type for Comma-Separated Values (CSV) Files' for more information) and can optionally be compressed into a .csv.gz file using the gzip command.
You must name the inbound file using the previous day's date in the format YYYYMMDD.csv or YYYYMMDD.csv.gz, and upload the file by 4 PM Eastern Standard Time/Eastern Daylight Time. If IAS fails when processing the file (for example, the file is corrupt) then IAS names the file with a ".FAILED" suffix, for example 20180411.csv.gz.2018_04_12_1000_05_EST.FAILED.
Note: All dates are in Eastern Standard Time/Eastern Daylight Time (EST/EDT).
For every compressed .csv.gz file uploaded to the SFTP, you can optionally create an .md5 file using the md5sum tool.
Finally, you can set a cutoff limit on partner_impression_count to exclude low traffic campaigns/placements, thereby limiting the input file size. Contact your IAS Technical Solutions representative for more information about setting a cutoff limit.
In summary, to verify the integration is within acceptable impression discrepancy levels, IAS requires you to create the subfolder video_integration in the inbound folder
Upload your impression counts to the inbound/video_integration folder as a .csv formatted file that may also optionally be compressed to a .csv.gz file, and optionally accompanied with a corresponding .md5 file
Feed CSV Fields
IAS allows you to use the following fields in your discrepancy report, using either the standard or select fields.
If you use the standard feed csv fields then all fields must be placed in the following order, with a header, with the following fields. All feed CSV fields must be included, otherwise the report is not processed. Leave fields empty if you do not require its data; do not input "NULL". Also, do not include commas in integers.
If you use select fields, you must create the file mapping.txt in the inbound folder to define the fields for your inbound CSV. See the section 'Select fields' for more information.
Note: IAS supports up to 10,000 rows per report
Standard fields
Here is the list of fields and the required order when using the standard feed CSV fields. These fields must be in this order if you do not include mapping.txt to select fields for the inbound file:
Order | Field | Type | Description | Open Web Example | Campaign Monitoring Example |
1 | impression_date | string | YYYY-mm-dd format date, EST/EDT time zone. For example, "2017-06-10". | 2017-06-01 | 2017-06-01 |
2 | ias_integration_package | string | Whether this is for Campaign Monitoring (CM) or Open Web. This field can be populated with exactly one of the following values: FW or CM | FW | CM |
3 | ias_publisher_type | string | Character to identify the publisher type. This field can be populated with exactly one of the following one-letter values: "p"= programmatic, "d" = direct, "u" = unknown | p | d |
4 | ias_integration_type | string | String to identify the integration type for video impressions. The field can be populated with exactly one of the following two and three character values: "avw" = ActionScript wrapper, "jvw" = HTML5 wrapper, "av2" = web video SDK 2 (formerly known as VANS 2), "av3" = ActionScript web video SDK 3 | jv3 | jv3 |
5 | ias_fw_adv_entity_id | integer | The advertiser entity ID; required for FW. | 27351 | |
6 | ias_fw_pub_entity_id | integer | The publisher entity ID which is required for the FW placement level report. | 97334 | |
7 | ias_cm_anid | integer | The advertiser-network ID which is required for CM. | 5858 | |
8 | ias_cm_adv_id | string | The advertiser ID which is required for CM. | 1461 | |
9 | ias_cm_campaign_id | string | The campaign ID which is required for CM. | 20167 | |
10 | ias_cm_publisher_id | string | The publisher ID which is required for CM. | 159 | |
11 | ias_cm_placement_id | string | The placement ID, which is required for CM placement level reports. | 69964 | |
12 | ias_fw_ext_placement_id | string | The external placement ID for FW placements and is required for FW. | 7866 | |
13 | ias_cm_channel_id | string | The channel ID which is required for CM. | 1471215439 | |
14 | impression_count | integer | Your count of impressions which includes non-human traffic but doesn't include blocked impressions. | 7000 | 566557 |
15 | valid_impression_count | integer | Your count of valid impressions, excluding known non-human impressions and blocked impressions. | 5122 | 566251 |
16 | total_impression_count | integer | Your count of impressions including non-human traffic and blocked impressions. | ||
17 | valid_total_impression_count | integer | Your count of impressions excluding non-human traffic, but including blocked impressions. |
Select fields
Select fields allows you to define which fields are in your inbound file. To use select fields, you must create the file mapping.txt in the inbound folder (inbound/mapping.txt). mapping.txt defines the field and heading names for the inbound CSV. See the section 'Standard fields' for details on of the fields.
Each line in mapping.txt is a new column field: field=Column Heading Name. For example:
ias_fw_adv_entity_id=ADVERTISER_ID
ias_fw_pub_entity_id=PUBLISHER_ID
total_impression_count=IMPRESSIONS
impression_date=DATE
Then in the inbound file CSV, the first row must be the header row, followed by data rows which follow the heading row structure. For example, this inbound file has a header row, then data for ias_fw_adv_entity_id of 1, 4, and 7:
PUBLISHER_ID,DATE,IMPRESSIONS,ADVERTISER_ID
2,2017-11-29,300000,1
5,2017-11-29,600000,4
8,2017-11-29,90000,7
Outbound files
After processing inbound files, IAS creates a discrepancy management report containing a .report, and .xlsx.gz and .xlsx.md5 in the outbound/video_integration folder:
$FTP_ROOT/outbound/video_integration/YYYYmmdd.xlsx.report
$FTP_ROOT/outbound/video_integration/YYYYmmdd.xlsx.gz
$FTP_ROOT/outbound/video_integration/YYYYmmdd.xlsx.md5
The .report includes the total number of records imported, or if IAS could not process the inbound file.
The .xlsx.gz is the discrepancy report file with the previous day's date, and the .xlsx.md5 file is the accompanying .md5 file for the report.