In this blog post, Pixalate explains X-Forwarded-For (XFF) and the best practices as it relates to detecting, targeting and even blocking the IP addresses associated with programmatic advertising transactions to reduce ad fraud and improve quality. This is a common issue digital marketers encounter, but it’s one that remains a mystery to many in the industry.
What is X-Forwarded-For (XFF)?
Proxying internet traffic through a server, gateway or peer-to-peer connection has become an increasingly common phenomenon on the web. However, when the internet protocol was originally designed, there was no standard method implemented to capture all the IP addresses involved in proxied transactions. As a result, a de facto standard has emerged, known as “X-Forwarded-For” (XFF).
The term “X-Forwarded-For” is added to an internet transaction to display the list of all IP addresses involved in a given transaction.
What does X-Forwarded-For look like in header data?
In the example web transaction below, this browser is using a proxy to reach its destination (www.pixalate.com). It is using the de facto X-Forwarded-For standard to send both the originating client IP address and the proxy IP address to the destination.
XFF IP addresses and OpenRTB
In this example, the X-Forwarded-For line appears at the end, showing both IP addresses: 184.108.40.206 and 220.127.116.11.
In the most recent publisher spec, OpenRTB 2.5*, and all prior specs, the IP address of the targeted client is sent in the Device Object as “IP” and defined as “IPv4 address closest to the device.”
- This means that if the XFF header is present, the IP address that should be used in any advertising transaction is the left-most, non-private address from the XFF line. This is the IP address that is closest to the device
- In the above example, this would be the IP address 18.104.22.168. It is the left-most non-private IP address on the “X-Forwarded-For” line in the sample header data.
- If the XFF header is not present, then the client IP address is the same as the “source” IP address.
If an SSP or exchange offering bid requests is not sending the correct IP address from the XFF header, they should be queried as to their methods of IP address detection.
Additionally, you can ask exchanges and SSPs to provide the full contents of the XFF header in the Device Object “ext,” or extension field. This field is reserved for customizations to the bid request made by agreements between you and your suppliers
Best practices for X-Forwarded-For (XFF) IP address filtration
- Case #1 - When the X-Forwarded-For (XFF) header exists
Per the OpenRTB guidelines, Pixalate specifies using the left-most IP address in the X-Forwarded-For header as the client IP address in all transactions including targeting, filtering and blocking.
When the XFF header is present, Pixalate also recommends checking all other IP values in the header for filtering or blocking, while not using them for targeting. In almost all cases, such IP addresses will result in false user-targeting, or even fraud.
- Case #2 - When the X-Forwarded-For header is absent
The source IP address from the header data should be used in the transaction for targeting, filtering or blocking.
- Case #3 - No access to the HTTP header
If you are not a direct party to the transaction and do not have access to HTTP header info, you should ensure that your partners are making this information available to you for every transaction in order to make your IP address targeting and filtration accurate.
If the information contained in the HTTP Header is simply not available in any context during the transaction, proceed with caution, and consider a solution for post-impression filtering and validation.
Direct vs. indirect client transactions
Direct Client Transaction
- When you have access to client direct transactions, you can read and filter the contents of the X-Forwarded-For header (where it exists) prior to sending or filling a bid request.
Indirect Client Transaction
- When you do not have direct access to the client and are receiving IP address information from a supplier, such as detailed in the OpenRTB specification, and the supplier cannot send you X-Forwarded-For information, you may choose to read and filter the contents of the X-Forwarded-For header after the impression is delivered.
- Suppliers should expect you to measure and validate such impressions and invalidate any ad spend that did not match the targeted IP address
*OpenRTB 3.0 is expected to be released in early 2018 and is not anticipated to change this part of the specification.
Want more data-driven insights? Sign up for our blog!