THE RISE OF PROGRAMMATIC BUYING
More and more online ad space is sold nowadays via a process called programmatic buying. What this means, is that every time a person visits a website or an app with an advertising space, this space is offered for sale on an auction. Advertisers can now bid for this person’s attention and the highest bidder wins and can place their advertisement. The whole process happens in a couple of milliseconds, so that the person who’s being ‘traded’ does not notice anything special.
RIGHT MESSAGE, RIGHT PERSON, RIGHT TIME
Similarly to traditional advertising, the holy grail of programmatic buying is showing the right message to the right person at the right time. In other words, the targeting of the programmatic ads is very important to maximise profit. Unlike with many traditional advertising channels, however, in the process of programmatic buying, a lot of data about the ‘traded’ people is being collected. Every time a person is ‘traded’, the advertiser gets a message (bid request) containing the person’s unique ID (not traceable to the actual person, though) and the website/app this person is using. Advertisers store these data points in their databases, and over time build rich profiles of their targets. This behavioral data can be leveraged to optimise ad targeting, if only the advertisers manage to unlock the value of this data.
INTERESTS ARE MORE IMPORTANT THAN DEMOGRAPHICS
Traditionally, ads are targeted based on demographics of the person being targeted: sex, age, geographic location, etc. This way of targeting however, has several limitations. First of all, even though ad publishers provide some demographic information about their users, this information is often incomplete or inaccurate. Most importantly, however, advertisers are interested in the actual interests of their targets, and not their demographics. They show an ad of a fast car to young males, because they think this group is interested in fast cars, not because they’re male.
The behavioral data collected during the programmatic buying process is a rich source of information of people’s habits and interests.
BEHAVIORAL TARGETING WITH RAX
One of our customers who does a lot programmatic buying asked us to help them with using their data to segment their ‘targets’ based on their behavior and interests rather than demographics. These segments can be then used for better ad targeting (i.e. show car ads only to people interested in cars).
To create these behavioral segments we use Rax to preprocess the data and turn it into features that can be then fed into a clustering algorithm. The output of this process is a number of clusters – segments – that contain users with distinct interests.
THE DATA
The data are records of impressions with time stamps collected from different online ads exchanges. There are about 9 million users; the data is labelled with user IDs. The goal of the project was to group users based on similarities of how they use applications; therefore we ignored the demographic variables, event though they were also present in the dataset.
FEATURE CONSTRUCTION
For this project, we only use mobile app usages. Since the number of different apps in the dataset was very large, we first divided the apps in categories. We built categories using information in Google Play, correcting some categories. (e.g. different alarm clocks end up in different categories tools vs productivity). For each app category, we computed a usage value based on the ideas from document clustering (so called tf-idf index). This way of constructing features has been proposed in the scientific literature for segmenting users on their web browsing histories. Apart from that, for each app category, we also constructed features describing the usage of this category at different times of the day. To create features from raw data, we used Rax running on top of Snowflake. This is the Rax code that computes the tf-idf index for each us
er/app category:
{[#:user_id, #:app_id, #:df]}: document_frequencies :=
fold [.user_id, .app_id, /count():df]
impressions;
{[#:app_id, #:tf]}: term_frequencies :=
fold [.app_id, /count(.user_id):tf]
impressions;
{[#:user_id, #:app_id, &:tf_idf]}: tf_idf :=
project [.user_id, .app_id, (&).tf/.df]
document_frequencies |><| term_frequencies;
CLUSTERING
For clustering we use the scikit-learn (Python) implementation of the K-means++ algorithm. We’ve also tried other algorithms, like hierarchical clustering, but the choice of the algorithm doesn’t make that much difference. So we’ve settled on K-means++ since it’s one of the fastest algorithms.
Rax integrates easily with scikit-learn library through the usage of the `dataMatrix plugin which outputs a CSV file with the features in the right format, and the <$: "| cmd"> operator to call external scripts:
<$: "| cmd">
Where the contents of kmeanspp.py looks like that:
from sklearn.cluster import KMeans
from numpy import loadtxt
X = loadtxt("tf_idf.csv", delimiter=",", skiprows=1)
user_ids = X[:,0]
features = X[:,1:]
clusterer = KMeans(n_clusters=9, random_state=1, n_init=20, init='k-means++')
cluster_labels = clusterer.fit_predict(X)
f = open('cluster_labels.csv', 'w')
for i in range(len(cluster_labels)):
f.write('{0}, {1}\n'.format(user_ids[i], cluster_labels[i]))
f.close()
THE CLUSTERS
The result are a number of clusters of users with distinct interests, that can be used for ad targeting. Some examples of groups that we found:
- News junkies: people who mostly use their phone for news consumption. This group can be further subdivided based on which news channel they’re using. The users of one of the channels were much more likely to also use their phone for online shopping.
- Game junkies: people who do a lot of gaming on their phones. They also tend to do a lot of online shopping.
- Car enthusiasts: spend a lot of time looking at cars. Also above average when it comes to online shopping
- Plain old telephone: people who use their smartphone only as a communication device.
BACK TO TARGETING
The insights obtained this way allow for more precise targeting of the advertising campains. For example, when advertising an online shop the advertiser can bid more on news junkies who use a particular news channel. Also bidding more on game junkies and car enthusiast makes sense, while the ‘plain-old-telephone’ users are not as likely to convert.