May 10, 2021
The Making of Copilot Insights
Copilot Insights: The Art and Science Behind Our AI Visualizations
The digital media and advertising industry has witnessed an explosion of innovation that has embedded Artificial Intelligence across a suite of buying and measurement technologies.
Whilst these developments have driven undeniable value for brands, many still question: “What is the AI doing?” and “Why did the technology make those decisions?”. Deconstructing machine learning and big data remains a formidable challenge in an industry where millions of buying decisions are made each second.
Recently the Copilot team invested in bridging this divide to better answer “what is Copilot doing?” and more for our clients.
In October of 2020 the Copilot team embarked on a journey of reimagining Insights through the lens of machine learning and data visualization. It was important for our team to find innovative solutions to showcase the value of Copilot decisions being made behind the scenes. With a team of Data Scientist, Engineers and visualization experts we began looking at the common threads that different algorithmic components derive decisions from. Our goal was to compartmentalize the various ways campaigns are optimized and visualize these in artistic and relatable visualizations.
The challenge was to communicate how an ad campaign performs and delivers and explain how our product, Copilot, adds incremental value with machine learning optimization. Adding to the complexity, the project took place during COVID-19 and required collaboration across various teams during an extended period of remote working.
To design Insights, we first connected with Copilot users across many countries to identify and organize pertinent campaign information and analytics. Through this process we discovered three key categories of questions that surface when running any Copilot campaign:
What did Copilot do?
How did Copilot perform?
What can I learn about my audience?
Next, our product and visualization teams organized “sketch parties” to engage a diverse group of experts across our Copilot team and crowdsource hand drawn ideas. Our own Katherine Mello walks through the processes and thinking behind this virtual brainstorm here. (video)
“Sketch parties” included presenting a specific design challenge to the team ie. “How to visualize budget being optimized over time” and giving members 5 minutes to sketch ideas. After short presentations and discussions, elements from these ideas were collated, evaluated and merged into low fidelity mocks for further input from our users.
Copilot Sketch Party, examples from Data Science and Marketing
Through ongoing user interviews in Asia, Latin America, Europe and North America we refined low fidelity mocks into high fidelity solutions. During this process we consolodiated concepts such as the color scale that communicates goal perfromance on every slide and widgets that actively translate data into human digestable language. Our aim was to make visual processing of the package streamlined and easy to digest.
Low Fidelity Mocks, bringing ideas and concepts to life
Copilot insights was launched March, 2021 and provides an automated, in-depth analysis of a campaign. The package is available across all major DSPs to compare and contrast campaign performance in a unified view.
The Copilot Insight package contains ten bespoke views split across three sections: • How a campaign performed.
The optimization that took place.
The characteristics of a campaign that drive performance.
Table of Contents, Copilot Insights
Machine learning is embedded in analytics to predict uplift generated by individual optimization components. These models actively deconstruct Copilot optimization to estimate uplift on a given goal type. Uplift models are embedded in Copilot Insight visualizations and highlighted across the package.
Estimated Uplift in CPCV, Copilot Activation
When reviewing Insights available in the industry, we observed analyses often lacked simple human interpretability. To improve the Insight experience, we embedded widgets that translate data into human digestible language to provide key takeaways in every view. The goal of this was to make everyone an expert when interpreting data. These additional takeaways communicate salient campaign information and easy ways for our clients to understand campaign management.
Example Widgets, Sites & Applications
It was important for our design specialists to provide analyses in divergent, interesting and aesthetic formats. In pulling together various analytics we aim to simplify the complexity in applying AI in media buying and targeting.
Copilot Targeting, Features exploited on Copilot campaigns
We are proud to have pulled off a massive design feat. In under 6 months all from our home offices. Copilot Insights are now available on demand for any clients working with Copilot.
Geographical Analysis, Which regions contributed to performance
Copilot Color Theory
A lot of technical thought goes into making sure we have the right colors to communicate the story of our AI. Let us take you through the evolution of Copilot’s color palette and some key decisions that were made to improve our designs along the way.