Our Thoughts

May 10, 2021

The Making of Copilot Insights

Copilot Insights: The Art and Science Behind Our AI Visualizations 

The digital media and advertising industry has witnessed an explosion of innovation that has  embedded Artificial Intelligence across a suite of buying and measurement technologies.  

Whilst these developments have driven undeniable value for brands, many still question: “What is the AI doing?” and “Why did the technology make those decisions?”. Deconstructing  machine learning and big data remains a formidable challenge in an industry where millions of  buying decisions are made each second.  

Recently the Copilot team invested in bridging this divide to better answer “what is Copilot  doing?” and more for our clients.  

The Vision 

In October of 2020 the Copilot team embarked on a journey of reimagining Insights through the  lens of machine learning and data visualization. It was important for our team to find innovative  solutions to showcase the value of Copilot decisions being made behind the scenes. With a  team of Data Scientist, Engineers and visualization experts we began looking at the common  threads that different algorithmic components derive decisions from. Our goal was to  compartmentalize the various ways campaigns are optimized and visualize these in artistic and  relatable visualizations.  

The Challenge 

The challenge was to communicate how an ad campaign performs and delivers and explain  how our product, Copilot, adds incremental value with machine learning optimization. Adding  to the complexity, the project took place during COVID-19 and required collaboration across  various teams during an extended period of remote working.  

The Method 

To design Insights, we first connected with Copilot users across many countries to identify and  organize pertinent campaign information and analytics. Through this process we discovered  three key categories of questions that surface when running any Copilot campaign: 

  • What did Copilot do? 

  • How did Copilot perform? 

  • What can I learn about my audience?  

Next, our product and visualization teams organized “sketch parties” to engage a diverse group  of experts across our Copilot team and crowdsource hand drawn ideas. Our own Katherine  Mello walks through the processes and thinking behind this virtual brainstorm here. (video)

“Sketch parties” included presenting a specific design challenge to the team ie. “How to  visualize budget being optimized over time” and giving members 5 minutes to sketch ideas.  After short presentations and discussions, elements from these ideas were collated, evaluated  and merged into low fidelity mocks for further input from our users. 

Copilot Sketch Party, examples from Data Science and Marketing 

Through ongoing user interviews in Asia, Latin America, Europe and North America we refined  low fidelity mocks into high fidelity solutions. During this process we consolodiated concepts  such as the color scale that communicates goal perfromance on every slide and widgets that  actively translate data into human digestable language. Our aim was to make visual processing  of the package streamlined and easy to digest.  

Low Fidelity Mocks, bringing ideas and concepts to life 

The Result 

Copilot insights was launched March, 2021 and provides an automated, in-depth analysis of a  campaign. The package is available across all major DSPs to compare and contrast campaign  performance in a unified view.  

The Copilot Insight package contains ten bespoke views split across three sections: • How a campaign performed. 

  • The optimization that took place. 

  • The characteristics of a campaign that drive performance.  

Table of Contents, Copilot Insights 

Machine learning is embedded in analytics to predict uplift generated by individual  optimization components. These models actively deconstruct Copilot optimization to estimate  uplift on a given goal type. Uplift models are embedded in Copilot Insight visualizations and  highlighted across the package.  

Estimated Uplift in CPCV, Copilot Activation 

When reviewing Insights available in the industry, we observed analyses often lacked simple  human interpretability. To improve the Insight experience, we embedded widgets that translate  data into human digestible language to provide key takeaways in every view. The goal of this  was to make everyone an expert when interpreting data. These additional takeaways communicate salient campaign information and easy ways for our clients to understand  campaign management. 

Example Widgets, Sites & Applications 

It was important for our design specialists to provide analyses in divergent, interesting and  aesthetic formats. In pulling together various analytics we aim to simplify the complexity in  applying AI in media buying and targeting.  

Copilot Targeting, Features exploited on Copilot campaigns 

We are proud to have pulled off a massive design feat. In under 6 months all from our home offices. Copilot Insights are now available on demand for any clients working with Copilot. 

Geographical Analysis, Which regions contributed to performance 

Suggested Articles

Katherine Mello

Copilot Color Theory

A lot of technical thought goes into making sure we have the right colors to communicate the story of our AI. Let us take you through the evolution of Copilot’s color palette and some key decisions that were made to improve our designs along the way.