Custom Animation Assets - Performance Lookback

Summary of Analysis and Findings

Hypothesis	Do Custom* assets (boxart and/or billboard) drive meaningful performance for Animation titles?
Impact	Custom assets are very expensive and time consuming to acquire. Understanding how they best serve animation titles helps PCS develop the best spend strategy for which Animated titles should get custom assets going forward.
RESULT	Vertical boxart: Custom assets DO appear to drive a measurable lift in title performance.  Billboard: Inconclusive
Additional Findings	The primary asset performs well consistently across all animation titles. 75% of primary assets are custom assets. There are strong compositional similarities in high performing and low performing asset types (custom or non-custom).

July 2020

Context and Summary

The PCS Producers and Assets Team members responsible for our Original Animation slate are finding it challenging to acquire custom assets (character poses, layered stills, environmental plates) for our fully owned OA titles. These custom assets are not consistently required by contract, so PCS is more frequently being asked to cover the cost of these assets, which are quite expensive (ranging from $10K to $375K). PCS wants to better understand how valuable these assets are within product artwork suites, so that the team can assess whether or not absorbing this additional cost is beneficial for the business.

Additional Reasons to Source Custom vs In-Stream Assets:

Late IMF delivery

EX: For Klaus, final IMF (with final animation in color vs animatics/line drawings) was delivered one month before launch day. In order to hit our Original Language artwork lock deadline (-5weeks) we had to work ahead of the film’s delivery with only custom assets.

Stylistic Limitations

There is a heightened complexity in building custom assets for Animated titles ourselves (outside of the original production/animation team), as the style is so unique to the content itself. We’re unable to pull from stock imagery (as we often do for Live Action) and cannot simply rebuild the world without risk of misrepresenting the unique animation style. 

Poor Legibility in Flat Artwork

When working with instream assets only, we’ve found it difficult to adjust lighting without layered assets. Characters often look discolored or are hidden by harsh shadows. This leads to poor legibility on characters.

Many background elements are overly complex and colorful in animated content. The depth of field is less apparent when capturing static/flattened stills, which adds friction -- muddying artwork and complicating focal point. 

Future Proofing for Flexible Assets (Universal Base)

It’s much more challenging to find a center-focused asset from in-stream assets alone (especially with all characters grouped together, eyes open, facing forward, etc). Custom poses and plates allow us to build out a center-focused asset that maintains flexibility across canvases.

Experiment Summary

Experiment Name	Method	# of Titles	# of Assets that Survived 1,000 profile per asset threshold	Date Period (Titles launched)
Custom Animation Assets Look-Back Analysis	Annotag; Producer Tagged 'custom' or 'not custom'	32  [title list]	Billboard: 7  Boxshot: 26	01-10-2020 through 07-10-2020

RAW TAGGED Data (Viewable in Bmuse) [BMuse all assets][BMuse tagged assets]

RESULTS (Provided by DSE, Full Deck Here )

Digging In

AOS conducted further investigation of image trends amongst high and low performing custom assets as well as high and low performing non-custom assets. When viewing the compositional patterns and similarities between the high and low performing assets (regardless of wether they are custom or non-custom assets) we begin to see strong similarities in the custom vs no-custom assets. 

The following compositional patterns have been identified by AOS with heavy guidance from to Animation producers, Brittany Hicks and Karen Bu. These patters are strictly observational and qualitative at this point in time, however, suggest a strong hypothesis should be tested to better understand how compositional patterns drive performance in animated titles. 

Compositional Patterns in High Performing Boxart for Animated titles 

• Single Character Focus, character face occupying 30-75% of pixel space
• Clear Emotional Expressions, BIG eyes (often but not always positive emotions)
• High Contrast, making it easy to visually distinguish character (or subject) from background
• Exaggerated Compositional Symmetry  

Compositional Patterns in Low Performing Boxart for Animated titles 

• Low Contrast, making it hard to visually distinguish characters (subjects) from their backgrounds
• Depiction of Conflict or Fear, angry, "mean" or unclear character expressions
• Dark, saturated colors
• General "busy-ness" or use of to many environmental/visual cues that lack context

Building on This Work

When we take a qualitative look at the results of the highest performing and lowest performing assets in the set, we begin to see clear compositional patterns suggesting that a stronger indicator of performance might be the the compositional content of the images rather than the source by which they were created. 

A follow up analysis is planned to dive deeper into understanding and validating the compositional similarities between high performing and low performing assets for Animated titles. Updates will be shared as they become available. 

Takeaways for PCS:

• There is evidence to suggest that custom assets drive a measurable lift in title performance. We believe this is highly correlated with composition.
• Because custom assets often provide more flexibility and clarity in composition, we believe it is still beneficial to source custom assets at this time.
• Some titles can achieve high quality composition without custom assets, so it is important to use your best judgement when commissioning custom assets for a title.

Questions to help frame your decision:

1. Does the cost align with potential reach and business impact of the title?

• If yes, consider sourcing custom assets
• If no, consider a lower number of custom assets or Instream assets only

2. Do the lighting/effects/animation style make it challenging to achieve quality composition and accurate representation of characters?

• If yes, consider sourcing custom assets
• If no, consider Instream assets only