December 19, 2005
By Elliot Mebane, Roguish.com
In September 2005 Macromedia released Macromedia
Studio 8, including Flash
8 and a new video codec from On2
that gives better video quality than earlier versions of Flash
video at the same file size, and also offers native alpha-channel
video transparency. Before this release of Flash, the Flash video
codec (made by Sorenson),
did not support transparency the new codec allows Flash developers
to composite video with alpha-channel transparency live at the
time of playback (including editable background text) in Flash
applications.
Whiskas wanted to release a virtual cat that would walk around on top of their website. They wanted users to be able to interact with it and it had to be real video of a cat Oh, and they wanted it in the first quarter of 2005, 6 months before the release of Flash 8 with native video transparency. So, Roguish and 2 other agencies worked together to create a method of faking transparent video.
A Solution... Abandoned...
The easy, highest-quality solution to the problem would have
been to shoot footage of the cat against greenscreen, key out
the background, then render out each clip as a series of .png
files with transparency. Flash supports the 24-bit transparent
.png format, so the .pngs could be imported into Flash and placed
on the timeline as a sequence of individual frames in a Flash
movie. Testing showed that this method results in file sizes
about 7-times greater than the file sizes of the movies we created
with our final solution.
The Real Fake Solution
While Flash 7 Video doesn't natively support transparency, Video
objects are treated just like all other objects in Flash and
can be masked by vector shapes. Do you see where I'm going with
this? If there were a way to create vector shapes around the
foreground subject in each frame of video (the cat), those vector
shapes could be used as masks to reveal the foreground subject
and hide the background, frame-by-frame. Flash 7 didn,t support
partial-transparency when applying masks, so with this solution
the foreground subject would have a solid edge. The tradeoff
is file size vs. natural feathered transitions from foreground
opacity to background transparency that can be achieved with
an 8-bit alpha channel.
From On-Set through Post-Production
It took a professional cat wrangler with his 3 nearly-identical
cats and 2-days of greenscreen capture to get 1200 clips to select
from. We culled 50 usable clips from the bunch. Careful placement
and sequencing of the final clips allows the cat to move through
a series of pre-defined motions, and the clips can also be dynamically
combined at runtime to allow users to interact with the cat -
feeding it, giving it a ball to play with and dangling a string
toy over its head.
Once the clips were selected, we used Keylight (bundled with AE 6.5 Pro) to extract the foreground subject from the background. Next, we created a new Solid layer and used the Auto-Trace feature (also part of AE) to automatically create a new mask for each frame in each clip based on its alpha channel. Auto-Trace was critical to making this project feasible within our timeframe and budget - without it, we would have had to resort to painstaking manual rotoscoping.
After some final hand-manipulation of the masks, 2 elements were exported from AE to be re-composited later in Flash: The masks were exported as vectors, and the videos were rendered as Quicktime movies for conversion to the Flash .flv video format using a third-party encoder.
More about the Vector Mask Exports
AE includes some basic exporting to the .swf format (Flash),
exporting vector elements such as masks as vector shapes, and
most other content as .jpgs. We wanted to include only the masks
in the .swf export, so before exporting the masks we disabled
the video layer. AE has a peculiarity when exporting masks to
the .swf format; The vector mask shapes are exported from AE,
but they are placed in the .swf as vector shapes that are masked
by a rectangular mask the size of the AE stage. This presented
some difficulty when the .swfs were brought into Flash to be
used as masks in the final composition (more details about this
problem and the solution in the Breeze recording tutorial linked
below).
More about the Quicktime Exports and
FLV Compression
At the time the project was being completed there were 3 commonly
used software packages for converting video to the .flv video
filetype that Flash uses: Sorenson
Squeeze, Wildform Flix,
and Macromedia's built-in encoder for Flash MX 2004 Pro. The
third party plug-ins created much higher quality than the built-in
encoder, and for this project we used the Wildform tool. FYI
- The story is a little different today; The encoder that now
ships with Flash Professional 8 encodes the newfangled Flash
8 video (still a .flv filename extension) and is quite good -
The 2 third-party encoders for the new format (On2 and Sorenson)
differentiate themselves from the built-in Macromedia encoder
with 2-pass VBR encoding for an incremental quality improvement
over the 1-pass CBR encoder that ships with Flash Professional.
For a look at the differences between codec manufacturers and
the new Flash 8 vs. old Flash 7 video formats, visit FlashVideoFAQ.com.
The site allows you to compare Flash 7 and Flash 8 video encoded
with the 3 major .flv-encoding software packages at a variety
of bit rates.
Bringing all the Pieces together in
Flash
After exporting the video layer from AE (without masking - just
premultiplied over black) and converting it to .flv format using
Wildform Flix, the .flv video was imported into a new Flash document
The associated .swf that was exported from AE including the vector
masks was also imported and placed on the Flash timeline in the
layer above the video. After some manipulation of the imported
.swf due to the quirky way that AE exports masks to the .swf
format, the vector mask layer was set to mask the underlying
video layer, and voila! Every frame of walking, jumping, sitting,
playing, eating, climbing and scratching cat was perfectly matched
with a vector outline, revealing only the cat and hiding the
background.
What is this Good For Today?
We would surely leverage the native alpha transparency in the
new Flash 8 video codec if we could deploy the Whiskas Virtual
Cat today, but that doesn,t mean that the process described here
isn,t useful in a variety of ways. Macromedia estimates that
it will take approximately 9 months after its release for the
Flash Player 8 (necessary to play the new Flash 8 video format)
to reach near-ubiquitous market penetration. This leaves a good
6 month window after the writing of this article for Flash developers
to use this process to deploy Flash video with transparency without
requiring their website visitors to download the latest Flash
Player plug-in. Additionally, the process can be used for a variety
of other purposes - vector shapes that trace the contour of an
object in video can be used as interactive and design elements
in Flash.
To view the Whiskas Virtual Cat, please visit the Whiskas USA website here, and select the USA version of the site:
The Breeze recording below is an audio/video tutorial of the procedure described above, detailing the entire process of faking transparency in Flash 7 video from the process of using Keylight to key-out the greenscreen through the process of using AE-generated masks in Flash: http://mmusergroup.breezecentral.com/p14271579/
copyright©Elliot Mebane2005
Elliot
Mebane is President of Roguish,
a multimedia production company in Los Angeles, CA with clients
including Apple, Yahoo!, Mars(Whiskas) TBWA Chiat\Day, ClearChannel,
Hilton Hotels, Toyota and McGraw-Hill.
Contact: info@roguish.com
Other resources mentioned in this
tutorial:
http://www.macromedia.com/
http://www.sorensonmedia.com/
http://www.wildform.com/
http://www.on2.com/
http://www.flashvideofaq.com/
