You are on page 1of 38

>

03
The Video Lifecycle is an
involved, multi-step process that
requires technical solutions and
services at every stage.

blinkx is the only technology


company to offer a full suite of
solutions for media owners’
and publishers’ requirements at
every step of the Video Lifecycle.
LOCATE
PROCESS
INDEX
CONTROL
MONETIZE
DELIVER
LOCATE BLINKX DISTRIBUTED
VIDEO FETCH SERVER (dVFS)
Today, most video content is captured and stored The blinkx distributed Video Fetch Server is a
in encoded digital files. These files may be stored configurable server that is capable of interfacing
in a private content management solution on a with a variety of video repositories (including
secure, private network or they may be public- over 50 distinct database and content manage-
ly available on the Internet. The remainder of ment systems) in order to locate and extract video
video content is maintained as legacy archives that is contained within them. The blinkx dVFS
in non-digital form. blinkx has developed the is architected as a parallel server that runs multi-
blinkx distributed Video Fetch Server (dVFS) ple spider modules which each trawl the reposi-
and blinkx offline Video Fetch Server (oVFS) to tories to be indexed. While some private systems
traverse and identify content in digital and non- can be relatively straightforward to interface
digital forms, respectively. with, offering some form of public interface or
export function, others are extremely complex
to spider. The blinkx dVFS is able to automati-
cally aggregate from all with minimal manual
configuration.
The award-winning blinkx dVFS Web Edi- In addition to static and dynamically generated
tion Spider has played a major role in building HTML, the blinkx dVFS is capable of analyzing
blinkx’s video index - now the largest in the and processing popular Web scripting languages
world. The Web Edition module is available ex- (including JavaScript/AJAX and Flash) and uses
clusively to our customers for spidering any sites variable substitution and code pre-compiling
that are of interest to them. techniques to infer content that is hidden in dy-
As with the standard dVFS, the Web Edition is namically generated pages.
fully parallelized in order to improve scalability
and is built on a complex process architecture
that supports dynamic resource allocation and

09
module self-replication in order to automati-
cally match the scale of the task set before it. The
spider trawls link structure, automatically bias-
ing towards likely sources of video content in or-
der to improve video yield.
BLINKX OFFLINE Parallel and massively scalable automated spiders
that intelligently focus to find audio/video content
VIDEO FETCH SERVER (oVFS)
If stored on tape or disk, offline video content Render page fully in memory, allowing videos’
must be captured and encoded before it can contextual metadata to be indexed
be used within a digital medium such as the
Ability to index any video from any page,
Internet. The blinkx offline Video Fetch Server
regardless of format (including Flash)
(oVFS) facilitates this process by interfacing
with standard video capture cards in order to Automatic generation of thumbnails, previews
capture streams of analog content from offline and word-timing
stores (such as tapes) or directly from live-air
Agnostic approach, metadata, video analysis,
broadcast. speech recognition and closed captioning

blinkx dVFS supports MySQL, Oracle and all other


common databases and standard C.M. systems

10
blinkx oVFS interfaces directly with offline video
sources including most tape and disk formats,
satellite, cable and terrestrial broadcast

Native support for over 50 common database


and content management systems
LOCATE PROCESS
PROCESS
INDEX Once a digital video has been found or created
CONTROL from an analog original, the next step is analysis.
This process involves breaking down the piece of
MONETIZE
content into several of its constituent compo-
DELIVER
nents and processing each one to fully under-
stand the video’s overall meaning. The resulting
data can serve as a foundation for functions such
as search, organization, selection or suggestion.
BLINKX VIDEOLOGGER
VIDEO ANALYSIS MODULE
12
index that accurately describes the video content.

blinkx processes video content using a multi- This precise, time-stamped index provides fine-
threaded server technology known as the Vid- grained access to the video content that can be
eoLogger. In constant development for the past used to efficiently search and locate a specific
ten years, blinkx’s VideoLogger is a central man- video segment for playback. Used together, the
agement service that is responsible for marshal- VFS and VideoLogger modules can simultane-
ling each piece of content through one or more ously index and digitize (encode) input content
analysis modules that extract information. to transform the video and audio assets into
accessible, Web-ready content.
The VideoLogger can be used in isolation but is
usually coupled directly with the Video Fetch Existing methods of making audio and video
Server (VFS) modules. searchable rely on either textual metadata (added
by professional editors or end-users as part of a
The VideoLogger analysis modules use a variety of ‘tagsonomy’) or closed caption data that is added
advanced image and audio analysis techniques to during the television production process. Both
automatically extract information about a video of these approaches are significantly flawed.
in real-time, facilitating the creation of a rich
Metadata is descriptive information (e.g., sum- Closed captioning is flawed primarily because it
maries and tags) created by a video’s original edi- is generated by human transcribers who can suf-
tor but often omits aspects of a video that may be fer from high error rates. Furthermore, closed
of interest to others. captioning is extremely rare on the Internet;
recent research by blinkx suggests that less than
Typically, it is only applied on a per-video 0.001% of all Internet video content contains any
basis, offering generic summaries of a video, and closed captioning. Even in cases where closed
is therefore an ineffective method of providing captioning exists, the majority of these videos
users with precise, granular descriptions of the only have basic titles that mark a content seg-
subtle details in a clip. Additionally, the prac- ment’s beginning and end - rather than a com-
tice of free tagging, especially when opened to a plete transcript.
community, is prone to spamming - where users
falsely apply descriptors to content to subvert the If they exist, the blinkx VideoLogger does ex-
search process. tract and use metadata and closed captioning as
the first step in the indexing process. In addition,
blinkx’s technology utilizes advanced speech
recognition and visual analysis techniques to
analyze and understand the spoken word and
visual content of an audio/video file, ensuring Once the video or audio stream has been indexed,
unparalleled comprehension of online multi- encoded and analyzed, the digital video files and
media content. the descriptive analysis output are stored in the
blinkx video index.
The VideoLogger can also control the encoding
process using third-party encoders that output
in popular formats including MP4, Flash, Real
and Windows Media. Using an output module,
the VideoLogger controls both indexing and
encoding processes to ensure synchronization
between the metadata captured from the video

14
asset and the associated digital file. The output
of these analytical modules are stored as further
metadata tracks alongside the digitally encoded
content itself; not only does blinkx know what
was said, blinkx knows exactly when it was said.
AUDIO ANALYSIS MODULE
blinkx’s audio analysis technology uses advanced The technology is based on decomposing digi-
statistical methods to deal with all aspects of tized speech into its phonetic constructs. The
processing the digital audio signal from an phonetic sequence is then analyzed in conjunc-
audio or video stream. It employs a wide range tion with the acoustic model and statistical prob-
of recognition technologies – from keyword- abilities to calculate the most probable sequence
and phrase-spotting to continuous vocabulary of words and utterances.
speech, speaker and language recognition.
LARGE VOCABULARY RECOGNITION
In order to analyze the spoken words of an audio Unlike traditional speech recognition systems
or video stream, blinkx uses audio analysis tech- which have fixed vocabularies, blinkx supports
niques that are based on neural network tech- large text corpora, including hundreds of mill-
nology and Hidden Markov Models (HMMs) to ions of words that can train the system and
construct an efficient acoustic model that can refine its accuracy according to the specific
provide a fast, accurate and dynamic solution requirements of its customers.
within rapidly changing acoustic environments,
such as radio and television.
Using patented predictive technology, blinkx’s was developed as a tool to maintain inter-speaker
speech analyzer can offer users the benefits of independence - not to be a single user transcrip-
a large vocabulary speech recognition system tion tool.
without the overhead of a vast search space.
Rather than relying solely on existing metadata NON-DICTATED SPEECH
to describe an audio or video clip, blinkx has the Information feeds, such as news broadcasts and
ability to retrieve a wide range of multimedia radio, are often difficult to transcribe due to
content based on the words spoken in the televi- noisy conditions and less-than-perfect articula-
sion or radio clip. tion. blinkx’s sophisticated signal processing and
statistical techniques enable the transcription
SPEAKER INDEPENDENCE engine to filter out extraneous noise, compensate
Whereas other approaches require training data for low volume levels and probabilistically pre-
from specific speakers to realize their full poten- dict intended dialogue.
tial, blinkx performs consistently well across a
wide variety of previously unrecognized speech
sources. Transcription of speech and segmenta-
tion by speaker requires no initial per-speaker
training because blinkx’s underlying technology
PHONEME-LEVEL PHRASE Boolean protocols, blinkx VideoLogger enables
AND WORD SPOTTING users to search audio data from a range of sources
blinkx breaks down all forms of speech into using multilingual natural language queries.
phrases, words and phonemes (the smallest
sound units in a particular language), offering IMAGE ANALYSIS MODULE
an exceptional granularity of understanding. blinkx VideoLogger’s advanced video capture
and analysis technology also utilize neural net-
MULTIPLE LANGUAGE works and HMMs to optimize the encoding of
MODEL ARCHITECTURE content in real-time. A comprehensive range
blinkx’s core technology is entirely language of media analysis plug-ins allow for the auto-
independent, enabling multiple languages to be matic creation of metadata and the ability to
simultaneously processed and searched. blinkx search entire media streams or clips by a range of
combines both phonetic and conceptual methods parameters such as audio, scene, speaker, loca-
to disambiguate the limitations inherent in tra- tion, key frame, image, on-screen text, face,
ditional approaches, combining more accurate token and concept.

17
language recognition with effective informa-
tion retrieval. Besides also supporting traditional
legacy techniques such as keyword-spotting and
By making video easy to identify, locate and time of day. Additionally, it utilizes techniques
re-use, blinkx’s VideoLogger allows the elements such as comparing object histories, motion detec-
to be assembled and repurposed faster and with tion, object sizing, object tracking, object count-
greater accuracy than ever before. blinkx is capa- ing and behavioral analysis; putting each object
ble of a wide range of intelligent video analytics and motion in context.
functions, including:

FACIAL IDENTIFICATION
LOGO AND SCENE-CHANGE DETECTION blinkx’s video analysis also offers powerful bio-
The blinkx VideoLogger is capable of automati- metric identification tools to enable facial recog-
cally detecting, analyzing and interpreting all nition. However, traditional 2-D facial recogni-
activity within video data and can, for exam- tion has fundamental limitations with regard to
ple, interpret and understand the significance posture, expression and lighting. The blinkx Vid-
of specific images or note scene changes. Using eoLogger employs superior three-dimensional
advanced techniques, the blinkx VideoLogger recognition techniques rather than 2-D facial
identifies and categorizes objects in a scene by matching processes for optimum performance.
size, shape, color, speed, direction, location and
19
ON-SCREEN CHARACTER RECOGNITION
Neural networks based on optical character rec-
ognition techniques allow the blinkx Video-
Logger to support advanced character recogni-
tion. Unlike template-matching used by other
systems, which is dependent on receiving high-
quality images, blinkx’s visual analysis techniques
provide much greater tolerance for matching
poorly-defined characters.

With the ability to integrate with multiple data-


bases and automatically cross-reference and cor-
relate identified characters with other data,
the blinkx VideoLogger offers the most sophis-
ticated, comprehensive, end-to-end solution
which encompasses every aspect of character
recognition together with advanced recording,
retrieval and analytical capabilities.
INDEX
LOCATE
PROCESS When a user requests a specific piece of content
INDEX or a suggestion of something new to watch, that
CONTROL request is processed by the blinkx Index.
MONETIZE
The Index uses a complex, multi-dimensional,
DELIVER
pattern-matching process to compare the request
to its records on each piece of available content,
and then uses its findings to create a list of the
most relevant suggestions. These suggestions are
fed back to the user, either in an ordered results
list that can be organized in a number of ways,
or by creating a channel or playlist of content
pieces that can be consumed sequentially.

The blinkx Index is a platform-indepedent serv-


er that typically runs as a single virtual service
supported by multiple actual nodes, running on the need through analysis of the many content
distinct physical machines that can be in mul- records at its disposal. These functions are split
tiple locations. blinkx’s Index architecture sup- into three broad groups: Search, Category and
ports linear scalability, redundancy and fault- Community.
tolerant functionality and is supported by a
fully-featured automated service architecture
that is able to automatically identify service re-
quirements and provision resources as required.

LOCATE
CONTROL
PROCESS
The blinkx Control layer is accessed through INDEX
a large family of functions - each of which CONTROL
represents a different way of expressing a partic- MONETIZE
ular requirement to the video index. Each causes DELIVER
the index to react in a different way, meeting
SEARCH FUNCTIONS
These functions allow a user to explore and keyword
search the blinkx Index with a high degree of
control and accuracy. blinkx’s technology sup-
boolean
ports keyword search, Boolean search, concep- phrase
tual search, automatic hyperlinking, fielded or conceptual
meta-search, federated search, parametric search
and guided-navigation.
contextual
hyperlinked
parametric
guided
clustered

22
CATEGORY FUNCTIONS
blinkx Category functionality facilitates the def-
inition of a manually designed or automatically - manually defined taxonomy and ontology
inferred taxonomy or ontology of subjects and
topics. This taxonomy can then be used as a ba- - automatically generated taxonomy and ontology
sis for the automatic organization of content—
incoming media is automatically sorted into - automatic categorization
suitable categories, allowing for easy retrieval.
- automatic topic clustering
Systems powered by blinkx are capable of intui-
tively sorting information on a grand scale, with - theme spotting
the assistance of Category functions. blinkx’s
Category functions also support Topic-based - user and community categorization
Clustering that can identify trends within in-
coming data. For example, Topic-based Cluster-
ing can spot breaking news topics or suggest a
taxonomy that will best describe a particular
corpus of video.
COMMUNITY FUNCTIONS
With the Community functions, a blinkx-pow- ‘leaders’, suggest user groups based on those who
ered system can infer profiles based upon the have similar interests and recommend content
consumption and creation of content by users based upon the creation, consumption and shar-
and groups. ing of content.

blinkx supports both implicitly generated pro- - thematic clustering


files that are built on automatic observation - explicit and implicit profiling
of user actions, and explicit profiling based on - alerting
user-driven preference setting and training. In - community suggestion
all cases, blinkx understands that individual us- - group selection
ers can have many diverse interests - blinkx pro- - audience/content demand profiling
files are multi-faceted by design.

These profiles can be used to support a wide va- SORTING AND FILTERING
riety of recommendation strategies: alert users Regardless of the function used, blinkx’s Index
to new content that will be of interest to them, results lists can also be sorted and filtered in a

24
automatically identify users who are topic number of ways:
SORTING FILTERING

BY CONTENT PROVIDER: SAFE FILTER:


preferential weighting of content from speci- pre-populated filter which blocks inappropriate
fied sources over all other content providers in content in order to facilitate family-friendly
an index results lists

BY RELEVANCE OR DATE: QUALITY FILTERS:


view content results based on best possible match ensure that only content of certain quality levels
to search query or freshness of content (measured by frame rate, resolution, bit rate or
destination network latency) are returned
BY ARBITRARY METADATA:
sort a results list by any of the fields of metadata FORMAT FILTERS:
which describe each piece of content (e.g., sort by block content encoded in formats the user does
author, security clearance, number of comments not wish to view, allowing for return of files
or number of views) only in Flash, Windows Media, RealPlayer any
required combination of content formats
MONETIZE
LOCATE
PROCESS Persuasive automated advertising must combine
INDEX historical, demographic knowledge of a user
CONTROL with an understanding of the content that he or
MONETIZE she consumes at a given moment in time.
DELIVER
Put simply, knowing who is watching what and
why makes it easy to select highly relevant ad-
vertising. blinkx’s ad platform, AdHoc, achieves
this by first capturing these inputs and then au-
tomatically synthesizing them to deliver a selec-
tion of advertising that aims to best monetize a
particular user or content event.
UNDERSTANDING PEOPLE UNDERSTANDING INTENT
When combined with a form of user identifica- Users often watch the same piece of content
tion harness (such as a user account or Internet for different reasons. The key to capturing the
browser cookies), blinkx’s profiling technol- intent of a given user at a specific point in time is
ogy can follow a specific user over an extended to follow the search activity that led them there.
period of time. blinkx’s analysis of users’ media If a user finds a piece of content based on search
consumption enables it to automatically build a activity through a blinkx-powered engine,
multi-faceted profile for specific users’ interests. blinkx is able to capture both the search leading
to the consumption of that content and also the
failed searches that preceded it.
UNDERSTANDING CONTENT
As described in the “Process” section, blinkx’s SYNTHESIS AND SELECTION
VideoLogger analysis modules can extract and blinkx’s AdHoc platform combines the access-
understand all information about a video clip. ible inputs with manually defined business rules
and applies the remaining analysis to databases
of available ads. blinkx partners can decide

27
which ad databases are leveraged – their own,
the blinkx platform or third party ad networks.
Once relevant ads have been identified, AdHoc BLINKX UN-ROLL UNIT
can deliver them in a variety of ways. The blinkx Un-roll unit allows the viewer to
engage with a brand continuously throughout a
BLINKX AD DELIVERY FORMATS video. The experience begins with a branded cur-
tain that draws back to reveal the video. As the
PRE-, POST- AND INTERSTITIAL-ROLL video plays, touch-points such as overlay ads and
These popular ads are full-screen video ads that logos appear at contextually relevant moments
are played before, after or during a piece of con- within the video, made possible by blinkx’s Ad-
tent (respectively). Though extremely arresting Hoc technology. The video ends with a clear
as they play directly in the video player window, call-to-action and the viewer has the option to
roll ads are less popular because they interfere continue to the advertiser’s Web site.
with the clip playback experience. Roll ads are
priced based on an impressions model. OVERLAY ADVERTISING
Like user-initiated ads, overlays appear in the
video player. In contrast, however, overlay ads
are display advertisements in themselves, con-
taining graphical elements and a form of mes-
saging or call-to-action that is always visible.
VIDEO SEO
29
While it is possible for users to further investi- Online video requires much more sophisticated
gate by clicking on the ad, overlay advertising is methods of Search Engine Optimization (SEO)
generally priced based on an impressions model. than traditional text-based content, due to
complexities and nuances inherent in its form.
TEXT AND BANNER ADS blinkx’s advanced technology solves this prob-
Text and banner ads are the most traditional form lem. Publishers can take advantage of blinkx’s
of Internet advertising and are used extensively unrivaled understanding of video content by in-
in non-video content. However, blinkx AdHoc is tegrating our technology with their Web prop-
format agnostic and can therefore apply its un- erties to greatly augment existing SEO initiatives
derstanding of a viewer and content to these ads, and strategies.
selecting keyword and banner ads that are of rel-
evance to the current user and/or content. AUTOMATIC VIDEO ANALYSIS
AND PROCESSING
To optimize video content for search engines, it
is necessary to create as much textual informa-
tion about it as possible, in order to maximize
ways the video can be searched and retrieved. ENTITY EXTRACTION
Like traditional text-based technologies, blinkx blinkx is able to generate a massive amount of
generates textual information about standard textual information about a given piece of video,
titles, categories, and user-created tags, but in so it’s critical to be able to refine searchable infor-
addition, blinkx actually listens to, watches, and mation into the most relevant descriptive units
reads video content. or entities, that describe the video’s most basic
components. blinkx’s technology automatically
This means blinkx has the power to analyze and and accurately assesses context and pulls out key
process not only textual content, but also audio words, so publishers attract the most relevant
and visual video components, using speech rec- possible audience to their video.
ognition and visual analysis technologies. These
processes greatly increase the number of words Video entities such as names, statistics, and lo-
associated with a given video, thus driving more cations are extracted from information associ-
traffic to it. blinkx’s enhanced sources of de- ated with a video object, like comments, titles,
scriptive data also enrich SEO content by allow- tags, and audio. After identifying these entities,
ing Web developers easy access to video content, blinkx tags a publisher’s content with broad de-
no infrastructure involvement necessary. scriptions of the video’s content.
These descriptions automatically populate the Video assets and generated tags are then scruti-
conventional components of a Web page, like ti- nized to identify overall concepts, to yield more
tles, html, or optimized URLs, to make a page as accurate, relevant results than are possible with
accurately searchable as possible. Additionally, keyword-based search technologies.
these extracted entities can be used to provide
superior navigation via site structure, categori- By recognizing complex concepts within the
zation, and taxonomical integration. video, rather than simply scanning inferior tag-
ging, blinkx determines relevance based on actu-
CONCEPTUAL UNDERSTANDING al content, not subjective human interpretation.
For optimum SEO, it is essential to understand blinkx delivers advantageous ways for search
the key concepts or themes associated with a vid- engines to drive relevant audiences to publish-
eo’s content, because they determine relevance er content by placing it in context, whether in
and ultimately drive navigation. In order to es- search results or associated links.
tablish these concepts, blinkx automatically ex-

31
tracts and indexes textual information about the
video to further its conceptual understanding.
LOCATE
PROCESS
INDEX STEP 1:
CONTROL DISPLAYING VIDEO
MONETIZE SEARCH RESULTS
DELIVER It makes sense for search engines to display results
for text-based web pages as text to efficiently
assist users. While many pieces of video content
DELIVER also have textual titles and metadata which could
be used to provide relevant summaries, these are
Once relevant videos and necessary advertising far-removed representations of the content itself.
have been matched, the content must be delivered They force the user to judge moving images and
to the viewer. This is a two-step process: first, the sound simply on the basis of words. In order
multiple pieces of content are displayed in the to allow for efficient user appraisal of results,
form of a search results list. Then, once a specific blinkx has developed “Moving Thumbnail
video has been selected for viewing, it must be Generation”, a unique, patented display method
played to the user. to summarize video clips.
BLINKX’S MOVING
THUMBNAIL GENERATION
33
For example, in the case of a longer form video
that covers more than one topic, the Thumbnail
blinkx’s patented Moving Thumbnail will feature screenshots related to the search
Generation technology analyzes every incoming query. A user is therefore able to swiftly view
video file and creates a number of visual and assess how relevant a given video is to their.
thumbnails – short, compressed video segments
that represent different points in time of a given STEP 2:
clip. Thumbnails are generated either arbitrarily DELIVERY OF CONTENT
(e.g., every minute, every 10 seconds, at the In-browser streamed video is an inexpensive
start and end, etc.) or, more typically, based on method of delivering content via popular
specifically identified events within the video streaming technologies such as Real streaming
(e.g., the utterance of key words, the appearance format, Windows Media, Flash Audio/Video
of a famous face, etc.). Later, when a video is and Apple’s Quicktime. blinkx supports all of
listed as relevant to a user’s search, blinkx not these formats and, in addition, can deliver in-
only returns the textual summary and title of the browser video using a number of lesser-known
relevant video, but it also displays the Moving standards.
Thumbnail that most closely demonstrates why Direct streaming of video, on the other hand, is
a given video is relevant to the search. an expensive process.
To guarantee high-quality content and low BLINKX HYBRID
latency, content providers often use a costly PEER-TO-PEER STREAMING
content delivery network that charges a high
price per data unit of transferred content. blinkx’s hybrid peer-to-peer technology supports
the on-demand streaming of television-quality
With regard to online video, these costs can content over a typical home broadband connec-
accumulate rapidly, especially in the case of tion.
popular clips that are viewed by thousands or
millions of viewers in short periods of time. For instance, BBTV uses seed servers to originate
To maintain both quality and latency while content and, as increasing numbers of users
dramatically reducing bandwidth costs, blinkx select and watch a given media file, redistributes
has built BBTV (blinkx Broadband TV) - a peer- the content to the network of peers. Peer-to-peer
to-peer streaming and download technology streaming shares distribution costs at various
that vastly improves the efficiency of providing levels of a network hierarchy, resulting in lower
large-footprint content to a user. delivery costs without sacrificing playback
quality.
BLINKX
DRIP-DOWNLOADING BLINKX DIGITAL RIGHTS MANAGEMENT

blinkx’s drip-download technology is a large-file


download manager that allows content owners In addition to employing blinkx’s peer-to-peer
to slowly download media to a user’s computer and drip-download technologies, BBTV also
over a period of time. supports the Microsoft Windows Digital Rights
Management (DRM) architecture.
The drip-download takes place unobtrusively
in the background, automatically throttling This technology ensures secure handling of
its download rate based on user activity, thus owners’ and licensors’ content. Only authorized
minimizing reduction of a user’s productivity. viewers can watch DRM-protected content and
Drip-downloading is an efficient, low-impact re-broadcast or share when explicitly allowed to
method of delivering extremely high quality do so.
content to a user when streaming which a peer-
to-peer approach simply cannot deliver. BBTV’s
drip-download service is typically used to deliver
HD or DVD-quality films at full-feature length.
DRM

You might also like