Archive for the 'Flash Video' Category

20
Feb

10 Most Popular Audio & Video file formats & Codec used by FlashPlayer

Video and audio files that are deployed on platforms with constrain and limitations such as Web and Mobile devices need to be compressed from the original format.

The compression is the usage of a codec (enCOder/DECoder) algorithm. Codecs, as their name implies, are programs that compress and decompress data. (There are audio codecs as well as video codecs). Compression has two purposes: reduced file size to speed up transmission as well as reduces the needed data storage space on the destination device.
Most codecs are lossy, which means some of the data is lost during compression and cannot be recovered during decompression. Lossless codecs preserve all original data, and are therefore a 100% faithful transcription of the original data set when uncompressed. Lossless compression is not well suited to the web or mobile devices.

A codec can either be a physical device or a software based process that, on the encoding side, manages the compression of raw digital audio or video data into files of reduced size, optimizing both download and playback performance. On the decoding side, the process is reversed, with the codec uncompressing the file to produce a high quality facsimile of the original content. Because the object of compression is to reduce the overall file size and streaming bandwidth requirements for a given segment of video, it is typically necessary or desirable to “throw away” some of the data during the compression process, meaning that when the codec reproduces content, it will have incrementally lower production values than the original. Compression, which discards some data in the compression and optimization process, is called lossy data compression.

At this point, we depart from the domain of science, and cross over into craft and art, because to create an acceptable result, compression algorithms must strike a complex balance between the visual quality of video and the volume of data necessary to render it. For purposes of multimedia content, the key measure of codec performance is the bit rate. In the context of transmitting multimedia data over the Internet or mobile carrier connections, bit rate quantifies the number of bits required per increment of playback time in order for the viewer to see smooth, uninterrupted content. For streaming video, this degree of playback quality is also called goodput. Goodput is the effective transmission rate supporting what the user actually sees on their device - in other words, it is the amount of data transferred after deducting things like internet, network and datalink layer protocol overhead; network congestion; and retransmission of data that was corrupted or lost in transit. The ability to empirically measure the performance of various codecs is key, because they have different strengths, and therefore, different applications.

Essentially, codecs are optimization tools, and they are many and diverse, often with thriving application genres based on them. The choice of a particular codec is driven by what rendering or transmission characteristics are the focus of optimization; what codecs a developer can reasonably assume to be present on the target platforms; and what post processing tools the developer has available for converting raw data into a video file format. It’s unsurprising that there is a great deal of competition among the developers of codec technology, because achieving a big advance in compression without a loss of quality would have tremendous commercial value. But, on the other hand, if all codec technologies were secret, there would be crippling fragmentation resulting from dozens of incompatible proprietary file formats for encoded video. This problem is neatly solved by an extensive, widely embraced standards-making process for video encoding.

Video codec designs are precisely specified by the Motion Picture Experts Group (MPEG), an international body, which includes 350 members representing media industries, universities, and research institutions. MPEG is chartered by the International Standards Organization (ISO) and is tasked with publishing standards documents that detail how various codecs work. What‘s interesting about this is that MPEG’s published specifications assume that the compression of video files is asymmetrical. In this sense, asymmetrical means that is it’s far more complex and difficult to compress data than to decompress it. As a standards making group, MPEG is exclusively interested in creating a framework for interoperability among various vendors’ codecs and products. This effectively means that only the decoding process needs to be enshrined in a public standard. The encoding process is not constrained by a published MPEG standard. As long as the compressed video files can be decoded as described in the MPEG spec, innovators are encouraged to design new and better encoders, achieving advances in optimizations, while secure in the knowledge they’ll reap the accompanying economic benefits. As encoder technology moves forward, the deployed decoder technology will continue to work, because the decoder side has no knowledge of encoder implementation and can’t be broken by encoder evolution.

Since there is a great deal at stake, the exact strategies of popular encoder designs are usually not public, but the nature of general recent advances is an open secret. Most codecs have transitioned from logic which compresses video data frame-by-frame to an object based model, where the encoder detects regions of frames that don’t change rapidly and caches those semi static portions. This is a tremendous advantage for bandwidth constrained scenarios like mobile video, because it prevents transmission of redundant data.

Both transmission speed and quality of the video rendering produced by decoding the results of various encoders can differ dramatically from one encoder implementation to another. In addition, there can be significant trade-offs in video codecs’ decoder runtime performance and resource utilization. It’s a subtle point, but an important one: Codec standards enable interoperability, but they do not imply uniformity of performance or quality across mobile devices. This potentially complicates life for content designers and developers, because it is necessary to know what codec is going to play your content back in order to ensure that video files provide acceptable playback performance. On desktop and laptop computers, there are frequently a variety of codecs available, and the presence or absence of a single one is rarely an issue for content developers. In any case, a desktop video app can request the user to download a needed codec if it isn’t already present. Not so with mobile devices.
Let’s go over some of the most popular audio and video formats.

1: FLV Format

FLV is the most popular video format available on the Internet with some of the best websites engaging their viewers with Flash based videos. This Flash Video format is also available Flash Player 6 as well as available on mobile phones from Flash Lite 3 player.

An FLV file encodes synchronized audio and video streams. The audio and video data within FLV files are encoded in the same way as audio and video within SWF files. SWF files published for Flash Player 6, Flash Player can exchange audio, video, and data over RTMP connections with Adobe Flash Media Server as well.

It is estimated that a one-minute video consumes 2 - 3MB of RAM, while a five-minute video consumes an average of 3 - 4MB. Longer videos play without requiring a linear increase in memory. This is true for progressive, streaming, local, and/or remote.

2: F4V And F4P Format

F4V is associated with FLV and many times you will see them attached together as the same format. What is F4V & F4P is simply the Adobe’s wrapper for the H.264 video. The reason there is even a need for a wrapper is to overcome the limitations that the H.264 format doesn’t support features such as alpha channel or cue points. F4V is available from Flash Player 9.0.r115 and higher. The format maintains dimensions and frame rate of source. The format also eliminates black borders. F4P is the protected video format.

FLV and F4V have open specification:
http://www.adobe.com/devnet/flv/pdf/video_file_format_spec_v10.pdf

3: MPEG-4 Format

MPEG-4 Part 14 also known as MP4 is a collection of audio and video encoding. MPEG-4 is considered the standard as many software companies such as Apple and Microsoft support the format. MPEG-4 is a container that allows you to combine audio and video (as well as other streams) into a single file. MPEG-4 video codec and H.264 are the included standards for video coding and compression. H.264 is the evolutionary step and improvement, providing improved capacity in terms of quality and efficiency. The format is available from Flash Player 9 and above.

4: H.264 Format

H.264 is the next-generation video compression technology in the MPEG-4 standard, also known as MPEG-4 Part 10. H.264 delivers excellent video quality across the entire bandwidth spectrum — right from 3G to High Definition Video Players. This format is preferred because it produces an incredible quality of video with the smallest amount of video data. This means you see crisp, clear video in much smaller files, saving you bandwidth and storage costs over previous generations of video codecs. The format is available from Flash Player 9 as well as Flash Lite 3.1.

5: MP3 format

Part of MPEG-1 standard, and also known as MPEG-1 Audio Layer 3. MP3 is a patented digital audio encoding format. The format is a popular audio format and the standard for lossy data compression of digital audio files.

Flash Player 6.0r40 and later are supporting MP3 and in fact audio in Flash Video files usually encoded as MP3. MP3 also supports ID3 metadata container, which allow passing data about the music file.

6: Advanced Audio Coding (AAC)

Supported by Flash Player 9 Update 3 and later Advanced Audio Coding (AAC) was designed to be the successor of the MP3 format. AAC is high-efficiency (HE) and high-fidelity (HiFi), low-bandwidth audio codec and a standardized, lossy compression data and encoding for digital audio. AAC is a higher quality format than MP3 and generally achieves better sound quality than MP3 at similar bit rates. The format is often packaged in a video format container.

7: MOV format

Available from Flash Player 9 Update 3 and up you can play MOV container using MPEG-4 codecs, it mostly interchangeable in a QuickTime-only environment. This is especially true on hardware devices, such as the Sony PSP and various DVD players; on the software side, most DirectShow.

8: 3GP and 3GPP Format

3GP is a simplified version of the MPEG-4 format; it is designed for mobile use. 3GP is based on MPEG-4 and H.263 video, and AAC or AMR audio The format is design to optimize the video content for mobile and specifically built to accommodate low bandwidths and little storage. 3GP format is a popular format within mobile devices and many support the 3GP format. The file extension is either .3gp for GSM-based Phones or .3g2 for CDMA-based Phone.

9: F4A and F4B format

From Flash Player 9 Adobe supports F4A & F4B audio/mp4 audio for Adobe Flash Player. The format is nothing more than MP4 audio file. F4A stands for an audio file while F4B stands for an audio book. The reason the format even exists is bridge and avoid compatibility issues between different platform such as Adobe Flash Player, Quicktime, iPod etc.

10: M4V and M4A

Flash Player 9 Update 3 supports M4V and M4A. While MP4 is the official extension, apple introduced M4V, M4A formats and are the standard file formats for videos and audio for iTune store, iPod and PlayStation portables.

M4V, M4A file formats are identical to MP4 and can be renamed to MP4. M4V stands for video while M4A by audio layer of MP4 movies.

Notice that M4V files contain DRM and the purchasing users info. You can use Requiem (http://undrm.info/remove-DRM-protection/Requiem-freeware-Mac-and-PC-DRM-remover-for-iTunes-files.htm) to remove the DRM.

So why did Apple created the format anyway? The difference in file extension allows to associate the file type with iTune and when you double click the file format you’ll have iTune opens up in case you have iTune installed. M4V are often used for movies, TV episodes, and music videos.

21
Jul

Getting Strarted with Open Source Media Framework (OSMF) formerly Strobe

The Open Source Media Framework (OSMF) formerly known as Strobe goes open source and provides a a framework for media playback of any kind.

To download the code visit:
http://opensource.adobe.com/wiki/display/osmf/Open+Source+Media+Framework

The video player ecosystem is complex. There are other elements that make a video player a complete composition, such as advertisement elements, social network elements, reporting, content management, DRM etc. Adobe recently announced a new framework called Adobe Open Source Media Framework that is aimed at solving these issues.

OSMF is an open source (vanity license) AS3 media framework that supports the workflow around video playback and monetization. Video players have different features set. The skins are different and the integration is different as well as the architecture workflow. But they do essentially the same thing and can be created using the OSMF framework. The framework is based on the quality of the video player (OVP) and addresses the common challenges.
Adobe vanity license means that it’s ok to use and redistribute. Modifications can be submitted to Adobe for redistribution.

The foundation of the framework is Qos (quality of service), which focuses on OVP and provides a quick start for playing videos (smallest buffer size needed to start the video), efficient connection logic, and switching bitrates dynamically (recall metric monitor service in OVP).

The framework by itself is not powerful without having the CDNs and the publishers onboard. Adobe OSMF is based on the “chicken and the egg” theory, meaning which will come first? In fact, Adobe is currently trying to get the publishers and the CDNs onboard and it seem that they got very positive responses. The idea is that each CDN will integrate their plugins to the OSMF framework and the publisher will be able to easily switch CDNs. This type of pluggable component can allow publisher to easily switch as well as test performance and service over different CDNs.
Advantages:

  • Reduce the barrier of entry for new publishers – by offering a framework to integrate the different pieces of the video player, new publishers can get started quickly and with fewer resources and scale up as requirements increase.
  • Provide flexible framework – OSMF provides an easy way to extend each component and allow these components to act as a Lego piece that can be compositable and extensible.
  • Leverage existing code – The OSMF framework uses the Flash Player from Open Video Player (OVP).
  • Drive Standard and allow custom workflows – many of the elements that connect to a video player are not standard yet and Adobe OSMF will help standardize these components as well as allow them to be configured.
  • No runtimes or framework dependency – the framework is based on Flash 10 AS3 and is not depend on any framework such as Flex SDK, Cairngorm or others. With that said some integrated elements may be created using a framework but these are loosely coupled and can be replaced if needed.
  • Partners focus – There are two partners: publishers and CDNs. Each can focused on their expertise. CDN can focus on services and integration and publisher can focus on user experience.
  • Integration with existing Adobe tools – Adobe OSMF will be integrated with other Adobe Suite tools and services such as Catalyst, Illustrator, FMS 3.5 and FMRMS.
  • Optimize – by the ability to separate the core framework and each element as a separate SWC you can increase performance by keeping file size to the minimum and based on components used.

The Adobe OSMF framework consists of three interests, which can be represented as an MVC pattern, as follows:

  • Model (Media elements) - media element (based on IMediaElement) may be video, image, SWF or others. Every media element has defined capabilities (traits based on IMediaTrait) and a life cycle, they are called Regions, for instance Region 1 (Image), Region 2 (Video) etc. Each media element implement IMediaElement contract, which contains the dynamic media traits (capabilities) and you can add or modify a trait. The traits are the building blocks and they can be set once and be reused.
  • Controller (Media compositions) – a media composition (IMediaFactory) is a collection of media elements. Each media can be rendered based on set of rules as well as media region that define how to use the capability and the life cycle.
  • View (Media configurations) – collection of media regions with properties that map to defined capabilities. Each media region can render each media element and can be spatially arranged and the entire media configuration can be the stage.

Let’s take a closer look at the higher architecture. The UI management is the set of tools we use to create our video player and skin it such as Flash Professional, Flex and soon we will be able to use Flash Catalyst for Flash 10 applications. Advertising companies such as DoubleClick or eyewonder allow us to add rich advertisement elements such as pre, mid and post roll ads, in playlist, or as companion ads or overlay ads. Applications provided by sites such as KickApps and gigya allowing embedding of the video player in social networks such as Facebook. The Flash Platform enable playback of video files. Companies such as Level(3) and Akamai provide syndications of feeds that can be consumed by other APIs. Lastly, stream management is available through Adobe’s server.

strobe1

The framework architecture was designed to allow you to integrate at multiple points and support different use cases, it also allow to use only the pieces you need, as well as the ability to scale up when ready. There are three types of implementation you may find useful.

  • Media Framework (level 1 integration) - small in size and include the basic integration such as the video player by OVP and pluggable CDNs. It allows hooking into your higher-level API, tweak the UI and access the needed classes and methods you define such as play and pause.
  • Media Framework and Composition Framework (level 2 integration) - In addition to level 1 level 2 allows creating a composition framework, which includes the ability to integrate to plugins that can provide monetization workflow for ads and tracking.
  • Configuration Framework, Media Framework and Composition Framework (level 3 integration) – in addition to the features that level 2 and level 1 offers, the level 3 integration offers dynamic chrome and syndication.

strobe2

Let’s see the framework in action

Let’s take a look at a simple implementation of the framework provided by Adobe based on the beta version. Keep in mind that the implementation was provided by Adobe and it’s under Adobe agreement; please refer to Adobe in regarding to the license agreement in term of usage.


package com.adobe.strobe.player
{
	import com.adobe.strobe.loaders.*;
	import com.adobe.strobe.media.*;
	import com.adobe.strobe.media.events.*;
	import com.adobe.strobe.traits.*;
	import com.adobe.strobe.traits.events.*;
	import com.adobe.strobe.view.*;

	import flash.events.*;
	import flash.net.*;

	public class MainWindow extends MainWindowLayout
	{

The childrenCreated method will be called automatically by the component once the child objects are created, since it’s replacing the default UIComponent method.
We create the MediaFactory, which holds all the media element using IMediaInfo. We than can call the registerDefaultMedia. And adds event listeners to the buttons. As well as bring back mediaPlayer object which is the component which is capable of playing any type of media by holding a media element that has all the traits we needs such as IPausible, which is a trait to pause and resume playing.


override protected function childrenCreated():void
		{
			super.childrenCreated();

			// Construct a loader factory:
 			factory = new MediaFactory();
			registerDefaultMedia(factory);

			// Add button handlers:
			buttonStop.addEventListener(MouseEvent.CLICK,onStopClick);
			buttonPlay.addEventListener(MouseEvent.CLICK,onPlayClick);
			buttonPause.addEventListener(MouseEvent.CLICK,onPauseClick);
			buttonResume.addEventListener(MouseEvent.CLICK,onResumeClick);
			buttonToggleMute.addEventListener(MouseEvent.CLICK,onToggleMuteClick);
			buttonLoad.addEventListener(MouseEvent.CLICK,onLoadClick);

			// Get a reference to our player:
			player = flexMediaPlayer.mediaPlayer;
			player.scaleMode = ScaleMode.LETTERBOX;
		}

registerDefaultMedia method creates all the media element that our application will be supporting, such as progressive download, streaming, audio files, images and SWFs.


private function registerDefaultMedia(factory:IMediaFactory):void
		{
 			var loader:ILoader = new VideoLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "progressive"
					, loader as IMediaResourceHandler
					, VideoElement
					, [VideoLoader]
					)
				);

 			loader = new RTMPLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "streaming"
					, loader as IMediaResourceHandler
					, VideoElement
					, [RTMPLoader]
					)
				);

			loader = new SoundLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "audio"
					, loader as IMediaResourceHandler
					, SoundElement
					, [SoundLoader]
					)
				);

			loader = new ImageLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "image"
					, loader as IMediaResourceHandler
					, ImageElement
					, [new ImageLoader()]
					)
				);

			loader = new SWFLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "swf"
					, loader as IMediaResourceHandler
					, ImageElement
					, [SWFLoader]
					)
				);
		}

loadMediaByURL method creates new media element based on the URL the user provided add listener and set it in our player component.


		private function loadMediaByURL(url:IURLResource):void
		{
			// Stop listening to the current media, if any.
			toggleMediaListeners(player.media,false);

			// Create the new media.
			var media:IMediaElement = factory.createMediaElement(url);

			// Listen for events related to the new media.
			toggleMediaListeners(media,true);

			// Set it on our media player.
			player.media = media;
		}

toggleMediaListeners method adds the event listeners to the media. Once the state have changed and the on flag is set to true it will call the event handler onMediaStateChange. The media element has a trait for loading the asset and once the onLoadableStateChange is captured the text field will display the information.


		private function toggleMediaListeners(media:IMediaElement,on:Boolean):void
		{
			if (media != null)
			{
				if (on)
				{
					media.addEventListener(MediaStateChangeEvent.STATE_CHANGE,onMediaStateChange);
				}
				else
				{
					media.removeEventListener(MediaStateChangeEvent.STATE_CHANGE,onMediaStateChange);
				}

				var loadable:ILoadable = media.getTrait(ILoadable) as ILoadable;
				if (loadable)
				{
					if (on)
					{
						loadable.addEventListener(LoadableEvent.LOADABLE_STATE_CHANGE,onLoadableStateChange);
					}
					else
					{
						loadable.removeEventListener(LoadableEvent.LOADABLE_STATE_CHANGE,onLoadableStateChange);
					}
				}
			}
		}

The event listener will handle the interaction with the media element based on the traits that got assign to the media.


		private function onStopClick(event:MouseEvent):void
		{
			player.stop();
		}

		private function onPlayClick(event:MouseEvent):void
		{
			player.play();
		}

		private function onLoadClick(event:MouseEvent):void
		{
			var url:String = urlInput.text;
			if (url.length > 0)
			{
				player.unload();
				loadMediaByURL(new URLResource(url));
			}
		}

		private function onLoadableStateChange(event:LoadableEvent):void
		{
			loadState.text = event.newState.toString();
		}

The onMediaStateChange event handler will handle the display and hiding of the toolbar.


		private function onMediaStateChange(event:MediaStateChangeEvent):void
		{
			if (player.media)
			{
				buttonStop.visible 	= buttonPlay.visible 	= player.media.getTrait(IPlayable) != null;
				buttonPause.visible = buttonResume.visible  = player.media.getTrait(IPlayable) != null;
				buttonToggleMute.visible = player.media.getTrait(IAudible) != null;
			}
		}

		private var factory:IMediaFactory;
		private var media:IMediaElement;
		private var player:MediaPlayer;

Constants to hold the different media files we can test.


		private static const REMOTE_PROGRESSIVE:String 	= "http://flipside.corp.adobe.com/pirates3/Pirates3-1.flv";
		private static const REMOTE_MP3:String 			= "http://flipside.corp.adobe.com/brian/strobe/media/Remember.mp3";
		private static const REMOTE_STREAM:String 		= "rtmp://flipside.corp.adobe.com/trailers/EvanAlmighty";
		private static const REMOTE_IMAGE:String 		= "http://www.adobe.com/products/mediaplayer/assets/images/amp_icon.jpg";
		private static const REMOTE_SWF:String 			= "http://flipside.corp.adobe.com/testing/swfPlayback/as3/Objects/Color/ObjClr_setRGB.swf"
	}
}

strobe3