Archive for the 'Flash Video' Category

19
Jun

Adobe Open Source Media Framework (OSMF) - revisit

About a year ago I have posted a blog post on how to get started with OSMF (see here). Since than the framework have gone a long way. OSMF 1.0 was released recently and I have given it a second look.

I want to give credit to Brian Riggs and David Hassoun for helping with the content of this blog entry.

OSMF is an open source AS3 media framework that supports the workflow around video playback and monetization. Video players have different features set. The skins are different, and the integration and architecture workflow are. They essentially do the same thing and can be created using the OSMF framework. The framework is based on the quality of the video player (OVP) and addresses the common challenges.
The foundation of the framework is Qos (quality of service) which focuses on Open Video Player (OVP) and provides a quick start for playing videos (smallest buffer size needed to start the video), efficient connection logic, and switching bitrates dynamically (recall the metric monitor service in OVP).

The framework can be used as follows:

  • Create a component user interface for integration of audio, videos, and images.
  • Plugin architecture for integration of the following: CDN, ad partners, publishers, analytics, social networks, and developers.

The framework by itself is not powerful without having partners embracing the frameworks – e.g. CDNs, publishers, ad analytics, social networks, and developers. This type of pluggable component can allow publishers to easily switch and test performance and service across different services.
Advantages:

  • Reduce the barrier of entry for new publishers – By offering a framework to integrate the different pieces of the video player, new publishers can get started quickly and with fewer resources and can scale up as requirements increase.
  • Provide flexible framework – OSMF provides an easy way to extend each component and allows these components to act as Lego pieces that can be compositable and extensible.
  • Leverage existing code – The OSMF framework uses the Flash Player from Open Video Player (OVP). No need to have duplication of efforts to solve basic problems.
  • Drive Standard and allow custom workflows – Many of the elements that connect to a video player are not standard yet, and Adobe OSMF will help standardize these components and allow them to be configured.
  • No runtimes or framework dependency – The framework is based on Flash 10 AS3 and is not dependant on any framework such as Flex SDK, Cairngorm, or others. With that said, some integrated elements may be created using a framework, but these are loosely coupled and can be replaced if needed.
  • Partners focus – There are two partners: publishers and CDNs. Each can focus on their expertise. CDN can focus on services and integration, and publishers can focus on user experience.
  • Integration with existing Adobe tools – Adobe OSMF will be integrated with other Adobe Suite tools and services such as Catalyst, Illustrator, FMS 3.5 and FMRMS.
  • Optimize – By having the ability to separate the core framework and each element as a separate SWC, you can increase performance by keeping file size to the minimum and remove components not used.

The Adobe OSMF framework consists of the following building blocks:

MediaPlayer - MediaPlayer class represents the controller class for media playback. You can play any type of media (video, audio, images, SWFs, etc.). Instead of using DisplayObject – use MediaPlayerSprite. To use the the media you can use the following methods: play(), pause(), seek() as well as the following properties: volume, autoRewind, loop. The events for the media are: seekingChange, volumeChange, complete

MediaElements & Traits – MediaElement class represents a unified media presentation (video, image, or a grouping of media that’s shown together). It takes a resource (URL, array of dynamic streams, etc.). You can than presents/plays the media using one or more MediaTraitBase. MediaTraitBase represents an intrinsic capability of a piece of media (ability to play, ability to seek, audio, etc.). This class is dynamic in nature, can come and go over the life of the media Trait APIs, it is also media-type-agnostic. Keep in mind that not all traits apply to all media types. For instace AudioElement doesn’t have DisplayObjectTrait and ImageElement doesn’t have PlayTrait.

Rules of thumb - A trait represents a media capability or characteristic. Trait cannot apply to every piece of media and must be something that a player developer might act upon. For instance: LoadTrait, PlayTrait, SeekTrait, DisplayObjectTrait.

As an example, take a look at the VideoElement traits in Figure:

110

Composite Elements – ParallelElement class represents a set of MediaElements that are presented in parallel. It is a media composition whose elements are presented in parallel. SerialElement class represents a set of MediaElements that are presented one after the other. These classes used together create two composite MediaElements that can represent complex, “tree-like” media experiences.

To better understand how it works take a look at Figure:

22

We have three video related MediaElements: (Episode), (Mid-Roll Ad) and (Episode, Continued). Together they create a media experience and are grouped as serialElement. Once you create the serialElement you can add that group to a parallelElement with other serialElement.

Composite Elements - A composite MediaElement is a MediaElement that exposes composite traits. Composite traits aggregate multiple traits of the same type. For instance, you can take two VideoElement and create a SerialElement and than have access to both playTrait properties from each VideoElement. Together these playTraits are CompositePlayTrait.

Proxy Elements – ProxyElement class wraps up (proxies) with another MediaElement. These class expose the same API. The class signature is as follow:


public function ProxyElement(wrappedElement:MediaElement)  

By default, all methods, properties and events are passed through subclasses and can change. This can be used to modify the behavior of another MediaElement. Clients think they’re working with a VideoElement, when they’re actually working with a ProxyElement that wraps a VideoElement. This is incredibly useful for plugins.

Here are two examples of ProxyElement:

  • User Analytics – ProxyElement, which listens for changes to the wrapped MediaElement and reports them to a server.
  • Seamless Video Switching – ProxyElement, which wraps up two VideoElements, and switches from one to the other without rebuffering.

Take a look at the example in Figure below. Here we created a AnalyticsProxyElement element. The proxy listens to events from wrapped MediaElement and can reports the data back to Omniture API. The AnalyticsProxyElement does that by wrapping an UnseekableProxyElement that prevents seeking of the wrapped MediaElement.

32

Plugins – The real challenge is the third party integration. There are many 3rd party vendors that make a player such as CDNs, Ad Servers/Networks, Analytics Providers, “Social” Providers and many others.

To allow integration between the Player and the vendors OSMF created Player/Plugin contract, which declare what the plugin capabilities are.

Plugins don’t have free rein over the player. Meaning that the Players can load plugins, but Players don’t integrate with custom plugin APIs directly. The way it works is that OSMF acts as the broker between the player and the plugin.

There are ranges of different plugin types:

  • Media Plugins - declares new media types. For examples: Video plugin, image plugin
  • Loader Plugins - declare new ways of “loading” media. For example: Akamai plugin for connection authentication.
  • Proxy Plugins - declare ways to modify (or listen to) the behavior of media. For example: Plugin, which prevents seeking of all videos, Omniture reporting plugin.
  • R

  • eference Plugins - Declare the range of media they’d like to reference - example: Plugin, which creates an overlay ad SWF that can pause the main video when displayed.

For info about creating plugins, check ASDOC PluginInfo:
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/org/osmf/media/PluginInfo.html

Metadata – There are two types of metadata: Resource-level Metadata and MediaElement Metadata.

Resource-level Metadata was created to address the challenge of how do I know what “http://example.com/myvideo” represents. MediaResourceBase.addMetadataValue method can be used to further qualify a resource with “static” data (e.g. MIME type). For instance, let’s say we want to assign the “video/x-flv” MIME type to the resource with a URL so we can know that it’s a video.

MediaElement Metadata was created to address the challenge of representing custom information about what’s playing. MediaElement.addMetadata method used to model metadata during playback.
For instance, let’s say that we have a dynamically generate “ad” vs. “episode” metadata, we want to be able to know that so we can have the UI update the chrome during ad breaks.

The OSMF Code is available here: www.osmf.org
To read more see the resources listed in this page: http://forums.adobe.com/message/2392184#2392184

Hello World example

To create a simple Hello Wrold visit the OSMF wesite:
http://blogs.adobe.com/osmf/2009/09/building_a_helloworld_app_with_osmf.html

Below is a minimalistic example of loading an image using Flex 4. We create sprite that contains a MediaPlayer to manage the display and control of the MediaElements we will be using. We then create and set the MediaElement (in our case ImageElement) with a resource and path. Lastly, we add the sprite to the UIComponent.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.ImageElement;
			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.media.URLResource;

			//path of media to be displayed: Image
			private static const MEDIA_PATH:String = "http://mediapm.edgesuite.net/strobe/content/test/train.jpg";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//creates and sets the MediaElement (ImageElement) with a resource and path
				playerSprite.media = new ImageElement( new URLResource( MEDIA_PATH ) );

				//Adds the sprite to the UIComponent defined in MXML
				mediaHolder.addChild( playerSprite );
			}

		]]>
	</fx:Script>

	<mx:UIComponent id="mediaHolder" />

</s:Application>

Below is a minimalistic example of creating a progressive download video player using OSMF and Flex 4. Just as in the image example we create a sprite that contains a MediaPlayer to manage the display and control of the MediaElements we will be using. We then create and set the MediaElement (in our case VideoElement) with a resource and path. Lastly, we add the sprite to the UIComponent.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="955" minHeight="600"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import mx.core.UIComponent;

			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.media.URLResource;

			//path of media to be displayed: Progressive Video
			private static const MEDIA_PATH:String = "http://mediapm.edgesuite.net/strobe/content/test/AFaerysTale_sylviaApostol_640_500_short.flv";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//creates and sets the MediaElement (VideoElement) with a resource and path
				playerSprite.media = new VideoElement( new URLResource( MEDIA_PATH ) );

				//Adds the sprite to the UIComponent defined in MXML
				var component:UIComponent = new UIComponent();
				component.addChild( playerSprite );
				mediaHolder.addElement( component );
			}

		]]>
	</fx:Script>

	<s:Group id="mediaHolder" />		   

</s:Application>

In case we want to serve an audio file via progressive download, just use AudioElement instead of VideoElement.

Here is a minimalistic example of streaming a video using OSMF and Flex 4. The URL points to a streaming server and the OVP player will be able to recognize that and provide streaming instead of progressive download.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.ImageElement;
			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.media.URLResource;

			//URI of connection/media to be displayed: RTMP - Streaming Video
			private static const MEDIA_PATH:String = "rtmp://cp67126.edgefcs.net/ondemand/mediapm/strobe/content/test/SpaceAloneHD_sounas_640_500_short";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//creates and sets the MediaElement (VideoElement) with a resource and path
				playerSprite.media = new VideoElement( new URLResource( MEDIA_PATH ) );

				//Adds the sprite to the UIComponent defined in MXML
				mediaHolder.addChild( playerSprite );
			}

		]]>
	</fx:Script>

	<mx:UIComponent id="mediaHolder" />

</s:Application>

Here is an example of dynamic streaming using FMS server. We set the host URL for the steaming server and than we can add the profile files for the videos and the dynamic switching will be handles automatically by the OVP player based on the user’s bandwidth.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.net.DynamicStreamingItem;
			import org.osmf.net.DynamicStreamingResource;

			//URI of host RTMP/E connection for streaming server
			private static const HOST:String = "rtmp://cp67126.edgefcs.net/ondemand";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//Resource containing the pointers, bitrate, width, and height of each DynamicStreamingItem to be in the set
				var dsr:DynamicStreamingResource = new DynamicStreamingResource( HOST );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_768x428_24.0fps_408kbps.mp4", 408, 768, 428) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_768x428_24.0fps_608kbps.mp4", 608, 768, 428) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1024x522_24.0fps_908kbps.mp4", 908, 1024, 522) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1024x522_24.0fps_1308kbps.mp4", 1308, 1024, 522) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1280x720_24.0fps_1708kbps.mp4", 1708, 1280, 720) );

				//Creates the MediaElement (VideoElement) adding the DynamicStreamingResource to it, and setting to as the mediaElement on the MediaPlayerSprite
				playerSprite.media = new VideoElement( dsr );

				//Adds the sprite to the UIComponent defined in MXML
				mediaHolder.addChild( playerSprite );
			}

		]]>
	</fx:Script>

	<mx:UIComponent id="mediaHolder" />

</s:Application>

You can download these examples from here:
http://www.elromdesign.com/blog/Flex/OSMF/OSMFExample.fxp

Last example (more advanced) shows you how to create a streaming video player and listen to events to recognize once the video is ready to play, as well as access properties in the video player and the net stream:


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.VideoElement;
			import org.osmf.events.DisplayObjectEvent;
			import org.osmf.events.MediaPlayerCapabilityChangeEvent;
			import org.osmf.media.MediaPlayer;
			import org.osmf.media.URLResource;

			private var playerContainer:Sprite = new Sprite;
			private var mediaPlayer:MediaPlayer = new MediaPlayer();

			private function playVideoURL(url:String):void
			{
				mediaPlayer.addEventListener( MediaPlayerCapabilityChangeEvent.CAN_PLAY_CHANGE, onVideoLoadedAndReady );
				mediaPlayer.addEventListener( DisplayObjectEvent.MEDIA_SIZE_CHANGE, onDimensionChange );

				var videoElement:VideoElement = new VideoElement(  new URLResource( url ) );

				mediaPlayer.media = videoElement;
				mediaHolder.addChild( playerContainer );
			}

			private function onVideoLoadedAndReady(event:MediaPlayerCapabilityChangeEvent):void
			{
				if (event.enabled  && mediaPlayer.canPlay)
					mediaPlayer.play()
			}

			private function onDimensionChange( event:DisplayObjectEvent ):void
			{
				mediaPlayer.displayObject.width = event.newWidth;
				mediaPlayer.displayObject.height = event.newHeight;

				mediaHolder.addChild( mediaPlayer.displayObject );
			}

		]]>
	</fx:Script>

	<s:Rect top="0" width="660" height="365"
			x="0" y="0">
		<s:fill>
			<s:SolidColor color="#000000"/>
		</s:fill>
	</s:Rect>	

	<mx:UIComponent id="mediaHolder" />

	<s:TextInput id="mediaPath" width="400"  x="48" y="330"
				 text="rtmp://cp67126.edgefcs.net/ondemand/mediapm/strobe/content/test/SpaceAloneHD_sounas_640_500_short"/>
	<s:Button x="456" y="331" label="Play" click="playVideoURL(mediaPath.text)"/>
	<s:Button x="534" y="331" label="Pause" click="mediaPlayer.pause()"/>

</s:Application> 

Where to go from here:

The toolkit and documentation is available for download at:
www.OpenSourceMediaFramework.com

OSMF developer forums:
http://forums.adobe.com/community/opensource/osmf/developers

OSMF User Group:
http://groups.adobe.com/groups/7af970e6e4/summary

20
Feb

10 Most Popular Audio & Video file formats & Codec used by FlashPlayer

Video and audio files that are deployed on platforms with constrain and limitations such as Web and Mobile devices need to be compressed from the original format.

The compression is the usage of a codec (enCOder/DECoder) algorithm. Codecs, as their name implies, are programs that compress and decompress data. (There are audio codecs as well as video codecs). Compression has two purposes: reduced file size to speed up transmission as well as reduces the needed data storage space on the destination device.
Most codecs are lossy, which means some of the data is lost during compression and cannot be recovered during decompression. Lossless codecs preserve all original data, and are therefore a 100% faithful transcription of the original data set when uncompressed. Lossless compression is not well suited to the web or mobile devices.

A codec can either be a physical device or a software based process that, on the encoding side, manages the compression of raw digital audio or video data into files of reduced size, optimizing both download and playback performance. On the decoding side, the process is reversed, with the codec uncompressing the file to produce a high quality facsimile of the original content. Because the object of compression is to reduce the overall file size and streaming bandwidth requirements for a given segment of video, it is typically necessary or desirable to “throw away” some of the data during the compression process, meaning that when the codec reproduces content, it will have incrementally lower production values than the original. Compression, which discards some data in the compression and optimization process, is called lossy data compression.

At this point, we depart from the domain of science, and cross over into craft and art, because to create an acceptable result, compression algorithms must strike a complex balance between the visual quality of video and the volume of data necessary to render it. For purposes of multimedia content, the key measure of codec performance is the bit rate. In the context of transmitting multimedia data over the Internet or mobile carrier connections, bit rate quantifies the number of bits required per increment of playback time in order for the viewer to see smooth, uninterrupted content. For streaming video, this degree of playback quality is also called goodput. Goodput is the effective transmission rate supporting what the user actually sees on their device - in other words, it is the amount of data transferred after deducting things like internet, network and datalink layer protocol overhead; network congestion; and retransmission of data that was corrupted or lost in transit. The ability to empirically measure the performance of various codecs is key, because they have different strengths, and therefore, different applications.

Essentially, codecs are optimization tools, and they are many and diverse, often with thriving application genres based on them. The choice of a particular codec is driven by what rendering or transmission characteristics are the focus of optimization; what codecs a developer can reasonably assume to be present on the target platforms; and what post processing tools the developer has available for converting raw data into a video file format. It’s unsurprising that there is a great deal of competition among the developers of codec technology, because achieving a big advance in compression without a loss of quality would have tremendous commercial value. But, on the other hand, if all codec technologies were secret, there would be crippling fragmentation resulting from dozens of incompatible proprietary file formats for encoded video. This problem is neatly solved by an extensive, widely embraced standards-making process for video encoding.

Video codec designs are precisely specified by the Motion Picture Experts Group (MPEG), an international body, which includes 350 members representing media industries, universities, and research institutions. MPEG is chartered by the International Standards Organization (ISO) and is tasked with publishing standards documents that detail how various codecs work. What‘s interesting about this is that MPEG’s published specifications assume that the compression of video files is asymmetrical. In this sense, asymmetrical means that is it’s far more complex and difficult to compress data than to decompress it. As a standards making group, MPEG is exclusively interested in creating a framework for interoperability among various vendors’ codecs and products. This effectively means that only the decoding process needs to be enshrined in a public standard. The encoding process is not constrained by a published MPEG standard. As long as the compressed video files can be decoded as described in the MPEG spec, innovators are encouraged to design new and better encoders, achieving advances in optimizations, while secure in the knowledge they’ll reap the accompanying economic benefits. As encoder technology moves forward, the deployed decoder technology will continue to work, because the decoder side has no knowledge of encoder implementation and can’t be broken by encoder evolution.

Since there is a great deal at stake, the exact strategies of popular encoder designs are usually not public, but the nature of general recent advances is an open secret. Most codecs have transitioned from logic which compresses video data frame-by-frame to an object based model, where the encoder detects regions of frames that don’t change rapidly and caches those semi static portions. This is a tremendous advantage for bandwidth constrained scenarios like mobile video, because it prevents transmission of redundant data.

Both transmission speed and quality of the video rendering produced by decoding the results of various encoders can differ dramatically from one encoder implementation to another. In addition, there can be significant trade-offs in video codecs’ decoder runtime performance and resource utilization. It’s a subtle point, but an important one: Codec standards enable interoperability, but they do not imply uniformity of performance or quality across mobile devices. This potentially complicates life for content designers and developers, because it is necessary to know what codec is going to play your content back in order to ensure that video files provide acceptable playback performance. On desktop and laptop computers, there are frequently a variety of codecs available, and the presence or absence of a single one is rarely an issue for content developers. In any case, a desktop video app can request the user to download a needed codec if it isn’t already present. Not so with mobile devices.
Let’s go over some of the most popular audio and video formats.

1: FLV Format

FLV is the most popular video format available on the Internet with some of the best websites engaging their viewers with Flash based videos. This Flash Video format is also available Flash Player 6 as well as available on mobile phones from Flash Lite 3 player.

An FLV file encodes synchronized audio and video streams. The audio and video data within FLV files are encoded in the same way as audio and video within SWF files. SWF files published for Flash Player 6, Flash Player can exchange audio, video, and data over RTMP connections with Adobe Flash Media Server as well.

It is estimated that a one-minute video consumes 2 - 3MB of RAM, while a five-minute video consumes an average of 3 - 4MB. Longer videos play without requiring a linear increase in memory. This is true for progressive, streaming, local, and/or remote.

2: F4V And F4P Format

F4V is associated with FLV and many times you will see them attached together as the same format. What is F4V & F4P is simply the Adobe’s wrapper for the H.264 video. The reason there is even a need for a wrapper is to overcome the limitations that the H.264 format doesn’t support features such as alpha channel or cue points. F4V is available from Flash Player 9.0.r115 and higher. The format maintains dimensions and frame rate of source. The format also eliminates black borders. F4P is the protected video format.

FLV and F4V have open specification:
http://www.adobe.com/devnet/flv/pdf/video_file_format_spec_v10.pdf

3: MPEG-4 Format

MPEG-4 Part 14 also known as MP4 is a collection of audio and video encoding. MPEG-4 is considered the standard as many software companies such as Apple and Microsoft support the format. MPEG-4 is a container that allows you to combine audio and video (as well as other streams) into a single file. MPEG-4 video codec and H.264 are the included standards for video coding and compression. H.264 is the evolutionary step and improvement, providing improved capacity in terms of quality and efficiency. The format is available from Flash Player 9 and above.

4: H.264 Format

H.264 is the next-generation video compression technology in the MPEG-4 standard, also known as MPEG-4 Part 10. H.264 delivers excellent video quality across the entire bandwidth spectrum — right from 3G to High Definition Video Players. This format is preferred because it produces an incredible quality of video with the smallest amount of video data. This means you see crisp, clear video in much smaller files, saving you bandwidth and storage costs over previous generations of video codecs. The format is available from Flash Player 9 as well as Flash Lite 3.1.

5: MP3 format

Part of MPEG-1 standard, and also known as MPEG-1 Audio Layer 3. MP3 is a patented digital audio encoding format. The format is a popular audio format and the standard for lossy data compression of digital audio files.

Flash Player 6.0r40 and later are supporting MP3 and in fact audio in Flash Video files usually encoded as MP3. MP3 also supports ID3 metadata container, which allow passing data about the music file.

6: Advanced Audio Coding (AAC)

Supported by Flash Player 9 Update 3 and later Advanced Audio Coding (AAC) was designed to be the successor of the MP3 format. AAC is high-efficiency (HE) and high-fidelity (HiFi), low-bandwidth audio codec and a standardized, lossy compression data and encoding for digital audio. AAC is a higher quality format than MP3 and generally achieves better sound quality than MP3 at similar bit rates. The format is often packaged in a video format container.

7: MOV format

Available from Flash Player 9 Update 3 and up you can play MOV container using MPEG-4 codecs, it mostly interchangeable in a QuickTime-only environment. This is especially true on hardware devices, such as the Sony PSP and various DVD players; on the software side, most DirectShow.

8: 3GP and 3GPP Format

3GP is a simplified version of the MPEG-4 format; it is designed for mobile use. 3GP is based on MPEG-4 and H.263 video, and AAC or AMR audio The format is design to optimize the video content for mobile and specifically built to accommodate low bandwidths and little storage. 3GP format is a popular format within mobile devices and many support the 3GP format. The file extension is either .3gp for GSM-based Phones or .3g2 for CDMA-based Phone.

9: F4A and F4B format

From Flash Player 9 Adobe supports F4A & F4B audio/mp4 audio for Adobe Flash Player. The format is nothing more than MP4 audio file. F4A stands for an audio file while F4B stands for an audio book. The reason the format even exists is bridge and avoid compatibility issues between different platform such as Adobe Flash Player, Quicktime, iPod etc.

10: M4V and M4A

Flash Player 9 Update 3 supports M4V and M4A. While MP4 is the official extension, apple introduced M4V, M4A formats and are the standard file formats for videos and audio for iTune store, iPod and PlayStation portables.

M4V, M4A file formats are identical to MP4 and can be renamed to MP4. M4V stands for video while M4A by audio layer of MP4 movies.

Notice that M4V files contain DRM and the purchasing users info. You can use Requiem (http://undrm.info/remove-DRM-protection/Requiem-freeware-Mac-and-PC-DRM-remover-for-iTunes-files.htm) to remove the DRM.

So why did Apple created the format anyway? The difference in file extension allows to associate the file type with iTune and when you double click the file format you’ll have iTune opens up in case you have iTune installed. M4V are often used for movies, TV episodes, and music videos.

21
Jul

Getting Strarted with Open Source Media Framework (OSMF) formerly Strobe

The Open Source Media Framework (OSMF) formerly known as Strobe goes open source and provides a a framework for media playback of any kind.

To download the code visit:
http://opensource.adobe.com/wiki/display/osmf/Open+Source+Media+Framework

The video player ecosystem is complex. There are other elements that make a video player a complete composition, such as advertisement elements, social network elements, reporting, content management, DRM etc. Adobe recently announced a new framework called Adobe Open Source Media Framework that is aimed at solving these issues.

OSMF is an open source (vanity license) AS3 media framework that supports the workflow around video playback and monetization. Video players have different features set. The skins are different and the integration is different as well as the architecture workflow. But they do essentially the same thing and can be created using the OSMF framework. The framework is based on the quality of the video player (OVP) and addresses the common challenges.
Adobe vanity license means that it’s ok to use and redistribute. Modifications can be submitted to Adobe for redistribution.

The foundation of the framework is Qos (quality of service), which focuses on OVP and provides a quick start for playing videos (smallest buffer size needed to start the video), efficient connection logic, and switching bitrates dynamically (recall metric monitor service in OVP).

The framework by itself is not powerful without having the CDNs and the publishers onboard. Adobe OSMF is based on the “chicken and the egg” theory, meaning which will come first? In fact, Adobe is currently trying to get the publishers and the CDNs onboard and it seem that they got very positive responses. The idea is that each CDN will integrate their plugins to the OSMF framework and the publisher will be able to easily switch CDNs. This type of pluggable component can allow publisher to easily switch as well as test performance and service over different CDNs.
Advantages:

  • Reduce the barrier of entry for new publishers – by offering a framework to integrate the different pieces of the video player, new publishers can get started quickly and with fewer resources and scale up as requirements increase.
  • Provide flexible framework – OSMF provides an easy way to extend each component and allow these components to act as a Lego piece that can be compositable and extensible.
  • Leverage existing code – The OSMF framework uses the Flash Player from Open Video Player (OVP).
  • Drive Standard and allow custom workflows – many of the elements that connect to a video player are not standard yet and Adobe OSMF will help standardize these components as well as allow them to be configured.
  • No runtimes or framework dependency – the framework is based on Flash 10 AS3 and is not depend on any framework such as Flex SDK, Cairngorm or others. With that said some integrated elements may be created using a framework but these are loosely coupled and can be replaced if needed.
  • Partners focus – There are two partners: publishers and CDNs. Each can focused on their expertise. CDN can focus on services and integration and publisher can focus on user experience.
  • Integration with existing Adobe tools – Adobe OSMF will be integrated with other Adobe Suite tools and services such as Catalyst, Illustrator, FMS 3.5 and FMRMS.
  • Optimize – by the ability to separate the core framework and each element as a separate SWC you can increase performance by keeping file size to the minimum and based on components used.

The Adobe OSMF framework consists of three interests, which can be represented as an MVC pattern, as follows:

  • Model (Media elements) - media element (based on IMediaElement) may be video, image, SWF or others. Every media element has defined capabilities (traits based on IMediaTrait) and a life cycle, they are called Regions, for instance Region 1 (Image), Region 2 (Video) etc. Each media element implement IMediaElement contract, which contains the dynamic media traits (capabilities) and you can add or modify a trait. The traits are the building blocks and they can be set once and be reused.
  • Controller (Media compositions) – a media composition (IMediaFactory) is a collection of media elements. Each media can be rendered based on set of rules as well as media region that define how to use the capability and the life cycle.
  • View (Media configurations) – collection of media regions with properties that map to defined capabilities. Each media region can render each media element and can be spatially arranged and the entire media configuration can be the stage.

Let’s take a closer look at the higher architecture. The UI management is the set of tools we use to create our video player and skin it such as Flash Professional, Flex and soon we will be able to use Flash Catalyst for Flash 10 applications. Advertising companies such as DoubleClick or eyewonder allow us to add rich advertisement elements such as pre, mid and post roll ads, in playlist, or as companion ads or overlay ads. Applications provided by sites such as KickApps and gigya allowing embedding of the video player in social networks such as Facebook. The Flash Platform enable playback of video files. Companies such as Level(3) and Akamai provide syndications of feeds that can be consumed by other APIs. Lastly, stream management is available through Adobe’s server.

strobe1

The framework architecture was designed to allow you to integrate at multiple points and support different use cases, it also allow to use only the pieces you need, as well as the ability to scale up when ready. There are three types of implementation you may find useful.

  • Media Framework (level 1 integration) - small in size and include the basic integration such as the video player by OVP and pluggable CDNs. It allows hooking into your higher-level API, tweak the UI and access the needed classes and methods you define such as play and pause.
  • Media Framework and Composition Framework (level 2 integration) - In addition to level 1 level 2 allows creating a composition framework, which includes the ability to integrate to plugins that can provide monetization workflow for ads and tracking.
  • Configuration Framework, Media Framework and Composition Framework (level 3 integration) – in addition to the features that level 2 and level 1 offers, the level 3 integration offers dynamic chrome and syndication.

strobe2

Let’s see the framework in action

Let’s take a look at a simple implementation of the framework provided by Adobe based on the beta version. Keep in mind that the implementation was provided by Adobe and it’s under Adobe agreement; please refer to Adobe in regarding to the license agreement in term of usage.


package com.adobe.strobe.player
{
	import com.adobe.strobe.loaders.*;
	import com.adobe.strobe.media.*;
	import com.adobe.strobe.media.events.*;
	import com.adobe.strobe.traits.*;
	import com.adobe.strobe.traits.events.*;
	import com.adobe.strobe.view.*;

	import flash.events.*;
	import flash.net.*;

	public class MainWindow extends MainWindowLayout
	{

The childrenCreated method will be called automatically by the component once the child objects are created, since it’s replacing the default UIComponent method.
We create the MediaFactory, which holds all the media element using IMediaInfo. We than can call the registerDefaultMedia. And adds event listeners to the buttons. As well as bring back mediaPlayer object which is the component which is capable of playing any type of media by holding a media element that has all the traits we needs such as IPausible, which is a trait to pause and resume playing.


override protected function childrenCreated():void
		{
			super.childrenCreated();

			// Construct a loader factory:
 			factory = new MediaFactory();
			registerDefaultMedia(factory);

			// Add button handlers:
			buttonStop.addEventListener(MouseEvent.CLICK,onStopClick);
			buttonPlay.addEventListener(MouseEvent.CLICK,onPlayClick);
			buttonPause.addEventListener(MouseEvent.CLICK,onPauseClick);
			buttonResume.addEventListener(MouseEvent.CLICK,onResumeClick);
			buttonToggleMute.addEventListener(MouseEvent.CLICK,onToggleMuteClick);
			buttonLoad.addEventListener(MouseEvent.CLICK,onLoadClick);

			// Get a reference to our player:
			player = flexMediaPlayer.mediaPlayer;
			player.scaleMode = ScaleMode.LETTERBOX;
		}

registerDefaultMedia method creates all the media element that our application will be supporting, such as progressive download, streaming, audio files, images and SWFs.


private function registerDefaultMedia(factory:IMediaFactory):void
		{
 			var loader:ILoader = new VideoLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "progressive"
					, loader as IMediaResourceHandler
					, VideoElement
					, [VideoLoader]
					)
				);

 			loader = new RTMPLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "streaming"
					, loader as IMediaResourceHandler
					, VideoElement
					, [RTMPLoader]
					)
				);

			loader = new SoundLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "audio"
					, loader as IMediaResourceHandler
					, SoundElement
					, [SoundLoader]
					)
				);

			loader = new ImageLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "image"
					, loader as IMediaResourceHandler
					, ImageElement
					, [new ImageLoader()]
					)
				);

			loader = new SWFLoader();
			factory.addMediaInfo
				( new MediaInfo
					( "swf"
					, loader as IMediaResourceHandler
					, ImageElement
					, [SWFLoader]
					)
				);
		}

loadMediaByURL method creates new media element based on the URL the user provided add listener and set it in our player component.


		private function loadMediaByURL(url:IURLResource):void
		{
			// Stop listening to the current media, if any.
			toggleMediaListeners(player.media,false);

			// Create the new media.
			var media:IMediaElement = factory.createMediaElement(url);

			// Listen for events related to the new media.
			toggleMediaListeners(media,true);

			// Set it on our media player.
			player.media = media;
		}

toggleMediaListeners method adds the event listeners to the media. Once the state have changed and the on flag is set to true it will call the event handler onMediaStateChange. The media element has a trait for loading the asset and once the onLoadableStateChange is captured the text field will display the information.


		private function toggleMediaListeners(media:IMediaElement,on:Boolean):void
		{
			if (media != null)
			{
				if (on)
				{
					media.addEventListener(MediaStateChangeEvent.STATE_CHANGE,onMediaStateChange);
				}
				else
				{
					media.removeEventListener(MediaStateChangeEvent.STATE_CHANGE,onMediaStateChange);
				}

				var loadable:ILoadable = media.getTrait(ILoadable) as ILoadable;
				if (loadable)
				{
					if (on)
					{
						loadable.addEventListener(LoadableEvent.LOADABLE_STATE_CHANGE,onLoadableStateChange);
					}
					else
					{
						loadable.removeEventListener(LoadableEvent.LOADABLE_STATE_CHANGE,onLoadableStateChange);
					}
				}
			}
		}

The event listener will handle the interaction with the media element based on the traits that got assign to the media.


		private function onStopClick(event:MouseEvent):void
		{
			player.stop();
		}

		private function onPlayClick(event:MouseEvent):void
		{
			player.play();
		}

		private function onLoadClick(event:MouseEvent):void
		{
			var url:String = urlInput.text;
			if (url.length > 0)
			{
				player.unload();
				loadMediaByURL(new URLResource(url));
			}
		}

		private function onLoadableStateChange(event:LoadableEvent):void
		{
			loadState.text = event.newState.toString();
		}

The onMediaStateChange event handler will handle the display and hiding of the toolbar.


		private function onMediaStateChange(event:MediaStateChangeEvent):void
		{
			if (player.media)
			{
				buttonStop.visible 	= buttonPlay.visible 	= player.media.getTrait(IPlayable) != null;
				buttonPause.visible = buttonResume.visible  = player.media.getTrait(IPlayable) != null;
				buttonToggleMute.visible = player.media.getTrait(IAudible) != null;
			}
		}

		private var factory:IMediaFactory;
		private var media:IMediaElement;
		private var player:MediaPlayer;

Constants to hold the different media files we can test.


		private static const REMOTE_PROGRESSIVE:String 	= "http://flipside.corp.adobe.com/pirates3/Pirates3-1.flv";
		private static const REMOTE_MP3:String 			= "http://flipside.corp.adobe.com/brian/strobe/media/Remember.mp3";
		private static const REMOTE_STREAM:String 		= "rtmp://flipside.corp.adobe.com/trailers/EvanAlmighty";
		private static const REMOTE_IMAGE:String 		= "http://www.adobe.com/products/mediaplayer/assets/images/amp_icon.jpg";
		private static const REMOTE_SWF:String 			= "http://flipside.corp.adobe.com/testing/swfPlayback/as3/Objects/Color/ObjClr_setRGB.swf"
	}
}

strobe3