14
Aug

Add Android Development Tools (ADT) plugin to your Eclipse/FlashBuilder for AIR Android development

In case you’re doing AIR Android development you’re familiar with Dalvik debug tool (just run ./ddms) and adb. You get information about the Android applications running and you can view/filter processors. You can view devices connected, terminate applications, filter, screen grab and many other features. What I usually do is filter for “flash” and you can see processors relate to AIR Android or Flash 10.1.

In case you’re doing AIR Android development using FlashBuilder 4.5 (pre-release), I highly recommend adding Android Development Tools (ADT) plugin to your Eclipse/FB. It will replace the need to use Dalvik and adb command line in many cases.

The installation is a bit of a challenge. ADT was built for Eclipse 3.3, however FlashBuilder is using newer version of Eclipse. What you will have to do is first install WST and only then you will be able to install ADT plugin.

To install WST add a remote location: galileo - http://download.eclipse.org/releases/galileo/
Then select: Web, XML, and Java EE Development > WST Server Adapter

Once WST is installed you can install the ADT plugin:
Add remote location:
ADT - https://dl-ssl.google.com/android/eclipse/

Lastly, point to the Android SDK location. Flash Builder > Preferences > Android

screen-shot-2010-08-14-at-50229-pm

And that’s it - you can add devices and logcat views to Eclipse and do many of operations of adb and Dalvik:

screen-shot-2010-08-10-at-84146-am

screen-shot-2010-08-10-at-84116-am

24
Jul

Speaking at the following upcoming Flash events

I am very excited and privileged to be attending and speaking at the following upcoming Flash events: CFUnited, RIAUnleashed, 360|Flex DC and AdobeMAX. It’s always great to network, meet new friends, as well as hang out with old conference buddies. In addition, it’s great to learn from others and see what people are working on these days.

CFUnited

screen-shot-2010-07-24-at-71147-am

CFUnited has been around the block for the past 12 years and this is the first time that I will be speaking at this event. This will be the last CFUnited and an end of an era. Don’t miss the last chance to attend this event!! It is only few days away and I can’t wait to be part of it. In addition to ColdFusion - CFUnited has many sessions about Flex, AIR, Best Practices for Code, UI Development, databases and much more. This is a real opportunity to extend your knowledge and to get a little bit out of your comfort zone. For instance, I am very excited to see some interesting presos such as:

  • Greg Wilson - Building Multi-Player games on the Flash Platform with ColdFusion, Flex and Data Services
  • Dan Wilson - Get the Lead Out - Practical Optimization
  • Kevin Schmidt - Exploring LCDS3 and ColdFusion 9 Applications
  • Greg Wilson - Adding real-time data visualization to your application or website
  • Brian Kotek - Clean Code: Why it matters and how it’s done

The amount of sessions and diversity is awesome!
Kudos to Tara Kossari and the rest of CFUnited organizers for putting this event together.

Dates: July 28 - 31, 2010
Place: Lansdowne resort

My sessions:

  • Thursday 7/29/2010: 1:00 PM 2:00 PM - Elad Elrom - Flex Data Binding Pitfalls 10 common misuse mistakes
  • Friday 7/30/2010 - 10:00 AM - 11:00 AM - 20 Security Threats to Flex Apps and how to avoid them

Follow on twitter: @cfunited
Website: http://cfunited.com/2010/
Buy ticket: http://cfunited.com/2010/prices

360|Flex DC

screen-shot-2010-07-24-at-80054-am

I am very excited that we have 360|Flex on the east coast this year. I have been speaking at 360|Flex for years now and once you attend a 360|Flex you’ll understand the appeal. This year I already spoke at San Jose 360|Flex. You can always find the top people in the industry attending including many of Adobe’s employees speaking and attending the event. The selection of topics is excellent and since the event is three days, you get a great chance to network! 360|Flex was the first Flash event and is still considered by many as the best of all Flex/AIR themed conferences. Early bird is $599 and worth every penny!! Three days of packed events and John Wilker always makes sure there are recordings and presentations available so once you get home — you’ll have a chance to watch again and digest the information or see the sessions you missed! This is also the first 360 event without Tom Ortega organizing - but I am sure we’ll see him around :)

Dates: September 19 - 21
Place: Washington DC

My session: September 20: 1:30 - 2:40 PM - Flex Data Binding Pitfalls: 10 Common Misuse Mistakes

Follow on twitter: @360FLEX
Website: http://www.360flex.com/
Buy ticket: http://www.360flex.com/register/

Adobe MAX 2010

screen-shot-2010-07-24-at-82633-am

What I can say: YOU MUST ATTEND THIS EVENT AT LEAST ONCE! I have been attending Adobe MAX for four years now and it was always worth it. Adobe MAX is the most prestigious Flash event. Thousands of people, over a hundred sessions, four days and a technology show off! You get to attend amazing sessions from top speakers (Adobe’s employees and community experts), network, party, MAX award, celebrities and more. You also get a chance to understand the direction Adobe is taking and attend sessions and forums with the team that created Adobe’s products. This year it will be super exciting with Mobile development being in the front line. I am super excited this year since it will be the first time I am speaking at Adobe MAX. Additionally, it’s a chance to get some more Adobe MAX Toys :)

Dates: October 23-27, 2010
Place: Los Angeles, CA

My session: Monday, October, 25th, 5:00 pm - 6:00 pm: Essential Eclipse Plug-ins and Tools for Flash Builder Developers

Follow on twitter: @AdobeMAX
Website: http://max.adobe.com/
Buy ticket: https://max.adobe.com/register

RIAUnleashed

screen-shot-2010-07-24-at-72704-am

I have attended this event for the first time last year and it was amazing! They had the top speakers in the industry, great topics and the event sold out! I wasn’t surprised that Brian Rinaldi will be doing the event again this year. This year they have an amazing line up and great topics (Check the speaker’s lineup: http://riaunleashed.com/page.cfm/speakers ). RIAUnleashed is offering a great opportunity to learn and network with people in the east coast and many visitors from around the USA and I am very pleased that we have this event on the east coast so you can get there from NYC, Philly, Upstate or anywhere in New England easily. Last year I took the Limoliner and my wife and baby girl Romi joined me so we made a great mini-vacation out of it. Don’t miss this event!

Dates: November 11 - November 12, 2010
Place: Boston

My session: November 11, 2010: 8:30-12:00 - Getting Started with AIR 2
Follow on twitter: @riaunleashed
Website: http://riaunleashed.com/
Buy ticket: http://riaunleashed.com/page.cfm/register

Early bird for the conference is $50 or for the conference and workshops it is $175 - this is a no brainer!

23
Jul

Reduce complexity from the DisplayList and increase performance

One of the easiest approaches to increase performance in Flash app is to remove any hidden objects from the Flash display list, and use the minimum amount of display objects. The code execution goes through the display list from the top layer, the Stage, and through the main Application class. Flash Player executes code and renders all the children in the display list from top to button until it reaches the display object. A container can be anything that extends the DisplayObjectContainer such as UIComponent, Group or a Sprite.

displaylisttree

Especially with MXML, it’s very easy to nest display objects, however it’s VERY expensive every time we nest display objects, since each display object added to the display list is extending a class that extends a class etc. To understand why it is so expensive to nest. Let’s take a look at the display list class hierarchy.

display_list

ActionScript 3

When it comes to the Display list, up to ActionScript 3 all display objects used to be MovieClip. MovieClip class is the core class for animation in Flash Professional and early version of Flash (AS 1 & AS 2). Each movie clip symbol includes behaviors and functionality of display objects, as well as the additional properties for controlling the timeline. There are many cases where you don’t need that extra layer and overhead of the timeline and in fact AS3 introduced a new objects call DisplayObject. DisplayObject is the low level base class for all objects that can be placed on the display list it extends EventDispatcher (which is based on an object). It includes classes to help manage the object such as cacheAsBitmap and visual properties.

Display object containers are types of display objects that can also include child objects that are display objects. The DisplayObjectContainer includes methods to handle managing the children such as addChild, numChildren and removeChild.

There are times where you need to display graphics but don’t need a timeline. That’s when you rather use a Sprite, which extends DisplayObjectContainer. As general rule of thumb — you rather use DisplayObject classes or classes that are subclasses such as Bitmap, Shape or Sprite over a MovieClip.

Flex 3

Next is Flex being thrown into the mix. Flex core class for the display objects is the FlexSprite class, which is nothing more Then a Sprite with the ability to returns a string indicating the location of this object. Then comes UIComponent which extends FlexSprite and is the base class for every visual component in Flex 3. UIComponent includes keyboard and mouse interactions, but also allow disabling these interactions for objects that don’t need that — such as labels. It also includes many features such as accessibility, Styles, States, Invalidation & Validation, Databinding, Effects, Embedding fonts and much much more.

Flex 4

Next is Flex 4. Flex 4 is using a Group as the base container class for visual elements in Spark (just like UIComponent for MX). Then you have a list of component that can be used such as Skin, Graphic, HGroup and VGroup.

In addition to Group you have the SkinnableComponent, which is the super class for all Spark components, it defines the base class for skinnable components. The skins used by a SkinnableComponent class are typically children classes of the Skin class.

For instance, take the Spark Button component as an example. The ButtonBase class is the base class for the all Spark button components. The Button and ToggleButtonBase classes are subclasses of ButtonBase. Similarly, looking at other component such as CheckBox and RadioButton classes there are subclasses of ToggleButtonBase. The Button component then extends the ButtonBase class and uses the default skin, which includes a text label. You can then define a custom skin class to add an image to the control or anything else you need. The default button skin is spark.skins.spark.ButtonSkin, which extends Skin. What we end up in Spark are components that can be easily skin and changed, however they are heavier.

So let’s do some testing. In term of memory usage of display objects:


Shape: 224 bytes
Sprite: 392 bytes
MovieClip: 428 bytes
UIComponent: 1036 bytes
Group: 1200 bytes

Next, let’s test to see the Flash Player code execution time on a frame. I will be using the FrameStats tool I created few days ago. What I have done is created a quick test harness and added the method to the FrameStats tool. It allows you to see information in regards to the code execution time in Milliseconds for a frame or few frames. I will be dropping the framerate to 1 second (so it’s more clear to test and watch the results) and Then pass a method that will be added to display object and add display objects to the display list. Then we can track the frame life cycle in the console.


			protected function creationCompleteHandler():void
			{
				frameStats = new FrameStats(FlexGlobals.topLevelApplication, false );
				frameRateControl = new FrameRateControl( FlexGlobals.topLevelApplication, false, false, 1, 1);

				var componenent:UIComponent = new UIComponent();
				componenent.addChild( frameStats );
				frameStatsHolder.addElement( componenent );

				trace( "Sprite: " + getSize(new Group()) );	

				uicom.addChild( sprite );
				frameStats.testingExecutionTimeOfMethod( methodToTest, 10000 );
			}

			private function methodToTest():void
			{
				sprite.addChild( new Group() );
			}

		]]>
	</fx:Script>

	<s:Group id="frameStatsHolder" />
	<mx:UIComponent id="uicom" />

Display Objects
When adding 10,000 display objects I am getting the following for the main base classes constructor code execution and their childern:


Shape
Constructor code of children executed: 276

Sprite
Constructor code of children executed: 399

UIComponent
Constructor code of children executed: 1078

Group
Constructor code of children executed: 1195

As expected, the execution of the constructor code and each child constructor code takes longer when the class has many nested objects. It good to know that the Flash Player usually turn methods into native code except for the constructor, so the execution time takes longer on constructor than other classes and we should keep constructors with the least amount of code possible.

Adding 1,000 display objects

Button

Although, I added 10,000 display objects SimpleButton it took only 33 milliseconds on a 1 fps Player framerate the reason is that SimpleButton is low level and nest at the same tree level as DisplayObject. Spark Button takes much longer to go through the constructor code than Halo — however, interesting to see that Spark Button uses less execution time to render on the display list.


SimpleButton
Constructor code of children executed: 33
Final user code executed: 730
Player renders changes display list: 304

mx.Button
Constructor code of children executed: 820
Player renders changes display list: 7075

s:Button
Constructor code of children executed: 2392
Player renders changes display list: 4539

Text

Adding text has interesting results as well. Spark Label did very well in term of rendering but still can’t beat Halo components in term of constructor time. TLF did very poorly on rendering (as expected) but decent on the constructor code.


TextField
Constructor code of children executed: 68
Player renders changes display list: 168

mx:Text
Constructor code of children executed: 743
Player renders changes display list: 685

S:Label
Constructor code of children executed: 1136
Player renders changes display list: 198

s:RichText
Constructor code of children executed: 416
Player renders changes display list: 3224

Conclusion

  • Use low level classes such as TextField, SimpleButton (when possible) over halo and Spark. It will require more coding but will improve performance.
  • Avoid using TLF
  • Use Halo components over Spark components.
  • When creating custom components use Sprite over MovieClip and UIComponent over Group
  • When creating vector graphics it’s recommended to use Shape display object.
20
Jul

Performance Monitor tool to watch internal of the Flash Player during runtime

My idea was to be able to easily add a performance monitor tool that will allow you to watch the internal of the Flash Player during Runtime. FrameStats will help you determine whether the player is stretching code execution, rendering or both.

The tool is platform independent and for Pure AS3 project just add the following line of code to your display object container:


this.addChild( new FrameStats(this) );

In Flex project you have to wrap it in a UIComponent since the class is a Sprite:


protected function creationCompleteHandler():void
{
      var componenent:UIComponent = new UIComponent();
       componenent.addChild( frameStats );
       frameStatsHolder.addElement( componenent );
}

<s:Group id="frameStatsHolder" />

screen-shot-2010-07-20-at-81015-am

The tools show the following phases:

  • Constructor Code - constructor code of display object container and it’s subclasses are executed. This is the time leaped between Event.ENTER_FRAME and Event.FRAME_CONSTRUCTED events
  • Frame Actions - code for the display object container and children are being executed. This is the timeline between Event.EXIT_FRAME and Event.FRAME_CONSTRUCTED
  • Final User Code - code that listen to Event.RENDER and last chance to change the display object
  • Display Changes - player renders changes to display list and changes are posted to the surface. This is the time elapsed since the final user code time and the next enter frame event.

elasticracetrace_monitor_tool

Code: http://github.com/EladElrom/Flash-Optimizing-Tools/blob/master/src/com/elad/optimize/memory/FrameStats.as

Explanation

The Flash Player is single threaded and on every frame the player tries to complete the code execution and rendering of the display list. Up to FP 10.1 there was no looseness. Meaning, the Flash Player will try to complete the two tasks of code execution and rendering for every single frame (and not faster than that). In case the Player cannot complete the cycle during that frame it will change the frame rate to accommodate. Less fps means less code to execute and less to render. It also means heavy usage of resources (CPU & memory) and a sluggish user experience.

10.1 is addressing these issues. For instance, 10.1 added the Adaptive player rate, which will skip the rendering phase (not code execution) of some frames to keep the framerate from falling below an acceptable frame rate. 10.1 is able to go to sleep mode when the player detects automatically that the backlight goes into off mode. At that point the framerate drops to 4 fps and rendering is paused. 10.1 also has the hasPriority flag and it will delay loading when the swf is not visible on the screen. Additionally, the FP will automatically pause swf playback when the swf is invisible or at the foreground application.

screen-shot-2010-07-20-at-75322-am

In a nutshell the Flash Player works as follows - first it loads a swf file. The swf consists of two actions: Byte code (being handled by the VM) and Control Tags (such as place objects, remove objects and set object properties).

There are two VM (the old AVM1 used up to FP8 for AS1 & AS2 kept not to break the web) and AVM2 (git works faster - introduced in FP9 to handle AS3 code). These VMs are the two virtual machines that feeds the display list, which ends up rendering and displays on user’s surface. The player is passive and unaware of the what’s on the user’s surface. Every frame cycle is the operation of the player handling the swf tags, action script and rendering operations.

There is really nothing you can do to have all the operations happening at the same time - unless you delegate execution of math calculations to Pixel Bender (see here), which is possible since Pixel Bender works on a separate thread.

In Flash Player 10 Adobe added the ability to track two more events: Event.FRAME_CONSTRUCTED and Event.EXIT_FRAME, which allows this tool to synthesize the framerate better. Let’s go through the cycle:

  • Frame starts - Event.ENTER_FRAME dispatched
  • Constructor code - Constructor code of display object container and it’s subclasses are executed. Event.FRAME_CONSTRUCTED dispatched once this phase is completed.
  • Frame actions executed - code for the display object container and children are executed
  • End of Frame cycle - Event.EXIT_FRAME marks the end of the frame. Dispatched when the display list is about to be updated and rendered. You can set handler to this event and used instead of Event.RENDER when you’re not sure if Stage.Invalidate will be set.
  • Pre-render phase starts - Event.RENDER dispatched when the display is dirty. You can force the event by setting the stage.invalidate();
  • Final user code is executed – for instance, mouse event handler or code to handle Event.RENDER. In case you need to show change few times during a frame you can force the Event.RENDER on Keyboard, Mouse and Timer events using the updateAfterEvent();
  • Player renders changes to display list - Flash Player updates the display list and changes are posted to the surface
  • Cycle completed

screen-shot-2010-07-20-at-13152-am

The tool shows the information on a graph and in a pie, but you can also run the tool in debug mode and watch the synthesize process.


this.addChild( new FrameStats(this, true) );

Running the Player at low 1-2 fps (using the framerate tool), I am getting the following in the console:


Constructor code of children executed: 12
frame actions &amp; children executed: 0
Event.RENDER
Final user code executed: 0
Player renders changes display list: 24
Event.ENTER_FRAME
info :: new framerate was set: 1
Constructor code of children executed: 3
frame actions &amp; children executed: 1
Final user code executed: 0
Player renders changes display list: 234
Event.ENTER_FRAME
Constructor code of children executed: 0
frame actions &amp; children executed: 0
Final user code executed: 702
Player renders changes display list: 208
Event.ENTER_FRAME
-------------- 1 second ---------------
Constructor code of children executed: 2
frame actions &amp; children executed: 1
Final user code executed: 777
Player renders changes display list: 235
Event.ENTER_FRAME
-------------- 2 second ---------------
Constructor code of children executed: 2
frame actions &amp; children executed: 0
Final user code executed: 609
Player renders changes display list: 361
Event.ENTER_FRAME
Constructor code of children executed: 1
frame actions &amp; children executed: 0
Final user code executed: 492
Player renders changes display list: 256
Event.ENTER_FRAME

Here’s the complete implementation code:


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="955" minHeight="600"
			   creationComplete="creationCompleteHandler()">
	<fx:Script>
		<![CDATA[
			import com.elad.optimize.memory.FrameStats;
			import com.elad.optimize.framerate.FrameRateControl;

			import mx.core.FlexGlobals;
			import mx.core.UIComponent;

			private var frameRateControl:FrameRateControl;
			private var frameStats:FrameStats;

			protected function creationCompleteHandler():void
			{
				frameStats = new FrameStats(FlexGlobals.topLevelApplication, true);
				frameRateControl = new FrameRateControl( FlexGlobals.topLevelApplication, true, false, 4, 1);

				var componenent:UIComponent = new UIComponent();
				componenent.addChild( frameStats );
				frameStatsHolder.addElement( componenent );
			}

		]]>
	</fx:Script>

	<s:Group id="frameStatsHolder" />

</s:Application>

You can also run the monitor tool and show the counter of each phase:


this.addChild( new FrameStats(this, true, true) );

Adding another ‘true’ param to the constructor will force the player to invalidate the stage after every mouse move event and UpdateAfterEvent();


this.addChild( new FrameStats(this, true, true, true) );

Download the tool:
http://github.com/EladElrom/Flash-Optimizing-Tools/blob/master/src/com/elad/optimize/memory/FrameStats.as

Keep in mind that this tool was created quickly and I would appreciate any feedback or comment to help improve.

Cheers :)

16
Jul

Easy way to store event listener references & prevent memory leaks

Many times when we find memory leaks in our applications, it’s due to listeners that have been set by us or someone else and never removed from memory. Although it’s recommended to use weak reference when setting event listeners, the fact that we have to put these three extra parameters often cause developers not to set them, which blocks the garbage collector from collecting these objects:


eventDispatcherObject.addEventListener(type,handler,false,0,true);

The solution is simple. All display objects such as Sprite, UIComponent, Spark etc are all extending event dispatcher class and if you set a line break point and look in the debugger you can see that any display object that the event listeners were set are actually stored in an array.

screen-shot-2010-07-15-at-105913-pm

The only problem is that the array is set to private, which prevents us from cleaning an object from it’s listeners.

See EventDispatcher class:


private function get listeners() : Array; 

Yesterday, I placed a CR in JIRA and recommended changing the access modifier in the EventDispatcher class to help prevent memory leaks and increase performance (please vote if you agree):
http://bugs.adobe.com/jira/browse/ASC-4104

I also recommended considering adding a method to EventDispatcher called clearAllListeners(); to allow removing any event listeners that have been created on an object. I am not sure it if makes total sense, since it may add an overhead to any class that extends EventDispatcher. However, having access to listeners() can allow us to remove all listeners, when needed.

With that said, even if Adobe’s SDK team agrees to make the change I highly doubt it if EventDispatcher will end up changing quickly, since it’s part of the core libraries: playerglobal.swc and these are not changing often.

Meanwhile, I created a quick solution to help out with keeping tabs of event listeners added to an object in an easy way.

The idea is to make it simple enough and lightweight so that it will be useful without the need to write any code to create the class or initialize a collection object (such as Array or a Dictionary). Also this class doesn’t have to be used at all times, but be there when you need it and when you have cases where you suspect you’re dealing with a memory leak that is related to an event listener that hasn’t been removed.

Implementation

I will start from the implementation, just to show how easy it is to use. We set an event just like before with one change. We hold a reference to the type and the handler while setting the listener.


movieClip.addEventListener( listeners.type = MouseEvent.CLICK, listeners.handler = onClick );
movieClip.addEventListener( listeners.type = MouseEvent.DOUBLE_CLICK, listeners.handler = onDoubleClick );

As you can see we just set the listeners.type and listeners.handler and it automatically creates a collection that stores references for you, while you add the event listener and then when you want to remove the listeners you just call the following method:


listeners.removeAllListeners( movieClip );

No need to do anything else… that’s it - it’s that simple!

API

Although it’s unconventional I set the class name (listeners) as lower case on purpose, so when you implement it will appear as if it’s part of your class when using the API.


package com.elad.optimize.memory
{
	import flash.events.IEventDispatcher;

	public class listeners
	{
		private static var listenerItem:ListenerItem = null;
		private static var listenerItems:Vector.<ListenerItem>;

		public static function set type( listenerType:String ):void
		{
			listenerItem = new ListenerItem;
			listenerItem.type = listenerType;
		}

		public static function set handler( eventHandlerReference:Function ):void
		{
			if (listenerItem == null)
				return;

			listenerItem.handler = eventHandlerReference;

			if ( listenerItems == null )
				listenerItems = new Vector.<ListenerItem>();

			listenerItems.push( listenerItem );
			listenerItem = null;
		}

		public static function removeAllListeners(eventDispatcherObject:IEventDispatcher, clearListenerItems:Boolean = true ):void
		{
			if ( listenerItems == null )
				return;

			var newListenerItems:Vector.<ListenerItem> = new Vector.<ListenerItem>();

			listenerItems.forEach( function callback(item:ListenerItem, index:int, vector:Vector.<ListenerItem>):void {

				if (eventDispatcherObject.hasEventListener( item.type ) )
				{
					eventDispatcherObject.removeEventListener( item.type, item.handler );

					if ( eventDispatcherObject.willTrigger( item.type ) )
						newListenerItems.push( item );
				}
				else
				{
					newListenerItems.push( item );
				}
			});

			if (clearListenerItems)
				listenerItems = null;
			else
				listenerItems = newListenerItems;
		}
	}
}

class ListenerItem {
	public var type:String;
	public var handler:Function;
}

Each time you set the type a new ListenerItem is created to hold the the type and handler.


		public static function set type( listenerType:String ):void
		{
			listenerItem = new ListenerItem;
			listenerItem.type = listenerType;
		}

Once both type and handler are set we can add them to the listenerItems collection.


		public static function set handler( eventHandlerReference:Function ):void
		{
			if (listenerItem == null)
				return;

			listenerItem.handler = eventHandlerReference;

			if ( listenerItems == null )
				listenerItems = new Vector.<ListenerItem>();

			listenerItems.push( listenerItem );
			listenerItem = null;
		}

When you’re ready to remove all the listeners you can use removeAllListeners method. There may be cases where you want to store the events added on more than one object so when it’s time to clean you can use clearListenerItems set to false, so it will keep the collection of handlers and we will be able to clean other objects.


		public static function removeAllListeners(eventDispatcherObject:IEventDispatcher, clearListenerItems:Boolean = true ):void
		{
			if ( listenerItems == null )
				return;

			var newListenerItems:Vector.<ListenerItem> = new Vector.<ListenerItem>();

			listenerItems.forEach( function callback(item:ListenerItem, index:int, vector:Vector.<ListenerItem>):void {

				if (eventDispatcherObject.hasEventListener( item.type ) )
				{
					eventDispatcherObject.removeEventListener( item.type, item.handler );

					if ( eventDispatcherObject.willTrigger( item.type ) )
						newListenerItems.push( item );
				}
				else
				{
					newListenerItems.push( item );
				}
			});

			if (clearListenerItems)
				listenerItems = null;
			else
				listenerItems = newListenerItems;
		}

Example

Using the API can help when you set an anonymous handler. It’s not recommended to set a weak reference to an anonymous handler, since the listener may be removed next time the garbage collector will do a round and it will appear as if our method sometimes is working and sometimes isn’t.

What I usually do is use argument.callee


event.currentTarget.removeEventListener(event.type, arguments.callee);

movieClip.addEventListener( MouseEvent.DOUBLE_CLICK, function(event:*):void {
					event.currentTarget.removeEventListener(event.type, arguments.callee);
				});

This will remove the listener after the first time the handler got called. However, there are cases when you want to use anonymous method and remove the listener later on. Using the API you can do that by just storing a reference to the method, just as we have done before:


movieClip.addEventListener( listeners.type = MouseEvent.DOUBLE_CLICK, listeners.handler = function(event:*):void {
					trace("DOUBLE_CLICK");
				});

Now let’s look at the complete example. We set two movie clips with event listeners and then each button allows us to remove a listener from each one of the movie clips. Check method allows us to check the status of the object and see if that event is still set on that object or not.

screen-shot-2010-07-15-at-105941-pm


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="955" minHeight="600"
			   creationComplete="check()">
	<fx:Script>
		<![CDATA[
			import com.elad.optimize.memory.listeners;

			private var movieClip:MovieClip = new MovieClip();
			private var movieClip2:MovieClip = new MovieClip();

			protected function btn_clickHandler(event:MouseEvent):void
			{
				movieClip.addEventListener( listeners.type = MouseEvent.CLICK, listeners.handler = onClick );
				movieClip2.addEventListener( listeners.type = MouseEvent.CLICK, listeners.handler = onClick );

				movieClip.addEventListener( listeners.type = MouseEvent.DOUBLE_CLICK, listeners.handler = function(event:*):void {
					trace("DOUBLE_CLICK");
				});

				movieClip2.addEventListener( listeners.type = MouseEvent.DOUBLE_CLICK, listeners.handler = function(event:*):void {
					trace("DOUBLE_CLICK");
				});				

				check();
			}			

			private function onClick(event:MouseEvent):void
			{
				trace("onClick");
			}

			private function check():void
			{
				trace("hasEventListener movieClip CLICK: " + movieClip.hasEventListener( MouseEvent.CLICK ) );
				trace("hasEventListener movieClip2 CLICK: " + movieClip2.hasEventListener( MouseEvent.CLICK ) );
				trace("hasEventListener movieClip DOUBLE_CLICK: " + movieClip.hasEventListener( MouseEvent.DOUBLE_CLICK ) );
				trace("hasEventListener movieClip2 DOUBLE_CLICK: " + movieClip2.hasEventListener( MouseEvent.DOUBLE_CLICK ) );
				trace("------------------------------------");
			}

		]]>
	</fx:Script>

	<s:Button id="btn" label="Add event listeners"
			  click="btn_clickHandler(event)"/>
	<s:Button label="remove movieClip listeners" x="158" y="0"
			  click="listeners.removeAllListeners( movieClip, false ); check();" width="171"/>

	<s:Button label="remove movieClip2 listeners" x="158" y="29"
			  click="listeners.removeAllListeners( movieClip2, false ); check();"/>	

</s:Application>

Console results:


application complete:

hasEventListener movieClip CLICK: false
hasEventListener movieClip2 CLICK: false
hasEventListener movieClip DOUBLE_CLICK: false
hasEventListener movieClip2 DOUBLE_CLICK: false
------------------------------------

Once we click to add the event listeners:

hasEventListener movieClip CLICK: true
hasEventListener movieClip2 CLICK: true
hasEventListener movieClip DOUBLE_CLICK: true
hasEventListener movieClip2 DOUBLE_CLICK: true
------------------------------------

Once we clear the movie clip object:

hasEventListener movieClip CLICK: false
hasEventListener movieClip2 CLICK: true
hasEventListener movieClip DOUBLE_CLICK: false
hasEventListener movieClip2 DOUBLE_CLICK: true
------------------------------------

Once we clear the second movie clip object:

hasEventListener movieClip CLICK: false
hasEventListener movieClip2 CLICK: false
hasEventListener movieClip DOUBLE_CLICK: false
hasEventListener movieClip2 DOUBLE_CLICK: false
------------------------------------

Project:
http://github.com/EladElrom/Flash-Optimizing-Tools
http://github.com/EladElrom/Flash-Optimizing-Tools/blob/master/src/com/elad/optimize/memory/listeners.as

Cheers :)

10
Jul

BitmapCaching API - help cache display object on Pure AS3, Flex and AIR projects

It is estimated that usually about 60-70% of the time that Flash is working it is because of rendering operations such as animating or drawing display objects. One of the most effective ways to increase the Flash Player rendering speed is caching the display objects. In fact, Flash allows caching as bitmap since Flash 8. Up until FP 10.1 the Flash Player has used software to render all the vector graphic and animation.

The way it works is that the player converts the content into a bitmap representation and stores the display object with all of it’s children in memory, so you end up with both: the original vector graphic and bitmap image representation.

It allows the player to use the bitmap image instead of to keep updating the display object on each frame when animating. However, in case visual change occurs, it will re-render the bitmap in memory again and again on every frame before displaying, which will tax the user’s machine.

So as you can imagine, working with bitmap caching is a double edge sword. When used correctly, you get to enjoy faster rendering of your movie clip with little updates for every frame, which will result in an increase of the application’s performance and smoothness of your animation. But when used in places where change to the visual occurs, it can often result in large memory footprint and abuse of the CPU. When used in conjunction with mobile devices where you have limited resources, that can be even more significant and noticeable and will result in draining the battery, especially when building games or animations.

A good usage for cacheAsBitmap

cacheAsBitmap should be used where the visual content doesn’t change often or doesn’t change at all. You will see the impact especially where there is a complex vector graphic involved.

Not a good usage

It should not be used when you need your visual appearance to change or expect the browser window to be resized often. For instance, cases when you need to change: alpha, rotate, scale, change width/height, filter, blendMode, opaqueBackground, transform or add more vector graphic or an image. These cases will result in the bitmap getting re-rendered in memory before displayed.

Additionally, the cached content can only be in the size of up to 2880×2880 pixels, since having a large bitmap dump stored in memory will defeat the purpose of increasing performance.

In some cases where it doesn’t makes much sense to cache as bitmap you can cache your display object manually. Additionally, Flash Player 10.1 offers new methods to help with content where the display object needs to change some visual properties often.

BitmapCaching API can be useful when you need to cache bitmap. Not only does it take into account the new features in Flash Player 10.1, it adds utilities that can help for a quick manual caching. The API includes the following static operations:

  • Manually cache as BitmapData
  • Manually cache as Bitmap
  • Cache all children as Bitmap
  • Cache UIComponent
  • Cache as BitmapMatrix

It also includes small utility methods to help you get your content cached and protect innocent bystanders.

Additionally, the API is cross platform and will work with Pure AS3, Flex, AIR as well as different versions of the player including FP 9 or FP 10.1, so it will allow you to use what you need based on the platform and FP version you’re using.

Manual cache as BitmapData or Bitmap

Using the cacheAsBitmap method instructs the Player to cached the content as a bitmap and to render it on screen. In case a change to the visual occurs, then each frame will result in updating the cached copy on memory before updating the screen.

The solution is to use a technique where you rasterized and create different bitmap references using the same BitmapData. Then, when you change the visual of the original BitmapData in memory the Flash Player will avoid redrawing, which will reduce CPU usage while keeping a smooth animation.

Take a look at the API. It allows you to quickly do a manual caching using the following static method manualCacheAsBitmapData.

We create a new matrix and then generate the BitmapData that will be used to create different bitmap objects.


		public static function manualCacheAsBitmapData( source:Sprite, main:Object = null, finalStageQuality:String = StageQuality.LOW ):BitmapData
		{
			var sourceRectangle:Rectangle = source.getBounds(source);
			var matrix:Matrix = new Matrix(); 

			matrix.translate( -sourceRectangle.x, -sourceRectangle.y ); 

			var bitmapData:BitmapData = new BitmapData( source.width + 1, source.height + 1, true, 0); 

			if ( main != null )
				main.stage.quality = StageQuality.HIGH;

			bitmapData.draw( source, matrix );

			if ( main != null )
				main.stage.quality = finalStageQuality;

			return bitmapData;
		}

Notice that I am using StageQuality.HIGH and then StageQuality.LOW. It can help the vector graphic appearance, since as the BitmapData is created we care about anti-aliasing and we want our bitmaps to be smoothed while the player creates the BitmapData. Then we can switch to StageQuality.LOW, since at that point we don’t want to tax the user’s CPU at all times. Once we are back at LOW our graphics won’t care that much about anti-aliased or bitmaps smoothed and we usually care about keeping performance and CPU usage low. Keep in mind that StageQuality.LOW is not available in AIR and the stage is always in HIGH or BEST quality.

This technique wont work well for the out of the box cacheAsBitmap, since the frame rate will get reduced as the player will redraw all the display objects that are set as cacheAsBitmap true.

You can then use manualCacheAsBitmap to wrap the display object as Bitmap when needed.


public static function manualCacheAsBitmap( source:Sprite, main:Object = null ):Bitmap
{
	return new Bitmap( manualCacheAsBitmapData( source, main ) );
}

Before I show you the implementation example, take a look at these two handy utilities that can help you create a MovieClip based on Sprite and create a Bitmap clip. You pass the graphic source (which is the sprite with the vector graphic or bitmap representation) and you pass a method that will set the event handlers (in case they are needed). These methods will come handy and will help us create the implementation examples. In case you use Flash Professional and the MovieClip is created, you wont need these methods.


		public static function createMovieClipBasedOnSprite(spriteSource:Sprite,
										isCacheAsBitmap:Boolean = false,
										setClipEventHandlers:Function = null):MovieClip
		{
			var movieClip:MovieClip = new MovieClip();
			var source:Sprite = spriteSource;

			movieClip.addChild( source );
			movieClip.cacheAsBitmap = isCacheAsBitmap;

			if (setClipEventHandlers != null)
				setClipEventHandlers( movieClip );

			return movieClip;
		}

		public static function createBitmapClip(bitmapData:BitmapData,
										setClipEventHandlers:Function = null):Bitmap
		{
			var movieClip:Bitmap = new Bitmap(bitmapData);
			movieClip.smoothing = true;

			if (setClipEventHandlers != null)
				setClipEventHandlers( movieClip );

			return movieClip;
		}

Now for the implementation. I will be creating 500 display objects that will move and change alpha, one using the built in bitmap caching mechanism and the other using the manual caching.


			private static const NUM_OF_DISPLAY_OBJECTS:int = 100; 

			private function bitmapCache():void
			{
				var destination:Sprite = new Sprite();
				var uicom:UIComponent = new UIComponent();
				uicom.addChild(destination);
				group.addElement( uicom );

				var movieClip:MovieClip;
				for (var i:int = 0; i < NUM_OF_DISPLAY_OBJECTS; i++ )
				{
					movieClip = BitmapCaching.createMovieClipBasedOnSprite( spriteSource, true, setClipEventHandlers);
					destination.addChild( movieClip );
				}
			}

			protected function manualCache():void
			{
				var source:Sprite = spriteSource;
				var destination:Sprite = new Sprite();

				var uicom:UIComponent = new UIComponent();
				uicom.addChild(destination);
				group.addElement( uicom );

				var bitmap:Bitmap;
				var bitmapData:BitmapData = BitmapCaching.manualCacheAsBitmapData( source, FlexGlobals.topLevelApplication );

				for (var i:int = 0; i < NUM_OF_DISPLAY_OBJECTS; i++)
				{
					bitmap = BitmapCaching.createBitmapClip( bitmapData, setClipEventHandlers );
					destination.addChild( bitmap );
				}
			}

			private function get spriteSource():Sprite
			{
				var source:Sprite = new Sprite();
				source.graphics.beginFill(0xFF88CC);
				source.graphics.drawRect(0, 0, 80, 80);

				source.filters = [new DropShadowFilter()];	

				return source;
			}

			private function setClipEventHandlers(clip:*):void
			{
				var newX:int;
				var newY:int;	

				clip.addEventListener( Event.ADDED_TO_STAGE, function( event:Event ):void {

					newX = Math.random() * ( stage.stageWidth - ( clip.width / 2 ) );
					newY = Math.random() * ( stage.stageHeight - (clip.height / 2 ) );

					clip.addEventListener( Event.ENTER_FRAME, function( event:Event ):void {

						clip.x -= ( clip.x - newX ) * 0.25;
						clip.y -= ( clip.y - newY ) * 0.25;
						clip.alpha = Math.random();

					});
				});
			}

When you run the example and switch you’ll notice that the difference is night and day. As you can see from the screenshots below, using the built-in Bitmap caching produced a jagged animation while the manual one runs smooth and in much better fps.

Bitmap caching:

screen-shot-2010-07-10-at-120301-pm

Manual caching:

screen-shot-2010-07-10-at-120320-pm

Bitmap caching turn off:

screen-shot-2010-07-10-at-120335-pm

Cache as bitmap matrix

Everything up to here is applicable since Flash Player 8; nothing really new other than a cool API to help you implement quicker. Now let’s fast forward to 2009. As Adobe created the packager for the iPhone it was soon clear that cacheAsBitmap is not enough since it was possible to take into account hardware acceleration (OpenGL ES 1.1 and soon OpenGL ES 2) and move the rendering of the display object from the CPU to the GPU. Using cacheAsBitmap the display matrix is used rather than the matrix itself - that’s where cacheAsBitmapMatrix comes into place.

A display object can be set with the cacheAsBitmapMatrix property to redraw only when the value of cacheAsBitmapMatrix changes. Once you set matrix at the cacheAsBitmapMatrix method, the GPU can use the matrix that created the transformation instead of the display object matrix. That makes a big difference since it allows the GPU to compose and animate even in cases where the object has changed. For instance, you can rotate, scale or use alpha and avoid the expensive process of re-rendering the display object on the CPU. So while you set cacheAsBitmapMatrix, you should set the cacheAsBitmap flag to false to avoid the display object to re-render. Even if you can’t use the GPU this method will speed things up.

Now back to the API. The cacheAsBitmapMatrix will allow you to pass a display object and cache a matrix as BitmapMatrix. You can also set the scale property, which will allow you to scale up or down the display object in case you don’t need a matrix representation in the same size. Note that just like cacheAsBitmap, cacheAsBitmapMatrix limits the content to be not more than 1020×1020 pixels and/or about four million pixels.


		public static function cacheAsBitmapMatrix(displayObject:*,
					scaleX:Number = 1, scaleY:Number = 1,
					isCacheAsBitmapMatrix:Boolean = true,
					isCacheAsBitmap:Boolean = false,
					isDebugMode:Boolean = false):void
		{
			var matrix:Matrix;

			if( displayObject.hasOwnProperty("cacheAsBitmapMatrix") )
			{
				if ( !isCacheAsBitmapMatrix )
				{
					if (isDebugMode)
						trace("remove cacheAsBitmapMatrix");

					displayObject.cacheAsBitmapMatrix = null;
					displayObject.cacheAsBitmap = isCacheAsBitmap;
					return;
				}

				if (scaleX != 1 && scaleY != 1 )
				{
					if (isDebugMode)
						trace("set cacheAsBitmapMatrix and scale");

					matrix = new Matrix()
					matrix.scale(scaleX, scaleY);
				}
				else
				{
					if (isDebugMode)
						trace("set cacheAsBitmapMatrix");

					matrix = displayObject.transform.concatenatedMatrix;
				}

				displayObject.cacheAsBitmapMatrix = matrix;
				displayObject.cacheAsBitmap = isCacheAsBitmap;
			}
			else
			{
				if (isDebugMode)
					trace("warning :: method cacheAsBitmapMatrix is only avaliable in FP10.1");
			}
		}

You will need to set the renderMode to GPU in the application’s descriptor initialWindow tag in order to improve performance and take advantage of GPU (where it’s supported). The GPU rendering engines are based on the device drivers so as Adobe is working with manufactures you should check which device offers support.

screen-shot-2010-07-10-at-111348-am

Here is an example using AIR 2.5:


<s:WindowedApplication xmlns:fx="http://ns.adobe.com/mxml/2009"
					   xmlns:s="library://ns.adobe.com/flex/spark"
					   xmlns:mx="library://ns.adobe.com/flex/mx"
					   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import com.elad.optimize.cache.BitmapCaching;
			import com.elad.optimize.cache.GarbageCollector;

			import flash.filters.DropShadowFilter;

			import mx.core.FlexGlobals;
			import mx.core.UIComponent;

			private static const NUM_OF_DISPLAY_OBJECTS:int = 100; 

			protected function creationCompleteHandler():void
			{
				var statsSprite:Stats = new Stats();
				var componenent:UIComponent = new UIComponent();

				componenent.addChild( statsSprite );
				stats.addElement( componenent );

				bitmapCache();
			}			

			private function bitmapCache():void
			{
				var destination:Sprite = new Sprite();
				var uicom:UIComponent = new UIComponent();
				uicom.addChild(destination);
				group.addElement( uicom );

				var movieClip:MovieClip;
				for (var i:int = 0; i< NUM_OF_DISPLAY_OBJECTS; i++ )
				{
					movieClip = BitmapCaching.createMovieClipBasedOnSprite( spriteSource, true, setClipEventHandlers);
					destination.addChild( movieClip );
				}
			}

			private function get spriteSource():Sprite
			{
				var source:Sprite = new Sprite();
				source.graphics.beginFill(0xFF88CC);
				source.graphics.drawRect(0, 0, 80, 80);

				source.filters = [new DropShadowFilter()];	

				return source;
			}

			private function setClipEventHandlers(clip:*):void
			{
				var newX:int;
				var newY:int;	

				clip.addEventListener( Event.ADDED_TO_STAGE, function( event:Event ):void {

					newX = Math.random() * ( stage.stageWidth - ( clip.width / 2 ) );
					newY = Math.random() * ( stage.stageHeight - (clip.height / 2 ) );

					clip.addEventListener( Event.ENTER_FRAME, function( event:Event ):void {

						clip.x -= ( clip.x - newX ) * 0.25;
						clip.y -= ( clip.y - newY ) * 0.25;
						clip.alpha = Math.random();

					});
				});
			}

		]]>
	</fx:Script>

	<s:Group id="group" />

	<s:Group id="stats" />

	<s:CheckBox id="checkBox" x="109" y="6" label="bitmap matrix cache all"
				click="GarbageCollector.forceGarbageCollector();
				BitmapCaching.setAllChildrenCachingPolicy( group.getElementAt(0), checkBox.selected, checkBox.selected, true );"
				selected="true"/>

</s:WindowedApplication>

screen-shot-2010-07-10-at-112234-am

screen-shot-2010-07-10-at-112246-am

We increased the memory usage from 17 to 23, however you get a better fps and smooth animation.

Set all children caching policy

It’s recommended that when using the cache bitmaps flag, only to use that for short periods of time

Notice that I used the setAllChildrenCachingPolicy in the example above. This method is part of the API and allows you to protect the poor innocent bystanders and remove the caching once you don’t need it any more or to set an entire display object and it’s children. The method iterate throws the class and sets every display object caching policy.


		public static function setAllChildrenCachingPolicy( displayObject:*,
				isCacheAsBitmap:Boolean = true,
				isCacheAsBitmapMatrix:Boolean = false,
				isDebugMode:Boolean = false,
				scaleXBitmapMatrix:Number = 1,
				scaleYBitmapMatrix:Number = 1 ):void
		{
			var len:int = 0;

			var description:XML = describeType(displayObject);
			var componentType:String = description.@name;
			var isSpark:Boolean = false;
			var isSprite:Boolean = false;

			if( componentType == "mx.core::UIComponent" ||
				componentType == "flash.display::Sprite" ||
				componentType == "flash.display::MovieClip")
			{
				isSprite = true;
				len = displayObject.numChildren;
			}
			else if ( componentType == "spark.components::Group" )
			{
				isSpark = true;
				len = displayObject.numElements;
			}

			if (!isSpark && !isSprite)
			{
				if (isDebugMode)
					trace("warning: " + componentType + " is not supported!");

				return;
			}

			if( displayObject.hasOwnProperty("cacheAsBitmap") )
			{
				// Cache as bitmap
				if (componentType == "mx.core::UIComponent")
					cacheUIComponent(displayObject, isCacheAsBitmap, "auto", true, isDebugMode);
				else
					displayObject.cacheAsBitmap = isCacheAsBitmap;

				// Cache as bitmapMatrix
				if ( displayObject.hasOwnProperty("cacheAsBitmapMatrix") )
					cacheAsBitmapMatrix(displayObject, scaleXBitmapMatrix, scaleYBitmapMatrix, isCacheAsBitmapMatrix, isCacheAsBitmap, isDebugMode);

				if (isDebugMode)
				{
					var isBitmapMatrix:Boolean = BitmapCaching.isCacheAsBitmapMatrix( displayObject );
					trace("cache status: " + displayObject.name + ", isCacheAsBitmap: " + isCacheAsBitmap + ", cacheAsBitmap: " +
						displayObject.cacheAsBitmap + ", isCacheAsBitmapMatrix: " + isBitmapMatrix);
				}
			}

			var dispalyObjectItem:*;

			for (var i:int=0; i<len; i++)
			{
				if( isSprite )
					dispalyObjectItem = displayObject.getChildAt(i);
				else if ( isSpark )
					dispalyObjectItem = displayObject.getElementAt(i);

				if( dispalyObjectItem.hasOwnProperty("cacheAsBitmap") )
				{
					setAllChildrenCachingPolicy( dispalyObjectItem,
						isCacheAsBitmap,
						isCacheAsBitmapMatrix,
						isDebugMode,
						scaleXBitmapMatrix,
						scaleYBitmapMatrix  );
				}
				else
				{
					if (isDebugMode)
						trace("warning: " + dispalyObjectItem.name + " doesn't support cacheAsBitmap");
				}
			}
		}

Check the status of all children

In case you want to know the cache policy of each display object, you can use the following method:


		private static function showAllChildrenCacheAsBitmapFlagStatus(displayObject:*):void
		{
			setAllChildrenCachingPolicy( displayObject, false, false, true );
		}

Lastly, here is the same application running on the Android using AIR. Attach the following pure AS 3 class:


package  {

	import flash.display.MovieClip;
	import com.elad.optimize.cache.BitmapCaching;
	import flash.events.Event;
	import flash.display.Sprite;
	import flash.filters.DropShadowFilter;
	import flash.ui.Mouse;
	import flash.events.MouseEvent;
	import com.elad.optimize.cache.GarbageCollector;

	public class Sqr extends MovieClip {

		private static const NUM_OF_DISPLAY_OBJECTS:int = 200;
		private var destination:Sprite;

		public function Sqr() {

			checkBox.addEventListener(MouseEvent.CLICK, function(event:MouseEvent):void {
				GarbageCollector.forceGarbageCollector();
				BitmapCaching.setAllChildrenCachingPolicy( destination, checkBox.selected, checkBox.selected, true );
			});

			bitmapCache();
			this.addChild( new Stats() );
		}

		private function bitmapCache():void
		{
			destination = new Sprite();
			this.addChild(destination);

			var movieClip:MovieClip;

			for (var i:int = 0; i< NUM_OF_DISPLAY_OBJECTS; i++ )
			{
				movieClip = BitmapCaching.createMovieClipBasedOnSprite( spriteSource, true, setClipEventHandlers);
				destination.addChild( movieClip );
			}
		}

		private function setClipEventHandlers(clip:*):void
		{
			var newX:int;
			var newY:int;	

			clip.addEventListener( Event.ADDED_TO_STAGE, function( event:Event ):void {

				newX = Math.random() * ( stage.stageWidth - ( clip.width / 2 ) );
				newY = Math.random() * ( stage.stageHeight - (clip.height / 2 ) );

				clip.addEventListener( Event.ENTER_FRAME, function( event:Event ):void {

					clip.x -= ( clip.x - newX ) * 0.25;
					clip.y -= ( clip.y - newY ) * 0.25;
					clip.alpha = Math.random();

				});
			});
		}	

		private function get spriteSource():Sprite
		{
			var source:Sprite = new Sprite();
			source.graphics.beginFill(0xFF88CC);
			source.graphics.drawRect(0, 0, 80, 80);

			source.filters = [new DropShadowFilter()];	

			return source;
		}
	}
}

device

This API is part of an ‘optimization tools’ project I started (see here). This project will provide tools that can help optimize a Flash application.

SWC:
http://github.com/EladElrom/Flash-Optimizing-Tools/tree/master/swc/

ASDOC:
http://github.com/EladElrom/Flash-Optimizing-Tools/tree/master/asdoc/

Project:
http://github.com/EladElrom/Flash-Optimizing-Tools

Cheers :)

08
Jul

FlashBuilder 4 tip: install ANT prospective in Eclipse version: 3.5.1

Right before FlashBuilder 4 came out I was speaking at 360|Flex SanJose and I told Andrew Shorten that I was unable to install JDT on the FB RC. I need JDT since I wanted to use ANT tasks in FlashBuilder 4. In previous versions of Eclipse I used to do the following:
Eclipse SDK > Uncategoriezed > Eclipse JDT Plug-in development resource. However it didn’t work with the current FB which is based on Eclipse version: 3.5.1.
It appears that there is a way, after spending hours I found a way — see below:

  • In the main menu navigate to: Help > Install New Software…
  • Add the following URL http://download.eclipse.org/releases/galileo/ with the name of galileo
  • In the table of features, expand Web, XML, and Java EE development > Eclipse Java EE developer Tools
  • Finish the wizard and restart Eclipse

screen-shot-2010-07-08-at-121600-pm

At this point you can do: Window > Other views and under Ant you’ll see the prospective.

screen-shot-2010-07-08-at-123305-pm

Cheers,
Elad

04
Jul

API to control player’s frame rate per second on Pure AS3/Flex/AIR apps

One of the most effective and easy ways to optimize your Flash application is controlling the player’s frame rate per second (fps). In fact, FP 10.1, which takes mobility into account, have optimized performance by adjusting the frame rate. FP 10.1 also help manage the SWFs that are loaded and delay loading SWFs based on priority, visibility as well as device memory and CPU resources.

Similarly, one of the biggest complaints for AIR is that apps consume too much CPU & memory and have large runtime sizes. Adobe has addressed some of the complaints and in fact in AIR 2.0 Adobe’s team put effort into increasing optimization and decreasing resources used by the AIR application by providing file improvement with smaller runtime size and less CPU/memory usage.

Many have spoken about the performance issues with AIR in regards to keeping the FPS low and in fact having our application obeying citizens in times where the Flash platform is criticized as being slow and inefficient is important.

I believe that it is partially our responsibility, as developers, to ensure our apps are obeying citizens and don’t consume too many resources.

With that said, for some reason, I haven’t seen an implementation that takes into account the three Flash platform pure AS3/ Flex and AIR and decided to create a small API to accommodate for that.

Consider the following. A user has a browser open with 5-10 tabs and each tab has about 2-3 swfs. Ideally, when you don’t use a swf it will reduce the memory footprint. How about few AIR apps and web browsers all running at the same time?

The idea is to have one utility class to help you control and adjust your fps and take into account the following:

  • Reducing fps when your app is inactive
  • Increase the fps once the app is active again
  • Increase fps while animation is playing to create a more smooth experience and keeping a stack of all the animations being played to know when we can drop the fps.
  • Provide a cross platform API (Pure AS3, Flex, AIR)

In the bigger picture I have started an ‘optimization tools’ project (see here). This project will provide tools that can help optimize a Flash application. FrameRateControl is just the starting point and the idea is to eventually create a manager that will provide helpful tools for a developer all bundled together into one small swc.

API explanation

Let’s look at the API.

Default constructor allows passing the configuration params that will be used in the API. For instance, setting whether it is debug mode and we will have trace statements indicating the fps, flag whether it’s an AIR app, the active, animation and sleep fps and call back method.

Notice that the main application is being passed instead since Flex, AIR and pure AS 3 holds the main application differently.

  • Flex 4 holds the main app here: FlexGlobals.topLevelApplication
  • Pure AS 3 holds the main app as the Sprite so it’s this
  • AIR holds the main app here: this when called from the main window.

public function FrameRateControl( main:*, isDebugMode:Boolean = false,
											isAIR:Boolean = false,
											sleepFramerate:int = 4,
											activeFramerate:int = 25,
											animationFramerate:int = 50,
											sleepModeCallback:Function = null,
											activeModeCallback:Function = null )

In this constructer we set parameters and set events to listen based on the type of app we're running.

<p>[sourcecode language='css']
			this.main = main;
			this.isAIR = isAIR;

			SLEEP_MODE_FRAME_RATE = sleepFramerate;
			ACTIVE_MODE_FRAME_RATE = activeFramerate;
			ANIMATION_FRAME_RATE = animationFramerate;

			this.sleepModeCallback = sleepModeCallback;
			this.activeModeCallback = activeModeCallback;

			this.isDebugMode = isDebugMode;

			if ( isAIR )
			{
				main.addEventListener( Event.DEACTIVATE, sleepModeHandler, false, 0, true );
				main.addEventListener( Event.ACTIVATE, activeModeHandler, false, 0, true );
			}
			else
			{
				main.addEventListener( MouseEvent.MOUSE_OUT, sleepModeHandler, false, 0, true );
			}

The setFrameRate method sets the frame rate on the main window. In case the user is in debug mode we want to displace trace statement in the console to show the change.


		public function setFrameRate(framerate:int):void
		{
			main.stage.frameRate = framerate;

			if ( isDebugMode )
				trace("info :: new framerate was set: " + framerate );
		}

Once the app goes into sleep mode the sleepModeHandler method is called and allows us to handle a few cases. In case this is a Flex/Pure AS app, the sleep mode gets dispatched from mouse event so we want to check whether there is a related object assign. In case there isn’t any related object assign, we know that the user has left the stage.

We then want to set an event to listen to when the user is back. AIR needs the ACTIVATE event constant name, while Flex/AS3 needs the MOUSE_MOVE constant.


		private function sleepModeHandler(event:*):void
		{
			if ( (event as Object).hasOwnProperty("relatedObject")
			 && event.relatedObject != null)
				return;

			if ( isAIR )
			{
				main.removeEventListener( Event.DEACTIVATE, sleepModeHandler );
				main.addEventListener( Event.ACTIVATE, activeModeHandler, false, 0, true );
			}
			else
			{
				main.removeEventListener( MouseEvent.MOUSE_OUT, sleepModeHandler );
				main.addEventListener( MouseEvent.MOUSE_MOVE, activeModeHandler, false, 0, true );
			}

			isSleepMode = true;
			setFrameRate( SLEEP_MODE_FRAME_RATE );

			if ( sleepModeCallback != null )
				sleepModeCallback();
		}

Once the app is in sleep mode it will wait for the event that indicates that the app is active again and will call activeModeHandler. In case this is an AIR app it will wait for the app to go into sleep mode again using Event.DEACTIVATE constant otherwise we will be using the MouseEvent.MOUSE_OUT event constant. We are also going to dispatch the call back function in case it was set.


		private function activeModeHandler(event:*):void
		{
			if ( isAIR )
			{
				main.addEventListener( Event.DEACTIVATE, sleepModeHandler );
				main.removeEventListener( Event.ACTIVATE, activeModeHandler );
			}
			else
			{
				main.addEventListener( MouseEvent.MOUSE_OUT, sleepModeHandler );
				main.removeEventListener( MouseEvent.MOUSE_MOVE, activeModeHandler );
			}

			isSleepMode = false;
			setFrameRate( ACTIVE_MODE_FRAME_RATE );

			if ( activeModeCallback != null )
				activeModeCallback();
		}

Once we are running an animation we may want to run it in a higher fps for a better user experience. The API by default set the fps for animation at 50 fps, but you can specify that when you set the constructer. Notice that we are using a dictionary to hold the key names for easy access.


		public function animate( animationNameKey:String ):void
		{
			animationsNamesMap[animationNameKey] = animationNameKey;

			setFrameRate( ANIMATION_FRAME_RATE );
		}

Once an animation is complete, we will delete the key from the dictionary and in case there isn’t any keys (meaning no more animation to consider) we will set the stage frame rate to active mode.


		public function clearAnimation( animationNameKey:String ):void
		{
			delete animationsNamesMap[animationNameKey];

			for (var key:* in animationsNamesMap)
			{
				return;
			}

			setFrameRate( ACTIVE_MODE_FRAME_RATE );
		}

clearAllAnimationKeys method is used to remove all animation names from the dictionary. So in case we want to start over or ensure the collection of names is empty we can use this method.


		public function clearAllAnimationKeys( useWeakReference:Boolean = true ):void
		{
			animationsNamesMap = new Dictionary( useWeakReference );
		}

The frameRate getter allows us to get the current fps in our application.


		public function get frameRate():int
		{
			return main.stage.frameRate;
		}

Implementation

No let’s take a look at a Flex 4 implementation. Once we have creation complete event we add Mr. doob stats to monitor fms and memory usage.

We set the API passing the main application instance:

frameRateControl = new FrameRateControl( FlexGlobals.topLevelApplication, true, true );


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="955" minHeight="600"
			   creationComplete="creationCompleteHandler()">
	<fx:Script>
		<![CDATA[
			import com.elad.optimize.framerate.FrameRateControl;

			import mx.core.FlexGlobals;
			import mx.core.UIComponent;

			private var frameRateControl:FrameRateControl;

			protected function creationCompleteHandler():void
			{
				var statsSprite:Stats = new Stats();
				var componenent:UIComponent = new UIComponent();

				componenent.addChild( statsSprite );
				stats.addElement( componenent );

				frameRateControl = new FrameRateControl( FlexGlobals.topLevelApplication, true );
			}

		]]>
	</fx:Script>

	<s:Group id="stats" />

I am setting two rotation animations to show you how to use the animation feature. One animation is shorter and one is longer.


	<fx:Declarations>
		<s:Rotate3D id="rotateAnimation" target="{rotateBtn}"
					angleYFrom="0" angleYTo="360"
					duration="10000"
					effectStart="frameRateControl.animate('rotateAnimation')"
					effectEnd="frameRateControl.clearAnimation('rotateAnimation')"/>

		<s:Rotate3D id="rotateAnimation2" target="{rotateBtn2}"
					angleYFrom="0" angleYTo="360"
					duration="3000"
					effectStart="frameRateControl.animate('rotateAnimation2')"
					effectEnd="frameRateControl.clearAnimation('rotateAnimation2')"/>
	</fx:Declarations>

Once you click the long rotate button you can see the fps rate goes up and you can also run the short animation and although the short animation is completed the fps will stay at 50 until the longer animation complete as well.

screen-shot-2010-07-04-at-93958-am


	<s:Button id="rotateBtn" label="long rotate" click="rotateAnimation.play();" width="200" x="40" y="314"/>
	<s:Button id="rotateBtn2" label="rotate" click="rotateAnimation2.play();" width="200" x="40" y="350"/>

</s:Application>

Test the application functionality by leaving the leaving the app and noticing that the fps drops to 4 fps.

To run the app on pure AS3 projects - initialize the API with “this” as the main instance:


var framerateManager:FrameRateOptimizer = new FrameRateOptimizer( this, true );

In AIR app just pass “true” for the third param, which will indicate that it’s an AIR app:


new FrameRateOptimizer( this, true, true );

SWC:
http://github.com/EladElrom/Flash-Optimizing-Tools/tree/master/swc/

ASDOC:
http://github.com/EladElrom/Flash-Optimizing-Tools/tree/master/asdoc/

Project:
http://github.com/EladElrom/Flash-Optimizing-Tools

19
Jun

Adobe Open Source Media Framework (OSMF) - revisit

About a year ago I have posted a blog post on how to get started with OSMF (see here). Since than the framework have gone a long way. OSMF 1.0 was released recently and I have given it a second look.

I want to give credit to Brian Riggs and David Hassoun for helping with the content of this blog entry.

OSMF is an open source AS3 media framework that supports the workflow around video playback and monetization. Video players have different features set. The skins are different, and the integration and architecture workflow are. They essentially do the same thing and can be created using the OSMF framework. The framework is based on the quality of the video player (OVP) and addresses the common challenges.
The foundation of the framework is Qos (quality of service) which focuses on Open Video Player (OVP) and provides a quick start for playing videos (smallest buffer size needed to start the video), efficient connection logic, and switching bitrates dynamically (recall the metric monitor service in OVP).

The framework can be used as follows:

  • Create a component user interface for integration of audio, videos, and images.
  • Plugin architecture for integration of the following: CDN, ad partners, publishers, analytics, social networks, and developers.

The framework by itself is not powerful without having partners embracing the frameworks – e.g. CDNs, publishers, ad analytics, social networks, and developers. This type of pluggable component can allow publishers to easily switch and test performance and service across different services.
Advantages:

  • Reduce the barrier of entry for new publishers – By offering a framework to integrate the different pieces of the video player, new publishers can get started quickly and with fewer resources and can scale up as requirements increase.
  • Provide flexible framework – OSMF provides an easy way to extend each component and allows these components to act as Lego pieces that can be compositable and extensible.
  • Leverage existing code – The OSMF framework uses the Flash Player from Open Video Player (OVP). No need to have duplication of efforts to solve basic problems.
  • Drive Standard and allow custom workflows – Many of the elements that connect to a video player are not standard yet, and Adobe OSMF will help standardize these components and allow them to be configured.
  • No runtimes or framework dependency – The framework is based on Flash 10 AS3 and is not dependant on any framework such as Flex SDK, Cairngorm, or others. With that said, some integrated elements may be created using a framework, but these are loosely coupled and can be replaced if needed.
  • Partners focus – There are two partners: publishers and CDNs. Each can focus on their expertise. CDN can focus on services and integration, and publishers can focus on user experience.
  • Integration with existing Adobe tools – Adobe OSMF will be integrated with other Adobe Suite tools and services such as Catalyst, Illustrator, FMS 3.5 and FMRMS.
  • Optimize – By having the ability to separate the core framework and each element as a separate SWC, you can increase performance by keeping file size to the minimum and remove components not used.

The Adobe OSMF framework consists of the following building blocks:

MediaPlayer - MediaPlayer class represents the controller class for media playback. You can play any type of media (video, audio, images, SWFs, etc.). Instead of using DisplayObject – use MediaPlayerSprite. To use the the media you can use the following methods: play(), pause(), seek() as well as the following properties: volume, autoRewind, loop. The events for the media are: seekingChange, volumeChange, complete

MediaElements & Traits – MediaElement class represents a unified media presentation (video, image, or a grouping of media that’s shown together). It takes a resource (URL, array of dynamic streams, etc.). You can than presents/plays the media using one or more MediaTraitBase. MediaTraitBase represents an intrinsic capability of a piece of media (ability to play, ability to seek, audio, etc.). This class is dynamic in nature, can come and go over the life of the media Trait APIs, it is also media-type-agnostic. Keep in mind that not all traits apply to all media types. For instace AudioElement doesn’t have DisplayObjectTrait and ImageElement doesn’t have PlayTrait.

Rules of thumb - A trait represents a media capability or characteristic. Trait cannot apply to every piece of media and must be something that a player developer might act upon. For instance: LoadTrait, PlayTrait, SeekTrait, DisplayObjectTrait.

As an example, take a look at the VideoElement traits in Figure:

110

Composite Elements – ParallelElement class represents a set of MediaElements that are presented in parallel. It is a media composition whose elements are presented in parallel. SerialElement class represents a set of MediaElements that are presented one after the other. These classes used together create two composite MediaElements that can represent complex, “tree-like” media experiences.

To better understand how it works take a look at Figure:

22

We have three video related MediaElements: (Episode), (Mid-Roll Ad) and (Episode, Continued). Together they create a media experience and are grouped as serialElement. Once you create the serialElement you can add that group to a parallelElement with other serialElement.

Composite Elements - A composite MediaElement is a MediaElement that exposes composite traits. Composite traits aggregate multiple traits of the same type. For instance, you can take two VideoElement and create a SerialElement and than have access to both playTrait properties from each VideoElement. Together these playTraits are CompositePlayTrait.

Proxy Elements – ProxyElement class wraps up (proxies) with another MediaElement. These class expose the same API. The class signature is as follow:


public function ProxyElement(wrappedElement:MediaElement)  

By default, all methods, properties and events are passed through subclasses and can change. This can be used to modify the behavior of another MediaElement. Clients think they’re working with a VideoElement, when they’re actually working with a ProxyElement that wraps a VideoElement. This is incredibly useful for plugins.

Here are two examples of ProxyElement:

  • User Analytics – ProxyElement, which listens for changes to the wrapped MediaElement and reports them to a server.
  • Seamless Video Switching – ProxyElement, which wraps up two VideoElements, and switches from one to the other without rebuffering.

Take a look at the example in Figure below. Here we created a AnalyticsProxyElement element. The proxy listens to events from wrapped MediaElement and can reports the data back to Omniture API. The AnalyticsProxyElement does that by wrapping an UnseekableProxyElement that prevents seeking of the wrapped MediaElement.

32

Plugins – The real challenge is the third party integration. There are many 3rd party vendors that make a player such as CDNs, Ad Servers/Networks, Analytics Providers, “Social” Providers and many others.

To allow integration between the Player and the vendors OSMF created Player/Plugin contract, which declare what the plugin capabilities are.

Plugins don’t have free rein over the player. Meaning that the Players can load plugins, but Players don’t integrate with custom plugin APIs directly. The way it works is that OSMF acts as the broker between the player and the plugin.

There are ranges of different plugin types:

  • Media Plugins - declares new media types. For examples: Video plugin, image plugin
  • Loader Plugins - declare new ways of “loading” media. For example: Akamai plugin for connection authentication.
  • Proxy Plugins - declare ways to modify (or listen to) the behavior of media. For example: Plugin, which prevents seeking of all videos, Omniture reporting plugin.
  • R

  • eference Plugins - Declare the range of media they’d like to reference - example: Plugin, which creates an overlay ad SWF that can pause the main video when displayed.

For info about creating plugins, check ASDOC PluginInfo:
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/org/osmf/media/PluginInfo.html

Metadata – There are two types of metadata: Resource-level Metadata and MediaElement Metadata.

Resource-level Metadata was created to address the challenge of how do I know what “http://example.com/myvideo” represents. MediaResourceBase.addMetadataValue method can be used to further qualify a resource with “static” data (e.g. MIME type). For instance, let’s say we want to assign the “video/x-flv” MIME type to the resource with a URL so we can know that it’s a video.

MediaElement Metadata was created to address the challenge of representing custom information about what’s playing. MediaElement.addMetadata method used to model metadata during playback.
For instance, let’s say that we have a dynamically generate “ad” vs. “episode” metadata, we want to be able to know that so we can have the UI update the chrome during ad breaks.

The OSMF Code is available here: www.osmf.org
To read more see the resources listed in this page: http://forums.adobe.com/message/2392184#2392184

Hello World example

To create a simple Hello Wrold visit the OSMF wesite:
http://blogs.adobe.com/osmf/2009/09/building_a_helloworld_app_with_osmf.html

Below is a minimalistic example of loading an image using Flex 4. We create sprite that contains a MediaPlayer to manage the display and control of the MediaElements we will be using. We then create and set the MediaElement (in our case ImageElement) with a resource and path. Lastly, we add the sprite to the UIComponent.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.ImageElement;
			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.media.URLResource;

			//path of media to be displayed: Image
			private static const MEDIA_PATH:String = "http://mediapm.edgesuite.net/strobe/content/test/train.jpg";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//creates and sets the MediaElement (ImageElement) with a resource and path
				playerSprite.media = new ImageElement( new URLResource( MEDIA_PATH ) );

				//Adds the sprite to the UIComponent defined in MXML
				mediaHolder.addChild( playerSprite );
			}

		]]>
	</fx:Script>

	<mx:UIComponent id="mediaHolder" />

</s:Application>

Below is a minimalistic example of creating a progressive download video player using OSMF and Flex 4. Just as in the image example we create a sprite that contains a MediaPlayer to manage the display and control of the MediaElements we will be using. We then create and set the MediaElement (in our case VideoElement) with a resource and path. Lastly, we add the sprite to the UIComponent.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="955" minHeight="600"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import mx.core.UIComponent;

			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.media.URLResource;

			//path of media to be displayed: Progressive Video
			private static const MEDIA_PATH:String = "http://mediapm.edgesuite.net/strobe/content/test/AFaerysTale_sylviaApostol_640_500_short.flv";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//creates and sets the MediaElement (VideoElement) with a resource and path
				playerSprite.media = new VideoElement( new URLResource( MEDIA_PATH ) );

				//Adds the sprite to the UIComponent defined in MXML
				var component:UIComponent = new UIComponent();
				component.addChild( playerSprite );
				mediaHolder.addElement( component );
			}

		]]>
	</fx:Script>

	<s:Group id="mediaHolder" />		   

</s:Application>

In case we want to serve an audio file via progressive download, just use AudioElement instead of VideoElement.

Here is a minimalistic example of streaming a video using OSMF and Flex 4. The URL points to a streaming server and the OVP player will be able to recognize that and provide streaming instead of progressive download.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.ImageElement;
			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.media.URLResource;

			//URI of connection/media to be displayed: RTMP - Streaming Video
			private static const MEDIA_PATH:String = "rtmp://cp67126.edgefcs.net/ondemand/mediapm/strobe/content/test/SpaceAloneHD_sounas_640_500_short";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//creates and sets the MediaElement (VideoElement) with a resource and path
				playerSprite.media = new VideoElement( new URLResource( MEDIA_PATH ) );

				//Adds the sprite to the UIComponent defined in MXML
				mediaHolder.addChild( playerSprite );
			}

		]]>
	</fx:Script>

	<mx:UIComponent id="mediaHolder" />

</s:Application>

Here is an example of dynamic streaming using FMS server. We set the host URL for the steaming server and than we can add the profile files for the videos and the dynamic switching will be handles automatically by the OVP player based on the user’s bandwidth.


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768"
			   creationComplete="creationCompleteHandler()">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.VideoElement;
			import org.osmf.media.MediaPlayerSprite;
			import org.osmf.net.DynamicStreamingItem;
			import org.osmf.net.DynamicStreamingResource;

			//URI of host RTMP/E connection for streaming server
			private static const HOST:String = "rtmp://cp67126.edgefcs.net/ondemand";

			protected function creationCompleteHandler():void
			{
				//sprite that contains a MediaPlayer to manage display and control of MediaElements
				var playerSprite:MediaPlayerSprite = new MediaPlayerSprite();

				//Resource containing the pointers, bitrate, width, and height of each DynamicStreamingItem to be in the set
				var dsr:DynamicStreamingResource = new DynamicStreamingResource( HOST );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_768x428_24.0fps_408kbps.mp4", 408, 768, 428) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_768x428_24.0fps_608kbps.mp4", 608, 768, 428) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1024x522_24.0fps_908kbps.mp4", 908, 1024, 522) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1024x522_24.0fps_1308kbps.mp4", 1308, 1024, 522) );
				dsr.streamItems.push( new DynamicStreamingItem( "mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1280x720_24.0fps_1708kbps.mp4", 1708, 1280, 720) );

				//Creates the MediaElement (VideoElement) adding the DynamicStreamingResource to it, and setting to as the mediaElement on the MediaPlayerSprite
				playerSprite.media = new VideoElement( dsr );

				//Adds the sprite to the UIComponent defined in MXML
				mediaHolder.addChild( playerSprite );
			}

		]]>
	</fx:Script>

	<mx:UIComponent id="mediaHolder" />

</s:Application>

You can download these examples from here:
http://www.elromdesign.com/blog/Flex/OSMF/OSMFExample.fxp

Last example (more advanced) shows you how to create a streaming video player and listen to events to recognize once the video is ready to play, as well as access properties in the video player and the net stream:


<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
			   xmlns:s="library://ns.adobe.com/flex/spark"
			   xmlns:mx="library://ns.adobe.com/flex/mx"
			   minWidth="1024" minHeight="768">

	<fx:Script>
		<![CDATA[
			import org.osmf.elements.VideoElement;
			import org.osmf.events.DisplayObjectEvent;
			import org.osmf.events.MediaPlayerCapabilityChangeEvent;
			import org.osmf.media.MediaPlayer;
			import org.osmf.media.URLResource;

			private var playerContainer:Sprite = new Sprite;
			private var mediaPlayer:MediaPlayer = new MediaPlayer();

			private function playVideoURL(url:String):void
			{
				mediaPlayer.addEventListener( MediaPlayerCapabilityChangeEvent.CAN_PLAY_CHANGE, onVideoLoadedAndReady );
				mediaPlayer.addEventListener( DisplayObjectEvent.MEDIA_SIZE_CHANGE, onDimensionChange );

				var videoElement:VideoElement = new VideoElement(  new URLResource( url ) );

				mediaPlayer.media = videoElement;
				mediaHolder.addChild( playerContainer );
			}

			private function onVideoLoadedAndReady(event:MediaPlayerCapabilityChangeEvent):void
			{
				if (event.enabled  && mediaPlayer.canPlay)
					mediaPlayer.play()
			}

			private function onDimensionChange( event:DisplayObjectEvent ):void
			{
				mediaPlayer.displayObject.width = event.newWidth;
				mediaPlayer.displayObject.height = event.newHeight;

				mediaHolder.addChild( mediaPlayer.displayObject );
			}

		]]>
	</fx:Script>

	<s:Rect top="0" width="660" height="365"
			x="0" y="0">
		<s:fill>
			<s:SolidColor color="#000000"/>
		</s:fill>
	</s:Rect>	

	<mx:UIComponent id="mediaHolder" />

	<s:TextInput id="mediaPath" width="400"  x="48" y="330"
				 text="rtmp://cp67126.edgefcs.net/ondemand/mediapm/strobe/content/test/SpaceAloneHD_sounas_640_500_short"/>
	<s:Button x="456" y="331" label="Play" click="playVideoURL(mediaPath.text)"/>
	<s:Button x="534" y="331" label="Pause" click="mediaPlayer.pause()"/>

</s:Application> 

Where to go from here:

The toolkit and documentation is available for download at:
www.OpenSourceMediaFramework.com

OSMF developer forums:
http://forums.adobe.com/community/opensource/osmf/developers

OSMF User Group:
http://groups.adobe.com/groups/7af970e6e4/summary

18
Jun

The lost blog post: Getting started with building applications for the iPhone, iPod touch, or iPad using Flash Tools

I have written this article last year and didn’t released it since the information was under NDA and when CS5 came out Apple released iOS SDK which included section 3.3.2 and blocked applications built with Flash and other engines. Since than Apple relaxed section 3.3.2 allowing popular game engines and libraries to publish content, while still blocking Flash.

Although we don’t know if in the future the Apple app store will accept applications built with CS5, currently it’s still possible to create applications for the iPhone platform using Adobe Flash Professional CS5 and I figured I will release this blog post.

  1. Create application for the iPhone and iPad and publish them for usage for devices that are jailbroken
  2. Use the iPhone as another testing platform and publish your application on the AIRAndroid or other devices.
  3. Better understand the process which is similar to AIR Android

In this article I will be focusing on building an application for apple mobile devices using Flash Professional CS5. The way it works is that CS5 includes a Packager for the iPod, iPhone and iPad, which allows compiling the ActionScript bytecode into a native application code.

The applications are distributed on the iTune store as application installer files (.ipa files), just like any other native application. To make that possible a front end Ahead of Time (AOT) compilation is using the Low Level Virtual Machine (LLVM), see http://llvm.org/, compiler to output native ARM assembly code, instead of using the Just in Time (JIT) compilation. See figure 1.

15

Creating your first iPhone application using CS5

  1. Developer certification – The first step is to apply for a developer certification from Apple. You can purchase the certificate from here: http://developer.apple.com/iphone/program/. Keep in mind that this step can take few days. To apply for a commercial certificate I had to fax over my LLC article of incorporations.
  2. Provisioning Profile - Once you complete the registrations process, you need to create and install a Provisioning Profile and an iPhone Development Certificate. You can see instructions here: http://developer.apple.com/iphone/library/documentation/Xcode/Conceptual/iphone_development/128-Managing_Devices/devices.html - //apple_ref/doc/uid/TP40007959-CH4-SW2, under “Obtaining Your Development Certificate”
  3. Upload a CSR file - Login to member center (http://tiny.cc/kjnp3) and upload the CSR file to Apple at the iPhone developer site.
  4. Add certifications, device and app Id - Under select: iPhone Provisioning Portal add: development certificates, add devices, add App IDs and provisioning. See ‘How To’ in development center for more information and all the steps needed to complete these parts.
  5. Convert certificate to P12 file – The last step is converting the developer certificate into a P12 file, needed for CS5: Open Keychain Access. Next, select the private key associated with your iPhone Development Certificate and click: File > Export Items. Place a password and you’re done.

Creating ‘Hello World’ application

You are ready to create your first application. We will create minimalistic code and deploy it on the apple’s devices.

  1. Open Flash Professional CS5 and create a new Flash document: File > New Document > Select iPhone OS, see figure 2:
  2. 21

  3. Add text box - click text tool element and change the text to: Hello World
  4. Compile the application - Select Control > Test Movie > In AIR Debug Launcher (Mobile). Save the application as HelloWorld.fla
  5. Icons - iPhone applications have icons that appear in iTunes and on the iPhone screen. You will need to create three icons with the sizes based on the names:
    You can just create empty icons with one color for this app:

    • 29×29.png
    • 57×57.png
    • 512×512.png

  6. Default image - create an image in the size of 320×480 and name it: Default.png. Place the image in the root directory of the application. This image will be displayed while the application loads on the iPhone.
  7. iPhone application settings & creating ipa file

  8. Compile the application - now we are ready to compile, Choose File > iPhone OS Settings. The iPhone Settings window opens up, see Figure 3:
  9. 31

    General Tab

    The General tab includes the following settings:

  10. Output file - The file name
  11. App name - The name of the application
  12. Version - the version number of your application
  13. Aspect ratio - initial aspect ratio of the app (portrait or landscape).
  14. Full screen - full screen, or displays the iPhone status bar.
  15. Auto orientation - allow the content to be reorient when the iPhone is reoriented.
  16. Rendering - decide how display objects are rendered: CPU, GPU or Auto (not yet implemented)
  17. Included files - Add any supporting files and directories.
  18. 41

    Icon tab

  19. Icons tab – allow you to point to the location of the icons.

51

Application descriptor

The application descriptor file, just like in any AIR app, is an XML file that holds all the application’s properties.

The iPhone application setting in CS5 generates the app descriptor and you can also edit the file manually.

The template descriptor file for AIR generated by Flash CS5 doesn’t include iPhone block at this point, so you have to add it in manually to publish an application for the iPad.

Add the Following to the descriptor file after the block:


<!-- iPhone/iPad -specific capabilities -->
<iPhone>
     <!-- A list of plist key/value pairs to be added to the application Info.plist -->
     <!-- Setting UIDeviceFamily will limit to which device is targeted. 1 is for iPhone/iPod. 2 is for iPad. -->
     <InfoAdditions>
          <![CDATA[
          <key>UIDeviceFamily</key>
          <array>
          <string>1</string>
          <string>2</string>
          </array>
          <key>UIStatusBarStyle</key>
          <string>UIStatusBarStyleBlackOpaque</string>
          <key>UIRequiresPersistentWiFi</key>
          <string>NO</string>
          ]]>
     </InfoAdditions>

</iPhone>

As you can see, in the under the UIDeviceFamily, 1 means iPhone and 2 means iPad.

When you create an iPad app you need to set your FLA document to the iPad side: 768×1004 in order to fill the whole screen.

Currently CS5 doesn’t allow debugging of iPad apps, however you can tab directly into the AIR debugger app located here:
/Applications/Adobe Flash CS5/AIK2.0/bin/adl

And debug your application manually. The command is as follow:


adl -screensize iPad -profile mobileDevice application.xml

For the iPad the command on my machine looks like this:


./adl -screensize iPad -profile mobileDevice /Users/Elad/Desktop/19/iPad\ Hello\ World/HelloWorld-app.xml

Installing your iPhone application

To install the application on your device you must first ensure you have added the provisioning profile file, see: Creating your first iPhone application using CS5 under: Provisioning Profile.

The easiest way to install the application is to double click the ipa file and it will be added to iTune and then you can just sync your device and your done.

Use FlashBuilder 4 to write your iPhone code

Flash Professional was criticized for creating projects with heavy ActionScript code projects and was one of the main motivation of using Eclipse as the IDE for developing AS3 code. You can use Flash Professional CS5 to compile and Flash Builder 4 to edit the ActionScript 3.0 content of your application.

File > New > iPhone OS. Save the project as Main.fla

Under properties click the pencil icon or place the class name and than hit the pencil, see Figure:

61

When it asks you which application should edit the ActionScript 3 class choose FlashBuilder.

7

At this point, you FlashBuilder will open up and create a new Flash Professional project, see Figure:

81

FlashBuilder opens up and ask you if you want to switch to Flash Prospective, select to switch and hit Ok.

The entry point class get populated automatically:


package
{
	import flash.display.Sprite;

	public class Main extends Sprite
	{
		public function Main()
		{

		}
	}
}

Let’s just add a text field with Hello World, see below;


package
{
     import flash.display.Sprite;
     import flash.text.TextField;

     public class Main extends Sprite
     {
          public function Main()
          {
               var textField:TextField = new TextField();
               textField.text = "Hello World";

               this.addChild( textField );
          }
     }
}

You can now publish your application. In FlashBuilder select:
Project > Publish Movie. You can also select: Command + Enter (Mac) or Control + Enter (PC). See Figure:

91