In Flash 10 Adobe added a compiler to handle filters, which is possible through Pixel Bender kernels. Pixel Bender kernels is a program that calculate a single pixel at run time.
This is how it works. The Pixel Bender graph is an XML language for combining individual pixel-processing operations called kernels, which are in PBK file format.
We can create the PBK XML file using Pixel Bender toolkit. After our kernel is ready we can then export it as a byte-code called PBJ. PBJ can then be used in the Flash 10 player.
Flex Gumbo Bender shader is the Pixel Bender kernels, which uses a separate thread than Flash Player to calculate a single pixel.
You can use the following math functions for in pixel bender for calculations:
1. sin(x) - Trigonometric sine function.
2. cos(x) - Cosine function.
3. tan(x) – Tangent function.
4. asin(x) - Arcsine (inverse sine) function.
5. acos(x) - inverse cosine (arccosine) function.
6. atan(x) - Arctangent (inverse tangent) function.
7. atan(x, y) - Arctangent (inverse tangent) function.
8. exp(x) - exponential function.
9. log(x) – Logarithm function.
10. pow(x, y) - power of function.
11. reciprocal(x) - multiplicative inverse function.
12. sqrt(x) - square root function.
The challenge with Flash related to single thread processing always caused issues, and while the Flash Player is processing information we cannot run another thread to do other things. We often find the player get “stuck” when doing heavy lifting. Pixel Bender can help in certain cases. I managed to create an example and an API that uses Pixel Bender to calculate information, while a video is playing. In the example here you can run calculations while the video is playing and compare performance with Flash Player doing the same calculation.
First play the video and hit the Pixel Bender calculation and while the video is playing calculations are being made in the background. The results are astonishing, using Flash Player to calculate 5M sine took about 10 seconds (on iMac 2.8GHz and 4GB memory) and the video paused. Using Pixel Bender the video had a light glitch and we received the results back for the 5M calculation after about 10 secounds, without the user noticing.
Create the pbj file in Pixel Bender Toolkit:
<languageVersion : 1.0;>
kernel SinCalculator
<
namespace : "pixelBender";
vendor : "Elad Elrom";
version : 1;
description : "Sin Calculator";
>
{
input image1 src;
output pixel3 result;
void evaluatePixel()
{
pixel1 value = pixel1(sin(sample(src, outCoord())));
result = pixel3(value, 0.0, 0.0);
}
}
The API handle calculating with Pixel bender by splitting the process into chucks of 5,000 per kernal, since Pixel Bender produce an error message on large calculations.
package com.elad.framework.pixelBender
{
import com.elad.framework.pixelBender.events.PixelBenderCalcEvent;
import flash.display.Shader;
import flash.display.ShaderJob;
import flash.events.Event;
import flash.events.EventDispatcher;
import flash.utils.ByteArray;
import flash.utils.Endian;
[Event(name="completed", type="com.elad.pixelBender.events.PixelBenderCalcEvent")]
/**
* The PixelBenderCalculator class is the base class for math calculation with pixel bender
*/
public class PixelBenderCalculator extends EventDispatcher
{
public var kernalClass:Class;
public var numberCollection:Array;
private var shader:Shader;
private var shaderJob:ShaderJob;
private var input:ByteArray;
private var output:ByteArray;
private var retCollection:Array;
private var requestsCounter:Number;
private var numberOfRequest:Number;
/**
* Number of calculations per kernal
*/
private const COLLECTION_SIZE:int = 5000;
/**
* Default constructor for the pixel bender calculator
*
* @param numberCollection collection contain numbers
* @param kernalClass the kernal class of type pbj to be used to run the calculation
*
*/
public function PixelBenderCalculator(numberCollection:Array, kernalClass:Class)
{
reset();
this.kernalClass = kernalClass;
this.numberCollection = numberCollection;
requestsCounter = numberCollection.length/COLLECTION_SIZE;
}
/**
* Method to start the calculation
*
*/
public function start():void
{
output = new ByteArray();
output.endian = Endian.LITTLE_ENDIAN;
var start:int = numberOfRequest*COLLECTION_SIZE;
var end:int = ( (numberOfRequest+1)*COLLECTION_SIZE > numberCollection.length) ? numberCollection.length : ((numberOfRequest+1)*COLLECTION_SIZE);
input = convertArrayToByteArray(numberCollection, start, end);
createShaderJob();
numberOfRequest++;
}
/**
* Creates a shader class based on the kernal to pass the numbers and start the calculations.
*
* @see flash.display.ShaderJob
* @see flash.display.Shader
*
*/
private function createShaderJob():void
{
var width:int = input.length >> 2;
var height:int = 1;
shader = new Shader(new kernalClass());
shader.data.src.width = width;
shader.data.src.height = height;
shader.data.src.input = input;
shaderJob = new ShaderJob(shader, output, width, height);
shaderJob.addEventListener(Event.COMPLETE, shaderJobCompleteHandler);
shaderJob.start();
}
/**
* Static method to convert the array given into byte array.
*
* @param array
* @return
*
*/
private static function convertArrayToByteArray(array:Array, start:int, end:int):ByteArray
{
var retVal:ByteArray = new ByteArray();
var number:Number;
retVal.endian = Endian.LITTLE_ENDIAN;
for (var i:int=start; i<end; i++)
{
number = Number(array[i]);
retVal.writeFloat(number);
}
retVal.position = 0;
return retVal;
}
/**
* Convert the a <code>ByteArray</code> into an <code>Array</code>
*
* @param byteArray
* @return an array collection
*
*/
private function addByteArrayToCollection(byteArray:ByteArray):void
{
var length:int = byteArray.length;
var number:Number;
for(var i:int=0; i<length; i+=4)
{
number = byteArray.readFloat();
if(i % 3 == 0)
{
retCollection.push(number);
}
}
}
/**
* Handler for the shader once job is completed.
*
* @param event
*
*/
private function shaderJobCompleteHandler(event:Event):void
{
output.position = 0;
addByteArrayToCollection(output);
input = null;
output = null;
if (requestsCounter>numberOfRequest)
{
start();
}
else
{
calculationCompleted();
}
}
/**
* Method to dispatch an event once calculation is completed and reset this class.
*
*/
private function calculationCompleted():void
{
this.dispatchEvent( new PixelBenderCalcEvent(retCollection) );
reset();
}
/**
* Method to clean up this class so we are not using un-needed memory.
*
*/
public function reset():void
{
retCollection = new Array();
numberCollection = new Array();
requestsCounter = 0;
numberOfRequest = 0;
numberCollection = null;
}
}
}
The implimentation is straight forward:
<FxApplication xmlns="http://ns.adobe.com/mxml/2009" xmlns:local="*" viewSourceURL="srcview/index.html">
<!--
////////////////////////////////////////////////////////////////////////////////
//
// Elad Elrom (elad@elromdesign.com)
// Copyright 2009 Elorm LLC,
// All Rights Reserved.
//
// NOTICE: Elad Elrom permits you to use, modify, and distribute this file
// in accordance with the terms of the license agreement accompanying it.
//
////////////////////////////////////////////////////////////////////////////////
@author Elad Elrom
-->
<Script>
<![CDATA[
import com.elad.framework.pixelBender.PixelBenderCalculator;
import com.elad.framework.pixelBender.events.PixelBenderCalcEvent;
import mx.collections.ArrayCollection;
import mx.collections.IList;
import mx.events.FlexEvent;
[Embed(source="SinCalculator.pbj", mimeType="application/octet-stream")]
private var kernalClass:Class;
private var pixelBenderCalc:PixelBenderCalculator;
protected function startCalculatorPixelBender():void
{
// create a number collection
var numberCollection:Array = new Array();
for (var i:int=0; i<5000000; i++)
{
numberCollection.push( i );
}
// calculate
pixelBenderCalc = new PixelBenderCalculator(numberCollection, kernalClass);
pixelBenderCalc.addEventListener(PixelBenderCalcEvent.COMPLETED, onComplete );
pixelBenderCalc.start();
}
private function startCalculatorFlashPlayer():void
{
// create a number collection
var numberCollection:Array = new Array();
for (var i:int=0; i<5000000; i++)
{
numberCollection.push( Math.sin(i) );
}
list.dataProvider = new ArrayCollection(numberCollection);
}
private function onComplete(event:PixelBenderCalcEvent):void
{
list.dataProvider = new ArrayCollection(event.numberCollection);
pixelBenderCalc.removeEventListener(PixelBenderCalcEvent.COMPLETED, onComplete);
}
]]>
</Script>
<List id="list" width="200" height="531.5"/>
<Button label="Calculate with Pixel Bender" width="200" height="20" y="545" click="startCalculatorPixelBender()"/>
<Button label="Calculate with Flash Player" width="200" height="20" y="574" click="startCalculatorFlashPlayer()"/>
<VideoDisplay id="vid" width="355" height="290"
source="http://thehq.tv/wp-content/uploads/flv/super-street-fighter-2-hd-round-1-trailer.flv"
autoPlay="false" x="209" y="-1"/>
<Button label="Play" click="vid.play();" x="213" y="303"/>
<local:MemoryDashBoard x="211" y="343"/>
</FxApplication>
