Flex image manipulation

Oct 7th

Posted by Giorgio Natili in code

2 comments

Image file formats are standardized means of organizing and storing images, let’s start to recap the building blocks of them in order to move faster in our flex applications.

Image file size (expressed as the number of bytes) increases with the number of pixels composing an image, and the color depth of the pixels and we use the words color depth or bit depth (usually a computer graphics term) in order to descibe the number of bits used to represent the color of a single pixel in a bitmapped image.

The color depths are summarized as following:

1-bit color (21 = 2 colors) monochrome, often black and white
2-bit color (22 = 4 colors) CGA, gray-scale early
3-bit color (23 = 8 colors) many early home computers with TV displays
4-bit color (24 = 16 colors) as used by EGA and by the least common denominator VGA standard at higher resolution
5-bit color (25 = 32 colors) Original Amiga chipset
6-bit color (26 = 64 colors) Original Amiga chipset
8-bit color (28 = 256 colors) most early color Unix workstations, VGA at low resolution, Super VGA, AGA, color Macintoshes.
12-bit color (212 = 4096 colors) some Silicon Graphics systems, Neo Geo, Color NeXTstation systems, and Amiga systems in HAM mode.
16-bit color (216 = 65536 colors) some color Macintoshes.

You can easily understand by the following images list (tanks to wikipedia) how much the color depth can impact the quality of your images.

There are two types of image file compression algorithms, lossless and lossy.

Lossless compression algorithms reduce file size without losing image quality, though they are not compressed into as small a file as a lossy compression file (when image quality is valued above file size, lossless algorithms are typically chosen).
Lossy compression algorithms take advantage of the inherent limitations of the human eye and discard invisible information, most lossy compression algorithms allow for variable quality levels (compression) and as these levels are increased, file size is reduced.

The most common image formats you can deal with in order to create image manipulation algorithms are: JPEG, TIFF, PNG and GIF.

JPEG (Joint Photographic Experts Group) files are (in most cases) a lossy format, the DOS filename extension is JPG (other operating systems may use JPEG).
Nearly every digital camera can save images in the JPEG format, which supports 8 bits per color (red, green, blue) for a 24-bit total, producing relatively small files so it’s usual that an application acquires images in this format.
JPEG files suffer generational degradation when repeatedly edited and saved are increased, file size is reduced so be careful to save too often the changes you do in your application.
The Exif (Exchangeable image file format) is an algorithm incorporated in the JPEG software used in most cameras, its purpose is to record and to standardize the exchange of data between digital cameras and editing and viewing software.The data is recorded for individual images and includes such things as camera settings, time and date, shutter speed, exposure, image size, compression, name of camera, color information, etc.
When images are viewed or edited by image editing software, all of this image information can be displayed and you can use them to create faster visualization tools.

The TIFF (Tagged Image File Format) is a flexible format that normally saves 8 bits or 16 bits per color (red, green, blue) for 24-bit and 48-bit totals, respectively, using either the TIFF or the TIF file extension. TIFFs compression algorithm are lossy and lossless and some offer relatively good lossless compression for bi-level (black & white) images.
The TIFF can handle device-specific color spaces, such as the CMYK defined by a particular set of printing press inks and for this reason is quite diffused in printing and in the photographer world.

The PNG (Portable Network Graphics) file format was created as the free, open-source successor to the GIF that supports true color (16 million colors) while the GIF supports only 256 colors.
The PNG file excels when the image has large, uniformly colored areas, the lossless PNG format is best suited for editing pictures, and the lossy formats, like JPG, are best for the final distribution of photographic images, because JPG files are smaller than PNG files.

The GIF (Graphics Interchange Format) format is limited to an 8-bit palette, or 256 colors, this makes the GIF format suitable for storing graphics with relatively few colors such as simple diagrams, shapes, logos and cartoon style images.
It uses a lossless compression that is more effective when large areas have a single color, and ineffective for detailed images or dithered images.

Each format has it’s own structure and in order to be able to manipulate images deeply you have to refer to each specific format, here is a list of all the file format specifications http://www.martinreddy.net/gfx/2d-hi.html.

You can consider the image manipulation process divided into two separate steps, the image transformation (rotation, zoom, vertical flip, mirroring, etc.) and the pixel manipulation.
In order to perform transformation the most efficient in Flex and in ActionScript is the affine geometry.

Affine geometry is a form of geometry featuring the unique parallel line property where the notion of angle is undefined and lengths cannot be compared in different directions, it is a generalization of Euclidean geometry characterized by slant and scale distortions.
Affine geometry can be developed in terms of the geometry of vectors, with or without the notion of coordinates, so an affine space is distinguished from a vector space of the same dimension by ‘forgetting’ the origin (sometimes known as free vectors).
Affine geometry can be seen as part of linear algebra, re-open your high school books and take a deep breath!

In linear algebra, linear transformations can be represented by matrices.
Matrices allow arbitrary linear transformations to be represented in a consistent format, suitable for computation (i.e. your transformation data are represented in a way that can be used with any software)…this also allows transformations to be concatenated easily (by multiplying their matrices).

For rotation by an angle θ clockwise about the origin, the functional form is x’ = xcosθ − ysinθ and y’ = xsinθ + ycosθ

Written in matrix form, this becomes

To represent affine transformations with matrices, we must use homogeneous coordinates, this means representing a 2-vector (x, y) as a 3-vector (x, y, 1), and similarly for higher dimensions.
All ordinary linear transformations are included in the set of affine transformations, and can be described as a simplified form of affine transformations hence, any linear transformation can be also represented by a general transformation matrix.

A matrix is a rectangular array (or table) of numbers consisting of any number of rows and columns it consists of m rows and n columns is known as an m x n matrix, this value represents the matrix’s dimensions.
You’ll commonly seen matrices with numbers in rows and columns surrounded by two large bracket symbols

Affine transformations are transformations that preserve collinearity and relative distancing in a transformed coordinate space, this means points on a line will remain in a line after an affine transformation is applied to the coordinate space in which that line exists.
It also means parallel lines remain parallel and that relative spacing or distancing, though it may scale, will always maintain at a consistent ratio.

Flash provides a 3 x 3 matrix in which u,v,w are sort of dummy here though their values remain as 0, 0, 1
You can manipulate the others properties (a,c,c,d,tx and ty) in order to get a transformation, following some examples

  • Translation: tx or ty changes would move either x pixels or y pixels
  • Scale: a or d changes would affect xScale or yScale
  • Skew: b or c changes skew either parallel to the x or y axis
  • Rotation: a,b,c and d changes affect the rotation (for an angle θ  a is cos(θ ), b is sin(θ ), c is -sin(θ ) and d is cos(θ ))

The properties of the Matrix class in ActionScript are basically just a collection of your main a, b, c, d, tx, and ty properties those you need to be concerned about when dealing with transformations, additional methods are also provided to make working with these matrices easier.
Some of the more common methods are:

translate(tx:Number, ty:Number) : voidscale(sx:Number, sy:Number) : void

rotate(angle:Number) : void

identity() : void

Flash Player 10 expands greatly on the drawing API in ActionScript, more so than any other version of the Flash Player since the initial introduction of the drawing API in Flash Player 6
New features include:

  • Use of vectors (typed arrays) for improved throughput and use of memory
  • Support for non-zero winding rules
  • An API for drawing triangles with support for 3D perspective rendering
  • Drawing API data objects

The introduction of vectors help a lot developers to make more efficient and powerful transformations

Vectors are almost exactly like arrays at their core, having pretty much the same API

There are only a few real differences:

  • Elements within a vector all share the same type
  • Vectors have an additional property, fixed, which determines whether or not the length of the vector can change dynamically
  • The vector constructor allows for two optional arguments, length and fixed
  • There is no equivalent to the bracket ([]) array constructor

The Graphics class contains now the drawTriangles method

public function drawTriangles(vertices:Vector.<Number>, indices:Vector.<int>=null, uvtData:Vector.<Number>=null, culling:String="none");

This method uses a Vector.<Number> to specify point locations for a path to be drawn, only with drawTriangles, the commands internally are predefined to use the point locations to draw triangles (Every 3 points, i.e. 6 numbers, represents a triangle path to be drawn)

It seems now that we are starting to talk about 3D because 3D models (in the end) are represented by a collection of triangles in space this is a good starting point for the 3D… Why talking about 3D in Flash if we are dealing with images manipulation?

Trough triangle the transformations are now faster and accurate, affine transformations and the division in triangles of an image help you to get a better image manipulation procedure, see the demo to understand that the drawing mechanism is totally different (no cutting on the sides of the image as if you are working with the classic bitmap transformation.

Let’s start now with some building blocks you need in order to play with pixels and better understand the code that is contained in the luminance, gray scale, tint and red eye correction demos you’ll get at the end of this post.

In order to perform advanced manipulation on an image you need to start to play with the pixels of an image, ActionScript provides a lot of class to help you on this task, the most relevant are

  • Bitmap
  • BitmapData

The Bitmap class represents display objects that represent bitmap images data, These can be images that you load with the flash.display.Loader class, or they can be images that you create with the Bitmap() constructor.

The Bitmap() constructor allows you to create a Bitmap object that contains a reference to a BitmapData object.
After you create a Bitmap object, use the addChild() or addChildAt() method of the parent DisplayObjectContainer instance to place the bitmap on the display list.
I said “image data”, the BitmapData class let you play with this data (i.e. pixels).
You can use the methods of the BitmapData class to create arbitrarily sized transparent or opaque bitmap images and manipulate them in various ways at runtime.
This class lets you separate bitmap rendering operations from the internal display updating routines of Flash Player and moreover by manipulating a BitmapData object directly, you can create complex images without incurring the per-frame overhead of constantly redrawing the content from vector data.
Each 32-bit integer is a combination of four 8-bit channel values (from 0 to 255) that describe the alpha transparency and the red, green, and blue (ARGB) values of the pixel.
There is a complete list of all the method of the class in the ActionScript documemntation, I would like to point your attention on the histogram method added to the player 10 that computes a 256-value binary number histogram of a BitmapData object

histogram(hRect:Rectangle = null):Vector.<Vector.<Number>>

This is one of the key to play with the luminance effect in Flex.
Trough the combination of different pixel manipulation methods you can get complex effects like luminance
Relative luminance follows the photometric definition of luminance, but with the values normalized to 1 or 100 for a reference white.
Like the photometric definition, it is related to the luminous flux density in a particular direction, which is radiant flux density weighted by the luminosity function of the CIE Standard Observer.
For RGB color spaces that use the ITU-R BT.709 primaries relative luminance can be calculated from linear RGB components:

Y = 0.2126 R + 0.7152 G + 0.0722 B

Let see the demos to see a lot of pixel manipulation in place, in order to get the source of the effects you see here please refer to the nabiro images packaging you can get here http://agile.gnstudio.com/nabiro.

Code tuning techniques

Oct 4th

Posted by Giorgio Natili in code

2 comments

Code tuning techniquesCode tuning differs a lot from refactoring because it doesn’t always improve code readability, and changes that you make are not meant to improve the internal structure of a software.
If the changes performed in the code don’t degrade the code’s readability, we believe that you are refactoring rather than fine-tuning.

One of the practices most of the fine tuning papers recommend is to stop testing a condition when an answer is known; this means that it could be better to split a short circuit evaluation into two separate evaluations, but this is not true in the Flash Player, if you make a simple test like this


private function testShortCircuit():void{
var start:int;
start = getTimer();

for(var i:int = 0; i < 100000000; i++){

    if(i > 3000000 && i < 4000000){

        break;

    }

}

trace("Short circuit", (getTimer() - start))

start = getTimer();

for(var j:int = 0; j < 100000000; j++){

    if(j > 3000000){

        if(j < 4000000){

            break;

        }

    }

}

trace("Not short circuit", (getTimer() - start));

you’ll get a result that clearly shows that short circuit in the Flash Player is faster

Short circuit 307
Not short circuit 331

Arrange tests in the conditional statements by frequency, putting the most common case as the first condition to evaluate is a good strategy for increasing performance.

We strongly recommend grouping together loops that operate on the same set of elements, so that you can remove a loop and acquire a lot of speed processing.

When dealing with nested loops we recommend always putting the busiest loop in the inside and moving the one with fewest iterations to the outside.

There are many common mathematical operations that can be done faster than usual

Operation                  Faster solution

Math.floor(1.5);         int(1.5);
Math.ceil(1.5);          int(1.5) + 1;
Math.abs(value);         var test:Number = value < 0 ? value * -1 : value;
value / 2;               value >> 1
value * 2;               value << 1

If you need some values multiple times in a component / class and these values need to be calculated we suggest that you calculate them only once and put the in constants for faster access

private const SOME_VALUE:Number = Math.sqrt(Math.pow(Math.PI, 10));

If the calculation varies during the execution of the class we suggest putting it in a method and avoiding having multiple pieces of code for the same calculation duplicated in the code.

Other resources

http://www.adobe.com/devnet/flex/articles/client_perf.html

Using the Abstract Factory pattern with Flex

Feb 13th

Posted by Giorgio Natili in design patterns

4 comments

Accordingly to the Gang of four the Abstract Factory pattern intent is to provide an interface for creating families of related or dependent objects without specifying their concrete classes.
This design pattern actually ensures that the patterns automatically get and use the correct object accordingly to the context in which the client is working.
There are many different situations in which this pattern can work, imagine for instance a system in which the drawing and printing method varies accordingly to the resolution supported by the system; the system has to use different drivers in different cases.
At a first glance you may be tempted to use two different switch case for the drawing and printing procedure in which your system reacts in a different way accordingly to the resolution but remember that switches may indicate the need of abstraction in your system and that it can bring your system to a combinatorial explosion (imagine to add new drivers for each different resolution and create and even more complex switch case).
A good way to solve this issue us to create a Factory that creates the appropriate object accordingly to the resolution supported by the system, in this way you avoid the combinatorial explosion I mentioned and you can keep the switches in a single place or even better accordingly to the language you are working on and to your code style you can avoid switches also in the factory.
We have talked since now about two possible families of objects to use, monitor and printer drivers. Give me a chance to introduce you to a more complex system, an e-commerce with a group of families related to the system payments

•    Credit Card
•    OnLine payment
•    Other payment

For each of these families you may have the need to define multiple objects when you system starts

•    Credit Card
o    Visa
o    American Express
o    Master Card
•    OnLine payment
o    Paypal
o    E-check
•    Other payment
o    Wire transfer
o    Check

and add even more objects when the system grows or when new payment methods will be released over the net.
In an e-commerce system you may also have the need to show to the user the appropriate payment system accordingly to his preferences, in order to increase the abstraction of your system you can define multiple Abstract classes and keep the system ignorant on which particular implementation is in use because the factories are the responsible to instantiate them.
Imagine the scenario I described an put it in an UML diagram

Abstract Factory

The application is composed by a form that contains a submit button and the form fields vary accordingly to the user preferences retrieved by the system.
The CheckOutFactory class is an abstract class that have two concrete implementations able to return to the client the UI needed for each payment (actually the MXML representation of the class). The PaymentContainer is a VBox with a property used to store a reference to the payment form created from the factory that implements an interface that defines two common methods of the payment forms

•    validateFields()
•    submit()

As you know ActionScript 3.0 doesn’t support abstract classes, so the CheckOutFactory simulates the abstraction of the class trough the use of an internal class in its constructor.
Another good way to implement the abstraction is the use of interfaces, this is the reason why I defined a common interface ICheckOut and two other interfaces (IOnlinePayment and ICreditCard) that extend the base interface and that will be implemented in the view of each payment form of the system.
Take a look to the class hierarchy in order to understand where we are and where we are going

class hierarchy

The abstraction of the on-line and credit card payments is reached through the interfaces put on the top of the onLinePayment and creditCardPayment packaging, the view and it’s logic has been keep separated through the Model View Presenter pattern used from each payment form.
Each presenter implements a public submit() method, in this way you can call in a centralized fashioned way the submit of each form trough the PaymentContainer property named element, each presenter will have the responsibility to recover the data from the view and each view, due to the fact that implements the ICheckOut interface, validate it’s fields (a good idea is to validate each credit card with a different Validator, this is the reason why you find an instance of the CreditCardValidator class in the  credit card payment forms.
In order to run this sample (view source enabled) I created an XML file that stores the name of some users and the payment preferences, each node has the following structure

<user name = "Giorgio Natili" mode = "OnlinePayment" type = "PayPal" />

In the main application a combo box is populated with this data and each time you change the selection a concrete factory is created and through the factory a new view is added to the PaymentContainer

private function onUserSelection(e:Event):void{
checkOutFactory = CheckOutFactory.getPaymentFacotry(e.target.selectedItem.@mode);
var s:String = e.target.selectedItem.@type;
checkOutModule = checkOutFactory.getUI(s.substr(0, 1).toLocaleLowerCase() + s.substring(1, s.length) + ".view." + s + "View");
paymentContainer.element = checkOutModule;
}

The checkOutFactory and the checkOutMopdule properties data type represents the layer of abstraction I’m searching for
private var checkOutFactory:CheckOutFactory;
private var checkOutModule:ICheckOut;
The method defined as a listener for the click event inside the PaymentContainer uses the methods defined in the ICheckOut interface to complete his task

private function doSubmit():void{
if(_element.validateData()){
_element.submit();
}else{
Alert.show("Invalid data in the form...", "Attention!");
}
}

At the end of the day we have a quite flexible sample, but which are the benefits of this pattern? It would be simpler to have a switch instead of all this code?
The main benefit is that if you have to add the support for another credit card you have to deal only with the logic stored in the MVP triad you need to add a payment form and the system automatically will be able to handle the new form.
The switch could be hard to maintain in a situation with more than 4 payment methods, moreover the abstraction layer that the system has reached give to you the flexibility to put each developer you want on the new payment forms the system needs without having to explain to the developer anything about the system itself.

There are other possible contexts in which the Abstract Factory pattern can be used

•    Handle different operating systems API in a cross platform application
•    Different traits for users of an application
•    Different version of an application
•    Different performance guidelines

At the end of the day we can recap with the following points the Abstract Factory pattern

•    You want to have families or sets of objects for particular clients
•    Families of related objects have to be instantiated
•    You want to coordinate the creation of families of objects
•    You need to isolates the rules of which objects are to be made

Obviously this is only a small sample and more complex implementation are out of the scope of this blog entry, so feel free to open a discussion on this topic.

«12345»...Last »