Reading Exif data with ActionScript 3.0

Most of the digital cameras use Exif file format to store images and through this format tons of information is available so that each software that deals with images can show relevant data loading few bytes.
The nice thing is that in the first 64 kbytes of an image all the information, thumbnail included, are stored in separate and well organized image file directories (IFDs), this is the reason why you don’t need to load a complete image to show it’s preview (if images are smaller than 64kbytes you can get anyway the information and there is no need to close the stream of data).
There are already some resources about exif data reading in ActionScript, you can get some great examples here:

•    http://code.shichiseki.jp/as3/ExifInfo/
•    http://patrickshyu.com/2009/04/jpg-exif-metadata-in-actionscript-3/

I wrote my own solution because the first one load all the image before starting to parse the data, because the second one created me some issues with a Motorola image and because I get also information for exif 1.1 data.
What I’m missing and I would like to improve is the visualization of thumbnails stored not with the JPG information and the visualization of GPS information that I already get but are not shown properly.
In order to avoid the issues I have had I would like to spent few words before getting you to the source code.

First of all in order to understand the format I really started to read the specification you can get here http://www.exif.org/Exif2-2.PDF. I know that it may not seem the most amazing piece of paper to read but was the only way to put in place a solution with all the comments needed to make people able to extend and improve it.

My second step was to find a good software able to read exif data in order to make a comparison between the information recovered from the Exif class I created and the ones recovered from a professional software, I get ACDSEE (http://www.acdsee.com/).
The third step was to get a lot of sample images and thankfully the exif official web site has a lot of them (by the way I embedded the images also in the sample application I did) available here http://www.exif.org/samples.html.
The last step was to get an hexadecimal reader to really parse and count the raw data I was going to play with, on Mac I found 0xED (http://www.suavetech.com/0xed/0xed.html), on Windows I get Hex Workshop (http://www.hexworkshop.com/).
Before starting to write down my solution I studied how other people implemented working solutions in other languages, two of the most inspiring for me were this one in Java http://www.java2s.com/Open-Source/Java-Document/Web-Server/Jigsaw/org.w3c.tools.jpeg.htm and this one http://mediachest.sourceforge.net/mediautil/javadocs/mediautil/image/jpeg/package-summary.html always written in Java.

Let’s start with some clarifications …

The Exif file format is based on the JPEG file format.  Exif inserts some of image / digital camera information and thumbnail image to JPEG in conformity to JPEG specification. Therefore you can view Exif formatted image files by JPEG compliant Internet browser / Picture viewer / Photo retouch software etc. as normal JPEG image files.
Every JPEG file starts from binary value ‘0xFFD8′, ends with binary value ‘0xFFD9′. There are several binary 0xFFXX data in JPEG data, they are known as ‘Marker’, and it starts a new part of JPEG information. 0xFFD8 means SOI (Start of image), 0xFFD9 means EOI (End of image). These two special Markers have no data following, the other markers have data with it.
The marker 0xFFE0~0xFFEF is named ‘Application Marker’, not necessary for decoding JPEG image. They are used by user application. For example, older olympus / canon / casio / agfa digital camera use JFIF (JPEG File Interchange Format) for storing images. JFIF uses APP0 (0xFFE0) marker for inserting digital camera configuration data and thumbnail image.
Exif format contains a thumbnail of the image (except Ricoh RDC-300Z). Usually it is located next to the IFD1. There are 3 formats for thumbnails; JPEG format (JPEG uses YCbCr), RGB TIFF format, YCbCr TIFF format. It seems that JPEG format and 160×120 pixels of size are recommended thumbnail format for Exif2.1 or later. By the DCF specification, thumbnail image MUST use JPEG format and image size is fixed to 160×120 pixels.  If the value of Compression (0×0103) Tag in IFD1 is ‘6′, thumbnail image format is JPEG. Most of Exif image uses JPEG format for thumbnail. In that case, you can get offset of thumbnail from JpegIFOffset (0×0201) Tag in IFD1, size of thumbnail from JpegIFByteCount (0×0202) Tag. Data format is ordinary JPEG format, starts from 0xFFD8 and ends by 0xFFD9.
The following terms and notation are used as follows in the exif standard:

•    “Tag” is used as a synonym of “field”,
•    “.H” appended to a numerical value means it is hexadecimal notation,
•    Unless otherwise indicated, other numerical values are given in decimal notation,
•    The initial IFD in a file is the “0th IFD,” with the next IFD being called the 1st IFD
•    The term Primary image refers to main image data
•    The term Thumbnail indicates a small image used to index the primary image
•    Exif is an abbreviation of Exchangeable image file format, used here as the general term for this standard and earlier versions of this standard

The main class of this solution is named Exif and contains all the exif tag uint values as class constants and a bunch of methods used to expose the data to the end user (i.g. the developer that use it in a project) and other methods to parse the information

The class extends the URLStream class so what you have to do in order to use it is creating and instance and load a valid URLRequest through it.
The events dispatched from the Exif class you have to aware of are

•    PARSE_COMPLETE
•    PARSE_FAILED
•    PARSING_ERRORS
•    DATA_READY
•    THUMBNAIL_READY

The first one is fired when the data are parsed completely, the second one indicates that a serious error happened (i.g, not a valid jpg, not valid exif data, etc.), the third one can be used to handle not serious errors (i.g. a tag is not properly recorded), the fourth one indicates that the exif data have been completely parsed and the last one is the event that inform you that the thumbnail is ready.
The public API can be used in order to get the thumbnail, the error logs, all the IFD recovered from the class and the amount of data loaded in order to parse the exif data.

The class usage is quite simple but the logic inside is a little bit more complex, in fact the core consists of two methods

•    exploreEntries
•    parseTag

The first one is the one that read all the entries (i.g. tags) stored in an IFD and that collaborates with the parseTag method in order to read the data as defined in the exif specification and with the Naming class used to handle a lot of situations like flash, orientation, etc. making these values human readable.

The following types are used in Exif (you see this types recovered in the first lines of the exploreEntries method):

•    1 = BYTE An 8-bit unsigned integer.,
•    2 = ASCII An 8-bit byte containing one 7-bit ASCII code. The final byte is terminated with NULL.,
•    3 = SHORT A 16-bit (2-byte) unsigned integer,
•    4 = LONG A 32-bit (4-byte) unsigned integer,
•    5 = RATIONAL Two LONGs. The first LONG is the numerator and the second LONG expresses the
denominator.,
•    7 = UNDEFINED An 8-bit byte that can take any value depending on the field definition,
•    9 = SLONG A 32-bit (4-byte) signed integer (2’s complement notation),
•    10 = SRATIONAL Two SLONGs. The first SLONG is the numerator and the second SLONG is the
denominator.

The Naming class can be passed to the Exif class in the constructor so that you can easily create your localized version of exif reader overriding the methods that initialize some arrays used to store values for the flash, the orientation, the exposure, etc.

Each time and IFD is found in the file the exif class create an IFD instance so that you can easily keep track of them and create customized helper classes to parse all these information, each time a valid exif tag is defined an instance of the ExfTag class is created, thorugh this class you can easily know the name and the value of a tag, use the public getTagValue method passing as an argument one of the 130 and more constants defined in the Exif class.

All these classes are available inside nabiro, the open source ActionScript library made up of a lot of components created during our daily job published here http://agile.gnstudio.com/nabiro.

Sample Usage
At the end of this post you find the sample application with the source code include, byt the way in order to use this tool you only need to instantiate the class, define a couple of event listeners and load a file

exif = new Exif();

exif.addEventListener(Exif.PARSING_ERRORS, onParsingErrors);
exif.addEventListener(Exif.DATA_READY, onExifDataRecovered);
exif.addEventListener(Exif.THUMBNAIL_READY, onThumbnailReady);

exif.load(new URLRequest(images.selectedItem.data));

The three event listeners populate a text area with the error log message, display a list of available data and load the image, so you can easily get this result

Download here the source and the application and please feel free to use it as you want but please provide me as much as possible your feedback.

Flex image manipulation

Image file formats are standardized means of organizing and storing images, let’s start to recap the building blocks of them in order to move faster in our flex applications.

Image file size (expressed as the number of bytes) increases with the number of pixels composing an image, and the color depth of the pixels and we use the words color depth or bit depth (usually a computer graphics term) in order to descibe the number of bits used to represent the color of a single pixel in a bitmapped image.

The color depths are summarized as following:

1-bit color (21 = 2 colors) monochrome, often black and white
2-bit color (22 = 4 colors) CGA, gray-scale early
3-bit color (23 = 8 colors) many early home computers with TV displays
4-bit color (24 = 16 colors) as used by EGA and by the least common denominator VGA standard at higher resolution
5-bit color (25 = 32 colors) Original Amiga chipset
6-bit color (26 = 64 colors) Original Amiga chipset
8-bit color (28 = 256 colors) most early color Unix workstations, VGA at low resolution, Super VGA, AGA, color Macintoshes.
12-bit color (212 = 4096 colors) some Silicon Graphics systems, Neo Geo, Color NeXTstation systems, and Amiga systems in HAM mode.
16-bit color (216 = 65536 colors) some color Macintoshes.

You can easily understand by the following images list (tanks to wikipedia) how much the color depth can impact the quality of your images.

There are two types of image file compression algorithms, lossless and lossy.

Lossless compression algorithms reduce file size without losing image quality, though they are not compressed into as small a file as a lossy compression file (when image quality is valued above file size, lossless algorithms are typically chosen).
Lossy compression algorithms take advantage of the inherent limitations of the human eye and discard invisible information, most lossy compression algorithms allow for variable quality levels (compression) and as these levels are increased, file size is reduced.

The most common image formats you can deal with in order to create image manipulation algorithms are: JPEG, TIFF, PNG and GIF.

JPEG (Joint Photographic Experts Group) files are (in most cases) a lossy format, the DOS filename extension is JPG (other operating systems may use JPEG).
Nearly every digital camera can save images in the JPEG format, which supports 8 bits per color (red, green, blue) for a 24-bit total, producing relatively small files so it’s usual that an application acquires images in this format.
JPEG files suffer generational degradation when repeatedly edited and saved are increased, file size is reduced so be careful to save too often the changes you do in your application.
The Exif (Exchangeable image file format) is an algorithm incorporated in the JPEG software used in most cameras, its purpose is to record and to standardize the exchange of data between digital cameras and editing and viewing software.The data is recorded for individual images and includes such things as camera settings, time and date, shutter speed, exposure, image size, compression, name of camera, color information, etc.
When images are viewed or edited by image editing software, all of this image information can be displayed and you can use them to create faster visualization tools.

The TIFF (Tagged Image File Format) is a flexible format that normally saves 8 bits or 16 bits per color (red, green, blue) for 24-bit and 48-bit totals, respectively, using either the TIFF or the TIF file extension. TIFFs compression algorithm are lossy and lossless and some offer relatively good lossless compression for bi-level (black & white) images.
The TIFF can handle device-specific color spaces, such as the CMYK defined by a particular set of printing press inks and for this reason is quite diffused in printing and in the photographer world.

The PNG (Portable Network Graphics) file format was created as the free, open-source successor to the GIF that supports true color (16 million colors) while the GIF supports only 256 colors.
The PNG file excels when the image has large, uniformly colored areas, the lossless PNG format is best suited for editing pictures, and the lossy formats, like JPG, are best for the final distribution of photographic images, because JPG files are smaller than PNG files.

The GIF (Graphics Interchange Format) format is limited to an 8-bit palette, or 256 colors, this makes the GIF format suitable for storing graphics with relatively few colors such as simple diagrams, shapes, logos and cartoon style images.
It uses a lossless compression that is more effective when large areas have a single color, and ineffective for detailed images or dithered images.

Each format has it’s own structure and in order to be able to manipulate images deeply you have to refer to each specific format, here is a list of all the file format specifications http://www.martinreddy.net/gfx/2d-hi.html.

You can consider the image manipulation process divided into two separate steps, the image transformation (rotation, zoom, vertical flip, mirroring, etc.) and the pixel manipulation.
In order to perform transformation the most efficient in Flex and in ActionScript is the affine geometry.

Affine geometry is a form of geometry featuring the unique parallel line property where the notion of angle is undefined and lengths cannot be compared in different directions, it is a generalization of Euclidean geometry characterized by slant and scale distortions.
Affine geometry can be developed in terms of the geometry of vectors, with or without the notion of coordinates, so an affine space is distinguished from a vector space of the same dimension by ‘forgetting’ the origin (sometimes known as free vectors).
Affine geometry can be seen as part of linear algebra, re-open your high school books and take a deep breath!

In linear algebra, linear transformations can be represented by matrices.
Matrices allow arbitrary linear transformations to be represented in a consistent format, suitable for computation (i.e. your transformation data are represented in a way that can be used with any software)…this also allows transformations to be concatenated easily (by multiplying their matrices).

For rotation by an angle θ clockwise about the origin, the functional form is x’ = xcosθ − ysinθ and y’ = xsinθ + ycosθ

Written in matrix form, this becomes

To represent affine transformations with matrices, we must use homogeneous coordinates, this means representing a 2-vector (x, y) as a 3-vector (x, y, 1), and similarly for higher dimensions.
All ordinary linear transformations are included in the set of affine transformations, and can be described as a simplified form of affine transformations hence, any linear transformation can be also represented by a general transformation matrix.

A matrix is a rectangular array (or table) of numbers consisting of any number of rows and columns it consists of m rows and n columns is known as an m x n matrix, this value represents the matrix’s dimensions.
You’ll commonly seen matrices with numbers in rows and columns surrounded by two large bracket symbols

Affine transformations are transformations that preserve collinearity and relative distancing in a transformed coordinate space, this means points on a line will remain in a line after an affine transformation is applied to the coordinate space in which that line exists.
It also means parallel lines remain parallel and that relative spacing or distancing, though it may scale, will always maintain at a consistent ratio.

Flash provides a 3 x 3 matrix in which u,v,w are sort of dummy here though their values remain as 0, 0, 1
You can manipulate the others properties (a,c,c,d,tx and ty) in order to get a transformation, following some expamples

  • Translation: tx or ty changes would move either x pixels or y pixels
  • Scale: a or d changes would affect xScale or yScale
  • Skew: b or c changes skew either parallel to the x or y axis
  • Rotation: a,b,c and d changes affect the rotation (for an angle θ  a is cos(θ ), b is sin(θ ), c is -sin(θ ) and d is cos(θ ))

The properties of the Matrix class in ActionScript are basically just a collection of your main a, b, c, d, tx, and ty properties those you need to be concerned about when dealing with transformations, additional methods are also provided to make working with these matrices easier.
Some of the more common methods are:

translate(tx:Number, ty:Number) : voidscale(sx:Number, sy:Number) : void

rotate(angle:Number) : void

identity() : void

Flash Player 10 expands greatly on the drawing API in ActionScript, more so than any other version of the Flash Player since the initial introduction of the drawing API in Flash Player 6
New features include:

  • Use of vectors (typed arrays) for improved throughput and use of memory
  • Support for non-zero winding rules
  • An API for drawing triangles with support for 3D perspective rendering
  • Drawing API data objects

The introduction of vectors help a lot developers to make more efficient and powerful transformations

Vectors are almost exactly like arrays at their core, having pretty much the same API

There are only a few real differences:

  • Elements within a vector all share the same type
  • Vectors have an additional property, fixed, which determines whether or not the length of the vector can change dynamically
  • The vector constructor allows for two optional arguments, length and fixed
  • There is no equivalent to the bracket ([]) array constructor

The Graphics class contains now the drawTriangles method

public function drawTriangles(vertices:Vector.<Number>, indices:Vector.<int>=null, uvtData:Vector.<Number>=null, culling:String="none");

This method uses a Vector.<Number> to specify point locations for a path to be drawn, only with drawTriangles, the commands internally are predefined to use the point locations to draw triangles (Every 3 points, i.e. 6 numbers, represents a triangle path to be drawn)

It seems now that we are starting to talk about 3D because 3D models (in the end) are represented by a collection of triangles in space this is a good starting point for the 3D… Why talking about 3D in Flash if we are dealing with images manipulation?

Trough triangle the transformations are now faster and accurate, affine transformations and the division in triangles of an image help you to get a better image manipulation procedure, see the demo to understand that the drawing mechanism is totally different (no cutting on the sides of the image as if you are working with the classic bitmap transformation.

Let’s start now with some building blocks you need in order to play with pixels and better understand the code that is contained in the luminance, gray scale, tint and red eye correction demos you’ll get at the end of this post.

In order to perform advanced manipulation on an image you need to start to play with the pixels of an image, ActionScript provides a lot of class to help you on this task, the most relevant are

  • Bitmap
  • BitmapData

The Bitmap class represents display objects that represent bitmap images data, These can be images that you load with the flash.display.Loader class, or they can be images that you create with the Bitmap() constructor.

The Bitmap() constructor allows you to create a Bitmap object that contains a reference to a BitmapData object.
After you create a Bitmap object, use the addChild() or addChildAt() method of the parent DisplayObjectContainer instance to place the bitmap on the display list.
I said “image data”, the BitmapData class let you play with this data (i.e. pixels).
You can use the methods of the BitmapData class to create arbitrarily sized transparent or opaque bitmap images and manipulate them in various ways at runtime.
This class lets you separate bitmap rendering operations from the internal display updating routines of Flash Player and moreover by manipulating a BitmapData object directly, you can create complex images without incurring the per-frame overhead of constantly redrawing the content from vector data.
Each 32-bit integer is a combination of four 8-bit channel values (from 0 to 255) that describe the alpha transparency and the red, green, and blue (ARGB) values of the pixel.
There is a complete list of all the method of the class in the ActionScript documemntation, I would like to point your attention on the histogram method added to the player 10 that computes a 256-value binary number histogram of a BitmapData object

histogram(hRect:Rectangle = null):Vector.<Vector.<Number>>

This is one of the key to play with the luminance effect in Flex.
Trough the combination of different pixel manipulation methods you can get complex effects like luminance
Relative luminance follows the photometric definition of luminance, but with the values normalized to 1 or 100 for a reference white.
Like the photometric definition, it is related to the luminous flux density in a particular direction, which is radiant flux density weighted by the luminosity function of the CIE Standard Observer.
For RGB color spaces that use the ITU-R BT.709 primaries relative luminance can be calculated from linear RGB components:

Y = 0.2126 R + 0.7152 G + 0.0722 B

Let see the demos to see a lot of pixel manipulation in place, in order to get the source of the effects you see here please refer to the nabiro images packaging you can get here http://agile.gnstudio.com/nabiro.

Code tuning techniques

Code tuning techniquesCode tuning differs a lot from refactoring because it doesn’t always improve code readability, and changes that you make are not meant to improve the internal structure of a software.
If the changes performed in the code don’t degrade the code’s readability, we believe that you are refactoring rather than fine-tuning.

One of the practices most of the fine tuning papers recommend is to stop testing a condition when an answer is known; this means that it could be better to split a short circuit evaluation into two separate evaluations, but this is not true in the Flash Player, if you make a simple test like this


private function testShortCircuit():void{
var start:int;
start = getTimer();

for(var i:int = 0; i < 100000000; i++){

    if(i > 3000000 && i < 4000000){

        break;

    }

}

trace("Short circuit", (getTimer() - start))

start = getTimer();

for(var j:int = 0; j < 100000000; j++){

    if(j > 3000000){

        if(j < 4000000){

            break;

        }

    }

}

trace("Not short circuit", (getTimer() - start));

you’ll get a result that clearly shows that short circuit in the Flash Player is faster

Short circuit 307
Not short circuit 331

Arrange tests in the conditional statements by frequency, putting the most common case as the first condition to evaluate is a good strategy for increasing performance.

We strongly recommend grouping together loops that operate on the same set of elements, so that you can remove a loop and acquire a lot of speed processing.

When dealing with nested loops we recommend always putting the busiest loop in the inside and moving the one with fewest iterations to the outside.

There are many common mathematical operations that can be done faster than usual

Operation                  Faster solution

Math.floor(1.5);         int(1.5);
Math.ceil(1.5);          int(1.5) + 1;
Math.abs(value);         var test:Number = value < 0 ? value * -1 : value;
value / 2;               value >> 1
value * 2;               value << 1

If you need some values multiple times in a component / class and these values need to be calculated we suggest that you calculate them only once and put the in constants for faster access

private const SOME_VALUE:Number = Math.sqrt(Math.pow(Math.PI, 10));

If the calculation varies during the execution of the class we suggest putting it in a method and avoiding having multiple pieces of code for the same calculation duplicated in the code.

Other reosurces

http://www.adobe.com/devnet/flex/articles/client_perf.html