Last week I was playing around with a little EaselJS experiment which required me to do collision detection against all items on the screen. This worked fine with a small number of items, but of course, the more items I added, the slower everything became.

I knew that I needed to optimize the code, and pare down the number of collision checks. I have done this before with a grid (even held a contest for it) and was going to port that AS3 code to JavaScript. However, Ralph Hauwert suggested I look at implemented a QuadTree, which should be more efficient.

A QuadTree is a data structure in which the coordinate space is broken up into regions / nodes that contain items. If too many items are added into a node, then that node is divided into 4 sub-nodes. This can provide very fast lookup of items based on the coordinates and coordinates and dimensions.

There are a couple of implementations in both JavaScript and ActionScript (Michael Baczynski has a nice AS3 implementation), but I decided that I wanted to learn a bit more and implement one from scratch.

The result if a QuadTree implemented in JavaScript that works with both 2D coordinates (x,y) and well as regions / items with dimensions.

I’ve released all of the code under an MIT License, and you can download it from my GitHub repository.

Here is an example of the tree in action:

This example shows how to use the QuadTree to pare down the number of items that need to be tested for collision detection.

I have created a couple of other simple examples:

• Collision Detection (shown above)
• Inserting points
• Inserting items with bounds
• Retrieving points
• Retrieving items with bounds

The basic code is pretty simple. Here is an example showing using the QuadTree to store and retrieve points:

var pointQuad = true;
var bounds = {
	x:0,
	y:0,
	width:canvas.width,
	height:canvas.height
}
 
var quad = new QuadTree(bounds, pointQuad);
 
//insert a random point
quad.insert({x:12, y:25});
 
var items = quad.retrieve({x:11, y:20});

And here is a simple example showing using the QuadTree to store and retrieve items with dimensions / bounds:

var bounds = {
	x:0,
	y:0,
	width:canvas.width,
	height:canvas.height
}
var quad = new QuadTree(bounds);
 
//insert a random point
quad.insert({
	x:12, 
	y:25,
	height:10,
	width:25
});
 
var items = quad.retrieve({x:11, y:20, height:10, width:20});

Again, you can download all of the code from my GitHub Repository. It seems fairly solid at this point, but if you find any issues, or have any suggestions either fork the project, and submit the changes to me, or post them in the comments.

There is still a lot of room for optimization and improvement in the implementation, such as pre-allocating the nodes, but Ill leave that for another day.

Btw, big thanks to Ralph Hauwert for pointing me in the right direction for my project.

Post any questions or suggestions in the comments.

If you have read my blog any this week, you have probably noticed that I have been doing some basic research on collision detection within the Flash Player. As part of this, I have put together a simple test suite, showing the performance of a couple of different techniques for checking for collision. This is by no means meant to be exhaustive (and currently tilts towards boundary collision). However, I wanted to post the results as the current information is useful (if nothing more than to confirm existing assumptions), and perhaps generate more tests an ideas around collision detection.

First, the test code (uses Flash Pro). clip1 and clip2 reference two overlapping MovieClips on the stage (clip2 is about 1/2 the size of clip1).

package
{
	import flash.display.MovieClip;
	import flash.geom.Point;
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.geom.Rectangle;

	import com.gskinner.utils.PerformanceTest;

	public class CollisionDetectionTests extends MovieClip
	{
		public var clip1:MovieClip;
		public var clip2:MovieClip;

		private var dynamicClip1:MovieClip;
		private var dynamicClip2:MovieClip;	

		var point1:Point = new Point();
		var point2:Point = new Point();

		private var clip1Rect:Rectangle;
		private var clip1ClipBmpData:BitmapData;
		private var clip2Rect:Rectangle;
		private var clip2ClipBmpData:BitmapData;

		public function CollisionDetectionTests()
		{
			super();

			init();
			checkCollisions();
			runTests();
		}

		private function init():void
		{

			clip1Rect = clip1.getBounds(this);
			clip1ClipBmpData = new BitmapData(clip1Rect.width, clip1Rect.height, true, 0);
			clip1ClipBmpData.draw(clip1);

			clip2Rect = clip2.getBounds(this);
			clip2ClipBmpData = new BitmapData(clip2Rect.width, clip2Rect.height, true, 0);
			clip2ClipBmpData.draw(clip2);

			dynamicClip1 = new MovieClip();

			dynamicClip1.graphics.beginFill(0xFF0000);
			dynamicClip1.graphics.drawEllipse(0,0, 100,100);
			dynamicClip1.cacheAsBitmap = true;

			dynamicClip1.x = 300;
			dynamicClip1.y = 300;

			addChild(dynamicClip1);

			dynamicClip2 = new MovieClip();

			dynamicClip2.graphics.beginFill(0x0000FF);
			dynamicClip2.graphics.moveTo(0, 0);
			dynamicClip2.graphics.lineTo(100, 0);
			dynamicClip2.graphics.lineTo(100, 100);
			dynamicClip2.graphics.lineTo(0, 100);
			dynamicClip2.graphics.lineTo(0, 0);
			dynamicClip2.cacheAsBitmap = true;

			dynamicClip2.x = 250;
			dynamicClip2.y = 250;

			addChild(dynamicClip2);
		}

		private function checkCollisions():void
		{
			//make sure everything is working and returns true
			trace(checkHitTest());
			trace(boundsIntersection());
			trace(checkHitTestReverse());
			trace(checkBoundsManually());
			trace(hitTestCircle());
			trace(checkBitmapDataHit());
			trace(checkBitmapDataHitReverse());
			trace(checkBitmapDataHitInternal());
			trace(checkBitmapDataHitDynamic());
		}

		private function runTests():void
		{
			var perfTest:PerformanceTest = PerformanceTest.getInstance();
			perfTest.out = out;

			//this is to work around bug in test lib
			perfTest.testSuite({});

			perfTest.testFunction(checkHitTest, 10000, "checkHitTest", "Uses DisplayObject.hitTest");
			perfTest.testFunction(checkHitTestReverse, 10000, "checkHitTestReverse", "Uses DisplayObject.hitTest with clips reversed");
			perfTest.testFunction(boundsIntersection, 10000, "boundsIntersection", "Uses Rectangle.intersects()");
			perfTest.testFunction(checkBoundsManually, 10000, "checkBoundsManually", "manually checks if bounds intersect.");
			perfTest.testFunction(hitTestCircle, 10000, "hitTestCircle", "Check if bounding circles intersect.");
			perfTest.testFunction(checkBitmapDataHit, 10000, "checkBitmapDataHit", "Uses BitmapData.hitTest to check for collision.");
			perfTest.testFunction(checkBitmapDataHitReverse, 10000, "checkBitmapDataHitReverse", "Uses BitmapData.hitTest to check for collision. Instances reversed.");
			perfTest.testFunction(checkBitmapDataHitInternal, 10000, "checkBitmapDataHitInternal", "Uses BitmapData.hitTest to check for collision. BitmapData not cached.");
			perfTest.testFunction(checkBitmapDataHitDynamic, 10000, "checkBitmapDataHitDynamic", "Uses BitmapData.hitTest to check for collision. BitmapData not cached. MovieClips dynamic.");
		}

		private function checkHitTest():Boolean
		{
			return clip1.hitTestObject(clip2);
		}

		private function checkHitTestReverse():Boolean
		{
			return clip2.hitTestObject(clip1);
		}

		private function boundsIntersection():Boolean
		{
			return clip1.getBounds(this).intersects(clip2.getBounds(this));
		}

		private function hitTestCircle():Boolean
		{
			var dx:Number = (clip2.x + clip2.width / 2) - (clip1.x + clip1.width / 2);
			var dy:Number = (clip2.y + clip2.height / 2) - (clip1.y + clip1.height / 2);
			var dist:Number = Math.sqrt(dx * dx + dy * dy);

			return (dist < ((clip1.width / 2) + (clip2.width / 2)));
		}

		private function checkBoundsManually():Boolean
		{
			if(clip1.x < clip2.x + clip2.width &&
			   clip2.x < clip1.x + clip1.width &&
			   clip1.y < clip2.y + clip2.height &&
			   clip2.y < clip1.y + clip1.height
			   )
			{
				return true;
			}

			return false;
		}

		private function checkBitmapDataHit():Boolean
		{
			point1.x = clip1.x;
			point1.y = clip1.y;

			point2.x = clip2.x;
			point2.y = clip2.y;
			if(clip1ClipBmpData.hitTest(point1,
										255,
										clip2ClipBmpData,
										point2,
										255

								  ))
			{
				return true;
			}

			return false;
		}

		private function checkBitmapDataHitInternal():Boolean
		{
			var _clip1Rect:Rectangle = clip1.getBounds(this);
			var _clip1ClipBmpData:BitmapData = new BitmapData(_clip1Rect.width, _clip1Rect.height, true, 0);
			_clip1ClipBmpData.draw(clip1);

			var _clip2Rect:Rectangle = clip2.getBounds(this);
			var _clip2ClipBmpData:BitmapData = new BitmapData(_clip2Rect.width, _clip2Rect.height, true, 0);
			_clip2ClipBmpData.draw(clip2);	

			point1.x = clip1.x;
			point1.y = clip1.y;

			point2.x = clip2.x;
			point2.y = clip2.y;

			var hits:Boolean = false;
			if(_clip1ClipBmpData.hitTest(point1,
										255,
										_clip2ClipBmpData,
										point2,
										255
								  ))
			{
				hits = true;
			}

			_clip1ClipBmpData.dispose();
			_clip2ClipBmpData.dispose();

			return hits;
		}

		private function checkBitmapDataHitDynamic():Boolean
		{
			var dynamicClip1Rect:Rectangle = dynamicClip1.getBounds(this);
			var dynamicClip1ClipBmpData:BitmapData = new BitmapData(dynamicClip1Rect.width,
																	dynamicClip1Rect.height, true, 0);
			dynamicClip1ClipBmpData.draw(dynamicClip1);

			var dynamicClip2Rect:Rectangle = dynamicClip2.getBounds(this);
			var dynamicClip2ClipBmpData:BitmapData = new BitmapData(dynamicClip2Rect.width,
																	dynamicClip2Rect.height, true, 0);
			dynamicClip2ClipBmpData.draw(dynamicClip2);	

			point1.x = dynamicClip1.x;
			point1.y = dynamicClip1.y;

			point2.x = dynamicClip2.x;
			point2.y = dynamicClip2.y;	

			var hits:Boolean = false;
			if(dynamicClip1ClipBmpData.hitTest(point1,
										255,
										dynamicClip2ClipBmpData,
										point2,
										255
								  ))
			{
				hits = true;
			}

			dynamicClip1ClipBmpData.dispose();
			dynamicClip2ClipBmpData.dispose();

			return hits;
		}

		private function checkBitmapDataHitReverse():Boolean
		{
			point1.x = clip1.x;
			point1.y = clip1.y;

			point2.x = clip2.x;
			point2.y = clip2.y;	

			if(clip2ClipBmpData.hitTest(point2,
										255,
										clip1ClipBmpData,
										point1,
										255

								  ))
			{
				return true;
			}

			return false;
		}

		private function out(str:*):void
		{
			trace(str);
		}
	}
}

You can download the code from here (requires Flash Pro and Grant Skinner’s Performance Testing Harness).

And here are the raw results:

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkHitTest (10000 iterations)
Uses DisplayObject.hitTest
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                    8     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkHitTestReverse (10000 iterations)
Uses DisplayObject.hitTest with clips reversed
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                    7     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
boundsIntersection (10000 iterations)
Uses Rectangle.intersects()
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                   33     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkBoundsManually (10000 iterations)
manually checks if bounds intersect.
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                   18     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
hitTestCircle (10000 iterations)
Check if bounding circles intersect.
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                   22     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkBitmapDataHit (10000 iterations)
Uses BitmapData.hitTest to check for collision.
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                    7     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkBitmapDataHitReverse (10000 iterations)
Uses BitmapData.hitTest to check for collision. Instances reversed.
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                    7     0.00
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkBitmapDataHitInternal (10000 iterations)
Uses BitmapData.hitTest to check for collision. BitmapData not cached.
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                 6332     0.63
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
checkBitmapDataHitDynamic (10000 iterations)
Uses BitmapData.hitTest to check for collision. BitmapData not cached.
Dynamic MovieClips
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
method...................................................ttl ms...avg ms
[function]                                                 1892     0.19
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

Most of the results are as expected, although there is one surprise.

Doing a BitmapData.hitTest on a MovieClip placed onto the stage at author time (in Flash Pro) is 3 times slower than testing against a MovieClip generated and drawn at runtime. The only thing I can think that is happening is that perhaps for some reason the garbage collector is being triggered in the checkBitmapDataHitInternal and not the checkBitmapDataHitDynamic test. I have asked around internally what might be causing it, but I would be curious if anyone else is seeing the same results.

So, some quick conclusions from this:

1. DisplayObject.hitTest is a fast way to do a quick boundaries check.
2. Having to dynamically generate BitmapData for BitmapData.hitTest leads to a big performance hit.
3. If you can cache the BitmapData being compared, then BitmapData.hitTest performance is on par with the other techniques / APIs tested. However, performance degrades as the dimensions of the clips increase (see here).
4. Getting the BitmapData from a MovieClip that has cacheAsBitmap = false is 3 times slower than getting it from a MovieClip with the property set to true (Thanks to Renaun Erickson who figured out the performance discrepancy).

I will update the post and results as I learn more techniques and get more information.

If you find any bugs with the test, or would like to add some more tests (such as a test using BitmapData.getColorBoundsRect), just post them in the comments.

Updated : July 6, 2009 : Added info on cacheAsBitmap property impact on performance. (See this comment).

Updated : November 6, 2009 : Added optimized circle test in comments : Approaches performance of hitTest.

Yesterday I blogged about how you can use the BitmapData.hitTest API to do collision detection between the visible parts of multiple DisplayObject instances. This works very well, but as some of the BitmapData apis can be cpu intensive (particularly new BitmapData and BitmapData.draw) you have to take care to make sure that performance does not get out of hand.

This post will discuss a number of approaches for optimizing collision detection when using BitmapData.hitTest.

Check boundary boxes first

The first thing to do is to check whether you need to check for visible collision detection in the first place. You can do this by checking whether the bounding boxes of the two DisplayObjects overlap, and only if they do, then use BitmapData.hitTest to see if their visible areas also overlap.

To do this, just use the DisplayObject.hitTestObject API like so:

if(displayObject1.hitTestObject(displayObject2))
{
	//do check with BitmapData.hitTest
}

When checking collisions against many items, this can dramatically improve performance, as it can remove most of the BitmapData api calls and comparisons.

Cache BitmapData

If one or more of your display objects will not have any transformations applied to them (such as rotation), then you can generate the BitmapData once, and cache it for future comparisons. Basically, you are trading improved CPU performance, for higher memory usage (since you keep the BitmapData in memory).

Here is a simple example:

private var bmpData:BitmapData;

private function checkCollisions():void
{
	if(!bmpData)
	{
		var blueRect:Rectangle = blueClip.getBounds(this);
		bmpData = new BitmapData(blueRect.width, blueRect.height, true, 0);
		bmpData.draw(blueClip)
	}

	//use bmpData.hitTest to check collisions
}

Furthermore, if you use multiple instances of a DisplayObject, then you can store the BitmapData for one instance, and use it for all instances of the DisplayObject. One thing I do, is the store it by the class type, which allows me to easily cache and retrieve BitmapData for multiple DisplayObject classes (I have custom classes that extend DisplayObject).

private var bmpDataLookup:Dictionary = new Dictionary();

private function checkCollisions():void
{
		var classRef:Class = blueClip["constructor"] as Class;

		bmpData = bmpDataLookup[classRef];

		if(!bmpData)
		{
			//run once per lifetime of app per DisplayObject subclass
			var blueRect:Rectangle = blueClip.getBounds(this);
			bmpData = new BitmapData(blueRect.width, blueRect.height, true, 0);
			bmpData.draw(blueClip)
		}

		//use bmpData.hitTest to check collisions
}

This example caches the BitmapData once for each Class type. i.e. if you have a lot of instances of a UFOClass (which extends DisplayObject), you would only have to cache one instance of BitmapData regardless of the number of instances of the UFOClass you had.

Again though, this only really works if your DisplayObjects will not have any transformations applied to them (since otherwise, their bounding boxes may change).

Set Thresholds for Caching

Even if your DisplayObjects will have transformations applied to them, you can still improve performance by only updating the BitmapData if the transformation has changed more than some threshold. Essentially, you can trade some hit detection accuracy, for better performance.

For example, in a game I am working on right now, I have a Ship that rotates based on the user input. I need to check if the ship collides with any enemies. Since the ship may rotate, I can’t use the technique above to cache the BitmapData. However, after some testing and profiling, I realized that I could cache the BitmapData and only update it if the rotation changes by more than 5. This means that the hit detection is slightly less accurate, but I can use cached BitmapData for many of the checks. In my case, I was able to remove 2/3rds of the BitmapData calls (basically use cached data 2/3rds of the time), without any noticeable change in ht detection accuracy.

private var shipBmpData:BitmapData;
private var oldShipHash:int = 0;

private function checkCollisions():void
{

	var shipBounds:Rectangle = ship.getBounds(this);	

	var shipBoundsHash:int = ship.rotation;

	//basically, we check the rotation to see if it has changed much
	//if it hasnt, we just use the bitmapdata from earlier frame(s)
	if(!shipBmpData ||
		shipBoundsHash < oldShipHash - 5 ||
		shipBoundsHash > oldShipHash + 5)
	{
		shipBmpData = new BitmapData(shipBounds.width, shipBounds.height, true, 0);

		//this shouldnt work in some cases because of the x/y changes
		//but collision detection seems to be working ok
		var shipOffset:Matrix = ship.transform.matrix;
		shipOffset.tx = ship.x - shipBounds.x;
		shipOffset.ty = ship.y - shipBounds.y;			

		shipBmpData.draw(ship, shipOffset);
	}

	oldShipHash = shipBoundsHash;

	//do collision detection with BitmapData.hitTest

}

Depending on the accuracy needed, this can provide large performance improvements.

Reuse Bitmap Data Instances

Another option is to reuse BitmapData instances, and use fillRect to reinitialize them before drawing new data in them. This removes the need to create a completely new BitmapData instance, but only works if the dimensions of your bonding box wont change. I wont go into detail on this technique (you can find a good description here), but in most cases where this could be useful, its probably better to just cache the BitmapData as discussed above.

Choose the right collision detection technique / API

Finally, make sure you are choosing the correct collision detection technique. For example, do you really need pixel perfect collision detection? Could you get away with just checking distances between items, or using some of the other existing APIs?

For example, if the visible area of your DisplayObjects fill most of there bounding boxes, then you could probably get away with just using DisplayObject.hitTestObject. Again, you trade some accurate for performance, but in a lot of cases, the user will not notice.

Is the visible areas of your DisplayObject mostly round? If so, you maybe be able to just check the distances between objects to test for collision.

The main point is that maybe BitmapData.hitTest is overkill for your needs.

Look at other collision detection optimizations

There are a lot of other general techniques for optimizing collision detection, many of which focus on reducing the number of tests needed. For example, you can divide your stage into a grid, and only check items that are in adjacent grids.

Many of these more general optimization techniques also apply when using BitmapData.hitTest, and you should explore them when optimizing your content.

Please post any other optimization techniques or suggestions in the comments.

The Flash Player contains a number of APIs for handling collision detection within Flash content. The DisplayObject class contains hitTest and hitTestPoint which can be useful if you need to detect bounding box collisions, or detect collisions between an individual point and bounding boxes or shapes.

However, BitmapData also contains a hitTest API, which can check collisions on BitmapData. Where the API really shines, is when you need to detect collisions between the visible areas of DisplayObjects (and not just of their bounding boxes). The API contains functionality for testing collisions between BitmapData and a Point, BitmapData and a Rectangle, and BitmapData and another BitmapData. It is the last item that I will focus on in this post.

Since you can get BitmapData from a DisplayObject the API can be used to detect very exact collisions between the visible areas of DisplayObjects, even if the DisplayObjects have irregular shapes.

Here is a simple example that uses the API to detect collisions between the visible areas of two MovieClips using BitmapData.hitTest.

And here is the code

var redRect:Rectangle = redClip.getBounds(this);
var redClipBmpData = new BitmapData(redRect.width, redRect.height, true, 0);
redClipBmpData.draw(redClip);

var blueRect:Rectangle = blueClip.getBounds(this);
var blueClipBmpData = new BitmapData(blueRect.width, blueRect.height, true, 0);
blueClipBmpData.draw(blueClip);

addEventListener(Event.ENTER_FRAME, enterFrame);

function enterFrame(e:Event):void
{
	blueClip.x = mouseX;
	blueClip.y = mouseY;

	if(redClipBmpData.hitTest(new Point(redClip.x, redClip.y),
								255,
								blueClipBmpData,
								new Point(blueClip.x, blueClip.y),
								255

						  ))
	{
		trace("hit");
		redClip.filters = [new GlowFilter()];
	}
	else
	{
		redClip.filters = [];
	}
}

Basically, the content contains two MovieClips on the timeline. We draw the graphics for each MovieClip into a BitmapData instance, and then use the BitmapData.hitTest API to see if the visible parts of the MovieClip (non-transparent) instances collide.

Couple of notes on the example:

1. The initial fill for the BitmapData is transparent pixels. This is necessary to detect just the visible areas of the DisplayObject.

2. The Point instances passed to hitTest are used to align the BitmapData instances for the comparisons.

This example works regardless of the contents or shape of the DisplayObjects. However, if either of the DisplayObjects have had any transformations applied to them (such as rotation), then the collision detection wont work correctly.

In order to get it to work, you need to apply a Matrix when drawing the BitmapData to account for the transformation applied to one or both DisplayObjects. If you haven’t worked with Matrices and DisplayObject transformations, this can be a little daunting at first (it was for me). If you are new to using Matrices, I suggest reading this excellent article on Matrices in Flash over at senocular.com.

Luckily for me, Trevor McCauley, who runs senocular.com, also happens to works for Adobe on the Flash Player team. He really helped me understand how how to get the hit detection to work when the DisplayObjects have had transformations applied to them.

Here is the example:

and the code:

addEventListener(Event.ENTER_FRAME, enterFrame);

function enterFrame(e:Event):void
{
	blueClip.x = mouseX;
	blueClip.y = mouseY;	

	redClip.rotation++;

	var blueRect:Rectangle = blueClip.getBounds(this);
	var blueOffset:Matrix = blueClip.transform.matrix;
	blueOffset.tx = blueClip.x - blueRect.x;
	blueOffset.ty = blueClip.y - blueRect.y;	

	var blueClipBmpData = new BitmapData(blueRect.width, blueRect.height, true, 0);
	blueClipBmpData.draw(blueClip, blueOffset);		

	var redRect:Rectangle = redClip.getBounds(this);
	var redClipBmpData = new BitmapData(redRect.width, redRect.height, true, 0);

	var redOffset:Matrix = redClip.transform.matrix;
	redOffset.tx = redClip.x - redRect.x;
	redOffset.ty = redClip.y - redRect.y;	

	redClipBmpData.draw(redClip, redOffset);	

	var rLoc:Point = new Point(redRect.x, redRect.y);
	var bLoc:Point = new Point(blueRect.x, blueRect.y);	

	if(redClipBmpData.hitTest(rLoc,
									255,
									blueClipBmpData,
									bLoc,
									255
						  		))
	{
		trace("hit");
		redClip.filters = [new GlowFilter()];
	}
	else
	{
		redClip.filters = [];
	}

	blueClipBmpData.dispose();
	redClipBmpData.dispose();
}

Note that we use the bounds of each DisplayObject to determine the BitmapData size and not the height / width of the DisplayObject. This is because the transformation will most likely modify the bounds, and we need to take this into account. In the first example, we could have just used the height and width of the DisplayObjects since they were the same as the bounds, but in general you should get in the habit of using the bounds.

We also pass a Matrix to each BitmapData.draw call to account for any transformations that may have been applied to the DisplayObjects (in this case rotations). I could try and explain what the Matrix is doing, but to be honest, im still trying to get my head around it. Instead, Ill let Trevor explain it (from an email he sent to me):

For this you need the transformation of the object as defined by DisplayObject.transform.matrix. Since everything is in the same container, we still won’t have to worry about walking up hierarchies to get contcatenated/global matrices – we can just use that which is directly applied to the target object (transform.matrix). This gives you the full transform of that object. But for the purposes of capturing a bitmap, we only want the non-translation parts of that transform. This is because when a bitmap is captured, it’s captured from the 0,0 location in the coordinate space of the captured object outward into positive space. The translation of the matrix of the desired object is its position in its parent coordinate space, not its own. HOWEVER, if that object’s own 0,0 location is within the middle of its graphic elements, drawing it into a Bitmap directly from 0,0 could cause cropping. So in terms of that translation as it is to be used with BitmapData.draw(), it’s still important to make sure everything in the target object is captured. And in doing that, it means shifting everything that would appear in negative coordinate space over into positive coordinate space.

Luckily, using getBounds (again) this is quite simple. We can use getBounds to determine the bounds of an object in its parent coordinate space – this includes its transformations. We also know the location of 0,0 within the parent coordinate space because that is the same location it places the target object using its x and y properties. With that, we can get the needed bitmap translation by simply subtracting the bounds x,y from the object’s own x,y. In code that essentially becomes

var b = t.getBounds(t.parent);
var m = t.transform.matrix;
m.tx = t.x – b.x;
m.ty = t.y – b.y;
var bmp = new BitmapData(b.width, b.height, true, 0);
bmp.draw(t, m);

where ‘t’ is the target display object.

Now, I will be the first to admit that I am still wrapping my head around some of this stuff, especially with the use of the Matrices to offset the transformation. However, the second example above is generic and robust enough to be used for general pixel perfect collision detection on visible portions of DisplayObjects.

One note, is that the examples above assumes that the items being compared share the same parent. If they don’t you need to make some additional adjustments (to take into consideration other transformations), but that is a post for another day.

Btw, if you don’t already, you need to read senocular.com. This is one of the best resources on understanding ActionScript 3 and the Flash Player.

Update : One optimzation which you can make is to first check if the bounding boxes of the DisplayObjects overlap (using DisplayObject.hitTestObject), and only if they do, use BitmapData.hitTest to see if their visible areas are touching.