I have just completed running all of the performance tests for the ActionScript 3 Development Task Contest #1.

The result below are tentative until I have confirmed that the code follows the rules (and spirit) of the contest. If you see any issues with any of the top 5 submissions, please post them in the comments. You can view all of the entries in the contest’s GitHub repository.

Here are the results:

You can download the raw results in CSV format from here, or view them on google docs here.

All examples were compiled with MXMLC from the Flex SDK 3.4 (Version 3.4.0 build 9271), with the following command:

mxmlc --target-player=10.0.0 -compiler.source-path ~/src/PerformanceTest/ -- TestRunner.as

Results were obtained by running TestRunner.as. TestRunner.as ran each test 300 times per run. The TestRunner was run a total of 5 times (with the SWF reloaded each time).

Test were run in Safari Version 4.0.4 (6531.21.10) on Mac 10.6.2 in Flash Player 10,0,32,18 (regular). The Mac was a MacPro 2 x 3 GHZ Quad Core Intel Xeon with 8 GB RAM.

I will finalize results and announce the winner either late tonight, on Wednesday (I am traveling on Tuesday). Check back here for updated information, or follow my twitter account at @mesh. I will be awarding prizes to multiple submissions.

UPDATE : (November 17th, 2009) : I am declaring the entries final. Arnaud Gatouillat is the overall winner, and wins Flex Builder, a tshirt, etc…. In addition, given some of the great results that we got from the submissions, I am also going to award a prize (their choice of a tshirt from threadless.org) to the next 3 top entries, which include Grant Skinner, Jim Cheng, and Jean Phillipp Auclair. If you are one of the top four entries, please email me at mesh@adobe.com and we will work out the details.

I have uploaded all of the entries for the ActionScript 3 Development Task Contest #1 to the contest’s GitHub repository.

If you sent me an entry and do not see it in the repository, email me ASAP (without the file). Adobe’s spam system can be a bit overzealous at times, but if I know you sent me something, I can find it.

Also, I have not validated all of the entries yet. I will do that later today or over the weekend. If you see any major issues in a submission, then post them in the comments (do not email me directly about it). My plan is to post the initial test results on Monday, and then finalize Monday night or Tuesday. However, I am traveling early next week, so that might slip a day.

All of the source code is fully commented, so please feel free to ask question, or comment on general techniques in the comments for this post. I think it is a great opportunity to learn from the code, and different approaches. My guess is that we can get something even faster by combining some techniques used by different submissions.

Note that I also submitted my entry, although I am not eligible for the contest. Also, I did not update mine any once I started receiving submissions.

Btw, if someone wants to modify the TestRunner to run all of the entries at once, then I will run it on multiple platforms and devices (including iphone), and report back the results.

Finally, thank you to everyone who submitted. When commenting, please keep in mind that there are various levels of developers who submitted content, so no “j00r (0D3 $U><0r$” comments.

Well, there is just a little over a day before the deadline for entries to the ActionScript Development Task Contest (AS3DTC) are due. I wanted to make a quick post with a couple of reminders to ensure everyone has their correctly working submissions in on time.

First, a couple items:

1. Make sure to include your name for the copyright in the ProximityManager.as file (replace my name at the top). This is really important, as you have the copyright, and it makez it easier for me to know who the submission is from.
2. Run your submissions again against the latest test files. There have been some minor changes to the test, so make sure they dont affect your results.
3. Test that you are returning valid results. See below for tips about how to do this.
4. Read through the comments in the original contest post, especially the comments that I made. There are some rule clarifications that are important.
5. Make sure you are using testing with the latest contest files. TestRunner.as has had some updates.

Regarding #4, here are some of the rule clarifications:

• For the purposes of the contest, you should use the x, y coordinates of the display objects to determine which cell they are contained within (i.e. you don’t have to worry about a display object straddling two cells).
• Basically, the only information your class has access to is that which is passed to its constructors and methods. Your code cannot make assumptions based on information in the test which is not passed to the ProximityManager class (i.e., how many items there are on the stage, or the fact that the items are not moving, etc…). If I change the parameters of the test that your code does not know about, then your class should still work correctly.
• The bounds argument is now required in ProximityManager constructor. This specifies the bounds of the collision detection / grid area.
• If the positions of the DisplayObjects change, update will be called again. Thus, when update is called, the code should treat it as new data.
• You must return a new Vector from getNeighbors. (You can’t re-use the same Vector instance).

If you have any additional questions about the rules, then please post them on the original post.

Validating Results

Inside of TestRunner.as, there is a checkResults method, which you can use to validate your results.

To run it, uncomment the runTests() method in onStageAdded, and and the add this code:

private function onStageAdded(e:Event):void
{
	...

	//runTests();

	proximityManager.update(items);
	checkResults(checkSprite_3, proximityManager.getNeighbors(checkSprite_3));
}

This will return something that should similar to this:

This test ensures that you are not missing any DisplayObjects which you should return.

To ensure you are not including any DisplayObjects which you should not be, change the for each loop in checkResults to:

private function checkResults(checkSprite:Sprite, items:Vector.<DisplayObject>):Boolean
{
	...

	for each(var disp:Sprite in items)
	{
		disp.graphics.beginFill( 0x0000FF , 1 );
		disp.graphics.drawCircle( 5 , 5 , 5 );
		disp.graphics.endFill();
		disp.alpha = 1;
	}

	...
}

You should not see any dark spots appear.

I have gone through and done an initial validation all of the submissions thus far. There were a couple of entries which were not returning the correct results, and I have emailed those developers. I also did a quick test run on my Mac Book Pro (2.4 Ghz Intel Core 2 Duo), and in general, right now, the bar to reach is an average of 2 – 3 ms per test run. Most submissions use take the same basic approach, but some have some interesting twists. I think there is still a opportunity for someone to take a novel approach and maybe jump ahead.

Remember, contest entries have to be received by me (mesh@adobe.com) by 12 Noon, PST, Friday, November 13.

I was working on some code over the weekend and was spending a lot of time trying to figure out the best way to approach a problem, and how to get the best performance from the solution. I thought it would be useful and interesting to see how other developers would approach the problem. This gave me an idea for an ActionScript 3 contest, which I am announcing today.

What: ActionScript 3 Development Task Contest (AS3DTC)
When: Submissions must be received by Mike Chambers (mesh@adobe.com) by 12 Noon, PST, Friday, November 13th.
Who: Anyone is eligible (see details below)
Prizes : Copy of Flex Builder 3, choice of a T-shirt from threadless.org, lots of random swag, and the admiration and adulation of your ActionScript developer peers.

Goals of contest

1. Get some solid implementations of classes that will be useful to everyone.
2. Get multiple examples of high performance code that everyone can learn from.
3. Drive discussions around implementations, and possible further optimizations (code and architectural).

Keep these goals in mind when writing your code and making your submission.

Contest Task

Implement a class named ProximityManager that divides the stage into a grid with cells sizes of 35px x 35px, and returns all of the display object instances within the same and adjacent cells as a specified display object (that will be passed in).

I have provided a TestRunner.as class that will be used to judge the contest. You cannot change this file. Your entry will consist of a class called ProximityManager.as that implements the constructor and public APIs in the included ProximityManager.as.

Update (November 11, 2009) : For the purposes of the contest, you can use the x, y coordinates of the display objects to determine which cell they are contained within (i.e. you dont have to worry about a display object straddling two cells).

The TestRunner will randomly place 10,000 display objects on the stage. It will then randomly place four target Sprites (checkSprites) instances on the stage.

The test that will be run is:

private function testProximityManager():void
{
	proximityManager.update(items);
	proximityManager.getNeighbors(checkSprite_1);
	proximityManager.getNeighbors(checkSprite_2);
	proximityManager.getNeighbors(checkSprite_3);
	proximityManager.getNeighbors(checkSprite_4);
}

Where items is a Vector of all DisplayObjects to be checked against, and checkSprite is the target Sprite, whose neighbors must be found and returned in a Vector of DisplayObjects.

Tests will be run and results determined using Grant Skinner’s ActionScript 3 Performance Test Harness.

Download Contest Files

All test files and entries will be posted to the ActionScript 3 Development Task Contests GitHub Repository.

Additional Rules

• Code must be release under an MIT license.
• Code must be fully commented.
• Code must be human readable.
• Code must implement the constructor and public APIs defined in the included ProximityManager.as file.
• Code must be implemented in the provided ProximityManager.as class.
• Contest is open to everyone.
• Code must be authored by the submitter.
• Code cannot have any external dependencies (including Flex).
• Code cannot require any additional tools, pre or post processing steps.
• All entry code must include the MIT license at the top, as well as the name of the developer who wrote and is submitting the code.
• Developers may enter more than once, but only entries that take a significantly different approach will be accepted.

Note, that the rules and TestRunner may be tweaked after I post them, but I will try and make any changes or clarifications within 1 day. Please make sure to check back to this post for any changes.

Don’t try to game the contest. If you are not sure if you are, then ping me. This is a community contest for fun and learning. I will make the final call on all disputes and entries. If I think an entry doesn’t abide by the rules, or is trying to game the contest then I may reject the entry. I am very serious about this, so dont mess around.

Update (November 10, 2009) : Basically, the only information your class has access to is that which is passed to its constructors and methods. Your code cannot make assumptions based on information in the test which is not passed to the ProximityManager class (i.e., how many items there are on the stage, or the fact that the items are not moving, etc…). If I change the parameters of the test that your code does not know about, then your class should still work correctly.

The winning entry will be determined by compiling the TestRunner.as (included) and submission class with the latest public release of the mxmlc compiler. The TestRunner SWF will be run 5 times and total times will be combined. The submission with the lowest total time over the test iterations will be the winner. The return values from the getNeighbors API will be validated (an example validation test is included in the TestRunner).

Tests will be run on the latest Flash Player 10 Release (non-debug) version in the Safari browser (latest release version) on an Intel Mac Pro. All submissions will be posted online.

Once the results are announced, ranked, and posted, there will be a short period before the contest is finalized. This will give the community a chance to identify any bugs in the code logic. I will make the final decision about whether any bugs in the logic will disqualify the entry. Basically, if it is a minor bug that will not affect performance, then I will let it pass. If it is a major bug, or has a major impact on performance, then I will probably disqualify the entry. I may give the developer a chance to correct the bug.

Since one of the main goals of the contest is to help generate knowledge on techniques and optimizations, the contest is open to everyone, including Adobe employees. However, if an Adobe employee wins, the prizes will go to the highest ranked non-Adobe employee.

This is the first time I have tried this, so there will be some kinks in the process. Please be patient, and keep the goals and spirit of the contest in mind. If it goes well, then we will do more contests, with different tasks.

If you find any errors / bugs in the TestRunner class, please post them in the comments, and I will get them fixed.

If you have any questions, or want clarification on something, post it in the comments. All clarifications made by me in this post and its comments should be considered part of the rule set.

Watch this blog post, and my twitter account (@mesh) for contest updates and reminders.

File Updates:

Release 0.7 (November 12, 2009)

• TestRunner now waits one second after it has loaded before it runs the performance tests.

Release 0.6 (November 12, 2009)

• checkResults now removes all result items from the stage. This makes it easier to view and validate results

Release 0.5 (November 12, 2009)

• All circles are now drawn so the top left of their bounds is at 0,0.

Release 0.4 (November 11, 2009)

• bounds argument is now required in ProximityManager constructor. This specifies the bounds of the collision detection / grid area.

Release 0.3 (November 11, 2009)

• Fixed Release dates in README.txt
• Added visual validation code to TestRunner.as (thanks to Sean Christmann for the help)
• Made a minor change to how circles are drawn in Display Objects on stage.
  From : disp.graphics.drawCircle( 0 , 0 , 5 );
  To : disp.graphics.drawCircle( 5 , 5 , 5 );

Release 0.2 (November 10, 2009)

• Move SWF meta data from ProximityManager to TestRunner
• Added README.txt file

Release 0.1 (November 10, 2009)

• Initial Release