At Infrared5, one thing we do to give us a good boost of speed in Flash is using a technique called (at least by me) Bitmap Injection.  The idea behind bitmap injection is to swap out pre-rendered frames of an animated clip into a bitmap object.  This removes the extra draw call of a MovieClip, thus speeding things up.  Since you can simply change the bitmapData property of a Bitmap, it simply reassigns what it is drawing, rather than having to copyPixels or do any of those performance killers.  Since we do it at work, and since Papervision is driven by bitmapData objects - I decided to whip up a quick material which could save your life.

A normal MovieMaterial, when set to animated, redraws the bitmap it is using for every frame of your rendering.  As you can imagine, this costs serious CPU, and if you have too many animated materials, your application will slow to a crawl.

So, I wrote up MovieCacheMaterial.  The name says it all really.  The material draws every frame of the movie into a global array for that particular asset.  Then, if the movie is ever on that same frame again, instead of redrawing the movie, it simply draws the 3D content using the already cached bitmap.  Then, any other DisplayObject3Ds using that same MovieClip will use the same cache as well.  So, your performance on the first loop will be the same as a normal MovieMaterial, but after that, the performance is equivalent to a normal BitmapMaterial - even though there is animation on it!

Check out the Demo

I wrote a hack work-around for the first loop issue, which might be of use to you.  You can call material.ripMovie(stage, frameRate, completeHandler).  You pass in the stage, how fast you want the movie to be ripped into the cache, and what function to call when it is complete.  The material will set your stage FPS to your desired number, then when it has ripped every frame of the MovieClip that was assigned to it, it will call the function you pass in as completeHandler.  This will make it so when you start your application you can have the best performance right off the bat.  Obviously it will take it longer to start up, however, since it has to rip your animation before beginning.

You are amazing Andy - What’s the catch?

Unfortunately, there are some limitations to using this material.  The first limitation is that it is a memory hog.  Since you draw every frame of your Movie into memory, that much more memory is going to be used.  To give you an idea of how much that equals:  the animation in the demo above is ~150 frames, and is 512×512.  When it is fully parsed it uses about 140 megs of memory.  Smaller movies will obviously use less.  It is definitely something you need to be aware of however.

Second limitation: it doesn’t work with real time dynamic movies or interactions.  Remember, it simply draws every frame of a movieclip.  That content can be dynamic the first loop, but after that, you are going to see exactly what happened the first time over and over again.  This obviously rules out any Virtual Mouse behaviors on a movie as well.  But, if you have a simple animation you want to loop - go for it!

And In the End

One other final thing to point out is that you can control your animation by changing your currentFrame on the MovieClip just like normal.  It won’t mess up the caching, so you can pause, play, skip, etc. just like normal.  In the demo, you can press “F” to jump to the middle of the animation - using a gotoAndPlay.

Anyways - thats about all there is to say about it.  You can see the performance difference on the demo.  For me, on 49 planes, i get about 25 FPS with MovieCacheMaterial, and about 5 FPS with regular animated MovieMaterial.

See the Demo Again

and

Get the Source

Feel free to report any bugs.  I probably won’t fix any, but it might help others :)  Also, you might have noticed I bolded certain words throughout the post.  If you can guess why I will give you a free copy of my upcoming top secret application (still a little ways off).

Enjoy!

25 Comments | In: BitmapData, as3, papervision | | #

Ha!  Honestly, the title was just to get your attention.  But to answer your next question, no, there isn’t a single line of Papervision in this demo (requires Flash Player 10).  This is, in fact, my first real test with Pixel Bender (and integration with Flash).  I had written one or two a few months ago the day it was released, but this was something that I had hoped would change the way I do some things in Flash.  It was inspired, (and some projection stuff borrowed) by the raycasting work of mike welsh, who made this great raycasting shader, which handled reflections, etc.  While it runs great in the toolkit, it can’t work in Flash (but even if it did you would only get about 1 FPS).  What I hoped would be a fast and elegant solution for a client project turned out to simply be a cool little experiment.  Not much application in the real world I don’t think - at least not till Flash gets some GPU support for it.

So, what did I learn?

1. Normalize doesn’t work in Flash - though it does in the toolkit.  I spent about 4 hours tracking down this nightmare.  My project worked great in the AIF Toolkit, but once I brought it over to Flash, the shading turned into a ghoulish wash.  Turns out that the Flash Player doesn’t read normalize.  Instead, you need to do something like this:

float3 vectorNorm = vector/ length(vector);

Or of course, you just do a /= to the original vector.

2.  No Debugging. This is always a struggle when making a shader.  The way you can debug is by simply outputting colors instead of debug statements.  I did end up making some very cool patterns in my filter by outputting various properties as color - but in the end it would be nice to have a simple trace statement.  For example, instead of doing:

out = sampleNearest(src, outCoord())

You can do

out = pixel4(norm.x, norm.y, norm.z, 1.0);

Then the image itself becomes a debug view of sorts.  Its a little less informative than a trace (though it does show trends well), but it helps tremendously when working with shaders.

3.  CPU != GPU.  Running in CPU mode flipped some normals on me.  It worked right in GPU and in Flash.  I read (somewhere) you should test in both to make sure it works all around.  Not sure if this means there will be problems on different computers or not…

4.  Pixel Bender isn’t nearly as fast as I wanted.  Sorry… its true.  It sucked going from the 800 FPS Toolkit to the 165% CPU destroyer (dual-core).  Turning the window mode to GPU took about 15% off the CPU, but it still wasn’t the performance I was hoping PIxel Bender would bring to the Flash Player.  I found the biggest thing that affected performance was simply how big the image was you were applying the filter too.  Apparently alot of math is faster in AS3 than in Pixel Bender (CPU).  I guess one solution would be to pass in more parameters with some of the math already taken care of.

Some Info About the Demo

I guess first thing to point out is that I use two input images.  src is first input image4, and the one that is automatically passed as an input to the shader.  This image4 is simply the object you are applying the shader too.  The second input, texture, is the texture we want to use for the mapping.  By using two images, we don’t have to process every pixel of the texture mapped, if we only want a 512×512 sphere.  In this demo the src image is only 350×350 - much better performance than running it on 1000×500.

Next, the performance sucks.  Well, to be fair, it runs okay - but it won’t run on older computers at all, and runs at max CPU on the newer ones.  I have tried some things to speed this up, such as taking the rotation matrix outside the filter and passing it in, but the change was negligible (I didn’t see a difference).  So, I opted to create the matrix in the filter per pixel and just keep the filter as self-contained as possible.  If anyone has some tips/tricks to make this filter run faster - let me know!

I’m happy with how it looks - sorry for some of the UV code in the shader itself - in order to remove some precision errors, I had to swap the side axis used for determining rotation depending on the location of the hit.  Makes things look better, though code looks a little worse.

Anyways:

Check out the Demo

and

Get the Source (once again, requires Flash CS4 and Flash Player 10)

Enjoy

16 Comments | In: BitmapData, Flash 10, as3, effects | | #

This is just a new experiment I made testing how well the pixel3d classes could stand up as a true particle system.  I was happy with the result.   Click to change particle stream color.

It grows 6,000 particles, has 2 fx layers, 3 total effects applied at 600x500 (no clipping).  I had it running with 6 gravity spheres floating around pushing and pulling the pixels, but all it did was lessen the effect (so I took them out).

See it here

(you can see gravity only stuff going on at a semi-done experiment here)

 [update]

I've uploaded a new experiment with particle color controlled by sound.  Check it out.

[/update]

17 Comments | In: BitmapData, as3, effects, papervision | | #

I just finished uploading a new branch to the Papervision repository.  If you hadn't noticed the slew of new files, take a look in /branches/Effects.  Its an API built into Papervision3D that I have been developing which allows developers to apply filters and other effects to their 3D objects.   I did a class on these techniques recently in NYC with John.  It seemed to go well and teach the main points of the API, so here is an online version of what was taught.

This demo involves a Borg cube collada - the finished source of which you can get here or at the end of this tutorial:

...

101 Comments | In: BitmapData, as3, effects, papervision | | #

If you haven't seen the Qwest Sand Art Commercials, check them out (http://www.qwest.com/business/sandart.html) They are basically the inspiration for this experimentation (along with Andre Michelle). The faster you move your mouse - the more you push the sand away - hold the CTRL/CMD key to magnitize/attract/whatever.

See it here

If anyone can actually draw something with it, let me know - personally i'm pretty terrible at freehand drawing on a PC

Enjoy!

[update]



Here is an expansion on the same concept - except this time it uses some fun new filters and is driven by a forcemap. Click and drag to attract, hold down CTRL to repel. If the map generated at first doesn't look too good, click reset and you might get one that forms the particles a little better

See it here

[/update]

11 Comments | In: BitmapData, as3 | | #