An exploration into the combination of 3D and 2D elements in Flash.

Papervision is awesome. I doubt anyone would disagree with that. It gives developers a tool to develop things that they could only wish for with previous versions of flash. Already, there are numerous sites and experiments out there which push 3D in Flash to the max. Something that really caught my interest, however, is how we can use a 3D engine like Papervision for more than just pure 3D environments. I was interested in more than simply using 3D as a transition between 2D states – or having a 3D object surrounded by a 2D interface. I started entertaining the idea of using both 3D and 2D elements to create effects that would not be possible with either one individually.  3D environments give us the ability to experience true depth - to view complex 3D models, cast shadows, or explore realistic environments. It's practically impossible to "simulate" 3D effects, or even simple navigation, around complex objects with proper texture mapping and depth in a 2D world. Animators can spend hours faking 3D on even the simplest of vector art. At the same time, some of the most stunning effects are possible in 2D (my favorites are done with the BitmapData class), as shown by some of the more prominent Flash experimentalists - such as Andre Michelle, Grant Skinner, and Jared Tarbell. Knowing that both 3D and 2D have such strengths – why should they be mutually exclusive? I don't think they should be.

My first real experiment with combining 3D and 2D was a 3D Forcemap. The physics aside, the really cool thing about this experiment was how it looked (at least I think so!). Not only that, it was the catalyst for further experimentation using 3D with the BitmapData class. The general idea behind this experiment, was that just because the 'particles' in the experiment were in 3D – their paths could be stored on a 2D surface. After all, everything in 3D is eventually projected into 2D. So, on every frame, the scene is simply drawn to a 2D canvas (BitmapData), which was then blurred and faded to have the trails fade over time.

I took a similar approach during phase 2 of my experimentation. I created a sound visualization which simply spit particles from a pseudo-sphere center – with 3D lines tracing the particles' paths and which faded the further the particles got from the center. A canvas was once again used, but was this time included in the drawing process. The canvas and the scene were drawn onto the canvas before being blurred, colored, faded, and translated. Using this technique, I was able to do 2 things. 1) Track the 3D Lines and particles on a 2D canvas, and 2) track the changes of the canvas itself as it was being transformed. The effect was an ethereal smoky one – and one I was quite happy with. By drawing a slightly transformed image back onto itself – you can create some amazing effects – whether it is done with webcams, particles, or yes, 3D.

Take the scene below:

and the same scene without the 2D:

As you can see, the 2D elements are what really add the flair to this movie. In fact, with the rendering limitations of Flash – it is impossible to get anything close to this effect with 3D alone. But doing this got me wondering – is it possible for these 2D effects to actually respond, in a realistic way, to the 3D environment? Can we have our effects actually based on perspective?

These questions sparked phase 3 of my experimentation. As you can see below, the effects in this comet demo actually do respond to the 3D world. Moreover, they are actually embedded in the 3D world (somewhat). Click and drag to see it.

I'm providing the source below, but here is a quick explanation of the code.

• There are actually 2 scenes in this movie. The first contains the comet, the second the stars in the background. This is important, since it is only the comet scene we want to apply any effects too.
• The comet scene and its effect canvas are never actually added to the stage. Instead, they use the self-rendering technique mentioned above (blur, displace, fade). The canvas, in turn, is then drawn on to a BitmapData that actually is on the stage – drawingBoard.
• We have to remove information from drawingBoard so we can see the stars underneath. For this example, I simply used a threshold test to make all black pixels transparent. This way, the stars will never be in front of our comet, but we can see them where the comet's trail doesn't go.
• To get the effect to respond to our perspective, we simply need to alter the direction our canvas translates based on the position of the camera. The method I used is a pretty big hack, but it works. I simply determine the angle of the view to the comet, and convert the translation variables to offset based on that angle.

From here it should be pretty easy to figure out what's going on in the source, which you can get here.

Enjoy!

In: general | | #