## Monday, August 11, 2008

### Perspective Correct Texturing in OpenGL

99.9% of the time, you don't have to worry about perspective correct texturing in OpenGL. If you draw a textured rectangle, it will look correct from any angle. But there are a few cases where you need to manage perspective yourself.

I blogged here on what perspective vs. non-perspective texturing looks like when a rectangle is deformed into another shape. This blog doesn't mention a minor detail: OpenGL will never give you that middle image - instead what you get is something like this. (What you see is the two triangles that OpenGL decomposes the quad into.)

Creating Perspective with the Q Coordinate

If you can easily identify the amount of "squishing" you have applied to one end of a quad, you can use the "Q" coordinate to produce perspective correct textures. We do this in X-Plane: the snow effect is a cylinder that wraps around the plane. In order to assure that the entire screen is covered, we close up one end to make a cone in some cases. When we do this, the quads that make the sides of the cylinders become trapezoids, then triangles. By using the Q coordinate technique, we keep our texture perspective correct.

There are two caveats of this technique:
1. You can't really make the texture go to infinite distance (that would be a Q coordinate of zero). If you do, all texture coordinates are zero and some of your texture information is lost. To hack around this, we clamp to a very small minimum Q. Don't tell anyone!
2. This technique will produce perspective changes in both directions of the texture! This is the "correct" perspective thing to do, but may be surprising. In particular, as we make the far end of our cylinder into a cone, because visually this is the equivalent of making it stretch out to much further away, the texture is stretched along the cylinder, making snow flakes that are close to us very tall and ones that are far away very short.
Why does this technique work? Well, basically texture coordinates are 4-component vectors, with the S & T coordinates you are used to divided by the Q coordinate (the 4th component). By having a smaller Q component, the division makes the S & T effectively larger, which means more texture in a small area, which looks "farther away".

But the real magic is in how interpolation happens. When you specify a per-vertex variable that is "varying", it is interpolated across the polygon. In our case, the Q coordinate is interpolated, creating a pattern of shrinking Q's that stretches the texture dynamically. The Y=1/X shape of this curve creates "perspective".

Creating Perspective Via Matrices

This snippet of code is the nuclear weapon of perspective. You simply give it a trapezoid (four corners) in 2-d and it calculates a matrix that applies the texture in a manner that looks like the appropriate perspective. In other words, it's effectively calculating the vanishing points.

We use this in X-Plane for the skewed instruments - when the user drags the instrument corners, we simply calculate a matrix that creates the correct perspective.

The snippet of code contains comments with the derivation - one interesting property: given 8 variables in a 2-d perspective matrix and 8 variables (four 2-d corners) there can exist only one perspective matrix for any given trapezoid.

One warning if you decide to use this with OpenGL: make sure your computed transform matrix doesn't produce negative W coordinates! (To do this, simply apply the resulting matrix to one of your input corners and check the resulting W. If it is negative, negate the entire matrix.)

I use LAPACK to solve the matrix in my version of this code, and LAPACK doesn't make a lot of guarantees for the "constant" term (a constant factor applied to all matrix values). If that constant is negative, you will get output coordinates where X, Y, Z and W are all negative - the perspective division would cancel this negation, but at least on the machines I've used, the negative "W" causes anarchy long before we get that far.

The reason you can use a projection matrix anywhere in OpenGL is because projection fundamentally relies on storing information in the "W" coordinate (the 4th coordinate) to be divided later - the "W" just gets carried along for the ride and then later produces the correct perspective effect.

1. I absolutely love you for explaining this to me. I've been looking for this solution for ages!

2. Ditto. I'm a friend of hao's, and this has been bugging me since I did a radial menu system with fan strips (so lots of quads in the shape you've shown). Putting textures onto them looked horrible. Now I might have a chance of fixing that!

3. Might be pushing my luck here, but I don't suppose you know the equivalent for Direct3D?

I'm using OpenGL from now on, but if you did, I could go back and fix some XNA things.

4. I've just been looking at the XNA documentation.

It looks like one could create a custom texture coordinate format using a Microsoft.Xna.Framework.Graphics.VertexBuffer and the flexible vertex format (VertexDeclaration, which uses multiple VertexElement structs to describe the format, each of which has a VertexElementFormat (like, 2d float vector, 3d float vector etc), and a VertexElementUsage (position, normal, colour, tex coord etc)). Now, I've no idea whether that would let you create a 4D texture coordinate but if it did then one might assume that one could place the Q value in the fourth dimension there. Might work!

5. hi guys. i really need to make a radial stretch of an image. like an ultrasound picture. are fan strips the way to go? i need to get rid of the perspective. according to this blogpost opengl will never give me that midde image (http://bp1.blogger.com/_TrRVoYy3Itc/SFF1aDGquZI/AAAAAAAAAUY/rPV86rOPljw/s1600-h/perspective.jpg). never? or never automatically? how do make a trapeziod like the middle one?

6. Sorry, bad news - the middle trapezoid distortion isn't available via hw texture interp...because it's not an affine transformation. At least, I think...

7. Thanks for your reply of the Mac OpenGL mailing list. This is just what I needed. I couldn't reply on the list because my copy of the email wasn't delivered. But, I was able to pull your reply off of the archive.

Thanks so much!

8. Thanks a bunch. I have been googling for exactly this.

I have a GL_QUAD with a texture. On top of that i have a freely adjustable 4 corner polygon that user can change to choose texture coordinates for mapping the same texture on a rectangle.

At the moment i see wrongly interpolated textures. I will probably have to use a pixel shader then.

9. Might it be possible to achieve the middle image by rotating a quad within an orthographic projection?

1. Definitely not! See the comments above about the limits of affine transformations, etc.