Cg Programming/Unity/Projectors

This tutorial covers projective texture mapping for projectors, which are particular rendering components of Unity.

It is based on Section “Cookies”. If you haven't read that tutorial yet, you should read it first.

Unity's Projectors

Unity's projectors are somewhat similar to spotlights. In fact, they can be used for similar applications. There is, however, an important technical difference: For spotlights, the shaders of all lit objects have to compute the lighting by the spotlight as discussed in Section “Cookies”. If the shader of an object ignores the spotlight, it just won't be lit by the spotlight. This is different for projectors: Each projector is associated with a material with a shader that is applied to any object in the projector's range. Thus, an object's shader doesn't need to deal with the projector; instead, the projector applies its shader to all objects in its range as an additional render pass in order to achieve certain effects, e.g. adding the light of a projected image or attenuating the color of an object to fake a shadow. In fact, various effects can be achieved by using different blend equations of the projector's shader. (Blend equations are discussed in Section “Transparency”.)

One might even consider projectors as the more “natural” way of implementing lights. However, the interaction between light and materials is usually specific to each material while the single shader of a projector cannot deal with all these differences. This limits the possibilities of projectors to three basic behaviors: adding light to an object, modulating an object's color, or both, adding light and modulating the object's color. We will look at adding light to an object and attenuating an object's colors as an example of modulating them.

Projectors for Adding Light

In order to create a projector, choose GameObject > Create Empty from the main menu and then (with the new object still selected) Component > Effects > Projector from the main menu. You have now a projector that can be manipulated similarly to a spotlight. The settings of the projector in the Inspector Window are discussed in Unity's manual. Here, the only important setting is the projector's Material, which will be applied to all objects in its range. Thus, we have to create another material and assign a suitable shader to it. This shader usually doesn't have access to the materials of the game objects, which it is applied to; therefore, it doesn't have access to their textures etc. Neither does it have access to any information about light sources. However, it has access to the attributes of the vertices of the game objects and its own shader properties.

A shader to add light to objects could be used to project any image onto other objects, similarly to an overhead projector or a movie projector. Thus, it should use a texture image similar to a cookie for spotlights (see Section “Cookies”) except that the RGB colors of the texture image should be added to allow for colored projections. We achieve this by setting the fragment color to the RGBA color of the texture image and using the blend equation

Blend One One

which just adds the fragment color to the color in the framebuffer. (Depending on the texture image, it might be better to use Blend SrcAlpha One in order to remove any colors with zero opacity.)

Another difference to the cookies of spotlights is that we should use the Unity-specific uniform matrix unity_Projector to transform positions from object space to projector space instead of the matrix _LightMatrix0. However, coordinates in projector space work very similar to coordinates in light space — except that the resulting $x$ and $y$ coordinates are in the correct range; thus, we don't have to bother with adding 0.5. Nonetheless, we have to perform the division by the $w$ coordinates (as always for projective texture mapping); either by explicitly dividing $x$ and $y$ by $w$ or by using tex2Dproj.

The line ZWrite Off makes sure that we don't change the depth buffer since we are only adding light to a mesh that has already been rasterized. Offset -1, -1 slightly changes the depth to pretend that we are a bit in front of the mesh to which we are adding light. This helps to make sure that nothing of what we rasterize now is occluded by that mesh. (When you copy and paste the code below, some editors add a space character (" ") after the first "-", which creates a syntax error. Just delete that space character.)

Shader "Cg projector shader for adding light" {
   Properties {
      _ShadowTex ("Projected Image", 2D) = "white" {}
   }
   SubShader {
      Pass {      
         Blend One One 
            // add color of _ShadowTex to the color in the framebuffer 
         ZWrite Off // don't change depths
         Offset -1, -1 // avoid depth fighting (should be "Offset -1, -1")

         CGPROGRAM
 
         #pragma vertex vert  
         #pragma fragment frag 
 
         // User-specified properties
         uniform sampler2D _ShadowTex; 
 
         // Projector-specific uniforms
         uniform float4x4 unity_Projector; // transformation matrix 
            // from object space to projector space 
 
          struct vertexInput {
            float4 vertex : POSITION;
            float3 normal : NORMAL;
         };
         struct vertexOutput {
            float4 pos : SV_POSITION;
            float4 posProj : TEXCOORD0;
               // position in projector space
         };
 
         vertexOutput vert(vertexInput input) 
         {
            vertexOutput output;
 
            output.posProj = mul(unity_Projector, input.vertex);
            output.pos = UnityObjectToClipPos(input.vertex);
            return output;
         }
 
 
         float4 frag(vertexOutput input) : COLOR
         {
            if (input.posProj.w > 0.0) // in front of projector?
            {
               return tex2D(_ShadowTex , 
                  input.posProj.xy / input.posProj.w); 
               // alternatively: return tex2Dproj(  
               //    _ShadowTex, input.posProj);
            }
            else // behind projector
            {
               return float4(0.0, 0.0, 0.0, 0.0);
            }
         }
 
         ENDCG
      }
   }  
   Fallback "Projector/Light"
}

Notice that we have to test whether $w$ is positive (i.e. the fragment is in front of the projector, not behind it). Without this test, the projector would also add light to objects behind it. Furthermore, the texture image has to be square and it is usually a good idea to use textures with wrap mode set to clamp.

Just in case you wondered: the shader property for the texture is called _ShadowTex in order to be compatible with the built-in shaders for projectors.

As described in Section “Cookies”, projective texture mapping comes sometimes with an unpleasant side effect: at the edges of the projection, the GPU uses a high mip map level, which can result in a visible border (in particular for texture maps with clamped texture coordinates). The easiest way to avoid this, is to deactivate mip maps for the texture image: find and select the texture image in the Project Window; then in the Inspector Window set Texture Type to Advanced and uncheck Generate Mip Maps. Don't forget to click the Apply button.

Projectors for Modulating Colors

The basic steps of creating a projector for modulating colors are the same as above. The only difference is the shader code. The following example adds a drop shadow by attenuating colors, in particular the floor's color. Note that in an actual application, the color of the shadow caster should not be attenuated. This can be achieved by assigning the shadow caster to a particular Layer (in the Inspector Window of the game object) and specifying this layer under Ignore Layers in the Inspector Window of the projector.

In order to give the shadow a certain shape, we use the alpha component of a texture image to determine how dark the shadow is. (Thus, we can use the cookie textures for lights in the standard assets.) In order to attenuate the color in the framebuffer, we should multiply it with 1 minus alpha (i.e. factor 0 for alpha equals 1). Therefore, the appropriate blend equation is:

Blend Zero OneMinusSrcAlpha

The Zero indicates that we don't add any light. Even if the shadow is too dark, no light should be added; instead, the alpha component should be reduced in the fragment shader, e.g. by multiplying it with a factor less than 1. For an independent modulation of the color components in the framebuffer, we would require Blend Zero SrcColor or Blend Zero OneMinusSrcColor.

The different blend equation is actually about the only change in the shader code compared to the version for adding light:

Shader "Cg projector shader for drop shadows" {
   Properties {
      _ShadowTex ("Projected Image", 2D) = "white" {}
   }
   SubShader {
      Pass {      
         Blend Zero OneMinusSrcAlpha // attenuate color in framebuffer 
            // by 1 minus alpha of _ShadowTex 
         ZWrite Off // don't change depths
         Offset -1, -1 // avoid depth fighting (should be "Offset -1, -1")
         
         CGPROGRAM
 
         #pragma vertex vert  
         #pragma fragment frag 
 
         // User-specified properties
         uniform sampler2D _ShadowTex; 
 
         // Projector-specific uniforms
         uniform float4x4 unity_Projector; // transformation matrix 
            // from object space to projector space 
 
          struct vertexInput {
            float4 vertex : POSITION;
            float3 normal : NORMAL;
         };
         struct vertexOutput {
            float4 pos : SV_POSITION;
            float4 posProj : TEXCOORD0;
               // position in projector space
         };
 
         vertexOutput vert(vertexInput input) 
         {
            vertexOutput output;
 
            output.posProj = mul(unity_Projector, input.vertex);
            output.pos = UnityObjectToClipPos(input.vertex);
            return output;
         }
 
 
         float4 frag(vertexOutput input) : COLOR
         {
            if (input.posProj.w > 0.0) // in front of projector?
            {
               return tex2D(_ShadowTex , 
                  input.posProj.xy / input.posProj.w); 
               // alternatively: return tex2Dproj(  
               //    _ShadowTex, input.posProj);
            }
            else // behind projector
            {
               return float4(0.0, 0.0, 0.0, 0.0);
            }
         }
 
         ENDCG
      }
   }  
   Fallback "Projector/Light"
}

Summary

Congratulations, this is the end of this tutorial. We have seen:

How Unity's projectors work.
How to implement a shader for a projector to add light to objects.
How to implement a shader for a projector to attenuate objects' colors.

Unity's Projectors

Projectors for Adding Light

Projectors for Modulating Colors

Summary

Further reading