This is a repost from the old blog, since this is one of those interesting stuff, which weren’t big enough to warrant a sample.
Working on my VTF tutorial, I came across the need to apply lighting to a terrain. For lighting, we need normals, of course, but I only had heightmaps available. When the terrain is static, this poses no problems at all. All we have to de is generate a normal map from the heightmaps with whatever tools we want. For example, NVIDIA has a Photoshop plugin for this here.
However, when we have dynamic terrain, with real time deformation, this is no longer an option. This is especially the case when we make the deformations on the displacement map. After lots of reading and searching, I’ve come across the Sobel Filter, which can be used to compute normal maps.
As a result, I made a pixel shader that takes as input the displacement map and outputs the normal map. At runtime, after modifying the displacement map, I apply this shader and put the result into a render target. This result is then used when drawing the terrain, when computing lighting in the pixel shader.
So here’s the HLSL code:
You should tweak normalStrength until the result you get is satisfactory.
And here is the result of applying that code to two heightmaps, and some terrain lit using a normal map generated by this shader.