Tuesday, January 6, 2009

Using hi-resolution external lightmap

Q3 engine and the 128x128 lightmap size limitation

That's something that have bored me since I started to map... How can I override this stupid limit of lightmap size.
When you are still new to mapping, you just don't understand why adding a key _lightmapscale with the value of 0.25 or 0.125 to the worldspawn give pretty good result on some part of the map, while being extremely ugly and blurry on some other. Because we all are very naive, we just think "let's reduce this damn value until it looks good", but it looks never good.

Why?
It is a little technical...

Q3 engine only handle 128x128 lightmap. So even if you specify a smaller value for lightmapscale or samplesize, or whatever, all your mergeable surface will be put by q3map2 in a 128x128 lightmap. This is why tiny brush have a good lightmap precision while large flat terrain have bad. Each tris can't have more than one lightmap, so larger tris can't have too much precision. And that's not all: if you have a flat surface made of multiple tris, all those tris share a single 128x128 lightmap. They are merged. Why? Because if they were not merged, you would not get a seamless lightmapping on this surface, because each tris would have a different lightmap precision, so shadows would be very sharp on small tris, and very blurry on some other, and that's a major problem especially if this is the same shadow that lie on multiple tris.

If you doubt about it, you can test it, for example by turning each ground brush you have into its own separated func_group. Face in a func_group will never be merged with face that are not in the same func_group (using this property is a common way to fix light leak). That was my first attempt to get more precision... That was awfull...



How to override the 128x128 engine limitation

I have searched the web forever... I have never seen someone getting it. I have found many forum thread talking about that, some people's answer was: "use the -lightmapsize switch of q3map2", but they have probably never tested it, cause this switch is for Wolfenstein/ET, if you use it for Q3, you get a full illuminated map, cause the engine just don't load those lightmap.

There is only one solution, I found it yesterday, you should use shaders. You should use an external lightmap. Ydnar (the author of q3map2) does that for this purpose! Too bad this feature is just badly documented (as many q3map2's feature). For each surface you want to have a greater lightmap, you have to write this shader:

textures/my_dir/my_texture_hi_res_lightmap
{
q3map_lightmapSize 1024 1024
q3map_lightmapBrightness 2.0
{
tcgen lightmap
map $lightmap
rgbGen identity
}
{
map textures/my_dir/my_texture.jpg
blendFunc filter
}
}

"q3map_lightmapSize 1024 1024" instruct the compiler to put the result of the lightmap for all face using this shaders into one or more external TGA image file (standard lightmap are directly stored into the BSP) with a 1024x1024 size, that will be written in your /map/name_of_your_map directory, with a name like lm_0000.tga, lm_0001.tga, and so on. Note that you can eventually modify them with your favorite image editor if something look bad.

Alamo's external hi-res 1024x1024 lightmap:


Then q3map2 auto-generate a shader file in the /script directory named q3map2_name_of_your_map_lightmap.shader. Here is what you found in this file:

// Custom shader file for my_map.bsp
// Generated by Q3Map2 (ydnar)
// Do not edit! This file is overwritten on recompiles.

my_map/7132892A3E304A3B1AEC29CEB002847F
{
q3map_lightmapSize 1024 1024
q3map_lightmapBrightness 2.0
{
tcgen lightmap
map maps/my_map/lm_0000.tga
rgbGen identity
}
{
map textures/my_dir/my_texture.jpg
blendFunc filter
}
}

That's the trick! Your external lightmap is just meant as a standard texture! Be carefull, cause q3map2 is forced to create one shader for every lightmap/texture combination. That's why it is best to use a huge lightmapsize, you have good chance that one texture will be combined with only one lightmap.

q3map_lightmapBrightness 2.0 instruct the compiler to render the surface twice as bright. You should have this to work cause standard lightmap use overbright bits, without this line, your surfaces will look darker than usual. With a value of 2.0, they will look exactly as bright as it used to be.

In the stage part of the shader (remember that stage are for the renderer, while q3map_* are directive for the compiler), there is an important line: tcgen lightmap. That's the trick I missed for days... In all usual shader (using true built-in lightmap), this line is implicit. But here we use a conventionnal texture that play the role of a lightmap, so the renderer should be aware that we want to use the lightmap coordinate calculated by q3map2 for this surface.

The end of those two shaders have nothing particular.

Note that if you want to use this trick for your terrain (modeled or indexed), you should add the directive q3map_lightmapMergable, because you want to merge all tris of your terrain into one continuous and seamless lightmap. I think you need (not tested) q3map_nonplanar and q3map_shadeangle N, and I suggest to tune the value of a q3map_lightmapSampleOffset N.



The result

I have just finished to patch Alamo, all the ground is merged into an external 1024x1024 lightmap. Look at those before/after screenshots!


Before, those wagon have no shadow and look just like if they were floating over the ground, after, they cast a nice shadow (even the thin wheels cast shadow).




Before, those palm tree were able to cast shadow over the wall, but not on the ground. After, they have pretty detailed shadow everywhere.




Before, like the wagon, all the scene looks like it floats over the ground, under an omni-directional light... After, it's just more realistic. Note that the arch cast shadow in the second screenshot.




External lightmap are good! Use them!

11 comments:

  1. Greetings.
    I was looking for something like this for ages now. Thx for putting this up. But i have a few questions.
    I tried the above steps and q3map2 stored the lightmaps as it should but it doesnt create any shader files for those lightmaps.So i have my lightmaps now but i dont know how to implement them on my map :(
    Any help would be much appreciated :)

    ReplyDelete
  2. For alamo.map, it creates a shader named q3map2_alamo.shader in your scripts directory.

    With shader's name like "alamo/7132892A3E304A3B1AEC29CEB002847F" (that are randomly generated), if it doesn't create it for you, you can't do it by yourself: you just can't guess the internal shader's name that q3map2 put in your bsp.

    Have you got the very last q3map2 ? Have you checked directory permission (if you are using an UNIX-like OS) ? Any error loged during the BSP stage ?

    ReplyDelete
  3. I think i found the error. I used -lightmapsize in my batch file which caused q3map2 not to write a shader. After i removed that from the batch file it worked fine. Thx for you fast response :)

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. I tested this in combination with normalimage and it worked but with a drawback. The true normal map is a layer that tiles across the surface, since using q3map_normalimage adds normal map info to the lightmap itself, to get a 1:1 normal map I'd to use a huge lightmap, huge enough to cover the whole map 1:1. If the lightmap is smaller than the surface it stretches, causing the normal info to get very blurry in game.

    ReplyDelete
  6. I will try this on HeH tonight. I have put it off for too long. Would be a nice thing to do for all maps if it's easy.

    ReplyDelete
  7. Is there a way to generate radiosity like in Half-Life maps?

    ReplyDelete
  8. Good write-up.

    Internal lightmaps are okay with "rgbGen identity" but because you are using an external lightmaps, you should explicitly specify "rgbGen identityLighting" in the lightmap stage and remove "q3map_lightmapBrightness".

    This will prevent problems with toggling r_overbrightbits.

    ReplyDelete
  9. For some reason this only works for me when applied to patches and not brushes (in the JK3 engine) why is this?

    ReplyDelete
  10. Thank you very much for this tutorial, very useful! Got me a couple of times to make it work.

    ReplyDelete