New monster color variations from scratch

by Paul Siramy - May 2010

 

Table Of Content :

Overview

What is a colormap ? It is a very useful trick largely used by [not so] old games. It allows to create different color variations of an original sprite, with just very few additional bytes. Tough all the variations are all using the SAME and unique one palette and the SAME and unique one original sprite, they all looks different from each other. This magic is done with the help of small color-conversion-tables, the so-called "colormaps".

This tutorial is divided into 2 parts.

The first part will explain colormaps in very deep details. It's here for reference, as it always help to undersand how the things works. If you're already familiar with colormaps tough, you can just skip this part.

The last part will demonstrate a method to desing new colors for a given monster, just by using your favorite image editor (as long as it can works with palettes). We'll take Izual as an example, and change his blue skin in green, while his golden armor will become red.

 

Tools

In this tutorial I'll use these programs :

  • Dr. Tester by SVR : from this page
  • Corel Paint Shop Pro Photo X2, but your favorite image editor should be enough
  • ColormapMaker v1.00

 

Part 1 : The technique beyond colormaps

 

   1. Why are they used ?

I'll take the example of an old console game, Streets of Rage 2. In this game, there is 1 type of a biker with many color variations :

     
Biker 1   Biker 2   Biker 3   Biker 4

As you can see, this is the exact same sprite, just some of the colors are changing from 1 image to the other. It could have been done :

  1. by using only 1 sprite, but with 4 different palettes
  2. by using 4 different sprites, but with only 1 palette

Assuming we're working in a 8bpp (256 colors) graphical mode, where each pixel requires 1 byte, let's compute the memory size each method requires :

  1. Sprites : 47 pixels width * 69 pixels height = 3,243 bytes.
    Palettes : 256 colors * 3 bytes for RGB componants * 4 palettes = 3,072 bytes.
    This solution use a total of 6,315 bytes for representing 4 different Bikers.

  2. Sprites : 47 pixels width * 69 pixels height * 4 sprites = 12,972 bytes.
    Palettes : 256 colors * 3 bytes for RGB componants = 768 bytes.
    This solution use a total of 13,740 bytes for representing 4 different Bikers.

In fact the game use a 3rd solution which involves colormaps. So it has only 1 sprite, and 1 palette. It works because it is using in addition 4 different colormaps. I'll explain what is a colormap, but for now let's assume that 1 colormap is 256 bytes. This solution use :

    Sprites : 47 pixels width * 69 pixels height = 3,243 bytes.
    Palettes : 256 colors * 3 bytes for RGB componants = 768 bytes.
    Colormaps : 256 bytes * 4 colormaps = 1,024 bytes
    This solution use a total of only 5,035 bytes for representing 4 different Bikers.

Now you understand the advantage of colormaps : it's a way to make lots of color variations of a given sprite, with just a little memory usage malus each time, all the way without having to create nor modify images and/or palettes.

 

  2. Size notes

  • The colormap system does NOT require the colormap table to be 256 bytes. For instance the example presented next after use only a colormap of 35 entries. That's because the biker's image contains only 35 different colors.

  • Strictly speaking, a colormap is composed of at least 1 entry, but has no maximum limit (since there isn't a limit either for a palette). In common practise tough, the colormap has the exact same number of entries than in the palette.

  • In Diablo II all the colormaps have 256 entries, just like the palettes.

 

  3. Technical demonstration

In a 8 bits-per-pixel (256 colors) graphical mode, it works like this : the image is split into 2 parts, the pixels, and the palette. The palette is an array of 256 entries, each entry containing the Red, Green and Blue componants values of 1 color. Each pixel is an index, which points into that palette. The game draw an image on screen easily :

    1. For a pixel, it reads its value. This value represent the Palette Entry Index
    2. Using the corresponding Palette Entry, it find the color componants
    3. It combines them and draw the result onto the screen

Here's a sample, the head of the Biker 1 :

Pixels
 
Pixels / Colors correspondance
Palette
 
Final Result on screen

As you see, the normal process of drawing a sprite on the screen is really straightforward :

Now, the idea is to insert a table between the Pixels values and the Palette. This table will "remap" Pixels value into another Palette entry. Such a table is called a Colormap :

The game draw an image on screen the same way as before, except for that additional step :

    1. For a pixel, it reads its value. This value represent the original Palette Entry Index
    2. Using that value, it find in the Colormap the corresponding new Palette Entry Index
    3. Using the corresponding Palette Entry, it find the color componants
    4. It combines them and draw the result onto the screen

Using this colormap, it's almost as if the original image had different Pixel values :

Original Biker 1
Pixel values :
 
     
Pixel values after
the colormap had
been used :
 
     

The final result
is the Biker 2 :

 

 

Pixels after
Colormap / Colors
correspondance :

 


Without modifying the Biker 1 original image nor the Palette, really just by using a small colormap, the game made a new sprite : Biker 2.

 

  4. Warning : colormaps can decrease quality

A colormap can decrease more or less the quality of the original image. It's not the case in the Bikers samples presented here, but take attention of how the colormap is build... the colormap presented to transform a Biker 1 into a Biker 2 is not using the complete range of the Palette : there are only 28 unique values in the colormap, while there are 35 palette entries :

  • The pixels 0 1 2 3 4 5 ... are mapped to 0 1 2 3 3 5 ...
  • The pixels 13 14 15 16 17 18 ... are mapped to 13 2 15 16 18 18 ...
  • And so on...

Since some values are present more than once in the colormap table, this means there will be less Palette colors available in the final result : if a sprite was using ALL the 35 original colors, and if we were applying this colormap onto it, then the final image would be made of only 28 different colors, therefore decreasing the quality of the variation compared to its original sprite.

That's the limitation of a colormap. It can transform colors of sprites, but it has to work with the colors that are present in the palette. It can not modify the palette itself.

 

  5. Array of colormaps : special effect simulation

A colormap alone is good to transform all the pixels of a sprite into a predefined set of other colors at once. But we can also use them to simulate in 256 colors graphical mode some special effects that are normaly only possible in high / true color grapical modes.

Diablo II is using some tables of colormaps when it is running in DirectDraw (2D) mode instead of Direct3D (3D) mode. Etheral items for instance are translucent. This special effect is made by building a 2-dimensional array of colormaps. One of the entry is the color of the background pixel, and the other entry is the color of the item pixel. The value in this 2-dimensional array is the color in the palette that is best representing the 50% blending of the 2 pixels together.

Such a 2-dimensional array of 256 * 256 entries takes 256 * 256 bytes = 65,536 bytes. Nowadays 64 kB means nothing in the memory of our PC.

256 * 256 arrays are needed for representing all the possible combinations of pixels between them, for a given special effect. But a special effect is not necessary a blend between 2 pixels. For instance it can be a combination between a pixel and a light level. Diablo II use 32 light levels when drawing shadows in DirectDraw mode. It is a table of 32 colormaps, therefore it's a 2-dimensional array of 32 * 256 bytes = 8,192 bytes : an entry is the original color of the pixel to draw, the other entry is the light level between 0 and 31.

 

  6. Implementation of Colormaps in Diablo II

Diablo II use colormaps for various situations while in DirectDraw (2D) graphical mode :

  • Inventory items
  • Player body parts
  • Monster body parts
  • State effects
  • Colored texts
  • 256 colors simulation of hicolor / truecolor effects
  • Hilight the cursor selection

Depending of their goals, the colormaps are build differently.

By design choice [except for 1 special case] the colormaps in Diablo II are NOT just a file of 256 bytes each. In fact, several colormaps are put together to make 1 bigger file.

Colormaps in Diablo II are stored in these files :

Path   File size
(bytes)
  Number of
colormaps
Data\Global\Items\Palette\*.dat   5,376   21
Data\Global\Monsters\GreenBlood.dat (1)   256   1
Data\Global\Monsters\RandTransforms.dat   7,680   30
Data\Global\Monsters\<2-letters token>\Cof\Palshift.dat   2,048   8
Data\Global\Palette\<palette name>\Pal.pl2 (2)   443,175   1,727

(1) The file GreenBlood.dat is not used. I think an option was planned to replace the Red blood that split from monsters into a Green blood, but it was not implemented in the final release.

(2) All the Pal.pl2 files contains a really huge collections of colormaps. They're all based on each available palette in the game. And in addition to their 1,727 colormaps, they also contains the original palette, and some text color datas for 3D mode.

 

Part 2 : Creating a new color variation for a monster from scratch

Izual will be the monster for which we will create a totally new color variation. We'll make his skin green and his armor red :
 
Original colors
With our future color map

 

   1. Creation of the sample image

First, we're going to build a sample image which contains several frames of Izual. The goal is to make this sample image as representative as possible of all the colors Izual is using. This is only to help choosing and adjusting the future color modifications in the image editor, so no need to choose them for 2 hours. Just 1 frame can be enough if you want.

The file data\global\excel\monstats.txt tells us that Izual is assigned the code "22", therefore the animations files of Izual are at the path data\global\monsters\22. Let's open d2data.mpq in Dr. Tester, and let's select the 1st COF file of Izual we find.

Now, we have to choose the appropriate Diablo II palette this animation was made with. Normally all monster's animations are using the "Units" palette, therefore the animation is not using any of the act-dependant colors. But for safety, we'll proceed as if it wasn't the case for Izual. Since this monster appears in Act 4, select the Act 4 palette :

Click on the button and it'll become . Click on it another time and it'll become . For both the Frames and Directions you can change the value associated with the and buttons. By selecting another "COF" file, you change the type of animation and see another new frames.

What we need now is to select as best representative images of Izual as possible. Just jump from a COF to another, explore directions and frames. Try to find some frames that show Izual with all the range of colors he's using. Changes directions to check the back of Izual... Take a screenshot each time and paste them into your favorite image editor, croping them to keep only Izual, and you might end with something similar to :

Once you have selected all the most representative frames, mix them together to create a sample image. Its dimensions are not important. As for me, I ended up with this image :

At this point, we need only 2 more thing before we're ready to play with colors : the sample image MUST be downsample to 256 colors using the appropriate Diablo II palette, and a special 16 * 16 pixels image looking like a palette MUST then be inserted somewhere in the sample image.

In order to do that, we'll first make the image editor able to work with Diablo II palettes. It's as simple as opening an image, and save its palette using the menus of the image editor. I'll take the example of Corel Paint Shop Pro Photo X2 here, as this is my current and favorite image editor, but the same logic should be reproductible with your own image editor.

Take this zip : http://paul.siramy.free.fr/_divers/d2_gif_palettes.zip

It contains all the Diablo II palettes, in the form of GIF images of size 16 * 16 pixels each. In case the link is dead for some reason, here are the images that it contains :

 
 
 
 
 
 
 
act0-black
 
act0-cyan
 
act1
 
act2
 
act3
 
act4
 
act5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
endgame
 
endgame2
 
fechar
 
loading
 
menu0
 
menu1
 
menu2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
menu3
 
menu4
 
sky
 
static
 
trademark
 
units
 
 

 
Open the d2-act0-black.gif image in your image editor, and save its palette somewhere on your hard disk :

While you're at it, proceed the same for all the other palettes, at least the "act" ones, since they'll probably come in handy one day. For Izual, you need to save at least the Act 4 palette.

Now, downsample your sample image (which is probably in true color mode right now), to 256 colors mode, using the "Act 4" palette you have just saved. It's important to NOT modify pixels, so DON'T choose any "Error diffusion dithering" method or something similar. On the contrary, select a basic pixel-per-pixel method, "Nearest color matching" beeing a perfect method :

Good, the sample image is now in 256 colors mode, using the act 4 palette. Normally it should looks exactly the same as before the downsampling.

Now insert the content of the d2-act4.gif image into the sample image. Since the sample image and that small palette-like image are using the exact same palette (act 4), the insertion should be done without problem nor color distorsion. Keep its original size of 16 * 16 pixels.

Once it's done, we should have a sample image that is using the Diablo II palette of Izual, and which contains a special image that will reflect all the color operations we'll make to the whole sample image a bit later :

Inserting the palette-looking small image produced an interesting side effect. The sample image is now using exactly 256 different colors, due to the presence of that small "palette" :

 

   2. Recoloring the sample image

Now start the fun part. In your image editor, alter the colors of your sample image by any means at your disposal. But use only the operations that do not modify pixels, just the colors. Depending of your image editor, it's possible that color operations must be done in truecolor mode. If this happen, don't hesitate and upsample back your image in true colors, and then proceed to the color operations you want to apply.

I'll show here the settings choosen in my Paint Shop Pro version to transform the skin and metal separatly.

Menus "Adjust" / "Hue and Saturation" / "Hue/Saturation/Lightness..."

Then I modify the "Yellows" and "Blues" settings like this :

This way the sample image become :

It looks like we have finish, but in fact we have a last but very important step to do !

The color operation made onto the image had altered the palette (if still in 256 colors mode), or the pixels (if you had to switch back to true color mode). In any case now the sample image is not using a Diablo II palette anymore. Therefore we have to recolor the image back to a Diablo II palette.

While we're at it, we're going to use not the act 4 palette again, but the "act0-black" palette. This way, whatever the original pixels using act-dependant colors, we'll "remap" them to any of the common colors. Guess what ? this operation of remaping pixels is exactly what the colormap we're creating will do later in-game.

Let's convert a last time the sample image, this time using the "act0-black" palette, and still using the "Nearest color matching" method :

It's done.

Indeed, if we compare before and after the conversion back to the "act0-black" palette, we see some differences. The right image is how Izual will looks within the game :
 
Before the conversion
After the palette becomes "act0-black"

And now we have the heart of our own colormap for Izual : the 16 * 16 pixels "palette" in the sample image. Select only this "palette", make a new image from it, and save it somewhere in the form of BMP, or PCX file. Be sure this little new image is still in 256 colors mode, and that it is still using the "act0-black" palette :

As you see, the image almost don't have any blue anymore, because they have been replaced by greens, despite blues are present in the palette itself.

Indeed, the 16 * 16 pixels palette-like image, despite looking like using 256 colors, is in fact using only 142 different colors of the "act 0" palette :

The 2 bottom rows of pixels, known as beeing the act-dependant colors in a Diablo II palette, are in fact using a palette index of a non-magenta color. It means that more than 2 pixels in the image are in fact using the same palette color entry... Yep, this magical 16 * 16 palette-like image is somehow a visual representation of a colormap !

The Izual.pcx file is the small image. It is our own new Izual colormap. It is not useable by the game in that form, so we'll use a specific tool to extract the binary componants of the colormap, based upon the pixels of the image.

 

   3. Creating Izual new palshift.dat

The hardest part is done.

Go to your ColormapMaker directory, and put the file Izual.pcx in there... Ok in fact it's already there because I made the tool especially for this tutorial ;)

Make a new.ini, put 8 times the filename of your new "colormap" PCX file.

Then create a new BAT file.

And run it :

Well done. Now, you just have to place this new palshift.dat file into your Diablo II mod directory :

You just have to launch Diablo II using the usual "-direct" method :

Run the game, and find Izual to verify that it works. Here, for this tutorial, I have modified the fortress DS1 to have him right in town :

As you see, it works well.