In this tutorial, you will learn how to add three new features by directly accessing and manipulating the positions of specific bones:

• Overriding the position of selected bones from your C# code so the head or arm can be rotated independently of the rest of the animation.
• Positioning an object relative to an animated bone so the character can hold a baseball bat.
• Attaching a set of bounding spheres to the animated character, which can be used for collision detection.

The first step is to download the Skinned Model Sample (Game Studio 4.0) from creators.xna.com. It is important to ensure you use version 4.0 of the sample because this tutorial will not match up properly if you are following along using an earlier version!

If you want to jump straight to the end result, the final code produced by following this tutorial is in the SkinnedModelExtensions subfolder.

Character animations are created by attaching vertices to a skeleton structure that specifies the current position of each bone. Individual vertices of the character model are moved according to whichever bone they are attached to, or if a vertex is on the joint between several bones, its position may be interpolated between more than one bone.

In the Content Pipeline, the skeleton structure is represented as a tree of BoneContent objects, each of which has a unique name string. The SkinnedModelProcessor provided by the Skinning Sample converts this tree of bones into a SkinningData object, which identifies bones by integer index rather than by name (because index format is more efficient for playing back the animation at runtime).

The three features we are adding require the ability to identify specific bones by name, so our first task is to extend the SkinningData class to include bone name information.

Open the SkinningData.cs file, and add this new property to the end of the class (at line 75):

    /// <summary>
    /// Dictionary mapping bone names to their indices in the preceding lists.
    /// </summary>
     [ContentSerializer]
    public Dictionary<string, int> BoneIndices { get; private set; }

We must also change the SkinningData constructor to initialize the new property. At line 24, replace the existing constructor with this version:

    /// <summary>
    /// Constructs a new skinning data object.
    /// </summary>
    public SkinningData(Dictionary<string, AnimationClip> animationClips,
                        List<Matrix> bindPose, List<Matrix> inverseBindPose,
                        List<int> skeletonHierarchy,
                        Dictionary<string, int> boneIndices)
    {
        AnimationClips = animationClips;
        BindPose = bindPose;
        InverseBindPose = inverseBindPose;
        SkeletonHierarchy = skeletonHierarchy;
        BoneIndices = boneIndices;
    }

Now we're going to change the custom processor so that it will gather the bone name data and pass it through to our modified constructor. Open SkinnedModelProcessor.cs, and insert this code at line 64 (after the declaration of the skeletonHierarchy field):

    Dictionary<string, int> boneIndices = new Dictionary<string, int>();

At line 71, immediately before the closing brace of the foreach loop, add this line:

    boneIndices.Add(bone.Name, boneIndices.Count);

Finally, at line 82, change the code that constructs the SkinningData object to pass an extra boneIndices argument, replacing the existing two lines with:

    model.Tag = new SkinningData(animationClips, bindPose,
                                 inverseBindPose, skeletonHierarchy,
                                 boneIndices);

Mission accomplished! If you compile and run the modified sample, it will appear exactly the same as before. The bone names are now available at runtime, but we have not yet written any code to use this new data.

There are three different coordinate systems involved in skeletal animation:

• Animation data is stored in bone space where the position of each bone is specified relative to its parent bone. This representation is compact and convenient to work with. For instance, if we apply a rotation to the arm bone, the hand and finger bones will automatically move along with it—even though their local transforms are not animated—because they will inherit the rotation from their parent bone.
• Bone transforms are converted into world space by multiplying each one by the world transform of its parent bone. World transforms are useful because they give the absolute position of each bone in 3D space.
• Skin space is calculated by multiplying each world space matrix by the inverse bind pose matrix for that bone. This leaves us with the difference between the current position of the bone and the position this bone had when the character model was first constructed, which is exactly what we need to render the skinned model geometry.

Open AnimationPlayer.cs, and look at the Update method (line 82). You will see that it just calls three helper methods, one after another:

• UpdateBoneTransforms extracts the latest bone space matrices from the animation data.
• UpdateWorldTransforms converts the bone space matrices into world space.
• UpdateSkinTransforms converts the world space matrices into skin space.

In order to override the position of specific bones, we want to change the bone transform matrices before they are passed to UpdateWorldTransforms. To make this possible, we must change the UpdateWorldTransforms method so it takes the bone transform matrices as a parameter, rather then looking them up from a field of the AnimationPlayer class.

At line 144 of AnimationPlayer.cs, change this method signature:

    public void UpdateWorldTransforms(Matrix rootTransform)

to:

    public void UpdateWorldTransforms(Matrix rootTransform, Matrix[] boneTransforms)

And at line 86, add this new parameter to the call site:

    UpdateWorldTransforms(rootTransform, boneTransforms);

Now we are going to open up SkinningSample.cs. At line 35 (after the animationPlayer field), add two new fields:

    SkinningData skinningData;
    Matrix[] boneTransforms;

At line 70, in the LoadContent method, change:

    SkinningData skinningData = currentModel.Tag as SkinningData;

to:

    skinningData = currentModel.Tag as SkinningData;

The SkinningData object will now be stored in a field rather than just locally to this method, so we can access it later on in our animation code.

At line 75 (the next line after the "This model does not contain a SkinningData tag." error handling code), add this line to initialize an array that will be used to hold temporary bone matrices:

    boneTransforms = new Matrix[skinningData.BindPose.Count];

Now for the fun stuff!

In the SkinningSample.Update method, at line 100 (after the UpdateCamera call), add this code for reading gamepad input:

    // Read gamepad inputs.
    float headRotation = currentGamePadState.ThumbSticks.Left.X;
    float armRotation = Math.Max(currentGamePadState.ThumbSticks.Left.Y, 0);

And this code for reading keyboard input:

    // Read keyboard inputs.
    if (currentKeyboardState.IsKeyDown(Keys.PageUp))
        headRotation = -1;
    else if (currentKeyboardState.IsKeyDown(Keys.PageDown))
        headRotation = 1;

    if (currentKeyboardState.IsKeyDown(Keys.Space))
        armRotation = 0.5f;

And this code that will create rotation matrices based on the inputs that we just read:

    // Create rotation matrices for the head and arm bones.
    Matrix headTransform = Matrix.CreateRotationX(headRotation);
    Matrix armTransform = Matrix.CreateRotationY(-armRotation);

Replace this line, which calls the AnimationPlayer.Update method:

    animationPlayer.Update(gameTime.ElapsedGameTime, true, Matrix.Identity);

With this version:

    // Tell the animation player to compute the latest bone transform matrices.
    animationPlayer.UpdateBoneTransforms(gameTime.ElapsedGameTime, true);

    // Copy the transforms into our own array, so we can safely modify the values.
    animationPlayer.GetBoneTransforms().CopyTo(boneTransforms, 0);

    // Tell the animation player to recompute the world and skin matrices.
    animationPlayer.UpdateWorldTransforms(Matrix.Identity, boneTransforms);
    animationPlayer.UpdateSkinTransforms();

This is basically just inlining the same three calls (UpdateBoneTransforms, UpdateWorldTransforms, and UpdateSkinTransforms) that AnimationPlayer.Update was previously doing for us. The only difference is that because it copies the bone transforms into our own array before passing them to UpdateWorldTransforms, we now have the ability to change these matrices en route.

How do we know which matrix in the array to change? This is where the SkinningData.BoneIndices dictionary (which we added in the first section of this tutorial) comes into play. Of course, this dictionary is only useful if we know the name of the bone we are looking for. To see what bone names are used in our skeleton, look in the SkinningSample\Content\obj\x86\Debug folder where you will find a file named dude_0.xml (if you are unable to find the file, you may need to build the project to create it). This contains a cached copy of the model data, and is created as a side effect of building the content. Load this file into Visual Studio, and search for "BoneContent." You will find a tree of nested elements that looks something like this:

    <Child Type="Graphics:BoneContent">
      <Name>Root</Name>
      <OpaqueData>
        <Data Key="liw" Type="bool">false</Data>
      </OpaqueData>
      <Transform>...</Transform>
      <Children>
        <Child Type="Graphics:BoneContent">
          <Name>Pelvis</Name>
          ...

Now we can see that this skeleton contains bones named "Root," "Pelvis," "Spine," "Neck," and so on.

Armed with this information, we can add this code to modify two selected bone matrices after the GetBoneTransforms call, but before UpdateWorldTransforms:

    // Modify the transform matrices for the head and upper-left arm bones.
    int headIndex = skinningData.BoneIndices["Head"];
    int armIndex = skinningData.BoneIndices["L_UpperArm"];

    boneTransforms[headIndex] = headTransform * boneTransforms[headIndex];
    boneTransforms[armIndex] = armTransform * boneTransforms[armIndex];

Now run the sample. To rotate the head, move the left thumbstick left and right, or press the PAGE UP and PAGE DOWN keys. To wave the left arm, move the left thumbstick up, or press the SPACEBAR key.

In this example, we multiplied our custom rotations with the existing bone transform matrices. It is also possible to entirely replace selected matrices, thus disabling animation for those bones, or to mix and match matrices from more than one AnimationPlayer in order to play back different animations on different parts of the body, or even to blend between two sets of matrices to cross fade between animations.

To make the character hold an object, we first need an object for the character to hold. Download the Object Placement On Avatar sample from Creators Club Online, open the Content folder, and then drag the baseballbat.fbx file into the SkinningSampleContent project, next to the existing dude.fbx file.

This baseball bat model was built at a different scale to our character model, but fortunately there is an easy way to fix that. Right-click the baseballbat.fbx node in Solution Explorer, and choose Properties. On the Properties tab, click the plus sign next to Content Processor; a list of custom processor parameters is shown. Change the Scale setting from 1 to 30, and the X Axis Rotation from 0 to 120.

In SkinningSample.cs, add this field to store the bat model at line 37 (after the boneTransforms field):

    Model baseballBat;

At line 78 (immediately before the "Create an animation player" comment) add this call to load the model:

    // Load the baseball bat model.
    baseballBat = Content.Load<Model>("baseballbat");

At the end of the Draw method, right before the base.Draw call, add this call:

    DrawBaseballBat(view, projection);

Finally, insert this new method to draw the bat immediately after the end of the existing Draw code:

    /// <summary>
    /// Draws the baseball bat.
    /// </summary>
    void DrawBaseballBat(Matrix view, Matrix projection)
    {
        int handIndex = skinningData.BoneIndices["L_Index1"];

        Matrix[] worldTransforms = animationPlayer.GetWorldTransforms();

        foreach (ModelMesh mesh in baseballBat.Meshes)
        {
            foreach (BasicEffect effect in mesh.Effects)
            {
                effect.World = worldTransforms[handIndex];
                effect.View = view;
                effect.Projection = projection;

                effect.EnableDefaultLighting();
            }

            mesh.Draw();
        }
    }

This looks up the current world space transform matrix for the left index finger bone, and uses this as the World transform when drawing the baseball bat model.

When you run the sample, you will see the character is now holding a bat, and the bat moves in sync with the animation. If you look closely at the bat, however, you'll notice that it appears to protrude through the character's finger. To correct this, we can insert a small translation in the DrawBaseballBat method we just added. Add the following code just before the two nested foreach statements. (The values were obtained through simple experimentation.)

    // Nudge the bat over so it appears between the left thumb and index finger.
    Matrix batWorldTransform = Matrix.CreateTranslation(-1.3f, 2.1f, 0.1f) *
                               worldTransforms[handIndex];
  

Then change the mesh effect world transform in the same DrawBaseballBat method from this:

effect.World = worldTransforms[handIndex];

to this:

effect.World = batWorldTransform;

Detecting collisions against animated character models can be tricky, because the collision shape needs to move in sync with the animation! One of the most common solutions is to approximate the shape of the model using a number of spheres: a couple for the torso, one for the head, two or three along each arm, and so on. This is obviously only an approximation of the true shape of the model, but it is easy to implement and efficient to test against, so many games find it to be a good approach. If you need more accuracy, you can always just use a larger number of smaller spheres to reduce the errors.

We are going to define our collision spheres by using an XML file to specify the size and parent bone of each sphere. First we need a new class to represent this data. Right-click the SkinnedModelWindows project, select Add | New Item | Class, and then enter SkinnedSphere.cs as the name. If you are working on the Xbox version of the sample, you will then have to Add | Existing Item on the SkinnedModelXbox project, and add SkinnedSphere.cs here as well.

Replace the contents of SkinnedSphere.cs with this code:

    using Microsoft.Xna.Framework;
    using Microsoft.Xna.Framework.Content;

    namespace SkinnedModel
    {
        public class SkinnedSphere
        {
            public string BoneName;
            public float Radius;

            [ContentSerializer(Optional = true)]
            public float Offset;
        }
    }

The Offset property positions the collision sphere along the length of the bone. This is used if we want several small spheres at different positions along a lengthy bone, for instance an upper arm bone might have one near the shoulder, one near the elbow, and a third in between. Not all spheres need an offset, so we use a ContentSerializer attribute to declare this field as optional.

Right-click the SkinningSampleContent project, choose Add | New Item | XML File, and then enter CollisionSpheres.xml as the name. Replace the contents of the new file with this XML:

    <?xml version="1.0" encoding="utf-8" ?>
    <XnaContent>
      <Asset Type="SkinnedModel.SkinnedSphere[]">
        <Item>
          <BoneName>Spine</BoneName>
          <Radius>16</Radius>
        </Item>
        <Item>
          <BoneName>Spine3</BoneName>
          <Radius>18</Radius>
        </Item>
        <Item>
          <BoneName>Head</BoneName>
          <Radius>10</Radius>
          <Offset>4</Offset>
        </Item>
      </Asset>
    </XnaContent>

So how do we know if these spheres are in the correct place? We need some kind of debug rendering in order to see where each sphere is positioned.

Download version 4.0 of the Primitives 3D Sample from creators.xna.com. Right-click the SkinningSample game project, click Add, click New Folder, and then enter Primitives3D as the name. Drag the files GeometricPrimitive.cs, SpherePrimitive.cs, and VertexPositionNormal.cs from the Primitives 3D sample, and drop them into this new folder in Solution Explorer.

At the top of the SkinningSample.cs file, we must add a new using statement at line 17:

    using Primitives3D;

We are going to add an option to toggle on and off debug sphere rendering. To do this, we must track the input state—both previous and current—in order to detect when a toggle button has been pressed. Add these fields at line 34 (after the existing currentKeyboardState and currentGamePadState fields):

    KeyboardState previousKeyboardState = new KeyboardState();
    GamePadState previousGamePadState = new GamePadState();

Also add these fields for animating and displaying the collision spheres:

    SkinnedSphere[] skinnedSpheres;
    BoundingSphere[] boundingSpheres;

    bool showSpheres;
    SpherePrimitive spherePrimitive;

Add this code at the end of the LoadContent method to load the SkinnedSphere data from the XML file we previously created, and to initialize the SpherePrimitive helper object that will be used to display the spheres:

    // Load the bounding spheres.
    skinnedSpheres = Content.Load<SkinnedSphere[]>("CollisionSpheres");
    boundingSpheres = new BoundingSphere[skinnedSpheres.Length];

    spherePrimitive = new SpherePrimitive(GraphicsDevice, 1, 12);

At the end of the Update method, between the calls to UpdateSkinTransforms and base.Update, add a new method call:

    UpdateBoundingSpheres();

This new method is implemented as follows:

    /// <summary>
    /// Updates the boundingSpheres array to match the current animation state.
    /// </summary>
    void UpdateBoundingSpheres()
    {
        // Look up the current world space bone positions.
        Matrix[] worldTransforms = animationPlayer.GetWorldTransforms();

        for (int i = 0; i < skinnedSpheres.Length; i++)
        {
            // Convert the SkinnedSphere description to a BoundingSphere.
            SkinnedSphere source = skinnedSpheres[i];
            Vector3 center = new Vector3(source.Offset, 0, 0);
            BoundingSphere sphere = new BoundingSphere(center, source.Radius);

            // Transform the BoundingSphere by its parent bone matrix,
            // and store the result into the boundingSpheres array.
            int boneIndex = skinningData.BoneIndices[source.BoneName];

            boundingSpheres[i] = sphere.Transform(worldTransforms[boneIndex]);
        }
    }

At the end of the Draw method, between the calls to DrawBaseballBat and base.Draw, add a new method call to display the resulting animated spheres:

    if (showSpheres)
    {
        DrawBoundingSpheres(view, projection);
    }

This new method is implemented as follows:

    /// <summary>
    /// Draws the animated bounding spheres.
    /// </summary>
    void DrawBoundingSpheres(Matrix view, Matrix projection)
    {
        GraphicsDevice.RasterizerState = Wireframe;

        foreach (BoundingSphere sphere in boundingSpheres)
        {
            Matrix world = Matrix.CreateScale(sphere.Radius) *
                           Matrix.CreateTranslation(sphere.Center);

            spherePrimitive.Draw(world, view, projection, Color.White);
        }

        GraphicsDevice.RasterizerState = RasterizerState.CullCounterClockwise;
    }
  

    static RasterizerState Wireframe = new RasterizerState
    {
        FillMode = FillMode.WireFrame
    };

Finally, we need a way to toggle on and off sphere display. Replace the HandleInput method with this new code:

    private void HandleInput()
    {
        previousKeyboardState = currentKeyboardState;
        previousGamePadState = currentGamePadState;
                
        currentKeyboardState = Keyboard.GetState();
        currentGamePadState = GamePad.GetState(PlayerIndex.One);

        // Check for exit.
        if (currentKeyboardState.IsKeyDown(Keys.Escape) ||
            currentGamePadState.Buttons.Back == ButtonState.Pressed)
        {
            Exit();
        }

        // Toggle the collision sphere display.
        if ((currentKeyboardState.IsKeyDown(Keys.Enter) &&
             previousKeyboardState.IsKeyUp(Keys.Enter)) ||
            (currentGamePadState.IsButtonDown(Buttons.A) &&
             previousGamePadState.IsButtonUp(Buttons.A)))
        {
            showSpheres = !showSpheres;
        }
    }

When you run the sample, you can now press the A button on the gamepad or the ENTER key, and three spheres will appear attached to the torso and head of the character model. More spheres can be added by extending the XML file. See the version of CollisionSpheres.xml in the SkinnedModelExtensions subfolder for a complete version that covers both arms, legs, and feet with a set of 25 spheres.

To check for collisions with the character, you can now simply loop over each BoundingSphere in the boundingSpheres array, and call BoundingSphere.Intersects for each.

Many games actually end up using more than one set of bounding spheres per character, so they can adjust the sphere sizes to improve the gameplay experience. For example, collision between the character and the environment might use quite large spheres to ensure the entire character model was inside the collision spheres, and, thus, the character could never put his arm through a wall. Whereas collision between the character and enemy bullet fire might use smaller spheres to ensure the collision detection will never register a false positive where a bullet that did not actually intersect the character model is still detected as a hit. Both forms of collision are only approximate, but depending on the gameplay situation, one may want to err on the side of being too generous, while the other errs on the side of being too cautious.