Quantcast
Channel: GameDev.net
Viewing all 17063 articles
Browse latest View live

DirectX 11 C++ Game Camera

$
0
0
So why do we need a camera anyway? Camera is a window, through which the player looks into 3D world. Usually it is placed at the eyes position of player's avatar in game. By controlling the camera movement you can create awesome immersion feel.

When you construct your 3D scene (every point in this space has three coordinates – x, y, z), you have to present it on the screen somehow. However, the monitor screen is a flat surface made up of pixels and thus has 2D coordinate system, not 3D. The camera abstraction class contains various transformation matrices inside itself and aims to address this contradiction. Additionally you can use this class when rendering scene from the light point of view to create shadow mapping or specify player’s behavior or to achieve other similar effects. It is extremely useful in rendering.

Basics


Firstly, let us discuss the concept of transformation matrices. We will need them for converting object points’ coordinates into camera-relative coordinates. If you know the basics, feel free to skip this part of the article and move on right to implementation part.

Take a look at the following picture:

Attached Image: pic1.png

As you can see, the exact same vector v can be defined by using different coordinate systems. In each of those systems, this vector will have its own coordinates, relative to the origin of system. How do we get vector coordinates relative to frame B, if we know its coordinates in frame A? Well, we can use transformation matrix, corresponding to coordinate system change:

[x', y', z', w'] = [x, y, z, w] * View

w component of the sequence in square brackets indicates if we want to convert coordinates of the vector or coordinates of the point. If w component is equal to zero - it is a vector, otherwise it is a point. This component is very important, because unlike vectors, points rely on exact position in space, thus, we need to consider coordinate system’s origin transformation as well. It is well illustrated on the following picture:

Attached Image: pic2.png

The view transformation matrix for our illustration looks like this:

Attached Image: pic2.1.png

Note that for vector (it means w component is equal to zero) Q is not needed, since multiplication by zero w component will discard it anyway. The matrix above helps us to change coordinate system from world to camera-relative.

Next, we have to project resulted camera-relative point coordinates onto 2D plane. The result M matrix will be:

M = View * Projection

Thus, the final equation will be:

[x', y', z', w'] = [x, y, z, w] * View * Projection

Let us discuss how to construct the projection matrix.

There are different types of projection matrices exist. One is called orthogonal matrix and other is called perspective matrix.

Orthogonal matrix serves for projecting 3D point into 2D without saving 3D illusion. This type of projection is often used for drawing UI sprites or text – those objects usually should not be resized regardless of their distance from camera. The following illustration roughly shows how orthogonal projection looks like:
Attached Image: pic3.png
View frustum is a spatial volume, limited by 4 planes (top, left, bottom and right) and other two planes, called near clip plane and far clip plane. All objects inside this frustum will be projected right onto the screen plane. Near and far clip planes are needed for preventing the rendering of too far or too close objects to the player. None of scene objects beyond far clip plane or closer to the camera than near clip plane will be presented on projection window. This greatly increases performance and avoid the waste of video card resources.

However, as we mentioned before, the picture produced by orthogonal projection is not realistic. Two objects with the same size, but with different distance from the camera will have absolutely the same size on the projection window. So what about realistic perspective projection?

Let us talk about the nature of human vision to understand concepts behind the perspective projection. The human eyes see objects in a distance as if those objects would be smaller than those placed closer to the eye would. In 3D, it is called perspective. Just look through the window and you can see that buildings outside it look smaller than the window itself – this is perspective effect. Artists usually use this effect to create 3D illusion on the paper. This is one of the simple but effective techniques contributing to the scene realism.

Another interesting feature of our vision is that it is limited by view angle – we cannot see everything around us at the same time. So how can we simulate our eyes behavior in game, taking everything we have discussed before into consideration?

Here comes the concept of perspective view frustum. The view frustum is a space volume, visible by our eyes, which we want to project onto the screen. The screen comes as a rough approximation of human viewing mechanism, which usually composed out of two images, received by two human eyes. Here is illustration of perspective view frustum:
Attached Image: pic4.png
Here is how we project objects onto the screen plane (recall that it is our screen window) depending on their distance from the camera:

Attached Image: pic5.png

Each 3D point will have a new coordinates (x', y', z', w') after applying transformation matrix M. If you divide x' and y' coordinates by w' you will get normalized device coordinates in range [-1; 1], which is then mapped by shaders and viewport right onto the screen. Remained z' coordinate serves for depth information, which helps to recognize the order in which primitives are drawn into the screen.

Implementation


Luckily for us, DirectX has a big number of built-in functions for managing view and projection matrices without extensive complex math involved on our side. DirectX uses left-handed coordinate system:

Attached Image: pic6.png

Thus our most involved functions will be:

XMMatrixLookAtLH(XMVECTOR position, XMVECTOR target, XMVECTOR up_direction)
XMMatrixPerspectiveFovLH(float radians, float ratio, float near_plane, float far_plane)
XMMatrixOrthographicLH(float view_width, float view_height, float near_plane, far_plane)

All of input parameters for those functions can be easily obtained from picture 3, so let us briefly define them:
  • position – camera position coordinates in world coordinate system;
  • target – camera’s focus target position. Note that this point must lie on the line of look at target vector in picture 3;
  • up_direction – camera coordinate system up vector. Usually this is (0, 1, 0), but if we want our camera to be free, we have to track this vector changes by our own;
  • radians – the angle between view frustum’s upper plane and bottom plane in radians, projection angle;
  • ratio – this is a result of division between projection’s width and height. Usually this is the same as screen_width/screen_height;
  • near_plane, far_plane – those parameters define the distance of corresponding clip planes from camera position measured along look-at-target axis;
  • view_width, view_height – the width and height of orthogonal frustum.
So now we know everything we need to write our own camera implementation. I will use C++ language, since this one is my favorite and the one widely used in DirectX games. This code will use DirectXMath library included in latest DirectX 11 Windows 8 SDK.

Why do we need a separate class for our camera anyway? Well, we can directly specify all transformation matrices right in our code. But soon or later we would want to make our look at the scene more dynamic – e.g. change its position according to keyboard or mouse input. So the process of managing matrices recreation will become a nightmare. Here comes our class, which will have a very human-friendly interfaces to control camera movement.

Let us define the functionality of this class first. It will contain camera view matrix, orthogonal and perspective projection matrices and provide interface for accessing those matrices. In addition, there will be a method, which will allow us to resize of our camera's view frustum. This is needed when we want to resize our application window or if we want to render more objects into texture etc. Moreover, our camera will have an ability to change its position, moving along specified axis, changing the target itself and of course rotating the camera around itself.

At instantiation time, we create a camera at (0, 0, -1) point coordinates, target (0, 0, 0) and up vector looking along y-axis. We will store those values in our class along with view and projection matrices and their parameters. Note that we will store the end-point of up-vector, not the vector itself. After camera object instantiation, class-consumer must call InitProjMatrix() or InitOrthoMatrix() methods or both. Those methods will construct initial projection matrices for our camera view frustum, so we can acess them later from camera-class users.

Here is the code for our camera-control class, but don't worry, we will discuss it below:

//GCamera.h
#pragma once
#include "GUtility.h"

namespace Game
{
////////////////////////////////////////////////////////
// Stores View and Projection matrices used by shaders
// to translate 3D world into 2D screen surface
// Camera can be moved and rotated. Also, user can change 
// camera's target and position
////////////////////////////////////////////////////////
class GCamera
{
public:
	// Constructs default camera looking at 0,0,0
	// placed at 0,0,-1 with up vector 0,1,0 (note that mUp is NOT a vector - it's vector's end)
    GCamera(void);
	// Create camera, based on another one
	GCamera(const GCamera& camera);
	// Copy all camera's parameters
    GCamera& operator=(const GCamera& camera);
    ~GCamera(void) {}

private:
	// Initialize camera's View matrix from mPosition, mTarget and mUp coordinates
	void initViewMatrix();

public:
	// Initialize camera's perspective Projection matrix
	void InitProjMatrix(const float angle, const float client_width, const float client_height, 
		const float nearest, const float farthest);
	// Initialize camera's orthogonal projection
	void InitOrthoMatrix(const float client_width, const float client_height,
		const float near_plane, const float far_plane);

	// Resize matrices when window size changes
	void OnResize(uint32_t new_width, uint32_t new_height);

	///////////////////////////////////////////////
	/*** View matrix transformation interfaces ***/
	///////////////////////////////////////////////

	// Move camera
	void Move(XMFLOAT3 direction);
	// Rotate camera around `axis` by `degrees`. Camera's position is a 
	// pivot point of rotation, so it doesn't change
	void Rotate(XMFLOAT3 axis, float degrees);
	// Set camera position coordinates
    void Position(XMFLOAT3& new_position);
	// Get camera position coordinates
	const XMFLOAT3& Position() const { return mPosition; }
	// Change camera target position
	void Target(XMFLOAT3 new_target);
	// Get camera's target position coordinates
	const XMFLOAT3& Target() const { return mTarget; }
	// Get camera's up vector
	const XMFLOAT3 Up() { return GMathVF(GMathFV(mUp) - GMathFV(mPosition)); }
	// Get camera's look at target vector
	const XMFLOAT3 LookAtTarget() { return GMathVF(GMathFV(mTarget) - GMathFV(mPosition)); }	
	// Returns transposed camera's View matrix	
    const XMFLOAT4X4 View() { return GMathMF(XMMatrixTranspose(GMathFM(mView))); }

	/////////////////////////////////////////////////////
	/*** Projection matrix transformation interfaces ***/
	/////////////////////////////////////////////////////

	// Set view frustum's angle
	void Angle(float angle);
	// Get view frustum's angle
	const float& Angle() const { return mAngle; }

	// Set nearest culling plane distance from view frustum's projection plane
	void NearestPlane(float nearest);
	// Set farthest culling plane distance from view frustum's projection plane
	void FarthestPlane(float farthest);

	// Returns transposed camera's Projection matrix
	const XMFLOAT4X4 Proj() { return GMathMF(XMMatrixTranspose(GMathFM(mProj))); }
	// Returns transposed orthogonal camera matrix
	const XMFLOAT4X4 Ortho() { return GMathMF(XMMatrixTranspose(GMathFM(mOrtho))); }

private:
    /*** Camera parameters ***/
    XMFLOAT3 mPosition;		// Camera's coordinates
    XMFLOAT3 mTarget;		// View target's coordinates
    XMFLOAT3 mUp;			// Camera's up vector end coordinates

	/*** Projection parameters ***/
	float mAngle;			// Angle of view frustum
	float mClientWidth;		// Window's width
	float mClientHeight;	// Window's height
	float mNearest;			// Nearest view frustum plane
	float mFarthest;		// Farthest view frustum plane

    XMFLOAT4X4  mView;		// View matrix
	XMFLOAT4X4	mProj;		// Projection matrix
	XMFLOAT4X4	mOrtho;		// Ortho matrix for drawing without tranformation
};

} // namespace Game

//GCamera.cpp
#include "GCamera.h"

namespace Game
{

GCamera::GCamera(void)
{
    mPosition		= XMFLOAT3(0.0f, 0.0f, -1.0f);
    mTarget			= XMFLOAT3(0.0f, 0.0f, 0.0f);
    mUp				= GMathVF(GMathFV(mPosition) + GMathFV(XMFLOAT3(0, 1, 0)));
	this->initViewMatrix();

	mAngle			= 0.0f;
	mClientWidth	= 0.0f;
	mClientHeight	= 0.0f;
	mNearest		= 0.0f;
	mFarthest		= 0.0f;

	XMStoreFloat4x4(&mView, XMMatrixIdentity());
	XMStoreFloat4x4(&mProj, XMMatrixIdentity());
	XMStoreFloat4x4(&mOrtho, XMMatrixIdentity());
}

GCamera::GCamera(const GCamera& camera)
{
	*this = camera;
}

GCamera& GCamera::operator=(const GCamera& camera)
{
    mPosition		= camera.mPosition;
    mTarget			= camera.mTarget;
    mUp				= camera.mUp;

	mAngle			= camera.mAngle;
	mClientWidth	= camera.mClientWidth;
	mClientHeight	= camera.mClientHeight;
	mNearest		= camera.mNearest;
	mFarthest		= camera.mFarthest;

    mView			= camera.mView;
	mProj			= camera.mProj;
	mOrtho			= camera.mOrtho;
    return *this;
}

void GCamera::initViewMatrix()
{
	XMStoreFloat4x4(&mView, XMMatrixLookAtLH(XMLoadFloat3(&mPosition), XMLoadFloat3(&mTarget), 
		XMLoadFloat3(&this->Up())));
}

void GCamera::InitProjMatrix(const float angle, const float client_width, const float client_height, 
								const float near_plane, const float far_plane)
{
	mAngle = angle;
	mClientWidth = client_width;
	mClientHeight = client_height;
	mNearest = near_plane;
	mFarthest = far_plane;
	XMStoreFloat4x4(&mProj, XMMatrixPerspectiveFovLH(angle, client_width/client_height, 
		near_plane, far_plane));
}

void GCamera::Move(XMFLOAT3 direction)
{
	mPosition = GMathVF(XMVector3Transform(GMathFV(mPosition), 
		XMMatrixTranslation(direction.x, direction.y, direction.z)));
	mTarget = GMathVF(XMVector3Transform(GMathFV(mTarget), 
		XMMatrixTranslation(direction.x, direction.y, direction.z)));
	mUp = GMathVF(XMVector3Transform(GMathFV(mUp), 
		XMMatrixTranslation(direction.x, direction.y, direction.z)));

	this->initViewMatrix();
}

void GCamera::Rotate(XMFLOAT3 axis, float degrees)
{
	if (XMVector3Equal(GMathFV(axis), XMVectorZero()) ||
		degrees == 0.0f)
		return;

	// rotate vectors
	XMFLOAT3 look_at_target = GMathVF(GMathFV(mTarget) - GMathFV(mPosition));
	XMFLOAT3 look_at_up = GMathVF(GMathFV(mUp) - GMathFV(mPosition));
	look_at_target = GMathVF(XMVector3Transform(GMathFV(look_at_target), 
		XMMatrixRotationAxis(GMathFV(axis), XMConvertToRadians(degrees))));
	look_at_up = GMathVF(XMVector3Transform(GMathFV(look_at_up), 
		XMMatrixRotationAxis(GMathFV(axis), XMConvertToRadians(degrees))));

	// restore vectors's end points mTarget and mUp from new rotated vectors
	mTarget = GMathVF(GMathFV(mPosition) + GMathFV(look_at_target));
	mUp = GMathVF(GMathFV(mPosition) + GMathFV(look_at_up));

	this->initViewMatrix();
}

void GCamera::Target(XMFLOAT3 new_target)
{
	if (XMVector3Equal(GMathFV(new_target), GMathFV(mPosition)) ||
		XMVector3Equal(GMathFV(new_target), GMathFV(mTarget)))
		return;

	XMFLOAT3 old_look_at_target = GMathVF(GMathFV(mTarget) - GMathFV(mPosition));	
	XMFLOAT3 new_look_at_target = GMathVF(GMathFV(new_target) - GMathFV(mPosition));
	float angle = XMConvertToDegrees(XMVectorGetX(
		XMVector3AngleBetweenNormals(XMVector3Normalize(GMathFV(old_look_at_target)), 
		XMVector3Normalize(GMathFV(new_look_at_target)))));
	if (angle != 0.0f && angle != 360.0f && angle != 180.0f)
	{
		XMVECTOR axis = XMVector3Cross(GMathFV(old_look_at_target), GMathFV(new_look_at_target));
		Rotate(GMathVF(axis), angle);
	}
	mTarget = new_target;
	this->initViewMatrix();
}

// Set camera position
void GCamera::Position(XMFLOAT3& new_position)
{
	XMFLOAT3 move_vector = GMathVF(GMathFV(new_position) - GMathFV(mPosition));
	XMFLOAT3 target = mTarget;
	this->Move(move_vector);
	this->Target(target);
}

void GCamera::Angle(float angle)
{
	mAngle = angle;
	InitProjMatrix(mAngle, mClientWidth, mClientHeight, mNearest, mFarthest);
}

void GCamera::NearestPlane(float nearest)
{
	mNearest = nearest;
	OnResize(mClientWidth, mClientHeight);
}

void GCamera::FarthestPlane(float farthest)
{
	mFarthest = farthest;
	OnResize(mClientWidth, mClientHeight);
}

void GCamera::InitOrthoMatrix(const float clientWidth, const float clientHeight,
		const float nearZ, const float fartherZ)
{
	XMStoreFloat4x4(&mOrtho, XMMatrixOrthographicLH(clientWidth, clientHeight, 0.0f, fartherZ));
}

void GCamera::OnResize(uint32_t new_width, uint32_t new_height)
{
	mClientWidth = new_width;
	mClientHeight = new_height;
	InitProjMatrix(mAngle, static_cast<float>(new_width), static_cast<float>(new_height), mNearest, mFarthest);
	InitOrthoMatrix(static_cast<float>(new_width), static_cast<float>(new_height), 0.0f, mFarthest);
}

}

// GUtility.h
#pragma once
#include <windows.h>

#include <d3d11.h>
#include <directxmath.h>
#include <iostream>

using namespace DirectX;

inline XMVECTOR GMathFV(XMFLOAT3& val)
{
	return XMLoadFloat3(&val);	
}

inline XMFLOAT3 GMathVF(XMVECTOR& vec)
{
	XMFLOAT3 val;
	XMStoreFloat3(&val, vec);
	return val;
}

First of all, let me briefly explain what is DirectXMath, so this code will become more clear for those who don’t know anything about it yet. DirectXMath is a set of wrappers for SSE operations over SIMD-compatible processor registers. They offer parallel processing of operations over matrices and vertices such as multiplication, division and so on. So theoretically, they give us a good speedup, if used right. It means you have to do as many operations as you can over loaded into SIMD-compatible processor registers data while it is still here. Obviously, the code above is not actually utilize this feature. I just used this library and not D3DX one because the latter is deprecated in Windows 8. You can disable this behavior in Visual Studio project settings so you will not get any parallel advantages.

Additionally, DirectXMath has a number of new types. They can be divided into storage data types and SIMD data types. The majority of operations are implemented for SIMD data types – so called worker data types. So, when you want to call them, you have to load data from storage type into SIMD type, process it and store it back. Storage classes are XMFLOAT2, XMFLOAT3, XMFLOAT4, XMINT2 and other similar for vectors, XMFLOAT3X3, XMFLOAT4X4 and similar for matrices. Vectorized data types are XMVECTOR and XMMATRIX – those will be your main workers.

You can read up a lot more about DirectXMath library here.

Therefore, we store camera position, target and up coordinates in XMFLOAT3 type. Matrices are stored within XMFLOAT4X4 data type. For operations over data, we load it into SIMD-types and process it.
Let us look at the code and explain some unclear moments. First of all, we have a constructor which initializes our camera view matrix with previously specified requirements for its default position, target and up coordinates. Constructor calls initViewMatrix() private class method, which is responsible for view matrix production from given inner position, target and up class properties.

Next, we have to construct our projection matrices outside of the class, on demand. InitProjMatrix() and InitOrthoMatrix() public methods are responsible for that. If you call any of those methods, they will store their input parameters inside the class private properties and produce corresponding matrix, which will be stored inside the class as well. As you can see, near plane for orthogonal matrix is always 0.0f – that’s because we don’t have to clip any UI geometry for which this matrix will be used. Also it is highly possible that Z buffer will be turned off when rendering UI, so depth information doesn’t really matter. You can change this behavior, if you are planning to clip some of the objects used in combination with orthogonal matrix in the future.
In order to handle screen resizing we provide OnResize() method, which will simply recreate our projection matrices with new parameters.

Take a special look at matrices accessors. As you can clearly see, they call XMMatrixTranspose before returning property value. That is because XMMATRIX stores its data in row-major order, but shader programs use column-major order. Since shaders are the common consumers for our matrices, we integrate matrix-transpose functionality right into accessor. You can change it, but do not forget about this behavior.

Now it is time to discuss move, rotate and change camera/target position functionality.

The movement is a very simple. Take a look at Move(XMFLOAT3) method. It changes position, target and up-end point coordinates of camera by adding input vector to the current ones. Thus, camera vectors will not change, hence camera will keep looking along the same direction and will not rotate anywhere. This effect can be illustrated if you keep looking ahead while jumping up or strafing left, right; moving back or forward.

Rotation. Rotation is the most interesting among other camera features. Along with camera rotation around specific angle, rotation will allow us to implement the change of camera position or camera’s target position in future without unneeded complexity. We will rotate our camera around axis, provided as function input, which must be in camera-bound 3D space. Also do not forget that angle in left-handed system is measured clockwise when looking from the end of the vector to its origin. This is very important.

The algorithm for camera rotation is following:

  1. We calculate two vectors – look-at-target vector and up vector.
  2. Construct the rotation matrix from passed into function axis and angle by calling XMMatrixRotationAxis(axis, angle).
  3. Modify both vectors by this matrix by calling XMMatrixTransform(vector, transform_matrix).
  4. Restore the coordinates of target and up end points and recreate camera view matrix. Camera position does not change in rotation.

So now we can take a look at camera target change function. Basically, we calculate two vectors – old look-at-target vector and new look-at-target vector. Next, we find an angle between those vectors and their cross product:
Attached Image: pic7.png
Since our system is left-handed, a x b(RH) is equal to b x a(LH). This cross-product vector will be an axis for our camera rotation. When we have an angle and axis, we can use Rotate(axis, angle), implemented earlier.
Almost the same algorithm goes for changing camera position, but we can cheat here by using previously implemented target changing method. Move our camera into new position, and then set the camera’s target to the old one.

There are so-called aircraft principal axes – pitch, roll and yaw:

Attached Image: pic8.png

Those are usually used in space-ship battle games, where you have free movements. By means of our camera class, implemented above, we can easily model this behavior:

GCamera camera;
// camera initialization
camera.Rotate(XMFLOAT3(0.0f, 0.0f, 1.0f), 90.0f); // roll by 90 degrees clockwise
// ...
camera.Rotate(XMFLOAT3(1.0f, 0.0f, 0.0f), 90.0f); // pitch by 90 degrees clockwise
// ...
camera.Rotate(XMFLOAT3(0.0f, -1.0f, 0.0f), 90.0f); // yaw by 90 degrees clockwise

Here is a complete code of our result:
https://code.google.com/p/camera-class/source/browse/#svn%2Ftrunk

Good luck and make some entertaining games! =)

D3D11 Tessellation In Depth

$
0
0

The Pipeline


Consider the typical flow of data through the programmable pipeline:

Input assembler -> Vertex shader -> Pixel shader

Buffers containing per-vertex data are bound to the input assembler. The vertex shader is executed once per vertex, and each execution is given one vertex worth of data from the input assembler.

Say, however, that we wish to process control points and patches instead. A patch consists of an ordered collection of control points that we'll use in the construction of a curve or surface. A control point consists of any data we might need during the construction of the curve or surface. For example, we might define a control point as representing positional and color information.

To relate control points and patches back to something more familiar, consider the standard primitive types such as lines and triangles. Think of the line primitive roughly as a patch consisting of two control points, and think of the triangle primitive roughly as a patch consisting of three control points. Typically, the control points will contain at least positional information, but more likely than not we'd also want to store other things we'd find useful in the programmable shading stages such as color and surface normal data.

D3D11 allows us to go beyond three control points per patch. The vertex shader by itself isn't particularly well-suited for handling the manipulation of patches; we could store the control points in a buffer and index with SV_VertexID, but this is not very efficient especially when dealing with 16+ control points per patch.

To solve this problem, D3D11 adds two new programmable stages: the hull shader and the domain shader. Consider the following pipeline.

Input assembler -> Vertex shader -> Hull shader -> Tessellator -> Domain shader -> Pixel shader

Normally, the input assembler can be configured to handle points, lines, line strips, triangles and triangle strips. It turns out that it is quite elegant to add new primitive types for patches. D3D11 adds 32 new primitive types: each represents a patch with a different number of control points. That is, it's possible to describe a patch with anywhere from 1 to 32 control points.

For the purpose of this example, say we've configured the input assembler to handle patches with 16 control points, and also that we're only rendering one patch. We will use a triangular patch domain.

Since we're rendering one patch, we'll need a buffer with 16 points in it -- in this context, these points are control points. This buffer is bound to the input assembler as usual. The vertex shader is executed once per control point, and each execution is given one control point worth of data from the input assembler. Similar to the non-patch primitive types, the vertex shader can only see one control point at a time; it cannot see all 16 of the control points on the patch.

When not using tessellation, the next shader stage is executed once the vertex shader has operated on all of the vertices of a single primitive. For example, when using the triangle primitive type, the next stage is run once per every three executions of the vertex shader. The same principle holds when using tessellation: the next stage isn't executed until all 16 control points have been transformed by 16 executions of the vertex shader.

Once all 16 control points have been transformed, the hull shader executes. The hull shader consists of two parts: a patch constant function and the hull program. The patch constant function is responsible for computing data that remains constant over the entire patch. The hull program is run per control point, but unlike the vertex shader, it can see all of the control points for the entire patch.

You might be wondering what the point of the hull program is. After all, we did already transform the control points in the vertex shader. The important part is that the hull program can take into account all of the control points when computing the further transformed output control points. D3D11 allows us to output a different number of control points from the hull program than we took in. This means we can perform basis transformations -- for example, using a little math we could transform 32 control points into 16 control points, which saves us some processing time later on down the pipeline. At this point, further clarification is helpful: the hull program runs once per output control point. So, if we've configured the hull program to output 4 control points, it will run 4 times total per patch. It will not run 16 times, even though we have 16 input control points.

The next stage is the tessellator unit itself. This stage is not programmable with HLSL, but has a number of properties that can be set. The tessellator is responsible for producing a tessellated mesh and nothing more; it does not care at all about any user-defined data or any of our control points. The one thing it does care about, however, are tessellation factors -- or, how much to tessellate regions of the patch. You may be wondering where we actually output these values. Since the tessellation factors are determined once per patch, we compute these in the patch constant function. Thus, the only thing given to the tessellator is the tessellation factors from the patch constant function.

The topologies produced by the tessellator vary depending on how it is setup. For this example, using a triangular domain means that the tessellator will produce a tessellated triangle topology described by 3D barycentric coordinates. How cool is that?

So, by this point we've transformed each control point in the vertex shader, performed a possible basis transformation of the control points in the hull program, and have determined the tessellation factors for this patch in the patch constant function, along with any other user-defined data. The tessellation factors have been run through the tessellation hardware, which has created a shiny new tessellated mesh: in this case, a tessellated triangle described with barycentric coordinates. I would like to emphasize once again that the tessellator does not care at all about anything besides the tessellation factors and a small number of configuration properties set at shader compile-time. This is what makes the D3D11 implementation so beautiful: it is very general and very powerful.

You're probably wishing we could transform the tessellated mesh in arbitrary ways, and, well... we can! The next stop is the domain shader. The domain shader can be thought of as a post-tessellation vertex shader; it is run once per tessellated vertex. It is handed all of our output control points, our patch constant data, as well as a special system value which describes the barycentric coordinate of the tessellated vertex we're operating on. Barycentric coordinates are very handy when working in triangular domains, since they allow us to interpolate data quite easily over the triangle.

At this point, the flow of data is familiar: the output from the domain shader is handed to the pixel shader. It is important to note that in general, 32 float4s can be passed between every shader stage. We can pass 32 float4s from the vertex shader to the hull shader, 32 float4s from the patch constant function to the domain shader, 32 float4s from the hull program to the domain shader, and 32 float4s from the domain shader to the pixel shader. In other words, a lot of data can be passed using interstage registers, not to mention we can also bind shader resource views to the vertex, hull, domain, geometry and pixel shader stages.

I have left the geometry shader out of this explanation to simplify things, but it is very possible to throw a geometry shader into the mix to do some very interesting things -- one example that comes to mind is eliminating portions of a patch, or breaking it up into individual triangles to form new topologies. It is also possible to use stream-out with tessellation.

Due to the general nature of the pipeline, we can even use tessellation without binding any actual control point data to the pipeline at all. Consider that the vertex shader is able to see the vertex ID (control point ID in this case) and instance ID. The hull and domain shaders can see the primitive ID (which is basically a patch ID). Using this information alone, very interesting and useful things can be accomplished: a good example is producing a large mesh consisting of many individual patches. The patches can be placed appropriately by using the primitive ID.

Earlier I touched on the tessellation stages having compile-time settings. These settings are specified with the hull program. Here is an example declaration of settings.

[domain("tri")]
[partitioning("integer")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(16)]
[patchconstantfunc("HSConst")]

domain(x) - This attribute specifies which domain we're using for our patches. In this example, I specified a triangle domain, but it's also possible to specify a quadrilateral or isoline domain.

partitioning(x) - This attribute tells the tessellator how it is to interpret our tessellation factors. Integer partitioning means the tessellation factors are interpreted as integral values; there are no "in-between" tessellated vertices. The other partitioning schemes are fractional_even, fractional_odd and pow2.

outputtopology(x) - This attribute tells the tessellator what kind of primitives we want to deal with after tessellation. In this case, triangle_cw means clockwise-wound triangles. Other possibilities are triangle_ccw and line.

outputcontrolpoints(x) - This attribute describes how many control points we will be outputting from the hull program. We can choose to output anywhere from 0 to 32 control points which are then fed into the domain shader.

patchconstantfunc(x) - This attribute specifies the name of the patch constant function, which is executed once per patch.

Each stage is given different data. To illustrate this, I will show one possible function signature for each stage.

VS_OUTPUT VS(IA_OUTPUT input, uint vertid : SV_VertexID, uint instid : SV_InstanceID);

HS_CONSTANT_OUTPUT HSConst(InputPatch<VS_OUTPUT, n> ip, OutputPatch<HS_OUTPUT, m> op, uint pid : SV_PrimitiveID);
HS_OUTPUT HS(InputPatch<VS_OUTPUT, n> ip, uint cpid : SV_OutputControlPointID, uint pid : SV_PrimitiveID);

DS_OUTPUT DomainShader(HS_CONSTANT_OUTPUT constdata, OutputPatch<HS_OUTPUT, m> op, uint pid : SV_PrimitiveID, float3 coord : SV_DomainLocation);

SV_DomainLocation's type depends on the chosen patch domain. For the triangular domain, SV_DomainLocation is a float3 For the quad domain, it is a float2. For the isoline domain, it is a float2 (for reasons which I will touch on in a future post). n stands for the number of input control points and m stands for the number of output control points.

As stated earlier, the patch constant function (HSConst in this case) is required to output at least the tessellation factors. The number of tessellation factors depends on the patch domain. For the triangular domain, there are 4 factors (3 sides, 1 inner). For the quadrilateral domain, there are 6 factors (4 sides, 2 inner). For the isoline domain, there are 2 factors (detail and density).

Let's take a look at the topology produced by the tessellator by using the wireframe rasterization mode, a quadrilateral domain, and integer partitioning.

In the following patch constant function, I have chosen to use hard-coded tessellation factors. In practice, the tessellation factors are computed dynamically. The tessellation factors are not required to be hard-coded constants!

struct HS_CONSTANT_OUTPUT
{
    float edges[4] : SV_TessFactor;
    float inside[2] : SV_InsideTessFactor;
};

HS_CONSTANT_OUTPUT HSConst()
{
    HS_CONSTANT_OUTPUT output;

    output.edges[0] = 1.0f;
    output.edges[1] = 1.0f;
    output.edges[2] = 1.0f;
    output.edges[3] = 1.0f;

    output.inside[0] = 1.0f;
    output.inside[1] = 1.0f;

    return output;
}


Sample Tessellations


The edge factors are held constant at 1, 1, 1, 1 and the inside factors at 1, 1. The tessellator produces the following mesh:
Attached Image: 1.png
What about edge factors of 3, 1, 1, 1 and inside factors of 1, 1?
Attached Image: 2.png
Edge factors of 5, 5, 5, 5 and inside factors of 1, 1:
Attached Image: 3.png
Edge factors of 1, 1, 1, 1 and inside factors of 2, 1:
Attached Image: 4.png
Edge factors of 1, 1, 1, 1 and inside factors of 4, 1:
Attached Image: 5.png
Edge factors of 1, 1, 1, 1 and inside factors of 4, 4:
Attached Image: 6.png
Edge factors of 4, 4, 4, 4 and inside factors of 4, 4:
(Same as edge factors of 3.5, 3.8, 3.9, 4.0 and inside factors of 3.1, 3.22!)
Attached Image: 7.png
Edge factors of 4, 4, 4, 1 and inside factors of 4, 4:
Attached Image: 8.png
It should be noted that when using integer partitioning, the implementation is essentially using the ceiling of the written tessellation factors. Let's take a look at the output from the fractional_even partitioning scheme.

Edge factors of 2, 1, 1, 1 and inside factors of 1, 1:
Attached Image: fraceven1.png
Edge factors of 2.1, 1, 1, 1 and inside factors of 1, 1:
Attached Image: fraceven2.png
Edge factors of 2.2, 1, 1, 1 and inside factors of 1, 1:
Attached Image: fraceven3.png
Edge factors of 2.5, 1, 1, 1 and inside factors of 1, 1:
Attached Image: fraceven4.png
Edge factors of 3, 1, 1, 1 and inside factors of 1, 1:
Attached Image: fraceven5.png
Here's a funky one with edge factors of 3, 3, 3, 3 and inside factors of 4, 6, using the fractional_odd partitioning scheme:
Attached Image: fracodd1.png
Obviously hard-coded tessellation factors are only so useful. The real usefulness of tessellation comes into play when computing the tessellation factors dynamically, per patch, in realtime based on factors such as level of detail in a height map, camera distance, or model detail.

Patches and Geometry Shaders


You might be wondering how the geometry shader interacts with the new shader stages and the new patch primitive types.

Consider a pipeline with a vertex shader, geometry shader and pixel shader. The vertex shader runs per-vertex, and once one primitive's worth of vertices have been processed, the geometry shader runs. The geometry shader runs per-primitive and outputs vertices of a potentially different primitive type.

Now add a hull and domain shader to the mix. Say the input assembler primitive type is a patch with n control points. The vertex shader runs n times per patch, then the hull shader runs another n times per patch, processing a total of m control points. The tessellator feeds the domain shader each tessellated vertex, and the domain shader outputs the processed vertex. From here, we head to the geometry shader.

Recall that the tessellator can produce lines or triangles. This determines the incoming primitive type to our geometry shader. For the sake of this example, assume the tessellator is configured to output a triangular topology. Say we're emitting points from the geometry shader. This means the signature to the geometry shader looks something like this:

void GS(triangle DOMAIN_SHADER_OUTPUT input[3], inout PointStream<geometry_shader_output> stream);

Had the tessellator been configured to deal with isolines, then we would be using line DOMAIN_SHADER_OUTPUT input[2] instead.

So, there you have it. The geometry shader integrates seamlessly into the tessellation model. Recall that the domain shader runs per tessellated vertex, and so we have little control over each individual triangle in that stage. A geometry shader can be used to break a patch up into individual primitives, which can be independently transformed, culled, duplicated, etc. Not to mention that geometry shaders can be instanced now... that's for a future post. :)

What happens if we configure the input assembler to use a patch primitive type, but do not bind hull and domain shaders to the pipeline? Remember that the geometry shader operates on primitives, and that the new patch types are primitives... therefore, the geometry shader can operate on patch primitives!

Here are some example geometry shader signatures.

Input primitive type of point:
void GS(point VERTEX_SHADER_OUTPUT pt[1], ...);

Input primitive type of line:
void GS(line VERTEX_SHADER_OUTPUT pt[2], ...);

Input primitive type of triangle:
void GS(triangle VERTEX_SHADER_OUTPUT pt[3], ...);

Input primitive type of a 25-point patch:
void GS(InputPatch<VERTEX_SHADER_OUTPUT, 25> pt, ...);

Excited yet? You should be! What this essentially means is that you can make up your own primitive types (with anywhere from 1 point to 32 points) that the geometry shader can operate on. Wish you had a quad primitive type? Use a 4-point patch with a geometry shader and emit two triangles!

Tesselation Example


Now that I have explained a bit about tessellation, it's time for an actual example. We'll start off with a basic cubic Bézier spline renderer.

Let's start by looking at the parametric function used to compute a cubic Bézier curve. The control points are represented by P0, P1, P2 and P3.

Vertex shader


Recall that the vertex shader is run once per control point. For this example, we just pass the control points through to the next stage.

struct IA_OUTPUT
{
    float3 cpoint : CPOINT;
};

struct VS_OUTPUT
{
    float3 cpoint : CPOINT;
};

VS_OUTPUT VS(IA_OUTPUT input)
{
    VS_OUTPUT output;
    output.cpoint = input.cpoint;
    return output;
}


Hull shader


The patch constant function (HSConst below) is executed once per patch (a cubic curve in our case). Recall that the patch constant function must at least output tessellation factors. The control point function (HS below) is executed once per output control point. In our case, we just pass the control points through unmodified.

struct VS_OUTPUT
{
    float3 cpoint : CPOINT;
};

struct HS_CONSTANT_OUTPUT
{
    float edges[2] : SV_TessFactor;
};

struct HS_OUTPUT
{
    float3 cpoint : CPOINT;
};

HS_CONSTANT_OUTPUT HSConst()
{
    HS_CONSTANT_OUTPUT output;

    output.edges[0] = 1.0f; // Detail factor (see below for explanation)
    output.edges[1] = 8.0f; // Density factor

    return output;
}

[domain("isoline")]
[partitioning("integer")]
[outputtopology("line")]
[outputcontrolpoints(4)]
[patchconstantfunc("HSConst")]
HS_OUTPUT HS(InputPatch<VS_OUTPUT, 4> ip, uint id : SV_OutputControlPointID)
{
    HS_OUTPUT output;
    output.cpoint = ip[id].cpoint;
    return output;
}



Tessellator


The actual tessellator is not programmable with HLSL, but it is worth noting that the actual tessellation takes place between the hull shader and the domain shader. The tessellation factors and compile-time settings (domain, partitioning, output topology, etc.) influence the tessellator.

Domain shader


Note that up until now, we have not used the cubic Bézier curve parametric function. The domain shader is where we use this function to compute the final position of the tessellated vertices.


struct HS_CONSTANT_OUTPUT
{
    float edges[2] : SV_TessFactor;
};

struct HS_OUTPUT
{
    float3 cpoint : CPOINT;
};

struct DS_OUTPUT
{
    float4 position : SV_Position;
};

[domain("isoline")]
DS_OUTPUT DS(HS_CONSTANT_OUTPUT input, OutputPatch<HS_OUTPUT, 4> op, float2 uv : SV_DomainLocation)
{
    DS_OUTPUT output;

    float t = uv.x;

    float3 pos = pow(1.0f - t, 3.0f) * op[0].cpoint + 3.0f * pow(1.0f - t, 2.0f) * t * op[1].cpoint + 3.0f * (1.0f - t) * pow(t, 2.0f) * op[2].cpoint + pow(t, 3.0f) * op[3].cpoint;

    output.position = float4(pos, 1.0f);

    return output;
}

Because this is an example, I omitted optimizations to maintain clarity.

Pixel shader


This is a simple pixel shader that produces black lines.

struct DS_OUTPUT
{
    float4 position : SV_Position;
};

float4 PS(DS_OUTPUT input) : SV_Target0
{
    return float4(0.0f, 0.0f, 0.0f, 1.0f);
}

API setup


Control points are treated the same way as vertices.

Input assembler signature:


D3D11_INPUT_ELEMENT_DESC desc[] =
{
    {"CPOINT", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, 0, D3D11_INPUT_PER_VERTEX_DATA, 0}
};
Input assembler binding code:

UINT strides[] = {3 * sizeof(float)}; // 3 dimensions per control point (x,y,z)
UINT offsets[] = {0};
g_pd3dDC->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_4_CONTROL_POINT_PATCHLIST); // 4 control points per primitive
g_pd3dDC->IASetInputLayout(layout);
g_pd3dDC->IASetVertexBuffers(0, 1, &controlpoints, strides, offsets);

// Bind the shaders
// ...

// Render 4 control points (1 patch in this example, since we're using 4-control-point primitives).
// Rendering 8 control points simply means we're processing two 4-control-point primitives, and so forth.
// Instancing and indexed rendering works as expected.
g_pd3dDC->Draw(4, 0);

Now that the shaders are out of the way, it is a good time to explain the purpose of two tessellation factors for isolines rather than just one. Recall that a single tessellation factor can be no greater than 64. When dealing with isolines, this number is rather small; it is desirable to render a single isoline patch with a high degree of tessellation. To alleviate this problem, D3D11 allows us to specify two isoline tessellation factors: a detail factor and a density factor.

To understand what these factors mean, visualize a square. Now imagine that the detail factor describes how much to divide up the y axis, while the density factor describes how much to divide up the x axis. Now imagine connecting the dots along the x axis to form lines.

Another way to think about this: the density factor describes how much to tessellate a line, while the detail factor describes how many times to instance the tessellated line. We can find the location within a tessellated line by using SV_DomainLocation.x and we can find which line we're evaluating by using SV_DomainLocation.y. This effectively lets us chain the lines together into one, ultra-tessellated line. Darn good use of parallelism if you ask me.

Back to the example at hand: let's run some control points through this shader and see what we end up with.

Consider the following control points:

P0 = [-1, -0.8, 0]
P1 = [ 4, -1,   0]
P2 = [-4,  1,   0]
P3 = [ 1,  0.8, 0]

Attached Image: cubic1.png

Keep in mind that we're using a hard-coded density tessellation factor of 8 here, which is why the result looks low-resolution. Let's up the factor to 64 and see what we get.

Attached Image: cubic2.png

Much better.

There are a number of things we could do to improve upon this example. For example, to obtain more than 64 divisions per patch, we can use the detail factor to "instance" the line up to 64 times, and piece together the instanced, divided lines in the domain shader. Another thing we could do is create a geometry shader which transforms lines into triangles. We could procedurally perturb the control points in the vertex shader for animation effects. We could compute the tessellation factors as a function of the control points.

DirectX Graphics Diagnostic

$
0
0

Debugging a captured frame is usually a real challenge in comparison with debugging C++ code. We are dealing with hundreds of thousands of more, pixels that are produced, and in addition, there might be several functions being processed by the GPU. Typically, in modern games, there are different passes on a frame constructing it; also, there are many post-process renderings that will be applied on the final result to increase the quality of the frame. All these processes make it quite difficult to find why a specific pixel is drawn with an unexpected color during debugging! Visual Studio 2012 comes with a series of tools that intend to assist game developers. The new DirectX graphics diagnostics tools are a set of development tools integrated with Microsoft Visual Studio 2012, which can help us to analyze and debug the frames captured from a Direct3D application. Some of the functionalities of these tools come from the PIX for a Windows tool, which is a part of DirectX SDK.


note that the DirectX graphics diagnostics tools are not supported by Visual Studio 2012 Express at the time of writing this article.


In this article, we are going to explain a complete example that shows how to use graphics diagnostics to capture and analyze the graphics information of a frame. Open the final project of this article, DirectX Graphics Diagnostics, and let's see what is going on with the GPU.


Intel Graphics Performance Analyzer (Intel GPA) is another suite of graphics analysis and optimization tools that are supported by Windows Store applications. At the time of writing this article, the final release of this suite (Intel GPA 2013 R2) is able to analyze Windows 8 Store applications, but tracing the captured frames is not supported yet. Also, Nvidia Nsight™ Visual Studio Edition 3.1 is another option which supports Visual Studio 2012 and Direct3D 11.1 for debugging, profiling, and tracing heterogeneous compute and graphics application.


Capturing the frame


To start debugging the application, press Alt + F5 or select the Start Diagnostics command from Debug | Graphics | Start Diagnostics, as shown in the following screenshot:


4803_04_07.png


You can capture graphics information by using the application in two ways. The first way is to use Visual Studio to manually capture the frame while it is running, and the second way is to use the programmatic capture API. The latter is useful when the application is about to run on a computer that does not have Visual Studio installed or when you would like to capture the graphics information from the Windows RT devices.


For in the first way, when the application starts, press the Print Screen key (Prt Scr).


For in the second way, for preparing the application to use the programmatic capture, you need to use the CaptureCurrentFrame API. So, make sure to add the following header to the pch.h file:


#include <vsgcapture.h> </p>

For Windows Store applications, the location of the temp directory is specific to each user and application, and can be found at C:\users\username\AppData\Local\Packages\package family name\TempState\.


Now you can capture your frame by calling the g_pVsgDbg->CaptureCurrentFrame() function. By default, the name of the captured file is default.vsglog.


Remember, do not start the graphics diagnostics when using the programmatic capture API, just run or debug your application.


The Graphics Experiment window


After a frame is captured, it is displayed in Visual Studio as Graphics Experiment.vsglog. Each captured frame will be added to the Frame List and is presented as a thumbnail image at the bottom of the Graphics Experiment tab. This logfile contains all the information needed for debugging and tracing. As you can see in the following screenshot, there are three subwindows: the left one displays the captured frames, the right one, which is named Graphics Events List, demonstrates the list of all DirectX events, and finally, the Graphics Pixel History subwindow in the middle is responsible for displaying the activities of the selected pixel in the running frame:


4803_04_08.png


Let's start with the Graphics Pixel History subwindow. As you can see in the preceding screenshot, we selected one of the pixels on the spaceship model. Now let us take a look at the Graphics Pixel History subwindow of that pixel, as shown in the following screenshot:


4803_04_09.png


The preceding screenshot shows how this pixel has been modified by each DirectX event; first it is initialized with a specific color, then it is changed to blue by the ClearRenderTargetView function and after this, it is changed to the color of our model by drawing indexed primitives. Open the collapsed DrawIndexed function to see what really happens in the Vertex Shader and Pixel Shader pipelines. The following screenshot shows the information about each of the vertices:


4803_04_10.png


The input layout of the vertex buffer is VertexPositionNormalTangentColorTexture. In this article, you can see each vertex's value of the model's triangle. Now, we would like to debug the Pixel Shader of this pixel, so just press the green triangular icon to start debugging. As you can see in the following screenshot, when the debug process is started, Visual Studio will navigate to the source code of the Pixel Shader:


4803_04_11.png


Now you can easily debug the Pixel Shader code of the specific pixel in the DrawIndexed stage. You can also right-click on each pixel of the captured frame and select Graphics Object Table to check the Direct3D object's data.


Following screenshot shows the Graphics Event List subwindow. Draw calls in this list are typically more important events:


4803_04_12.png


The event that is displayed with the 4803_04_13.png icon marks a draw event, the one that is displayed with the 4803_04_14.png icon marks an event that occurs before the captured frame, and the user-defined event marker or the group shown with the 4803_04_15.png icon can be defined inside the application code. In this example, we mark an event (Start rendering the model) before rendering the model and mark another event (The model has been rendered) after the model is rendered. You can create these events by using the ID3DUserDefinedAnnotation:: BeginEvent, ID3DUserDefinedAnnotation:: EndEvent, and ID3DUserDefinedAnnotation:: SetMarker interfaces.


Investigating a missing object


This section will demonstrate how to investigate an error that occurs in the Vertex Shader stage so we can find out why an object is missing from our scene. First we need to find the draw call for the missing geometry. Select the Draw Event from the event list then select Graphics Pipeline Stage by navigating to Debug | Graphics | Pipeline Stages.


First select Input Assembler and view the object's geometry inside the Model Editor. If we make sure that it is the missing geometry, we must find out why it is not shown in our scene. The following are some of the common reasons:
  • Incompatible input layout: When you click on the DrawIndexed event, you will see a window that shows the device context information. Make sure the input element description of the vertex buffer is the same as the Vertex Shader input structure.
  • Error in the Vertex Shader code: The input assembler provides the right data for the Vertex Shader, so we need to see what is going on inside the Vertex Shader. Use the HLSL debugger to examine the state of variables in the Vertex Shader, especially for the output position of the Vertex Shader.
  • Error in the application source code: If neither of the previous two solutions solve the rendering issue, then we must find out where the constant buffer is being set in the source code of the program. Open the Graphics Event Call Stack window by navigating to Debug | Graphics | Event Call Stack and check the source code.
  • Let's check the device state: Open the Graphics Object Table tool and examine the device state, especially the depth stencil view, to see whether the primitives have failed the depth test or not.

Summary


In this article, you have learned DirectX graphics diagnostic, how to capture the frame, and the graphics Experiment window.

Further reading


If you are interested to explore DirectX 11.1 in more detail, check DirectX 11.1 Game Programming book by Pooya Eimandar, published by PackT publishing.

Visual Studio Graphics Content Pipeline for C# Projects

$
0
0
In this article we will look at how to use the graphics content pipeline for C# in both Visual Studio 2012 and Visual Studio 2013 for Desktop and Windows Store apps.

Since Visual Studio 2012 there has been a new graphics content pipeline and graphics debugger – including a DirectX frame debugger and HLSL debugger. The graphics content pipeline provides a number of build targets for converting common 3D and graphics assets into a usable format for DirectX applications, this includes the compilation of common mesh formats such as Collada (.dae), AutoDesk FBX (.fbx), and Wavefront (.obj) into a compiled mesh object (.cmo) file, and converting regular images into .DDS files.

Unfortunately the graphics content pipeline tasks don’t work out-of-the-box with C# because the MSBuild targets are not compatible.

Graphics content pipeline for C#


Thankfully it is quite simple to get this to work for our C# projects by making a few minor tweaks to the MSBuild target XML definitions. These build targets are defined in files named *Task.targets within the directories
  • C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Microsoft\VsGraphics
    OR
  • C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\VsGraphics
You can download the updated graphics content tasks that work with C# projects for Visual Studio 2012 and Visual Studio 2013 for both Desktop apps and Windows Store apps that are attached to this article.

After extracting the contents of the archive we can use the content tasks by right clicking our project in the solution explorer and select Unload Project, then select Edit yourprojectname.csproj. At the end of the project file insert the following:

<Project ...>
...
  <Import Project="ImageContentTask.targets" />
  <Import Project="MeshContentTask.targets" />
  <Import Project="ShaderGraphContentTask.targets" />
</Project>

Reload your project, select a graphics resource such as a 3D model and then apply the appropriate Build Action, as shown in the following screenshot.


Attached Image: MeshContentTask.png


This will result in a Character.cmo file being generated in the project’s build output directory.

Controlling the graphics content pipeline task properties


In order to pass through options to the task for C# projects it is necessary to edit the associated *.props file. This contains a section for default settings. For example the ImageContentTask allows you to determine whether or not to generate mip levels. The following XML shows the available ImageContentTask parameters found in ImageContentTask.target

<ImageContentTask
Source = "%(ImageContentTask.Identity)"
ContentOutput = "%(ImageContentTask.ContentOutput)"
Compress = "%(ImageContentTask.Compress)"
GeneratePremultipliedAlpha = "%(ImageContentTask.GeneratePremultipliedAlpha)"
GenerateMips = "%(ImageContentTask.GenerateMips)"
TLogDir = "$(ProjectDir)obj\$(ConfigurationName)"
IsDebugBuild = "$(UseDebugLibraries)"
 />

And the following XML extract shows the appropriate section within ImageContentTask.props that you would need to update.

<ImageContentTask>
    <!--Enter Defaults Here-->
    <ContentOutput Condition="'%(ImageContentTask.ContentOutput)' == '' and !$(DefineConstants.Contains('NETFX_CORE'))">$(OutDir)%(RelativeDir)%(Filename).dds</ContentOutput>
    <ContentOutput Condition="'%(ImageContentTask.ContentOutput)' == '' and $(DefineConstants.Contains('NETFX_CORE'))">$(OutDir)AppX\%(RelativeDir)%(Filename).dds</ContentOutput>
</ImageContentTask>

Conclusion


Visual Studio 2012 brought with it some significant improvements for graphics / Direct3D programming for the C++ world, however it left C# developers a little short. By integrating the graphics content pipeline with your C# project you can then make use of these great features.

Further reading


Direct3D Rendering Cookbook by Justin Stenning and published by Packt Publishing provides further examples of how to work with Direct3D in C# using SharpDX and the Visual Studio graphics content pipeline.

Article Update Log


20 Mar 2014: Initial release

Animating Characters with DirectX

$
0
0
DirectX 9 is good! It allows you to import .x files into your game projects unlike newer versions of DirectX. Why not ask yourself, "What do I want in my game?" well, the answer for me is 3D game characters and a nice 3D scene. I am writing this tutorial so that you may follow the steps required to create such a scene. Well at least the characters. I’ll leave the scene up to you. The scene will serve as an area for our characters to walk around. Unfortunately 3D studio max is not free for many people. I have no answer for this because I was unsuccessful in using Blender for animations; maybe now that I have experience I'd manage however if you are able to get hold of max, then great! There's nothing stopping you. There are plugin exporters available that do the job. And the aim of this tutorial is purely for teaching you how to create 3D DirectX games using any of the exporters available and as a refresher course for myself. DirectX is not easy but it is a lot of fun. So let's get started!

Note:  If you are familiar with the basics of setting up DirectX and Win32 applications, then you can skip straight to discussions of loading and animating models


Setting up Direct3D


Note:  if you are using a unicode character set for your projects, be sure to place an L before strings e.g. L"my string";


Let's start from the beginning: Creating a window. Setup a new empty Win32 project in Visual Studio; you will need a version of the DirectX SDK and to link in the include and library directories of the SDK into your project. This is done via project properties. In the linker->Input Additional dependencies add d3d9.lib and d3dx9.lib.

First we will include <windows.h> and create a WinMain() function and a Windows procedure WinProc(). WinMain is the entry point of the program that is executed first in a standard Windows application. Our WinMain function is outlined as:

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, PSTR cmdLine, int showCmd);

And the outline of our WndProc is:

LRESULT CALLBACK WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam);

Place these declarations at the top of the cpp file after includes so that they can be referenced throughout other functions.

Now we will want an object to handle various application events such as game initialization, updating the game and cleaning of resources. We will create a simple Game object for this. Within the Init() function we will create a window and hold a handle to it, m_mainWindow:

class Game{
public:
	Game();
	~Game();

	HRESULT Init(HINSTANCE hInstance);
	void Update(float deltaTime);
	void Cleanup();
	
private:
	HWND m_mainWindow;
};

Don't forget the semi-colon after class declaration.

We create the window class that describes our window and register it with the operating system in the Game::Init() function. We will use a basic window class for this:

	WNDCLASS wc;
	//Prepare class for use
	memset(&wc, 0, sizeof(WNDCLASS));
	
	wc.style=CS_HREDRAW | CS_VREDRAW; //Redraws the window if width or height changes.
	//Associate the window procedure with the window
	wc.lpfnWndProc=(WNDPROC)WndProc;

	wc.hInstance=hInstance;
	wc.lpszClassName="Direct3D App";

Register the window class with the OS:

	if(FAILED(RegisterClass(&wc))){
		return E_FAIL;
	}

And finally, create the window; We use WS_OVERLAPPEDWINDOW for a standard minimize, maximize and close options:

m_mainWindow = CreateWindow("Direct3D App", //The window class to use
			      "Direct3D App", //window title
			      WS_OVERLAPPEDWINDOW, //window style
			      200, //x
			      200, //y
			      CW_USEDEFAULT, //Default width
			      CW_USEDEFAULT, //Default height
			      NULL, //Parent Window
			      NULL, //Menu
			      hInstance, //Application instance
			      0); //Pointer to value parameter, lParam of WndProc
if(!m_mainWindow){return E_FAIL;}
	And after CreateWindow, add:
ShowWindow(m_mainWindow, SW_SHOW);
UpdateWindow(m_mainWindow);

//Function completed successfully
return S_OK;

In the WinMain function we must create an instance of Game and call the Init function to initialize the window:

	Game mygame;

	if(FAILED(mygame.Init(hInstance))){
		MessageBox(NULL, "Failed to initialize game", NULL, MB_OK);
		return 1;
	}

Every Windows application has a message pump. The message pump is a loop that forwards Windows messages to the window procedure by "looking" or peeking at the internal message queue. We "look" at a message within our message pump, remove it from the queue with PM_REMOVE and send it to the window procedure typically for a game. PeekMessage is better than GetMessage in the case where we want to execute game code because it returns immediately. GetMessage on the other hand would force us to wait for a message.

Now for our message pump; place this in the WinMain function:

MSG msg;
memset(&msg, 0, sizeof(MSG));
while(WM_QUIT != msg.message){
	while(PeekMessage(&msg, 0, 0, 0, PM_REMOVE) != 0){
		//Put the message in a recognizable form
		TranslateMessage(&msg);
		//Send the message to the window procedure
		DispatchMessage(&msg);
	}

	//Update the game
	//Don't worry about this just now
}

At the end of WinMain, call the application cleanup function and return with a message code:

mygame.Cleanup();
return (int)msg.wParam;

Now for our window procedure implementation; We will use a basic implementation to handle messages:

LRESULT CALLBACK WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam){
	//Handle window creation and destruction events
	switch(msg){
	case WM_CREATE:
		break;
	case WM_DESTROY:
		PostQuitMessage(0);
		break;
	}

	//Handle all other events with default procedure
	return DefWindowProc(hWnd, msg, wParam, lParam);
}

Check that the program works so far. You should be able to build the project and run it and display the window. If all goes well, we can start with Direct3D.

Now that we have created a window, let's setup Direct3D and render a background colour to the screen. We will add to our Game object so we can easily work with different parts of our application in a clear and secure manner such as creation, updating the game and application destruction. With our game object we can intialize Direct3D in an Init() function and update our game with Game::Update() and clean up DirectX resources with Game::Cleanup(). Name this class to suit your preference. I will call it Game. The m_ prefix is Hungarian notation and was developed by someone called Charles Simonyi, a Microsoft programmer. It is short for member variable. Other prefixes might be nCount, n for number, s_ for static variables. Anyway here is the Game object with added functions and variables specific to Direct3D:

#include "d3d9.h"
#include "d3dx9.h"

class Game{
public:
	Game();
	~Game();

	HRESULT Init(HINSTANCE hInstance);
	void Update(float deltaTime);
	void Render();
	void Cleanup();
	
	void OnDeviceLost();
	void OnDeviceGained();
private:
	HWND m_mainWindow;
	D3DPRESENT_PARAMETERS m_pp;
	bool m_deviceStatus;
};

Further reading recommended: Character Animation with Direct3D - Carl Granberg

We could do with a helper function to release, free up memory used by DirectX COM objects in a safe manner. We use a generic template class for this or you could create a macro for the same purpose:

dxhelper.h
template<class T>
inline void SAFE_RELEASE(T t)
{
	if(t)t->Release();
	t = NULL;
}

Don't forget to include it in your main cpp file. #include "dxhelper.h"

In our Init function we fill out the present parameters and setup Direct3D;

HRESULT Game::Init(){
	//... Window Creation Code Here
	//...

	//Direct3D Initialization

	//Create the Direct3D object
	IDirect3D9* d3d9 = Direct3DCreate9(D3D_SDK_VERSION);

	if(d3d9 == NULL)
		return E_FAIL;

	memset(&m_pp, 0, sizeof(D3DPRESENT_PARAMETERS));
	m_pp.BackBufferWidth = 800;	
	m_pp.BackBufferHeight = 600;
	m_pp.BackBufferFormat = D3DFMT_A8R8G8B8;
	m_pp.BackBufferCount = 1;
	m_pp.MultiSampleType = D3DMULTISAMPLE_NONE;
	m_pp.MultiSampleQuality = 0;
	m_pp.SwapEffect = D3DSWAPEFFECT_DISCARD;
	m_pp.hDeviceWindow = m_mainWindow;
	m_pp.Windowed = true;
	m_pp.EnableAutoDepthStencil = true;
	m_pp.AutoDepthStencilFormat = D3DFMT_D24S8;
	m_pp.Flags = 0;
	m_pp.FullScreen_RefreshRateInHz = D3DPRESENT_RATE_DEFAULT;
	m_pp.PresentationInterval = D3DPRESENT_INTERVAL_IMMEDIATE;

	if(FAILED(d3d9->CreateDevice(D3DADAPTER_DEFAULT,
			     D3DDEVTYPE_HAL,
			     m_mainWindow,
			     D3DCREATE_HARDWARE_VERTEXPROCESSING,
			     &m_pp,
			     &g_pDevice)))
		return E_FAIL;

	//We no longer need the Direct3D object
	SAFE_RELEASE(d3d9);

	//Success the Direct3D device was created
	return S_OK;
}

Let's go over the parameters quickly; BackBufferWidth specifies the width of the offscreen buffer, the back buffer. The offscreen buffer is a memory segment that we render a scene to and it becomes the on-screen buffer, what we actually see on the screen when it is flipped with the front buffer, the on-screen buffer. Similarly with the BackBufferHeight. We specify 8 bits for alpha, 8 bits for red, 8 bits for green and 8 bits for blue so this is 32-bit true colour allocated for our buffer. We specify that there will only be one back buffer; the reason we might have two back buffers is to speed up the rendering to the screen e.g. while the onscreen buffer is diplayed you could prepare two offscreen buffers so you could flip one and then the next is ready to be displayed so you could flip the other immediately. Multisampling is a technique that improves the quality of an image but takes up more processing time. So we specify D3DMULTISAMPLE_NONE. We specify SWAPEFFECT_DISCARD to remove the oncreen buffer when it is swapped with a backbuffer; so the backbuffer becomes the front and the old front is deleted. m_pp.hDeviceWindow is the window to render to. Windowed can be true, displaying the scene in a window, or false to display the scene fullscreen. We set m_pp.EnableAutoDepthStencil to true to enable depth bufferring; where a depth buffer is used effectively causes 3D objects in the world to overlap correctly; a z value will be specified for each pixel of the depth buffer and this will effectively enable depth testing which is basically a test comparison of the z value of each pixel; If a pixel's z value is less, nearer to the screen, than another pixels z value, it is closer to the screen so will be written to the offscreen buffer. We use a default refresh rate and immediate buffer swapping. The other type of buffer swapping could be D3DPRESENT_INTERVAL_DEFAULT, default interval, which might be the screen refresh rate.

Next we create the device with a call to d3d9->CreateDevice().

We need to specify a global Direct3D device object so that we can use the device anywhere in the program.

IDirect3DDevice9* g_pDevice = NULL;

Then the CreateDevice() function will create the device object. We use the default display adapter and then D3DDEVTYPE_HAL to use the hardware abstraction layer, hardware graphics acceleration for our rendering of the scene. This is much faster as opposed to software rendering. Hardware means the graphics card in this case. We specify to use hardware vertex processing too. And then we pass the present parameters structure that describes properties of the device to create. And lastly we pass the g_pDevice variable to retrieve a handle to the newly created device.

Now, before we continue with animation we must do a bit of device handling. For example if the user does ALT+TAB our device might be lost and we need to reset it so that our resources are maintained. You'll notice we have onDeviceLost and onDeviceGained functions in our game object. These will work hand-in-hand with the deviceStatus variable to handle a device lost situation. After a device has been lost we must reconfigure it with onDeviceGained().

To check for a lost device we check the device cooperative level for D3D_OK. If the cooperative level is not D3D_OK then our device can be in a lost state or in a lost and not reset state. This should be regularly checked so we will put the code in our Game::Update() function.

#define DEVICE_LOSTREADY 0
#define DEVICE_NOTRESET 1
HRESULT coop = g_pDevice->TestCooperativeLevel();

if(coop != D3D_OK)
{
	if(coop == D3DERR_DEVICELOST)
	{
		if(m_deviceStatus == DEVICE_LOSTREADY)
			OnDeviceLost();		
	}
	else if(coop == D3DERR_DEVICENOTRESET)
	{
		if(m_deviceStatus == DEVICE_NOTRESET)
			OnDeviceGained();
	}
}

Our OnDeviceLost function and OnDeviceGained look like:

void Game::OnDeviceLost()
{
	try
	{
		//Add OnDeviceLost() calls for DirectX COM objects
		m_deviceStatus = DEVICE_NOTRESET;
	}
	catch(...)
	{
		//Error handling code
	}
}

void Game::OnDeviceGained()
{
	try
	{
		g_pDevice->Reset(&m_pp);
		//Add OnResetDevice() calls for DirectX COM objects
		m_deviceStatus = DEVICE_LOSTREADY;
	}
	catch(...)
	{
		//Error handling code
	}
}

When the program starts we have to set the m_deviceStatus variable so that we can use it. So in our Game::Game() constructor set the variable:

Game::Game(){
	m_deviceStatus = DEVICE_LOSTREADY;
}

Now we need to implement the Render and Cleanup functions. We will just clear the screen and free up memory in these functions.

void Game::Render()
{
	g_pDevice->Clear(0, NULL, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, 0xff00ff00, 1.0f, 0);

	if(SUCCEEDED(g_pDevice->BeginScene()))
	{
		// Perform some rendering to back buffer.

		g_pDevice->EndScene();
	}

	// Swap buffers.
	g_pDevice->Present(NULL, NULL, NULL, NULL);
}

void Game::Cleanup()
{
	SAFE_RELEASE(g_pDevice);
}

Finally, we want to render our scene and update it. Remember the Update() method handles device lost events.

We want our game to run with a consistent frame rate so that Update() updates the game at the same frame rate on all PCs. If we just updated the game inconsistently without frame rate, our game would be faster on faster computers and slower on slower computers and also might speed up or slow down if we do not specify a frame rate.

Therefore we use GetTickCount() and pass a change in time to our Update() function. GetTickCount() returns the number of milliseconds that have elapsed since the system was started. We record the start time, and subtract this from the current time of each iteration of the loop. We then set the new start time; repeat this calculation and get the change in time and pass this value to Update(deltaTime).

Our message loop now is defined as:

//Get the time in milliseconds
DWORD startTime = GetTickCount();
float deltaTime = 0;

MSG msg;
memset(&msg, 0, sizeof(MSG));
while(msg.message != WM_QUIT){
	if(PeekMessage(&msg, 0, 0, 0, PM_REMOVE)){
		//Put the message in a recognizable form
		TranslateMessage(&msg);
		//Send the message to the window procedure
		DispatchMessage(&msg);
	}
	else{
		//Update the game
		DWORD t=GetTickCount();
		deltaTime=float(t-startTime)*0.001f;
		//Pass time in seconds
		mygame.Update(deltaTime);
		//Render the world
		mygame.Render();
		startTime = t;
	}
}

Now we have a complete Direct3D framework we can begin with loading an animated model. I will supply you the code so far and then talk about how we can load an animation hierarchy:

#include <windows.h>
#include "d3d9.h"
#include "d3dx9.h"
#include "dxhelper.h"

#define DEVICE_LOSTREADY 0
#define DEVICE_NOTRESET 1

IDirect3DDevice9* g_pDevice = NULL;

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, PSTR cmdLine, int showCmd);
LRESULT CALLBACK WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam);

class Game{
public:
	Game();
	~Game();

	HRESULT Init(HINSTANCE hInstance);
	void Update(float deltaTime);
	void Render();
	void Cleanup();
	
	void OnDeviceLost();
	void OnDeviceGained();
private:
	HWND m_mainWindow;
	D3DPRESENT_PARAMETERS m_pp;
	bool m_deviceStatus;
};

Game::Game(){
	m_deviceStatus = DEVICE_LOSTREADY;
}
Game::~Game(){
}
HRESULT Game::Init(HINSTANCE hInstance){
	WNDCLASS wc;
	//Prepare class for use
	memset(&wc, 0, sizeof(WNDCLASS));
	
	wc.style=CS_HREDRAW | CS_VREDRAW; //Redraws the window if width or height changes.
	//Associate the window procedure with the window
	wc.lpfnWndProc=(WNDPROC)WndProc;

	wc.hInstance=hInstance;
	wc.lpszClassName=L"Direct3D App";

	if(FAILED(RegisterClass(&wc))){
		return E_FAIL;
	}

	m_mainWindow = CreateWindow(L"Direct3D App", //The window class to use
			      L"Direct3D App", //window title
				  WS_OVERLAPPEDWINDOW, //window style
			      200, //x
			      200, //y
			      CW_USEDEFAULT, //Default width
			      CW_USEDEFAULT, //Default height
				  NULL,
				  NULL,
			      hInstance, //Application instance
			      0); //Pointer to value parameter, lParam of WndProc

	if(!m_mainWindow){return E_FAIL;}

	ShowWindow(m_mainWindow, SW_SHOW);
	UpdateWindow(m_mainWindow);

	//Direct3D Initialization

	//Create the Direct3D object
	IDirect3D9* d3d9 = Direct3DCreate9(D3D_SDK_VERSION);

	if(d3d9 == NULL)
		return E_FAIL;

	memset(&m_pp, 0, sizeof(D3DPRESENT_PARAMETERS));
	m_pp.BackBufferWidth = 800;
	m_pp.BackBufferHeight = 600;
	m_pp.BackBufferFormat = D3DFMT_A8R8G8B8;
	m_pp.BackBufferCount = 1;
	m_pp.MultiSampleType = D3DMULTISAMPLE_NONE;
	m_pp.MultiSampleQuality = 0;
	m_pp.SwapEffect = D3DSWAPEFFECT_DISCARD;
	m_pp.hDeviceWindow = m_mainWindow;
	m_pp.Windowed = true;
	m_pp.EnableAutoDepthStencil = true;
	m_pp.AutoDepthStencilFormat = D3DFMT_D24S8;
	m_pp.Flags = 0;
	m_pp.FullScreen_RefreshRateInHz = D3DPRESENT_RATE_DEFAULT;
	m_pp.PresentationInterval = D3DPRESENT_INTERVAL_IMMEDIATE;

	if(FAILED(d3d9->CreateDevice(D3DADAPTER_DEFAULT,
			     D3DDEVTYPE_HAL,
			     m_mainWindow,
			     D3DCREATE_HARDWARE_VERTEXPROCESSING,
			     &m_pp,
			     &g_pDevice)))
		return E_FAIL;

	//We no longer need the Direct3D object
	SAFE_RELEASE(d3d9);

	//Success the Direct3D device was created
	return S_OK;
}

void Game::Update(float deltaTime){

	HRESULT coop = g_pDevice->TestCooperativeLevel();

	if(coop != D3D_OK)
	{
		if(coop == D3DERR_DEVICELOST)
		{
			if(m_deviceStatus == DEVICE_LOSTREADY)
				OnDeviceLost();		
		}
		else if(coop == D3DERR_DEVICENOTRESET)
		{
			if(m_deviceStatus == DEVICE_NOTRESET)
				OnDeviceGained();
		}
	}

}
void Game::Cleanup()
{
	SAFE_RELEASE(g_pDevice);
}

void Game::OnDeviceLost()
{
	try
	{
		//Add OnDeviceLost() calls for DirectX COM objects
		m_deviceStatus = DEVICE_NOTRESET;
	}
	catch(...)
	{
		//Error handling code
	}
}

void Game::OnDeviceGained()
{
	try
	{
		g_pDevice->Reset(&m_pp);
		//Add OnResetDevice() calls for DirectX COM objects
		m_deviceStatus = DEVICE_LOSTREADY;
	}
	catch(...)
	{
		//Error handling code
	}
}

void Game::Render()
{
	if(m_deviceStatus==DEVICE_LOSTREADY){
		g_pDevice->Clear(0, NULL, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, 0xff00ff00, 1.0f, 0);

		if(SUCCEEDED(g_pDevice->BeginScene()))
		{
			// Perform some rendering to back buffer.

			g_pDevice->EndScene();
		}

		// Swap buffers.
		g_pDevice->Present(NULL, NULL, NULL, NULL);
	}
}

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, PSTR cmdLine, int showCmd){

	Game mygame;

	if(FAILED(mygame.Init(hInstance))){
		MessageBox(NULL, L"Failed to initialize game", NULL, MB_OK);
		return 1;
	}

	//Get the time in milliseconds
	DWORD startTime = GetTickCount();
	float deltaTime = 0;

	MSG msg;
	memset(&msg, 0, sizeof(MSG));
	while(WM_QUIT != msg.message){
		while(PeekMessage(&msg, 0, 0, 0, PM_REMOVE) != 0){
			//Put the message in a recognizable form
			TranslateMessage(&msg);
			//Send the message to the window procedure
			DispatchMessage(&msg);
		}

		//Update the game
		DWORD t=GetTickCount();
		deltaTime=float(t-startTime)*0.001f;
		//Pass time in seconds
		mygame.Update(deltaTime);
		//Render the world
		mygame.Render();
		startTime = t;
	}

	mygame.Cleanup();
	return (int)msg.wParam;
}

LRESULT CALLBACK WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam){
	//Handle window creation and destruction events
	switch(msg){
	case WM_CREATE:
		break;
	case WM_DESTROY:
		PostQuitMessage(0);
		break;
	}

	//Handle all other events with default procedure
	return DefWindowProc(hWnd, msg, wParam, lParam);
}

Loading an Animation Hierarchy


Hopefully you have found the tutorial useful up until now. In this part I will start by setting the perspective projection and then cover the steps needed to load and animate a mesh. We will be animating our mesh object with hardware acceleration. This involves creating an HLSL .fx file.

Requirements

You will need a method of producing an .x file with animation info. For this I will be using 3d studio max 2014 and an exporter plugin. There are a number of plugins available. If you can't get hold of this then you may have to use Blender, however I don't know if the .x files produced with Blender are sufficient for this tutorial. And I don't have the incentive to find out. So I leave this to the reader. Hopefully you will find a way to produce an .x file with an animation hierarchy in a suitable format for loading into a DirectX application.

Setting up a perspective projection

We have three matrices to create. World, View and Projection. The World matrix represents the world transformation for a set of objects such as meshes. This is the position of these meshes in the game world. The view matrix represents the camera. It has a lookAt component, an Up component that specifies the up direction of the world, y or z. And also an eye component that is the "look from" component; the position of the camera.

Our perspective matrix defines the "fish eye" factor or field-of-view, which is the angle range we can see from our eye. This is typically 45 degrees. We must also specify the aspect ratio, which is the number of width pixels in correspondence with the number of height pixels; the ratio of width to height. Typically set this to WINDOW_WIDTH / WINDOW_HEIGHT. And lastly, the z near plane and z far plane must be specified, these are the cut-off points of our view. Typically the z near plane of a projection frustum is z=1.0f. Z here is the point of cut-off; anything closer to the eye than 1.0 is cut-off, not rendered. Similarly anything further away than zfar is cut-off, not rendered too.

We add the describing transform states with the D3DTS_ constants in the device SetTranform function passing each tranform matrix as parameters. This is done in our Render() function.

D3DXMATRIX view, proj, world;
D3DXMatrixIdentity(&world); //Set to no transform

//Position the camera behind the z axis(z=-10)
//Make Y the up direction of the world
//And look at the centre of the world origin(0,0,0)
D3DXMatrixLookAtLH(&view, &D3DXVECTOR3(0,0,-10.0f), &D3DXVECTOR3(0.0f, 1.0f, 0.0f), &D3DXVECTOR3(0.0f, 0.0f, 0.0f));

//Make the field of view 45 degrees
//Use the window dimensions for aspect ratio so rendered image is not
//stretched when window is resized.
//Set znear to 1.0f and zfar to 10000.0f
RECT rc;
GetClientRect(m_mainWindow, &rc);
D3DXMatrixPerspectiveFovLH(&proj, D3DXToRadian(45.0), (float)rc.width / (float)rc.height, 1.0f, 10000.0f);

g_pDevice->SetTransform(D3DTS_WORLD, &world);
g_pDevice->SetTransform(D3DTS_VIEW, &view);
g_pDevice->SetTransform(D3DTS_PROJECTION, &proj);

Now that we have the scene set up, the basic camera and projection ready - it's time to load a model with animation.

Loading an Animation Hierarchy

In real life humans and animals have bones. Likewise, our game characters have bones. A bone in DirectX is represented with the D3DXFRAME structure. You may just think of this structure as a bone. A bone may have a parent and a sibling. For example a parent bone might be the upper arm and a child bone might be the lower arm. When you move your upper arm, your lower arm, forearm, moves with it. That is why a forearm is the child. If you move your lower arm however, the upper arm is not affected. And hence that is why the upper arm is a parent. Each bone may have a transformation, a rotation and position for example. A sibling is just a bone that shares the same parent as another bone(D3DXFRAME). Let's look at the D3DXFRAME structure to see how we can represent a bone in code.

struct D3DXFRAME{
	LPSTR Name;
	D3DXMATRIX TransformationMatrix;
	LPD3DXMESHCONTAINER pMeshContainer;
	D3DXFRAME* pFrameSibling;
	D3DXFRAME* pFrameFirstChild;
};

A bone has a name, a transformation, and optionally a mesh associated with it; and optionally a sibling and a child. With this structure we can represent a whole hierarchy or skeleton in other words. By associating sibling bones and child bones we can link all the D3DXFRAME bones together, which in turn will be a representation of a skeleton such as a human or an animal.

The names of the bones could be "right leg", "left leg", "right forearm" for example to give you an idea. The transformations defined in a basic D3DXFRAME are the local bone transformations. These are local to the bones in contrast with the world transformations, which are the actual transformations in the world; the final transforms. In our game we require the world transformations of the bones to render them at the exact positions in the world; so we will extend this structure to include them. For easy reference we will call the new structure Bone.

struct Bone: public D3DXFRAME
{
	D3DXMATRIX WorldMatrix;
};

The WorldMatrix is the combination of a bone's local matrix with its parent's WorldMatrix. This is simply a multiplication of the two so that the child bone inherits the transformation of its parent.

We must traverse the bone hierarchy to calculate each of the new matrices. This is done with the following recursive function:

void CalculateWorldMatrices(Bone* child, D3DXMATRIX* parentMatrix){
	D3DXMatrixMultiply(&child->WorldMatrix, &child->TransformationMatrix, parentMatrix);

	//Each sibling has same parent as child
	//so pass the parent matrix of this child
	if(child->pFrameSibling){
		CalculateWorldMatrices((Bone*)child->pFrameSibling, parentMatrix);
	}

	//Pass child matrix as parent to children
	if(child->pFrameFirstChild){
		CalculateWorldMatrices((Bone*)child->pFrameFirstChild, &child->WorldMatrix);
	}
}

Then to calculate all of the bones matrices that make up the skeleton we call CalculateWorldMatrices on the root node, the parent bone of the hierarchy with the identity matrix as the parent. This will traverse all children and siblings of those children and build each of the world matrices.

To load a bone hierarchy from an .x file we must implement the ID3DXAllocateHierarchy interface. This interface defines 4 functions that we must implement ourselves. Then we pass the implemented object in a call to D3DXLoadMeshHierarchyFromX(). That will create a skeleton of a character. And after we have the skeleton we can apply skinning by using an HLSL effect file to make our character effectively have skin.

To implement the functions declared in ID3DXAllocateHierarchy we provide a new class definition that inherits from it. We will call this class AllocateHierarchyImp. The new class and inherited functions effectively looks like this:

Note:  STDMETHOD is a macro defined as virtual HRESULT __stdcall. It declares a virtual function that is a function that must be implemented by an inheriting class and uses the standard calling convention __stdcall.


class AllocateHierarchyImp : public ID3DXAllocateHierarchy{
public:
	STDMETHOD(CreateFrame)(LPCSTR Name, 
					      LPD3DXFRAME* ppNewFrame);

	STDMETHOD(CreateMeshContainer)(LPCSTR Name, 
			CONST D3DXMESHDATA* pMeshData,
			CONST D3DXMATERIAL* pMaterials,
			CONST D3DXEFFECTINSTANCE* pEffectInstances,
			DWORD NumMaterials,
			CONST DWORD* pAdjacency,
			LPD3DXSKININFO pSkinInfo,
			LPD3DXMESHCONTAINER* ppNewMeshContainer);

	STDMETHOD(DestroyFrame)(LPD3DXFRAME pFrameToFree);

	STDMETHOD(DestroyMeshContainer)(LPD3DXMESHCONTAINER pMeshContainerBase);
};

In these functions we handle the allocation of memory for bones and the deallocation of memory we allocated ourselves for bones and associated bone meshes. CreateFrame is fairly simple; we just allocate memory for a bone with new Bone; and allocate memory for the name of the bone with new char[strlen(Name)+1];.

CreateMeshContainer on the other hand is more complicated. Bear in mind these functions are called by the D3DXLoadMeshHierarchyFromX() function. Information about the mesh we are loading is passed to these functions.

Before I jump into the code for these functions we should consider a class that will provide the mechanism of loading a skinned mesh separately for each of our animated characters. Thus we will create a class called SkinnedMesh that caters for each individual character. This class is outlined as:

SkinnedMesh.h
class SkinnedMesh{
public:
	SkinnedMesh();
	~SkinnedMesh();
	void Load(WCHAR filename[]);
	void Render(Bone* bone);
private:
	void CalculateWorldMatrices(Bone* child, D3DXMATRIX* parentMatrix);
	void AddBoneMatrixPointers(Bone* bone);

	D3DXFRAME* m_pRootNode;
};

We need to define a mesh container structure so that we can hold a mesh associated with each bone and prepare for skinning the mesh. Like with the Bone when we extended the D3DXFRAME, we extend the D3DXMESHCONTAINER to represent a mesh associated with a bone. The D3DXMESHCONTAINER looks like this:

struct D3DXMESHCONTAINER{
	LPSTR Name;
	D3DXMESHDATA MeshData;
	LPD3DXMATERIAL pMaterials;
	LPD3DXEFFECTINSTANCE pEffects;
	DWORD NumMaterials;
	DWORD* pAdjacency;
	LPD3DXSKININFO pSkinInfo;
	D3DXMESHCONTAINER* pNextMeshContainer;
};

Time for a cup of coffee.

MeshData holds the actual mesh. pMaterials holds material and texture info. pEffects may hold effects associated with the mesh. pAdjacency holds adjacency info, which is indices of faces, triangles, adjacent to each other. And pSkinInfo holds skinning info such as vertex weights and bone offset matrices that are used to add a skin effect to our animations.

And our extended version to cater for skinning looks like this:

SkinnedMesh.h
struct BoneMesh: public D3DXMESHCONTAINER
{
	vector<D3DMATERIAL9> Materials;
	vector<IDirect3DTexture9*> Textures;

	DWORD NumAttributeGroups;
	D3DXATTRIBUTERANGE* attributeTable;
	D3DXMATRIX** boneMatrixPtrs;
	D3DXMATRIX* boneOffsetMatrices;
	D3DXMATRIX* localBoneMatrices;
};

The attribute table is an array of D3DXATTRIBUTERANGE objects. The AttribId of this object corresponds to the material or texture to use for rendering a subset of a mesh. Because we are working with a COM object for the mesh, the memory for it will be deallocated after the function completes unless we add a reference using AddRef(). So we call AddRef on the pMesh of MeshData: pMeshData->pMesh->AddRef().

Notice boneMatrixPtrs; these are pointers to world bone transformation matrices in the D3DXFRAME structures. This means if we change the world transform of a bone these will be affected or in other words these will point to the changed matrices. We need the world transformations to perform rendering. When we animate the model, we call CalculateWorldMatrices to calculate these new world matrices that represent the pose of the bones, the bone transformations. A boneMesh may be affected by a number of bones so we use pointers to these bones matrices instead of saving them twice or multiple times for bone meshes and also so we only have to update one bone matrix with our animation controller for that bone to take effect. Using the modified local transformations of the model from the animation controller, we get these matrices from the bones that influence a mesh. We then use the boneoffset matrices to calclate the local transformations as these are not stored and offset matrices are stored in pSkinInfo. When we multiply a bone offset matrix with a bone's world matrix we get the bone's local transformation without it's affecting parent's world transformation. We want to do this when we pass the transformation to the skinning shader. The mesh in this case is the mesh associated with one of the bones, found in the D3DXFRAME structure pMeshContainer.

Now that you understand the Bone and BoneMesh structures somewhat we can begin to implement the ID3DXAllocateHierachy. We'll start with CreateFrame. In this function we allocate memory for the name of the bone as well as memory for the bone itself:

SkinnedMesh.cpp
HRESULT AllocateHierarchyImp::CreateFrame(LPCSTR Name, LPD3DXFRAME *ppNewFrame)
{
	Bone *bone = new Bone;
	memset(bone, 0, sizeof(Bone));

	if(Name != NULL)
	{
		//Allocate memory for name
		bone->Name = new char[strlen(Name)];
		strcpy(bone->Name, Name);
	}

	//Prepare Matrices
	D3DXMatrixIdentity(&bone->TransformationMatrix);
	D3DXMatrixIdentity(&bone->WorldMatrix);

	//Return the new bone
	*ppNewFrame = (D3DXFRAME*)bone;

	return S_OK;
}

And the DestroyFrame function should deallocate memory allocated in CreateFrame:

HRESULT AllocateHierarchyImp::DestroyFrame(LPD3DXFRAME pFrameToFree) 
{
	if(pFrameToFree)
	{
		//Free up memory
		if(pFrameToFree->Name != NULL)
			delete [] pFrameToFree->Name;

		delete pFrameToFree;
	}
	pFrameToFree = NULL;

    return S_OK; 
}

A single mesh can have a number of bones influencing it. Each bone has a set of vertex weights associated with it corresponding to each vertex of the mesh. The weights determine how much a vertex is affected by each bone. The greater the weight of a bone, the more a vertex will be affected by the transformation of that bone. This is how the skin works. The weights of each vertex of the model that correspond to affecting bones are passed to the HLSL effect file that performs the skinning on the GPU. And the bones that influence the vertex are passed to the HLSL file as well through the BLENDINDICES0 semantic. These weights and bones are stored in the .x file and therefore are loaded in the MeshData of the D3DXMESHCONTAINER BoneMesh. By rendering a subset of the BoneMesh, we pass these weight and bone parameters to the effect file, which in turn performs the skinning based on these values. We pass the values to the HLSL file during the SkinnedMesh rendering function.

Look back at the BoneMesh structure. Bone offset matrices are inverse matrices that tranform a bone from world space to local space; that is to the transformation uneffected by its parent. These are stored in the .x file and can be retrieved with pSkinInfo of D3DXSKININFO.

To skin a mesh with hardware skinning we need to put the vertex data of each mesh to render in a format that has vertex weight and influencing bone indices. The bone indices are indices of bones that affect the vertex. Each bone in the .x file has a set of vertices that are "attached" to that bone and a weight for each vertex that determines how much the bone affects that vertex. To include this information in our mesh, we must convert the mesh to an "indexed blended" mesh. When we convert the mesh to an indexed blended mesh, the additional bone indices and vertex weights are added to our vertex information within the mesh.

Now is a good time to show you how to load a mesh container since you know about the elements that make up one. Here it is again:

struct BoneMesh: public D3DXMESHCONTAINER
{
	vector<D3DMATERIAL9> Materials;
	vector<IDirect3DTexture9*> Textures;

	DWORD NumAttributeGroups;
	D3DXATTRIBUTERANGE* attributeTable;
	D3DXMATRIX** boneMatrixPtrs;
	D3DXMATRIX* boneOffsetMatrices;
	D3DXMATRIX* localBoneMatrices;
};

localBoneMatrices are calculated when we render the mesh by using boneMatrixPtrs and boneOffsetMatrices so we can pass the local bone matrix array to the shader to perform skinning. In our CreateMeshContainer function we must allocate memory for these matrix arrays and obtain pointers to the world transformations of our bones.

HRESULT AllocateHierarchyImp::CreateMeshContainer(LPCSTR Name,				CONST D3DXMESHDATA *pMeshData,
			CONST D3DXMATERIAL *pMaterials,
			CONST D3DXEFFECTINSTANCE *pEffectInstances,
			DWORD NumMaterials,
			CONST DWORD *pAdjacency,
			LPD3DXSKININFO pSkinInfo,
			LPD3DXMESHCONTAINER *ppNewMeshContainer)
{
	//Allocate memory for the new bone mesh
	//and initialize it to zero
	BoneMesh *boneMesh = new BoneMesh;
	memset(boneMesh, 0, sizeof(BoneMesh));

	//Add a reference to the mesh so the load function doesn't get rid of 	//it
	pMeshData->pMesh->AddRef();
	//Get the device
	IDirect3DDevice9 *pDevice = NULL;
	pMeshData->pMesh->GetDevice(&pDevice);

	//Get the mesh materials and create related textures
	D3DXMATERIAL mtrl;
	for(int i=0;i<NumMaterials;i++){
		memcpy(&mtrl, &pMaterials[i], sizeof(D3DXMATERIAL));

		boneMesh->Materials.push_back(mtrl.MatD3D);

		IDirect3DTexture9* pTexture = NULL;
		//If there is a texture associated with this material, load it into
		//the program
		if(mtrl.pTextureFilename != NULL){
			wchar_t fname[MAX_PATH];
			memset(fname, 0, sizeof(wchar_t)*MAX_PATH);
			mbstowcs(fname, mtrl.pTextureFilename, MAX_PATH);
			D3DXCreateTextureFromFile(pDevice, fname, &pTexture);
			boneMesh->Textures.push_back(pTexture);
		}
		else{
			//Make sure we have the same number of elements in 				//Textures as we do Materials
			boneMesh->Textures.push_back(NULL);
		}
	}

	//Now we need to prepare the mesh for hardware skinning; as 	//mentioned earlier we need the bone offset matrices, and these are
	//stored in pSkinInfo.  Here we get the bone offset matrices and 	//allocate memory for the local bone matrices that influence the 	//mesh.  But of course this is only if skinning info is available.
	if(pSkinInfo != NULL){
		boneMesh->pSkinInfo = pSkinInfo;
		pSkinInfo->AddRef();

		DWORD maxVertInfluences = 0;
		DWORD numBoneComboEntries = 0;
		ID3DXBuffer* boneComboTable = 0;

		//Convert mesh to indexed blended mesh to add additional 			//vertex components; weights and influencing bone indices.
		//Store the new mesh in the bone mesh.
		pSkinInfo->ConvertToIndexedBlendedMesh(pMeshData->pMesh, 
			D3DXMESH_MANAGED | D3DXMESH_WRITEONLY,  
			30, 
			0, //Not used
			0, //Not used
			0, //Not used
			0, //Not used
			&maxVertInfluences,
			&numBoneComboEntries, 
			&boneComboTable,
			&boneMesh->MeshData.pMesh);

		if(boneComboTable != NULL) //Not used
			boneComboTable->Release();

		//As mentioned, the attribute table is used for selecting 				//materials and textures to render on the mesh.  So we aquire 			//it here.
		boneMesh->MeshData.pMesh->GetAttributeTable(NULL, &boneMesh->NumAttributeGroups);
		boneMesh->attributeTable = new D3DXATTRIBUTERANGE[boneMesh->NumAttributeGroups];
		boneMesh->MeshData.pMesh->GetAttributeTable(boneMesh->attributeTable, NULL);

		//Next we load the offset matrices and allocate memory for 			//the local bone matrices.  skin info holds the number of bones 		//that influence this mesh in terms of the bones used to create 		//the skin.
		int NumBones = pSkinInfo->GetNumBones();
		boneMesh->boneOffsetMatrices = new D3DXMATRIX[NumBones];
		boneMesh->localBoneMatrices = new D3DXMATRIX[NumBones];

		for(int i=0;i < NumBones;i++){
			boneMesh->boneOffsetMatrices[i] = *(boneMesh->pSkinInfo->GetBoneOffsetMatrix(i));
		}
	}

	//Return new mesh
	*ppNewMeshContainer = boneMesh;
	return S_OK;
}

Hopefully you understood that code to create a mesh ready for animating.

But before we animate it we have to provide the mesh deallocation implementation. This is simply a case of deallocating the memory we allocated ourselves and releasing the COM objects used:

HRESULT AllocateHierarchyImp::DestroyMeshContainer(LPD3DXMESHCONTAINER pMeshContainerBase)
{
	BoneMesh* boneMesh = (BoneMesh*)pMeshContainerBase;

	//Release textures
	int nElements = boneMesh->Textures.size();
	for(int i=0;i<nElements;i++){
		if(boneMesh->Textures[i] != NULL)
			boneMesh->Textures[i]->Release();
	}

	//Delete local bone matrices and offset if we have skin info
	if(boneMesh->pSkinInfo != NULL){
		delete[] boneMesh->localBoneMatrices;
		delete[] boneMesh->boneOffsetMatrices;
		delete[] boneMesh->attributeTable;
	}

	//Release mesh and skin info
	if(boneMesh->pSkinInfo){boneMesh->pSkinInfo->Release();}
	if(boneMesh->MeshData.pMesh){boneMesh->MeshData.pMesh->Release();}

	return S_OK;
}

Now that the AllocateHierarchy functions are implemented we can go ahead and call D3DXLoadMeshHierarchyFromX passing the AllocateHierarchy object to it. This is done in the SkinnedMesh::Load function. When we call this function we retrieve a pointer to the root bone of the hierarchy that allows us to traverse the whole hierarchy with this one bone to calculate new matrices for animation for example. With just the root node we can add matrix pointers to each of our meshes that correspond to the world transformations of each bone and effectively will point to matrices that make up the animation when we animate the model.

In our SkinnedMesh::Load function is where we call D3DXLoadMeshHierarchyFromX. First we need to create an instance of the AllocateHierarchy; Our SkinnedMesh implementation then becomes:

SkinnedMesh::SkinnedMesh(){
}
SkinnedMesh::~SkinnedMesh(){
}
void SkinnedMesh::Load(WCHAR filename[]){
	AllocateHierarchyImp boneHierarchy;

	D3DXLoadMeshHierarchyFromX(filename, D3DXMESH_MANAGED, 
							   g_pDevice, &boneHierarchy,
							   NULL, &m_pRootNode, NULL);
}
void SkinnedMesh::Render(Bone *bone){
}
void SkinnedMesh::CalculateWorldMatrices(Bone *child, D3DXMATRIX *parentMatrix){
	D3DXMatrixMultiply(&child->WorldMatrix, &child->TransformationMatrix, parentMatrix);

	//Each sibling has same parent as child
	//so pass the parent matrix of this child
	if(child->pFrameSibling){
		CalculateWorldMatrices((Bone*)child->pFrameSibling, parentMatrix);
	}

	//Pass child matrix as parent to children
	if(child->pFrameFirstChild){
		CalculateWorldMatrices((Bone*)child->pFrameFirstChild, &child->WorldMatrix);
	}
}
void SkinnedMesh::AddBoneMatrixPointers(Bone *bone){
}

Now in our game object we can test whether the hierarchy of one of our .x files can be loaded. Place SkinnedMesh model1; in the private members of your Game object. And call model1.Load("Your file.x") in the Init() function of Game. You will need to put an .x file containing a bone hierarchy into the directory that the game runs from. You can test whether a hierarchy was loaded using a break point. Hopefully all is good. You loaded a bone hierarchy.

We still have a few things to setup before model animation can occur. With the HLSL shader that we create, we define an interpolation of vertices from a start pose to a final pose. Each start and end pose of an animation is known as an animation set and these should typically be stored in the .x file. Whenever we pass a vertex to the HLSL effect it will be updated to a new position and then that new position will be passed the HLSL effect file and be updated and this is how we create a skinned animation but we need to setup the matrices that make this animation work. We must pass the bone tranformations to this effect file uneffected by their parents in order to calculate new vertex positions with the vertex shader. For these tranformations we calculate them in our Render function using bone pointer matrices and bone offset matrices. So we need to add bone matrix pointers to each bone mesh of the hierarchy. Then when we render the mesh we can easily access these from the array of matrix pointers to the world matrices of the bones that affect the mesh skin.

These bone matrix pointers are added to each mesh that is affected by bones; we use pointers so that when a bone's transformation changes the pointers will be affected and resultingly the mesh skin too. To add pointers we must traverse the whole hierarchy and check for a bone mesh; if one exists and has skinning information, we add the matrix pointers to affecting bones. The affecting bones are the bones contained in pSkinInfo:

void SkinnedMesh::AddBoneMatrixPointers(Bone *bone){
	if(bone->pMeshContainer != NULL){
		BoneMesh* boneMesh=(BoneMesh*)bone->pMeshContainer;

		if(boneMesh->pSkinInfo != NULL){
			//Get the bones affecting this mesh' skin.
			int nBones=boneMesh->pSkinInfo->GetNumBones();

			//Allocate memory for the pointer array
			boneMesh->boneMatrixPtrs = new D3DXMATRIX*[nBones];

			for(int i=0;i<nBones;i++){
				Bone* bone=(Bone*)D3DXFrameFind(m_pRootNode, boneMesh->pSkinInfo->GetBoneName(i));
				if(bone != NULL){
					boneMesh->boneMatrixPtrs[i]=&bone->WorldMatrix;
				}
				else{
					boneMesh->boneMatrixPtrs[i]=NULL;
				}
			}
		}
	}

	//Traverse Hierarchy
	if(bone->pFrameSibling){AddBoneMatrixPointers((Bone*)bone->pFrameSibling);}
	if(bone->pFrameFirstChild){AddBoneMatrixPointers((Bone*)bone->pFrameFirstChild);}
}

We call this function passing the root node to it to setup all the pointers to influencing mesh bones world matrices. This is done after loading the hierarchy. Also add a call to CalculateWorldMatrices in the Load function to add world matrices to each of the bones. If you don't add these the model will be displayed as a jumble of meshes.

void SkinnedMesh::Load(WCHAR filename[]){
	AllocateHierarchyImp boneHierarchy;

	if(SUCCEEDED(D3DXLoadMeshHierarchyFromX(filename, D3DXMESH_MANAGED, 
							   g_pDevice, &boneHierarchy,
							   NULL, &m_pRootNode, NULL))){
		D3DXMATRIX identity;
		D3DXMatrixIdentity(&identity);
		CalculateWorldMatrices((Bone*)m_pRootNode, &identity);

		AddBoneMatrixPointers((Bone*)m_pRootNode);
	}
}

Now we need to free the memory of the matrix pointers when the skinned mesh is destroyed. This again involves traversing the hierarchy and freeing memory used by pointers. Add the following function to the SkinnedMesh class:

void SkinnedMesh::FreeBoneMatrixPointers(Bone *bone){
	if(bone->pMeshContainer != NULL){
		BoneMesh* boneMesh=(BoneMesh*)bone->pMeshContainer;

		if(boneMesh->boneMatrixPtrs != NULL){
			delete[] boneMesh->boneMatrixPtrs;
		}
	}

	//Traverse Hierarchy
	if(bone->pFrameSibling){FreeBoneMatrixPointers((Bone*)bone->pFrameSibling);}
	if(bone->pFrameFirstChild){FreeBoneMatrixPointers((Bone*)bone->pFrameFirstChild);}
}

And call it on SkinnedMesh destruction.

SkinnedMesh::~SkinnedMesh(){
	FreeBoneMatrixPointers((Bone*)m_pRootNode);
}

Animating a Hierarchy


Finally everything is set up for us to add the skinning effect and animate the model with an animation controller. First we create the effect; we will make this global so we can use it in the Render function of our SkinnedMesh.

ID3DXEffect* g_pEffect=NULL;

This effect will be our interface to the HLSL effect file. We can upload variables to the file through this interface. Create a new effect file called skinning.fx; this is simply an ASCII text file. This will be our shader that performs skinning. We create the effect with D3DXCreateEffectFromFile(). Call this from the Init function of your Game object. Just set flags to D3DXSHADER_DEBUG for now because the shader is not known to work yet.

//Create effect
//Only continue application if effect compiled successfully
if(FAILED(D3DXCreateEffectFromFile(g_pDevice, L"skinning.fx", NULL, NULL, D3DXSHADER_DEBUG, NULL, &g_pEffect, NULL))){
	return E_FAIL;
}

Modify OnDeviceLost and OnDeviceGained to cater for the effect file:

void Game::OnDeviceLost()
{
	try
	{
		//Add OnDeviceLost() calls for DirectX COM objects
		g_pEffect->OnLostDevice();
		m_deviceStatus = DEVICE_NOTRESET;
	}
	catch(...)
	{
		//Error handling code
	}
}

void Game::OnDeviceGained()
{
	try
	{
		g_pDevice->Reset(&m_pp);
		//Add OnResetDevice() calls for DirectX COM objects
		g_pEffect->OnResetDevice();
		m_deviceStatus = DEVICE_LOSTREADY;
	}
	catch(...)
	{
		//Error handling code
	}
}

Now we need to implement the Render function of the SkinnedMesh and the HLSL file. In this file we calculate both skinning and lighting of the model. We first define our vertex structure that corresponds to the vertex structure of the index blended mesh; this will be the input vertex data to the shader:

struct VS_INPUT_SKIN
{
     	float4 position: POSITION0;
   	float3 normal: NORMAL;
     	float2 tex0: TEXCOORD0;
	float4 weights: BLENDWEIGHT0;
     	int4 boneIndices: BLENDINDICES0;
};

Here we get the position of the vertex that we will modify in the shader; and we get the normal, which is the direction vector of the vertex and used in lighting. The weights are the weights of the affecting bones, which can be found by the bone indices. We use the weights and the bone matrices to determine the new position of the vertex for each vertex and normals. Therefore we store the matrices in a matrix array as follows:

extern float4x4 BoneMatrices[40];

To calculate the new vertex positions we apply each bone weight to a multiplication of the original vertex position and the bone transformation matrix and sum up the results. However there is one more thing - the combination of weights must add up to 1, which is equivelent to 100%. See, each weight applies a percentage of effect on a vertex so they must add up to 1. Therefore we calculate the last weight as 1-totalWeights; one minus the sum total so that they definitely add up to one.

Here is the complete shader for performing skin and lighting:

//World and View*Proj Matrices
matrix matWorld;
matrix matVP;
//Light Position
float3 lightPos;
//Texture
texture texDiffuse;

//Skinning variables
extern float4x4 BoneMatrices[40]; 
extern int MaxNumAffectingBones = 2;

//Sampler
sampler DiffuseSampler = sampler_state
{
   Texture = (texDiffuse);
   MinFilter = Linear;   MagFilter = Linear;   MipFilter = Linear;
   AddressU  = Wrap;     AddressV  = Wrap;     AddressW  = Wrap;
   MaxAnisotropy = 16;
};

//Vertex Output / Pixel Shader Input
struct VS_OUTPUT
{
     float4 position : POSITION0;
     float2 tex0     : TEXCOORD0;
     float  shade	 : TEXCOORD1;
};

//Vertex Input
struct VS_INPUT_SKIN
{
     float4 position : POSITION0;
     float3 normal   : NORMAL;
     float2 tex0     : TEXCOORD0;
	 float4 weights  : BLENDWEIGHT0;
     int4   boneIndices : BLENDINDICES0;
};

VS_OUTPUT vs_Skinning(VS_INPUT_SKIN IN)
{
    VS_OUTPUT OUT = (VS_OUTPUT)0;

    float4 v = float4(0.0f, 0.0f, 0.0f, 1.0f);
    float3 norm = float3(0.0f, 0.0f, 0.0f);
    float lastWeight = 0.0f;
    
    IN.normal = normalize(IN.normal);
    
    for(int i = 0; i < MaxNumAffectingBones-1; i++)
    {
	//Multiply position by bone matrix
	v += IN.weights[i] * mul(IN.position, BoneMatrices[IN.boneIndices[i]]);
	norm += IN.weights[i] * mul(IN.normal, BoneMatrices[IN.boneIndices[i]]);
	    
	//Sum up the weights
	lastWeight += IN.weights[i];
    }
    //Make sure weights add up to 1
    lastWeight = 1.0f - lastWeight;
    
    //Apply last bone
    v += lastWeight * mul(IN.position, BoneMatrices[IN.boneIndices[MaxNumAffectingBones-1]]);
    norm += lastWeight * mul(IN.normal, BoneMatrices[IN.boneIndices[MaxNumAffectingBones-1]]);
    
    //Get the world position of the vertex
    v.w = 1.0f;
	float4 posWorld = mul(v, matWorld);
    OUT.position = mul(posWorld, matVP);
    //Output texture coordinate is same as input
    OUT.tex0 = IN.tex0;
    
	//Calculate Lighting
    norm = normalize(norm);
    norm = mul(norm, matWorld);
	OUT.shade = max(dot(norm, normalize(lightPos - posWorld)), 0.2f);
     
    return OUT;
}

//Pixel Shader
float4 ps_Lighting(VS_OUTPUT IN) : COLOR0
{
	float4 color = tex2D(DiffuseSampler, IN.tex0);
	return color * IN.shade;
}

technique Skinning
{
	pass P0
	{
		VertexShader = compile vs_2_0 vs_Skinning();
		PixelShader  = compile ps_2_0 ps_Lighting();
	}
}

The technique tells the system to use vertex and pixel shader version 2 and to pass the output of vs_Skinning to the input of ps_Lighting. The pixel shader ps_Lighting essentially "Lights" the pixels of the texture.

Now that we have the vertex shader, we can use it. And render the model. In this Render function we get a handle to the technique with g_pEffect->GetTechniqueByName("Skinning") and pass the mesh to it; and glory be the shader will perform its work. The Render function will be called on the root bone and traverse the bone hierarchy and render each of the mesh containers associated with the bones of the hierarchy.

Here is the Render function:

void SkinnedMesh::Render(Bone *bone){
	//Call the function with NULL parameter to use root node
	if(bone==NULL){
		bone=(Bone*)m_pRootNode;
	}

	//Check if a bone has a mesh associated with it
	if(bone->pMeshContainer != NULL)
	{
		BoneMesh *boneMesh = (BoneMesh*)bone->pMeshContainer;

		//Is there skin info?
		if (boneMesh->pSkinInfo != NULL)
		{		
			//Get the number of bones influencing the skin
			//from pSkinInfo.
			int numInflBones = boneMesh->pSkinInfo->GetNumBones();
			for(int i=0;i < numInflBones;i++)
			{
				//Get the local bone matrices, uneffected by parents
				D3DXMatrixMultiply(&boneMesh->localBoneMatrices[i],
								   &boneMesh->boneOffsetMatrices[i], 
								   boneMesh->boneMatrixPtrs[i]);
			}

			//Upload bone matrices to shader.
			g_pEffect->SetMatrixArray("BoneMatrices", boneMesh->localBoneMatrices, boneMesh->pSkinInfo->GetNumBones());

			//Set world transform to identity; no transform.
			D3DXMATRIX identity;				
			D3DXMatrixIdentity(&identity);

			//Render the mesh
			for(int i=0;i < (int)boneMesh->NumAttributeGroups;i++)
			{
				//Use the attribute table to select material and texture attributes
				int mtrlIndex = boneMesh->attributeTable[i].AttribId;
				g_pDevice->SetMaterial(&(boneMesh->Materials[mtrlIndex]));
				g_pDevice->SetTexture(0, boneMesh->Textures[mtrlIndex]);
				g_pEffect->SetMatrix("matWorld", &identity);
				//Upload the texture to the shader
				g_pEffect->SetTexture("texDiffuse", boneMesh->Textures[mtrlIndex]);
				D3DXHANDLE hTech = g_pEffect->GetTechniqueByName("Skinning");
				g_pEffect->SetTechnique(hTech);
				g_pEffect->Begin(NULL, NULL);
				g_pEffect->BeginPass(0);

				//Pass the index blended mesh to the technique
				boneMesh->MeshData.pMesh->DrawSubset(mtrlIndex);

				g_pEffect->EndPass();
				g_pEffect->End();
			}
		}
	}

	//Traverse the hierarchy; Rendering each mesh as we go
	if(bone->pFrameSibling!=NULL){Render((Bone*)bone->pFrameSibling);}
	if(bone->pFrameFirstChild!=NULL){Render((Bone*)bone->pFrameFirstChild);}
}

Now that we have the shader in place and the render function we can aquire an animation controller and control the animations of the model.

We get an animation controller from the D3DXLoadHierarchyFromX function. We will add this controller to the SkinnedMesh class in the private members:

ID3DXAnimationController* m_pAnimControl;

Then in our SkinnedMesh Load function add this as a parameter to D3DXLoadMeshHierarchyFromX:

D3DXLoadMeshHierarchyFromX(filename, D3DXMESH_MANAGED, 
							 g_pDevice, &boneHierarchy,
							 NULL, &m_pRootNode, 								 	&m_pAnimControl)

The way animation works is there are a number of animation sets stored in the model or .x file that correspond to different animation cycles such as a walk animation or jump. Depending on the character, each animation has a name and we can set the current animation using its name. If we have different characters we will want to access or set different animations per character using name strings, e.g. SetAnimation("Walk"), SetAnimation("Sit") like that. For this we can use a map of strings to animation IDs. With this map we can get the ID of each animation set by using the name along with the map. A map has an array of keys and values associated with those keys. The key here is the name of the animation and the value is its animation set ID.

Lastly once we have the animation sets we can play an animation by calling m_pAnimControl->AdvanceTime(time, NULL);

First let's get the animation sets and store their names and IDs in a map. For this we will create a function in our SkinnedMesh class void GetAnimationSets(). Create a map by including <map> and make sure using namespace std; is at the top of the SkinnedMesh cpp file. Create the map called map<string, dword="DWORD">animationSets; For this you will also need to include <string>. using namespace std means we can use map and string without std::map or std::string for example if you didn't know already. We will also add another two functions void SetAnimation(string name) and void PlayAnimation(float time).

The skinned mesh now looks like:
class SkinnedMesh{
public:
	SkinnedMesh();
	~SkinnedMesh();
	void Load(WCHAR filename[]);
	void Render(Bone* bone);

private:
	void CalculateWorldMatrices(Bone* child, D3DXMATRIX* parentMatrix);
	void AddBoneMatrixPointers(Bone* bone);
	void FreeBoneMatrixPointers(Bone* bone);
	//Animation functions
	void GetAnimationSets();

	D3DXFRAME* m_pRootNode;
	ID3DXAnimationController* m_pAnimControl;
	map<string, int>animationSets;
public:
	void SetAnimation(string name);
	void PlayAnimation(D3DXMATRIX world, float time);
};

We get and save the animation sets to our map with the following function:

void SkinnedMesh::GetAnimationSets(){
	ID3DXAnimationSet* pAnim=NULL;

	for(int i=0;i<(int)m_pAnimControl->GetMaxNumAnimationSets();i++)
	{
		pAnim=NULL;
		m_pAnimControl->GetAnimationSet(i, &pAnim);

		//If we found an animation set, add it to the map
		if(pAnim != NULL)
		{
			string name = pAnim->GetName();
			animationSets[name]=i;//Creates an entry
			pAnim->Release();
		}
	}
}

We set the active animation set with SetAnimation:

void SkinnedMesh::SetAnimation(string name){
	ID3DXAnimationSet* pAnim = NULL;
	//Get the animation set from the name.
	m_pAnimControl->GetAnimationSet(animationSets[name], &pAnim);

	if(pAnim != NULL)
	{
		//Set the current animation set
		m_pAnimControl->SetTrackAnimationSet(0, pAnim);
		pAnim->Release();
	}
}

When we update the game we call the following function to play the active animation:

void SkinnedMesh::PlayAnimation(D3DXMATRIX world, float time){
	//The world matrix here allows us to position the model in the scene.
	m_pAnimControl->AdvanceTime(time, NULL);//Second parameter not used.
	//Update the matrices that represent the pose of animation.
	CalculateWorldMatrices((Bone*)m_pRootNode, &world);
}

Usage:

In the SkinnedMesh::Load function add the following line on hierarchy load success:

//Save names of sets to map
GetAnimationSets();

In the Game::Init function, set the active animation set - for example if we have a walk cycle animation:

string set="Walk";
model1.SetAnimation(set);

And then play the animation in Game::Update():

D3DXMATRIX identity;
D3DXMatrixIdentity(&identity);
model1.PlayAnimation(identity, deltaTime*0.5f);

There is one more thing we often want to do. That is render static non-moving meshes that are part of the hierarchy. We might for example have a mesh that doesn't have skin info; so we need to save this; Notice that we create an indexed blended mesh only if there is skin info. Well now we need to save the mesh if there is no skin info:

In our CreateMeshContainer function:

if(pSkinInfo != NULL){
	//...
}
else{
	//We have a static mesh
	boneMesh->MeshData.pMesh = pMeshData->pMesh;
	boneMesh->MeshData.Type = pMeshData->Type;
}

Now we have the static meshes saved in BoneMesh structures we can render them. But we need first to light the mesh with a lighting shader; Add this to the shader file:

//Vertex Input
struct VS_INPUT
{
     float4 position : POSITION0;
     float3 normal   : NORMAL;
     float2 tex0     : TEXCOORD0;
};

VS_OUTPUT vs_Lighting(VS_INPUT IN)
{
    VS_OUTPUT OUT = (VS_OUTPUT)0;

	float4 posWorld = mul(IN.position, matWorld);
    float4 normal = normalize(mul(IN.normal, matWorld));
    
    OUT.position = mul(posWorld, matVP);
    
	//Calculate Lighting
	OUT.shade = max(dot(normal, normalize(lightPos - posWorld)), 0.2f);
	
	 //Output texture coordinate is same as input
    OUT.tex0=IN.tex0;
     
    return OUT;
}

technique Lighting
{
    pass P0
    {	
        VertexShader = compile vs_2_0 vs_Lighting();
        PixelShader  = compile ps_2_0 ps_Lighting();        
    }
}

Now we can render the static meshes of our model; We have a static mesh if there is no pSkinInfo. Here we set the lighting technique to active and render the mesh with texturing:

if (boneMesh->pSkinInfo != NULL)
{
	//...
}
else{
	//We have a static mesh; not animated.
	g_pEffect->SetMatrix("matWorld", &bone->WorldMatrix);

	D3DXHANDLE hTech = g_pEffect->GetTechniqueByName("Lighting");
	g_pEffect->SetTechnique(hTech);

	//Render the subsets of this mesh with Lighting
	for(int mtrlIndex=0;mtrlIndex<(int)boneMesh->Materials.size();mtrlIndex++){
		g_pEffect->SetTexture("texDiffuse", boneMesh->Textures[mtrlIndex]);
		g_pEffect->Begin(NULL, NULL);
		g_pEffect->BeginPass(0);

		//Pass the index blended mesh to the technique
		boneMesh->MeshData.pMesh->DrawSubset(mtrlIndex);

		g_pEffect->EndPass();
		g_pEffect->End();
	}
}

That's us done! We have an animated model that we can work with. We can set the active animation and render it with skinning!

Saving an .x File


In this part we will export an animation hierarchy from 3d studio max.

Note:  The vertex shader only supports a fixed number of bones for a character so you will need to create a model with about max 40 bones.


  1. Place the exporter plugin in the plugins directory of max. Fire up 3d Studio Max. New Empty Scene.
  2. Go to Helpers in the right-hand menu.
  3. Select CATParent.
  4. Click and drag in the perspective viewport to create the CATParent triangle object.
  5. Double click Base Human in the CATRig Load Save list.
  6. Model a rough human shaped character around the bones. E.g. create new box edit mesh.
  7. Click on the CATRig triangle. Go to motion in the menu tabs.
  8. Click and hold Abs and select the little man at the bottom.
  9. Press the stop sign to activate the walk cycle.
  10. Go to modifiers and select skin modifier. On the properties sheet you will see Bones:; click Add. Select all the bones. And click Select. Now if you play the animation the skin should follow the bones.
  11. Lastly, you need to add a material to the mesh, click the material editor icon and drag a material to the mesh.
  12. Now export the scene or selected model using the exporter plugin. You will need to export vertex normals animation and select Y up as the worlds up direction to suit the game. Select Animation in the export dialog. And select skinning! can't forget that. Set a frame range e.g. 0 to 50 and call the set "walk". Click Add Animation Set. And Save the model.
  13. Now you can check that the model exported successfully using DXViewer utility that comes with the DirectX SDK.
  14. Now you can try loading the model into the program.

You may have to adjust the camera. E.g.

D3DXMatrixLookAtLH(&view, &D3DXVECTOR3(0,0.0f,-20.0f), &D3DXVECTOR3(0.0f, 10.0f, 0.0f), &D3DXVECTOR3(0.0f, 1.0f, 0.0f));

Article Update Log


12 April 2014: Initial release
17 April 2014: Updated included download
22 April 2014: Updated
24 April 2014: Updated

Tom Sloper's Format for Game Design Specifications

$
0
0

June 26, 1997

GAME TITLE(TM)

Game System (Hardware)

The primary goals of a game design are to (1) excite and (2) inform the reader. First paragraph must excite the reader and make the reader want to read more of the first page. The first page must be interesting, concise, and informative, and must make the reader want to read the remaining pages. When the reader has finished reading the entire design, if the reader does not have a clear understanding of what you want the game to be, you have failed to communicate your vision of the game.

I …

Game Design : Immersiveness

$
0
0

According to Webster's dictionary immersiveness is not a word, however in the world of computer game design it is quite a crucial word. Webster defines immerse to mean, "to plunge into something that surrounds or covers" and to engross and absorb. This is exactly what you want your game to do to your players, engross and absorb them. Therefore to talk about the quality a game can have to engross or absorb players would be the game's immersiveness.

Gameplay

Meaningful Interactions

Every computer game is made up of two basic components, output for the user to interpret such …

Practical use of Vector Math in Games

$
0
0

For a beginner, geometry in 3D space may seem a bit daunting. 2D on paper was hard enough, but now, geometry in 3D? Good news: use of trigonometry in graphics is rare and avoided for multitude of reasons. We have other tools which are easier to understand and use. You may recognize our old friend here - a vector. 

file(2).png

This article will introduce you to 3D vectors and will walk you through several real-world usage examples. Even though it focuses on 3D, most things explained here also work for 2D. Article assumes familiarity with algebra and geometry, some programming language …


'U-MMMO' DevLog - Day 14

$
0
0

Another week has passed and more progress has been made.

I ended the last dev log on the fact that the character creation process worked but was pretty rough. I'm happy to report that has been fixed. Upon account creation, the server now assigns a new character token automatically to the player account which is then referenced at character creation for whether not the account CAN make a character. This token is also durable, so if a player wishes to make more than one character, the token can be reused and not have to be granted again like it …

Dev Diary #103 - Last Week, In Summary 02

$
0
0

Hello and welcome to this weeks Dev Diary - the Summary Edition!

I have decided to use summaries in this Dev Diary for the time being, in order to concentrate more on research, project planning and creation and less on writing about it, until I have something more major to update. My aim is to do at least 3 things a week, for at least couple of hours per day and write a summary about the things I have concentrated on the last week.

Here is what my last week consisted of regarding the game project;

  • Research on ancient Uralic cultures …

New Hero Added 01-09-2020

New Hero Added 01-20-2020

A Journey into Video Game Art

$
0
0

Have you ever wondered how someone gets into making art for video games as a freelancer? Well today I have an interview with @IroPagis for you an up and coming video game artiest.

Hey. So alright, I’ll tell you about my journey, even though it’s kinda all over the place.I started drawing early on and kept at it. I wanted to make games, then I wanted to make traditional art, then I wanted to make MANGA~ So I was making manga when I was 12 years old all the way to 19 yro.

When I was 18 I finished …

Now made with Unity!

$
0
0

Finally! There it is, the new version of Imagine Earth using the Unity graphics engine. It took us a while to port all visualizations and UI elements, but the result it totally worth it – especially because we can move much faster now based on a modern graphics engine with a lot of tools and features that make the live a lot easier for us.

Full savegame compatibility

Before we have a look into what is new, we want to point out that your existing savegames will still work. If you experience any problems related to that, please note that we …

Fragment’s Moonrise | #39 Damage, Resistance, Status Effects, Auras, and Elements, Part 6

$
0
0

Welcome to our thirty-ninth blog post!

 

 

Before we begin, we would like to announce we’ll be working on launching a Kickstarter campaign soon, so please look forward to it!

Our last posts consisted of us going over Status Effect variables. In this one, we just have a few remaining variables to discuss, before we can proceed onto Units and Research.

invisGif

 

 

Invisibility will play a key role in a lot of both Unit and Boss designs, even being a huge factor in a future race we want to develop for this game, the Gyen.

It also …


Life finds a way

$
0
0

A demo of a tight game inside Toy Hunter

The Elixir of life

Characters

Buildings

Warmaster Line Up

Viewing all 17063 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>