|Warning! This article has yet to be cleaned up to a higher standard of quality, per our Cleanup Project. It may contain factual errors and nonsense, as well as spelling, grammar and structure issues, or simply structure problems. Reader's discretion is advised until fixing is done.|
You can help clean up this page by correcting spelling and grammar, removing factual errors and rewriting sections to ensure they are clear and concise, and moving some elements when appropriate.
The Source engine is a game engine developed by Valve Corporation. Its unique features include a large degree of mod-ability, flexibility, an artist-driven, shader-based renderer, industry-leading lip-sync and facial expression technology, and a powerful, efficient, and completely network-enabled physics system.
Source supports both 32-bit and 64-bit environments and the Microsoft Windows, macOS, Linux, Android, Xbox, Xbox 360, and PlayStation 3 platforms. The Source engine debuted in October 2004 with Half-Life 2 and Counter-Strike: Source. Since its release it has been heavily modernized by Valve, with the "Orange Box Update", when the aging Source engine was updated with new features for the release of Half-Life 2: Episode Two, Team Fortress 2 and Portal.
- 1 Technology overview
- 2 Modularity and notable upgrades
- 2.1 High dynamic range rendering
- 2.2 Facial animation 2
- 2.3 Dynamic lighting and shadowing 2
- 2.4 Next-gen renderer
- 2.5 Landscape and Flora Rendering
- 2.6 Soft-Particle system
- 2.7 Cinematic physics
- 2.8 Multiprocessor optimizations
- 2.9 Unconfirmed future technology
- 3 Origins
- 4 Common issues
- 5 Valve Developer Community
- 6 Licensed Source games
- 7 References
- 8 External links
- For a full overview, see Source Engine Features on the Valve Developer Community Wiki.
- DirectX 9 with Shader Model 3.0, OpenGL, OpenGL ES
- 3D skybox; an area can be displayed as a skybox at up to 16x its actual size with full 3D parallax.
- High dynamic range rendering
- Any animation can merge seamlessly with any other animation at any time.
- Inverse kinematics to ensure that characters' limbs react to environments.
- Networked physics
- Originate from Havok 2 physics engine
- Highly-tuned by Valve
- Software only
- Fully networked with low bandwidth requirements
- Vehicle physics including torque, power, gears, tire material, suspension and mode.
- Full DSP system
- Full surround sound support
- Software only
- Low and high frequency components merged depending on surrounding area and relative position of origin.
- Supports DirectX 6 and upwards
- Modularity allows for all current and future Source projects to scale back to DirectX 6 if they desire.
- Facial expressions
- Full range of human and non-human facial movements
- Based on Professor Paul Ekman's Facial Action Coding System
- Over eighty-four "digital muscles"
- Works in tandem with facial expressions
- Auto-generated, but completely configurable
- Stored in sound file itself for localization
Modularity and notable upgrades
Source is designed from the ground up to be highly modular. This allows for the easy upgrade and modification of certain features without breaking other areas of the engine, or breaking engine continuity (that is to say, there need be no 'version jumps' from 1.0 to 2.0). When coupled with Steam, these updates can be distributed retroactively and automatically. For instance, if Source is upgraded to support hardware physics, every Source title on Steam will instantly benefit. Entirely new features such as High Dynamic Range (HDR) Rendering have been shown to require developer input, however.
High dynamic range rendering
First seen in Day of Defeat: Source, then shortly after in Half-Life 2: Lost Coast, HDR was the first major instance of Source's modularity in use. However, whilst in theory all Source engine games and mods were able to use HDR immediately after its release, the game code required to 'hook in' to the new system was not made available to modders until eleven months later, on August 4, 2006. Official licensees and Valve themselves have all made use of the technology since its release.
Facial animation 2
When Half-Life 2: Episode One was released on 1 June 2006, it introduced the second version of Valve's proprietary facial animation system. Ken Birdwell explains the upgrade's features in the game's commentary track:
"When we designed the Half-Life 2 facial system back in 2000, our goal was to get a natural-looking performance at a moderate distance. For Episode One, we wanted to extend the characters' facial systems to support more intense performances with a wider range of facial expressions, that would hold up better at close range. These facial improvements included increasing the detail around the eyes and mouth, increasing the number of facial shape targets – think of these as movements of muscle groups – by about 50%, rewriting the rules that control how these shapes blend, and increasing the intensity of many of our existing shapes."
Dynamic lighting and shadowing 2
A new dynamic lighting and shadow mapping system was released for Source, replacing the somewhat limp existing system. It launched with the various other new Source features with Half-Life 2: Episode Two.
|Current Lighting and Shadowing System||Dynamic Lighting and Shadowing 2|
|Dynamic shadows in a map always come from the same predetermined direction.||Dynamic shadows in a map always react dynamically to every light source.|
|Models do not self-shadow or cast shadows onto other objects.||Models can self-shadow and cast shadows onto the world and other objects.|
|Dynamic shadows do not blend with lightmapped shadows and cast through all objects except world geometry.||Dynamic shadows are more unified with static shadows and don't cast through models.|
|Every object is allowed only one dynamic shadow.||Any object can cast multiple dynamic soft shadows.|
|The player's flashlight merely illuminates an area.||The player's flashlight casts shadows from models and world geometry.|
Landscape and Flora Rendering
Large, open natural environments with heavy foliage, traditionally a weakness for the Source engine, have been supported as of Half-Life 2: Episode Two. The updates were also made available for MOD teams to use shortly after the release.
During the July 2006 Electronic Arts Summer Showcase press conference, Gabe Newell mentioned that a new soft-particle system would be introduced into the Source Engine in the upcoming title Half-Life 2: Episode Two. It was first demonstrated in the July 19 Team Fortress 2 teaser, which showed a remarkably realistic flamethrower in its closing moments.
During the July 2006 Electronic Arts Summer Showcase press conference, it was revealed that former Weta Digital employee Gray Horsfield, special effects destruction lead on The Return of the King and King Kong among other roles, was building a "Cinematic Physics" system for Source. GameSpy described the new system in their conference report:
"The idea behind this is to give players the opportunity to experience in-game physics in action on a grander scale. As an example of Cinematic Physics in action, a clip from Half-Life 2: Episode Two was shown of a huge bridge collapsing across a vast ravine."
The system adds the following features to Source's physics simulator:
- Deforming objects — before, physics models could not be modified except through animation
- Dynamic crumbling of brush geometry — before, lines of separation had to be specified by the mapper
Cinematic Physics supports a keyframe system, but its exact nature is currently unclear. It could be that an animator creates a largely complete but low-detail sequence which then sees details added by the physics system, or it could be that an animator creates a handful of single-frame states which are then used as motion targets for the ensuing simulation (in a manner not dissimilar to the Endorphin NaturalMotion technology).
Either method results in a drastic reduction of developer input, thus allowing the creation of far more complex scenes than before with the same budget. It is currently unclear both whether or not keyframes are strictly required, and what number are needed to create a scene as complex as the bridge collapse demonstration.
As a part of the Source engine's transition to next-generation consoles, multiprocessor optimizations have been added, resulting in faster processing on PC hardware with dual or quad core systems and the Xbox 360 and PlayStation 3 consoles. Gabe Newell:
"Yes. We definitely think that content needs to move forward. For example, one of the things we're reacting to is the speed at which microprocessors are coming out. So, Intel has very aggressively moved up delivery of desktop processors with four different cores; we'll have support for that in Episode Two, and we'll definitely go back to affect, Episode One or Half-Life 2 or Counter-Strike Source, so they can take advantage of that. We'll definitely try to keep the existing games - especially the multiplayer games - current as technology evolves."
Valve has demonstrated the new multi core optimizations which use a multi-threading style they dub "hybrid threading." A Source multi-threading update and benchmark are expected to be released before Half-Life 2: Episode Two, though a date for either and the content of the benchmark are currently unknown.
Unconfirmed future technology
With color correction and film grain already released, Valve intends to add other cinematic effects such as motion blur and depth of field to Source when hardware is able to render it to their satisfaction. The effects are accomplished with an accumulation buffer for quality, creating enormous overhead; for instance, twenty to thirty motion blur frames need to be rendered for every one frame that the user sees. For a constant frame rate of thirty frames per second, a video card is required to produce between six hundred and nine hundred frames per second. This causes late 2005-era hardware to require a full two seconds to render each frame.
Motion blur and depth of field can be seen in several of Valve's promotional videos, including:
Image-Based Rendering is a technique in which 2D elements are manipulated to appear in a 3D world. In the context of a 3D game, it delivers a significant performance boost by replacing 3D geometry that is far enough away for the transition to be imperceptible with a 2D image. Implementation of the technology in this role can be found applied to Rome: Total War's soldiers, Far Cry's forests, and various objects in Guild Wars' environments, such as buildings and flora.
"There's this technology that was really exciting that I’d like to see us get into production, which is a different approach to rendering complexity: Moving things into and out of an image domain and then seamlessly interpolating between those motions as the player moves around. So that everything close to the player is physical and geometry, and everything really far away from the player is an image, but the player has no way of telling that if the player does it properly and things can fly out and come back."
The June 2006 Day of Defeat: Source update included an "image-based texture blending shader", and Half-Life 2: Episode Two's expansive environments seem ideally suited to the technology, but it has yet to be dated or even officially announced.
One of the technologies developed for Half-Life 2's Xbox release was file streaming, wherein a map's resources could be loaded as the player moved around in it rather than in one operation before playability. With the system in place, loading times were reduced to as little as fifteen seconds. The system expanded on the caching system already implemented. There is no time frame for its release, as implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems (see Stutter section).
Although Valve has explicitly stated that the Source engine has been built internally from the ground up, rumors and myths persist that it is instead merely derived from the original Quake codebase via Valve's GoldSrc offshoot. The primary reasons for this are the manner in which the engine uses similar development interfaces to GoldSrc (to aid transitioning developers), and John Carmack's comment on his blog that "there are still bits of early Quake code in Half-Life 2", expanded through hearsay to be a confirmation that large swathes of code are identical, when no such conclusion can be drawn from the statement. There remains no solid proof that Source is derived from the GoldSrc codebase — and indeed, given the fact that the 2003 code leak did not produce any such claims it can only be assumed that no incriminating evidence was to be found.
However, it is known that Source was developed part-by-part, slowly replacing the GoldSrc engine in Valve's internal projects. This explains its modular nature, and suggests that, even if Source was not derived from GoldSrc, GoldSrc was at the very least modified to plug into it during development.
The Source Engine uses a caching system, whereby the loading of certain resources is handled and managed on the fly, rather than in a single operation behind a traditional loading screen. Texture and sound data are the primary areas in which this occurs. Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested.
Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.
While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).
When Half-Life 2 was first released and stuttering became a widely-known problem, community member Mark McWilliams set up a page covering the issue and Valve's communication and work on resolving it. Several updates were released by Valve, the effects of which varied from complete fixes for some users to previously smooth systems becoming "infected" with the problem.
- Example (.WMV), showing a relatively low level of stutter. The hitches coincide with the drawing of new areas (a process that is by design not visible without debugging tools).
More recently (February 7, 2006), changes to the Source engine were introduced alongside a beta test of Steam's Friends chat service, with the aim of 'narrowing down' the problem. The update featured a limited implementation of Source's file streaming system (see above). Generally, the response was very positive.
In Left 4 Dead 2, this had been resolved. Instead of stuttering, the sounds will simply just stop.
The Source Engine suffers from an error whereby the asynchronous loading (see Stutter, above) of a new sound file will cause the engine to lock up with looping audio. Because of the nature of DirectX, once the engine enters such a state it will remain on the screen unless the user can blindly terminate the program, or reboot their computer. The error occurs in a standard Windows library associated with on-board sound cards, and in some cases can be resolved by decreasing DirectSound Acceleration.
Reports of looping audio crashes increased around the release of Episode One. While it is likely that the spike was simply because there was an unusually high number of people playing the game, changes to the engine, of which there were many for the game, cannot be ruled out. It has been noted that people who have been able to play Half-Life 2 without any crashes and audio errors in fact do find Episode 1 to be more prone to that problem. A steampowered.com forum thread is dedicated to discussing the problem and attempting to work out solutions, although the experience of the thread contributors indicates that most suggested workarounds and fixes do not remedy the problem.
Valve Developer Community
On 28 June 2005, Valve opened the Valve Developer Community Wiki. The VDC replaced the previously available Source SDK documentation with a full MediaWiki-powered community site. Within a matter of days Valve reported that "the number of useful articles [had] nearly doubled". These new articles covered the previously undocumented Counter-Strike: Source bot (added by the bot's author, Mike Booth) and Half-Life 2 AI, source control for Source engine mods, and more.
Licensed Source games
- Alien Swarm
- Counter-Strike: Source
- Counter-Strike: Global Offensive
- Day of Defeat: Source
- Dota 2
- Half-Life: Source
- Half-Life 2
- Half-Life 2: Deathmatch
- Half-Life 2 episodic series (Episode One and Episode Two)
- Half-Life 2: Lost Coast
- Left 4 Dead
- Left 4 Dead 2
- Portal 2
- Team Fortress 2
- Other developers
- The Crossing
- Dark Messiah of Might and Magic
- Deep 6
- The History Channel's ShootOut! The Game
- Kuma\War 2
- Natural Selection 2 (No longer using Source engine)
- Postal 3 (PC, PS3 and Xbox 360)
- SiN Episodes
- The Ship
- They Hunger: Lost Souls
- Twilight War: After the Fall (production resumed)
- Vampire: The Masquerade - Bloodlines
- Mabinogi Heroes
- Untitled Threewave Software game
- Zeno Clash
- Source engine Official website.
- DirectX Versions. Valve Developer Community. Retrieved on 9 July 2006.
- New technology for EP2. HL2World.com.
- Half-Life 2: Episode Two Impressions. GameSpot (24 August 2006). Retrieved on 24 August 2006.
- Valve Half-Life 2: Episode One Interview. FiringSquad.com.
- Team Fortress 2 Hands-On Preview. GameInformer.com (28 March 2007). Retrieved on 2007-04-01.
- Half-Life 2: Episode Two Update. IGN.com (July 13, 2006). Retrieved on 22 July 2006.
- Interview: Gabe Newell. PC Zone (2006-09-11). Retrieved on 2006-09-20.
- Multi-Core in the Source Engine Core. Bit-tech (2006-11-2). Retrieved on 2006-11-2.
- Valve Loves Quad Core. ExtremeTech (2006-11-2). Retrieved on 2006-11-2.
- Day Of Defeat Source Summer Update Coming June 28. Steam News.
- Cinematic Effects in Source. bit-tech.net (December 9, 2005). Retrieved on 19 July 2006.
- Eike Umlauf (24 August 2004) (PDF). Image-Based Rendering of Forests. VRVis. www.vrvis.at/publications/pdfs/VRVis_2005_11_14_16_06_19.pdf. Retrieved on 21 August 2006.
- Interview with Gabe Newell. DriverHeaven.net.
- Valve Week. 1UP.com. Retrieved on 14 July 2006.
- Day of Defeat: Source Update Released. Steam news. Retrieved on 14 July 2006.
- Info received from valve. ValveTime.net forums.
- Welcome, Q3 source, Graphics. John Carmack's Blog. Retrieved on 2004-12-31.
- Half-Life 2: Raising the Bar
- The Half-Life 2 Stuttering Problem. Retrieved on 2004-11-20.
- Friends Beta. Valve Developer Community. Retrieved on 2006-02-07.
- Problem: Games freeze or crash with looping sounds. Steam Support. Retrieved on 2006-05-26.
- Ep 1 - Looping Sound Crash Discussion. Steampowered.com forums.
- Left 4 Dead Announced. Voodoo Extreme. Retrieved on 2006-11-20.
- The Crossing Revealed. ValveTime.net. Retrieved on 2006-12-23.
- Postal 3 Officially Announced. Firing Squad. Retrieved on 2006-10-18.
- Mabinogi Heroes on MMO Hut
- First-Person Advertiser - Talking In-Game Ads With Threewave Software. Gamasutra. Retrieved on 2006-09-16.
- Valve's Source licensing site
- GameSpy: "Original Half-Life to be Released in Source Engine"
- Half-Life 2 Fallout.net's comprehensive "Source: Belly of the Beast" article
- DevMaster.net Source Engine details
- Bit-tech.net's in depth look at Cinematic Effects in Source