Linker

Do you know what you are linking?
I'm surprised on how many people are confused about the linking process in general. This discussion will focus on MSVC in particular but this discussion stands for the gnu toolset too. First let's define:

• A compile unit as a single C++ (.cc) source file generating a single object (.obj) file.
• A PE is a single output unit from the linker.
• A symbol is a single named component globally visible, like a global function, class definition, member function, global variable, imported symbol with dllimport or template instance; this list is obviously not exhaustive.

Compilation units
When linking, all inputs are linked together to extracts symbols and put them in the output file. This is important for the linker to have compilation units be easy to differentiate one to each other. The reason is that the compilation unit's name (the source file name) is used in the symbol mangling, especially for file static global variable. This is also important since often even if the source files are in various subdirectories, the output objects are in a single intermediary directory. This can cause problems when you have 2 sources files with the same name. For example, libjingle, up to revision 7, has 2 compile units with the same name: constants.cc and constants.cc. If both arrive in the same output dir, you're in trouble. But how come it works in this specific case? Well, when the developer adds the second file in the Visual Studio IDE, the IDE will automatically set the ObjectFile attribute to to add a '1' postfix. See libjingle.vcproj.

PE and incremental linking
Every output PE must have a unique name in a solution. This is something we are hitting in the Chromium project since we have chrome.exe and chrome.dll. The issue we face is that both generate chrome.ilk for incremental linking. This cause that incremntal linking fails every time. The workaround is to output one of the binaries in the intermediary output directory and hardlink it back at the right place. Lame but works.

Every symbol is not on equal footing
When defining the same symbol multiple times into a PE, the linker will give errors on multiple defined symbols. To alleviate this problem, you can define the symbol as selectany, e.g. a weak symbol. But if the two symbol definitions are not exactly the same, you can get into real trouble since one is selected at random at link time.

The interesting twist comes with static libraries. If a symbol is defined in one or multiple static libraries and in an object file, the symbol defined in the object file will be used. The best commonly used example is DllMain. When creating a DLL, if it is defined in a .obj file, your definition will be used. If not defined in an object file, the definition provided in the CRT static library will be used.

So in short, the linker has different priorities when linking its inputs: static libraries are second class citizens to object files.

So if a symbol defined in a static library is not referenced by a symbol in a object file, it will not be linked in. That's why many people will use the /INCLUDE parameter to force a symbol from a static library to be included. ATL is one doing that with "#pragma comment(linker, "/include:_forceAtlDllManifest")" in atlbase.h.

Scalability issue
But linking large static libraries takes time. For the Chromium project, some of our libraries are significantly large, browser.lib and webcore.lib being respectively 320 and 408 megs as revision 27335. One way to help speed up is to use Use Library Dependency Inputs. This permits skipping over the static library linking step. Why? Since static libraries cannot use incremental linking, they are much slower to link in debug than actual PE.

But all the rules described above change with Use Library Dependency Inputs since the final linker step links only object files and no static libraries. You see it coming, every symbols are now on equal footing. Which mean a lot of fun and helps find a lot of problems.

I saw unit tests including another compile unit. That results in duplicate symbols. I saw another project having 2 different main() functions. Same failure. I saw an installer including unit tests. Same failure. Why does that work without the ULDI flag? Because of different symbol priority between object file and static library. When a symbol exists in an object file, the symbols defined in static libraries are silently discarded. If all the symbols defined in static libraries are now on higher priority, that breaks on every symbols that are ambiguous.

That concludes my short explanation.