Prebinding.html   [plain text]

<!--This file created 3/15/01 3:04 PM by Claris Home Page version 3.0-->
   <TITLE>Prebinding Notes</TITLE>
   <META NAME=GENERATOR CONTENT="Claris Home Page 3.0">
<P><FONT SIZE="-1">MacOS X 10.1 Prebinding Notes Copyright &copy; 2001 by
Apple Computer, Inc. All Rights Reserved.</FONT></P>

<H1>Mac OS X 10.1 Prebinding Notes</H1>

<P>This document describes an optimization called <b>prebinding</b> which
enables fast launching of applications on Mac OS X.</P>

<P>Normally, when you build an application or dynamic library, the static
linker (<tt>ld</tt>) records the names of the symbols the executable links
against, and marks references to these symbols as "undefined."</P>

<P>When an application is launched, the dynamic linker (<tt>dyld</tt>)
must bind the needed undefined references from the executable and dynamic
libraries to their respective definitions. The binding process can take time,
since the linker must map the library to an unoccupied address range and
calculate the address of each referenced symbol in the library.</P>

<P>Building a dynamic library with prebinding enabled eliminates the normal
binding overhead by predefining the library at a specified address range. When
an executable or other dynamic library is built against a prebound library,
the linker can directly reference symbols in the prebound library by address,
instead of leaving the addresses undefined.</P>

<P>The static linker also records the time stamp of libraries dependent
libraries.  When the program is executed, the dynamic linker checks to see
that all the build time stamps match and that the prebound address ranges of
all code does not overlap. If both of these conditions are met, the binding
of undefined references is already done, which saves a considerable amount
of time. If the time stamps don't match, or prebound executable addresses
overlap, the prebinding is undone and the program is bound normally.</P>

<P>Note that prebinding is only applicable to Mach-O executables. CFM PEF
binaries do not support prebinding.</P>

<H3><A NAME="#prebindreqs">Requirements for Prebinding</A></H3>

<P>To work properly, libraries and executables must meet several important requirements:</P>
<LI>Libraries must not have overlapping preferred addresses.  For
each release of Mac OS X the Apple-supplied libraries are all
prebound and do not have overlapping addresses. Your libraries must not
overlap with any of your own libraries, and also must not overlap with
any of the Apple-supplied libraries that are installed with Mac OS X.
There is currently no way to automatically select an address for a
library to ensure that it does not overlap the addresses of other libraries.
Non-Apple libraries can use any address in the range <tt>0x00000000
</tt> to <tt>0x3FFFFFFF</tt>. For the Mac OS X 10.0 release, all addresses
above <tt> 0x90000000 </tt> are also available. To set the address of a library, pass
either the <tt>-seg1addr</tt> flag or the <tt>-seg_addr_table</tt> flag
to <tt>ld(1)</tt>. All executables should be at the default address,
<tt>0x00000000</tt>. The default address is <tt>0x00000000</tt>. When selecting
addresses for libraries, the address range of the largest executable
using the libraries should be avoided.</LI>

<LI>There can be no undefined symbols. In most cases, this means that
you must link against your dependent libraries.  It also means that
there can be no circular dependencies (cases where library A calls a
function in library B, and a function in library B also calls a function
in library A).  Circular dependencies can be removed
by changing the code to dynamically look up the function instead of
directly referencing it.</LI>

<LI>You cannot override symbols that are referenced in flat namespace
images used by the dependent libraries.  For example, you can't define
your own <tt>malloc</tt> and then prebind using flat namespace

<H3>Producing a Prebound Build</H3>

<P>All libraries must be built in dependent order and built prebound.
That is, libraries must be built before the libraries (and executables)
that link against them are built.  To build prebound, either pass the
<tt>-prebind</tt> flag to <tt>ld(1)</tt>, or define the <tt>LD_PREBIND</tt>
environment variable.</P>

<P>You do not need to relink every time the prebinding of a library might
change. You can use the <tt>redo_prebinding(1)</tt> tool and the
<tt>update_prebinding(1)</tt> tool to update prebindings. However, in
order to use these tools, you need to link against libraries that were
initially prebound.  If a library was not prebound when it was originally
linked against, <tt>redo_prebinding(1)</tt> cannot prebind it.  It must be
initially built as a prebound library, as specified above.

<H3>Building Your Project Prebound</H3>

<P>The easy way to do this is to set the environment variable
<tt>LD_PREBIND</tt> before building your projects.  For example,
if you are using <tt>tcsh(1)</tt>:</P>

<P><tt>% setenv LD_PREBIND</tt></P>

<P>This has the same effect as passing <tt>-prebind</tt> to the
<tt>ld(1)</tt>.  For Project Builder framework and library projects
you can add -prebind to <tt>OTHER_LDFLAGS</tt>.  For application
projects, Project Builder adds <tt>-prebind</tt> to the linker flags by default.</P>

<H3>If your project's prebinding is disabled when built, here's how to
fix it:</H3>

<P>If prebinding fails you will see a warning message in the build
log.  Prebinding has a number of requirements to allow it to work
<A HREF="#prebindreqs">(see above)</A>.  If any of these requirements
aren't met prebinding will disabled and a warning message will be printed
by the static linker when building.</P>

<H4>The most common message you might see is:</H4>

<P><tt>/usr/bin/ld: warning prebinding disabled because of symbols
overridden in dependent dynamic shared libraries:<BR>
definition of _swap_mach_header in section (__TEXT,__text)<BR>
/System/Library/Frameworks/System.framework/System(swap.o) definition
of _swap_mach_header</tt></P>

<P>In this case, this happened for the objc4 project because it had
a copy of System framework's swap.c in the project.  It did this long
ago as a workaround when the System framework's swap.c was out of date.
The fix for this was to remove the objc4 project's copy of swap.c,
thus eliminating the symbol override.</P>

<P>If you need to track down what object is referencing a specified
symbol then <tt>ld(1)</tt>'s <tt>-ysymbol</tt> flag can be used.  If you need to
determine why a specific module is being linked in from a library
<tt>ld(1)</tt>'s <tt>-whyload</tt> flag can be used.</P>

<H4>Another example:</H4>

<P><tt>ld: warning prebinding disabled because of symbols overridden in
dependent dynamic shared libraries:<BR>
definition of _regcomp in section (__TEXT,__text)<BR>
definition of _regcomp</tt></P>

<P>In this case PBDevKit was using a different version of regcomp,
regexec and regfree than the System framework's version.  In this
case, PBDevKit needs this different version but does not need the code
in System to use it's version. The best fix for this problem is to hide
the names of these three functions in PBDevKit and their uses.  This
can be done in a number of ways:</P>
<LI>PBDevKit could declare these symbols as <tt>__private_extern__</tt></LI>
<LI> PBDevKit could remove these symbols from the list of exported symbols using the nmedit(1) tool</LI>
<LI>If the System framework was built as a two-level namespace library and
PBDevKit was built as a two-level namespace library, this would not be a problem, because these symbols would be declared in different namespaces and therefore not overlap.</LI>

<H4>Another message you might see is:</H4>

<P><tt>/usr/bin/ld: warning prebinding disabled because dependent
/System/Library/Frameworks/System.framework/Versions/B/System is not

<P>This happened because the project was built against a version of
System.framework that was not prebound. This might be a build order problem.
If you have programs or frameworks that depend on other frameworks you
must build them in the right order, and you must ensure that they are prebound.</P>

<H4>Another example is:</H4>

<P><tt>/usr/bin/ld: warning prebinding disabled because dependent
is not prebound</tt></P>

<P>In this case the prebinding of PBDevKit was disabled so that
caused the prebinding of the program linking against it to be disabled
(see the above example).</P>

<H3>Checking If A Binary Is Prebound</H3>

<P>To check if a binary was built prebound you can use <tt>otool(1)</tt> to
look for the <tt>PREBOUND</tt> flag in the mach header.  For example:</P>

<P><tt>% otool -hv /bin/cat<BR>
Mach header<BR>
      magic cputype cpusubtype   filetype ncmds sizeofcmds     
   MH_MAGIC     PPC        ALL    EXECUTE    10       1456   NOUNDEFS

<P>A prebound binary has the build time stamps of its dependent
libraries recorded in them.  To see the build time stamps again:</P>

<P><tt>% otool -Lv /bin/cat<BR>
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
current version 50.0.0)<BR>
        time stamp 982996740 Fri Feb 23 22:39:00 2001</tt></P>

<P>And viewing the dependent libraries build time stamp can also be done
with otool(1):</P>

<P><tt>% otool -Lv /usr/lib/libSystem.B.dylib<BR>
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
current version 50.0.0)<BR>
        time stamp 982996740 Fri Feb 23 22:39:00 2001</tt></P>

<H3>Checking If Prebinding Is Successful</H3>

<P>Set the environment variable <tt>DYLD_PREBIND_DEBUG</tt> and run the
program.  For example using <tt>tcsh(1)</tt> and checking the program

% sync<BR>
dyld: sync: prebinding enabled</tt></P>

<P>If a program is not built prebound but all the libraries are
built prebound then dyld may try to use the prebound libraries.  For

<P><tt>% setenv DYLD_PREBIND_DEBUG<BR>
% cc -v<BR>
dyld: cc: prebinding enabled using only prebound libraries</tt></P>

<P>This happened because when <tt>cc(1)</tt> was built the System framework it
was built against was not prebound.  <tt>dyld(1)</tt> can fail trying to use
prebound libraries if the program overrides symbols defined in it's
dependent libraries that are used by a dependent library.  For

<P><tt>% /usr/bin/objcunique<BR>
dyld: /usr/bin/objcunique: trying to use prebound libraries failed
due to overridden symbols</tt></P>

<P>Currently <tt>dyld(1)</tt> only tries to use the prebound libraries
if the program uses only one prebound library.  This is because with flat
namespace libraries the checking needed to make sure all the
libraries don't override any of each other symbols is more time
consuming than the optimization would generally save.</P>

<H3>Updating the Prebinding</H3>

<P>Another way prebinding may fail is that the build time stamps of
the libraries a program was built against with do not match the
libraries the program is run against. In the following example, sync1
was built using using a different version of the System.framework:</P>

    % /tmp/sync1<BR>
    % dyld: /tmp/sync1: prebinding disabled because time stamp
of library:<BR>
    /System/Library/Frameworks/System.framework/Versions/B/System did
not match<BR>

<P>The program will run correctly but the prebinding optimization
will be undone and the program dynamically bound as usual.  Releases
from Apple should never be in this state.  However, if you change a
framework on your system you can update the dependent programs (or
frameworks) without rebuilding them by running <tt>redo_prebinding(1)</tt> on
them.  For example, using the case above:</P>

<P><tt>    % redo_prebinding /tmp/sync1<BR>
    % /tmp/sync1<BR>
    dyld: /tmp/sync1: prebinding enabled<BR>
    % </tt></P>

<P>The program <tt>update_prebinding(1)</tt> can also be used to update the
prebinding and is used as part of the Installer's processing during installation.</P>

<H3>Invalidating the Prebinding</H3>

<P>To force the prebinding to be out of date one can run <tt>strip(1)</tt> or
<tt>nmedit(1)</tt> on one of the dependent libraries to cause the built time
stamp to be changed. A common way to do this and not effect the
symbols contained in the library is to run <tt>strip(1)</tt> with the <tt>-X</tt>
option as all the local symbols starting with <tt>'L'</tt> are already
stripped by the assembler by default.</P>

<H3>Prebinding and Library Initialization</H3>

<P>Since prebinding requires that the entire set of libraries be
initialized when loaded, libraries may be initialized in a different
order than they would without prebinding.  This can uncover latent
problems in the order of library initializations that went undetected
without prebinding, particularly if there were library initialization
routines which did not explicitly call the initialization routines
for libraries on which they depend. As a result, unexpected problems
can occur when prebinding is enabled.</P>