More doc updates

9 files changed, 396 insertions, 267 deletions

diff --git a/doc/filters.txt b/doc/filters.txt
index 3e337c3c3..7df1da5af 100644
--- a/doc/filters.txt
+++ b/doc/filters.txt
@@ -109,9 +109,12 @@ filter in a pipe. The output of the compressor is sent to the
 ``DataSink_Stream``. Anything written to a ``DataSink_Stream`` is
 written to a file; the filter produces no output. As soon as the
 compression algorithm finishes up a block of data, it will send it
-along, at which point it will immediately be written to disk; if you
-were to call ``pipe.read_all()`` after ``pipe.end_msg()``, you'd get
-an empty vector out.
+along to the sink filter, which will immediately write it to the
+stream; if you were to call ``pipe.read_all()`` after
+``pipe.end_msg()``, you'd get an empty vector out. This is
+particularly useful for cases where you are processing a large amount
+of data, as it means you don't have to store everything in memory at
+once.
 
 Here's an example using two computational filters::
 
@@ -127,6 +130,30 @@ Here's an example using two computational filters::
    encryptor.end_msg(); // flush buffers, complete computations
    std::cout << encryptor;
 
+You can read from a pipe while you are still writing to it, which
+allows you to bound the amount of memory that is in use at any one
+time. A common idiom for this is::
+
+   pipe.start_msg();
+   SecureBuffer<byte, 4096> buffer;
+   while(infile.good())
+      {
+      infile.read((char*)&buffer[0], buffer.size());
+      const size_t got_from_infile = infile.gcount();
+      pipe.write(buffer, got_from_infile);
+
+      if(infile.eof())
+         pipe.end_msg();
+
+      while(pipe.remaining() > 0)
+         {
+         const size_t buffered = pipe.read(buffer, buffer.size());
+         outfile.write((const char*)&buffer[0], buffered);
+         }
+      }
+   if(infile.bad() || (infile.fail() && !infile.eof()))
+      throw Some_Exception();
+
 Fork
 ---------------------------------
 
@@ -243,34 +270,34 @@ Sources and Sinks
 Data Sources
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-A ``DataSource`` is a simple abstraction for a thing that stores bytes. This
-type is used heavily in the areas of the API related to ASN.1
-encoding/decoding. The following types are ``DataSource``s: ``Pipe``,
-``SecureQueue``, and a couple of special purpose ones:
+A ``DataSource`` is a simple abstraction for a thing that stores
+bytes. This type is used heavily in the areas of the API related to
+ASN.1 encoding/decoding. The following types are ``DataSource``s:
+``Pipe``, ``SecureQueue``, and a couple of special purpose ones:
 ``DataSource_Memory`` and ``DataSource_Stream``.
 
-You can create a ``DataSource_Memory`` with an array of bytes and a length
-field. The object will make a copy of the data, so you don't have to worry
-about keeping that memory allocated. This is mostly for internal use, but if it
-comes in handy, feel free to use it.
+You can create a ``DataSource_Memory`` with an array of bytes and a
+length field. The object will make a copy of the data, so you don't
+have to worry about keeping that memory allocated. This is mostly for
+internal use, but if it comes in handy, feel free to use it.
 
 A ``DataSource_Stream`` is probably more useful than the memory based
 one. Its constructors take either a ``std::istream`` or a
 ``std::string``. If it's a stream, the data source will use the
-``istream`` to satisfy read requests (this is particularly useful to use
-with ``std::cin``). If the string version is used, it will attempt to open
-up a file with that name and read from it.
+``istream`` to satisfy read requests (this is particularly useful to
+use with ``std::cin``). If the string version is used, it will attempt
+to open up a file with that name and read from it.
 
 Data Sinks
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-A ``DataSink`` (in ``data_snk.h``) is a ``Filter`` that
-takes arbitrary amounts of input, and produces no output. This means
-it's doing something with the data outside the realm of what
-``Filter``/``Pipe`` can handle, for example, writing it to a
-file (which is what the ``DataSink_Stream`` does). There is no
-need for ``DataSink``s that write to a ``std::string`` or memory
-buffer, because ``Pipe`` can handle that by itself.
+A ``DataSink`` (in ``data_snk.h``) is a ``Filter`` that takes
+arbitrary amounts of input, and produces no output. This means it's
+doing something with the data outside the realm of what
+``Filter``/``Pipe`` can handle, for example, writing it to a file
+(which is what the ``DataSink_Stream`` does). There is no need for
+``DataSink``s that write to a ``std::string`` or memory buffer,
+because ``Pipe`` can handle that by itself.
 
 Here's a quick example of using a ``DataSink``, which encrypts
 ``in.txt`` and sends the output to ``out.txt``. There is
@@ -416,7 +443,7 @@ Functions in ``Pipe`` related to reading include:
 
   Returns how many bytes are left in the message
 
-.. cpp:function:: Pipe::message_id default_msg()
+.. cpp:function:: Pipe::message_id Pipe::default_msg()
 
   Returns the current default message number
 
diff --git a/doc/index.txt b/doc/index.txt
index ca3795e88..2f55ea362 100644
--- a/doc/index.txt
+++ b/doc/index.txt
@@ -14,12 +14,14 @@ Contents:
    pubkey
    x509
    ssl
+   bigint
    lowlevel
+   secmem
+   kdf
+   pbkdf
    passhash
    rng
-   kdf
    fpe
-   bigint
    examples
    algos
    faq
diff --git a/doc/lowlevel.txt b/doc/lowlevel.txt
index 5b3ec42b2..bea6edc4e 100644
--- a/doc/lowlevel.txt
+++ b/doc/lowlevel.txt
@@ -2,13 +2,13 @@
 The Low-Level Interface
 =================================
 
-Botan has two different interfaces. The one documented in this section is meant
-more for implementing higher-level types (see the section on filters, earlier in
-this manual) than for use by applications. Using it safely requires a solid
-knowledge of encryption techniques and best practices, so unless you know, for
-example, what CBC mode and nonces are, and why PKCS #1 padding is important,
-you should avoid this interface in favor of something working at a higher level
-(such as the CMS interface).
+Botan has two different interfaces. The one documented in this section
+is meant more for implementing higher-level types (see the section on
+filters, earlier in this manual) than for use by applications. Using
+it safely requires a solid knowledge of encryption techniques and best
+practices, so unless you know, for example, what CBC mode and nonces
+are, and why PKCS #1 padding is important, you should avoid this
+interface in favor of something working at a higher level.
 
 Basic Algorithm Abilities
 ---------------------------------
@@ -201,18 +201,17 @@ The second method of using final is to call it with no arguments at
 all, as shown in the second prototype. It will return the hash/mac
 value in a memory buffer.
 
-There is also a pair of functions called ``process``. They are
-a combination of a single ``update``, and ``final``.
-Both versions return the final value, rather than placing it an
-array. Calling ``process`` with a single byte value isn't
-available, mostly because it would rarely be useful.
+There is also a pair of functions called ``process``. They are a
+combination of a single ``update``, and ``final``. Both versions
+return the final value, rather than placing it an array. Calling
+``process`` with a single byte value isn't available, mostly because
+it would rarely be useful.
 
 A MAC can be viewed (in most cases) as a keyed hash function, so
 classes that are derived from ``MessageAuthenticationCode`` have
-``update`` and ``final`` classes just like a
-``HashFunction`` (and like a ``HashFunction``, after
-``final`` is called, it can be used to make a new MAC right
-away; the key is kept around).
+``update`` and ``final`` classes just like a ``HashFunction`` (and
+like a ``HashFunction``, after ``final`` is called, it can be used to
+make a new MAC right away; the key is kept around).
 
 A MAC has the ``SymmetricAlgorithm`` interface in addition to the
 ``BufferedComputation`` interface.
@@ -227,84 +226,3 @@ ones being that the output size is very small (usually in the range of
 intended purpose (error checking), they perform very well. Some
 examples of checksums included in Botan are the Adler32 and CRC32
 checksums.
-
-Secure Memory
----------------------------------
-
-A major concern with mixing modern multiuser OSes and cryptographic
-code is that at any time the code (including secret keys) could be
-swapped to disk, where it can later be read by an attacker. Botan
-stores almost everything (and especially anything sensitive) in memory
-buffers that a) clear out their contents when their destructors are
-called, and b) have easy plugins for various memory locking functions,
-such as the ``mlock`` call on many Unix systems.
-
-Two of the allocation method used ("malloc" and "mmap") don't
-require any extra privileges on Unix, but locking memory does. At
-startup, each allocator type will attempt to allocate a few blocks
-(typically totaling 128k), so if you want, you can run your
-application ``setuid`` ``root``, and then drop privileges
-immediately after creating your ``LibraryInitializer``. If you end
-up using more than what's been allocated, some of your sensitive data
-might end up being swappable, but that beats running as ``root``
-all the time.
-
-These classes should also be used within your own code for storing
-sensitive data. They are only meant for primitive data types (int,
-long, etc): if you want a container of higher level Botan objects, you
-can just use a ``std::vector``, since these objects know how to clear
-themselves when they are destroyed. You cannot, however, have a
-``std::vector`` (or any other container) of ``Pipe`` objects or
-filters, because these types have pointers to other filters, and
-implementing copy constructors for these types would be both hard and
-quite expensive (vectors of pointers to such objects is fine, though).
-
-These types are not described in any great detail: for more information,
-consult the definitive sources~--~the header files ``secmem.h`` and
-``allocate.h``.
-
-``SecureBuffer`` is a simple array type, whose size is specified at
-compile time. It will automatically convert to a pointer of the
-appropriate type, and has a number of useful functions, including
-``clear()``, and ``size_t`` ``size()``, which returns the length of
-the array. It is a template that takes as parameters a type, and a
-constant integer which is how long the array is (for example:
-``SecureBuffer<byte, 8> key;``).
-
-``SecureVector`` is a variable length array. Its size can be increased
-or decreased as need be, and it has a wide variety of functions useful
-for copying data into its buffer. Like ``SecureBuffer``, it implements
-``clear`` and ``size``.
-
-Allocators
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The containers described above get their memory from allocators. As a
-user of the library, you can add new allocator methods at run time for
-containers, including the ones used internally by the library, to
-use. The interface to this is in ``allocate.h``. Code needing to
-allocate or deallocate memory calls ``get_allocator``, which returns a
-pointer to an allocator object. This pointer should not be freed: the
-caller does not own the allocator (it is shared among multiple
-allocatore users, and uses a mutex to serialize access internally if
-necessary). It is possible to call ``get_allocator`` with a specific
-name to request a particular type of allocator, otherwise, a default
-allocator type is returned.
-
-At start time, the only allocator known is a ``Default_Allocator``,
-which just allocates memory using ``malloc``, and zeroizes it when the
-memory is released. It is known by the name "malloc". If you ask for
-another type of allocator ("locking" and "mmap" are currently used),
-and it is not available, some other allocator will be returned.
-
-You can add in a new allocator type using ``add_allocator_type``. This
-function takes a string and a pointer to an allocator. The string gives this
-allocator type a name to which it can be referred when one is requesting it
-with ``get_allocator``. If an error occurs (such as the name being
-already registered), this function returns false. It will return true if the
-allocator was successfully registered. If you ask it to,
-``LibraryInitializer`` will do this for you.
-
-Finally, you can set the default allocator type that will be returned
-using the policy setting "default_alloc" to the name of any previously
-registered allocator.
diff --git a/doc/passhash.txt b/doc/passhash.txt
index e9a179c1a..8ce3cf805 100644
--- a/doc/passhash.txt
+++ b/doc/passhash.txt
@@ -1,51 +1,4 @@
 
-.. _pbkdf:
-
-PBKDF Algorithms
-========================================
-
-There are various procedures (usually ad-hoc) for turning a
-passphrase into a (mostly) arbitrary length key for a symmetric
-cipher. A general interface for such algorithms is presented in
-``pbkdf.h``. The main function is ``derive_key``, which
-takes a passphrase, a salt, an iteration count, and the desired length
-of the output key, and returns a key of that length, deterministically
-produced from the passphrase and salt. If an algorithm can't produce a
-key of that size, it will throw an exception (most notably, PKCS #5's
-PBKDF1 can only produce strings between 1 and $n$ bytes, where $n$ is
-the output size of the underlying hash function).
-
-The purpose of the iteration count is to make the algorithm take
-longer to compute the final key (reducing the speed of brute-force
-attacks of various kinds). Most standards recommend an iteration count
-of at least 10000. Currently defined PBKDF algorithms are
-"PBKDF1(digest)", "PBKDF2(digest)", and "OpenPGP-S2K(digest)"; you can
-retrieve any of these using the ``get_pbkdf``, found in
-``lookup.h``. As of this writing, "PBKDF2(SHA-256)" with 10000
-iterations and a 16 byte salt is recommend for new applications.
-
-OpenPGP S2K
-----------------------------------------
-
-There are some oddities about OpenPGP's S2K algorithms that are
-documented here. For one thing, it uses the iteration count in a
-strange manner; instead of specifying how many times to iterate the
-hash, it tells how many *bytes* should be hashed in total
-(including the salt). So the exact iteration count will depend on the
-size of the salt (which is fixed at 8 bytes by the OpenPGP standard,
-though the implementation will allow any salt size) and the size of
-the passphrase.
-
-To get what OpenPGP calls "Simple S2K", set iterations to 0, and do
-not specify a salt. To get "Salted S2K", again leave the iteration
-count at 0, but give an 8-byte salt. "Salted and Iterated S2K"
-requires an 8-byte salt and some iteration count (this should be
-significantly larger than the size of the longest passphrase that
-might reasonably be used; somewhere from 1024 to 65536 would probably
-be about right). Using both a reasonably sized salt and a large
-iteration count is highly recommended to prevent password guessing
-attempts.
-
 Password Hashing
 ========================================
 
diff --git a/doc/pbkdf.txt b/doc/pbkdf.txt
new file mode 100644
index 000000000..f24dcd188
--- /dev/null
+++ b/doc/pbkdf.txt
@@ -0,0 +1,76 @@
+
+.. _pbkdf:
+
+PBKDF Algorithms
+========================================
+
+There are various procedures for turning a passphrase into a arbitrary
+length key for use with a symmetric cipher. A general interface for
+such algorithms is presented in ``pbkdf.h``. The main function is
+``derive_key``, which takes a passphrase, a salt, an iteration count,
+and the desired length of the output key, and returns a key of that
+length, deterministically produced from the passphrase and salt. If an
+algorithm can't produce a key of that size, it will throw an exception
+(most notably, PKCS #5's PBKDF1 can only produce strings between 1 and
+$n$ bytes, where $n$ is the output size of the underlying hash
+function).
+
+The purpose of the iteration count is to make the algorithm take
+longer to compute the final key (reducing the speed of brute-force
+attacks of various kinds). Most standards recommend an iteration count
+of at least 10000. Currently defined PBKDF algorithms are
+"PBKDF1(digest)", "PBKDF2(digest)", and "OpenPGP-S2K(digest)"; you can
+retrieve any of these using the ``get_pbkdf``, found in
+``lookup.h``. As of this writing, "PBKDF2(SHA-256)" with 10000
+iterations and a 16 byte salt is recommend for new applications.
+
+.. cpp:function:: OctetString PBKDF::derive_key( \
+   size_t output_len, const std::string& passphrase, \
+   const byte* salt, size_t salt_len, \
+   size_t iterations) const
+
+   Computes a key from *passphrase* and the *salt* (of length
+   *salt_len* bytes) using an algorithm-specific interpretation of
+   *iterations*, producing a key of length *output_len*.
+
+   Use an iteration count of at least 10000. The salt should be
+   randomly chosen by a good random number generator (see
+   :ref:`random_number_generators` for how), or at the very least
+   unique to this usage of the passphrase.
+
+   If you call this function again with the same parameters, you will
+   get the same key.
+
+::
+
+   PBKDF* pbkdf = get_pbkdf("PBKDF2(SHA-256)");
+   AutoSeeded_RNG rng;
+
+   SecureVector<byte> salt = rng.random_vec(16);
+   OctetString aes256_key = pbkdf->derive_key(32, "password",
+                                              &salt[0], salt.size(),
+                                              10000);
+
+
+OpenPGP S2K
+----------------------------------------
+
+There are some oddities about OpenPGP's S2K algorithms that are
+documented here. For one thing, it uses the iteration count in a
+strange manner; instead of specifying how many times to iterate the
+hash, it tells how many *bytes* should be hashed in total
+(including the salt). So the exact iteration count will depend on the
+size of the salt (which is fixed at 8 bytes by the OpenPGP standard,
+though the implementation will allow any salt size) and the size of
+the passphrase.
+
+To get what OpenPGP calls "Simple S2K", set iterations to 0, and do
+not specify a salt. To get "Salted S2K", again leave the iteration
+count at 0, but give an 8-byte salt. "Salted and Iterated S2K"
+requires an 8-byte salt and some iteration count (this should be
+significantly larger than the size of the longest passphrase that
+might reasonably be used; somewhere from 1024 to 65536 would probably
+be about right). Using both a reasonably sized salt and a large
+iteration count is highly recommended to prevent password guessing
+attempts.
+
diff --git a/doc/pubkey.txt b/doc/pubkey.txt
index fe24c719b..f12aae0b5 100644
--- a/doc/pubkey.txt
+++ b/doc/pubkey.txt
@@ -78,8 +78,8 @@ Nyberg-Rueppel key pairs with
   value. Normally, you would leave the value as zero, letting the
   class generate a new random key.
 
-Finally, given an ``EC_Group`` object, you can create a new
-ECDSA, ECDH, or GOST 34.10 private key with
+Finally, given an ``EC_Group`` object, you can create a new ECDSA,
+ECDH, or GOST 34.10-2001 private key with
 
 .. cpp:function:: ECDSA_PrivateKey::ECDSA_PrivateKey(RandomNumberGenerator& rng, \
    const EC_Group& domain, const BigInt& x = 0)
@@ -357,49 +357,55 @@ hides the underlying mathematical properties of the algorithm.
 Additionally, they will add randomness, so encrypting the same
 plaintext twice produces two different ciphertexts.
 
-The primary interface for encryption is ``PK_Encryptor``, which
-provides the following interface:
+The primary interface for encryption is 
 
-.. cpp:function:: SecureVector<byte> PK_Encryptor::encrypt(const byte* in, \
-   size_t length, RandomNumberGenerator& rng) const
+.. cpp:class:: PK_Encryptor
 
-.. cpp:function:: SecureVector<byte> PK_Encryptor::encrypt(const MemoryRegion<byte>& in, \
-   RandomNumberGenerator& rng) const
+   .. cpp:function:: SecureVector<byte> encrypt( \
+         const byte* in, size_t length, RandomNumberGenerator& rng) const
 
+   .. cpp:function:: SecureVector<byte> encrypt( \
+      const MemoryRegion<byte>& in, RandomNumberGenerator& rng) const
 
-.. cpp:function::  size_t PK_Encryptor::maximum_input_size() const
+      These encrypt a message, returning the ciphertext.
 
-   This function returns the maximum size of the message that can
-   be processed, in bytes. If you call ``encrypt`` with a value
-   larger than this the operation will fail with an exception.
+   .. cpp:function::  size_t maximum_input_size() const
 
-``PK_Encryptor`` is only an interface; to use one you have to create
-an implementation; there are currently two availabie in the library,
-``PK_Encryptor_EME`` and ``DLIES_Encryptor``. DLIES is a standard
-method (from IEEE 1363) that uses a key agreement technique such as DH
-or ECDH to perform message encryption. Normally, public key encryption
-is done using algorithms which support it directly, such as RSA or
-ElGamal; these use ``PK_Encryptor_EME``. The construction method is
-simple; call
+      Returns the maximum size of the message that can be processed,
+      in bytes. If you call :cpp:func:`PK_Encryptor::encrypt` with a
+      value larger than this the operation will fail with an
+      exception.
 
-.. cpp:function:: PK_Encryptor_EME::PK_Encryptor_EME(const Public_Key& key, std::string eme)
+:cpp:class:`PK_Encryptor` is only an interface - to actually encrypt
+you have to create an implementation, of which there are currently two
+available in the library, :cpp:class:`PK_Encryptor_EME` and
+:cpp:class:`DLIES_Encryptor`. DLIES is a standard method (from IEEE
+1363) that uses a key agreement technique such as DH or ECDH to
+perform message encryption. Normally, public key encryption is done
+using algorithms which support it directly, such as RSA or ElGamal;
+these use the EME class:
 
-  With *key* being the key you want to encrypt messages to. The
-  padding method to use is specified in *eme*.
+.. cpp:class:: PK_Encryptor_EME
 
-  The recommended values for *eme* is "EME1(SHA-1)" or
-  "EME1(SHA-256)". If you need compatability with protocols using the
-  PKCS #1 v1.5 standard, you can also use "EME-PKCS1-v1_5".
+   .. cpp:function:: PK_Encryptor_EME(const Public_Key& key, std::string eme)
 
-The DLIES encryptor is defined in the header ``dlies.h``, and
-is created by the constructor:
+     With *key* being the key you want to encrypt messages to. The
+     padding method to use is specified in *eme*.
 
-.. cpp:function:: DLIES_Encryptor::DLIES_Encryptor(const PK_Key_Agreement_Key&, \
-   KDF* kdf, MessageAuthenticationCode* mac, size_t mac_key_len = 20)
+     The recommended values for *eme* is "EME1(SHA-1)" or
+     "EME1(SHA-256)". If you need compatability with protocols using
+     the PKCS #1 v1.5 standard, you can also use "EME-PKCS1-v1_5".
 
-  Where *kdf* is a key derivation function (see
-  :ref:`key_derivation_function`) and *mac* is a
-  MessageAuthenticationCode.
+.. cpp:class:: DLIES_Encryptor
+
+   Available in the header ``dlies.h``
+
+   .. cpp:function:: DLIES_Encryptor(const PK_Key_Agreement_Key& key, \
+         KDF* kdf, MessageAuthenticationCode* mac, size_t mac_key_len = 20)
+
+      Where *kdf* is a key derivation function (see
+      :ref:`key_derivation_function`) and *mac* is a
+      MessageAuthenticationCode.
 
 The decryption classes are named ``PK_Decryptor``,
 ``PK_Decryptor_EME``, and ``DLIES_Decryptor``. They are created in the
@@ -410,53 +416,96 @@ function is named ``decrypt``.
 Signatures
 ---------------------------------
 
+Signature generation is performed using
+
+.. cpp:class:: PK_Signer
+
+   .. cpp:function:: PK_Signer(const Private_Key& key, \
+      const std::string& emsa, \
+      Signature_Format format = IEEE_1363)
+
+     Constructs a new signer object for the private key *key* using the
+     signature format *emsa*. The key must support signature operations.
+     In the current version of the library, this includes RSA, DSA,
+     ECDSA, GOST 34.10-2001, Nyberg-Rueppel, and Rabin-Williams. Other
+     signature schemes may be supported in the future.
+
+     Currently available values for *emsa* include EMSA1, EMSA2, EMSA3,
+     EMSA4, and Raw. All of them, except Raw, take a parameter naming a
+     message digest function to hash the message with. The Raw encoding
+     signs the input directly; if the message is too big, the signing
+     operation will fail. Raw is not useful except in very specialized
+     applications. Examples are "EMSA1(SHA-1)" and "EMSA4(SHA-256)".
+
+     For RSA, use EMSA4 (also called PSS) unless you need compatability
+     with software that uses the older PKCS #1 v1.5 standard, in which
+     case use EMSA3 (also called "EMSA-PKCS1-v1_5"). For DSA, ECDSA,
+     GOST 34.10-2001, and Nyberg-Rueppel, you should use EMSA1.
+
+     The *format* defaults to ``IEEE_1363`` which is the only available
+     format for RSA. For DSA and ECDSA, you can also use
+     ``DER_SEQUENCE``, which will format the signature as an ASN.1
+     SEQUENCE value.
+
+   .. cpp:function:: void update(const byte* in, size_t length)
+   .. cpp:function:: void update(const MemoryRegion<byte>& in)
+   .. cpp:function:: void update(byte in)
+
+      These add more data to be included in the signature
+      computation. Typically, the input will be provided directly to a
+      hash function.
+
+   .. cpp:function:: SecureVector<byte> signature(RandomNumberGenerator& rng)
+
+      Creates the signature and returns it
+
+   .. cpp:function:: SecureVector<byte> sign_message( \
+      const byte* in, size_t length, RandomNumberGenerator& rng)
+
+   .. cpp:function:: SecureVector<byte> sign_message( \
+      const MemoryRegion<byte>& in, RandomNumberGenerator& rng)
+
+      These functions are equivalent to calling
+      :cpp:func:`PK_Signer::update` and then
+      :cpp:func:`PK_Signer::signature`. Any data previously provided
+      using ``update`` will be included.
+
+Signatures are verified using
+
+.. cpp:class:: PK_Verifier
+
+   .. cpp:function:: PK_Verifier(const Public_Key& pub_key, \
+          const std::string& emsa, Signature_Format format = IEEE_1363)
+
+      Construct a new verifier for signatures assicated with public
+      key *pub_key*. The *emsa* and *format* should be the same as
+      that used by the signer.
+
+   .. cpp:function:: void update(const byte* in, size_t length)
+   .. cpp:function:: void update(const MemoryRegion<byte>& in)
+   .. cpp:function:: void update(byte in)
+
+      Add further message data that is purportedly assocated with the
+      signature that will be checked.
+
+   .. cpp:function:: bool check_signature(const byte* sig, size_t length)
+   .. cpp:function:: bool check_signature(const MemoryRegion<byte>& sig)
+
+      Check to see if *sig* is a valid signature for the message data
+      that was written in. Return true if so. This function clears the
+      internal message state, so after this call you can call
+      :cpp:func:`PK_Verifier::update` to start verifying another
+      message.
+
+   .. cpp:function:: bool verify_message(const byte* msg, size_t msg_length, \
+                                         const byte* sig, size_t sig_length)
+
+   .. cpp:function:: bool verify_message(const MemoryRegion<byte>& msg, \
+                                         const MemoryRegion<byte>& sig)
 
-The signature algorithms look quite a bit like the hash functions. You
-can repeatedly call ``update``, giving more and more of a message you
-wish to sign, and then call ``signature``, which will return a
-signature for that message. If you want to do it all in one shot, call
-``sign_message``, which will just call ``update`` with its argument
-and then return whatever ``signature`` returns. Generating a signature
-requires random numbers with some schemes, so ``signature`` and
-``sign_message`` both take a ``RandomNumberGenerator&``.
-
-You can validate a signature by updating the verifier class, and
-finally seeing the if the value returned from ``check_signature`` is
-true (you pass the supposed signature to the ``check_signature``
-function as a byte array and a length or as a
-``MemoryRegion<byte>``). There is another function,
-``verify_message``, which takes a pair of byte array/length pairs (or
-a pair of ``MemoryRegion<byte>`` objects), the first of which is the
-message, the second being the (supposed) signature. It returns true if
-the signature is valid and false otherwise.
-
-Available public key signature algorithms in Botan are RSA, DSA,
-ECDSA, GOST-34.11, Nyberg-Rueppel, and Rabin-Williams. Signature
-encoding methods include EMSA1, EMSA2, EMSA3, EMSA4, and Raw. All of
-them, except Raw, take a parameter naming a message digest function to
-hash the message with. The Raw encoding signs the input directly; if
-the message is too big, the signing operation will fail. Raw is not
-useful except in very specialized applications.
-
-There are various interactions that make certain encoding schemes and
-signing algorithms more or less useful.
-
-EMSA2 is the usual method for encoding Rabin-William signatures, so
-for compatibility with other implementations you may have to use
-that. EMSA4 (also called PSS), also works with Rabin-Williams. EMSA1
-and EMSA3 do *not* work with Rabin-Williams.
-
-RSA can be used with any of the available encoding methods. EMSA4 is
-by far the most secure, but is not (as of now) widely
-implemented. EMSA3 (also called "EMSA-PKCS1-v1_5") is commonly used
-with RSA (for example in SSL). EMSA1 signs the message digest
-directly, without any extra padding or encoding. This may be useful,
-but is not as secure as either EMSA3 or EMSA4. EMSA2 may be used but
-is not recommended.
-
-For DSA, ECDSA, GOST-34.11, and Nyberg-Rueppel, you should use
-EMSA1. None of the other encoding methods are particularly useful for
-these algorithms.
+      These are equivalent to calling :cpp:func:`PK_Verifier::update`
+      on *msg* and then calling :cpp:func:`PK_Verifier::check_signature`
+      on *sig*.
 
 Here is an example of DSA signature generation
 
diff --git a/doc/rng.txt b/doc/rng.txt
index 756dccfc9..66679271d 100644
--- a/doc/rng.txt
+++ b/doc/rng.txt
@@ -1,31 +1,48 @@
 .. _random_number_generators:
 
 Random Number Generators
----------------------------------
+========================================
 
 The random number generators provided in Botan are meant for creating
 keys, IVs, padding, nonces, and anything else that requires 'random'
 data. It is important to remember that the output of these classes
-will vary, even if they are supplied with ethe same seed (ie, two
-``Randpool`` objects with similar initial states will not produce
-the same output, because the value of high resolution timers is added
-to the state at various points).
+will vary, even if they are supplied with the same seed (ie, two
+``Randpool`` objects with similar initial states will not produce the
+same output, because the value of high resolution timers is added to
+the state at various points).
 
-To ensure good quality output, a PRNG needs to be seeded with truly
-random data (such as that produced by a hardware RNG). Typically, you
-will use an ``EntropySource`` (see below). To add some (typically
-application specific) entropy to a PRNG, you can use
+To create a random number generator, instantiate a ``AutoSeeded_RNG``
+object. This object will handle choosing the right algorithms from the
+set of enabled ones and doing seeding using OS specific
+routines. The main service a RandomNumberGenerator provides is, of
+course, random numbers:
+
+.. cpp:function:: byte RandomNumberGenerator::random()
+
+  Generates a single random byte and returns it
 
-.. cpp:function:: void add_entropy(const byte* data, size_t length)
+.. cpp:function:: void RandomNumberGenerator::randomize(byte* data, size_t length)
+
+  Places *length* bytes into the array pointed to by *data*
+
+To ensure good quality output, a PRNG needs to be seeded with truly
+random data. Normally this is done for you. However it may happen that
+your application has access to data that is potentially unpredictable
+to an attacker. If so, use
 
-Once a PRNG has been initialized, you can get a single byte of random
-data by
+.. cpp:function:: void RandomNumberGenerator::add_entropy(const byte* data, \
+                                                          size_t length)
 
-.. cpp:function:: byte random()
+which incorporates the data into the current randomness state. Don't
+worry about filtering the data or doing any kind of cryptographic
+preprocessing (such as hashing); the RNG objects in botan are designed
+such that you can feed them any arbitrary non-random or even
+maliciously chosen data - as long as at some point some of the seed
+data was good the output will be secure.
 
-or get a large block by calling
 
-.. cpp:function:: void randomize(byte* data, size_t length)
+Implementation Notes
+----------------------------------------
 
 Randpool
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/secmem.txt b/doc/secmem.txt
new file mode 100644
index 000000000..eba3462de
--- /dev/null
+++ b/doc/secmem.txt
@@ -0,0 +1,81 @@
+
+Secure Memory Containers
+========================================
+
+A major concern with mixing modern multiuser OSes and cryptographic
+code is that at any time the code (including secret keys) could be
+swapped to disk, where it can later be read by an attacker. Botan
+stores almost everything (and especially anything sensitive) in memory
+buffers that a) clear out their contents when their destructors are
+called, and b) have easy plugins for various memory locking functions,
+such as the ``mlock`` call on many Unix systems.
+
+Two of the allocation method used ("malloc" and "mmap") don't
+require any extra privileges on Unix, but locking memory does. At
+startup, each allocator type will attempt to allocate a few blocks
+(typically totaling 128k), so if you want, you can run your
+application ``setuid`` ``root``, and then drop privileges
+immediately after creating your ``LibraryInitializer``. If you end
+up using more than what's been allocated, some of your sensitive data
+might end up being swappable, but that beats running as ``root``
+all the time.
+
+These classes should also be used within your own code for storing
+sensitive data. They are only meant for primitive data types (int,
+long, etc): if you want a container of higher level Botan objects, you
+can just use a ``std::vector``, since these objects know how to clear
+themselves when they are destroyed. You cannot, however, have a
+``std::vector`` (or any other container) of ``Pipe`` objects or
+filters, because these types have pointers to other filters, and
+implementing copy constructors for these types would be both hard and
+quite expensive (vectors of pointers to such objects is fine, though).
+
+These types are not described in any great detail: for more information,
+consult the definitive sources~--~the header files ``secmem.h`` and
+``allocate.h``.
+
+``SecureBuffer`` is a simple array type, whose size is specified at
+compile time. It will automatically convert to a pointer of the
+appropriate type, and has a number of useful functions, including
+``clear()``, and ``size_t`` ``size()``, which returns the length of
+the array. It is a template that takes as parameters a type, and a
+constant integer which is how long the array is (for example:
+``SecureBuffer<byte, 8> key;``).
+
+``SecureVector`` is a variable length array. Its size can be increased
+or decreased as need be, and it has a wide variety of functions useful
+for copying data into its buffer. Like ``SecureBuffer``, it implements
+``clear`` and ``size``.
+
+Allocators
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The containers described above get their memory from allocators. As a
+user of the library, you can add new allocator methods at run time for
+containers, including the ones used internally by the library, to
+use. The interface to this is in ``allocate.h``. Code needing to
+allocate or deallocate memory calls ``get_allocator``, which returns a
+pointer to an allocator object. This pointer should not be freed: the
+caller does not own the allocator (it is shared among multiple
+allocatore users, and uses a mutex to serialize access internally if
+necessary). It is possible to call ``get_allocator`` with a specific
+name to request a particular type of allocator, otherwise, a default
+allocator type is returned.
+
+At start time, the only allocator known is a ``Default_Allocator``,
+which just allocates memory using ``malloc``, and zeroizes it when the
+memory is released. It is known by the name "malloc". If you ask for
+another type of allocator ("locking" and "mmap" are currently used),
+and it is not available, some other allocator will be returned.
+
+You can add in a new allocator type using ``add_allocator_type``. This
+function takes a string and a pointer to an allocator. The string gives this
+allocator type a name to which it can be referred when one is requesting it
+with ``get_allocator``. If an error occurs (such as the name being
+already registered), this function returns false. It will return true if the
+allocator was successfully registered. If you ask it to,
+``LibraryInitializer`` will do this for you.
+
+Finally, you can set the default allocator type that will be returned
+using the policy setting "default_alloc" to the name of any previously
+registered allocator.
diff --git a/doc/x509.txt b/doc/x509.txt
index 3375af8b4..05adeaaad 100644
--- a/doc/x509.txt
+++ b/doc/x509.txt
@@ -264,10 +264,10 @@ which key was used to sign the certificate). Implementing the function
 is optional, and by default will return no CRLs. If it is available,
 it will be used by ``X509_CRL``.
 
-As for using such a store, you have to tell ``X509_Store`` about
-it, by calling the ``X509_Store`` member function
+As for using such a store, you have to tell ``X509_Store`` about it
+with
 
-.. cpp:function:: void add_new_certstore(Certificate_Store* new_store)
+.. cpp:function:: void X509_Store::add_new_certstore(Certificate_Store* new_store)
 
   The store object will be owned by (and deleted by) ``X509_Store``,
   so make sure to allocate it with ``new``.
@@ -365,7 +365,8 @@ on how to create one) or one issued by another CA (see the section on
 PKCS #10 requests). Creating a CA object is done by the following
 constructor:
 
-.. cpp:function:: X509_CA(const X509_Certificate& cert, const Private_Key& key)
+.. cpp:function:: X509_CA::X509_CA(const X509_Certificate& cert, \
+   const Private_Key& key)
 
 The private key is the private key corresponding to the public key in the
 CA's certificate.
@@ -378,7 +379,12 @@ verify that the name in the request corresponds to the name of the
 entity who requested it), and then signed by a CA key, generating a
 new certificate:
 
-.. cpp:function:: X509_Certificate sign_request(const PKCS10_Request& req) const
+.. cpp:function:: X509_Certificate \
+   X509_CA::sign_request(const PKCS10_Request& req) const
+
+Here's an example:
+
+.. literalinclude examples/ca.cpp
 
 Generating CRLs
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^