path: root/doc/lowlevel.txt

The Low-Level Interface
=================================

Botan has two different interfaces. The one documented in this section is meant
more for implementing higher-level types (see the section on filters, earlier in
this manual) than for use by applications. Using it safely requires a solid
knowledge of encryption techniques and best practices, so unless you know, for
example, what CBC mode and nonces are, and why PKCS #1 padding is important,
you should avoid this interface in favor of something working at a higher level
(such as the CMS interface).

Basic Algorithm Abilities
---------------------------------

There are a small handful of functions implemented by most of Botan's
algorithm objects. Among these are:

.. cpp:function:: std::string name()

Returns a human-readable string of the name of this
algorithm. Examples of names returned are "AES-128" and
"HMAC(SHA-512)". You can turn names back into algorithm objects using
the functions in ``lookup.h``.

.. cpp:function:: void clear()

Clear out the algorithm's internal state. A block cipher object will
"forget" its key, a hash function will "forget" any data put into it,
etc. The object will look and behave as it did when you initially
allocated it.

.. cpp:function:: T* clone()

This function is central to Botan's name-based interface. The
``clone`` has many different return types, such as ``BlockCipher``\*
and ``HashFunction``\*, depending on what kind of object it is called
on. Note that unlike Java's clone, this returns a new object in a
"pristine" state; that is, operations done on the initial object
before calling ``clone`` do not affect the initial state of the new
clone.

Cloned objects can (and should) be deallocated with the C++ ``delete``
operator.

Keys and IVs
---------------------------------

Both symmetric keys and initialization values can be considered byte
(or octet) strings. These are represented by the classes
``SymmetricKey`` and ``InitializationVector``, which are
subclasses of ``OctetString``.

Since often it's hard to distinguish between a key and IV, many things
(such as key derivation mechanisms) return ``OctetString`` instead of
``SymmetricKey`` to allow its use as a key or an IV.

.. cpp:function:: OctetString::OctetString(RandomNumberGenerator& rng, size_t length)

  This constructor creates a new random key ``length`` bytes long
  using the random number generator.

.. cpp:function:: OctetString::OctetString(std::string str)

  The argument ``str`` is assumed to be a hex string; it is converted
  to binary and stored. Whitespace is ignored.

.. cpp:function:: OctetString::OctetString(const byte* input, size_t length)

  This constructor copies its input.

Symmetrically Keyed Algorithms
---------------------------------

Block ciphers, stream ciphers, and MACs are all keyed operations; to
be useful, they have to be set to use a particular key, which is a
randomly chosen string of bits of a specified length.  The length
required by any particular algorithm may vary, depending on both the
algorithm specification and the implementation. You can query any
botan object to find out what key length(s) it supports.

To make this similarity in terms of keying explicit, all algorithms of
those types are derived from the ``SymmetricAlgorithm`` base
class. This type provides functions for setting the key, and querying
restrictions on the size of the key:

.. cpp:function:: void set_key(const byte* key, size_t length)

  This sets the key to the value specified. Most algorithms only
  accept keys of certain lengths. If you attempt to call ``set_key``
  with a key length that is not supported, the exception
  ``Invalid_Key_Length`` will be thrown. There is also another version
  of ``set_key`` that takes a ``SymmetricKey`` as an argument.

  In all cases, ``set_key`` must be called on an object before any
  data processing (encryption, decryption, etc) is done by that
  object. If this is not done, the results are undefined, but probably
  will not be good.

.. cpp:function:: bool valid_keylength(size_t length) const

  This function returns true if and only if ``length`` is a valid keylength
  for the algorithm.

.. cpp:function:: size_t minimum_keylength() const

  Return the smallest key length (in bytes) that is acceptible for the
  algorithm.

.. cpp:function:: size_t maximum_keylength() const

  Return the largest key length (in bytes) that is acceptible for the
  algorithm

Block Ciphers
---------------------------------

Block ciphers implement the interface ``BlockCipher``, found in
``block_cipher.h``, as well as the ``SymmetricAlgorithm`` interface.

.. cpp:function:: size_t block_size() const

  Returns the block size of the cipher in bytes

.. cpp:function:: void encrypt_n(const byte* in, byte* out, size_t blocks) const

  Encrypt ``blocks`` blocks of data, taking the input from ``in`` and
  placing the ciphertext in ``out``. The two pointers may be
  identical, but should not overlap ranges.

.. cpp:function:: void encrypt(const byte* in, byte* out) const

  Encrypt a single block, taking the input from ``in`` and placing it
  in ``out``.

.. cpp:function:: void encrypt(byte* block) const

  Identical to ``encrypt(block, block)``.

.. cpp:function:: void decrypt_n(const byte* in, byte* out, size_t blocks) const

  Decrypt ``blocks`` blocks of data, taking the input from ``in`` and
  placing the plaintext in ``out``. The two pointers may be identical,
  but should not overlap ranges.

.. cpp:function:: void decrypt(const byte* in, byte* out) const

  Decrypt a single block, taking the input from ``in`` and placing it
  in ``out``.

.. cpp:function:: void decrypt(byte* block) const

  Identical to ``decrypt(block, block)``.


Stream Ciphers
---------------------------------

Stream ciphers are somewhat different from block ciphers, in that
encrypting data results in changing the internal state of the
cipher. Also, you may encrypt any length of data in one go (in byte
amounts).

.. cpp:function:: void encrypt(const byte* in, byte* out, size_t length)

.. cpp:function:: void encrypt(byte* data, size_t length)

Stream ciphers implement the ``SymmetricAlgorithm`` interface.

Hash Functions / Message Authentication Codes
----------------------------------------------

Hash functions take their input without producing any output, only producing
anything when all input has already taken place. MACs are very similar, but are
additionally keyed. Both of these are derived from the base class
``BufferedComputation``, which has the following functions.

.. cpp:function:: size_t output_length()

Return the size of the output of this function.

.. cpp:function:: void update(const byte* input, size_t length)

.. cpp:function:: void update(byte input)

.. cpp:function:: void update(const std::string& input)

Updates the hash/mac calculation with ``input``.

.. cpp:function:: void final(byte* out)

.. cpp:function:: SecureVector<byte> final()

Complete the hash/MAC calculation and place the result into ``out``.
For the argument taking an array, exactly ``output_length`` bytes will
be written. After you call ``final``, the hash function is reset to
its initial state, so it may be reused immediately.

The second method of using final is to call it with no arguments at
all, as shown in the second prototype. It will return the hash/mac
value in a memory buffer.

There is also a pair of functions called ``process``. They are
a combination of a single ``update``, and ``final``.
Both versions return the final value, rather than placing it an
array. Calling ``process`` with a single byte value isn't
available, mostly because it would rarely be useful.

A MAC can be viewed (in most cases) as a keyed hash function, so
classes that are derived from ``MessageAuthenticationCode`` have
``update`` and ``final`` classes just like a
``HashFunction`` (and like a ``HashFunction``, after
``final`` is called, it can be used to make a new MAC right
away; the key is kept around).

A MAC has the ``SymmetricAlgorithm`` interface in addition to the
``BufferedComputation`` interface.

User Interfaces
---------------------------------

Botan has recently changed some infrastructure to better accommodate
more complex user interfaces, in particular ones that are based on
event loops. Primary among these was the fact that when doing
something like loading a PKCS #8 encoded private key, a passphrase
might be needed, but then again it might not (a PKCS #8 key doesn't
have to be encrypted). Asking for a passphrase to decrypt an
unencrypted key is rather pointless. Not only that, but the way to
handle the user typing the wrong passphrase was complicated,
undocumented, and inefficient.

So now Botan has an object called ``User_Interface``, which provides a
simple interface for the aspects of user interaction the library has
to be concerned with. Currently, this means getting a passphrase from
the user, and that's it (``User_Interface`` will probably be extended
in the future to support other operations as they are needed). The
base ``User_Interface`` class is very stupid, because the library
can't directly assume anything about the environment that it's running
under (for example, if there will be someone sitting at the terminal,
if the application is even *attached* to a terminal, and so on). But
since you can subclass ``User_Interface`` to use whatever method
happens to be appropriate for your application, this isn't a big deal:

.. cpp:function:: std::string User_Interface::get_passphrase(const std::string& what, const std::string& source, UI_Result& result) const

  The ``what`` argument specifies what the passphrase is needed for
  (for example, PKCS #8 key loading passes ``what`` as "PKCS #8
  private key"). This lets you provide the user with some indication
  of *why* your application is asking for a passphrase; feel free to
  pass the string through ``gettext(3)`` or moral equivalent for i18n
  purposes. Similarly, ``source`` specifies where the data in question
  came from, if available (for example, a file name). If the source is
  not available for whatever reason, then ``source`` will be an empty
  string; be sure to account for this possibility when writing a
  ``User_Interface`` subclass.

  The function returns the passphrase as the return value, and a
  status code in ``result`` (either ``OK`` or ``CANCEL_ACTION``). If
  ``CANCEL_ACTION`` is returned in ``result``, then the return value
  will be ignored, and the caller will take whatever action is
  necessary (typically, throwing an exception stating that the
  passphrase couldn't be determined). In the specific case of PKCS #8
  key decryption, a ``Decoding_Error`` exception will be thrown; your
  UI should assume this can happen, and provide appropriate error
  handling (such as putting up a dialog box informing the user of the
  situation, and canceling the operation in progress).

There is an example ``User_Interface`` that uses GTK+ available on the
web site. The ``GTK_UI`` code is cleanly separated from the rest of
the example, so if you happen to be using GTK+, you can copy (and/or
adapt) that code for your application. If you write a
``User_Interface`` object for another windowing system (Win32, Qt,
wxWidgets, FOX, etc), and would like to make it available to users in
general (ideally under a permissive license such as public domain or
MIT/BSD), feel free to send in a copy.


Checksums
---------------------------------

Checksums are very similar to hash functions, and in fact share the same
interface. But there are some significant differences, the major ones being
that the output size is very small (usually in the range of 2 to 4 bytes), and
is not cryptographically secure. But for their intended purpose (error
checking), they perform very well. Some examples of checksums included in Botan
are the Adler32 and CRC32 checksums.

Threads and Mutexes
---------------------------------

Botan includes a mutex system, which is used internally to lock some shared
data structures that must be kept shared for efficiency reasons (mostly, these
are in the allocation systems~--~handing out 1000 separate allocators hurts
performance and makes caching memory blocks useless). This system is supported
by the ``mux_pthr`` module, implementing the ``Mutex`` interface for
systems that have POSIX threads.

If your application is using threads, you *must* add the option
"thread_safe" to the options string when you create the
``LibraryInitializer`` object. If you specify this option and no mutex
type is available, an exception is thrown, since otherwise you would
probably be facing a nasty crash.

Secure Memory
---------------------------------

A major concern with mixing modern multiuser OSes and cryptographic
code is that at any time the code (including secret keys) could be
swapped to disk, where it can later be read by an attacker. Botan
stores almost everything (and especially anything sensitive) in memory
buffers that a) clear out their contents when their destructors are
called, and b) have easy plugins for various memory locking functions,
such as the ``mlock`` call on many Unix systems.

Two of the allocation method used ("malloc" and "mmap") don't
require any extra privileges on Unix, but locking memory does. At
startup, each allocator type will attempt to allocate a few blocks
(typically totaling 128k), so if you want, you can run your
application ``setuid`` ``root``, and then drop privileges
immediately after creating your ``LibraryInitializer``. If you end
up using more than what's been allocated, some of your sensitive data
might end up being swappable, but that beats running as ``root``
all the time.

These classes should also be used within your own code for storing
sensitive data. They are only meant for primitive data types (int,
long, etc): if you want a container of higher level Botan objects, you
can just use a ``std::vector``, since these objects know how to clear
themselves when they are destroyed. You cannot, however, have a
``std::vector`` (or any other container) of ``Pipe`` objects or
filters, because these types have pointers to other filters, and
implementing copy constructors for these types would be both hard and
quite expensive (vectors of pointers to such objects is fine, though).

These types are not described in any great detail: for more information,
consult the definitive sources~--~the header files ``secmem.h`` and
``allocate.h``.

``SecureBuffer`` is a simple array type, whose size is specified at compile
time. It will automatically convert to a pointer of the appropriate type, and
has a number of useful functions, including ``clear()``, and
``size_t`` ``size()``, which returns the length of the array. It is a
template that takes as parameters a type, and a constant integer which is how
long the array is (for example: ``SecureBuffer<byte, 8> key;``).

``SecureVector`` is a variable length array. Its size can be increased or
decreased as need be, and it has a wide variety of functions useful for copying
data into its buffer. Like ``SecureBuffer``, it implements ``clear``
and ``size``.

Allocators
---------------------------------

The containers described above get their memory from allocators. As a
user of the library, you can add new allocator methods at run time for
containers, including the ones used internally by the library, to
use. The interface to this is in ``allocate.h``. Code needing
to allocate or deallocate memory calls ``get_allocator``,
which returns a pointer to an allocator object. This pointer should
not be freed: the caller does not own the allocator (it is shared
among multiple allocatore users, and uses a mutex to serialize access
internally if necessary). It is possible to call
``get_allocator`` with a specific name to request a particular
type of allocator, otherwise, a default allocator type is returned.

At start time, the only allocator known is a ``Default_Allocator``,
which just allocates memory using ``malloc``, and zeroizes it when the
memory is released. It is known by the name "malloc". If you ask for
another type of allocator ("locking" and "mmap" are currently used),
and it is not available, some other allocator will be returned.

You can add in a new allocator type using ``add_allocator_type``. This
function takes a string and a pointer to an allocator. The string gives this
allocator type a name to which it can be referred when one is requesting it
with ``get_allocator``. If an error occurs (such as the name being
already registered), this function returns false. It will return true if the
allocator was successfully registered. If you ask it to,
``LibraryInitializer`` will do this for you.

Finally, you can set the default allocator type that will be returned
using the policy setting "default_alloc" to the name of any previously
registered allocator.