diff options
author | Jack Lloyd <[email protected]> | 2016-04-12 23:03:14 -0400 |
---|---|---|
committer | Jack Lloyd <[email protected]> | 2016-04-21 09:18:54 -0400 |
commit | 8b85b7805151ab8fce5ac9d214c71c4eeb3d6075 (patch) | |
tree | 40cbc2af481dfc2f84e32330308523a5e8f68e44 | |
parent | a4358c96a0de1ab7afc0b437ab79bfc35f2e1824 (diff) |
Remove Transform base class
With sufficient squinting, Transform provided an abstract base
interface that covered both cipher modes and compression algorithms.
However it mapped on neither of them particularly well. In addition
this API had the same problem that has made me dislike the Pipe/Filter
API: given a Transform&, what does it do when you put bits in? Maybe
it encrypts. Maybe it compresses. It's a floor wax and a dessert topping!
Currently the Cipher_Mode interface is left mostly unchanged, with the
APIs previously on Transform just moved down the type hierarchy. I
think there are some definite improvements possible here, wrt handling
of in-place encryption, but left for a later commit.
The compression API is split into two types, Compression_Algorithm and
Decompression_Algorithm. Compression_Algorithm's start() call takes
the compression level, allowing varying compressions with a single
object. And flushing the compression state is moved to a bool param on
`Compression_Algorithm::update`. All the nonsense WRT compression
algorithms having zero length nonces, input granularity rules, etc
as a result of using the Transform interface goes away.
35 files changed, 678 insertions, 664 deletions
diff --git a/doc/manual/compression.rst b/doc/manual/compression.rst index c58ba58a6..808b7f0dc 100644 --- a/doc/manual/compression.rst +++ b/doc/manual/compression.rst @@ -9,39 +9,75 @@ formats), bzip2, and lzma. You should always compress *before* you encrypt, because encryption seeks to hide the redundancy that compression is supposed to try to find and remove. -All compressors provide the `Transform` interface through a subclass -`Compression_Transform` (defined in compression.h). The compression algorithms -have some limitations in terms of the standard API, in particular the -`output_length` function simply throws an exception since the value cannot be -determined merely from the input length for such an algorithm. - -The transformations work much like any other - calling `update` on a vector -returns the (de)compressed result, calling `finish` completes the computation. -All (de)compression algorithms will accept inputs of any size -(update_granularity is 1) and do not require any final data be saved to be -passed to `finish`. - -On `Compression_Transform` an additional function function `flush` is available -which (in addition to always acting as equivalent to an `update`) signals the -compression function to flush as much output as possible immediately, regardless -of considerations of compression ratio. Any compressor or decompressor may -ignore this and treat it as equivalent to a normal update. +Compression is done through the ``Compression_Algorithm`` and +``Decompression_Algorithm`` classes, both defined in `compression.h` + +Compression and decompression both work in three stages: starting a +message (``start``), continuing to process it (``update``), and then +finally completing processing the stream (``finish``). + +.. cpp:class:: Compression_Algorithm + + .. cpp:function:: void start(size_t level) + + Initialize the compression engine. This must be done before calling + ``update`` or ``finish``. The meaning of the `level` parameter varies by + the algorithm but generally takes a value between 1 and 9, with higher + values implying typically better compression from and more memory and/or + CPU time consumed by the compression process. The decompressor can always + handle input from any compressor. + + .. cpp:function:: void update(secure_vector<byte>& buf, \ + size_t offset = 0, bool flush = false) + + Compress the material in the in/out parameter ``buf``. The leading + ``offset`` bytes of ``buf`` are ignored and remain untouched; this can be + useful for ignoring packet headers. If ``flush`` is true, the + compression state is flushed, allowing the decompressor to recover the + entire message up to this point without having the see the rest of the + compressed stream. + + .. cpp::function:: void finish(secure_vector<byte>& buf, size_t offset = 0) + + Finish compressing a message. The ``buf`` and ``offset`` parameters are + treated as in ``update``. It is acceptable to call ``start`` followed by + ``finish`` with the entire message, without any intervening call to + ``update``. + +.. cpp:class:: Decompression_Algorithm + + .. cpp:function:: void start() + + Initialize the decompression engine. This must be done before calling + ``update`` or ``finish``. No level is provided here; the decompressor + can accept input generated by any compression parameters. + + .. cpp:function:: void update(secure_vector<byte>& buf, \ + size_t offset = 0) + + Decompress the material in the in/out parameter ``buf``. The leading + ``offset`` bytes of ``buf`` are ignored and remain untouched; this can be + useful for ignoring packet headers. + + This function may throw if the data seems to be invalid. + + .. cpp::function:: void finish(secure_vector<byte>& buf, size_t offset = 0) + + Finish decompressing a message. The ``buf`` and ``offset`` parameters are + treated as in ``update``. It is acceptable to call ``start`` followed by + ``finish`` with the entire message, without any intervening call to + ``update``. + + This function may throw if the data seems to be invalid. The easiest way to get a compressor is via the functions -.. cpp:function:: Compression_Transform* make_compressor(std::string type, size_t level) -.. cpp:function:: Compression_Transform* make_decompressor(std::string type) +.. cpp:function:: Compression_Algorithm* make_compressor(std::string type) +.. cpp:function:: Decompression_Algorithm* make_decompressor(std::string type) Supported values for `type` include `zlib` (raw zlib with no checksum), `deflate` (zlib's deflate format), `gzip`, `bz2`, and `lzma`. A null pointer -will be returned if the algorithm is unavailable. The meaning of the `level` -parameter varies by the algorithm but generally takes a value between 1 and 9, -with higher values implying typically better compression from and more memory -and/or CPU time consumed by the compression process. The decompressor can always -handle input from any compressor. - -As with any consumer of complex formats, a decompressor may throw an exception -(from either `update` or `finish`) if the input is invalid or corrupt. +will be returned if the algorithm is unavailable. To use a compression algorithm in a `Pipe` use the adaptor types `Compression_Filter` and `Decompression_Filter` from `comp_filter.h`. The diff --git a/src/cli/compress.cpp b/src/cli/compress.cpp index c573eaa20..d5b2c4736 100644 --- a/src/cli/compress.cpp +++ b/src/cli/compress.cpp @@ -6,42 +6,13 @@ #include "cli.h" -#include <botan/transform.h> - #if defined(BOTAN_HAS_COMPRESSION) #include <botan/compression.h> #endif namespace Botan_CLI { -namespace { - -void do_compress(Botan::Transform& comp, - std::ifstream& in, - std::ostream& out, - size_t buf_size) - { - Botan::secure_vector<uint8_t> buf; - - comp.start(); - - while(in.good()) - { - buf.resize(buf_size); - in.read(reinterpret_cast<char*>(&buf[0]), buf.size()); - buf.resize(in.gcount()); - - comp.update(buf); - - out.write(reinterpret_cast<const char*>(&buf[0]), buf.size()); - } - - buf.clear(); - comp.finish(buf); - out.write(reinterpret_cast<const char*>(&buf[0]), buf.size()); - } - -} +#if defined(BOTAN_HAS_COMPRESSION) class Compress final : public Command { @@ -70,12 +41,12 @@ class Compress final : public Command void go() override { const std::string comp_type = get_arg("type"); + const size_t buf_size = get_arg_sz("buf-size"); + const size_t comp_level = get_arg_sz("level"); - std::unique_ptr<Botan::Transform> compress; + std::unique_ptr<Botan::Compression_Algorithm> compress; -#if defined(BOTAN_HAS_COMPRESSION) - compress.reset(Botan::make_compressor(comp_type, get_arg_sz("level"))); -#endif + compress.reset(Botan::make_compressor(comp_type)); if(!compress) { @@ -97,7 +68,25 @@ class Compress final : public Command throw CLI_IO_Error("writing", out_file); } - do_compress(*compress, in, out, get_arg_sz("buf-size")); + Botan::secure_vector<uint8_t> buf; + + compress->start(comp_level); + + while(in.good()) + { + buf.resize(buf_size); + in.read(reinterpret_cast<char*>(&buf[0]), buf.size()); + buf.resize(in.gcount()); + + compress->update(buf); + + out.write(reinterpret_cast<const char*>(&buf[0]), buf.size()); + } + + buf.clear(); + compress->finish(buf); + out.write(reinterpret_cast<const char*>(&buf[0]), buf.size()); + out.close(); } }; @@ -122,6 +111,7 @@ class Decompress final : public Command void go() override { + const size_t buf_size = get_arg_sz("buf-size"); const std::string in_file = get_arg("file"); std::string out_file, suffix; parse_extension(in_file, out_file, suffix); @@ -131,11 +121,9 @@ class Decompress final : public Command if(!in.good()) throw CLI_IO_Error("reading", in_file); - std::unique_ptr<Botan::Transform> decompress; + std::unique_ptr<Botan::Decompression_Algorithm> decompress; -#if defined(BOTAN_HAS_COMPRESSION) decompress.reset(Botan::make_decompressor(suffix)); -#endif if(!decompress) throw CLI_Error_Unsupported("Decompression", suffix); @@ -144,10 +132,30 @@ class Decompress final : public Command if(!out.good()) throw CLI_IO_Error("writing", out_file); - do_compress(*decompress, in, out, get_arg_sz("buf-size")); + Botan::secure_vector<uint8_t> buf; + + decompress->start(); + + while(in.good()) + { + buf.resize(buf_size); + in.read(reinterpret_cast<char*>(&buf[0]), buf.size()); + buf.resize(in.gcount()); + + decompress->update(buf); + + out.write(reinterpret_cast<const char*>(&buf[0]), buf.size()); + } + + buf.clear(); + decompress->finish(buf); + out.write(reinterpret_cast<const char*>(&buf[0]), buf.size()); + out.close(); } }; BOTAN_REGISTER_COMMAND("decompress", Decompress); +#endif + } diff --git a/src/cli/speed.cpp b/src/cli/speed.cpp index c767b2a55..595b4bd20 100644 --- a/src/cli/speed.cpp +++ b/src/cli/speed.cpp @@ -613,13 +613,13 @@ class Speed final : public Command timer.run([&] { srcs.poll_just(accum, src); }); #if defined(BOTAN_HAS_COMPRESSION) - std::unique_ptr<Botan::Compressor_Transform> comp(Botan::make_compressor("zlib", 9)); + std::unique_ptr<Botan::Compression_Algorithm> comp(Botan::make_compressor("zlib")); Botan::secure_vector<uint8_t> compressed; if(comp) { compressed.assign(entropy.begin(), entropy.end()); - comp->start(); + comp->start(9); comp->finish(compressed); } #endif diff --git a/src/lib/base/algo_registry.h b/src/lib/base/algo_registry.h index ebc23bfca..f7e66b3e2 100644 --- a/src/lib/base/algo_registry.h +++ b/src/lib/base/algo_registry.h @@ -294,10 +294,6 @@ make_new_T_1X(const typename Algo_Registry<T>::Spec& spec) #define BOTAN_REGISTER_NAMED_T_2LEN(T, type, name, provider, len1, len2) \ BOTAN_REGISTER_TYPE(T, type, name, (make_new_T_2len<type,len1,len2>), provider, BOTAN_DEFAULT_ALGORITHM_PRIO) -// TODO move elsewhere: -#define BOTAN_REGISTER_TRANSFORM(name, maker) BOTAN_REGISTER_T(Transform, name, maker) -#define BOTAN_REGISTER_TRANSFORM_NOARGS(name) BOTAN_REGISTER_T_NOARGS(Transform, name) - } #endif diff --git a/src/lib/base/info.txt b/src/lib/base/info.txt index 33d22a279..7553007e5 100644 --- a/src/lib/base/info.txt +++ b/src/lib/base/info.txt @@ -8,15 +8,12 @@ secmem.h scan_name.h sym_algo.h symkey.h -transform.h </header:public> <header:internal> algo_registry.h </header:internal> -define TRANSFORM 20131209 - <requires> block hash diff --git a/src/lib/base/transform.cpp b/src/lib/base/transform.cpp deleted file mode 100644 index 8f05a33ad..000000000 --- a/src/lib/base/transform.cpp +++ /dev/null @@ -1,20 +0,0 @@ -/* -* (C) 2015 Jack Lloyd -* -* Botan is released under the Simplified BSD License (see license.txt) -*/ - -#include <botan/internal/algo_registry.h> -#include <botan/transform.h> - -namespace Botan { - -Transform* get_transform(const std::string& specstr, - const std::string& provider, - const std::string& dirstr) - { - Algo_Registry<Transform>::Spec spec(specstr, dirstr); - return Algo_Registry<Transform>::global_registry().make(spec, provider); - } - -} diff --git a/src/lib/base/transform.h b/src/lib/base/transform.h deleted file mode 100644 index cd4ee9880..000000000 --- a/src/lib/base/transform.h +++ /dev/null @@ -1,182 +0,0 @@ -/* -* Transforms of data -* (C) 2013 Jack Lloyd -* -* Botan is released under the Simplified BSD License (see license.txt) -*/ - -#ifndef BOTAN_TRANSFORM_H__ -#define BOTAN_TRANSFORM_H__ - -#include <botan/secmem.h> -#include <botan/key_spec.h> -#include <botan/exceptn.h> -#include <botan/symkey.h> -#include <botan/scan_name.h> -#include <string> -#include <vector> - -namespace Botan { - -/** -* Interface for general transformations on data -*/ -class BOTAN_DLL Transform - { - public: - typedef SCAN_Name Spec; - - /** - * Begin processing a message. - * @param nonce the per message nonce - */ - template<typename Alloc> - secure_vector<byte> start(const std::vector<byte, Alloc>& nonce) - { - return start(nonce.data(), nonce.size()); - } - - /** - * Begin processing a message. - * @param nonce the per message nonce - */ - template<typename Alloc> - BOTAN_DEPRECATED("Use Transform::start") - secure_vector<byte> start_vec(const std::vector<byte, Alloc>& nonce) - { - return start(nonce.data(), nonce.size()); - } - - /** - * Begin processing a message. - * @param nonce the per message nonce - * @param nonce_len length of nonce - */ - secure_vector<byte> start(const byte nonce[], size_t nonce_len) - { - return start_raw(nonce, nonce_len); - } - - /** - * Begin processing a message. - */ - secure_vector<byte> start() - { - return start_raw(nullptr, 0); - } - - virtual secure_vector<byte> start_raw(const byte nonce[], size_t nonce_len) = 0; - - /** - * Process some data. Input must be in size update_granularity() byte blocks. - * @param blocks in/out parameter which will possibly be resized - * @param offset an offset into blocks to begin processing - */ - virtual void update(secure_vector<byte>& blocks, size_t offset = 0) = 0; - - /** - * Complete processing of a message. - * - * @param final_block in/out parameter which must be at least - * minimum_final_size() bytes, and will be set to any final output - * @param offset an offset into final_block to begin processing - */ - virtual void finish(secure_vector<byte>& final_block, size_t offset = 0) = 0; - - /** - * Returns the size of the output if this transform is used to process a - * message with input_length bytes. Will throw if unable to give a precise - * answer. - */ - virtual size_t output_length(size_t input_length) const = 0; - - /** - * @return size of required blocks to update - */ - virtual size_t update_granularity() const = 0; - - /** - * @return required minimium size to finalize() - may be any - * length larger than this. - */ - virtual size_t minimum_final_size() const = 0; - - /** - * Return the default size for a nonce - */ - virtual size_t default_nonce_length() const = 0; - - /** - * Return true iff nonce_len is a valid length for the nonce - */ - virtual bool valid_nonce_length(size_t nonce_len) const = 0; - - /** - * Return some short name describing the provider of this tranformation. - * Useful in cases where multiple implementations are available (eg, - * different implementations of AES). Default "core" is used for the - * 'standard' implementation included in the library. - */ - virtual std::string provider() const { return "core"; } - - virtual std::string name() const = 0; - - virtual void clear() = 0; - - virtual ~Transform() {} - }; - -class BOTAN_DLL Keyed_Transform : public Transform - { - public: - /** - * @return object describing limits on key size - */ - virtual Key_Length_Specification key_spec() const = 0; - - /** - * Check whether a given key length is valid for this algorithm. - * @param length the key length to be checked. - * @return true if the key length is valid. - */ - bool valid_keylength(size_t length) const - { - return key_spec().valid_keylength(length); - } - - template<typename Alloc> - void set_key(const std::vector<byte, Alloc>& key) - { - set_key(key.data(), key.size()); - } - - void set_key(const SymmetricKey& key) - { - set_key(key.begin(), key.length()); - } - - /** - * Set the symmetric key of this transform - * @param key contains the key material - * @param length in bytes of key param - */ - void set_key(const byte key[], size_t length) - { - if(!valid_keylength(length)) - throw Invalid_Key_Length(name(), length); - key_schedule(key, length); - } - - private: - virtual void key_schedule(const byte key[], size_t length) = 0; - }; - -typedef Transform Transformation; - -BOTAN_DLL Transform* get_transform(const std::string& specstr, - const std::string& provider = "", - const std::string& dirstr = ""); - -} - -#endif diff --git a/src/lib/compression/bzip2/bzip2.cpp b/src/lib/compression/bzip2/bzip2.cpp index d9ada84f6..565eb09fc 100644 --- a/src/lib/compression/bzip2/bzip2.cpp +++ b/src/lib/compression/bzip2/bzip2.cpp @@ -39,6 +39,14 @@ class Bzip2_Compression_Stream : public Bzip2_Stream public: explicit Bzip2_Compression_Stream(size_t block_size) { + /* + * Defaults to 900k blocks as the computation cost of + * compression is not overly affected by the size, though + * more memory is required. + */ + if(block_size == 0 || block_size >= 9) + block_size = 9; + int rc = BZ2_bzCompressInit(streamp(), block_size, 0, 0); if(rc == BZ_MEM_ERROR) @@ -98,9 +106,9 @@ class Bzip2_Decompression_Stream : public Bzip2_Stream } -Compression_Stream* Bzip2_Compression::make_stream() const +Compression_Stream* Bzip2_Compression::make_stream(size_t comp_level) const { - return new Bzip2_Compression_Stream(m_block_size); + return new Bzip2_Compression_Stream(comp_level); } Compression_Stream* Bzip2_Decompression::make_stream() const diff --git a/src/lib/compression/bzip2/bzip2.h b/src/lib/compression/bzip2/bzip2.h index 06c80cb8e..18216f7eb 100644 --- a/src/lib/compression/bzip2/bzip2.h +++ b/src/lib/compression/bzip2/bzip2.h @@ -19,21 +19,9 @@ namespace Botan { class BOTAN_DLL Bzip2_Compression final : public Stream_Compression { public: - /** - * @param block_size in 1024 KiB increments, in range from 1 to 9. - * - * Lowering this does not noticably modify the compression or - * decompression speed, though less memory is required for both - * compression and decompression. - */ - Bzip2_Compression(size_t block_size = 9) : m_block_size(block_size) {} - std::string name() const override { return "Bzip2_Compression"; } - private: - Compression_Stream* make_stream() const override; - - const size_t m_block_size; + Compression_Stream* make_stream(size_t comp_level) const override; }; /** diff --git a/src/lib/compression/bzip2/info.txt b/src/lib/compression/bzip2/info.txt index ea2efa6f1..bc8d780be 100644 --- a/src/lib/compression/bzip2/info.txt +++ b/src/lib/compression/bzip2/info.txt @@ -1,4 +1,4 @@ -define BZIP2_TRANSFORM 20141118 +define BZIP2 20160412 load_on vendor diff --git a/src/lib/compression/compress_utils.h b/src/lib/compression/compress_utils.h index 2a830ac4e..a60ae3c22 100644 --- a/src/lib/compression/compress_utils.h +++ b/src/lib/compression/compress_utils.h @@ -86,8 +86,8 @@ class Zlib_Style_Stream : public Compression_Stream }; #define BOTAN_REGISTER_COMPRESSION(C, D) \ - BOTAN_REGISTER_T_1LEN(Transform, C, 9); \ - BOTAN_REGISTER_T_NOARGS(Transform, D) + BOTAN_REGISTER_T_NOARGS(Compression_Algorithm, C); \ + BOTAN_REGISTER_T_NOARGS(Decompression_Algorithm, D) } diff --git a/src/lib/compression/compression.cpp b/src/lib/compression/compression.cpp index 54faec7b8..f289aa79f 100644 --- a/src/lib/compression/compression.cpp +++ b/src/lib/compression/compression.cpp @@ -54,9 +54,7 @@ void Compression_Alloc_Info::do_free(void* ptr) } } -namespace { - -Compressor_Transform* do_make_compressor(const std::string& type, const std::string& suffix) +Compression_Algorithm* make_compressor(const std::string& type) { const std::map<std::string, std::string> trans{ {"zlib", "Zlib"}, @@ -73,31 +71,29 @@ Compressor_Transform* do_make_compressor(const std::string& type, const std::str if(i == trans.end()) return nullptr; - const std::string t_name = i->second + suffix; - - std::unique_ptr<Transform> t(get_transform(t_name)); - - if(!t) - return nullptr; - - Compressor_Transform* r = dynamic_cast<Compressor_Transform*>(t.get()); - if(!r) - throw Exception("Bad cast of compression object " + t_name); - - t.release(); - return r; + const SCAN_Name t_name(i->second + "_Compression"); + return Algo_Registry<Compression_Algorithm>::global_registry().make(t_name); } -} - -Compressor_Transform* make_compressor(const std::string& type, size_t level) +Decompression_Algorithm* make_decompressor(const std::string& type) { - return do_make_compressor(type, "_Compression(" + std::to_string(level) + ")"); - } + const std::map<std::string, std::string> trans{ + {"zlib", "Zlib"}, + {"deflate", "Deflate"}, + {"gzip", "Gzip"}, + {"gz", "Gzip"}, + {"bzip2", "Bzip2"}, + {"bz2", "Bzip2"}, + {"lzma", "LZMA"}, + {"xz", "LZMA"}}; -Compressor_Transform* make_decompressor(const std::string& type) - { - return do_make_compressor(type, "_Decompression"); + auto i = trans.find(type); + + if(i == trans.end()) + return nullptr; + + const SCAN_Name t_name(i->second + "_Decompression"); + return Algo_Registry<Decompression_Algorithm>::global_registry().make(t_name); } void Stream_Compression::clear() @@ -105,13 +101,9 @@ void Stream_Compression::clear() m_stream.reset(); } -secure_vector<byte> Stream_Compression::start_raw(const byte[], size_t nonce_len) +void Stream_Compression::start(size_t level) { - if(!valid_nonce_length(nonce_len)) - throw Invalid_IV_Length(name(), nonce_len); - - m_stream.reset(make_stream()); - return secure_vector<byte>(); + m_stream.reset(make_stream(level)); } void Stream_Compression::process(secure_vector<byte>& buf, size_t offset, u32bit flags) @@ -154,16 +146,10 @@ void Stream_Compression::process(secure_vector<byte>& buf, size_t offset, u32bit buf.swap(m_buffer); } -void Stream_Compression::update(secure_vector<byte>& buf, size_t offset) - { - BOTAN_ASSERT(m_stream, "Initialized"); - process(buf, offset, m_stream->run_flag()); - } - -void Stream_Compression::flush(secure_vector<byte>& buf, size_t offset) +void Stream_Compression::update(secure_vector<byte>& buf, size_t offset, bool flush) { BOTAN_ASSERT(m_stream, "Initialized"); - process(buf, offset, m_stream->flush_flag()); + process(buf, offset, flush ? m_stream->flush_flag() : m_stream->run_flag()); } void Stream_Compression::finish(secure_vector<byte>& buf, size_t offset) @@ -178,14 +164,9 @@ void Stream_Decompression::clear() m_stream.reset(); } -secure_vector<byte> Stream_Decompression::start_raw(const byte[], size_t nonce_len) +void Stream_Decompression::start() { - if(!valid_nonce_length(nonce_len)) - throw Invalid_IV_Length(name(), nonce_len); - m_stream.reset(make_stream()); - - return secure_vector<byte>(); } void Stream_Decompression::process(secure_vector<byte>& buf, size_t offset, u32bit flags) diff --git a/src/lib/compression/compression.h b/src/lib/compression/compression.h index 66aaacdc4..81b863dcf 100644 --- a/src/lib/compression/compression.h +++ b/src/lib/compression/compression.h @@ -8,32 +8,76 @@ #ifndef BOTAN_COMPRESSION_TRANSFORM_H__ #define BOTAN_COMPRESSION_TRANSFORM_H__ -#include <botan/transform.h> +#include <botan/secmem.h> +#include <botan/scan_name.h> namespace Botan { -class BOTAN_DLL Compressor_Transform : public Transform +class BOTAN_DLL Compression_Algorithm { public: - size_t update_granularity() const override final { return 1; } + typedef SCAN_Name Spec; + + /** + * Begin compressing. Most compression algorithms offer a tunable + * time/compression tradeoff parameter generally represented by + * an integer in the range of 1 to 9. + * + * If 0 or a value out of range is provided, a compression algorithm + * specific default is used. + */ + virtual void start(size_t comp_level = 0) = 0; + + /** + * Process some data. Input must be in size update_granularity() byte blocks. + * @param blocks in/out parameter which will possibly be resized or swapped + * @param offset an offset into blocks to begin processing + * @param flush if true the compressor will be told to flush state + */ + virtual void update(secure_vector<byte>& buf, size_t offset = 0, bool flush = false) = 0; + + /** + * Finish compressing + * + * @param final_block in/out parameter + * @param offset an offset into final_block to begin processing + */ + virtual void finish(secure_vector<byte>& final_block, size_t offset = 0) = 0; + + virtual std::string name() const = 0; + + /** + * Reset the state and abort the current message; start can be + * called again to process a new message. + */ + virtual void clear() = 0; + + virtual ~Compression_Algorithm() {} + }; + +class BOTAN_DLL Decompression_Algorithm + { + public: + typedef SCAN_Name Spec; - size_t minimum_final_size() const override final { return 0; } + /** + * Decompression does not support levels + */ + virtual void start() = 0; - size_t default_nonce_length() const override final { return 0; } + virtual void update(secure_vector<byte>& buf, size_t offset = 0) = 0; - bool valid_nonce_length(size_t nonce_len) const override final - { return nonce_len == 0; } + virtual void finish(secure_vector<byte>& final_block, size_t offset = 0) = 0; - virtual void flush(secure_vector<byte>& buf, size_t offset = 0) { update(buf, offset); } + virtual std::string name() const = 0; - size_t output_length(size_t) const override final - { - throw Exception(name() + " output length indeterminate"); - } + virtual void clear() = 0; + + virtual ~Decompression_Algorithm() {} }; -BOTAN_DLL Compressor_Transform* make_compressor(const std::string& type, size_t level); -BOTAN_DLL Compressor_Transform* make_decompressor(const std::string& type); +BOTAN_DLL Compression_Algorithm* make_compressor(const std::string& type); +BOTAN_DLL Decompression_Algorithm* make_decompressor(const std::string& type); class Compression_Stream { @@ -55,39 +99,37 @@ class Compression_Stream virtual bool run(u32bit flags) = 0; }; -class BOTAN_DLL Stream_Compression : public Compressor_Transform +class Stream_Compression : public Compression_Algorithm { public: - void update(secure_vector<byte>& buf, size_t offset = 0) final override; - - void flush(secure_vector<byte>& buf, size_t offset = 0) final override; + void update(secure_vector<byte>& buf, size_t offset, bool flush) final override; - void finish(secure_vector<byte>& buf, size_t offset = 0) final override; + void finish(secure_vector<byte>& buf, size_t offset) final override; void clear() final override; private: - secure_vector<byte> start_raw(const byte[], size_t) final override; + void start(size_t level) final override; void process(secure_vector<byte>& buf, size_t offset, u32bit flags); - virtual Compression_Stream* make_stream() const = 0; + virtual Compression_Stream* make_stream(size_t level) const = 0; secure_vector<byte> m_buffer; std::unique_ptr<Compression_Stream> m_stream; }; -class BOTAN_DLL Stream_Decompression : public Compressor_Transform +class Stream_Decompression : public Decompression_Algorithm { public: - void update(secure_vector<byte>& buf, size_t offset = 0) final override; + void update(secure_vector<byte>& buf, size_t offset) final override; - void finish(secure_vector<byte>& buf, size_t offset = 0) final override; + void finish(secure_vector<byte>& buf, size_t offset) final override; void clear() final override; private: - secure_vector<byte> start_raw(const byte[], size_t) final override; + void start() final override; void process(secure_vector<byte>& buf, size_t offset, u32bit flags); diff --git a/src/lib/compression/lzma/info.txt b/src/lib/compression/lzma/info.txt index 443a96919..7a712338e 100644 --- a/src/lib/compression/lzma/info.txt +++ b/src/lib/compression/lzma/info.txt @@ -1,4 +1,4 @@ -define LZMA_TRANSFORM 20141118 +define LZMA 20160412 load_on vendor diff --git a/src/lib/compression/lzma/lzma.cpp b/src/lib/compression/lzma/lzma.cpp index 3cc03a098..18701278d 100644 --- a/src/lib/compression/lzma/lzma.cpp +++ b/src/lib/compression/lzma/lzma.cpp @@ -58,6 +58,11 @@ class LZMA_Compression_Stream : public LZMA_Stream public: explicit LZMA_Compression_Stream(size_t level) { + if(level == 0) + level = 6; // default + else if(level > 9) + level = 9; // clamp to maximum allowed value + lzma_ret rc = ::lzma_easy_encoder(streamp(), level, LZMA_CHECK_CRC64); if(rc == LZMA_MEM_ERROR) @@ -84,9 +89,9 @@ class LZMA_Decompression_Stream : public LZMA_Stream } -Compression_Stream* LZMA_Compression::make_stream() const +Compression_Stream* LZMA_Compression::make_stream(size_t level) const { - return new LZMA_Compression_Stream(m_level); + return new LZMA_Compression_Stream(level); } Compression_Stream* LZMA_Decompression::make_stream() const diff --git a/src/lib/compression/lzma/lzma.h b/src/lib/compression/lzma/lzma.h index d9ea10091..63722b064 100644 --- a/src/lib/compression/lzma/lzma.h +++ b/src/lib/compression/lzma/lzma.h @@ -20,19 +20,10 @@ namespace Botan { class BOTAN_DLL LZMA_Compression final : public Stream_Compression { public: - /** - * @param level how much effort to use on compressing (0 to 9); - * higher levels are slower but tend to give better - * compression - */ - LZMA_Compression(size_t level = 6) : m_level(level) {} - std::string name() const override { return "LZMA_Compression"; } private: - Compression_Stream* make_stream() const override; - - const size_t m_level; + Compression_Stream* make_stream(size_t level) const override; }; /** diff --git a/src/lib/compression/zlib/info.txt b/src/lib/compression/zlib/info.txt index 8b722350f..6c82c49a3 100644 --- a/src/lib/compression/zlib/info.txt +++ b/src/lib/compression/zlib/info.txt @@ -1,4 +1,4 @@ -define ZLIB_TRANSFORM 20141118 +define ZLIB 20160412 load_on vendor diff --git a/src/lib/compression/zlib/zlib.cpp b/src/lib/compression/zlib/zlib.cpp index 6df5ee931..836925a68 100644 --- a/src/lib/compression/zlib/zlib.cpp +++ b/src/lib/compression/zlib/zlib.cpp @@ -50,20 +50,25 @@ class Zlib_Compression_Stream : public Zlib_Stream { wbits = compute_window_bits(wbits, wbits_offset); - int rc = deflateInit2(streamp(), level, Z_DEFLATED, wbits, - 8, Z_DEFAULT_STRATEGY); + if(level >= 9) + level = 9; + else if(level == 0) + level = 6; + + int rc = ::deflateInit2(streamp(), level, Z_DEFLATED, wbits, 8, Z_DEFAULT_STRATEGY); + if(rc != Z_OK) throw Exception("zlib deflate initialization failed"); } ~Zlib_Compression_Stream() { - deflateEnd(streamp()); + ::deflateEnd(streamp()); } bool run(u32bit flags) override { - int rc = deflate(streamp(), flags); + int rc = ::deflate(streamp(), flags); if(rc == Z_MEM_ERROR) throw Exception("zlib memory allocation failure"); @@ -79,7 +84,7 @@ class Zlib_Decompression_Stream : public Zlib_Stream public: Zlib_Decompression_Stream(int wbits, int wbits_offset = 0) { - int rc = inflateInit2(streamp(), compute_window_bits(wbits, wbits_offset)); + int rc = ::inflateInit2(streamp(), compute_window_bits(wbits, wbits_offset)); if(rc == Z_MEM_ERROR) throw Exception("zlib memory allocation failure"); @@ -89,12 +94,12 @@ class Zlib_Decompression_Stream : public Zlib_Stream ~Zlib_Decompression_Stream() { - inflateEnd(streamp()); + ::inflateEnd(streamp()); } bool run(u32bit flags) override { - int rc = inflate(streamp(), flags); + int rc = ::inflate(streamp(), flags); if(rc == Z_MEM_ERROR) throw Exception("zlib memory allocation failure"); @@ -145,9 +150,9 @@ class Gzip_Decompression_Stream : public Zlib_Decompression_Stream } -Compression_Stream* Zlib_Compression::make_stream() const +Compression_Stream* Zlib_Compression::make_stream(size_t level) const { - return new Zlib_Compression_Stream(m_level, 15); + return new Zlib_Compression_Stream(level, 15); } Compression_Stream* Zlib_Decompression::make_stream() const @@ -155,9 +160,9 @@ Compression_Stream* Zlib_Decompression::make_stream() const return new Zlib_Decompression_Stream(15); } -Compression_Stream* Deflate_Compression::make_stream() const +Compression_Stream* Deflate_Compression::make_stream(size_t level) const { - return new Deflate_Compression_Stream(m_level, 15); + return new Deflate_Compression_Stream(level, 15); } Compression_Stream* Deflate_Decompression::make_stream() const @@ -165,9 +170,9 @@ Compression_Stream* Deflate_Decompression::make_stream() const return new Deflate_Decompression_Stream(15); } -Compression_Stream* Gzip_Compression::make_stream() const +Compression_Stream* Gzip_Compression::make_stream(size_t level) const { - return new Gzip_Compression_Stream(m_level, 15, m_os_code); + return new Gzip_Compression_Stream(level, 15, m_os_code); } Compression_Stream* Gzip_Decompression::make_stream() const diff --git a/src/lib/compression/zlib/zlib.h b/src/lib/compression/zlib/zlib.h index 6a8cead14..0cedb1eab 100644 --- a/src/lib/compression/zlib/zlib.h +++ b/src/lib/compression/zlib/zlib.h @@ -19,20 +19,9 @@ namespace Botan { class BOTAN_DLL Zlib_Compression final : public Stream_Compression { public: - /** - * @param level how much effort to use on compressing (0 to 9); - * higher levels are slower but tend to give better - * compression - */ - - Zlib_Compression(size_t level = 6) : m_level(level) {} - std::string name() const override { return "Zlib_Compression"; } - private: - Compression_Stream* make_stream() const override; - - const size_t m_level; + Compression_Stream* make_stream(size_t level) const override; }; /** @@ -42,7 +31,6 @@ class BOTAN_DLL Zlib_Decompression final : public Stream_Decompression { public: std::string name() const override { return "Zlib_Decompression"; } - private: Compression_Stream* make_stream() const override; }; @@ -53,19 +41,9 @@ class BOTAN_DLL Zlib_Decompression final : public Stream_Decompression class BOTAN_DLL Deflate_Compression final : public Stream_Compression { public: - /** - * @param level how much effort to use on compressing (0 to 9); - * higher levels are slower but tend to give better - * compression - */ - Deflate_Compression(size_t level = 6) : m_level(level) {} - std::string name() const override { return "Deflate_Compression"; } - private: - Compression_Stream* make_stream() const override; - - const size_t m_level; + Compression_Stream* make_stream(size_t level) const override; }; /** @@ -75,7 +53,6 @@ class BOTAN_DLL Deflate_Decompression final : public Stream_Decompression { public: std::string name() const override { return "Deflate_Decompression"; } - private: Compression_Stream* make_stream() const override; }; @@ -86,20 +63,11 @@ class BOTAN_DLL Deflate_Decompression final : public Stream_Decompression class BOTAN_DLL Gzip_Compression final : public Stream_Compression { public: - /** - * @param level how much effort to use on compressing (0 to 9); - * higher levels are slower but tend to give better - * compression - */ - Gzip_Compression(size_t level = 6, byte os_code = 255) : - m_level(level), m_os_code(os_code) {} + Gzip_Compression(byte os_code = 255) : m_os_code(os_code) {} std::string name() const override { return "Gzip_Compression"; } - private: - Compression_Stream* make_stream() const override; - - const size_t m_level; + Compression_Stream* make_stream(size_t level) const override; const byte m_os_code; }; @@ -110,7 +78,6 @@ class BOTAN_DLL Gzip_Decompression final : public Stream_Decompression { public: std::string name() const override { return "Gzip_Decompression"; } - private: Compression_Stream* make_stream() const override; }; diff --git a/src/lib/filters/aead_filt.h b/src/lib/filters/aead_filt.h index a97b580bd..c86739fc8 100644 --- a/src/lib/filters/aead_filt.h +++ b/src/lib/filters/aead_filt.h @@ -8,7 +8,7 @@ #ifndef BOTAN_AEAD_FILTER_H__ #define BOTAN_AEAD_FILTER_H__ -#include <botan/transform_filter.h> +#include <botan/cipher_filter.h> #include <botan/aead.h> namespace Botan { @@ -16,10 +16,10 @@ namespace Botan { /** * Filter interface for AEAD Modes */ -class BOTAN_DLL AEAD_Filter : public Transform_Filter +class AEAD_Filter : public Cipher_Mode_Filter { public: - AEAD_Filter(AEAD_Mode* aead) : Transform_Filter(aead) {} + AEAD_Filter(AEAD_Mode* aead) : Cipher_Mode_Filter(aead) {} /** * Set associated data that is not included in the ciphertext but diff --git a/src/lib/filters/cipher_filter.cpp b/src/lib/filters/cipher_filter.cpp new file mode 100644 index 000000000..f91cf3aa2 --- /dev/null +++ b/src/lib/filters/cipher_filter.cpp @@ -0,0 +1,114 @@ +/* +* Filter interface for Cipher_Modes +* (C) 2013,2014 Jack Lloyd +* +* Botan is released under the Simplified BSD License (see license.txt) +*/ + +#include <botan/cipher_filter.h> +#include <botan/internal/rounding.h> + +namespace Botan { + +namespace { + +size_t choose_update_size(size_t update_granularity) + { + const size_t target_size = 1024; + + if(update_granularity >= target_size) + return update_granularity; + + return round_up(target_size, update_granularity); + } + +} + +Cipher_Mode_Filter::Cipher_Mode_Filter(Cipher_Mode* mode) : + Buffered_Filter(choose_update_size(mode->update_granularity()), + mode->minimum_final_size()), + m_nonce(mode->default_nonce_length() == 0), + m_mode(mode), + m_buffer(m_mode->update_granularity()) + { + } + +std::string Cipher_Mode_Filter::name() const + { + return m_mode->name(); + } + +void Cipher_Mode_Filter::Nonce_State::update(const InitializationVector& iv) + { + m_nonce = unlock(iv.bits_of()); + m_fresh_nonce = true; + } + +std::vector<byte> Cipher_Mode_Filter::Nonce_State::get() + { + BOTAN_ASSERT(m_fresh_nonce, "The nonce is fresh for this message"); + + if(!m_nonce.empty()) + m_fresh_nonce = false; + return m_nonce; + } + +void Cipher_Mode_Filter::set_iv(const InitializationVector& iv) + { + m_nonce.update(iv); + } + +void Cipher_Mode_Filter::set_key(const SymmetricKey& key) + { + m_mode->set_key(key); + } + +Key_Length_Specification Cipher_Mode_Filter::key_spec() const + { + return m_mode->key_spec(); + } + +bool Cipher_Mode_Filter::valid_iv_length(size_t length) const + { + return m_mode->valid_nonce_length(length); + } + +void Cipher_Mode_Filter::write(const byte input[], size_t input_length) + { + Buffered_Filter::write(input, input_length); + } + +void Cipher_Mode_Filter::end_msg() + { + Buffered_Filter::end_msg(); + } + +void Cipher_Mode_Filter::start_msg() + { + send(m_mode->start(m_nonce.get())); + } + +void Cipher_Mode_Filter::buffered_block(const byte input[], size_t input_length) + { + while(input_length) + { + const size_t take = std::min(m_mode->update_granularity(), input_length); + + m_buffer.assign(input, input + take); + m_mode->update(m_buffer); + + send(m_buffer); + + input += take; + input_length -= take; + } + } + +void Cipher_Mode_Filter::buffered_final(const byte input[], size_t input_length) + { + secure_vector<byte> buf(input, input + input_length); + m_mode->finish(buf); + send(buf); + } + +} diff --git a/src/lib/filters/transform_filter.h b/src/lib/filters/cipher_filter.h index 2ecc5cecb..e0681f82a 100644 --- a/src/lib/filters/transform_filter.h +++ b/src/lib/filters/cipher_filter.h @@ -1,6 +1,6 @@ /* -* Filter interface for Transforms -* (C) 2013 Jack Lloyd +* Filter interface for ciphers +* (C) 2013,2016 Jack Lloyd * * Botan is released under the Simplified BSD License (see license.txt) */ @@ -8,20 +8,20 @@ #ifndef BOTAN_TRANSFORM_FILTER_H__ #define BOTAN_TRANSFORM_FILTER_H__ -#include <botan/transform.h> +#include <botan/cipher_mode.h> #include <botan/key_filt.h> #include <botan/buf_filt.h> namespace Botan { /** -* Filter interface for Transforms +* Filter interface for cipher modes */ -class BOTAN_DLL Transform_Filter : public Keyed_Filter, - private Buffered_Filter +class BOTAN_DLL Cipher_Mode_Filter : public Keyed_Filter, + private Buffered_Filter { public: - explicit Transform_Filter(Transform* t); + explicit Cipher_Mode_Filter(Cipher_Mode* t); void set_iv(const InitializationVector& iv) override; @@ -34,9 +34,9 @@ class BOTAN_DLL Transform_Filter : public Keyed_Filter, std::string name() const override; protected: - const Transform& get_transform() const { return *m_transform; } + const Cipher_Mode& get_mode() const { return *m_mode; } - Transform& get_transform() { return *m_transform; } + Cipher_Mode& get_mode() { return *m_mode; } private: void write(const byte input[], size_t input_length) override; @@ -59,10 +59,12 @@ class BOTAN_DLL Transform_Filter : public Keyed_Filter, }; Nonce_State m_nonce; - std::unique_ptr<Transform> m_transform; + std::unique_ptr<Cipher_Mode> m_mode; secure_vector<byte> m_buffer; }; +// deprecated aliases, will be removed before 2.0 +typedef Cipher_Mode_Filter Transform_Filter; typedef Transform_Filter Transformation_Filter; } diff --git a/src/lib/filters/comp_filter.cpp b/src/lib/filters/comp_filter.cpp index ab11526dd..a89fd68ae 100644 --- a/src/lib/filters/comp_filter.cpp +++ b/src/lib/filters/comp_filter.cpp @@ -1,6 +1,6 @@ /* * Filter interface for compression -* (C) 2014,2015 Jack Lloyd +* (C) 2014,2015,2016 Jack Lloyd * (C) 2015 Matej Kenda * * Botan is released under the Simplified BSD License (see license.txt) @@ -8,44 +8,83 @@ #include <botan/comp_filter.h> #include <botan/compression.h> +#include <botan/exceptn.h> namespace Botan { Compression_Filter::Compression_Filter(const std::string& type, size_t level, size_t bs) : - Compression_Decompression_Filter(make_compressor(type, level), bs) + m_comp(make_compressor(type)), + m_buffersize(std::max<size_t>(bs, 256)), + m_level(level) { + if(!m_comp) + { + throw Invalid_Argument("Compression type '" + type + "' not found"); + } } -Decompression_Filter::Decompression_Filter(const std::string& type, size_t bs) : - Compression_Decompression_Filter(make_decompressor(type), bs) +std::string Compression_Filter::name() const { + return m_comp->name(); } -Compression_Decompression_Filter::Compression_Decompression_Filter(Transform* transform, size_t bs) : - m_buffersize(std::max<size_t>(256, bs)), m_buffer(m_buffersize) +void Compression_Filter::start_msg() { - if (!transform) + m_comp->start(m_level); + } + +void Compression_Filter::write(const byte input[], size_t input_length) + { + while(input_length) { - throw Invalid_Argument("Transform is null"); + const size_t take = std::min(m_buffersize, input_length); + BOTAN_ASSERT(take > 0, "Consumed something"); + + m_buffer.assign(input, input + take); + m_comp->update(m_buffer); + + send(m_buffer); + + input += take; + input_length -= take; } - m_transform.reset(dynamic_cast<Compressor_Transform*>(transform)); - if(!m_transform) + } + +void Compression_Filter::flush() + { + m_buffer.clear(); + m_comp->update(m_buffer, 0, true); + send(m_buffer); + } + +void Compression_Filter::end_msg() + { + m_buffer.clear(); + m_comp->finish(m_buffer); + send(m_buffer); + } + +Decompression_Filter::Decompression_Filter(const std::string& type, size_t bs) : + m_comp(make_decompressor(type)), + m_buffersize(std::max<size_t>(bs, 256)) + { + if(!m_comp) { - throw Invalid_Argument("Transform " + transform->name() + " is not a compressor"); + throw Invalid_Argument("Compression type '" + type + "' not found"); } } -std::string Compression_Decompression_Filter::name() const +std::string Decompression_Filter::name() const { - return m_transform->name(); + return m_comp->name(); } -void Compression_Decompression_Filter::start_msg() +void Decompression_Filter::start_msg() { - send(m_transform->start()); + m_comp->start(); } -void Compression_Decompression_Filter::write(const byte input[], size_t input_length) +void Decompression_Filter::write(const byte input[], size_t input_length) { while(input_length) { @@ -53,7 +92,7 @@ void Compression_Decompression_Filter::write(const byte input[], size_t input_le BOTAN_ASSERT(take > 0, "Consumed something"); m_buffer.assign(input, input + take); - m_transform->update(m_buffer); + m_comp->update(m_buffer); send(m_buffer); @@ -62,17 +101,10 @@ void Compression_Decompression_Filter::write(const byte input[], size_t input_le } } -void Compression_Decompression_Filter::flush() - { - m_buffer.clear(); - m_transform->flush(m_buffer); - send(m_buffer); - } - -void Compression_Decompression_Filter::end_msg() +void Decompression_Filter::end_msg() { m_buffer.clear(); - m_transform->finish(m_buffer); + m_comp->finish(m_buffer); send(m_buffer); } diff --git a/src/lib/filters/comp_filter.h b/src/lib/filters/comp_filter.h index bb928b640..4d347c42d 100644 --- a/src/lib/filters/comp_filter.h +++ b/src/lib/filters/comp_filter.h @@ -1,6 +1,6 @@ /* * Filter interface for compression -* (C) 2014,2015 Jack Lloyd +* (C) 2014,2015,2016 Jack Lloyd * * Botan is released under the Simplified BSD License (see license.txt) */ @@ -12,46 +12,50 @@ namespace Botan { -class Transform; -class Compressor_Transform; +class Compression_Algorithm; +class Decompression_Algorithm; /** -* Filter interface for compression/decompression +* Filter interface for compression */ -class BOTAN_DLL Compression_Decompression_Filter : public Filter +class BOTAN_DLL Compression_Filter : public Filter { public: void start_msg() override; void write(const byte input[], size_t input_length) override; void end_msg() override; - std::string name() const override; + void flush(); - protected: - Compression_Decompression_Filter(Transform* t, size_t bs); + std::string name() const override; - void flush(); + Compression_Filter(const std::string& type, + size_t compression_level, + size_t buffer_size = 4096); private: - std::unique_ptr<Compressor_Transform> m_transform; - std::size_t m_buffersize; + std::unique_ptr<Compression_Algorithm> m_comp; + size_t m_buffersize, m_level; secure_vector<byte> m_buffer; }; -class BOTAN_DLL Compression_Filter : public Compression_Decompression_Filter +/** +* Filter interface for decompression +*/ +class BOTAN_DLL Decompression_Filter : public Filter { public: - Compression_Filter(const std::string& type, - size_t compression_level, - size_t buffer_size = 4096); + void start_msg() override; + void write(const byte input[], size_t input_length) override; + void end_msg() override; - using Compression_Decompression_Filter::flush; - }; + std::string name() const override; -class BOTAN_DLL Decompression_Filter : public Compression_Decompression_Filter - { - public: Decompression_Filter(const std::string& type, size_t buffer_size = 4096); + private: + std::unique_ptr<Decompression_Algorithm> m_comp; + std::size_t m_buffersize; + secure_vector<byte> m_buffer; }; } diff --git a/src/lib/filters/info.txt b/src/lib/filters/info.txt index fbecd9c87..217baea1d 100644 --- a/src/lib/filters/info.txt +++ b/src/lib/filters/info.txt @@ -1,4 +1,4 @@ -define FILTERS 20131128 +define FILTERS 20160415 <source> algo_filt.cpp @@ -14,7 +14,7 @@ pipe_io.cpp pipe_rw.cpp secqueue.cpp threaded_fork.cpp -transform_filter.cpp +cipher_filter.cpp </source> <header:public> @@ -27,7 +27,7 @@ filters.h key_filt.h pipe.h secqueue.h -transform_filter.h +cipher_filter.h </header:public> <header:internal> diff --git a/src/lib/filters/key_filt.cpp b/src/lib/filters/key_filt.cpp index 2cc350268..0f6a67da9 100644 --- a/src/lib/filters/key_filt.cpp +++ b/src/lib/filters/key_filt.cpp @@ -5,7 +5,7 @@ */ #include <botan/key_filt.h> -#include <botan/transform_filter.h> +#include <botan/cipher_filter.h> namespace Botan { @@ -14,7 +14,7 @@ Keyed_Filter* get_cipher(const std::string& algo_spec, { std::unique_ptr<Cipher_Mode> c(get_cipher_mode(algo_spec, direction)); if(c) - return new Transform_Filter(c.release()); + return new Cipher_Mode_Filter(c.release()); throw Algorithm_Not_Found(algo_spec); } diff --git a/src/lib/filters/transform_filter.cpp b/src/lib/filters/transform_filter.cpp deleted file mode 100644 index 27a22f9bd..000000000 --- a/src/lib/filters/transform_filter.cpp +++ /dev/null @@ -1,119 +0,0 @@ -/* -* Filter interface for Transforms -* (C) 2013,2014 Jack Lloyd -* -* Botan is released under the Simplified BSD License (see license.txt) -*/ - -#include <botan/transform_filter.h> -#include <botan/internal/rounding.h> - -namespace Botan { - -namespace { - -size_t choose_update_size(size_t update_granularity) - { - const size_t target_size = 1024; - - if(update_granularity >= target_size) - return update_granularity; - - return round_up(target_size, update_granularity); - } - -} - -Transform_Filter::Transform_Filter(Transform* transform) : - Buffered_Filter(choose_update_size(transform->update_granularity()), - transform->minimum_final_size()), - m_nonce(transform->default_nonce_length() == 0), - m_transform(transform), - m_buffer(m_transform->update_granularity()) - { - } - -std::string Transform_Filter::name() const - { - return m_transform->name(); - } - -void Transform_Filter::Nonce_State::update(const InitializationVector& iv) - { - m_nonce = unlock(iv.bits_of()); - m_fresh_nonce = true; - } - -std::vector<byte> Transform_Filter::Nonce_State::get() - { - BOTAN_ASSERT(m_fresh_nonce, "The nonce is fresh for this message"); - - if(!m_nonce.empty()) - m_fresh_nonce = false; - return m_nonce; - } - -void Transform_Filter::set_iv(const InitializationVector& iv) - { - m_nonce.update(iv); - } - -void Transform_Filter::set_key(const SymmetricKey& key) - { - if(Keyed_Transform* keyed = dynamic_cast<Keyed_Transform*>(m_transform.get())) - keyed->set_key(key); - else if(key.length() != 0) - throw Exception("Transform " + name() + " does not accept keys"); - } - -Key_Length_Specification Transform_Filter::key_spec() const - { - if(Keyed_Transform* keyed = dynamic_cast<Keyed_Transform*>(m_transform.get())) - return keyed->key_spec(); - return Key_Length_Specification(0); - } - -bool Transform_Filter::valid_iv_length(size_t length) const - { - return m_transform->valid_nonce_length(length); - } - -void Transform_Filter::write(const byte input[], size_t input_length) - { - Buffered_Filter::write(input, input_length); - } - -void Transform_Filter::end_msg() - { - Buffered_Filter::end_msg(); - } - -void Transform_Filter::start_msg() - { - send(m_transform->start(m_nonce.get())); - } - -void Transform_Filter::buffered_block(const byte input[], size_t input_length) - { - while(input_length) - { - const size_t take = std::min(m_transform->update_granularity(), input_length); - - m_buffer.assign(input, input + take); - m_transform->update(m_buffer); - - send(m_buffer); - - input += take; - input_length -= take; - } - } - -void Transform_Filter::buffered_final(const byte input[], size_t input_length) - { - secure_vector<byte> buf(input, input + input_length); - m_transform->finish(buf); - send(buf); - } - -} diff --git a/src/lib/misc/pbes2/pbes2.h b/src/lib/misc/pbes2/pbes2.h index 90aa4f84b..ea2f9aa1d 100644 --- a/src/lib/misc/pbes2/pbes2.h +++ b/src/lib/misc/pbes2/pbes2.h @@ -8,8 +8,7 @@ #ifndef BOTAN_PBE_PKCS_v20_H__ #define BOTAN_PBE_PKCS_v20_H__ -#include <botan/secmem.h> -#include <botan/transform.h> +#include <botan/rng.h> #include <botan/alg_id.h> #include <chrono> diff --git a/src/lib/modes/aead/aead.cpp b/src/lib/modes/aead/aead.cpp index 3d04887d0..88e6cbeaa 100644 --- a/src/lib/modes/aead/aead.cpp +++ b/src/lib/modes/aead/aead.cpp @@ -41,8 +41,8 @@ BOTAN_REGISTER_BLOCK_CIPHER_MODE_LEN2(CCM_Encryption, CCM_Decryption, 16, 3); #endif #if defined(BOTAN_HAS_AEAD_CHACHA20_POLY1305) -BOTAN_REGISTER_TRANSFORM_NOARGS(ChaCha20Poly1305_Encryption); -BOTAN_REGISTER_TRANSFORM_NOARGS(ChaCha20Poly1305_Decryption); +BOTAN_REGISTER_T_NOARGS(Cipher_Mode, ChaCha20Poly1305_Encryption); +BOTAN_REGISTER_T_NOARGS(Cipher_Mode, ChaCha20Poly1305_Decryption); #endif #if defined(BOTAN_HAS_AEAD_EAX) diff --git a/src/lib/modes/cipher_mode.cpp b/src/lib/modes/cipher_mode.cpp index acd5e23e2..e7040772c 100644 --- a/src/lib/modes/cipher_mode.cpp +++ b/src/lib/modes/cipher_mode.cpp @@ -29,10 +29,13 @@ namespace Botan { +#define BOTAN_REGISTER_CIPHER_MODE(name, maker) BOTAN_REGISTER_T(Cipher_Mode, name, maker) +#define BOTAN_REGISTER_CIPHER_MODE_NOARGS(name) BOTAN_REGISTER_T_NOARGS(Cipher_Mode, name) + #if defined(BOTAN_HAS_MODE_ECB) template<typename T> -Transform* make_ecb_mode(const Transform::Spec& spec) +Cipher_Mode* make_ecb_mode(const Cipher_Mode::Spec& spec) { std::unique_ptr<BlockCipher> bc(BlockCipher::create(spec.arg(0))); std::unique_ptr<BlockCipherModePaddingMethod> pad(get_bc_pad(spec.arg(1, "NoPadding"))); @@ -41,14 +44,14 @@ Transform* make_ecb_mode(const Transform::Spec& spec) return nullptr; } -BOTAN_REGISTER_TRANSFORM(ECB_Encryption, make_ecb_mode<ECB_Encryption>); -BOTAN_REGISTER_TRANSFORM(ECB_Decryption, make_ecb_mode<ECB_Decryption>); +BOTAN_REGISTER_CIPHER_MODE(ECB_Encryption, make_ecb_mode<ECB_Encryption>); +BOTAN_REGISTER_CIPHER_MODE(ECB_Decryption, make_ecb_mode<ECB_Decryption>); #endif #if defined(BOTAN_HAS_MODE_CBC) template<typename CBC_T, typename CTS_T> -Transform* make_cbc_mode(const Transform::Spec& spec) +Cipher_Mode* make_cbc_mode(const Cipher_Mode::Spec& spec) { std::unique_ptr<BlockCipher> bc(BlockCipher::create(spec.arg(0))); @@ -65,8 +68,8 @@ Transform* make_cbc_mode(const Transform::Spec& spec) return nullptr; } -BOTAN_REGISTER_TRANSFORM(CBC_Encryption, (make_cbc_mode<CBC_Encryption,CTS_Encryption>)); -BOTAN_REGISTER_TRANSFORM(CBC_Decryption, (make_cbc_mode<CBC_Decryption,CTS_Decryption>)); +BOTAN_REGISTER_CIPHER_MODE(CBC_Encryption, (make_cbc_mode<CBC_Encryption,CTS_Encryption>)); +BOTAN_REGISTER_CIPHER_MODE(CBC_Decryption, (make_cbc_mode<CBC_Decryption,CTS_Decryption>)); #endif #if defined(BOTAN_HAS_MODE_CFB) @@ -83,14 +86,17 @@ Cipher_Mode* get_cipher_mode(const std::string& algo_spec, Cipher_Dir direction) const char* dir_string = (direction == ENCRYPTION) ? "_Encryption" : "_Decryption"; - std::unique_ptr<Transform> t; + Cipher_Mode::Spec spec(algo_spec, dir_string); - t.reset(get_transform(algo_spec, provider, dir_string)); + std::unique_ptr<Cipher_Mode> cipher_mode( + Algo_Registry<Cipher_Mode>::global_registry().make( + Cipher_Mode::Spec(algo_spec, dir_string), + provider) + ); - if(Cipher_Mode* cipher = dynamic_cast<Cipher_Mode*>(t.get())) + if(cipher_mode) { - t.release(); - return cipher; + return cipher_mode.release(); } const std::vector<std::string> algo_parts = split_on(algo_spec, '/'); @@ -115,24 +121,32 @@ Cipher_Mode* get_cipher_mode(const std::string& algo_spec, Cipher_Dir direction) const std::string mode_name = mode_info[0] + alg_args.str(); const std::string mode_name_directional = mode_info[0] + dir_string + alg_args.str(); - t.reset(get_transform(mode_name_directional, provider)); + cipher_mode.reset( + Algo_Registry<Cipher_Mode>::global_registry().make( + Cipher_Mode::Spec(mode_name_directional), + provider) + ); - if(Cipher_Mode* cipher = dynamic_cast<Cipher_Mode*>(t.get())) + if(cipher_mode) { - t.release(); - return cipher; + return cipher_mode.release(); } - t.reset(get_transform(mode_name, provider)); + cipher_mode.reset( + Algo_Registry<Cipher_Mode>::global_registry().make( + Cipher_Mode::Spec(mode_name), + provider) + ); - if(Cipher_Mode* cipher = dynamic_cast<Cipher_Mode*>(t.get())) + if(cipher_mode) { - t.release(); - return cipher; + return cipher_mode.release(); } if(auto sc = StreamCipher::create(mode_name, provider)) + { return new Stream_Cipher_Mode(sc.release()); + } return nullptr; } diff --git a/src/lib/modes/cipher_mode.h b/src/lib/modes/cipher_mode.h index 63821005d..73a5f7d96 100644 --- a/src/lib/modes/cipher_mode.h +++ b/src/lib/modes/cipher_mode.h @@ -8,17 +8,123 @@ #ifndef BOTAN_CIPHER_MODE_H__ #define BOTAN_CIPHER_MODE_H__ -#include <botan/transform.h> -#include <botan/stream_cipher.h> +#include <botan/secmem.h> +#include <botan/key_spec.h> +#include <botan/exceptn.h> +#include <botan/symkey.h> +#include <botan/scan_name.h> +#include <string> +#include <vector> namespace Botan { /** * Interface for cipher modes */ -class BOTAN_DLL Cipher_Mode : public Keyed_Transform +class BOTAN_DLL Cipher_Mode { public: + typedef SCAN_Name Spec; + + virtual ~Cipher_Mode() {} + + /** + * Begin processing a message. + * @param nonce the per message nonce + */ + template<typename Alloc> + secure_vector<byte> start(const std::vector<byte, Alloc>& nonce) + { + return start(nonce.data(), nonce.size()); + } + + /** + * Begin processing a message. + * @param nonce the per message nonce + */ + template<typename Alloc> + BOTAN_DEPRECATED("Use Transform::start") + secure_vector<byte> start_vec(const std::vector<byte, Alloc>& nonce) + { + return start(nonce.data(), nonce.size()); + } + + /** + * Begin processing a message. + * @param nonce the per message nonce + * @param nonce_len length of nonce + */ + secure_vector<byte> start(const byte nonce[], size_t nonce_len) + { + return start_raw(nonce, nonce_len); + } + + /** + * Begin processing a message. + */ + secure_vector<byte> start() + { + return start_raw(nullptr, 0); + } + + virtual secure_vector<byte> start_raw(const byte nonce[], size_t nonce_len) = 0; + + /** + * Process some data. Input must be in size update_granularity() byte blocks. + * @param blocks in/out parameter which will possibly be resized + * @param offset an offset into blocks to begin processing + */ + virtual void update(secure_vector<byte>& blocks, size_t offset = 0) = 0; + + /** + * Complete processing of a message. + * + * @param final_block in/out parameter which must be at least + * minimum_final_size() bytes, and will be set to any final output + * @param offset an offset into final_block to begin processing + */ + virtual void finish(secure_vector<byte>& final_block, size_t offset = 0) = 0; + + /** + * Returns the size of the output if this transform is used to process a + * message with input_length bytes. Will throw if unable to give a precise + * answer. + */ + virtual size_t output_length(size_t input_length) const = 0; + + /** + * @return size of required blocks to update + */ + virtual size_t update_granularity() const = 0; + + /** + * @return required minimium size to finalize() - may be any + * length larger than this. + */ + virtual size_t minimum_final_size() const = 0; + + /** + * Return the default size for a nonce + */ + virtual size_t default_nonce_length() const = 0; + + /** + * Return true iff nonce_len is a valid length for the nonce + */ + virtual bool valid_nonce_length(size_t nonce_len) const = 0; + + /** + * Return some short name describing the provider of this tranformation. + * Useful in cases where multiple implementations are available (eg, + * different implementations of AES). Default "core" is used for the + * 'standard' implementation included in the library. + */ + virtual std::string provider() const { return "core"; } + + virtual std::string name() const = 0; + + virtual void clear() = 0; + /** * Returns true iff this mode provides authentication as well as * confidentiality. @@ -29,6 +135,47 @@ class BOTAN_DLL Cipher_Mode : public Keyed_Transform * Return the size of the authentication tag used (in bytes) */ virtual size_t tag_size() const { return 0; } + + /** + * @return object describing limits on key size + */ + virtual Key_Length_Specification key_spec() const = 0; + + /** + * Check whether a given key length is valid for this algorithm. + * @param length the key length to be checked. + * @return true if the key length is valid. + */ + bool valid_keylength(size_t length) const + { + return key_spec().valid_keylength(length); + } + + template<typename Alloc> + void set_key(const std::vector<byte, Alloc>& key) + { + set_key(key.data(), key.size()); + } + + void set_key(const SymmetricKey& key) + { + set_key(key.begin(), key.length()); + } + + /** + * Set the symmetric key of this transform + * @param key contains the key material + * @param length in bytes of key param + */ + void set_key(const byte key[], size_t length) + { + if(!valid_keylength(length)) + throw Invalid_Key_Length(name(), length); + key_schedule(key, length); + } + + private: + virtual void key_schedule(const byte key[], size_t length) = 0; }; /** diff --git a/src/lib/modes/mode_utils.h b/src/lib/modes/mode_utils.h index 44524c4f6..3e4b7370b 100644 --- a/src/lib/modes/mode_utils.h +++ b/src/lib/modes/mode_utils.h @@ -18,7 +18,7 @@ namespace Botan { template<typename T> -T* make_block_cipher_mode(const Transform::Spec& spec) +T* make_block_cipher_mode(const Cipher_Mode::Spec& spec) { if(std::unique_ptr<BlockCipher> bc = BlockCipher::create(spec.arg(0))) return new T(bc.release()); @@ -26,7 +26,7 @@ T* make_block_cipher_mode(const Transform::Spec& spec) } template<typename T, size_t LEN1> -T* make_block_cipher_mode_len(const Transform::Spec& spec) +T* make_block_cipher_mode_len(const Cipher_Mode::Spec& spec) { if(std::unique_ptr<BlockCipher> bc = BlockCipher::create(spec.arg(0))) { @@ -38,7 +38,7 @@ T* make_block_cipher_mode_len(const Transform::Spec& spec) } template<typename T, size_t LEN1, size_t LEN2> -T* make_block_cipher_mode_len2(const Transform::Spec& spec) +T* make_block_cipher_mode_len2(const Cipher_Mode::Spec& spec) { if(std::unique_ptr<BlockCipher> bc = BlockCipher::create(spec.arg(0))) { @@ -51,16 +51,16 @@ T* make_block_cipher_mode_len2(const Transform::Spec& spec) } #define BOTAN_REGISTER_BLOCK_CIPHER_MODE(E, D) \ - BOTAN_REGISTER_NAMED_T(Transform, #E, E, make_block_cipher_mode<E>); \ - BOTAN_REGISTER_NAMED_T(Transform, #D, D, make_block_cipher_mode<D>) + BOTAN_REGISTER_NAMED_T(Cipher_Mode, #E, E, make_block_cipher_mode<E>); \ + BOTAN_REGISTER_NAMED_T(Cipher_Mode, #D, D, make_block_cipher_mode<D>) #define BOTAN_REGISTER_BLOCK_CIPHER_MODE_LEN(E, D, LEN) \ - BOTAN_REGISTER_NAMED_T(Transform, #E, E, (make_block_cipher_mode_len<E, LEN>)); \ - BOTAN_REGISTER_NAMED_T(Transform, #D, D, (make_block_cipher_mode_len<D, LEN>)) + BOTAN_REGISTER_NAMED_T(Cipher_Mode, #E, E, (make_block_cipher_mode_len<E, LEN>)); \ + BOTAN_REGISTER_NAMED_T(Cipher_Mode, #D, D, (make_block_cipher_mode_len<D, LEN>)) #define BOTAN_REGISTER_BLOCK_CIPHER_MODE_LEN2(E, D, LEN1, LEN2) \ - BOTAN_REGISTER_NAMED_T(Transform, #E, E, (make_block_cipher_mode_len2<E, LEN1, LEN2>)); \ - BOTAN_REGISTER_NAMED_T(Transform, #D, D, (make_block_cipher_mode_len2<D, LEN1, LEN2>)) + BOTAN_REGISTER_NAMED_T(Cipher_Mode, #E, E, (make_block_cipher_mode_len2<E, LEN1, LEN2>)); \ + BOTAN_REGISTER_NAMED_T(Cipher_Mode, #D, D, (make_block_cipher_mode_len2<D, LEN1, LEN2>)) } diff --git a/src/lib/stream/stream_cipher.h b/src/lib/stream/stream_cipher.h index 8c9b28147..bff1fd1a6 100644 --- a/src/lib/stream/stream_cipher.h +++ b/src/lib/stream/stream_cipher.h @@ -8,7 +8,6 @@ #ifndef BOTAN_STREAM_CIPHER_H__ #define BOTAN_STREAM_CIPHER_H__ -#include <botan/transform.h> #include <botan/sym_algo.h> #include <botan/scan_name.h> diff --git a/src/tests/test_compression.cpp b/src/tests/test_compression.cpp index fd247803a..d76daf30f 100644 --- a/src/tests/test_compression.cpp +++ b/src/tests/test_compression.cpp @@ -66,11 +66,10 @@ class Compression_Tests : public Test { Test::Result result(algo + " compression"); - std::unique_ptr<Botan::Compressor_Transform> c1(Botan::make_compressor(algo, 1)); - std::unique_ptr<Botan::Compressor_Transform> c9(Botan::make_compressor(algo, 9)); - std::unique_ptr<Botan::Compressor_Transform> d(Botan::make_decompressor(algo)); + std::unique_ptr<Botan::Compression_Algorithm> c(Botan::make_compressor(algo)); + std::unique_ptr<Botan::Decompression_Algorithm> d(Botan::make_decompressor(algo)); - if(!c1 || !c9 || !d) + if(!c || !d) { result.note_missing(algo); continue; @@ -83,14 +82,14 @@ class Compression_Tests : public Test const uint8_t* textb = reinterpret_cast<const uint8_t*>(text_str); const Botan::secure_vector<uint8_t> text(textb, textb + text_len); - const size_t c1_e = run_compression(result, *c1, *d, empty); - const size_t c9_e = run_compression(result, *c9, *d, empty); - const size_t c1_z = run_compression(result, *c1, *d, all_zeros); - const size_t c9_z = run_compression(result, *c9, *d, all_zeros); - const size_t c1_r = run_compression(result, *c1, *d, random_binary); - const size_t c9_r = run_compression(result, *c9, *d, random_binary); - const size_t c1_t = run_compression(result, *c1, *d, text); - const size_t c9_t = run_compression(result, *c9, *d, text); + const size_t c1_e = run_compression(result, 1, *c, *d, empty); + const size_t c9_e = run_compression(result, 9, *c, *d, empty); + const size_t c1_z = run_compression(result, 1, *c, *d, all_zeros); + const size_t c9_z = run_compression(result, 9, *c, *d, all_zeros); + const size_t c1_r = run_compression(result, 1, *c, *d, random_binary); + const size_t c9_r = run_compression(result, 9, *c, *d, random_binary); + const size_t c1_t = run_compression(result, 1, *c, *d, text); + const size_t c9_t = run_compression(result, 9, *c, *d, text); result.test_gte("Empty input L1 compresses to non-empty output", c1_e, 1); result.test_gte("Empty input L9 compresses to non-empty output", c9_e, 1); @@ -118,13 +117,14 @@ class Compression_Tests : public Test // Returns # of bytes of compressed message size_t run_compression(Test::Result& result, - Botan::Compressor_Transform& c, - Botan::Transform& d, + size_t level, + Botan::Compression_Algorithm& c, + Botan::Decompression_Algorithm& d, const Botan::secure_vector<uint8_t>& msg) { Botan::secure_vector<uint8_t> compressed = msg; - c.start(); + c.start(level); c.finish(compressed); const size_t c_size = compressed.size(); diff --git a/src/tests/test_entropy.cpp b/src/tests/test_entropy.cpp index 406dc7c41..42d2cf536 100644 --- a/src/tests/test_entropy.cpp +++ b/src/tests/test_entropy.cpp @@ -77,7 +77,7 @@ class Entropy_Source_Tests : public Test continue; } #endif - std::unique_ptr<Botan::Compressor_Transform> comp(Botan::make_compressor(comp_algo, 9)); + std::unique_ptr<Botan::Compression_Algorithm> comp(Botan::make_compressor(comp_algo)); if(comp) { @@ -87,7 +87,7 @@ class Entropy_Source_Tests : public Test { Botan::secure_vector<byte> compressed; compressed.assign(entropy.begin(), entropy.end()); - comp->start(); + comp->start(9); comp->finish(compressed); comp1_size = compressed.size(); |