forked from boostorg/unordered
Add Boost.Unordered and add to the documentation. Not fully integrated yet.
[SVN r42528]
This commit is contained in:
17
doc/Jamfile.v2
Normal file
17
doc/Jamfile.v2
Normal file
@@ -0,0 +1,17 @@
|
||||
|
||||
# Copyright 2005 Daniel James.
|
||||
# Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
# file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
|
||||
|
||||
using quickbook ;
|
||||
|
||||
xml unordered : unordered.qbk ;
|
||||
boostbook standalone : unordered :
|
||||
<xsl:param>boost.root=../../../..
|
||||
<xsl:param>boost.libraries=../../../libraries.htm
|
||||
<xsl:param>html.stylesheet=../../../../doc/html/boostbook.css
|
||||
<xsl:param>chunk.first.sections=1
|
||||
<xsl:param>chunk.section.depth=2
|
||||
<xsl:param>generate.section.toc.level=2
|
||||
<xsl:param>toc.section.depth=1
|
||||
<xsl:param>toc.max.depth=1 ;
|
||||
146
doc/buckets.qbk
Normal file
146
doc/buckets.qbk
Normal file
@@ -0,0 +1,146 @@
|
||||
[/ Copyright 2006-2007 Daniel James.
|
||||
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) ]
|
||||
|
||||
[section:buckets The Data Structure]
|
||||
|
||||
The containers are made up of a number of 'buckets', each of which can contain
|
||||
any number of elements. For example, the following diagram shows an [classref
|
||||
boost::unordered_set unordered_set] with 7 buckets containing 5 elements, `A`,
|
||||
`B`, `C`, `D` and `E` (this is just for illustration, containers will typically
|
||||
have more buckets).
|
||||
|
||||
[$../../libs/unordered/doc/diagrams/buckets.png]
|
||||
|
||||
In order to decide which bucket to place an element in, the container applies
|
||||
the hash function, `Hash`, to the element's key (for `unordered_set` and
|
||||
`unordered_multiset` the key is the whole element, but is referred to as the key
|
||||
so that the same terminology can be used for sets and maps). This returns a
|
||||
value of type `std::size_t`. `std::size_t` has a much greater range of values
|
||||
then the number of buckets, so that container applies another transformation to
|
||||
that value to choose a bucket to place the element in.
|
||||
|
||||
Retreiving the elements for a given key is simple. The same process is applied
|
||||
to the key to find the correct bucket. Then the key is compared with the
|
||||
elements in the bucket to find any elements that match (using the equality
|
||||
predicate `Pred`). If the hash function has worked well the elements will be
|
||||
evenly distributed amongst the buckets so only a small number of elements will
|
||||
need to be examined.
|
||||
|
||||
There is [link unordered.hash_equality more information on hash functions and
|
||||
equality predicates in the next section].
|
||||
|
||||
You can see in the diagram that `A` & `D` have been placed in the same bucket.
|
||||
When looking for elements in this bucket up to 2 comparisons are made, making
|
||||
the search slower. This is known as a collision. To keep things fast we try to
|
||||
keep collisions to a minimum.
|
||||
|
||||
[table Methods for Accessing Buckets
|
||||
[[Method] [Description]]
|
||||
|
||||
[
|
||||
[``size_type bucket_count() const``]
|
||||
[The number of buckets.]
|
||||
]
|
||||
[
|
||||
[``size_type max_bucket_count() const``]
|
||||
[An upper bound on the number of buckets.]
|
||||
]
|
||||
[
|
||||
[``size_type bucket_size(size_type n) const``]
|
||||
[The number of elements in bucket `n`.]
|
||||
]
|
||||
[
|
||||
[``size_type bucket(key_type const& k) const``]
|
||||
[Returns the index of the bucket which would contain k]
|
||||
]
|
||||
[
|
||||
[``
|
||||
local_iterator begin(size_type n);
|
||||
local_iterator end(size_type n);
|
||||
const_local_iterator begin(size_type n) const;
|
||||
const_local_iterator end(size_type n) const;
|
||||
const_local_iterator cbegin(size_type n) const;
|
||||
const_local_iterator cend(size_type n) const;
|
||||
``]
|
||||
[Return begin and end iterators for bucket `n`.]
|
||||
]
|
||||
]
|
||||
|
||||
[h2 Controlling the number of buckets]
|
||||
|
||||
As more elements are added to an unordered associative container, the number
|
||||
of elements in the buckets will increase causing performance to degrade.
|
||||
To combat this the containers increase the bucket count as elements are inserted.
|
||||
You can also tell the container to change the bucket count (if required) by
|
||||
calling `rehash`.
|
||||
|
||||
The standard leaves a lot of freedom to the implementor to decide how the
|
||||
number of buckets are chosen, but it does make some requirements based on the
|
||||
container's 'load factor', the average number of elements per bucket.
|
||||
Containers also have a 'maximum load factor' which they should try to keep the
|
||||
load factor below.
|
||||
|
||||
You can't control the bucket count directly but there are two ways to
|
||||
influence it:
|
||||
|
||||
* Specify the minimum number of buckets when constructing a container or
|
||||
when calling `rehash`.
|
||||
* Suggest a maximum load factor by calling `max_load_factor`.
|
||||
|
||||
`max_load_factor` doesn't let you set the maximum load factor yourself, it just
|
||||
lets you give a /hint/. And even then, the draft standard doesn't actually
|
||||
require the container to pay much attention to this value. The only time the
|
||||
load factor is /required/ to be less than the maximum is following a call to
|
||||
`rehash`. But most implementations will try to keep the number of elements
|
||||
below the max load factor, and set the maximum load factor to be the same as
|
||||
or close to the hint - unless your hint is unreasonably small or large.
|
||||
|
||||
[table Methods for Controlling Bucket Size
|
||||
[[Method] [Description]]
|
||||
|
||||
[
|
||||
[``float load_factor() const``]
|
||||
[The average number of elements per bucket.]
|
||||
]
|
||||
[
|
||||
[``float max_load_factor() const``]
|
||||
[Returns the current maximum load factor.]
|
||||
]
|
||||
[
|
||||
[``float max_load_factor(float z)``]
|
||||
[Changes the container's maximum load factor, using `z` as a hint.]
|
||||
]
|
||||
[
|
||||
[``void rehash(size_type n)``]
|
||||
[Changes the number of buckets so that there at least n buckets, and
|
||||
so that the load factor is less than the maximum load factor.]
|
||||
]
|
||||
|
||||
]
|
||||
|
||||
[h2 Iterator Invalidation]
|
||||
|
||||
It is not specified how member functions other than `rehash` affect
|
||||
the bucket count, although `insert` is only allowed to invalidate iterators
|
||||
when the insertion causes the load factor to be greater than or equal to the
|
||||
maximum load factor. For most implementations this means that insert will only
|
||||
change the number of buckets when this happens. While iterators can be
|
||||
invalidated by calls to `insert` and `rehash`, pointers and references to the
|
||||
container's elements are never invalidated.
|
||||
|
||||
In a similar manner to using `reserve` for `vector`s, it can be a good idea
|
||||
to call `rehash` before inserting a large number of elements. This will get
|
||||
the expensive rehashing out of the way and let you store iterators, safe in
|
||||
the knowledge that they won't be invalidated. If you are inserting `n`
|
||||
elements into container `x`, you could first call:
|
||||
|
||||
x.rehash((x.size() + n) / x.max_load_factor() + 1);
|
||||
|
||||
[blurb Note: `rehash`'s argument is the minimum number of buckets, not the
|
||||
number of elements, which is why the new size is divided by the maximum load factor. The
|
||||
`+ 1` guarantees there is no invalidation; without it, reallocation could occur
|
||||
if the number of bucket exactly divides the target size, since the container is
|
||||
allowed to rehash when the load factor is equal to the maximum load factor.]
|
||||
|
||||
[endsect]
|
||||
164
doc/comparison.qbk
Normal file
164
doc/comparison.qbk
Normal file
@@ -0,0 +1,164 @@
|
||||
[/ Copyright 2006-2007 Daniel James.
|
||||
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) ]
|
||||
|
||||
[section:comparison Comparison with Associative Containers]
|
||||
|
||||
[table Interface differences.
|
||||
[[Associative Containers] [Unordered Associative Containers]]
|
||||
|
||||
[
|
||||
[Parameterized by an ordering relation `Compare`]
|
||||
[Parameterized by a function object `Hash` and an equivalence relation
|
||||
`Pred`]
|
||||
]
|
||||
[
|
||||
[Keys can be compared using `key_compare` which is accessed by member function `key_comp()`,
|
||||
values can be compared using `value_compare` which is accessed by member function `value_comp()`.]
|
||||
[Keys can be hashed using `hasher` which is accessed by member function `hash_function()`,
|
||||
and checked for equality using `key_equal` which is accessed by member function `key_eq()`.
|
||||
There is no function object for compared or hashing values.]
|
||||
]
|
||||
[
|
||||
[Constructors have optional extra parameters for the comparison object.]
|
||||
[Constructors have optional extra parameters for the initial minimum
|
||||
number of buckets, a hash function and an equality object.]
|
||||
]
|
||||
|
||||
[
|
||||
[Keys `k1`, `k2` are considered equivalent if
|
||||
`!Compare(k1, k2) && !Compare(k2, k1)`]
|
||||
[Keys `k1`, `k2` are considered equivalent if `Pred(k1, k2)`]
|
||||
]
|
||||
[
|
||||
[Member function `lower_bound(k)` and `upper_bound(k)`]
|
||||
[No equivalent. Since the elements aren't ordered `lower_bound` and
|
||||
`upper_bound` would be meaningless.]
|
||||
]
|
||||
[
|
||||
[`equal_range(k)` returns an empty range at the position that k
|
||||
would be inserted if k isn't present in the container.]
|
||||
[`equal_range(k)` returns a range at the end of the container if
|
||||
k isn't present in the container. It can't return a positioned
|
||||
range as k could be inserted into multiple place. To find out the
|
||||
bucket that k would be inserted into use `bucket(k)`. But remember
|
||||
that an insert can cause the container to rehash - meaning that the
|
||||
element can be inserted into a different bucket.]
|
||||
]
|
||||
[
|
||||
[`iterator`, `const_iterator` are of the bidirectional category.]
|
||||
[`iterator`, `const_iterator` are of at least the forward category.]
|
||||
]
|
||||
[
|
||||
[Iterators, pointers and references to the container's elements are
|
||||
never invalidated.]
|
||||
[[link unordered.buckets.iterator_invalidation Iterators can
|
||||
be invalidated by calls to insert or rehash]. Pointers and
|
||||
references to the container's elements are never invalidated.]
|
||||
]
|
||||
[
|
||||
[Iterators iterate through the container in the order defined by
|
||||
the comparison object.]
|
||||
[Iterators iterate through the container in an arbitrary order, that
|
||||
can change as elements are inserted. Although, equivalent elements
|
||||
are always adjacent.]
|
||||
]
|
||||
[
|
||||
[No equivalent]
|
||||
[Local iterators can be used to iterate through individual buckets.
|
||||
(I don't think that the order of local iterators and iterators are
|
||||
required to have any correspondence.)]
|
||||
]
|
||||
[
|
||||
[Can be compared using the `==`, `!=`, `<`, `<=`, `>`, `>=` operators.]
|
||||
[No comparison operators are defined in the standard, although
|
||||
[link unordered.rationale.equality_operator
|
||||
implementations might extend the containers to support `==` and
|
||||
`!=`].]
|
||||
]
|
||||
[
|
||||
[]
|
||||
[When inserting with a hint, implementations are permitted to ignore
|
||||
the hint.]
|
||||
]
|
||||
[
|
||||
[`erase` never throws an exception]
|
||||
[The containers hash or predicate function can throw exceptions
|
||||
from `erase`]
|
||||
]
|
||||
]
|
||||
|
||||
[table Complexity Guarantees
|
||||
[[Operation] [Associative Containers] [Unordered Associative Containers]]
|
||||
[
|
||||
[Construction of empty container]
|
||||
[constant]
|
||||
[O(/n/) where /n/ is the minimum number of buckets.]
|
||||
]
|
||||
[
|
||||
[Construction of container from a range of /N/ elements]
|
||||
[O(/N/ log /N/), O(/N/) if the range is sorted with `value_comp()`]
|
||||
[Average case O(/N/), worst case
|
||||
O(/N/'''<superscript>2</superscript>''')]
|
||||
]
|
||||
[
|
||||
[Insert a single element]
|
||||
[logarithmic]
|
||||
[Average case constant, worst case linear]
|
||||
]
|
||||
[
|
||||
[Insert a single element with a hint]
|
||||
[Amortized constant if t elements inserted right after hint,
|
||||
logarithmic otherwise]
|
||||
[Average case constant, worst case linear (ie. the same as
|
||||
a normal insert).]
|
||||
]
|
||||
[
|
||||
[Inserting a range of /N/ elements]
|
||||
[ /N/ log(`size()`+/N/) ]
|
||||
[Average case O(/N/), worst case O(/N/ * `size()`)]
|
||||
]
|
||||
[
|
||||
[Erase by key, `k`]
|
||||
[O(log(`size()`) + `count(k)`)]
|
||||
[Average case: O(`count(k)`), Worst case: O(`size()`)]
|
||||
]
|
||||
[
|
||||
[Erase a single element by iterator]
|
||||
[Amortized constant]
|
||||
[Average case: O(1), Worst case: O(`size()`)]
|
||||
]
|
||||
[
|
||||
[Erase a range of /N/ elements]
|
||||
[O(log(`size()`) + /N/)]
|
||||
[Average case: O(/N/), Worst case: O(`size()`)]
|
||||
]
|
||||
[
|
||||
[Clearing the container]
|
||||
[O(`size()`)]
|
||||
[O(`size()`)]
|
||||
]
|
||||
[
|
||||
[Find]
|
||||
[logarithmic]
|
||||
[Average case: O(1), Worst case: O(`size()`)]
|
||||
]
|
||||
[/ TODO: Average case is probably wrong. ]
|
||||
[
|
||||
[Count]
|
||||
[O(log(`size()`) + `count(k)`)]
|
||||
[Average case: O(1), Worst case: O(`size()`)]
|
||||
]
|
||||
[
|
||||
[`equal_range(k)`]
|
||||
[logarithmic]
|
||||
[Average case: O(`count(k)`), Worst case: O(`size()`)]
|
||||
]
|
||||
[
|
||||
[`lower_bound`,`upper_bound`]
|
||||
[logarithmic]
|
||||
[n/a]
|
||||
]
|
||||
]
|
||||
|
||||
[endsect]
|
||||
BIN
doc/diagrams/buckets.dia
Normal file
BIN
doc/diagrams/buckets.dia
Normal file
Binary file not shown.
BIN
doc/diagrams/buckets.png
Normal file
BIN
doc/diagrams/buckets.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
71
doc/hash_equality.qbk
Normal file
71
doc/hash_equality.qbk
Normal file
@@ -0,0 +1,71 @@
|
||||
[/ Copyright 2006-2007 Daniel James.
|
||||
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) ]
|
||||
|
||||
[section:hash_equality Equality Predicates and Hash Functions]
|
||||
|
||||
While the associative containers use an ordering relation to specify how the
|
||||
elements are stored, the unordered associative containers use an equality
|
||||
predicate and a hash function. For example, [classref boost::unordered_set]
|
||||
is declared as:
|
||||
|
||||
template<typename Value,
|
||||
typename Hash = ``[classref boost::hash]``<Value>,
|
||||
typename Pred = std::equal_to<Value>,
|
||||
typename Alloc = std::allocator<Value> >
|
||||
class ``[classref boost::unordered_set unordered_set]``;
|
||||
|
||||
The hash function comes first as you might want to change the hash function
|
||||
but not the equality predicate, while if you were to change the behavior
|
||||
of the equality predicate you would have to change the hash function to match
|
||||
it. So, if you wanted to use the
|
||||
[@http://www.isthe.com/chongo/tech/comp/fnv/ FNV-1 hash] you could write:
|
||||
|
||||
``[classref boost::unordered_set]``<std::string, hash::fnv_1> words;
|
||||
|
||||
An example implementation of FNV-1, and some other hash functions are supplied
|
||||
in the examples directory.
|
||||
|
||||
Alternatively, you might wish to use a different equality function. If you do
|
||||
this you will need to use a hash function that matches it. So to implement a
|
||||
case-insensitive dictionary:
|
||||
|
||||
[import src_code/insensitive.cpp]
|
||||
[case_insensitive_functions]
|
||||
[case_insensitive_dictionary]
|
||||
|
||||
This is a simplified version of the example at
|
||||
[@../../libs/unordered/examples/case_insensitive.hpp /libs/unordered/examples/case_insensitive.hpp]
|
||||
which supports other locales and string types.
|
||||
|
||||
[h2 Custom Types]
|
||||
|
||||
Similarly, a custom hash function can be used for custom types:
|
||||
|
||||
[import src_code/point1.cpp]
|
||||
[point_example1]
|
||||
|
||||
Although, [link hash.custom extending boost::hash to support the type] is
|
||||
probably a better solution:
|
||||
|
||||
[import src_code/point2.cpp]
|
||||
[point_example2]
|
||||
|
||||
See the [link hash.custom Boost.Hash documentation] for more detail on how to
|
||||
do this. Remember that it relies on extensions to the draft standard - so it
|
||||
won't work on other implementations of the unordered associative containers.
|
||||
|
||||
[table Methods for accessing the hash and equality functions.
|
||||
[[Method] [Description]]
|
||||
|
||||
[
|
||||
[``hasher hash_function() const``]
|
||||
[Returns the container's hash function.]
|
||||
]
|
||||
[
|
||||
[``key_equal key_eq() const``]
|
||||
[Returns the container's key equality function.]
|
||||
]
|
||||
]
|
||||
|
||||
[endsect]
|
||||
128
doc/intro.qbk
Normal file
128
doc/intro.qbk
Normal file
@@ -0,0 +1,128 @@
|
||||
[/ Copyright 2006-2007 Daniel James.
|
||||
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) ]
|
||||
|
||||
[def __tr1__
|
||||
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1836.pdf
|
||||
C++ Standard Library Technical Report]]
|
||||
[def __boost-tr1__
|
||||
[@http://www.boost.org/doc/html/boost_tr1.html
|
||||
Boost.TR1]]
|
||||
[def __draft__
|
||||
[@http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2007/n2461.pdf
|
||||
Working Draft of the C++ Standard]]
|
||||
[def __hash-table__ [@http://en.wikipedia.org/wiki/Hash_table
|
||||
hash table]]
|
||||
[def __hash-function__ [@http://en.wikipedia.org/wiki/Hash_function
|
||||
hash function]]
|
||||
|
||||
[section:intro Introduction]
|
||||
|
||||
For accessing data based on key lookup, the C++ standard library offers `std::set`,
|
||||
`std::map`, `std::multiset` and `std::multimap`. These are generally
|
||||
implemented using balanced binary trees so that lookup time has
|
||||
logarithmic complexity. That is generally okay, but in many cases a
|
||||
__hash-table__ can perform better, as accessing data has constant complexity,
|
||||
on average. The worst case complexity is linear, but that occurs rarely and
|
||||
with some care, can be avoided.
|
||||
|
||||
Also, the existing containers require a 'less than' comparison object
|
||||
to order their elements. For some data types this is impossible to implement
|
||||
or isn't practical. In contrast, a hash table only needs an equality function
|
||||
and a hash function for the key.
|
||||
|
||||
With this in mind, the __tr1__ introduced the unordered associative containers,
|
||||
which are implemented using hash tables, and they have now been added to the
|
||||
__draft__.
|
||||
|
||||
This library supplies an almost complete implementation of the specification in
|
||||
the __draft__, (it doesn't support `emplace` yet, see the [link
|
||||
unordered.rationale.future_developments Implementation Rationale] section for more
|
||||
details). If accepted the containers should also be added to __boost-tr1__.
|
||||
|
||||
`unordered_set` and `unordered_multiset` are defined in the header
|
||||
<[headerref boost/unordered_set.hpp]>
|
||||
|
||||
namespace boost {
|
||||
template <
|
||||
class Key,
|
||||
class Hash = ``[classref boost::hash]``<Key>,
|
||||
class Pred = std::equal_to<Key>,
|
||||
class Alloc = std::allocator<Key> >
|
||||
class ``[classref boost::unordered_set unordered_set]``;
|
||||
|
||||
template<
|
||||
class Key,
|
||||
class Hash = ``[classref boost::hash]``<Key>,
|
||||
class Pred = std::equal_to<Key>,
|
||||
class Alloc = std::allocator<Key> >
|
||||
class ``[classref boost::unordered_multiset unordered_multiset]``;
|
||||
}
|
||||
|
||||
`unordered_map` and `unordered_multimap` are defined in the header
|
||||
<[headerref boost/unordered_map.hpp]>
|
||||
|
||||
namespace boost {
|
||||
template <
|
||||
class Key, class Mapped,
|
||||
class Hash = ``[classref boost::hash]``<Key>,
|
||||
class Pred = std::equal_to<Key>,
|
||||
class Alloc = std::allocator<Key> >
|
||||
class ``[classref boost::unordered_map unordered_map]``;
|
||||
|
||||
template<
|
||||
class Key, class Mapped,
|
||||
class Hash = ``[classref boost::hash]``<Key>,
|
||||
class Pred = std::equal_to<Key>,
|
||||
class Alloc = std::allocator<Key> >
|
||||
class ``[classref boost::unordered_multimap unordered_multimap]``;
|
||||
}
|
||||
|
||||
When using Boost.TR1, these classes are included from `<unordered_set>` and
|
||||
`<unordered_map>`, with the classes added to the `std::tr1` namespace.
|
||||
|
||||
The containers are used in a similar manner to the normal associative
|
||||
containers:
|
||||
|
||||
#include <``[headerref boost/unordered_map.hpp]``>
|
||||
#include <cassert>
|
||||
|
||||
int main()
|
||||
{
|
||||
boost::unordered_map<std::string, int> x;
|
||||
x["one"] = 1;
|
||||
x["two"] = 2;
|
||||
x["three"] = 3;
|
||||
|
||||
assert(x["one"] == 1);
|
||||
assert(x["missing"] == 0);
|
||||
}
|
||||
|
||||
But since the elements aren't ordered, the output of:
|
||||
|
||||
BOOST_FOREACH(map::value_type i, x) {
|
||||
std::cout<<i.first<<","<<i.second<<"\n";
|
||||
}
|
||||
|
||||
can be in any order. For example, it might be:
|
||||
|
||||
two,2
|
||||
one,1
|
||||
three,3
|
||||
missing,0
|
||||
|
||||
To store an object in an unordered associative container requires both an
|
||||
key equality function and a hash function. The default function objects in
|
||||
the standard containers support a few basic types including integer types,
|
||||
floating point types, pointer types, and the standard strings. Since
|
||||
Boost.Unordered uses [classref boost::hash] it also supports some other types,
|
||||
including standard containers. To use any types not supported by these methods
|
||||
you have to [link hash.custom extend Boost.Hash to support the type] or use
|
||||
your own custom equality predicates and hash functions. See the
|
||||
[link unordered.hash_equality Equality Predicates and Hash Functions] section
|
||||
for more details.
|
||||
|
||||
There are other differences, which are listed in the
|
||||
[link unordered.comparison Comparison with Associative Containers] section.
|
||||
|
||||
[endsect]
|
||||
212
doc/rationale.qbk
Normal file
212
doc/rationale.qbk
Normal file
@@ -0,0 +1,212 @@
|
||||
[/ Copyright 2006-2007 Daniel James.
|
||||
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) ]
|
||||
|
||||
[def __wang__
|
||||
[@http://www.concentric.net/~Ttwang/tech/inthash.htm
|
||||
Thomas Wang's article on integer hash functions]]
|
||||
[def __n2345__
|
||||
[@http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2007/n2345.pdf
|
||||
N2345, 'Placement Insert for Containers']]
|
||||
[def __n2369__
|
||||
[@http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2007/n2369.pdf
|
||||
the August 2008 version of the working draft standard]]
|
||||
|
||||
[section:rationale Implementation Rationale]
|
||||
|
||||
The intent of this library is to implement the unordered
|
||||
containers in the draft standard, so the interface was fixed. But there are
|
||||
still some implementation decisions to make. The priorities are
|
||||
conformance to the standard and portability.
|
||||
|
||||
The [@http://en.wikipedia.org/wiki/Hash_table wikipedia article on hash tables]
|
||||
has a good summary of the implementation issues for hash tables in general.
|
||||
|
||||
[h2 Data Structure]
|
||||
|
||||
By specifying an interface for accessing the buckets of the container the
|
||||
standard pretty much requires that the hash table uses chained addressing.
|
||||
|
||||
It would be conceivable to write a hash table that uses another method. For
|
||||
example, it could use open addressing, and use the lookup chain to act as a
|
||||
bucket but there are a some serious problems with this:
|
||||
|
||||
* The draft standard requires that pointers to elements aren't invalidated, so
|
||||
the elements can't be stored in one array, but will need a layer of
|
||||
indirection instead - losing the efficiency and most of the memory gain,
|
||||
the main advantages of open addressing.
|
||||
|
||||
* Local iterators would be very inefficient and may not be able to
|
||||
meet the complexity requirements.
|
||||
|
||||
* There are also the restrictions on when iterators can be invalidated. Since
|
||||
open addressing degrades badly when there are a high number of collisions the
|
||||
restrictions could prevent a rehash when it's really needed. The maximum load
|
||||
factor could be set to a fairly low value to work around this - but the
|
||||
standard requires that it is initially set to 1.0.
|
||||
|
||||
* And since the standard is written with a eye towards chained
|
||||
addressing, users will be surprised if the performance doesn't reflect that.
|
||||
|
||||
So chained addressing is used.
|
||||
|
||||
For containers with unique keys I store the buckets in a single-linked list.
|
||||
There are other possible data structures (such as a double-linked list)
|
||||
that allow for some operations to be faster (such as erasing and iteration)
|
||||
but the possible gain seems small compared to the extra memory needed.
|
||||
The most commonly used operations (insertion and lookup) would not be improved
|
||||
at all.
|
||||
|
||||
But for containers with equivalent keys a single-linked list can degrade badly
|
||||
when a large number of elements with equivalent keys are inserted. I think it's
|
||||
reasonable to assume that users who choose to use `unordered_multiset` or
|
||||
`unordered_multimap` do so because they are likely to insert elements with
|
||||
equivalent keys. So I have used an alternative data structure that doesn't
|
||||
degrade, at the expense of an extra pointer per node.
|
||||
|
||||
This works by adding storing a circular linked list for each group of equivalent
|
||||
nodes in reverse order. This allows quick navigation to the end of a group (since
|
||||
the first element points to the last) and can be quickly updated when elements
|
||||
are inserted or erased. The main disadvantage of this approach is some hairy code
|
||||
for erasing elements.
|
||||
|
||||
[h2 Number of Buckets]
|
||||
|
||||
There are two popular methods for choosing the number of buckets in a hash
|
||||
table. One is to have a prime number of buckets, another is to use a power
|
||||
of 2.
|
||||
|
||||
Using a prime number of buckets, and choosing a bucket by using the modulus
|
||||
of the hash function's result will usually give a good result. The downside
|
||||
is that the required modulus operation is fairly expensive.
|
||||
|
||||
Using a power of 2 allows for much quicker selection of the bucket
|
||||
to use, but at the expense of loosing the upper bits of the hash value.
|
||||
For some specially designed hash functions it is possible to do this and
|
||||
still get a good result but as the containers can take arbitrary hash
|
||||
functions this can't be relied on.
|
||||
|
||||
To avoid this a transformation could be applied to the hash function, for an
|
||||
example see __wang__. Unfortunately, a transformation like Wang's requires
|
||||
knowledge of the number of bits in the hash value, so it isn't portable enough.
|
||||
This leaves more expensive methods, such as Knuth's Multiplicative Method
|
||||
(mentioned in Wang's article). These don't tend to work as well as taking the
|
||||
modulus of a prime, and the extra computation required might negate
|
||||
efficiency advantage of power of 2 hash tables.
|
||||
|
||||
So, this implementation uses a prime number for the hash table size.
|
||||
|
||||
[h2 Active Issues and Proposals]
|
||||
|
||||
[h3 Removing unused allocator functions]
|
||||
|
||||
In
|
||||
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2257.html
|
||||
N2257, removing unused allocator functions],
|
||||
Matt Austern suggests removing the `construct`, `destroy` and `address` member
|
||||
functions - all of which Boost.Unordered calls. Changing this will simplify the
|
||||
implementation, as well as make supporting `emplace` easier, but means that the
|
||||
containers won't support allocators which require these methods to be called.
|
||||
Detlef Vollmann opposed this change in
|
||||
[@http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2007/n2339.htm N2339].
|
||||
|
||||
[h3 Swapping containers with unequal allocators]
|
||||
|
||||
It isn't clear how to swap containers when their allocators aren't equal.
|
||||
This is
|
||||
[@http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html#431
|
||||
Issue 431: Swapping containers with unequal allocators].
|
||||
|
||||
Howard Hinnant wrote about this in
|
||||
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1599.html N1599]
|
||||
and suggested swapping both the allocators and the containers' contents.
|
||||
But the committee have now decided that `swap` should do a fast swap if the
|
||||
allocator is Swappable and a slow swap using copy construction otherwise. To
|
||||
make this distinction requires concepts.
|
||||
|
||||
In
|
||||
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2387.pdf
|
||||
N2387, Omnibus Allocator Fix-up Proposals],
|
||||
Pablo Halpern suggests that there are actually two distinct allocator models,
|
||||
"Moves with Value" and "Scoped" which behave differently:
|
||||
|
||||
[:
|
||||
When allocators are allowed to have state, it is necessary to have a model for
|
||||
determining from where an object obtains its allocator. We’ve identified two such
|
||||
models: the “Moves with Value” allocator model and the “Scoped” allocator model.
|
||||
|
||||
In the “Moves with Value” allocator model, the copy constructor of an allocator-aware
|
||||
class will copy both the value and the allocator from its argument. This is the model
|
||||
specified in the C++03 standard. With this model, inserting an object into a container
|
||||
usually causes the new container item to copy the allocator from the object that was
|
||||
inserted. This model can be useful in special circumstances, e.g., if the items within a
|
||||
container use an allocator that is specially tuned to the item’s type.
|
||||
|
||||
In the “Scoped” allocator model, the allocator used to construct an object is determined
|
||||
by the context of that object, much like a storage class. With this model, inserting an
|
||||
object into a container causes the new container item to use the same allocator as the
|
||||
container. To avoid allocators being used in the wrong context, the allocator is never
|
||||
copied during copy or move construction. Thus, it is possible using this model to use
|
||||
allocators based on short-lived resources without fear that an object will transfer its
|
||||
allocator to a copy that might outlive the (shared) allocator resource. This model is
|
||||
reasonably safe and generally useful on a large scale. There was strong support in the
|
||||
2005 Tremblant meeting for pursuing an allocator model that propagates allocators
|
||||
from container to contained objects.
|
||||
]
|
||||
|
||||
With these models the choice becomes clearer:
|
||||
|
||||
[:
|
||||
I introduced the “Moves with Value” allocator model and the
|
||||
“Scoped” allocator model. In the former case, the allocator is copied when the container
|
||||
is copy-constructed. In the latter case it is not. Swapping the allocators is the right thing
|
||||
to do if the containers conform to the “Moves with Value” allocator model and
|
||||
absolutely the wrong thing to do if the containers conform to the “Scoped” allocator
|
||||
model. With the two allocator models well-defined, the desired behavior becomes clear.
|
||||
]
|
||||
|
||||
The proposal is that allocators are swapped if the allocator follows the
|
||||
"Moves with Value" model and the allocator is swappable. Otherwise a slow swap
|
||||
is used. Since containers currently only support the "Moves with Value" model
|
||||
this is consistent with the committee's current recommendation (although it
|
||||
suggests using a trait to detect if the allocator is swappable rather than a
|
||||
concept).
|
||||
|
||||
Since there is currently neither have a swappable trait or concept for
|
||||
allocators this implementation always performs a slow swap.
|
||||
|
||||
[h3 Are insert and erase stable for unordered_multiset and unordered_multimap?]
|
||||
|
||||
It is not specified if `unordered_multiset` and `unordered_multimap` preserve the order
|
||||
of elements with equivalent keys (i.e. if they're stable under `insert` and `erase`).
|
||||
This is [@http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html#518 issue 581].
|
||||
The current proposal is that insert, erase and rehash are stable - so they are here.
|
||||
|
||||
[h3 const_local_iterator cbegin, cend missing from TR1]
|
||||
|
||||
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2482.html#691
|
||||
Issue 691] is that `cbegin` and `cend` are missing for local iterators.
|
||||
The current resolution is that they'll be added, so I've added them.
|
||||
|
||||
[h2 Future Developments]
|
||||
|
||||
[h3 Support for `emplace`]
|
||||
|
||||
In __n2369__ a new member function, `emplace` was added to the containers to
|
||||
allow placement insert, as described in __n2345__. To fully implement this
|
||||
`std::forward` is required, along with new functions in `std::allocator` and
|
||||
new constructors in `std::pair`. But partial support is possible - especially
|
||||
if I don't use the `construct` member of allocators.
|
||||
|
||||
[h3 Equality operator]
|
||||
|
||||
While `operator==` and `operator!=` are not included in the standard, it's
|
||||
possible to implement them for all the containers - this is helped by having
|
||||
stable order of elements with equivalent keys. They will need to be specified
|
||||
differently to the standard associative containers, probably comparing keys
|
||||
using the equality predicate rather than `operator==`. This is inconsistent
|
||||
with the other containers but it is probably closer to user's expectations.
|
||||
|
||||
If these are added then a `hash_value` free function should also be added.
|
||||
|
||||
[endsect]
|
||||
2537
doc/ref.xml
Normal file
2537
doc/ref.xml
Normal file
File diff suppressed because it is too large
Load Diff
79
doc/src_code/insensitive.cpp
Normal file
79
doc/src_code/insensitive.cpp
Normal file
@@ -0,0 +1,79 @@
|
||||
|
||||
// Copyright 2006-2007 Daniel James.
|
||||
// Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
|
||||
|
||||
#include <boost/unordered_map.hpp>
|
||||
#include <boost/detail/lightweight_test.hpp>
|
||||
#include <boost/algorithm/string/predicate.hpp>
|
||||
|
||||
//[case_insensitive_functions
|
||||
struct iequal_to
|
||||
: std::binary_function<std::string, std::string, bool>
|
||||
{
|
||||
bool operator()(std::string const& x,
|
||||
std::string const& y) const
|
||||
{
|
||||
return boost::algorithm::iequals(x, y, std::locale());
|
||||
}
|
||||
};
|
||||
|
||||
struct ihash
|
||||
: std::unary_function<std::string, std::size_t>
|
||||
{
|
||||
std::size_t operator()(std::string const& x) const
|
||||
{
|
||||
std::size_t seed = 0;
|
||||
std::locale locale;
|
||||
|
||||
for(std::string::const_iterator it = x.begin();
|
||||
it != x.end(); ++it)
|
||||
{
|
||||
boost::hash_combine(seed, std::toupper(*it, locale));
|
||||
}
|
||||
|
||||
return seed;
|
||||
}
|
||||
};
|
||||
|
||||
struct word_info;
|
||||
//]
|
||||
|
||||
struct word_info {
|
||||
int tag;
|
||||
explicit word_info(int t = 0) : tag(t) {}
|
||||
};
|
||||
|
||||
int main() {
|
||||
//[case_insensitive_dictionary
|
||||
boost::unordered_map<std::string, word_info, ihash, iequal_to>
|
||||
idictionary;
|
||||
//]
|
||||
|
||||
BOOST_TEST(idictionary.empty());
|
||||
|
||||
idictionary["one"] = word_info(1);
|
||||
BOOST_TEST(idictionary.size() == 1);
|
||||
BOOST_TEST(idictionary.find("ONE") != idictionary.end() &&
|
||||
idictionary.find("ONE") == idictionary.find("one"));
|
||||
|
||||
idictionary.insert(std::make_pair("ONE", word_info(2)));
|
||||
BOOST_TEST(idictionary.size() == 1);
|
||||
BOOST_TEST(idictionary.find("ONE") != idictionary.end() &&
|
||||
idictionary.find("ONE")->first == "one" &&
|
||||
idictionary.find("ONE")->second.tag == 1);
|
||||
|
||||
idictionary["One"] = word_info(3);
|
||||
BOOST_TEST(idictionary.size() == 1);
|
||||
BOOST_TEST(idictionary.find("ONE") != idictionary.end() &&
|
||||
idictionary.find("ONE")->first == "one" &&
|
||||
idictionary.find("ONE")->second.tag == 3);
|
||||
|
||||
idictionary["two"] = word_info(4);
|
||||
BOOST_TEST(idictionary.size() == 2);
|
||||
BOOST_TEST(idictionary.find("two") != idictionary.end() &&
|
||||
idictionary.find("TWO")->first == "two" &&
|
||||
idictionary.find("Two")->second.tag == 4);
|
||||
|
||||
return boost::report_errors();
|
||||
}
|
||||
46
doc/src_code/point1.cpp
Normal file
46
doc/src_code/point1.cpp
Normal file
@@ -0,0 +1,46 @@
|
||||
|
||||
// Copyright 2006-2007 Daniel James.
|
||||
// Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
|
||||
|
||||
#include <boost/unordered_set.hpp>
|
||||
#include <boost/detail/lightweight_test.hpp>
|
||||
|
||||
//[point_example1
|
||||
struct point {
|
||||
int x;
|
||||
int y;
|
||||
};
|
||||
|
||||
bool operator==(point const& p1, point const& p2)
|
||||
{
|
||||
return p1.x == p2.x && p1.y == p2.y;
|
||||
}
|
||||
|
||||
struct point_hash
|
||||
: std::unary_function<point, std::size_t>
|
||||
{
|
||||
std::size_t operator()(point const& p) const
|
||||
{
|
||||
std::size_t seed = 0;
|
||||
boost::hash_combine(seed, p.x);
|
||||
boost::hash_combine(seed, p.y);
|
||||
return seed;
|
||||
}
|
||||
};
|
||||
|
||||
boost::unordered_multiset<point, point_hash, std::equal_to<point> >
|
||||
points;
|
||||
//]
|
||||
|
||||
int main() {
|
||||
point x[] = {{1,2}, {3,4}, {1,5}, {1,2}};
|
||||
for(int i = 0; i < sizeof(x) / sizeof(point); ++i)
|
||||
points.insert(x[i]);
|
||||
BOOST_TEST(points.count(x[0]) == 2);
|
||||
BOOST_TEST(points.count(x[1]) == 1);
|
||||
point y = {10, 2};
|
||||
BOOST_TEST(points.count(y) == 0);
|
||||
|
||||
return boost::report_errors();
|
||||
}
|
||||
43
doc/src_code/point2.cpp
Normal file
43
doc/src_code/point2.cpp
Normal file
@@ -0,0 +1,43 @@
|
||||
|
||||
// Copyright 2006-2007 Daniel James.
|
||||
// Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
|
||||
|
||||
#include <boost/unordered_set.hpp>
|
||||
#include <boost/functional/hash.hpp>
|
||||
#include <boost/detail/lightweight_test.hpp>
|
||||
|
||||
//[point_example2
|
||||
struct point {
|
||||
int x;
|
||||
int y;
|
||||
};
|
||||
|
||||
bool operator==(point const& p1, point const& p2)
|
||||
{
|
||||
return p1.x == p2.x && p1.y == p2.y;
|
||||
}
|
||||
|
||||
std::size_t hash_value(point const& p) {
|
||||
std::size_t seed = 0;
|
||||
boost::hash_combine(seed, p.x);
|
||||
boost::hash_combine(seed, p.y);
|
||||
return seed;
|
||||
}
|
||||
|
||||
// Now the default function objects work.
|
||||
boost::unordered_multiset<point> points;
|
||||
//]
|
||||
|
||||
int main() {
|
||||
point x[] = {{1,2}, {3,4}, {1,5}, {1,2}};
|
||||
for(int i = 0; i < sizeof(x) / sizeof(point); ++i)
|
||||
points.insert(x[i]);
|
||||
BOOST_TEST(points.count(x[0]) == 2);
|
||||
BOOST_TEST(points.count(x[1]) == 1);
|
||||
point y = {10, 2};
|
||||
BOOST_TEST(points.count(y) == 0);
|
||||
|
||||
return boost::report_errors();
|
||||
}
|
||||
|
||||
26
doc/unordered.qbk
Normal file
26
doc/unordered.qbk
Normal file
@@ -0,0 +1,26 @@
|
||||
[/ Copyright 2006-2007 Daniel James.
|
||||
/ Distributed under the Boost Software License, Version 1.0. (See accompanying
|
||||
/ file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) ]
|
||||
|
||||
[library Unordered Associative Containers
|
||||
[quickbook 1.4]
|
||||
[authors [James, Daniel]]
|
||||
[copyright 2003 2004 Jeremy B. Maitin-Shepard]
|
||||
[copyright 2005 2006 2007 Daniel James]
|
||||
[purpose std::tr1 compliant hash containers]
|
||||
[id unordered]
|
||||
[dirname unordered]
|
||||
[license
|
||||
Distributed under the Boost Software License, Version 1.0.
|
||||
(See accompanying file LICENSE_1_0.txt or copy at
|
||||
[@http://www.boost.org/LICENSE_1_0.txt]
|
||||
]
|
||||
]
|
||||
|
||||
[include:unordered intro.qbk]
|
||||
[include:unordered buckets.qbk]
|
||||
[include:unordered hash_equality.qbk]
|
||||
[include:unordered comparison.qbk]
|
||||
[include:unordered rationale.qbk]
|
||||
|
||||
[xinclude ref.xml]
|
||||
Reference in New Issue
Block a user