contrib.lookup.IdTableWithHashBuckets
tf.contrib.lookup.IdTableWithHashBuckets
class tf.contrib.lookup.IdTableWithHashBuckets
Defined in tensorflow/contrib/lookup/lookup_ops.py
.
String to Id table wrapper that assigns out-of-vocabulary keys to buckets.
For example, if an instance of IdTableWithHashBuckets
is initialized with a string-to-id table that maps: - emerson -> 0 - lake -> 1 - palmer -> 2
The IdTableWithHashBuckets
object will performs the following mapping: - emerson -> 0 - lake -> 1 - palmer -> 2 -
If input_tensor is ["emerson", "lake", "palmer", "king", "crimson"], the lookup result is [0, 1, 2, 4, 7]
If table
is None, only out-of-vocabulary buckets are used.
Example usage:
num_oov_buckets = 3 input_tensor = tf.constant(["emerson", "lake", "palmer", "king", "crimnson"]) table = tf.IdTableWithHashBuckets( tf.HashTable(tf.TextFileIdTableInitializer(filename), default_value), num_oov_buckets) out = table.lookup(input_tensor). table.init.run() print out.eval()
The hash function used for generating out-of-vocabulary buckets ID is handled by hasher_spec
.
Properties
init
The table initialization op.
key_dtype
The table key dtype.
name
The name of the table.
value_dtype
The table value dtype.
Methods
__init__
__init__( table, num_oov_buckets, hasher_spec=tf.contrib.lookup.FastHashSpec, name=None, key_dtype=None )
Construct a IdTableWithHashBuckets
object.
Args:
-
table
: Table that mapstf.string
ortf.int64
keys totf.int64
ids. -
num_oov_buckets
: Number of buckets to use for out-of-vocabulary keys. -
hasher_spec
: AHasherSpec
to specify the hash function to use for assignation of out-of-vocabulary buckets (optional). -
name
: A name for the operation (optional). -
key_dtype
: Data type of keys passed tolookup
. Defaults totable.key_dtype
iftable
is specified, otherwisetf.string
. Must be string or integer, and must be castable totable.key_dtype
.
Raises:
-
ValueError
: whentable
in None andnum_oov_buckets
is not positive. -
TypeError
: whenhasher_spec
is invalid.
check_table_dtypes
check_table_dtypes( key_dtype, value_dtype )
Check that the given key_dtype and value_dtype matches the table dtypes.
Args:
-
key_dtype
: The key data type to check. -
value_dtype
: The value data type to check.
Raises:
-
TypeError
: when 'key_dtype' or 'value_dtype' doesn't match the table data types.
lookup
lookup( keys, name=None )
Looks up keys
in the table, outputs the corresponding values.
It assigns out-of-vocabulary keys to buckets based in their hashes.
Args:
-
keys
: Keys to look up. May be either aSparseTensor
or denseTensor
. -
name
: Optional name for the op.
Returns:
A SparseTensor
if keys are sparse, otherwise a dense Tensor
.
Raises:
-
TypeError
: whenkeys
doesn't match the table key data type.
size
size(name=None)
Compute the number of elements in this table.
© 2017 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/contrib/lookup/IdTableWithHashBuckets