tfm.nlp.encoders.XLNetEncoderConfig

XLNet encoder configuration.

Inherits From: Config, ParamsDict

tfm.nlp.encoders.XLNetEncoderConfig(
    default_params: dataclasses.InitVar[Optional[Mapping[str, Any]]] = None,
    restrictions: dataclasses.InitVar[Optional[List[str]]] = None,
    vocab_size: int = 32000,
    num_layers: int = 24,
    hidden_size: int = 1024,
    num_attention_heads: int = 16,
    head_size: int = 64,
    inner_size: int = 4096,
    inner_activation: str = 'gelu',
    dropout_rate: float = 0.1,
    attention_dropout_rate: float = 0.1,
    attention_type: str = 'bi',
    bi_data: bool = False,
    tie_attention_biases: bool = False,
    memory_length: int = 0,
    same_length: bool = False,
    clamp_length: int = -1,
    reuse_length: int = 0,
    use_cls_mask: bool = False,
    embedding_width: int = 1024,
    initializer_range: float = 0.02,
    two_stream: bool = False
)

BUILDER

default_params Dataclass field restrictions Dataclass field vocab_size Dataclass field num_layers Dataclass field hidden_size Dataclass field num_attention_heads Dataclass field head_size Dataclass field inner_size Dataclass field inner_activation Dataclass field dropout_rate Dataclass field attention_dropout_rate Dataclass field attention_type Dataclass field bi_data Dataclass field tie_attention_biases Dataclass field memory_length Dataclass field same_length Dataclass field clamp_length Dataclass field reuse_length Dataclass field use_cls_mask Dataclass field embedding_width Dataclass field initializer_range Dataclass field two_stream Dataclass field

Methods

as_dict

View source

as_dict()

Returns a dict representation of params_dict.ParamsDict.

For the nested params_dict.ParamsDict, a nested dict will be returned.

from_args

View source

@classmethod
from_args(
    *args, **kwargs
)

Builds a config from the given list of arguments.

from_json

View source

@classmethod
from_json(
    file_path: str
)

Wrapper for from_yaml.

from_yaml

View source

@classmethod
from_yaml(
    file_path: str
)

get

View source

get(
    key, value=None
)

Accesses through built-in dictionary get method.

lock

View source

lock()

Makes the ParamsDict immutable.

override

View source

override(
    override_params, is_strict=True
)

Override the ParamsDict with a set of given params.

Args

override_params a dict or a ParamsDict specifying the parameters to be overridden. is_strict a boolean specifying whether override is strict or not. If True, keys in override_params must be present in the ParamsDict. If False, keys in override_params can be different from what is currently defined in the ParamsDict. In this case, the ParamsDict will be extended to include the new keys.

replace

View source

replace(
    **kwargs
)

Overrides/returns a unlocked copy with the current config unchanged.

validate

View source

validate()

Validate the parameters consistency based on the restrictions.

This method validates the internal consistency using the pre-defined list of restrictions. A restriction is defined as a string which specifies a binary operation. The supported binary operations are {'==', '!=', '<', '<=', '>', '>='}. Note that the meaning of these operators are consistent with the underlying Python immplementation. Users should make sure the define restrictions on their type make sense.

For example, for a ParamsDict like the following

a:
  a1: 1
  a2: 2
b:
  bb:
    bb1: 10
    bb2: 20
  ccc:
    a1: 1
    a3: 3

one can define two restrictions like this ['a.a1 == b.ccc.a1', 'a.a2 <= b.bb.bb2']

What it enforces are

  • a.a1 = 1 == b.ccc.a1 = 1
  • a.a2 = 2 <= b.bb.bb2 = 20

Raises

KeyError if any of the following happens (1) any of parameters in any of restrictions is not defined in ParamsDict, (2) any inconsistency violating the restriction is found. ValueError if the restriction defined in the string is not supported.

__contains__

View source

__contains__(
    key
)

Implements the membership test operator.

__eq__

__eq__(
    other
)

IMMUTABLE_TYPES (<class 'str'>, <class 'int'>, <class 'float'>, <class 'bool'>, <class 'NoneType'>) RESERVED_ATTR ['_locked', '_restrictions'] SEQUENCE_TYPES (<class 'list'>, <class 'tuple'>) attention_dropout_rate 0.1 attention_type 'bi' bi_data False clamp_length -1 default_params None dropout_rate 0.1 embedding_width 1024 head_size 64 hidden_size 1024 initializer_range 0.02 inner_activation 'gelu' inner_size 4096 memory_length 0 num_attention_heads 16 num_layers 24 restrictions None reuse_length 0 same_length False tie_attention_biases False two_stream False use_cls_mask False vocab_size 32000