tfm.vision.configs.image_classification.DataConfig

Input config for training.

Inherits From: DataConfig, Config, ParamsDict

tfm.vision.configs.image_classification.DataConfig(
    default_params: dataclasses.InitVar[Optional[Mapping[str, Any]]] = None,
    restrictions: dataclasses.InitVar[Optional[List[str]]] = None,
    input_path: Union[Sequence[str], str, tfm.hyperparams.Config] = '',
    tfds_name: Union[str, tfm.hyperparams.Config] = '',
    tfds_split: str = '',
    global_batch_size: int = 0,
    is_training: bool = True,
    drop_remainder: bool = True,
    shuffle_buffer_size: int = 10000,
    cache: bool = False,
    cycle_length: int = 10,
    block_length: int = 1,
    deterministic: Optional[bool] = None,
    sharding: bool = True,
    enable_tf_data_service: bool = False,
    tf_data_service_address: Optional[str] = None,
    tf_data_service_job_name: Optional[str] = None,
    tfds_data_dir: str = '',
    tfds_as_supervised: bool = False,
    tfds_skip_decoding_feature: str = '',
    enable_shared_tf_data_service_between_parallel_trainers: bool = False,
    apply_tf_data_service_before_batching: bool = False,
    trainer_id: Optional[str] = None,
    seed: Optional[int] = None,
    prefetch_buffer_size: Optional[int] = None,
    autotune_algorithm: Optional[str] = None,
    weights: Optional[tfm.hyperparams.Config] = None,
    dtype: str = 'float32',
    is_multilabel: bool = False,
    aug_rand_hflip: bool = True,
    aug_crop: Optional[bool] = True,
    crop_area_range: Optional[Tuple[float, float]] = tfm.vision.configs.image_classification.DataConfig.crop_area_range,
    aug_type: Optional[tfm.vision.configs.common.Augmentation] = None,
    three_augment: bool = False,
    color_jitter: float = 0.0,
    random_erasing: Optional[tfm.vision.configs.common.RandomErasing] = None,
    file_type: str = 'tfrecord',
    image_field_key: str = 'image/encoded',
    label_field_key: str = 'image/class/label',
    decode_jpeg_only: bool = True,
    mixup_and_cutmix: Optional[tfm.vision.configs.common.MixupAndCutmix] = None,
    decoder: Optional[tfm.vision.configs.common.DataDecoder] = dataclasses.field(default_factory=common.DataDecoder),
    aug_policy: Optional[str] = None,
    randaug_magnitude: Optional[int] = 10,
    center_crop_fraction: Optional[float] = 0.875,
    tf_resize_method: str = 'bilinear',
    repeated_augment: Optional[int] = None
)

BUILDER

default_params Dataclass field restrictions Dataclass field input_path Dataclass field tfds_name Dataclass field tfds_split Dataclass field global_batch_size Dataclass field is_training Dataclass field drop_remainder Dataclass field shuffle_buffer_size Dataclass field cache Dataclass field cycle_length Dataclass field block_length Dataclass field deterministic Dataclass field sharding Dataclass field enable_tf_data_service Dataclass field tf_data_service_address Dataclass field tf_data_service_job_name Dataclass field tfds_data_dir Dataclass field tfds_as_supervised Dataclass field tfds_skip_decoding_feature Dataclass field enable_shared_tf_data_service_between_parallel_trainers Dataclass field apply_tf_data_service_before_batching Dataclass field trainer_id Dataclass field seed Dataclass field prefetch_buffer_size Dataclass field autotune_algorithm Dataclass field weights Dataclass field dtype Dataclass field is_multilabel Dataclass field aug_rand_hflip Dataclass field aug_crop Dataclass field crop_area_range Dataclass field aug_type Dataclass field three_augment Dataclass field color_jitter Dataclass field random_erasing Dataclass field file_type Dataclass field image_field_key Dataclass field label_field_key Dataclass field decode_jpeg_only Dataclass field mixup_and_cutmix Dataclass field decoder Dataclass field aug_policy Dataclass field randaug_magnitude Dataclass field center_crop_fraction Dataclass field tf_resize_method Dataclass field repeated_augment Dataclass field

Methods

as_dict

View source

as_dict()

Returns a dict representation of params_dict.ParamsDict.

For the nested params_dict.ParamsDict, a nested dict will be returned.

from_args

View source

@classmethod
from_args(
    *args, **kwargs
)

Builds a config from the given list of arguments.

from_json

View source

@classmethod
from_json(
    file_path: str
)

Wrapper for from_yaml.

from_yaml

View source

@classmethod
from_yaml(
    file_path: str
)

get

View source

get(
    key, value=None
)

Accesses through built-in dictionary get method.

lock

View source

lock()

Makes the ParamsDict immutable.

override

View source

override(
    override_params, is_strict=True
)

Override the ParamsDict with a set of given params.

Args

override_params a dict or a ParamsDict specifying the parameters to be overridden. is_strict a boolean specifying whether override is strict or not. If True, keys in override_params must be present in the ParamsDict. If False, keys in override_params can be different from what is currently defined in the ParamsDict. In this case, the ParamsDict will be extended to include the new keys.

replace

View source

replace(
    **kwargs
)

Overrides/returns a unlocked copy with the current config unchanged.

validate

View source

validate()

Validate the parameters consistency based on the restrictions.

This method validates the internal consistency using the pre-defined list of restrictions. A restriction is defined as a string which specifies a binary operation. The supported binary operations are {'==', '!=', '<', '<=', '>', '>='}. Note that the meaning of these operators are consistent with the underlying Python immplementation. Users should make sure the define restrictions on their type make sense.

For example, for a ParamsDict like the following

a:
  a1: 1
  a2: 2
b:
  bb:
    bb1: 10
    bb2: 20
  ccc:
    a1: 1
    a3: 3

one can define two restrictions like this ['a.a1 == b.ccc.a1', 'a.a2 <= b.bb.bb2']

What it enforces are

  • a.a1 = 1 == b.ccc.a1 = 1
  • a.a2 = 2 <= b.bb.bb2 = 20

Raises

KeyError if any of the following happens (1) any of parameters in any of restrictions is not defined in ParamsDict, (2) any inconsistency violating the restriction is found. ValueError if the restriction defined in the string is not supported.

__contains__

View source

__contains__(
    key
)

Implements the membership test operator.

__eq__

__eq__(
    other
)

IMMUTABLE_TYPES (<class 'str'>, <class 'int'>, <class 'float'>, <class 'bool'>, <class 'NoneType'>) RESERVED_ATTR ['_locked', '_restrictions'] SEQUENCE_TYPES (<class 'list'>, <class 'tuple'>) apply_tf_data_service_before_batching False aug_crop True aug_policy None aug_rand_hflip True aug_type None autotune_algorithm None block_length 1 cache False center_crop_fraction 0.875 color_jitter 0.0 crop_area_range (0.08, 1.0) cycle_length 10 decode_jpeg_only True default_params None deterministic None drop_remainder True dtype 'float32' enable_shared_tf_data_service_between_parallel_trainers False enable_tf_data_service False file_type 'tfrecord' global_batch_size 0 image_field_key 'image/encoded' input_path '' is_multilabel False is_training True label_field_key 'image/class/label' mixup_and_cutmix None prefetch_buffer_size None randaug_magnitude 10 random_erasing None repeated_augment None restrictions None seed None sharding True shuffle_buffer_size 10000 tf_data_service_address None tf_data_service_job_name None tf_resize_method 'bilinear' tfds_as_supervised False tfds_data_dir '' tfds_name '' tfds_skip_decoding_feature '' tfds_split '' three_augment False trainer_id None weights None