example.fashionMNIST.data_preprocessing package

Submodules

example.fashionMNIST.data_preprocessing.MNIST_dataset module

load_dataset_FashionMNIST_with_standardization(dataset_path, valid_ratio=0.2, num_threads=4, batch_size=128)

Load the FashionMNIST dataset with standardize data

Tip

For example with a valid_ratio = 0.2, we going to use 80%/20% split for train/valid.

Parameters:

  • dataset_path (str.) – Path where the dataset will be read if already present, downloaded else.

  • valid_ratio (float.) – Percentage of the FashionMNIST train dataset that will be used or the test process (validation data). Between 0.0 and 1.0.

  • num_threads (int.) – Number of threads used for this task.

  • batch_size (int.) – Size of the batchs.

Returns:

a tuple of three dataloader: train, validation and test.

Return type:

Tuple of three torch.utils.data.DataLoader.

example.fashionMNIST.data_preprocessing.data_utils module

class DatasetTransformer(base_dataset, transform)

Wrapper of the pytorch Dataset class. Add a transformation process to apply to each sample during the data loading.

compute_mean_std(loader)

Compute the mean over minibatches.

Parameters:

loader (torch.utils.data.DataLoader.) – Dataloader on which to iterate.