2D Data Loading

class deepobs.two_d.two_d_input.data_loading(batch_size, train_size=1000, noise_level=6)

Class providing the data loading functionality for simple 2D stochastic loss functions.

Parameters:

• batch_size (int) -- Batch size. No default value is given.
• train_size (int) -- Size of the training set. Defaults to 1000.
• noise_level (float) -- Noise level of the training set. All training points are sampled from a gaussian distribution with the noise level as the standard deviation. Defaults to 6.

batch_size

Batch size.

Type:int

train_size

Size of the training set. Defaults to 1000.

Type:int

noise_level

Noise level of the training set. All training points are sampled from a gaussian distribution with the noise level as the standard deviation. Defaults to 6.

Type:float

D_train

The training data set.

Type:tf.data.Dataset

D_train_eval

The training evaluation data set. It is the same data as D_train but we go through it separately.

Type:tf.data.Dataset

D_test

The test data set. We use the mean of the data points. Thus, the test data set has just a single data point.

Type:tf.data.Dataset

phase

Variable to describe which phase we are currently in. Can be "train", "train_eval" or "test". The phase variable can determine the behaviour of the network, for example deactivate dropout during evaluation.

Type:tf.Variable

iterator

A single iterator for all three data sets. We us the initialization operators (see below) to switch this iterator to the data sets.

Type:tf.data.Iterator

X

Tensor holding data points. It has dimension batch_size.

Type:tf.Tensor

y

Tensor holding the labels of the data points. It has dimension batch_size.

Type:tf.Tensor

train_init_op

A TensorFlow operation to be performed before starting every training epoch. It sets the phase variable to "train" and initializes the iterator to the training data set.

Type:tf.Operation

train_eval_init_op

A TensorFlow operation to be performed before starting every training eval phase. It sets the phase variable to "train_eval" and initializes the iterator to the training eval data set.

Type:tf.Operation

test_init_op

A TensorFlow operation to be performed before starting every test evaluation phase. It sets the phase variable to "test" and initializes the iterator to the test data set.

Type:tf.Operation

load()

Returns the data (X and y ) and the phase variable.

Returns:Tupel consisting of the data points (X), (y) and the phase variable (phase). Return type:tupel

make_dataset(data_x, data_y, batch_size, one_hot=True, shuffle=True, shuffle_buffer_size=10000, num_prefetched_batches=10)

Creates a data set from given data points.

Parameters:

• data_x (np.array) -- Numpy array containing the X values of the data points.
• data_y (np.array) -- Numpy array containing the y values of the data points.
• batch_size (int) -- Batch size of the input-output pairs.
• one_hot (bool) -- Switch to turn on or off one-hot encoding of the labels. Defaults to True.
• shuffle (bool) -- Switch to turn on or off shuffling of the data set. Defaults to True.
• shuffle_buffer_size (int) -- Size of the shuffle buffer. Defaults to 10000 the size of the test and train eval data set, meaning that they will be completely shuffled.
• num_prefetched_batches (int) -- Number of prefeteched batches, defaults to 10.

Returns:

Data set object created from the images and label files.

Return type:

tf.data.Dataset

test_dataset(batch_size)

Creates the test data set.

Parameters:batch_size (int) -- Batch size. Just a single data point is created, with the mean value of the Gaussian distributions of the training data set. Returns:The test data set. Return type:tf.data.Dataset

train_dataset(batch_size, size, noise_level)

Creates the training data set.

Parameters:

• batch_size (int) -- Batch size of the data points.
• size (int) -- Size of the training data set, i.e. the number of data points in the train set.
• noise_level (float) -- Standard deviation of the data points around the mean. The data points are drawn from a Gaussian distribution.

Returns:

The training data set.

Return type:

tf.data.Dataset

train_eval_dataset(batch_size, size, noise_level)

Creates the train eval data set.

Parameters:

• batch_size (int) -- Batch size of the data points.
• size (int) -- Size of the train eval data set, i.e. the number of data points in the train eval set.
• noise_level (float) -- Standard deviation of the data points around the mean. The data points are drawn from a Gaussian distribution.

Returns:

The train eval data set.

Return type:

tf.data.Dataset