CIFAR-100 All-CNN-C

class deepobs.cifar100.cifar100_allcnnc.set_up(batch_size=128, weight_decay=0.0005)[source]

"Class providing the functionality for the all convolutional architecture (All-CNN-C) from the Striving for simplicity paper on CIFAR-100.

Details about the architecture can be found in the paper. The All-CNN-C network consits of multiple convolutional layers, with two dropout layers in between. The paper does not comment on initialization; here we use Xavier for conv filters and constant 0.1 for biases.

Basis data augmentation (random crop, left-right flip, lighting augmentation) is done on the training images.

The training setting in the paper were: Batch size of 256, weight decay of 0.0005, total training time of 350 epochs, with a base learning rate of 0.05 and a decrease by factor 10 after 200, 250 and 300 epochs. Training was done using momentum with a momentum parameter of 0.9.

Parameters:
  • batch_size (int) -- Batch size of the data points. Defaults to 128.
  • weight_decay (float) -- Weight decay factor, which is only applied to the weights and not the biases. Defaults to 0.0005.
data_loading

Data loading class for CIFAR-100, cifar100_input.data_loading.

Type:deepobs.data_loading
losses

Tensor of size batch_size containing the individual losses per data point.

Type:tf.Tensor
accuracy

Tensor containing the accuracy of the model.

Type:tf.Tensor
train_init_op

A TensorFlow operation to be performed before starting every training epoch.

Type:tf.Operation
train_eval_init_op

A TensorFlow operation to be performed before starting every training eval epoch.

Type:tf.Operation
test_init_op

A TensorFlow operation to be performed before starting every test evaluation phase.

Type:tf.Operation
bias_variable(name, shape, init_val)[source]

Creates a bias variable of given shape and initialized to a given value.

Parameters:
  • name (str) -- Name of the bias variable.
  • shape (list) -- Dimensionality of the bias variable.
  • init_val (float) -- Initial value of the bias variable.
Returns:

Bias variable.
Return type:

tf.Variable
conv2d(x, W, stride=1, padding='VALID')[source]

Creates a two dimensional convolutional layer on top of a given input.

Parameters:
  • x (tf.Variable) -- Input to the layer.
  • W (tf.Variable) -- Weight variable of the convolutional layer.
  • stride (int) -- Stride of the convolution. Defaults to 1.
  • padding (str) -- Padding of the convolutional layers. Can be SAME or VALID. Defaults to VALID.
Returns:

Output after the convolutional layer.
Return type:

tf.Variable
conv_filter(name, shape)[source]

Creates a convolutional filter matrix, initialized by the Xavier-initializer.

Parameters:
  • name (str) -- Name of the filter variable.
  • shape (list) -- Dimensionality of the filter variable.
Returns:

Filter variable.
Return type:

tf.Variable
get()[source]

Returns the losses and the accuray of the model.

Returns:Tupel consisting of the losses and the accuracy.
Return type:tupel
set_up(weight_decay)[source]

Sets up the test problem.

Parameters:weight_decay (float) -- Weight decay factor, which is only applied to the weights and not the biases.
Returns:Tupel consisting of the losses and the accuracy.
Return type:tupel