Conv Net

---

Has the following structure :

Input -> Conv -> MaxPool -> Conv -> MaxPool -> Dropout -> Hidden -> Dropout -> Softmax

Imports

%matplotlib inline 

import numpy as np
import matplotlib
import matplotlib.pyplot as plt

# Data Loader
import sys
sys.path.insert(0, '/home/licet/Documents/tamilclassifier/data_loader')
from load_data import load_data

Data Loader

load_data - function that takes in as input the path to the dataset and returns numpy matrices

# Load the data 
X_tr, y_tr, X_ts, y_ts, X_val, y_val = load_data('/home/licet/Documents/tamilclassifier/dataset/tamil_final/')

print "\nData Loaded"

print "X Train ", X_tr.shape
print "y Train ", y_tr.shape
print "X Test ", X_ts.shape
print "y Test ", y_ts.shape
print "X Val ", X_val.shape
print "y Val ", y_val.shape
print "Data Type ", X_tr.dtype

Visualising ...
Visulaised Output  [32]
Processed Samples training:  17100
Processed Samples test:  4275
Processed Samples validation:  3600

Data Loaded
X Train  (17100, 3025)
y Train  (17100,)
X Test  (4275, 3025)
y Test  (4275,)
X Val  (3600, 3025)
y Val  (3600,)
Data Type  float64

Reshape according to theano's conventions

(no_of_training_examples, no_of_channels, image_width, image_height)

# Reshape the Input Examples to the dimensions required by theano 
# (no_of_egs, no_of_channels, width, height)
X_tr = X_tr.reshape(-1,1,55,55)
X_val = X_val.reshape(-1,1,55,55)
X_ts = X_ts.reshape(-1,1,55,55)

Training

The fit method takes in numpy matrices, builds the specifed network, trains them, calculates validation and test set accuracies and then returns a reference to the last layer of the neural network

# Now Start training by a creating a ModernNet Instance
from ConvNet import ConvNet

net = ConvNet()
model, trn_loss, val_loss = net.fit(X_tr, y_tr, X_val, y_val, X_ts, y_ts)

Checking..........
Starting training...
Epoch 1 of 500 took 114.679s
  training loss:		3.806524
  validation loss:		3.786465
  validation accuracy:		6.00 %
Train Batches  57
Val batches 12
Epoch 2 of 500 took 98.240s
  training loss:		3.776461
  validation loss:		3.744579
  validation accuracy:		12.67 %
Train Batches  57
Val batches 12
Epoch 3 of 500 took 98.428s
  training loss:		3.721543
  validation loss:		3.648787
  validation accuracy:		22.56 %
Train Batches  57
Val batches 12
Epoch 4 of 500 took 98.308s
  training loss:		3.613587
  validation loss:		3.485731
  validation accuracy:		37.14 %
Train Batches  57
Val batches 12
Epoch 5 of 500 took 97.784s
  training loss:		3.384892
  validation loss:		3.096493
  validation accuracy:		47.06 %
Train Batches  57
Val batches 12
Epoch 6 of 500 took 97.407s
  training loss:		2.977509
  validation loss:		2.327409
  validation accuracy:		58.08 %
Train Batches  57
Val batches 12
Epoch 7 of 500 took 111.553s
  training loss:		2.401269
  validation loss:		1.519190
  validation accuracy:		64.72 %
Train Batches  57
Val batches 12
Epoch 8 of 500 took 92.077s
  training loss:		1.918249
  validation loss:		1.272502
  validation accuracy:		68.11 %
Train Batches  57
Val batches 12
Epoch 9 of 500 took 107.468s
  training loss:		1.694096
  validation loss:		1.113189
  validation accuracy:		71.42 %
Train Batches  57
Val batches 12
Epoch 10 of 500 took 115.392s
  training loss:		1.528854
  validation loss:		1.009822
  validation accuracy:		73.47 %
Train Batches  57
Val batches 12
Epoch 11 of 500 took 111.143s
  training loss:		1.422359
  validation loss:		0.924013
  validation accuracy:		75.03 %
Train Batches  57
Val batches 12
Epoch 12 of 500 took 111.034s
  training loss:		1.321333
  validation loss:		0.869723
  validation accuracy:		76.33 %
Train Batches  57
Val batches 12
Epoch 13 of 500 took 98.293s
  training loss:		1.247027
  validation loss:		0.821147
  validation accuracy:		77.58 %
Train Batches  57
Val batches 12
Epoch 14 of 500 took 92.390s
  training loss:		1.179103
  validation loss:		0.769481
  validation accuracy:		78.89 %
Train Batches  57
Val batches 12
Epoch 15 of 500 took 110.856s
  training loss:		1.111372
  validation loss:		0.731835
  validation accuracy:		79.58 %
Train Batches  57
Val batches 12
Epoch 16 of 500 took 113.518s
  training loss:		1.057563
  validation loss:		0.693902
  validation accuracy:		80.97 %
Train Batches  57
Val batches 12
Epoch 17 of 500 took 112.959s
  training loss:		1.009278
  validation loss:		0.660473
  validation accuracy:		81.64 %
Train Batches  57
Val batches 12
Epoch 18 of 500 took 113.933s
  training loss:		0.952069
  validation loss:		0.628977
  validation accuracy:		82.33 %
Train Batches  57
Val batches 12
Epoch 19 of 500 took 100.911s
  training loss:		0.922475
  validation loss:		0.607139
  validation accuracy:		82.67 %
Train Batches  57
Val batches 12
Epoch 20 of 500 took 105.009s
  training loss:		0.873922
  validation loss:		0.577006
  validation accuracy:		84.19 %
Train Batches  57
Val batches 12
Epoch 21 of 500 took 97.565s
  training loss:		0.855452
  validation loss:		0.563446
  validation accuracy:		84.33 %
Train Batches  57
Val batches 12
Epoch 22 of 500 took 104.682s
  training loss:		0.815212
  validation loss:		0.540325
  validation accuracy:		85.08 %
Train Batches  57
Val batches 12
Epoch 23 of 500 took 116.782s
  training loss:		0.772300
  validation loss:		0.520219
  validation accuracy:		85.39 %
Train Batches  57
Val batches 12
Epoch 24 of 500 took 95.293s
  training loss:		0.756282
  validation loss:		0.498718
  validation accuracy:		85.94 %
Train Batches  57
Val batches 12
Epoch 25 of 500 took 117.488s
  training loss:		0.732919
  validation loss:		0.486958
  validation accuracy:		86.36 %
Train Batches  57
Val batches 12
Epoch 26 of 500 took 110.161s
  training loss:		0.708811
  validation loss:		0.472041
  validation accuracy:		86.97 %
Train Batches  57
Val batches 12
Epoch 27 of 500 took 98.678s
  training loss:		0.686339
  validation loss:		0.459995
  validation accuracy:		86.72 %
Train Batches  57
Val batches 12
Epoch 28 of 500 took 107.468s
  training loss:		0.660344
  validation loss:		0.450087
  validation accuracy:		86.78 %
Train Batches  57
Val batches 12
Epoch 29 of 500 took 95.798s
  training loss:		0.641118
  validation loss:		0.436013
  validation accuracy:		87.44 %
Train Batches  57
Val batches 12
Epoch 30 of 500 took 95.989s
  training loss:		0.623110
  validation loss:		0.426152
  validation accuracy:		87.97 %
Train Batches  57
Val batches 12
Epoch 31 of 500 took 111.977s
  training loss:		0.606959
  validation loss:		0.415227
  validation accuracy:		88.28 %
Train Batches  57
Val batches 12
Epoch 32 of 500 took 106.427s
  training loss:		0.603506
  validation loss:		0.405368
  validation accuracy:		88.42 %
Train Batches  57
Val batches 12
Epoch 33 of 500 took 79.561s
  training loss:		0.583816
  validation loss:		0.399371
  validation accuracy:		88.28 %
Train Batches  57
Val batches 12
Epoch 34 of 500 took 79.739s
  training loss:		0.573959
  validation loss:		0.392991
  validation accuracy:		88.28 %
Train Batches  57
Val batches 12
Epoch 35 of 500 took 89.846s
  training loss:		0.549931
  validation loss:		0.383030
  validation accuracy:		88.78 %
Train Batches  57
Val batches 12
Epoch 36 of 500 took 96.048s
  training loss:		0.537505
  validation loss:		0.376134
  validation accuracy:		89.31 %
Train Batches  57
Val batches 12
Epoch 37 of 500 took 95.801s
  training loss:		0.533318
  validation loss:		0.370242
  validation accuracy:		89.19 %
Train Batches  57
Val batches 12
Epoch 38 of 500 took 95.732s
  training loss:		0.518054
  validation loss:		0.367270
  validation accuracy:		89.19 %
Train Batches  57
Val batches 12
Epoch 39 of 500 took 95.787s
  training loss:		0.511189
  validation loss:		0.358466
  validation accuracy:		89.61 %
Train Batches  57
Val batches 12
Epoch 40 of 500 took 95.826s
  training loss:		0.504803
  validation loss:		0.356010
  validation accuracy:		89.58 %
Train Batches  57
Val batches 12
Epoch 41 of 500 took 95.775s
  training loss:		0.491277
  validation loss:		0.350692
  validation accuracy:		89.64 %
Train Batches  57
Val batches 12
Epoch 42 of 500 took 95.724s
  training loss:		0.486566
  validation loss:		0.347352
  validation accuracy:		89.92 %
Train Batches  57
Val batches 12
Epoch 43 of 500 took 95.738s
  training loss:		0.474360
  validation loss:		0.341151
  validation accuracy:		89.78 %
Train Batches  57
Val batches 12
Epoch 44 of 500 took 95.782s
  training loss:		0.463551
  validation loss:		0.334287
  validation accuracy:		90.14 %
Train Batches  57
Val batches 12
Epoch 45 of 500 took 95.852s
  training loss:		0.463349
  validation loss:		0.331272
  validation accuracy:		90.19 %
Train Batches  57
Val batches 12
Epoch 46 of 500 took 95.837s
  training loss:		0.439916
  validation loss:		0.327157
  validation accuracy:		90.47 %
Train Batches  57
Val batches 12
Epoch 47 of 500 took 95.809s
  training loss:		0.431039
  validation loss:		0.325251
  validation accuracy:		90.36 %
Train Batches  57
Val batches 12
Epoch 48 of 500 took 95.955s
  training loss:		0.432412
  validation loss:		0.323481
  validation accuracy:		90.03 %
Train Batches  57
Val batches 12
Epoch 49 of 500 took 95.821s
  training loss:		0.426676
  validation loss:		0.317279
  validation accuracy:		90.64 %
Train Batches  57
Val batches 12
Epoch 50 of 500 took 95.880s
  training loss:		0.425139
  validation loss:		0.314064
  validation accuracy:		90.58 %
Train Batches  57
Val batches 12
Epoch 51 of 500 took 82.115s
  training loss:		0.420389
  validation loss:		0.309352
  validation accuracy:		90.81 %
Train Batches  57
Val batches 12
Epoch 52 of 500 took 90.087s
  training loss:		0.413144
  validation loss:		0.306611
  validation accuracy:		90.86 %
Train Batches  57
Val batches 12
Epoch 53 of 500 took 97.013s
  training loss:		0.401547
  validation loss:		0.306135
  validation accuracy:		91.08 %
Train Batches  57
Val batches 12
Epoch 54 of 500 took 97.138s
  training loss:		0.397609
  validation loss:		0.300611
  validation accuracy:		91.00 %
Train Batches  57
Val batches 12
Epoch 55 of 500 took 97.323s
  training loss:		0.393952
  validation loss:		0.298362
  validation accuracy:		91.56 %
Train Batches  57
Val batches 12
Epoch 56 of 500 took 97.438s
  training loss:		0.383002
  validation loss:		0.294840
  validation accuracy:		91.25 %
Train Batches  57
Val batches 12
Epoch 57 of 500 took 97.369s
  training loss:		0.389335
  validation loss:		0.293972
  validation accuracy:		91.14 %
Train Batches  57
Val batches 12
Epoch 58 of 500 took 97.154s
  training loss:		0.373198
  validation loss:		0.291373
  validation accuracy:		91.44 %
Train Batches  57
Val batches 12
Epoch 59 of 500 took 96.998s
  training loss:		0.365410
  validation loss:		0.288981
  validation accuracy:		91.61 %
Train Batches  57
Val batches 12
Epoch 60 of 500 took 96.944s
  training loss:		0.353863
  validation loss:		0.284390
  validation accuracy:		91.92 %
Train Batches  57
Val batches 12
Epoch 61 of 500 took 97.034s
  training loss:		0.355296
  validation loss:		0.281045
  validation accuracy:		91.83 %
Train Batches  57
Val batches 12
Epoch 62 of 500 took 97.103s
  training loss:		0.354555
  validation loss:		0.279103
  validation accuracy:		92.11 %
Train Batches  57
Val batches 12
Epoch 63 of 500 took 97.115s
  training loss:		0.348804
  validation loss:		0.276837
  validation accuracy:		91.81 %
Train Batches  57
Val batches 12
Epoch 64 of 500 took 97.054s
  training loss:		0.341340
  validation loss:		0.274890
  validation accuracy:		92.19 %
Train Batches  57
Val batches 12
Epoch 65 of 500 took 106.602s
  training loss:		0.337993
  validation loss:		0.273471
  validation accuracy:		92.28 %
Train Batches  57
Val batches 12
Epoch 66 of 500 took 115.127s
  training loss:		0.330920
  validation loss:		0.271580
  validation accuracy:		91.94 %
Train Batches  57
Val batches 12
Epoch 67 of 500 took 106.840s
  training loss:		0.324925
  validation loss:		0.269899
  validation accuracy:		92.03 %
Train Batches  57
Val batches 12
Epoch 68 of 500 took 98.988s
  training loss:		0.333798
  validation loss:		0.266321
  validation accuracy:		92.31 %
Train Batches  57
Val batches 12
Epoch 69 of 500 took 106.222s
  training loss:		0.324811
  validation loss:		0.267525
  validation accuracy:		92.50 %
Train Batches  57
Val batches 12
Epoch 70 of 500 took 116.297s
  training loss:		0.315295
  validation loss:		0.264593
  validation accuracy:		92.31 %
Train Batches  57
Val batches 12
Epoch 71 of 500 took 106.288s
  training loss:		0.318852
  validation loss:		0.261880
  validation accuracy:		92.44 %
Train Batches  57
Val batches 12
Epoch 72 of 500 took 96.837s
  training loss:		0.312631
  validation loss:		0.261178
  validation accuracy:		92.61 %
Train Batches  57
Val batches 12
Epoch 73 of 500 took 88.815s
  training loss:		0.308520
  validation loss:		0.260149
  validation accuracy:		92.53 %
Train Batches  57
Val batches 12
Epoch 74 of 500 took 114.877s
  training loss:		0.303951
  validation loss:		0.259513
  validation accuracy:		92.33 %
Train Batches  57
Val batches 12
Epoch 75 of 500 took 106.987s
  training loss:		0.304786
  validation loss:		0.257592
  validation accuracy:		92.67 %
Train Batches  57
Val batches 12
Epoch 76 of 500 took 108.091s
  training loss:		0.303511
  validation loss:		0.256170
  validation accuracy:		92.50 %
Train Batches  57
Val batches 12
Epoch 77 of 500 took 88.911s
  training loss:		0.300318
  validation loss:		0.251397
  validation accuracy:		92.44 %
Train Batches  57
Val batches 12
Epoch 78 of 500 took 99.400s
  training loss:		0.286139
  validation loss:		0.251514
  validation accuracy:		92.67 %
Train Batches  57
Val batches 12
Epoch 79 of 500 took 96.583s
  training loss:		0.290957
  validation loss:		0.252513
  validation accuracy:		92.56 %
Train Batches  57
Val batches 12
Epoch 80 of 500 took 96.839s
  training loss:		0.287554
  validation loss:		0.248518
  validation accuracy:		92.53 %
Train Batches  57
Val batches 12
Epoch 81 of 500 took 119.067s
  training loss:		0.282169
  validation loss:		0.248926
  validation accuracy:		92.64 %
Train Batches  57
Val batches 12
Epoch 82 of 500 took 93.780s
  training loss:		0.280229
  validation loss:		0.244377
  validation accuracy:		92.92 %
Train Batches  57
Val batches 12
Epoch 83 of 500 took 97.984s
  training loss:		0.278158
  validation loss:		0.244887
  validation accuracy:		92.81 %
Train Batches  57
Val batches 12
Epoch 84 of 500 took 111.994s
  training loss:		0.277130
  validation loss:		0.242980
  validation accuracy:		92.92 %
Train Batches  57
Val batches 12
Epoch 85 of 500 took 110.873s
  training loss:		0.280261
  validation loss:		0.244568
  validation accuracy:		92.69 %
Train Batches  57
Val batches 12
Epoch 86 of 500 took 114.696s
  training loss:		0.268718
  validation loss:		0.241421
  validation accuracy:		92.97 %
Train Batches  57
Val batches 12
Epoch 87 of 500 took 111.645s
  training loss:		0.266642
  validation loss:		0.241685
  validation accuracy:		93.25 %
Train Batches  57
Val batches 12
Epoch 88 of 500 took 110.094s
  training loss:		0.261207
  validation loss:		0.240921
  validation accuracy:		92.86 %
Train Batches  57
Val batches 12
Epoch 89 of 500 took 97.914s
  training loss:		0.261712
  validation loss:		0.238103
  validation accuracy:		93.08 %
Train Batches  57
Val batches 12
Epoch 90 of 500 took 102.214s
  training loss:		0.255430
  validation loss:		0.238174
  validation accuracy:		92.94 %
Train Batches  57
Val batches 12
Epoch 91 of 500 took 109.959s
  training loss:		0.251862
  validation loss:		0.237246
  validation accuracy:		93.28 %
Train Batches  57
Val batches 12
Epoch 92 of 500 took 103.517s
  training loss:		0.250286
  validation loss:		0.235816
  validation accuracy:		93.14 %
Train Batches  57
Val batches 12
Epoch 93 of 500 took 89.067s
  training loss:		0.252084
  validation loss:		0.233972
  validation accuracy:		93.19 %
Train Batches  57
Val batches 12
Epoch 94 of 500 took 95.803s
  training loss:		0.248299
  validation loss:		0.235665
  validation accuracy:		93.22 %
Train Batches  57
Val batches 12
Epoch 95 of 500 took 98.946s
  training loss:		0.247197
  validation loss:		0.234320
  validation accuracy:		93.28 %
Train Batches  57
Val batches 12
Epoch 96 of 500 took 99.522s
  training loss:		0.243902
  validation loss:		0.233543
  validation accuracy:		93.22 %
Train Batches  57
Val batches 12
Epoch 97 of 500 took 79.468s
  training loss:		0.241842
  validation loss:		0.232499
  validation accuracy:		93.31 %
Train Batches  57
Val batches 12
Epoch 98 of 500 took 79.594s
  training loss:		0.241590
  validation loss:		0.232100
  validation accuracy:		93.36 %
Train Batches  57
Val batches 12
Epoch 99 of 500 took 80.630s
  training loss:		0.239081
  validation loss:		0.234260
  validation accuracy:		93.03 %
Train Batches  57
Val batches 12
Epoch 100 of 500 took 79.977s
  training loss:		0.232798
  validation loss:		0.232647
  validation accuracy:		93.08 %
Train Batches  57
Val batches 12
Epoch 101 of 500 took 112.589s
  training loss:		0.234511
  validation loss:		0.228947
  validation accuracy:		93.44 %
Train Batches  57
Val batches 12
Epoch 102 of 500 took 106.321s
  training loss:		0.230612
  validation loss:		0.229772
  validation accuracy:		93.33 %
Train Batches  57
Val batches 12
Epoch 103 of 500 took 119.925s
  training loss:		0.223918
  validation loss:		0.228274
  validation accuracy:		93.33 %
Train Batches  57
Val batches 12
Epoch 104 of 500 took 120.203s
  training loss:		0.221479
  validation loss:		0.226336
  validation accuracy:		93.39 %
Train Batches  57
Val batches 12
Epoch 105 of 500 took 93.474s
  training loss:		0.228658
  validation loss:		0.228312
  validation accuracy:		93.44 %
Train Batches  57
Val batches 12
Epoch 106 of 500 took 118.347s
  training loss:		0.222163
  validation loss:		0.226735
  validation accuracy:		93.50 %
Train Batches  57
Val batches 12
Epoch 107 of 500 took 103.302s
  training loss:		0.214605
  validation loss:		0.224725
  validation accuracy:		93.25 %
Train Batches  57
Val batches 12
Epoch 108 of 500 took 111.672s
  training loss:		0.218419
  validation loss:		0.223311
  validation accuracy:		93.42 %
Train Batches  57
Val batches 12
Epoch 109 of 500 took 119.130s
  training loss:		0.216648
  validation loss:		0.222945
  validation accuracy:		93.44 %
Train Batches  57
Val batches 12
Epoch 110 of 500 took 118.812s
  training loss:		0.205840
  validation loss:		0.225135
  validation accuracy:		93.28 %
Train Batches  57
Val batches 12
Epoch 111 of 500 took 118.862s
  training loss:		0.213173
  validation loss:		0.221448
  validation accuracy:		93.64 %
Train Batches  57
Val batches 12
Epoch 112 of 500 took 91.010s
  training loss:		0.203304
  validation loss:		0.220902
  validation accuracy:		93.58 %
Train Batches  57
Val batches 12
Epoch 113 of 500 took 81.071s
  training loss:		0.209204
  validation loss:		0.220831
  validation accuracy:		93.39 %
Train Batches  57
Val batches 12
Epoch 114 of 500 took 86.397s
  training loss:		0.204922
  validation loss:		0.220772
  validation accuracy:		93.50 %
Train Batches  57
Val batches 12
Epoch 115 of 500 took 111.004s
  training loss:		0.208566
  validation loss:		0.220652
  validation accuracy:		93.67 %
Train Batches  57
Val batches 12
Epoch 116 of 500 took 97.916s
  training loss:		0.196895
  validation loss:		0.219227
  validation accuracy:		93.25 %
Train Batches  57
Val batches 12
Epoch 117 of 500 took 100.333s
  training loss:		0.200188
  validation loss:		0.219978
  validation accuracy:		93.31 %
Train Batches  57
Val batches 12
Epoch 118 of 500 took 105.343s
  training loss:		0.198008
  validation loss:		0.218246
  validation accuracy:		93.53 %
Train Batches  57
Val batches 12
Epoch 119 of 500 took 80.149s
  training loss:		0.198926
  validation loss:		0.219316
  validation accuracy:		93.58 %
Train Batches  57
Val batches 12
Epoch 120 of 500 took 80.457s
  training loss:		0.199142
  validation loss:		0.217717
  validation accuracy:		93.81 %
Train Batches  57
Val batches 12
Epoch 121 of 500 took 80.845s
  training loss:		0.194098
  validation loss:		0.218270
  validation accuracy:		93.64 %
Train Batches  57
Val batches 12
Epoch 122 of 500 took 81.349s
  training loss:		0.193100
  validation loss:		0.216387
  validation accuracy:		93.61 %
Train Batches  57
Val batches 12
Epoch 123 of 500 took 94.026s
  training loss:		0.195104
  validation loss:		0.215883
  validation accuracy:		93.42 %
Train Batches  57
Val batches 12
Epoch 124 of 500 took 117.483s
  training loss:		0.192873
  validation loss:		0.216059
  validation accuracy:		93.69 %
Train Batches  57
Val batches 12
Epoch 125 of 500 took 119.597s
  training loss:		0.188902
  validation loss:		0.214481
  validation accuracy:		93.58 %
Train Batches  57
Val batches 12
Epoch 126 of 500 took 119.806s
  training loss:		0.189927
  validation loss:		0.216486
  validation accuracy:		93.53 %
Train Batches  57
Val batches 12
Epoch 127 of 500 took 109.713s
  training loss:		0.190904
  validation loss:		0.214116
  validation accuracy:		93.61 %
Train Batches  57
Val batches 12
Epoch 128 of 500 took 93.263s
  training loss:		0.187816
  validation loss:		0.209977
  validation accuracy:		93.89 %
Train Batches  57
Val batches 12
Epoch 129 of 500 took 97.863s
  training loss:		0.187809
  validation loss:		0.212516
  validation accuracy:		93.78 %
Train Batches  57
Val batches 12
Epoch 130 of 500 took 97.960s
  training loss:		0.183220
  validation loss:		0.214038
  validation accuracy:		93.72 %
Train Batches  57
Val batches 12
Epoch 131 of 500 took 97.877s
  training loss:		0.181752
  validation loss:		0.214001
  validation accuracy:		93.56 %
Train Batches  57
Val batches 12
Epoch 132 of 500 took 110.325s
  training loss:		0.181105
  validation loss:		0.212863
  validation accuracy:		93.36 %
Train Batches  57
Val batches 12
Epoch 133 of 500 took 118.778s
  training loss:		0.184801
  validation loss:		0.210637
  validation accuracy:		93.47 %
Train Batches  57
Val batches 12
Epoch 134 of 500 took 118.843s
  training loss:		0.173826
  validation loss:		0.212601
  validation accuracy:		93.56 %
Train Batches  57
Val batches 12
Epoch 135 of 500 took 118.804s
  training loss:		0.174254
  validation loss:		0.211649
  validation accuracy:		93.78 %
Train Batches  57
Val batches 12
Epoch 136 of 500 took 118.850s
  training loss:		0.168324
  validation loss:		0.210737
  validation accuracy:		93.69 %
Train Batches  57
Val batches 12
Epoch 137 of 500 took 98.033s
  training loss:		0.172386
  validation loss:		0.209975
  validation accuracy:		93.75 %
Train Batches  57
Val batches 12
Epoch 138 of 500 took 79.425s
  training loss:		0.175827
  validation loss:		0.211586
  validation accuracy:		93.56 %
Train Batches  57
Val batches 12
Epoch 139 of 500 took 79.369s
  training loss:		0.164464
  validation loss:		0.211644
  validation accuracy:		93.83 %
Train Batches  57
Val batches 12
Epoch 140 of 500 took 79.507s
  training loss:		0.165520
  validation loss:		0.211385
  validation accuracy:		93.69 %
Train Batches  57
Val batches 12
Epoch 141 of 500 took 79.558s
  training loss:		0.160496
  validation loss:		0.209290
  validation accuracy:		93.83 %
Train Batches  57
Val batches 12
Epoch 142 of 500 took 79.346s
  training loss:		0.174292
  validation loss:		0.209317
  validation accuracy:		93.72 %
Train Batches  57
Val batches 12
Epoch 143 of 500 took 82.044s
  training loss:		0.161107
  validation loss:		0.209426
  validation accuracy:		93.75 %
Train Batches  57
Val batches 12
Epoch 144 of 500 took 79.514s
  training loss:		0.162197
  validation loss:		0.207719
  validation accuracy:		93.97 %
Train Batches  57
Val batches 12
Epoch 145 of 500 took 83.092s
  training loss:		0.156616
  validation loss:		0.206080
  validation accuracy:		94.06 %
Train Batches  57
Val batches 12
Epoch 146 of 500 took 95.296s
  training loss:		0.165174
  validation loss:		0.208265
  validation accuracy:		93.86 %
Train Batches  57
Val batches 12
Epoch 147 of 500 took 95.320s
  training loss:		0.155722
  validation loss:		0.206591
  validation accuracy:		93.72 %
Train Batches  57
Val batches 12
Epoch 148 of 500 took 95.316s
  training loss:		0.159006
  validation loss:		0.207945
  validation accuracy:		93.86 %
Train Batches  57
Val batches 12
Epoch 149 of 500 took 95.328s
  training loss:		0.158340
  validation loss:		0.211595
  validation accuracy:		93.75 %
Train Batches  57
Val batches 12
Epoch 150 of 500 took 95.321s
  training loss:		0.152038
  validation loss:		0.210488
  validation accuracy:		93.83 %
Train Batches  57
Val batches 12
Epoch 151 of 500 took 95.427s
  training loss:		0.153465
  validation loss:		0.206859
  validation accuracy:		93.83 %
Train Batches  57
Val batches 12
Epoch 152 of 500 took 95.389s
  training loss:		0.163238
  validation loss:		0.206457
  validation accuracy:		93.64 %
Train Batches  57
Val batches 12
Epoch 153 of 500 took 95.534s
  training loss:		0.154132
  validation loss:		0.205054
  validation accuracy:		93.81 %
Train Batches  57
Val batches 12
Epoch 154 of 500 took 95.455s
  training loss:		0.152657
  validation loss:		0.204217
  validation accuracy:		94.03 %
Train Batches  57
Val batches 12
Epoch 155 of 500 took 95.407s
  training loss:		0.152322
  validation loss:		0.206866
  validation accuracy:		93.92 %
Train Batches  57
Val batches 12
Epoch 156 of 500 took 95.310s
  training loss:		0.149913
  validation loss:		0.204595
  validation accuracy:		93.94 %
Train Batches  57
Val batches 12
Epoch 157 of 500 took 95.484s
  training loss:		0.150634
  validation loss:		0.202926
  validation accuracy:		93.89 %
Train Batches  57
Val batches 12
Epoch 158 of 500 took 95.375s
  training loss:		0.146337
  validation loss:		0.204805
  validation accuracy:		93.97 %
Train Batches  57
Val batches 12
Epoch 159 of 500 took 95.361s
  training loss:		0.147129
  validation loss:		0.202848
  validation accuracy:		93.89 %
Train Batches  57
Val batches 12
Epoch 160 of 500 took 95.356s
  training loss:		0.146607
  validation loss:		0.203456
  validation accuracy:		93.78 %
Train Batches  57
Val batches 12
Epoch 161 of 500 took 95.421s
  training loss:		0.149097
  validation loss:		0.202620
  validation accuracy:		94.00 %
Train Batches  57
Val batches 12
Epoch 162 of 500 took 95.376s
  training loss:		0.150011
  validation loss:		0.201177
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 163 of 500 took 95.363s
  training loss:		0.136721
  validation loss:		0.200479
  validation accuracy:		94.06 %
Train Batches  57
Val batches 12
Epoch 164 of 500 took 95.413s
  training loss:		0.146865
  validation loss:		0.203737
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 165 of 500 took 95.374s
  training loss:		0.142065
  validation loss:		0.200635
  validation accuracy:		94.06 %
Train Batches  57
Val batches 12
Epoch 166 of 500 took 95.403s
  training loss:		0.136600
  validation loss:		0.202140
  validation accuracy:		94.17 %
Train Batches  57
Val batches 12
Epoch 167 of 500 took 95.401s
  training loss:		0.138512
  validation loss:		0.202943
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 168 of 500 took 95.413s
  training loss:		0.141686
  validation loss:		0.200661
  validation accuracy:		94.33 %
Train Batches  57
Val batches 12
Epoch 169 of 500 took 95.464s
  training loss:		0.141708
  validation loss:		0.202342
  validation accuracy:		93.92 %
Train Batches  57
Val batches 12
Epoch 170 of 500 took 95.403s
  training loss:		0.139972
  validation loss:		0.200679
  validation accuracy:		93.94 %
Train Batches  57
Val batches 12
Epoch 171 of 500 took 95.442s
  training loss:		0.139629
  validation loss:		0.199815
  validation accuracy:		93.94 %
Train Batches  57
Val batches 12
Epoch 172 of 500 took 95.492s
  training loss:		0.133428
  validation loss:		0.201518
  validation accuracy:		93.89 %
Train Batches  57
Val batches 12
Epoch 173 of 500 took 95.405s
  training loss:		0.136718
  validation loss:		0.200390
  validation accuracy:		94.03 %
Train Batches  57
Val batches 12
Epoch 174 of 500 took 95.399s
  training loss:		0.136084
  validation loss:		0.199910
  validation accuracy:		94.11 %
Train Batches  57
Val batches 12
Epoch 175 of 500 took 95.312s
  training loss:		0.133985
  validation loss:		0.202316
  validation accuracy:		94.03 %
Train Batches  57
Val batches 12
Epoch 176 of 500 took 95.364s
  training loss:		0.130515
  validation loss:		0.200479
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 177 of 500 took 95.305s
  training loss:		0.130006
  validation loss:		0.202640
  validation accuracy:		93.94 %
Train Batches  57
Val batches 12
Epoch 178 of 500 took 95.313s
  training loss:		0.124927
  validation loss:		0.202260
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 179 of 500 took 95.275s
  training loss:		0.131192
  validation loss:		0.201008
  validation accuracy:		94.14 %
Train Batches  57
Val batches 12
Epoch 180 of 500 took 95.308s
  training loss:		0.132605
  validation loss:		0.204304
  validation accuracy:		94.00 %
Train Batches  57
Val batches 12
Epoch 181 of 500 took 95.384s
  training loss:		0.129883
  validation loss:		0.200217
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 182 of 500 took 95.281s
  training loss:		0.129965
  validation loss:		0.199501
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 183 of 500 took 95.269s
  training loss:		0.125141
  validation loss:		0.201151
  validation accuracy:		94.17 %
Train Batches  57
Val batches 12
Epoch 184 of 500 took 95.283s
  training loss:		0.126806
  validation loss:		0.198738
  validation accuracy:		94.11 %
Train Batches  57
Val batches 12
Epoch 185 of 500 took 87.756s
  training loss:		0.128362
  validation loss:		0.197619
  validation accuracy:		94.17 %
Train Batches  57
Val batches 12
Epoch 186 of 500 took 89.890s
  training loss:		0.123783
  validation loss:		0.198266
  validation accuracy:		94.19 %
Train Batches  57
Val batches 12
Epoch 187 of 500 took 106.941s
  training loss:		0.129859
  validation loss:		0.197350
  validation accuracy:		94.14 %
Train Batches  57
Val batches 12
Epoch 188 of 500 took 94.342s
  training loss:		0.124416
  validation loss:		0.199453
  validation accuracy:		94.06 %
Train Batches  57
Val batches 12
Epoch 189 of 500 took 100.916s
  training loss:		0.120542
  validation loss:		0.198048
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 190 of 500 took 94.125s
  training loss:		0.123882
  validation loss:		0.197935
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 191 of 500 took 88.031s
  training loss:		0.120593
  validation loss:		0.195866
  validation accuracy:		94.14 %
Train Batches  57
Val batches 12
Epoch 192 of 500 took 81.124s
  training loss:		0.113717
  validation loss:		0.197408
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 193 of 500 took 81.326s
  training loss:		0.121314
  validation loss:		0.195647
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 194 of 500 took 92.065s
  training loss:		0.118204
  validation loss:		0.197285
  validation accuracy:		93.97 %
Train Batches  57
Val batches 12
Epoch 195 of 500 took 81.352s
  training loss:		0.119929
  validation loss:		0.194619
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 196 of 500 took 81.606s
  training loss:		0.114947
  validation loss:		0.195316
  validation accuracy:		94.17 %
Train Batches  57
Val batches 12
Epoch 197 of 500 took 81.317s
  training loss:		0.117992
  validation loss:		0.195636
  validation accuracy:		94.17 %
Train Batches  57
Val batches 12
Epoch 198 of 500 took 89.137s
  training loss:		0.112341
  validation loss:		0.198016
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 199 of 500 took 81.719s
  training loss:		0.116631
  validation loss:		0.197143
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 200 of 500 took 81.647s
  training loss:		0.120721
  validation loss:		0.194792
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 201 of 500 took 90.598s
  training loss:		0.115147
  validation loss:		0.193272
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 202 of 500 took 86.757s
  training loss:		0.111250
  validation loss:		0.194338
  validation accuracy:		94.28 %
Train Batches  57
Val batches 12
Epoch 203 of 500 took 81.622s
  training loss:		0.114788
  validation loss:		0.193853
  validation accuracy:		94.08 %
Train Batches  57
Val batches 12
Epoch 204 of 500 took 81.561s
  training loss:		0.108321
  validation loss:		0.194840
  validation accuracy:		93.97 %
Train Batches  57
Val batches 12
Epoch 205 of 500 took 81.506s
  training loss:		0.108533
  validation loss:		0.195307
  validation accuracy:		94.14 %
Train Batches  57
Val batches 12
Epoch 206 of 500 took 81.649s
  training loss:		0.109647
  validation loss:		0.196629
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 207 of 500 took 81.496s
  training loss:		0.113493
  validation loss:		0.193933
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 208 of 500 took 81.394s
  training loss:		0.109884
  validation loss:		0.192883
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 209 of 500 took 98.719s
  training loss:		0.106745
  validation loss:		0.195722
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 210 of 500 took 81.610s
  training loss:		0.109585
  validation loss:		0.198455
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 211 of 500 took 88.683s
  training loss:		0.116371
  validation loss:		0.192360
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 212 of 500 took 95.334s
  training loss:		0.108469
  validation loss:		0.193786
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 213 of 500 took 95.352s
  training loss:		0.110881
  validation loss:		0.195613
  validation accuracy:		94.03 %
Train Batches  57
Val batches 12
Epoch 214 of 500 took 95.355s
  training loss:		0.109274
  validation loss:		0.190259
  validation accuracy:		94.44 %
Train Batches  57
Val batches 12
Epoch 215 of 500 took 95.353s
  training loss:		0.103668
  validation loss:		0.192675
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 216 of 500 took 95.383s
  training loss:		0.108395
  validation loss:		0.193530
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 217 of 500 took 95.380s
  training loss:		0.107603
  validation loss:		0.194587
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 218 of 500 took 95.385s
  training loss:		0.105091
  validation loss:		0.193301
  validation accuracy:		94.11 %
Train Batches  57
Val batches 12
Epoch 219 of 500 took 95.406s
  training loss:		0.107839
  validation loss:		0.195288
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 220 of 500 took 95.453s
  training loss:		0.098240
  validation loss:		0.194316
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 221 of 500 took 95.393s
  training loss:		0.104238
  validation loss:		0.195418
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 222 of 500 took 95.447s
  training loss:		0.097919
  validation loss:		0.194175
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 223 of 500 took 95.373s
  training loss:		0.107303
  validation loss:		0.195577
  validation accuracy:		94.33 %
Train Batches  57
Val batches 12
Epoch 224 of 500 took 95.438s
  training loss:		0.099251
  validation loss:		0.193850
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 225 of 500 took 95.358s
  training loss:		0.102498
  validation loss:		0.198388
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 226 of 500 took 95.410s
  training loss:		0.098919
  validation loss:		0.193739
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 227 of 500 took 95.444s
  training loss:		0.101921
  validation loss:		0.197744
  validation accuracy:		94.33 %
Train Batches  57
Val batches 12
Epoch 228 of 500 took 95.368s
  training loss:		0.104932
  validation loss:		0.192531
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 229 of 500 took 95.373s
  training loss:		0.099995
  validation loss:		0.196214
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 230 of 500 took 95.411s
  training loss:		0.097812
  validation loss:		0.194707
  validation accuracy:		94.25 %
Train Batches  57
Val batches 12
Epoch 231 of 500 took 95.450s
  training loss:		0.096456
  validation loss:		0.196362
  validation accuracy:		94.36 %
Train Batches  57
Val batches 12
Epoch 232 of 500 took 95.431s
  training loss:		0.097620
  validation loss:		0.194669
  validation accuracy:		94.44 %
Train Batches  57
Val batches 12
Epoch 233 of 500 took 95.464s
  training loss:		0.092762
  validation loss:		0.194926
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 234 of 500 took 95.478s
  training loss:		0.096463
  validation loss:		0.195318
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 235 of 500 took 95.454s
  training loss:		0.094711
  validation loss:		0.196029
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 236 of 500 took 95.439s
  training loss:		0.094109
  validation loss:		0.195722
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 237 of 500 took 95.468s
  training loss:		0.095315
  validation loss:		0.196743
  validation accuracy:		94.17 %
Train Batches  57
Val batches 12
Epoch 238 of 500 took 95.446s
  training loss:		0.100715
  validation loss:		0.194204
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 239 of 500 took 95.436s
  training loss:		0.095308
  validation loss:		0.195695
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 240 of 500 took 95.454s
  training loss:		0.094999
  validation loss:		0.195233
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 241 of 500 took 95.445s
  training loss:		0.089222
  validation loss:		0.195766
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 242 of 500 took 95.482s
  training loss:		0.091173
  validation loss:		0.195417
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 243 of 500 took 95.430s
  training loss:		0.094168
  validation loss:		0.194820
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 244 of 500 took 95.421s
  training loss:		0.093032
  validation loss:		0.194397
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 245 of 500 took 95.446s
  training loss:		0.091347
  validation loss:		0.192317
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 246 of 500 took 95.475s
  training loss:		0.096685
  validation loss:		0.193604
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 247 of 500 took 95.483s
  training loss:		0.091481
  validation loss:		0.193378
  validation accuracy:		94.44 %
Train Batches  57
Val batches 12
Epoch 248 of 500 took 95.442s
  training loss:		0.087614
  validation loss:		0.193774
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 249 of 500 took 95.450s
  training loss:		0.092300
  validation loss:		0.195376
  validation accuracy:		94.47 %
Train Batches  57
Val batches 12
Epoch 250 of 500 took 95.424s
  training loss:		0.090465
  validation loss:		0.194817
  validation accuracy:		94.19 %
Train Batches  57
Val batches 12
Epoch 251 of 500 took 95.421s
  training loss:		0.088512
  validation loss:		0.192013
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 252 of 500 took 95.468s
  training loss:		0.094723
  validation loss:		0.193691
  validation accuracy:		94.44 %
Train Batches  57
Val batches 12
Epoch 253 of 500 took 95.469s
  training loss:		0.086458
  validation loss:		0.195919
  validation accuracy:		94.36 %
Train Batches  57
Val batches 12
Epoch 254 of 500 took 95.416s
  training loss:		0.087685
  validation loss:		0.192922
  validation accuracy:		94.22 %
Train Batches  57
Val batches 12
Epoch 255 of 500 took 97.253s
  training loss:		0.087734
  validation loss:		0.196454
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 256 of 500 took 101.865s
  training loss:		0.085232
  validation loss:		0.197200
  validation accuracy:		94.47 %
Train Batches  57
Val batches 12
Epoch 257 of 500 took 98.103s
  training loss:		0.084699
  validation loss:		0.194478
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 258 of 500 took 98.080s
  training loss:		0.088897
  validation loss:		0.193600
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 259 of 500 took 98.054s
  training loss:		0.087343
  validation loss:		0.197889
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 260 of 500 took 98.042s
  training loss:		0.085440
  validation loss:		0.194823
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 261 of 500 took 98.043s
  training loss:		0.083495
  validation loss:		0.195848
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 262 of 500 took 98.061s
  training loss:		0.083699
  validation loss:		0.196443
  validation accuracy:		94.31 %
Train Batches  57
Val batches 12
Epoch 263 of 500 took 98.005s
  training loss:		0.082944
  validation loss:		0.194111
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 264 of 500 took 98.030s
  training loss:		0.083780
  validation loss:		0.194377
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 265 of 500 took 98.072s
  training loss:		0.086257
  validation loss:		0.192818
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 266 of 500 took 98.098s
  training loss:		0.080933
  validation loss:		0.192138
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 267 of 500 took 98.171s
  training loss:		0.085133
  validation loss:		0.195858
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 268 of 500 took 98.485s
  training loss:		0.083867
  validation loss:		0.194895
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 269 of 500 took 98.088s
  training loss:		0.086758
  validation loss:		0.192785
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 270 of 500 took 98.022s
  training loss:		0.086007
  validation loss:		0.192258
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 271 of 500 took 98.142s
  training loss:		0.085019
  validation loss:		0.193772
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 272 of 500 took 98.057s
  training loss:		0.081943
  validation loss:		0.191618
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 273 of 500 took 98.011s
  training loss:		0.087437
  validation loss:		0.193467
  validation accuracy:		94.42 %
Train Batches  57
Val batches 12
Epoch 274 of 500 took 95.920s
  training loss:		0.080905
  validation loss:		0.193765
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 275 of 500 took 95.339s
  training loss:		0.078704
  validation loss:		0.195384
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 276 of 500 took 95.366s
  training loss:		0.079119
  validation loss:		0.195861
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 277 of 500 took 95.356s
  training loss:		0.077247
  validation loss:		0.194332
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 278 of 500 took 95.348s
  training loss:		0.080859
  validation loss:		0.195867
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 279 of 500 took 95.379s
  training loss:		0.082164
  validation loss:		0.196851
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 280 of 500 took 95.327s
  training loss:		0.079006
  validation loss:		0.194576
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 281 of 500 took 95.315s
  training loss:		0.079189
  validation loss:		0.193270
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 282 of 500 took 95.383s
  training loss:		0.077977
  validation loss:		0.194174
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 283 of 500 took 95.393s
  training loss:		0.077543
  validation loss:		0.192754
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 284 of 500 took 95.363s
  training loss:		0.076564
  validation loss:		0.195174
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 285 of 500 took 95.377s
  training loss:		0.076272
  validation loss:		0.195494
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 286 of 500 took 95.345s
  training loss:		0.077326
  validation loss:		0.195014
  validation accuracy:		94.47 %
Train Batches  57
Val batches 12
Epoch 287 of 500 took 95.366s
  training loss:		0.078852
  validation loss:		0.192826
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 288 of 500 took 95.377s
  training loss:		0.074527
  validation loss:		0.197607
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 289 of 500 took 95.413s
  training loss:		0.071912
  validation loss:		0.198663
  validation accuracy:		94.39 %
Train Batches  57
Val batches 12
Epoch 290 of 500 took 95.373s
  training loss:		0.074599
  validation loss:		0.194831
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 291 of 500 took 95.378s
  training loss:		0.073452
  validation loss:		0.200773
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 292 of 500 took 95.375s
  training loss:		0.072489
  validation loss:		0.194465
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 293 of 500 took 95.419s
  training loss:		0.081962
  validation loss:		0.192497
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 294 of 500 took 95.396s
  training loss:		0.079526
  validation loss:		0.196914
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 295 of 500 took 95.318s
  training loss:		0.066959
  validation loss:		0.194006
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 296 of 500 took 95.347s
  training loss:		0.076943
  validation loss:		0.194164
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 297 of 500 took 95.342s
  training loss:		0.070951
  validation loss:		0.195945
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 298 of 500 took 95.331s
  training loss:		0.072360
  validation loss:		0.195442
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 299 of 500 took 95.325s
  training loss:		0.074942
  validation loss:		0.197389
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 300 of 500 took 95.330s
  training loss:		0.075575
  validation loss:		0.198750
  validation accuracy:		94.50 %
Train Batches  57
Val batches 12
Epoch 301 of 500 took 95.306s
  training loss:		0.073082
  validation loss:		0.194800
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 302 of 500 took 95.452s
  training loss:		0.075805
  validation loss:		0.197400
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 303 of 500 took 95.303s
  training loss:		0.074692
  validation loss:		0.195981
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 304 of 500 took 95.416s
  training loss:		0.067997
  validation loss:		0.194699
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 305 of 500 took 95.335s
  training loss:		0.071375
  validation loss:		0.196462
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 306 of 500 took 95.292s
  training loss:		0.071089
  validation loss:		0.197499
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 307 of 500 took 95.291s
  training loss:		0.073333
  validation loss:		0.195308
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 308 of 500 took 95.275s
  training loss:		0.071841
  validation loss:		0.195032
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 309 of 500 took 95.313s
  training loss:		0.073914
  validation loss:		0.199265
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 310 of 500 took 95.318s
  training loss:		0.070382
  validation loss:		0.196064
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 311 of 500 took 95.303s
  training loss:		0.068466
  validation loss:		0.198195
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 312 of 500 took 95.331s
  training loss:		0.068888
  validation loss:		0.197346
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 313 of 500 took 95.254s
  training loss:		0.067339
  validation loss:		0.198002
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 314 of 500 took 95.335s
  training loss:		0.072657
  validation loss:		0.198502
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 315 of 500 took 95.310s
  training loss:		0.070115
  validation loss:		0.197248
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 316 of 500 took 95.321s
  training loss:		0.070078
  validation loss:		0.195980
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 317 of 500 took 95.317s
  training loss:		0.066764
  validation loss:		0.197210
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 318 of 500 took 95.315s
  training loss:		0.068951
  validation loss:		0.198249
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 319 of 500 took 95.335s
  training loss:		0.063472
  validation loss:		0.195000
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 320 of 500 took 96.479s
  training loss:		0.068787
  validation loss:		0.197323
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 321 of 500 took 96.532s
  training loss:		0.069135
  validation loss:		0.196163
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 322 of 500 took 96.587s
  training loss:		0.064766
  validation loss:		0.195954
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 323 of 500 took 96.614s
  training loss:		0.067834
  validation loss:		0.197305
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 324 of 500 took 96.567s
  training loss:		0.070916
  validation loss:		0.196858
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 325 of 500 took 96.586s
  training loss:		0.069576
  validation loss:		0.197763
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 326 of 500 took 96.781s
  training loss:		0.067128
  validation loss:		0.195976
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 327 of 500 took 96.764s
  training loss:		0.066297
  validation loss:		0.198074
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 328 of 500 took 96.678s
  training loss:		0.065090
  validation loss:		0.197720
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 329 of 500 took 96.772s
  training loss:		0.069581
  validation loss:		0.194922
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 330 of 500 took 96.745s
  training loss:		0.066414
  validation loss:		0.199427
  validation accuracy:		94.53 %
Train Batches  57
Val batches 12
Epoch 331 of 500 took 96.654s
  training loss:		0.063037
  validation loss:		0.201204
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 332 of 500 took 96.802s
  training loss:		0.067347
  validation loss:		0.197175
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 333 of 500 took 96.858s
  training loss:		0.067020
  validation loss:		0.199878
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 334 of 500 took 96.694s
  training loss:		0.066675
  validation loss:		0.198347
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 335 of 500 took 96.624s
  training loss:		0.065019
  validation loss:		0.197487
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 336 of 500 took 96.583s
  training loss:		0.064408
  validation loss:		0.195337
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 337 of 500 took 96.609s
  training loss:		0.060544
  validation loss:		0.198126
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 338 of 500 took 96.594s
  training loss:		0.064427
  validation loss:		0.198661
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 339 of 500 took 96.575s
  training loss:		0.067999
  validation loss:		0.197028
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 340 of 500 took 96.536s
  training loss:		0.059771
  validation loss:		0.195897
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 341 of 500 took 96.566s
  training loss:		0.064557
  validation loss:		0.195756
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 342 of 500 took 96.790s
  training loss:		0.064844
  validation loss:		0.195442
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 343 of 500 took 96.665s
  training loss:		0.062938
  validation loss:		0.196201
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 344 of 500 took 96.558s
  training loss:		0.058765
  validation loss:		0.196885
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 345 of 500 took 96.648s
  training loss:		0.058698
  validation loss:		0.199226
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 346 of 500 took 96.633s
  training loss:		0.062287
  validation loss:		0.197954
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 347 of 500 took 96.691s
  training loss:		0.058524
  validation loss:		0.196911
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 348 of 500 took 96.653s
  training loss:		0.062030
  validation loss:		0.197317
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 349 of 500 took 96.599s
  training loss:		0.061198
  validation loss:		0.196599
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 350 of 500 took 96.646s
  training loss:		0.060727
  validation loss:		0.193247
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 351 of 500 took 96.641s
  training loss:		0.057781
  validation loss:		0.194710
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 352 of 500 took 96.639s
  training loss:		0.061517
  validation loss:		0.197552
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 353 of 500 took 96.580s
  training loss:		0.062853
  validation loss:		0.197699
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 354 of 500 took 96.618s
  training loss:		0.060573
  validation loss:		0.194796
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 355 of 500 took 96.617s
  training loss:		0.057517
  validation loss:		0.195866
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 356 of 500 took 96.638s
  training loss:		0.059041
  validation loss:		0.197989
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 357 of 500 took 96.634s
  training loss:		0.059234
  validation loss:		0.196511
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 358 of 500 took 96.582s
  training loss:		0.058966
  validation loss:		0.194573
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 359 of 500 took 96.647s
  training loss:		0.059823
  validation loss:		0.198409
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 360 of 500 took 96.610s
  training loss:		0.060066
  validation loss:		0.194727
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 361 of 500 took 96.646s
  training loss:		0.057639
  validation loss:		0.195489
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 362 of 500 took 96.607s
  training loss:		0.057216
  validation loss:		0.195509
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 363 of 500 took 96.601s
  training loss:		0.057495
  validation loss:		0.197449
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 364 of 500 took 96.613s
  training loss:		0.057676
  validation loss:		0.196511
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 365 of 500 took 96.627s
  training loss:		0.056527
  validation loss:		0.199498
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 366 of 500 took 96.622s
  training loss:		0.055786
  validation loss:		0.196853
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 367 of 500 took 96.616s
  training loss:		0.056845
  validation loss:		0.194269
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 368 of 500 took 80.371s
  training loss:		0.056928
  validation loss:		0.196187
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 369 of 500 took 79.425s
  training loss:		0.055386
  validation loss:		0.196657
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 370 of 500 took 79.466s
  training loss:		0.055067
  validation loss:		0.192760
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 371 of 500 took 79.400s
  training loss:		0.055087
  validation loss:		0.195636
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 372 of 500 took 79.378s
  training loss:		0.060771
  validation loss:		0.193961
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 373 of 500 took 79.360s
  training loss:		0.053122
  validation loss:		0.194754
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 374 of 500 took 117.594s
  training loss:		0.061318
  validation loss:		0.196477
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 375 of 500 took 117.694s
  training loss:		0.059552
  validation loss:		0.195481
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 376 of 500 took 89.782s
  training loss:		0.057664
  validation loss:		0.191815
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 377 of 500 took 82.071s
  training loss:		0.057347
  validation loss:		0.195500
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 378 of 500 took 117.486s
  training loss:		0.055481
  validation loss:		0.197890
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 379 of 500 took 117.383s
  training loss:		0.058827
  validation loss:		0.195788
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 380 of 500 took 117.363s
  training loss:		0.057370
  validation loss:		0.194527
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 381 of 500 took 117.395s
  training loss:		0.056740
  validation loss:		0.192262
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 382 of 500 took 117.370s
  training loss:		0.054682
  validation loss:		0.196959
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 383 of 500 took 87.797s
  training loss:		0.055467
  validation loss:		0.195688
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 384 of 500 took 79.383s
  training loss:		0.056816
  validation loss:		0.195417
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 385 of 500 took 79.399s
  training loss:		0.058031
  validation loss:		0.195895
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 386 of 500 took 79.364s
  training loss:		0.057139
  validation loss:		0.195688
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 387 of 500 took 79.415s
  training loss:		0.058834
  validation loss:		0.195499
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 388 of 500 took 79.479s
  training loss:		0.057510
  validation loss:		0.197097
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 389 of 500 took 79.453s
  training loss:		0.052638
  validation loss:		0.199001
  validation accuracy:		94.56 %
Train Batches  57
Val batches 12
Epoch 390 of 500 took 79.419s
  training loss:		0.053871
  validation loss:		0.197114
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 391 of 500 took 79.441s
  training loss:		0.060300
  validation loss:		0.193509
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 392 of 500 took 79.484s
  training loss:		0.054070
  validation loss:		0.199425
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 393 of 500 took 79.403s
  training loss:		0.053723
  validation loss:		0.194074
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 394 of 500 took 97.024s
  training loss:		0.052151
  validation loss:		0.194336
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 395 of 500 took 117.685s
  training loss:		0.056288
  validation loss:		0.196981
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 396 of 500 took 117.689s
  training loss:		0.052411
  validation loss:		0.198099
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 397 of 500 took 117.657s
  training loss:		0.052929
  validation loss:		0.199888
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 398 of 500 took 117.743s
  training loss:		0.054210
  validation loss:		0.196505
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 399 of 500 took 117.683s
  training loss:		0.052435
  validation loss:		0.198433
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 400 of 500 took 117.718s
  training loss:		0.056195
  validation loss:		0.195974
  validation accuracy:		95.08 %
Train Batches  57
Val batches 12
Epoch 401 of 500 took 117.720s
  training loss:		0.050604
  validation loss:		0.195567
  validation accuracy:		95.06 %
Train Batches  57
Val batches 12
Epoch 402 of 500 took 89.341s
  training loss:		0.056170
  validation loss:		0.196609
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 403 of 500 took 79.493s
  training loss:		0.055078
  validation loss:		0.197302
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 404 of 500 took 79.377s
  training loss:		0.053250
  validation loss:		0.192937
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 405 of 500 took 79.482s
  training loss:		0.055573
  validation loss:		0.193808
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 406 of 500 took 79.398s
  training loss:		0.053473
  validation loss:		0.195704
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 407 of 500 took 79.533s
  training loss:		0.049639
  validation loss:		0.195994
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 408 of 500 took 79.452s
  training loss:		0.053843
  validation loss:		0.195515
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 409 of 500 took 79.541s
  training loss:		0.051810
  validation loss:		0.194029
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 410 of 500 took 111.676s
  training loss:		0.052033
  validation loss:		0.195812
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 411 of 500 took 85.689s
  training loss:		0.050092
  validation loss:		0.196717
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 412 of 500 took 79.467s
  training loss:		0.050363
  validation loss:		0.194973
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 413 of 500 took 79.397s
  training loss:		0.050930
  validation loss:		0.195414
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 414 of 500 took 79.448s
  training loss:		0.055173
  validation loss:		0.200617
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 415 of 500 took 79.492s
  training loss:		0.050584
  validation loss:		0.198292
  validation accuracy:		94.97 %
Train Batches  57
Val batches 12
Epoch 416 of 500 took 79.685s
  training loss:		0.052265
  validation loss:		0.194580
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 417 of 500 took 95.370s
  training loss:		0.051760
  validation loss:		0.198092
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 418 of 500 took 95.340s
  training loss:		0.048281
  validation loss:		0.198154
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 419 of 500 took 95.385s
  training loss:		0.047284
  validation loss:		0.194622
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 420 of 500 took 95.445s
  training loss:		0.050642
  validation loss:		0.198629
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 421 of 500 took 95.421s
  training loss:		0.050354
  validation loss:		0.197493
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 422 of 500 took 95.461s
  training loss:		0.048093
  validation loss:		0.197017
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 423 of 500 took 95.394s
  training loss:		0.047748
  validation loss:		0.198678
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 424 of 500 took 95.413s
  training loss:		0.053111
  validation loss:		0.198457
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 425 of 500 took 115.374s
  training loss:		0.051336
  validation loss:		0.199316
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 426 of 500 took 117.989s
  training loss:		0.048375
  validation loss:		0.197412
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 427 of 500 took 97.496s
  training loss:		0.051248
  validation loss:		0.197727
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 428 of 500 took 109.398s
  training loss:		0.050542
  validation loss:		0.198347
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 429 of 500 took 97.282s
  training loss:		0.050952
  validation loss:		0.197004
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 430 of 500 took 97.222s
  training loss:		0.049410
  validation loss:		0.199687
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 431 of 500 took 97.172s
  training loss:		0.049966
  validation loss:		0.199479
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 432 of 500 took 98.519s
  training loss:		0.052185
  validation loss:		0.198668
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 433 of 500 took 98.497s
  training loss:		0.050794
  validation loss:		0.201764
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 434 of 500 took 95.925s
  training loss:		0.048914
  validation loss:		0.197835
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 435 of 500 took 95.891s
  training loss:		0.044758
  validation loss:		0.197683
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 436 of 500 took 103.509s
  training loss:		0.048477
  validation loss:		0.200061
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 437 of 500 took 79.533s
  training loss:		0.048760
  validation loss:		0.200462
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 438 of 500 took 79.579s
  training loss:		0.045325
  validation loss:		0.198835
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 439 of 500 took 118.288s
  training loss:		0.050281
  validation loss:		0.199170
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 440 of 500 took 118.217s
  training loss:		0.051301
  validation loss:		0.199280
  validation accuracy:		94.67 %
Train Batches  57
Val batches 12
Epoch 441 of 500 took 119.622s
  training loss:		0.051678
  validation loss:		0.200141
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 442 of 500 took 90.006s
  training loss:		0.045051
  validation loss:		0.197640
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 443 of 500 took 79.660s
  training loss:		0.046938
  validation loss:		0.199533
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 444 of 500 took 79.661s
  training loss:		0.048679
  validation loss:		0.197820
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 445 of 500 took 79.639s
  training loss:		0.046448
  validation loss:		0.198849
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 446 of 500 took 79.678s
  training loss:		0.046646
  validation loss:		0.197586
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 447 of 500 took 119.409s
  training loss:		0.046001
  validation loss:		0.199832
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 448 of 500 took 119.505s
  training loss:		0.046611
  validation loss:		0.199498
  validation accuracy:		94.58 %
Train Batches  57
Val batches 12
Epoch 449 of 500 took 119.465s
  training loss:		0.049808
  validation loss:		0.199023
  validation accuracy:		94.64 %
Train Batches  57
Val batches 12
Epoch 450 of 500 took 118.150s
  training loss:		0.043956
  validation loss:		0.200460
  validation accuracy:		94.61 %
Train Batches  57
Val batches 12
Epoch 451 of 500 took 99.213s
  training loss:		0.047170
  validation loss:		0.197638
  validation accuracy:		94.72 %
Train Batches  57
Val batches 12
Epoch 452 of 500 took 118.117s
  training loss:		0.047385
  validation loss:		0.200907
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 453 of 500 took 82.180s
  training loss:		0.049617
  validation loss:		0.196259
  validation accuracy:		94.78 %
Train Batches  57
Val batches 12
Epoch 454 of 500 took 97.146s
  training loss:		0.046139
  validation loss:		0.200764
  validation accuracy:		94.69 %
Train Batches  57
Val batches 12
Epoch 455 of 500 took 97.190s
  training loss:		0.044711
  validation loss:		0.199965
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 456 of 500 took 97.157s
  training loss:		0.043996
  validation loss:		0.199495
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 457 of 500 took 97.213s
  training loss:		0.045989
  validation loss:		0.200085
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 458 of 500 took 97.167s
  training loss:		0.045545
  validation loss:		0.197456
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 459 of 500 took 95.773s
  training loss:		0.048293
  validation loss:		0.201824
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 460 of 500 took 79.800s
  training loss:		0.046968
  validation loss:		0.200358
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 461 of 500 took 79.566s
  training loss:		0.043100
  validation loss:		0.200757
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 462 of 500 took 79.676s
  training loss:		0.044746
  validation loss:		0.200232
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 463 of 500 took 119.327s
  training loss:		0.046825
  validation loss:		0.201226
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 464 of 500 took 80.737s
  training loss:		0.045237
  validation loss:		0.199684
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 465 of 500 took 79.713s
  training loss:		0.045339
  validation loss:		0.198098
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 466 of 500 took 97.274s
  training loss:		0.046962
  validation loss:		0.197558
  validation accuracy:		94.94 %
Train Batches  57
Val batches 12
Epoch 467 of 500 took 97.157s
  training loss:		0.045388
  validation loss:		0.198306
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 468 of 500 took 97.153s
  training loss:		0.047112
  validation loss:		0.199271
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 469 of 500 took 97.175s
  training loss:		0.044642
  validation loss:		0.201970
  validation accuracy:		94.86 %
Train Batches  57
Val batches 12
Epoch 470 of 500 took 97.187s
  training loss:		0.048433
  validation loss:		0.200103
  validation accuracy:		95.17 %
Train Batches  57
Val batches 12
Epoch 471 of 500 took 97.890s
  training loss:		0.046216
  validation loss:		0.198901
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 472 of 500 took 97.206s
  training loss:		0.044612
  validation loss:		0.196926
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 473 of 500 took 97.339s
  training loss:		0.045859
  validation loss:		0.199809
  validation accuracy:		94.75 %
Train Batches  57
Val batches 12
Epoch 474 of 500 took 97.329s
  training loss:		0.045313
  validation loss:		0.202338
  validation accuracy:		94.83 %
Train Batches  57
Val batches 12
Epoch 475 of 500 took 97.221s
  training loss:		0.040038
  validation loss:		0.200950
  validation accuracy:		95.06 %
Train Batches  57
Val batches 12
Epoch 476 of 500 took 97.183s
  training loss:		0.046125
  validation loss:		0.200588
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 477 of 500 took 97.125s
  training loss:		0.045873
  validation loss:		0.201296
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 478 of 500 took 97.173s
  training loss:		0.045354
  validation loss:		0.201426
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 479 of 500 took 97.204s
  training loss:		0.043450
  validation loss:		0.200770
  validation accuracy:		95.11 %
Train Batches  57
Val batches 12
Epoch 480 of 500 took 97.168s
  training loss:		0.042639
  validation loss:		0.201986
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 481 of 500 took 97.171s
  training loss:		0.043459
  validation loss:		0.205269
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 482 of 500 took 97.164s
  training loss:		0.041353
  validation loss:		0.204676
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 483 of 500 took 97.131s
  training loss:		0.041198
  validation loss:		0.204937
  validation accuracy:		94.81 %
Train Batches  57
Val batches 12
Epoch 484 of 500 took 96.629s
  training loss:		0.042474
  validation loss:		0.200512
  validation accuracy:		94.89 %
Train Batches  57
Val batches 12
Epoch 485 of 500 took 96.520s
  training loss:		0.045640
  validation loss:		0.200822
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 486 of 500 took 118.785s
  training loss:		0.043462
  validation loss:		0.201091
  validation accuracy:		95.08 %
Train Batches  57
Val batches 12
Epoch 487 of 500 took 103.540s
  training loss:		0.043721
  validation loss:		0.202282
  validation accuracy:		94.92 %
Train Batches  57
Val batches 12
Epoch 488 of 500 took 97.954s
  training loss:		0.041942
  validation loss:		0.202868
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 489 of 500 took 117.449s
  training loss:		0.042886
  validation loss:		0.205222
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 490 of 500 took 117.553s
  training loss:		0.040955
  validation loss:		0.205183
  validation accuracy:		95.11 %
Train Batches  57
Val batches 12
Epoch 491 of 500 took 88.559s
  training loss:		0.039848
  validation loss:		0.205305
  validation accuracy:		95.08 %
Train Batches  57
Val batches 12
Epoch 492 of 500 took 106.542s
  training loss:		0.041428
  validation loss:		0.204769
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 493 of 500 took 117.441s
  training loss:		0.040954
  validation loss:		0.205377
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 494 of 500 took 117.474s
  training loss:		0.041768
  validation loss:		0.204337
  validation accuracy:		95.08 %
Train Batches  57
Val batches 12
Epoch 495 of 500 took 118.868s
  training loss:		0.045026
  validation loss:		0.205002
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 496 of 500 took 118.832s
  training loss:		0.040122
  validation loss:		0.207104
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 497 of 500 took 118.826s
  training loss:		0.043581
  validation loss:		0.207430
  validation accuracy:		95.00 %
Train Batches  57
Val batches 12
Epoch 498 of 500 took 118.830s
  training loss:		0.042708
  validation loss:		0.208112
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Epoch 499 of 500 took 118.832s
  training loss:		0.044749
  validation loss:		0.206631
  validation accuracy:		95.11 %
Train Batches  57
Val batches 12
Epoch 500 of 500 took 118.930s
  training loss:		0.041753
  validation loss:		0.205096
  validation accuracy:		95.03 %
Train Batches  57
Val batches 12
Final results:
  test loss:			0.183397
  test accuracy:		94.86 %

/home/licet/anaconda2/envs/TC/lib/python2.7/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
  "downsample module has been moved to the theano.tensor.signal.pool module.")

Network Architecture

# Print the Layer Structure 
import lasagne
import theano 

lyrs = lasagne.layers.get_all_layers(model)

print "Neural Network Architecture\n"

for i in lyrs:
    print i
    print lasagne.layers.get_output_shape(i)
    print "------------------------------------------------------------------"

Neural Network Architecture

<lasagne.layers.input.InputLayer object at 0x7f8e38bb6850>
(None, 1, 55, 55)
------------------------------------------------------------------
<lasagne.layers.conv.Conv2DLayer object at 0x7f8e38bb68d0>
(None, 32, 55, 55)
------------------------------------------------------------------
<lasagne.layers.pool.MaxPool2DLayer object at 0x7f8e38bb6890>
(None, 32, 27, 27)
------------------------------------------------------------------
<lasagne.layers.conv.Conv2DLayer object at 0x7f8e38b4f810>
(None, 64, 27, 27)
------------------------------------------------------------------
<lasagne.layers.pool.MaxPool2DLayer object at 0x7f8e38bb6910>
(None, 64, 13, 13)
------------------------------------------------------------------
<lasagne.layers.noise.DropoutLayer object at 0x7f8e38b4f850>
(None, 64, 13, 13)
------------------------------------------------------------------
<lasagne.layers.dense.DenseLayer object at 0x7f8e38b4fb10>
(None, 250)
------------------------------------------------------------------
<lasagne.layers.noise.DropoutLayer object at 0x7f8e38b4fad0>
(None, 250)
------------------------------------------------------------------
<lasagne.layers.dense.DenseLayer object at 0x7f8e38b62150>
(None, 45)
------------------------------------------------------------------

Loss History Graphs

# Show Loss History 
print model

<lasagne.layers.dense.DenseLayer object at 0x7f8e38b62150>

Test the network again

Randomly sample an image from the test set, pass it through the network, print the output along with the top 5 classes and their probabilities

from utils import get_character
import random 
import matplotlib.cm as cm
# Load a sample of the test data and show the output class for it 
# Prediction done using the get_output helper 
# Correct Class is in y_ts
# No of test samples - 4275

for i in range(1):
    # Randomise training sample
    row = random.randint(0,4275)
    character = X_ts[row]
    character = character.reshape(55, 55) * 255
    # character will (3025,)
    plt.imshow(character, cmap=cm.Greys_r)
    plt.show()
    to_do = theano.tensor.tensor4()
    to_do = X_ts[row].reshape(-1,1,55,55)
    print X_ts[row].shape
    # Now print output 
    #X_ts[row] = X_ts[row].reshape(-1,1,55,55)
    print X_ts[row].shape
    guess = lasagne.layers.get_output(model, to_do, deterministic=True)
    print "My guess is that it is ", guess.argmax().eval()
    print get_character(guess.argmax().eval())    
    print "Correct Answer is", y_ts[row]
    
    # Show top 5 
    print "-" * 50
    print "The Top 5 Classes"
    top_5 = guess.eval().argsort()[0][::-1][:5]
    for i in top_5 :
        print "Class ", get_character(i), " Probability ", guess[0][i].eval()
    print "-" * 50    

(1, 55, 55)
(1, 55, 55)
My guess is that it is  19
லி
Correct Answer is 19
--------------------------------------------------
The Top 5 Classes
Class  லி  Probability  0.999988608407
Class  ளி  Probability  1.08837903029e-05
Class  யி  Probability  4.18422305063e-07
Class  னி  Probability  8.57565312452e-08
Class  ல  Probability  2.38226436777e-09
--------------------------------------------------

Visualisation

from conv_visualise import plot_conv_weights
from conv_visualise import plot_conv_activity

Weight visualization

plot_conv_weights(lyrs[1])

<module 'matplotlib.pyplot' from '/home/licet/anaconda2/envs/TC/lib/python2.7/site-packages/matplotlib/pyplot.pyc'>

Activity Visualization

# Get the input for visalization 
inp = X_ts[1200]
inp_vis = theano.tensor.tensor4()
inp_vis = inp.reshape(-1,1,55,55) 
plot_conv_activity(lyrs[1], inp_vis)

<module 'matplotlib.pyplot' from '/home/licet/anaconda2/envs/TC/lib/python2.7/site-packages/matplotlib/pyplot.pyc'>

Real Time Test

Path to file being saved

PATH = '/home/licet/Documents/tamilclassifier/app'
im_file = '/char_resized.tiff'

from scipy.misc import imread

Covert the image to a suitable format that can be sent into the trained network

pix = imread(PATH + im_file) # 1 X xs
pix = pix.astype('float64')
pix = pix / 255
pix = pix.ravel()
pix = pix.reshape(1, 55, 55)
usr_img = pix.reshape(55, 55) 
plt.imshow(usr_img, cmap=cm.Greys_r)
plt.show()

to_do = theano.tensor.tensor4()
to_do = pix.reshape(-1,1,55,55)

Test the image on the trained network

usr_guess = lasagne.layers.get_output(model, to_do, deterministic=True)
print "My guess is that it is ", usr_guess.argmax().eval()
print get_character(usr_guess.argmax().eval())    

# Show top 5 
print "-" * 50
print "The Top 5 Classes"
top_5 = usr_guess.eval().argsort()[0][::-1][:5]
for i in top_5 :
    print "Class ", get_character(i), " Probability ", usr_guess[0][i].eval()
print "-" * 50    

My guess is that it is  0
அ
--------------------------------------------------
The Top 5 Classes
Class  அ  Probability  0.776827384315
Class  வு  Probability  0.201602649248
Class  ஆ  Probability  0.0213056678546
Class  ன்  Probability  0.000253278633656
Class  த்  Probability  4.15644063454e-06
--------------------------------------------------

plot_conv_activity(lyrs[1], to_do)

<module 'matplotlib.pyplot' from '/home/licet/anaconda2/envs/TC/lib/python2.7/site-packages/matplotlib/pyplot.pyc'>