Exercises weeks 43 and 44

Exercises weeks 43 and 44¶

October 23-27, 2023

Date: Deadline is Sunday November 5 at midnight

You can hand in the exercises from week 43 and week 44 as one exercise and get a total score of two additional points.

Overarching aims of the exercises weeks 43 and 44¶

The aim of the exercises this week and next week is to get started with writing a neural network code of relevance for project 2.

During week 41 we discussed three different types of gates, the so-called XOR, the OR and the AND gates. In order to develop a code for neural networks, it can be useful to set up a simpler system with only two inputs and one output. This can make it easier to debug and study the feed forward pass and the back propagation part. In the exercise this and next week, we propose to study this system with just one hidden layer and two hidden nodes. There is only one output node and we can choose to use either a simple regression case (fitting a line) or just a binary classification case with the cross-entropy as cost function.

Their inputs and outputs can be summarized using the following tables, first for the OR gate with inputs $x_1$ and $x_2$ and outputs $y$:

$x_1$	$x_2$	$y$
0	0	0
0	1	1
1	0	1
1	1	1

The AND and XOR Gates¶

The AND gate is defined as

$x_1$	$x_2$	$y$
0	0	0
0	1	0
1	0	0
1	1	1

And finally we have the XOR gate

$x_1$	$x_2$	$y$
0	0	0
0	1	1
1	0	1
1	1	0

Representing the Data Sets¶

Our design matrix is defined by the input values $x_1$ and $x_2$. Since we have four possible outputs, our design matrix reads

\[\begin{split} \boldsymbol{X}=\begin{bmatrix} 0 & 0 \\ 0 & 1 \\ 1 & 0 \\ 1 & 1 \end{bmatrix}, \end{split}\]

while the vector of outputs is $\boldsymbol{y}^T=[0,1,1,0]$ for the XOR gate, $\boldsymbol{y}^T=[0,0,0,1]$ for the AND gate and $\boldsymbol{y}^T=[0,1,1,1]$ for the OR gate.

Your tasks here are

Set up the design matrix with the inputs as discussed above and a vector containing the output, the so-called targets. Note that the design matrix is the same for all gates. You need just to define different outputs.
Construct a neural network with only one hidden layer and two hidden nodes using the Sigmoid function as activation function.
Set up the output layer with only one output node and use again the Sigmoid function as activation function for the output.
Initialize the weights and biases and perform a feed forward pass and compare the outputs with the targets.
Set up the cost function (cross entropy for classification of binary cases).
Calculate the gradients needed for the back propagation part.
Use the gradients to train the network in the back propagation part. Think of using automatic differentiation.
Train the network and study your results and compare with results obtained either with scikit-learn or TensorFlow.

Everything you develop here can be used directly into the code for the project.

Setting up dimensionalities by hand¶

It can be useful to test the dimensionalities for the network. Let us assume we have performed an optimization for XOR gate and found that the weights for the hidden layer are given by

\[\begin{split} \boldsymbol{W_h}=\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}, \end{split}\]

Multiplying $\boldsymbol{X}$ and $\boldsymbol{W}$ gives

\[\begin{split} \boldsymbol{X}{W}_h=\begin{bmatrix} 0 & 0 \\ 1 & 1 \\ 1 & 1 \\ 2 & 2 \end{bmatrix}, \end{split}\]

Assume also that the bias vector for the hidden layer is

\[\begin{split} \boldsymbol{b}_h=\begin{bmatrix} 0 \\ -1\end{bmatrix}, \end{split}\]

Adding it gives us the input to the activation function of the hidden layer

\[\begin{split} \boldsymbol{z}_h=\boldsymbol{X}\boldsymbol{W}_h+\boldsymbol{b}_h=\begin{bmatrix} 0 & -1 \\ 1 & 0 \\ 1 & 0 \\ 2 & 1 \end{bmatrix}, \end{split}\]

Let us then assume that our activation function is the RELU function, which simply means that we take the max of $0$ and the elements of the input argument $\boldsymbol{z}_h$, that is we have

\[\begin{split} \boldsymbol{a}_h=\mathrm{RELU}(\boldsymbol{z}_h=\boldsymbol{X}\boldsymbol{W}_h+\boldsymbol{b}_h)=\begin{bmatrix} 0 & 0 \\ 1 & 0 \\ 1 & 0 \\ 2 & 1 \end{bmatrix}, \end{split}\]

Assume also that the bias of the output layer is zero and that the weights of the output layer are

\[\begin{split} \boldsymbol{w}_o=\begin{bmatrix} 1 \\ -2\end{bmatrix}, \end{split}\]

and multiplying with $\boldsymbol{a}_h$ gives the output

\[\begin{split} \boldsymbol{a}_o=\begin{bmatrix} 0 & 0 \\ 1 & 0 \\ 1 & 0 \\ 2 & 1 \end{bmatrix}\begin{bmatrix} 1 \\ -2\end{bmatrix}=\begin{bmatrix} 0 \\ 1 \\ 1 \\0\end{bmatrix}, \end{split}\]

the wanted result. Pay attention to the dimensionalities as well.

Setting up the Neural Network¶

We define first our design matrix and the various output vectors for the different gates.

%matplotlib inline

"""
Simple code that tests XOR, OR and AND gates with linear regression
"""

# import necessary packages
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets

def sigmoid(x):
    return 1/(1 + np.exp(-x))

def feed_forward(X):
    # weighted sum of inputs to the hidden layer
    z_h = np.matmul(X, hidden_weights) + hidden_bias
    # activation in the hidden layer
    a_h = sigmoid(z_h)
    
    # weighted sum of inputs to the output layer
    z_o = np.matmul(a_h, output_weights) + output_bias
    # softmax output
    # axis 0 holds each input and axis 1 the probabilities of each category
    probabilities = sigmoid(z_o)
    return probabilities


# ensure the same random numbers appear every time
np.random.seed(0)

# Design matrix
X = np.array([ [0, 0], [0, 1], [1, 0],[1, 1]],dtype=np.float64)

# The XOR gate
yXOR = np.array( [ 0, 1 ,1, 0])
# The OR gate
yOR = np.array( [ 0, 1 ,1, 1])
# The AND gate
yAND = np.array( [ 0, 0 ,0, 1])

# Defining the neural network
n_inputs, n_features = X.shape
n_hidden_neurons = 2
n_categories = 1
n_features = 2

# we make the weights normally distributed using numpy.random.randn

# weights and bias in the hidden layer
hidden_weights = np.random.randn(n_features, n_hidden_neurons)
hidden_bias = np.zeros(n_hidden_neurons) + 0.01

# weights and bias in the output layer
output_weights = np.random.randn(n_hidden_neurons, n_categories)
output_bias = np.zeros(n_categories) + 0.01

probabilities = feed_forward(X)
print(probabilities)

[[0.61238907]
 [0.61939429]
 [0.73482109]
 [0.70115106]]

Not an impressive result, but this was our first forward pass with randomly assigned weights. Let us now add the full network with the back-propagation algorithm discussed above.

The Code using Scikit-Learn¶

# import necessary packages
import numpy as np
import matplotlib.pyplot as plt
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import accuracy_score
import seaborn as sns

# ensure the same random numbers appear every time
np.random.seed(0)

# Design matrix
X = np.array([ [0, 0], [0, 1], [1, 0],[1, 1]],dtype=np.float64)

# The XOR gate
yXOR = np.array( [ 0, 1 ,1, 0])
# The OR gate
yOR = np.array( [ 0, 1 ,1, 1])
# The AND gate
yAND = np.array( [ 0, 0 ,0, 1])

# Defining the neural network
n_hidden_neurons = 2

eta_vals = np.logspace(-5, 1, 7)
lmbd_vals = np.logspace(-5, 1, 7)
# store models for later use
DNN_scikit = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)
epochs = 100

for i, eta in enumerate(eta_vals):
    for j, lmbd in enumerate(lmbd_vals):
        dnn = MLPClassifier(hidden_layer_sizes=(n_hidden_neurons), activation='logistic',
                            alpha=lmbd, learning_rate_init=eta, max_iter=epochs)
        dnn.fit(X, yXOR)
        DNN_scikit[i][j] = dnn
        print("Learning rate  = ", eta)
        print("Lambda = ", lmbd)
        print("Accuracy score on data set: ", dnn.score(X, yXOR))
        print()

sns.set()
test_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
for i in range(len(eta_vals)):
    for j in range(len(lmbd_vals)):
        dnn = DNN_scikit[i][j]
        test_pred = dnn.predict(X)
        test_accuracy[i][j] = accuracy_score(yXOR, test_pred)

fig, ax = plt.subplots(figsize = (10, 10))
sns.heatmap(test_accuracy, annot=True, ax=ax, cmap="viridis")
ax.set_title("Test Accuracy")
ax.set_ylabel("$\eta$")
ax.set_xlabel("$\lambda$")
plt.show()

Learning rate  =  1e-05
Lambda =  1e-05
Accuracy score on data set:  0.5

Learning rate  =  1e-05
Lambda =  0.0001
Accuracy score on data set:  0.5

Learning rate  =  1e-05
Lambda =  0.001
Accuracy score on data set:  0.5

Learning rate  =  1e-05
Lambda =  0.01
Accuracy score on data set:  0.5

Learning rate  =  1e-05
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  1e-05
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  1e-05
Lambda =  10.0
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  1e-05
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  0.0001
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  0.001
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  0.01
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  0.0001
Lambda =  10.0
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  1e-05
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  0.0001
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  0.001
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  0.01
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  0.001
Lambda =  10.0
Accuracy score on data set:  0.5

Learning rate  =  0.01
Lambda =  1e-05
Accuracy score on data set:  0.25

Learning rate  =  0.01
Lambda =  0.0001
Accuracy score on data set:  0.75

Learning rate  =  0.01
Lambda =  0.001
Accuracy score on data set:  0.5

Learning rate  =  0.01
Lambda =  0.01
Accuracy score on data set:  0.75

Learning rate  =  0.01
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  0.01
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  0.01
Lambda =  10.0
Accuracy score on data set:  0.5

Learning rate  =  0.1
Lambda =  1e-05
Accuracy score on data set:  0.5

Learning rate  =  0.1
Lambda =  0.0001
Accuracy score on data set:  0.5

Learning rate  =  0.1
Lambda =  0.001
Accuracy score on data set:  1.0

Learning rate  =  0.1
Lambda =  0.01
Accuracy score on data set:  1.0

Learning rate  =  0.1
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  0.1
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  0.1
Lambda =  10.0
Accuracy score on data set:  0.5

Learning rate  =  1.0
Lambda =  1e-05
Accuracy score on data set:  0.75

Learning rate  =  1.0
Lambda =  0.0001
Accuracy score on data set:  0.75

Learning rate  =  1.0
Lambda =  0.001
Accuracy score on data set:  0.75

Learning rate  =  1.0
Lambda =  0.01
Accuracy score on data set:  0.5

Learning rate  =  1.0
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  1.0
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  1.0
Lambda =  10.0
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  1e-05
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  0.0001
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  0.001
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  0.01
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  0.1
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  1.0
Accuracy score on data set:  0.5

Learning rate  =  10.0
Lambda =  10.0
Accuracy score on data set:  0.5

/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(
/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:692: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
  warnings.warn(

Building a neural network code¶

Here we present a flexible object oriented codebase for a feed forward neural network, along with a demonstration of how to use it. Before we get into the details of the neural network, we will first present some implementations of various schedulers, cost functions and activation functions that can be used together with the neural network.

The codes here were developed by Eric Reber and Gregor Kajda during spring 2023.

Learning rate methods¶

The code below shows object oriented implementations of the Constant, Momentum, Adagrad, AdagradMomentum, RMS prop and Adam schedulers. All of the classes belong to the shared abstract Scheduler class, and share the update_change() and reset() methods allowing for any of the schedulers to be seamlessly used during the training stage, as will later be shown in the fit() method of the neural network. Update_change() only has one parameter, the gradient ($δ^l_ja^{l−1}_k$), and returns the change which will be subtracted from the weights. The reset() function takes no parameters, and resets the desired variables. For Constant and Momentum, reset does nothing.

import autograd.numpy as np

class Scheduler:
    """
    Abstract class for Schedulers
    """

    def __init__(self, eta):
        self.eta = eta

    # should be overwritten
    def update_change(self, gradient):
        raise NotImplementedError

    # overwritten if needed
    def reset(self):
        pass


class Constant(Scheduler):
    def __init__(self, eta):
        super().__init__(eta)

    def update_change(self, gradient):
        return self.eta * gradient
    
    def reset(self):
        pass


class Momentum(Scheduler):
    def __init__(self, eta: float, momentum: float):
        super().__init__(eta)
        self.momentum = momentum
        self.change = 0

    def update_change(self, gradient):
        self.change = self.momentum * self.change + self.eta * gradient
        return self.change

    def reset(self):
        pass


class Adagrad(Scheduler):
    def __init__(self, eta):
        super().__init__(eta)
        self.G_t = None

    def update_change(self, gradient):
        delta = 1e-8  # avoid division ny zero

        if self.G_t is None:
            self.G_t = np.zeros((gradient.shape[0], gradient.shape[0]))

        self.G_t += gradient @ gradient.T

        G_t_inverse = 1 / (
            delta + np.sqrt(np.reshape(np.diagonal(self.G_t), (self.G_t.shape[0], 1)))
        )
        return self.eta * gradient * G_t_inverse

    def reset(self):
        self.G_t = None


class AdagradMomentum(Scheduler):
    def __init__(self, eta, momentum):
        super().__init__(eta)
        self.G_t = None
        self.momentum = momentum
        self.change = 0

    def update_change(self, gradient):
        delta = 1e-8  # avoid division ny zero

        if self.G_t is None:
            self.G_t = np.zeros((gradient.shape[0], gradient.shape[0]))

        self.G_t += gradient @ gradient.T

        G_t_inverse = 1 / (
            delta + np.sqrt(np.reshape(np.diagonal(self.G_t), (self.G_t.shape[0], 1)))
        )
        self.change = self.change * self.momentum + self.eta * gradient * G_t_inverse
        return self.change

    def reset(self):
        self.G_t = None


class RMS_prop(Scheduler):
    def __init__(self, eta, rho):
        super().__init__(eta)
        self.rho = rho
        self.second = 0.0

    def update_change(self, gradient):
        delta = 1e-8  # avoid division ny zero
        self.second = self.rho * self.second + (1 - self.rho) * gradient * gradient
        return self.eta * gradient / (np.sqrt(self.second + delta))

    def reset(self):
        self.second = 0.0


class Adam(Scheduler):
    def __init__(self, eta, rho, rho2):
        super().__init__(eta)
        self.rho = rho
        self.rho2 = rho2
        self.moment = 0
        self.second = 0
        self.n_epochs = 1

    def update_change(self, gradient):
        delta = 1e-8  # avoid division ny zero

        self.moment = self.rho * self.moment + (1 - self.rho) * gradient
        self.second = self.rho2 * self.second + (1 - self.rho2) * gradient * gradient

        moment_corrected = self.moment / (1 - self.rho**self.n_epochs)
        second_corrected = self.second / (1 - self.rho2**self.n_epochs)

        return self.eta * moment_corrected / (np.sqrt(second_corrected + delta))

    def reset(self):
        self.n_epochs += 1
        self.moment = 0
        self.second = 0

Usage of the above learning rate schedulers¶

To initalize a scheduler, simply create the object and pass in the necessary parameters such as the learning rate and the momentum as shown below. As the Scheduler class is an abstract class it should not called directly, and will raise an error upon usage.

momentum_scheduler = Momentum(eta=1e-3, momentum=0.9)
adam_scheduler = Adam(eta=1e-3, rho=0.9, rho2=0.999)

Here is a small example for how a segment of code using schedulers could look. Switching out the schedulers is simple.

weights = np.ones((3,3))
print(f"Before scheduler:\n{weights=}")

epochs = 10
for e in range(epochs):
    gradient = np.random.rand(3, 3)
    change = adam_scheduler.update_change(gradient)
    weights = weights - change
    adam_scheduler.reset()

print(f"\nAfter scheduler:\n{weights=}")

Before scheduler:
weights=array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])

After scheduler:
weights=array([[0.993993  , 0.993993  , 0.99399301],
       [0.99399308, 0.99399315, 0.99399301],
       [0.99399301, 0.99399309, 0.99399301]])

Cost functions¶

Here we discuss cost functions that can be used when creating the neural network. Every cost function takes the target vector as its parameter, and returns a function valued only at $x$ such that it may easily be differentiated.

import autograd.numpy as np

def CostOLS(target):
    
    def func(X):
        return (1.0 / target.shape[0]) * np.sum((target - X) ** 2)

    return func

def CostLogReg(target):

    def func(X):
        
        return -(1.0 / target.shape[0]) * np.sum(
            (target * np.log(X + 10e-10)) + ((1 - target) * np.log(1 - X + 10e-10))
        )

    return func

def CostCrossEntropy(target):
    
    def func(X):
        return -(1.0 / target.size) * np.sum(target * np.log(X + 10e-10))

    return func

Below we give a short example of how these cost function may be used to obtain results if you wish to test them out on your own using AutoGrad’s automatics differentiation.

from autograd import grad

target = np.array([[1, 2, 3]]).T
a = np.array([[4, 5, 6]]).T

cost_func = CostCrossEntropy
cost_func_derivative = grad(cost_func(target))

valued_at_a = cost_func_derivative(a)
print(f"Derivative of cost function {cost_func.__name__} valued at a:\n{valued_at_a}")

Derivative of cost function CostCrossEntropy valued at a:
[[-0.08333333]
 [-0.13333333]
 [-0.16666667]]

Activation functions¶

Finally, before we look at the neural network, we will look at the activation functions which can be specified between the hidden layers and as the output function. Each function can be valued for any given vector or matrix X, and can be differentiated via derivate().

import autograd.numpy as np
from autograd import elementwise_grad

def identity(X):
    return X


def sigmoid(X):
    try:
        return 1.0 / (1 + np.exp(-X))
    except FloatingPointError:
        return np.where(X > np.zeros(X.shape), np.ones(X.shape), np.zeros(X.shape))


def softmax(X):
    X = X - np.max(X, axis=-1, keepdims=True)
    delta = 10e-10
    return np.exp(X) / (np.sum(np.exp(X), axis=-1, keepdims=True) + delta)


def RELU(X):
    return np.where(X > np.zeros(X.shape), X, np.zeros(X.shape))


def LRELU(X):
    delta = 10e-4
    return np.where(X > np.zeros(X.shape), X, delta * X)


def derivate(func):
    if func.__name__ == "RELU":

        def func(X):
            return np.where(X > 0, 1, 0)

        return func

    elif func.__name__ == "LRELU":

        def func(X):
            delta = 10e-4
            return np.where(X > 0, 1, delta)

        return func

    else:
        return elementwise_grad(func)

Below follows a short demonstration of how to use an activation function. The derivative of the activation function will be important when calculating the output delta term during backpropagation. Note that derivate() can also be used for cost functions for a more generalized approach.

z = np.array([[4, 5, 6]]).T
print(f"Input to activation function:\n{z}")

act_func = sigmoid
a = act_func(z)
print(f"\nOutput from {act_func.__name__} activation function:\n{a}")

act_func_derivative = derivate(act_func)
valued_at_z = act_func_derivative(a)
print(f"\nDerivative of {act_func.__name__} activation function valued at z:\n{valued_at_z}")

Input to activation function:
[[4]
 [5]
 [6]]

Output from sigmoid activation function:
[[0.98201379]
 [0.99330715]
 [0.99752738]]

Derivative of sigmoid activation function valued at z:
[[0.19824029]
 [0.19721923]
 [0.19683648]]

The Neural Network¶

Now that we have gotten a good understanding of the implementation of some important components, we can take a look at an object oriented implementation of a feed forward neural network. The feed forward neural network has been implemented as a class named FFNN, which can be initiated as a regressor or classifier dependant on the choice of cost function. The FFNN can have any number of input nodes, hidden layers with any amount of hidden nodes, and any amount of output nodes meaning it can perform multiclass classification as well as binary classification and regression problems. Although there is a lot of code present, it makes for an easy to use and generalizeable interface for creating many types of neural networks as will be demonstrated below.

import math
import autograd.numpy as np
import sys
import warnings
from autograd import grad, elementwise_grad
from random import random, seed
from copy import deepcopy, copy
from typing import Tuple, Callable
from sklearn.utils import resample

warnings.simplefilter("error")


class FFNN:
    """
    Description:
    ------------
        Feed Forward Neural Network with interface enabling flexible design of a
        nerual networks architecture and the specification of activation function
        in the hidden layers and output layer respectively. This model can be used
        for both regression and classification problems, depending on the output function.

    Attributes:
    ------------
        I   dimensions (tuple[int]): A list of positive integers, which specifies the
            number of nodes in each of the networks layers. The first integer in the array
            defines the number of nodes in the input layer, the second integer defines number
            of nodes in the first hidden layer and so on until the last number, which
            specifies the number of nodes in the output layer.
        II  hidden_func (Callable): The activation function for the hidden layers
        III output_func (Callable): The activation function for the output layer
        IV  cost_func (Callable): Our cost function
        V   seed (int): Sets random seed, makes results reproducible
    """

    def __init__(
        self,
        dimensions: tuple[int],
        hidden_func: Callable = sigmoid,
        output_func: Callable = lambda x: x,
        cost_func: Callable = CostOLS,
        seed: int = None,
    ):
        self.dimensions = dimensions
        self.hidden_func = hidden_func
        self.output_func = output_func
        self.cost_func = cost_func
        self.seed = seed
        self.weights = list()
        self.schedulers_weight = list()
        self.schedulers_bias = list()
        self.a_matrices = list()
        self.z_matrices = list()
        self.classification = None

        self.reset_weights()
        self._set_classification()

    def fit(
        self,
        X: np.ndarray,
        t: np.ndarray,
        scheduler: Scheduler,
        batches: int = 1,
        epochs: int = 100,
        lam: float = 0,
        X_val: np.ndarray = None,
        t_val: np.ndarray = None,
    ):
        """
        Description:
        ------------
            This function performs the training the neural network by performing the feedforward and backpropagation
            algorithm to update the networks weights.

        Parameters:
        ------------
            I    X (np.ndarray) : training data
            II   t (np.ndarray) : target data
            III  scheduler (Scheduler) : specified scheduler (algorithm for optimization of gradient descent)
            IV   scheduler_args (list[int]) : list of all arguments necessary for scheduler

        Optional Parameters:
        ------------
            V    batches (int) : number of batches the datasets are split into, default equal to 1
            VI   epochs (int) : number of iterations used to train the network, default equal to 100
            VII  lam (float) : regularization hyperparameter lambda
            VIII X_val (np.ndarray) : validation set
            IX   t_val (np.ndarray) : validation target set

        Returns:
        ------------
            I   scores (dict) : A dictionary containing the performance metrics of the model.
                The number of the metrics depends on the parameters passed to the fit-function.

        """

        # setup 
        if self.seed is not None:
            np.random.seed(self.seed)

        val_set = False
        if X_val is not None and t_val is not None:
            val_set = True

        # creating arrays for score metrics
        train_errors = np.empty(epochs)
        train_errors.fill(np.nan)
        val_errors = np.empty(epochs)
        val_errors.fill(np.nan)

        train_accs = np.empty(epochs)
        train_accs.fill(np.nan)
        val_accs = np.empty(epochs)
        val_accs.fill(np.nan)

        self.schedulers_weight = list()
        self.schedulers_bias = list()

        batch_size = X.shape[0] // batches

        X, t = resample(X, t)

        # this function returns a function valued only at X
        cost_function_train = self.cost_func(t)
        if val_set:
            cost_function_val = self.cost_func(t_val)

        # create schedulers for each weight matrix
        for i in range(len(self.weights)):
            self.schedulers_weight.append(copy(scheduler))
            self.schedulers_bias.append(copy(scheduler))

        print(f"{scheduler.__class__.__name__}: Eta={scheduler.eta}, Lambda={lam}")

        try:
            for e in range(epochs):
                for i in range(batches):
                    # allows for minibatch gradient descent
                    if i == batches - 1:
                        # If the for loop has reached the last batch, take all thats left
                        X_batch = X[i * batch_size :, :]
                        t_batch = t[i * batch_size :, :]
                    else:
                        X_batch = X[i * batch_size : (i + 1) * batch_size, :]
                        t_batch = t[i * batch_size : (i + 1) * batch_size, :]

                    self._feedforward(X_batch)
                    self._backpropagate(X_batch, t_batch, lam)

                # reset schedulers for each epoch (some schedulers pass in this call)
                for scheduler in self.schedulers_weight:
                    scheduler.reset()

                for scheduler in self.schedulers_bias:
                    scheduler.reset()

                # computing performance metrics
                pred_train = self.predict(X)
                train_error = cost_function_train(pred_train)

                train_errors[e] = train_error
                if val_set:
                    
                    pred_val = self.predict(X_val)
                    val_error = cost_function_val(pred_val)
                    val_errors[e] = val_error

                if self.classification:
                    train_acc = self._accuracy(self.predict(X), t)
                    train_accs[e] = train_acc
                    if val_set:
                        val_acc = self._accuracy(pred_val, t_val)
                        val_accs[e] = val_acc

                # printing progress bar
                progression = e / epochs
                print_length = self._progress_bar(
                    progression,
                    train_error=train_errors[e],
                    train_acc=train_accs[e],
                    val_error=val_errors[e],
                    val_acc=val_accs[e],
                )
        except KeyboardInterrupt:
            # allows for stopping training at any point and seeing the result
            pass

        # visualization of training progression (similiar to tensorflow progression bar)
        sys.stdout.write("\r" + " " * print_length)
        sys.stdout.flush()
        self._progress_bar(
            1,
            train_error=train_errors[e],
            train_acc=train_accs[e],
            val_error=val_errors[e],
            val_acc=val_accs[e],
        )
        sys.stdout.write("")

        # return performance metrics for the entire run
        scores = dict()

        scores["train_errors"] = train_errors

        if val_set:
            scores["val_errors"] = val_errors

        if self.classification:
            scores["train_accs"] = train_accs

            if val_set:
                scores["val_accs"] = val_accs

        return scores

    def predict(self, X: np.ndarray, *, threshold=0.5):
        """
         Description:
         ------------
             Performs prediction after training of the network has been finished.

         Parameters:
        ------------
             I   X (np.ndarray): The design matrix, with n rows of p features each

         Optional Parameters:
         ------------
             II  threshold (float) : sets minimal value for a prediction to be predicted as the positive class
                 in classification problems

         Returns:
         ------------
             I   z (np.ndarray): A prediction vector (row) for each row in our design matrix
                 This vector is thresholded if regression=False, meaning that classification results
                 in a vector of 1s and 0s, while regressions in an array of decimal numbers

        """

        predict = self._feedforward(X)

        if self.classification:
            return np.where(predict > threshold, 1, 0)
        else:
            return predict

    def reset_weights(self):
        """
        Description:
        ------------
            Resets/Reinitializes the weights in order to train the network for a new problem.

        """
        if self.seed is not None:
            np.random.seed(self.seed)

        self.weights = list()
        for i in range(len(self.dimensions) - 1):
            weight_array = np.random.randn(
                self.dimensions[i] + 1, self.dimensions[i + 1]
            )
            weight_array[0, :] = np.random.randn(self.dimensions[i + 1]) * 0.01

            self.weights.append(weight_array)

    def _feedforward(self, X: np.ndarray):
        """
        Description:
        ------------
            Calculates the activation of each layer starting at the input and ending at the output.
            Each following activation is calculated from a weighted sum of each of the preceeding
            activations (except in the case of the input layer).

        Parameters:
        ------------
            I   X (np.ndarray): The design matrix, with n rows of p features each

        Returns:
        ------------
            I   z (np.ndarray): A prediction vector (row) for each row in our design matrix
        """

        # reset matrices
        self.a_matrices = list()
        self.z_matrices = list()

        # if X is just a vector, make it into a matrix
        if len(X.shape) == 1:
            X = X.reshape((1, X.shape[0]))

        # Add a coloumn of zeros as the first coloumn of the design matrix, in order
        # to add bias to our data
        bias = np.ones((X.shape[0], 1)) * 0.01
        X = np.hstack([bias, X])

        # a^0, the nodes in the input layer (one a^0 for each row in X - where the
        # exponent indicates layer number).
        a = X
        self.a_matrices.append(a)
        self.z_matrices.append(a)

        # The feed forward algorithm
        for i in range(len(self.weights)):
            if i < len(self.weights) - 1:
                z = a @ self.weights[i]
                self.z_matrices.append(z)
                a = self.hidden_func(z)
                # bias column again added to the data here
                bias = np.ones((a.shape[0], 1)) * 0.01
                a = np.hstack([bias, a])
                self.a_matrices.append(a)
            else:
                try:
                    # a^L, the nodes in our output layers
                    z = a @ self.weights[i]
                    a = self.output_func(z)
                    self.a_matrices.append(a)
                    self.z_matrices.append(z)
                except Exception as OverflowError:
                    print(
                        "OverflowError in fit() in FFNN\nHOW TO DEBUG ERROR: Consider lowering your learning rate or scheduler specific parameters such as momentum, or check if your input values need scaling"
                    )

        # this will be a^L
        return a

    def _backpropagate(self, X, t, lam):
        """
        Description:
        ------------
            Performs the backpropagation algorithm. In other words, this method
            calculates the gradient of all the layers starting at the
            output layer, and moving from right to left accumulates the gradient until
            the input layer is reached. Each layers respective weights are updated while
            the algorithm propagates backwards from the output layer (auto-differentation in reverse mode).

        Parameters:
        ------------
            I   X (np.ndarray): The design matrix, with n rows of p features each.
            II  t (np.ndarray): The target vector, with n rows of p targets.
            III lam (float32): regularization parameter used to punish the weights in case of overfitting

        Returns:
        ------------
            No return value.

        """
        out_derivative = derivate(self.output_func)
        hidden_derivative = derivate(self.hidden_func)

        for i in range(len(self.weights) - 1, -1, -1):
            # delta terms for output
            if i == len(self.weights) - 1:
                # for multi-class classification
                if (
                    self.output_func.__name__ == "softmax"
                ):
                    delta_matrix = self.a_matrices[i + 1] - t
                # for single class classification
                else:
                    cost_func_derivative = grad(self.cost_func(t))
                    delta_matrix = out_derivative(
                        self.z_matrices[i + 1]
                    ) * cost_func_derivative(self.a_matrices[i + 1])

            # delta terms for hidden layer
            else:
                delta_matrix = (
                    self.weights[i + 1][1:, :] @ delta_matrix.T
                ).T * hidden_derivative(self.z_matrices[i + 1])

            # calculate gradient
            gradient_weights = self.a_matrices[i][:, 1:].T @ delta_matrix
            gradient_bias = np.sum(delta_matrix, axis=0).reshape(
                1, delta_matrix.shape[1]
            )

            # regularization term
            gradient_weights += self.weights[i][1:, :] * lam

            # use scheduler
            update_matrix = np.vstack(
                [
                    self.schedulers_bias[i].update_change(gradient_bias),
                    self.schedulers_weight[i].update_change(gradient_weights),
                ]
            )

            # update weights and bias
            self.weights[i] -= update_matrix

    def _accuracy(self, prediction: np.ndarray, target: np.ndarray):
        """
        Description:
        ------------
            Calculates accuracy of given prediction to target

        Parameters:
        ------------
            I   prediction (np.ndarray): vector of predicitons output network
                (1s and 0s in case of classification, and real numbers in case of regression)
            II  target (np.ndarray): vector of true values (What the network ideally should predict)

        Returns:
        ------------
            A floating point number representing the percentage of correctly classified instances.
        """
        assert prediction.size == target.size
        return np.average((target == prediction))
    def _set_classification(self):
        """
        Description:
        ------------
            Decides if FFNN acts as classifier (True) og regressor (False),
            sets self.classification during init()
        """
        self.classification = False
        if (
            self.cost_func.__name__ == "CostLogReg"
            or self.cost_func.__name__ == "CostCrossEntropy"
        ):
            self.classification = True

    def _progress_bar(self, progression, **kwargs):
        """
        Description:
        ------------
            Displays progress of training
        """
        print_length = 40
        num_equals = int(progression * print_length)
        num_not = print_length - num_equals
        arrow = ">" if num_equals > 0 else ""
        bar = "[" + "=" * (num_equals - 1) + arrow + "-" * num_not + "]"
        perc_print = self._format(progression * 100, decimals=5)
        line = f"  {bar} {perc_print}% "

        for key in kwargs:
            if not np.isnan(kwargs[key]):
                value = self._format(kwargs[key], decimals=4)
                line += f"| {key}: {value} "
        sys.stdout.write("\r" + line)
        sys.stdout.flush()
        return len(line)

    def _format(self, value, decimals=4):
        """
        Description:
        ------------
            Formats decimal numbers for progress bar
        """
        if value > 0:
            v = value
        elif value < 0:
            v = -10 * value
        else:
            v = 1
        n = 1 + math.floor(math.log10(v))
        if n >= decimals - 1:
            return str(round(value))
        return f"{value:.{decimals-n-1}f}"

Before we make a model, we will quickly generate a dataset we can use for our linear regression problem as shown below

import autograd.numpy as np
from sklearn.model_selection import train_test_split

def SkrankeFunction(x, y):
    return np.ravel(0 + 1*x + 2*y + 3*x**2 + 4*x*y + 5*y**2)

def create_X(x, y, n):
    if len(x.shape) > 1:
        x = np.ravel(x)
        y = np.ravel(y)

    N = len(x)
    l = int((n + 1) * (n + 2) / 2)  # Number of elements in beta
    X = np.ones((N, l))

    for i in range(1, n + 1):
        q = int((i) * (i + 1) / 2)
        for k in range(i + 1):
            X[:, q + k] = (x ** (i - k)) * (y**k)

    return X

step=0.5
x = np.arange(0, 1, step)
y = np.arange(0, 1, step)
x, y = np.meshgrid(x, y)
target = SkrankeFunction(x, y)
target = target.reshape(target.shape[0], 1)

poly_degree=3
X = create_X(x, y, poly_degree)

X_train, X_test, t_train, t_test = train_test_split(X, target)

Now that we have our dataset ready for the regression, we can create our regressor. Note that with the seed parameter, we can make sure our results stay the same every time we run the neural network. For inititialization, we simply specify the dimensions (we wish the amount of input nodes to be equal to the datapoints, and the output to predict one value).

input_nodes = X_train.shape[1]
output_nodes = 1

linear_regression = FFNN((input_nodes, output_nodes), output_func=identity, cost_func=CostOLS, seed=2023)

We then fit our model with our training data using the scheduler of our choice.

linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights

scheduler = Constant(eta=1e-3)
scores = linear_regression.fit(X_train, t_train, scheduler)

Constant: Eta=0.001, Lambda=0

  [----------------------------------------] 0.000% | train_error: 3.69 

  [----------------------------------------] 1.000% | train_error: 3.67

  [----------------------------------------] 2.000% | train_error: 3.65

  [>---------------------------------------] 3.000% | train_error: 3.64

  [>---------------------------------------] 4.000% | train_error: 3.62

  [=>--------------------------------------] 5.000% | train_error: 3.60

  [=>--------------------------------------] 6.000% | train_error: 3.58

  [=>--------------------------------------] 7.000% | train_error: 3.57

  [==>-------------------------------------] 8.000% | train_error: 3.55

  [==>-------------------------------------] 9.000% | train_error: 3.53

  [===>------------------------------------] 10.00% | train_error: 3.52

  [===>------------------------------------] 11.00% | train_error: 3.50

  [===>------------------------------------] 12.00% | train_error: 3.48

  [====>-----------------------------------] 13.00% | train_error: 3.47

  [====>-----------------------------------] 14.00% | train_error: 3.45

  [=====>----------------------------------] 15.00% | train_error: 3.43

  [=====>----------------------------------] 16.00% | train_error: 3.42

  [=====>----------------------------------] 17.00% | train_error: 3.40

  [======>---------------------------------] 18.00% | train_error: 3.38

  [======>---------------------------------] 19.00% | train_error: 3.37

  [=======>--------------------------------] 20.00% | train_error: 3.35

  [=======>--------------------------------] 21.00% | train_error: 3.34

  [=======>--------------------------------] 22.00% | train_error: 3.32

  [========>-------------------------------] 23.00% | train_error: 3.31

  [========>-------------------------------] 24.00% | train_error: 3.29

  [=========>------------------------------] 25.00% | train_error: 3.27

  [=========>------------------------------] 26.00% | train_error: 3.26

  [=========>------------------------------] 27.00% | train_error: 3.24

  [==========>-----------------------------] 28.00% | train_error: 3.23

  [==========>-----------------------------] 29.00% | train_error: 3.21

  [===========>----------------------------] 30.00% | train_error: 3.20

  [===========>----------------------------] 31.00% | train_error: 3.18

  [===========>----------------------------] 32.00% | train_error: 3.17

  [============>---------------------------] 33.00% | train_error: 3.15

  [============>---------------------------] 34.00% | train_error: 3.14

  [=============>--------------------------] 35.00% | train_error: 3.12

  [=============>--------------------------] 36.00% | train_error: 3.11

  [=============>--------------------------] 37.00% | train_error: 3.09

  [==============>-------------------------] 38.00% | train_error: 3.08

  [==============>-------------------------] 39.00% | train_error: 3.06

  [===============>------------------------] 40.00% | train_error: 3.05

  [===============>------------------------] 41.00% | train_error: 3.03

  [===============>------------------------] 42.00% | train_error: 3.02

  [================>-----------------------] 43.00% | train_error: 3.00

  [================>-----------------------] 44.00% | train_error: 2.99

  [=================>----------------------] 45.00% | train_error: 2.98

  [=================>----------------------] 46.00% | train_error: 2.96

  [=================>----------------------] 47.00% | train_error: 2.95

  [==================>---------------------] 48.00% | train_error: 2.93

  [==================>---------------------] 49.00% | train_error: 2.92

  [===================>--------------------] 50.00% | train_error: 2.91

  [===================>--------------------] 51.00% | train_error: 2.89

  [===================>--------------------] 52.00% | train_error: 2.88

  [====================>-------------------] 53.00% | train_error: 2.86

  [====================>-------------------] 54.00% | train_error: 2.85

  [=====================>------------------] 55.00% | train_error: 2.84

  [=====================>------------------] 56.00% | train_error: 2.82

  [=====================>------------------] 57.00% | train_error: 2.81

  [======================>-----------------] 58.00% | train_error: 2.80

  [======================>-----------------] 59.00% | train_error: 2.78

  [=======================>----------------] 60.00% | train_error: 2.77

  [=======================>----------------] 61.00% | train_error: 2.76

  [=======================>----------------] 62.00% | train_error: 2.74

  [========================>---------------] 63.00% | train_error: 2.73

  [========================>---------------] 64.00% | train_error: 2.72

  [=========================>--------------] 65.00% | train_error: 2.70

  [=========================>--------------] 66.00% | train_error: 2.69

  [=========================>--------------] 67.00% | train_error: 2.68

  [==========================>-------------] 68.00% | train_error: 2.67

  [==========================>-------------] 69.00% | train_error: 2.65

  [===========================>------------] 70.00% | train_error: 2.64

  [===========================>------------] 71.00% | train_error: 2.63

  [===========================>------------] 72.00% | train_error: 2.62

  [============================>-----------] 73.00% | train_error: 2.60

  [============================>-----------] 74.00% | train_error: 2.59

  [=============================>----------] 75.00% | train_error: 2.58

  [=============================>----------] 76.00% | train_error: 2.57

  [=============================>----------] 77.00% | train_error: 2.55

  [==============================>---------] 78.00% | train_error: 2.54

  [==============================>---------] 79.00% | train_error: 2.53

  [===============================>--------] 80.00% | train_error: 2.52

  [===============================>--------] 81.00% | train_error: 2.51

  [===============================>--------] 82.00% | train_error: 2.49

  [================================>-------] 83.00% | train_error: 2.48

  [================================>-------] 84.00% | train_error: 2.47

  [=================================>------] 85.00% | train_error: 2.46

  [=================================>------] 86.00% | train_error: 2.45

  [=================================>------] 87.00% | train_error: 2.44

  [==================================>-----] 88.00% | train_error: 2.42

  [==================================>-----] 89.00% | train_error: 2.41

  [===================================>----] 90.00% | train_error: 2.40

  [===================================>----] 91.00% | train_error: 2.39

  [===================================>----] 92.00% | train_error: 2.38

  [====================================>---] 93.00% | train_error: 2.37

  [====================================>---] 94.00% | train_error: 2.36

  [=====================================>--] 95.00% | train_error: 2.34

  [=====================================>--] 96.00% | train_error: 2.33

  [=====================================>--] 97.00% | train_error: 2.32

  [======================================>-] 98.00% | train_error: 2.31

  [======================================>-] 99.00% | train_error: 2.30

  [=======================================>] 100.0% | train_error: 2.30

Due to the progress bar we can see the MSE (train_error) throughout the FFNN’s training. Note that the fit() function has some optional parameters with defualt arguments. For example, the regularization hyperparameter can be left ignored if not needed, and equally the FFNN will by default run for 100 epochs. These can easily be changed, such as for example:

linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights

scores = linear_regression.fit(X_train, t_train, scheduler, lam=1e-4, epochs=1000)

Constant: Eta=0.001, Lambda=0.0001

  [----------------------------------------] 0.000% | train_error: 3.69 

  [----------------------------------------] 0.1000% | train_error: 3.67

  [----------------------------------------] 0.2000% | train_error: 3.65

  [----------------------------------------] 0.3000% | train_error: 3.64

  [----------------------------------------] 0.4000% | train_error: 3.62

  [----------------------------------------] 0.5000% | train_error: 3.60

  [----------------------------------------] 0.6000% | train_error: 3.58

  [----------------------------------------] 0.7000% | train_error: 3.57

  [----------------------------------------] 0.8000% | train_error: 3.55

  [----------------------------------------] 0.9000% | train_error: 3.53

  [----------------------------------------] 1.000% | train_error: 3.52

  [----------------------------------------] 1.100% | train_error: 3.50

  [----------------------------------------] 1.200% | train_error: 3.48

  [----------------------------------------] 1.300% | train_error: 3.47

  [----------------------------------------] 1.400% | train_error: 3.45

  [----------------------------------------] 1.500% | train_error: 3.43

  [----------------------------------------] 1.600% | train_error: 3.42

  [----------------------------------------] 1.700% | train_error: 3.40

  [----------------------------------------] 1.800% | train_error: 3.38

  [----------------------------------------] 1.900% | train_error: 3.37

  [----------------------------------------] 2.000% | train_error: 3.35

  [----------------------------------------] 2.100% | train_error: 3.34

  [----------------------------------------] 2.200% | train_error: 3.32

  [----------------------------------------] 2.300% | train_error: 3.31

  [----------------------------------------] 2.400% | train_error: 3.29

  [>---------------------------------------] 2.500% | train_error: 3.27

  [>---------------------------------------] 2.600% | train_error: 3.26

  [>---------------------------------------] 2.700% | train_error: 3.24

  [>---------------------------------------] 2.800% | train_error: 3.23

  [>---------------------------------------] 2.900% | train_error: 3.21

  [>---------------------------------------] 3.000% | train_error: 3.20

  [>---------------------------------------] 3.100% | train_error: 3.18

  [>---------------------------------------] 3.200% | train_error: 3.17

  [>---------------------------------------] 3.300% | train_error: 3.15

  [>---------------------------------------] 3.400% | train_error: 3.14

  [>---------------------------------------] 3.500% | train_error: 3.12

  [>---------------------------------------] 3.600% | train_error: 3.11

  [>---------------------------------------] 3.700% | train_error: 3.09

  [>---------------------------------------] 3.800% | train_error: 3.08

  [>---------------------------------------] 3.900% | train_error: 3.06

  [>---------------------------------------] 4.000% | train_error: 3.05

  [>---------------------------------------] 4.100% | train_error: 3.03

  [>---------------------------------------] 4.200% | train_error: 3.02

  [>---------------------------------------] 4.300% | train_error: 3.00

  [>---------------------------------------] 4.400% | train_error: 2.99

  [>---------------------------------------] 4.500% | train_error: 2.98

  [>---------------------------------------] 4.600% | train_error: 2.96

  [>---------------------------------------] 4.700% | train_error: 2.95

  [>---------------------------------------] 4.800% | train_error: 2.93

  [>---------------------------------------] 4.900% | train_error: 2.92

  [=>--------------------------------------] 5.000% | train_error: 2.91

  [=>--------------------------------------] 5.100% | train_error: 2.89

  [=>--------------------------------------] 5.200% | train_error: 2.88

  [=>--------------------------------------] 5.300% | train_error: 2.86

  [=>--------------------------------------] 5.400% | train_error: 2.85

  [=>--------------------------------------] 5.500% | train_error: 2.84

  [=>--------------------------------------] 5.600% | train_error: 2.82

  [=>--------------------------------------] 5.700% | train_error: 2.81

  [=>--------------------------------------] 5.800% | train_error: 2.80

  [=>--------------------------------------] 5.900% | train_error: 2.78

  [=>--------------------------------------] 6.000% | train_error: 2.77

  [=>--------------------------------------] 6.100% | train_error: 2.76

  [=>--------------------------------------] 6.200% | train_error: 2.74

  [=>--------------------------------------] 6.300% | train_error: 2.73

  [=>--------------------------------------] 6.400% | train_error: 2.72

  [=>--------------------------------------] 6.500% | train_error: 2.70

  [=>--------------------------------------] 6.600% | train_error: 2.69

  [=>--------------------------------------] 6.700% | train_error: 2.68

  [=>--------------------------------------] 6.800% | train_error: 2.67

  [=>--------------------------------------] 6.900% | train_error: 2.65

  [=>--------------------------------------] 7.000% | train_error: 2.64

  [=>--------------------------------------] 7.100% | train_error: 2.63

  [=>--------------------------------------] 7.200% | train_error: 2.62

  [=>--------------------------------------] 7.300% | train_error: 2.60

  [=>--------------------------------------] 7.400% | train_error: 2.59

  [==>-------------------------------------] 7.500% | train_error: 2.58

  [==>-------------------------------------] 7.600% | train_error: 2.57

  [==>-------------------------------------] 7.700% | train_error: 2.55

  [==>-------------------------------------] 7.800% | train_error: 2.54

  [==>-------------------------------------] 7.900% | train_error: 2.53

  [==>-------------------------------------] 8.000% | train_error: 2.52

  [==>-------------------------------------] 8.100% | train_error: 2.51

  [==>-------------------------------------] 8.200% | train_error: 2.49

  [==>-------------------------------------] 8.300% | train_error: 2.48

  [==>-------------------------------------] 8.400% | train_error: 2.47

  [==>-------------------------------------] 8.500% | train_error: 2.46

  [==>-------------------------------------] 8.600% | train_error: 2.45

  [==>-------------------------------------] 8.700% | train_error: 2.44

  [==>-------------------------------------] 8.800% | train_error: 2.42

  [==>-------------------------------------] 8.900% | train_error: 2.41

  [==>-------------------------------------] 9.000% | train_error: 2.40

  [==>-------------------------------------] 9.100% | train_error: 2.39

  [==>-------------------------------------] 9.200% | train_error: 2.38

  [==>-------------------------------------] 9.300% | train_error: 2.37

  [==>-------------------------------------] 9.400% | train_error: 2.36

  [==>-------------------------------------] 9.500% | train_error: 2.34

  [==>-------------------------------------] 9.600% | train_error: 2.33

  [==>-------------------------------------] 9.700% | train_error: 2.32

  [==>-------------------------------------] 9.800% | train_error: 2.31

  [==>-------------------------------------] 9.900% | train_error: 2.30

  [===>------------------------------------] 10.00% | train_error: 2.29

  [===>------------------------------------] 10.10% | train_error: 2.28

  [===>------------------------------------] 10.20% | train_error: 2.27

  [===>------------------------------------] 10.30% | train_error: 2.26

  [===>------------------------------------] 10.40% | train_error: 2.25

  [===>------------------------------------] 10.50% | train_error: 2.23

  [===>------------------------------------] 10.60% | train_error: 2.22

  [===>------------------------------------] 10.70% | train_error: 2.21

  [===>------------------------------------] 10.80% | train_error: 2.20

  [===>------------------------------------] 10.90% | train_error: 2.19

  [===>------------------------------------] 11.00% | train_error: 2.18

  [===>------------------------------------] 11.10% | train_error: 2.17

  [===>------------------------------------] 11.20% | train_error: 2.16

  [===>------------------------------------] 11.30% | train_error: 2.15

  [===>------------------------------------] 11.40% | train_error: 2.14

  [===>------------------------------------] 11.50% | train_error: 2.13

  [===>------------------------------------] 11.60% | train_error: 2.12

  [===>------------------------------------] 11.70% | train_error: 2.11

  [===>------------------------------------] 11.80% | train_error: 2.10

  [===>------------------------------------] 11.90% | train_error: 2.09

  [===>------------------------------------] 12.00% | train_error: 2.08

  [===>------------------------------------] 12.10% | train_error: 2.07

  [===>------------------------------------] 12.20% | train_error: 2.06

  [===>------------------------------------] 12.30% | train_error: 2.05

  [===>------------------------------------] 12.40% | train_error: 2.04

  [====>-----------------------------------] 12.50% | train_error: 2.03

  [====>-----------------------------------] 12.60% | train_error: 2.02

  [====>-----------------------------------] 12.70% | train_error: 2.01

  [====>-----------------------------------] 12.80% | train_error: 2.00

  [====>-----------------------------------] 12.90% | train_error: 1.99

  [====>-----------------------------------] 13.00% | train_error: 1.98

  [====>-----------------------------------] 13.10% | train_error: 1.97

  [====>-----------------------------------] 13.20% | train_error: 1.96

  [====>-----------------------------------] 13.30% | train_error: 1.96

  [====>-----------------------------------] 13.40% | train_error: 1.95

  [====>-----------------------------------] 13.50% | train_error: 1.94

  [====>-----------------------------------] 13.60% | train_error: 1.93

  [====>-----------------------------------] 13.70% | train_error: 1.92

  [====>-----------------------------------] 13.80% | train_error: 1.91

  [====>-----------------------------------] 13.90% | train_error: 1.90

  [====>-----------------------------------] 14.00% | train_error: 1.89

  [====>-----------------------------------] 14.10% | train_error: 1.88

  [====>-----------------------------------] 14.20% | train_error: 1.87

  [====>-----------------------------------] 14.30% | train_error: 1.86

  [====>-----------------------------------] 14.40% | train_error: 1.86

  [====>-----------------------------------] 14.50% | train_error: 1.85

  [====>-----------------------------------] 14.60% | train_error: 1.84

  [====>-----------------------------------] 14.70% | train_error: 1.83

  [====>-----------------------------------] 14.80% | train_error: 1.82

  [====>-----------------------------------] 14.90% | train_error: 1.81

  [=====>----------------------------------] 15.00% | train_error: 1.80

  [=====>----------------------------------] 15.10% | train_error: 1.79

  [=====>----------------------------------] 15.20% | train_error: 1.79

  [=====>----------------------------------] 15.30% | train_error: 1.78

  [=====>----------------------------------] 15.40% | train_error: 1.77

  [=====>----------------------------------] 15.50% | train_error: 1.76

  [=====>----------------------------------] 15.60% | train_error: 1.75

  [=====>----------------------------------] 15.70% | train_error: 1.74

  [=====>----------------------------------] 15.80% | train_error: 1.74

  [=====>----------------------------------] 15.90% | train_error: 1.73

  [=====>----------------------------------] 16.00% | train_error: 1.72

  [=====>----------------------------------] 16.10% | train_error: 1.71

  [=====>----------------------------------] 16.20% | train_error: 1.70

  [=====>----------------------------------] 16.30% | train_error: 1.69

  [=====>----------------------------------] 16.40% | train_error: 1.69

  [=====>----------------------------------] 16.50% | train_error: 1.68

  [=====>----------------------------------] 16.60% | train_error: 1.67

  [=====>----------------------------------] 16.70% | train_error: 1.66

  [=====>----------------------------------] 16.80% | train_error: 1.65

  [=====>----------------------------------] 16.90% | train_error: 1.65

  [=====>----------------------------------] 17.00% | train_error: 1.64

  [=====>----------------------------------] 17.10% | train_error: 1.63

  [=====>----------------------------------] 17.20% | train_error: 1.62

  [=====>----------------------------------] 17.30% | train_error: 1.62

  [=====>----------------------------------] 17.40% | train_error: 1.61

  [======>---------------------------------] 17.50% | train_error: 1.60

  [======>---------------------------------] 17.60% | train_error: 1.59

  [======>---------------------------------] 17.70% | train_error: 1.59

  [======>---------------------------------] 17.80% | train_error: 1.58

  [======>---------------------------------] 17.90% | train_error: 1.57

  [======>---------------------------------] 18.00% | train_error: 1.56

  [======>---------------------------------] 18.10% | train_error: 1.56

  [======>---------------------------------] 18.20% | train_error: 1.55

  [======>---------------------------------] 18.30% | train_error: 1.54

  [======>---------------------------------] 18.40% | train_error: 1.53

  [======>---------------------------------] 18.50% | train_error: 1.53

  [======>---------------------------------] 18.60% | train_error: 1.52

  [======>---------------------------------] 18.70% | train_error: 1.51

  [======>---------------------------------] 18.80% | train_error: 1.50

  [======>---------------------------------] 18.90% | train_error: 1.50

  [======>---------------------------------] 19.00% | train_error: 1.49

  [======>---------------------------------] 19.10% | train_error: 1.48

  [======>---------------------------------] 19.20% | train_error: 1.48

  [======>---------------------------------] 19.30% | train_error: 1.47

  [======>---------------------------------] 19.40% | train_error: 1.46

  [======>---------------------------------] 19.50% | train_error: 1.46

  [======>---------------------------------] 19.60% | train_error: 1.45

  [======>---------------------------------] 19.70% | train_error: 1.44

  [======>---------------------------------] 19.80% | train_error: 1.43

  [======>---------------------------------] 19.90% | train_error: 1.43

  [=======>--------------------------------] 20.00% | train_error: 1.42

  [=======>--------------------------------] 20.10% | train_error: 1.41

  [=======>--------------------------------] 20.20% | train_error: 1.41

  [=======>--------------------------------] 20.30% | train_error: 1.40

  [=======>--------------------------------] 20.40% | train_error: 1.39

  [=======>--------------------------------] 20.50% | train_error: 1.39

  [=======>--------------------------------] 20.60% | train_error: 1.38

  [=======>--------------------------------] 20.70% | train_error: 1.37

  [=======>--------------------------------] 20.80% | train_error: 1.37

  [=======>--------------------------------] 20.90% | train_error: 1.36

  [=======>--------------------------------] 21.00% | train_error: 1.35

  [=======>--------------------------------] 21.10% | train_error: 1.35

  [=======>--------------------------------] 21.20% | train_error: 1.34

  [=======>--------------------------------] 21.30% | train_error: 1.34

  [=======>--------------------------------] 21.40% | train_error: 1.33

  [=======>--------------------------------] 21.50% | train_error: 1.32

  [=======>--------------------------------] 21.60% | train_error: 1.32

  [=======>--------------------------------] 21.70% | train_error: 1.31

  [=======>--------------------------------] 21.80% | train_error: 1.30

  [=======>--------------------------------] 21.90% | train_error: 1.30

  [=======>--------------------------------] 22.00% | train_error: 1.29

  [=======>--------------------------------] 22.10% | train_error: 1.29

  [=======>--------------------------------] 22.20% | train_error: 1.28

  [=======>--------------------------------] 22.30% | train_error: 1.27

  [=======>--------------------------------] 22.40% | train_error: 1.27

  [========>-------------------------------] 22.50% | train_error: 1.26

  [========>-------------------------------] 22.60% | train_error: 1.26

  [========>-------------------------------] 22.70% | train_error: 1.25

  [========>-------------------------------] 22.80% | train_error: 1.24

  [========>-------------------------------] 22.90% | train_error: 1.24

  [========>-------------------------------] 23.00% | train_error: 1.23

  [========>-------------------------------] 23.10% | train_error: 1.23

  [========>-------------------------------] 23.20% | train_error: 1.22

  [========>-------------------------------] 23.30% | train_error: 1.21

  [========>-------------------------------] 23.40% | train_error: 1.21

  [========>-------------------------------] 23.50% | train_error: 1.20

  [========>-------------------------------] 23.60% | train_error: 1.20

  [========>-------------------------------] 23.70% | train_error: 1.19

  [========>-------------------------------] 23.80% | train_error: 1.19

  [========>-------------------------------] 23.90% | train_error: 1.18

  [========>-------------------------------] 24.00% | train_error: 1.17

  [========>-------------------------------] 24.10% | train_error: 1.17

  [========>-------------------------------] 24.20% | train_error: 1.16

  [========>-------------------------------] 24.30% | train_error: 1.16

  [========>-------------------------------] 24.40% | train_error: 1.15

  [========>-------------------------------] 24.50% | train_error: 1.15

  [========>-------------------------------] 24.60% | train_error: 1.14

  [========>-------------------------------] 24.70% | train_error: 1.14

  [========>-------------------------------] 24.80% | train_error: 1.13

  [========>-------------------------------] 24.90% | train_error: 1.13

  [=========>------------------------------] 25.00% | train_error: 1.12

  [=========>------------------------------] 25.10% | train_error: 1.11

  [=========>------------------------------] 25.20% | train_error: 1.11

  [=========>------------------------------] 25.30% | train_error: 1.10

  [=========>------------------------------] 25.40% | train_error: 1.10

  [=========>------------------------------] 25.50% | train_error: 1.09

  [=========>------------------------------] 25.60% | train_error: 1.09

  [=========>------------------------------] 25.70% | train_error: 1.08

  [=========>------------------------------] 25.80% | train_error: 1.08

  [=========>------------------------------] 25.90% | train_error: 1.07

  [=========>------------------------------] 26.00% | train_error: 1.07

  [=========>------------------------------] 26.10% | train_error: 1.06

  [=========>------------------------------] 26.20% | train_error: 1.06

  [=========>------------------------------] 26.30% | train_error: 1.05

  [=========>------------------------------] 26.40% | train_error: 1.05

  [=========>------------------------------] 26.50% | train_error: 1.04

  [=========>------------------------------] 26.60% | train_error: 1.04

  [=========>------------------------------] 26.70% | train_error: 1.03

  [=========>------------------------------] 26.80% | train_error: 1.03

  [=========>------------------------------] 26.90% | train_error: 1.02

  [=========>------------------------------] 27.00% | train_error: 1.02

  [=========>------------------------------] 27.10% | train_error: 1.01

  [=========>------------------------------] 27.20% | train_error: 1.01

  [=========>------------------------------] 27.30% | train_error: 1.00

  [=========>------------------------------] 27.40% | train_error: 0.999

  [==========>-----------------------------] 27.50% | train_error: 0.994

  [==========>-----------------------------] 27.60% | train_error: 0.990

  [==========>-----------------------------] 27.70% | train_error: 0.985

  [==========>-----------------------------] 27.80% | train_error: 0.980

  [==========>-----------------------------] 27.90% | train_error: 0.976

  [==========>-----------------------------] 28.00% | train_error: 0.971

  [==========>-----------------------------] 28.10% | train_error: 0.966

  [==========>-----------------------------] 28.20% | train_error: 0.962

  [==========>-----------------------------] 28.30% | train_error: 0.957

  [==========>-----------------------------] 28.40% | train_error: 0.953

  [==========>-----------------------------] 28.50% | train_error: 0.948

  [==========>-----------------------------] 28.60% | train_error: 0.944

  [==========>-----------------------------] 28.70% | train_error: 0.939

  [==========>-----------------------------] 28.80% | train_error: 0.935

  [==========>-----------------------------] 28.90% | train_error: 0.930

  [==========>-----------------------------] 29.00% | train_error: 0.926

  [==========>-----------------------------] 29.10% | train_error: 0.922

  [==========>-----------------------------] 29.20% | train_error: 0.917

  [==========>-----------------------------] 29.30% | train_error: 0.913

  [==========>-----------------------------] 29.40% | train_error: 0.909

  [==========>-----------------------------] 29.50% | train_error: 0.904

  [==========>-----------------------------] 29.60% | train_error: 0.900

  [==========>-----------------------------] 29.70% | train_error: 0.896

  [==========>-----------------------------] 29.80% | train_error: 0.891

  [==========>-----------------------------] 29.90% | train_error: 0.887

  [===========>----------------------------] 30.00% | train_error: 0.883

  [===========>----------------------------] 30.10% | train_error: 0.879

  [===========>----------------------------] 30.20% | train_error: 0.875

  [===========>----------------------------] 30.30% | train_error: 0.870

  [===========>----------------------------] 30.40% | train_error: 0.866

  [===========>----------------------------] 30.50% | train_error: 0.862

  [===========>----------------------------] 30.60% | train_error: 0.858

  [===========>----------------------------] 30.70% | train_error: 0.854

  [===========>----------------------------] 30.80% | train_error: 0.850

  [===========>----------------------------] 30.90% | train_error: 0.846

  [===========>----------------------------] 31.00% | train_error: 0.842

  [===========>----------------------------] 31.10% | train_error: 0.838

  [===========>----------------------------] 31.20% | train_error: 0.834

  [===========>----------------------------] 31.30% | train_error: 0.830

  [===========>----------------------------] 31.40% | train_error: 0.826

  [===========>----------------------------] 31.50% | train_error: 0.822

  [===========>----------------------------] 31.60% | train_error: 0.818

  [===========>----------------------------] 31.70% | train_error: 0.814

  [===========>----------------------------] 31.80% | train_error: 0.811

  [===========>----------------------------] 31.90% | train_error: 0.807

  [===========>----------------------------] 32.00% | train_error: 0.803

  [===========>----------------------------] 32.10% | train_error: 0.799

  [===========>----------------------------] 32.20% | train_error: 0.795

  [===========>----------------------------] 32.30% | train_error: 0.792

  [===========>----------------------------] 32.40% | train_error: 0.788

  [============>---------------------------] 32.50% | train_error: 0.784

  [============>---------------------------] 32.60% | train_error: 0.780

  [============>---------------------------] 32.70% | train_error: 0.777

  [============>---------------------------] 32.80% | train_error: 0.773

  [============>---------------------------] 32.90% | train_error: 0.769

  [============>---------------------------] 33.00% | train_error: 0.766

  [============>---------------------------] 33.10% | train_error: 0.762

  [============>---------------------------] 33.20% | train_error: 0.759

  [============>---------------------------] 33.30% | train_error: 0.755

  [============>---------------------------] 33.40% | train_error: 0.751

  [============>---------------------------] 33.50% | train_error: 0.748

  [============>---------------------------] 33.60% | train_error: 0.744

  [============>---------------------------] 33.70% | train_error: 0.741

  [============>---------------------------] 33.80% | train_error: 0.737

  [============>---------------------------] 33.90% | train_error: 0.734

  [============>---------------------------] 34.00% | train_error: 0.730

  [============>---------------------------] 34.10% | train_error: 0.727

  [============>---------------------------] 34.20% | train_error: 0.723

  [============>---------------------------] 34.30% | train_error: 0.720

  [============>---------------------------] 34.40% | train_error: 0.717

  [============>---------------------------] 34.50% | train_error: 0.713

  [============>---------------------------] 34.60% | train_error: 0.710

  [============>---------------------------] 34.70% | train_error: 0.706

  [============>---------------------------] 34.80% | train_error: 0.703

  [============>---------------------------] 34.90% | train_error: 0.700

  [=============>--------------------------] 35.00% | train_error: 0.696

  [=============>--------------------------] 35.10% | train_error: 0.693

  [=============>--------------------------] 35.20% | train_error: 0.690

  [=============>--------------------------] 35.30% | train_error: 0.687

  [=============>--------------------------] 35.40% | train_error: 0.683

  [=============>--------------------------] 35.50% | train_error: 0.680

  [=============>--------------------------] 35.60% | train_error: 0.677

  [=============>--------------------------] 35.70% | train_error: 0.674

  [=============>--------------------------] 35.80% | train_error: 0.670

  [=============>--------------------------] 35.90% | train_error: 0.667

  [=============>--------------------------] 36.00% | train_error: 0.664

  [=============>--------------------------] 36.10% | train_error: 0.661

  [=============>--------------------------] 36.20% | train_error: 0.658

  [=============>--------------------------] 36.30% | train_error: 0.655

  [=============>--------------------------] 36.40% | train_error: 0.652

  [=============>--------------------------] 36.50% | train_error: 0.649

  [=============>--------------------------] 36.60% | train_error: 0.646

  [=============>--------------------------] 36.70% | train_error: 0.642

  [=============>--------------------------] 36.80% | train_error: 0.639

  [=============>--------------------------] 36.90% | train_error: 0.636

  [=============>--------------------------] 37.00% | train_error: 0.633

  [=============>--------------------------] 37.10% | train_error: 0.630

  [=============>--------------------------] 37.20% | train_error: 0.627

  [=============>--------------------------] 37.30% | train_error: 0.624

  [=============>--------------------------] 37.40% | train_error: 0.622

  [==============>-------------------------] 37.50% | train_error: 0.619

  [==============>-------------------------] 37.60% | train_error: 0.616

  [==============>-------------------------] 37.70% | train_error: 0.613

  [==============>-------------------------] 37.80% | train_error: 0.610

  [==============>-------------------------] 37.90% | train_error: 0.607

  [==============>-------------------------] 38.00% | train_error: 0.604

  [==============>-------------------------] 38.10% | train_error: 0.601

  [==============>-------------------------] 38.20% | train_error: 0.598

  [==============>-------------------------] 38.30% | train_error: 0.596

  [==============>-------------------------] 38.40% | train_error: 0.593

  [==============>-------------------------] 38.50% | train_error: 0.590

  [==============>-------------------------] 38.60% | train_error: 0.587

  [==============>-------------------------] 38.70% | train_error: 0.584

  [==============>-------------------------] 38.80% | train_error: 0.582

  [==============>-------------------------] 38.90% | train_error: 0.579

  [==============>-------------------------] 39.00% | train_error: 0.576

  [==============>-------------------------] 39.10% | train_error: 0.573

  [==============>-------------------------] 39.20% | train_error: 0.571

  [==============>-------------------------] 39.30% | train_error: 0.568

  [==============>-------------------------] 39.40% | train_error: 0.565

  [==============>-------------------------] 39.50% | train_error: 0.563

  [==============>-------------------------] 39.60% | train_error: 0.560

  [==============>-------------------------] 39.70% | train_error: 0.557

  [==============>-------------------------] 39.80% | train_error: 0.555

  [==============>-------------------------] 39.90% | train_error: 0.552

  [===============>------------------------] 40.00% | train_error: 0.549

  [===============>------------------------] 40.10% | train_error: 0.547

  [===============>------------------------] 40.20% | train_error: 0.544

  [===============>------------------------] 40.30% | train_error: 0.542

  [===============>------------------------] 40.40% | train_error: 0.539

  [===============>------------------------] 40.50% | train_error: 0.537

  [===============>------------------------] 40.60% | train_error: 0.534

  [===============>------------------------] 40.70% | train_error: 0.532

  [===============>------------------------] 40.80% | train_error: 0.529

  [===============>------------------------] 40.90% | train_error: 0.527

  [===============>------------------------] 41.00% | train_error: 0.524

  [===============>------------------------] 41.10% | train_error: 0.522

  [===============>------------------------] 41.20% | train_error: 0.519

  [===============>------------------------] 41.30% | train_error: 0.517

  [===============>------------------------] 41.40% | train_error: 0.514

  [===============>------------------------] 41.50% | train_error: 0.512

  [===============>------------------------] 41.60% | train_error: 0.509

  [===============>------------------------] 41.70% | train_error: 0.507

  [===============>------------------------] 41.80% | train_error: 0.505

  [===============>------------------------] 41.90% | train_error: 0.502

  [===============>------------------------] 42.00% | train_error: 0.500

  [===============>------------------------] 42.10% | train_error: 0.498

  [===============>------------------------] 42.20% | train_error: 0.495

  [===============>------------------------] 42.30% | train_error: 0.493

  [===============>------------------------] 42.40% | train_error: 0.491

  [================>-----------------------] 42.50% | train_error: 0.488

  [================>-----------------------] 42.60% | train_error: 0.486

  [================>-----------------------] 42.70% | train_error: 0.484

  [================>-----------------------] 42.80% | train_error: 0.481

  [================>-----------------------] 42.90% | train_error: 0.479

  [================>-----------------------] 43.00% | train_error: 0.477

  [================>-----------------------] 43.10% | train_error: 0.475

  [================>-----------------------] 43.20% | train_error: 0.472

  [================>-----------------------] 43.30% | train_error: 0.470

  [================>-----------------------] 43.40% | train_error: 0.468

  [================>-----------------------] 43.50% | train_error: 0.466

  [================>-----------------------] 43.60% | train_error: 0.463

  [================>-----------------------] 43.70% | train_error: 0.461

  [================>-----------------------] 43.80% | train_error: 0.459

  [================>-----------------------] 43.90% | train_error: 0.457

  [================>-----------------------] 44.00% | train_error: 0.455

  [================>-----------------------] 44.10% | train_error: 0.453

  [================>-----------------------] 44.20% | train_error: 0.451

  [================>-----------------------] 44.30% | train_error: 0.448

  [================>-----------------------] 44.40% | train_error: 0.446

  [================>-----------------------] 44.50% | train_error: 0.444

  [================>-----------------------] 44.60% | train_error: 0.442

  [================>-----------------------] 44.70% | train_error: 0.440

  [================>-----------------------] 44.80% | train_error: 0.438

  [================>-----------------------] 44.90% | train_error: 0.436

  [=================>----------------------] 45.00% | train_error: 0.434

  [=================>----------------------] 45.10% | train_error: 0.432

  [=================>----------------------] 45.20% | train_error: 0.430

  [=================>----------------------] 45.30% | train_error: 0.428

  [=================>----------------------] 45.40% | train_error: 0.426

  [=================>----------------------] 45.50% | train_error: 0.424

  [=================>----------------------] 45.60% | train_error: 0.422

  [=================>----------------------] 45.70% | train_error: 0.420

  [=================>----------------------] 45.80% | train_error: 0.418

  [=================>----------------------] 45.90% | train_error: 0.416

  [=================>----------------------] 46.00% | train_error: 0.414

  [=================>----------------------] 46.10% | train_error: 0.412

  [=================>----------------------] 46.20% | train_error: 0.410

  [=================>----------------------] 46.30% | train_error: 0.408

  [=================>----------------------] 46.40% | train_error: 0.406

  [=================>----------------------] 46.50% | train_error: 0.404

  [=================>----------------------] 46.60% | train_error: 0.402

  [=================>----------------------] 46.70% | train_error: 0.400

  [=================>----------------------] 46.80% | train_error: 0.398

  [=================>----------------------] 46.90% | train_error: 0.397

  [=================>----------------------] 47.00% | train_error: 0.395

  [=================>----------------------] 47.10% | train_error: 0.393

  [=================>----------------------] 47.20% | train_error: 0.391

  [=================>----------------------] 47.30% | train_error: 0.389

  [=================>----------------------] 47.40% | train_error: 0.387

  [==================>---------------------] 47.50% | train_error: 0.386

  [==================>---------------------] 47.60% | train_error: 0.384

  [==================>---------------------] 47.70% | train_error: 0.382

  [==================>---------------------] 47.80% | train_error: 0.380

  [==================>---------------------] 47.90% | train_error: 0.378

  [==================>---------------------] 48.00% | train_error: 0.377

  [==================>---------------------] 48.10% | train_error: 0.375

  [==================>---------------------] 48.20% | train_error: 0.373

  [==================>---------------------] 48.30% | train_error: 0.371

  [==================>---------------------] 48.40% | train_error: 0.370

  [==================>---------------------] 48.50% | train_error: 0.368

  [==================>---------------------] 48.60% | train_error: 0.366

  [==================>---------------------] 48.70% | train_error: 0.364

  [==================>---------------------] 48.80% | train_error: 0.363

  [==================>---------------------] 48.90% | train_error: 0.361

  [==================>---------------------] 49.00% | train_error: 0.359

  [==================>---------------------] 49.10% | train_error: 0.358

  [==================>---------------------] 49.20% | train_error: 0.356

  [==================>---------------------] 49.30% | train_error: 0.354

  [==================>---------------------] 49.40% | train_error: 0.353

  [==================>---------------------] 49.50% | train_error: 0.351

  [==================>---------------------] 49.60% | train_error: 0.349

  [==================>---------------------] 49.70% | train_error: 0.348

  [==================>---------------------] 49.80% | train_error: 0.346

  [==================>---------------------] 49.90% | train_error: 0.344

  [===================>--------------------] 50.00% | train_error: 0.343

  [===================>--------------------] 50.10% | train_error: 0.341

  [===================>--------------------] 50.20% | train_error: 0.339

  [===================>--------------------] 50.30% | train_error: 0.338

  [===================>--------------------] 50.40% | train_error: 0.336

  [===================>--------------------] 50.50% | train_error: 0.335

  [===================>--------------------] 50.60% | train_error: 0.333

  [===================>--------------------] 50.70% | train_error: 0.332

  [===================>--------------------] 50.80% | train_error: 0.330

  [===================>--------------------] 50.90% | train_error: 0.328

  [===================>--------------------] 51.00% | train_error: 0.327

  [===================>--------------------] 51.10% | train_error: 0.325

  [===================>--------------------] 51.20% | train_error: 0.324

  [===================>--------------------] 51.30% | train_error: 0.322

  [===================>--------------------] 51.40% | train_error: 0.321

  [===================>--------------------] 51.50% | train_error: 0.319

  [===================>--------------------] 51.60% | train_error: 0.318

  [===================>--------------------] 51.70% | train_error: 0.316

  [===================>--------------------] 51.80% | train_error: 0.315

  [===================>--------------------] 51.90% | train_error: 0.313

  [===================>--------------------] 52.00% | train_error: 0.312

  [===================>--------------------] 52.10% | train_error: 0.310

  [===================>--------------------] 52.20% | train_error: 0.309

  [===================>--------------------] 52.30% | train_error: 0.308

  [===================>--------------------] 52.40% | train_error: 0.306

  [====================>-------------------] 52.50% | train_error: 0.305

  [====================>-------------------] 52.60% | train_error: 0.303

  [====================>-------------------] 52.70% | train_error: 0.302

  [====================>-------------------] 52.80% | train_error: 0.300

  [====================>-------------------] 52.90% | train_error: 0.299

  [====================>-------------------] 53.00% | train_error: 0.298

  [====================>-------------------] 53.10% | train_error: 0.296

  [====================>-------------------] 53.20% | train_error: 0.295

  [====================>-------------------] 53.30% | train_error: 0.293

  [====================>-------------------] 53.40% | train_error: 0.292

  [====================>-------------------] 53.50% | train_error: 0.291

  [====================>-------------------] 53.60% | train_error: 0.289

  [====================>-------------------] 53.70% | train_error: 0.288

  [====================>-------------------] 53.80% | train_error: 0.287

  [====================>-------------------] 53.90% | train_error: 0.285

  [====================>-------------------] 54.00% | train_error: 0.284

  [====================>-------------------] 54.10% | train_error: 0.283

  [====================>-------------------] 54.20% | train_error: 0.281

  [====================>-------------------] 54.30% | train_error: 0.280

  [====================>-------------------] 54.40% | train_error: 0.279

  [====================>-------------------] 54.50% | train_error: 0.277

  [====================>-------------------] 54.60% | train_error: 0.276

  [====================>-------------------] 54.70% | train_error: 0.275

  [====================>-------------------] 54.80% | train_error: 0.273

  [====================>-------------------] 54.90% | train_error: 0.272

  [=====================>------------------] 55.00% | train_error: 0.271

  [=====================>------------------] 55.10% | train_error: 0.270

  [=====================>------------------] 55.20% | train_error: 0.268

  [=====================>------------------] 55.30% | train_error: 0.267

  [=====================>------------------] 55.40% | train_error: 0.266

  [=====================>------------------] 55.50% | train_error: 0.265

  [=====================>------------------] 55.60% | train_error: 0.263

  [=====================>------------------] 55.70% | train_error: 0.262

  [=====================>------------------] 55.80% | train_error: 0.261

  [=====================>------------------] 55.90% | train_error: 0.260

  [=====================>------------------] 56.00% | train_error: 0.259

  [=====================>------------------] 56.10% | train_error: 0.257

  [=====================>------------------] 56.20% | train_error: 0.256

  [=====================>------------------] 56.30% | train_error: 0.255

  [=====================>------------------] 56.40% | train_error: 0.254

  [=====================>------------------] 56.50% | train_error: 0.253

  [=====================>------------------] 56.60% | train_error: 0.251

  [=====================>------------------] 56.70% | train_error: 0.250

  [=====================>------------------] 56.80% | train_error: 0.249

  [=====================>------------------] 56.90% | train_error: 0.248

  [=====================>------------------] 57.00% | train_error: 0.247

  [=====================>------------------] 57.10% | train_error: 0.246

  [=====================>------------------] 57.20% | train_error: 0.244

  [=====================>------------------] 57.30% | train_error: 0.243

  [=====================>------------------] 57.40% | train_error: 0.242

  [======================>-----------------] 57.50% | train_error: 0.241

  [======================>-----------------] 57.60% | train_error: 0.240

  [======================>-----------------] 57.70% | train_error: 0.239

  [======================>-----------------] 57.80% | train_error: 0.238

  [======================>-----------------] 57.90% | train_error: 0.237

  [======================>-----------------] 58.00% | train_error: 0.235

  [======================>-----------------] 58.10% | train_error: 0.234

  [======================>-----------------] 58.20% | train_error: 0.233

  [======================>-----------------] 58.30% | train_error: 0.232

  [======================>-----------------] 58.40% | train_error: 0.231

  [======================>-----------------] 58.50% | train_error: 0.230

  [======================>-----------------] 58.60% | train_error: 0.229

  [======================>-----------------] 58.70% | train_error: 0.228

  [======================>-----------------] 58.80% | train_error: 0.227

  [======================>-----------------] 58.90% | train_error: 0.226

  [======================>-----------------] 59.00% | train_error: 0.225

  [======================>-----------------] 59.10% | train_error: 0.224

  [======================>-----------------] 59.20% | train_error: 0.223

  [======================>-----------------] 59.30% | train_error: 0.222

  [======================>-----------------] 59.40% | train_error: 0.221

  [======================>-----------------] 59.50% | train_error: 0.219

  [======================>-----------------] 59.60% | train_error: 0.218

  [======================>-----------------] 59.70% | train_error: 0.217

  [======================>-----------------] 59.80% | train_error: 0.216

  [======================>-----------------] 59.90% | train_error: 0.215

  [=======================>----------------] 60.00% | train_error: 0.214

  [=======================>----------------] 60.10% | train_error: 0.213

  [=======================>----------------] 60.20% | train_error: 0.212

  [=======================>----------------] 60.30% | train_error: 0.211

  [=======================>----------------] 60.40% | train_error: 0.210

  [=======================>----------------] 60.50% | train_error: 0.209

  [=======================>----------------] 60.60% | train_error: 0.209

  [=======================>----------------] 60.70% | train_error: 0.208

  [=======================>----------------] 60.80% | train_error: 0.207

  [=======================>----------------] 60.90% | train_error: 0.206

  [=======================>----------------] 61.00% | train_error: 0.205

  [=======================>----------------] 61.10% | train_error: 0.204

  [=======================>----------------] 61.20% | train_error: 0.203

  [=======================>----------------] 61.30% | train_error: 0.202

  [=======================>----------------] 61.40% | train_error: 0.201

  [=======================>----------------] 61.50% | train_error: 0.200

  [=======================>----------------] 61.60% | train_error: 0.199

  [=======================>----------------] 61.70% | train_error: 0.198

  [=======================>----------------] 61.80% | train_error: 0.197

  [=======================>----------------] 61.90% | train_error: 0.196

  [=======================>----------------] 62.00% | train_error: 0.195

  [=======================>----------------] 62.10% | train_error: 0.194

  [=======================>----------------] 62.20% | train_error: 0.194

  [=======================>----------------] 62.30% | train_error: 0.193

  [=======================>----------------] 62.40% | train_error: 0.192

  [========================>---------------] 62.50% | train_error: 0.191

  [========================>---------------] 62.60% | train_error: 0.190

  [========================>---------------] 62.70% | train_error: 0.189

  [========================>---------------] 62.80% | train_error: 0.188

  [========================>---------------] 62.90% | train_error: 0.187

  [========================>---------------] 63.00% | train_error: 0.186

  [========================>---------------] 63.10% | train_error: 0.186

  [========================>---------------] 63.20% | train_error: 0.185

  [========================>---------------] 63.30% | train_error: 0.184

  [========================>---------------] 63.40% | train_error: 0.183

  [========================>---------------] 63.50% | train_error: 0.182

  [========================>---------------] 63.60% | train_error: 0.181

  [========================>---------------] 63.70% | train_error: 0.180

  [========================>---------------] 63.80% | train_error: 0.180

  [========================>---------------] 63.90% | train_error: 0.179

  [========================>---------------] 64.00% | train_error: 0.178

  [========================>---------------] 64.10% | train_error: 0.177

  [========================>---------------] 64.20% | train_error: 0.176

  [========================>---------------] 64.30% | train_error: 0.176

  [========================>---------------] 64.40% | train_error: 0.175

  [========================>---------------] 64.50% | train_error: 0.174

  [========================>---------------] 64.60% | train_error: 0.173

  [========================>---------------] 64.70% | train_error: 0.172

  [========================>---------------] 64.80% | train_error: 0.171

  [========================>---------------] 64.90% | train_error: 0.171

  [=========================>--------------] 65.00% | train_error: 0.170

  [=========================>--------------] 65.10% | train_error: 0.169

  [=========================>--------------] 65.20% | train_error: 0.168

  [=========================>--------------] 65.30% | train_error: 0.168

  [=========================>--------------] 65.40% | train_error: 0.167

  [=========================>--------------] 65.50% | train_error: 0.166

  [=========================>--------------] 65.60% | train_error: 0.165

  [=========================>--------------] 65.70% | train_error: 0.164

  [=========================>--------------] 65.80% | train_error: 0.164

  [=========================>--------------] 65.90% | train_error: 0.163

  [=========================>--------------] 66.00% | train_error: 0.162

  [=========================>--------------] 66.10% | train_error: 0.161

  [=========================>--------------] 66.20% | train_error: 0.161

  [=========================>--------------] 66.30% | train_error: 0.160

  [=========================>--------------] 66.40% | train_error: 0.159

  [=========================>--------------] 66.50% | train_error: 0.158

  [=========================>--------------] 66.60% | train_error: 0.158

  [=========================>--------------] 66.70% | train_error: 0.157

  [=========================>--------------] 66.80% | train_error: 0.156

  [=========================>--------------] 66.90% | train_error: 0.156

  [=========================>--------------] 67.00% | train_error: 0.155

  [=========================>--------------] 67.10% | train_error: 0.154

  [=========================>--------------] 67.20% | train_error: 0.153

  [=========================>--------------] 67.30% | train_error: 0.153

  [=========================>--------------] 67.40% | train_error: 0.152

  [==========================>-------------] 67.50% | train_error: 0.151

  [==========================>-------------] 67.60% | train_error: 0.151

  [==========================>-------------] 67.70% | train_error: 0.150

  [==========================>-------------] 67.80% | train_error: 0.149

  [==========================>-------------] 67.90% | train_error: 0.149

  [==========================>-------------] 68.00% | train_error: 0.148

  [==========================>-------------] 68.10% | train_error: 0.147

  [==========================>-------------] 68.20% | train_error: 0.146

  [==========================>-------------] 68.30% | train_error: 0.146

  [==========================>-------------] 68.40% | train_error: 0.145

  [==========================>-------------] 68.50% | train_error: 0.144

  [==========================>-------------] 68.60% | train_error: 0.144

  [==========================>-------------] 68.70% | train_error: 0.143

  [==========================>-------------] 68.80% | train_error: 0.142

  [==========================>-------------] 68.90% | train_error: 0.142

  [==========================>-------------] 69.00% | train_error: 0.141

  [==========================>-------------] 69.10% | train_error: 0.141

  [==========================>-------------] 69.20% | train_error: 0.140

  [==========================>-------------] 69.30% | train_error: 0.139

  [==========================>-------------] 69.40% | train_error: 0.139

  [==========================>-------------] 69.50% | train_error: 0.138

  [==========================>-------------] 69.60% | train_error: 0.137

  [==========================>-------------] 69.70% | train_error: 0.137

  [==========================>-------------] 69.80% | train_error: 0.136

  [==========================>-------------] 69.90% | train_error: 0.135

  [===========================>------------] 70.00% | train_error: 0.135

  [===========================>------------] 70.10% | train_error: 0.134

  [===========================>------------] 70.20% | train_error: 0.134

  [===========================>------------] 70.30% | train_error: 0.133

  [===========================>------------] 70.40% | train_error: 0.132

  [===========================>------------] 70.50% | train_error: 0.132

  [===========================>------------] 70.60% | train_error: 0.131

  [===========================>------------] 70.70% | train_error: 0.131

  [===========================>------------] 70.80% | train_error: 0.130

  [===========================>------------] 70.90% | train_error: 0.129

  [===========================>------------] 71.00% | train_error: 0.129

  [===========================>------------] 71.10% | train_error: 0.128

  [===========================>------------] 71.20% | train_error: 0.128

  [===========================>------------] 71.30% | train_error: 0.127

  [===========================>------------] 71.40% | train_error: 0.126

  [===========================>------------] 71.50% | train_error: 0.126

  [===========================>------------] 71.60% | train_error: 0.125

  [===========================>------------] 71.70% | train_error: 0.125

  [===========================>------------] 71.80% | train_error: 0.124

  [===========================>------------] 71.90% | train_error: 0.124

  [===========================>------------] 72.00% | train_error: 0.123

  [===========================>------------] 72.10% | train_error: 0.122

  [===========================>------------] 72.20% | train_error: 0.122

  [===========================>------------] 72.30% | train_error: 0.121

  [===========================>------------] 72.40% | train_error: 0.121

  [============================>-----------] 72.50% | train_error: 0.120

  [============================>-----------] 72.60% | train_error: 0.120

  [============================>-----------] 72.70% | train_error: 0.119

  [============================>-----------] 72.80% | train_error: 0.119

  [============================>-----------] 72.90% | train_error: 0.118

  [============================>-----------] 73.00% | train_error: 0.117

  [============================>-----------] 73.10% | train_error: 0.117

  [============================>-----------] 73.20% | train_error: 0.116

  [============================>-----------] 73.30% | train_error: 0.116

  [============================>-----------] 73.40% | train_error: 0.115

  [============================>-----------] 73.50% | train_error: 0.115

  [============================>-----------] 73.60% | train_error: 0.114

  [============================>-----------] 73.70% | train_error: 0.114

  [============================>-----------] 73.80% | train_error: 0.113

  [============================>-----------] 73.90% | train_error: 0.113

  [============================>-----------] 74.00% | train_error: 0.112

  [============================>-----------] 74.10% | train_error: 0.112

  [============================>-----------] 74.20% | train_error: 0.111

  [============================>-----------] 74.30% | train_error: 0.111

  [============================>-----------] 74.40% | train_error: 0.110

  [============================>-----------] 74.50% | train_error: 0.110

  [============================>-----------] 74.60% | train_error: 0.109

  [============================>-----------] 74.70% | train_error: 0.109

  [============================>-----------] 74.80% | train_error: 0.108

  [============================>-----------] 74.90% | train_error: 0.108

  [=============================>----------] 75.00% | train_error: 0.107

  [=============================>----------] 75.10% | train_error: 0.107

  [=============================>----------] 75.20% | train_error: 0.106

  [=============================>----------] 75.30% | train_error: 0.106

  [=============================>----------] 75.40% | train_error: 0.105

  [=============================>----------] 75.50% | train_error: 0.105

  [=============================>----------] 75.60% | train_error: 0.104

  [=============================>----------] 75.70% | train_error: 0.104

  [=============================>----------] 75.80% | train_error: 0.103

  [=============================>----------] 75.90% | train_error: 0.103

  [=============================>----------] 76.00% | train_error: 0.102

  [=============================>----------] 76.10% | train_error: 0.102

  [=============================>----------] 76.20% | train_error: 0.101

  [=============================>----------] 76.30% | train_error: 0.101

  [=============================>----------] 76.40% | train_error: 0.101

  [=============================>----------] 76.50% | train_error: 0.100

  [=============================>----------] 76.60% | train_error: 0.0996

  [=============================>----------] 76.70% | train_error: 0.0992

  [=============================>----------] 76.80% | train_error: 0.0987

  [=============================>----------] 76.90% | train_error: 0.0983

  [=============================>----------] 77.00% | train_error: 0.0978

  [=============================>----------] 77.10% | train_error: 0.0974

  [=============================>----------] 77.20% | train_error: 0.0969

  [=============================>----------] 77.30% | train_error: 0.0965

  [=============================>----------] 77.40% | train_error: 0.0961

  [==============================>---------] 77.50% | train_error: 0.0956

  [==============================>---------] 77.60% | train_error: 0.0952

  [==============================>---------] 77.70% | train_error: 0.0948

  [==============================>---------] 77.80% | train_error: 0.0943

  [==============================>---------] 77.90% | train_error: 0.0939

  [==============================>---------] 78.00% | train_error: 0.0935

  [==============================>---------] 78.10% | train_error: 0.0930

  [==============================>---------] 78.20% | train_error: 0.0926

  [==============================>---------] 78.30% | train_error: 0.0922

  [==============================>---------] 78.40% | train_error: 0.0918

  [==============================>---------] 78.50% | train_error: 0.0914

  [==============================>---------] 78.60% | train_error: 0.0910

  [==============================>---------] 78.70% | train_error: 0.0905

  [==============================>---------] 78.80% | train_error: 0.0901

  [==============================>---------] 78.90% | train_error: 0.0897

  [==============================>---------] 79.00% | train_error: 0.0893

  [==============================>---------] 79.10% | train_error: 0.0889

  [==============================>---------] 79.20% | train_error: 0.0885

  [==============================>---------] 79.30% | train_error: 0.0881

  [==============================>---------] 79.40% | train_error: 0.0877

  [==============================>---------] 79.50% | train_error: 0.0873

  [==============================>---------] 79.60% | train_error: 0.0869

  [==============================>---------] 79.70% | train_error: 0.0865

  [==============================>---------] 79.80% | train_error: 0.0861

  [==============================>---------] 79.90% | train_error: 0.0858

  [===============================>--------] 80.00% | train_error: 0.0854

  [===============================>--------] 80.10% | train_error: 0.0850

  [===============================>--------] 80.20% | train_error: 0.0846

  [===============================>--------] 80.30% | train_error: 0.0842

  [===============================>--------] 80.40% | train_error: 0.0838

  [===============================>--------] 80.50% | train_error: 0.0835

  [===============================>--------] 80.60% | train_error: 0.0831

  [===============================>--------] 80.70% | train_error: 0.0827

  [===============================>--------] 80.80% | train_error: 0.0823

  [===============================>--------] 80.90% | train_error: 0.0820

  [===============================>--------] 81.00% | train_error: 0.0816

  [===============================>--------] 81.10% | train_error: 0.0812

  [===============================>--------] 81.20% | train_error: 0.0809

  [===============================>--------] 81.30% | train_error: 0.0805

  [===============================>--------] 81.40% | train_error: 0.0801

  [===============================>--------] 81.50% | train_error: 0.0798

  [===============================>--------] 81.60% | train_error: 0.0794

  [===============================>--------] 81.70% | train_error: 0.0791

  [===============================>--------] 81.80% | train_error: 0.0787

  [===============================>--------] 81.90% | train_error: 0.0784

  [===============================>--------] 82.00% | train_error: 0.0780

  [===============================>--------] 82.10% | train_error: 0.0776

  [===============================>--------] 82.20% | train_error: 0.0773

  [===============================>--------] 82.30% | train_error: 0.0770

  [===============================>--------] 82.40% | train_error: 0.0766

  [================================>-------] 82.50% | train_error: 0.0763

  [================================>-------] 82.60% | train_error: 0.0759

  [================================>-------] 82.70% | train_error: 0.0756

  [================================>-------] 82.80% | train_error: 0.0752

  [================================>-------] 82.90% | train_error: 0.0749

  [================================>-------] 83.00% | train_error: 0.0746

  [================================>-------] 83.10% | train_error: 0.0742

  [================================>-------] 83.20% | train_error: 0.0739

  [================================>-------] 83.30% | train_error: 0.0736

  [================================>-------] 83.40% | train_error: 0.0732

  [================================>-------] 83.50% | train_error: 0.0729

  [================================>-------] 83.60% | train_error: 0.0726

  [================================>-------] 83.70% | train_error: 0.0723

  [================================>-------] 83.80% | train_error: 0.0719

  [================================>-------] 83.90% | train_error: 0.0716

  [================================>-------] 84.00% | train_error: 0.0713

  [================================>-------] 84.10% | train_error: 0.0710

  [================================>-------] 84.20% | train_error: 0.0707

  [================================>-------] 84.30% | train_error: 0.0703

  [================================>-------] 84.40% | train_error: 0.0700

  [================================>-------] 84.50% | train_error: 0.0697

  [================================>-------] 84.60% | train_error: 0.0694

  [================================>-------] 84.70% | train_error: 0.0691

  [================================>-------] 84.80% | train_error: 0.0688

  [================================>-------] 84.90% | train_error: 0.0685

  [=================================>------] 85.00% | train_error: 0.0682

  [=================================>------] 85.10% | train_error: 0.0679

  [=================================>------] 85.20% | train_error: 0.0676

  [=================================>------] 85.30% | train_error: 0.0673

  [=================================>------] 85.40% | train_error: 0.0670

  [=================================>------] 85.50% | train_error: 0.0667

  [=================================>------] 85.60% | train_error: 0.0664

  [=================================>------] 85.70% | train_error: 0.0661

  [=================================>------] 85.80% | train_error: 0.0658

  [=================================>------] 85.90% | train_error: 0.0655

  [=================================>------] 86.00% | train_error: 0.0652

  [=================================>------] 86.10% | train_error: 0.0649

  [=================================>------] 86.20% | train_error: 0.0646

  [=================================>------] 86.30% | train_error: 0.0643

  [=================================>------] 86.40% | train_error: 0.0641

  [=================================>------] 86.50% | train_error: 0.0638

  [=================================>------] 86.60% | train_error: 0.0635

  [=================================>------] 86.70% | train_error: 0.0632

  [=================================>------] 86.80% | train_error: 0.0629

  [=================================>------] 86.90% | train_error: 0.0626

  [=================================>------] 87.00% | train_error: 0.0624

  [=================================>------] 87.10% | train_error: 0.0621

  [=================================>------] 87.20% | train_error: 0.0618

  [=================================>------] 87.30% | train_error: 0.0615

  [=================================>------] 87.40% | train_error: 0.0613

  [==================================>-----] 87.50% | train_error: 0.0610

  [==================================>-----] 87.60% | train_error: 0.0607

  [==================================>-----] 87.70% | train_error: 0.0605

  [==================================>-----] 87.80% | train_error: 0.0602

  [==================================>-----] 87.90% | train_error: 0.0599

  [==================================>-----] 88.00% | train_error: 0.0597

  [==================================>-----] 88.10% | train_error: 0.0594

  [==================================>-----] 88.20% | train_error: 0.0591

  [==================================>-----] 88.30% | train_error: 0.0589

  [==================================>-----] 88.40% | train_error: 0.0586

  [==================================>-----] 88.50% | train_error: 0.0584

  [==================================>-----] 88.60% | train_error: 0.0581

  [==================================>-----] 88.70% | train_error: 0.0578

  [==================================>-----] 88.80% | train_error: 0.0576

  [==================================>-----] 88.90% | train_error: 0.0573

  [==================================>-----] 89.00% | train_error: 0.0571

  [==================================>-----] 89.10% | train_error: 0.0568

  [==================================>-----] 89.20% | train_error: 0.0566

  [==================================>-----] 89.30% | train_error: 0.0563

  [==================================>-----] 89.40% | train_error: 0.0561

  [==================================>-----] 89.50% | train_error: 0.0558

  [==================================>-----] 89.60% | train_error: 0.0556

  [==================================>-----] 89.70% | train_error: 0.0553

  [==================================>-----] 89.80% | train_error: 0.0551

  [==================================>-----] 89.90% | train_error: 0.0549

  [===================================>----] 90.00% | train_error: 0.0546

  [===================================>----] 90.10% | train_error: 0.0544

  [===================================>----] 90.20% | train_error: 0.0541

  [===================================>----] 90.30% | train_error: 0.0539

  [===================================>----] 90.40% | train_error: 0.0537

  [===================================>----] 90.50% | train_error: 0.0534

  [===================================>----] 90.60% | train_error: 0.0532

  [===================================>----] 90.70% | train_error: 0.0530

  [===================================>----] 90.80% | train_error: 0.0527

  [===================================>----] 90.90% | train_error: 0.0525

  [===================================>----] 91.00% | train_error: 0.0523

  [===================================>----] 91.10% | train_error: 0.0520

  [===================================>----] 91.20% | train_error: 0.0518

  [===================================>----] 91.30% | train_error: 0.0516

  [===================================>----] 91.40% | train_error: 0.0514

  [===================================>----] 91.50% | train_error: 0.0511

  [===================================>----] 91.60% | train_error: 0.0509

  [===================================>----] 91.70% | train_error: 0.0507

  [===================================>----] 91.80% | train_error: 0.0505

  [===================================>----] 91.90% | train_error: 0.0503

  [===================================>----] 92.00% | train_error: 0.0500

  [===================================>----] 92.10% | train_error: 0.0498

  [===================================>----] 92.20% | train_error: 0.0496

  [===================================>----] 92.30% | train_error: 0.0494

  [===================================>----] 92.40% | train_error: 0.0492

  [====================================>---] 92.50% | train_error: 0.0490

  [====================================>---] 92.60% | train_error: 0.0487

  [====================================>---] 92.70% | train_error: 0.0485

  [====================================>---] 92.80% | train_error: 0.0483

  [====================================>---] 92.90% | train_error: 0.0481

  [====================================>---] 93.00% | train_error: 0.0479

  [====================================>---] 93.10% | train_error: 0.0477

  [====================================>---] 93.20% | train_error: 0.0475

  [====================================>---] 93.30% | train_error: 0.0473

  [====================================>---] 93.40% | train_error: 0.0471

  [====================================>---] 93.50% | train_error: 0.0469

  [====================================>---] 93.60% | train_error: 0.0467

  [====================================>---] 93.70% | train_error: 0.0465

  [====================================>---] 93.80% | train_error: 0.0463

  [====================================>---] 93.90% | train_error: 0.0461

  [====================================>---] 94.00% | train_error: 0.0459

  [====================================>---] 94.10% | train_error: 0.0457

  [====================================>---] 94.20% | train_error: 0.0455

  [====================================>---] 94.30% | train_error: 0.0453

  [====================================>---] 94.40% | train_error: 0.0451

  [====================================>---] 94.50% | train_error: 0.0449

  [====================================>---] 94.60% | train_error: 0.0447

  [====================================>---] 94.70% | train_error: 0.0445

  [====================================>---] 94.80% | train_error: 0.0443

  [====================================>---] 94.90% | train_error: 0.0441

  [=====================================>--] 95.00% | train_error: 0.0439

  [=====================================>--] 95.10% | train_error: 0.0437

  [=====================================>--] 95.20% | train_error: 0.0435

  [=====================================>--] 95.30% | train_error: 0.0434

  [=====================================>--] 95.40% | train_error: 0.0432

  [=====================================>--] 95.50% | train_error: 0.0430

  [=====================================>--] 95.60% | train_error: 0.0428

  [=====================================>--] 95.70% | train_error: 0.0426

  [=====================================>--] 95.80% | train_error: 0.0424

  [=====================================>--] 95.90% | train_error: 0.0423

  [=====================================>--] 96.00% | train_error: 0.0421

  [=====================================>--] 96.10% | train_error: 0.0419

  [=====================================>--] 96.20% | train_error: 0.0417

  [=====================================>--] 96.30% | train_error: 0.0415

  [=====================================>--] 96.40% | train_error: 0.0414

  [=====================================>--] 96.50% | train_error: 0.0412

  [=====================================>--] 96.60% | train_error: 0.0410

  [=====================================>--] 96.70% | train_error: 0.0408

  [=====================================>--] 96.80% | train_error: 0.0407

  [=====================================>--] 96.90% | train_error: 0.0405

  [=====================================>--] 97.00% | train_error: 0.0403

  [=====================================>--] 97.10% | train_error: 0.0401

  [=====================================>--] 97.20% | train_error: 0.0400

  [=====================================>--] 97.30% | train_error: 0.0398

  [=====================================>--] 97.40% | train_error: 0.0396

  [======================================>-] 97.50% | train_error: 0.0395

  [======================================>-] 97.60% | train_error: 0.0393

  [======================================>-] 97.70% | train_error: 0.0391

  [======================================>-] 97.80% | train_error: 0.0389

  [======================================>-] 97.90% | train_error: 0.0388

  [======================================>-] 98.00% | train_error: 0.0386

  [======================================>-] 98.10% | train_error: 0.0385

  [======================================>-] 98.20% | train_error: 0.0383

  [======================================>-] 98.30% | train_error: 0.0381

  [======================================>-] 98.40% | train_error: 0.0380

  [======================================>-] 98.50% | train_error: 0.0378

  [======================================>-] 98.60% | train_error: 0.0376

  [======================================>-] 98.70% | train_error: 0.0375

  [======================================>-] 98.80% | train_error: 0.0373

  [======================================>-] 98.90% | train_error: 0.0372

  [======================================>-] 99.00% | train_error: 0.0370

  [======================================>-] 99.10% | train_error: 0.0369

  [======================================>-] 99.20% | train_error: 0.0367

  [======================================>-] 99.30% | train_error: 0.0365

  [======================================>-] 99.40% | train_error: 0.0364

  [======================================>-] 99.50% | train_error: 0.0362

  [======================================>-] 99.60% | train_error: 0.0361

  [======================================>-] 99.70% | train_error: 0.0359

  [======================================>-] 99.80% | train_error: 0.0358

  [======================================>-] 99.90% | train_error: 0.0356

  [=======================================>] 100.0% | train_error: 0.0356

We see that given more epochs to train on, the regressor reaches a lower MSE.

Let us then switch to a binary classification. We use a binary classification dataset, and follow a similar setup to the regression case.

from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import MinMaxScaler

wisconsin = load_breast_cancer()
X = wisconsin.data
target = wisconsin.target
target = target.reshape(target.shape[0], 1)

X_train, X_val, t_train, t_val = train_test_split(X, target)

scaler = MinMaxScaler()
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_val = scaler.transform(X_val)

input_nodes = X_train.shape[1]
output_nodes = 1

logistic_regression = FFNN((input_nodes, output_nodes), output_func=sigmoid, cost_func=CostLogReg, seed=2023)

We will now make use of our validation data by passing it into our fit function as a keyword argument

logistic_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights

scheduler = Adam(eta=1e-3, rho=0.9, rho2=0.999)
scores = logistic_regression.fit(X_train, t_train, scheduler, epochs=1000, X_val=X_val, t_val=t_val)

Adam: Eta=0.001, Lambda=0

  [----------------------------------------] 0.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.1000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.2000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.3000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.4000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.5000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.6000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.7000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.8000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 0.9000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 1.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 2.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 2.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 2.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 2.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [----------------------------------------] 2.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 2.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 2.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 2.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 2.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 2.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 3.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [>---------------------------------------] 4.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 5.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 6.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 7.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 7.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [=>--------------------------------------] 7.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [=>--------------------------------------] 7.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [=>--------------------------------------] 7.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 7.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 7.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 7.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 7.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 7.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 8.900% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.000% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.100% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.200% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.300% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.400% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.500% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.600% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.700% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.800% | train_error: 12.9 | train_acc: 0.376 | val_error: 12.8 | val_acc: 0.385 

  [==>-------------------------------------] 9.900% | train_error: 13.0 | train_acc: 0.373 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.00% | train_error: 13.0 | train_acc: 0.373 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.10% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.20% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.30% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.40% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.50% | train_error: 13.0 | train_acc: 0.371 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.60% | train_error: 13.0 | train_acc: 0.371 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.70% | train_error: 13.0 | train_acc: 0.371 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.80% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 10.90% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.00% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.10% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.20% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.30% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.40% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.50% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.60% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.70% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.80% | train_error: 13.1 | train_acc: 0.369 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 11.90% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 12.00% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 12.10% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 12.20% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 12.30% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [===>------------------------------------] 12.40% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 12.50% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 12.60% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 12.70% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 12.80% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 12.90% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.00% | train_error: 13.1 | train_acc: 0.366 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.10% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.20% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.30% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.40% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.50% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.60% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.70% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.8 | val_acc: 0.385 

  [====>-----------------------------------] 13.80% | train_error: 13.3 | train_acc: 0.359 | val_error: 12.9 | val_acc: 0.378 

  [====>-----------------------------------] 13.90% | train_error: 13.4 | train_acc: 0.354 | val_error: 12.9 | val_acc: 0.378 

  [====>-----------------------------------] 14.00% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.0 | val_acc: 0.371 

  [====>-----------------------------------] 14.10% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.0 | val_acc: 0.371 

  [====>-----------------------------------] 14.20% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.0 | val_acc: 0.371 

  [====>-----------------------------------] 14.30% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.0 | val_acc: 0.371 

  [====>-----------------------------------] 14.40% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.0 | val_acc: 0.371 

  [====>-----------------------------------] 14.50% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.0 | val_acc: 0.371 

  [====>-----------------------------------] 14.60% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.2 | val_acc: 0.364 

  [====>-----------------------------------] 14.70% | train_error: 13.4 | train_acc: 0.354 | val_error: 13.2 | val_acc: 0.364 

  [====>-----------------------------------] 14.80% | train_error: 13.4 | train_acc: 0.352 | val_error: 13.2 | val_acc: 0.364 

  [====>-----------------------------------] 14.90% | train_error: 13.4 | train_acc: 0.352 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.00% | train_error: 13.5 | train_acc: 0.347 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.10% | train_error: 13.6 | train_acc: 0.345 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.20% | train_error: 13.6 | train_acc: 0.345 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.30% | train_error: 13.8 | train_acc: 0.336 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.40% | train_error: 13.8 | train_acc: 0.336 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.50% | train_error: 13.8 | train_acc: 0.336 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.60% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.70% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.80% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 15.90% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.00% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.10% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.20% | train_error: 13.8 | train_acc: 0.333 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.30% | train_error: 14.0 | train_acc: 0.326 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.40% | train_error: 14.0 | train_acc: 0.326 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.50% | train_error: 14.0 | train_acc: 0.326 | val_error: 13.2 | val_acc: 0.364 

  [=====>----------------------------------] 16.60% | train_error: 14.0 | train_acc: 0.326 | val_error: 13.3 | val_acc: 0.357 

  [=====>----------------------------------] 16.70% | train_error: 14.1 | train_acc: 0.322 | val_error: 13.5 | val_acc: 0.350 

  [=====>----------------------------------] 16.80% | train_error: 14.1 | train_acc: 0.322 | val_error: 13.5 | val_acc: 0.350 

  [=====>----------------------------------] 16.90% | train_error: 14.1 | train_acc: 0.319 | val_error: 13.6 | val_acc: 0.343 

  [=====>----------------------------------] 17.00% | train_error: 14.1 | train_acc: 0.319 | val_error: 13.6 | val_acc: 0.343 

  [=====>----------------------------------] 17.10% | train_error: 14.1 | train_acc: 0.319 | val_error: 13.6 | val_acc: 0.343 

  [=====>----------------------------------] 17.20% | train_error: 14.2 | train_acc: 0.317 | val_error: 13.6 | val_acc: 0.343 

  [=====>----------------------------------] 17.30% | train_error: 14.2 | train_acc: 0.317 | val_error: 13.6 | val_acc: 0.343 

  [=====>----------------------------------] 17.40% | train_error: 14.2 | train_acc: 0.317 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 17.50% | train_error: 14.2 | train_acc: 0.317 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 17.60% | train_error: 14.3 | train_acc: 0.312 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 17.70% | train_error: 14.4 | train_acc: 0.308 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 17.80% | train_error: 14.4 | train_acc: 0.308 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 17.90% | train_error: 14.4 | train_acc: 0.308 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.00% | train_error: 14.4 | train_acc: 0.308 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.10% | train_error: 14.4 | train_acc: 0.305 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.20% | train_error: 14.4 | train_acc: 0.305 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.30% | train_error: 14.4 | train_acc: 0.305 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.40% | train_error: 14.5 | train_acc: 0.298 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.50% | train_error: 14.5 | train_acc: 0.298 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.60% | train_error: 14.6 | train_acc: 0.296 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.70% | train_error: 14.6 | train_acc: 0.293 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.80% | train_error: 14.7 | train_acc: 0.291 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 18.90% | train_error: 14.7 | train_acc: 0.291 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 19.00% | train_error: 14.7 | train_acc: 0.291 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 19.10% | train_error: 14.7 | train_acc: 0.291 | val_error: 13.6 | val_acc: 0.343 

  [======>---------------------------------] 19.20% | train_error: 14.7 | train_acc: 0.291 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.30% | train_error: 14.8 | train_acc: 0.286 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.40% | train_error: 14.8 | train_acc: 0.284 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.50% | train_error: 14.8 | train_acc: 0.284 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.60% | train_error: 14.9 | train_acc: 0.282 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.70% | train_error: 14.9 | train_acc: 0.282 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.80% | train_error: 14.9 | train_acc: 0.282 | val_error: 13.5 | val_acc: 0.350 

  [======>---------------------------------] 19.90% | train_error: 14.9 | train_acc: 0.282 | val_error: 13.5 | val_acc: 0.350 

  [=======>--------------------------------] 20.00% | train_error: 14.9 | train_acc: 0.279 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.10% | train_error: 15.1 | train_acc: 0.272 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.20% | train_error: 15.1 | train_acc: 0.272 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.30% | train_error: 15.1 | train_acc: 0.272 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.40% | train_error: 15.1 | train_acc: 0.270 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.50% | train_error: 15.1 | train_acc: 0.270 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.60% | train_error: 15.4 | train_acc: 0.258 | val_error: 13.6 | val_acc: 0.343 

  [=======>--------------------------------] 20.70% | train_error: 15.4 | train_acc: 0.258 | val_error: 13.8 | val_acc: 0.336 

  [=======>--------------------------------] 20.80% | train_error: 15.4 | train_acc: 0.258 | val_error: 13.8 | val_acc: 0.336 

  [=======>--------------------------------] 20.90% | train_error: 15.4 | train_acc: 0.258 | val_error: 13.8 | val_acc: 0.336 

  [=======>--------------------------------] 21.00% | train_error: 15.4 | train_acc: 0.258 | val_error: 13.8 | val_acc: 0.336 

  [=======>--------------------------------] 21.10% | train_error: 15.4 | train_acc: 0.258 | val_error: 13.8 | val_acc: 0.336 

  [=======>--------------------------------] 21.20% | train_error: 15.3 | train_acc: 0.261 | val_error: 13.9 | val_acc: 0.329 

  [=======>--------------------------------] 21.30% | train_error: 15.5 | train_acc: 0.254 | val_error: 13.9 | val_acc: 0.329 

  [=======>--------------------------------] 21.40% | train_error: 15.5 | train_acc: 0.254 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 21.50% | train_error: 15.5 | train_acc: 0.254 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 21.60% | train_error: 15.5 | train_acc: 0.254 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 21.70% | train_error: 15.4 | train_acc: 0.256 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 21.80% | train_error: 15.4 | train_acc: 0.256 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 21.90% | train_error: 15.4 | train_acc: 0.256 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 22.00% | train_error: 15.4 | train_acc: 0.256 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 22.10% | train_error: 15.6 | train_acc: 0.249 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 22.20% | train_error: 15.6 | train_acc: 0.249 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 22.30% | train_error: 15.6 | train_acc: 0.249 | val_error: 14.2 | val_acc: 0.315 

  [=======>--------------------------------] 22.40% | train_error: 15.7 | train_acc: 0.244 | val_error: 14.3 | val_acc: 0.308 

  [========>-------------------------------] 22.50% | train_error: 15.7 | train_acc: 0.242 | val_error: 14.3 | val_acc: 0.308 

  [========>-------------------------------] 22.60% | train_error: 15.7 | train_acc: 0.242 | val_error: 14.3 | val_acc: 0.308 

  [========>-------------------------------] 22.70% | train_error: 15.7 | train_acc: 0.242 | val_error: 14.5 | val_acc: 0.301 

  [========>-------------------------------] 22.80% | train_error: 15.7 | train_acc: 0.242 | val_error: 14.5 | val_acc: 0.301 

  [========>-------------------------------] 22.90% | train_error: 15.8 | train_acc: 0.239 | val_error: 14.5 | val_acc: 0.301 

  [========>-------------------------------] 23.00% | train_error: 15.9 | train_acc: 0.235 | val_error: 14.5 | val_acc: 0.301 

  [========>-------------------------------] 23.10% | train_error: 15.9 | train_acc: 0.235 | val_error: 14.5 | val_acc: 0.301 

  [========>-------------------------------] 23.20% | train_error: 15.9 | train_acc: 0.232 | val_error: 14.6 | val_acc: 0.294 

  [========>-------------------------------] 23.30% | train_error: 15.9 | train_acc: 0.232 | val_error: 14.6 | val_acc: 0.294 

  [========>-------------------------------] 23.40% | train_error: 15.9 | train_acc: 0.232 | val_error: 14.6 | val_acc: 0.294 

  [========>-------------------------------] 23.50% | train_error: 15.9 | train_acc: 0.232 | val_error: 14.9 | val_acc: 0.280 

  [========>-------------------------------] 23.60% | train_error: 15.9 | train_acc: 0.232 | val_error: 14.8 | val_acc: 0.287 

  [========>-------------------------------] 23.70% | train_error: 15.9 | train_acc: 0.232 | val_error: 14.8 | val_acc: 0.287 

  [========>-------------------------------] 23.80% | train_error: 16.0 | train_acc: 0.230 | val_error: 14.8 | val_acc: 0.287 

  [========>-------------------------------] 23.90% | train_error: 16.0 | train_acc: 0.230 | val_error: 14.9 | val_acc: 0.280 

  [========>-------------------------------] 24.00% | train_error: 16.0 | train_acc: 0.230 | val_error: 14.9 | val_acc: 0.280 

  [========>-------------------------------] 24.10% | train_error: 16.0 | train_acc: 0.230 | val_error: 15.1 | val_acc: 0.273 

  [========>-------------------------------] 24.20% | train_error: 16.0 | train_acc: 0.230 | val_error: 15.1 | val_acc: 0.273 

  [========>-------------------------------] 24.30% | train_error: 16.0 | train_acc: 0.230 | val_error: 15.1 | val_acc: 0.273 

  [========>-------------------------------] 24.40% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.2 | val_acc: 0.266 

  [========>-------------------------------] 24.50% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.4 | val_acc: 0.259 

  [========>-------------------------------] 24.60% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.4 | val_acc: 0.259 

  [========>-------------------------------] 24.70% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.4 | val_acc: 0.259 

  [========>-------------------------------] 24.80% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.5 | val_acc: 0.252 

  [========>-------------------------------] 24.90% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.5 | val_acc: 0.252 

  [=========>------------------------------] 25.00% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.5 | val_acc: 0.252 

  [=========>------------------------------] 25.10% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.5 | val_acc: 0.252 

  [=========>------------------------------] 25.20% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.5 | val_acc: 0.252 

  [=========>------------------------------] 25.30% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.5 | val_acc: 0.252 

  [=========>------------------------------] 25.40% | train_error: 16.0 | train_acc: 0.228 | val_error: 15.7 | val_acc: 0.245 

  [=========>------------------------------] 25.50% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 25.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 25.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 25.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 25.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.00% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.10% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.20% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.30% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.40% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.50% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 26.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 27.00% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 27.10% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.8 | val_acc: 0.238 

  [=========>------------------------------] 27.20% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [=========>------------------------------] 27.30% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [=========>------------------------------] 27.40% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 27.50% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 27.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 27.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 27.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 27.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.00% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.10% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.20% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.30% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.40% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.50% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 28.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.00% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.10% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.20% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.30% | train_error: 16.0 | train_acc: 0.230 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.40% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.50% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [==========>-----------------------------] 29.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.00% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.10% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.20% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.30% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.40% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.50% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.60% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.70% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.80% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 30.90% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.8 | val_acc: 0.238 

  [===========>----------------------------] 31.00% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.10% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.20% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.30% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.40% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.50% | train_error: 15.7 | train_acc: 0.242 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.60% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.70% | train_error: 15.5 | train_acc: 0.251 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.80% | train_error: 15.5 | train_acc: 0.251 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 31.90% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 32.00% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 32.10% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 32.20% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 32.30% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [===========>----------------------------] 32.40% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 32.50% | train_error: 15.6 | train_acc: 0.246 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 32.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 32.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 32.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 32.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.00% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.10% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.20% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.30% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.40% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.50% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.60% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.70% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.80% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 33.90% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.7 | val_acc: 0.245 

  [============>---------------------------] 34.00% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.10% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.20% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.30% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.40% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.50% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.60% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.70% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.80% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [============>---------------------------] 34.90% | train_error: 15.9 | train_acc: 0.235 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.00% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.10% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.20% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.30% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.40% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.50% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.60% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.70% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.80% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 35.90% | train_error: 15.8 | train_acc: 0.237 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 36.00% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 36.10% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 36.20% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.5 | val_acc: 0.252 

  [=============>--------------------------] 36.30% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 36.40% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 36.50% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 36.60% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 36.70% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 36.80% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 36.90% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 37.00% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 37.10% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 37.20% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 37.30% | train_error: 15.7 | train_acc: 0.244 | val_error: 15.4 | val_acc: 0.259 

  [=============>--------------------------] 37.40% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.4 | val_acc: 0.259 

  [==============>-------------------------] 37.50% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 37.60% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 37.70% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 37.80% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 37.90% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.00% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.10% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.20% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.30% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.40% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.50% | train_error: 15.6 | train_acc: 0.249 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.60% | train_error: 15.4 | train_acc: 0.256 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.70% | train_error: 15.3 | train_acc: 0.261 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.80% | train_error: 15.3 | train_acc: 0.261 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 38.90% | train_error: 15.4 | train_acc: 0.258 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.00% | train_error: 15.4 | train_acc: 0.258 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.10% | train_error: 15.4 | train_acc: 0.258 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.20% | train_error: 15.4 | train_acc: 0.258 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.30% | train_error: 15.2 | train_acc: 0.268 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.40% | train_error: 15.2 | train_acc: 0.268 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.50% | train_error: 15.2 | train_acc: 0.268 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.60% | train_error: 15.2 | train_acc: 0.268 | val_error: 15.2 | val_acc: 0.266 

  [==============>-------------------------] 39.70% | train_error: 15.2 | train_acc: 0.268 | val_error: 15.1 | val_acc: 0.273 

  [==============>-------------------------] 39.80% | train_error: 15.2 | train_acc: 0.268 | val_error: 15.1 | val_acc: 0.273 

  [==============>-------------------------] 39.90% | train_error: 15.1 | train_acc: 0.272 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.00% | train_error: 15.1 | train_acc: 0.272 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.10% | train_error: 15.1 | train_acc: 0.272 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.20% | train_error: 15.1 | train_acc: 0.272 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.30% | train_error: 15.1 | train_acc: 0.272 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.40% | train_error: 15.1 | train_acc: 0.272 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.50% | train_error: 15.0 | train_acc: 0.277 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.60% | train_error: 15.0 | train_acc: 0.277 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.70% | train_error: 15.0 | train_acc: 0.277 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.80% | train_error: 15.0 | train_acc: 0.277 | val_error: 15.1 | val_acc: 0.273 

  [===============>------------------------] 40.90% | train_error: 15.0 | train_acc: 0.277 | val_error: 14.9 | val_acc: 0.280 

  [===============>------------------------] 41.00% | train_error: 15.0 | train_acc: 0.277 | val_error: 14.9 | val_acc: 0.280 

  [===============>------------------------] 41.10% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.9 | val_acc: 0.280 

  [===============>------------------------] 41.20% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.6 | val_acc: 0.294 

  [===============>------------------------] 41.30% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.5 | val_acc: 0.301 

  [===============>------------------------] 41.40% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.5 | val_acc: 0.301 

  [===============>------------------------] 41.50% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.3 | val_acc: 0.308 

  [===============>------------------------] 41.60% | train_error: 15.0 | train_acc: 0.277 | val_error: 14.3 | val_acc: 0.308 

  [===============>------------------------] 41.70% | train_error: 15.0 | train_acc: 0.277 | val_error: 14.3 | val_acc: 0.308 

  [===============>------------------------] 41.80% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.3 | val_acc: 0.308 

  [===============>------------------------] 41.90% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [===============>------------------------] 42.00% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [===============>------------------------] 42.10% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [===============>------------------------] 42.20% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [===============>------------------------] 42.30% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [===============>------------------------] 42.40% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 42.50% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 42.60% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 42.70% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 42.80% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 42.90% | train_error: 14.9 | train_acc: 0.279 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.00% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.1 | val_acc: 0.322 

  [================>-----------------------] 43.10% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.1 | val_acc: 0.322 

  [================>-----------------------] 43.20% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.30% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.40% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.50% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.60% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.70% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.80% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 43.90% | train_error: 14.9 | train_acc: 0.282 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.00% | train_error: 14.8 | train_acc: 0.284 | val_error: 14.1 | val_acc: 0.322 

  [================>-----------------------] 44.10% | train_error: 14.8 | train_acc: 0.284 | val_error: 14.1 | val_acc: 0.322 

  [================>-----------------------] 44.20% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.1 | val_acc: 0.322 

  [================>-----------------------] 44.30% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.40% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.50% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.60% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.70% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.80% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [================>-----------------------] 44.90% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [=================>----------------------] 45.00% | train_error: 14.7 | train_acc: 0.289 | val_error: 14.2 | val_acc: 0.315 

  [=================>----------------------] 45.10% | train_error: 14.7 | train_acc: 0.291 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.20% | train_error: 14.6 | train_acc: 0.296 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.30% | train_error: 14.6 | train_acc: 0.296 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.40% | train_error: 14.5 | train_acc: 0.300 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.50% | train_error: 14.5 | train_acc: 0.300 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.60% | train_error: 14.4 | train_acc: 0.308 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.70% | train_error: 14.4 | train_acc: 0.308 | val_error: 14.1 | val_acc: 0.322 

  [=================>----------------------] 45.80% | train_error: 14.4 | train_acc: 0.308 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 45.90% | train_error: 14.4 | train_acc: 0.308 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.00% | train_error: 14.3 | train_acc: 0.310 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.10% | train_error: 14.3 | train_acc: 0.310 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.20% | train_error: 14.3 | train_acc: 0.312 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.30% | train_error: 14.2 | train_acc: 0.315 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.40% | train_error: 14.2 | train_acc: 0.315 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.50% | train_error: 14.2 | train_acc: 0.315 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.60% | train_error: 14.2 | train_acc: 0.315 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.70% | train_error: 14.2 | train_acc: 0.315 | val_error: 13.9 | val_acc: 0.329 

  [=================>----------------------] 46.80% | train_error: 14.1 | train_acc: 0.319 | val_error: 13.8 | val_acc: 0.336 

  [=================>----------------------] 46.90% | train_error: 14.1 | train_acc: 0.319 | val_error: 13.8 | val_acc: 0.336 

  [=================>----------------------] 47.00% | train_error: 14.1 | train_acc: 0.322 | val_error: 13.8 | val_acc: 0.336 

  [=================>----------------------] 47.10% | train_error: 14.1 | train_acc: 0.322 | val_error: 13.8 | val_acc: 0.336 

  [=================>----------------------] 47.20% | train_error: 14.1 | train_acc: 0.322 | val_error: 13.8 | val_acc: 0.336 

  [=================>----------------------] 47.30% | train_error: 14.0 | train_acc: 0.324 | val_error: 13.8 | val_acc: 0.336 

  [=================>----------------------] 47.40% | train_error: 14.0 | train_acc: 0.324 | val_error: 13.8 | val_acc: 0.336 

  [==================>---------------------] 47.50% | train_error: 14.0 | train_acc: 0.324 | val_error: 13.8 | val_acc: 0.336 

  [==================>---------------------] 47.60% | train_error: 14.0 | train_acc: 0.326 | val_error: 13.6 | val_acc: 0.343 

  [==================>---------------------] 47.70% | train_error: 13.9 | train_acc: 0.329 | val_error: 13.5 | val_acc: 0.350 

  [==================>---------------------] 47.80% | train_error: 13.9 | train_acc: 0.329 | val_error: 13.5 | val_acc: 0.350 

  [==================>---------------------] 47.90% | train_error: 13.7 | train_acc: 0.338 | val_error: 13.5 | val_acc: 0.350 

  [==================>---------------------] 48.00% | train_error: 13.6 | train_acc: 0.343 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.10% | train_error: 13.6 | train_acc: 0.343 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.20% | train_error: 13.6 | train_acc: 0.343 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.30% | train_error: 13.6 | train_acc: 0.343 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.40% | train_error: 13.6 | train_acc: 0.343 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.50% | train_error: 13.5 | train_acc: 0.347 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.60% | train_error: 13.5 | train_acc: 0.347 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.70% | train_error: 13.5 | train_acc: 0.347 | val_error: 13.3 | val_acc: 0.357 

  [==================>---------------------] 48.80% | train_error: 13.5 | train_acc: 0.347 | val_error: 13.2 | val_acc: 0.364 

  [==================>---------------------] 48.90% | train_error: 13.4 | train_acc: 0.352 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.00% | train_error: 13.4 | train_acc: 0.352 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.10% | train_error: 13.4 | train_acc: 0.352 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.20% | train_error: 13.4 | train_acc: 0.352 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.30% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.40% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.50% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.60% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.70% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.80% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [==================>---------------------] 49.90% | train_error: 13.2 | train_acc: 0.362 | val_error: 12.9 | val_acc: 0.378 

  [===================>--------------------] 50.00% | train_error: 13.2 | train_acc: 0.364 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 50.10% | train_error: 13.2 | train_acc: 0.364 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.20% | train_error: 13.0 | train_acc: 0.373 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.30% | train_error: 13.0 | train_acc: 0.373 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.40% | train_error: 13.0 | train_acc: 0.371 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.50% | train_error: 13.0 | train_acc: 0.371 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.60% | train_error: 13.0 | train_acc: 0.371 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.70% | train_error: 12.8 | train_acc: 0.380 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.80% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 50.90% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 51.00% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.10% | train_error: 12.7 | train_acc: 0.385 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.20% | train_error: 12.7 | train_acc: 0.385 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.30% | train_error: 12.7 | train_acc: 0.385 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.40% | train_error: 12.7 | train_acc: 0.385 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.50% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.60% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.70% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 51.80% | train_error: 12.8 | train_acc: 0.380 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 51.90% | train_error: 12.8 | train_acc: 0.380 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 52.00% | train_error: 12.8 | train_acc: 0.380 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 52.10% | train_error: 12.8 | train_acc: 0.380 | val_error: 12.6 | val_acc: 0.392 

  [===================>--------------------] 52.20% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 52.30% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.5 | val_acc: 0.399 

  [===================>--------------------] 52.40% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 52.50% | train_error: 12.8 | train_acc: 0.383 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 52.60% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 52.70% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 52.80% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 52.90% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.00% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.10% | train_error: 12.6 | train_acc: 0.390 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.20% | train_error: 12.6 | train_acc: 0.392 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.30% | train_error: 12.6 | train_acc: 0.392 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.40% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.50% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.60% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.70% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.80% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 53.90% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 54.00% | train_error: 12.6 | train_acc: 0.394 | val_error: 12.5 | val_acc: 0.399 

  [====================>-------------------] 54.10% | train_error: 12.5 | train_acc: 0.397 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.20% | train_error: 12.5 | train_acc: 0.397 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.30% | train_error: 12.5 | train_acc: 0.397 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.40% | train_error: 12.5 | train_acc: 0.397 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.50% | train_error: 12.5 | train_acc: 0.397 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.60% | train_error: 12.4 | train_acc: 0.401 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.70% | train_error: 12.4 | train_acc: 0.404 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.80% | train_error: 12.2 | train_acc: 0.411 | val_error: 12.3 | val_acc: 0.406 

  [====================>-------------------] 54.90% | train_error: 12.1 | train_acc: 0.415 | val_error: 12.2 | val_acc: 0.413 

  [=====================>------------------] 55.00% | train_error: 12.1 | train_acc: 0.415 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.10% | train_error: 12.1 | train_acc: 0.415 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.20% | train_error: 12.1 | train_acc: 0.415 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.30% | train_error: 12.1 | train_acc: 0.418 | val_error: 12.2 | val_acc: 0.413 

  [=====================>------------------] 55.40% | train_error: 11.7 | train_acc: 0.437 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.50% | train_error: 11.7 | train_acc: 0.434 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.60% | train_error: 11.7 | train_acc: 0.434 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.70% | train_error: 11.7 | train_acc: 0.437 | val_error: 12.0 | val_acc: 0.420 

  [=====================>------------------] 55.80% | train_error: 11.7 | train_acc: 0.437 | val_error: 11.9 | val_acc: 0.427 

  [=====================>------------------] 55.90% | train_error: 11.7 | train_acc: 0.437 | val_error: 11.9 | val_acc: 0.427 

  [=====================>------------------] 56.00% | train_error: 11.7 | train_acc: 0.437 | val_error: 11.9 | val_acc: 0.427 

  [=====================>------------------] 56.10% | train_error: 11.7 | train_acc: 0.437 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.20% | train_error: 11.7 | train_acc: 0.437 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.30% | train_error: 11.7 | train_acc: 0.437 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.40% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.50% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.60% | train_error: 11.6 | train_acc: 0.441 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.70% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.7 | val_acc: 0.434 

  [=====================>------------------] 56.80% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [=====================>------------------] 56.90% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [=====================>------------------] 57.00% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [=====================>------------------] 57.10% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [=====================>------------------] 57.20% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [=====================>------------------] 57.30% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [=====================>------------------] 57.40% | train_error: 11.3 | train_acc: 0.453 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 57.50% | train_error: 11.4 | train_acc: 0.451 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 57.60% | train_error: 11.4 | train_acc: 0.451 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 57.70% | train_error: 11.4 | train_acc: 0.451 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 57.80% | train_error: 11.4 | train_acc: 0.451 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 57.90% | train_error: 11.3 | train_acc: 0.455 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.00% | train_error: 11.2 | train_acc: 0.460 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.10% | train_error: 10.9 | train_acc: 0.472 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.20% | train_error: 10.9 | train_acc: 0.472 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.30% | train_error: 10.9 | train_acc: 0.472 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.40% | train_error: 10.9 | train_acc: 0.472 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.50% | train_error: 10.9 | train_acc: 0.472 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.60% | train_error: 10.9 | train_acc: 0.472 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.70% | train_error: 10.8 | train_acc: 0.477 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.80% | train_error: 10.8 | train_acc: 0.477 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 58.90% | train_error: 10.8 | train_acc: 0.477 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 59.00% | train_error: 10.7 | train_acc: 0.484 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 59.10% | train_error: 10.7 | train_acc: 0.484 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 59.20% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 59.30% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.4 | val_acc: 0.448 

  [======================>-----------------] 59.40% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 59.50% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 59.60% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 59.70% | train_error: 10.6 | train_acc: 0.488 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 59.80% | train_error: 10.6 | train_acc: 0.488 | val_error: 11.6 | val_acc: 0.441 

  [======================>-----------------] 59.90% | train_error: 10.6 | train_acc: 0.491 | val_error: 11.6 | val_acc: 0.441 

  [=======================>----------------] 60.00% | train_error: 10.5 | train_acc: 0.493 | val_error: 11.6 | val_acc: 0.441 

  [=======================>----------------] 60.10% | train_error: 10.5 | train_acc: 0.493 | val_error: 11.4 | val_acc: 0.448 

  [=======================>----------------] 60.20% | train_error: 10.5 | train_acc: 0.493 | val_error: 11.3 | val_acc: 0.455 

  [=======================>----------------] 60.30% | train_error: 10.5 | train_acc: 0.493 | val_error: 11.0 | val_acc: 0.469 

  [=======================>----------------] 60.40% | train_error: 10.5 | train_acc: 0.493 | val_error: 11.0 | val_acc: 0.469 

  [=======================>----------------] 60.50% | train_error: 10.5 | train_acc: 0.493 | val_error: 11.0 | val_acc: 0.469 

  [=======================>----------------] 60.60% | train_error: 10.3 | train_acc: 0.502 | val_error: 11.0 | val_acc: 0.469 

  [=======================>----------------] 60.70% | train_error: 10.3 | train_acc: 0.502 | val_error: 10.7 | val_acc: 0.483 

  [=======================>----------------] 60.80% | train_error: 10.3 | train_acc: 0.502 | val_error: 10.7 | val_acc: 0.483 

  [=======================>----------------] 60.90% | train_error: 10.3 | train_acc: 0.502 | val_error: 10.7 | val_acc: 0.483 

  [=======================>----------------] 61.00% | train_error: 10.2 | train_acc: 0.507 | val_error: 10.7 | val_acc: 0.483 

  [=======================>----------------] 61.10% | train_error: 10.2 | train_acc: 0.507 | val_error: 10.7 | val_acc: 0.483 

  [=======================>----------------] 61.20% | train_error: 10.2 | train_acc: 0.509 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.30% | train_error: 10.2 | train_acc: 0.509 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.40% | train_error: 10.2 | train_acc: 0.509 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.50% | train_error: 10.2 | train_acc: 0.509 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.60% | train_error: 10.1 | train_acc: 0.512 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.70% | train_error: 10.1 | train_acc: 0.512 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.80% | train_error: 10.1 | train_acc: 0.512 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 61.90% | train_error: 10.1 | train_acc: 0.512 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 62.00% | train_error: 10.0 | train_acc: 0.516 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 62.10% | train_error: 9.97 | train_acc: 0.519 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 62.20% | train_error: 9.97 | train_acc: 0.519 | val_error: 10.6 | val_acc: 0.490 

  [=======================>----------------] 62.30% | train_error: 9.88 | train_acc: 0.523 | val_error: 10.4 | val_acc: 0.497 

  [=======================>----------------] 62.40% | train_error: 9.88 | train_acc: 0.523 | val_error: 10.4 | val_acc: 0.497 

  [========================>---------------] 62.50% | train_error: 9.78 | train_acc: 0.528 | val_error: 10.4 | val_acc: 0.497 

  [========================>---------------] 62.60% | train_error: 9.68 | train_acc: 0.533 | val_error: 10.3 | val_acc: 0.503 

  [========================>---------------] 62.70% | train_error: 9.68 | train_acc: 0.533 | val_error: 10.00 | val_acc: 0.517 

  [========================>---------------] 62.80% | train_error: 9.53 | train_acc: 0.540 | val_error: 10.00 | val_acc: 0.517 

  [========================>---------------] 62.90% | train_error: 9.44 | train_acc: 0.545 | val_error: 10.00 | val_acc: 0.517 

  [========================>---------------] 63.00% | train_error: 9.39 | train_acc: 0.547 | val_error: 10.00 | val_acc: 0.517 

  [========================>---------------] 63.10% | train_error: 9.29 | train_acc: 0.552 | val_error: 10.00 | val_acc: 0.517 

  [========================>---------------] 63.20% | train_error: 9.29 | train_acc: 0.552 | val_error: 10.00 | val_acc: 0.517 

  [========================>---------------] 63.30% | train_error: 9.24 | train_acc: 0.554 | val_error: 9.85 | val_acc: 0.524 

  [========================>---------------] 63.40% | train_error: 9.24 | train_acc: 0.554 | val_error: 9.85 | val_acc: 0.524 

  [========================>---------------] 63.50% | train_error: 8.85 | train_acc: 0.573 | val_error: 9.85 | val_acc: 0.524 

  [========================>---------------] 63.60% | train_error: 8.85 | train_acc: 0.573 | val_error: 9.85 | val_acc: 0.524 

  [========================>---------------] 63.70% | train_error: 8.76 | train_acc: 0.577 | val_error: 9.56 | val_acc: 0.538 

  [========================>---------------] 63.80% | train_error: 8.76 | train_acc: 0.577 | val_error: 9.56 | val_acc: 0.538 

  [========================>---------------] 63.90% | train_error: 8.76 | train_acc: 0.577 | val_error: 9.56 | val_acc: 0.538 

  [========================>---------------] 64.00% | train_error: 8.76 | train_acc: 0.577 | val_error: 9.56 | val_acc: 0.538 

  [========================>---------------] 64.10% | train_error: 8.71 | train_acc: 0.580 | val_error: 9.56 | val_acc: 0.538 

  [========================>---------------] 64.20% | train_error: 8.71 | train_acc: 0.580 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.30% | train_error: 8.71 | train_acc: 0.580 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.40% | train_error: 8.56 | train_acc: 0.587 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.50% | train_error: 8.56 | train_acc: 0.587 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.60% | train_error: 8.51 | train_acc: 0.589 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.70% | train_error: 8.46 | train_acc: 0.592 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.80% | train_error: 8.32 | train_acc: 0.599 | val_error: 9.42 | val_acc: 0.545 

  [========================>---------------] 64.90% | train_error: 8.22 | train_acc: 0.603 | val_error: 9.42 | val_acc: 0.545 

  [=========================>--------------] 65.00% | train_error: 8.12 | train_acc: 0.608 | val_error: 9.42 | val_acc: 0.545 

  [=========================>--------------] 65.10% | train_error: 8.08 | train_acc: 0.610 | val_error: 9.27 | val_acc: 0.552 

  [=========================>--------------] 65.20% | train_error: 8.03 | train_acc: 0.613 | val_error: 9.27 | val_acc: 0.552 

  [=========================>--------------] 65.30% | train_error: 8.03 | train_acc: 0.613 | val_error: 9.27 | val_acc: 0.552 

  [=========================>--------------] 65.40% | train_error: 8.03 | train_acc: 0.613 | val_error: 9.13 | val_acc: 0.559 

  [=========================>--------------] 65.50% | train_error: 8.03 | train_acc: 0.613 | val_error: 9.13 | val_acc: 0.559 

  [=========================>--------------] 65.60% | train_error: 8.03 | train_acc: 0.613 | val_error: 8.98 | val_acc: 0.566 

  [=========================>--------------] 65.70% | train_error: 8.03 | train_acc: 0.613 | val_error: 8.98 | val_acc: 0.566 

  [=========================>--------------] 65.80% | train_error: 8.03 | train_acc: 0.613 | val_error: 8.84 | val_acc: 0.573 

  [=========================>--------------] 65.90% | train_error: 7.98 | train_acc: 0.615 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.00% | train_error: 7.98 | train_acc: 0.615 | val_error: 8.55 | val_acc: 0.587 

  [=========================>--------------] 66.10% | train_error: 7.88 | train_acc: 0.620 | val_error: 8.55 | val_acc: 0.587 

  [=========================>--------------] 66.20% | train_error: 7.88 | train_acc: 0.620 | val_error: 8.55 | val_acc: 0.587 

  [=========================>--------------] 66.30% | train_error: 7.88 | train_acc: 0.620 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.40% | train_error: 7.88 | train_acc: 0.620 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.50% | train_error: 7.88 | train_acc: 0.620 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.60% | train_error: 7.73 | train_acc: 0.627 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.70% | train_error: 7.73 | train_acc: 0.627 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.80% | train_error: 7.73 | train_acc: 0.627 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 66.90% | train_error: 7.73 | train_acc: 0.627 | val_error: 8.70 | val_acc: 0.580 

  [=========================>--------------] 67.00% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.55 | val_acc: 0.587 

  [=========================>--------------] 67.10% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.55 | val_acc: 0.587 

  [=========================>--------------] 67.20% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.41 | val_acc: 0.594 

  [=========================>--------------] 67.30% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.41 | val_acc: 0.594 

  [=========================>--------------] 67.40% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.26 | val_acc: 0.601 

  [==========================>-------------] 67.50% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.12 | val_acc: 0.608 

  [==========================>-------------] 67.60% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.12 | val_acc: 0.608 

  [==========================>-------------] 67.70% | train_error: 7.69 | train_acc: 0.629 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 67.80% | train_error: 7.54 | train_acc: 0.636 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 67.90% | train_error: 7.54 | train_acc: 0.636 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.00% | train_error: 7.54 | train_acc: 0.636 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.10% | train_error: 7.44 | train_acc: 0.641 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.20% | train_error: 7.44 | train_acc: 0.641 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.30% | train_error: 7.30 | train_acc: 0.648 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.40% | train_error: 7.30 | train_acc: 0.648 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.50% | train_error: 7.30 | train_acc: 0.648 | val_error: 7.83 | val_acc: 0.622 

  [==========================>-------------] 68.60% | train_error: 7.10 | train_acc: 0.657 | val_error: 7.83 | val_acc: 0.622 

  [==========================>-------------] 68.70% | train_error: 7.01 | train_acc: 0.662 | val_error: 7.83 | val_acc: 0.622 

  [==========================>-------------] 68.80% | train_error: 6.91 | train_acc: 0.667 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 68.90% | train_error: 6.91 | train_acc: 0.667 | val_error: 7.97 | val_acc: 0.615 

  [==========================>-------------] 69.00% | train_error: 6.81 | train_acc: 0.671 | val_error: 7.83 | val_acc: 0.622 

  [==========================>-------------] 69.10% | train_error: 6.81 | train_acc: 0.671 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.20% | train_error: 6.81 | train_acc: 0.671 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.30% | train_error: 6.81 | train_acc: 0.671 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.40% | train_error: 6.76 | train_acc: 0.674 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.50% | train_error: 6.76 | train_acc: 0.674 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.60% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.70% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.80% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.68 | val_acc: 0.629 

  [==========================>-------------] 69.90% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.68 | val_acc: 0.629 

  [===========================>------------] 70.00% | train_error: 6.91 | train_acc: 0.667 | val_error: 7.68 | val_acc: 0.629 

  [===========================>------------] 70.10% | train_error: 6.91 | train_acc: 0.667 | val_error: 7.68 | val_acc: 0.629 

  [===========================>------------] 70.20% | train_error: 6.86 | train_acc: 0.669 | val_error: 7.68 | val_acc: 0.629 

  [===========================>------------] 70.30% | train_error: 6.86 | train_acc: 0.669 | val_error: 7.68 | val_acc: 0.629 

  [===========================>------------] 70.40% | train_error: 6.86 | train_acc: 0.669 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 70.50% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 70.60% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 70.70% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 70.80% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 70.90% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 71.00% | train_error: 6.71 | train_acc: 0.676 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 71.10% | train_error: 6.66 | train_acc: 0.678 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 71.20% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.54 | val_acc: 0.636 

  [===========================>------------] 71.30% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.39 | val_acc: 0.643 

  [===========================>------------] 71.40% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.39 | val_acc: 0.643 

  [===========================>------------] 71.50% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.39 | val_acc: 0.643 

  [===========================>------------] 71.60% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.39 | val_acc: 0.643 

  [===========================>------------] 71.70% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.25 | val_acc: 0.650 

  [===========================>------------] 71.80% | train_error: 6.52 | train_acc: 0.685 | val_error: 7.25 | val_acc: 0.650 

  [===========================>------------] 71.90% | train_error: 6.52 | train_acc: 0.685 | val_error: 7.25 | val_acc: 0.650 

  [===========================>------------] 72.00% | train_error: 6.52 | train_acc: 0.685 | val_error: 7.25 | val_acc: 0.650 

  [===========================>------------] 72.10% | train_error: 6.52 | train_acc: 0.685 | val_error: 7.25 | val_acc: 0.650 

  [===========================>------------] 72.20% | train_error: 6.47 | train_acc: 0.688 | val_error: 7.25 | val_acc: 0.650 

  [===========================>------------] 72.30% | train_error: 6.42 | train_acc: 0.690 | val_error: 7.10 | val_acc: 0.657 

  [===========================>------------] 72.40% | train_error: 6.42 | train_acc: 0.690 | val_error: 7.10 | val_acc: 0.657 

  [============================>-----------] 72.50% | train_error: 6.42 | train_acc: 0.690 | val_error: 7.10 | val_acc: 0.657 

  [============================>-----------] 72.60% | train_error: 6.42 | train_acc: 0.690 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 72.70% | train_error: 6.37 | train_acc: 0.692 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 72.80% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 72.90% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.00% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.10% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.20% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.30% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.40% | train_error: 6.32 | train_acc: 0.695 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.50% | train_error: 6.23 | train_acc: 0.700 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.60% | train_error: 6.23 | train_acc: 0.700 | val_error: 6.81 | val_acc: 0.671 

  [============================>-----------] 73.70% | train_error: 6.23 | train_acc: 0.700 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 73.80% | train_error: 6.23 | train_acc: 0.700 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 73.90% | train_error: 6.18 | train_acc: 0.702 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.00% | train_error: 5.98 | train_acc: 0.711 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.10% | train_error: 5.93 | train_acc: 0.714 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.20% | train_error: 5.84 | train_acc: 0.718 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.30% | train_error: 5.84 | train_acc: 0.718 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.40% | train_error: 5.84 | train_acc: 0.718 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.50% | train_error: 5.79 | train_acc: 0.721 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.60% | train_error: 5.64 | train_acc: 0.728 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.70% | train_error: 5.64 | train_acc: 0.728 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.80% | train_error: 5.59 | train_acc: 0.730 | val_error: 6.67 | val_acc: 0.678 

  [============================>-----------] 74.90% | train_error: 5.59 | train_acc: 0.730 | val_error: 6.38 | val_acc: 0.692 

  [=============================>----------] 75.00% | train_error: 5.59 | train_acc: 0.730 | val_error: 6.38 | val_acc: 0.692 

  [=============================>----------] 75.10% | train_error: 5.50 | train_acc: 0.735 | val_error: 6.23 | val_acc: 0.699 

  [=============================>----------] 75.20% | train_error: 5.30 | train_acc: 0.744 | val_error: 6.23 | val_acc: 0.699 

  [=============================>----------] 75.30% | train_error: 5.30 | train_acc: 0.744 | val_error: 6.23 | val_acc: 0.699 

  [=============================>----------] 75.40% | train_error: 5.30 | train_acc: 0.744 | val_error: 6.23 | val_acc: 0.699 

  [=============================>----------] 75.50% | train_error: 5.16 | train_acc: 0.751 | val_error: 6.23 | val_acc: 0.699 

  [=============================>----------] 75.60% | train_error: 5.11 | train_acc: 0.754 | val_error: 6.09 | val_acc: 0.706 

  [=============================>----------] 75.70% | train_error: 5.06 | train_acc: 0.756 | val_error: 5.94 | val_acc: 0.713 

  [=============================>----------] 75.80% | train_error: 5.06 | train_acc: 0.756 | val_error: 5.94 | val_acc: 0.713 

  [=============================>----------] 75.90% | train_error: 5.06 | train_acc: 0.756 | val_error: 5.94 | val_acc: 0.713 

  [=============================>----------] 76.00% | train_error: 5.06 | train_acc: 0.756 | val_error: 5.94 | val_acc: 0.713 

  [=============================>----------] 76.10% | train_error: 5.01 | train_acc: 0.758 | val_error: 5.94 | val_acc: 0.713 

  [=============================>----------] 76.20% | train_error: 5.01 | train_acc: 0.758 | val_error: 5.94 | val_acc: 0.713 

  [=============================>----------] 76.30% | train_error: 4.91 | train_acc: 0.763 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 76.40% | train_error: 4.91 | train_acc: 0.763 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 76.50% | train_error: 4.91 | train_acc: 0.763 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 76.60% | train_error: 4.91 | train_acc: 0.763 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 76.70% | train_error: 4.91 | train_acc: 0.763 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 76.80% | train_error: 4.91 | train_acc: 0.763 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 76.90% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 77.00% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 77.10% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.80 | val_acc: 0.720 

  [=============================>----------] 77.20% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.65 | val_acc: 0.727 

  [=============================>----------] 77.30% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.65 | val_acc: 0.727 

  [=============================>----------] 77.40% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.65 | val_acc: 0.727 

  [==============================>---------] 77.50% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.65 | val_acc: 0.727 

  [==============================>---------] 77.60% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.65 | val_acc: 0.727 

  [==============================>---------] 77.70% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.36 | val_acc: 0.741 

  [==============================>---------] 77.80% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.36 | val_acc: 0.741 

  [==============================>---------] 77.90% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.36 | val_acc: 0.741 

  [==============================>---------] 78.00% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.36 | val_acc: 0.741 

  [==============================>---------] 78.10% | train_error: 4.67 | train_acc: 0.775 | val_error: 5.36 | val_acc: 0.741 

  [==============================>---------] 78.20% | train_error: 4.62 | train_acc: 0.777 | val_error: 5.36 | val_acc: 0.741 

  [==============================>---------] 78.30% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.22 | val_acc: 0.748 

  [==============================>---------] 78.40% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.22 | val_acc: 0.748 

  [==============================>---------] 78.50% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.22 | val_acc: 0.748 

  [==============================>---------] 78.60% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.22 | val_acc: 0.748 

  [==============================>---------] 78.70% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.22 | val_acc: 0.748 

  [==============================>---------] 78.80% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.22 | val_acc: 0.748 

  [==============================>---------] 78.90% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.07 | val_acc: 0.755 

  [==============================>---------] 79.00% | train_error: 4.52 | train_acc: 0.782 | val_error: 5.07 | val_acc: 0.755 

  [==============================>---------] 79.10% | train_error: 4.52 | train_acc: 0.782 | val_error: 4.93 | val_acc: 0.762 

  [==============================>---------] 79.20% | train_error: 4.43 | train_acc: 0.786 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.30% | train_error: 4.38 | train_acc: 0.789 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.40% | train_error: 4.13 | train_acc: 0.800 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.50% | train_error: 4.13 | train_acc: 0.800 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.60% | train_error: 4.13 | train_acc: 0.800 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.70% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.80% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.78 | val_acc: 0.769 

  [==============================>---------] 79.90% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.64 | val_acc: 0.776 

  [===============================>--------] 80.00% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.64 | val_acc: 0.776 

  [===============================>--------] 80.10% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.49 | val_acc: 0.783 

  [===============================>--------] 80.20% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.49 | val_acc: 0.783 

  [===============================>--------] 80.30% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.49 | val_acc: 0.783 

  [===============================>--------] 80.40% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.49 | val_acc: 0.783 

  [===============================>--------] 80.50% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.35 | val_acc: 0.790 

  [===============================>--------] 80.60% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.35 | val_acc: 0.790 

  [===============================>--------] 80.70% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.35 | val_acc: 0.790 

  [===============================>--------] 80.80% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.35 | val_acc: 0.790 

  [===============================>--------] 80.90% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.35 | val_acc: 0.790 

  [===============================>--------] 81.00% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.10% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.20% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.30% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.40% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.50% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.60% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.70% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.20 | val_acc: 0.797 

  [===============================>--------] 81.80% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.06 | val_acc: 0.804 

  [===============================>--------] 81.90% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.06 | val_acc: 0.804 

  [===============================>--------] 82.00% | train_error: 4.28 | train_acc: 0.793 | val_error: 4.06 | val_acc: 0.804 

  [===============================>--------] 82.10% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [===============================>--------] 82.20% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [===============================>--------] 82.30% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [===============================>--------] 82.40% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 82.50% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 82.60% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 82.70% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 82.80% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 82.90% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.00% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.10% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.20% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.30% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.40% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.50% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.60% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 83.70% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 83.80% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 83.90% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.20 | val_acc: 0.797 

  [================================>-------] 84.00% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.10% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.20% | train_error: 4.33 | train_acc: 0.791 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.30% | train_error: 4.33 | train_acc: 0.791 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.40% | train_error: 4.33 | train_acc: 0.791 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.50% | train_error: 4.33 | train_acc: 0.791 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.60% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.70% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.80% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [================================>-------] 84.90% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.00% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.10% | train_error: 4.23 | train_acc: 0.796 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.20% | train_error: 4.18 | train_acc: 0.798 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.30% | train_error: 4.18 | train_acc: 0.798 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.40% | train_error: 4.18 | train_acc: 0.798 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.50% | train_error: 4.18 | train_acc: 0.798 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.60% | train_error: 4.13 | train_acc: 0.800 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.70% | train_error: 4.13 | train_acc: 0.800 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.80% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 85.90% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.06 | val_acc: 0.804 

  [=================================>------] 86.00% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.10% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.20% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.30% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.40% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.50% | train_error: 4.09 | train_acc: 0.803 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.60% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.70% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.80% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 86.90% | train_error: 3.99 | train_acc: 0.808 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 87.00% | train_error: 3.89 | train_acc: 0.812 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 87.10% | train_error: 3.89 | train_acc: 0.812 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 87.20% | train_error: 3.89 | train_acc: 0.812 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 87.30% | train_error: 3.89 | train_acc: 0.812 | val_error: 4.20 | val_acc: 0.797 

  [=================================>------] 87.40% | train_error: 3.79 | train_acc: 0.817 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 87.50% | train_error: 3.79 | train_acc: 0.817 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 87.60% | train_error: 3.89 | train_acc: 0.812 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 87.70% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 87.80% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 87.90% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 88.00% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 88.10% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.20 | val_acc: 0.797 

  [==================================>-----] 88.20% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.30% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.40% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.50% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.60% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.70% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.80% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 88.90% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 89.00% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.06 | val_acc: 0.804 

  [==================================>-----] 89.10% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.20% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.30% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.40% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.50% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.60% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.70% | train_error: 3.70 | train_acc: 0.822 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.80% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.91 | val_acc: 0.811 

  [==================================>-----] 89.90% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.91 | val_acc: 0.811 

  [===================================>----] 90.00% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.91 | val_acc: 0.811 

  [===================================>----] 90.10% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.91 | val_acc: 0.811 

  [===================================>----] 90.20% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.91 | val_acc: 0.811 

  [===================================>----] 90.30% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.91 | val_acc: 0.811 

  [===================================>----] 90.40% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 90.50% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 90.60% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 90.70% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 90.80% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 90.90% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.00% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.10% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.20% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.30% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.40% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.50% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.60% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.70% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.80% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 91.90% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 92.00% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 92.10% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 92.20% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 92.30% | train_error: 3.60 | train_acc: 0.826 | val_error: 3.77 | val_acc: 0.818 

  [===================================>----] 92.40% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 92.50% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 92.60% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 92.70% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 92.80% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 92.90% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.00% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.10% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.20% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.30% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.40% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.50% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.60% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.70% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.80% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 93.90% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.00% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.10% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.20% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.30% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.40% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.50% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.60% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.70% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.80% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [====================================>---] 94.90% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.00% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.10% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.20% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.30% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.40% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.50% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.62 | val_acc: 0.825 

  [=====================================>--] 95.60% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 95.70% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 95.80% | train_error: 3.65 | train_acc: 0.824 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 95.90% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.00% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.10% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.20% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.30% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.40% | train_error: 3.55 | train_acc: 0.829 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.50% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.60% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.70% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.80% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 96.90% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 97.00% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 97.10% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 97.20% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 97.30% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [=====================================>--] 97.40% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 97.50% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 97.60% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 97.70% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 97.80% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 97.90% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.00% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.10% | train_error: 3.50 | train_acc: 0.831 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.20% | train_error: 3.45 | train_acc: 0.833 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.30% | train_error: 3.45 | train_acc: 0.833 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.40% | train_error: 3.41 | train_acc: 0.836 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.50% | train_error: 3.36 | train_acc: 0.838 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.60% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.48 | val_acc: 0.832 

  [======================================>-] 98.70% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 98.80% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 98.90% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.00% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.10% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.20% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.30% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.40% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.50% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.60% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.70% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.62 | val_acc: 0.825 

  [======================================>-] 99.80% | train_error: 3.26 | train_acc: 0.843 | val_error: 3.77 | val_acc: 0.818 

  [======================================>-] 99.90% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.77 | val_acc: 0.818 

  [=======================================>] 100.0% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.77 | val_acc: 0.818 

Finally, we will create a neural network with 2 hidden layers with activation functions.

input_nodes = X_train.shape[1]
hidden_nodes1 = 100
hidden_nodes2 = 30
output_nodes = 1

dims = (input_nodes, hidden_nodes1, hidden_nodes2, output_nodes)

neural_network = FFNN(dims, hidden_func=RELU, output_func=sigmoid, cost_func=CostLogReg, seed=2023)

neural_network.reset_weights() # reset weights such that previous runs or reruns don't affect the weights

scheduler = Adam(eta=1e-4, rho=0.9, rho2=0.999)
scores = neural_network.fit(X_train, t_train, scheduler, epochs=1000, X_val=X_val, t_val=t_val)

Adam: Eta=0.0001, Lambda=0

  [----------------------------------------] 0.000% | train_error: 11.3 | train_acc: 0.453 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 0.1000% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 0.2000% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 0.3000% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 0.4000% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 0.5000% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 0.6000% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 0.7000% | train_error: 11.5 | train_acc: 0.444 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 0.8000% | train_error: 11.5 | train_acc: 0.444 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 0.9000% | train_error: 11.3 | train_acc: 0.453 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 1.000% | train_error: 11.3 | train_acc: 0.453 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 1.100% | train_error: 11.3 | train_acc: 0.453 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 1.200% | train_error: 11.3 | train_acc: 0.453 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.300% | train_error: 11.3 | train_acc: 0.453 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.400% | train_error: 11.4 | train_acc: 0.448 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.500% | train_error: 11.4 | train_acc: 0.451 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.600% | train_error: 11.5 | train_acc: 0.444 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.700% | train_error: 11.5 | train_acc: 0.444 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.800% | train_error: 11.6 | train_acc: 0.441 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 1.900% | train_error: 11.6 | train_acc: 0.441 | val_error: 10.4 | val_acc: 0.497 

  [----------------------------------------] 2.000% | train_error: 11.6 | train_acc: 0.441 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 2.100% | train_error: 11.6 | train_acc: 0.439 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 2.200% | train_error: 11.6 | train_acc: 0.439 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 2.300% | train_error: 11.8 | train_acc: 0.430 | val_error: 10.6 | val_acc: 0.490 

  [----------------------------------------] 2.400% | train_error: 11.9 | train_acc: 0.425 | val_error: 10.6 | val_acc: 0.490 

  [>---------------------------------------] 2.500% | train_error: 11.9 | train_acc: 0.425 | val_error: 10.6 | val_acc: 0.490 

  [>---------------------------------------] 2.600% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.6 | val_acc: 0.490 

  [>---------------------------------------] 2.700% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.6 | val_acc: 0.490 

  [>---------------------------------------] 2.800% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.7 | val_acc: 0.483 

  [>---------------------------------------] 2.900% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.7 | val_acc: 0.483 

  [>---------------------------------------] 3.000% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.7 | val_acc: 0.483 

  [>---------------------------------------] 3.100% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.7 | val_acc: 0.483 

  [>---------------------------------------] 3.200% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.7 | val_acc: 0.483 

  [>---------------------------------------] 3.300% | train_error: 11.9 | train_acc: 0.425 | val_error: 10.9 | val_acc: 0.476 

  [>---------------------------------------] 3.400% | train_error: 11.9 | train_acc: 0.427 | val_error: 10.9 | val_acc: 0.476 

  [>---------------------------------------] 3.500% | train_error: 11.9 | train_acc: 0.427 | val_error: 11.0 | val_acc: 0.469 

  [>---------------------------------------] 3.600% | train_error: 11.8 | train_acc: 0.432 | val_error: 11.0 | val_acc: 0.469 

  [>---------------------------------------] 3.700% | train_error: 11.6 | train_acc: 0.439 | val_error: 11.0 | val_acc: 0.469 

  [>---------------------------------------] 3.800% | train_error: 11.6 | train_acc: 0.439 | val_error: 11.0 | val_acc: 0.469 

  [>---------------------------------------] 3.900% | train_error: 11.7 | train_acc: 0.434 | val_error: 11.3 | val_acc: 0.455 

  [>---------------------------------------] 4.000% | train_error: 11.7 | train_acc: 0.434 | val_error: 11.3 | val_acc: 0.455 

  [>---------------------------------------] 4.100% | train_error: 11.8 | train_acc: 0.430 | val_error: 11.2 | val_acc: 0.462 

  [>---------------------------------------] 4.200% | train_error: 11.9 | train_acc: 0.425 | val_error: 11.2 | val_acc: 0.462 

  [>---------------------------------------] 4.300% | train_error: 12.0 | train_acc: 0.423 | val_error: 11.2 | val_acc: 0.462 

  [>---------------------------------------] 4.400% | train_error: 11.9 | train_acc: 0.427 | val_error: 11.2 | val_acc: 0.462 

  [>---------------------------------------] 4.500% | train_error: 11.8 | train_acc: 0.430 | val_error: 11.2 | val_acc: 0.462 

  [>---------------------------------------] 4.600% | train_error: 11.8 | train_acc: 0.432 | val_error: 11.2 | val_acc: 0.462 

  [>---------------------------------------] 4.700% | train_error: 11.8 | train_acc: 0.430 | val_error: 11.3 | val_acc: 0.455 

  [>---------------------------------------] 4.800% | train_error: 11.7 | train_acc: 0.434 | val_error: 11.3 | val_acc: 0.455 

  [>---------------------------------------] 4.900% | train_error: 11.5 | train_acc: 0.446 | val_error: 11.3 | val_acc: 0.455 

  [=>--------------------------------------] 5.000% | train_error: 11.5 | train_acc: 0.446 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 5.100% | train_error: 11.5 | train_acc: 0.446 | val_error: 11.3 | val_acc: 0.455 

  [=>--------------------------------------] 5.200% | train_error: 11.5 | train_acc: 0.446 | val_error: 11.2 | val_acc: 0.462 

  [=>--------------------------------------] 5.300% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.0 | val_acc: 0.469 

  [=>--------------------------------------] 5.400% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.0 | val_acc: 0.469 

  [=>--------------------------------------] 5.500% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.0 | val_acc: 0.469 

  [=>--------------------------------------] 5.600% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.3 | val_acc: 0.455 

  [=>--------------------------------------] 5.700% | train_error: 11.5 | train_acc: 0.444 | val_error: 11.3 | val_acc: 0.455 

  [=>--------------------------------------] 5.800% | train_error: 11.9 | train_acc: 0.425 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 5.900% | train_error: 11.9 | train_acc: 0.425 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 6.000% | train_error: 12.0 | train_acc: 0.420 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 6.100% | train_error: 12.0 | train_acc: 0.420 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 6.200% | train_error: 12.1 | train_acc: 0.418 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 6.300% | train_error: 12.1 | train_acc: 0.418 | val_error: 11.4 | val_acc: 0.448 

  [=>--------------------------------------] 6.400% | train_error: 11.9 | train_acc: 0.425 | val_error: 11.7 | val_acc: 0.434 

  [=>--------------------------------------] 6.500% | train_error: 11.9 | train_acc: 0.427 | val_error: 11.9 | val_acc: 0.427 

  [=>--------------------------------------] 6.600% | train_error: 11.8 | train_acc: 0.432 | val_error: 12.0 | val_acc: 0.420 

  [=>--------------------------------------] 6.700% | train_error: 11.5 | train_acc: 0.444 | val_error: 12.2 | val_acc: 0.413 

  [=>--------------------------------------] 6.800% | train_error: 11.4 | train_acc: 0.448 | val_error: 12.3 | val_acc: 0.406 

  [=>--------------------------------------] 6.900% | train_error: 11.2 | train_acc: 0.458 | val_error: 12.5 | val_acc: 0.399 

  [=>--------------------------------------] 7.000% | train_error: 11.2 | train_acc: 0.458 | val_error: 12.3 | val_acc: 0.406 

  [=>--------------------------------------] 7.100% | train_error: 11.3 | train_acc: 0.455 | val_error: 12.5 | val_acc: 0.399 

  [=>--------------------------------------] 7.200% | train_error: 11.3 | train_acc: 0.455 | val_error: 12.6 | val_acc: 0.392 

  [=>--------------------------------------] 7.300% | train_error: 11.3 | train_acc: 0.455 | val_error: 12.5 | val_acc: 0.399 

  [=>--------------------------------------] 7.400% | train_error: 11.3 | train_acc: 0.453 | val_error: 12.3 | val_acc: 0.406 

  [==>-------------------------------------] 7.500% | train_error: 11.3 | train_acc: 0.455 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 7.600% | train_error: 11.5 | train_acc: 0.446 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 7.700% | train_error: 11.3 | train_acc: 0.453 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 7.800% | train_error: 11.4 | train_acc: 0.448 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 7.900% | train_error: 11.4 | train_acc: 0.448 | val_error: 12.3 | val_acc: 0.406 

  [==>-------------------------------------] 8.000% | train_error: 11.4 | train_acc: 0.451 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 8.100% | train_error: 11.4 | train_acc: 0.448 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 8.200% | train_error: 11.3 | train_acc: 0.453 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 8.300% | train_error: 11.3 | train_acc: 0.453 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 8.400% | train_error: 11.4 | train_acc: 0.451 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 8.500% | train_error: 11.4 | train_acc: 0.451 | val_error: 12.5 | val_acc: 0.399 

  [==>-------------------------------------] 8.600% | train_error: 11.4 | train_acc: 0.451 | val_error: 12.3 | val_acc: 0.406 

  [==>-------------------------------------] 8.700% | train_error: 11.2 | train_acc: 0.458 | val_error: 12.3 | val_acc: 0.406 

  [==>-------------------------------------] 8.800% | train_error: 11.2 | train_acc: 0.458 | val_error: 12.2 | val_acc: 0.413 

  [==>-------------------------------------] 8.900% | train_error: 11.2 | train_acc: 0.458 | val_error: 12.0 | val_acc: 0.420 

  [==>-------------------------------------] 9.000% | train_error: 11.2 | train_acc: 0.458 | val_error: 12.0 | val_acc: 0.420 

  [==>-------------------------------------] 9.100% | train_error: 11.2 | train_acc: 0.458 | val_error: 11.9 | val_acc: 0.427 

  [==>-------------------------------------] 9.200% | train_error: 11.2 | train_acc: 0.458 | val_error: 11.9 | val_acc: 0.427 

  [==>-------------------------------------] 9.300% | train_error: 11.2 | train_acc: 0.460 | val_error: 11.9 | val_acc: 0.427 

  [==>-------------------------------------] 9.400% | train_error: 11.0 | train_acc: 0.469 | val_error: 11.9 | val_acc: 0.427 

  [==>-------------------------------------] 9.500% | train_error: 11.0 | train_acc: 0.469 | val_error: 11.9 | val_acc: 0.427 

  [==>-------------------------------------] 9.600% | train_error: 10.8 | train_acc: 0.479 | val_error: 11.9 | val_acc: 0.427 

  [==>-------------------------------------] 9.700% | train_error: 10.7 | train_acc: 0.484 | val_error: 11.7 | val_acc: 0.434 

  [==>-------------------------------------] 9.800% | train_error: 10.7 | train_acc: 0.484 | val_error: 11.7 | val_acc: 0.434 

  [==>-------------------------------------] 9.900% | train_error: 10.7 | train_acc: 0.484 | val_error: 11.7 | val_acc: 0.434 

  [===>------------------------------------] 10.00% | train_error: 10.7 | train_acc: 0.484 | val_error: 11.7 | val_acc: 0.434 

  [===>------------------------------------] 10.10% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.7 | val_acc: 0.434 

  [===>------------------------------------] 10.20% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.6 | val_acc: 0.441 

  [===>------------------------------------] 10.30% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.6 | val_acc: 0.441 

  [===>------------------------------------] 10.40% | train_error: 10.7 | train_acc: 0.486 | val_error: 11.4 | val_acc: 0.448 

  [===>------------------------------------] 10.50% | train_error: 10.4 | train_acc: 0.500 | val_error: 11.4 | val_acc: 0.448 

  [===>------------------------------------] 10.60% | train_error: 10.4 | train_acc: 0.500 | val_error: 11.4 | val_acc: 0.448 

  [===>------------------------------------] 10.70% | train_error: 10.4 | train_acc: 0.500 | val_error: 11.4 | val_acc: 0.448 

  [===>------------------------------------] 10.80% | train_error: 10.4 | train_acc: 0.500 | val_error: 11.4 | val_acc: 0.448 

  [===>------------------------------------] 10.90% | train_error: 10.2 | train_acc: 0.507 | val_error: 11.3 | val_acc: 0.455 

  [===>------------------------------------] 11.00% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.3 | val_acc: 0.455 

  [===>------------------------------------] 11.10% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.3 | val_acc: 0.455 

  [===>------------------------------------] 11.20% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.3 | val_acc: 0.455 

  [===>------------------------------------] 11.30% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 11.40% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 11.50% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 11.60% | train_error: 9.83 | train_acc: 0.526 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 11.70% | train_error: 9.73 | train_acc: 0.531 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 11.80% | train_error: 9.63 | train_acc: 0.535 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 11.90% | train_error: 9.63 | train_acc: 0.535 | val_error: 11.2 | val_acc: 0.462 

  [===>------------------------------------] 12.00% | train_error: 9.53 | train_acc: 0.540 | val_error: 10.9 | val_acc: 0.476 

  [===>------------------------------------] 12.10% | train_error: 9.53 | train_acc: 0.540 | val_error: 10.9 | val_acc: 0.476 

  [===>------------------------------------] 12.20% | train_error: 9.49 | train_acc: 0.542 | val_error: 10.9 | val_acc: 0.476 

  [===>------------------------------------] 12.30% | train_error: 9.49 | train_acc: 0.542 | val_error: 10.9 | val_acc: 0.476 

  [===>------------------------------------] 12.40% | train_error: 9.44 | train_acc: 0.545 | val_error: 10.9 | val_acc: 0.476 

  [====>-----------------------------------] 12.50% | train_error: 9.44 | train_acc: 0.545 | val_error: 10.9 | val_acc: 0.476 

  [====>-----------------------------------] 12.60% | train_error: 9.39 | train_acc: 0.547 | val_error: 10.9 | val_acc: 0.476 

  [====>-----------------------------------] 12.70% | train_error: 9.39 | train_acc: 0.547 | val_error: 10.7 | val_acc: 0.483 

  [====>-----------------------------------] 12.80% | train_error: 9.39 | train_acc: 0.547 | val_error: 10.7 | val_acc: 0.483 

  [====>-----------------------------------] 12.90% | train_error: 9.34 | train_acc: 0.549 | val_error: 10.7 | val_acc: 0.483 

  [====>-----------------------------------] 13.00% | train_error: 9.10 | train_acc: 0.561 | val_error: 10.6 | val_acc: 0.490 

  [====>-----------------------------------] 13.10% | train_error: 9.05 | train_acc: 0.563 | val_error: 10.6 | val_acc: 0.490 

  [====>-----------------------------------] 13.20% | train_error: 9.05 | train_acc: 0.563 | val_error: 10.6 | val_acc: 0.490 

  [====>-----------------------------------] 13.30% | train_error: 8.80 | train_acc: 0.575 | val_error: 10.6 | val_acc: 0.490 

  [====>-----------------------------------] 13.40% | train_error: 8.76 | train_acc: 0.577 | val_error: 10.4 | val_acc: 0.497 

  [====>-----------------------------------] 13.50% | train_error: 8.56 | train_acc: 0.587 | val_error: 10.4 | val_acc: 0.497 

  [====>-----------------------------------] 13.60% | train_error: 8.56 | train_acc: 0.587 | val_error: 10.4 | val_acc: 0.497 

  [====>-----------------------------------] 13.70% | train_error: 8.42 | train_acc: 0.594 | val_error: 10.4 | val_acc: 0.497 

  [====>-----------------------------------] 13.80% | train_error: 8.42 | train_acc: 0.594 | val_error: 10.3 | val_acc: 0.503 

  [====>-----------------------------------] 13.90% | train_error: 8.42 | train_acc: 0.594 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.00% | train_error: 8.42 | train_acc: 0.594 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.10% | train_error: 8.32 | train_acc: 0.599 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.20% | train_error: 8.32 | train_acc: 0.599 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.30% | train_error: 8.22 | train_acc: 0.603 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.40% | train_error: 8.22 | train_acc: 0.603 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.50% | train_error: 8.22 | train_acc: 0.603 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.60% | train_error: 8.22 | train_acc: 0.603 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.70% | train_error: 8.17 | train_acc: 0.606 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.80% | train_error: 8.17 | train_acc: 0.606 | val_error: 10.1 | val_acc: 0.510 

  [====>-----------------------------------] 14.90% | train_error: 8.17 | train_acc: 0.606 | val_error: 10.1 | val_acc: 0.510 

  [=====>----------------------------------] 15.00% | train_error: 8.08 | train_acc: 0.610 | val_error: 9.85 | val_acc: 0.524 

  [=====>----------------------------------] 15.10% | train_error: 7.98 | train_acc: 0.615 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.20% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.30% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.40% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.50% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.60% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.70% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.71 | val_acc: 0.531 

  [=====>----------------------------------] 15.80% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.56 | val_acc: 0.538 

  [=====>----------------------------------] 15.90% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.27 | val_acc: 0.552 

  [=====>----------------------------------] 16.00% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.27 | val_acc: 0.552 

  [=====>----------------------------------] 16.10% | train_error: 7.93 | train_acc: 0.617 | val_error: 9.27 | val_acc: 0.552 

  [=====>----------------------------------] 16.20% | train_error: 7.88 | train_acc: 0.620 | val_error: 9.27 | val_acc: 0.552 

  [=====>----------------------------------] 16.30% | train_error: 7.69 | train_acc: 0.629 | val_error: 9.27 | val_acc: 0.552 

  [=====>----------------------------------] 16.40% | train_error: 7.69 | train_acc: 0.629 | val_error: 9.13 | val_acc: 0.559 

  [=====>----------------------------------] 16.50% | train_error: 7.69 | train_acc: 0.629 | val_error: 9.13 | val_acc: 0.559 

  [=====>----------------------------------] 16.60% | train_error: 7.69 | train_acc: 0.629 | val_error: 9.13 | val_acc: 0.559 

  [=====>----------------------------------] 16.70% | train_error: 7.69 | train_acc: 0.629 | val_error: 8.98 | val_acc: 0.566 

  [=====>----------------------------------] 16.80% | train_error: 7.59 | train_acc: 0.634 | val_error: 8.84 | val_acc: 0.573 

  [=====>----------------------------------] 16.90% | train_error: 7.49 | train_acc: 0.638 | val_error: 8.84 | val_acc: 0.573 

  [=====>----------------------------------] 17.00% | train_error: 7.39 | train_acc: 0.643 | val_error: 8.70 | val_acc: 0.580 

  [=====>----------------------------------] 17.10% | train_error: 7.39 | train_acc: 0.643 | val_error: 8.55 | val_acc: 0.587 

  [=====>----------------------------------] 17.20% | train_error: 7.30 | train_acc: 0.648 | val_error: 8.41 | val_acc: 0.594 

  [=====>----------------------------------] 17.30% | train_error: 7.30 | train_acc: 0.648 | val_error: 8.26 | val_acc: 0.601 

  [=====>----------------------------------] 17.40% | train_error: 7.15 | train_acc: 0.655 | val_error: 8.26 | val_acc: 0.601 

  [======>---------------------------------] 17.50% | train_error: 7.05 | train_acc: 0.660 | val_error: 8.12 | val_acc: 0.608 

  [======>---------------------------------] 17.60% | train_error: 7.05 | train_acc: 0.660 | val_error: 7.97 | val_acc: 0.615 

  [======>---------------------------------] 17.70% | train_error: 6.91 | train_acc: 0.667 | val_error: 7.97 | val_acc: 0.615 

  [======>---------------------------------] 17.80% | train_error: 6.86 | train_acc: 0.669 | val_error: 7.83 | val_acc: 0.622 

  [======>---------------------------------] 17.90% | train_error: 6.86 | train_acc: 0.669 | val_error: 7.83 | val_acc: 0.622 

  [======>---------------------------------] 18.00% | train_error: 6.86 | train_acc: 0.669 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.10% | train_error: 6.76 | train_acc: 0.674 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.20% | train_error: 6.76 | train_acc: 0.674 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.30% | train_error: 6.62 | train_acc: 0.681 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.40% | train_error: 6.57 | train_acc: 0.683 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.50% | train_error: 6.47 | train_acc: 0.688 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.60% | train_error: 6.47 | train_acc: 0.688 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.70% | train_error: 6.37 | train_acc: 0.692 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.80% | train_error: 6.32 | train_acc: 0.695 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 18.90% | train_error: 6.28 | train_acc: 0.697 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 19.00% | train_error: 6.28 | train_acc: 0.697 | val_error: 7.54 | val_acc: 0.636 

  [======>---------------------------------] 19.10% | train_error: 6.28 | train_acc: 0.697 | val_error: 7.39 | val_acc: 0.643 

  [======>---------------------------------] 19.20% | train_error: 6.28 | train_acc: 0.697 | val_error: 7.39 | val_acc: 0.643 

  [======>---------------------------------] 19.30% | train_error: 6.28 | train_acc: 0.697 | val_error: 7.10 | val_acc: 0.657 

  [======>---------------------------------] 19.40% | train_error: 6.18 | train_acc: 0.702 | val_error: 7.10 | val_acc: 0.657 

  [======>---------------------------------] 19.50% | train_error: 6.18 | train_acc: 0.702 | val_error: 6.81 | val_acc: 0.671 

  [======>---------------------------------] 19.60% | train_error: 6.13 | train_acc: 0.704 | val_error: 6.81 | val_acc: 0.671 

  [======>---------------------------------] 19.70% | train_error: 6.13 | train_acc: 0.704 | val_error: 6.67 | val_acc: 0.678 

  [======>---------------------------------] 19.80% | train_error: 6.13 | train_acc: 0.704 | val_error: 6.52 | val_acc: 0.685 

  [======>---------------------------------] 19.90% | train_error: 6.13 | train_acc: 0.704 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.00% | train_error: 6.13 | train_acc: 0.704 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.10% | train_error: 6.08 | train_acc: 0.707 | val_error: 6.38 | val_acc: 0.692 

  [=======>--------------------------------] 20.20% | train_error: 6.03 | train_acc: 0.709 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.30% | train_error: 5.64 | train_acc: 0.728 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.40% | train_error: 5.69 | train_acc: 0.725 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.50% | train_error: 5.69 | train_acc: 0.725 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.60% | train_error: 5.59 | train_acc: 0.730 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 20.70% | train_error: 5.59 | train_acc: 0.730 | val_error: 6.67 | val_acc: 0.678 

  [=======>--------------------------------] 20.80% | train_error: 5.55 | train_acc: 0.732 | val_error: 6.67 | val_acc: 0.678 

  [=======>--------------------------------] 20.90% | train_error: 5.50 | train_acc: 0.735 | val_error: 6.52 | val_acc: 0.685 

  [=======>--------------------------------] 21.00% | train_error: 5.35 | train_acc: 0.742 | val_error: 6.38 | val_acc: 0.692 

  [=======>--------------------------------] 21.10% | train_error: 5.35 | train_acc: 0.742 | val_error: 6.23 | val_acc: 0.699 

  [=======>--------------------------------] 21.20% | train_error: 5.30 | train_acc: 0.744 | val_error: 6.23 | val_acc: 0.699 

  [=======>--------------------------------] 21.30% | train_error: 5.21 | train_acc: 0.749 | val_error: 6.23 | val_acc: 0.699 

  [=======>--------------------------------] 21.40% | train_error: 5.06 | train_acc: 0.756 | val_error: 6.23 | val_acc: 0.699 

  [=======>--------------------------------] 21.50% | train_error: 5.06 | train_acc: 0.756 | val_error: 6.09 | val_acc: 0.706 

  [=======>--------------------------------] 21.60% | train_error: 5.06 | train_acc: 0.756 | val_error: 6.09 | val_acc: 0.706 

  [=======>--------------------------------] 21.70% | train_error: 5.01 | train_acc: 0.758 | val_error: 6.09 | val_acc: 0.706 

  [=======>--------------------------------] 21.80% | train_error: 5.01 | train_acc: 0.758 | val_error: 6.09 | val_acc: 0.706 

  [=======>--------------------------------] 21.90% | train_error: 4.96 | train_acc: 0.761 | val_error: 6.09 | val_acc: 0.706 

  [=======>--------------------------------] 22.00% | train_error: 4.96 | train_acc: 0.761 | val_error: 5.65 | val_acc: 0.727 

  [=======>--------------------------------] 22.10% | train_error: 4.72 | train_acc: 0.772 | val_error: 5.65 | val_acc: 0.727 

  [=======>--------------------------------] 22.20% | train_error: 4.62 | train_acc: 0.777 | val_error: 5.65 | val_acc: 0.727 

  [=======>--------------------------------] 22.30% | train_error: 4.52 | train_acc: 0.782 | val_error: 5.65 | val_acc: 0.727 

  [=======>--------------------------------] 22.40% | train_error: 4.62 | train_acc: 0.777 | val_error: 5.65 | val_acc: 0.727 

  [========>-------------------------------] 22.50% | train_error: 4.52 | train_acc: 0.782 | val_error: 5.51 | val_acc: 0.734 

  [========>-------------------------------] 22.60% | train_error: 4.57 | train_acc: 0.779 | val_error: 5.65 | val_acc: 0.727 

  [========>-------------------------------] 22.70% | train_error: 4.52 | train_acc: 0.782 | val_error: 5.65 | val_acc: 0.727 

  [========>-------------------------------] 22.80% | train_error: 4.33 | train_acc: 0.791 | val_error: 5.51 | val_acc: 0.734 

  [========>-------------------------------] 22.90% | train_error: 4.33 | train_acc: 0.791 | val_error: 5.36 | val_acc: 0.741 

  [========>-------------------------------] 23.00% | train_error: 4.33 | train_acc: 0.791 | val_error: 5.36 | val_acc: 0.741 

  [========>-------------------------------] 23.10% | train_error: 4.33 | train_acc: 0.791 | val_error: 5.36 | val_acc: 0.741 

  [========>-------------------------------] 23.20% | train_error: 4.33 | train_acc: 0.791 | val_error: 5.07 | val_acc: 0.755 

  [========>-------------------------------] 23.30% | train_error: 4.33 | train_acc: 0.791 | val_error: 5.07 | val_acc: 0.755 

  [========>-------------------------------] 23.40% | train_error: 4.28 | train_acc: 0.793 | val_error: 5.07 | val_acc: 0.755 

  [========>-------------------------------] 23.50% | train_error: 4.28 | train_acc: 0.793 | val_error: 5.07 | val_acc: 0.755 

  [========>-------------------------------] 23.60% | train_error: 4.13 | train_acc: 0.800 | val_error: 5.07 | val_acc: 0.755 

  [========>-------------------------------] 23.70% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.93 | val_acc: 0.762 

  [========>-------------------------------] 23.80% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.78 | val_acc: 0.769 

  [========>-------------------------------] 23.90% | train_error: 4.04 | train_acc: 0.805 | val_error: 4.78 | val_acc: 0.769 

  [========>-------------------------------] 24.00% | train_error: 3.89 | train_acc: 0.812 | val_error: 4.64 | val_acc: 0.776 

  [========>-------------------------------] 24.10% | train_error: 3.84 | train_acc: 0.815 | val_error: 4.64 | val_acc: 0.776 

  [========>-------------------------------] 24.20% | train_error: 3.79 | train_acc: 0.817 | val_error: 4.64 | val_acc: 0.776 

  [========>-------------------------------] 24.30% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.64 | val_acc: 0.776 

  [========>-------------------------------] 24.40% | train_error: 3.70 | train_acc: 0.822 | val_error: 4.49 | val_acc: 0.783 

  [========>-------------------------------] 24.50% | train_error: 3.65 | train_acc: 0.824 | val_error: 4.35 | val_acc: 0.790 

  [========>-------------------------------] 24.60% | train_error: 3.55 | train_acc: 0.829 | val_error: 4.35 | val_acc: 0.790 

  [========>-------------------------------] 24.70% | train_error: 3.55 | train_acc: 0.829 | val_error: 4.35 | val_acc: 0.790 

  [========>-------------------------------] 24.80% | train_error: 3.50 | train_acc: 0.831 | val_error: 4.64 | val_acc: 0.776 

  [========>-------------------------------] 24.90% | train_error: 3.45 | train_acc: 0.833 | val_error: 4.64 | val_acc: 0.776 

  [=========>------------------------------] 25.00% | train_error: 3.21 | train_acc: 0.845 | val_error: 4.35 | val_acc: 0.790 

  [=========>------------------------------] 25.10% | train_error: 3.21 | train_acc: 0.845 | val_error: 4.20 | val_acc: 0.797 

  [=========>------------------------------] 25.20% | train_error: 3.11 | train_acc: 0.850 | val_error: 4.20 | val_acc: 0.797 

  [=========>------------------------------] 25.30% | train_error: 3.11 | train_acc: 0.850 | val_error: 4.35 | val_acc: 0.790 

  [=========>------------------------------] 25.40% | train_error: 3.11 | train_acc: 0.850 | val_error: 4.06 | val_acc: 0.804 

  [=========>------------------------------] 25.50% | train_error: 3.11 | train_acc: 0.850 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 25.60% | train_error: 3.11 | train_acc: 0.850 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 25.70% | train_error: 3.11 | train_acc: 0.850 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 25.80% | train_error: 3.11 | train_acc: 0.850 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 25.90% | train_error: 3.11 | train_acc: 0.850 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 26.00% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.10% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.20% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.30% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.40% | train_error: 3.21 | train_acc: 0.845 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.50% | train_error: 3.02 | train_acc: 0.854 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.60% | train_error: 2.77 | train_acc: 0.866 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.70% | train_error: 2.77 | train_acc: 0.866 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.80% | train_error: 2.72 | train_acc: 0.869 | val_error: 3.91 | val_acc: 0.811 

  [=========>------------------------------] 26.90% | train_error: 2.68 | train_acc: 0.871 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 27.00% | train_error: 2.53 | train_acc: 0.878 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 27.10% | train_error: 2.48 | train_acc: 0.880 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 27.20% | train_error: 2.48 | train_acc: 0.880 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 27.30% | train_error: 2.38 | train_acc: 0.885 | val_error: 3.77 | val_acc: 0.818 

  [=========>------------------------------] 27.40% | train_error: 2.38 | train_acc: 0.885 | val_error: 3.77 | val_acc: 0.818 

  [==========>-----------------------------] 27.50% | train_error: 2.38 | train_acc: 0.885 | val_error: 3.62 | val_acc: 0.825 

  [==========>-----------------------------] 27.60% | train_error: 2.29 | train_acc: 0.890 | val_error: 3.62 | val_acc: 0.825 

  [==========>-----------------------------] 27.70% | train_error: 2.24 | train_acc: 0.892 | val_error: 3.62 | val_acc: 0.825 

  [==========>-----------------------------] 27.80% | train_error: 2.24 | train_acc: 0.892 | val_error: 3.62 | val_acc: 0.825 

  [==========>-----------------------------] 27.90% | train_error: 2.19 | train_acc: 0.894 | val_error: 3.48 | val_acc: 0.832 

  [==========>-----------------------------] 28.00% | train_error: 2.19 | train_acc: 0.894 | val_error: 3.33 | val_acc: 0.839 

  [==========>-----------------------------] 28.10% | train_error: 2.19 | train_acc: 0.894 | val_error: 3.19 | val_acc: 0.846 

  [==========>-----------------------------] 28.20% | train_error: 2.19 | train_acc: 0.894 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 28.30% | train_error: 2.19 | train_acc: 0.894 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 28.40% | train_error: 2.14 | train_acc: 0.897 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 28.50% | train_error: 2.09 | train_acc: 0.899 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 28.60% | train_error: 2.04 | train_acc: 0.901 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 28.70% | train_error: 2.09 | train_acc: 0.899 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 28.80% | train_error: 2.09 | train_acc: 0.899 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 28.90% | train_error: 2.09 | train_acc: 0.899 | val_error: 2.90 | val_acc: 0.860 

  [==========>-----------------------------] 29.00% | train_error: 2.09 | train_acc: 0.899 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.10% | train_error: 2.09 | train_acc: 0.899 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.20% | train_error: 2.09 | train_acc: 0.899 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.30% | train_error: 2.04 | train_acc: 0.901 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.40% | train_error: 2.04 | train_acc: 0.901 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.50% | train_error: 2.04 | train_acc: 0.901 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.60% | train_error: 2.04 | train_acc: 0.901 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.70% | train_error: 2.04 | train_acc: 0.901 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.80% | train_error: 1.99 | train_acc: 0.904 | val_error: 3.04 | val_acc: 0.853 

  [==========>-----------------------------] 29.90% | train_error: 1.95 | train_acc: 0.906 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 30.00% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 30.10% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 30.20% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.19 | val_acc: 0.846 

  [===========>----------------------------] 30.30% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.19 | val_acc: 0.846 

  [===========>----------------------------] 30.40% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.19 | val_acc: 0.846 

  [===========>----------------------------] 30.50% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.19 | val_acc: 0.846 

  [===========>----------------------------] 30.60% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.19 | val_acc: 0.846 

  [===========>----------------------------] 30.70% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 30.80% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 30.90% | train_error: 1.90 | train_acc: 0.908 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.00% | train_error: 1.75 | train_acc: 0.915 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.10% | train_error: 1.75 | train_acc: 0.915 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.20% | train_error: 1.61 | train_acc: 0.923 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.30% | train_error: 1.61 | train_acc: 0.923 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.40% | train_error: 1.56 | train_acc: 0.925 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.50% | train_error: 1.56 | train_acc: 0.925 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.60% | train_error: 1.56 | train_acc: 0.925 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.70% | train_error: 1.51 | train_acc: 0.927 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.80% | train_error: 1.51 | train_acc: 0.927 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 31.90% | train_error: 1.51 | train_acc: 0.927 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 32.00% | train_error: 1.51 | train_acc: 0.927 | val_error: 3.04 | val_acc: 0.853 

  [===========>----------------------------] 32.10% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [===========>----------------------------] 32.20% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [===========>----------------------------] 32.30% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [===========>----------------------------] 32.40% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 32.50% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 32.60% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 32.70% | train_error: 1.41 | train_acc: 0.932 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 32.80% | train_error: 1.31 | train_acc: 0.937 | val_error: 3.04 | val_acc: 0.853 

  [============>---------------------------] 32.90% | train_error: 1.26 | train_acc: 0.939 | val_error: 3.04 | val_acc: 0.853 

  [============>---------------------------] 33.00% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 33.10% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 33.20% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 33.30% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.90 | val_acc: 0.860 

  [============>---------------------------] 33.40% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 33.50% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 33.60% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 33.70% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 33.80% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 33.90% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 34.00% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.61 | val_acc: 0.874 

  [============>---------------------------] 34.10% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.61 | val_acc: 0.874 

  [============>---------------------------] 34.20% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.61 | val_acc: 0.874 

  [============>---------------------------] 34.30% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.61 | val_acc: 0.874 

  [============>---------------------------] 34.40% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.61 | val_acc: 0.874 

  [============>---------------------------] 34.50% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 34.60% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 34.70% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 34.80% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [============>---------------------------] 34.90% | train_error: 1.22 | train_acc: 0.941 | val_error: 2.75 | val_acc: 0.867 

  [=============>--------------------------] 35.00% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.75 | val_acc: 0.867 

  [=============>--------------------------] 35.10% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.75 | val_acc: 0.867 

  [=============>--------------------------] 35.20% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.75 | val_acc: 0.867 

  [=============>--------------------------] 35.30% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.61 | val_acc: 0.874 

  [=============>--------------------------] 35.40% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.61 | val_acc: 0.874 

  [=============>--------------------------] 35.50% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.61 | val_acc: 0.874 

  [=============>--------------------------] 35.60% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.61 | val_acc: 0.874 

  [=============>--------------------------] 35.70% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.61 | val_acc: 0.874 

  [=============>--------------------------] 35.80% | train_error: 1.02 | train_acc: 0.951 | val_error: 2.61 | val_acc: 0.874 

  [=============>--------------------------] 35.90% | train_error: 0.876 | train_acc: 0.958 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.00% | train_error: 0.876 | train_acc: 0.958 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.10% | train_error: 0.778 | train_acc: 0.962 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.20% | train_error: 0.778 | train_acc: 0.962 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.30% | train_error: 0.778 | train_acc: 0.962 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.40% | train_error: 0.730 | train_acc: 0.965 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.50% | train_error: 0.730 | train_acc: 0.965 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.60% | train_error: 0.730 | train_acc: 0.965 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.70% | train_error: 0.730 | train_acc: 0.965 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.80% | train_error: 0.730 | train_acc: 0.965 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 36.90% | train_error: 0.730 | train_acc: 0.965 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 37.00% | train_error: 0.681 | train_acc: 0.967 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 37.10% | train_error: 0.681 | train_acc: 0.967 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 37.20% | train_error: 0.681 | train_acc: 0.967 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 37.30% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [=============>--------------------------] 37.40% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 37.50% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 37.60% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 37.70% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 37.80% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 37.90% | train_error: 0.632 | train_acc: 0.969 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.00% | train_error: 0.535 | train_acc: 0.974 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.10% | train_error: 0.535 | train_acc: 0.974 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.20% | train_error: 0.486 | train_acc: 0.977 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.30% | train_error: 0.341 | train_acc: 0.984 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.40% | train_error: 0.486 | train_acc: 0.977 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.50% | train_error: 0.341 | train_acc: 0.984 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.60% | train_error: 0.486 | train_acc: 0.977 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.70% | train_error: 0.341 | train_acc: 0.984 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.80% | train_error: 0.486 | train_acc: 0.977 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 38.90% | train_error: 0.341 | train_acc: 0.984 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.00% | train_error: 0.486 | train_acc: 0.977 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.10% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.20% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.30% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.40% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.50% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.60% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.70% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.80% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [==============>-------------------------] 39.90% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 40.00% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 40.10% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 40.20% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 40.30% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 40.40% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 40.50% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 40.60% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 40.70% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 40.80% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 40.90% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 41.00% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 41.10% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 41.20% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 41.30% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 41.40% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 41.50% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 41.60% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 41.70% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 41.80% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 41.90% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.17 | val_acc: 0.895 

  [===============>------------------------] 42.00% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [===============>------------------------] 42.10% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.17 | val_acc: 0.895 

  [===============>------------------------] 42.20% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 42.30% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [===============>------------------------] 42.40% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 42.50% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 42.60% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 42.70% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 42.80% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 42.90% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 43.00% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 43.10% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 43.20% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 43.30% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 43.40% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 43.50% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 43.60% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 43.70% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.17 | val_acc: 0.895 

  [================>-----------------------] 43.80% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 43.90% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [================>-----------------------] 44.00% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 44.10% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [================>-----------------------] 44.20% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.46 | val_acc: 0.881 

  [================>-----------------------] 44.30% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [================>-----------------------] 44.40% | train_error: 0.292 | train_acc: 0.986 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 44.50% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [================>-----------------------] 44.60% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 44.70% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [================>-----------------------] 44.80% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.32 | val_acc: 0.888 

  [================>-----------------------] 44.90% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.00% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.32 | val_acc: 0.888 

  [=================>----------------------] 45.10% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.20% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.30% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.40% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.50% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.60% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.70% | train_error: 0.243 | train_acc: 0.988 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.80% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 45.90% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 46.00% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 46.10% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 46.20% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 46.30% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 46.40% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 46.50% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 46.60% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 46.70% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 46.80% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 46.90% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 47.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 47.10% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 47.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [=================>----------------------] 47.30% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [=================>----------------------] 47.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 47.50% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 47.60% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 47.70% | train_error: 0.195 | train_acc: 0.991 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 47.80% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 47.90% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 48.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 48.10% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 48.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 48.30% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 48.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 48.50% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 48.60% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 48.70% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 48.80% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 48.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 49.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 49.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 49.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 49.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 49.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 49.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 49.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 49.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [==================>---------------------] 49.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [==================>---------------------] 49.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 50.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 50.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 50.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 50.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 50.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 50.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 50.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 50.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 50.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 50.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 51.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 51.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 51.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 51.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 51.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 51.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 51.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 51.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 51.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 51.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 52.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.17 | val_acc: 0.895 

  [===================>--------------------] 52.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 52.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 52.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [===================>--------------------] 52.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 52.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 52.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 52.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 52.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 52.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 53.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 53.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 53.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 53.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 54.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 54.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 54.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 54.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 54.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 54.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 54.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 54.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [====================>-------------------] 54.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [====================>-------------------] 54.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 55.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 55.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 55.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 55.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 55.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 55.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 55.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 55.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 55.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 55.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 56.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 56.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 56.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 56.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 56.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 56.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 56.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 56.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 56.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 56.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 57.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 57.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 57.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 2.03 | val_acc: 0.902 

  [=====================>------------------] 57.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=====================>------------------] 57.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 57.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 57.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 57.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 57.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 57.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 58.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [======================>-----------------] 59.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 60.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 61.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 62.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 62.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 62.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 62.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [=======================>----------------] 62.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 62.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 62.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 62.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 62.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 62.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 63.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [========================>---------------] 64.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 64.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [========================>---------------] 64.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 64.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [========================>---------------] 64.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 64.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [========================>---------------] 64.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 64.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [========================>---------------] 64.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [========================>---------------] 64.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 65.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 65.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 65.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 65.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 65.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 65.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 65.60% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 65.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 65.80% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 65.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 66.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 66.10% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 66.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 66.30% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 66.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 66.50% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 66.60% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 66.70% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 66.80% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 66.90% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 67.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 67.10% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 67.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [=========================>--------------] 67.30% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [=========================>--------------] 67.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 67.50% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 67.60% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 67.70% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 67.80% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 67.90% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 68.00% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 68.10% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 68.20% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 68.30% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 68.40% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 68.50% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 68.60% | train_error: 0.146 | train_acc: 0.993 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 68.70% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 68.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 68.90% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 69.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 69.10% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 69.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 69.30% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 69.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 69.50% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 69.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 69.70% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [==========================>-------------] 69.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.88 | val_acc: 0.909 

  [==========================>-------------] 69.90% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.10% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.30% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.50% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.70% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 70.90% | train_error: 0.0486 | train_acc: 0.998 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 71.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 72.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 72.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 72.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 72.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===========================>------------] 72.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 72.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 72.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 72.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 72.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 72.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 73.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [============================>-----------] 74.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 75.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 76.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 77.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 77.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=============================>----------] 77.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=============================>----------] 77.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=============================>----------] 77.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 77.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 77.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 77.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 77.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 77.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 78.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 78.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 78.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 78.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 78.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 78.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 78.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 78.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 78.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 78.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 79.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 79.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 79.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 79.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 79.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 79.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 79.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 79.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==============================>---------] 79.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==============================>---------] 79.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 80.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 80.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 80.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 80.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 80.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 80.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 80.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 80.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 80.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 80.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 81.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 81.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 81.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 81.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 81.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 81.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 81.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 81.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 81.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 81.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 82.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 82.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 82.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===============================>--------] 82.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===============================>--------] 82.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 82.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 82.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 82.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 82.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 82.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 83.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 83.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 83.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 83.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 83.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 83.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 83.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 83.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 83.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 83.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 84.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 84.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 84.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 84.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 84.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 84.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 84.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 84.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [================================>-------] 84.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [================================>-------] 84.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 85.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 85.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 85.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 85.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 85.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 85.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 85.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 85.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 85.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 85.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 86.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 86.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 86.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 86.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 86.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 86.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 86.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 86.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 86.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 86.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 87.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 87.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 87.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=================================>------] 87.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [=================================>------] 87.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 87.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 87.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 87.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 87.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 87.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 88.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 88.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 88.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 88.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 88.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 88.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 88.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 88.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 88.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 88.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 89.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 89.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 89.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 89.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 89.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 89.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 89.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 89.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [==================================>-----] 89.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [==================================>-----] 89.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.59 | val_acc: 0.923 

  [===================================>----] 90.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 90.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 91.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 92.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 92.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 92.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 92.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [===================================>----] 92.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 92.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 92.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 92.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 92.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 92.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 93.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [====================================>---] 94.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 95.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 96.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 97.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 97.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 97.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 97.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=====================================>--] 97.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 97.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 97.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 97.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 97.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 97.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 98.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.00% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.10% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.20% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.30% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.40% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.50% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.60% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.70% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.80% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [======================================>-] 99.90% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

  [=======================================>] 100.0% | train_error: 0.0973 | train_acc: 0.995 | val_error: 1.74 | val_acc: 0.916 

Multiclass classification¶

Finally, we will demonstrate the use case of multiclass classification using our FFNN with the famous MNIST dataset, which contain images of digits between the range of 0 to 9.

from sklearn.datasets import load_digits

def onehot(target: np.ndarray):
    onehot = np.zeros((target.size, target.max() + 1))
    onehot[np.arange(target.size), target] = 1
    return onehot

digits = load_digits()

X = digits.data
target = digits.target
target = onehot(target)

input_nodes = 64
hidden_nodes1 = 100
hidden_nodes2 = 30
output_nodes = 10

dims = (input_nodes, hidden_nodes1, hidden_nodes2, output_nodes)

multiclass = FFNN(dims, hidden_func=LRELU, output_func=softmax, cost_func=CostCrossEntropy)

multiclass.reset_weights() # reset weights such that previous runs or reruns don't affect the weights

scheduler = Adam(eta=1e-4, rho=0.9, rho2=0.999)
scores = multiclass.fit(X, target, scheduler, epochs=1000)

Adam: Eta=0.0001, Lambda=0

  [----------------------------------------] 0.000% | train_error: 1.83 | train_acc: 0.823 

  [----------------------------------------] 0.1000% | train_error: 1.83 | train_acc: 0.823 

  [----------------------------------------] 0.2000% | train_error: 1.83 | train_acc: 0.823 

  [----------------------------------------] 0.3000% | train_error: 1.83 | train_acc: 0.823 

  [----------------------------------------] 0.4000% | train_error: 1.83 | train_acc: 0.823 

  [----------------------------------------] 0.5000% | train_error: 1.83 | train_acc: 0.823 

  [----------------------------------------] 0.6000% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 0.7000% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 0.8000% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 0.9000% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.000% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.100% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.200% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.300% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.400% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.500% | train_error: 1.82 | train_acc: 0.824 

  [----------------------------------------] 1.600% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.700% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.800% | train_error: 1.83 | train_acc: 0.824 

  [----------------------------------------] 1.900% | train_error: 1.82 | train_acc: 0.824 

  [----------------------------------------] 2.000% | train_error: 1.82 | train_acc: 0.824 

  [----------------------------------------] 2.100% | train_error: 1.82 | train_acc: 0.824 

  [----------------------------------------] 2.200% | train_error: 1.82 | train_acc: 0.824 

  [----------------------------------------] 2.300% | train_error: 1.82 | train_acc: 0.824 

  [----------------------------------------] 2.400% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 2.500% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 2.600% | train_error: 1.83 | train_acc: 0.824 

  [>---------------------------------------] 2.700% | train_error: 1.83 | train_acc: 0.824 

  [>---------------------------------------] 2.800% | train_error: 1.83 | train_acc: 0.824 

  [>---------------------------------------] 2.900% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.000% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.100% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.200% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.300% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.400% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.500% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.600% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.700% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.800% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 3.900% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 4.000% | train_error: 1.82 | train_acc: 0.824 

  [>---------------------------------------] 4.100% | train_error: 1.83 | train_acc: 0.824 

  [>---------------------------------------] 4.200% | train_error: 1.83 | train_acc: 0.824 

  [>---------------------------------------] 4.300% | train_error: 1.83 | train_acc: 0.824 

  [>---------------------------------------] 4.400% | train_error: 1.83 | train_acc: 0.823 

  [>---------------------------------------] 4.500% | train_error: 1.83 | train_acc: 0.823 

  [>---------------------------------------] 4.600% | train_error: 1.83 | train_acc: 0.823 

  [>---------------------------------------] 4.700% | train_error: 1.83 | train_acc: 0.823 

  [>---------------------------------------] 4.800% | train_error: 1.83 | train_acc: 0.823 

  [>---------------------------------------] 4.900% | train_error: 1.83 | train_acc: 0.823 

  [=>--------------------------------------] 5.000% | train_error: 1.83 | train_acc: 0.823 

  [=>--------------------------------------] 5.100% | train_error: 1.83 | train_acc: 0.823 

  [=>--------------------------------------] 5.200% | train_error: 1.83 | train_acc: 0.823 

  [=>--------------------------------------] 5.300% | train_error: 1.83 | train_acc: 0.823 

  [=>--------------------------------------] 5.400% | train_error: 1.83 | train_acc: 0.823 

  [=>--------------------------------------] 5.500% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 5.600% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 5.700% | train_error: 1.84 | train_acc: 0.822 

  [=>--------------------------------------] 5.800% | train_error: 1.84 | train_acc: 0.822 

  [=>--------------------------------------] 5.900% | train_error: 1.84 | train_acc: 0.822 

  [=>--------------------------------------] 6.000% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.100% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.200% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.300% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.400% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.500% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.600% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.700% | train_error: 1.84 | train_acc: 0.823 

  [=>--------------------------------------] 6.800% | train_error: 1.84 | train_acc: 0.822 

  [=>--------------------------------------] 6.900% | train_error: 1.85 | train_acc: 0.822 

  [=>--------------------------------------] 7.000% | train_error: 1.85 | train_acc: 0.822 

  [=>--------------------------------------] 7.100% | train_error: 1.85 | train_acc: 0.821 

  [=>--------------------------------------] 7.200% | train_error: 1.85 | train_acc: 0.821 

  [=>--------------------------------------] 7.300% | train_error: 1.85 | train_acc: 0.821 

  [=>--------------------------------------] 7.400% | train_error: 1.85 | train_acc: 0.821 

  [==>-------------------------------------] 7.500% | train_error: 1.86 | train_acc: 0.821 

  [==>-------------------------------------] 7.600% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 7.700% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 7.800% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 7.900% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.000% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.100% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.200% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.300% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.400% | train_error: 1.87 | train_acc: 0.820 

  [==>-------------------------------------] 8.500% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.600% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.700% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.800% | train_error: 1.86 | train_acc: 0.820 

  [==>-------------------------------------] 8.900% | train_error: 1.87 | train_acc: 0.820 

  [==>-------------------------------------] 9.000% | train_error: 1.87 | train_acc: 0.820 

  [==>-------------------------------------] 9.100% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.200% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.300% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.400% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.500% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.600% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.700% | train_error: 1.87 | train_acc: 0.819 

  [==>-------------------------------------] 9.800% | train_error: 1.88 | train_acc: 0.819 

  [==>-------------------------------------] 9.900% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.00% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.10% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.20% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.30% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.40% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.50% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.60% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.70% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.80% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 10.90% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 11.00% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 11.10% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 11.20% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 11.30% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 11.40% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 11.50% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 11.60% | train_error: 1.88 | train_acc: 0.819 

  [===>------------------------------------] 11.70% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 11.80% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 11.90% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 12.00% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 12.10% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 12.20% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 12.30% | train_error: 1.88 | train_acc: 0.818 

  [===>------------------------------------] 12.40% | train_error: 1.88 | train_acc: 0.818 

  [====>-----------------------------------] 12.50% | train_error: 1.88 | train_acc: 0.818 

  [====>-----------------------------------] 12.60% | train_error: 1.88 | train_acc: 0.818 

  [====>-----------------------------------] 12.70% | train_error: 1.88 | train_acc: 0.819 

  [====>-----------------------------------] 12.80% | train_error: 1.88 | train_acc: 0.819 

  [====>-----------------------------------] 12.90% | train_error: 1.88 | train_acc: 0.819 

  [====>-----------------------------------] 13.00% | train_error: 1.88 | train_acc: 0.819 

  [====>-----------------------------------] 13.10% | train_error: 1.87 | train_acc: 0.819 

  [====>-----------------------------------] 13.20% | train_error: 1.87 | train_acc: 0.819 

  [====>-----------------------------------] 13.30% | train_error: 1.87 | train_acc: 0.819 

  [====>-----------------------------------] 13.40% | train_error: 1.87 | train_acc: 0.819 

  [====>-----------------------------------] 13.50% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 13.60% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 13.70% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 13.80% | train_error: 1.86 | train_acc: 0.820 

  [====>-----------------------------------] 13.90% | train_error: 1.86 | train_acc: 0.820 

  [====>-----------------------------------] 14.00% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 14.10% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 14.20% | train_error: 1.86 | train_acc: 0.820 

  [====>-----------------------------------] 14.30% | train_error: 1.86 | train_acc: 0.820 

  [====>-----------------------------------] 14.40% | train_error: 1.86 | train_acc: 0.820 

  [====>-----------------------------------] 14.50% | train_error: 1.86 | train_acc: 0.820 

  [====>-----------------------------------] 14.60% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 14.70% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 14.80% | train_error: 1.87 | train_acc: 0.820 

  [====>-----------------------------------] 14.90% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 15.00% | train_error: 1.86 | train_acc: 0.820 

  [=====>----------------------------------] 15.10% | train_error: 1.86 | train_acc: 0.820 

  [=====>----------------------------------] 15.20% | train_error: 1.86 | train_acc: 0.820 

  [=====>----------------------------------] 15.30% | train_error: 1.86 | train_acc: 0.820 

  [=====>----------------------------------] 15.40% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 15.50% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 15.60% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 15.70% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 15.80% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 15.90% | train_error: 1.87 | train_acc: 0.820 

  [=====>----------------------------------] 16.00% | train_error: 1.87 | train_acc: 0.819 

  [=====>----------------------------------] 16.10% | train_error: 1.87 | train_acc: 0.819 

  [=====>----------------------------------] 16.20% | train_error: 1.87 | train_acc: 0.819 

  [=====>----------------------------------] 16.30% | train_error: 1.87 | train_acc: 0.819 

  [=====>----------------------------------] 16.40% | train_error: 1.87 | train_acc: 0.819 

  [=====>----------------------------------] 16.50% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 16.60% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 16.70% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 16.80% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 16.90% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 17.00% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 17.10% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 17.20% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 17.30% | train_error: 1.88 | train_acc: 0.819 

  [=====>----------------------------------] 17.40% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 17.50% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 17.60% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 17.70% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 17.80% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 17.90% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 18.00% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 18.10% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 18.20% | train_error: 1.87 | train_acc: 0.820 

  [======>---------------------------------] 18.30% | train_error: 1.87 | train_acc: 0.820 

  [======>---------------------------------] 18.40% | train_error: 1.87 | train_acc: 0.819 

  [======>---------------------------------] 18.50% | train_error: 1.87 | train_acc: 0.820 

  [======>---------------------------------] 18.60% | train_error: 1.87 | train_acc: 0.819 

  [======>---------------------------------] 18.70% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 18.80% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 18.90% | train_error: 1.87 | train_acc: 0.819 

  [======>---------------------------------] 19.00% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 19.10% | train_error: 1.87 | train_acc: 0.819 

  [======>---------------------------------] 19.20% | train_error: 1.87 | train_acc: 0.819 

  [======>---------------------------------] 19.30% | train_error: 1.87 | train_acc: 0.819 

  [======>---------------------------------] 19.40% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 19.50% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 19.60% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 19.70% | train_error: 1.88 | train_acc: 0.819 

  [======>---------------------------------] 19.80% | train_error: 1.88 | train_acc: 0.818 

  [======>---------------------------------] 19.90% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.00% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.10% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.20% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.30% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.40% | train_error: 1.88 | train_acc: 0.818 

  [=======>--------------------------------] 20.50% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.60% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.70% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.80% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 20.90% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 21.00% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 21.10% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 21.20% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 21.30% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 21.40% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 21.50% | train_error: 1.88 | train_acc: 0.818 

  [=======>--------------------------------] 21.60% | train_error: 1.88 | train_acc: 0.818 

  [=======>--------------------------------] 21.70% | train_error: 1.88 | train_acc: 0.818 

  [=======>--------------------------------] 21.80% | train_error: 1.88 | train_acc: 0.818 

  [=======>--------------------------------] 21.90% | train_error: 1.88 | train_acc: 0.818 

  [=======>--------------------------------] 22.00% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 22.10% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 22.20% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 22.30% | train_error: 1.88 | train_acc: 0.819 

  [=======>--------------------------------] 22.40% | train_error: 1.88 | train_acc: 0.819 

  [========>-------------------------------] 22.50% | train_error: 1.88 | train_acc: 0.819 

  [========>-------------------------------] 22.60% | train_error: 1.88 | train_acc: 0.819 

  [========>-------------------------------] 22.70% | train_error: 1.88 | train_acc: 0.819 

  [========>-------------------------------] 22.80% | train_error: 1.88 | train_acc: 0.819 

  [========>-------------------------------] 22.90% | train_error: 1.88 | train_acc: 0.819 

  [========>-------------------------------] 23.00% | train_error: 1.87 | train_acc: 0.819 

  [========>-------------------------------] 23.10% | train_error: 1.87 | train_acc: 0.819 

  [========>-------------------------------] 23.20% | train_error: 1.87 | train_acc: 0.820 

  [========>-------------------------------] 23.30% | train_error: 1.87 | train_acc: 0.820 

  [========>-------------------------------] 23.40% | train_error: 1.86 | train_acc: 0.820 

  [========>-------------------------------] 23.50% | train_error: 1.86 | train_acc: 0.820 

  [========>-------------------------------] 23.60% | train_error: 1.86 | train_acc: 0.820 

  [========>-------------------------------] 23.70% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 23.80% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 23.90% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 24.00% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 24.10% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 24.20% | train_error: 1.86 | train_acc: 0.820 

  [========>-------------------------------] 24.30% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 24.40% | train_error: 1.86 | train_acc: 0.821 

  [========>-------------------------------] 24.50% | train_error: 1.85 | train_acc: 0.821 

  [========>-------------------------------] 24.60% | train_error: 1.85 | train_acc: 0.821 

  [========>-------------------------------] 24.70% | train_error: 1.85 | train_acc: 0.822 

  [========>-------------------------------] 24.80% | train_error: 1.85 | train_acc: 0.821 

  [========>-------------------------------] 24.90% | train_error: 1.85 | train_acc: 0.822 

  [=========>------------------------------] 25.00% | train_error: 1.85 | train_acc: 0.822 

  [=========>------------------------------] 25.10% | train_error: 1.84 | train_acc: 0.822 

  [=========>------------------------------] 25.20% | train_error: 1.84 | train_acc: 0.822 

  [=========>------------------------------] 25.30% | train_error: 1.84 | train_acc: 0.822 

  [=========>------------------------------] 25.40% | train_error: 1.84 | train_acc: 0.822 

  [=========>------------------------------] 25.50% | train_error: 1.84 | train_acc: 0.822 

  [=========>------------------------------] 25.60% | train_error: 1.84 | train_acc: 0.823 

  [=========>------------------------------] 25.70% | train_error: 1.84 | train_acc: 0.823 

  [=========>------------------------------] 25.80% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 25.90% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 26.00% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 26.10% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 26.20% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 26.30% | train_error: 1.83 | train_acc: 0.824 

  [=========>------------------------------] 26.40% | train_error: 1.83 | train_acc: 0.824 

  [=========>------------------------------] 26.50% | train_error: 1.83 | train_acc: 0.824 

  [=========>------------------------------] 26.60% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 26.70% | train_error: 1.83 | train_acc: 0.823 

  [=========>------------------------------] 26.80% | train_error: 1.83 | train_acc: 0.824 

  [=========>------------------------------] 26.90% | train_error: 1.82 | train_acc: 0.824 

  [=========>------------------------------] 27.00% | train_error: 1.82 | train_acc: 0.824 

  [=========>------------------------------] 27.10% | train_error: 1.82 | train_acc: 0.824 

  [=========>------------------------------] 27.20% | train_error: 1.82 | train_acc: 0.824 

  [=========>------------------------------] 27.30% | train_error: 1.82 | train_acc: 0.825 

  [=========>------------------------------] 27.40% | train_error: 1.82 | train_acc: 0.825 

  [==========>-----------------------------] 27.50% | train_error: 1.82 | train_acc: 0.825 

  [==========>-----------------------------] 27.60% | train_error: 1.82 | train_acc: 0.825 

  [==========>-----------------------------] 27.70% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 27.80% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 27.90% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 28.00% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 28.10% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 28.20% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 28.30% | train_error: 1.81 | train_acc: 0.825 

  [==========>-----------------------------] 28.40% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 28.50% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 28.60% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 28.70% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 28.80% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 28.90% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 29.00% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 29.10% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 29.20% | train_error: 1.80 | train_acc: 0.827 

  [==========>-----------------------------] 29.30% | train_error: 1.80 | train_acc: 0.827 

  [==========>-----------------------------] 29.40% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 29.50% | train_error: 1.80 | train_acc: 0.827 

  [==========>-----------------------------] 29.60% | train_error: 1.80 | train_acc: 0.827 

  [==========>-----------------------------] 29.70% | train_error: 1.80 | train_acc: 0.826 

  [==========>-----------------------------] 29.80% | train_error: 1.79 | train_acc: 0.827 

  [==========>-----------------------------] 29.90% | train_error: 1.79 | train_acc: 0.827 

  [===========>----------------------------] 30.00% | train_error: 1.79 | train_acc: 0.827 

  [===========>----------------------------] 30.10% | train_error: 1.79 | train_acc: 0.828 

  [===========>----------------------------] 30.20% | train_error: 1.79 | train_acc: 0.827 

  [===========>----------------------------] 30.30% | train_error: 1.79 | train_acc: 0.827 

  [===========>----------------------------] 30.40% | train_error: 1.79 | train_acc: 0.827 

  [===========>----------------------------] 30.50% | train_error: 1.79 | train_acc: 0.828 

  [===========>----------------------------] 30.60% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 30.70% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 30.80% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 30.90% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 31.00% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 31.10% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 31.20% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 31.30% | train_error: 1.78 | train_acc: 0.828 

  [===========>----------------------------] 31.40% | train_error: 1.78 | train_acc: 0.829 

  [===========>----------------------------] 31.50% | train_error: 1.78 | train_acc: 0.829 

  [===========>----------------------------] 31.60% | train_error: 1.77 | train_acc: 0.829 

  [===========>----------------------------] 31.70% | train_error: 1.78 | train_acc: 0.829 

  [===========>----------------------------] 31.80% | train_error: 1.78 | train_acc: 0.829 

  [===========>----------------------------] 31.90% | train_error: 1.77 | train_acc: 0.829 

  [===========>----------------------------] 32.00% | train_error: 1.77 | train_acc: 0.829 

  [===========>----------------------------] 32.10% | train_error: 1.76 | train_acc: 0.830 

  [===========>----------------------------] 32.20% | train_error: 1.77 | train_acc: 0.829 

  [===========>----------------------------] 32.30% | train_error: 1.76 | train_acc: 0.830 

  [===========>----------------------------] 32.40% | train_error: 1.76 | train_acc: 0.830 

  [============>---------------------------] 32.50% | train_error: 1.76 | train_acc: 0.830 

  [============>---------------------------] 32.60% | train_error: 1.76 | train_acc: 0.830 

  [============>---------------------------] 32.70% | train_error: 1.76 | train_acc: 0.830 

  [============>---------------------------] 32.80% | train_error: 1.76 | train_acc: 0.830 

  [============>---------------------------] 32.90% | train_error: 1.76 | train_acc: 0.831 

  [============>---------------------------] 33.00% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.10% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.20% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.30% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.40% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.50% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.60% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.70% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.80% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 33.90% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 34.00% | train_error: 1.75 | train_acc: 0.831 

  [============>---------------------------] 34.10% | train_error: 1.74 | train_acc: 0.832 

  [============>---------------------------] 34.20% | train_error: 1.74 | train_acc: 0.832 

  [============>---------------------------] 34.30% | train_error: 1.74 | train_acc: 0.832 

  [============>---------------------------] 34.40% | train_error: 1.74 | train_acc: 0.832 

  [============>---------------------------] 34.50% | train_error: 1.74 | train_acc: 0.832 

  [============>---------------------------] 34.60% | train_error: 1.73 | train_acc: 0.833 

  [============>---------------------------] 34.70% | train_error: 1.73 | train_acc: 0.833 

  [============>---------------------------] 34.80% | train_error: 1.73 | train_acc: 0.833 

  [============>---------------------------] 34.90% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 35.00% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 35.10% | train_error: 1.73 | train_acc: 0.834 

  [=============>--------------------------] 35.20% | train_error: 1.73 | train_acc: 0.834 

  [=============>--------------------------] 35.30% | train_error: 1.72 | train_acc: 0.834 

  [=============>--------------------------] 35.40% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 35.50% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 35.60% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 35.70% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 35.80% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 35.90% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.00% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 36.10% | train_error: 1.73 | train_acc: 0.833 

  [=============>--------------------------] 36.20% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.30% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.40% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.50% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.60% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.70% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.80% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 36.90% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 37.00% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 37.10% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 37.20% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 37.30% | train_error: 1.74 | train_acc: 0.832 

  [=============>--------------------------] 37.40% | train_error: 1.74 | train_acc: 0.832 

  [==============>-------------------------] 37.50% | train_error: 1.74 | train_acc: 0.832 

  [==============>-------------------------] 37.60% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 37.70% | train_error: 1.72 | train_acc: 0.834 

  [==============>-------------------------] 37.80% | train_error: 1.73 | train_acc: 0.834 

  [==============>-------------------------] 37.90% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 38.00% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 38.10% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 38.20% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 38.30% | train_error: 1.72 | train_acc: 0.834 

  [==============>-------------------------] 38.40% | train_error: 1.73 | train_acc: 0.834 

  [==============>-------------------------] 38.50% | train_error: 1.73 | train_acc: 0.834 

  [==============>-------------------------] 38.60% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 38.70% | train_error: 1.73 | train_acc: 0.834 

  [==============>-------------------------] 38.80% | train_error: 1.72 | train_acc: 0.834 

  [==============>-------------------------] 38.90% | train_error: 1.72 | train_acc: 0.834 

  [==============>-------------------------] 39.00% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.10% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.20% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.30% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.40% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.50% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.60% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.70% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.80% | train_error: 1.73 | train_acc: 0.833 

  [==============>-------------------------] 39.90% | train_error: 1.73 | train_acc: 0.833 

  [===============>------------------------] 40.00% | train_error: 1.73 | train_acc: 0.833 

  [===============>------------------------] 40.10% | train_error: 1.72 | train_acc: 0.834 

  [===============>------------------------] 40.20% | train_error: 1.72 | train_acc: 0.834 

  [===============>------------------------] 40.30% | train_error: 1.72 | train_acc: 0.834 

  [===============>------------------------] 40.40% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 40.50% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 40.60% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 40.70% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 40.80% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 40.90% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 41.00% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 41.10% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 41.20% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 41.30% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 41.40% | train_error: 1.71 | train_acc: 0.835 

  [===============>------------------------] 41.50% | train_error: 1.70 | train_acc: 0.836 

  [===============>------------------------] 41.60% | train_error: 1.70 | train_acc: 0.836 

  [===============>------------------------] 41.70% | train_error: 1.70 | train_acc: 0.836 

  [===============>------------------------] 41.80% | train_error: 1.70 | train_acc: 0.836 

  [===============>------------------------] 41.90% | train_error: 1.70 | train_acc: 0.836 

  [===============>------------------------] 42.00% | train_error: 1.70 | train_acc: 0.836 

  [===============>------------------------] 42.10% | train_error: 1.69 | train_acc: 0.837 

  [===============>------------------------] 42.20% | train_error: 1.69 | train_acc: 0.837 

  [===============>------------------------] 42.30% | train_error: 1.69 | train_acc: 0.837 

  [===============>------------------------] 42.40% | train_error: 1.69 | train_acc: 0.837 

  [================>-----------------------] 42.50% | train_error: 1.69 | train_acc: 0.837 

  [================>-----------------------] 42.60% | train_error: 1.69 | train_acc: 0.837 

  [================>-----------------------] 42.70% | train_error: 1.68 | train_acc: 0.838 

  [================>-----------------------] 42.80% | train_error: 1.68 | train_acc: 0.838 

  [================>-----------------------] 42.90% | train_error: 1.68 | train_acc: 0.838 

  [================>-----------------------] 43.00% | train_error: 1.68 | train_acc: 0.838 

  [================>-----------------------] 43.10% | train_error: 1.68 | train_acc: 0.838 

  [================>-----------------------] 43.20% | train_error: 1.67 | train_acc: 0.839 

  [================>-----------------------] 43.30% | train_error: 1.67 | train_acc: 0.839 

  [================>-----------------------] 43.40% | train_error: 1.67 | train_acc: 0.839 

  [================>-----------------------] 43.50% | train_error: 1.67 | train_acc: 0.839 

  [================>-----------------------] 43.60% | train_error: 1.66 | train_acc: 0.840 

  [================>-----------------------] 43.70% | train_error: 1.66 | train_acc: 0.840 

  [================>-----------------------] 43.80% | train_error: 1.65 | train_acc: 0.840 

  [================>-----------------------] 43.90% | train_error: 1.65 | train_acc: 0.841 

  [================>-----------------------] 44.00% | train_error: 1.65 | train_acc: 0.841 

  [================>-----------------------] 44.10% | train_error: 1.64 | train_acc: 0.841 

  [================>-----------------------] 44.20% | train_error: 1.64 | train_acc: 0.842 

  [================>-----------------------] 44.30% | train_error: 1.62 | train_acc: 0.843 

  [================>-----------------------] 44.40% | train_error: 1.62 | train_acc: 0.843 

  [================>-----------------------] 44.50% | train_error: 1.62 | train_acc: 0.843 

  [================>-----------------------] 44.60% | train_error: 1.62 | train_acc: 0.844 

  [================>-----------------------] 44.70% | train_error: 1.61 | train_acc: 0.844 

  [================>-----------------------] 44.80% | train_error: 1.61 | train_acc: 0.845 

  [================>-----------------------] 44.90% | train_error: 1.60 | train_acc: 0.846 

  [=================>----------------------] 45.00% | train_error: 1.59 | train_acc: 0.846 

  [=================>----------------------] 45.10% | train_error: 1.59 | train_acc: 0.846 

  [=================>----------------------] 45.20% | train_error: 1.59 | train_acc: 0.847 

  [=================>----------------------] 45.30% | train_error: 1.58 | train_acc: 0.847 

  [=================>----------------------] 45.40% | train_error: 1.58 | train_acc: 0.847 

  [=================>----------------------] 45.50% | train_error: 1.58 | train_acc: 0.848 

  [=================>----------------------] 45.60% | train_error: 1.57 | train_acc: 0.848 

  [=================>----------------------] 45.70% | train_error: 1.57 | train_acc: 0.849 

  [=================>----------------------] 45.80% | train_error: 1.56 | train_acc: 0.849 

  [=================>----------------------] 45.90% | train_error: 1.56 | train_acc: 0.850 

  [=================>----------------------] 46.00% | train_error: 1.55 | train_acc: 0.850 

  [=================>----------------------] 46.10% | train_error: 1.55 | train_acc: 0.851 

  [=================>----------------------] 46.20% | train_error: 1.55 | train_acc: 0.851 

  [=================>----------------------] 46.30% | train_error: 1.54 | train_acc: 0.852 

  [=================>----------------------] 46.40% | train_error: 1.53 | train_acc: 0.852 

  [=================>----------------------] 46.50% | train_error: 1.53 | train_acc: 0.852 

  [=================>----------------------] 46.60% | train_error: 1.53 | train_acc: 0.853 

  [=================>----------------------] 46.70% | train_error: 1.52 | train_acc: 0.853 

  [=================>----------------------] 46.80% | train_error: 1.52 | train_acc: 0.854 

  [=================>----------------------] 46.90% | train_error: 1.52 | train_acc: 0.854 

  [=================>----------------------] 47.00% | train_error: 1.51 | train_acc: 0.854 

  [=================>----------------------] 47.10% | train_error: 1.51 | train_acc: 0.854 

  [=================>----------------------] 47.20% | train_error: 1.50 | train_acc: 0.855 

  [=================>----------------------] 47.30% | train_error: 1.50 | train_acc: 0.855 

  [=================>----------------------] 47.40% | train_error: 1.49 | train_acc: 0.856 

  [==================>---------------------] 47.50% | train_error: 1.47 | train_acc: 0.858 

  [==================>---------------------] 47.60% | train_error: 1.47 | train_acc: 0.858 

  [==================>---------------------] 47.70% | train_error: 1.47 | train_acc: 0.858 

  [==================>---------------------] 47.80% | train_error: 1.46 | train_acc: 0.859 

  [==================>---------------------] 47.90% | train_error: 1.46 | train_acc: 0.859 

  [==================>---------------------] 48.00% | train_error: 1.46 | train_acc: 0.859 

  [==================>---------------------] 48.10% | train_error: 1.46 | train_acc: 0.859 

  [==================>---------------------] 48.20% | train_error: 1.46 | train_acc: 0.859 

  [==================>---------------------] 48.30% | train_error: 1.46 | train_acc: 0.859 

  [==================>---------------------] 48.40% | train_error: 1.45 | train_acc: 0.860 

  [==================>---------------------] 48.50% | train_error: 1.44 | train_acc: 0.861 

  [==================>---------------------] 48.60% | train_error: 1.44 | train_acc: 0.861 

  [==================>---------------------] 48.70% | train_error: 1.44 | train_acc: 0.861 

  [==================>---------------------] 48.80% | train_error: 1.44 | train_acc: 0.861 

  [==================>---------------------] 48.90% | train_error: 1.43 | train_acc: 0.862 

  [==================>---------------------] 49.00% | train_error: 1.42 | train_acc: 0.863 

  [==================>---------------------] 49.10% | train_error: 1.41 | train_acc: 0.864 

  [==================>---------------------] 49.20% | train_error: 1.40 | train_acc: 0.865 

  [==================>---------------------] 49.30% | train_error: 1.39 | train_acc: 0.866 

  [==================>---------------------] 49.40% | train_error: 1.39 | train_acc: 0.866 

  [==================>---------------------] 49.50% | train_error: 1.39 | train_acc: 0.866 

  [==================>---------------------] 49.60% | train_error: 1.39 | train_acc: 0.866 

  [==================>---------------------] 49.70% | train_error: 1.38 | train_acc: 0.867 

  [==================>---------------------] 49.80% | train_error: 1.38 | train_acc: 0.867 

  [==================>---------------------] 49.90% | train_error: 1.37 | train_acc: 0.868 

  [===================>--------------------] 50.00% | train_error: 1.36 | train_acc: 0.869 

  [===================>--------------------] 50.10% | train_error: 1.35 | train_acc: 0.869 

  [===================>--------------------] 50.20% | train_error: 1.35 | train_acc: 0.870 

  [===================>--------------------] 50.30% | train_error: 1.34 | train_acc: 0.870 

  [===================>--------------------] 50.40% | train_error: 1.34 | train_acc: 0.870 

  [===================>--------------------] 50.50% | train_error: 1.34 | train_acc: 0.870 

  [===================>--------------------] 50.60% | train_error: 1.34 | train_acc: 0.871 

  [===================>--------------------] 50.70% | train_error: 1.33 | train_acc: 0.872 

  [===================>--------------------] 50.80% | train_error: 1.32 | train_acc: 0.873 

  [===================>--------------------] 50.90% | train_error: 1.31 | train_acc: 0.873 

  [===================>--------------------] 51.00% | train_error: 1.31 | train_acc: 0.874 

  [===================>--------------------] 51.10% | train_error: 1.31 | train_acc: 0.874 

  [===================>--------------------] 51.20% | train_error: 1.31 | train_acc: 0.874 

  [===================>--------------------] 51.30% | train_error: 1.31 | train_acc: 0.874 

  [===================>--------------------] 51.40% | train_error: 1.31 | train_acc: 0.874 

  [===================>--------------------] 51.50% | train_error: 1.31 | train_acc: 0.874 

  [===================>--------------------] 51.60% | train_error: 1.30 | train_acc: 0.874 

  [===================>--------------------] 51.70% | train_error: 1.30 | train_acc: 0.875 

  [===================>--------------------] 51.80% | train_error: 1.30 | train_acc: 0.875 

  [===================>--------------------] 51.90% | train_error: 1.29 | train_acc: 0.875 

  [===================>--------------------] 52.00% | train_error: 1.29 | train_acc: 0.875 

  [===================>--------------------] 52.10% | train_error: 1.29 | train_acc: 0.875 

  [===================>--------------------] 52.20% | train_error: 1.29 | train_acc: 0.875 

  [===================>--------------------] 52.30% | train_error: 1.29 | train_acc: 0.876 

  [===================>--------------------] 52.40% | train_error: 1.28 | train_acc: 0.876 

  [====================>-------------------] 52.50% | train_error: 1.28 | train_acc: 0.876 

  [====================>-------------------] 52.60% | train_error: 1.28 | train_acc: 0.877 

  [====================>-------------------] 52.70% | train_error: 1.28 | train_acc: 0.877 

  [====================>-------------------] 52.80% | train_error: 1.27 | train_acc: 0.877 

  [====================>-------------------] 52.90% | train_error: 1.27 | train_acc: 0.877 

  [====================>-------------------] 53.00% | train_error: 1.27 | train_acc: 0.877 

  [====================>-------------------] 53.10% | train_error: 1.27 | train_acc: 0.878 

  [====================>-------------------] 53.20% | train_error: 1.26 | train_acc: 0.878 

  [====================>-------------------] 53.30% | train_error: 1.26 | train_acc: 0.878 

  [====================>-------------------] 53.40% | train_error: 1.26 | train_acc: 0.879 

  [====================>-------------------] 53.50% | train_error: 1.25 | train_acc: 0.879 

  [====================>-------------------] 53.60% | train_error: 1.25 | train_acc: 0.880 

  [====================>-------------------] 53.70% | train_error: 1.24 | train_acc: 0.880 

  [====================>-------------------] 53.80% | train_error: 1.24 | train_acc: 0.880 

  [====================>-------------------] 53.90% | train_error: 1.24 | train_acc: 0.880 

  [====================>-------------------] 54.00% | train_error: 1.23 | train_acc: 0.881 

  [====================>-------------------] 54.10% | train_error: 1.22 | train_acc: 0.882 

  [====================>-------------------] 54.20% | train_error: 1.22 | train_acc: 0.882 

  [====================>-------------------] 54.30% | train_error: 1.21 | train_acc: 0.883 

  [====================>-------------------] 54.40% | train_error: 1.21 | train_acc: 0.883 

  [====================>-------------------] 54.50% | train_error: 1.21 | train_acc: 0.883 

  [====================>-------------------] 54.60% | train_error: 1.21 | train_acc: 0.883 

  [====================>-------------------] 54.70% | train_error: 1.21 | train_acc: 0.884 

  [====================>-------------------] 54.80% | train_error: 1.20 | train_acc: 0.884 

  [====================>-------------------] 54.90% | train_error: 1.20 | train_acc: 0.884 

  [=====================>------------------] 55.00% | train_error: 1.20 | train_acc: 0.884 

  [=====================>------------------] 55.10% | train_error: 1.20 | train_acc: 0.884 

  [=====================>------------------] 55.20% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 55.30% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 55.40% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 55.50% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 55.60% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 55.70% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 55.80% | train_error: 1.18 | train_acc: 0.886 

  [=====================>------------------] 55.90% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 56.00% | train_error: 1.19 | train_acc: 0.885 

  [=====================>------------------] 56.10% | train_error: 1.18 | train_acc: 0.886 

  [=====================>------------------] 56.20% | train_error: 1.18 | train_acc: 0.886 

  [=====================>------------------] 56.30% | train_error: 1.17 | train_acc: 0.887 

  [=====================>------------------] 56.40% | train_error: 1.17 | train_acc: 0.887 

  [=====================>------------------] 56.50% | train_error: 1.17 | train_acc: 0.887 

  [=====================>------------------] 56.60% | train_error: 1.16 | train_acc: 0.888 

  [=====================>------------------] 56.70% | train_error: 1.16 | train_acc: 0.888 

  [=====================>------------------] 56.80% | train_error: 1.15 | train_acc: 0.889 

  [=====================>------------------] 56.90% | train_error: 1.15 | train_acc: 0.889 

  [=====================>------------------] 57.00% | train_error: 1.14 | train_acc: 0.890 

  [=====================>------------------] 57.10% | train_error: 1.14 | train_acc: 0.890 

  [=====================>------------------] 57.20% | train_error: 1.14 | train_acc: 0.890 

  [=====================>------------------] 57.30% | train_error: 1.14 | train_acc: 0.890 

  [=====================>------------------] 57.40% | train_error: 1.13 | train_acc: 0.891 

  [======================>-----------------] 57.50% | train_error: 1.13 | train_acc: 0.891 

  [======================>-----------------] 57.60% | train_error: 1.13 | train_acc: 0.891 

  [======================>-----------------] 57.70% | train_error: 1.12 | train_acc: 0.891 

  [======================>-----------------] 57.80% | train_error: 1.13 | train_acc: 0.891 

  [======================>-----------------] 57.90% | train_error: 1.12 | train_acc: 0.892 

  [======================>-----------------] 58.00% | train_error: 1.11 | train_acc: 0.893 

  [======================>-----------------] 58.10% | train_error: 1.11 | train_acc: 0.893 

  [======================>-----------------] 58.20% | train_error: 1.09 | train_acc: 0.894 

  [======================>-----------------] 58.30% | train_error: 1.09 | train_acc: 0.895 

  [======================>-----------------] 58.40% | train_error: 1.09 | train_acc: 0.895 

  [======================>-----------------] 58.50% | train_error: 1.09 | train_acc: 0.895 

  [======================>-----------------] 58.60% | train_error: 1.09 | train_acc: 0.895 

  [======================>-----------------] 58.70% | train_error: 1.09 | train_acc: 0.895 

  [======================>-----------------] 58.80% | train_error: 1.08 | train_acc: 0.895 

  [======================>-----------------] 58.90% | train_error: 1.08 | train_acc: 0.895 

  [======================>-----------------] 59.00% | train_error: 1.08 | train_acc: 0.896 

  [======================>-----------------] 59.10% | train_error: 1.08 | train_acc: 0.896 

  [======================>-----------------] 59.20% | train_error: 1.08 | train_acc: 0.896 

  [======================>-----------------] 59.30% | train_error: 1.07 | train_acc: 0.896 

  [======================>-----------------] 59.40% | train_error: 1.07 | train_acc: 0.897 

  [======================>-----------------] 59.50% | train_error: 1.06 | train_acc: 0.898 

  [======================>-----------------] 59.60% | train_error: 1.05 | train_acc: 0.898 

  [======================>-----------------] 59.70% | train_error: 1.05 | train_acc: 0.899 

  [======================>-----------------] 59.80% | train_error: 1.04 | train_acc: 0.899 

  [======================>-----------------] 59.90% | train_error: 1.04 | train_acc: 0.899 

  [=======================>----------------] 60.00% | train_error: 1.04 | train_acc: 0.899 

  [=======================>----------------] 60.10% | train_error: 1.04 | train_acc: 0.900 

  [=======================>----------------] 60.20% | train_error: 1.03 | train_acc: 0.900 

  [=======================>----------------] 60.30% | train_error: 1.03 | train_acc: 0.901 

  [=======================>----------------] 60.40% | train_error: 1.03 | train_acc: 0.901 

  [=======================>----------------] 60.50% | train_error: 1.02 | train_acc: 0.901 

  [=======================>----------------] 60.60% | train_error: 1.02 | train_acc: 0.902 

  [=======================>----------------] 60.70% | train_error: 1.01 | train_acc: 0.902 

  [=======================>----------------] 60.80% | train_error: 1.01 | train_acc: 0.903 

  [=======================>----------------] 60.90% | train_error: 1.00 | train_acc: 0.903 

  [=======================>----------------] 61.00% | train_error: 1.00 | train_acc: 0.903 

  [=======================>----------------] 61.10% | train_error: 1.00 | train_acc: 0.903 

  [=======================>----------------] 61.20% | train_error: 1.00 | train_acc: 0.903 

  [=======================>----------------] 61.30% | train_error: 0.999 | train_acc: 0.904 

  [=======================>----------------] 61.40% | train_error: 0.998 | train_acc: 0.904 

  [=======================>----------------] 61.50% | train_error: 0.998 | train_acc: 0.904 

  [=======================>----------------] 61.60% | train_error: 0.994 | train_acc: 0.904 

  [=======================>----------------] 61.70% | train_error: 0.994 | train_acc: 0.904 

  [=======================>----------------] 61.80% | train_error: 0.991 | train_acc: 0.904 

  [=======================>----------------] 61.90% | train_error: 0.987 | train_acc: 0.905 

  [=======================>----------------] 62.00% | train_error: 0.986 | train_acc: 0.905 

  [=======================>----------------] 62.10% | train_error: 0.978 | train_acc: 0.906 

  [=======================>----------------] 62.20% | train_error: 0.976 | train_acc: 0.906 

  [=======================>----------------] 62.30% | train_error: 0.976 | train_acc: 0.906 

  [=======================>----------------] 62.40% | train_error: 0.973 | train_acc: 0.906 

  [========================>---------------] 62.50% | train_error: 0.972 | train_acc: 0.906 

  [========================>---------------] 62.60% | train_error: 0.968 | train_acc: 0.907 

  [========================>---------------] 62.70% | train_error: 0.968 | train_acc: 0.907 

  [========================>---------------] 62.80% | train_error: 0.965 | train_acc: 0.907 

  [========================>---------------] 62.90% | train_error: 0.966 | train_acc: 0.907 

  [========================>---------------] 63.00% | train_error: 0.962 | train_acc: 0.907 

  [========================>---------------] 63.10% | train_error: 0.954 | train_acc: 0.908 

  [========================>---------------] 63.20% | train_error: 0.951 | train_acc: 0.908 

  [========================>---------------] 63.30% | train_error: 0.948 | train_acc: 0.909 

  [========================>---------------] 63.40% | train_error: 0.946 | train_acc: 0.909 

  [========================>---------------] 63.50% | train_error: 0.943 | train_acc: 0.909 

  [========================>---------------] 63.60% | train_error: 0.935 | train_acc: 0.910 

  [========================>---------------] 63.70% | train_error: 0.926 | train_acc: 0.911 

  [========================>---------------] 63.80% | train_error: 0.926 | train_acc: 0.911 

  [========================>---------------] 63.90% | train_error: 0.926 | train_acc: 0.911 

  [========================>---------------] 64.00% | train_error: 0.914 | train_acc: 0.912 

  [========================>---------------] 64.10% | train_error: 0.914 | train_acc: 0.912 

  [========================>---------------] 64.20% | train_error: 0.906 | train_acc: 0.913 

  [========================>---------------] 64.30% | train_error: 0.903 | train_acc: 0.913 

  [========================>---------------] 64.40% | train_error: 0.900 | train_acc: 0.913 

  [========================>---------------] 64.50% | train_error: 0.895 | train_acc: 0.914 

  [========================>---------------] 64.60% | train_error: 0.894 | train_acc: 0.914 

  [========================>---------------] 64.70% | train_error: 0.891 | train_acc: 0.914 

  [========================>---------------] 64.80% | train_error: 0.893 | train_acc: 0.914 

  [========================>---------------] 64.90% | train_error: 0.893 | train_acc: 0.914 

  [=========================>--------------] 65.00% | train_error: 0.890 | train_acc: 0.914 

  [=========================>--------------] 65.10% | train_error: 0.889 | train_acc: 0.914 

  [=========================>--------------] 65.20% | train_error: 0.883 | train_acc: 0.915 

  [=========================>--------------] 65.30% | train_error: 0.880 | train_acc: 0.915 

  [=========================>--------------] 65.40% | train_error: 0.878 | train_acc: 0.915 

  [=========================>--------------] 65.50% | train_error: 0.876 | train_acc: 0.915 

  [=========================>--------------] 65.60% | train_error: 0.875 | train_acc: 0.916 

  [=========================>--------------] 65.70% | train_error: 0.874 | train_acc: 0.916 

  [=========================>--------------] 65.80% | train_error: 0.875 | train_acc: 0.916 

  [=========================>--------------] 65.90% | train_error: 0.873 | train_acc: 0.916 

  [=========================>--------------] 66.00% | train_error: 0.871 | train_acc: 0.916 

  [=========================>--------------] 66.10% | train_error: 0.868 | train_acc: 0.916 

  [=========================>--------------] 66.20% | train_error: 0.866 | train_acc: 0.916 

  [=========================>--------------] 66.30% | train_error: 0.864 | train_acc: 0.917 

  [=========================>--------------] 66.40% | train_error: 0.855 | train_acc: 0.918 

  [=========================>--------------] 66.50% | train_error: 0.851 | train_acc: 0.918 

  [=========================>--------------] 66.60% | train_error: 0.850 | train_acc: 0.918 

  [=========================>--------------] 66.70% | train_error: 0.850 | train_acc: 0.918 

  [=========================>--------------] 66.80% | train_error: 0.848 | train_acc: 0.918 

  [=========================>--------------] 66.90% | train_error: 0.844 | train_acc: 0.919 

  [=========================>--------------] 67.00% | train_error: 0.845 | train_acc: 0.918 

  [=========================>--------------] 67.10% | train_error: 0.842 | train_acc: 0.919 

  [=========================>--------------] 67.20% | train_error: 0.841 | train_acc: 0.919 

  [=========================>--------------] 67.30% | train_error: 0.841 | train_acc: 0.919 

  [=========================>--------------] 67.40% | train_error: 0.836 | train_acc: 0.919 

  [==========================>-------------] 67.50% | train_error: 0.821 | train_acc: 0.921 

  [==========================>-------------] 67.60% | train_error: 0.808 | train_acc: 0.922 

  [==========================>-------------] 67.70% | train_error: 0.805 | train_acc: 0.922 

  [==========================>-------------] 67.80% | train_error: 0.800 | train_acc: 0.923 

  [==========================>-------------] 67.90% | train_error: 0.799 | train_acc: 0.923 

  [==========================>-------------] 68.00% | train_error: 0.792 | train_acc: 0.924 

  [==========================>-------------] 68.10% | train_error: 0.791 | train_acc: 0.924 

  [==========================>-------------] 68.20% | train_error: 0.782 | train_acc: 0.925 

  [==========================>-------------] 68.30% | train_error: 0.774 | train_acc: 0.925 

  [==========================>-------------] 68.40% | train_error: 0.766 | train_acc: 0.926 

  [==========================>-------------] 68.50% | train_error: 0.763 | train_acc: 0.926 

  [==========================>-------------] 68.60% | train_error: 0.757 | train_acc: 0.927 

  [==========================>-------------] 68.70% | train_error: 0.753 | train_acc: 0.927 

  [==========================>-------------] 68.80% | train_error: 0.754 | train_acc: 0.927 

  [==========================>-------------] 68.90% | train_error: 0.747 | train_acc: 0.928 

  [==========================>-------------] 69.00% | train_error: 0.740 | train_acc: 0.929 

  [==========================>-------------] 69.10% | train_error: 0.746 | train_acc: 0.928 

  [==========================>-------------] 69.20% | train_error: 0.737 | train_acc: 0.929 

  [==========================>-------------] 69.30% | train_error: 0.744 | train_acc: 0.928 

  [==========================>-------------] 69.40% | train_error: 0.736 | train_acc: 0.929 

  [==========================>-------------] 69.50% | train_error: 0.745 | train_acc: 0.928 

  [==========================>-------------] 69.60% | train_error: 0.737 | train_acc: 0.929 

  [==========================>-------------] 69.70% | train_error: 0.736 | train_acc: 0.929 

  [==========================>-------------] 69.80% | train_error: 0.724 | train_acc: 0.930 

  [==========================>-------------] 69.90% | train_error: 0.722 | train_acc: 0.930 

  [===========================>------------] 70.00% | train_error: 0.718 | train_acc: 0.931 

  [===========================>------------] 70.10% | train_error: 0.718 | train_acc: 0.931 

  [===========================>------------] 70.20% | train_error: 0.717 | train_acc: 0.931 

  [===========================>------------] 70.30% | train_error: 0.712 | train_acc: 0.931 

  [===========================>------------] 70.40% | train_error: 0.713 | train_acc: 0.931 

  [===========================>------------] 70.50% | train_error: 0.710 | train_acc: 0.931 

  [===========================>------------] 70.60% | train_error: 0.708 | train_acc: 0.932 

  [===========================>------------] 70.70% | train_error: 0.705 | train_acc: 0.932 

  [===========================>------------] 70.80% | train_error: 0.702 | train_acc: 0.932 

  [===========================>------------] 70.90% | train_error: 0.701 | train_acc: 0.932 

  [===========================>------------] 71.00% | train_error: 0.695 | train_acc: 0.933 

  [===========================>------------] 71.10% | train_error: 0.694 | train_acc: 0.933 

  [===========================>------------] 71.20% | train_error: 0.691 | train_acc: 0.933 

  [===========================>------------] 71.30% | train_error: 0.687 | train_acc: 0.934 

  [===========================>------------] 71.40% | train_error: 0.683 | train_acc: 0.934 

  [===========================>------------] 71.50% | train_error: 0.682 | train_acc: 0.934 

  [===========================>------------] 71.60% | train_error: 0.677 | train_acc: 0.935 

  [===========================>------------] 71.70% | train_error: 0.672 | train_acc: 0.935 

  [===========================>------------] 71.80% | train_error: 0.669 | train_acc: 0.935 

  [===========================>------------] 71.90% | train_error: 0.669 | train_acc: 0.935 

  [===========================>------------] 72.00% | train_error: 0.668 | train_acc: 0.936 

  [===========================>------------] 72.10% | train_error: 0.665 | train_acc: 0.936 

  [===========================>------------] 72.20% | train_error: 0.658 | train_acc: 0.936 

  [===========================>------------] 72.30% | train_error: 0.655 | train_acc: 0.937 

  [===========================>------------] 72.40% | train_error: 0.654 | train_acc: 0.937 

  [============================>-----------] 72.50% | train_error: 0.657 | train_acc: 0.937 

  [============================>-----------] 72.60% | train_error: 0.652 | train_acc: 0.937 

  [============================>-----------] 72.70% | train_error: 0.647 | train_acc: 0.938 

  [============================>-----------] 72.80% | train_error: 0.645 | train_acc: 0.938 

  [============================>-----------] 72.90% | train_error: 0.645 | train_acc: 0.938 

  [============================>-----------] 73.00% | train_error: 0.630 | train_acc: 0.939 

  [============================>-----------] 73.10% | train_error: 0.637 | train_acc: 0.939 

  [============================>-----------] 73.20% | train_error: 0.624 | train_acc: 0.940 

  [============================>-----------] 73.30% | train_error: 0.630 | train_acc: 0.939 

  [============================>-----------] 73.40% | train_error: 0.618 | train_acc: 0.940 

  [============================>-----------] 73.50% | train_error: 0.623 | train_acc: 0.940 

  [============================>-----------] 73.60% | train_error: 0.611 | train_acc: 0.941 

  [============================>-----------] 73.70% | train_error: 0.612 | train_acc: 0.941 

  [============================>-----------] 73.80% | train_error: 0.614 | train_acc: 0.941 

  [============================>-----------] 73.90% | train_error: 0.608 | train_acc: 0.941 

  [============================>-----------] 74.00% | train_error: 0.605 | train_acc: 0.942 

  [============================>-----------] 74.10% | train_error: 0.609 | train_acc: 0.941 

  [============================>-----------] 74.20% | train_error: 0.604 | train_acc: 0.942 

  [============================>-----------] 74.30% | train_error: 0.601 | train_acc: 0.942 

  [============================>-----------] 74.40% | train_error: 0.600 | train_acc: 0.942 

  [============================>-----------] 74.50% | train_error: 0.601 | train_acc: 0.942 

  [============================>-----------] 74.60% | train_error: 0.599 | train_acc: 0.942 

  [============================>-----------] 74.70% | train_error: 0.593 | train_acc: 0.943 

  [============================>-----------] 74.80% | train_error: 0.592 | train_acc: 0.943 

  [============================>-----------] 74.90% | train_error: 0.593 | train_acc: 0.943 

  [=============================>----------] 75.00% | train_error: 0.589 | train_acc: 0.943 

  [=============================>----------] 75.10% | train_error: 0.590 | train_acc: 0.943 

  [=============================>----------] 75.20% | train_error: 0.585 | train_acc: 0.944 

  [=============================>----------] 75.30% | train_error: 0.590 | train_acc: 0.943 

  [=============================>----------] 75.40% | train_error: 0.580 | train_acc: 0.944 

  [=============================>----------] 75.50% | train_error: 0.579 | train_acc: 0.944 

  [=============================>----------] 75.60% | train_error: 0.574 | train_acc: 0.945 

  [=============================>----------] 75.70% | train_error: 0.578 | train_acc: 0.944 

  [=============================>----------] 75.80% | train_error: 0.560 | train_acc: 0.946 

  [=============================>----------] 75.90% | train_error: 0.566 | train_acc: 0.945 

  [=============================>----------] 76.00% | train_error: 0.564 | train_acc: 0.946 

  [=============================>----------] 76.10% | train_error: 0.563 | train_acc: 0.946 

  [=============================>----------] 76.20% | train_error: 0.559 | train_acc: 0.946 

  [=============================>----------] 76.30% | train_error: 0.560 | train_acc: 0.946 

  [=============================>----------] 76.40% | train_error: 0.549 | train_acc: 0.947 

  [=============================>----------] 76.50% | train_error: 0.563 | train_acc: 0.946 

  [=============================>----------] 76.60% | train_error: 0.542 | train_acc: 0.948 

  [=============================>----------] 76.70% | train_error: 0.558 | train_acc: 0.946 

  [=============================>----------] 76.80% | train_error: 0.542 | train_acc: 0.948 

  [=============================>----------] 76.90% | train_error: 0.556 | train_acc: 0.946 

  [=============================>----------] 77.00% | train_error: 0.536 | train_acc: 0.948 

  [=============================>----------] 77.10% | train_error: 0.549 | train_acc: 0.947 

  [=============================>----------] 77.20% | train_error: 0.529 | train_acc: 0.949 

  [=============================>----------] 77.30% | train_error: 0.536 | train_acc: 0.948 

  [=============================>----------] 77.40% | train_error: 0.534 | train_acc: 0.949 

  [==============================>---------] 77.50% | train_error: 0.532 | train_acc: 0.949 

  [==============================>---------] 77.60% | train_error: 0.528 | train_acc: 0.949 

  [==============================>---------] 77.70% | train_error: 0.526 | train_acc: 0.949 

  [==============================>---------] 77.80% | train_error: 0.515 | train_acc: 0.950 

  [==============================>---------] 77.90% | train_error: 0.517 | train_acc: 0.950 

  [==============================>---------] 78.00% | train_error: 0.510 | train_acc: 0.951 

  [==============================>---------] 78.10% | train_error: 0.511 | train_acc: 0.951 

  [==============================>---------] 78.20% | train_error: 0.510 | train_acc: 0.951 

  [==============================>---------] 78.30% | train_error: 0.504 | train_acc: 0.951 

  [==============================>---------] 78.40% | train_error: 0.510 | train_acc: 0.951 

  [==============================>---------] 78.50% | train_error: 0.507 | train_acc: 0.951 

  [==============================>---------] 78.60% | train_error: 0.509 | train_acc: 0.951 

  [==============================>---------] 78.70% | train_error: 0.499 | train_acc: 0.952 

  [==============================>---------] 78.80% | train_error: 0.506 | train_acc: 0.951 

  [==============================>---------] 78.90% | train_error: 0.499 | train_acc: 0.952 

  [==============================>---------] 79.00% | train_error: 0.504 | train_acc: 0.951 

  [==============================>---------] 79.10% | train_error: 0.503 | train_acc: 0.952 

  [==============================>---------] 79.20% | train_error: 0.495 | train_acc: 0.952 

  [==============================>---------] 79.30% | train_error: 0.495 | train_acc: 0.952 

  [==============================>---------] 79.40% | train_error: 0.488 | train_acc: 0.953 

  [==============================>---------] 79.50% | train_error: 0.482 | train_acc: 0.953 

  [==============================>---------] 79.60% | train_error: 0.473 | train_acc: 0.954 

  [==============================>---------] 79.70% | train_error: 0.476 | train_acc: 0.954 

  [==============================>---------] 79.80% | train_error: 0.477 | train_acc: 0.954 

  [==============================>---------] 79.90% | train_error: 0.468 | train_acc: 0.955 

  [===============================>--------] 80.00% | train_error: 0.472 | train_acc: 0.954 

  [===============================>--------] 80.10% | train_error: 0.466 | train_acc: 0.955 

  [===============================>--------] 80.20% | train_error: 0.473 | train_acc: 0.954 

  [===============================>--------] 80.30% | train_error: 0.464 | train_acc: 0.955 

  [===============================>--------] 80.40% | train_error: 0.467 | train_acc: 0.955 

  [===============================>--------] 80.50% | train_error: 0.458 | train_acc: 0.956 

  [===============================>--------] 80.60% | train_error: 0.461 | train_acc: 0.955 

  [===============================>--------] 80.70% | train_error: 0.449 | train_acc: 0.957 

  [===============================>--------] 80.80% | train_error: 0.464 | train_acc: 0.955 

  [===============================>--------] 80.90% | train_error: 0.446 | train_acc: 0.957 

  [===============================>--------] 81.00% | train_error: 0.456 | train_acc: 0.956 

  [===============================>--------] 81.10% | train_error: 0.449 | train_acc: 0.957 

  [===============================>--------] 81.20% | train_error: 0.454 | train_acc: 0.956 

  [===============================>--------] 81.30% | train_error: 0.446 | train_acc: 0.957 

  [===============================>--------] 81.40% | train_error: 0.443 | train_acc: 0.957 

  [===============================>--------] 81.50% | train_error: 0.443 | train_acc: 0.957 

  [===============================>--------] 81.60% | train_error: 0.429 | train_acc: 0.959 

  [===============================>--------] 81.70% | train_error: 0.434 | train_acc: 0.958 

  [===============================>--------] 81.80% | train_error: 0.427 | train_acc: 0.959 

  [===============================>--------] 81.90% | train_error: 0.422 | train_acc: 0.959 

  [===============================>--------] 82.00% | train_error: 0.419 | train_acc: 0.960 

  [===============================>--------] 82.10% | train_error: 0.424 | train_acc: 0.959 

  [===============================>--------] 82.20% | train_error: 0.424 | train_acc: 0.959 

  [===============================>--------] 82.30% | train_error: 0.422 | train_acc: 0.959 

  [===============================>--------] 82.40% | train_error: 0.417 | train_acc: 0.960 

  [================================>-------] 82.50% | train_error: 0.413 | train_acc: 0.960 

  [================================>-------] 82.60% | train_error: 0.408 | train_acc: 0.961 

  [================================>-------] 82.70% | train_error: 0.401 | train_acc: 0.961 

  [================================>-------] 82.80% | train_error: 0.402 | train_acc: 0.961 

  [================================>-------] 82.90% | train_error: 0.396 | train_acc: 0.962 

  [================================>-------] 83.00% | train_error: 0.402 | train_acc: 0.961 

  [================================>-------] 83.10% | train_error: 0.399 | train_acc: 0.962 

  [================================>-------] 83.20% | train_error: 0.401 | train_acc: 0.961 

  [================================>-------] 83.30% | train_error: 0.389 | train_acc: 0.962 

  [================================>-------] 83.40% | train_error: 0.397 | train_acc: 0.962 

  [================================>-------] 83.50% | train_error: 0.386 | train_acc: 0.963 

  [================================>-------] 83.60% | train_error: 0.389 | train_acc: 0.963 

  [================================>-------] 83.70% | train_error: 0.386 | train_acc: 0.963 

  [================================>-------] 83.80% | train_error: 0.385 | train_acc: 0.963 

  [================================>-------] 83.90% | train_error: 0.385 | train_acc: 0.963 

  [================================>-------] 84.00% | train_error: 0.382 | train_acc: 0.963 

  [================================>-------] 84.10% | train_error: 0.378 | train_acc: 0.963 

  [================================>-------] 84.20% | train_error: 0.374 | train_acc: 0.964 

  [================================>-------] 84.30% | train_error: 0.372 | train_acc: 0.964 

  [================================>-------] 84.40% | train_error: 0.371 | train_acc: 0.964 

  [================================>-------] 84.50% | train_error: 0.369 | train_acc: 0.964 

  [================================>-------] 84.60% | train_error: 0.364 | train_acc: 0.965 

  [================================>-------] 84.70% | train_error: 0.372 | train_acc: 0.964 

  [================================>-------] 84.80% | train_error: 0.368 | train_acc: 0.964 

  [================================>-------] 84.90% | train_error: 0.362 | train_acc: 0.965 

  [=================================>------] 85.00% | train_error: 0.364 | train_acc: 0.965 

  [=================================>------] 85.10% | train_error: 0.355 | train_acc: 0.966 

  [=================================>------] 85.20% | train_error: 0.356 | train_acc: 0.966 

  [=================================>------] 85.30% | train_error: 0.349 | train_acc: 0.966 

  [=================================>------] 85.40% | train_error: 0.341 | train_acc: 0.967 

  [=================================>------] 85.50% | train_error: 0.347 | train_acc: 0.966 

  [=================================>------] 85.60% | train_error: 0.349 | train_acc: 0.966 

  [=================================>------] 85.70% | train_error: 0.345 | train_acc: 0.967 

  [=================================>------] 85.80% | train_error: 0.346 | train_acc: 0.967 

  [=================================>------] 85.90% | train_error: 0.338 | train_acc: 0.968 

  [=================================>------] 86.00% | train_error: 0.348 | train_acc: 0.966 

  [=================================>------] 86.10% | train_error: 0.344 | train_acc: 0.967 

  [=================================>------] 86.20% | train_error: 0.346 | train_acc: 0.967 

  [=================================>------] 86.30% | train_error: 0.340 | train_acc: 0.967 

  [=================================>------] 86.40% | train_error: 0.339 | train_acc: 0.967 

  [=================================>------] 86.50% | train_error: 0.336 | train_acc: 0.968 

  [=================================>------] 86.60% | train_error: 0.343 | train_acc: 0.967 

  [=================================>------] 86.70% | train_error: 0.336 | train_acc: 0.968 

  [=================================>------] 86.80% | train_error: 0.339 | train_acc: 0.967 

  [=================================>------] 86.90% | train_error: 0.330 | train_acc: 0.968 

  [=================================>------] 87.00% | train_error: 0.338 | train_acc: 0.967 

  [=================================>------] 87.10% | train_error: 0.328 | train_acc: 0.969 

  [=================================>------] 87.20% | train_error: 0.326 | train_acc: 0.969 

  [=================================>------] 87.30% | train_error: 0.317 | train_acc: 0.969 

  [=================================>------] 87.40% | train_error: 0.329 | train_acc: 0.968 

  [==================================>-----] 87.50% | train_error: 0.317 | train_acc: 0.969 

  [==================================>-----] 87.60% | train_error: 0.316 | train_acc: 0.970 

  [==================================>-----] 87.70% | train_error: 0.318 | train_acc: 0.969 

  [==================================>-----] 87.80% | train_error: 0.315 | train_acc: 0.970 

  [==================================>-----] 87.90% | train_error: 0.309 | train_acc: 0.970 

  [==================================>-----] 88.00% | train_error: 0.308 | train_acc: 0.970 

  [==================================>-----] 88.10% | train_error: 0.296 | train_acc: 0.972 

  [==================================>-----] 88.20% | train_error: 0.302 | train_acc: 0.971 

  [==================================>-----] 88.30% | train_error: 0.298 | train_acc: 0.971 

  [==================================>-----] 88.40% | train_error: 0.300 | train_acc: 0.971 

  [==================================>-----] 88.50% | train_error: 0.296 | train_acc: 0.971 

  [==================================>-----] 88.60% | train_error: 0.291 | train_acc: 0.972 

  [==================================>-----] 88.70% | train_error: 0.287 | train_acc: 0.972 

  [==================================>-----] 88.80% | train_error: 0.283 | train_acc: 0.973 

  [==================================>-----] 88.90% | train_error: 0.280 | train_acc: 0.973 

  [==================================>-----] 89.00% | train_error: 0.285 | train_acc: 0.973 

  [==================================>-----] 89.10% | train_error: 0.277 | train_acc: 0.973 

  [==================================>-----] 89.20% | train_error: 0.292 | train_acc: 0.972 

  [==================================>-----] 89.30% | train_error: 0.289 | train_acc: 0.972 

  [==================================>-----] 89.40% | train_error: 0.292 | train_acc: 0.972 

  [==================================>-----] 89.50% | train_error: 0.287 | train_acc: 0.972 

  [==================================>-----] 89.60% | train_error: 0.285 | train_acc: 0.973 

  [==================================>-----] 89.70% | train_error: 0.280 | train_acc: 0.973 

  [==================================>-----] 89.80% | train_error: 0.283 | train_acc: 0.973 

  [==================================>-----] 89.90% | train_error: 0.277 | train_acc: 0.973 

  [===================================>----] 90.00% | train_error: 0.285 | train_acc: 0.973 

  [===================================>----] 90.10% | train_error: 0.278 | train_acc: 0.973 

  [===================================>----] 90.20% | train_error: 0.266 | train_acc: 0.974 

  [===================================>----] 90.30% | train_error: 0.264 | train_acc: 0.975 

  [===================================>----] 90.40% | train_error: 0.271 | train_acc: 0.974 

  [===================================>----] 90.50% | train_error: 0.265 | train_acc: 0.974 

  [===================================>----] 90.60% | train_error: 0.265 | train_acc: 0.974 

  [===================================>----] 90.70% | train_error: 0.258 | train_acc: 0.975 

  [===================================>----] 90.80% | train_error: 0.263 | train_acc: 0.975 

  [===================================>----] 90.90% | train_error: 0.251 | train_acc: 0.976 

  [===================================>----] 91.00% | train_error: 0.248 | train_acc: 0.976 

  [===================================>----] 91.10% | train_error: 0.248 | train_acc: 0.976 

  [===================================>----] 91.20% | train_error: 0.250 | train_acc: 0.976 

  [===================================>----] 91.30% | train_error: 0.243 | train_acc: 0.977 

  [===================================>----] 91.40% | train_error: 0.241 | train_acc: 0.977 

  [===================================>----] 91.50% | train_error: 0.239 | train_acc: 0.977 

  [===================================>----] 91.60% | train_error: 0.240 | train_acc: 0.977 

  [===================================>----] 91.70% | train_error: 0.239 | train_acc: 0.977 

  [===================================>----] 91.80% | train_error: 0.239 | train_acc: 0.977 

  [===================================>----] 91.90% | train_error: 0.235 | train_acc: 0.977 

  [===================================>----] 92.00% | train_error: 0.233 | train_acc: 0.978 

  [===================================>----] 92.10% | train_error: 0.234 | train_acc: 0.977 

  [===================================>----] 92.20% | train_error: 0.240 | train_acc: 0.977 

  [===================================>----] 92.30% | train_error: 0.238 | train_acc: 0.977 

  [===================================>----] 92.40% | train_error: 0.226 | train_acc: 0.978 

  [====================================>---] 92.50% | train_error: 0.226 | train_acc: 0.978 

  [====================================>---] 92.60% | train_error: 0.229 | train_acc: 0.978 

  [====================================>---] 92.70% | train_error: 0.226 | train_acc: 0.978 

  [====================================>---] 92.80% | train_error: 0.218 | train_acc: 0.979 

  [====================================>---] 92.90% | train_error: 0.219 | train_acc: 0.979 

  [====================================>---] 93.00% | train_error: 0.220 | train_acc: 0.979 

  [====================================>---] 93.10% | train_error: 0.216 | train_acc: 0.979 

  [====================================>---] 93.20% | train_error: 0.217 | train_acc: 0.979 

  [====================================>---] 93.30% | train_error: 0.216 | train_acc: 0.979 

  [====================================>---] 93.40% | train_error: 0.216 | train_acc: 0.979 

  [====================================>---] 93.50% | train_error: 0.213 | train_acc: 0.979 

  [====================================>---] 93.60% | train_error: 0.213 | train_acc: 0.979 

  [====================================>---] 93.70% | train_error: 0.213 | train_acc: 0.979 

  [====================================>---] 93.80% | train_error: 0.205 | train_acc: 0.980 

  [====================================>---] 93.90% | train_error: 0.210 | train_acc: 0.980 

  [====================================>---] 94.00% | train_error: 0.211 | train_acc: 0.980 

  [====================================>---] 94.10% | train_error: 0.209 | train_acc: 0.980 

  [====================================>---] 94.20% | train_error: 0.206 | train_acc: 0.980 

  [====================================>---] 94.30% | train_error: 0.204 | train_acc: 0.980 

  [====================================>---] 94.40% | train_error: 0.210 | train_acc: 0.980 

  [====================================>---] 94.50% | train_error: 0.198 | train_acc: 0.981 

  [====================================>---] 94.60% | train_error: 0.198 | train_acc: 0.981 

  [====================================>---] 94.70% | train_error: 0.202 | train_acc: 0.981 

  [====================================>---] 94.80% | train_error: 0.201 | train_acc: 0.981 

  [====================================>---] 94.90% | train_error: 0.202 | train_acc: 0.981 

  [=====================================>--] 95.00% | train_error: 0.195 | train_acc: 0.981 

  [=====================================>--] 95.10% | train_error: 0.203 | train_acc: 0.980 

  [=====================================>--] 95.20% | train_error: 0.197 | train_acc: 0.981 

  [=====================================>--] 95.30% | train_error: 0.201 | train_acc: 0.981 

  [=====================================>--] 95.40% | train_error: 0.187 | train_acc: 0.982 

  [=====================================>--] 95.50% | train_error: 0.197 | train_acc: 0.981 

  [=====================================>--] 95.60% | train_error: 0.187 | train_acc: 0.982 

  [=====================================>--] 95.70% | train_error: 0.195 | train_acc: 0.981 

  [=====================================>--] 95.80% | train_error: 0.195 | train_acc: 0.981 

  [=====================================>--] 95.90% | train_error: 0.190 | train_acc: 0.982 

  [=====================================>--] 96.00% | train_error: 0.190 | train_acc: 0.982 

  [=====================================>--] 96.10% | train_error: 0.194 | train_acc: 0.981 

  [=====================================>--] 96.20% | train_error: 0.187 | train_acc: 0.982 

  [=====================================>--] 96.30% | train_error: 0.190 | train_acc: 0.982 

  [=====================================>--] 96.40% | train_error: 0.190 | train_acc: 0.982 

  [=====================================>--] 96.50% | train_error: 0.195 | train_acc: 0.981 

  [=====================================>--] 96.60% | train_error: 0.188 | train_acc: 0.982 

  [=====================================>--] 96.70% | train_error: 0.188 | train_acc: 0.982 

  [=====================================>--] 96.80% | train_error: 0.190 | train_acc: 0.982 

  [=====================================>--] 96.90% | train_error: 0.191 | train_acc: 0.982 

  [=====================================>--] 97.00% | train_error: 0.188 | train_acc: 0.982 

  [=====================================>--] 97.10% | train_error: 0.189 | train_acc: 0.982 

  [=====================================>--] 97.20% | train_error: 0.175 | train_acc: 0.983 

  [=====================================>--] 97.30% | train_error: 0.189 | train_acc: 0.982 

  [=====================================>--] 97.40% | train_error: 0.183 | train_acc: 0.982 

  [======================================>-] 97.50% | train_error: 0.189 | train_acc: 0.982 

  [======================================>-] 97.60% | train_error: 0.181 | train_acc: 0.983 

  [======================================>-] 97.70% | train_error: 0.188 | train_acc: 0.982 

  [======================================>-] 97.80% | train_error: 0.179 | train_acc: 0.983 

  [======================================>-] 97.90% | train_error: 0.190 | train_acc: 0.982 

  [======================================>-] 98.00% | train_error: 0.186 | train_acc: 0.982 

  [======================================>-] 98.10% | train_error: 0.185 | train_acc: 0.982 

  [======================================>-] 98.20% | train_error: 0.182 | train_acc: 0.982 

  [======================================>-] 98.30% | train_error: 0.182 | train_acc: 0.982 

  [======================================>-] 98.40% | train_error: 0.171 | train_acc: 0.984 

  [======================================>-] 98.50% | train_error: 0.178 | train_acc: 0.983 

  [======================================>-] 98.60% | train_error: 0.170 | train_acc: 0.984 

  [======================================>-] 98.70% | train_error: 0.181 | train_acc: 0.983 

  [======================================>-] 98.80% | train_error: 0.166 | train_acc: 0.984 

  [======================================>-] 98.90% | train_error: 0.175 | train_acc: 0.983 

  [======================================>-] 99.00% | train_error: 0.173 | train_acc: 0.983 

  [======================================>-] 99.10% | train_error: 0.173 | train_acc: 0.983 

  [======================================>-] 99.20% | train_error: 0.171 | train_acc: 0.984 

  [======================================>-] 99.30% | train_error: 0.168 | train_acc: 0.984 

  [======================================>-] 99.40% | train_error: 0.167 | train_acc: 0.984 

  [======================================>-] 99.50% | train_error: 0.174 | train_acc: 0.983 

  [======================================>-] 99.60% | train_error: 0.158 | train_acc: 0.985 

  [======================================>-] 99.70% | train_error: 0.173 | train_acc: 0.983 

  [======================================>-] 99.80% | train_error: 0.164 | train_acc: 0.984 

  [======================================>-] 99.90% | train_error: 0.175 | train_acc: 0.983 

  [=======================================>] 100.0% | train_error: 0.175 | train_acc: 0.983 

Testing the XOR gate and other gates¶

Let us now use our code to test the XOR gate.

X = np.array([ [0, 0], [0, 1], [1, 0],[1, 1]],dtype=np.float64)

# The XOR gate
yXOR = np.array( [[ 0], [1] ,[1], [0]])

input_nodes = X.shape[1]
output_nodes = 1

logistic_regression = FFNN((input_nodes, output_nodes), output_func=sigmoid, cost_func=CostLogReg, seed=2023)
logistic_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights
scheduler = Adam(eta=1e-1, rho=0.9, rho2=0.999)
scores = logistic_regression.fit(X, yXOR, scheduler, epochs=1000)

Adam: Eta=0.1, Lambda=0

  [----------------------------------------] 0.000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.1000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.2000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.3000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.4000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.5000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.6000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.7000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.8000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 0.9000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.100% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.200% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.300% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.400% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.500% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.600% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.700% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.800% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 1.900% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 2.000% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 2.100% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 2.200% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 2.300% | train_error: 10.4 | train_acc: 0.500 

  [----------------------------------------] 2.400% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 2.500% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 2.600% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 2.700% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 2.800% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 2.900% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.000% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.100% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.200% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.300% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.400% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.500% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.600% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.700% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.800% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 3.900% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.000% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.100% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.200% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.300% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.400% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.500% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.600% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.700% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.800% | train_error: 10.4 | train_acc: 0.500 

  [>---------------------------------------] 4.900% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.000% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.100% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.200% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.300% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.400% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.500% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.600% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.700% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.800% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 5.900% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.000% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.100% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.200% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.300% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.400% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.500% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.600% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.700% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.800% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 6.900% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 7.000% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 7.100% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 7.200% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 7.300% | train_error: 10.4 | train_acc: 0.500 

  [=>--------------------------------------] 7.400% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 7.500% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 7.600% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 7.700% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 7.800% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 7.900% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.000% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.100% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.200% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.300% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.400% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.500% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.600% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.700% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.800% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 8.900% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.000% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.100% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.200% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.300% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.400% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.500% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.600% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.700% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.800% | train_error: 10.4 | train_acc: 0.500 

  [==>-------------------------------------] 9.900% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.00% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.10% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.20% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.30% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.40% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.50% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.60% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.70% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.80% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 10.90% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.00% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.10% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.20% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.30% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.40% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.50% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.60% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.70% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.80% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 11.90% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 12.00% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 12.10% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 12.20% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 12.30% | train_error: 10.4 | train_acc: 0.500 

  [===>------------------------------------] 12.40% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 12.50% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 12.60% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 12.70% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 12.80% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 12.90% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.00% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.10% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.20% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.30% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.40% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.50% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.60% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.70% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.80% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 13.90% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.00% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.10% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.20% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.30% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.40% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.50% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.60% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.70% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.80% | train_error: 10.4 | train_acc: 0.500 

  [====>-----------------------------------] 14.90% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.00% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.10% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.20% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.30% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.40% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.50% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.60% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.70% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.80% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 15.90% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.00% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.10% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.20% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.30% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.40% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.50% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.60% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.70% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.80% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 16.90% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 17.00% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 17.10% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 17.20% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 17.30% | train_error: 10.4 | train_acc: 0.500 

  [=====>----------------------------------] 17.40% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 17.50% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 17.60% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 17.70% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 17.80% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 17.90% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.00% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.10% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.20% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.30% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.40% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.50% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.60% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.70% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.80% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 18.90% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.00% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.10% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.20% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.30% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.40% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.50% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.60% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.70% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.80% | train_error: 10.4 | train_acc: 0.500 

  [======>---------------------------------] 19.90% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.00% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.10% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.20% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.30% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.40% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.50% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.60% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.70% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.80% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 20.90% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.00% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.10% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.20% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.30% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.40% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.50% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.60% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.70% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.80% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 21.90% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 22.00% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 22.10% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 22.20% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 22.30% | train_error: 10.4 | train_acc: 0.500 

  [=======>--------------------------------] 22.40% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 22.50% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 22.60% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 22.70% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 22.80% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 22.90% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.00% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.10% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.20% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.30% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.40% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.50% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.60% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.70% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.80% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 23.90% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.00% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.10% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.20% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.30% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.40% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.50% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.60% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.70% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.80% | train_error: 10.4 | train_acc: 0.500 

  [========>-------------------------------] 24.90% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.00% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.10% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.20% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.30% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.40% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.50% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.60% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.70% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.80% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 25.90% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.00% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.10% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.20% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.30% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.40% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.50% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.60% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.70% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.80% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 26.90% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 27.00% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 27.10% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 27.20% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 27.30% | train_error: 10.4 | train_acc: 0.500 

  [=========>------------------------------] 27.40% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 27.50% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 27.60% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 27.70% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 27.80% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 27.90% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.00% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.10% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.20% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.30% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.40% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.50% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.60% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.70% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.80% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 28.90% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.00% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.10% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.20% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.30% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.40% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.50% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.60% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.70% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.80% | train_error: 10.4 | train_acc: 0.500 

  [==========>-----------------------------] 29.90% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.00% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.10% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.20% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.30% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.40% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.50% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.60% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.70% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.80% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 30.90% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.00% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.10% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.20% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.30% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.40% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.50% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.60% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.70% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.80% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 31.90% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 32.00% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 32.10% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 32.20% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 32.30% | train_error: 10.4 | train_acc: 0.500 

  [===========>----------------------------] 32.40% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 32.50% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 32.60% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 32.70% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 32.80% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 32.90% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.00% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.10% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.20% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.30% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.40% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.50% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.60% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.70% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.80% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 33.90% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.00% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.10% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.20% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.30% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.40% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.50% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.60% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.70% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.80% | train_error: 10.4 | train_acc: 0.500 

  [============>---------------------------] 34.90% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.00% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.10% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.20% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.30% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.40% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.50% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.60% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.70% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.80% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 35.90% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.00% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.10% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.20% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.30% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.40% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.50% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.60% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.70% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.80% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 36.90% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 37.00% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 37.10% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 37.20% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 37.30% | train_error: 10.4 | train_acc: 0.500 

  [=============>--------------------------] 37.40% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 37.50% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 37.60% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 37.70% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 37.80% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 37.90% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.00% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.10% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.20% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.30% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.40% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.50% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.60% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.70% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.80% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 38.90% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.00% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.10% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.20% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.30% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.40% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.50% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.60% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.70% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.80% | train_error: 10.4 | train_acc: 0.500 

  [==============>-------------------------] 39.90% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.00% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.10% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.20% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.30% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.40% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.50% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.60% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.70% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.80% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 40.90% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.00% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.10% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.20% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.30% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.40% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.50% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.60% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.70% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.80% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 41.90% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 42.00% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 42.10% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 42.20% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 42.30% | train_error: 10.4 | train_acc: 0.500 

  [===============>------------------------] 42.40% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 42.50% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 42.60% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 42.70% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 42.80% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 42.90% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.00% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.10% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.20% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.30% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.40% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.50% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.60% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.70% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.80% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 43.90% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.00% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.10% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.20% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.30% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.40% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.50% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.60% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.70% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.80% | train_error: 10.4 | train_acc: 0.500 

  [================>-----------------------] 44.90% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.00% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.10% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.20% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.30% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.40% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.50% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.60% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.70% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.80% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 45.90% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.00% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.10% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.20% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.30% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.40% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.50% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.60% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.70% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.80% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 46.90% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 47.00% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 47.10% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 47.20% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 47.30% | train_error: 10.4 | train_acc: 0.500 

  [=================>----------------------] 47.40% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 47.50% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 47.60% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 47.70% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 47.80% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 47.90% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.00% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.10% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.20% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.30% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.40% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.50% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.60% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.70% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.80% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 48.90% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.00% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.10% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.20% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.30% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.40% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.50% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.60% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.70% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.80% | train_error: 10.4 | train_acc: 0.500 

  [==================>---------------------] 49.90% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 50.00% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 50.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 50.20% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 50.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 50.40% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 50.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 50.60% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 50.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 50.80% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 50.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 51.00% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 51.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 51.20% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 51.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 51.40% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 51.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 51.60% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 51.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 51.80% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 51.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 52.00% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 52.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 52.20% | train_error: 10.4 | train_acc: 0.500 

  [===================>--------------------] 52.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================>--------------------] 52.40% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 52.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 52.60% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 52.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 52.80% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 52.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 53.00% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 53.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 53.20% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 53.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 53.40% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 53.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 53.60% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 53.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 53.80% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 53.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 54.00% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 54.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 54.20% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 54.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 54.40% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 54.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 54.60% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 54.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================>-------------------] 54.80% | train_error: 10.4 | train_acc: 0.500 

  [====================>-------------------] 54.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 55.00% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 55.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 55.20% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 55.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 55.40% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 55.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 55.60% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 55.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 55.80% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 55.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 56.00% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 56.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 56.20% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 56.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 56.40% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 56.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 56.60% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 56.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 56.80% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 56.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 57.00% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 57.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 57.20% | train_error: 10.4 | train_acc: 0.500 

  [=====================>------------------] 57.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================>------------------] 57.40% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 57.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 57.60% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 57.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 57.80% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 57.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 58.00% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 58.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 58.20% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 58.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 58.40% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 58.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 58.60% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 58.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 58.80% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 58.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 59.00% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 59.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 59.20% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 59.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 59.40% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 59.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 59.60% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 59.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================>-----------------] 59.80% | train_error: 10.4 | train_acc: 0.500 

  [======================>-----------------] 59.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 60.00% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 60.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 60.20% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 60.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 60.40% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 60.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 60.60% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 60.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 60.80% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 60.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 61.00% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 61.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 61.20% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 61.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 61.40% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 61.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 61.60% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 61.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 61.80% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 61.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 62.00% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 62.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 62.20% | train_error: 10.4 | train_acc: 0.500 

  [=======================>----------------] 62.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================>----------------] 62.40% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 62.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 62.60% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 62.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 62.80% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 62.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 63.00% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 63.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 63.20% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 63.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 63.40% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 63.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 63.60% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 63.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 63.80% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 63.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 64.00% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 64.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 64.20% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 64.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 64.40% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 64.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 64.60% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 64.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [========================>---------------] 64.80% | train_error: 10.4 | train_acc: 0.500 

  [========================>---------------] 64.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 65.00% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 65.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 65.20% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 65.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 65.40% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 65.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 65.60% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 65.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 65.80% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 65.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 66.00% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 66.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 66.20% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 66.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 66.40% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 66.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 66.60% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 66.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 66.80% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 66.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 67.00% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 67.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 67.20% | train_error: 10.4 | train_acc: 0.500 

  [=========================>--------------] 67.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=========================>--------------] 67.40% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 67.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 67.60% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 67.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 67.80% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 67.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 68.00% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 68.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 68.20% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 68.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 68.40% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 68.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 68.60% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 68.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 68.80% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 68.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 69.00% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 69.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 69.20% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 69.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 69.40% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 69.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 69.60% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 69.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==========================>-------------] 69.80% | train_error: 10.4 | train_acc: 0.500 

  [==========================>-------------] 69.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 70.00% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 70.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 70.20% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 70.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 70.40% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 70.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 70.60% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 70.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 70.80% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 70.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 71.00% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 71.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 71.20% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 71.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 71.40% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 71.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 71.60% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 71.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 71.80% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 71.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 72.00% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 72.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 72.20% | train_error: 10.4 | train_acc: 0.500 

  [===========================>------------] 72.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===========================>------------] 72.40% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 72.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 72.60% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 72.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 72.80% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 72.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 73.00% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 73.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 73.20% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 73.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 73.40% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 73.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 73.60% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 73.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 73.80% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 73.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 74.00% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 74.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 74.20% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 74.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 74.40% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 74.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 74.60% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 74.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [============================>-----------] 74.80% | train_error: 10.4 | train_acc: 0.500 

  [============================>-----------] 74.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.00% | train_error: 10.4 | train_acc: 0.500 

  [=============================>----------] 75.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.20% | train_error: 10.4 | train_acc: 0.500 

  [=============================>----------] 75.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.40% | train_error: 10.4 | train_acc: 0.500 

  [=============================>----------] 75.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 75.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 76.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 77.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 77.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 77.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 77.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=============================>----------] 77.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 77.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 77.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 77.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 77.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 77.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 78.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [==============================>---------] 79.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 80.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 81.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 82.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 82.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 82.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 82.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===============================>--------] 82.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 82.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 82.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 82.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 82.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 82.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 83.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [================================>-------] 84.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 85.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 86.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 87.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 87.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 87.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 87.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=================================>------] 87.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 87.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 87.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 87.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 87.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 87.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 88.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [==================================>-----] 89.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 90.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 91.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 92.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 92.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 92.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 92.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [===================================>----] 92.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 92.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 92.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 92.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 92.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 92.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 93.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [====================================>---] 94.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 95.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 96.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 97.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 97.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 97.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 97.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [=====================================>--] 97.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 97.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 97.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 97.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 97.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 97.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 98.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.00% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.10% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.20% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.30% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.40% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.50% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.60% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.70% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.80% | train_error: -0.0000000010 | train_acc: 1.00 

  [======================================>-] 99.90% | train_error: -0.0000000010 | train_acc: 1.00 

  [=======================================>] 100.0% | train_error: -0.0000000010 | train_acc: 1.00 

Not bad, but the results depend strongly on the learning reate. Try different learning rates.

Week 42 Constructing a Neural Network code with introduction to Tensor flow

Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations

Applied Data Analysis and Machine Learning