Digit Recognition - Sequential Neural Network in Keras

Last article I have demostrated a basic neural network in digit recognition achieving a 94% accurancy without tuning. Well , this result is not nice. Err.. let's try another neural network to try whether we can increase the accurancy a bit without tuning too much.

Before you run code script , you gotta install the Tensorflow and Keras , those 2 packages. Tensorflow is a neural network-based encapsulation in python , then Keras is a higher encapsulation based on Tensorflow using python. So normally , you of course use the advanced Keras rather than the basic Tensorflow.

import keras
import numpy as np
import pandas as pd
from sklearn import datasets
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation

1. create a model

model  = Sequential() #create a model called sequential

#input layer
model.add(Dense(128, input_shape=(784,)))# Dense is called fully-connected layer with 128 neurons inside
model.add(Activation("sigmoid")) # activation function

#hidden layer
model.add(Dense(160))# Dense is fully-connected layer with 160 neurons inside
model.add(Activation("sigmoid")) #activation function

#output layer
model.add(Dense(10))# Dense is fully-connected layer with 10 neurons
model.add(Activation("softmax")) # output only consist of 0 and 1

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=["accuracy"])

Hidden layer's neurons' number is a tuning param, so normally , you need to try & error various to see how is the result goes.

Output layer's neuron's number is a fixed number, it depends on your output, eg : digits has 10 numbers from 0 to 9 , so i put 10 there.

2. read & train the data

from keras.utils import np_utils

dataset = pd.read_csv("train.csv")
target = dataset.iloc[:,0].values.ravel()
train = dataset.iloc[:,1:].values
test = pd.read_csv("test.csv").values

train_x = train
train_y = np_utils.to_categorical(target, 10) # convert into one-hot-encoder

# "verbose" is to print the result,  "epoch" is training frequency
history = model.fit(train_x, train_y, epochs=650,  verbose=1) 


print (history.history['acc'])
print (history.history['loss'])

Epoch 1/650
42000/42000 [==============================] - 11s - loss: 0.5749 - acc: 0.8446    
Epoch 2/650
42000/42000 [==============================] - 11s - loss: 0.3669 - acc: 0.8891    
Epoch 3/650
42000/42000 [==============================] - 12s - loss: 0.3301 - acc:

epochs is how many times you want to train your model, this is the param you gotta tune. noramlly just try & error a big number to see whether the graph in the next phase is convergent .if it is converging well, then it proves that this 650 in epochs is a good param.

np_utils.to_categorical is you convert the 0 to 9 , 10 numbers into binary expression , because binary is consiste with only 0 and 1, so it can fill the sigmoid function to feed the model , this is very important.

3.draw a graph to tune the model

%matplotlib inline

import matplotlib
import matplotlib.pyplot as plt


plt.xlabel('x')
plt.ylabel('y')
plt.title('acc graph')

plt.plot(range(len(history.history['acc'])), history.history['acc'])

plt.show()

This Step is the most important , because you only tune your params based on this graph , as you can see, the line is convergent and less fluctuated which means your model's prediction go stable in the later phase . This is a good sign. 

The way you tune your model is to let the graph go as less fluctuated as so possible . So in the end , it go alomost a horizontal line which should be the best result you want to achieve in theory. 

4. draw a sequence diagram and show here

from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot

SVG(model_to_dot(model).create(prog='dot', format='svg'))

This part is to draw a flow chart basicly to see how your model goes, it is not a must. however, it 's good to have.

5. predict

y_pred = model.predict_classes(test) ## predict, = predict

np.savetxt('neural_5.csv', np.c_[range(1,len(test)+1),y_pred], delimiter=',', 
           header = 'ImageId,Label', comments = '', fmt='%d')

27296/28000 [============================>.] - ETA: 0s

Ok the flow is simple and of fun, can't wait to see how the result go , isn't it?

upload the result to kaggle to see how much the result is improved.

It should go between 96 % to 99% depends how do you choose your hidden layer's neurons and epoch's. Anyway, if you find any part is blur for you, please leave a msg

References ：

http://www.graphviz.org/Download_windows.php
https://keras.io/models/sequential/
https://keras.io/layers/core/
https://transcranial.github.io/keras-js/#/