Digit Recognition - Multi-layer Perceptron classifier

Last Article I described some basic classifers to train the hand-written digits. After you run , you may notice that the accurancy can only reach up tp 92 %  in kaggle . which is not so nice. This article I will introduce a basic neural network - multi-Layer Perceptron classfier ,In this way , the result will be slightly improved . Ok , throw aways those boring math deductions and theories, Let's paste the Script directly here.

%%time

import numpy as np
import pandas as pd
from sklearn import metrics
from sklearn import datasets
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import KFold


dataset = pd.read_csv("train.csv")
target = dataset[[0]].values.ravel()
train = dataset.iloc[:,1:].values
test = pd.read_csv("test.csv").values

# testing set
x_finaltest = test

kf = KFold(n_splits = 10)

total_score1 = []
# 3 hidden layers, each layer 50 neutrons， try 50 first , if not good , then increase
model1 = MLPClassifier(hidden_layer_sizes=(50,50,50), alpha=1e-4,solver='lbfgs',random_state=1) 

for train_index ,test_index in kf.split(train):
    x_train = train[train_index]
    y_train = target[train_index]
    x_test  = train[test_index]
    y_test  = target[test_index]
      
    #train the model
    model1.fit(x_train, y_train)
       
    # test model
    y_predict1 = model1.predict(x_test)

    total_score1.append(metrics.accuracy_score(y_predict1,y_test))

#average score
print(np.mean(total_score1))

y_pred1 = model1.predict(x_finaltest)

np.savetxt('neural_4.csv', np.c_[range(1,len(x_finaltest)+1),y_pred1], delimiter=',',
           header = 'ImageId,Label', comments = '', fmt='%d')

0.947785714286
Wall time: 47min

Well ,After you run, you may notice that the score is improved from 92 % to 94.77%. Even though the score is still not so nice , yet you may notice the kaggle score is similar to your score and your score is indeed improved. So you can conclude that they may also use another neural network model to score your model.

I believe once you tune the parameters in this MLP you can still improve, but do it in your spare time to modify the hidden layer neutrons and layers, and also the solvers to see the results.

It's ok ,we will catch up bit by bit in future and the score will be finally nice