Deep Learning using SAP Leonardo ML Foundation: Image Classification

[former_member517463]

This is the first article in a series of articles to come in the space of Deep Learning and how to use SAP Leonardo ML Foundation for the same. These articles will cover the complete process of a deep learning project starting for data preparation to prediction.

The first is a very popular and most successful application of deep learning: Image Classification.

Problem Statement: Make a deep learning model to classify images into various categories.

Dataset: Any image classification dataset. In this article, we will use the famous Fashion MNIST dataset. This dataset contains 70000 (60000 train + 10000 test) images of clothes having each image belonging to one of the 10 categories. You can get the dataset from here.  On unzipping, you will find train.zip and test.zip. The ten categories are:

1. T-shirt/top.


2. Trouser


3. Pullover


4. Dress


5. Coat


6. Sandal


7. Shirt


8. Sneaker


9. Bag


10. Ankle boot

Technological stack:

1. Platform: Train Your Own Model functionality of SAP Leonardo ML Foundation.


2. Deep Learning library: Keras.


3. Programming language: Python 2

Let’s start the work! Major steps are:

1. Data preparation.


2. Uploading data to SAP Leonardo ML foundation.


3. Making model and training.


4. Making predictions using the trained model.

 

Data Preparation

The images are 28*28 in size, they are already very small, so we need not crop them. Also, images are black and white already.

But we should still do data augmentation as more the data, better the performance for deep learning.

The dataset structure is:

- fashion_mnist.zip

- train.zip

- train/

- train.csv

- test.zip

- test/

- test.csv

 

The train folder has 60000 images and test folder has 10000 images. We will first divide this dataset to train (85%=51000 images) and validation (15%=9000 images). Also, we will create a separate subfolder for each of the 10 categories in both train and validation folder. The directory structure should look like this:

-train

-0

-1

…

-9

-validation

-0

-1

…

-9

 

Let’s do this on our local systems first and then upload it to SAP Leonardo ML foundation. The python code to do this (run the code in the folder where data is unzipped):

import pandas as pd

import cv2

import os



train = pd.read_csv('train.csv')



os.mkdir('train_folders')

os.mkdir('validation_folders')



for i in range(10):

    images = train['id'][train['label'] == i].tolist()

    num_images = len(images)

    

    os.mkdir('train_folders/' + str(i))

    for image in images[:int(0.85*num_images)]:

        img = cv2.imread('train/' + str(image)+ '.png')

        cv2.imwrite('train_folders/' + str(i) + '/' + str(image) + '.png', img)

        

    os.mkdir('validation_folders/' + str(i))

    for image in images[int(0.85*num_images):]:

        img = cv2.imread('train/' + str(image)+ '.png')

        cv2.imwrite('validation_folders/' + str(i) + '/' + str(image) + '.png', img)

 

This code will result in two folders named train_folders and validation_folders. These folders should contain 10 folders each numbered from 0 to 9 having images of the folder name class.

Uploading data to SAP Leonardo ML foundation:

1. Open command prompt and login to your respective space.


2. Upload the data:
   

   cf sapml fs put train_folders fashion_mnist/​
   
   cf sapml fs put validation_folders fashion_mnist/

   
   

 

This will create a folder named fashion_mnist and two folders in it in SAP Leonardo ML foundation.

 

Making model and training:

Since the dataset is not huge, we will use pre-trained models from keras trained on the imagenet dataset. The idea is to retrain the pre-trained models like ResNet50 or InceptionV3 which are already trained on the imagenet dataset and fine tune them with our dataset. Here, I won’t go in details of fine tuning of models, but we will see how to accomplish this in code on SAP Leonardo ML foundation.

Let’s write code for training:

All the necessary libraries are imported. Note that we are going to use inception_resnet_v2 model from keras.

## necessary imports

import pandas as pd

import numpy as np



import keras

from keras.preprocessing.image import ImageDataGenerator

from keras.applications.inception_resnet_v2 import preprocess_input

from keras.models import Model, load_model

from keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau

from keras import optimizers, losses, activations, models

from keras.layers import Convolution2D, Dense, Input, Flatten, Dropout, MaxPooling2D, BatchNormalization, GlobalMaxPool2D

 

Let's define some constants

## some constants

SIZE = 28

NUM_CLASSES = 10

BATCH_SIZE = 16

SEED = 1



train_data_dir = '../../fashion_mnist/train_folders/'

validation_data_dir = '../../fashion_mnist/validation_folders/'



nb_train_samples = 51000

nb_validation_samples = 9000

 

An image data generator is used to generate images from training folders, apply augmentation and feeds to the learning model. Here, we define two image generators, one for training and other for validation.

image_gen = ImageDataGenerator(rotation_range=15,

width_shift_range=0.2,

height_shift_range=0.2,

zoom_range=0.2,

horizontal_flip=True,

rescale=1./255,

preprocessing_function=preprocess_input)



val_image_gen = ImageDataGenerator(rescale=1./255, preprocessing_function=preprocess_input)



train_gen = image_gen.flow_from_directory(train_data_dir, batch_size=BATCH_SIZE, seed=SEED, target_size=(SIZE, SIZE), shuffle=True, class_mode = "categorical")



valid_gen = val_image_gen.flow_from_directory(validation_data_dir, batch_size=BATCH_SIZE, seed=SEED, target_size=(SIZE, SIZE), shuffle=True, class_mode = "categorical")

 

Defining the model. Here, we will load the pre-trained inception_resnet_v2 add a couple of dense layers and dropout layers and compile with SGD optimizer. We will keep all the layers except added ones as non-trainable because they are already pre-trained. This function returns the compiled model.

def get_model():



    keras.backend.clear_session()



    model = keras.applications.inception_resnet_v2.InceptionResNetV2(include_top=False, 

    weights='imagenet', input_tensor=None, input_shape=(SIZE, SIZE, 3), pooling='avg')



    y = Dense(32, activation='relu')(model.layers[-1].output)

    y = Dropout(0.3)(y)

    y = Dense(32, activation='relu')(y)

    y = Dropout(0.2)(y)

    y = Dense(NUM_CLASSES, activation='softmax')(y)

    model = Model(inputs=model.input, outputs=y)



    for l in model.layers[:-1]:

        l.trainable = False



    sgd = keras.optimizers.SGD(lr=0.001, momentum=0.9)

    model.compile(optimizer=sgd, loss=losses.categorical_crossentropy, metrics=['acc'])

    model.summary()

    return model

 

Also, define some callbacks: checkpoint, early stopping and reducing learning rate.

filepath = '../../fashion_mnist/model-inception-resnet-v2-retrain-{epoch:02d}-{val_loss:.2f}-{val_acc:.4f}.hdf5'



checkpointer = ModelCheckpoint(filepath, verbose=1, save_best_only=True)

early = EarlyStopping(monitor='val_acc', min_delta=0, patience=3, verbose=1, mode='auto')

reduce_lr = ReduceLROnPlateau(monitor='val_acc', factor=0.1, patience=2, min_lr=0.000001, verbose=1, min_delta=0.001)



callbacks_list = [checkpointer, early, reduce_lr]

 

It's time to train the model. Training logs are shown for each epoch. Note that the number of epochs is only 3, you may increase it. It's less because later we will fine tune all the layers again with more epochs.

keras.backend.clear_session()

model = get_model()



model.fit_generator(train_gen,

                      steps_per_epoch=nb_train_samples//BATCH_SIZE, 

                      epochs=3, 

                      verbose=1, 

                      callbacks=callbacks_list, 

                      validation_data=valid_gen, 

                      validation_steps=nb_validation_samples//BATCH_SIZE,

                      max_queue_size=20, 

                      shuffle=True,

                      workers=4)

 

Now, since the added dense layers are trained for 3 epochs (or less depending on callbacks), its time to fine tune the whole model with a smaller learning rate, so we unfreeze the whole model.

for l in model.layers[:-1]:

    l.trainable = True



sgd = keras.optimizers.SGD(lr=0.0001, momentum=0.9)

model.compile(optimizer=sgd, loss=losses.categorical_crossentropy, metrics=['acc'])

model.summary()



model.fit_generator(train_gen,

steps_per_epoch=nb_train_samples//BATCH_SIZE,

epochs=200,

verbose=1,

callbacks=callbacks_list,

validation_data=valid_gen,

validation_steps=nb_validation_samples//BATCH_SIZE,

max_queue_size=20,

shuffle=True,

workers=4)

 

Now, we have the trained model, let's save the weights for predicting on new images in the same fashion_mnist folder.

model.save('../../fashion_mnist/inception_resnet_v2_retrain_model.h5')

Running code on SAP Leonardo ML Foundation

Save all of the above training code as training.py in a folder code in the current directory. Also, create a yaml file by name fashion_mnist.yaml having following contents in the current directory:

job:

  name: "train-fashion-mnist"

  execution:

    image: "tensorflow/tensorflow:1.5.0-gpu"

    command: "pip install keras --upgrade && python training.py"

    completionTime: "10"

    resources:

      cpus: 1

      memory: 10000

      gpus: 1

Now, we have to upload the code folder having training.py and fashion_mnist.yaml to SAP Leonardo ML foundation as a job. As soon as the job is submitted it starts to run and you can also see the logs of the process. To upload, run in command prompt from the current directory:

cf sapml job submit -f fashion_mnist.yaml code

Predicting on new images

All we have to do now is to load the saved model and start predicting, making sure that we apply the same preprocessing steps as we did on the training dataset. But first, let's download test data from the same place (if you haven't done yet). It's a zip, extracting it will give you a folder names test. Make another folder testing and put this test folder in the testing folder (we are doing this as this is how keras needs for prediction). Upload testing folder to SAP Leonardo ML foundation.

cf sapml fs put testing fashion_mnist/

Let's start. Make a new python file prediction.py and write following code in it. After running this code, a final file having predictions for all the test images named submit.csv will get generated in the fashion_mnist folder in SAP Leonardo ML foundation.

## import libraries

import numpy as np

import pandas as pd



import os



import keras

from keras.preprocessing.image import ImageDataGenerator

from keras.models import load_model

from keras.preprocessing import image

from keras.applications.inception_resnet_v2 import preprocess_input



img_width, img_height = 28, 28

test_data_dir = '../../fashion_mnist/testing/'



datagen = ImageDataGenerator(rescale=1./255, preprocessing_function=preprocess_input)



model = load_model('../../fashion_mnist/inception_resnet_v2_retrain_model.h5')



generator = datagen.flow_from_directory(

        test_data_dir,

        target_size=(img_width, img_height),

        batch_size=128,

        seed=27,

        class_mode=None,  # only data, no labels

        shuffle=False)  # keep data in same order as labels

		

img_names = generator.filenames



probs = model.predict_generator(generator, verbose=1)



predictions = np.argmax(probs, axis=1)



df = pd.DataFrame(columns=['Image-Id', 'Category'])

df['id'] = img_names

df['label'] = predictions

df.to_csv('../../fashion_mnist/submit.csv', index=False)

The process to run the prediction is same as training:

1. Put prediction.py in the same code folder.


2. Change the yaml file, this should be the new content:

job:

  name: "predict-fashion-mnist"

  execution:

    image: "tensorflow/tensorflow:1.5.0-gpu"

    command: "pip install keras --upgrade && python prediction.py"

    completionTime: "5"

    resources:

      cpus: 1

      memory: 10000

      gpus: 1

3. Upload the prediction job:

cf sapml job submit -f fashion_mnist.yaml code

4. You can download the submit.csv file to your local system as well.

cf sapml fs get fashion_mnist/submit.csv .

 

Through this article:

1. Doing basic image classification using keras and python.


2. Uploading and downloading data from SAP Leonardo ML foundation.


3. Running programs as jobs on SAP Leonardo ML foundation using Train Your Own Model (TYOM).