Tensorflow Model Building

Dec 20 2019

This is all taken from sentdex's excellent Tensorflow tutorial series. Check it all out here:
https://www.youtube.com/playlist?list=PLQVvvaa0QuDfhTox0AjmQ6tvTgMBZBEXN

After we have imported our own data as discussed in the previous post. We can begin building and training our model. There are different ways this can be approached, one way is to try different models until you get something that clicks, even a little, and then expand from there.

The below could be an example of code that clicks but that hasn't been optimized or expanded upon yet:

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
from tensorflow.keras.callbacks import TensorBoard
import pickle
import time
import numpy as np


gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.8)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
# The above just designates how much gpu memory you're designating to this particular model.
# Can be useful for when you're running multiple models at the same time.

NAME = f"Cats-vs-dogs-cnn-64x2-{int(time.time())}"
tensorboard = TensorBoard(log_dir=f"logs\\{NAME}")

x = np.asarray(pickle.load(open("x.pickle", "rb")))
y = np.asarray(pickle.load(open("y.pickle", "rb")))

x = x/255.0

model = Sequential()
model.add(Conv2D(64, (3,3), input_shape=x.shape[1:]))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Conv2D(64, (3,3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation("relu"))

model.add(Dense(1))
model.add(Activation("sigmoid"))

model.compile(loss="binary_crossentropy", 
              optimizer="adam", 
              metrics=["accuracy"])

model.fit(x, y, batch_size=32, epochs=10, validation_split=0.15, callbacks=[tensorboard])

Some things to point out:
1) x = x/255.0 is an attempt to normalize the data before we pass it through the neural network. The easiest way to normalize the data is to scale that data. In this case we're using imagery data, we know the min: 0 and max: 255 values for pixel data, so we can just divide by 255 here. Otherwise we may have to use keras.utils.normalize(x, axis=-1, order=2) which normalizes a Numpy array.

2) model.add(Conv2D(64, (3,3), input_shape=x.shape[1:])) the input_shape uses [1:] to skip the detail of how many of the feature sets we had. So this dynamically will give us the input_shape, something like 50x50x1.

3) model.fit(x, y, batch_size=32, epochs=10, validation_split=0.15, callbacks=[tensorboard]) the batch_size is how many samples you're passing at a time out of our total ~25000. The epochs are how many cycles it goes through, if you do too few epochs your model won't learn as well at it could but if you do too many it'll over-learn/over-fit which means it just starts memorizing the data itself rather than generalizing. The validation_split allows you to choose the percentage of your data that you want to dedicate to measuring validation such as val_acc and val_loss. Here we're using the last 15% of our training data to measure how well the model actually works.