This food vision project aims to build a transfer learning model that recognizes food images. As our benchmark, we have kept the DeepFood research paper from 2016, which uses CNN to achieve a 77.4% test accuracy after training for three days on the Food101 dataset.
We turn our data from 101,000 image tensors and labels (train and test combined) into batches of 32 image and label pairs, thus enabling it to fit into the memory of our GPU. We shuffle, augment, batch, prefetch, and cache our data for efficient memory use and robust training. We also set up mixed precision training to convert a single-precision floating-point format (float12) to a half-precision floating-point format (float16) to use less memory where possible and, in turn, run faster (using less memory per tensor means more tensors can be computed on simultaneously). As a result, we improve performance on modern GPUs by up to 3x.
The first model we test is a transfer learning model that uses EffecientNetB0 weights to predict the type of fruit. The model is as follows:
InputLayer -> Functional (EfficientNetB0 [non-trainable]) -> GlobalAverage Pooling -> Dense -> Activation
Fine-tuning is to freeze a pre-trained base model and then train only the output layers for a few iterations to update their weights in line with your custom data (feature extraction). And then, unfreeze a number or all of the layers in the base model and continue training until the model stops improving.
We will also use the ModelCheckpoint, EarlyStoppingand ReduceLROnPlateaucallbacks so that we train only till convergence and prevent the model from overfitting.
InputLayer -> Functional (EfficientNetB0 [trainable]) -> GlobalAverage Pooling -> Dense -> Activation
Model 1 has an accuracy of approximately 70%. Model 2's accuracy was ~79.69% and it was trained in less than 20 minutes. The loss and accuracy curves are given below.
We further predict on random test data and obtain good results.
