Abstract

Road sign detection is a very critical and trending concept for autonomous vehicles. The creation of an intelligent system to detect the road signs and convert them to speech commands can be the integral part of such autonomous vehicles in very near future.This paper presents a study to recognize traffic sign patterns using Neural Networks technique. The images are pre-processed with several image processing techniques, such as threshold techniques, Gaussian filter, Canny edge detection, Contour and Fit Ellipse. Then, the Neural Networks stages are performed to recognize the traffic sign patterns.

High-level architecture of the system:

Method

The first step of the preprocessing is to adjust the image size, reducing the space occupied when the resolution of the input image is too high. If the image size were too high, it would slow down the execution of the algorithms to the point that it would restrict the program execution. The next step is to transform the input image into a gray scale image, so the edge detection algorithm can be applied .The algorithm used to detect edges in the image is the Canny Edge Detection, which takes a grayscale image as input and produces another grayscale image as output, where only the edges are shown. Then, Morphological Operations such as dilation and erosion are applied. After this, Region of Interests or Contours are identified to locate the subject i.e. traffic sign (Rectangle, Triangle, Circle, Inverted Triangle). There are many types of traffic signs, but their shapes are very limited and characteristic, so knowing the shape allows us to determine if an ROI contains a potential traffic sign or not.

The Convolutional Neural Network (CNN) is a deep learning approach that stacks several convolutional, subsampling and non-linear activation layers in sequence. Recently, the CNN has become a breakthrough technique in the field of artificial intelligence for object classification and pattern recognition applications such as handwritten digit recognition, speech recognition, object classification, and face identification. A fully trainable Convolutional Neural Network (CNN) is used for the detection and recognition of traffic signs.

Image pre-processing steps:

The following graph shows to what extent classifier is able to correctly classify the traffic sign from a given set of classes of traffic sign dataset:

The graph shows to what extent validator is able to accurately predict the traffic sign from a given set of classes of traffic sign data-set:

The first layers of the network C1 up to S2 function as a trainable feature extractor. All the network layers contain neuron models as in a classical Multi-Layer Perceptron (MLP) network. The feature extraction layers C1 up to S2 have specific constraints such as local connectivity and weight sharing. With these constraints, the first layers are able to extract position invariant features from two-dimensional shapes. The classification layers n1 and n2 at the output are fully connected MLPs. These layers use the extracted local features to perform classification of the input image. The details of the three different types of layers are described in the next subsections.

Result

The Belgian Traffic Sign Dataset is used for this project, it has two directories:\ 1.BelgiumTSC Training (171.3MB) - contains about 4000 images.\ 2.BelgiumTSC Testing (76.5MBytes) - contains around 100 real images.\

Each of the two directories above has 62 sub-directories named sequentially from 00000 to 00062. The directory name represents the code (or label) and the images inside the directory are examples of that label.

After completion of building this system, the system was able to correctly identify the traffic signals. To determine the accuracy of the classifier, a graph of all the predicted images and compared it with the images used for training was generated. The speed sign recognition application is trained with greyscale images to classify speed signs based on their shape of at least 32x32 pixel values. The exploration resulted in an efficient network architecture which has very good recognition accuracy.

Ground truth and prediction comparison result for 10 random images:

Loss function Gradient vs the percentage of training finished:

Figure depicting optimal resolution vs accuracy :

One observation derived from experimenting with the resolution of the test images was that on reducing the resolution of images by half, the system find it difficult to identify the sign board in the test image. The minimum resolution at which the system is able to correctly identify the signs is found to be 32 x 32.

conclusion

This project implements a road sign board detection and recognition system based on a fully trainable Convolutional Neural Network (CNN) (TensorFlow). To train the application a dataset containing around 4500 real sign board images from Belgium are collected. The dataset is completed with 2904 background images of other road signs, the cars and other non-speed sign images. The speed sign recognition application is trained with greyscale images to classify speed signs based on their shape of at least 32x32 pixel values. The exploration resulted in an efficient network architecture which has very good recognition accuracy. The optimal resolution was found to be 32 x 32.

Authors and Contributors

Authors: @amiteshmahajan and @vaibhavsahu.

Support or Contact

Having trouble with Pages? Check out our documentation or contact support and we’ll help you sort it out.