Designed a CNN inspired by LeNet architecture
The pipeline will be tested on some images and videos provided by Udacity. The following assumptions are made:
Here you can find the project.
I will use the following picture to show you all the steps:
Firstly, I applied a color filtering to suppress non-yellow and non-white colors. The pixels that were above the thresholds have been retained, and pixels below the threshold have been blacked out. This is the result:
I will keep aside this mask and use it later.
The original image is converted in grayscale. In this way we have only one channel:
Before running the Canny detector, I applied a Gaussian smoothing which is essentially a way of suppressing noise and spurious gradients by averaging. The Canny allows detecting the edges in the images. To improve the result, I also used the OpenCV function
In some cases, the Canny edge detector fails to find the lines. For example, when there is not enough contrast between the asphalt and the line, as in the challenge video (see section
Optional challenge). The color selection, on the other hand, doesn’t have this problem. For this reason, I decided to merge the result of the Canny detector and the color selection:
I defined a left and right trapezoidal Region Of Interest (ROI) based on the image size. Since that the front facing camera is mounted in a fix position, we supposed here that the lane lines will always appear in the same region of the image.
The Hough transform is used to detect lines in the images. At this step, I applied a slope filter to get rid of horizontal lines. This is the result:
Now I need to average/extrapolate the result of the Hough transform and draw the two lines onto the image. I used the function
fitLine, after having extrapolated the points from the Hough tranform result with the OpenCV function
findNonZero. I did this two times, once for the right line and another time for the left line. As a result, I got the slopes of the lines, and I could draw them onto the original picture:
Here some results on test images provided by Udacity:
You can find the original pictures and the results in the folder
Here some results on test videos provided by Udacity:
You can find the video files here: video1, video2.
While I got a satisfactory result on the first two videos provided by Udacity, it was not the case for the challenge video. In the challenge video we can identify more difficulties:
To overcome theses problems, I introduced the color mask and resized the ROI. This is the result, using only the color mask (without the canny detection):
You can find the video file here: video_challenge
The right line is a little jumpy mainly because of the curve: the function
fitline is trying to fit a line on a curvy lane. It would be useful to shrink the ROI in this case, but I preferred to keep the same ROI size used in the first two videos.
If we analyze the steps using a snapshot from the challenge video, we can notice that the Canny detector is not very useful:
while the color mask is able to detect the lines:
Indeed, as you can see in the following picture, we lose valuable color information when we convert the image in grayscale. Moreover, the Canny operator find a lot of edges when we have shadows on the road.
Just out of curiosity, I wanted to test the pipeline on a video extracted from Youtube (see the original video here ).
I noticed that the color selection was not working properly in this case, so I had to tune a little bit the thresholds values. This is the new result using both color selection and Canny:
You can find the video file here: video_extra
It would be wiser to transform the image in the HSV space and to apply the color selection, instead of doing it on the RGB images.
Some possible improvements: