Headlight (or Blob) Detection

I tried experimenting with Nicholas Howe's Segmentation through Graph Cuts but was largely unsuccessful on night images. I lowered the threshold as Nicholas Howe suggested but it still could not catch anything in the test video clips.

I implemented the sliding average(currently set at n=15 frames) to compute the background image for the motion segmentation. It produces much better results than just the previous frame.

Additionally, I worked on blob detection a little. After I get my image difference, I convert the frame into binary and do a connected component labeling on it. This is thresholded based on area. As you can see in the sample videos, the connected component labeling finds the headlights, but it also catches a lot of other false positives.

night time bike:




car at 5pm:



One idea I have is somehow to incorporate directional motion(like gradient with respect to y) as a possible feature. Since each camera will only be responsible for one traffic stop, we know the direction that the motorbike will be traveling. In this case, we can check if the blob is heading south. If this can be implemented successfully, we can get rid of the cross traffic detection.

Trials In Motion Segmentation

I'm working on segmenting the objects out of a video feed.

Most of the research papers I have read in regards to stationary traffic surveillance use simple background subtraction such as the frame before it or where the background image is taken when no traffic are present. ["Vision-based Detection of Activity for Traffic Control”] [“Real Time Detection of Crossing Pedestrians For Traffic Adaptive Signal Control”] Erosion and Dilation can be applied after to reduce noise. During our last class, Serge suggested perhaps taking the last N frames and compute an average from it to use as the background image. Oddly, I can't get the perfect segmentation pictures as found in the various research papers.

Here's a few runs in both day and night using different methods:
bike at night:

Subtracting from the Previous Frame Bike at Night:

Subtracting from the Average across all frames(not technically realistic since we can't see frames ahead of the current one) Working on doing just previous N frames.
bike at night:


car 5pm:


subtract previous frame car 5pm:


subtract avg car 5pm:

I also was looking other segmentation methods. I found an interesting one, “Better Foreground Segmentation Through Graph Cuts” by Nicholas Howe and Alexandra Deschamps. They had the Matlab code available for usage but it seems only to work with the demo movie file they included.
Here's how the sample video turned out. Looks good for blob detection(the next step), I'll take a more detailed look at it.

Improved method of capturing sample data

So it seems taking pictures is a poor way of capturing data. A better approach is just to take video footage and cut the frames out of it. I figured out the simple matlab commands for it:

FILENAME = 'MVI_1619.avi';
file_info = aviinfo(FILENAME);
num_frames = file_info.NumFrames;
for current_frame=1:num_frames
%grab the movie frame
movieframe = aviread(FILENAME, current_frame);
%convert to an image and save into cell
imageframe{current_frame} = frame2im(movieframe);
end

I read the research paper in the previous post in more detail. To answer my own question regarding why motion detection is needed rather than just work with a single image, motion detection is used for background detection. By subtracting two frames of a time interval, we can set the foreground to the objects that moved.

Research Papers

I found an interesting research paper that many of the ideas for segmentation that Serge suggested. It is titled "Vision-based Detection of Activity for Traffic Control."
The paper discusses how a single algorithm to cover all cases of traffic detection is inefficient. I tend to agree. There are variations between daylight and night that can be exploited to achieve better results.

The approach in the paper suggests two detection modes based on the environmental conditions. The first stage applies motion detection by background differentiation, ghost removal, segment filtering and an adaptive background update. Then, a dual processing scheme is applied that uses different methods to refine the motion parameters based on the contrast in the current image. A selection algorithm that uses HSV color space is used to determine different conditions(weather, day, night). In high contrast, shadows with moving objects need to be detected to avoid being merged with the foreground objects. In nighttime and low contrast, only the headlights is needed.

My question seems to for detection, why is motion detection utilized? Why not just analyze a static image? We can apply background subtraction, then look at the window of interest(the lanes) for key feature points such as the headlights.

As for testing images, i found a good spot at the intersection of Villa La Jolla and the Gilman Parking structure. here are some pics:


At first I took the image from the ground floor, then I took it from the 2nd floor of the structure. I think it's a more realistic approach to take it from a higher ground since thats where cameras are usually mounted.

Also, found this one that might of interest to the group, Vision-Based Human Tracking and Activity Recognition

Sample Images

I found this site, http://www.metrokc.gov/kcdot/mycommute/allcams.cfm, that has webcams setup near Seattle, WA.

Assuming I try to based my algorithm on detecting headlights, we see two immediate problems given the images below: glare and reflections. Detecting a motorcycle at night time i think is the most vital because it is when cars are less likely to be on the road, thus not set off the lights.

When I take test images on campus, I will try to mimic the perspective. I will also try to work with low resolutions to mimic a poor government bought camera.

100 Ave NE @ Juan-Wood Way



Novelty Hill @ 208 Ave NE (SE)





W Valley Hwy @ S 180th St


W Valley Hwy @ S 180th St (W)




Talbot Rd @ S 43rd St/Carr Rd



SE 176th St/Carr Rd @ SR515



116th SE @ Petro Rd




Iss Hob Rd @ SE May Vly Rd