RANSAC Lines

To solve last week's problem of fitting partial lines, Serge suggested RANSAC. I implemented RANSAC for line fitting and the results are good as expected. RANSAC basically takes n random points, creates a model based on these points, calculate the error for all the data points from this model and add to a list of possible models if within a threshold count.More details can be found at wikipedia

Here is a line fitting to the motorcycle frame from last week:


This is one from a car in the background:


As seen in the first image, we now have a more robust model for detecting motion towards a traffic stop.

Partial Line Fittings for Classification

My problem now is to see how to fit partial lines. Originally, I wanted the trajectory creation to make a line like Fig 1. All incoming traffic will have a path that is similar to this path given that it is tracked fine..

Fig. 1 ideal case

The tracking system I had used is still vulnerable to mis-labeling. This is even after using minimum distance and area difference to retain a previous labeling across frames.

On Fig.1, we see a stolen trajectory from an opposing traffic, but it still possess the last portion which can be used to determine if it is incoming traffic. It still is able the same slope and direction movement as Fig. 1. So how can we just use the last part of the line instead of the whole trajectory path?

both min. dist and area difference for labeling:


Fig.1a,b Stolen trajectory from opposing traffic.


Fig. 2 Opposing traffic.


Harris Corner Detection
Curious to see how harris interest pts would work out. I used N. True's method for his parking example: Do a harris corner detect over the region of interest and sum up the points. The sum should different from empty traffic because pavement produces no corners. I've tried this with N. True's openCV implementation and it is able to pick up interest points on the bike.

The biggest problem I see is how occlusion of cross traffic can greatly affect it which is why motion can be used to help separate correct incoming traffic.

Thoughts on Classification

In regards to the classification, I was thinking of the possible problems I might run into with my idea. My idea is to have a human label the correct incoming traffic object in the training set. The system will save these object's trajectories. For testing, when given a video footage(but in the same location) the system has not seen before, it will do the blob detection and check the current trajectory if it near matches the ones in the training set.

The problems I see are:
- occlusion will happen sometimes so the incoming traffic will might only have half the trajectory.
- a vehicle traveling north in the opposing lane will have the close to the same points as the incoming traffic. --> need to use time/frame numbers to help.



On a side note, check out the power of human low-resolution recognition

Opponent Color Subtraction Result

I've implemented image subtraction using the blue/yellow channel as opposed to just rgb.
I also had to dilate the image before doing the connected component labeling for the daytime bike to be more apparent.

This is a video clip of the rgb subtract, notice no bike spotted in the image subtraction.


This is a video clip of the blue/yellow channel subtract with dilation.


This is the same capture as above with the footage shown.


Using this blue/yellow channel image subtraction seems to hurt night time bike tracking.The bike is first labeled as 2 and then when the car drives by, it becomes a new blob.


Maybe use RGB for night time and B/Y on daytime?

--
For the classification rules, I'm thinking of something that trains on a set of known trajectories that are known to be bike traffic. Then each actual labeled blob is compared to this set and check if the distance between the two trajectories are within a certain bounds.

Opponent Colors Image Subtraction

I did some more experimenting on image subtraction in different color space. I used two frames from the daytime footage. Serge suggested using opponent colors rather than LAB for better debugging. Opponent Color channels has a simpler model.
Below is plain RGB subtraction.


These two show the subtraction using opponent color channels. I ignore The first one is the green/red one and the second one is blue/yellow. The Blue/yellow seems to bring out difference more, but nosier than RGB.

Daytime Clip and Blob Position Plots

I recorded a clip with a bike at 1 pm. With the same settings as used in the gilman car 5pm and bike at night, it was not able to recognize the bike correctly. Perhaps the image is too saturated with light.

I found a L*A*B* colorspace function for Matlab. Here are the different components for a snapshot of the above video.


----
I plotted the positions of different objects in a 3d plot with x,y and frame number as the axes.

This one is of the car at 5pm gilman. Only posted ones with more than 10 positions.

This one is of the motorbike at night gilman. Only posted ones with more than 5 positions.



TODO Next:
-Fix daytime issue.
-Come up with classification rules.

Tracking Blobs

Since last week's discussion, I have decided to change my detection algorithm by adding a tracking component. After image subtraction to get the moving blobs and thresholding, I track the moving blobs to determine which direction it is moving in. Since the camera traffic light triggering is intended for only traffic stop, we can then determine if the blob is moving in the direction of the targeted traffic light.

My algorithm for tracking is simple:

For all the frames in the video:
Compute the background by a sliding average.

Do an image subtraction of the background image from the current image.

Convert the resultant image to a binary image and use a connected component labeling

threshold on area to reduce noise

for all the blobs on the current frame:
compute position(centroid), area, boundingbox

compare the current blob's position, area against the global blob set
if positions are the nearest and within a certain threshold, add position to the matching global blob
if the positions are not near, add to global blob set

if a blob's trajectory has shown the intended direction path in the specified area, turn on boolean flag for triggering

Results:
night time bike:

car at 5pm:


The cross traffic does not track as well because they are moving faster where as the incoming traffic is moving slowly towards a stop sign and are able to be tracked.

TODO:
-Experiment and grab more training data in the daytime.
-->Expect to run into issues in the day with more cross-traffic, people, lighting conditions.