This week we looked at two ways to do object recognition.
Kevin went through the steps involved in finding an object with a particular colour in an image. He started on an image with six circles each of a different colour, and demonstrated finding a green circle in the image. Then he stepped through the Python code and explained each task.
Here’s the image:
OpenCV, the Open Source Computer Vision library, has lots of functions for transforming and processing images.
We started with a standard RGB (red, green, blue) JPEG image, which OpenCV stores in memory as BGR (blue, red, green). Then we transformed the image to HSV (hue, saturation, value) format. The HSV colour space has a very useful property: colours are described by their hue and saturation. The value represents the intensity of the colour. This means that we can use H and S to find a colour, without having to worry much about lighting or shadows.
Next we used the known value for the green circle to apply a threshold to the image: any colours above or below the threshold are converted to black. Any colour at the threshold is converted to white. Here’s what the thresholded image looked like:
Then we found the white area in the image. To do that, we used an OpenCV function that gets the co-ordinates for contours that can be drawn around the boundaries of all the white regions in the image. We calculated the areas of each contour, and took the largest. We’ll find out why this is useful later.
To show that we had found the right circle, we calculated the co-ordinates of its centre-point. Finally, we drew a small cross at that centre-point to mark it, and displayed the full image on screen. This is what we ended up with:
Since we had a contour, we also used that contour to draw a line around the perimeter of the circle.
Next, we took a photo of a blue marker, found the HSV value of the blue, and used that value to find the marker in a live image. We held the marker in front of a laptop webcam, moved the marker around, and watched as the spot moved with it. Our method for finding a particular colour works for any shape, since we use a contour, not just circles.
Michael introduced us to TensorFlow, a machine learning library. Once trained, TensorFlow can identify specific objects by name. It’s a lot more sophisticated than finding something by colour. We spent some time setting the library up on a Raspberry Pi. The Pi isn’t powerful enough to train the software, but it is capable of doing the recognition after training models on more powerful computers.
Or final goal is to build an autonomous robot to play a game of hide and seek. We can use one of our remote-controlled battlebots from last year to hide, and the new robot to do the seeking on its own. One way to do the seeking would be to go after the biggest thing in the robot’s field of view that has a particular colour – the colour of the hiding robot. Another way to do the seeking would be to train a TensorFlow model with pictures of the hiding robot, so that the seeker can recognise it from different angles.
It’s going to take us a while to figure out what works best, and then we have to work out how to control our robot. It should be an interesting new year.