Applied Object Recognition

John Keogh | May 15, 2013

An earlier blog post discussed a simple approach to object recognition and included a complete project in C#. I ported the code to Objective-C and made a video of the use of the code for object recognition for a small robot that I am building. This short post mostly adds a few points to the video. The earlier post referred to previously is where most of the code is. The robot uses an iPod running EyesBot Driver, which will be submitted to the Apple App Store soon, hopefully next week.

There were two primary problems to solve: how to teach the robot what an object is called, and how to later recognize the object.

Learning

The sequence of events that occurs when the robot learns an object is: This turned out to be quite simple to implement, and permits easy communication between the robot and user (see the video to see the learning in action). The code for this choreography is:

  
-(void)learnNewObject:(NSString *)objectName{
    
    [self performSelector:@selector(turnOffLights) 
       withObject:nil afterDelay:0.1];
    [self performSelector:@selector(saveBackgroundPhoto) 
       withObject:nil afterDelay:2.0];

    //flash light once to let user know to place object 
    //to learn in field of view
    [self performSelector:@selector(turnOnLights) 
       withObject:nil afterDelay:2.5];
    [self performSelector:@selector(turnOffLights) 
       withObject:nil afterDelay:3.5];
    [self performSelector:@selector(saveObjectPhoto) 
       withObject:objectName  afterDelay:5.0];

    //finally, learn the object
    [self performSelector:@selector(analyzeLearningPhotos:) 
       withObject:objectName afterDelay:6.0];
}
  
 

Recognition

The recognition is simple determining whether the palette present in the object corresponds to any of the learned objects. Please see the project in the other blog entry for a complete project that implements this approach, but the code to test the extracted color palette against the learned color palettes is:

  
 NSString *matchedObject = @"";
 NSInteger bestMatches = 0;
 NSInteger currentMatches = 0;
 for(LearnedObject *object in learnedObjects){
    currentMatches=0;
    NSString *objectName = object.name;
    for(NSString *color in object.pallette.keyEnumerator){
      if([imageColors valueForKey:color]!=nil){
        currentMatches++;
        if(currentMatches>bestMatches){
          matchedObject=objectName;
          bestMatches=currentMatches;
        }
      }
    }
  }
  
 

Limitations

The main problem is that this only looks for a palette, whereas objects also have a visual texture and morphology, and the colors for an object should be present in an image in a way that is consistent with the expected visual texture and object morphology. This simple approach is useful for a quick screen to narrow down the possible objects for further screening, but also have the problem of light level changes affecting the apparent color palette.

Future

I'm looking forward to releasing EyesBot Driver in the next few weeks, but it will likely not contain this functionality until the functionality is much more mature. It is quite useful for navigation (since rooms tend to have distinctive color palettes) and also as presumptive object recognition. Lots of work left to do, though.

Blog

Applied Object Recognition

Make a VR Robot

Make a Robot controlled with EyesBot App

Simple Computer Vision with Javascript

Simple Obstacle Avoidance

Robots and Cassandra

Robotics: Controlling CIM Motors with Jaguar Motor Controllers using an Arduino

Building a Robot Controlled with an iPod - Easier Version

Simple Computer Vision with Web Services

Robot Reacting to Visual Stimulus

First Look at the MyRobots API

Calling Web Services from a Robot

Motion Detection and Tracking in C#

Building a Robot Controlled with an iPod - Part One

Building a Robot Controlled with an iPod - Part Two

Measure the Volume of Liquid in a Container using an Arduino and a Pressure Sensor

Applied Object Recognition

Robotics: How to Use an LED to Determine Distances

Communicating with an Arduino or Hardware from an iOS Device

Using NPN and PNP Transistors

Optimum Deployment of Vision Based Security Devices

Object Recognition

Applied Physical Computing

Eyesbot Company

Download EyesBot Driver Free!

EyesBot

EyesBot Company

Developers

EyesBot Blog