OCR's Power !!

Robots learn to read the writing on the wall !!

A new breed of robot is using text-spotting software, dictionaries and internet access to learn to read anything, anywhere

INGMAR POSNER wants to develop robots that can see through walls. Not by equipping them with X-ray specs or specialised radar technology, but simply by teaching them to read.

"By reading a label on a closed door you can sometimes get a good idea of what can be found behind it," says Posner, a romanticist at the University of Oxford. "Reading can help you detect things you cannot directly see."

Roboticists have spent years teaching their robots a range of skills to help them get by in the real world. Robots have learned to map their surroundings and to pick up and manipulate cumbersome objects. Some have even shown signs of becoming self-aware. But, remarkably, they remain illiterate.

With the written word so prevalent in the human world - from road signs to shop names - a non-reading robot trying to prove its worth is placed at a severe disadvantage, says Paul Newman, who works alongside Posner. Along with Peter Corke at the Queensland University of Technology in Brisbane, Australia, the team are trying to help robots level the playing field.

Teaching robots to read should, in principle, be relatively simple. After all,optical character recognition (OCR) software packages already exist. These automatically turn scanned images of books into text, and many researchers are using them to turn robots' attention towards posters and signs on city streets. Last year, for example, Google launched Goggles, a smartphone application for just that task. Since May, Goggles has been able to translate languages, helping tourists work out what to order from a menu, for instance.

Current 'reading' software tries to force everything a robot sees into text, such as walls and chimneys

"The OCR software doesn't cater for the fact that it might not be seeing text," says Posner. "It tries its level best to force everything into text - brick walls, chimney stacks, everything." The result is a nonsensical muddle.

To get round this problem, the team developed text-spotting software. This relies on the fact that there is often a horizontal area of uniform colour just above and below text on a sign, but lots of two-tone colour variation within the text itself. Once the software has identified text, an image of it is passed onto the OCR software to read.

Even then, the results returned by the OCR software are often error-strewn. So the team has loaded their test robot, Marge, with a dictionary and spellchecker. This allows it to work out that "roodbond" is most likely a misreading of "broadband", while "nqkio" should be read as "Nokia".

To understand names it reads in its environment, Marge turns to news websites, such as The New York Times and BBC Online. The robot trawls the sites for appearances of the word it has read, and analyses how often keywords like "restaurant" or "bank" appear in the same stories. This allows it to make strong semantic connections between frequent matches. Using this approach, Marge has learned that Strada is a UK restaurant chain, and that Barclays is a UK bank.

With those systems in place, Marge is now ready to read and exploit text in the world in the same way a human does - a "seriously exciting" prospect, says Posner. The work was presented at the International Conference on Intelligent Robots and Systems in Taipei, Taiwan, last month.

One potential problem is identifying words that are difficult to read because of the viewing perspective, says Majid Mirmehdi at the University of Bristol, UK, whose team has developed its own software to help robots to read. Words printed on a curved surface can appear distorted, making them tricky for a robot to understand. Mirmehdi's team is working on improving their software to overcome this, so a humanoid robot with dextrous hands can manipulate objects - like cylindrical paint cans - to read them more easily.

Posner hopes his team's work will allow mobile robots to carry out tasks more easily by following signs the same way a human can. For example, a search-and-rescue robot in a building wouldn't need to gradually build its own map of the building - it could read any available signs to find its way around.

While such end results are still a long way off, other romanticists agree that reading projects are worth pursuing. The work is "refreshingly original in the robotics context", says Gregory Dudek at McGill University in Montreal, Canada. "I personally believe that exploiting OCR methods in a mobile robotics context makes a lot of sense," he adds. "In fact, once you reflect on it, there is no doubt it will be highly useful."

Courtesy