Letting go the Fear: machine vision and image analysis

Successfully developing a vision-based application can require so much specialized knowledge that many would-be developers refuse to attempt the task, turning instead to consultants who have built their careers on mastery of the technology's many nuances. Often, a consultant can save you not only many times his or her fee, but also a lot of valuable time. Even so, new shrink-wrapped software packages for vision-based-system development are increasing the number of projects that those without machine-vision- or image-analysis experience can tackle with impunity.

If you lack appropriate experience, a good first step is to try to determine which tasks require outside assistance and which tasks you are likely to be able to accomplish expeditiously on your own using prepackaged software. Suppliers of development tools and hardware can often help you to make this judgment. In many cases, their Web sites contain tools to help with the decision. A phone call to such a vendor will usually put you in touch with an application engineer who can gather information about your application. When appropriate, most suppliers will refer you to consultants with whose work they are familiar. Frequently, the most economical approach is to use consulting help only for certain parts of a project, such as lighting.

Image analysis and machine vision are related but different fields. In one sense, image analysis is part of machine vision. In another sense, however, image analysis is a broader discipline. In fact, the lines of demarcation between the fields often become blurry.

Machine-vision applications usually have a commercial flavor. For example, machine vision is a vital part of many manufacturing processes. On the other hand, "image analysis"-as most people understand the term-is more likely to find application in scientific-research laboratories. Some experts say that image analysis often deals with operations that are less defined than those of machine vision. An example is characterizing or classifying images of unknown objects, such as animal-tissue cells in an academic laboratory (Figure 1) or even in a clinical pathology laboratory.


In machine vision, you usually have a general idea of what the camera or image sensor is looking at, but you need to obtain more specific information. Product-inspection applications fit this category. For example, you know which model of PC board an image depicts, but you must determine whether all of the components are of the correct type and are in the proper positions. Determining whether the parts are correct and properly oriented certainly involves image analysis, but the analyses are more straightforward than those in the clinical lab.

Categorizing machine-vision tasks
Several experts categorize the major machine-vision tasks as follows:

• Counting components, such as washers, nuts, and bolts, and extracting the visual information from a noisy background.


• Measuring (also called gauging) angles, dimensions, and relative positions.


• Reading, which includes operations such as getting information from bar codes, OCR (optical-character recognition) of characters etched on semiconductor wafers, and reading 2-D DataMatrix codes.


• Comparing items, for example, finding manufacturing defects, such as missing components or labels, by comparing units on a production line with a KGU (known-good unit) of the same type. The comparison can consist of simple pattern subtraction or can involve geometric- or vector-pattern-matching algorithms, which you must use if the compared objects are of different sizes or orientations. Types of comparisons include detecting objects' presence and absence, matching colors, and comparing print quality. The inspected items can be as simple as aspirin tablets, whose correct marking requires verification before packaging.


Because the preceding list is specific, it may suggest that you can create machine-vision applications using menu-driven graphics-based development tools as opposed to writing code in a text-based language, such as C++. You can indeed use one of several menu-driven, graphical, application-development packages, although developers who have long experience with text-based programming of machine-vision applications generally prefer to stick with tools they have successfully used for years. Whereas some in the industry decry this reluctance to change, ask yourself how you would feel if a consultant you engaged to work on an application made your job the first on which she or he tried a new tool set.

Even among graphics-based tools, suppliers differentiate those that provide real programmability from those that allow users to merely configure applications. The configurable approach lets you more quickly get an application running and provides all of the flexibility that many developers need. Programming provides still more flexibility at the expense of somewhat longer development times-especially for those using a tool for the first time. In some cases, both the configurable and the programmable approaches produce outputs in the same language, enabling you to use programming to modify or enhance applications that you created using the configurable approach (Figure 2). The potential benefits of such flexibility are great: You can use the more powerful tool to perfect an application, which, with the aid of the more basic tool, you quickly made work at a primitive level. This approach reduces the likelihood of wasting time on perfecting approaches in which you later discover fundamental flaws.


Ongoing tweaking
Perhaps more important are the ways in which the ability to move easily between approaches can simplify the ongoing tweaking that is inescapable in many machine-vision applications. In AOI (automated optical inspection), for example, you might want to reject any UUT (unit under test) that differs from the KGU. Alas, if you take this tack, the inspection process will probably reject most of the units you produce, even though most of them perform acceptably. A simple example of an unimportant difference that might cause an AOI system to reject a good assembly is the UUT's use of a component whose date code differs from that of the equivalent component on the KGU.

Now, during design of the application, you might foresee the data-code problem and ensure that the system ignores image differences in regions that contain date codes. Unfortunately, though, other unimportant differences are more difficult to predict, and you must expect the need to modify the application as you discover them. In fact, some AOI systems' software almost automatically makes such changes; if you tell the system that it has rejected a good unit, the software compares the unit's image with that of the original KGU and eliminates differing regions from inspections of subsequent units.

Such approaches sometimes produce unsatisfactory results, however. Suppose that the inspection system is in a room in which some light enters through windows from outside, causing variations in the UUT illumination. Although a human inspector accommodates them without a second thought, such variations can cause a vision system to classify identical objects' images as different, thereby causing unpredictable inspection failures. Although blacking out the windows can prevent entry of external light, it may be more cost-effective to tweak the test program to pass KGUs under all lighting extremes.

Even so, this example points out the importance of lighting in machine vision and image analysis. Lighting is a science or art in its own right. Various lighting techniques have varying strengths and weaknesses, and the method of illuminating a UUT can solve or ameliorate common machine-vision problems (Reference 1).

Project cost and time frame
Costs of machine-vision projects vary widely. A few such projects cost no more than $5000 each, including the costs of hardware, prepackaged software-development tools, and the application developer's time. Such low project costs are likely to exclude the costs of tweaking and tuning the application to achieve satisfactory performance, however. At the other end of the spectrum, projects cost in excess of $1 million. The most common of these are probably capital improvements to automated manufacturing lines in the automotive and aerospace industries. According to several suppliers, the most common projects typically range in cost from a few tens of thousands of dollars to slightly more than $100,000. Project duration from the time when management approves the start of work until the vision system is in routine use in production is generally less than six months and is often only a couple of months.

Not surprisingly, nearly all vision projects begin by getting answers to basic questions. The answers to these questions substantially determine the cost of the vision-system hardware: How many cameras do you need? What image resolution must you have? Is color imaging necessary? How many frames per second must you acquire? Will you use cameras that produce an analog output? If so, you will need to select a frame-grabber board to convert the analog signals to digital form and, if necessary, synchronize the acquisition of image frames with external trigger events (Reference 2).

Although some frame grabbers for analog cameras can accept simultaneous inputs from several cameras, boards that provide an interface to one camera at a time are more common. If you choose cameras that have a digital interface, will you use "smart" cameras that perform image processing as well as image acquisition, or will the cameras send raw (unprocessed) image data to the host PC for processing? Also, which interface standard or bus will the digital cameras use to communicate with the host PC? Digital cameras for some buses require frame grabbers. However, unlike frame grabbers for analog cameras, frame grabbers for digital cameras don't perform analog-to-digital conversion.

Hardware-related considerations can go beyond these questions. Moreover, some of the questions make a tacit assumption that is usually correct: that the vision system's host computer is a PC that runs a standard version of Windows (www.micro-soft.com). Machine-vision systems sometimes run under real-time operating systems, and image-analysis software often runs under Unix or Linux. Moreover, like other real-time systems, many real-time vision systems use CPUs other than Pentium (www.intel.com) or Athlon (www.amd.com) devices.

Camera interfacing
Interfacing cameras to host computers remains a key issue in vision-system design. The selection of the camera interface still merits careful consideration despite the emergence of digitally interfaced cameras and imaging systems' acceptance of IEEE 1394 (also known as FireWire and i-Link) for camera interfacing. (USB 2.0, which is fast becoming the dominant high-speed PC-peripheral interface, is not a factor in industrial imaging, largely because, despite a data-transfer rate of 480Mbps, which is nominally higher than that of the initial version of FireWire, USB 2.0's host-centric protocol is slower than FireWire for imaging.)

FireWire is a popular high-speed serial bus in consumer-video and home-entertainment systems. The plug-and-play bus uses a multidrop architecture and a peer-to-peer communication protocol. The standard's initial embodiment covered data transfers at rates to 400Mbps. Speeds will eventually reach 3.2Gbps. In January, the IEEE announced 1394b, and adherents expect to soon see the 800Mbps version in vision hardware. Still, the choice among industrial FireWire cameras remains limited despite the cameras' reasonable cost; their growing availability for consumer applications (in which the required resolution-and sometimes the frame rate-are more modest than those in industrial applications); the convenience of the slender, flexible serial cables; and the noise immunity of the bus's digital technology.

Cost may be limiting FireWire's popularity in industrial imaging. Industrial FireWire cameras cost more than industrial analog-output units of similar frame rate and resolution. On the other hand, cost comparisons with analog cameras can sometimes mislead you. On systems with built-in FireWire ports, cameras usually require no additional interface hardware. The cameras include an ADC, whereas analog cameras require frame grabbers to perform the necessary ADC function (Figure 3).


FireWire cameras use IEEE 1394's isochronous protocol, which guarantees bandwidth and guarantees that data packets-if they arrive at all-arrive in the order in which they were sent. The standard's other protocol, asynchronous, guarantees message delivery but not the arrival of packets in the order in which they were sent. Each isochronous device can issue a bandwidth request as often as every 125µsec-that is, at an 8kHz maximum rate. The device that is acting as bus manager grants each device that issues a request the right to transmit a predeter-mined number of packets within the ensuing 125µsec.

The more isochronous devices on the bus, the less bandwidth each is entitled to. When it is the only camera on a FireWire bus, a 12803960-pixel monochrome camera can transmit perhaps 15 frames/sec. A 6403480-pixel FireWire color camera can transmit approximately 30 frames/sec. Although neither of these examples appears to use all of the bus's available data-transmission capacity, the number of bits per pixel and the way in which the camera formats the data can affect the maximum frame rate. By the way, higher resolution isn't always better. Higher resolution cameras are not only more expensive and generally slower than lower resolution units, but also more readily reveal inconsequential differences between UUTs and the KGUs, thus raising the rate at which AOI systems incorrectly detect failures.

More camera interfaces
Besides FireWire, interfacing options for digital-output cameras include RS 422 parallel interfaces and Camera Link (Table 1). RS 422 camera interfaces are not completely standardized, so they usually need camera-specific interface cards. These cards are not frame grabbers in the sense that interface cards for analog-output cameras are, but they usually plug into the host PC's PCI bus in the same way. A parallel interface can prove awkward because it sometimes requires more than 50 wires. Nevertheless, RS 422 digital cameras remain popular and continue to be widely available.


The AIA's Camera Link is the highest performance standard for interfacing digital-output cameras. Unlike FireWire, Camera Link allows only one camera per bus, although many PCs can accommodate multiple Camera Link buses. Camera Link transmits data at speeds as high as 4.8Gbps using SERDES (serialization/ deserialization) technology
on parallel combinations of unidirectional, serial, point-to-point links. Each link carries data from seven channels and uses LVDS (low-voltage differential-signaling) technology, which requires two wires per link. The number of channels determines a Camera Link bus's maximum data rate.

A fully configured bus would contain 76 channels, including 11 links and 22 wires, but the standard also allows for buses with 28 and 56 channels (four and eight links and eight and 16 wires). Each Camera Link bus normally requires a separate interface card in the PC.

A choice of Camera Link also currently involves writing additional software. Shrink-wrapped application-development packages usually lack Camera Link drivers because the cards
that generate the Camera Link bus in the PC are both rare and not completely standardized. Still, if you need Camera Link's blazing speed, you have few alternatives.
Sometimes, you can decrease the amount of data that vision systems have to handle by using smart cameras, which process, or compress, the data they acquire before transmitting it to the host PC. Such cameras can sometimes pay for their greater cost by reducing both the required data rate between the cameras and the host and the processing load in the host. However, you need to be sure that the data compression is either truly lossless or that you won't need the data you lose through compression.

Author Information
In his 16 years at EDN, Dan Strassberg, who recently "graduated" to Contributing Technical Editor, has often been fascinated by the confluence of image analysis and data acquisition. Apparently, he shares this interest with several vendors that started in data acquisition and have successfully branched into image analysis and machine vision. Strassberg holds a BSEE from Rensselaer Polytechnic Institute and an MSEE from Massachusetts Institute of Technology. You can reach him at StrassbergEDN@att.net.

References
1. Keyence Corp, Machine-vision on-line interactive seminar, http://world.keyence.com/topics/vision_ seminar/lighting.html.


2. Test & Measurement World, www.tmworld.com. Click on Other Searches and Entire Site. Entering into the search-criteria field immediately below such terms as "machine vision," "image analysis," or "automated inspection," brings up dozens of relevant articles.

At a glance


• Not all vision-related projects require the services of consultants; with the aid of suppliers of the hardware and development tools, developers lacking experience in vision can often do much-if not all-of the work and save their companies money, too.


• Before you begin a vision-system development, you need to answer approximately half a dozen questions; your answers will largely determine the system's hardware cost.


• You can greatly enhance your efficiency by choosing development tools that let you begin application development in a menu-driven environment and then refine your program via graphical or syntactical programming.


• Get used to the idea that vision systems can require care and feeding after installation; often, you can't foresee all of the reasons that tweaking the algorithms may be necessary after the system has run for a while.