Vision guided robots have been around now in the trade shows for quite some time and easily impress me to watch. Learning to do a vision-guided application myself was a bit scary I must admit, but nonetheless, I was very excited to get an application to use two products that I am very familiar with. I've used Intelligent Actuators IX series robots many times and really like what the robot can do. I've also used Cognex vision systems, and very familiar with vision systems and lighting. So why not make the two work together? That's what I've done finally! Granted, I have seen other applications such as Fanuc's Vision software, which is easy to use, but this application couldn't justify the cost of a six-axis Fanuc, instead we used a four-axis SCARA robot, which fits our application.
If you've never done a vision-guided robot, your probably asking yourself the same questions I asked when I got started. How do I get the vision system X, Y, and theta information in the form of pixels to translate into X, Y, and theta the robot can understand? I was really concerned that I would have to come up with my own complicated trigonometry formulas to be able to do this application. Luckily for you and me, Cognex saves the day with a trigonometry function built into the camera software called TRANSPATTERNTOWORLD in conjunction with a CALIB "Calibrate" function, more on this later.
The other thing I kept asking myself is how do I get the information from the Cognex to the IAI robot? With a little legwork, I found out that the IAI IX controller supports ASCII using serial by purchasing a dedicated RS-232 card with my controller. Cost was $550 for two RS-232 DB-9 ports installed in the IX controller. I don't advise that you use the DB-25 teach port, but rather fork the bill for the additional RS-232 ports so you can still have a DB-25 teach port for your laptop. I thought about using Ethernet, but I never got a warm and fuzzy feeling about getting Ethernet to work between the IAI robot and Cognex camera. So I went with what I knew, plain old RS-232 and ASCII.
After establishing a direction, I waited until everything was assembled on the machine and went to work. The application I had was to find the theta of a part using the camera and reporting that information back to the robot so it could place the part in the correction orientation on to a dial index table. The part was bowl feed to the robot so the orientation was unknown. The robot would always pick up the part the same, then read the camera data, upon placing the part, a calculation was done to subtract the pickup theta from the theta the camera said the part was located. This information was used to determine the final theta correction factor needed to place and orient the part on the dial table. Originally, I misunderstood our Mechanical Engineer thinking that I would need to know the X, Y and theta of the part so I wrote all my software to handle all three positions even though I ended up only using theta (degrees). The camera was mounted upside down to the robot, this kind of concerned me at first and didn't know really how to handle this, but after using the calibration function in the Cognex camera, it seemed to not make any difference.
I started to develop my Cognex application first. I started with the robot calibration wizard program that Cognex has in their online support download section. After playing with this wizard, I end up only using the four-point calibration method from the wizard. The way this works is you teach four points with a pattern match tool of your part. I can't show you the part in this article for disclosure reasons, but picture a part with a pattern tool selected around it. For example, what I did was pick up the part in the robot gripper, ran the pattern match tool to get the X, Y, and theta in pixels of the part. While the robot was still holding the part, I looked at the real world robot position and typed them into the Cognex camera. So in my example, point "0" was 370.527, 323.711 pixels, which was equal to the robot X and Y value of 176.427, 69.727.
The calibration function needs four or more points entered to work correctly. You'll notice that theta is not included in the calibration. The theta is dealt with a little differently all on it's own. It's important that you move the part to the extreme portions of your field of view to get a better, more accurate calibration. Since vision lens have a natural "fish-eye" effect, teaching points in the center and towards the outside of the lens field of view will give you the best accuracy. Of course, you can use more than four points if you want, but four points was enough to give me an accuracy of about +- 0.1 mm. Because of where I had to mount my camera, I was forced into using an 8mm lens, which is a very bad lens to give off fish-eye distortion. If you can use a 25mm or a tele-centric lens (50mm), I would definitely recommend it if you can fit your camera mounting location using those size lenses. Use a better lens especially if you need better than +- 0.1 mm accuracy for your application.
You can see the four points that are calibrated in this screen shot. Since the robot can only move to these maximum pick up positions, I calibrated to the outside of the tool, not the outside of the screen.
Once your calibrated, next you'll need to throw the TRANSPATTERNTOWORLD function on your sheet. Basically it needs to be pointed to the calibration points you set earlier. Once is has these calibration points, the idea is you can feed this function any pixel pattern X, Y and it will tell you what the real world robot X, Y is of the part you just took an image of. I ended up separating my X, Y from my theta in my application because in the beginning of the project I was confused at what I was suppose to be doing at first. I found this more flexible in the end, so I created two different TRANSPATTERNTOWORLD functions but used the same calibration points. Mainly I needed this for a part lookup function I was using. My application needed to run different parts. So in the end it worked out ok, plus I was able to prove to myself I can do all three positions. (X , Y and Theta).
The pattern find tool in the Cognex camera reports the theta of the pattern. This is the same theta I needed to determine the rotation in degrees for my robot. It was accurate enough for me to use this value as the pattern tool found the rotation of the part. One thing I did notice was the pattern find tool does not report back a full 360 degrees. At degree 0 you will get a 0 Angle from the pattern find tool. At degree 179 you'll get a +179 Angle. At degree 182 you'll get a 178 Angle.
I didn't really find any fault with this logic, other than the robot might make a few extra turns to get 178 degrees. Or so it would make two complete revolutions sometimes if it found a part pattern that was at 178 degrees (angle), which was really 182 degrees to the robot, but I went ahead and sent the 178 degrees and made the robot do the two complete revolutions. I had enough cycle time in my application to allow this. At the end of my project I wanted to come back and write some logic to automatically correct this 178 degrees issue I had found, but I just ran out of time and it was working ok. So I never went back and wrote something that would fix 178 Angle to read 182 degrees.
With the vision application taking images, now solving X, Y and theta, and reporting pixel patterns to the TRANSPATTERNTOWORLD function, it's time to generate an ASCII strings to send to the robot. When working with ASCII, I found it easier to make the ASCII data separated or delimited as I send it to whatever RS-232 device I am dealing with. In this case, I borrowed the example from the Cognex robot calibration wizard, and used a TAB space delimiter. I also read a little bit about the WRITESTRING function in the Cognex online help. I found out that the data needed to always change in order for the function to generate the string and send it out. So what Cognex did in the robot calibration wizard was make a counter that would always change the string data on an image update. I thought this was a good idea, and I used the same idea. My first data character in my string is the counter value that is sent to the robot. I really just ignore this value once it arrives at the robot but it's a good idea for keeping the ASCII strings up-to-date. Next, I arranged my data in some orderly fashion. Once again modeling what I seen done in the Cognex robot calibration wizard software, I put the X, Y and theta information in cells. I also created a cell with a value "Stringf("%c",09)" This denotes a TAB SPACE in ASCII. Now I need to Concatenate all the string values. Basically if you look at cell E70 in my Cognex application you'll notice this function "Concatenate($D$64,$D$68,$D$65,$D$68,$D$66,$D$68,$D$67)" What I am doing here is compacting all the information I need into one big string to send to the robot. First I grab the counter value, then add a TAB SPACE delimiter. For X, I get the X value, and then add another TAB SPACE delimiter. I do the same for Y, by grabbing the Y value, and then another TAB SPACE delimiter. Finally I get the theta value. Once all of these values are concatenated, I can send this string to the robot, and let the robot dissect the data I need at the robot end.
Stereo Vision