KSR10 Robotic arm programmed to follow big red squares using OpenCV

Hi, there. I know it has been a long time. But no worries I am back with a small project. I got my hands on a KSR10. The KSR10 is a simple robotic arm that any 13 year old (or 13 year old at heart) can put together. The great part of it though is even though the manual says it has no support 64-bit computers you can easily find a Python library that does. Now you know all know how much I love writing in Python. Even a broken down computer and coming home tired from work could not stop me work on the project!

I strapped a simple web-cam on the robotic arm to make it 100 times cooler. Now the robotic arm follows faces and big red squares!! I used the OpenCV library for the project. Below you can find a nice little video;

The library for the KSR10 could be found below;

ksr10.py

# this file contains the base control set for the KSR10 Robot arm
#
# Written by Niels Bosboom, n.bosboom@gmail.com, july 2012
# (based on ideas provided by http://www.aonsquared.co.uk)

# imports
import usb.core
import time

class ksr10_class:

    def __init__(self):
        self.dev = usb.core.find(idVendor=0x1267, idProduct=0x0000)
        if self.dev is None:
            print "Unable to initialize ksr10..."
        # set status to stop all movement and dim lights        
        self.status = dict()
        self.status["lights"] = 0
        self.stop()

    def sendcommand(self):
        # build command from status
        byte1 = self.status["shoulder"]*64 + self.status["elbow"]*16 + self.status["wrist"]*4 + self.status["grip"]        
        # build command        
        comm_bytes = (byte1, self.status["base"], self.status["lights"])
        # print "commando : ",comm_bytes
        # send command to device
        try:
            self.dev.ctrl_transfer(0x40, 6, 0x100, 0, comm_bytes, 1000)
        except AttributeError:
            print "KSR10 is not connected..."

    def lights(self):
        # swap lights
        if (self.status["lights"]== 0):
            self.status["lights"] = 1
        else:
            self.status["lights"] = 0
        #send command
        self.sendcommand()

    def move(self,part,direction):
        #determine which part to move:        

        # determine direction
        if (direction=='right') or (direction=='up') or (direction=='close'):
            dir_command = 1
        elif (direction=='left') or (direction=='down') or (direction=='open'):
            dir_command = 2
        else:
            dir_command = 0        
        self.status[part] = dir_command

        # send command
        self.sendcommand()

    def stop(self):
        # stop all movement and dim lights        
        self.status["shoulder"] = 0
        self.status["base"] = 0
        self.status["elbow"] = 0
        self.status["wrist"] = 0
        self.status["grip"] = 0
        # send command
        self.sendcommand()

if __name__ == '__main__':
    print "No main in this module"

And next is our neat little code that actually does the image processing;

import cv
import ksr10
import time

def find_car(image):

	size = cv.GetSize(image)

	#prepare memory

	car = cv.CreateImage(size, 8, 1)

	red = cv.CreateImage(size, 8, 1)

	hsv = cv.CreateImage(size, 8, 3)

	sat = cv.CreateImage(size, 8, 1)

	#split image into hsv, grab the sat

	cv.CvtColor(image, hsv, cv.CV_BGR2HSV)

	cv.Split(hsv, None, sat, None, None)

	#split image into rgb

	cv.Split(image, None, None, red, None)

	#find the car by looking for red, with high saturation

	cv.Threshold(red, red, 128, 255, cv.CV_THRESH_BINARY)

	cv.Threshold(sat, sat, 128, 255, cv.CV_THRESH_BINARY)

	#AND the two thresholds, finding the car

	cv.Mul(red, sat, car)

	#remove noise, highlighting the car

	cv.Erode(car, car, iterations=6)

	cv.Dilate(car, car, iterations=6)

	storage = cv.CreateMemStorage(0)

	obj = cv.FindContours(car, storage, cv.CV_RETR_CCOMP, cv.CV_CHAIN_APPROX_SIMPLE)

	cv.ShowImage('Countour Vision', car)

	if not obj:

	return(0, 0, 0, 0)

	else:

	return cv.BoundingRect(obj)

points = []

capture = cv.CaptureFromCAM(0)

if not capture:

print "Error opening capture device"

sys.exit(1)

cv.SetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH, 640)

cv.SetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT, 480)

# create object

ksr = ksr10.ksr10_class()

# turn on lights

#ksr.lights()

prevx = 0

while 1:

	original = cv.QueryFrame(capture)

	car_rect = find_car(original)

	x,y,width,height = car_rect

	print car_rect

	if width>50 :

	if ((0<x)and(x 5 and abs(points[-1][1] - middle[1]) > 10:

	points.append(middle)

	cv.Rectangle(original,

	(car_rect[0], car_rect[1]),

	(car_rect[0] + car_rect[2], car_rect[1] + car_rect[3]),

	(255, 0, 0),

	1,

	8,

	0)

	for point in points:

	cv.Circle(original,

	point,

	1,

	(0, 0, 255),

	-1,

	8,

	0)

cv.ShowImage('Computer Vision', original)

k = cv.WaitKey(27)

Aaand that is how it is done!

I would like to thank Niels Bosboom for making my life easier by writing the robotic arm library.

So, have fun and code on!