This page collects all the code designed to control the robot and devices.
It may happen that after all the checks and tests the proposed code contains bugs (who is able to code without bugs?). If you found it, or simply want to report someone who you think improvements can be made my email is always at your disposal.

In addition to the code used on the Rover, it is available a collection of tutorials on Computer Vision and general programming.

Note: Title in “italian language” means that the article has not an english translation yet. You can read the article in your language using automatic translation tool available in article page under the title.

Microchip PIC

• Ambiente di Sviluppo SW PIC. A little tutorial about installation and configuration of the C programming environment for Microchip PIC [Old Site Version].
• Controllo Servo R/C . C code written for PIC 16f628 to control a DC Servo Motor [Old site version]

Computer Vision

• Qt Widget based on OpenGL to render OpenCV images [First Part]. first part of the tutorial about Qt Widget realization
• Qt Widget based on OpenGL to render OpenCV images [Second Part].. second part of the tutorial about Qt Widget usage.
• [Back soon] Face Recognition using OpenCV: tutorial about Face Recognition using OpenCV library.
• [Back soon] Background Subtraction: tutorial about Background Segmentation using OpenCV.
• OpenCV e il riconoscimento dei contorni. Tutorial to illustrate how it is easy to detect contours in an image using OpenCV library [Old site version]

After the creation of the Qt Widget to show an OpenCV image in a GUI in the first part of the tutorial, now we are going to see how to use it.

The simple application that we are going to create allows us to render the streaming video from a webcam. This is the base for each application that uses OpenCV.

This tutorial requires basic knowledge of Qt Creator and Qt. If you think that any step is not clear or sufficiently explained, please do not hesitate to contact me via email.

Let’s start by opening the application QtCreator, Qt can also be used in other work environments (Eclipse, Visual Studio, …), but honestly I prefer to use its original environment as it has been specially designed to take advantage of all the features of Qt.

Let’s create a new project “Qt Widget Project” -> “Qt GUI Application”.

Let’s copy in the project’s folder the files we created in the first part of the tutorial.

Add to the project the exisitng files “cqtopencvviewergl.h” and “cqtopencvviewergl.cpp”.

Let’s add to the GUI the widget OpenGL-OpenCV:

• open the form
• take a standard widget from “Containers” sub-list
• name the widget “openCVviewer”
• click on the widget with the right button and select “Promote to”
• nclick “Add” and insert “CQtOpenCVViewerGl” in “Promoted Class Name”
• click on the Form with the right button, select “Layout” and then “Layout in a Grid” to expand the widget on the whole available area

The base of the interface is ready. Now we need to configure the application so that it can use OpenCV:

• Let’s open the “pro” file and add:INCLUDEPATH += your openCV path

  (for Linux: INCLUDEPATH += /usr/local/opencv2/)

• aggiungiamo ancheLIBS     += -lopencv_core -lopencv_highgui

Now that the application is designed to using OpenCV we can enter the code to display the images from the webcam.

Still open the form and insert an item “Start” int themenu:

• select the menu bar where the label “Type here” and insert “Camera”
• select the item “Camera” and in the submenu “Type here” type “Start”

In the “Action Editor” menu click with the right mouse button on “actionStart” and select “Go to slot…”->”triggered()”.

QtCreator in this way automatically creates the function that will be called when the user selects “Camera”->”Start”.

Our function is very simple:

void QtGLWebcamDemo::on_actionStart_triggered()
{
    if( !mCapture.isOpened() )
        if( !mCapture.open( 0 ) )
            return;

    startTimer(50);
}

mCapture is a private variable of our GUI:

    cv::VideoCapture mCapture;

Remember also that in order to use VideoCapture we must include the OpenCV module “HighGUI” in our application:

#include

The application must handle the timer that we just created so that every 50 msec it asks a new webcam image and sends it to our widget opengl.

Add to our main class the handling of the Timer:

protected:
     void timerEvent(QTimerEvent *event);

That will contain the following simple code:

void QtGLWebcamDemo::timerEvent(QTimerEvent *event)
{
    cv::Mat image;
    mCapture >> image;

    // Do what you want with the image <img class="wp-smiley" alt=":-)" src="/web/20120417211944im_/http://www.robot-home.it/blog/wp-includes/images/smilies/icon_smile.gif" /> 

    // Show the image
    ui-&gt;openCVviewer-&gt;showImage( image );
}

We arrived at the end. The simple application to display the video stream from a webcam using Qt, OpenCV and OpenGL is ready, you just have to compile and try it.

If you want to download the project files for this tutorial, follow this link .

Have fun

This tutorial is about the problem of showing an OpenCV image in a Qt GUI, we will create a Qt Widget based on QGLWidget.
The widget allows a better image rendering and the “width/height” ratio is maintained during resize operation.

The tutorial assumes basic knowledge of C++, Qt4 framework, development environment and the library QtCreator OpenCV2. The sample code is designed for Windows 7, but the project file attachments can be adapted to Linux or MacOS without great difficulty, only by setting the correct “path” to the libraries in the file. “.pro” following the  comments.

First part will describe widget creation.

CQtOpenCVViewerGl: creation

In QtCreator software:

• File -> New File or Project -> C++ -> C++ Class -> Choose
• Class name: CQtOpenCVViewerGl
• Base Class: QGLWidget [pay attention to letter cases!]

At this point you have two new project files: cqtopencvviewergl.cpp e cqtopencvviewergl.h

Open cqtopencvviewergl.h and add two private variables:

private:
   bool mSceneChanged; /// Indicates when OpenGL view is to be redrawn

   QImage mRenderQtImg; /// Qt image to be rendered
   cv::Mat mOrigImage; /// original OpenCV image to be shown

   QColor mBgColor; /// Background color

   int mOutH; /// Resized Image height
   int mOutW; /// Resized Image width
   float mImgratio; /// height/width ratio

   int mPosX; /// Top left X position to render image in the center of widget
   int mPosY; /// Top left Y position to render image in the center of widget

• mSceneChanged: indicates if the frames has to be rendered
• mRenderQtImg: contains the image to be rendered in Qt format
• mOrigImage: contains the image to be rendered in OpenCV format
• mBgColor: background color for the zones not covered by the image
• mOutH & mOutW: image real rendering sizes
• mImgratio: contains information about image size ratio
• mPosX & mPosY: top/left image coordinates, allow to render image in the center of the widget

At this point a few functions are needed.
First of all we add one signal and one slot:

signals:
   void imageSizeChanged( int outW, int outH ); /// Used to resize the image outside the widget

public slots:
   bool showImage( cv::Mat image ); /// Used to set the image to be viewed

• void imageSizeChanged( int outW, int outH ): it is needed to communicate to “Qt world” that the Widget has been resized and that the rendered image will have “outW” and “outH” size. This is useful if we want to pass to the widget a just scaled image just to not oblige the widget to make rescaling operations for each rendering operation.
• bool showImage( cv::Mat image ): it is the function used to pass to the widget the image to be rendered. It is a “slot” just to allow the utilization of signal/slot functionality.

Finally we insert the five functions used to render the image:

protected:
   void initializeGL(); /// OpenGL initialization
   void paintGL(); /// OpenGL Rendering
   void resizeGL(int width, int height); /// Widget Resize Event

   void updateScene(); /// Forces a scene update
   void renderImage(); /// Render image on openGL frame

• void initializeGL(): openGL initialization
• void paintGL(): openGL rendering
• void resizeGL(int width, int height): openGL resizing
• void updateScene(): force rendering operations
• void renderImage(): called by paintGL to effectively render the image

Now it is time to pass to variable initialization and function definition in cqtopencvviewergl.cpp file.

Constructor:

CQtOpenCVViewerGl::CQtOpenCVViewerGl(QWidget *parent) :
QGLWidget(parent)
{
   mSceneChanged = false;
   mBgColor = QColor::fromRgb(150, 150, 150);

   mOutH = 0;
   mOutW = 0;
   mImgratio = 4.0f/3.0f; // Default image ratio

   mPosX = 0;
   mPosY = 0;
}

Other functions:

void CQtOpenCVViewerGl::initializeGL()
{
   makeCurrent();
   qglClearColor(mBgColor.darker());
}

Sets background color. This function, like next, begins with makeCurrent, such to allows GUI to use more than one OpenGL widget.

void CQtOpenCVViewerGl::resizeGL(int width, int height)
{
   makeCurrent();
   glViewport(0, 0, (GLint)width, (GLint)height);

   glMatrixMode(GL_PROJECTION);
   glLoadIdentity();

   glOrtho(0, width, 0, height, 0, 1); // To Draw image in the center of the area

   glMatrixMode(GL_MODELVIEW);

   // ---> Scaled Image Sizes
   mOutH = width/mImgratio;
   mOutW = width;

   if(mOutH>height)
   {
      mOutW = height*mImgratio;
      mOutH = height;
   }

   emit imageSizeChanged( mOutW, mOutH );
   // < --- Scaled Image Sizes

   mPosX = (width-mOutW)/2;
   mPosY = (height-mOutH)/2;

   mSceneChanged = true;

   updateScene();
}

This function is called each time the widget is resized:

• Rows 3-12: OpenGL layout initialization
• Rows 14-21: image sizes calculation such to maintain constant the width/height ratio
• Row 23: “imageSizeChanged” signal
• Rows 26-27: top/left image point such that it is centered in the widget
• Rows 29-31: the widget must know that there is a new image to be rendered

void CQtOpenCVViewerGl::updateScene()
{
   if( mSceneChanged && this->isVisible() )
      updateGL();
}

updateScene is needed to "force" the rendering of the scene after that the image has been updated. updateGL is called and the rendering is done only if it is really necessary.

void CQtOpenCVViewerGl::paintGL()
{
   makeCurrent();

   if( !mSceneChanged )
      return;

   glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

   renderImage();

   mSceneChanged = false;
}

paintGL is the function called by Qt when the area of an OpenGL widget is to be updated. In our case the function cleans the scene and calls renderImage to effectively render the image.

void CQtOpenCVViewerGl::renderImage()
{
    makeCurrent();
    
    glClear(GL_COLOR_BUFFER_BIT);
    
    if (!mRenderQtImg.isNull())
    {
        glLoadIdentity();
        
        QImage image; // the image rendered
        
        glPushMatrix();
        {
            int imW = mRenderQtImg.width();
            int imH = mRenderQtImg.height();
            
            // The image is to be resized to fit the widget?
            if( imW != this->size().width() &&
                    imH != this->size().height() )
            {
                
                image = mRenderQtImg.scaled( //this->size(),
                                             QSize(mOutW,mOutH),
                                             Qt::IgnoreAspectRatio,
                                             Qt::SmoothTransformation
                                             );
                
                //qDebug( QString( "Image size: (%1x%2)").arg(imW).arg(imH).toAscii() );
            }
            else
                image = mRenderQtImg;
            
            // --->Centering image in draw area
            
            glRasterPos2i( mPosX, mPosY );
            // < --- Centering image in draw area
            
            imW = image.width();
            imH = image.height();
            
            glDrawPixels( imW, imH, GL_RGBA, GL_UNSIGNED_BYTE, image.bits());
        }
        glPopMatrix();
        
        // end
        glFlush();
    }
}

renderImage is the main widget function, in fact it draws the image.

• Rows 19-32: these rows make the resizing of the image such that is is entirely visible in the widget. Note that is image sizes are correct, heavy resize operation are not processed. For this scope I inserted imageSizeChanged signal to allow other GUI function to perform resizing only when original image is created and not each time it is rendered.
• Row 36: indicates top/left image pixsel such that it is centered in the widget.
• Row 42: real rendering using OpenGL glDrawPixels function.

bool CQtOpenCVViewerGl::showImage( cv::Mat image )
{
    image.copyTo(mOrigImage);
    
    mImgratio = (float)image.cols/(float)image.rows;
    
    if( mOrigImage.channels() == 3)
        mRenderQtImg = QImage((const unsigned char*)(mOrigImage.data),
                              mOrigImage.cols, mOrigImage.rows,
                              mOrigImage.step, QImage::Format_RGB888).rgbSwapped();
    else if( mOrigImage.channels() == 1)
        mRenderQtImg = QImage((const unsigned char*)(mOrigImage.data),
                              mOrigImage.cols, mOrigImage.rows,
                              mOrigImage.step, QImage::Format_Indexed8);
    else
        return false;
    
    mRenderQtImg = QGLWidget::convertToGLFormat(mRenderQtImg);
    
    mSceneChanged = true;
    
    updateScene();
    
    return true;
}

Last, but not less important, is the function needed to communicate to the widget which is the image to render. showImage works on 8 bit OpenCV (cv::Mat) with one or 3 channels and converts them to the correct format for QGLWidget.
The function calculates and memorizes the image ratio (row 5).

Widget code is available using this link.

An example of use is available in the second part of the tutorial.