Qt and serial ports

Update Jun 22, 2011 : Fixed a bug for compiling under visual studio 2010.

This is intended to be a small add-on to the article I’ve posted on my website which explains how to use boost’s asio library to interface with serial ports from C++ code. This blog post focuses on a topic not covered in the article: using serial ports in Qt GUI applications.

Since serial ports are used primarily to interface a computer with a microcontroller, we’ll focus on this case.

The additional problem that occurs when the application has a GUI is the result of two restrictions that GUI applications have:

  • The thread that runs the GUI event loop should not block for long periods of time (say, more than a second). If this happens, the GUI “freezes” and this annoys users who think the application has crashed and usually force-quit it.
  • Code that updates the GUI should not be called from threads different from the main thread.

Now, if the microcontroller never sends data to the serial port unless the computer sends a command to it, and replies are always fast there is no problem.

Let’s show a basic protocol of this kind:

  • The PC can send the command “A” to the microcontroller.
  • If the microcontroller receives “A”, it reads an ADC channel and sends the value back to the PC

As said, in this case there is no problem. A possible GUI would have a button to read the value, and a label to show the result.

Such an application can easily use the TimeoutSerial class, with code to write the letter “A” in the callback of the button, immediately followed by a read from the serial port to get the result and show it on the GUI. The timeout, set to an appropriate value (such as 500ms) would ensure that the GUI won’t freeze even if something goes wrong, and the programmer can catch the TimeoutException to show an error message to the user.

But what to do if the command takes much time to complete, say 10 seconds? In this case this simple approach won’t work, because the GUI will freeze for ten seconds with the main thread waiting for the response from the serial port.

And what to do with a protocol like this?

  • The microcontroller sends the value read from the ADC 20 times a second.

In this case the PC application should always listen on the serial port for incoming data.

Well, who has read my article will probably think of using the AsyncSerial class, which allows to register a callback that will be called every time some data arrives from the serial port.

But also this approach won’t work, because of the constraint that the GUI code can be called only from the main thread, but AsyncSerial calls the callback from a separate thread.

To show how to solve this problem I’ve written a class, called QAsyncSerial which is a wrapper class for AsyncSerial that uses Qt’s system of signals and slots. It basically emits a signal every time a line is received from the serial port, and because signals and slots are thread safe, the code works as expected.

To show everything in practice, this is a simple GUI that uses QAsyncSerial running on Linux:

This simple GUI allows to select a serial port and open it (the baud rate is fixed to 115200, 8N1 format). Once it is opened, there is a line where to write text to send to the serial port while received text is showed below.

The code of the QAsyncSerial class and of the GUI example is posted at the end of the original article (here). It is meant to be compiled with QtCreator. If you try to compile it don’t forget to edit the line “-L/usr/local/lib” in the SerialGUI.pro file to point to where you have the boost libraries installed.

Last note, since the Qt libraries are cross platform, here’s the same application running on a Mac:

Tags: , ,

63 Responses to “Qt and serial ports”

  1. Kevin Waite Says:

    I found your article tonight “Serial ports and C++” and I thought it was so good I saved and printed it. Currently I am attempting to use the boost libraries [ asio ] for my own “virtual com-port” communication in both Linux and Windows, using the FTDI chipset which has worked very well for me. I am very familiar with C [ at least for micros ! ] but a complete noob for C++. Therefore I really appreciated the article’s style- I felt I could understand everything right away except the very last part concerning mutex and multi-threading.
    Thank you so much for writing and posting this article.

    Which do you like more, QT4 or wxWidgets? I bought the book on wxWidgets but now I’m not sure if I shouldn’t have purchased the QT4 book instead.
    Do we actually need C++ or could we write an application that is acceptably fast in Python [ pySerial ]? I don’t know python either, usually I use Java. I fear C++ – my “primer book” for beginners is over 1000 pages long….. 🙂

  2. fedetft Says:

    Thanks for the comment 🙂
    Well, C++ has a steeper learning curve than C, but once you’ve learned at least the basics it allows you to write code faster without losing performance (and by the way, you can program also microcontrollers in C++, at least some of them such as ARM or even AVR).
    The asio library is very good, it has asynchronous features which are good for performance, but while for maximum ease of use there are socket iostreams that allow to make a TCP connection in just a couple of lines of code, there is no such thing for serial ports.
    That’s why I wrote these classes, both to be used when you need to access a serial port in a couple of lines of code, and to show how to use asio by reading how they’re implemented.

    Regarding wx and Qt, I’m not that GUI expert, but I’ve tried both, and Qt seems more modern, consistent and high level. Sometimes wx has a slightly different behaviour on Windows and Linux, which is annoying when developing, while Qt does not. Plus, Qt’s signal and slots seem very powerful and easy to use.
    If you use QtCreator you also get good documentation for Qt which helps learning the framework.

    Regarding which language to choose, it depends on the application. Both Java and Python have libraries for serial ports. C++ will probably result in a faster application which uses less RAM memory. Also, if have a primer book for C++ it probably means you want to learn it anyway, but these are just guesses. In the end I can’t answer this question, you will have to make the decision.

    PS: if you plan on learning C++, and you come from C, don’t forget to learn about shared_ptr (http://www.boost.org/doc/libs/1_42_0/libs/smart_ptr/shared_ptr.htm). It is smart pointer with reference counting that will manage memory (and in general resources) for you.

  3. Kevin Waite Says:

    Thanks so much!!
    This is helpful. I already have QTcreator installed on my openSuse dist but since I bought the wxWidgets book I basically ignored it. I have read other users also commenting on the signals and slots but I have no idea what it means [ I will find out !]. But since your reply I will go back and try the QT4 – I have nothing to lose at this point.

    The best thing about your article above is that you lead us step by step from the simple to the complex- so I can perform a quick test right away to verify if all hardware and software are holding hands properly. Then later I can use the threaded classes. This is excellent style and I wish more people would adopt it.

    Basically I am developing a DDS waveform generator that is controlled by the user via the virtual serial port. All of the hardware is working properly – I could just use MS C# and call it a day, but I live most of my life in Linux and many other people do too …. so your article is very useful to me.

    I will take your advice and try to learn C++ – if it results in a faster program with less memory usage than that seems the way to go.
    I’m just a hardware hacker stumbling my way through the PC software!

    P.S. thanks for the shared pointer link!!
    Kevin

  4. fedetft Says:

    Signal and slots is a Qt specific feature (http://en.wikipedia.org/wiki/Signals_and_slots) to make event driven systems easy. They are basically used to handle GUI events. For example pushing a button emits a signal in Qt. So you can register a slot (which is basically a member function of a class) to be called when the user pushes a button. If you’ve programmed GUIs before regardless of the language, you’re surely familiar with this way of doing things.

    DDS chips are cool, but I’ve never used one, though.
    I’ve generated sine waves with a microcontroller + DAC + low pass filter built with discrete components before (frequencies in the audio range, so doable in this way). The good of DDS chips is that they contain everything and are capable of going up into the 100MHz range.
    Are you keeping a blog for your project? If yes, post the link here 😉

  5. Kevin Waite Says:

    Initially I wanted to come up with a better signal generator that I could build, as the MAX038 is basically no longer available. This was mainly to be for testing inductors and capacitors … In any case Analog Devices has quite a few of the DDS – I settled on the AD9833 but plan to upgrade in the future. They are not widely used outside industry due to the difficulty of soldering them [ tiny footprint ] and complex SPI setup. Nevertheless I have successfully driven the DDS using both Atmel and PSOC chips.

    I had quite a difficult time getting the Simple Serial class to work – I struggled with wxWidgets but am now having success with the Qt. I did not have any problems with the boost libraries, but I’m only a C programmer and not a great one at that! I am struggling learning both C++ and Qt at the same time, but at least I am at the point where I can input “/dev/ttyUSB” into a text box and I am able to write / read strings from my micro using Qt.

    Another difficulty in Linux is that when the USB cable is plugged in, the virtual COM port that is created is only accessible by root. So one has to change permissions on the device before you can use the software. I am sure this is solvable without running the software as root which would just be wrong! At least the FTDI drivers are loaded automatically.

    Thank you for the Qt link for the wrapper – apparently and sadly there is no real Qt access for serial ports. There was Qextserial [ perhaps not spelled right ] but it appears dead.

    IFtools https://iftools.com/start/index.de.php ctb library looked good to me, and their wxWidgets wxterm works really great! Finally I found a hyperterminal replacement for Linux. Unfortunately I was not able to make their ctb serial port library work for me – you could probably do it but I cannot. It is touted also as being “non-blocking” and is much much smaller than BOOST.
    There is just not enough good documentation for me to use their library but I am looking forward to using the Asynchronous Class you wrote!

    Thanks,
    Kevin

  6. Judy Duncan Says:

    I’m trying to use the boost library with Qt in order to use your implementation of QAsyncSerial on Windows 7 with Qt 4.5. I downloaded boost 1.43 and built it with bjam. I’ve picked up the system, thread, and date_time.dll and .dll.a files and placed them in the same folder with your SerialGUI project. When I try to build the project, I get a warning from the boost headers telling me to define _WIN32_WINNT_ or _WIN32_WINDOWS_ “appropriately”. Do you what the expected value is and how/where I should define it?

    If I can ever get things set up, I can understand them fairly well – but building the libraries and trying to sort out the results is very confusing.

    Thanks,
    Judy

  7. fedetft Says:

    I’ve come across this issue earlier with boost.asio. It apparently uses some low-level functionality that is different across windows versions, so you have to define that macro with the correct version.
    Searching on the web it seems that there are only two possible values,
    0x0500 for win2000 and earlier, or 0x0501 for win xp and above.

    Unfortunately I don’t have a windows machine right now to do the test and make sure it works, but you’ll have to add a
    #define _WIN32_WINNT 0x0501
    before you #include any asio headers (asio should be a header only library, so this looks enough)
    Given you’re using QAsyncSerial, open the file AsyncSerial.h (QasyncSerial depends on it) and add that macro before the line that includes boost/asio.hpp

    Let me know if it works.

  8. Rinie Says:

    QextSerialPort is updated at
    http://code.google.com/p/qextserialport/

    Will look into this one as well

  9. Johan Says:

    Hi fede.tft,

    thanks for your serial port classes. Do you want bug reports? In that case: If you create an instance of the BufferedAsyncSerial class without arguments and later call open() then it will never receive anything.

    This is how I fixed it:
    – the AsyncSerial class open() method should be virtual.
    – BufferedAsyncSerial implements its own open() which first calls AsyncSerial::open() and then sets up the callback.

  10. Johan Says:

    While I’m at it, have you considered submitting your Serial Port project to sourceforge.net or launchpad.net? I haven’t found any cross platform serial port class out there, that’s better than yours.

  11. fedetft Says:

    Sure, bug reports are always welcome.
    Today I updated the code to fix this issue. The solution was even simpler than yours, I just forgot a call to setReadCallback() in the default constructor. Adding that call fixed the bug.

    About creating a project on sourceforge, you’re right, the blog is not a good way to keep track of bugs and making new releases. Now I’m a bit busy, but I’ll consider this possibility.

  12. Kalle H Says:

    Hey, thanks for the great serial port tutorial! I used some of your code (the second one, with the async read and timeouts) in one of my projects but I’m having a little problem. I’m trying to change the serial port speed on the fly, but first I have to send a command to the device in the serial port to change the speed, and then I have to reconfigure the serial_port device for the new speed. But the problem is that the write() function returns before the device has actually received the command to change the speed, and the set_option seems to discard the current (hardware) serial_port buffer (or something, anyway the device does not receive the command to change the speed if I do set_option after the command in my code).

    Oh, and your code does _not_ seem to work if the timeout is 0. First read works, following ones just hang. If I configure the timeout to boost::posix_time::seconds(1) (or anything else than 0) the code works fine.

  13. fedetft Says:

    Hi, I fixed the bug of the zero timeout.
    On the other hand, I couldn’t reproduce the bug about changing baudrates.
    Also, a new iostream-compliant serial port class was added.

  14. seank Says:

    Awesome work, is there anyway to flush the buffer without calling read? Sometimes there is garbage sitting in the buffer before I power up the uC. If I call read and there aren’t any chars in there it will throw an exception. In plain C I think its called tcflush(). Sorry for the dumb question.

  15. fedetft Says:

    Which class are you using? And what exception are you getting? Maybe timeout_exception? If it is timeout_exception you could simply try to increase the timeout value, or catch it and retry reading.

  16. hiabai Says:

    Great job! Thanks.
    At the begining, I used Qt Creator – the IDE to build the project and got some errors from boost head files. It took my several hours to find what’s wrong. While the answer is that I should use the MinGW’s make to build the SerialGUI.pro. 😦

  17. fedetft Says:

    Yes, you need the boost libraries to use QAsyncSerial. Either build them from source, or get them pre-built. On windows use MinGW distro (http://nuwen.net/mingw.html), while on linux you usually find them using your package manager. In this case be sure to get the -dev or -devel packages as well.

  18. bluehash Says:

    I’m haveing trouble integrating Boost libraries with my project. My path does contain MinGW\bin. Also included the following in the .pro file
    LIBS += -LF:\Dev\BOOST\MinGW\lib \
    -llibboost_system \
    -llibboost_thread \
    -llibboost_date_time \
    -lwsock32 -lws2_32

  19. bluehash Says:

    Forgot to add my errors:

    In file included from mainwindow.h:5,

    from main.cpp:2:

    QAsyncSerial.h:10:32: error: boost/shared_ptr.hpp: No such file or directory

    In file included from mainwindow.h:5,

    from main.cpp:2:

    QAsyncSerial.h:84: error: ‘boost’ has not been declared

    QAsyncSerial.h:84: error: ISO C++ forbids declaration of ‘shared_ptr’ with no type

    QAsyncSerial.h:84: error: expected ‘;’ before ‘<' token

  20. fedetft Says:

    If you get errors while compiling it’s a matter of include path, the compiler can’t find the required .h files. Missing libraries cause errors when linking, not compiling.
    Are you using the GCC in MinGW distro (http://nuwen.net/mingw.html) that should have the boost libraries already in the search path?
    Otherwise you can use the INCLUDEPATH command in the .pro file to tell the compiler where are the header files of boost, see http://www.qtcentre.org/threads/1974-qmake-INCLUDEPATH-with-spaces.

    Last note, when you say
    LIBS += -LF:\Dev\BOOST\MinGW\lib \
    -llibboost_system \
    -llibboost_thread \
    -llibboost_date_time \
    -lwsock32 -lws2_32
    this is actually wrong, you have to use the library name without the lib prefix and the .dll (or .a, or .lib) suffix. So the correct one is:
    LIBS += -LF:\Dev\BOOST\MinGW\lib \
    -lboost_system \
    -lboost_thread \
    -lboost_date_time \
    -lwsock32 -lws2_32

  21. bluehash Says:

    Thanks. I’m getting closer:

    Warning: .drectve `-aligncomm:”___hexdig_D2A”,2′ unrecognized

    Warning: .drectve `-aligncomm:”___mingw_gMTRemoveKeyDtor”,2 ‘ unrecognized

    Warning: .drectve `-aligncomm:”___mingw_gMTKeyDtor”,2 ‘ unrecognized

    Warning: .drectve `-aligncomm:”___mingw_usemthread_dll”,2′ unrecognized

    F:/Dev/BOOST/MinGW/lib/libmingw32.a(tlssup.o):tlssup.c:(.text+0x5c): undefined reference to `LoadLibraryA@4′

    F:/Dev/BOOST/MinGW/lib/libmingw32.a(tlssup.o):tlssup.c:(.text+0xd4): undefined reference to `FreeLibrary@4’

    Is it because boost may be compiled with gcc and not g++?

  22. fedetft Says:

    Boost is a C++ library and therefore must be compiled with g++, but to build it you usually use the build script that come with it, and they do the right thing.
    I don’t recognize those linker errors, except the last two which probably mean you forgot to link with -lkernel32 (http://msdn.microsoft.com/en-us/library/ms684175%28VS.85%29.aspx)

  23. bluehash Says:

    Thanks for your help, unfortunately its still there. If I find out the problem, I’ll get back to you.

  24. bluehash Says:

    Ok.. I had to rebuild my libraries and not use the one form the nuwen link above.

    Also, had to change library names as qt couldn’t find it:
    LIBS += -LC:/Temp/boost_1_46_1/stage/lib/ -llibboost_system-mgw44-1_46_1 -llibboost_thread-mgw44-mt-1_46_1 -llibboost_date_time-mgw44-1_46_1

    LIBS += -lkernel32 -lwsock32 -lws2_32

    Hope this helps someone.

  25. nev008@hotmail.com Says:

    Thanks for your article on serial ports. As a newbie to Qt and C++ i have found the info a great help.
    On a GUI i was trying to figure out the best way to read what COM ports are available and show them in a combo box. I came up with the following:

    void MainWindow::on_updateportlist_clicked()
    {
    int i;
    QString testPortName;
    const int MAX_COM_PORT = 100; // The maximum number of COM ports that will be searched for.

    for(i = 1; i cboComm->addItem(testPortName,0);

    }
    }
    }

    The “serial.open(testPortName,115200);” line crashes the program when a COM port that is not available is detected. Is there any modification I can do to your QAsyncSerial class that will work?

    Apologies in advance if this already exists and I just haven’t figured it.
    Using XP.

  26. nev Says:

    Code snipet didnt paste quite right. Try this.

    void MainWindow::on_updateportlist_clicked()
    {
    int i;
    QString testPortName;
    const int MAX_COM_PORT = 100; // The maximum number of COM ports that will be searched for.

    //cboComm.Clear

    for(i = 1; i cboComm->addItem(testPortName,0);

    }
    }
    }

  27. nev Says:

    and again

    void MainWindow::on_updateportlist_clicked()
    {
    int i;
    QString testPortName;
    const int MAX_COM_PORT = 100; // The maximum number of COM ports that will be searched for.

    //cboComm.Clear

    for(i = 1; i cboComm->addItem(testPortName,0);
    }

    }

    }

  28. fedetft Says:

    Try modifying open() in QAsyncSerial.cpp from:

    void QAsyncSerial::open(QString devname, unsigned int baudrate)
    {
    try {
    pimpl->serial.open(devname.toStdString(),baudrate);
    } catch(boost::system::system_error& e)
    {
    //Errors during open
    }
    pimpl->serial.setCallback(bind(&QAsyncSerial::readCallback,this, _1, _2));
    }

    to:

    void QAsyncSerial::open(QString devname, unsigned int baudrate)
    {
    pimpl->serial.open(devname.toStdString(),baudrate);
    pimpl->serial.setCallback(bind(&QAsyncSerial::readCallback,this, _1, _2));
    }

    and surround the call to open() in your code in a try..catch block.
    In this way you should be able to check if the serial port was opened successfully (no exception thrown) or not (exception thrown).

  29. nev Says:

    Thanks for your quick response.

    I tried this and the error is not being caught by the try catch (the program crashes). The error occurs when the

    pimpl->serial.open(devname.toStdString(),baudrate);

    line is being executed. The error is a segmentation fault. Any suggestions greatly received.

  30. fedetft Says:

    Ok, it’s not an exception, it’s a crash.
    Unfortunately I couldn’t reproduce your error. I modified the MainWindow::on_openCloseButton_clicked() function like this to better see what happens when a nonexisting serial port is opened:

    […]
    serial.open(ui->portName->currentText(),115200);
    //if(!serial.isOpen() || serial.errorStatus()) return;
    bool fail=false;
    if(!serial.isOpen()) { cout<<"not open\n"; fail=true; }
    if(serial.errorStatus()) { cout<openCloseButton->setText(“Close”);
    […]

    But on my PC I get (correctly)
    not open
    error
    so the attempt to open a nonexisting port leaves the object not opened and with the error flag set to true.

    Try to compile in debug mode and single step your code till when you find the crash.

  31. johpe Says:

    Hi, I really found your post and article about boost serial ports very interesting and educational. I started playing around with the Callback version of your serial implementation and I’ve gotten it to work pretty well.

    BUT (there always is a but right?) I have tried to use what the boost docs seem to suggest to keep the io_service::run from returning (http://www.boost.org/doc/libs/1_46_1/doc/html/boost_asio/reference/io_service.html) and I can not get that to work.

    They suggest that instead of posting a doRead as you do before starting the threads you create a io_service::work object with the io_service as a reference. But if I do that I can post write operations to the io_service without problems but I never get back in the doRead function (which I do if I use your trick with posting a doRead in the beginning).

    Do you know why the suggest boost way of doing it is not working here?

    What I’ve done is adding the following changes:
    To the pimpl class I’ve added the work member:
    boost::shared_ptr keepAliveWork;
    And in the open function I’ve replaced your line:
    m_pimpl->io.post(boost::bind(&AsyncSerial::doRead, this));
    with this one:
    m_pimpl->keepAliveWork.reset(new boost::asio::io_service::work(m_pimpl->io));

    and it does not work (as I thought it would…)!? I’d be really happy to know if you have any idea why this does not work.

  32. fedetft Says:

    Well, if I understood correctly what you’re trying to do, the behaviour you’re seeing is perfectly normal.
    The way asio is built, is that you tell it to do something asynchronously, and get a callback when the work is completed.
    So, if you post a write request on a serial port (or socket), the code returns immediately, and after it has sent the data you get the callback.
    The same is for reading: you post a read request, and when data arrives, the callback is called; the fact that there can be a long time between the read request and when data actually arrives is unimportant. If you don’t post a read request, you’re basically telling asio you don’t expect to receive anything, and of course get no callback if data is received.
    The code in the documentation to prevent io_service::run() from returning is useful when there can be times where no operation is being performed, but on a serial port it is never the case, you usually want it to always look for incoming data.

    • johpe Says:

      Thank you very much for your reply. As I understand it from your response the boost::asio::io_service::work(m_pimpl->io) call will put work in the io_service::run() function so it is always busy and will not be able to serve any other post then?

      I’ve seen a lot of other people pasting code above so I will try it as well to give you the full picture of the changes I’ve done to the AsyncSerial class (and its pimpl).

      /// Size defines.
      static const int IO_BUFFER_SIZE = 512;
      static const int READ_CIRCULAR_BUFFER_SIZE = 1024;

      class AsyncSerialImpl : private boost::noncopyable
      {
      public:
      AsyncSerialImpl() :
      io(), port(io),
      open(false), error(false)
      {
      writeQueue.reserve(IO_BUFFER_SIZE);
      receivedDataBuffer.set_capacity(READ_CIRCULAR_BUFFER_SIZE);
      }
      /// IO Service object.
      boost::asio::io_service io;
      /// Serial port object.
      boost::asio::serial_port port;

      /// Background thread for writing.
      boost::shared_ptr backgroundWriteThread;
      /// Background thread for reading.
      boost::shared_ptr backgroundReadThread;

      /// Initial work for the io service object so its run function wont return.
      boost::shared_ptr keepAliveWork;

      /// To know if the port is open.
      bool open;
      /// To know if we’ve had an error.
      bool error;
      /// Mutex to protect the error flag.
      mutable boost::mutex errorMutex;
      /// Data is queued here before it goes in writeBuffer.
      std::vector writeQueue;
      /// Data being written to port.
      boost::shared_array writeBuffer;
      /// Size of writeBuffer.
      size_t writeBufferSize;
      /// Mutex to protect the writeQueue.
      boost::mutex writeQueueMutex;
      /// Data being read from port.
      char readBuffer[IO_BUFFER_SIZE];
      /// Size of data most recently read from port.
      size_t bytes_received;
      /// Growing circular buffer storing data read from port.
      boost::circular_buffer receivedDataBuffer;
      /// Mutex to protect the receivedDataBuffer.
      boost::mutex receivedDataMutex;
      /// Read complete callback.
      boost::function callback;
      };

      AsyncSerial::AsyncSerial() :
      m_pimpl(new AsyncSerialImpl())
      {
      }

      AsyncSerial::AsyncSerial(const std::string& devname,
      unsigned int baud_rate,
      boost::asio::serial_port_base::parity opt_parity,
      boost::asio::serial_port_base::character_size opt_csize,
      boost::asio::serial_port_base::flow_control opt_flow,
      boost::asio::serial_port_base::stop_bits opt_stop) :
      m_pimpl(new AsyncSerialImpl)
      {
      open(devname, baud_rate, opt_parity, opt_csize, opt_flow, opt_stop);
      }

      void AsyncSerial::open(const std::string& devname,
      unsigned int baud_rate,
      boost::asio::serial_port_base::parity opt_parity,
      boost::asio::serial_port_base::character_size opt_csize,
      boost::asio::serial_port_base::flow_control opt_flow,
      boost::asio::serial_port_base::stop_bits opt_stop)
      {
      if (isOpen()) {
      close();
      }

      setErrorStatus(true); // If an exception is thrown, error_ remains true
      m_pimpl->port.open(devname);
      m_pimpl->port.set_option(boost::asio::serial_port_base::baud_rate(baud_rate));
      m_pimpl->port.set_option(opt_parity);
      m_pimpl->port.set_option(opt_csize);
      m_pimpl->port.set_option(opt_flow);
      m_pimpl->port.set_option(opt_stop);

      // This gives some work to the io_service before it is started
      m_pimpl->keepAliveWork.reset(new boost::asio::io_service::work(m_pimpl->io));

      // Create two threads (read and write) running the io_service::run function.
      m_pimpl->backgroundWriteThread =
      boost::shared_ptr(
      new boost::thread(boost::bind(&boost::asio::io_service::run,
      &m_pimpl->io)));
      m_pimpl->backgroundReadThread =
      boost::shared_ptr(
      new boost::thread(boost::bind(&boost::asio::io_service::run,
      &m_pimpl->io)));

      setErrorStatus(false); // If we get here, no error
      m_pimpl->open = true; // Port is now open
      }

      What I’ve done is adding a io_service::work object and I use two thread instead. The two threads I’ve added since I was thinking that there might actually be reading and writing at the same time on a full-duplex serial port, I don’t know if it makes sense?? Then there is the io_service::work object which actually seems to be blocking all subsequent posts to the io_service!?? The rest of the implementation is basically the same as the one that you had. Any more insights to this would be greatly appreciated, thanks…

  33. fedetft Says:

    I think I’m missing why you’ve decided to modify my serial class. I don’t understand if you’re simply tring to learn asio, or you’ve found a use case where my original class doesn’t work and you’re trying to fix it (in this case, I’d like to know exactly what doesn’t work).

    “the boost::asio::io_service::work(m_pimpl->io) call will put work in the io_service::run() function so it is always busy and will not be able to serve any other post then? ”

    That’s not true. You can post multiple tasks to an io_service and they will be executed at the same time, transaprently, without the need to expicitly create threads. That’s why I start the io_service with the task of reading from the serial port directly in the constructor. I can later on post the task of writing data to the serial port (when user code calls a write member function) without disturbing the serial port read operation being carried on in the same io_service, at the same time. That’s the advantage of the asynchronous programming model.

    • johpe Says:

      The reason behind modifying your serial class was mostly to learn a bit about asio and serial ports and because I wanted to understand the code before actually using it in a spare-time project of mine including z-wave devices. I’ve also added a circular buffer to store data read from the serial port before the ui/main thread fetches the data for processing/displaying. And since I’m using Qt as well, it is only the main thread that is allowed to modify ui components so I’ve also added a context synchronization object to make sure the callback actually reports in the correct thread context.

      What I have failed to read (or understand) in the somewhat lacking boost documentation is if its even at all necessary to create threads for the io_service? Since the calls to read and write are asynchronous there shouldn’t really be any need for one thread let alone two threads? I added a second thread thinking that I might receive and send data at the same time, but maybe that is not really needed, even though I can see from debug printing thread ids that the two threads actually take turns on servicing the write and read calls (so booth are being used).

  34. fedetft Says:

    Perfect, now that I know what you’re tring to do it’s easier to answer 🙂
    You’re correct when you say that there’s no need to use threads with io_services, but as long as you can live with the main process blocking within the io_service::run() member function. Take for example this code:

    void event_1() { /*do stuff*/ }
    void event_2() { /*do stuff*/ }
    […]
    void event_n() { /*do stuff*/ }

    int main()
    {
    io_service io;
    /* register event handlers 1..n with the io_service. They may handle serial ports, timers, sockets, whatever… */
    io.run(); //BLOCKING call
    }

    If you can structure your application like this, with a setup phase and then a blocking call to io_service::run(), you can do everything asynchronously without using threads.
    However, think about my serial class, it wouldn’t have been a good idea to design it in a way that blocks the control flow, right? So I’ve added a thread and let the io_service::run() run in the separate thread. That’s it, it’s the only reason why there’s a thread in the code.
    Also, you’re right when you say that when multiple threads call run() on the same io_service, both are used. In fact, when only one thread calls run(), the callbacks are called from that thread only. This means that if two events happen simultaneously, their callbacks can’t be called at the same time, they will be serialized by the io_service. That’s an useful guarantee as it solevs thres safety issues, but in very high performance applications you might want multiple callbacks to execute in parallel. To do so, simply call run() from N threads, effectively making a thread pool capable of serving up to N requests in parallel.

    Hope everything is more clear now.

  35. Jerome Says:

    Hello,

    Thanks a lot for your great job.

    I try to use your QT class to dialog with a pic microcontroler.
    The communication as to be made in hex.
    I made some test and I modified the “void MainWindow::on_pushButton_clicked()” fonction just to test if I can send à command

    // the command I have to send (to turn on a motor)
    static const QChar command[] = {
    0x0002, 0x000b, 0x00FF, 0x0044, 0x0000, 0x0000, 0x0000, 0x0000, 0x00fe, 0x00ff, 0x0003};
    // size of the command
    int size = sizeof(command) / sizeof(QChar);
    // QString to be transmited
    QString test(command, size);
    serial.write(test);

    The command is send correctly, I have a good result on the microcontroler, the motor is running.

    The pic send a message every 600 ms. I have to receive this king of message
    0x0002 0x0022 0x00FF 0x00F1 0x0000 0x00FF 0x00FF 0x000A 0x0000 0x0000 0x0050 0x0050 0x0000 0x0000 0x00CA 0x0000 0x0032 0x0000 0x0020 0x0003 0x00C8 0x0000 0x00C8 0x0000 0x0000 0x0000 0x00F4 0x0001 0x00FE 0x0003

    But on the serial GUI nothing happened?

    Could you help me?

    Best regards,

    Jérôme

  36. fedetft Says:

    As a first look, it may be because the QAsyncSerial buffer serial port data on a line basis. So if your pic never sends a \n after each message, it never emits any signal and nothing appears on screen. Can you do a test sending a \n to confirm that the problem is this (not necessarily using the pic, in case you can’t modify the firmware on it to make it output a newline)?

    • Jerome Says:

      Thanks a lot for your answer. Indeed I put a /n at the end of the command send by the pic and it is working fine. Thank you very much it help me

      I have another question. I build your sample in debug mode without any problem. The program is fully working.
      But now I tried on release mode and it is not linking, I have the following warning :

      d:/developpements/qt/2010.05/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../libboost_thread.a(thread.o):thread.cpp:(.text+0x3c11): undefined reference to `std::out_of_range::~out_of_range()’

      d:/developpements/qt/2010.05/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../libboost_thread.a(thread.o):thread.cpp:(.text+0x49e6): undefined reference to `std::out_of_range::~out_of_range()’

      d:/developpements/qt/2010.05/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../libboost_thread.a(thread.o):thread.cpp:(.text$_ZN5boost9gregorian9bad_monthD1Ev[boost::gregorian::bad_month::~bad_month()]+0xb): undefined reference to `std::out_of_range::~out_of_range()’

      d:/developpements/qt/2010.05/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../libboost_thread.a(thread.o):thread.cpp:(.text$_ZN5boost9gregorian9bad_monthD0Ev[boost::gregorian::bad_month::~bad_month()]+0x12): undefined reference to `std::out_of_range::~out_of_range()’

      d:/developpements/qt/2010.05/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../libboost_thread.a(thread.o):thread.cpp:(.text$_ZN5boost9gregorian8bad_yearD1Ev[boost::gregorian::bad_year::~bad_year()]+0xb): undefined reference to `std::out_of_range::~out_of_range()’

      d:/developpements/qt/2010.05/mingw/bin/../lib/gcc/mingw32/4.4.0/../../../libboost_thread.a(thread.o):thread.cpp:(.text$_ZN5boost9gregorian8bad_yearD0Ev[boost::gregorian::bad_year::~bad_year()]+0x12): more undefined references to `std::out_of_range::~out_of_range()’ follow

      collect2: ld returned 1 exit status

      mingw32-make[1]: *** [release\SerialGUI.exe] Error 1

      mingw32-make: *** [release] Error 2

      Le processus “D:/Developpements/Qt/2010.05/mingw/bin/mingw32-make.exe” s’est terminé avec le code %2.
      Erreur à la compilation du projet SerialGUI (cible : Bureau)
      Lors de l’exécution de l’étape ‘Make’

      Can you give me any advice to avoid that

      Thanks a lot

      Jérôme

  37. fedetft Says:

    I haven’t encountered errors like that, so unfortunately I don’t know how to help. It looks like when linking, some function within the boost libraries can’t find some other function in libstdc++. Try a make clean; make, or reinstall the boost libraries, but these are just guesses.

    • Jerome Says:

      Thanks for your reply, Itried what you said. I also tried on another computer. It is the same.
      What is your configuration?
      I am on PC computer, windows Xp 32bit, QTSDK (QT 4.7.3 for desktop, MinGW 4.4)
      I also use as you say the MinGW-distro

      Thanks a lot

      Jérôme

  38. fedetft Says:

    Well, I mostly use Linux, and I’m not an expert of compiling under Windows. I compiled the serial port examples on a Win XP virtual machine using MinGW distro, and they worked, so I have no idea about how to reproduce your link time bug, nor how to fix it.

  39. Jerome Says:

    Ok I finally found, it was a mixing of code.
    Thanks for your help.

    I have a new question. My communication works well. I use a usb – RS232 converter.
    My GUI is used to program several boards. During the programmation, my colleague who do the job, forgot sometimes to close the port before change the board and the GUI crashed. It is due to the fact that the port is not close properly.
    Is it possible to know if the communication is broken and to reset the boost library or something like that

    Thanks a lot

    Jérôme

    • Edinson Says:

      Hello Jerome, I have the same linking problem like you show. I got the same warning message when try on realease mode. Can you give me any advice to avoid that

      Thanks a lot…..

      Edinson Aedo

  40. fedetft Says:

    It may be that the call to QAsyncSerial::write() throws an exception if the port is not open. Do a quick test to see if it’s the case, for example by making a simple program that repeatedly writes to the serial port and disconnect the USB to serial adapter while it is writing.
    If it is the case, you can fix it in two ways:
    1) guard calls to QAsyncSerial::write() with an
    if(serial.errorStatus()==false)
    {
    serial.write(s);
    } else {
    //Show a popup to the user saying the serial port has closed unexpectedly
    }
    2) even better, catch the C++ exception
    try {
    serial.write(s);
    } catch(exception& e) {
    //Show a popup to the user saying the serial port has closed unexpectedly
    }

  41. Jerome Says:

    Thanks for your help. It is working well.
    I made a dialog box which warn that the port has closed unexpectedly and I close the complete application. The user need to restart the application to have access to the board.

    Is it possible to just make a kind of reset of the port; I tried to close the port after the cable was unplug but I catch a mistake

    Regards

    Jérôme

  42. fedetft Says:

    I tried, but it looks like it’s something inside the boost libraries.

  43. Barry Callahan Says:

    I have a couple questions:
    1) Might this boost bug report have any impact on the apple-specific conditional blocks?

    2) I’m planning a project where the serial communication will be primarily challenge-response, which would seem to be a perfect application for the TimeoutSerial class. BUT I’ll also need to be able to monitor the port for any spontaneous incoming traffic and deal with it accordingly, which would be more appropriate for AsyncSerial. I was wondering if you’d be able to suggest an approach.

    I could open the port in an instance of AsyncSerial for monitoring, and closing it there and reopening it in a TimeoutSerial class when I need to poll the device at the other end of the port for info. But that seems kind of silly.

    Another idea I had was to modify CallbackAsyncSerial. Add an optional boolean argument to the write* functions to indicate whether they should return immediately, or wait until pimpl->writeQueue is empty. Of course, that still leaves implementing analogs of TimeoutSerial’s read*() functions.

  44. Barry Callahan Says:

    grr. Typed a not-an-email-address in the form in the above comment.

    Also, it should be noted that while I’m planning on working with Qt, I’d much rather see the functionality added at the boost::asio level that way it’s available even if you’re not using Qt.

  45. fedetft Says:

    Sorry for the late reply, I’ve been a bit busy and didn’t check my blog for some time.
    For the Mac OSX issue, I’m currently no longer using this OS, so I’d need someone to test it.

    For the issue of a partly challenge-response and partly async communication, I don’t recommend closing and opening different implementations of the serial port classes, you’d risk missing some bytes sent in the short time while the port is closed.
    I’m currently developing code that uses asio directly, and I must admit it’s pretty comfortable once you get in the async-coding-style mindset. So my advice would be to instantiate an io_service, a serial_port and a deadline_timer directly in your code. When sending data you could call write() on the port, and immediately after start the timer for the timeout. In the meantime always keep an active call of async_read_some() or async_read_until() so that when data comes, either as a reply to one of your command, or “unexpected” you get it.
    If you can’t put blocking code in your main, just spawn a thread and execute the io_service::run() call there. Also in this case asio helps you with the io_service::post() method to cause, from your main thread, a call of a callback in the context of the background thread, so that often explicit thread synchronization isn’t required at all.

  46. Anonymous Says:

    Hello.

    I particulary interested in your application for Qt Gui, but the code isn’t yet available for download.
    Should you do something for it ?

    Thanks

  47. Bringing Star Trek to life: LCARS home automation with Arduino and Raspberry Pi #piday #raspberrypi @Raspberry_Pi « adafruit industries blog Says:

    […] usb connection. This gave me the answer: http://www.webalice.it/fede.tft/seria&#8230; (part 5) and https://fedetft.wordpress.com/2010/04/&#8230; . Several connected clients can talk to each other via the interface and share for example their […]

  48. LCARS home automation with Arduino and Raspberry PI: QT redesign | SECURITY EXPO Says:

    […] usb connection. This gave me the answer: http://www.webalice.it/fede.tft/seria&#8230; (part 5) and https://fedetft.wordpress.com/2010/04/&#8230; . Several connected clients can talk to each other via the interface and share for example their […]

  49. loki Says:

    Hi fedetft,

    I know it’s been a while you post it but I’m using your excellent project for doing a multithreading channels com devices. Thanks for sharing it. It lighted me the shadow path of using boost asio asynch. There’s just a point where I’m bit confused. BTW I’m far behind to be an expert and I’m still blocking in many things in programmation so appologies for my dumps questions.
    In your project I’m doing Qdialog windows for settings the serial port. When we instanciate the QAsynchSerial in background of the UI Qt we just set the namedevice and the baudrate. Could we set the other param of the serial even if the are initialize by default in the AsyncSerial Constructor ? aren’t there side effect on the behave the buffer ? Could we made the AsyncSerial abstract ?

    • fedetft Says:

      If I understood correctly, you want to set other serial port parameters, such as flow control and parity? You could just extend the open() member function in QAsyncSerial to take additional parameters, and maybe add default values so as to remain compatible with code that just passes the first two. Then in the implementation of open() in QAsyncSerial, just pass those parameters to the CallbackAsyncSerial open() member function.

  50. loki Says:

    thanks a lot for your quick reply fedetft. Yes this is my aim and launch at least several threads for reading serials ports. I was doing something else like instanciat at first the QAsyncSerial object in the mainwindow constructor and making setter in QAsyncSerial to set Parity stopbits flowcontrol and dataBits but it seems to be not a good way.

  51. loki Says:

    BTW I change a small issu you got for higher version Qt (Qt 5.6) :
    Line 75 in QAsyncSerial pimpl->receivedData+=QString::fromAscii(data,size);
    to
    pimpl->receivedData+=QString::fromLatin1(data, size);

    The fromAscii become obsolete has changed by fromLatin1

  52. William JCM Says:

    Hello, fedetft.

    I am currently using your serial port classes in a school project, and I have two questions.

    I’m making a GUI program that needs to communicate with a CAN bus via a RS-232/CAN converter.

    Nodes on the CAN bus could send data to the computer at any time, so I created a thread to handle serial communication.

    I’m using wxWidgets as my GUI toolkit, and I can create and send events to the main event loop at any time, including from a thread.

    Question one: which class should I use, between AsyncSerial and CallbackAsyncSerial ?

    Question two: if CallbackAsyncSerial is the better choice, where would I need to add the callback method ?

    Thanks in advance for your answer.

    • fedetft Says:

      Long ago I used wxWidgets a little bit, but I don’t remember threading considerations about it. If you use CallbackAsyncSerial, the callback is called by a background thread, and that callback needs to post an event to the main thread in a thread-safe way. Did you have a look at the Qt version? Maybe you can replicate the same approach for integrating serial ports and wx.

  53. Elektron Says:

    Dear fedetft,

    I really enjoyed your comprehensible explanation.

    Unfortunately, the links to the sources are down. Therefore, it is not anymore possible to play with our examples.

    Would you mind to upload them again.

    Best Regards
    Elektron

  54. Elektron Says:

    Well, meanwhile I have found the project on github: https://github.com/fedetft/serial-port

Leave a reply to fedetft Cancel reply