Archive for the ‘Programming’ Category

Remote models

The Christmas break came and I have finally found some free time to finish some little projects and experiments I started some time ago and I decided to publish preliminary (yet working) results of one of them.

The problem

Qt4 supports the model-view paradigm through the Interview framework, but unfortunately it doesn’t handle models which store data on some remote host. “Hey, of course it can operate on models with remote datasets – just take a look at the SQL models available!” – one might say and basically this is true. But this is just a partial truth – have you tried using those models (for example QSqlTableModel) on a large dataset, let’s say… with thousand records holding some image and textual data? The problem with SQL models is that when the model initialises (when select() is called) it retrieves all the data from the SQL DBMS and blocks the GUI until all of the data is ready. This is not a big problem with local databases – 1k records of 50kB of data each gives 50MB of data that needs to be transfered – over a Fast Ethernet (100Mbps) network it’ll take not more than a few seconds, but try doing the same using a 1Mb network link… After 6 minutes your model will be ready and your application could continue.

Possible solutions

The problem would be easy to overcome if one of two things were true – either QSqlQuery would work asynchronously or QSqlTableModel would begin with an empty model, execute the query in the background and fill the model after the data is ready. The result of both of these solutions would be simmilar – the data would be added to the model dynamically as it is retrieved from the distant server.
Unfortunately QSql*Model classes don’t support such behaviours. In general the same problem applies to all data sources that are blocking.
The proper solution to the problem is to use a thread or set of threads that fetch the data in the background and insert it into the model using signals and slots. Because of queued connections across threads, updating the model is thread safe and it doesn’t block the GUI thread.

As a proof of concept I implemented a subclass of QAbstractTableModel that uses a thread and a simple updater object to transmit data between the application and a distant data storage. The concept uses signals and custom events to do the job and developers only need to implement the updater object and call proper methods in the model.

How does it work?

Here is an example implementation of a remote model:

class MyModel : public RemoteTableModel {
public:
  MyModel() : RemoteTableModel(){
    setUpdaterFactory(new MyUpdaterFactory()); // terrible hack here
    start(); // start filling the model
  }
  int columnCount ( const QModelIndex & parent = QModelIndex() ) const{
    if(parent.isValid()) return 0;
    return 2; // two column flat model
  }
  int rowCount( const QModelIndex &parent = QModelIndex() ) const {
    if(parent.isValid()) return 0;
    return m_rows.count();
  }
  QVariant data ( const QModelIndex & index, int role = Qt::DisplayRole ) const{
    if(!index.isValid() || role!=Qt::DisplayRole) return QVariant();
    int row = index.row();
    if(row>=m_rows.count()) return QVariant();
    if(index.column()==1){
      return m_rows.at(row).title;
    } else {
      return m_rows.at(row).tid;
    }
  }
protected:
  void addRow(const QVariant &data){
    // add a row retrieved from a remote data source
    beginInsertRows(QModelIndex(), m_rows.size(), m_rows.size());
    QVariantList vlist = data.toList();
    m_rows << st(vlist.at(0).toString(), vlist.at(1).toInt());
    endInsertRows();
  }
private:
  /**
   *  Internal data structure
   */
  struct st {
    st(QString t, int i){ title = t; tid = i; }
    QString title; // column 1 data
    int tid;       // column 0 data
  };
  QList m_rows; // data container
};

One also needs to implement an updater object that will do the actual fetching and storing. You can see an example in the tar bundle attached at the end of this post. Basically what it does is to fetch a list of threads in QtCentre’s Qt Programming forum using QtCentre’s archive features over the HTTP protocol. QHttp works asynchronously so I could implement that particular example directly without using threads too, but it’s just an example – I use it successfully to fetch and store data in a SQL database, but I don’t have a public database to use in an example, so the HTTP example has to suffice for now.

Next thing that needs to be done is to split the data transfer functionality from the model interface so that it could be used for hierarchical models or even some other things as well and to clean the implementation a little
(now I’m using an ugly hack to prevent creating a QObject in the context of a wrong thread).

Example code

GLModel part 3

Today I posted the third chapter of the GLModel article series. You can find a link to the article on the sidebar of the blog.

QUiViewer

I have written a small application today which aims at viewing UI files from Qt4. Nothing special, but can be very usefull. I based the idea on KUIViewer program which allows the user to view Qt3 UI files (unfortunately it crashes when fed with an UI from Qt4).

You can get the app tarball here.

Syntax highlighter

Today my thoughts about making syntax highlighting for my blog came back to me. As my blog is based on bash scripts, I needed an application that would do the actual highlighting. I know there exists a plugin for nanoblogger that enables syntax highlighting, but it’s fair from being mature. So I sat down and implemented
my own generic syntax highlighter. It’s not complete yet, but it already does some highlighting and it only took about an hour to implement. Qt was very helpful, as usual :)

Testing the highlighter should include trying to parse the code of the highlighter itself and so I did. You can see the result below.

  1. #include
  2. #include
  3. #include
  4. #include
  5. #include
  6. #include
  7. #include
  8. classQHighlighter{
  9. public:
  10. ��QHighlighter(QIODevice�*indevice);
  11. ��void�readConfig(QIODevice�*configdevice);
  12. ��QString�parse();
  13. protected:
  14. ��QString�isKeyword(constQString�&);
  15. ��QString�parseNumber();
  16. ��QString�parseIdentifier();
  17. ��QString�parseString();
  18. ��QString�parseComment();
  19. ��QString�parsePreproc();
  20. ��QString�parseSLComment();
  21. ��void�makeLines(bool�count=false);
  22. ��QIODevice�*_indevice;
  23. ��bool�_perror;
  24. ��QString�output;
  25. ��QMap<QString,�QStringList>�keywords;
  26. ��
  27. };
  28. void�QHighlighter::readConfig(QIODevice�*configdevice){
  29. ����QDomDocument�doc;
  30. ����if(!doc.setContent(configdevice)){
  31. ��������qDebug(“CONFIG ERROR”);
  32. ��������return;
  33. ����}
  34. ����QDomElement�root�=�doc.documentElement();
  35. ����for(QDomElement�keywselem�=�root.firstChildElement(“keywords”);�
  36. ��������!keywselem.isNull();�
  37. ��������keywselem�=�keywselem.nextSiblingElement(“keywords”))
  38. ����{
  39. ��������QStringList�kw;
  40. ��������for(QDomElement�key�=�keywselem.firstChildElement(“keyword”);�
  41. ������������!key.isNull();
  42. ������������key�=�key.nextSiblingElement(“keyword”))
  43. ��������{
  44. ������������kw�<<�key.text().trimmed();
  45. ��������}
  46. ��������QDomElement�keystyle�=�keywselem.firstChildElement(“style”);
  47. ��������QString�kstyle�=�keystyle.text().replace(“$”,�“%”).replace(“[“,�“<").replace(“]–>,�“>”);
  48. ��������keywords[kstyle]�=�kw;
  49. ����}
  50. }
  51. QString�QHighlighter::isKeyword(constQString�&s){
  52. ��foreach(QString�k,�keywords.keys()){
  53. ����const�QStringList�&slist�=�keywords[k];
  54. ����foreach(QString�el,�slist){
  55. ������if(QRegExp(el).exactMatch(s))�return�k.arg(s);
  56. ����}
  57. �}
  58. return�s;
  59. }
  60. QHighlighter::QHighlighter(QIODevice�*indevice){
  61. ��_indevice�=�indevice;
  62. ��_perror�=�false;
  63. }
  64. void�QHighlighter::makeLines(bool�countLines){
  65. ��if(!countLines)�output�=�output.replace(“\n”,�
    );
  66. ��else�{
  67. ����QString�n�=�
      “;

    1. ����foreach(QString�line,�output.split(“\n”)){
    2. ������n+=QString(
    3. %1
    4. \n”).arg(line);

    5. ����}
    6. ����n+=

    ;

  68. ����output�=�n;
  69. ��}
  70. }
  71. QString�QHighlighter::parse(){
  72. ��QString�current;
  73. ��while(!_indevice->atEnd()){
  74. ����char�c�=�_indevice->peek(1).at(0);
  75. ����QChar�ch(c);
  76. ����if(ch.isNumber())�{
  77. ������current�=�parseNumber();
  78. ������output�+=QString(%1).arg(current);
  79. ����}�elseif(ch.isLetter()�||�ch==‘_’)�{
  80. ������current�=�parseIdentifier();
  81. ������output�+=�isKeyword(current);
  82. ����}�elseif(c==‘”‘||�c==‘\”�)�{
  83. ������current�=�parseString().replace(“<“,�“<“);
  84. ������output�+=QString(%1).arg(current);
  85. ����}�elseif(c==‘/’�&&�_indevice->peek(2)==“/*”)�{
  86. ������current�=�parseComment().replace(“<“,�“<“);
  87. ������output�+=QString(%1).arg(current);
  88. ����}�elseif(c==‘/’�&&�_indevice->peek(2)==“//”){
  89. ������current�=�parseSLComment().replace(“<“,�“<“);
  90. ������output�+=�QString(%1).arg(current);
  91. ����}�elseif(c==‘#’){
  92. ������current�=�parsePreproc().replace(“<“,�“<“);
  93. ������output�+=�QString(%1).arg(current);
  94. ����}�elseif(c==‘<‘){
  95. ����_indevice->getChar(&c);
  96. ������output�+=�“<“;
  97. ����}�else�{
  98. ������_indevice->getChar(&c);
  99. ������if(c!=‘ ‘)
  100. ������output�+=�c;
  101. ������else�output�+=“�”;
  102. ����}
  103. ��}
  104. ��makeLines(true);
  105. ��return�output;
  106. }
  107. QString�QHighlighter::parsePreproc(){
  108. ��return�_indevice->readLine();
  109. }
  110. QString�QHighlighter::parseSLComment(){
  111. ��return�_indevice->readLine();
  112. }
  113. QString�QHighlighter::parseNumber(){
  114. ��QString�result;
  115. ��char�c;
  116. ��while(!_indevice->atEnd()){
  117. ����if(!_indevice->getChar(&c)){
  118. ������_perror�=�true;
  119. ������return�result;
  120. ����}
  121. ����QChar�ch(c);
  122. ����if(ch.isNumber()�||�c==‘.’){
  123. ������result+=c;
  124. ����}�else�{
  125. ������_indevice->ungetChar(c);
  126. ������return�result;
  127. ����}
  128. ��}
  129. ��return�result;
  130. }
  131. QString�QHighlighter::parseComment(){
  132. ��QString�result;
  133. ��char�c=‘\0′;
  134. ��char�prev=‘\0′;
  135. ��while(!_indevice->atEnd()){
  136. ����if(!_indevice->getChar(&c)){
  137. ������_perror�=�true;
  138. ������return�result;
  139. ����}
  140. ����// match */
  141. ����if(c==‘/’�&&�prev==‘*’){
  142. ��������result�+=c;
  143. ��������return�result;
  144. ����}
  145. ����prev�=�c;
  146. ����result+=c;
  147. ��}
  148. ���_perror�=�true;
  149. ��return�result;
  150. }
  151. QString�QHighlighter::parseIdentifier(){
  152. ��QString�result;
  153. ��char�c;
  154. ��while(!_indevice->atEnd()){
  155. ����if(!_indevice->getChar(&c)){
  156. ������_perror�=�true;
  157. ������return�result;
  158. ����}
  159. ����QChar�ch(c);
  160. ����if(ch.isLetter()�||�ch.isDigit()�||�c==‘_’�||�c==‘:’){
  161. ������result+=c;
  162. ����}�else�{
  163. ������_indevice->ungetChar(c);
  164. ������return�result;
  165. ����}
  166. ��}
  167. ��return�result;
  168. }
  169. QString�QHighlighter::parseString(){
  170. ��QString�result;
  171. ��char�c;
  172. ��char�sep;
  173. ��_indevice->getChar(&sep);
  174. ��result+=sep;
  175. ��while(!_indevice->atEnd()){
  176. ����if(!_indevice->getChar(&c)){
  177. ������_perror�=�true;
  178. ������return�result;
  179. ����}
  180. ����if(c==‘\\’){
  181. ������if(!_indevice->getChar(&c)){
  182. ��������_perror�=�true;
  183. ��������return�result;
  184. ������}
  185. ������result+=‘\\’;
  186. ������result+=c;
  187. ����}�elseif(c!=sep)�
  188. ������result+=c;
  189. �����else�{
  190. ������// _indevice->ungetChar(c);
  191. ��������result+=c;
  192. �������return�result;
  193. �����}
  194. ��}
  195. ��_perror�=�true;
  196. ��return�result;
  197. }
  198. int�main(int�argc,�char�**argv){
  199. ��QFile�infile(argv[1]);
  200. ��if(!infile.open(QFile::ReadOnly))�return2;
  201. ��QHighlighter�h(&infile);
  202. ��QFile�config(“h.config”);
  203. ��if(!config.open(QFile::ReadOnly))�return2;
  204. ��h.readConfig(&config);
  205. ��QString�result�=�h.parse();
  206. ��QFile�outfile(“hh.html”);
  207. ��outfile.open(QFile::WriteOnly);
  208. // outfile.write(”
    ");

  209. ��outfile.write(result.toLocal8Bit());
  210. // outfile.write(“”);

  211. ��return0;
  212. }

  213. ]]>

Designer Tutorial

Some time ago I created and published a video tutorial for Qt4 Designer. Nothing funky, some small speaking problems, but hopefully next tutorial will be better.

I received a good feedback from people who have seen the tutorial before I published it, so hopefully someone will find my work useful.

You can download the tutorial from my webpage.

GLModel part 2

Today, motivated by a comment on my blog and Johan’s blog post, I have published the next article about making a custom Qt model that holds information about objects in a 3D scene. This article aims to show how to create an internal representation of data used by the model.

Charts 0.2

I’ve been sick for a few days now and I don’t feel like doing anything serious, so I have a lot free time to do “other things”.

I started thinking and finally decided that I should release an update to my MVC charts. I wrote the updated class some time ago but never put it on the Web. Now it’s time to do that. Nothing funky, I added some stuff for manipulating the chart title and axis descriptions. Unfortunately something weird causes charts to eat up all my cpu cycles and I can’t find the reason. I even tried profiling the code but without success. Oh well… maybe I’ll go back to the application and try to trace what’s going on. Better that than lying in bed whole day.

You can download the bundle from my website.

OpenGL Model articles

I decided to write a series of articles about implementing a complete data model based on Qt4 model-view framework called Interview which will hold object data from 3D scenes for use with OpenGL (a series of articles about implementing an OpenGL view in Qt4 will follow).

First two articles are ready:

If you have any commments or requests about those or future articles, drop me a line here or on QtCentre.