Encouraged by some recent posts on Qt Centre I decided to point out a common issue when dealing with Qt model-view framework.

Many people often treat models as a burden that needs to be carried to be able to display some data using Itemviews. One needs to perform a tedious task of implementing the QAbstractItemModel interface or filling one of the ready made models such as QStandardItemModel with data. Most often the model needs to represent some data which is contained in some structure and it is not rare to spot people write code similar to this one:

struct MyStruct {
  int x;
  int y;
};
 
class Model : ... {
  //...
  QList <MyStruct> export() const { ... }
  void import(const QList <MyStruct> &){  ... }
  //...
};

What happens here is that they provide export and import functions to convert between their own storage format and the model. It’s a similar thing if they instantiate QStandardItems and feed it with data from some datasource. In both cases the data is duplicated – it is kept both in the model and in the external container.

Instead the model should be perceived only as an interface to an existing data – no matter if it is kept inside or outside the model object, a good example being QSqlTableModel and family – however the data is cached within the model, it is really kept on some remote server.

Instead of the above snippet of code, the skeleton could look like the following:

class Model : public QAbstractItemModel {
public:
  Model(QList<MyStruct> *dat, QObject *parent=0)
  : QAbstractItemModel(parent){
    m_data = dat;
  }
  QVariant rowCount(const QModelIndex &parent) const {
    if(parent.isValid() || m_data.isNull()) return 0;
    return m_data->count();
  }
  //...
private:
  QPointer<QList <MyStruct> > m_data;
};

As you can see the model operates on an external list of items and as long as the list is valid, it will return correct data. Of course the same applies to the rest of the interface – one can add rows and modify data kept in the list.

The only problem that remains is that the list can be modified outside the model, causing all components using the model (views included) to not update themselves upon changes in the list. Fortunately there are ways to overcome it.

The first good solution is to use a data source that can emit changes made to it using Qt signals. Then one can connect those signals to the model to be able to emit proper signals from the model to its environment.

The second solution is to disallow access to the data source outside the model, but then it makes more sense to embed the source within the model itself, either with “has-a” or “is-a” relation (remember, multiple inheritance is a beautiful thing 😎 ). It is probably the best solution if you fully control the data source and you can modify the code that uses it. It is worth mentioning that you don’t have to use the methods provided by QAbstractItemModel to access the data if it feels odd or difficult in your situation. You can define custom methods to access the data through the model – both in read and write mode (for example using MyStruct as the carrier). As long as you emit proper model signals from those methods, your views will continue to work properly.

The third option is to trigger manual or periodic updates of the model – for flat models things like row and column count can be cached and compared against the real source. If they are out of sync, one can try emiting one of generic signals – layoutChanged() or modelReset() to save the situation. Just remember that calling modelReset() will cause all your views to loose selections and item visibility. For simpler models (or better data sources) it might be possible to trace exact changes in the source and emit proper signals upon detecting changes.

Remember, the more complex the data gets, the easier to fall out of sync when using two separate data structures. Save yourself time and nerves and use a single data set instead of two whenever you can, even if it doesn’t seem obvious from the start how to do it. A good design always pays off.

The Christmas break came and I have finally found some free time to finish some little projects and experiments I started some time ago and I decided to publish preliminary (yet working) results of one of them.

The problem

Qt4 supports the model-view paradigm through the Interview framework, but unfortunately it doesn’t handle models which store data on some remote host. “Hey, of course it can operate on models with remote datasets – just take a look at the SQL models available!” – one might say and basically this is true. But this is just a partial truth – have you tried using those models (for example QSqlTableModel) on a large dataset, let’s say… with thousand records holding some image and textual data? The problem with SQL models is that when the model initialises (when select() is called) it retrieves all the data from the SQL DBMS and blocks the GUI until all of the data is ready. This is not a big problem with local databases – 1k records of 50kB of data each gives 50MB of data that needs to be transfered – over a Fast Ethernet (100Mbps) network it’ll take not more than a few seconds, but try doing the same using a 1Mb network link… After 6 minutes your model will be ready and your application could continue.

Possible solutions

The problem would be easy to overcome if one of two things were true – either QSqlQuery would work asynchronously or QSqlTableModel would begin with an empty model, execute the query in the background and fill the model after the data is ready. The result of both of these solutions would be simmilar – the data would be added to the model dynamically as it is retrieved from the distant server.
Unfortunately QSql*Model classes don’t support such behaviours. In general the same problem applies to all data sources that are blocking.
The proper solution to the problem is to use a thread or set of threads that fetch the data in the background and insert it into the model using signals and slots. Because of queued connections across threads, updating the model is thread safe and it doesn’t block the GUI thread.

As a proof of concept I implemented a subclass of QAbstractTableModel that uses a thread and a simple updater object to transmit data between the application and a distant data storage. The concept uses signals and custom events to do the job and developers only need to implement the updater object and call proper methods in the model.

How does it work?

Here is an example implementation of a remote model:

class MyModel : public RemoteTableModel {
public:
  MyModel() : RemoteTableModel(){
    setUpdaterFactory(new MyUpdaterFactory()); // terrible hack here
    start(); // start filling the model
  }
  int columnCount ( const QModelIndex & parent = QModelIndex() ) const{
    if(parent.isValid()) return 0;
    return 2; // two column flat model
  }
  int rowCount( const QModelIndex &parent = QModelIndex() ) const {
    if(parent.isValid()) return 0;
    return m_rows.count();
  }
  QVariant data ( const QModelIndex & index, int role = Qt::DisplayRole ) const{
    if(!index.isValid() || role!=Qt::DisplayRole) return QVariant();
    int row = index.row();
    if(row>=m_rows.count()) return QVariant();
    if(index.column()==1){
      return m_rows.at(row).title;
    } else {
      return m_rows.at(row).tid;
    }
  }
protected:
  void addRow(const QVariant &data){
    // add a row retrieved from a remote data source
    beginInsertRows(QModelIndex(), m_rows.size(), m_rows.size());
    QVariantList vlist = data.toList();
    m_rows << st(vlist.at(0).toString(), vlist.at(1).toInt());
    endInsertRows();
  }
private:
  /**
   *  Internal data structure
   */
  struct st {
    st(QString t, int i){ title = t; tid = i; }
    QString title; // column 1 data
    int tid;       // column 0 data
  };
  QList m_rows; // data container
};

One also needs to implement an updater object that will do the actual fetching and storing. You can see an example in the tar bundle attached at the end of this post. Basically what it does is to fetch a list of threads in QtCentre’s Qt Programming forum using QtCentre’s archive features over the HTTP protocol. QHttp works asynchronously so I could implement that particular example directly without using threads too, but it’s just an example – I use it successfully to fetch and store data in a SQL database, but I don’t have a public database to use in an example, so the HTTP example has to suffice for now.

Next thing that needs to be done is to split the data transfer functionality from the model interface so that it could be used for hierarchical models or even some other things as well and to clean the implementation a little
(now I’m using an ugly hack to prevent creating a QObject in the context of a wrong thread).

Example code

Today I posted the third chapter of the GLModel article series. You can find a link to the article on the sidebar of the blog.

Today, motivated by a comment on my blog and Johan’s blog post, I have published the next article about making a custom Qt model that holds information about objects in a 3D scene. This article aims to show how to create an internal representation of data used by the model.