KDE Frameworks 5 Tech Preview released, with updated ThreadWeaver
Today, the KDE Community released a tech preview of the upcoming KDE 5 Frameworks, the new, modularised incarnation of what was previously distributed simply as the KDE libraries. The new frameworks are drop-in extensions to Qt applications, with minimal and well-documented dependencies for easier deployment. The tech preview contains two frameworks that are marked as mature, namely KArchive and ThreadWeaver. The updated ThreadWeaver was my major piece of library coding work in 2013, and was finished just in time for the release. Even though it is a tech preview, it is stable, and no major (or even significant but minor) changes in the current API are expected until the final release. Programmers are already encouraged to use it, and provide feedback and bug reports.
ThreadWeaver is a concurrent execution scheduler written in C++. Available for all target platforms of the Qt framework, including desktop, mobile and embedded environments, ThreadWeaver delivers concurrent execution of tasks, load balancing with regard to user-defined criteria, multiple independent queues, processing graph modelling, aggregate jobs and other comprehensive features. As all other KDE frameworks, ThreadWeaver is Free Software. Its only dependency is Qt, which makes it a tier 1 framework in KDE’s lingo.1
A number of the new features of ThreadWeaver were announced at Akademy 2013. Jobs, the unit of concurrent execution in ThreadWeaver, are now managed by the queue using shared pointers, meaning that auto-delete behaviour is implicit and controlled by the user. Helper templates are available to queue stack or member variables, so allocation of jobs can be static or dynamic. Functors or lambda functions can be used to construct jobs. Job aggregates like collections and sequences now execute their own run() method before queueing their elements, so that aggregates can generate their own elements. Success and queueing state of jobs are now integrated into a single status. Jobs can signal the result of execution by setting a status, but also using exceptions, simplifying error reporting in more complex job classes. Jobs can be decorated, and no more inherit QObject by default. Decorators can be used to add signals, change priorities or modify just about any behaviour of jobs independently of the actual job class used. The construction of the global queue can now be customised using a queue factory. The QueueStream API greatly simplifies queueing jobs with a familiar iostream-like C++ syntax.
ThreadWeaver follows the Unix idiom of doing one thing, and doing it right. Similar to how small Unix programs can be combined to create an practically infinite space of computing solutions, ThreadWeaver offers itself to programmers as an add-on module with minimal dependencies. Including it extends an application with concurrent scheduling capability. But the same Unix idiom is also applied in a second sense. Within ThreadWeaver, a few basic concepts – jobs and their aggregates, queues and policies – are implemented that again provide simple building blocks that can be combined creatively, offering a vast space of potential solutions within the scope of the application.
The history of ThreadWeaver goes back to KDE 3. The idea of implementing a thread pool based execution scheduler that manages dependencies between jobs was implemented as a proof of concept using Qt 3. However it turned out to be difficult to implement and use because of the lack of thread-safe reference counting of the implicitly shared classes at the time. These fundamental problems have been solved with the release of Qt 4. Additionally, the introduction of cross-thread signal-slot connections further simplified the communication between jobs and the application’s user interface. The first production ready version of ThreadWeaver was released as part of KDELibs with KDE 4.0. For KDE Frameworks 5, it was almost completely re-written to simplify memory management of jobs, make use of new Qt 5 features like atomic variables, and in part to reflect new language constructs in C++11 like lambda functions. ThreadWeaver comes with an extensive set of unit tests that all pass in the tech preview (hear, hear).
In the following weeks and months, the framework will be polished and debugged based on user feedback. Also, a series of posts here on this blog will introduce individual ThreadWeaver concepts and features in depth, mostly based on example programs, including contrasting it to thread handling in Qt using QThread or Qt Concurrent. ThreadWeaver is very close to production quality, having been tested continuously in the last couple of months. There may still be smaller, source compatible changes to the framework. We ask interested programmers out there to provide feedback and bug reports to make ThreadWeaver what it should be – a worry-free, easy to use and powerful add-on to Qt that programmers enjoy using. Have fun!
