Sunday, January 24, 2016

QNanoPainter - a brief update

QNanoPainter has been now publicly available for about a month and during this time has seen some nice improvements.

Gunnar first added Windows support, then worked on reducing QNanoQuickItemPainter creation time and memory consumption with multiple items. Niels and Jean-Michaƫl did some performance testing on Linux & NVIDIA shield, results discussed in issue 6. Examples Gallery was restructured and now contains 3 examples: QNanoPainter Features, Mouse Event Boxes, and Freehand Painting.

Here’s what they look on Nexus 6:

(best viewed in HD)

So, if you are working with Qt5 and have a need for custom QQuickItems, I warmly recommend checking out QNanoPainter. Thanks to everyone involved!

Friday, December 25, 2015

QNanoPainter Available!

About 2.5 months ago I introduced QNanoPainter project for implementing custom QQuickItems. We’ve been really busy (and continue to be so) with other things but took now the time to push QNanoPainter publicly available for wider audience in here:

It’s not ready (as software never is), but already quite capable for serious kicking of tires. Just clone it, build with Qt Creator / qmake and run the provided examples. And when you want to try implementing own items, straightforward instructions to get started are available at front-page README.

If you dig it and want to assist in improving QNanoPainter, here are few ideas for patches:

  • We have tested QNanoPainter on OS X, Linux, Android and iOS. What’s missing at least is making it work on Windows (with dynamic OpenGL build of Qt).
  • Documentation with QDoc has been started but it’s not complete. There are plenty of ”TODO: Write more documentation here” comments in classes and documentation could overall be improved and styled.
  • Current NanoVG version used is from fork at bfbac9a35e, with some additions. Custom additions are marked with ”// ### Added”. Some reasons to go with this fork for now instead of upstream are discussed in What should be done is to update NanoVG to some latest version, apply additions and test properly. Maybe own NanoVG fork with additions should be maintained?
  • Instead of stb_truetype and stb_image code could be ported to use Qt for images and texts. I think at least text rendering would require private Qt headers and additions in Qt side, so may not be that straightforward.
  • Bugfixes!

We received some good comments already and hopefully now more & patches when sources are publicly available. I’ll write some more blog posts later, but now.. Clone!

Sunday, October 4, 2015

Introducing QNanoPainter

Qt World Summit is just about to start! I’m not participating this time, but wanted to contribute one-more-thing to discuss in UI groups at the heart of Berlin ;)

During the past about six months we at QUIt Coding have had a side-side-project called QNanoPainter. This library is designed for implementing custom QQuickItems into Qt5 scene graph. Currently if you want to implement custom Qt Quick item, options are at least:

  • QQuickItem: Offers best performance due to Qt5 scene graph integration. But QSG* classes are relatively low-level with vertices, indices & materials. Painting more complicated things requires quite an expert and even then productivity is not very high.
  • QQuickFramebufferObject: This is a good option if you want to draw with OpenGL into QQuickItem. But as we all know, OpenGL is also quite low-level API so no productivity wins here.
  • QQuickPaintedItem: This means painting with good old QPainter C++ API. Weakness of QPainter with modern graphics accelerated hardware is that you either get good antialiased quality (QImage rendering target) or fast OpenGL painting (FBO rendering target) but not both. Qt raster engine is very good CPU backend but GPUs are ruling the world these days. And QPainter OpenGL backend doesn’t itself support antialiasing, it requires multisampling with GL_EXT_framebuffer_multisample and GL_EXT_framebuffer_blit extensions as explained here by Laszlo. In case you set FBO target and antialiasing on with hardware that doesn’t support these (e.g. Nexus6 and iPhone6), target with automatically switch back to QImage & raster with performance dipping on most devices.
  • QtQuick Canvas: Canvas item is implemented on top of QPainter so it has mostly same cons and pros as QQuickPaintedItem. HTML5 Canvas API is nice and popular but there is some overhead from (QPainter) API mismatch and JavaScript especially if more drawing data needs to be exported from C++ side.
  • QML ShaderEffect: As you can’t really mess with the mesh, this approach suits for simple cases where fragment shader is enough. But we are now thinking a bit more complicated items where this option isn’t enough or at least becomes complicated and slow.

So isn’t these options enough, still need something else? Yes, I think so.

QNanoPainter is our novel approach for trying to offer performance, productivity and rendering quality all in one. In a nutshell QNanoPainter is:

  • Excellent NanoVG OpenGL library as a drawing backend with some patches for performance & features.
  • QNanoPainter C++ API on top of NanoVG (which is written with C). This API is mixture of QPainter and HTML5 canvas APIs. Offers vector drawing, images and text rendering with classes such as QNanoPainter, QNanoFont, QNanoImagePattern, QNanoLinearGradient etc.
  • QNanoQuickItem to implement items into Qt Quick scene graph. Basically it’s like QQuickPaintedItem, but instead of QPainter offers QNanoPainter API for drawing.
  • Some examples like QNanoPainter vs. QPainter demo and Gallery, more to come.

One thing we have tried is to implement demo with same painting result using QPainter (QQuickPaintedItem) and QNanoPainter (QNanoQuickItem). Performance comparison is always tricky (and it’s not just about performance), but with comparable antialiased painting we are seeing 3-10 times better performance with QNanoPainter depending on hardware, content, item sizes etc. Here’s a short video showing this demo running on iPhone6 and Nexus6 with all tests on (iPhone6 in fact painting everything twice to get more juice out of it):

(best viewed in HD)

QNanoPainter is still work-in-progress and not ready for releasing. But we would like to hear what others think: Which (of the above) options do you use for custom QML items? Have you had issues finding suitable option for some use-cases? If yes, what features have you missed and would like to have?

Just comment here, tweet with #qnanopainter or email to info{at}, thanks!

Sunday, March 9, 2014

QML and Recursive Shaders

Let's say that you want your Qt5/QML UI to contain sort of a graph. Instead of presenting just the current value, your want user to see few seconds into history how the value has changed. Something like this:

To implement the above component with Qt5 & QML, there are at least 3 possibilities:
1) Use QML Canvas and draw the graph using JavaScript.
2) Use QQuickPaintedItem and draw the graph using C++ QPainter APIs.
3) Use QQuickItem and draw using OpenGL and QSG* helpers.

First of these is "easiest", no C++ is required but performance and features go along HTML5 canvas API. Second one is good for those familiar with QPainter, but it also has a small performance cost as rendering goes either through QImage or FBO. Third QQuickItem option performs the best and is optimal for Qt5 scene graph, but working with scene graph QSG* classes and OpenGL can feel a bit too low level.

As the title of this blog post suggest, there is also a candidate number 4: Use a recursive shader.

QML ShaderEffectSource provides a property "recursive" which doesn't seem to be widely known. Story goes that Kim #1 asked Kim #2 to provide this feature and it was implemented during summer 11' with this commit. But none of the Qt examples (even now at Qt 5.2.1) demonstrate the usage so it got lost somehow... until now, my precious *gollum* *gollum*.

When setting the recursive property to true, ShaderEffectSource allocates an extra texture to store the rendering result and this texture can then be used as a source for ShaderEffect. Effectively this means that instead of rendering into a clean sheet, ShaderEffect can utilize the previous frame. And this of course opens up many possibilities.

Our component which demonstrates recursive shader usage is called 'QUItMeterComponent'. As explained, recursive ShaderEffectSource needs to allocate an extra texture of its size. Keeping the texture small increases performance and reduces graphics memory usage, so this sounds like a perfect opportunity to utilize modified LedScreen component presented earlier. As each pixel equals to a led, it allows presenting small source (left) as a nice ledscreen (right).

Now we just add a bit sugar with a mirror effect and feed our QUItMeterComponent with QtSensors accelerometer data. Outcome is G-force app which looks in Nexus4 like this:

If you think this component might suit your needs and/or want get a small example of using recursive ShaderEffectSource, sources of this G-force app are available from:

Thursday, October 24, 2013

QUIt demos on Android

Qt 5.2 Beta was released on yesterday, whey! As many of us know already, Qt 5.2 will be the first Qt version with an official Android support. Qt 5.2 will also contain quite a big changes on the UI side, with the new V4 JavaScript engine and the Scene Graph renderer. So how well does this beta run existing Qt Quick 2.0 applications on Android?

We have made numerous UI demos in the past so I decided to install Qt 5.2 beta into Nexus 7 (2013) and put it into test! Here's video:

Overall I'm impressed by the quality. Qt Creator is able to setup Android projects with ease, things work and performance is very good. What more could one ask?

I encourage everyone to test this beta and report bugs so we'll get things close to perfection before 5.2.0 release, due by the end of November. Keep calm and hack on!

Monday, May 13, 2013

Qt5 Battery Component

After the QUItIndicator trilogy which introduced idea, design and performance of a specific Qt5 QML component there's room for more, right?! Something like this:

This time we have a dynamic QML component for showing the remaining power of your mobile device battery. As a recap, with "Dynamic QML component" I mean someting which utilizes not only basic QML animation properties (position, opacity, scale etc.) but also new Qt5 features (shaders, particles, scenegraph powa!) to appear "more dynamic". Maybe it's just me, but I would love to see UIs really utilizing modern GPUs... and accelerating this progress is one of the reasons why I code these examples and blog about them. Another reason being to rule-the-world, obviously ;-P

Instead of explaining design & features of QUItBattery component I'll let this video to do that:

If you want to use this liquid battery component in your UI: Download the sources from here, copy QUItBatteryComponent directory, import it in your QML and off you go. Happy hacking!

Thursday, May 2, 2013

QUItIndicators: Performance considerations

(This is part III, please check also parts I and II)

Even with a small component like this, there are plenty of possibilities to improve (or sink) the performance. To make sure that our indicators perform as expected, we'll test them on Nokia N9 and and on Raspberry Pi.

These both devices are relatively low-end by current standards. N9 contains 1GHz Cortex-A8 CPU which is still quite beefy, but GPU (SGX530) on the other hand is getting old and unable to handle more complicated fragment shaders. For RPi these are just the opposite: CPU is slowish 700MHz ARM11 while the GPU (VideoCore IV) is more performant than N9 SGX530. Because of these qualities (and because both support Qt5, naturally) this duo is excellent for our performance ensurement.

So here's a short video showing how our indicators perform on Nokia N9 and on RaspberryPi:

Performs pretty OK on both, right? ProgressIndicator stress test isn't smooth on RPi, which indicates that its CPU can't handle 100 indicators animated like that. So stress test does what it's supposed to, normal use cases perform well.

Even with declarative QML language, good performance doesn't just happen, you need to work towards it. By looking at the sources of these indicators, at least these tips/notes can be made:

  • BusyIndicator and ProgressIndicator are separate components instead of just one component with e.g. "indeterminate" property. This allows using a more light-weight (only one texture, animating only vertex shader, less properties etc.) BusyIndicator component with indeterminate use-cases. Lesson to learn is to avoid making too general/bloat QML components.
  • There are four different sizes for indicators, from small (64x64px) to huge (512x512px). These sizes were selected because power-of-two texture sizes are more optimal for GPU. SourceSize property is used to scale and cache exactly the correct sized textures.
  • BusyIndicator animation is achieved purely on vertex shader. As vertex shader is run once for every vertex instead of once for every fragment (pixel), it can be much more GPU-friendly. In case of an 256x256px indicator, our vertex shader is executed over 650(!) times less than fragment shader.
  • To keep the amount of vertices as small as possible while making sure animation looks still smooth, the mesh size is allocated based on indicator size. GridMesh resolution for 256x256 indicator is 10x10 while 64x64 indicator manages with a 4x4 mesh.
  • When using items&images as sources for ShaderEffect and not needing the original one, remember to set its visibility to false to prevent it from rendering.
  • ProgressIndicator can show percentages in the center and applies vertex animation also for it, which requires that the whole Item is used as a source for ShaderEffect. Initialize of Item as a ShaderEffect source is slightly slower than initialize of Image. This is because Items (with their child Items) need to be rendered into FBO first while Image textures are instantly available. When percentages are disabled (showPercentages: false), Image is used directly and stress test of 162 ProgressIndicators starts pretty instantly. So if you need instantly appearing ProgressIndicators, don't show percentages on them.
  • Minimize the amount of property changes. Property bindings in QML are so easy to make that without paying attention you may forget to disable those when not needed. As an example, ProgressIndicator percentages Text element visibility was set to false when disabled and it wasn't even used as part of the shader source. But because of an oversight, its text property was still updated whenever ProgressIndicator value changed. That's bad, fixed now to update text only when showPercentages is true. Qt Creator QML Profiler is the tool to use to analyze your code.

So with all this, does it mean that these indicator components are fully optimized? Well of course not! There are still some generalization left and room for improvements. Additinally, although we have tried to offload most of the work from CPU to GPU, indicator animations like these should be run in a separate thread instead of the GUI thread. Only this would allow smooth 60fps even when processing under heavy CPU load. For details, please read this blog post by Gunnar:

Summing up briefly: When implementing QML components for applications, developers and designers should co-operate closely to bend the design to be as CPU&GPU friendly as possible while still making designers happy. Designers dig perfect pixels, but they also appreciate smooth 60fps. By making wise compromises and utilizing Qt5 correctly, we can deliver both.

Sources of QUItIndicator components & examples are available from: