Scene Graph - OpenGL Under QML

The OpenGL under QML example shows how an application can make use of the QQuickWindow::beforeRendering() signal to draw custom OpenGL content under a Qt Quick scene. This signal is emitted at the start of every frame, before the scene graph starts its rendering, thus any OpenGL draw calls that are made as a response to this signal, will stack under the Qt Quick items.

As an alternative, applications that wish to render OpenGL content on top of the Qt Quick scene, can do so by connecting to the QQuickWindow::afterRendering() signal.

In this example, we will also see how it is possible to have values that are exposed to QML which affect the OpenGL rendering. We animate the threshold value using a NumberAnimation in the QML file and this value is used by the OpenGL shader program that draws the squircles.

The example is equivalent in most ways to the Direct3D 11 Under QML, Metal Under QML, and Vulkan Under QML examples, they all render the same custom content, just via different native APIs.

 class Squircle : public QQuickItem
 {
     Q_OBJECT
     Q_PROPERTY(qreal t READ t WRITE setT NOTIFY tChanged)
     QML_ELEMENT

 public:
     Squircle();

     qreal t() const { return m_t; }
     void setT(qreal t);

 signals:
     void tChanged();

 public slots:
     void sync();
     void cleanup();

 private slots:
     void handleWindowChanged(QQuickWindow *win);

 private:
     void releaseResources() override;

     qreal m_t;
     SquircleRenderer *m_renderer;
 };

First of all, we need an object we can expose to QML. This is a subclass of QQuickItem so we can easily access QQuickItem::window(). We expose it to QML using the QML_ELEMENT macro.

 class SquircleRenderer : public QObject, protected QOpenGLFunctions
 {
     Q_OBJECT
 public:
     ~SquircleRenderer();

     void setT(qreal t) { m_t = t; }
     void setViewportSize(const QSize &size) { m_viewportSize = size; }
     void setWindow(QQuickWindow *window) { m_window = window; }

 public slots:
     void init();
     void paint();

 private:
     QSize m_viewportSize;
     qreal m_t = 0.0;
     QOpenGLShaderProgram *m_program = nullptr;
     QQuickWindow *m_window = nullptr;
 };

Then we need an object to take care of the rendering. This instance needs to be separated from the QQuickItem because the item lives in the GUI thread and the rendering potentially happens on the render thread. Since we want to connect to QQuickWindow::beforeRendering(), we make the renderer a QObject. The renderer contains a copy of all the state it needs, independent of the GUI thread.

Lets move on to the implementation.

 Squircle::Squircle()
     : m_t(0)
     , m_renderer(nullptr)
 {
     connect(this, &QQuickItem::windowChanged, this, &Squircle::handleWindowChanged);
 }

The constructor of the Squircle class simply initializes the values and connects to the window changed signal which we will use to prepare our renderer.

 void Squircle::handleWindowChanged(QQuickWindow *win)
 {
     if (win) {
         connect(win, &QQuickWindow::beforeSynchronizing, this, &Squircle::sync, Qt::DirectConnection);
         connect(win, &QQuickWindow::sceneGraphInvalidated, this, &Squircle::cleanup, Qt::DirectConnection);

Once we have a window, we attach to the QQuickWindow::beforeSynchronizing() signal which we will use to create the renderer and to copy state into it safely. We also connect to the QQuickWindow::sceneGraphInvalidated() signal to handle the cleanup of the renderer.

         // Ensure we start with cleared to black. The squircle's blend mode relies on this.
         win->setColor(Qt::black);
     }
 }

The default behavior of the scene graph is to clear the framebuffer before rendering. This is fine since we will insert our own rendering code after this clear is enqueued. Make sure however that we clear to the desired color (black).

 void Squircle::sync()
 {
     if (!m_renderer) {
         m_renderer = new SquircleRenderer();
         connect(window(), &QQuickWindow::beforeRendering, m_renderer, &SquircleRenderer::init, Qt::DirectConnection);
         connect(window(), &QQuickWindow::beforeRenderPassRecording, m_renderer, &SquircleRenderer::paint, Qt::DirectConnection);
     }
     m_renderer->setViewportSize(window()->size() * window()->devicePixelRatio());
     m_renderer->setT(m_t);
     m_renderer->setWindow(window());
 }

We use the sync() function to initialize the renderer and to copy the state in our item into the renderer. When the renderer is created, we also connect the QQuickWindow::beforeRendering() and QQuickWindow::beforeRenderPassRecording() to the renderer's init() and paint() slots.

 void Squircle::cleanup()
 {
     delete m_renderer;
     m_renderer = nullptr;
 }

 class CleanupJob : public QRunnable
 {
 public:
     CleanupJob(SquircleRenderer *renderer) : m_renderer(renderer) { }
     void run() override { delete m_renderer; }
 private:
     SquircleRenderer *m_renderer;
 };

 void Squircle::releaseResources()
 {
     window()->scheduleRenderJob(new CleanupJob(m_renderer), QQuickWindow::BeforeSynchronizingStage);
     m_renderer = nullptr;
 }

 SquircleRenderer::~SquircleRenderer()
 {
     delete m_program;
 }

In the cleanup() function we delete the renderer which in turn cleans up its own resources. This is complemented by reimplementing QQuickWindow::releaseResources() since just connecting to the sceneGraphInvalidated() signal is not sufficient on its own to handle all cases.

 void Squircle::setT(qreal t)
 {
     if (t == m_t)
         return;
     m_t = t;
     emit tChanged();
     if (window())
         window()->update();
 }

When the value of t changes, we call QQuickWindow::update() rather than QQuickItem::update() because the former will force the entire window to be redrawn, even when the scene graph has not changed since the last frame.

 void SquircleRenderer::init()
 {
     if (!m_program) {
         QSGRendererInterface *rif = m_window->rendererInterface();
         Q_ASSERT(rif->graphicsApi() == QSGRendererInterface::OpenGL);

         initializeOpenGLFunctions();

         m_program = new QOpenGLShaderProgram();
         m_program->addCacheableShaderFromSourceCode(QOpenGLShader::Vertex,
                                                     "attribute highp vec4 vertices;"
                                                     "varying highp vec2 coords;"
                                                     "void main() {"
                                                     "    gl_Position = vertices;"
                                                     "    coords = vertices.xy;"
                                                     "}");
         m_program->addCacheableShaderFromSourceCode(QOpenGLShader::Fragment,
                                                     "uniform lowp float t;"
                                                     "varying highp vec2 coords;"
                                                     "void main() {"
                                                     "    lowp float i = 1. - (pow(abs(coords.x), 4.) + pow(abs(coords.y), 4.));"
                                                     "    i = smoothstep(t - 0.8, t + 0.8, i);"
                                                     "    i = floor(i * 20.) / 20.;"
                                                     "    gl_FragColor = vec4(coords * .5 + .5, i, i);"
                                                     "}");

         m_program->bindAttributeLocation("vertices", 0);
         m_program->link();

     }
 }

In the SquircleRenderer's init() function we start by initializing the shader program if not yet done. The OpenGL context is current on the thread when the slot is invoked.

 void SquircleRenderer::paint()
 {
     // Play nice with the RHI. Not strictly needed when the scenegraph uses
     // OpenGL directly.
     m_window->beginExternalCommands();

     m_program->bind();

     m_program->enableAttributeArray(0);

     float values[] = {
         -1, -1,
         1, -1,
         -1, 1,
         1, 1
     };

     // This example relies on (deprecated) client-side pointers for the vertex
     // input. Therefore, we have to make sure no vertex buffer is bound.
     glBindBuffer(GL_ARRAY_BUFFER, 0);

     m_program->setAttributeArray(0, GL_FLOAT, values, 2);
     m_program->setUniformValue("t", (float) m_t);

     glViewport(0, 0, m_viewportSize.width(), m_viewportSize.height());

     glDisable(GL_DEPTH_TEST);

     glEnable(GL_BLEND);
     glBlendFunc(GL_SRC_ALPHA, GL_ONE);

     glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

     m_program->disableAttributeArray(0);
     m_program->release();

     m_window->endExternalCommands();
 }

We use the shader program to draw the squircle in paint().

 int main(int argc, char **argv)
 {
     QGuiApplication app(argc, argv);

     QQuickWindow::setGraphicsApi(QSGRendererInterface::OpenGL);

     QQuickView view;
     view.setResizeMode(QQuickView::SizeRootObjectToView);
     view.setSource(QUrl("qrc:///scenegraph/openglunderqml/main.qml"));
     view.show();

     return QGuiApplication::exec();
 }

The application's main() function instantiates a QQuickView and launches the main.qml file.

 import QtQuick
 import OpenGLUnderQML

 Item {

     width: 320
     height: 480

     Squircle {
         SequentialAnimation on t {
             NumberAnimation { to: 1; duration: 2500; easing.type: Easing.InQuad }
             NumberAnimation { to: 0; duration: 2500; easing.type: Easing.OutQuad }
             loops: Animation.Infinite
             running: true
         }
     }

We import the Squircle QML type with the name we registered in the main() function. We then instantiate it and create a running NumberAnimation on its t property.

     Rectangle {
         color: Qt.rgba(1, 1, 1, 0.7)
         radius: 10
         border.width: 1
         border.color: "white"
         anchors.fill: label
         anchors.margins: -10
     }

     Text {
         id: label
         color: "black"
         wrapMode: Text.WordWrap
         text: qsTr("The background here is a squircle rendered with raw OpenGL using the 'beforeRender()' signal in QQuickWindow. This text label and its border is rendered using QML")
         anchors.right: parent.right
         anchors.left: parent.left
         anchors.bottom: parent.bottom
         anchors.margins: 20
     }
 }

Then we overlay a short descriptive text, so that it is clearly visible that we are in fact rendering OpenGL under our Qt Quick scene.

Example project @ code.qt.io