Overview
In this project, I explore NPR (Non-Photorealistic Rendering) techniques with a custom post process shader aiming to create the effect of brushstrokes on a model in realtime. I aimed to create a shader which was able to create the appearance of a painted scene utilizing artistic techniques with shaders in realtime. For research, I used refrence papers Paint By Numbers: Abstract Image Representations by Paul Haeberli [1] and Amy and Bruce Gooch's A Non-Photorealistic Lighting Model for Automatic Technical Illustration [2]. I implemented the NPR shaders in Opengl using C++, GLFW, and GLEW, as well as the boilerplate provided by Opengl Tutorial [3].
A showcase of the result of my work is shown below. Beneath that, I have some documentation of my progress on the project and how I achieved my results. Finally, I also speculate on alternate avenues for achieving artisitc effects through shaders. Here is the Github link for the project.
Video Showcase
Process
Setup
This being one of my first major projects with OpenGL rather than webGL, I began first by learning OpenGL. For this I used the online resource opengl-tutorial.org, which I can highly recommend as it provides all the necessary basics you might need for getting started with any OpenGL project. Since there's a lot that goes into a build system (and since I didn't want to spend time debugging build problems), I used the CmakeLists and project structure provided by the tutorial website. In the end, this added a lot of bloat to my project structure, so for anyone begining with OpenGL, I recommend using this skeleton code provided by Tomasz Galaj for setting up the build system for your own projects.
Lighting
The tutorial's code provided all the necessary baseline code for rendering 3D objects with a camera. It also provided a simple lighting model Phong Illumination. However, Phong Illumination did not achieve the artistic appearance that I wanted, so I instead used the Gooch Illumination model.
The Gooch Illumination model works, at a high level, by taking what would normally be in darkness and adding cool tones to the model, while adding warm tones to the illuminated side. This is what artists normally do in their work, and for my rendering, it resulting in a more pleasing appearance that is softer on the harsh shadows of the Phong lighting model.
Brush Stokes
So now, I had to decide on how to approach the problem of rendering brushstrokes in realtime for my shader. I had initially thought about drawing individual brushstrokes on a texture using samples from the scene rendered to texture, however, doing this would be difficult to do in parallel on the GPU, as each brush stroke would have to be layered on the same shared texture. In addition, this would also create a new brush texture each frame, sure to be extrememly expensive.
Creating a static frame of brush strokes simplified the problem significantly. I would create a series of quads in screen space that each had a brush stroke texture. I created the quads on the CPU with a set of parametrizable variables, such as BRUSH_SIZE, BRUSH_SIZE_VARIANCE, ANGLE, ANGLE_VARIANCE, and COLOR_VARIANCE. Brush size is pretty self explanatory, being the size of an individual brush stroke. Angle is the angle at which the brush strokes would be applied on the screen. Color variance is the range at which a random amount of color would be added to brush stroke. Variance for size and angle also added a random amount to the brush size and angle within a given range.
To render the result, I simply had to render the scene I had previously to texture, then read from that texture in the brush stroke shaders. To make sure each brush stroke had fairly uniform color, I would sample the rendered texture only at the vertices of the brush stroke. The results are shown below.
As you can see, this resulted in some strange shadow-like artifacts on the final render. This effect was particularly apparent with larger brush sizes. I found that the problem came from some brushstrokes in the lower right having a majority of the brush stroke quad with the background color value, but one or two values of the object itself. This resulted in a blended color between the two for a brush stroke. A somewhat poor solution to this problem, I simply discarded all fragments that had a color close to the background color.
After fixing that, the shader really came together. Playing with the parameters to the brush strokes a bit, I came to some good looking renders:
Results
