RenderSpud

Cycles development

2011-12-28T21:25:00.000-08:00

I got recruited by an excited horde of blender artists to work on Cycles and give Brecht Van Lommel a little sidekick action. I was flattered they would ask. I will still be working on RenderSpud, although not as much; my wife reminded me that I have a fixed amount of time (and free time) in the day, so all of these fun projects get to jostle for that fixed-size pie.

A special thanks goes to my employer, Tippett Studio, for giving me permission to work on these projects. That flexibility is one reason why that's such an awesome place to work. Thanks, Phil, Jules, and Sanjay!

My progress on the requested Cycles features will be cataloged here.

Wallpaper attempt #1

2011-11-01T09:27:00.000-07:00

I saw a wallpaper that inspired me a little, so I decided to make one of my own.

I used Blender (with RenderSpud) to throw the scene together, but I noticed that Blender's default uv-sphere doesn't have UVs, and it shares the top and bottom vertex such that you can't get the proper texture coordinates for a latitude-longitude texture. I ended up writing a python script to generate an OBJ sphere with the right texture coordinates and separate vertices, and then importing that into Blender.

The lightprobe came from here; thanks Bernhard Vogl!

There are a couple of images, the first is 1920x1080, and the other is resized/cropped to be 1600x1200.

Feel free to use these, but for commercial users please credit these as rendered in RenderSpud.

Volumetric Rendering

2011-10-26T23:33:00.000-07:00

I recently joined the land of the living and got an android phone. I found a wallpaper that is a volumetric rendering of a mushroom cloud explosion, complete with pyroclastic noise. That wallpaper kept staring me down, so I finally started working on the volumetric rendering portion of RenderSpud for real (after a few false starts over the years).

The interface to the volumetric rendering consists in an integrator that can be assigned to the main renderer (e.g. a path tracer can have a single-scattering volumetric integrator attached to it), such that during main bounces from point to point the in-scattering, out-scattering, and emission can be taken into account along the ray's normal path. Then, when tracing shadow rays, the light can be attenuated by the transmission along the ray.

Volumetric primitives are given a special status so that they can be gathered out of the scene into a list early, and then the integrator can query them for intersections and scattering properties as it marches along rays. A volumetric material is also assigned that determines phase response (Henyey-Greenstein, Schlick, etc). The approach loosely matches PBRT's way of doing things.

Here is a progression of images as I worked on it this week, with an explanation of each.

The first is an image of single-scattering, with a volume filling the whole interior of the box. This was one of my first successful renders. The volume primitive is homogeneous, so my raymarch step size was pretty big (5% or so of the volume dimension) and I could still get away with it. I don't remember for sure, but I think it took an hour or so to render at 640x480.

The next is an emission-only integrator, so it essentially only takes into account reducing transmission and not any light scattering. As you can see, much is lost, but it could be useful for certain scenarios (and it's faster, as it has to trace no shadow rays).

For reference, here is the original scene with no volume. It didn't take long to render, maybe 15 minutes.

By this point, I had fixed a number of bugs and optimized some. This shows a still-homogeneous volume, but restricted to just a region. This was finding ray extents that actually intersect with the volume to reduce ray marching and the number of shadow rays. This also was running with a pretty small step size (about 0.5% of the volume extent), so it took over an hour to render.

Here I added some perlin-noise density that generates a spherical-displaced noise pattern into a 100x100x100 grid. There's no optimization on the grid access, and again I used a small step size. This took about 2.5 hours to render.

And this image represents me reaching my goal for the week: pyroclastic noise. I added a parameter to allow the step size when calculating only the light transmission to be larger; in the above case the step size was about 2% of the volume extent, and the transmission-only step size was 8% or so. This resulted in a render time of about 35 minutes, which isn't too bad.

You'll note there are some grid-like artifacts on the pyroclastic noise; you are seeing the 100x100x100 grid resolution, as I am doing no filtered lookups of the density values. I plan on making my grid primitive into a two-level grid, where the dense grids don't have to be loaded/generated unless they are hit by rays, and all the grid cells are laid out in cache-coherent tiles. Then, I'll do some trilinear interpolation between cells to help smooth out the cell-induced grid artifacts.

By the way, a great reference for generating and dealing with volumetric primitives and rendering can be had at the Siggraph 2011 Production Volume Rendering session page. Thanks, Magnus and Nafees for putting that all together this year!

Cycles car (in RenderSpud)

2011-07-09T18:38:00.000-07:00

Just a quick update: I finally got around to rendering Mike Pan's BMW series 3 coupe (which he used to show off Cycles, a future Blender renderer). I also rendered this out of Blender, with 1024 samples/pixel (probably overkill), using Mike Pan's studio lighting setup from a slightly older version of his blend file. It took about 2 hrs 40 minutes or so.

Thanks for the model, Mike!

More thin primitive rendering

2011-06-08T22:28:00.000-07:00

I still haven't come up with an appropriate curvature-based refinement for curves, but there are a handful of other bugs fixed. Render times are about 10% faster, and it now supports variable-width curves.

I fixed a completely different bug with multiple importance sampling and area lights that was throwing noise all over the scene (I can't believe how long I lived with that...) and rebalanced how my emission material is set up relative to lights and their power rating. Area-lit scenes look notably better.

Here is a rendering with fewer curves, but variable widths on the curves. At 640x480 with 256 samples/pixel, it took about 7 minutes:

If you look closely, you'll notice an artifact along the convex edges of the curves. I haven't yet figured that one out.

Thin primitive rendering (curves)

2011-05-31T00:15:00.000-07:00

Thin primitive rendering for geometry such as hair, fuzz, whiskers, and so forth requires a slightly different set of assumptions about how rays intersect than standard primitives. While thin primitives have a width to them, it is relatively minuscule compared to their length. A thin primitive is best represented as a curve; I have chosen cubic Bezier curves as the basis for RenderSpud's thin primitives (note, for simplicity they are regular beziers, not rational beziers, as of this writing). When viewing a thin primitive, generally its surface is oriented "towards the viewer" such that the surface normal is oriented to be perpendicular to the surface, but as close to facing the viewer as possible.

Here is a render with 1024 samples/pixel, with about 500 thin primitives (each refined into 20 cubic beziers). It took about 45 minutes for 640x480 resolution, which is way too long for my tastes, so I have some serious optimization work to do.

Refinement

RenderSpud also supports cubic NURBS curves decomposed into cubic beziers by knot insertion until each knot has multiplicity of three (see Dr. Shene's course notes, unit 6, for a decent reference on the algorithms involved for B-splines). Those cubic beziers are then further refined as needed; see below for details.

Bezier curves are tessellated by De Casteljau refinement, currently based on a fixed number of splits at predefined locations. I'm working on an appropriate curvature measure to split based on need, hopefully to reduce the number of primitives generated and put the data where it's needed. The final refined cubic beziers are then put in a bounding box and then into an acceleration structure (BVH, BIH, etc).

Ray Intersection

Ray intersection assumes that, once the curve is refined the segments between control points are close enough to the actual curve. We can just intersect with the lines connecting control points directly rather than doing any sort of ray-bezier intersection with Newton-Raphson root finding or anything of that nature. Note this means to find the nearest intersection for this refined bezier we must intersect with all three of its line segments, finding the nearest (unless we are doing shadow rays, in which case we bail after any is found).

The ray intersections with each segment proceed by finding the closest point of approach on the ray and on the segment, clipping to the ray's and the segments actual ranges. If that closest points distance is larger than the curve radius at that point it isn't an intersection.

While root finding can be more accurate and plenty fast, it can also be error prone (finding the wrong root) and expensive, especially in light of the fact that we are finding the nearest point of passage of a ray and the curve, and the curve has a width that may vary from place to place. Currently RenderSpud does not support a variable width per control point, but it will eventually.

At the actual intersection it is fairly trivial to compute dP/dv along the curve, as well as dP/du based on the width. The normal is back towards the incoming ray, but corrected to be perpendicular to dP/dv. Natural parametrization u,v are easy to compute as well based on distance from the actual curve segment (taking care to note which side the intersection is on) and the distance along the curve compared to its v-range. Each segment stores the v start and end from the original curve, so that curve refinement doesn't lose sight of what the parametrization of the original unrefined curve was.

More jellowy animation

2011-03-13T20:28:00.000-07:00

I couldn't help it, honest. (And jello is good stuff, especially when you mix it partly with cream cheese...mmm....but I digress.)

This was 128 samples/pixel path tracing at 854x480 (16:9 widescreen 480p), taking less than 5 minutes / frame to render on average. Of course, I have no render farm at home, so it took...a long time on my machine. There is a memory leak in the RenderSpud Blender plugin, and so I couldn't do the whole animation at once. I wrote a python script that would invoke Blender on the commandline (UI-less rendering) and process about 30 frames at a time.

The one artifact that bothers me is the grid-like specular flashes on the sides/top of some of the jello cubes near the beginning. I believe this is jitter from the soft-body simulation in Blender and is not from RenderSpud itself. Although, the lightprobe used for the background/lighting is pretty dang contrasty, so that probably contributed. In any case, it isn't wrong per se, but it is unnerving.

Jellowy

2011-03-08T22:59:00.000-08:00

Rendered with PyBlenderSpud (RenderSpud Blender plugin). 16 samples/pixel path tracing at 720x480, 15-20 seconds per frame to render the 70-frame sequence.

Yay, animation!

Everybody loves them some cars (part deux)

2011-03-05T17:34:00.000-08:00

With a few bug fixes and optimizations out of the way, I fixed up the Audi A5 render to be a little more...interesting. A set of texture coordinates on the ground object didn't make it for some reason that I'm still investigating, but putting the car into an environment (the Pisa lightprobe from Debevec's original set) helped out notably. The following was rendered via the Blender plugin and took about 50 minutes, at 1920x1080 with 256 samples/pixel:

Update: I fixed the texture mapping issue. I upgraded to OpenImageIO 0.9, and I turned off some of the texture system attributes I was imposing. It turns out that I was making thing extra difficult for OIIO to do its job. So, I took the mesh and turned it into a Catmull-Clark subdivision surface, added a little depth-of-field to integrate it into the background a little better, and I got a little better result:

Vector displacement

2011-02-26T23:13:00.000-08:00

Creating the illusion of having more mesh detail than you really do is a mainstay in computer graphics; a myriad of techniques are intended to do exactly that: bump mapping, normal mapping, height displacement, procedural mesh generation, and so forth. Even texture mapping itself technically is meant to provide the appearance of mesh detail that was never actually modeled. But the most advanced (and difficult) way to do so is vector displacement, where via procedural means or a map (or layers of maps; see here about a Siggraph talk where I helped setup the technology for an extreme example of this) one may displace a mesh into another nearly arbitrary and detailed mesh, as long as they cover the same S,T texture space.

I'm totally serious about that. As a test at Tippett Studio we literally displaced a sphere into a creature while animating it. This was in Pixar's Renderman, but I digress.

The actual technique is pretty simple. Given a mesh, you dice or tesselate it until each polygon is small enough (in Renderman, that means it takes up about a pixel or so on screen), and then displace each corner vertex in a displacement shader, where the displacement can be in any arbitrary direction, not just along the normal (which is what 'regular' displacement is). Then you render the final tiny polygons. Seems pretty straightforward, no?

But if you're here reading this blog, you already knew the devil is in the details. Tons of tiny polygons means lots of RAM used, perhaps too much to fit into RAM on a typical machine. Tons of tiny polygons and fine tesselation means you just spent a bunch of time running that displacement shader. Oh, and for a ray tracer, it means you have to be able to ray trace against all those polygons quickly. Plus, often in a ray tracer you don't know how big the polygons should be, because they could be magnified in a reflection or refraction, and many of them that would normally be displaced might never been seen at all. Seems pretty complicated, no?

I won't discuss how Renderman/REYES accomplishes all of this (although given their constraints and knowledge of what's going on at the pixel level, it's pretty slick), but instead I'll focus on ray tracing. For reference, a paper by Matt Pharr and Pat Hanrahan is an excellent item to have for this discussion.

Memory

First order of business: RAM. In a nutshell, you need two techniques: lazy displacement, and a cache of displaced polygons where you can drop out chunks of them when you need to make room for newly-displaced polygons. You can stream the old ones to disk, or just re-displace them as needed (I'll use the latter, as disk I/O is more complicated and potentially slower than just redoing the polygons, and the polygons may never be needed again anyway). The strategy for deciding which ones to nuke when you need space is an open question, but generally the tried-and-true least-recently-used (LRU) choice is a good one.

Speed

Second order of business: performance. The first thing you need to know is that barring any amazing insight or automatic computations, you don't know how far the mesh will be displaced. So you make the user specify displacement bounds, in my case a simple distance that indicates a radius of maximum displacement. Performance will be decent when the user specifies this bound well, and performance will positively suck rocks when they don't.

I place all of my original coarse mesh triangles into an acceleration structure (a BIH or BVH tree), where the bounds of each triangle are its regular bounds enlarged to add in the displacement radius. When a ray intersects one of these larger bounds, I check if I have tesselated that source triangle yet into its displaced counterparts. If not, I tesselate it to full resolution (based on a maximum edge length in world space), splitting each triangle into four as I go down using edge midpoints. See Pharr's paper for more information on when to split, and when to displace, as there are some subtleties here to avoid cracking. When that is done, I place it into its own acceleration structure (right now an agglomerative BVH). Then I can intersect the ray (and all future rays) against that acceleration structure relatively quickly.

Note that the test/lazily-displace action takes place inside of a mutex so that other render threads will only wait on the displacement if they particularly need that chunk of polygons. So far, I haven't had any trouble with thread contention that I can discern, so this seems to a fine approach.

Accuracy and Flexibility

Of course, the devil is in the details. I do not yet coalesce back together neighboring triangles that are flat enough as Pharr does. I also found that I needed to track all of the undisplaced vertices and normals along with the displaced versions as I tesselate and displace, because I also allow the displacement shader to modify the surface normals for shading.

The surface normals provide endless fun in glitch images when things go wrong. Does the original mesh have only per-face normals or non-smooth normals? You'll get surface cracking like crazy; do it with smooth normals or a subdivision surface (which implies smooth normals), or else you're probably doomed. Even with orignal smooth normals per-vertex, Pharr recommended just using the face normals after displacement was all done; but that precludes allowing the user to specify normals in their shader. Ideally, there should be the capability to, say, put a normal map on top of a displaced surface, but do it only at the displacement resolution. Here's what it looks like if I keep the undisplaced per-face normals, per Pharr's recommendation:

Face normals on a displaced spherical mesh (which has 512 polygons originally).

Even though there is no cracking (neighboring triangles that separated when they should have stuck together), the discontinuous normals are not tolerable. I could allow displacement, and then do a post-smoothing on the normals and run a change-the-normals-only shader as a solution.

However, in order to avoid breaking the displacement shading up into two passes (one for displacement, the other to then perturb the normals), I do something that would probably strike the REYES crowd as pretty odd. I have the user perturb the undisplaced normal in undisplaced space in addition to calculating positional displacement in their shader; then the tesselation/displacement machinery will compute a rotation from the undisplaced geometric normal to the displaced geometric normal for the polygon, and apply it to all of the perturbed normals via a quaternion into their displaced space. If that's not intuitively obvious what that does, imagine a triangle before displacement, then after displacement. Take the flat face normal of the triangle before, and then the normal after displacement, and compute the rotation that gets you from the one normal to the other. Apply that same rotation to the munged normals the shader gave back to you so that they are oriented naturally with the new displaced triangle.

It's surprisingly intuitive in the shader: you are given the undisplaced position, shading normal, dp/du (change in position with respect to U direction), dp/dv, du/dx, du/dy (for filtered texture lookups), the flat face geometric normal, and the U,V location. You output a displaced position and a modified shading normal in the space of the original dpdu and dpdv and geometric normal, without regard to how the whole triangle will end up oriented. The displacement machinery takes care of rotating that into the proper displaced tangent space for you.

And this is what you get:

Predisplaced normal pertubation, post-displaced normals fixup via quaternion rotation.

I apologize for using a different displacement map, but this one was a little less busy, and easier to see what's going on. Note the perfectly smooth shading on the undisplaced areas, and even the smoothness in the (uniformly displaced) areas on top of the letters. This render took a little less than 30 minutes, because I had it tesselating the mesh to an insane resolution: there are about 16 million triangles total in this scene after displacement. It used about 4 GB of RAM overall (displacement data was around 3.5 GB of that), as again I am not coalescing triangles that are mostly similar to their neighbors in surface normal, and I actually am not applying any cache management yet, because I haven't yet run out of RAM. Believe me, the day RenderSpud starts paging into swap space, I will have that LRU caching strategy implemented lickety-split.

Another interesting note about the above rendering: the rumpled edges you see along the displaced areas are due to the source displacement map, not the size of the tesselated triangles. Each displacement map used for this blog entry was 512x512 pixels, and in particular the above had a contrast filter applied to it so that the transitions are sharp. You literally see some of the pixelization from the map in the resulting displacement.

To prove this is actually vector displacement I implemented, I present the following image, which has some overhangs in it (they are somewhat subtle, sorry...I'm not much of an artist):

Overhangs (middle-right side has a little bit), which regular displacement can't do.

Everybody loves them some cars!

2011-01-19T10:18:00.000-08:00

Uses: RenderSpud Blender plugin, 256 samples/pixel path tracing, physical sky model. Took 14 minutes to render. Model by Tahseen.

It was surprisingly easy to get this to look decent out of the box, but there are a handful of improvements to be made in the materials and in the model. Thanks Tahseen for making the model available!

Metropolis Light Transport (MLT), fixed Mitchell-Netravali filter

2011-01-01T21:22:00.000-08:00

I implemented Kelemen-style MLT over vanilla path tracing over the last couple of days (and blew right through the New Year celebrations in the process). I had a dickens of a time with getting my rejected/accepted weights right because I had a bug in my pixel filtering code, as well as a slightly incorrect version of the Mitchell-Netravali filter.

I fixed the M-N pixel filter, added a proper sharpness parameter (inspired by Brent Burley), and got it hooked up in Blender so that I could test it out with an interesting scene. Queue a couple of indirect-lit renders:

The above has only 128 mutations per pixel average, and is lit by a large area light out past a small hole in the wall to the left of the camera (and an additional little light behind the box). Using path tracing, this scene is unbearably noisy.

This uses models from here, although I'm not sure what the original source is. This has 256 mutations per pixel, again very indirectly lit.

And the kitchen scene again, this time with a not-so-stupid table color, and a different light probe outside. This rendered in about 1 hour, with 400 mutations per pixel. If you look closely, you'll see the typical MLT favoritism of brighter areas, while the darker areas that have perceptually less noise get neglected a little more.

Update: Iliyan Geogriev asked about the actual implementation of the sharpness parameter in the Mitchell-Netravali filter. I tried to explain it, but for everyone's benefit I'm just going to post my actual pixel filters code. I'm including the box filter so you can all get an idea of how the M-N filter code relates.

class PixelFilter
{
public:
    PixelFilter(float width = 1.0f, float height = 1.0f)
        : m_width(width), m_height(height) { }

    virtual ~PixelFilter() { }

    virtual float value(float xOffset, float yOffset) = 0;

    float  width() const     { return m_width; }
    float& width()           { return m_width; }
    void   setWidth(float w) { m_width = w; }

    float  height() const     { return m_height; }
    float& height()           { return m_height; }
    void   setHeight(float h) { m_height = h; }

protected:
    float m_width, m_height;
};

class BoxFilter : public PixelFilter
{
public:
    BoxFilter(float width = 1.0f, float height = 1.0f)
        : PixelFilter(width, height) { }

    virtual float value(float xOffset, float yOffset)
    {
        return 1.0f;
    }
};

class MitchellNetravaliFilter : public PixelFilter
{
public:
    MitchellNetravaliFilter(float width = 4.0f, float height = 4.0f)
        :   PixelFilter(width, height), m_b(1.0f / 3.0f),
            m_c(1.0f / 3.0f), m_widthFactor(4.0f / width),
            m_heightFactor(4.0f / height) { }
    // Sharpness = 2*c, b + 2*c = 1, so b + sharpness = 1
    MitchellNetravaliFilter(float sharpness, float width, float height)
        :   PixelFilter(width, height), m_b(1.0f - sharpness),
            m_c(sharpness * 0.5f), m_widthFactor(4.0f / width),
            m_heightFactor(4.0f / height) { }
    // Recommended that b + 2*c = 1
    MitchellNetravaliFilter(float b, float c, float width, float height)
        :   PixelFilter(width, height), m_b(b), m_c(c),
            m_widthFactor(4.0f / width), m_heightFactor(4.0f / height) { }

    virtual float value(float xOffset, float yOffset)
    {
        return mitchellNetravali(xOffset * m_widthFactor) *
               mitchellNetravali(yOffset * m_heightFactor);
    }

    float mitchellNetravali(float v)
    {
        float av = ::fabsf(v);
        if (av > 1.0f)
        {
            return ((-m_b - m_c * 6.0f) * av * av * av +
                    (m_b * 6.0f + m_c * 30.0f) * av * av +
                    (m_b * -12.0f - m_c * 48.0f) * av +
                    m_b * 8.0f + m_c * 24.0f) / 6.0f;
        }
        else
        {
            return ((12.0f - m_b * 9.0f - m_c * 6.0f) * av * av * av +
                    (m_b * 12.0f + m_c * 6.0f - 18.0f) * av * av +
                    6.0f - m_b * 2.0f) / 6.0f;
        }
    }

protected:
    float m_b, m_c;
    float m_widthFactor, m_heightFactor;
};

Of course, actually depositing each sample over the 4x4 (or whatever size you choose) window of pixels is another question, but I'll leave that as an exercise for the reader.

The cabin (more PyBlenderSpud)

2010-11-20T00:42:00.000-08:00

The new Blender integration with RenderSpud makes it a lot easier to get at interesting data. I opened up the cabin CGTalk/3DRender lighting challenge Blender file, ajusted a few render settings, and let 'er rip. 30 minutes later, here's what I got:

And here's an updated version (the physical sky model is now integrated):

PyBlenderSpud

2010-11-19T00:35:00.000-08:00

I have managed to shoehorn in a few python bindings (using boost::python) and then get some integration with Blender 2.55 as a result. There is a lot that is missing still, such as lights (I just have a bland ambient dome light hard-coded for now), decent materials translation and new RenderSpud material panels in Blender, no physical sky integration, etc. But hey, it's a start:

The renderer does update once per second. The above image took about 3 minutes to render.

Bottom-up (agglomerative) BVH construction

2010-10-17T16:53:00.000-07:00

I implemented Fast Agglomerative Clustering for Rendering by Walter, et. al in an attempt to get faster ray-tree intersections. Bottom-up tree construction usually results in higher quality trees, and it handles the surface area preferences pretty well without having to implement SAH split candidate testing.

So far, for scenes with very similarly-sized triangles (such as scanned data) the final performance is roughly the same as the fairly highly-optimized top-down BIH implementation. For scenes with more complexity it may have a slight edge, but I am not seeing the 30%-ish increases that the paper authors were. It could be a bug or other problem, so I'm hoping to speed it up in the future; but for now, it's no worse a choice than my BIH implementation.

There were a couple of tricky parts to the agglomerative implementation. I had to be very careful about comparisons (e.g. less-than vs. less-than-or-equal-to) and their suggestion about when you find A matches B matches C when finding potential matches is very important (because it definitely happens for large models). Also, the checking of the nearest child first in the k-d tree that bootstraps creation was very important. If you screw any part of the bootstrap k-d tree up, you end up with very, very low quality trees that take eons to render. Implement it carefully. C'est la vie.

In any case, my litmus test was to throw a tough scene at it and see if it held up. Here is the Natural History scene from 3drender.com's lighting challenges. I have rendered this before, but it really lacked good materials and decent lighting. Here it is naturally lit by the physical sky model, but it still has the same wussy materials assigned to it. On my Core i7 920, it took 3.3 GB RAM and 90 minutes to render, at 1024 paths/pixel:

Since I do not have bidirectional path tracing implemented, I had to remove all of the window glass geometry so that the lighting that reaches the interior is actually directly-lit. It worked out fairly well (this looks noticeably better than the last attempt), but it still needs improved materials.

Arbitrary Output Variables (AOV) and Light Path Expressions

2010-05-05T22:51:00.000-07:00

Larry Gritz and the other kind folks at Sony Pictures Imageworks have been putting a lot of effort into a (now open-source) project, Open Shading Language (see here for more info). As part of that, they released a document detailing a very neat way of capturing arbitrary output variables, or AOVs, using what they've termed "Light Path Expressions".

AOVs are used extensively in production rendering; they essentially are a method of capturing, in image form, some part of the render or some extra data that can be produced by your shaders or the renderer itself during the process of rendering. Several examples of AOVs would be diffuse-only reflection, perfect-specular-only reflection, diffuse reflection from a certain object, or glossy reflections from a certain light. Or they could even be just surfaces tagged with specific colors, or rendering out normals only, or nearly anything else. These AOV images can be used for post processing in the comp (compositing afterwards), or just as debugging/diagnostic data for improving lighting or fixing bugs. It is readily apparent that they can be extremely handy.

SPI have provided a pretty neat method of capturing AOVs in their light path expressions. Essentially, you can use a regular-expression-style syntax to specify the bounces a set of rays go through (or rather, the reverse, tracing from the camera to the light, background, or surfaces with emission), and then every ray path that matches that light path regular expression will then go into the AOV image associated with it. Please see their document for more details. It's a very elegant way of capturing complex ray/surface/light interactions.

I have implemented nearly all of the specification in that document, with the exception of some of the background handling, and the "everything but this label" syntax: [^'label'] This is due to the fact that I used pcre (and pcrecpp), whereas the SPI guys have created their own custom regex parser that handles the light path expressions in a specific way. In order to use pcre, I had to do some pretty interesting regex rewriting; happily, the only feature lost was the [^'label'] feature as far as I can tell. SPI says they will release the source to their implementation hopefully fairly soon, and I intend to switch to theirs when that occurs.

For now, however, my implementation works pretty well, and covers what I need. Given this final rendering:

Here is the diffuse-only capture, using the light path expression: CD[DGS]*[LO]

Here is the glossy-only capture, using the light path expression: CG[DGS]*[LO]

And finally, here is the specular-only capture, using the light path expression: CS[DGS]*[LO]

If you want to know what hideousness that gets rewritten into for pcre, here is a teaser (this is the diffuse-only expression):

(?:<C.'(?:[^'])|(?:\\'))*'>)(?:<.D'(?:(?:[^'])|(?:\\'))*'>)(?:<.[DGS]'(?:(?:[^'])|(?:\\'))*'>)*(?:<[LO].'(?:(?:[^'])|(?:\\'))*'>)

Inside the path tracer in RenderSpud, it ends up writing paths that look like this:

<Cx''><RD'LambertTex'><RS'ReflectiveWhite'><Lx'AreaLight1'>

Which is then matched against the pcre version of the expression.

Note #1: RenderSpud is now using OpenImageIO for the texture sampling. This is another great Larry Gritz project. He lives near me, so I owe him lunch and a big thank-you for providing such great source code out in the open. I am aware of the fact that he was a major driving force in getting OSL itself open-sourced, and his long history of sharing his hard work with the rest of the rendering community is one that does not go unnoticed. Thanks, Larry!

Note #2: Larry tells me Alejandro Conty was responsible for the idea and SPI's implementation of light path expressions. So thank you too, Alex!

Look Ma, your kitchen is clean!

2010-04-30T12:57:00.000-07:00

Some additions to the file reading were added, such that I can include additional GTOs or OBJs from within a GTO scene. It works out nicely for assembling more interesting items to render:

This comes from this thread at ompf.org, and when they said it was a great stress test of acceleration tree construction, and they were right. It definitely didn't perform as well as I would have hoped. It took 5.5 hours to render on my quad-core i7 920 (at 640x480, 1024 samples per pixel). It definitely is *not* sampling the Pisa lightprobe very efficiently, hence the high number of samples to remove the noise.

Nevertheless, I'm pleased with the results. Next to Pomesuba's results in that thread, though, it doesn't stack up quite as well, but it represents a pretty decent stab at it.

Please share with the class...

2010-04-28T22:25:00.001-07:00

I had a couple of questions from viewon01 at ompf.org sent to me from that I figured I would let everyone hear the answers to. Here is what he asked, and my (edited for space) responses compose the rest of the post.

Hi Mike,

I have follow the development of RenderSpud and would like to know if it is still in progress and if a day you expect (or not) to set the project as open source.

I'm very interested to view your source code and have your feedback about creating a rendering engine in C#. Does it suffer of performance problem compare to a C++ version ?

Thanks

Thanks for your interest! I don't know if I'm ever going to open-source RenderSpud (I'm likely to open-source at least some parts of it), but if you have specific questions I'd be happy to share example code from RenderSpud that shows how I tried to do it.

I am actively working on RenderSpud, although I haven't posted as often lately as I would like to about it. I don't have a huge amount of free time outside of my regular job.

The C# version of RenderSpud has not been updated in a while; instead, I've recently been reworking a faster SSE-optimized C++ render core. The C# version was decently fast, but the optimized C++ version is much faster. I didn't have time to go back and use Mono.Simd in the C# version to speed it up, but perhaps someday I'll try that out. Mono didn't perform as well as the .net runtime due to garbage collection performance, and I think that situation is improving. [Note: I could do preallocations and pooling in C# as well, I just hadn't gotten around to it.] But really, memory performance was another place where the C++ version does much better; I use memory pools in C++ that make its inner rendering loops pretty speedy, since it doesn't have to do any allocations.

-Mike

Twisting Suzanne (or, Deformation Motion Blur)

2010-04-10T22:45:00.000-07:00

Motion blur in ray tracing is an interesting beast, due to the need to play nice with acceleration structures. The process of building acceleration structures is often at odds with dynamic data, especially when you must find discrete parts of it spatially very quickly.

Handling motion blur for regular transformations (matrices) is relatively straightforward: store time-sampled versions of the transformation, and when a ray comes in at a given time, interpolate the transformation (carefully, this is not quite as easy as it sounds), use the inverse to transform the ray into a local space, and intersect against the untransformed element. Transform the intersection results back out to world space, and you're done.

Deformation motion blur requires a substantially different approach. Instead of transforming the ray itself into a "time local" space by interpolating transformations, you must instead find all candidate primitives at any time with the ray in its regular space, and then interpolate each candidate primitive itself before performing a final intersection test.

Consider the following method for handling a triangle mesh that is deforming over time. First, instead of just creating a k-d or BIH tree over the triangles, you must create the tree over the swept triangles (which are really a volume after being swept). These swept volumes bound each triangle as it sweeps through time, and are what the tree itself contains. Incoming rays are traced against the swept volumes to find candidates to perform fine-grained intersection tests with.

The fine-grained intersection is to find the two motion samples of the candidate swept triangle that surround the time value associated with the ray. The triangle's vertices (and normals, STs, etc.) are then interpolated from the two samples to find where it exists at the ray's time, and then the ray is tested against this interpolated triangle (in the same way a non-motion-blurred mesh's triangles are tested).

Additional issues do come into play: if there is multi-sample motion blur (e.g. more than just a shutter-open and shutter-close version of the primitives; I'll refer to this as just MSMB), then you likely will want to construct one acceleration structure for each pair of consecutive motion samples. Also, if the primitives are swept for a very long distance (extreme motion blur), the acceleration structures will tend to become more inefficient with long, skinny volumes it must store. In that case, you might decide to automatically break up the motion into MSMB so that the acceleration structures can function more efficiently. There is a memory and overhead vs. speed trade off here, and it would be interesting to explore what heuristics generate optimal results.

I create a simple twisting deformation of the venerable Suzanne mesh in Blender 2.5, and rendered the results with a two-sample deformation motion blur. I am not doing any extra optimization (automatically turning things into MSMB is not done, for example). The first image is the reference, with no motion blur, which had a throughput of about 2 million rays/second:

The second rendering contains a slight (and fairly typical) amount of motion blur. It ran at about 1.5 million rays/second, showing a moderate speed hit for incorporating the deformation motion blur:

And the final image, with a more extreme motion blur. This ran at about 0.5 million rays per second, showing the effect of extreme movement on the acceleration structure performance:

Where do I start?

2010-03-07T14:11:00.000-08:00

I had a very relevant comment on an earlier post, that I thought was worth re-posting and then responding to:

You have a very educational blog, especially in a field that has a very fragmented base of knowledge to consult.

I have been following the development of "unbiased" renderers since before Maxwell Render made it's debut. I have also been watching and using many CG applications that convince me of the potential, but at the same time of some huge roadblocks to a really intuitive environment for photorealistic content creators.

My work in this field is completely at a non-commercial educational level and I am reaching the conclusion that I need to get a more intimate understanding of the core algorithms and techniques for producing the kind of images I see on your site and that are produced by commercial apps like Maxwell and Arion (of RandomControl.com).

My problem is that I hardly know where to begin. I have a programming background, but mainly in object oriented and scripting languages. However, I have deep experience in application architecture and user interface design. Now I would like to get a deeper working knowledge of the algorithms used in these cutting edge rendering technologies (and eventually, parallel CUDA programming and physics simulation).

Using a self-taught approach, where would you recommend that I begin if I wanted to get an understanding and see samples of these rendering algorithms and how they can be used in an application? (Don't laugh, I know that it will take considerable time, but I've already wasted 15 years seeing how others have tried.) I'm sure that you may have learned some lessons going down this road yourself that perhaps I could benefit from before attempting to embark on the same journey.

Sincerely,

Rob

His questions really go to the heart of why I created a blog about my renderer development in the first place (aside from being a tiny bit of a showoff, natch), which is to help others to know how to decide on your approach, what gotchas you will encounter, and what the theories behind all of this are.

First:

My problem is that I hardly know where to begin.... Using a self-taught approach, where would you recommend that I begin?

With a programming background, you need to make sure you have the right tools in your toolbox. For the long term:

You need at least a JIT-compiled language such as Java or C# under your belt, and be pretty comfortable with it. Preferable? A systems-level language that allows you to go to the metal, but this is not necessary (and indeed, if you look over this whole blog, I went from a C++ incarnation to a C# incarnation, then finally back to a SSE-based C++ incarnation). Another language will work, but don't expect to see performance numbers in line with the faster renderers out there. If you're not too concerned about that, then use any language you're comfortable with.
You need a decent understanding of how 3D scene graphs are usually organized. There are several approaches to this, each having its own strengths and problems. I really should put together a post on this by itself, so show a couple of approaches and how they work.
Algorithms and data structures must be staples of your programming diet. You don't have to be an expert, but you do need to be able to learn a new technique without too much trouble. Here are a few concepts that you'll undoubtedly run into in one way or another while writing a renderer, be it ray tracing or not: regular grids, k-d trees, BSP trees, BVH trees, sorting algorithms (radix sort, merge sort, quicksort, sweep and prune), k-nearest-neighbors queries (kNN), big-O notation (algorithmic complexity). For getting started, you won't need to know all of those, but eventually you'll get mighty familiar with them. Variations on these concepts are where most of the recent speedups in real-time ray tracing have come from.
A basic understanding of physics-related areas, such as optics and heat transfer. You won't need a lot of detail here, but if someone says "Snell's Law", you should know conceptually what that is (refraction). If someone mentions spectral-this or spectral-that, you know what the difference is between a spectral color and RGB colors. This is less critical, but even knowing things like "flux" vs. "radiance" vs. "irradiance" can be helpful, although not even all rendering experts can get that stuff right.
Math! You should be familiar with vector and linear algebra, basic calculus and differentiation, and when you start reading about Monte-Carlo integration, you should be able to relate that back to the problem at hand, even if you don't understand all the details. Make sure you know or have a handy reference for things like trigonometric identities, matrix algebra identities, definitions of things like dot products, cross products, what orthogonal or orthonormal bases are, etc. You don't need it all memorized, but you should be able to see a geometric problem to solve, and have that twinge in the back of your mind happen when you know there's something that applies that you ought to go look up, in order to solve the problem faster or more elegantly. You'd be surprised how often this happens, even in just writing a ray tracer.
A good set of places to ask questions, and to research solutions before you go asking. Yeah, this goes without saying, but I'm saying it anyway. More on this later.

Once you have the ball rolling on all of the above, you're ready to start tinkering. Having watched a lot of programmers (including myself) learn how to do something, I've come to the conclusion that the single best way to learn something is to actually try it. Rob said:

Don't laugh, I know that it will take considerable time, but I've already wasted 15 years seeing how others have tried.

Read and study enough that you get the general gist of things before you start writing some code, but you do not need more than this to get started. On the flip side, you must, must assume that your first attempt (or three, if you're me) is going to be very educational, fun, but not work very well, if much at all. You can't learn to prize the good without experiencing the bad; so as long as you keep your understanding of what you're learning from getting rigid (and thus get into bad habits), each time you try again it will get better and better. Pretty soon, you'll have something that you can feel proud of, and which you can then use to help you get a job or teach others some aspects of the trade.

Case in point, RenderSpud. My first foray into ray tracing was as a 16-17 year old in high school as part of a science & tech fair project. I was comparing per-vertex lit rasterized shading, per-pixel lit rasterized shading, and ray tracing. I wrote all of the rasterizer and ray tracer (with some help from my dad, who is also a programmer and a graphics guy), but honestly, it sucked. Because I was so young, the judges all realized this was definitely a first attempt, and I picked up a Math Teachers Association award, an award from Intel for $100, and a superior rating on the project. That first taste of getting something to work like that was enough to convince me to never turn back. I wrote another ray tracer in Java a year later, ported it to C++, then during college I finally started on RenderSpud in my junior year (which was not based on that C++ code). At the behest of a professor, I wrote a paper on renderer architectures that got published (see here), although I look back and know now that I could do a much better job on that paper. I scrapped that for the C# version, and now I'm doing a parallel optimized C++ version again based on the architecture from the C# version.

So, if you really think about it, I count, oh, six ray tracers so far for me. And I'm quite sure this won't be the last core renderer rewrite I do in my life. Yes, I'm still totally excited to do it; I'm not getting bored yet. But what propels me is not just what I see others do or write about it, but it is the act of getting down and dirty with code and hammering it out. Very few things are more satisfying than getting a pretty image rendered after a lot of your own sleepless nights, sweat and blood (er, well, hopefully not blood). The day I showed an image from my renderer to someone and they thought it was a photograph was a very happy day in my life.

But, why go through all of the pain and irritation of solving problems someone has already solved?

I'm sure that you may have learned some lessons going down this road yourself that perhaps I could benefit from before attempting to embark on the same journey.

To some extent, you can learn from others' mistakes without having to experience them yourself. I hope this blog serves as a resource to help people learn what to avoid, and what to keep an eye on as they go. There are another pile of resources to keep handy as well that can help minimize the most frustrating aspects of writing a renderer from scratch, which I'll list below. Plus, I'll admit that the days you have to chase NaNs through your code that blew up your render, or when you have something go wrong and you get snow-like sparklies all over are plenty frustrating.

But really, will you remember the solutions or be able to apply them well in other situations if all the gotchas are solved for you up front? Let me illustrate a different philosophy with a personal story. My father is a programmer (a really good one), and he practically created the factory simulation software industry with the product that he mostly wrote himself. He's a graphics guy as well, and the software he wrote shows graphically the loads moving through a fab or whatever facility. When I was first learning to program (before high school), I gravitated towards 2D and 3D graphics very quickly, and I would try to ask him questions about programming issues and graphics issues. He was a busy man, but in addition to that, if he thought I had any chance of figuring it out on my own, he would not really help me and would let me flounder around until I got it. It was really frustrating to me at the time, but I've since seen a lot of wisdom in it: after many hours (or days) trying to fix something, and I figured it out, it stuck. Forever. I wouldn't make those mistakes again very often. As such, I literally taught myself how to program, how to do 3D, how to debug, how to find answers, and the value of persistence. (To be fair, my dad did help me with my first ray tracer for the science and tech fair project, and that was a really fun memory for me.)

When you are doing programming for a job, there is a balance that must be struck between getting help from other programmers/experts and figuring it out on your own. Your goal there is to maximize productivity, and if someone else has a bit of time and can show you the solution in 5 minutes as opposed to you spending hours trying to figure it out, then for the company's sake that 5 minutes is well spent in tutelage, despite the "life lessons" that might have been learned otherwise. On the other hand, if it's for non-commercial reasons you are doing the project, the "how much time is too much?" threshold for working it out yourself goes much higher. You ought to spend extra time researching it and figuring it out on your own, for your own benefit as a programmer.

Now, for the resources:

This blog, and others like it (for a long time, Nick Chapman had a blog about his development on Indigo which was quite interesting).
Shirley and Morley's Realistic Ray Tracing (see here).
Jensen's book on photon mapping
PBRT (this is good because you get all the source code, and can tinker with it by doing some of the exercises, although it might be a little much to start with; PBRT is a great resource nonetheless)
ompf.org forums have tons of questions asked, answered, and shooed away. Tons of knowledgeable people there, pointers to more resources, a place to show off/critique your work, and very interesting discussions on both simple and complex issues in ray tracing and rendering. Please register here, lurk, and ask questions when you can't find the answer anywhere.

Those are just a few of the places to start. There are more books, a gazillion papers, etc, but the above items will serve as a good gateway to more if you aren't familiar with them already. If you are, then get started writing or modifying some code already! =)

Of course, if you are looking to see some code that actually implements the rendering algorithms, I would pick an open source renderer (such as sunflow) or get the PBRT book and the code. Start modifying it or using it, and then start writing your own modeled after their general approach; writing your own will give you maximal understanding, although it obviously takes a long time.

Finally, Rob brought up something that I think really needs to be pointed out:

I have also been watching and using many CG applications that convince me of the potential, but at the same time of some huge roadblocks to a really intuitive environment for photorealistic content creators.

I agree completely. There are complete modeling/animation/rendering packages out there, and in my job I hear people complain endlessly about the ones that are out there. Some look good, but lack things they need; some are ugly and clunky, but have tons and tons of functionality (read: they're bloated). Many are just plain not easy to use.

Building a good user interface around a renderer is a very interesting problem, and some people have solved parts of it fairly well. I don't think there is one answer, and there are many avenues to be explored. Making an interface that fits a photorealistic or a production-quality renderer is different as well; this conflates rendering itself with material assignments, light assignments, render pass management, and data conversion/acquisition from previous stages of the human pipeline (modeling, animation, etc).

This is an area that I'm exploring more over time, but I don't have a good enough idea of what I think is the right approach yet.

GUI viewer progress

2010-02-23T23:58:00.000-08:00

In the spirit of bravery (or a habit of programmers to walk around in their underwear, so to speak), I'm showing the very clunky, very incomplete beginnings to a scene viewer and manipulator for RenderSpud.

If you look closely, you might spot a new material, too (*cough* anisotropic *cough*).

Thus far it can read a GTO scene or load a built-in scene, it will do a relatively fast one-bounce single-sample ray trace for the preview area, and show you the scene graph. And then you can kick off a high-detail render, with exposure/gamma controls and ability to save to EXR.

Visual update for January 2010

2010-01-30T18:53:00.000-08:00

I haven't been very good about posting lately, but since a picture is worth a thousand words, I'll mostly just post a few renderings that have been done lately.

First, instancing. The following examples used very little memory over what is required to just load the Ajax bust once, and the render times were pretty reasonable. If you think about it, the second image has approximately 1,000,000 triangles in each model, and there are 10,000 of them (that's around 10 billion triangles):

The next image is a bare-bones render of a 3dRender.com lighting challenge scene (#17, natural history). The scene was modeled by Luna Bautista and Joel Andersdon:

I hope to get some interesting materials in there at some point, and get bidirectional path tracing working so that I can use it with the physical sky model in a dusk scenario at some point in the future.

GTO file reading

2009-11-03T11:19:00.000-08:00

Just a quick update with a couple of renders. I have GTO scene/material reading working (first image is from a GTO scene file), and have been playing around with/fixing a few bugs here and there. Nothing fancy, but here they are:

Physical Sky (again)

2009-08-15T00:50:00.000-07:00

I have reimplemented A Practical Analytic Model for Daylight in RenderSpud again, this time with better spectral fidelity. The improvement over the older implementation is fairly noticeable.

It produces color values in the range of 5000.0 to 6000.0, which is pretty stinking bright, so I also had to add a few range controls to my rendering console. When saved out as EXR it is easy to get them in the right range though for conversion to PNG.

Here are a couple of renders, each taking a little less than 6 minutes, with the same daylight settings, but with different turbidities (see the paper for details).

HDR Light Domes

2009-07-18T11:03:00.001-07:00

The EXR texture loading was partially a segue towards loading HDR environments to act as lights. RenderSpud now sports an implementation of the Pharr/Humphreys method of importance sampling for environment lights. This light probe is the Pisa courtyard from the ICT Graphics Lab set.

Note there is no ground plane to shadow, and I haven't built in the functionality to produce shadowing on the ground without also seeing the proxy geometry. That will come later (a shadow-only material, or somesuch).

Texture Mapping

2009-07-17T22:47:00.000-07:00

Using a variant of the simple scene where I tend to test my materials, I added a quick implementation of EXR texture maps. This is nothing special, but it's a bit of progress. Next up are HDR light domes.

Car paint (iridescence)

2009-07-10T14:39:00.000-07:00

I just wanted to post a quick couple of images that I've been using for testing lately. They feature motion blur, an iridescent material (meant to model car paint), and an ambient light dome. They're nothing special (and there are bugs/issues), but they were worthy of a quick post. These each are 640x480 resolution and took about 22 minutes or so, the second being the jotero.com Ajax model:

Gundam Robot

2009-07-03T22:58:00.000-07:00

Here is another excellent model from my friend Jack, who also modeled Iron Man and let me render it. This one is a Japanese gundam robot. The render settings are largely the same as they were for the Iron Man render (2048 samples/pixel, 640x480 resolution), except I threw in a little bit of depth-of-field and positioned the camera to help sell the scale of this model. The robots are meant to be quite large (50+ feet tall), and it's sometimes hard to get a rendering to appear that way.

The render time was approximately 30 minutes on my Intel Core i7 920. Note a few potential artifacts: he is mostly a faceted model, so there are some harsh edges in the lighting. Also, the Blinn material on some of the metal was picking up very wide highlights from the area light above (and behind the camera), but that is actually correct. It's just ugly, and I could use a better lighting setup.

I gotta get that spectral sky model put back in soon.

Iron Man

2009-07-03T20:30:00.000-07:00

A modeler (who must have busted his butt to make this) gave me a chance to render his Iron Man replica, inspired by the first movie. No ILM models were pilfered or hurt to bring you this rendering.

This was rendered at 640x480 with 2048 samples/pixel, one area light above, and an area light attached to each of his hands, his chest, and his eyes, and I fudged the materials until they looked decent. I realize the color on this guy is not quite the same as what is in the movie; but hey, it's a start. This rendered in about 20 minutes on my new Intel Core i7 920 machine I built recently.

He also made a Japanese Gundam model, so that's up next. This is helping me fix a few bugs with memory usage and OBJ loading.

Transform motion blur, and a note about production rendering

2009-05-04T09:45:00.000-07:00

The artists I know that have worked with Renderman/REYES style renderers are all scared of ray tracing. They believe that certain features of scanline/REYES rendering are not possible in ray tracing, or at least not without insane render times.

My experience with the overall rendering process (with GI, shadow maps, etc.) in REYES is that the overall time to render a high-quality frame is not more than it would be with ray tracing, and ray tracing requires fewer passes (usually just one), and it can achieve feature parity with REYES in a similar amount of time on modern hardware. In addition, there are certain features that simply cannot be achieved or not at all easily with REYES that ray tracing handles nicely, such as reflections, refractions, blurry reflections, caustics, generalized volumetric rendering, true GI in difficult lighting situations, and true soft shadows. That list should make any artist cringe, and realize what REYES is natively lacking.

But, what REYES does offer and do well at are very important. It can achieve high geometric complexity without high memory requirements fairly easily (this is part of the algorithm), although this can be achieved in ray tracing rendering with some ingenuity. REYES can also handle direct-view effects such as motion blur (both deforming the geometry, or just changing the transformation matrix) handily, albeit with some noise. It can also do motion blur without a big memory hit. It does this by creating micropolygons for all of the blurred positions of a polygon that touch any pixel, or creating micropolygons for displaced surfaces as a natural result of the algorithm, and when the polygons are rendered, they are thrown away. This can be done incrementally, such that the total number of micropolygons in flight at any one time can be minimized.

Ray tracing, on the other hand, can produce a ray that will intersect any geometry in the scene at almost any time. This creates the apparent requirement that all of the geometry must be in memory at any given time (although this is not actually true, but the potential for this to happen with incoherent rays is nonetheless real). This, and the fact that noise reduction in ray tracing usually requires a heavy-handed "just trace more rays" approach, scares artists away who are familiar with Renderman. In production rendering, motion blur and displacements are used quite extensively, so I don't blame them for feeling sheepish about it.

But they need not be scared. Over a period of time, I'm going to implement production-renderer features that will demonstrate that feature parity, along with controllability, and adequate speed are possible in a ray tracer, with the additional flexibility and simplicity that ray tracing affords to boot. We'll start with transform motion blur (deformation motion blur is harder, and I'll explain why in a later post).

The above rendering was produced in RenderSpud the other day, at a render time of just a couple of minutes on my dual-core athlon 64 4400+. This is the Ajax bust, with a a second motion sample rotating it in a clockwise direction over the course of the exposure. It was rendered with approximately 64 samples/pixel, hardly enough for a production rendering, so there is still plenty of noise, but if it were rendered with more samples that would go away; I left the noise in to demonstrate what it looks like. What you should notice also is that this image has full global illumination (notice the slight caustics on the ground square from the shiny ball bearing-like objects), soft shadows that incorporate the motion-blurred object, and even caustics from the balls onto the motion-blurred ajax base in front. Achieving all of that in a REYES render would be difficult and expensive time-wise, but in RenderSpud, I implemented a transform motion blur object that you attach another object to, and all of the other features automatically work. Ray tracing algorithms tend to lend themselves to that type of result.

How this is achieved here is quite simple. The motion blur object asks the object it is blurring for a bounding box. It then transforms the bounding box by each motion sample's matrix (and you can have as many samples at whatever times you like), and collects the results into a bounding box that contains the entire motion-blurred sequence. Then, when rays are traced, it find the motion samples that bound the ray's assigned time (which is assigned when the ray leaves the camera), transforms the ray into local space for the object based on interpolating the two motion samples' transforms, and then traces the ray locally to hit the object. The results are then transformed back out by the motion samples' transforms, interpolated linearly to find the real world-space position/normal/etc. of the intersection, and off we go.

That's it. Transform into local space according to the motion samples, interpolating the ray position/direction, intersect, transform back out, interpolate the intersection according to the motion samples, and plug that into your normal ray tracing framework.

View from the Podium

2009-02-28T23:34:00.001-08:00

A long overdue update, I have been (slowly) working on improving the C++ render core and shaping it up. There are some transformation/matrix bugs in my SSE-ized implementation, so I had to effectively freeze transforms for the area lights in the following render, but it's showing promise for sure.

Something about the conference scene is giving the BIH trees some real trouble; it generates a large amount of RAM usage, and the performance is a lot less than I would expect. Even so, the render time isn't totally atrocious. This was rendered in 1 hour 22 mins with two threads, at 256 samples/pixel and 400x400 resolution:

This next one was rendered in about 52 minutes, using 2048 samples/pixel. It showcases a few materials, including a new iridescence material (which uses a Fresnel blend from diffuse green to Blinn gray glossy reflection), Blinn glossy reflections (particularly the blue sphere and the ground), and the usual reflective material with a conductor Fresnel on it:

I intend to post more about where things are heading soon.

Experiments

2008-07-28T13:00:00.000-07:00

I apologize for not posting in a while, but I've been pretty busy with other projects as of late. However, I've had the help of a friend in optimizing RenderSpud's C# code to perform better. We fixed a number of multithreading bugs as well, but in the process I've discovered that I've hit somewhat of a brick wall performance-wise. It seems that, due to garbage collection and the lack of good vectorization (e.g. the JIT turning code into SSE instructions, which it doesn't do) that it won't get a whole lot faster from here. That's unfortunate, because, well, it's not very fast.

It must be said, though, that prototyping the actual raytracing core in C# has been really good. It has made it much easier to focus on algorithms and correctness, and since I had to figure out how to cast the PBRT paradigm into a different language it forced me to learn and notice a lot more details than I would have otherwise.

That said, I've been experimenting with creating an optimized C++ backend, which will largely be responsible for handling ray queries and shading. I might even go so far as to bundle up each renderer implementation in there as well, so that all rendering loops are as fast as possible. I fully intend to then expose what is needed to a C# frontend, where it can deal with inter-node communication and load balancing, image display, UI, etc.

What I have made in the last few weeks is quite a bit faster, but it's not as fast as it will be by a long shot. It currently has a few primitives (triangle meshes, spheres, planes), cameras, groups, and it even has the BIH acceleration structure ported over. With minimal shading and shadow rays, it can trace approximately 325k rays/second on a single core on my machine with the Ajax Bust model (~500k polygons). However, I fully expect to be able to triple that, even with some lighting/shading thrown in.

I have made vector, point, matrix, and orthonormal basis classes that feature both generic and SSE variants (using intrinsics and gcc vector extensions), but they currently support only one ray at a time. Inspired by Jacco Bikker's approach in Arauna, I'll make it support 2x2 SIMD packets mostly transparently; e.g. via operator overloading, typedefs, and the like I'll make it so that most of the code can deal with 4 rays at a time without having any idea that it is doing it (except where the rays are packaged up and then the results extracted back out). It's not too hard, and if you are curious about how to approach it, see Jacco's article about it.

If all goes well, I'll see the 3x throughput on rays Jacco did; with a little more care, I hope to be above one million rays/second on each core. If I were doing real-time raytracing, that would put me into striking distance of a fluid framerate at low resolutions. I'm not too concerned about producing a real-time display yet, but it sure would be nice if my 5 hour renders were reduced to something more along the lines of 5 minutes (I was only getting about 30k rays/second with the C# raytracing core, so that speedup is actually possible).

Props to cool people and places

2008-02-10T23:59:00.000-08:00

I have been lurking and occasionally commenting on things over at ompf.org, and if you are at all interested in real-time ray tracing or ray tracing rendering of any kind, you should go there. You'll love it; they have great discussions and top-notch people.

I'll mention one in particular, Toxie. He's one of the authors of the BIH tree paper, and he just put his thesis up for review. It has BIH info in it, as well as some great discussion on QMC (quasi-Monte-Carlo techniques), I presume because he studies under someone well-known for QMC research. Toxie's work is excellent. He mentioned in his thesis that RenderSpud is one of the few hobbyist renderers implementing BIH, which totally made my day.

To ompf.org, and Lycium, tbp, Phantom, Toxie, and everyone else over there: you guys rock. Just thought I'd let y'all know.

Bidirectional path tracing

2008-02-10T22:31:00.001-08:00

The bidirectional path tracing renderer has languished for a while, as there were some bugs that I wasn't sure how to fix. With the help of Ian, I was able to get down to the root of what was messing up (as far as I can tell), so I think I'm ready to show it off in action.

The first problem is that I was weighting my generated paths all wrong; a flat average totally falls flat on its face (no pun intended) when combining bidrectional paths. Instead, for each edge on the path where both vertices are diffuse, I increment the 'number of techniques' variable. I also increment it once for the technique of sampling the whole path the way I did, specular vertices and all. Please see Ian's comments in the post Comments, Updates for more info, or go search for Eric Veach's thesis and his discussion on bidirectional path tracing.

Note that RenderSpud's implementation does not follow Veach's formulation exactly; I still am taking a few shortcuts. First it does not account for the leg of the journey from the camera to the first vertex, as I never vary the camera vertex location anyway. It also seems to have a problem where if I increment the number of techniques when the last vertex before the light on the light path is diffuse, it gets too dark, so I skip that too. Finally, I'm not doing any sophisticated MIS (multiple-importance-sampling) yet either, so accuracy on glossy surfaces may suffer somewhat at low sampling rates. I also have to fudge my lights' PDF function a little right now, but that is probably because I need to convert from a PDF with respect to area on the light to a PDF with respect to solid angle. Yes, it all makes my head spin, too.

However, despite the above, it does reduce noise noticeably more quickly than regular path tracing for most scenes. I tested with some scenes that should not show a huge difference between regular path tracing an bidirectional path tracing to make sure the noise reduction was all I was seeing. Here is a path-traced version of the three spheres scene, with an area light above them. I have deliberately limited the number of rays/pixel so you can see the difference in noise (I think all of these had 20 samples/pixel, and 1-2 minute render times):

And here is the bidirectional path-traced version. Note the shadows have resolved more quickly (and the fact that they thus appear more dark is correct, as far as I can tell), as well as the glossy reflections and specular bounces:

Here is the same scene, but with the ambient light dome replaced with a single rectangular area light (which is not directly visible; I'm fixing that with an area light implementation soon). Here is the regular path-traced version :

Here is the bidirectional path-traced version. Note the resolved shadows, the less noisy caustics, although the reflection in the blue blinn ball (try saying that three times really fast) is darker in the bidirectional version; I'm not sure if that's a bug or not:

Bug fixes, HDR tone mapping

2007-11-06T10:03:00.000-08:00

I fixed a few bugs in the sun vs. sky handling. The blown out colors prompted me to also implement a couple of post-process operators, namely gamma correction and Reinhard tone mapping. It made the bright colors much more manageable (although I still have to scale them down, but the scale is much more intuitive now). The proof is in the pudding:

The top was rendered with 500 rays/pixel, and the bottom was rendered with 50 rays/pixel. I used Reinhard tone mapping with a burnout allowance of 5% and an exposure close to 1.0 (meaning very little modification).

Physical sky model

2007-11-05T10:27:00.000-08:00

There wasn't a lot of time lately to work on this, but I finally managed to get moving on the physical sky model (implementing the fine paper A Practical Analytic Model for Daylight by Preetham, Shirley, and Smits). I had to scale the intensity of the colors way down to get it to a displayable range, but I can fix that later (as it's just a scale value, after all).

Once again, I apologize for the ugly scene, but it's the pretty sunlight that counts. Here is a 400x400 image, with 100 rays/pixel at a 30 min render time. It is positioned near Salt Lake City (where the fine chaps who created the model are) at 6:00 PM during the late summer:

Here is the same, but I left the depth of field turned up too much. The time is 5:00 PM instead of 6:00 PM, and it is rendered with 500 rays/pixel (at 2 hrs of render time):

Note the colors in the background seem a little wierd; I'm not sure if that is just the model or if I made a mistake porting the example sunsky code to C# (or if my scale factor is messing it up, and I just need to go implement a good HDR output format to compensate).

Comments, updates

2007-10-19T15:05:00.000-07:00

I just noticed today that Thomas (I assume Thomas An of Maxwell Render fame) posted a comment on the biased-vs-unbiased discussion. [Update: that wasn't him, the real culprit was lycium from ompf.org, but my praise of Maxwell Render that follows still stands.] I sincerely apologize for not responding sooner, but I didn't have comment moderation or notification turned on. He pointed out a mistake about the biased-ness of photon mapping, and I have since updated the post on why he was correct. I hope I haven't offended anybody on the Maxwell team (their work is excellent), as I was poking fun at the marketing for Maxwell and not the product. The actual renderer, as I said before, produces beautiful renders, and since they have a product to sell and bread to put on the table they have to put their best foot forward with their marketing. (Secretly I envy them, so I hope they come here and visit/comment more often; I welcome their in-depth knowledge on all things rendering.)

On another note, I am starting to implement A Practical Analytic Model for Daylight now that my spectral rendering seems to be keeping up. When you have a proper spectral model for a given light source, the results are just incredible. I work with Andrew Gardner, who worked under Paul Debevec working on various productions, papers, and even HDRShop, and he's pointed me at some good spectral emittance curves for regular lightbulbs and for flourescent lights. I'll have to try those soon as well to see what kind of a difference they make over converting from RGB and then doing spectral rendering. Stay tuned.

Rectangle lights, improved dielectrics/refraction

2007-10-12T21:55:00.000-07:00

Maybe others would have noticed this sooner, but I only noticed the other day that my refractive surfaces were generating a lot more noise than they should (see color correction at the bottom for a visual example). The problem was that when there is more than one BRDF to choose from on a surface, such as a reflection vs. refraction on a glass ball, it would have a 1/N chance of choosing a given BRDF where N is the number of BRDFs for the surface. That's fine and all for most things, but when each BRDF shows up more than the others at certain points on the surface it doesn't cut it.

For transmissive surfaces, the amount of reflection vs. refraction is modulated by Fresnel's equation for dielectrics, so where the surface points toward the camera it should have a higher chance of picking the refractive BRDF, whereas toward the edges when the surface points mostly away from the camera it should more likely pick the reflective BRDF. The material (which holds the BRDFs) can now provide a weighting based on the surface and ray directions, which improves the quality noticeably. Here's another scene with the improvement in BRDF selection, and note that the reflections and refractions are no more noisy than the rest of the scene:

On another note, I got sick of point lights and my dome lights, so I implemented a simple rectangular area light (really, it can be a parallelogram, but whatever). Rays can't get directly intersect the light except for shadow testing (no emission yet), so I had to give it the option of being double-sided so that the ceiling plane would pick up the light and have reflections show up in the models below it. If I didn't, the reflective materials would look odd.

New renderers, XML scene format

2007-10-02T23:22:00.000-07:00

I switched jobs recently and am now working as a graphics software engineer for a movie visual effects studio in Berkeley, California. It was quite a move (from Utah, where I was working for EA in their Salt Lake studio), but we're settling in. I haven't had much time to work on RenderSpud, but I have made some improvements over the last few months.

Bidirectional path tracing is implemented, but there are some bugs (hence no screenshots yet). With certain numbers of bounces it gets too dark in shadowed areas, probably because it is throwing out one of the paths when it shouldn't be. It also doesn't handle the probability distributions of dome lights appropriately, but I can't figure out what is wrong with the math. My estimation was that BDPT was going to be fairly straightforward to implement...but no. The devil is in the details, and I think it is just a few details away from being showable. It definitely renders with less noise for the same render time, if only it was correct when it did.

I also implemented a very simple recursive Monte Carlo raytracing renderer. I call it the 'Whitted' renderer due to historical reasons, but if you give it some diffuse bounces it will gather more than a traditional Whitted raytracer would normally do. It works well, but it bogs down in a hurry when you allow it to have very many diffuse samples, because those can then have diffuse samples, and so forth. Perhaps an early cutoff for the diffuse samples would be helpful, but that would also bias it quite a bit.

Finally, I fleshed out the scene XML format and code for reading and writing the scene files that I have almost switched over to using them exclusively instead of mucking around with hard-coded test scenes. The notable things left to do are support mesh reading of external files such as OBJs, and making the app discover for itself all of the classes via reflection that can read/write parts of the scene XML. The latter is useful for plugins; it would allow anyone to more easily add a custom type of scene element, material/brdf, renderer, image format, fresnel reflection function, microfacet distribution, etc. RenderSpud would auto-recognize them as it loaded the appropriate DLLs.

Enter the requisite renders. Both of these are rendered with path tracing at 400x400, 500 samples/pixel. They were both loaded from a scene XML file (note the lack of meshes...) and have just point lights and some varied materials in the second image. Yes, yes, my colors are hideous, but I'm a programmer and need artists to make my mathematical goop shine:

Bug fixes, more color correction

2007-07-22T15:19:00.000-07:00

Over the weekend, I fixed a number of crashes and other code quality bugs. I was also finally bothered enough that my white surfaces weren't looking very white that I figured out what was wrong; along with it, the spectral rendering mode tended to favor reddish colors. RenderSpud was converting from spectral to XYZ, and then to RGB, where in the latter conversion you could specify a color system (sRGB, NTSC, CIE equal-energy), whitepoint, etc.

So, using the sRGB profile is either not set up correctly, or else displaying an sRGB-ized image on an sRGB display just tends to darken colors besides red. Once it was switched to just using the equal-energy CIE model, my color balance was much better.

On the white not being white front, what was happening was that during the conversion of a spectral color to XYZ coordinates, it was normalizing the XYZ coordinates (as it should). I would track the highest-valued wavelength/intensity pair, and then rescale the resulting final RGB color by that on the way out. It picked up relative color differences, and I thought that was all it needed.

The trick was that when converting to XYZ representation, white would end up with a Y coordinate (luminance) of 1/3. When you want that to be a full 1.0, and then you have a max intensity of 1.0, you end up with 1/3 of white in the end after rescaling. Ugh. But, since that max intensity is really what I want the color to be in proportion to, I can make my rescale value something like "maxIntensity / Y" so that the final luminance is roughly in line with the max brightness of the original spectral color. It's possible to make it more sophisticated but that is probably not necessary.

Here is the Ajax with two spheres scene with corrected colors. Note that I changed the gold metallic sphere to a diffuse green sphere, and the ground plane is set to 80% white now. It still looks much better, considering the ambient light dome is emitting a full white. Rendered at 400x400, 256 samples/pixel, approx. 2:35:00 rendertime:

Color conversion, scene files

2007-07-13T23:07:00.000-07:00

I fixed an issue where converting from RGB to spectral representation was totally wrong. I now use Brian Smits' method of adding a spectral curve for white, then one of cyan, magenta, or yellow, and finally one of red, green or blue. It looks pretty good, but there is still some bug with the other direction (spectral to CIE xyz then to RGB) where everything in the final output color but red is still too dark for some reason. I can flip a switch and have RenderSpud do regular RGB rendering, and then flip back and compare to full-fledged spectral rendering; the color conversion at the end has some wonky problem where it just ends up too dark. Maybe my selection of a color model and whitepoint sucks, but I need to figure out where the hangup is.

The scene XML reader has been started, along with render job and job-parts support. Once I get to where I can set up image settings, renderer settings and a decent scene, I'll probably start creating a client GUI app and I'll flesh out the CLI interface as well. At that point I might be brave enough to do a limited demo for a few artists around to play with.

I'll start collecting all of the "if only it did this" talk that comes out after that and prioritize for the next round of features. If I can get more texture support, HDR lightdome speed improvements, output image support, a physical sky model, and some normal mapping, I'd be pretty tempted to let it out to a wider audience. Let me know if you'd like to be on the list.

Depth of field, and some optimizations

2007-06-25T10:26:00.000-07:00

After posting about what I want to do with RenderSpud, I decided it wouldn't hurt to take the time to implement one little itty-bitty feature. I picked depth of field effects, and it turned out to be surprisingly easy. All that needs to be done is take the regular pinhole-ray that it already computes, compute a random point on a disk simulating the lens (which you can set the radius of), compute the intersection of the original ray with the focal plane, and make a new ray from the random lens point going through that focal-plane intersection location. Piece of cake. What's more, is it is easy to control in the camera: set a focal distance, set a lens radius, and boom, instant DOF.

I was getting a little depressed about the long rendertimes though, so I also set about speeding things up a bit. I only had a few hours to make the attempt, so I didn't do a lot, but I was able to squeeze approximately 30% more speed out of it. Not bad for a small code massage. First, I changed my vector, point, and texture coordinate classes into structs so that they would be not contribute to excessive garbage collection. That gained it about a 20% speed increase. The last 10% came from adjusting the main thread's sleep patterns. It would spawn the worker threads that do the actual rendering, and then spin and sleep as quickly as it could, checking to see when the other threads were done. Sleeping just to allow a context switch in the OS is bad, as it keeps landing back in the run queue too quickly. I changed the sleep to last about 200 ms, which is fine considering that's tiny compared to minutes or even hours of render time. Voila, another modest speed increase.

I adjusted the termination probabilities and compared renders, and for nearly the same rendertime I can now get a lot better quality. Here is a DOF render before and after the optimization, with the second one taking only a little longer than the first (but looks much better):

Both of these were rendered using the typical 400x400@500 rays/pixel settings.

Quick update

2007-06-16T22:11:00.000-07:00

I haven't posted anything for a while; I apologize for that. I have been busy with some other projects, and the last bite of the I-wanna-make-pretty-pictures bug wasn't enough to keep me away from my other obligations forever.

What is planned for the next round of motivation, in some semblance of priority order:

Scene XML loading, so I can stop tweaking code to create test scenes
A basic render node/client with a GUI that displays render results as they come in

This requires render job support, and so I'll make sure jobs are splittable in prep for rendering on more than one node
I think I'll use Gtk# for the GUI, as that is about as cross-platform for C# as it gets for GUI toolkits at the moment

I'll probably implement Pharr/Humphries infinite light sampling (see here for links) so that HDR images as backgrounds aren't so stinking noisy.
MTL handling during OBJ loading
Vanilla Monte-Carlo raytracer with lots of adjustable nobs so I can preview features faster. Sample controls will include:

Diffuse BRDFs, glossy BRDFs (separate nobs)
Shadow rays
View rays

Saving of images in some format other than .ppm so on Windows it has a clue what to do with it
Basic texture mapping (I might not get to anisotropic sampling of image textures this round, though), and bump mapping if I can swing it
Depth-of-field effects on the camera
Optimizations for the BIH tree to speed up rendertimes

Profile the dang thing and get loads of stats to see if our tree depths and balances are terrible
Do something about the large triangle problem that BIH tends to degrade heavily on
Better handling of ugly node splits
BVH2-style clipping off of empty space

Once I get to that point, I should be ready for some bigger and more interesting features in the next round. I'll have a solid enough basis to tackle:

Physically-based sky models
Displacement mapping (probably will do it with a micro-polygon displacement engine)
Subsurface scattering, volumetric rendering
Bi-directional path tracing
Metropolis Light Transport, on top of the bi-dir path tracing
Basic networking support, so we can distribute jobs out to potentially many nodes

If you aim for the sun, you'll probably hit the moon. I should at least make some progress, and have a blast doing it!

Hey, as I noted slyly above about "stinking noisy" renders using HDRI backgrounds, here's what I mean. I threw a lot of samples at this thing, and it still looks like crud due to the dynamic range:

New materials, support for spectral rendering

2007-04-11T22:08:00.000-07:00

RenderSpud now sports a few more surface materials. I fixed a number of bugs relating to the specular reflection and refraction code, and I also added a Torrance-Sparrow microfacet model, that can have different pluggable microfacet distributions. Currently, the only one supported is the Blinn distribution.

I also, under the hood, fleshed out my alternate color implementation that works based purely on wavelength and intensity data. It allows spreads of values across the spectrum, but also allows narrow (currently 10 nm wide) spikes. The representation is somewhat like a histogram of wavelengths. As a result, I needed a sane method to convert to RGB for writing the output render, so I added support for converting from wavelength to CIE XYZ representation, and then from there to RGB based on a color system and a whitepoint (with built-in color systems such as CIE equal-energy, HDTV, sRGB, NTSC, EBU, etc.). It supports whitepoints of: A, B, C, D50, D55, D65, D75, E, 9300, F2, F7, and F11. I'm hoping I got all of the ones anyone cares about.

Currently, RenderSpud does not do any splitting of rays or selection of wavelengths, so no prism or diamond effects yet. I don't think it will be too hard to do now that the basis is in place for it, but I really need to take this one step at a time and make sure I'm getting things right.

Without further ado, here is a screenshot showing off the newest materials, rendered into the sRGB color system. I forgot to record the rendertime, but I think it was around 2 hours using the usual configuration at 400x400, 500 rays/pixel. This shows a red blinn BRDF, a gold metal sphere using Fresnel's law for conductors, and a glass sphere with an IOR of 1.5 and using Fresnel's law for dielectrics:

TopMod, specular reflection

2007-04-06T16:40:00.000-07:00

A specular reflection BRDF has been sitting around for a while now, but while I was fixing up my BIH trees and ray intersection problems I held off on putting up renders using it. I figured I need to not be embarrassed to show my mistakes, so here is the first one.

Last night, I finally decided to learn how to use TopMod to create some additional interesting meshes for rendering. Hey, everyone is doing it. I can be trendy too! Actually, while TopMod is a little bit unstable, it is really interesting and fun to use. Kudos go to its creator, Ergun Akleman. I am particularly inspired by the meshes that the Jotero guys have come up with, so a nod goes to them too for their Ajax mesh and their incredible renders.

Anyway, 'nuff jabbering. It is not smoothed (so you see some faceting artifacts), but here is my first attempt at modern digital art (2 hrs 5 mins, 400x400 with 500 rays/pixel, 2 cores):

Instancing

2007-04-01T10:46:00.000-07:00

RenderSpud can now do instancing (use only one set of geometry to save memory, but multiple instances of the geometry in the scene). I didn't quite get the yellow and purple instances far enough apart, but oh well. There are quite a few fireflies in this render, and I'm not sure why, but I'll improve that over time if possible, while retaining the unbiased nature of the renderer.

Here it is, originally rendered at 800x800 with 500 rays/pixel, 1.63 million polygons, using both cores of my machine. It took a whopping 12 hrs, 5 mins at full size:

BIH trees, much progress

2007-03-31T22:28:00.000-07:00

I apologize for not posting recently, but I've been very busy. In my spare time, I have managed to improve the renderer in the last month or so quite a bit. First order of business, I fixed my boxes to have correct surface normals, and fixed a few other bugs relating to normals (8 mins 24 sec, 200x200 with 200 rays/pixel):

Sorry about the noisy render, but I had bigger fish to fry rather than wait around for this. Next up, I implemented Wachter and Keller's BIH tree (bounding interval hierarchy; see http://en.wikipedia.org/wiki/Bounding_interval_hierarchy for more info) as my first acceleration structure. The build times are very fast, and the intersection performance is pretty good. I haven't done anything really fancy, such as the BVH2 nodes, but I do a few tricks in mine to make it practical, and so far it has performed well as a result. Here is the obligatory render, showing my first real stress-test (1.5 hours, 400x400 with 500 rays/pixel, ~5000 polys, 2 parallel threads):

I know this was a long time coming, but I decided spheres and boxes just weren't exciting enough anymore, so I added triangle mesh support, and added a Wavefront OBJ loader. It isn't terribly sophisticated (no MTL support), but it gets the job done. Rendering models from here exposed a number of bugs with how I was accounting for intersections, and problems with my BIH implementation. Here's one of the in-progress renders of Jotero's Ajax bust showing a somewhat broken state (1.5 hours, 400x400 with 500 rays/pixel, 544k polys, 2 parallel threads):

It looks like it is sandblasted, but that is really due to not tracking a minimum t-value on my rays the way I should. I thought I could get away with it, but my joyride didn't last forever. After fixing all of the bugs I've run into, I was able to produce a much more pleasing result (1.5 hours, 400x400 with 500 rays/pixel, 544k polys, 2 parallel threads):

I implemented instancing today as well, but I still have a render going right now to show that off. I'll post again later with that render and its render time.

HDR dome lights

2006-11-26T18:26:00.000-08:00

RenderSpud is now able to read Radiance RGBE files (.hdr extension) for High Dynamic Range image support. Specifically, it knows about regular textures with HDR data, or spherical-format HDR textures used in sky domes.

I used a number of images from Paul Debevec's Light Probe resource, but some of them had a really high contrast ratio that made all the renderings very noisy. I opted to use Galileo's Tomb to show off, as it was still noisy, but not so bad that you couldn't tell the features and see the HDR lighting in action.

It is still quite noisy, however, because I use an extremely naive and brute-force approach; there is no importance sampling of the actual image yet. That will eventually get fixed, probably using either the approach by Matt Pharr and Greg Humphreys (detailed here), or a more complex approach as done by Jensen et al (seen here). Both will give RenderSpud a much faster convergence rate for lighting coming from HDR images. Currently, it took 1 hour 30 minutes to get to the image above (at 800 samples/pixel, 2 parallel rendering threads), so there is clearly much to be improved upon.

I haven't decided yet how much of a priority it is to implement EXR support. I had to hand-port the RGBE/.hdr file support to C#, and openEXR is much bigger and more complex. I'm likely to just do interop to use it, but I'm loathe to lose my cross-platform-ness of RenderSpud.

Boxes, Take 2

2006-11-06T07:40:00.000-08:00

I did a smoother/bigger render last night of that boxes scene, because the last one was so noisy. I'm really beginning to think that there is a problem with the normals or the intersection calculations, because the color bleeding seems to have some errors in it.

For example, there is some slight blue in the base of the pedestal in front of the blue box, when the normal points directly away and even hemispherical sampling would never reach the blue box. The left and right sides of the boxes seem particularly dark as well, indicating a similar problem. Finally, there is a green firefly or two on the top of the red box, when the normal (with hemispherical sampling) would not possibly allow it to sample the green box to even get a firefly that is green in this scene.

Come to think of it, maybe that's a white firefly, which is acceptable, but the blue color bleeding is still wrong...

Boxes!

2006-10-28T23:40:00.000-07:00

I got sick of the same old sphere primitive for testing, so I implemented boxes. Yeah, I know they are just so incredibly cool, but it'll hold me over until I get k-d trees in and can have arbitrary meshes rendering at a decent speed. The above is a 100 samples/pixel (noisy) render I used for testing. I'm not sure why the red and green box inside faces are so dark; I probably have a surface normal pointed the wrong direction. I'll get that fixed later.

Ambient lightdomes, lighting improvements

2006-10-28T18:26:00.000-07:00

The experimental renderer (currently it is using just path tracing) now has support for querying any type of light. The only lighting not supported yet in the actual renderer is area lights from arbitrary objects or meshes in the scene.

I finally discovered the bug in my diffuse (lambertian) material that was causing things to be too dark; I was computing the PDF of it multiplying by Pi instead of dividing by it. I had been driving up the power and proximity of my light sources to compensate, and when that got fixed my scenes were much brighter. Of course, it also spread the noise out as well, so my images take longer to get smooth. Oh, well, you win some and you lose some.

Ambient lightdomes are now in, although they only support a flat color. Next up is to add support for textures, and then I'll allow the lightdome to use HDR images. It will be pretty naive at first, and so any renders using HDR lightdomes will be rather grainy, but I'll eventually get around to doing some importance sampling on them so it's more intelligent.

Here are a couple of renders to whet the appetite.

The above render was 42 minutes rendertime (2 threads, 2 cores, Althon 4400+), with 400 samples/pixel.

The second render was 58 minutes rendertime (2 threads, 2 cores, Althon 4400+), with 400 samples/pixel. The difference in rendertime is almost entirely the closer camera (and thus more ray hits).

Wavelength-based light transport

2006-10-26T09:46:00.000-07:00

Wavelength-based lighting calculations allow for a level of realism that is hard to attain any other way. Imagine trying to simulate sunlight or a flourescent light (even if simplified) with just RGB color components. It would never do those types of light sources justice.

Accounting for a more granular distribution of wavelengths allows for more accurate lighting computations, at the cost of some speed. When adding lighting contributions at a surface, you can just add the computed colors by adding together each of the red, green, and blue components, respectively. It's very quick, and works well enough for quite a few scenes. But again, add sunlight, and it's hard to get the realism to stick. Add a harsh arc light, and it just isn't quite as believable.

In contrast, colors with varied wavelength distributions have to add whole functions of light values together, and also multiply them together, etc. As with all issues in computers, this is a hard problem to solve, so we usually just break it down into discreet (quantized) chunks. We could break down the visible light spectrum into 10 different wavelengths, or 20, or whatever. RGB colors just happen to break it down into 3 wavelengths, placed strategically to match the cones and rods in the human eye. Wavelength-based rendering, then, is just a generalized color representation that allows much more flexibility than the simplified RGB model.

There will be three methods of dealing with color implement ed in RenderSpud, such that we could choose at runtime (hopefully) which model to use. There will be the traditional RGB color model, for those who want speed and don't care about a loss in accuracy. Next will be a model with however many wavelengths spread evenly across the visible spectrum (how many that will be can change). This will be more accurate, but still could be improved upon. Last will be an adaptive model that should allow for the most accurate computations, although it will take some time before it is working well.

The adaptive wavelength colors will allow you to arbitrarily specify wavelengths and associated intensities (at the peak), and a spread value measuring how far from the peak value that still has light associated with it. Think of it like each wavelength location has a bell curve that peaks right at that wavelength, and the intensity is how high the peak is and the spread is two standard deviations away from the center of the curve. Adding these together gets a smooth graph of blended bell curves looking like the profile of smooth hills in china, with the occasional sharp high peak for flourescence. Dealing with these curves is the interesting part. Issues such as how do you represent white (unmodified color) in this scheme, and then combine colors from there? How bright is white?

Another nice thing about dealing with better color representations is that you can model interesting effects such as Cauchy dispersion and get prism effects, as well as add thin-film models (bubbles, woohoo!). Again, there is a cost hit for these, but for some scenes it is well worth it.

Bidirectional Path Tracing

2006-10-22T20:02:00.000-07:00

Bidirectional path tracing improves on regular path tracing in that it doesn't just construct ray paths from the camera; it also starts a ray from a random light source and constructs a path from there. Then, the two paths are connected at their ending bounce (vertex) locations, and then every other bounce location from the camera path is connected to every path vertex on the light path. All the lighting contributions from all of these connected paths are added up (and divided by the number of connected paths total).

As noted in PBRT, you get some statistical correlation between these paths, but the technique is still unbiased overall. It reduces variance quite a bit over vanilla path tracing. There are some optimizations you can make to the technique, but in a nutshell this is it.

Improvements to to make include: any time a light path is used that only has the one ray location on the light source itself, it shouldn't just use that point. Use another sample on the light to get some variation in the lighting calculations. Another improvement is to use multiple-importance sampling (MIS) to take into account all the different paths generated from the original light and camera paths, and reweight the lighting contributions accordingly. (Exactly how the weighting is done, I have no idea, so I'll have to research that more before I can implement that optimization.)

The nice thing about bidirectional path tracing is that you get many fewer paths that have low lighting contributions, so you can handle tricky indirect lighting better and reduce noise quite a bit for the same rendering time. For some scenes, of course, the extra overhead of increased calculations might not be helpful, but in general the decreased variance will far exceed any disadvantages over unidirectional path tracing.

Another interesting tidbit is that you can use these generated paths in both unidirectional and bidirectional path tracing as the basis for Metropolis Light Transport. Once a path is generated, you can randomly mutate and modify the path to walk around the scene, but it will perform more efficiently if the original paths are generated such that they have a higher lighting contribution. Bidirectional path tracing comes out on top here again.

Unidirectional Path Tracing

2006-10-22T15:08:00.000-07:00

While development is progressing, I've been implementing a test renderer that I can experiment with and exercise the rest of RenderSpud. Thus far it has been a simple path tracing implementation.

First, we set a number of rays per pixel to trace, and sweep across the camera's view plane sending that many rays per pixel, randomly situated within the pixel. I could improve this by stratifying those random samples, such that we divide up the pixel into sub-pixel rectangular regions and then randomly pick a spot within each of those. You get good coverage of the pixel area, while still retaining all the aliasing reduction benefits of random sampling.

Even better might be to concentrate more samples toward the center of the pixel, and fewer toward the outside (a tent filter, or cubic filter on the samples), but the increased computation time might not be worth it. Stratifying it at all makes the biggest difference.

As each ray is traced, it gets sent into the scene to find the nearest intersection. Once found (if any), it will pick a random light to sample and pick a random direction based on the material properties at the intersection point.

The light contribution is added in, but there are some neat details in that computation. It uses multiple-importance sampling, such that it sample the light according to its position, and it also samples the light according to the material properties. They are then combined using weights (from the power heuristic, if you are interested). This might increase noise some for simple lighting and material situations, but in general it has the opposite effect for most scenes.

When the material is sampled to find an outgoing direction, that then forms the next leg of the journey for the ray. RenderSpud uses Russian Roulette termination, so after about 4 bounces or so it will then have a 50% chance of being terminated at each bounce. If the ray is terminated, the light contribution from that bounce will be doubled, to account for all the bounces that didn't happen due to the termination. This introduces more variance, but in actuality keeps the algorithm unbiased (go figure).

Lather, rinse, repeat. Millions of times.

I'll talk about bidirectional path tracing next, and later I'll start getting into Metropolis light transport (MLT).

Multithreaded rendering

2006-10-20T14:07:00.000-07:00

Last night I was able to get a couple of threads running concurrently and rendering. I had to get my random number generator to have multiple actual number generators so that I didn't have to lock a mutex every time I needed a random number, but otherwise I set it up so that there is almost zero locking among the threads (making for uninhibited parallelism).

I didn't run any specific timed tests between one and two threads, but I will do that soon (as running with one render thread is pretty much the same as before I implemented the multithreading). I did add some console output telling the actual rendertime, though, so it should be easy enough to gather some data.

As part of my tests, I added more varied data to the test scene, so that I could give it something more difficult to chew on. I added all the sides of a box except the side where the camera is, another sphere, and some more color variations. This added more noise, and gives a more realistic rendering scenario, even though it is still very simplistic (all diffuse surfaces, etc.). I also made my (hard-coded) lights into area lights to soften the scene.

To counterbalance the noise, I ran a render that lasted 11 hours 15 mins. (on two cores) with 200 samples/pixel in path tracing. It's a very naive method, not doing any stratified sampling of pixels or lights, and is only uni-directional path tracing (bi-dir path tracing would require fewer samples for the same image quality, and is thus faster). Here's a scaled pic:

Biased vs Unbiased Rendering

2006-10-17T23:11:00.000-07:00

Coders have really funny reactions to buzzwords in the tech industry. We start hearing marketing people, and then subsequently users, talk about a particular technique or new technology as if it were just the greatest thing since sliced bread. They throw the terms associated with it around and make it sound really important. Sometimes it really is important, but most of the time we geeks just laugh about all the hype and talk.

Enter 'unbiased rendering.' Maxwell Render, Indigo, and a few others are able to do unbiased rendering, and quite frankly, they produce beautiful renders. I would argue, however, that being completely unbiased is not why, or is not the primary reason why. (On a side note, a beef I have with the Maxwell Render guys is that they said they were the only ones doing unbiased rendering, which is pure baloney. They didn't even do a google search or look in academia if they truly thought they were first.)

Let me explain why unbiased rendering is not always the primary feature to be sought after. Unbiased rendering quite simply means that there is nothing about the algorithms or techniques used that stops them from statistically having at least some chance of sending rays anywhere or accounting for any possible light. Basically, if you let an unbiased renderer run for long enough, it will eventually get the image right and converge on the correct solution. There is obviously some mathematical goup that allows you to prove a rendering technique is biased or not, but in a nutshell, that's it.

It sounds really nice, except for one thing: by ensuring a renderer is unbiased, you undoubtedly will introduce extra noise that then takes longer to get rid of. Yes, those famed fireflies are worse when using perfectly unbiased algorithms, for the same amount of render time. As with all things in life, it often boils down to compromises, balancing acts, and heuristics.

So, take a technique such as Jensen's two-pass photon mapping. For a sufficiently small gather radius and enough photons, it eventually becomes unbiased, but no one is going to jam billions of photons into an average scene (except those density-estimation guys, but that's mostly academic). Instead, the average artist will do 500,000 to 2 million photons in each of the caustic, global, and volumetric photon maps, and be done with it. Render times are relatively fast, the blotchiness usually goes away, and it looks pretty darn close to what an unbiased renderer will spit out. Oh, and throw in a little (biased, of course) irradiance caching, and the noise nearly disappears in some cases.

[UPDATE: the above "throw enough photons at it" to become unbiased is not true; it is still actually biased. This is because whatever photons you shot in the first pass are the only ones you can sample, and the aribitrary "hmm, I'll send out x number of photons" is where the bias is introduced. You can theoretically shoot enough photons that you'll converge extremely closely to what an unbiased rendering method would produce, but that unfortunately does not mean it is unbiased. Thanks, Thomas, for pointing that out.]

Matt Pharr and Greg Humphries in PBRT teach an important principle that is SO true in the real world of deadlines and demands: sometimes it is less objectionable to introduce bias to reduce noise (or render time) than to have a perfectly unbiased render.

But the truth be told, I absolutely love the output from unbiased renderers. If you spend enough time studying renders and rendering algorithms, those issues start to become more apparent. To satiate my split personality on the issue, RenderSpud will incorporate both unbiased rendering and biased rendering so that it is at least practical, but also accurate when the needs are such. At least four rendering algorithms will be present: Metropolis Light Transport over bi-directional path tracing (unbiased, if done right), vanilla bi-directional path tracing (unbiased), uni-directional path tracing (unbiased), and photon mapping (usually biased).

There will also be options in the unbiased renderers to introduce a little bit of bias in the interest of speed. Nick Chapman, who is the author of Indigo Renderer, has done similar things with his MLT rendering to keep it from hanging on the same pixel for too long (force accepting), for example. Another example would be to simply introduce a hard maximum ray trace depth instead of purely trying to terminate a ray's path using Russian Roulette. Again, a little bit of bias exchanged for some saner render times, but only when the user desires it.

History

2006-10-17T10:52:00.000-07:00

Since I have a link in my profile to the original RenderSpud, I think I had better explain some of the history of it. I originally got into raytracing back in high school when I did a Science and Tech Fair project on the differences between raytracing vs. traditional Phong/per-pixel shading (essentially, image-based vs. object-based rendering). My raytracer stunk, but it was enough to get me some awards and a pat on the back from Intel about it. They probably figured it wasn't typical for a junior in high school to be brave enough to make the attempt.

Right after I graduated from high school I wrote a raytracer in Java just to see if I could do it right. It worked much better, so I ported it to C++ to see what the difference in speed was; it was something on the order of 10x increase. I integrated it as the non-realtime renderer into a real-time graphics engine that I've slowly worked on over the years. It wasn't that great, but it was decent for a little whitted-style raytracer.

That got left that alone for a number of years, and in the summer of 2004 I bought Henrik Wann Jensen's book "Realistic Image Synthesis Using Photon Mapping" and (unsurprisingly) got the itch to write a photon mapping renderer. RenderSpud, the first incarnation, was born and within a few months it was doing color bleeding, caustics and all sorts of other goodies. A grad-level graphics class in the spring of 2005 helped motivate me to add HDR post-processing, a regular raytracer, a path tracer, and a bounding-volume-hierarchy implementation to speed up rendering. I also added butterfly sub-D surfaces and a few other tidbits.

Then, I graduated (from Utah State University) and got a job doing real-time rendering work and, well, my free time got whittled down quite a bit. At work, I get to hang out with artists sometimes, and many of them love a good offline/GI renderer. They have been encouraging me to keep up on RenderSpud, and I finally got the itch again to work on it some more.

I have come to the realization that yet again I didn't quite get it right (although I got closer with the first incarnation of RenderSpud). So, I started a rewrite, and I think I have a much better architecture for the render core than before. I also chose to do the rewrite in C# using .NET and Mono. Now, before everyone cries foul and screams something about it being too slow, hear me out. I am a C++ guy. I generally love C++, and I also am very concerned about speed. In a GI renderer, speed is paramount, and I wrestled with this decision for a long time. Suffice it to say, I did a lot of research, read a lot of flame wars on forums, that sort of thing to answer these questions: "Will the JIT engine actually optimize my math well enough?" "Will ngen.exe and AOT work well?" "Will garbage collection hurt my performance?" "Will Microsoft take over the world and make it so my stuff isn't cross-platform anymore?"

The answer to all the above questions is generally yes, or at least Microsoft wouldn't mind the last question being yes. In the meantime, Mono does a decent job (I have developed most of the render core using MonoDevelop in Ubuntu Linux 6.06), although the .NET runtime on Windows is definitely faster. Recall my render time from the first post? A 1680x1050 image at 9 samples/pixel, using path tracing with two point lights and three objects renders in about 5 minutes on my machine (which is an Athlon 4400+ dual-core, using only one core at the time). That doesn't seem very slow to me.

The other aspect of this is that, despite my general dislike of Microsoft and Windows (I grew up on Macs and really like Mac OS X and Linux), C# is a great language. It is well-designed, it is easy to write good solid code in it fairly quickly, and it is definitely more readable and maintainable than C++. The ONLY issue is of speed, and I think that generally C# will execute at 90-95% of the speed of an equivalent C++ app, not counting any initial JIT compile time. Much of that last 5-10% can be gained back by ahead-of-time optimization (AOT on Mono and ngen.exe in the .NET framework tools). That last 3-5% potential speed loss may be made up for by writing better code, and also by the fact that algorithmic changes and multi-threading/distributed rendering (see this post) will make a MUCH bigger difference in performance and reduction in rendertime.

Note briefly, I did not choose Java. Java is a great language, and I have had several jobs where Java was the language I used (the Space Dynamics Laboratory at USU, USU Math dept., my own projects). Java to me feels restrictive, and has some rules of the language, compiler, and runtime that drive me nuts. I realize that a great many people do not feel this way, and I say "more power to you" if you want to use Java. I'll write in Java when I have to, but that's about it.

Speaking of extra cores...

2006-10-16T15:22:00.000-07:00

I slyly mentioned using 'only one of my cores' in my last post. I managed to scrape enough money and permission from the wife to build a dual-core athlon SLI system, primarily for my graphics development. My old machine just wasn't cutting it anymore.

I have fairly substantial plans for distributed and parallel rendering support:

Threading. Full internal support for any number of rendering threads. It will break up the image to be rendered into multiple pieces and let each instance of the renderer go off and do its thing. The nice thing about ray-tracing-based renderers is that they are (usually) almost trivially parallelizable. There is usually very little locking that needs to be done during the actual rendering process, which means it actually can get close to 2x the speed on two cores, and 4x the speed on 4 cores, which is usually hard to do for a lot of algorithms when they are parallelized.
Server vs. nodes. Every instance of the renderer can have a list of other renderers that it can send parts of the rendering to take care of (much like it divides it up among threads). You can get a roomful of machines with an instance of the renderer running on each, set up your config files to talk to them, and voila, instant renderfarm.
Scalability and clustering. Each instance of the renderer can load its own configuration set, and thus set of nodes it can distribute any render jobs it receives. A renderer might hand out parts of a job to its nodes (and start rendering, itself), and each of those nodes might then hand out subdivided parts of their job out to yet more nodes, and so forth. Each node additionally might break up its own part of the job into pieces that it lets render among multiple threads. There is potential for a clustered rays-at-home setup where people can participate in render jobs all over the place, and make one of their machines the front-man that accepts render jobs and hands them out to other places within their LAN (to keep the WAN traffic minimized).

The trick with this stuff is to architect the rendering to be able to handle this sort of thing right from the beginning. I'll be adding in the threading support really soon, and the (rudimentary) rendering core already present understands that it might only be rendering a portion of the final image. Handling animations is something I plan on doing eventually as well, but that's harder.

The potato comes to life

2006-10-16T13:50:00.000-07:00

I finally made a screenshot last night that I thought was worthy of actually being seen in public. I brought it to work and let a few people see it; some were impressed, but one of the guys I wanted to impress seemed wholly unmoved by it. My next attempt will need to be more glamorous, I think.

Well, it wasn't much, I suppose. Two point lights, two diffuse spheres (one white, one red) on a diffuse ground plane. But it had color bleeding, and even fireflies in it. Yeah, yeah, most people aren't excited about fireflies and noise in their renders, but for me that meant it was actually working and mathematically doing what it should. I did the render at 1680x1050 with 9 samples/pixel. For a path-traced image, it actually looked rather smooth (despite the fireflies), and it only took about 5 minutes or so, and I was only using one of my cores.

So, I'm really feeling brave, and I am putting up some this render for the whole world to see. I'm definitely going to show the progression of renders as the functionality gets introduced, because it's pretty cool to watch the evolution of a renderer in development.

Here is a scaled-down version (and the fireflies are gone, sorry).