SSR

Introduction

In typical deferred rendering engine, GBuffer will include all pixel's normal and depth information. As we all know, pixel position is just the eye-ray in view space. Then, it is possible to do ray tracing in screen space. This feature has already been supported by many games and game engines recently([Bartlomiej14]). 

     1. CryTek3 

    The Art and Technology behind Crysis3: http://www.crytek.com/download/fmx2013_c3_art_tech_donzallaz_sousa.pdf

    Update in RYSE: http://www.crytek.com/download/2014_03_25_CRYENGINE_GDC_Schultz.pdf

    2. Unreal4

    https://docs.unrealengine.com/latest/INT/Resources/Showcases/Reflections/index.html

    3. Killzone: Shadow Fall

    Reflections and volumetrics of killzone shadow fall: http://advances.realtimerendering.com/s2014/valient/Valient_Siggraph14_Killzone.pptx

    4. Assassin's Creed 4: Black Flag

    Assassin's Creed 4: Black Flag Road to next-gen graphics: http://bartwronski.files.wordpress.com/2014/03/ac4_gdc.pdf

    5. Thief3

    Reflection System in Thief: http://advances.realtimerendering.com/s2014/eidos/eidos_siggraph_2014.pptx

This method is very flexible,suit for any kind of surface and has almost constant performance.

General Solution

All reflection system is combined by three levels: local reflection, local cubemap, global cubemap. 'Local reflection' is calculated in screen space by ray trace depth buffer, this is always generated in realtime; 'Local Cubemap' is a cubemap for local area environment, is generated offline; 'Global Cubemap' stand for global environment and is generated offline too. Following is a typical scene with all reflection environment.

RED

Large radius. Reflects bulk of the level and the background.

BLUE

More localized reflection. Captures individual rooms.

GREEN

Detail reflection with small radius. Placed in areas where small important details are required.

Because SSR works on screen space, in many cases, the reflection point information will be missed. Then local cubemap will be used to get information. If there's no local cubemap, global cubemap will be used. Local cubemap is generated offline. Artist need place probe to help with setting properties to generate it. This will cause some issue for our UGC game type, maybe some similar feature need to be implemented in in-game editor. Normally, global cubemap is the skybox cubemap.

Ray Trace

 Screen space ray-tracing is 2.5D ray-tracing through depth-buffer. Raw ray trace will cost a lot of GPU time and should be optimized in practice.

1. Quarter resolution depth
   On additional pass to down sample depth buffer to 1/2 * 1/2 size. Performance will be 4X time fast. By keeping last 4 frames quarter depth buffer, temporal supper sample can be used to acquire original visual quality.
2. Hierarchy depth buffer( Killzone: Shadow Fall )
   Generate hierarchy of depth buffer( min/max ), ray trace can be done with large steps. [Michal09]
3. Const sample count( Unreal4 )
   Change reflection ray length according how far the pixel from eye, always average sample const points in reflection ray. Thus, performance are configurable by number of sample points. It become more easy to balance performance and visual quality.

Some issues will affect ray trace result a lot:

1. How to determine intersection point?
   Depth delta value is a good and fast parameter to check intersection point. But it's hard to define a perfect threshold for it. Too large threshold will generate obvious wrong result, too little threshold cause too much intersection result is skipped.
2. How to move forward ray trace sample point?
   Simple constant step will have better performance, inconstant step will have better result.
3. How to avoid intersection with start point?
   Normally, ray will stop at start point immediately, because start point is a intersection point too. Some offset should be added before following ray tracing. The offset value need be determined. Too large value will ignore near intersection points.
4. How to get intersection result?
   It's very easy to treating last ray tracing sample point as intersection point. To achieve better result, need more ray tracing with small steps or interpolate between last two sample points.

Glossy

Roughness is a very import parameter to determine reflection property of surface. Very similar, different roughness surface will have different reflection effect. Screen space ray trace only works for ideal reflection. Some more operation should be added to process roughness/glossy effect.

1. Hierarchy reflection result
   After reflection result is generated by screen space ray trace, Generate a serial mipmap by blur and down-sampling. Then, fetch result from different mipmap level according to roughness;
2. Dither reflection ray
   Using multiply reflection ray, all of them will be dithered and average blend all screen space ray trace result.
3. Direct blur with roughness
   After getting raw reflection result, using roughness to do blur operation, more large roughness will have more large blur kernel size. For better quality, the blur can be applied in separate horizontal and vertical direction.

Remove Noise

Normally, the raw reflection result will be full of noise. They come from small glossy surface, lack of information or precision error. Some process will take care of them.

1. Remove noise points
   All small noise points can be removed;
2. Blur reflection result
   Blur reflection result by Average blur, Gaussian blur, Bilateral blur or Separable blur.
3. Apply temporal filter
   Add temporal filter, it will upgrade visual result a lot.

Temporal filter

Temporal supersampling and antialiasing will help improving graphic quality a lot with very tiny performance cost.  http://bartwronski.com/2014/03/15/temporal-supersampling-and-antialiasing/

Implemented Solution

In current prototype, we combine advantages in several game engine, and solve some issues.

All C++ code and HLSL code can be found in ScreenSpaceReflection prototype. 

Prepare

What ever front rendering or deferred lighting, normal and roughness value need to be wrote into gbuffer for following usage. All SSR render pass should be applied after tonemapping for better visual quality. Still need to evaluate after integrating into final engine. Besides, screen size information and project/unproject matrix are needed.

Screen Space Ray Trace

Whole screen space ray trace algorithm is ported from unreal 4 engine, ray trace line is 1.5 screen size (u,v[0,1]). It's divided into 'NUM_SAMPLES_PER_RAY' sample points. GPU will check depth difference of all sample points from start points. 'GLOSSY_RAY_TRACE' is defined for adjusting reflect direction a little for better visual result. We always use only one ray for one pixel, because larger 'NUM_SAMPLES_PER_RAY' value will have better result. In unreal4 original ray trace, a lot of ray will stop at start pont area (first four sample points), so, most of reflection color is start point color. By adding 2.5 offset for all ray, most ray will intersect to assumed point. Then come out an obvious band visual issue at start area. Short ray trace step will reduce the issue, so, we change ray trace step dynamically. First 1/4 ray trace sample points will have very small step.

Ray trace cost many GPU time, to optimize performance, some pixel will be early out by checking roughness and angle between eye with face normal. They also improve visual quality. Another optimization is keeping screen space ray trace result to 960x544 pixels and ignoring original screen size. This will keep whole GPU time usage low than 2ms. During my testing, small gbuffer texture size will have better performance (texture cache?), final implementation should reuse some small texture with other features.

Separable Blur

The raw reflection color result will be full of noise points normally. To have better visual quality, two separable blur passes are added. By testing several filtering, average 7 near pixels color will have better result, it will blur noise points into a combined shape.

Mipmap Generation

Differ from unreal4 and similar with AC4 & Killzone, we will generate several mipmap level of blur reflection texture. This texture will be used to simulate roughness result.

Postprocess Combine

Combine lighting color with reflection color, roughness will used to fetch different mipmap level of reflection texture. By adding 0.5 bias to mipmap level, reflection color looks a little blur and looks better. How to blend reflection color with original color still need some tweak.

Following is a screen shot of current implementation:

BTW, To remove flicking, remove using frame number as part of random seed, but if we want to simulate rainy surface, this can be added back.

Conclusion

By using unreal4 similar ray trace, it avoid some bad band issue. It's also very flexible to balance performance with visual quality. After adding separable blur for reflection color buffer, noise points cloud become into blur shape, it looks more nature. But still the method need balance between performance and quality. In some view angle, the reflection color will exist some error caused by precision (similar with all existing implementation).

Reference

1. [Bartlomiej14] The future of screenspace reflections: http://www.gamasutra.com/blogs/BartlomiejWronski/20140129/209609/The_future_of_screenspace_reflections.php
2. [Cryteck] Crysis 2 DirectX 11 Ultra upgrade Effects Comparison: http://www.geforce.com/whats-new/articles/crysis-2-directx-11-ultra-upgrade-page-2
3. [Michal13] Killzone: Shadow Fall Demo Postmortem: http://www.guerrilla-games.com/presentations/Valient_Killzone_Shadow_Fall_Demo_Postmortem.pdf
4. [Bartlomiej14] Assassin's Creed 4: Black Flag Road to next-gen graphics: http://bartwronski.files.wordpress.com/2014/03/ac4_gdc_nomovies.pptx
5. Unreal4 Screen Space Reflection: https://docs.unrealengine.com/latest/INT/Engine/Rendering/PostProcessEffects/ScreenSpaceReflection/index.html
6. [Michal09] Quadtree Displacement Mapping with Height Blending: http://drobot.org/pub/M_Drobot_Programming_Quadtree%20Displacement%20Mapping.pdf
7. [Doug11] Dynamic Resolution Rendering Article: https://software.intel.com/en-us/articles/dynamic-resolution-rendering-article

Some general website:

http://www.iryoku.com/

http://blog.selfshadow.com/

---

OLD content

Warning: Keep updating for screen-shots from trials fusion game.

some additional references:

• Taking Killzone shadow fall image quality into next generation: http://www.gdcvault.com/play/1020770/Taking-Killzone-Shadow-Fall-Image
• Screen space cone tracing for glossy reflections: http://www.tobias-franke.eu/publications/hermanns14ssScreen Space Cone Tracing For Glossy Reflectionsct/
• Improving Unreal engine 4 Screen space Reflections: http://oliverm-h.blogspot.com/2014/08/ue4-improving-ssr.html?spref=tw
• A gentle intorduction to bilateral filtering and its applications: http://people.csail.mit.edu/sparis/bf_course/
• Wiener filter: http://en.wikipedia.org/wiki/Wiener_filter
• The future of screenspace reflections: http://www.gamasutra.com/blogs/BartlomiejWronski/20140129/209609/The_future_of_screenspace_reflections.php
• Filtering and noise reduction: http://www.matdat.life.ku.dk/ia/sessions/session2-4up.pdf
• Image Filtering: http://eeweb.poly.edu/~yao/EE3414/image_filtering.pdf
• Fast and Efficient Algorithm to Remove Gaussian Noise in Digital Images: http://www.iaeng.org/IJCS/issues_v37/issue_1/IJCS_37_1_09.pdf
• Variance Invariant Adaptive Temporal Supersampling for Motion Blurring: http://webdocs.cs.ualberta.ca/~dneilson/publications/dneilson.variance_invariant_motion.pdf
• Advanced Topics in Computer Graphics Sampling Techniques: http://web.cs.wpi.edu/~emmanuel/courses/cs563/S10/talks/wk3_p1_wadii_sampling_techniques.pdf
• Temporal supersampling, flipquads and real time raytracing: http://blog.demofox.org/2014/08/11/temporal-supersampling-flipquads-and-real-time-raytracing/
• Temporal supersampling and antialiasing: http://bartwronski.com/2014/03/15/temporal-supersampling-and-antialiasing/
• Advance real time rendering 2013: http://advances.realtimerendering.com/s2013/index.html
• Realtime global illumination and reflections in Dust 514: http://advances.realtimerendering.com/s2012/CCP/Malan-Dust_514_GI_reflections(Siggraph2012).pptx