Explore alternatives to FXAA (SMAA / TAA / SHAA)

[bryanedds]

Our visual fidelity can be quite decent thanks to PBR, but the next level of improvement probably lies in improved anti-aliasing techniques.

[bryanedds]

Convo about specular-highlight anti-aliasing (SHAA?) -

be me -

the trade-off is a lot of eye-catching specular aliasing for SMAA
isn't there a type of aliasing that can detect such sparkling fragments specifically and blur just them?
compare the color intensity differentials in a 3x3 sample and blend the bright one conditionally.

be chat-gpt -

You're on the right track! Specular aliasing often results in "sparkling" or "flickering" effects in the highlights, and some advanced anti-aliasing methods specifically target those high-intensity areas to reduce this type of artifact without blurring the entire image. What you're describing — detecting areas with high intensity differential and selectively blurring them — is a targeted approach that can be very effective in reducing specular aliasing without the more general blurring applied by techniques like SMAA or FXAA.

Let's break down the concept:

Targeted Anti-Aliasing for Specular Highlights

Instead of applying general post-processing anti-aliasing to the entire scene, you can target specular aliasing directly by analyzing the intensity of the specular highlights in a localized manner and applying selective filtering or blending to those specific areas.

Basic Steps to Implement This Approach:

1. Intensity Differential Detection:
   • A key idea is to detect "sparkling" or "flickering" specular highlights, which usually result from high-frequency detail in the specular reflection. These highlights typically have sharp intensity changes between adjacent pixels.
   • To detect this, you can compute the intensity differential between neighboring pixels, such as a 3x3 neighborhood. This gives you a sense of whether there's a significant intensity difference that might be causing aliasing artifacts.
     • Formula: For each pixel, calculate the difference between the central pixel and its 8 neighbors:

where I is the intensity of the specular component at each pixel.

2. Conditional Blending Based on Intensity:

   • Once you have the intensity differential for each pixel, you can threshold this value to determine whether it indicates an aliasing artifact. If the intensity differential is above a certain threshold (indicating high-frequency specular reflections), you can apply a selective blur to smooth out the aliasing.
   • This can be done by blending the pixel with its neighbors, using something like a Gaussian blur or a simpler bilateral filter that preserves edges while smoothing the highlights.

3. Edge-Respecting Filter:

   • To avoid blurring edges or areas where aliasing is not present (like sharp geometry edges), you can use an edge-preserving filter like a bilateral filter or guided filter. These filters will apply smoothing selectively, depending on both intensity and spatial proximity.
   • This means you'll apply the blur only in areas where the specular highlights have high intensity differential, effectively reducing the shimmering effect in those highlights without affecting the entire image.

Example Implementation:

Here's an outline of how this process could be implemented in a shader:

1. Sampling Intensity:

   • Sample the specular highlights in a 3x3 region around each pixel.

2. Intensity Difference Calculation:

   • Compute the intensity difference between the central pixel and its neighbors.

3. Conditional Blurring:

   • If the intensity difference exceeds a certain threshold, apply a blur to the specular highlight (you could use a Gaussian kernel or bilateral filtering based on the intensity).
   • Optionally, you could mix the blurred result with the original specular value based on how strong the aliasing is.

4. Edge Preservation:

   • If you're using a bilateral filter or something similar, ensure that the blur is applied only to high-intensity areas while preserving edges where specular aliasing isn't an issue.

Shader Code Pseudocode:

// Define constants for the neighborhood size and intensity threshold
const int neighborhoodSize = 3;
const float intensityThreshold = 0.1; // This is the threshold for detecting aliasing

// Sample the 3x3 neighborhood
vec3 currentSpecular = texture(specularTexture, fragCoord);
vec3 neighbors[8];
int i = 0;
for (int x = -1; x <= 1; ++x) {
    for (int y = -1; y <= 1; ++y) {
        if (x == 0 && y == 0) continue;
        neighbors[i++] = texture(specularTexture, fragCoord + vec2(x, y));
    }
}

// Compute intensity difference between the center and neighbors
float intensityDiff = 0.0;
for (int i = 0; i < 8; ++i) {
    intensityDiff += abs(length(currentSpecular) - length(neighbors[i]));
}

// If the intensity difference exceeds a threshold, apply selective blurring
if (intensityDiff > intensityThreshold) {
    vec3 blurredSpecular = vec3(0.0);
    float blurWeight = 0.0;
    for (int i = 0; i < 8; ++i) {
        float weight = exp(-length(currentSpecular - neighbors[i]) * 2.0);
        blurredSpecular += neighbors[i] * weight;
        blurWeight += weight;
    }
    blurredSpecular /= blurWeight;
    
    // Mix the blurred specular with the original based on the intensity difference
    currentSpecular = mix(currentSpecular, blurredSpecular, 0.5);
}

// Output the result
fragColor = vec4(currentSpecular, 1.0);

Key Points:

• Detection of high-frequency details: By comparing intensity differentials between neighboring fragments, you can identify areas of high-frequency detail that are likely to cause specular aliasing.
• Selective blurring: Only blur the high-intensity specular highlights that exhibit aliasing behavior, leaving other parts of the image unaffected.
• Edge-preserving: Use a filter that preserves edges while smoothing specular reflections, such as bilateral or guided filters.

Summary:

The approach you're proposing — analyzing the intensity differentials in a neighborhood and selectively applying a blur to high-intensity areas — is a great way to reduce specular aliasing artifacts in a targeted manner. This allows you to smooth out only the problematic specular highlights (the sparkling fragments) without affecting other parts of the image, leading to a cleaner, sharper result compared to general anti-aliasing methods like SMAA or FXAA, which often blur the entire image.