Skip to main content

Post-processing

Real-time selfie segmentation can be susceptible to flickering, which occurs when the foreground subject appears to flicker or shift around in the video. This can be caused by variations in lighting, motion blur, or other factors that affect the image quality. Flickering can be distracting and can negatively impact the user experience, making it important to address.

Post-processing is a technique used to improve the quality of the selfie segmentation output and reduce the likelihood of flickering. Post-processing involves applying a series of image processing techniques to the output of the selfie segmentation algorithm.

Post-processing selfie segmentation can have an impact on the CPU, as it requires additional processing time and resources to apply the image processing techniques.

Enabling/Disabling the post-processing​

If you want to enable/disable explicitly the post-processing, you can use the selfieSegmentationType field of the renderingOptions.

As explained https://vonage.github.io/ml-transformers-docs/docs/renderers#post-processing, post-processing is only available for WebGL.

// Disable
const webglProcessor = await createVonageMediaProcessor({
    transformerType: 'BackgroundBlur',
    renderingOptions: { 
        type: RenderingType.WEBGL,
        selfieSegmentationType: WebglSelfieSegmentationType.FAST
    }
});

// Enable
const webglProcessor = await createVonageMediaProcessor({
    transformerType: 'BackgroundBlur',
    renderingOptions: { 
        type: RenderingType.WEBGL,
        selfieSegmentationType: WebglSelfieSegmentationType.PRECISE
    }
});

Default value​

When the post-processing option is not explicitly specified, it defaults to enabling post-processing only for the background image and background video.

Since blurred backgrounds typically have less contrast between the foreground subject and the background, any flickering or visual artifacts that do occur are less noticeable. As a result, there may be less of a need to apply post-processing techniques to the foreground subject.