Problem

Augmented reality (AR) systems are widely used in social media, virtual meetings, and interactive applications, yet little foundational research has been conducted on how AR renders darker skin tones compared to lighter tones. Emerging reports suggest that darker skin tones may appear washed out, less visible, or inconsistently displayed, but the underlying mechanisms behind these transparency effects remain largely unexplored.

Existing AR models rely on depth detection, color processing, and lighting adaptation to track and render users, yet these systems are often optimized using limited skin tone datasets, leading to potential biases. Far less is known about how AR systems interpret shade, transparency, and color accuracy across different skin tones, particularly in varying lighting and background conditions.

This study seeks to establish baseline data on AR transparency effects through controlled experiments and qualitative perception assessments, providing the first empirical evidence on how AR systems process and display different skin tones. By systematically investigating factors such as lighting, background complexity, and shading, this research aims to fill a critical gap in AR accessibility and inform more inclusive AR development.

Process

User Study Design: Judging Skin Tone Representation in AR

To analyze how AR represents different skin tones, I designed a controlled user study where participants evaluated the accuracy of different shades in AR under varying lighting conditions.

• Skin Tone Representation in AR: Participants evaluated how AR rendered different shades, ranging from light to dark, across various conditions.

• Lighting Conditions: Participants compared AR display accuracy under low, bright, and mixed lighting to assess when transparency effects were most pronounced.

• Background Complexity: Participants tested AR in front of simple and complex backgrounds to determine whether background details influenced skin tone visibility and accuracy.

• Transparency Effect Analysis: The study examined when and under what conditions darker skin tones appeared faded, washed out, or transparent in AR.

User Feedback: Evaluating Perceived Transparency in AR

After testing AR effects, participants provided structured responses on:

• How accurately different shades were represented in AR across lighting conditions and backgrounds.

• Whether darker tones appeared less defined or more transparent.

• How lighting and background complexity influenced the visibility of certain shades.

By collecting both quantitative AR tracking data and qualitative perception ratings, the study provided a data-driven and user-informed understanding of AR transparency effects.

Results & Ongoing Research

This study provided foundational insights into how AR systems detect and render skin tones, revealing that darker tones are more prone to transparency effects, particularly in specific environmental conditions. While AR transparency issues were previously thought to be influenced by depth detection, our findings indicate that depth has no effect on shade detection accuracy or transparency.

By combining AR tracking data with user perception ratings, we confirmed that background complexity and shading significantly impact AR’s ability to accurately render darker skin tones. Contrary to expectations, more complex backgrounds and higher shading levels actually improve AR skin tone detection, rather than making transparency worse. These findings suggest that background-aware processing models and shading adjustments may play a larger role in improving AR inclusivity than previously assumed.

Building on these results, this ongoing research is now focused on:

• Refining grayscale-to-color mapping to enhance AR’s ability to differentiate skin tones across varying conditions.

• Optimizing transparency algorithms to reduce fading effects, incorporating insights on shading and background complexity.

• Exploring background-aware rendering techniques to determine how AR can better distinguish skin tones in real-world environments.

• Testing additional lighting conditions to further understand how AR performance can be stabilized across different user settings.

These findings lay the groundwork for improving AR skin tone representation, with continued research aimed at ensuring all users experience accurate and equitable digital representation, regardless of skin tone or environmental conditions.