Meta's Patent for a Multi-Directional Waveguide Eye-Tracking System

Introduction

Eye-tracking functionality has the potential to significantly enhance the services and interaction quality provided by head-mounted devices. However, when imaging the eyes from the periphery, eyelashes and eyelids can obstruct and degrade the signal quality obtained from the eyes. Imaging the eye's light reflections from directly in front of the eye is a much better position, but placing a camera directly in front of the eye can obstruct the user's line of sight and may reduce the quality of experience when using a head-mounted display. Meta has proposed a solution to this problem with its multi-directional waveguide eye-tracking system, as detailed in a recent patent application.

Multi-Directional Waveguide Eye-Tracking System

The multi-directional waveguide eye-tracking system proposed by Meta aims to guide light from the lens component to image sensors located on or within the frame of the head-mounted device. The waveguide system can include one or more waveguides configured to direct light from a window region to various image sensors, thereby expanding the window region from which reflections can be detected.

System Components and Functionality

In one embodiment, the head-mounted device includes a multi-directional waveguide eye-tracking system. This system can guide light reflections from the user's eyes to multiple image sensors, enabling interference-free and in-field imaging of the eyes. The system comprises a first and second waveguide system, configured to receive light from different parts of the eye box region and direct it to the first and second image sensors.

• First Waveguide System: This includes a first waveguide, a first input Diffractive Optical Element (DOE), and a first output DOE. The first input DOE is configured to couple light from the first part of the window region and guide it in a first direction within the first waveguide. The first output DOE then couples the light out to the first image sensor, which is coupled to the frame at a first position.

• Second Waveguide System: This includes a second waveguide, a second input DOE, and a second output DOE. The second input DOE is configured to couple light from the second part of the window region and guide it in a second direction within the second waveguide. The second output DOE then couples the light out to the second image sensor, which is coupled to the frame at a second position.

In one embodiment, the first and second waveguide systems share a single waveguide, which includes two input DOEs configured to guide light in the first and second directions. In another embodiment, the first waveguide includes multiple input DOEs, and the second waveguide includes multiple input DOEs, further increasing the light detection from the window region.

Head-Mounted Device Configuration

Figure 1 illustrates an example of a head-mounted device with a multi-directional waveguide eye-tracking system. The system includes waveguide systems 101 and 102, which are configured to couple light from the user's eye field of view. Waveguide system 101 includes input DOE 103, which couples light and directs it towards image sensor 104. Waveguide system 102 includes input DOE 105, which couples light and directs it towards image sensor 107. The input DOEs 103 and 105 can operate together to expand the window region from which light can be coupled and imaged.

The waveguide systems use Total Internal Reflection (TIR) to guide light from the input DOEs to the output DOEs, which then couple the light out to the image sensors. The image sensors can be CMOS sensors and may be configured to capture images of invisible light. They are coupled to the frame and can be partially or fully enclosed within it.

The head-mounted device also includes multiple light sources, such as infrared LEDs, which are configured to emit light towards the window region. A controller is communicatively coupled to the image sensors and light sources, controlling the illumination timing of the light sources and synchronizing them with the operation of the image sensors to capture reflections of the emitted light.

Eye Environment and Operation

Figure 2 illustrates the eye environment, showing the head-mounted device and the user's eye within the window region. Scene light from the scene side passes through the lens component and the waveguide system to the window side and the user's eye. The light sources emit non-visible light, which is coupled by the waveguide systems to illuminate the eye. The waveguide systems are configured to couple light from different parts of the window region, covering a larger area than a single input DOE.

The waveguide systems can extend into the frame of the head-mounted device, with the output DOEs enclosed within the frame and optically coupled to the image sensors. The image sensors can be read via communication channels to obtain image data for eye-tracking operations.

Various Implementations

Figures 3A, 3B, 3C, and 3D illustrate various implementations of the multi-directional waveguide eye-tracking system. In one example, the lens component includes a waveguide optical layer and a display optical layer, with the waveguide systems integrated into these layers. The input DOEs can be configured to guide light in opposite directions towards the waveguide's opposite ends, with gaps between them to couple light from different parts of the window region.

In another implementation, the waveguide systems are located in different optical layers of the lens component, allowing light to be directed to different positions within the frame. Additional waveguides with multiple input DOEs can be included to further increase light detection from the window region.

Eye-Tracking Process

Figure 5 presents a flowchart of the process for eye-tracking using the multi-directional waveguide system. The process involves directing light towards the window region to illuminate the user's eye, receiving reflected light with the waveguide system having multi-directional DOEs, coupling light in different directions with the input DOEs, coupling the light out to the image sensors with the output DOEs, receiving the light with the image sensors, and determining the orientation of the user's eye based on the image data received from the image sensors.

Conclusion

Meta's multi-directional waveguide eye-tracking system offers a novel approach to enhancing eye-tracking capabilities in head-mounted devices. By guiding light reflections from the user's eyes to multiple image sensors through a sophisticated waveguide system, the system can achieve interference-free and high-quality imaging of the eyes. This technology has the potential to significantly improve the user experience in AR/VR applications by enabling more accurate and reliable eye-tracking.

The patent application for the "Multi-Directional Waveguide Eye-Tracking System" was initially filed in March 2023 and was recently published by the U.S. Patent and Trademark Office. Further details can be found in the original patent document.