Firmware Development for Augmented Reality Glasses

Solution

We developed a driver for the accelerometer and gyroscope that supports data transmission via the I2C protocol.

The client’s team used a standard display controller in their device which is compatible with a Raspberry Pi driver. We used it as a basis for creating the custom driver for the customer. Our developers added the capability to transmit data to four displays simultaneously and to adjust image contrast in real time within a range from 0 to 255.

Since Android operating systems do not allow applications to interact directly with drivers, we were also required to develop a HAL (Hardware Abstraction Layer) module. HAL is a software layer that acts as an intermediary between user applications and the driver. It provides Android applications with a convenient interface for interacting with hardware and translates their calls into commands that the driver can understand, and thus directly control the display.

We built a custom OS image based on the Android Open Source Project, incorporating the developed driver and HAL module into it.

Major Issues Resolved

Bypassing the Linux Security System

The smart glasses are designed as a closed device: no access to the firmware, no ability to install applications on it. They were intended for corporate clients, not for the mass market. Therefore, the advanced functionality of SELinux—the built-in security system that controls application and module access to OS resources—seemed excessive. So, we disabled it to speed up development and reduce the costs.

8-bit Mode and Performance

The device used customized displays that operate in 24 bits per pixel mode by default but don’t have a color mask. To ensure the image is displayed correctly on such screens, it was necessary to implement support for an 8-bit grayscale mode in the driver.

We also paid special attention to optimizing the software performance to enhance the device’s speed and responsiveness. The requirement was at least 10 fps, but the team managed to achieve an average frame rate of 20 fps while operating four displays simultaneously.

Pixel mapping tables

Due to the specifics of the device’s optical system, it was necessary to implement support for pixel mapping tables in the driver.

Scope of Work

• Accelerometer and gyroscope driver development
• Video driver development
• HAL module creation
• Android Open Source Project customization

Technologies

• The C language for driver development.
• C++ and Java for the HAL module creation.

Result

Thanks to our expertise, the customer was able to complete the firmware for their augmented reality glasses, meeting all stated requirements: multiple color modes support, simultaneous operation of four displays, real-time contrast adjustment, and calibration tables integration.

With Integra Sources’ support, the client:

• Compensated for the lack of domain expertise and accelerated device development by several months;
• Obtained reliable software fully compatible with the hardware platform and Android specifics, which improved the stability of the device;
• Was able to exceed the expected performance: 20 fps instead of required 10 fps.

Integra Sources helped the customer speed up the development of the new device, while our team gained experience working with an interesting cutting-edge technology. The delivered solution lays the foundation for the client’s further product development in the field of high-tech electronics.