Virtual Camera Driver and Desktop Application for macOS
Video blogging has become a very popular activity over recent years. Some people shoot and publish videos as a hobby. Others make money by creating high-quality content. Creating professional video content requires a team and costly equipment. So, an Asian company came up with an idea of a device that could replace both.
They created a motion tracking device that works in tandem with any smartphone. The user attaches his or her phone to the device and launches the mobile application. The application uses the camera on the phone to take pictures and shoot videos. At the same time, the application controls the tracking device and its wide range of functions. Users can select cameras, change the tracking mode, choose the position of the figure in the frame, control brightness, and so on.
The movement tracking device can rotate 360° horizontally and 60° vertically. The mobile app uses artificial intelligence to recognize and follow objects by controlling the device. The face tracking mode makes the device keep a person’s face in the frame. The figure-follow mode allows users to select an object in the frame so that the camera follows it. The device has other functions as well, such as auto zoom, auto alignment, predictive follow, and more.
The device is relevant for video blogging, live broadcasting, and video conferences.
The customer already had a completed device and a mobile application. But they needed a macOS desktop application that would receive the video and audio data from the mobile application, process it, and transmit it to a virtual camera. The virtual camera would then send the video and audio to other applications. Our job was to develop such an application.
Another task was to develop macOS video and audio drivers for the virtual camera. The drivers had to work with a wide range of applications and websites, such as Skype, Zoom, YouTube, and Facebook to name a few.
Our team had to complete three tasks: to create a macOS desktop application, as well as to design a video driver and an audio driver.
The whole system was supposed to work as follows. The mobile application on a smartphone controls the movement tracking device and sends multimedia data (video and audio) to the desktop application via Wi-Fi. The desktop application demultiplexes the data into video and audio. After that, the data is sent to a virtual camera which is, in turn, connected to a video streaming application or website.
The team created the video driver by writing a DAL virtual camera plugin on the CoreMediaIO framework in Objective C and C++. Other video streaming applications can receive video data from this plugin which serves as a virtual camera.
The driver allows the desktop application to control the camera feed, namely brightness, contrast, color, video settings, and so on.
The audio driver was developed on the CoreAudio and IOKit frameworks in Objective C and C++.
The installation packages of the desktop application and the drivers for macOS were created in PKG.
Major Issues Resolved
One of the first problems the team faced was related to video data transmission from the desktop application to the DAL plugin. The video frames from the application required a special interface to be sent to the plugin. The transmission failed when dealing with large amounts of data. Initially, the team tried to create this interface with pasteboards. However, this attempt caused a lot of problems. So, we decided to use a socket connection instead.
Another major problem was synchronizing the video and audio data flows. We solved this issue algorithmically.
When developing the desktop application, we faced a demultiplexer issue: this function did not work properly. Also, sometimes videos did not fit the screen. But both these problems were solved when the team developed the Windows version of the application. At this phase, they did not need to be addressed.
The Scope of Work
- Video kernel extension (KEXT) development for macOS
- Audio kernel extension (KEXT) development for macOS
- Desktop application development
- The desktop application was developed on the Electron framework.
- The application was implemented using C++ and TypeScript.
- We developed the video and audio KEXTs in Objective C and C++.
- The video KEXT was implemented using the CoreMediaIO framework.
- The audio KEXT was developed on the CoreAudio and IOKit frameworks.
Integra Sources developed a desktop application for macOS, as well as video and audio drivers for the virtual camera so that the application could receive data from the mobile application and transmit it as a standard video to other applications and websites.
You might also like...
Graphing Calculator Development
The Integra Sources team has created a graphing calculator for advanced math, science, and coding. The device has a 2.8-inch color display.
Wearable Blood Pressure Monitor Development
The portable blood pressure monitor measures blood pressure, heart rate, blood saturation, and posture and transmits the data to a tablet app.