Path: EDN Asia >> Design Centre >> Consumer Electronics >> Off-screen video frame composition, decomposition
Consumer Electronics Share print

Off-screen video frame composition, decomposition

10 Jan 2014  | Sharath Bhat

Share this page with your friends

Due to the device limitation of a maximum one simultaneous encode and decode of HD frames, it is impossible for low-end tablets and smart phones to perform a conference call involving multiple decode and single encode of high-definition frames. In order to enable HD video conferencing between such systems, they involve a high-end video communication bridge to which all the devices dial in. Figure 3 illustrates the functionality of a video bridge.

Figure 3: Use case of a video conference using a bridge.


The video bridge receives multiple encoded input video streams from all the devices and then it decodes them to frames. The decoded frames from multiple parties are then composited to a single frame. The bridge generates the composited frame for every user device connected.

The composited frame from the video bridge for a user device will have video frames from every other user except the frames from that device. The bridge then encodes and streams the composited frames to all the users in that video conference call. Thus it is necessary to have at least a single bridge to enable video conferencing between all the devices. This bridge can be a high-end video card or can be one of these devices with more encode and decode capability.

As we see in the above example, the embedded devices that play the role of a video bridge need to have an off-screen video frame compositor component.


External display
On each of the devices that are connected to a video bridge in the use case of video conferencing explained above, it might be required to extract/decomposite the composited video frames from the incoming single video channels and display them on to various display devices or display them on different windows on-screen at the receiving end for presentation etc. Figure 4 illustrates such a use case, where in the frame from User A, C and D are displayed on the LCD screen of User A in separate windows on the screen and frames from User B will be displayed on an external display like HDMI TV.


Figure 4: Use case of routing frames to external display and as separate windows on screen.


3D video conferencing
In the case of 3D video conferencing application shown in figure 5, it is required to composite the incoming frames from the two camera sensors, and encode and stream it over the network. On the receiving end it is required to extract/decomposite the frames from each of the camera sensors, so that the fames of individual channels can be rendered on to different channels of display for stereoscopic vision.


Implementation
The off-screen frame compositor/decompositor functionality can be implemented in multiple ways:


Using platform independent algorithms
This method employs the generic algorithms which can be run of the CPU. Though this implementation provides portability, it results in heavy CPU loading. This is because the process of frame composition/decomposition involves following operations in general:

 • Scaling or resizing of the video frame. (Up scaling or down scaling)
 • Positioning of the input video frame in the output frame.
 • Cropping of input frame.
 • Cropping of output frame.
 • Alpha blending [Ref 13] of input channels.
 • Rotation of the input/output frames

Figure 5: Use case of 3-D video communication.



 First Page Previous Page 1 • 2 • 3 Next Page Last Page


Want to more of this to be delivered to you for FREE?

Subscribe to EDN Asia alerts and receive the latest design ideas and product news in your inbox.

Got to make sure you're not a robot. Please enter the code displayed on the right.

Time to activate your subscription - it's easy!

We have sent an activate request to your registerd e-email. Simply click on the link to activate your subscription.

We're doing this to protect your privacy and ensure you successfully receive your e-mail alerts.


Add New Comment
Visitor (To avoid code verification, simply login or register with us. It is fast and free!)
*Verify code:
Tech Impact

Regional Roundup
Control this smart glass with the blink of an eye
K-Glass 2 detects users' eye movements to point the cursor to recognise computer icons or objects in the Internet, and uses winks for commands. The researchers call this interface the "i-Mouse."

GlobalFoundries extends grants to Singapore students
ARM, Tencent Games team up to improve mobile gaming


News | Products | Design Features | Regional Roundup | Tech Impact