Added Camera overlay to the pc client (now also with video)

Firmware/software/electronics/mechanics
Post Reply
omwdunkley
Expert
Posts: 162
Joined: Thu Jun 06, 2013 9:56 pm
Location: Munich

Added Camera overlay to the pc client (now also with video)

Post by omwdunkley »

EDIT: Added video

Hey guys!

Many of you asked / pmed / etc me about my live HUD (http://forum.bitcraze.se/viewtopic.php?f=6&t=491#p3145) so I thought I would integrate a very simple version directly into the client.

Here is the code:
https://bitbucket.org/omwdunkley/crazyf ... e711f6b5dd

Here is a video:
http://www.youtube.com/watch?v=ewN3v4wLO08

and a picture:
Image

I tried to use the existing libraries as much as possible. So far there are no new dependencies. It uses pygame to query the available cameras and read images from them. It captures at the maximum camera fps, but is capped at 60hz. People had performance problems with the attitude indicator (AI) updating too fast and hogging lots of resources. But to have a good AI+camera overlay (HUD) it needs to update at around 30hz to look good. I reimplemented the repainting logic so it aggregates updates. The IMU data is displayed at 30hz in the AI and around 5hz in the GUI. So hopefully this should work for most people without killing their system. The camera resolution is set to the maximum the camera can handle, capped at 720p. As for now we only know how to add analoge cameras to the flie, so the most commonly used resolution would then be around 720x576.

I wanted to add the functionality to make it full screen but I couldnt figure out how to maximise the widget. So for now if you double click on it it uses the whole tab area by hiding the other elements. Maybe someone else can figure this out?

It is also designed so that if the camera fails it still shows the IMU data and vice versa.

This is just supposed to be a very simple proof of concept to allow others to tinker/extend it.
Lots of things could be done to improve it, eg
- move some of the config data to the menu bars
- display a lot more information in the HUD (as in my c++/ROS version) [temp, pressure, acceleration, altitude, motor power, thrust, heading, compass, inputs, etc]
- add opencv dependencies to clean up the image a lot (eg interpolating the even image rows from the odd ones to remove deinterlacing artifacts)
- add modifiable camera settings (resolution, update rate, etc)

Either way let me know what you think and if you need pointers where to or how to add functionality let me know!

PS: I coded this rather quickly so I have not tested it for strange situations (unplugging the camera while it is streaming, etc)

Cheers
Last edited by omwdunkley on Wed Jan 15, 2014 10:32 am, edited 1 time in total.
tobias
Bitcraze
Posts: 2339
Joined: Mon Jan 28, 2013 7:17 pm
Location: Sweden

Re: Added Camera overlay to the pc client

Post by tobias »

Awesome omwdunkley!

This is definitely something we should continue to build upon. And a very nice thing is that it worked right away using a webcam when we tested. Now we just have to get a good digitizer. Do you have any to recommend? How is the latency on the "Hauppauge 610" you seem to be using?
omwdunkley
Expert
Posts: 162
Joined: Thu Jun 06, 2013 9:56 pm
Location: Munich

Re: Added Camera overlay to the pc client

Post by omwdunkley »

tobias wrote:Awesome omwdunkley!

This is definitely something we should continue to build upon.
I agree! It has a lot of potential :) Id like to see other play around with adding custom HUD elements. The qt painting framework is pretty nice and easy to use for that.
tobias wrote:Now we just have to get a good digitizer. Do you have any to recommend?
I tried two digitisers, some "LogiLink" and the Hauppage 610 USB-Live2. I would stay away from the LogiLinks, there seem to be tons of different types that look exactly the same. I had a really hard time getting it to work with Ubuntu. Not worth the effort. Rather spend a little more and get the hauppage - at least you know what you are getting then! And it worked out of the box.

tobias wrote:How is the latency on the "Hauppauge 610" you seem to be using?
There is measurable latency, but it is not too obvious. Ill need to do some tests to make an exact measurement. Not sure how the Hauppage stacks up against other digitisers in terms of lag though!
Post Reply