[Video] Alternative Kinect Control (using depth images)

[Veerachart]

I think I have done with the fake joy part, but I lost one of my flie when trying to change the motor holder. A new one also still not working as I have posted here. If you have any idea, please

I hope to get it back to work ASAP.

[crazyflie007]

Hi,
I was wondering if it's possible to draw a 3D trajectory and make the Flie follow it. The main probem here would be to draw the trajectory and correctly get the depth points. Rest would be just setting the set point to the next point on the trajectory.

[omwdunkley]

I think I have done with the fake joy part, but I lost one of my flie when trying to change the motor holder. A new one also still not working as I have posted here. If you have any idea, please

I hope to get it back to work ASAP.

Sorry man, not sure!
What is your goal with the fake joy part anyway?

Hi,
I was wondering if it's possible to draw a 3D trajectory and make the Flie follow it. The main probem here would be to draw the trajectory and correctly get the depth points. Rest would be just setting the set point to the next point on the trajectory.

Should be possible Guess there are three things that need to be taken care of:

• following logic: how do you want to follow the path. Should be easy enough..if you get close enough to one point, set the next as goal. There are better ways to do this, but this might be a good first step.
• visualization: using rviz would be pretty easy. Publish your path as a line
• user interaction: creating the path with some gui or just preloading a path. Not sure how I would draw a 3d path in 2d without resorting to a leap motion or something

Should be fun

[Veerachart]

What is your goal with the fake joy part anyway?

Well, it's just because that I don't have a joy, and my final goal doesn't need manual control. The way of publishing fake joy signals seemed easier than changing the code to work without joy signals (as I'll have to go through lots of codes.)

I have tested (without a ready-to-fly quad) and the PID controllers appear to be reset when I press a key to toggle L1 key to "pressed" state. Anyway I can't yet see the result.

[Veerachart]

Hi guys,

Just got a chance to test it with a new crazyflie. I already got quite a good result with my fake joy publisher

Anyway, what I found after some tries are:

• The tracking sometimes has some delays (sometimes up to ~1 sec), and that usually causes oscillations and sometimes leads to instability of the crazyflie. I noticed that in the subscriber of the depth image, buffer size is not set, and the defaul is 64 kB. From my experience, having some buffer helps avoiding frame loss, but also results in delays.
  
  With the kinect not so high from the ground (around 1 meter), when the crazyflie got higher than the kinect, its detected size will be larger, and therefore if the detection size is set just for the detection on the ground. the flie can be lost from detection and causing loss of control.

I also wonder if we can change from freenect to openni or not. I will try looking for this possibility.

Anyway, thanks for this

[omwdunkley]

Thanks for your input!

Yup, some where I introduced some delay into tracking. Oops. Quick tests show that all the drawing is consuming most of the time. I guess I should really write a c++ application and minimize opencv drawing calls, not sure why they are taking so long in python :/ I suggest you just comment them all out for now.

Its only really a proof of concept for now and requires some tuning Esp the PID values.

Good point with the crazyflie size changing when viewing from below/above. Ive not had the issues yet but it makes sense. I guess some kind of adaptive size filtering would make sense; knowing the the pitch of the kinect and guessing where the crazyflie is being seen in the image (and assuming the crazyflie is level) would probably be enough to compensate for this.

I think openni should work fine - all you need a depth images

Keep us posted!

[Veerachart]

Some more investigations showed me that trying to do screencast uses quite considerable resources and cause some delays. Without doing that (and a little bit modification on numpy matrix manipulations - not sure how much this helps), I experience the delay less; most of the time no problem, but sometimes it just comes out of nowhere.

About the detection size: for my case I start using the Kinect to control the flie since taking off. To make it detectable, I have to put a box on the floor as the take-off stand, so that the difference from the background is large enough. Also in my case, the detected size is smaller when the flie is on the box, as the part closer to the box has too small difference to be detected. Setting wider range of detection size already solved this problem (or at least seems so.) (I also found a little problem with part of my lab's wall, that it somehow reflects the IR light weirdly, causing too much noise which are around the same size as the flie, so they got detected. Anyway, I solved this by using partitions to block those materials and everything comes to a peaceful state )

Changing from freenect to OpenNI also works well, just need to change rosdep in the manifest file.

[tobias]

Any pictures of your setup? Would be really nice to see!

[DerP0LE]

Hello everyone,

im new here and sry for my, I think bad english.
My problem is that i cant run this programm I just go from one error to the next. Maybe Im to stupid for Linux because I only use Windows.
Could anyone help me with a Teamviever session? or Teamspeak, Skype whatever. And maybe here is a Person who can speak german
Best regards from Germany

[omwdunkley]

Just responded to your PM. Tldr; I just started a new job in a new country so I'm very very short on time these days! Sorry