[Video] Alternative Kinect Control (using depth images)

[omwdunkley]

Hey guys,

Im tight on time but thought I would spend a weekend to see if I could get the flie to reliably fly with the kinect. I have my own custom cf client running and use ROS to do acquire the depth images from the kinect and do some pose transformations.

The kinect tracker by Bitcraze uses color images for segmentation. My approach just uses depth images.
If someone wants to port my tracking part of code to theirs, there relevant function is here: IncomingDepthImage
If you want to run it yourself (you will need ROS), here is the code : GitHub.

Questions, etc welcome.



Cheers,
-Oliver

[tobias]

Great work as always Oliver!

[junoon53]

Amazing! This is what I've been planning to do haha.

[Patrik]

Hi Oliver!

We are three students at Lund Uni making a project based on your code. Our goal is to first get your code going (which has turned out to be a challenge) and then try to make some improvements by using an Asus Xtion camera and using a Kalman filter for control.
Since we are new to both linux and ROS we are having some problem setting up the requirements for your code. Currently we have problems connecting the joy node and the cfjoy node. When we test the joy node we get readings from the controller but when we test the cfjoy node the output angles doesn't change. It does however notice if we activate hover mode. Is there a problem in the transformation between the controller input to roll, pitch, yaw and thrust? Or might it be something wrong with the ROS topics?

We would really appreciate if you could help us with this

Btw we found a mistake in your code that made it impossible to run rosmake. A # was missing in cmd.msg.

Best regards

Patrik, Emil and Jens

[omwdunkley]

Hi guys! Thanks for you interest.

First up, the code isnt really "public ready" or anything, I hacked it together over a weekend. Im sure you will bump into lots of problems along the way! I appreciate you contacting me via the forums, that gives everyone else who wants to use this a heads up too

Good that you mentioned the missing #, that means you are in the wrong branch Ive not documented it anywhere or so, but the default branch is rather outdated. You probably want to use the joyManager branch or so. The gui looks quite different, and a lot of the functionality is still missing. Lots of place holders etc.

Try that branch and see if it works better. I am very short on time these days and have not run that code in a while, but please post your progress and problems here! Some stuff could be trivial to me (eg "oh yeah I forgot to mention you need to check that checkbox else nothing works etc") but might take you a while to figure out.

Are you using a ps3 controller?

I put some notes here. The are probably outdated too but Ill try to update them with the feedback I get from this post!

Cheers and good luck

Hi Oliver!

We are three students at Lund Uni making a project based on your code. Our goal is to first get your code going (which has turned out to be a challenge) and then try to make some improvements by using an Asus Xtion camera and using a Kalman filter for control.
Since we are new to both linux and ROS we are having some problem setting up the requirements for your code. Currently we have problems connecting the joy node and the cfjoy node. When we test the joy node we get readings from the controller but when we test the cfjoy node the output angles doesn't change. It does however notice if we activate hover mode. Is there a problem in the transformation between the controller input to roll, pitch, yaw and thrust? Or might it be something wrong with the ROS topics?

We would really appreciate if you could help us with this

Btw we found a mistake in your code that made it impossible to run rosbuild. A # was missing in cmd.msg.

Best regards

Patrik, Emil and Jens

[Patrik]

Thanks for the quick answer

Yes we are using a ps3 controller.

I've downloaded the right branch now but the problem remains :/ Still don't get a reading when we test cfjoy on the angles. Still just hover mode register.

This is the scheme we get from rqt_graph:



I've made some hints on some of the things we've struggled with so far:

--About installing ROS

• Make sure that your linux distribution is supported. If you are using bitcraze's virtual machine use VM 0.3 because it is based on ubuntu 13.04 which is supported by ROS hydro (hydro is the latest version of ROS).
• Follow the following instructions on how to install ROS: http://wiki.ros.org/hydro/Installation/Ubuntu
• To get started with ROS have a look at this tutorial: http://wiki.ros.org/ROS/Tutorials

--After you installed ROS

• Make sure you download the right git branch
• Remember to set permissions on your downloaded files (I used: chmod -R a+rwx /crazyflieROS . Maybe you shouldn't do this but I'm a linux newbie and it worked.)
• Add the following line in your ~/bashrc (open with gedit): export ROS_PACKAGE_PATH=/your/path/to/workspace:$ROS_PACKAGE_PATH where /your/path/to/workspace is where you placed Olivers code.
• Download ROS joy package (http://wiki.ros.org/joy?distro=hydro) and follow this guide to get your joystick going. http://wiki.ros.org/joy/Tutorials/Confi ... uxJoystick
• Run rosmake on crazyflieROS

Hopefully this will be of some use for someone in the future

Patrik

[omwdunkley]

Hey Patrik!

Your graph looks good. Not sure what the problem is! Ill rerun all the nodes and see if I run into any similar issues early next week when I have access to a ps3 controller.

Ive actually had a few problems with later ubuntu kernels dealing well with specific bluetooth devices and the ps3 controller. But if you can get a "hover mode" command through, it should be okay.
You might also want to test "rostopic hz /cfjoy". Ideally it would be around 100hz. Once again, with later ubuntu kernels/ros versions ive not quite reach this number...might just be my hardware.

The launch file sets up the joy_node to send data at 100hz, which in turn drives the crazyfliejoystickDriver node.

It may sound stupid, but I might have forgotten to mention:
I have a killswitch!! The hover button is actually also the kill switch. It has two modes: light press = disable kill switch, strong press = enter hover mode. So maybe this is why you can enter hover mode as you accidentally also disabled the kill switch.

Hope this helps!

[Patrik]

It was the kill switch... >.<

Now I get a reading on cfjoy and driver.py starts up fine and connects. But I still can't control the flie. After some time though the flie gets all crazy (lol) and just shoots off. In the console i get the following errors:

File "/home/bitcraze/projects/crazyflieROS/src/crazyflieROS/rosTools.py", line 102, in receiveCrazyflieLog
self.pub(getGroup(log), self.genMsg(log, tsROS), f=thrustToPercentage)
TypeError: pub() got an unexpected keyword argument 'f'
Traceback (most recent call last):
File "/home/bitcraze/projects/crazyflieROS/src/crazyflieROS/rosTools.py", line 102, in receiveCrazyflieLog
self.pub(getGroup(log), self.genMsg(log, tsROS), f=thrustToPercentage)
TypeError: pub() got an unexpected keyword argument 'f'
Traceback (most recent call last):
File "/home/bitcraze/projects/crazyflieROS/src/crazyflieROS/rosTools.py", line 102, in receiveCrazyflieLog
self.pub(getGroup(log), self.genMsg(log, tsROS), f=thrustToPercentage)
TypeError: pub() got an unexpected keyword argument 'f'
[WARN] [WallTime: 1397512604.247174] Maximum Packets Changed from 1655/s -> 1683/s
[ERROR] [WallTime: 1397512604.257436] Logging error with motor: Block too large

And after that the GUI freezes and I have to terminate the node. I checked the frequency on cfjoy and its about 96 hz. Any ideas?

Patrik

[omwdunkley]

Hi, not sure, not seen that error before. I can have a look on thursday!

Something you might consider (and I probably forgot to mention):
Once you start the flie and connect, go to the options tab or whatever it is called and on the bottom you will see something that resembled "Generate ROS Msgs from Table of Contents". Press it! This basically generates ros messages based on the available logs and their contents the flie exposes. Maybe this helps...just a wild guess!

Actually, screw that. Just checked out the code. Had a bracket typo!
See this revision.

Thanks for the feedback!

[omwdunkley]

Just noticed the real error:

Code: Select all

[ERROR] [WallTime: 1397512604.257436] Logging error with motor: Block too large

That message comes directly from the flie driver. Not really sure what to do about that! Obviously I should handle it more graciously and the GUI shouldnt crash.

Are you running the latest firmware? I tested with 9a67fa8542dc. Early ones probably will not work as many changes happened around that time.

-Oll