> For the complete documentation index, see [llms.txt](https://campus-rover.gitbook.io/lab-notebook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://campus-rover.gitbook.io/lab-notebook/fiiva/reports/2020/gesture-recognition/leap_motion.md). # leap\_motion.md ## **Intro** As a very last minute and spontaneous approach, we decided to use a Leap Motion device. Leap Motion uses an Orion SDK, two IR camerad and three infared LEDs. This is able to generate a roughly hemispherical area where the motions are tracked. ![](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-8beedbbbfdbfb5f51180f449f665b70d1e16f3e8%2Fleap.png?alt=media) ![](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-f00d7c492ffc4bd21163c8801b73b513a3a54c7a%2Fsetup.png?alt=media) It has a smaller observation area dn higher resolution of the device that differentiates the product from using a Kinect (which is more of whole body tracking in a large space). This localized apparatus makes it easier to just look for a hand and track those movements. The set up is relatively simple and just involved downloading for the appropriate OS. In this case, Linux (x86 for a 32 bit Ubuntu system). ## **Steps to downloading Leap Motion and getting it started:** 1. download the SDK from ; you can extract this package and you will find two DEB files that can be installed on Ubuntu. 2. Open Terminal on the extracted location and install the DEB file using the following command (for 64-bit PCs): **$ sudo dpkg -install Leap-\*-x64.deb** **If you are installing it on a 32-bit PC, you can use the following command:** **sudo dpkg -install Leap-\*-x86.deb** 3. plug in leap motion and type dmesg in terminal to see if it is detected 4. clone ros drivers: **$ git clone ** 5. edit .bashrc: **export LEAP\_SDK=$LEAP\_SDK:$HOME/LeapSDK** **export PYTHONPATH=$PYTHONPATH:$HOME/LeapSDK/lib:$HOME/LeapSDK/lib/x64** 6. save bashrc and restart terminal then run: **sudo cp $LeapSDK/lib/x86/libLeap.so /usr/local/lib** **sudo ldconfig** **catkin\_make install --pkg leap\_motion** 7. to test run: **sudo leapd** **roslaunch leap\_motion sensor\_sender.launch** **rostopic list** Once having Leap Motion installed, we were able to simulate it on RViz. We decided to program our own motion controls based on angular and linear parameters (looking at directional and normal vectors that leap motion senses): ![](https://github.com/campusrover/labnotebook/blob/master/reports/2020/gesture-recognition/labnotebook/images/code.png) This is what the Leap Motion sees (the raw info): ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-d7c364cacbfe2e88d06a835740e1d3017bacdb2d%2Fleapmotionss.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-10e99524eb66c538ae9ad7f7fd720f195867384f%2Fleapmotionss2.png?alt=media) In the second image above, the x y and z parameters indicate where the leap motion detects a hand (pictured in the first photo) **This is how the hand gestures looked relative to the robot's motion:** ### **Stationary** ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-3d3c5610d3597c3134bfd6f4a3b3cf548b5aafe0%2Fstat.png?alt=media) ### **Forward** ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-d788472da6e8ac555575ac8e5522ce0604f02794%2Fforward.jpg?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-9130e3149e0066ffd1944796036c66bae6b99ff6%2Fforward2.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-f0cb6b4531b2ecb79dcdd6bb7206e19fc974f846%2Fforward3.png?alt=media) ### **Backward** ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-5b59ba0d97c90e2cdb9a19d046a56c478d3e24ea%2Fbackwards.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-1ec9215fd0fec1ff251e606c090df4dbe0aee58e%2Fbackwards2.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-63f6c337dc4b04a74f6d388657139348fcd42632%2Fbackwards3.png?alt=media) ### **Left Rotation** ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-e504cea9ff21bed71977ebf526063c4f48cc15ab%2Fleft.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-2ae2f85e0b9573c5db00f7f2360531b3f1e2a249%2Fleft2.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-092714fd87470c923f61d795f0855112f059a135%2Fleft3.png?alt=media) ### **Right Rotation** ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-73a773bdcf3598bf2d83c508ef5f19092b93fd0e%2Fright.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-487d08dfaac9bc724374a75d1a463d846b2699c5%2Fright2.png?alt=media) ![Your image title](https://4196336753-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LULg3WmBk4r0dWAdK9A%2Fuploads%2Fgit-blob-f69968c490fc32898120fa20a3360db669eb215e%2Fright3.png?alt=media) ## **Conclusion** So, we got the Leap Motion to successfully work and are able to have the robot follow our two designated motion. We could have done many more if we had discovered this solution earlier. One important thing to note is that at this moment we are not able to mount the Leap Motion onto the physical robot as LeapMotion is not supported by the Raspberry Pi (amd64). If we are able to obtain an Atomic Pi, this project should be able to be furthered explored. Leap Motion is a very powerful and accurate piece of technology that was much easier to work with than the Kinect, but I advise still exploring both options. ## **GIT HUB REPO** --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://campus-rover.gitbook.io/lab-notebook/fiiva/reports/2020/gesture-recognition/leap_motion.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.