Mediapipe Object Detection Solution to OSC

 

Download here: https://github.com/little-scale/mediapipe-object-osc

Adds basic OSC output for the Mediapipe object detection solution

By default, OSC will be sent to 127.0.0.1 localhost on port 3000 and with address /mediapipe/objects. This can be changed in utils.py

Each OSC message contains: Category name, object detection instance within the frame, normalised x position, normalised y position and certainty

An example Ableton Max for Live device is provided which allows mapping an object class with x and y position to Ableton parameters

ArUco2osc

Send ArUco marker detections as OSC messages

Download here: https://github.com/little-scale/ArUco2osc 

Repo contains 50 4x4 markers

To use:

• pip install opencv-contrib-python
• pip install python-osc
• python aruco2osc.py

Args are:

• --input: camera to use; default 0
• --address: OSC message address to use; default /aruco/marker
• --ip: OSC IP address to use; default 127.0.0.1
• --port: OSC port to use; default 3001

Each message contains:

• marker identity
• size as a normalised ratio
• angle in degrees
• x value of midpoint of marker as a normalised value
• y value of midpoint of marker as a normalised value

To change the marker type used for detection, change DICT_4X4_50 to a different value

Example Max Patch for receiving ArUCo detections: 

Running labelimg Under Ventura

Download labelimg to desktop

/usr/bin/python3 -m venv env

source env/bin/activate 

pip install --upgrade pip

pip install PyQt5

cd desktop/labelImg-master

pip3 install pyqt5 lxml

make qt5py3

python3 labelImg.py 

Mediapipe Facemesh Landmarks Map

 

Mediapipe Hand Solution on Raspberry Pi to OSC to Modular CV / Gate

View this post on Instagram

A post shared by Sebastian Tomczak (@littlescale)

Setup: 

1. Set up a Raspberry Pi 4 with a clean install of Raspberry Pi OS: https://www.raspberrypi.com/software/ 
2. Install Mediapipe as per the instructions here: https://github.com/googlesamples/mediapipe/tree/main/examples/pose_landmarker/raspberry_pi
3. Install Python OSC on the Raspberry Pi: https://pypi.org/project/python-osc/ 
4. Add detect_osc_pi.py (https://github.com/little-scale/Tele-o-Module/blob/main/software/dectect_pi_osc.py) to mediapipe/examples/hand_landmarker/rasperry_pi
5. If using my Tele-o module (https://little-scale.blogspot.com/2023/01/tele-o-wifi-bluetooth-to-cv-gate.html) replace ip in detect_osc_pi.py with the address of the ESP32 in the module. 
6. If using a different OSC endpoint, replace ip in detect_osc_pi.py with the appropriate address. 
7. It may be beneficial to change the port variable in detect_osc_pi.py - the default is 3000
8. Pressing together or releasing the thumb and index finger sends a 1 or 0 on OSC path /gate/0
9. The distance between the thumb and index finger sends a float 0 - 1 on OSC path /cv/0
10. The X and Y co-ordinates of the index finger are sent as floats of 0 -1 on OSC paths /cv/1 and /cv/2
11. These OSC paths can be changed with the variables near the top of detect_osc_pi.py

YOLOv7 to OSC

 

YOLOv7 pose to OSC output including support for multiple pose detection: https://github.com/little-scale/yolov72osc/ 

Based on the Yolov7 implementation: https://github.com/WongKinYiu/yolov7

Add /p.py, replace /utils/plots.py and download the yolov7-w6-pose.pt model weights from https://github.com/WongKinYiu/yolov7/releases

Default OSC address is /yolo7/pose, IP is localhost and port is 4500. These can be changed in plots.py