This whole project is meant to be run in a Raspberry Pi, with the heaviest Neural Network, the handLandmarks NN, accelerated by a Coral USB Accelerator.
In this first part I will only discuss the basic components in the Pi, for now HandCommander and the underlying MediaPipe framework will be run in the development PC, while the generated commands will be sent to the RPi to be converted into “physical” actions.
Those components are:
- Raspbian Buster, Nothing special about this, mentioned just for reference
- LIRC,Now since Kernel 4.19 does not include lirc_dev, it was a real PITA to make it work, start with this post and wish you the best of luck 😀 .Seriously, it was very difficult, at the extreme to having to use an Arduino and the IRremote Library by z3t0 with a second pair of IR receiver/transmitter just to debug LIRC signals and ended using the Arduino to capture the codes of my Sony tv and making the config file by hand.
- Mosquitto, Installed this one and many more in docker with the excellent tutorial made by Andreas Spiess (subscribing to this guy’s channel is a must, excellent material) in this video https://www.youtube.com/watch?v=a6mjt8tWUws
- Node-RED, Even though Node-RED is installed with the previous tutorial, since I needed direct access to LIRC, I disabled the Node-RED container and installed it on the host. Now, I know that container to host communication is perfectly doable, but again this is a big project and some corners need to be cut in order to release a fully working beta in a reasonable time frame
- An InfraRed Transceiver, Very simple:
The data flow is easy to understand:
HandCommander(PC)->Mosquitto(RPi)->Node-RED->LIRC->InfraRed command to TV
Node-RED is the intermediary, “gluing” all together, it subscribes to the mqtt broker and actuates on the LIRC module to send ir commands, this is the Flow for that task
A LIRC node does exist but in my case it wasn’t reliable, so, I switch to an Exec Node that will just execute a predefined terminal command inserting the received payload at the end. Works just fine.
Also added a very simple mqtt injector to test the above
I moved to Denmark a few weeks ago (amazing country!!!). In my apartment the cable TV is, to be honest, very expensive and the content is boring at best. Since I still needed to test the channel changing functions, a way around this limitation was to create a VLC playlist with public streaming channels
#EXTM3U #EXTINF:0,SW:NORWAY http://bgo1.cdn.s3m.no/fs/live/live/chunklist.m3u8 #EXTINF:0,SW:kanal10 http://cdn-kanal10.crossnet.net:1935/kanal10/mp4:mpegts.stream/chunklist_w1523238254.m3u8 #EXTINF:0,SW:kkr http://stream.kkr.dk/live/kkr/chunklist_w832247479.m3u8 #EXTINF:0,SW: OPPNA KANALEN http://18.104.22.168/edge/_451iw2h_/451iw2h/chunklist_w348058882.m3u8
VLC does not yet support mqtt subscribing, but they do have a telnet server
To send telnet commands to VLC, another Flow was created in Node-RED, with a mqtt in node subscribed to the handCommander/VLC topic, again, using a Exec node to execute a telnet command appending the mqtt payload at the end
* xxxx is the VLC telnet password
And again, a test injector