About this project
This max patch, developed as a final project for my first semester of Max MSP class in grad school, takes a video feed of my fish tank and uses color tracking to follow his movements. His position in the tank selects a color from a color chart, and the color of my hue light bulbs change to match the color the fish has selected.
The camera input goes through color tracking (and in the shareable patch, the user may click on the video to select a new color), which is then converted to hue and lightness parameters. Those parameters are sent to a color wheel (swatch object), slowed so as to not overload the hue API, and then converted from RGB to CIE values via javascript. The values are then sent to the hue-ems.js which formats and sends the bulb parameters to the Hue API via the internet.
Once the camera is turned off, there is a slight pause, and the bulbs will all turn white.
The Set Up
My fish tank is in my hallway, so I set my laptop up on the chair I keep there. For filming, this was a good positioning as my laptop faced the doors to my living room, where the Hue bulbs are, so I could use its webcam to show how the lights are changing.
The webcam has a wide angle lens so I could get as close to the tank as possible while still getting its full width. I had struggled a bit with camera placement before realizing I really needed a tripod to get the positioning I wanted.
The towel draped over the light is also taped to the sides of the tank, acting as a barn door for light spill in the space between the lid and the top of the water. Without it, the image became very washed out and Max had trouble reading colors accurately.
Music Credits:
--------------------------------------------
All files can be found in a ZIP folder in the above link. There are 4:
- bulbs-to-share201213.maxpat -- the actual patch
- hue-ems.js -- the javascript that sends information to the Philips Hue API
- maxhue_config.json -- the JSON file where the user needs to input their API information for the patch to work (NOTE: because this is going on my blog, my API & IP information has been removed)
- rgb-to-cie.js -- the javascript that converts max's RGB colors into CIE values for the Hue bulbs
References
- The original version of the hue-ems.js file is from natcl on github. I expanded on this patch & code: https://github.com/natcl/maxhue
- The video processing/color tracking is based on a patch by Ryan Holsopple that was shared in Class 10 of this course.
- This article from GoToGuy Blog was a useful reference and set me on the right direction in accessing the Hue API: https://gotoguy.blog/2020/05/21/remote-authentication-and-controlling-philips-hue-api-using-postman/
- These articles from Philips itself were instrumental (note: for some of these, you need a free API developer account):
- Ryan Holsopple helped greatly with the rgb-to-cie javascript, and for that we worked off of this reddit post by u/Croques: https://www.reddit.com/r/tasker/comments/4mzd01/using_rgb_colours_with_philips_hue_bulbs/
Future Plans
I'd love to use this idea of color tracking to change a larger environment in other realms, such as applying it to an immersive performance.
Additionally, larger community fish tanks often have a 'centerpiece' fish that is quite different visually from the other fish and could be easy to track. With a larger, more active tank, the lighting results might be quite different!
It could also easily be adapted for use with other pets (IE, a pet bird) to introduce ways of non-verbal communication.
