beyhan/rust_bifrost
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Beyhan Oğur
2026-04-26 22:29:38 +03:00
commit 427856cd3a
176 changed files with 27613 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
doc/bifrost.service.ex Normal file
View File

@@ -0,0 +1,36 @@
[Unit]
Description=Bifrost Bridge
After=network.target
[Service]
Type=simple
# Make it possible for unprivileged processes to bind to low ports (< 1024)
# This is needed to run port 80 + 443 without being root.
AmbientCapabilities=CAP_NET_BIND_SERVICE
# If bifrost should fail for some reason, wait 20s and restart it,
# no matter the cause
Restart=always
RestartSec=20s
# To use these settings, create a bifrost user + group:
#
# adduser --group bifrost --system bifrost
#
User=bifrost
Group=bifrost
# This assumes you want to run the bifrost server in:
#
# /data/bifrost/
#
# with the executable at:
#
# /data/bifrost/bifrost
#
WorkingDirectory=/data/bifrost
ExecStart=/data/bifrost/bifrost
[Install]
WantedBy=multi-user.target

69
doc/comparison-with-diyhue.md Normal file
View File

@@ -0,0 +1,69 @@
## Comparison with diyHue
You might already be familiar with [diyHue](https://github.com/diyhue/diyHue),
an existing project that aims to emulate a Philips Hue Bridge.
diyHue is a well-established project, that integrates with countless
servers/services/light systems, and emulates many Hue Bridge features.
However, I have been frustrated with diyHue's MQTT integration, and its fairly
poor performance when operating more than a handful of lights at a time. Since
diyHue always sends individual messages to each light in a group, large rooms
can get quite slow (multiple seconds for every adjustment, no matter how minor).
Currently, diyHue does not support Zigbee groups (or MQTT groups) at all,
whereas Bifrost is written specifically to present Zigbee2MQTT groups as Hue
Bridge "rooms". For zigbee/mqtt use cases, this massively increases performance
and reliability.
Another thing about diyHue that frustrates me to no end, is the lack of
(working) support for push notifications. If you use the Hue App to control a
diyHue bridge, you will notice that it does not react to any changes from other
phones, home automation, etc. Also, the reported light states (on/off, color,
temperature, etc) are sometimes just wrong.
Overall, diyHue can do an impressive number of things, but it seems to have some
pretty rough edges.
Just to clarify, I've enjoyed using diyHue, and I wish them all the best. It's
also very useful, both as a home automation service, and a reverse engineering
resource.
However, if you're also using one or more Zigbee2MQTT servers to control Zigbee
devices, feel free to give Bifrost a try. It might be a better fit for your use
case.
In any case, feedback always welcome.
| Feature | diyHue | Bifrost |
|--------------------------------------|-----------------------------------------|-------------------------------------------|
| Language | Python | Rust |
| Project scope | Broad (supports countless integrations) | Narrow (specifically targets Zigbee2MQTT) |
| Use Hue Bridge as backend | ✅ | ❌ |
| Usable from Homeassistant | ✅ (as a Hue Bridge) | ✅ (as a Hue Bridge) |
| Control individual lights | ✅ | ✅ |
| Good performance for groups of light | ❌ (sends a message per light) | ✅ (uses zigbee groups) |
| Connect to Zigbee2MQTT | (✅) (but only one server) | ✅ (multiple servers supported) |
| Auto-detection of color features | ❌ (needs manual configuration) | ✅ |
| Create Zigbee2MQTT scenes | ❌ | ✅ |
| Recall Zigbee2MQTT scenes | ❌ | ✅ |
| Learn Zigbee2MQTT scenes | ❌ | ✅ |
| Delete Zigbee2MQTT scenes | ❌ | ✅ |
| Join new zigbee lights | ✅ | ❌ |
| Add/remove lights to rooms | ❌ | ✅ |
| Live state of lights in Hue app | ❌ [^1] | ✅ |
| Multiple type of backends | ✅ | ❌ (only Zigbee2MQTT) |
| Entertainment zones | ✅ | ✅ |
| Zigbee Entertainment mode support | ❌ | ✅ |
| Hue effects (fireplace, candle, etc) | (✅) (partial) | ✅ |
| Routines / Wake up / Go to sleep | ✅ | ❌ (planned) |
| Remote services | (✅) (only with Hue essentials) | ❌ |
| Add custom lights and switches | ✅ | ❌ |
[^1]: Light state synchronization (i.e. consistency between hue emulator, hue
app and reality) seems to be, unfortunately, somewhat brittle in diyHue. See
for example:
* https://github.com/diyhue/diyHue/issues/883
* https://github.com/diyhue/diyHue/issues/835
* https://github.com/diyhue/diyHue/issues/795

178
doc/config-reference.md Normal file
View File

@@ -0,0 +1,178 @@
## Configuration reference
Bifrost
```yaml
# Bifrost section [optional!]
#
# Contains bifrost server settings
# [usually omitted, to use defaults]
bifrost:
# name of yaml file to write state database to
state_file: "state.yaml"
# name of x509 certificate for https
#
# if this file is missing, bifrost will generate one for you
#
# if this file exists, bifrost will check that the mac address
# matches the specified server mac address
#
# to generate a fresh certificate, rename/move this file
# (this might require pairing the Hue App again)
cert_file: "cert.pem"
# Bridge section
#
# Settings for hue bridge emulation
bridge:
name: Bifrost
mac: 00:11:22:33:44:55
ipaddress: 10.0.0.12
netmask: 255.255.255.0
gateway: 10.0.0.1
timezone: Europe/Copenhagen
# HTTP port for emulated bridge
#
# beware: most client programs do NOT support non-standard ports.
# This is for advanced users (e.g. bifrost behind a reverse proxy)
http_port: 80
# HTTPS port for emulated bridge
#
# beware: most client programs do NOT support non-standard ports.
# This is for advanced users (e.g. bifrost behind a reverse proxy)
https_port: 443
# DTLS port for emulated bridge (Hue Entertainment streaming)
#
# beware: client programs do NOT support non-standard ports.
# For advanced users (e.g. bifrost behind a port forwarded firewall)
entm_port: 2100
# Zigbee2mqtt section
#
# Make a sub-section for each zigbee2mqtt server you want to connect
#
# The server names ("some-server", "other-with-tls") are used for logging,
# but have no functional impact.
#
# NOTE: Be sure to use DIFFERENT names for different servers.
# Otherwise the yaml parser will consider it the same server!
z2m:
some-server:
# The websocket url for z2m, starting with "ws://".
#
# For z2m version 2.x, the url must end in `/api?token=<token>`.
# For z2m version 1.x, this is optional, but supported.
#
# Therefore, Bifrost will adjust the urls if needed.
# A message will be logged with the rewritten url if this happens.
#
# NOTE: The z2m default token is literally the string "your-secret-token",
# so if unsure, append "/api?token=your-secret-token".
#
# Example:
#
# If your z2m frontend is listening on 10.00.0.100:8080, this
# is the resuling config:
#
url: ws://10.00.0.100:8080/api?token=your-secret-token
other-with-tls:
# This will work, but Bifrost will generate a warning that the url has been
# adapted to include "/api?token=your-secret-token".
#
# NOTE: Using "wss://" instead of "ws://" enables TLS for this connection.
url: wss://10.10.0.102:8080
# Disable TLS verify [optional!]
#
# If this parameter is included, and has a value of "true", TLS certificate
# verification will be disabled!
#
# NOTE: From a security standpoint, this is almost as bad as disabling
# encryption entirely. If having a secure connection is important to you,
# DO NOT enable this option.
#
# If you're using self-signed certificates, enabling this option will allow
# Bifrost to connect to your z2m server.
disable_tls_verify: false
# Group prefix [optional!]
#
# If you specify this parameter, *only* groups with this prefix
# will be visible from this z2m server. The prefix will be removed.
#
# Example:
#
# With a group_prefix of "bifrost_", the group "bifrost_kitchen"
# will be available as "kitchen", but the group "living_room" will
# be hidden instead.
#
group_prefix: bifrost_
# Streaming mode ("Entertainment mode" / "Hue Sync") maximum frames per second
# [optional!]
#
# This is the maximum number of light updates attempted per second.
#
# The incoming data stream (from a Sync Box, Hue Sync for Windows/Mac,
# or some other client) determines the maximum possible fps.
#
# For example, if Bifrost only receives light updates at 10 fps, setting
# this limit to 20 will still only cause the lights to update at 10 fps.
#
# On the other hand, if the streaming client sends faster than this limit,
# frames will be dropped to avoid going over it.
#
# If not specified, uses a default of 20, which is an attempt to balance
# responsiveness against load on the Zigbee mesh.
#
# Because of the smoothing algorithm Bifrost uses, the results will look
# *better* if this is not set higher than needed.
#
# For example, 30 fps content will look good at 10, 20 or 30 streaming_fps,
# but worse at streaming_fps: 60, because the frame-to-frame transition
# time will be wrong for the content.
#
# Rules of thumb(s), for best results:
# - Higher numbers mean greater load on your Zigbee mesh.
# - If your mesh starts lagging or becoming unresponsive, try a lower number.
# - Even values as low as 5 fps looks pretty good.
# - There usually no reason to go above 60.
# - Have fun experimenting :-)
streaming_fps: 20
...
# Rooms section [optional!]
#
# This section allows you to map zigbee2mqtt "friendly names" to
# a human-readable description you provide.
#
# Each entry under "rooms" must match a zigbee2mqtt "friendly name",
# and can contain the following keys: (both are optional)
#
# name: The human-readable name presented in the API (for the Hue App, etc)
#
# icon: The icon to use for this room. Must be selected from the following
# list of icons supported by the Hue App:
#
# attic balcony barbecue bathroom bedroom carport closet computer dining
# downstairs driveway front_door garage garden guest_room gym hallway
# home kids_bedroom kitchen laundry_room living_room lounge man_cave
# music nursery office other pool porch reading recreation staircase
# storage studio terrace toilet top_floor tv upstairs
#
rooms:
office_group:
name: Office 1
icon: office
carport_group:
name: Carport Lights
icon: carport
...
```

35
doc/docker-compose-install.md Normal file
View File

@@ -0,0 +1,35 @@
## Building From Source
When you have these things available, you can install Bifrost by running these commands:
```sh
git clone https://github.com/chrivers/bifrost
cd bifrost
```
Then rename or copy our `config.example.yaml`:
```sh
cp config.example.yaml config.yaml
```
And edit it with your favorite editor to your liking (see
[configuration reference](config-reference.md)).
If you want to put your configuration file or the certificates Bifrost creates somewhere
else, you also need to adjust the mount paths in the `docker-compose.yaml`. Otherwise,
just leave the default values.
Now you are ready to run the app with:
```sh
docker compose up -d
```
This will build and then start the app on your Docker instance.
To view the logs, run the following command:
```sh
docker logs bifrost
```

29
doc/docker-image-install.md Normal file
View File

@@ -0,0 +1,29 @@
## Using Docker Pull
Pull the latest image from Github Container Registry:
```sh
docker pull ghcr.io/chrivers/bifrost:latest
```
Curl and rename the example configuration file:
```sh
curl -O https://raw.githubusercontent.com/chrivers/bifrost/master/config.example.yaml
cp config.example.yaml config.yaml
```
And edit it with your favorite editor to your liking (see
[configuration reference](config-reference.md)).
Now run the Docker Container:
```sh
docker run -v $(pwd)/config.yaml:/app/config.yaml ghcr.io/chrivers/bifrost:latest
```
To view the logs, run the following command:
```sh
docker logs bifrost
```

17
doc/how-to-find-mac-linux.md Normal file
View File

@@ -0,0 +1,17 @@
## How to find your mac address (Linux)
On Linux, you can use the `ip -c addr` command to find the mac address:
```
$ ip -c addr
...
3: enp36s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:11:22:33:44:55 brd ff:ff:ff:ff:ff:ff
inet 10.12.0.20/24 brd 10.12.0.255 scope global enp36s0
valid_lft forever preferred_lft forever
...
```
Here we see an interface called `enp36s0` that has the mac address `00:11:22:33:44:55`.
You will see multiple interfaces - use the one with your IP address listed.

336
doc/hue-zigbee-clusters.md Normal file
View File

@@ -0,0 +1,336 @@
# The Color of Magic: Reversing the Hue Zigbee Clusters
This document, which builds on the [initial work](hue-zigbee-format.md), aims to
compile all available information about the custom Zigbee messages used by
Philips Hue devices, and in particular, lights.
The following text refers to commands and attributes on Hue devices. This has
been researched using the following units:
## "Hue Bulb"
- "Hue white and color ambiance E27 1100lm"
- Model `LCA006`
- Firmware 1.122.2 (20240902)
## "Hue Gradient strip"
- "Hue Play gradient lightstrip for PC"
- Model `LCX005`
- Firmware 1.122.2 (20240902)
## Nomenclature
The following short names are used to refer to zigbee data types and concepts:
| Name used here | Zigbee meaning |
|----------------|---------------------------------------------|
| N/S | Attribute not supported |
| u8 | Unsigned, 8-bit integer |
| u16 | Unsigned, 16-bit integer |
| i16 | Signed, 16-bit integer |
| b8 | 8-bit bitmap value |
| b32 | 32-bit bitmap value |
| e8 | 8-bit enum value |
| hex | "Octet string" (byte array) in hex notation |
# Cluster 0xFC00: Hue Button events
Used by hue buttons to report button events and other state changes.
## Cluster-specific commands
### Command 0: Button Event
These are mostly documented elsewhere, and because they are button events, they
are not the main focus of this document.
# Cluster 0xFC01: Entertainment
This cluster is used to control "Entertainment Zones", a defining feature of the
Hue ecosystem.
## Cluster-specific commands
### Command 1: Update entertainment zone
This is the major command used to send a "frame" of Hue Entertainment data.
Sending it to a Hue bulb will cause that bulb to repeat it in broadcast mode,
for other devices to pick up.
```text
┌───────────┬───┬───┬───┬───┬───┬───┬───┬───┐
│ Byte Bit ► 7 │ 6 │ 5 │ 4 │ 3 │ 2 │ 1 │ 0 │
├─▼─────────┼───┴───┴───┴───┴───┴───┴───┴───┤
│ 0 │ .counter │
│ │ │
│ 1 │ │
│ │ │
│ 2 │ │
│ │ │
│ 3 │ │
├───────────┼───────────────────────────────┤
│ 4 │ .smoothing │\
│ │ Defaults to 0x0400 │ } Smoothing factor
│ 5 │ (encoded as "0004") │/
├───────────┼───────────────────────────────┤
│ 6 │ Light data block 0 │\
│ │ │ \
│ .. │ │ } Repeated for each light
│ │ │ /
│ 12 │ │/
├───────────┼───────────────────────────────┤
: 13 : Light data block 1 :
: : :
```
The "smoothing factor" is a value that controls how agressively the
color/brightness will change from the previous frame. A value of `0x0000` is the
fastest possible (and generally not very pleasant to look at), while a value of
`0x1000` is quite slow, giving very smooth animations, but without any quick changes.
Very high values (e.g. above `0x4000`) are so slow that they are unlikely to be
useful in most cases.
The existing Hue Entertainment clients all seem to use `0x0400`, which is a
reasonable starting point. Note that this property does NOT seem to be exposed
over any known API, but it is available over Bifrost.
Each "light data block" is a 7-byte packed structure describing the desired
state for a light (a bulb, or single segment of a multi-segment light source).
```text
┌───────────┬───┬───┬───┬───┬───┬───┬───┬───┐
│ Byte Bit ► 7 │ 6 │ 5 │ 4 │ 3 │ 2 │ 1 │ 0 │
├─▼─────────┼───┴───┴───┴───┴───┴───┴───┴───┤
│ 0 │ .addr │
│ │ Zigbee address (or alias) │
│ 1 │ for the target light │
├───────────┼───────────┬───────────────────┤
│ 2 │(low 3 bit)│ .mode (5 bit enum)│
│ │─ ─ ─ ─ ─ ─└───────────────────┤
│ 3 │ .brightnes (high 8 bits) │
├───────────┼───────────────────────────────┤
│ 4 │ .color_x (low 8 bits) │\
│ ├───────────────┐─ ─ ─ ─ ─ ─ ─ ─│ \
│ 5 │ (low 4 bits) │ (high 4 bits) │ same format as for composite updates
│ │─ ─ ─ ─ ─ ─ ─ ─└───────────────┤ /
│ 6 │ .color_y (high 8 bits) │/
└───────────┴───────────────────────────────┘
```
The `.mode` field is an odd one. Only two values have ever been observed:
```rust
// the names might change, as we learn more about these bits
enum LightRecordMode {
Segment = 0b00000,
Device = 0b01011,
}
```
Normal bulbs must be contacted with the `LightRecordMode::Device` option, while
updates for segments on a gradient strip must use the `LightRecordMode::Segment`
mode. Otherwise, the entire segment only lights up in the first color.
Current hypothesis: This values determines if real network addresses or virtual
segment addresses are used, but this is currently not tested.
### Command 3: Synchronize entertainment zone
This command is used to synchronize the sequence number in an entertainment
group. The first two bytes are unknown.
```c
struct {
x0: u8, // only seen as 0
x1: u8, // seen as 0 or 1. unknown function
counter: u32, // frame counter for entertainment group
}
```
### Command 4: Retrieve segment mapping
This command is used to retrieve the segment mapping for a hue multi-segment
light.
#### Request
A single byte is sent. Only observed as `00` (might be an index for highly
addressable devices?).
#### Response
```c
struct Response {
x0: u8, // unknown
x1: u8, // unknown
count: u8, // number of segments
segments: [Segment], // segment descriptors
}
struct Segment {
start: u8, // start index for segment
length: u8, // segment length
}
```
As an example, the following is a real response from a Hue Gradient light strip:
```
┌───┬───First segment descriptor
│ │
00 00 07 00 01 01 01 02 01 03 01 04 01 05 01 06 01
│ │ │ │
└header┘ └───────Seven segment descriptors───────┘
```
This tells us the segments are arranged thus:
- Start at `00`, length `01`
- Start at `01`, length `01`
- Start at `02`, length `01`
- ...
These are all length 1. In other words, the layout is:
`0, 1, 2, 3, 4, 5, 6`
### Command 7: Configure segments for entertainment mode (req/rsp)
Hue Entertainment frames consists of brightness and color data for up to 10
lights, all in a single frame.
Each light is identified by 2 bytes containing its zigbee network (short)
address.
For Hue devices that contain multiple lights (such as gradient strips), this
presents a problem, since the entire strip only has a single zigbee address!
To solve that problem, this command can be used on multi-segment devices to
configure each segment with a virtual address.
#### Request
```c
struct {
count: u16,
addresses: [count x u16],
}
```
Here is an example of a command that sets seven virtual addresses for a gradient
light strip with 7 segments:
```
┌───┬───Segment index 0
│ │
00 07 97 d2 98 d2 99 d2 9a d2 9b d2 9c d2 9d d2
│ │ │ │
└cnt┘ └───────Seven segment indices───────────┘
```
After this, the segments will respond the these addresses:
- `0xD297`
- `0xD298`
- `0xD299`
- `0xD29A`
- `0xD29B`
- `0xD29C`
- `0xD29D`
#### Response
```c
struct {
x0: u16,
}
```
The only observed response is `0000`, which probably indicates success.
Running this command on a Hue device that does not have multiple segments (i.e,
a regular Hue bulb) gets a "Command Not Supported" standard Zigbee response, so
returning `0000` seems to be a safe way to detect success.
## Attributes
| Attr | Type | Desc | Strip | Bulb | Firmware |
|--------|------|----------------------------|-------|------|----------------------------------------|
| `0000` | `b8` | ? | `0F` | `0B` | |
| `0001` | `e8` | ? | `00` | `00` | |
| `0002` | `u8` | Probably max segment count | `0A` | N/S | |
| `0003` | `u8` | Probably gradient-related | `04` | N/S | |
| `0004` | `u8` | Probably segment count | `07` | N/S | |
| `0005` | `u8` | Light balance factor | `FE` | `FE` | Fails on `1.76.11`, works on `1.122.2` |
Notice that attributes `0002`, `0003` and `0004` are not present on the hue
bulb. This supports the idea that these attributes are gradient-related.
So far the only attribute known on this cluster is `0x005`, which sets the light
level balancing for entertainment mode.
This is a feature where lights can be dimmed relatively, so certain lights
aren't blindingly bright. Just like regular brightness updates, the valid range
is `0x01` to `0xFE`. This should always be set to `0xFE`, unless you want to dim
the light in entertainment mode.
# Cluster 0xFC02
Never seen. Maybe they skipped a number?
# Cluster 0xFC03: Gradients, Effects, Animations
## Cluster-specific commands
### Command 0: Write combined state
This is perhaps the single most complicated Hue command. It is used to
simultaneously set all supported properties of a Hue bulb.
It has been extensively [documented in a separate document](hue-zigbee-format.md).
After setting the state with this command, it can be read back as property
`0x0002` (see below).
## Attributes
Sample values:
| Attr | Type | Desc | Strip | Bulb |
|--------|-------|-----------------|--------------------|--------------------|
| `0001` | `b32` | ? | `0000000F` | `00000007` |
| `0002` | `hex` | Composite state | `0700010a6e01` | `070001176f01` |
| `0010` | `b16` | ? | `0001` | `0001` |
| `0011` | `b64` | ? | `000000000003FE0E` | `000000000003FE0E` |
| `0012` | `b32` | ? | `00000003` | `00000000` |
| `0013` | `b16` | ? | `0007` | N/S |
| `0031` | `u16` | ? | `04E2` | N/S |
| `0032` | `u8` | ? | `00` | N/S |
| `0033` | `u8` | ? | `00` | N/S |
| `0034` | `u8` | ? | `03` | N/S |
| `0035` | `u8` | ? | `FE` | N/S |
| `0036` | `u8` | ? | `4F` | N/S |
| `0038` | `u16` | ? | `0007` | N/S |
The bulb supports noticably fewer properties, which makes it likely that the
missing ones are related to gradient handling.
# Cluster 0xFC04
Very rarely observed. Only seen with ZCL: Read Attributes.
## Attributes
| Attr | Type | Desc | Strip | Bulb |
|--------|-------|------|------------|------------|
| `0000` | `b16` | ? | `1007` | `1007` |
| `0001` | `b16` | ? | `0000` | `0000` |
| `0002` | `b16` | ? | `0000` | `0000` |
| `0010` | `u32` | ? | `00000000` | `00000000` |
| `0011` | `u32` | ? | `00000000` | `00000000` |
| `0012` | `u32` | ? | `00000000` | `00000000` |
| `0013` | `u32` | ? | `00000000` | `00000000` |

457
doc/hue-zigbee-format.md Normal file
View File

@@ -0,0 +1,457 @@
# Zigbee format for Philips Hue manufacturer-specific light updates
## Introduction
Philips hue lights support zigbee frames in a manufacturer-specific format, on cluster 0xFC03.
This type of message is necessary to support many of the advanced features in Hue lights, such as:
- Multiple colors ("gradient") in LED strips
- Light Effects ("Candle", "Fireplace", etc)
- Combining effects with color settings
Several attempts have been made to reverse this format before, but none have
managed to get everything decoded, although many attempts and techniques have
been employed. A few examples of the ongoing work:
- <https://github.com/kjagiello/hue-gradient-command-wizard/blob/main/src/modes/CustomGradient/utils.tsx>
- <https://github.com/Koenkk/zigbee-herdsman-converters/pull/5192>
- <https://github.com/zigpy/zha-device-handlers/issues/2517>
The best (and newest) work so far, is probably Krzysztof Jagiełło's "Hue Gradient Command Wizard":
- <https://kjagiello.github.io/hue-gradient-command-wizard/>
This one gets much right, but is still missing quite a few details.
Another invaluable resource when researching XY-based lights, is Thomas Lochmatter's RGB/XY converter:
- <https://viereck.ch/hue-xy-rgb/>
## Examples
Here are some examples of the zigbee messages discussed in this document (hex encoded):
- `50010000135000fffff3620c400f5bf4120d400f5b0cf4f43858`
- `ab00012e6f2f40100f7f`
- `51010104000d30040000fa441eb7cb49bff65f1800`
- `19000132518f530400`
- `1100000800`
- `bb0001feb575156904000a80`
- `51010104001350020000fa441e590834b7cb49ff8857bff65f2800`
At first glance, there's no obvious repeating pattern or fixed header in this
format, but with a combination of careful analysis and applied elbow grease, we
have managed to reverse the format in its entirety.
## Current state of the art (in zigbee-herdsman-converters)
The current state of the art in zigbee-herdsman-converters
(`srd/lib/philips.ts`) has patchy support for a few advanced features, but is
riddled with errors and inaccuracies. It also suffers from being written before a
complete understanding of the format was available, widely using "magic" numbers
that happen to work, although they may not be a good fit in the bigger picture.
There is great potential for improving zigbee-herdsman-converters, using the
information found in this repository (and in particular, this file).
# Frame format
Okay, let's start looking at the actual format now.
Philips hue lights work as simple I/O devices with up to 9 properties.
Each message to cluster 0xFC03 can update any chosen subset of these properties
as desired.
When sending an update, only the included properties will be affected. All other
properties will retain their previous values.
### Header
The first two bytes of the message form a little-endian integer that contains
these flags:
```text
FEDCBA98 76543210
xxxxxxxx xxxxxxxx
|||||||| ||||||||
|||||||| |||||||'--> ON_OFF
|||||||| ||||||'---> BRIGHTNESS
|||||||| |||||'----> COLOR_MIREK
|||||||| ||||'-----> COLOR_XY
|||||||| |||'------> FADE_SPEED
|||||||| ||'-------> EFFECT_TYPE
|||||||| |'--------> GRADIENT_PARAMS
|||||||| '---------> EFFECT_SPEED
||||||||
|||||||'-----------> GRADIENT_COLORS
||||||'------------> UNUSED_9
|||||'-------------> UNUSED_A
||||'--------------> UNUSED_B
|||'---------------> UNUSED_C
||'----------------> UNUSED_D
|'-----------------> UNUSED_E
'------------------> UNUSED_F
```
As an example, let us consider the message:
`530101c00400135000000094031fda1955b98347f00468c426792800`
The first bytes are [0x53, 0x01], which is 0x0153 in little-endian:
```text
0x01 0x53
/ \ / \
.......1 01010011
| ||||||||
| |||||||'--> ON_OFF
| ||||||'---> BRIGHTNESS
| |||||'----> -
| ||||'-----> -
| |||'------> FADE_SPEED
| ||'-------> -
| |'--------> GRADIENT_PARAMS
| '---------> -
|
'-----------> GRADIENT_COLORS
```
Now we can read the properties that have their corresponding flag set.
## Field order
The fields are always read this this order:
| Field | Size |
|-------------------|----------|
| `ON_OFF` | 1 byte |
| `BRIGHTNESS` | 1 byte |
| `COLOR_MIREK` | 2 bytes |
| `COLOR_XY` | 4 bytes |
| `FADE_SPEED` | 2 bytes |
| `EFFECT_TYPE` | 1 byte |
| `GRADIENT_COLORS` | variable |
| `EFFECT_SPEED` | 1 byte |
| `GRADIENT_PARAMS` | 2 bytes |
After reading the message in this manner, there shouldn't be any bytes left over.
However, this is seemingly a protocol that has been extended a few times (as can
be observed from the newest flags occupying the highest bits, for instance).
This would theoretically allow older devices to simply ignore all unknown flags,
and the corresponding "tail" of the message, while still reacting to the
properties they understand.
It is unknown if Hue devices actually operate in this way, or if they would
reject messages they do not fully understand.
Certainly, invalid messages with known flags are readily reject (and thus,
completely ignored), if they cannot be parsed 100% successfully.
### Property: `ON_OFF`
Size: 1 byte.
Light is turned off if `0`, or on otherwise.
### Property: `BRIGHTNESS`
Size: 1 byte.
NOTE: Values `0` and `255` are INVALID.
Valid range is `1..254` (dimmest to brightest, respectively)
### Property: `COLOR_MIREK`
Size: 2 bytes (little-endian)
Contains the color temperature in MIREK.
Typically valid range is `153` - `500` (both inclusive).
### Property: `COLOR_XY`
Size: 2 + 2 bytes (X, Y)
The color of the light, in XY format.
These coordinates are encoded as 16-bit little-endian integers, each
representing a fixed-point number in the range `0`..`1`.
Here 0 represents `0.0` and `0xFFFF` represents `1.0`.
### Property: `FADE_SPEED`
Size: 2 bytes (little-endian)
This number sets the transition speed for applying the new properties.
A value of 0 makes all transitions as fast as possible (practically
instant). Typical practical values are in the range `2..8`.
While values above 0x100 are possible, these cause very slow
transitions. However, the animation is running inside the light, so this could
be a good way to enable smooth, lightweight light transitions.
### Property: `EFFECT_TYPE`
Size: 1 byte (specifically, [`zigbee::EffectType`])
| Name | Value |
|--------------|-------|
| `NoEffect` | 0x00 |
| `Candle` | 0x01 |
| `Fireplace` | 0x02 |
| `Prism` | 0x03 |
| `Sunrise` | 0x09 |
| `Sparkle` | 0x0a |
| `Opal` | 0x0b |
| `Glisten` | 0x0c |
| `Sunset` | 0x0d |
| `Underwater` | 0x0e |
| `Cosmos` | 0x0f |
| `Sunbeam` | 0x10 |
| `Enchant` | 0x11 |
This enables one of the specific, known effects in the [`zigbee::EffectType`]
enum. Most (all?) effects allow setting other properties (such as color xy or
color temperature) while the effect is active.
This is how custom effects like "Purple Fireplace" or "Blue Candle" from the Hue
app are activated.
### Property: `GRADIENT_COLORS`
For gradient light strips, this property allows setting a number of independent
colors at once.
The gradient colors black is the most complicated of the property data blocks
used in this format. It has the following layout:
```text
┌───────────┬───┬───┬───┬───┬───┬───┬───┬───┐
│ Byte Bit ► 7 │ 6 │ 5 │ 4 │ 3 │ 2 │ 1 │ 0 │
├─▼─────────┼───┴───┴───┴───┴───┴───┴───┴───┤
│ 0 │ .size (excluding this field) │
├───────────┼───────────────┬───────────────┤
│ 1 │ .color_count │ MUST be zero! │
├───────────┼───────────────┴───────────────┤
│ 2 │ .gradient_style │
├───────────┼───────────────────────────────┤
│ 3 │ Reserved (seems unused) │
│ │ │
│ 4 │ │
├───────────┼───────────────────────────────┤
│ 5+3*index │ .color_x (low 8 bits) │\
│ ├───────────────┐ - - - - - - - │ \
│ 6+3*index │ (low 4 bits) │ (high 4 bits) │ Repeated {.color_count} times
│ │ - - - - - - - └───────────────┤ /
│ 7+3*index │ .color_y (high 8 bits) │/
├───────────┼───────────────────────────────┤
: : :
: : :
```
The gradient style *must* be one of these values:
```rust
pub enum GradientStyle {
Linear = 0x00,
Scattered = 0x02,
Mirrored = 0x04,
}
```
### Property: `GRADIENT_COLORS`: Color encoding
The gradient colors format can specify up to and including 9 colors, even for
light strips with fewer than 9 segments. Any attempt to specify 10 or more
colors will result in the entire message being rejected.
Each color is packed into 3 bytes, representing 12 bits for the `X` and `Y`
color coordinate, respectively. These bytes are packed in an odd way. The
following code snippet demonstrates how to unpack them:
```rust
let bytes: [u8; 3] = [0x11, 0x22, 0x33];
let x = u16::from(bytes[0]) | u16::from(bytes[1] & 0x0F) << 8;
let y = u16::from(bytes[2]) << 4 | u16::from(bytes[1] >> 4);
```
And packing:
```rust,no_run
let x = 0x123;
let y = 0x456;
let bytes: [u8; 3] = [
(x & 0xFF) as u8,
(((x >> 8) & 0x0F) | ((y & 0x0F) << 4)) as u8,
(y >> 4 & 0xFF) as u8,
];
```
These 12-bit values are fractional values, but NOT in the unit range 0..1 as
might be expected.
Instead, the coordinates are scaled so that precision is not wasted on useless
coordinates outside the visible light spectrum, for example.
Other implementations all seem to make this guess about the scaling:
```rust
const max_x: f32 = 0.7347;
const max_y: f32 = 0.8431;
```
As far as I can tell, these numbers appeared at one point in someone's
implementation (probably as a best guess), and have been mercilessly copy-pasted
since then. If anyone can show a good source for why these numbers would be
correct, please let me know!
Now, the X coordinate makes a lot of sense, and is right as far as I can tell.
The value 0.7347 is the maximum X value inside the visible light spectrum.
For a visual illustration of this, see <https://viereck.ch/hue-xy-rgb/>.
The "Wide" color gamut also has this exact number in its specification, as the X
value of the "Red" coordinate, specifically.
However, the Y value doesn't match any source I can find.
If the scaling matches the Wide Gamut, the Y value (maximum height) should be
`0.8264`. If it matched the top of the visible light area, it should be around
`0.836`. I have no idea where `0.8431` comes from!
From experimentation, I have determined that the most likely candidates are the
outer bounds of the wide gamut, leading to the following values:
```rust
const MAX_X: f64 = 0.7347;
const MAX_Y: f64 = 0.8264;
```
These are then the scaling values used when serializing/deserializing the 24-bit
(X,Y) values in the gradient colors.
In other words, X values from `0` to `0xFFF` represent the X-coordinates `0.0`
to `0.7347`, while Y values in the same range represent Y-coordinates from `0.0`
to `0.8264`.
### Property: `EFFECT_SPEED`
Size: 1 byte
This property controls the animation speed for effects (see `EFFECT_TYPE`).
All values in the range `0`..`255` seem to be allowed, with `0` being the
slowest and `255` the fastest.
Curiously, the Hue app does not use the full range for all effects, and at high
values, some animations are rendered so quickly that they start to break down.
A good starting point seems to be 128 (representing 0.5).
### Property: `GRADIENT_PARAMS`
Size: 2 bytes (`scale`, `offset`)
The gradient parameters block contain two bytes, describing the `scale` (first
byte) and `offset` (second byte).
Both bytes are in fixed-point format, with the upper 5 bits representing the
integer portion, and the lower 3 bits the fractional part:
```text
| 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
\ / \ /
\________________/ \________/
integer fraction
```
Here are some examples of translating to/from this fixed-point format:
| Encoded value | Quotient | Numeric value |
|---------------|----------|---------------|
| 0x00 | 0/8 | 0.0 |
| 0x01 | 1/8 | 0.125 |
| 0x04 | 4/8 | 0.5 |
| 0x08 | 8/8 | 1.0 |
| 0x38 | 56/8 | 7.0 |
| 0x39 | 57/8 | 7,125 |
| 0x3a | 58/8 | 7.25 |
#### Property: `GRADIENT_PARAMS`: `scale`
For a gradient light strip, the `scale` value determines how "wide" the gradient
colors are rendered. Specifically, a gradient light strip will scale the
gradient colors to fit "scale" colors on the strip.
As an example, the Hue Play gradient lightstrip for PC (model `LCX005`) has 42
LEDs, but only 7 independent sections. Each group of 6 LEDs can thus be thought
of as one "pixel".
With such a low pixel count, `scale` greatly affects the resulting colors when
updating the gradient strip.
For sharp, clear colors, the scale should match the number of segments in the
light strip. Again using the `LCX005` as an example, a good value for `Linear`
gradient mode is `0x38` (since this represents `7.0` in the fixed-point
notation).
This allows each of the colors to fit exactly in a segment on the strip, but
other values are possible too, of course. For example, `0x08` (= `1.0`) will
show one color on the entire strip, while `0x10` (= `2.0`) will show a smooth
transition between the first and second colors.
The above example is for the `Linear` gradient style only. The `Mirrored` mode
uses the middle segment as the base, and thus has 3 available (mirrored)
segments on either side, on a 7-segment light strip.
The `Scattered` mode always shows colors aligned with segments. As a result,
`scale` is ignored in this mode.
| Gradient style: | Linear | Mirrored | Scattered |
|-----------------------------------|--------|----------|-----------|
| Scale to show single color | 0x08 | 0x08 | N/A |
| Scale to fade between 2 colors | 0x10 | 0x10 | N/A |
| Scale to fit colors to 7 segments | 0x38 | 0x20 | N/A |
NOTE: The `scale` value MUST BE at least `0x08` (= `1.0`)
NOTE: The `scale` value `0x00` is a special condition. It stretches the gradient
colors to exactly fit the gradient strip. Kind of a "zoom to fit" option.
#### Property: `GRADIENT_PARAMS`: `offset`
This property is simpler than `scale`. When rendering the gradient colors to the
light strip, the first `offset` lights are skipped.
For example, assume we have these abstract values for gradient light colors:
```text
|-------------------|
| A | B | C | D | E |
|-------------------|
```
With `offset` set to `0`, the colors will be rendered starting with `A`, so
[`A`, `B`, `C`, ...].
With `offset` set to `0x08` (= `1.0` in the fixed-point format), the first color
shown will be `B`, so [`B`, `C`, `D`, ...], and so forth.
For *fractional* offset values, proportional blending is used to emulate the
sub-pixel offset. With an offset of `0x04` (= `0.5`), the rendered colors will be:
- `50% A + 50% B`
- `50% B + 50% C`
- `50% C + 50% D`
- ...
If unsure, a good value for `offset` is `0x00`.

47
doc/implementation-status.md Normal file
View File

@@ -0,0 +1,47 @@
## Implementation status
### Legacy (V1 API)
| Feature | Endpoint | Status |
|-------------|--------------------------------------|--------------|
| Minimal API | `/api/config`, `/api/:userid/config` | ✅ |
| Lights | `/api/:user/lights` | ✅ (partial) |
| Groups | `/api/:user/groups` | ✅ (partial) |
| Scenes | `/api/:user/scenes` | ✅ (partial) |
| Sensors | `/api/:user/sensors` | ❌ |
| Endpoint | GET | PUT | POST | DELETE |
|----------------------------|-----|-----|------|--------|
| `/` | - | - | ✅ | - |
| `/config` | ✅ | - | - | - |
| `/:user` | ✅ | - | - | - |
| `/:user/config` | ✅ | ❌ | ❌ | ❌ |
| `/:user/lights` | ✅ | ❌ | ❌ | ❌ |
| `/:user/groups` | ✅ | ❌ | ❌ | ❌ |
| `/:user/scenes` | ✅ | ❌ | ❌ | ❌ |
| `/:user/capabilities` | ✅ | ❌ | ❌ | ❌ |
| `/:user/<other>` | ❌ | ❌ | ❌ | ❌ |
| `/:user/lights/:id` | ✅ | - | - | ❌ |
| `/:user/groups/:id` | ✅ | - | - | ❌ |
| `/:user/scenes/:id` | ✅ | - | - | ❌ |
| `/:user/lights/:id/state` | - | ✅ | - | - |
| `/:user/groups/:id/action` | - | ✅ | - | - |
### Modern (V2 API)
| Feature | Implemented | Notes |
|-----------------|-------------|----------------------------------------------------------------------------------------------------------|
| Authentication | ❌ | No authentication! Everybody has full access |
| Config | ✅ | |
| Event streaming | ✅ | Can send updates for lights, groups, rooms, scenes |
| Lights | ✅ | Supports on/off, color temperature, full color |
| Groups | ✅ | Automatically mapped to rooms |
| Scenes | ✅ | Scenes can be created, recalled, deleted. Scenes found in zigbee2mqtt will be imported, and auto-learned |
| Feature | GET | POST | PUT | DELETE |
|---------------------|-----|------|--------------|--------|
| Lights | ✅ | - | ✅ (partial) | - |
| Groups | ✅ | ❌ | ✅ (partial) | ❌ |
| Scenes | ✅ | ✅ | ✅ (partial) | ✅ |
| Entertainment Zones | ✅ | ✅ | ✅ | ❌ |

BIN
doc/logo-title-320x80.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

BIN
doc/logo-title-640x160.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

150
doc/logo-title.svg Normal file
View File

@@ -0,0 +1,150 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
version="1.1"
xlink="http://www.w3.org/1999/xlink"
width="512"
height="128"
viewBox="10 10 512 128"
id="svg6"
sodipodi:docname="logo-title.svg"
inkscape:version="1.2.2 (b0a8486541, 2022-12-01)"
inkscape:export-filename="logo-title.png"
inkscape:export-xdpi="60"
inkscape:export-ydpi="60"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<defs
id="defs10">
<linearGradient
inkscape:collect="always"
id="linearGradient3045">
<stop
style="stop-color:#060667;stop-opacity:1;"
offset="0"
id="stop3041" />
<stop
style="stop-color:#2020e2;stop-opacity:1;"
offset="0.78752887"
id="stop3043" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient2984">
<stop
style="stop-color:#f8ffff;stop-opacity:1;"
offset="0"
id="stop2980" />
<stop
style="stop-color:#000000;stop-opacity:1;"
offset="1"
id="stop2982" />
</linearGradient>
<linearGradient
inkscape:collect="always"
id="linearGradient2900">
<stop
style="stop-color:#ffffff;stop-opacity:1;"
offset="0"
id="stop2896" />
<stop
style="stop-color:#ffffff;stop-opacity:0;"
offset="1"
id="stop2898" />
</linearGradient>
<linearGradient
id="linearGradient2884"
inkscape:swatch="solid">
<stop
style="stop-color:#000000;stop-opacity:1;"
offset="0"
id="stop2882" />
</linearGradient>
<rect
x="145.39399"
y="28.545941"
width="231.60111"
height="70.066312"
id="rect289" />
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient2900"
id="linearGradient2902"
x1="150.29653"
y1="58.880343"
x2="329.76849"
y2="58.880343"
gradientUnits="userSpaceOnUse" />
<radialGradient
inkscape:collect="always"
xlink:href="#linearGradient2984"
id="radialGradient2968"
cx="74"
cy="74"
fx="74"
fy="74"
r="63.999989"
gradientUnits="userSpaceOnUse" />
<radialGradient
inkscape:collect="always"
xlink:href="#linearGradient3045"
id="radialGradient3049"
cx="74"
cy="74"
fx="74"
fy="74"
r="63.999989"
gradientUnits="userSpaceOnUse" />
</defs>
<sodipodi:namedview
id="namedview8"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
inkscape:showpageshadow="2"
inkscape:pageopacity="0"
inkscape:pagecheckerboard="true"
inkscape:deskcolor="#d1d1d1"
showgrid="false"
inkscape:zoom="2.8750344"
inkscape:cx="228.34509"
inkscape:cy="95.129297"
inkscape:window-width="2391"
inkscape:window-height="1394"
inkscape:window-x="0"
inkscape:window-y="0"
inkscape:window-maximized="0"
inkscape:current-layer="g954"
showborder="true"
shape-rendering="auto" />
<g
id="g954"
style="stroke:url(#radialGradient2968)">
<circle
style="fill:url(#radialGradient3049);stroke:url(#radialGradient2968);stroke-width:13.3838;stroke-dasharray:none;paint-order:stroke fill markers;stop-color:#000000;fill-opacity:1"
id="path345"
cx="74"
cy="74"
r="57.30809" />
<g
fill="#ffffff"
id="g4"
style="fill:#ffffff;stroke:url(#radialGradient2968)"
transform="matrix(0.72365752,0,0,0.72365752,23.817124,23.817124)">
<path
d="M 69.346173,18.146169 18.146169,69.346173 69.346173,120.54618 120.54618,69.346173 Z M 30.21273,69.346173 69.346173,30.21273 85.896575,46.763131 79.862654,52.797052 69.346173,42.283131 42.283131,69.346173 52.795772,79.862654 46.761851,85.896575 Z m 54.133765,0 L 69.346173,84.350335 54.349692,69.346173 69.346173,54.349692 Z m -31.550723,22.584322 6.03392,-6.03392 10.516481,10.512641 27.066883,-27.063043 -10.516481,-10.516481 6.03392,-6.03392 16.549125,16.550401 -39.133447,39.133447 z"
id="path2"
style="fill:#ffffff;stroke-width:1.28;stroke:url(#radialGradient2968)" />
</g>
</g>
<text
xml:space="preserve"
transform="matrix(1.7297511,0,0,1.7297511,-98.188589,-27.848338)"
id="text287"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:56px;line-height:125%;font-family:MaxTF-SemiBold;-inkscape-font-specification:MaxTF-SemiBold;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect289);shape-padding:0.250766;display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:#ffffff;stroke-width:0.578118;stroke-linecap:round;stroke-linejoin:bevel;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:normal"><tspan
x="145.64453"
y="79.572343"
id="tspan3079">Bifrost</tspan></text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 5.1 KiB

52
doc/logo.svg Normal file
View File

@@ -0,0 +1,52 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
version="1.1"
xlink="http://www.w3.org/1999/xlink"
width="128"
height="128"
viewBox="10 10 128 128"
id="svg6"
sodipodi:docname="logo.svg"
inkscape:version="1.2.2 (b0a8486541, 2022-12-01)"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<defs
id="defs10" />
<sodipodi:namedview
id="namedview8"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
showgrid="false"
inkscape:zoom="5.2546875"
inkscape:cx="-39.583705"
inkscape:cy="51.002081"
inkscape:window-width="5120"
inkscape:window-height="1361"
inkscape:window-x="0"
inkscape:window-y="0"
inkscape:window-maximized="1"
inkscape:current-layer="svg6" />
<circle
style="fill:#0044aa;stroke:#000000;stroke-width:13.3838;stroke-dasharray:none;paint-order:stroke fill markers;stop-color:#000000"
id="path345"
cx="74"
cy="74"
r="57.30809" />
<g
fill="#ffffff"
id="g4"
style="fill:#ffffff"
transform="matrix(0.72365752,0,0,0.72365752,23.817124,23.817124)">
<path
d="M 69.346173,18.146169 18.146169,69.346173 69.346173,120.54618 120.54618,69.346173 Z M 30.21273,69.346173 69.346173,30.21273 85.896575,46.763131 79.862654,52.797052 69.346173,42.283131 42.283131,69.346173 52.795772,79.862654 46.761851,85.896575 Z m 54.133765,0 L 69.346173,84.350335 54.349692,69.346173 69.346173,54.349692 Z m -31.550723,22.584322 6.03392,-6.03392 10.516481,10.512641 27.066883,-27.063043 -10.516481,-10.516481 6.03392,-6.03392 16.549125,16.550401 -39.133447,39.133447 z"
id="path2"
style="fill:#ffffff;stroke-width:1.28" />
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB