All In with Home Assistant

I’ve spent parts of the last 9 months playing with Home Assistant and have written about some of my adventures. A few weeks ago, I finally decided to go all in with Home Assistant and ditch my Vera. I bought an Aeotec Z-Stick Gen5 Z-Wave dongle and starting moving all my devices over to it. Within a few days, I had all my devices moved over and unplugged my Vera. Everything was running great on my Raspberry Pi B, but I noticed that the History and Logbook features were slow. I like looking at the history to look at temperature fluctuations in the house.

History graph

I had read that switching from the SQLite database to a MySQL database would speed things up. So I installed MariaDB (a fork of MySQL) on my Raspberry Pi and saw a slight increase in speed, but not much. Next was to move MariaDB to a separate server using Docker. Again, a slight increase in speed, but it still lagged. At this point everything I read pointed to running Home Assistant on an Intel NUC or another computer. I didn’t want to invest that kind of money in this, so I took a look at what I had and started down the path of installing Ubuntu on my old Mac mini which was completely overkill for it (Intel Quad Core i7, 16 GB RAM, 1 TB SSD). Then I remembered that I had read about a virtual machine image for Hass.io and decided to give that a try.

After some experimenting, I managed to get Home Assistant installed on a virtual machine running in VMWare on my Mac Pro. (A few days after I did this, I saw that someone posted an article documenting this.) I gave the VM 8 GB of RAM, 2 cores (the Mac Pro has 12) and 50 GB of storage. Wow, the speed improvement was significant and history now shows up almost instantly (the database is running in a separate VM)! I was so pleased with this, I decided to unplug the Raspberry Pi and make the virtual machine my home automation hub. There were a few tricks, however. The virtual machine’s main disk had to be setup as a SATA drive (the default SCSI wouldn’t boot), suspending the VM confused it, and the Z-Wave stick wouldn’t reconnect upon restart. After much digging, I found the changes I needed to make to the .vmx file in the virtual machine:

    suspend.disabled = "TRUE"
    usb.autoConnect.device0 = "name:Sigma\ Designs\ Modem"

(The USB auto connect is documented deep down on VMWare’s site.)

I’ve rebooted the Mac Pro a few times and everything comes up without a problem very quickly, so I’m now good to go with this setup. Z-Wave takes about 2.5 minutes to finish startup vs. 5 or 6 on the Pi. A friend asked if I was OK with running a “mission critical” component on a VM. I said that I was because the Mac Pro has been rock solid for a long time and my virtual machines have been performing well. I could change my mind later on, but I see no reason to spin up another machine when I have a perfectly overpowered machine that is idle 95% of the time.

What next? Now that I have more power for my automation, I may look at more pretty graphs and statistics. I may also just cool it for awhile as I’ve poured a lot of time into this lately to get things working to my satisfaction. This has definitely been an adventure and am glad that I embarked on it.

Setting up a Leviton VRCZ4-M0Z for use with Home Assistant

I’ve been so pleased with Home Assistant that I decided to see if I could migrate completely away from Vera and run all my Z-Wave devices on Home Assistant. Currently Home Assistant uses OpenZwave as the base and has basic support for a lot of Z-Wave devices. While OpenZwave isn’t as mature as Vera in its implementation, I found that with the exception of 4 Leviton VRCZ4-M0Z zone controllers, everything worked well. I’ve read that some people have had problems with Z-Wave on Home Assistant, but so far things have been going quite well for me using an Aeotec Z-Stick Gen5 as the controller. I suspect that my success is due to the type of devices I have (only outlets, switches, 1 light bulb, 2 portable MiniMote controllers and these VRCZ4s); I don’t have any sensors and only 2 of my devices are battery powered.

In order to complete my transition away from Vera, I had to get the 4 VRCZ4s to work. A Google search turned up very little information on how to do this with the exception of one post that gave me some clues. The post uses a third party piece of software that I don’t have to program the controllers, so that was pretty much out. However, the post did talk about the SCENE_CONTROLLER_CONF Z-Wave command class. With this information in hand, I decided to see if I could program my controllers.

Here’s what I eventually did.

Controller Programming

  1. Sign up for a developer account on Silicon Labs.
  2. Download the Z-Wave Developer kit, specifically the Z-Wave PC Controller software. (Windows only, but works fine in VMWare Fusion)
  3. Reset the VRCZ4 by pressing and holding the left side of buttons 1 & 3 until it blinks amber and then remains a solid red.
  4. Press left side of buttons 1 & 3 of the VRCZ4.
  5. Add node in Z-Wave config in HA
    Add Node
  6. Wait until node is added. I like to look the the OZW_Log.txt to check status. The node will say Complete next to it when it is done.
  7. Select node in Z-Wave config called Leviton VRCZ4-M0Z. Make a note of the node number.
    VRCZ4 Node
  8. Rename Entity by going to Node Information, selecting the gear and entering a new node.
  9. Under Node Group Associations, select Group 1
  10. Make sure Aeotec Z-Stick (node 1) is selected and click Add to Group.
    Screen Shot 2019 01 03 at 9 33 22 AM
  11. Repeat the above 2 steps for Groups 2-4. The VRCZ4 needs to know where to send commands when the buttons are pressed.
  12. If you have other devices you want to control such as lights or outlets and they support Z-Wave association, associate the devices with the VRCZ4 now. Groups 1-4 corresponds with buttons 1-4. So if you want button 1 to control an outlet, associate it with the outlet.
  13. If all the buttons on the VRCZ4 are going to be associated with other devices, you can stop here as the associations will cause the buttons to turn on/off the associated devices. You can even associate more than 1 device to a button (i.e. 2 backyard lights).
  14. If you want of the buttons to control scripts or automations on Home Assistant, you’ll have to do some more steps.
  15. Shutdown the HA box (not just HA, but the entire box).
  16. Connect Aeotec Z-Stick to computer with Z-Wave PC Controller software.
  17. Launch PC Controller software.
    Z-Wave PC Controller
  18. Click Command Classes
  19. Locate node of controller on left side.
  20. Click on it and then click Node Info.
    Command Classes
  21. Double click SCENE_CONTROLLER_CONF in the lower left.
  22. Change command to SCENE_CONTROLLER_CONF_SET.
    Change Command Class
  23. Click the button in the lower right to show the log.
  24. At this point, you need to program each button where you are not using associations to control devices.
  25. The Group IDs for the buttons are as follows:
    Button 1 on - 1
    Button 2 on - 2
    Button 3 on - 3
    Button 4 on - 4
    Button 1 off - 5
    Button 2 off - 6
    Button 3 off - 7
    Button 4 off - 8
    
  26. For each group ID, you have to assign a scene ID. Apparently the scene IDs should be unique across the entire Z-Wave network, but for my case I assigned the same scene IDs for each controller and can distinguish between the controllers in Home Assistant. You can choose whatever numbering scheme you want, but I went with the following:
    Button 1 on - 1
    Button 1 off - 2
    Button 2 on - 3
    Button 2 off - 4
    Button 3 on - 5
    Button 3 off - 6
    Button 4 on - 7
    Button 4 off - 8
    
  27. For each group ID, assign a scene ID and click send.
  28. You’ll see some message in the logs.
  29. To verify that the scene IDs are set, press a button and in the log you will see a SCENE_ACTIVATION_SET message with the scene ID.
    Z-Wave Log
  30. Note that the group IDs and the scene IDs are in hex. Since I’m only using 1-8, it doesn’t matter but if you use different scene IDs, be aware of this.
  31. Repeat for each button you want to set.
  32. Unplug the Aeotec Stick from the computer and plug it back into HA and reboot HA.

Using the Scenes

The programming has now been done, so the next step is to use the new commands in Home Assistant. I use Node-RED and have setup a sequence that handles all of my controllers.

Node-RED Sequence
1. Drag an events node (top left in Home Assistant) to the workspace and set it up to only look at zwave.scene_activated events.
Scene Events
2. Use a switch node to differentiate between the controllers.
Switch Node
3. Use a change node to extract the scene ID.
Extract Scene ID
4. Next use a switch node to separate out the scenes. If you didn’t follow my scene numbering, you will have to enter whatever scene IDs you used.
Separate out scenes
5. Connect up nodes for each scene.

While this seems like a lot of work, I probably took longer for me to write this up then to actually configure a controller! I’m not an expert in Home Assistant, Node-RED or Z-Wave so send feedback if you have any.

Enjoy!

Home Assistant and Node-RED, automation perfected?

Earlier this year I started experimenting with Home Assistant and wrote some thoughts about it. It brought together enough pieces that I keep tinkering with it. A friend of mine had mentioned a project called Node-RED which is supposed to more easily link together IoT components and perform actions based on different inputs. At the time, I brushed it off as I didn’t want to bother figuring out how to install it.

Fast forward to last week when I noticed that Home Assistant had a Node-RED add-on. Installing the add-on was quite easy and I was presented with a blank canvas. After a few minutes, I figured out how to make a simple sequence that took the state of a door sensor (at the time connected via my Vera) and turned on a light. Nothing too fancy, but I was able to hook it up, hit Deploy and test. It worked! This was light years ahead of the YAML based automations in Home Assistant and much faster to setup than Vera. I was hooked almost immediately. Could I convert all my automation logic to this? I spent the next few days trying.

Wiring some basic automations was quite easy as in this example that turns off lights in my bathroom if there is no motion. Node-RED resets the timer if there is more motion, making the sequence very straightforward.
Basic Node-RED Sequence

Unfortunately not all my automations are this simple.

The most important sequences deal with my front and back motion sensors. I could just use something like the above sequence, but I only want the lights on at night, I want to be able to disable the motion sensors (for example on Halloween if we’re not home, I don’t want lights coming on), and if I turn on the lights manually I don’t want them turning off a few minutes after there is motion. The tricky part here was to determine if the lights were manually turned on or triggered by a motion sensor. After a bit of experimenting, I decided to record the last time there was motion into a variable. Then when the lights turn on I check the variable to see how long ago it occurred. If it was less than 30 seconds ago, call it triggered my motion. With that, I was able to set on off timer based on how the lights came on. The one bit of “code” I had to write was to determine how long ago the motion was tripped.

    var motionTimestamp = flow.get('lastFrontMotion') || -1;
    var difference = (Date.now() - motionTimestamp) / 1000;
    if (motionTimestamp === -1)
    {
        difference = -1
    }
    msg.payload = difference;
    return msg;

The flow may look complicated, but to me it is quite readable.

Complicated Node-RED Flow

The visual aspect of Node-RED makes it easier to setup automations, but calling automation simple is far from the truth as I’ve shown above. As a professional software developer, writing code doesn’t scare me but for a hobby, this visual approach (with a little code as necessary) is much nicer. When I come back to this in 6 months, I have no doubt that I can read what is going on and troubleshoot as necessary.

For the last 5 years, I’ve been using my Vera to control everything using a plugin called PLEG which stands for Program Logic Event Generator. It has worked quite well, but it has been so long since I setup most of the automations, I have no idea what I did. PLEG, while functional, was a bit difficult for me to wrap my head around and it pained me every time I had to touch it. Also, when I touched it, I seem to recall having to restart Vera and wait only to find out that I needed to change something.

I’m so impressed with Node-RED that I’ve decided to see if I can move all my Z-Wave devices to Home Assistant using an Aeotec Z-Stick Gen5 plugged into the Home Assistant. The goal with this move is to speed up messaging; right now the Home Assistant polls (I think) Vera all the time looking for changes. This isn’t very efficient and pressing a button can take a second or two to have the message reach Home Assistant. Will this work? I hope so!

I know that I’m just scratching the surface with this, but I am very excited over the prospects!

Experimenting with Home Assistant

Last week I read that Ubiquiti Networks had hired the main author of the Home Assistant home automation project. I looked at the project and at first I couldn’t understand what the project would do, but after poking at it, I realized that it is the glue that connects disparate automation systems. I wrote about putting together various pieces together and thought that maybe Home Assistant could put all the pieces together in one little box.

Over the course of a few hours last weekend, I installed Home Assistant on a Raspberry Pi 2, configured it to connect with my Vera and set it up for HomeKit and Amazon Echo. Right away Home Assistant removed Homebridge and HA Bridge from my system; fewer parts means it is easier to maintain.

Looking at the list of available components, it is clear that Home Assistant could replace my Vera and could control everything without me having to put together all the little parts. It has components for Envisalink, my Russound audio distribution units, my Squeezebox devices and everything else I could throw at it. If I put a Z-Wave stick on the Pi, Home Assistant could also natively handle Z-Wave. In order to replace my Vera, I’d have to convert my schedules and my PLEG actions over to Home Assistant which is not an insignificant task.

While I’m not ready to say that Home Assistant is the clear winner in the home automation game as configuring it is quite painful (most of it has to be configured via specifically formatted YAML files), it is very intriguing. If the author does what he has said he’s going to do to bring more of the configuration to the GUI, but leave advanced features to the YAML files, I’ll be quite happy. It isn’t for the faint of heart, but well worth a look for any home automation enthusiast.

Cobbling Together a Home Automation System

I was introduced to home automation at a young age when my father installed an X10 system hooked to our alarm system. The basic gist of the automation was to turn on lights when we were away and to have all the lights in the house turn on if the alarm system activated. This was more than 30 years ago and was pretty simple in what he wanted to accomplish. Since then I’ve dabbled with the X10 Firecacker hooked to a Linux box, but really didn’t get serious about home automation until about four and a half years ago when we purchased our current house. We were remodeling the house we purchased, so I used the opportunity to design an automation system and briefly wrote about it.

Many lights, several outlets, and the thermostat in our house are using the ZWave protocol. This only covers part of the automation. All of the sensors (motion, window, and door) are on a standard DSC security system hooked up with an Envisalink module. The brains of the system is a VeraEdge with several third party plugins. Out of the box, the VeraEdge won’t talk to the sensors and has very basic decision making capability, so the Program Logic Event Generator plugin is needed. In addition, I put together a plugin for controlling my Russound audio distribution units that works by sending commands to a PortServer TS 4 MEI and then to the Russound’s RS-232 serial ports.

This whole system worked well for several years, but I wasn’t happy with the iOS app, so I created my own app a few years ago.

If this sounds like a lot of pieces and a lot of work, it has been a labor of love and an interesting hobby. Of course having a system working well isn’t a lot of fun if you like tinkering! When I received my first Amazon Echo two years ago, I was intrigued by the ability to control things by voice. My Vera (at the time) had no integration with the Echo, so I turned to a third party solution called HA Bridge by BWS Systems. It required more setup, but exposed scenes and devices so that they could be controlled by the Echo. (Vera now has native Echo support, but the way it is setup creates a delay in the response so I continue to use HA Bridge.) The new Routines in the Amazon Echo like adding one for “Good night” are an excellent addition, but has me scratching my head on if I should set that kind of thing up on the Echo, use the Vera natively, or use HA Bridge.

As an Apple fan, you’ll probably notice that I haven’t mentioned HomeKit, yet. HomeKit has been interesting, but due to the lack of interoperable components and hackability, I’ve pretty much ignored it. Some enterprising developers have created Homebridge which I’ve also setup. Why HomeKit and Echo support? Simple, with the HomeKit app on iOS 11 and the widget, I can easily turn lights on or off right from my iPad without switching to my app and without having to yell at the Echo. In addition, this allows me to control lights with my Apple Watch. I run HA Bridge and Homebridge on a Raspberry Pi so that it can easily sit on my IoT VLAN (I separate out IoT traffic from my main network).

This automation setup has been about 5 years in the making and I’m sure I’m not completely done with it. I keep looking at ways to simplify it with different components like maybe a new hub or new switches. However, every time I start looking into it I realize that I’d lose functionality. My wife tolerates my hobby and kind of likes some of the aspects of it as she warms up to them. Changing anything would be a change for the worse.

The more time and effort I put into my automation system, the more I realize that turnkey home automation is still a pipe dream. If someone has very simple needs then maybe he or she can get away with one system. The moment you want a choice in your components or want to do something that the major companies that do automation don’t want you to do, you have to either give up or cobble together a system.

Products used in my system:

Hardware

  • VeraEdge – main automation hub
  • DSC alarm system
  • Envisalink – alarm system to network
  • PortServer TS 4 MEI
  • Russound audio distribution units (no longer made) with RS-232 ports
  • Squeezebox audio units (no longer made)
  • Raspberry Pi for running Homebridge and HA Bridge
  • Amazon Echo
  • iPad/iPhone/Apple Watch

Software

Feel free to ask questions about my system. I’m always open to ideas and will keep actively exploring ways to improve my life through automation!