OS X Server and the runaway process

Yesterday I heard the fan running like mad on my Mac Mini. My mini is my workhorse machine that is my media center, surveillance system, and build server. This is not uncommon as it processes video at times and when it builds certain projects (Swift seems to cause it more), the processor has to work harder. Normally the load is about 30% (Quad Core i7), but when the fan comes on loud enough for me to notice, it is hitting the 50-90% range. I did a little poking at the machine and saw a process called sdmd hogging the CPU. I had never heard of this process, but a quick search found some references on Apple's support discussion threads.

I read through all the responses and turned off "Create personal folders when users connect on iOS".

Unfortunately that was only half the solution. The other half of the answer that I ignored on the first pass was a setting in each Shared Folder that enables iOS sharing. I had recently added a shared item and didn't notice the iOS setting.

Once I unchecked iOS, the load started going down and little files stopped being created. By this time, I had over 150,000 little files! These were thumbnails from my cameras collected over the last week. Using the Finder to delete the files was an exercise in waiting, so I searched for a UNIXy way to do this and found the answer. Switch to the /Library/Server/ServerDocs/Data directory and issue:

sudo find . -type f -print -delete

This quickly nukes all files in the current directory.

I still have mixed feelings about OS X Server; it is working pretty well for me right now to perform basic functions including Time Machine backups, but sometimes it does magic where it takes time for me to figure out how to undo its magic.

Review: Amazon Echo

[19 Jan 2015, 7:38 AM - Minor Edits]

When Apple first announced HomeKit, I was excited for an easy to use system for home automation. Unfortunately or fortunately, they left the actual implementation to the manufacturers. The first HomeKit devices that came out were pretty simplistic and would allow you to turn lights on and off. To me, this was a bit useless. In order for automation to be truly valuable, there had to be rules for different things to happen based on inputs that worked even if my phone wasn't around (what good is automation if it can't turn on lights in the middle of the night when you're not home to scare someone away?). Now that more manufacturers are getting into automation, we're seeing hubs such as Insteon incorporate HomeKit; I'm not exactly sure what it does, but it is a start.

I glanced at the forums for the Vera hub that I use periodically and saw that someone had created a Vera HomeKit Bridge. I had some time one day and installed it and picked up one of the apps on the iOS app store that had an Apple Watch component and gave it a whirl. Telling Siri on my watch to turn a light on and off was interesting, but due to the speed of Siri, it was more of a gimmick than anything else.

I was now intrigued with voice control of my automation, but it had to be more seamless and work even when I didn't have my watch on my wrist (raising my wrist to yell at Siri to turn on a light was pretty lame). I saw that someone created a Bridge Application for Amazon Echo and I started looking at the Amazon Echo. The Echo looked very gimmicky with a lot of promise. This bridge, if it worked better than the HomeKit bridge, could be the next step in my home automation. The cost of the Echo was a bit more than I was willing to spend on an experiment, so I put it on my wishlist.

Much to my surprise, one Saturday evening, the Echo arrived (someone bought it for me off my wishlist) and I quickly set it up, told it to discover devices (the Echo Bridge emulates a Hue Bridge), and I was ready to start experimenting. I was turning lights on and off in no time. My son got a bit too excited about the Echo and basically annoyed my wife to no end; the Echo was banished to my office. After a few days, my wife let me put the Echo back in our kitchen (we have a split/tri level house with the kitchen/living area in the middle). My son soon learned to not annoy my wife and I started routinely using the Echo to turn lights on and off. For instance, when I go out back to put compost in the bin, I tell the Echo to turn on the back lights (unfortunately due to some choices I made, the switch for the back lights is not near the door).

So now I had a "toy" to control my automation. In a few short weeks, the "toy" became a tool where I routinely told the Echo to turn off lights that my family had left on around the house. It wasn't until one day my wife walked upstairs, sat on the couch and used the Echo to turn off the downstairs light, that I knew the Echo was a keeper. Since we have an open floor plan with our living room adjacent to our kitchen, the Echo can "hear" us anywhere in our main living quarters which is about 500 sq feet. (Our house isn't all that big and the light switch was literally 5 feet from the couch.) While I initially thought that turning lights on and off was too simplistic, it is really something that is done all the time in a house that can't be automated.

Other than the Echo controlling my automation, I use it occasionally to hear news and weather and we sometimes ask it questions.

The Amazon Echo has tons of potential and if it wasn't for the Echo Bridge, it would be an expensive, seldom used gadget. The bridge makes the Echo an excellent addition to my home; however, the price is a bit much for what it does. Developers are adding more "skills" to it all the time and maybe it will make sense for it to be used by more than just early adopters.

Some people have privacy concerns over an always on microphone. As far as I understand it, the trigger word processing "Alexa" is all done locally and doesn't transmit until the trigger word is heard. To me, this is fine to have the processing done remotely after the trigger word is said; nothing I say to the Echo is all that interesting.

Pros

• Voice recognition works well.
• Microphone picks up voice from across the room.
• Easy setup.
• Developers are extending built in functionality.

Cons

• Limited utility for most people.
• High cost for a gadget.
• May have privacy concerns.
• Doesn't integrate well into iOS ecosystem (tasks and reminders need IFTTT to get to the device).
• Current skills are pretty mundane.

Summary

For me, the Amazon Echo has been an excellent addition to my home automation system. I don't think a day goes by that we don't use it. However, without the bridge to my Vera automation system, the Echo would not be used. The current price is a bit much without a particular use. I wouldn't recommend purchasing the Echo unless you have a particular use in mind and are willing to get used to using your voice to control things. The Echo Bridge is a great piece of software that completes the Echo; while it was designed around the Vera, anything you can control via a URL can be controlled via the Bridge.

At this point, I think I'd feel like one hand was tied behind my back if I didn't have the Echo!

Economics of Water Conservation

Last week I received my latest City of San Diego water bill; we're billed every 2 months. Our bill has a base fee for water, base fee for sewer, a few other fixed costs and then the actual cost of water that, of course, varies based on usage. The problem that I immediately noticed is that water for Tier 1 was $3.9183 per HCF (1 HCF = 748 gallons). We used 6 HCF for a total cost of $23.51 in variable costs; the fixed costs total over $100. Just for reference, the average usage in my area is 13 HCF, so we're doing pretty well in conserving.

While I can completely understand that fixed costs handle infrastructure and personnel costs, the actual water cost doesn't provide any incentive to conserve. One of the fixed fees is adjusted each year based on usage, but it is still fixed for the entire year. If I save 1 HCF of water every 2 months (16% reduction), I'll reduce my bill a whopping $3.92! If on the other hand I double my usage, I'll pay an additional $23.51 which is nothing.

What I propose is that the fee structure change such that each HCF of water costs say $20 and have no fixed costs. Throw in a minimum fee that say includes 4 or 6 HCF of water and now there is incentive for people in my area to reduce usage. At the current rate structure, the average user in my area pays about $155 (I think the base sewer fee is a bit higher, but I'm not sure); under my proposed $20 per HCF, the average user would pay $260. Now, people reading are probably saying that this is outrageous to increase the water rates so much, but think about how this would encourage conservation while still covering fixed costs for infrastructure and personnel.

Of course, my proposed number is simply based on taking my total bill and dividing by the number of HCF used, so it could easily be adjusted. There is probably something I'm missing here as this simple pricing change seems like it would go a long way to encouraging conservation.

Attempting to perfect TV viewing system

As I've written about in the past, I've gone to great lengths to record and play TV shows (I don't torrent and I get my shows over the air). My system up until now has relied on El Gato's EyeTV 3 software running on my Mac Mini combined with a few scripts and now a command line program I wrote called EyeTVExporter to drive ffmpeg. The whole system has worked well, but one problem with EyeTV is that I have to screen share to the Mac Mini to add or modify schedules. Also, this process is far too complex to maintain in the long run. Also, I don't currently have faith that EyeTV will continue to be updated; for instance the latest issue I have with it is that the Export routines (either in the app or via AppleScript) convert all the audio to stereo instead of preserving the AC3 5.1 audio that is recorded on many shows and exists in the MPEG2 file that EyeTV internally stores. When I've tried to just export to MPEG2 with the cut marks which mark the commercials, it has garbled some of the video.

Never content with the status quo, I decided to give MythTV a try this weekend. I had tried to install it before, but the pre-canned installers failed. So this time, I decided to build MythTV using MacPorts using the instructions on the MythTV wiki. Other than a few minor hiccups due to old cruft lying around, the install succeeded after awhile of building. I also installed mythweb to manage the whole thing (I changed the port on the version of Apache installed so I didn't have to futz with the standard OS X Server apache install.)

After the initial configuration and purchasing a 2 month trial of Schedules Direct for the guide data, I was off and running. First step was to record. That went smoothly and the built in commercial skipping seemed adequate for my needs; having it built in instead of using an external component was a big win for me. The next part became trickier and required me to work on a script. I created a user job in the mythbackend that has the following calling parameters:

plexexport %DIR% %FILE% %CHANID% %STARTTIMEUTC% "%TITLE%" %SEASON% %EPISODE%

I then created a script that I placed in /usr/local/bin/plexexport (chmod +x the file after placing it there). Basically the script uses the commercial flags that MythTV inserted, converts those to a cut list and then exports to the appropriate location for Plex to pick up. My ffmpeg settings seem to work well for 1080i and 720p video and preserves the 5.1 audio. I did find that ffmpeg didn't stop when it was done and duplicating my test video for 9 hours of video; using a little script magic and ffprobe, I cleared up that problem by explicitly telling ffmpeg the duration.

With MythTV you can have it run the user job right after the show records, but I want everything to happen at night when I don't have to hear the box if I'm in my office, so I had to create a simple script based on a sample I found that I invoke every night (I use Lingon to set it up). This script tells MythTV to locate all shows that are less than 24 hours old and to queue a commercial flagging job and a Plex Export job. I have MythTV setup to do 1 job at a time so I don't have to worry about the processor getting overloaded.

The last piece in this puzzle is deleting old recordings automatically and yes, I have another script to accomplish that task.

This may sound like it was more complicated than in the past, but I had 4 scripts before and 1 custom program to do the same work. If this all works, my time will have been worth it.

This allows me to use a web browser to schedule shows and easily view upcoming recordings. While the web interface isn't great, it is usable. One caveat is that mucking with some things can have cause lots of problems as the web interface does very little error checking.

Does this save me money? No, Schedules Direct is $25/year while the EyeTV guide data is $20/year, so it is $5 more. However, by undertaking this now, I've proven to myself that when/if EyeTV stops functioning, I have a viable alternative. Hopefully this experiment pans out.