Wireless Link Becoming Limiting Factor

I’ve been experimenting with WiFi access points in the past few weeks and have tested the performance of the wireless link. The max speed of 802.11 n (most of my devices are n devices with a few ac and even a g device or two) is 300 Mbps or 450 Mbps depending on the number of antennas. That may sound fast, but actual performance is a lot less than that and in my testing, I was able to get about 225 Mbps using Iperf. Usually I can get between 50 and 150 Mbps. That should be plenty fast enough to max out many Internet connections; I currently pay for 50 Mbps down/5 Mbps up.

At the end of November, Time Warner Cable should be rolling out its MAXX service here in San Diego and will have a maximum speed of 300 Mbps down/20 Mbps up. I’ll be opting for 200/20 which will lower my bill each month at the same time quadrupling my Internet speed. So where does that put me? That would put my wireless link at about the same speed as my Internet service! I really didn’t think this day would come where the airlink can’t keep up with my Internet. Of course all these speeds are theoretical and performance will varying depending on conditions, but it now becomes even more important to tune my WiFi network in order to get the most performance out of it otherwise I could be paying for Internet speeds that I can’t use.

Determination or insanity?

For the last week I’ve been tracking down a bug in my current project. I’ve spent just about every spare minute including the weekend trying to figure out why my code works in a test app, but not the main app. I rewrote a huge piece of some communications code to no avail. I tried threading the communications, putting in proper locks, changed how I processed the bytes, etc. Basically when I was receiving large amounts of data from a Bluetooth device, some data was lost in the transmission and the checksum was incorrect; for small amounts of data, this problem didn’t show up. At this moment in time, I believe I’m the only person in the world working with this combination of hardware and software which made using Stackoverflow useless.

A co-worker of mine kept sending me ideas to try as I was completely out of ideas. His latest idea was to put the files from my test app into the main app to make sure the project settings weren’t the problem. That didn’t solve the problem, so I started ripping out pieces of the code and came across some code we had put in for a demo.

All of the communications code that I was working on was using the External Accessory protocol over Bluetooth. The demo code we had in there was using Multipeer Connectivity which uses WiFi and Bluetooth to discover peers. Once I removed that code, all of my communication issues went away! Now I should file a Radar issue with Apple on issue, but I can’t reproduce it outside of the particular hardware I’m using, networking library I’ve written, and a few other pieces so a bug report would be quite lacking.

I must have spent 40+ hours on this one issue which has caused me to lose sleep and be a bit on edge all week. As failure was not an option and I had no place to turn, I had to solve this. I’m sure that others would have given up as 40+ hours is far too long to spend on a bug. For me, however, I actually spent 4 months full time chasing down a bug in a network router for a cellular base station; like this one, I did eventually discover the fix.

Lesson learned…be careful of using different communication libraries in an app.

Troubleshooting OS X Server’s Websites

Today I was trying to install a program on my OS X Server and the software complained that something was listening on port 8080. I couldn’t figure it out, so after a bit of searching I found a command that could get me closer:

sudo lsof -iTCP -sTCP:LISTEN -P -n | grep 8080

The output had in it:

httpd     21950        _www    7u  IPv6 0xc086bca85e9b29f7      0t0  TCP *:8080 (LISTEN)
httpd     65328        _www    7u  IPv6 0xc086bca85e9b29f7      0t0  TCP *:8080 (LISTEN)

OK, so now I know that the _www user was running the process. Next up was to do:

ps awuwx | grep _www

Which gave me:

_www            23128   0.0  0.0  2518828   1132   ??  S     8:41PM   0:00.00 /usr/sbin/httpd -D FOREGROUND -f /Library/Server/Web/Config/apache2/httpd_server_app.conf

Ah, this tells me that OS X’s Server App is running Apache which is fine, but I had Websites turned off in OS X Server!

Screen Shot 2015 09 01 at 8 33 06 PM

Hmmm, how do I disable Apache without hacking away on the http files? Well, I noticed on the Websites section I had an entry listening on port 8080. I removed the entry and presto, my server stopped listening on port 8080. Very odd because Websites is off, but OS X Server decided that if Websites is off, it should still listen and serve up a page that says “Websites are turned off.”

Screen Shot 2015 09 01 at 8 46 55 PM

Besides Apple ending the text with a preposition, this is absolutely the wrong behavior in my opinion. I understand that certain services (CalDAV, WebDAV, etc.) run through Apache and OS X Server has to have it on to work, but Apple needs to have better messaging to indicate that it is listening on various ports and how to turn off the websites.

Maybe I’m not the average OS X Server user in that I run various other services including Jenkins, but OS X Server still is not up to snuff; I’m not sure why I keep running it. OK, I do know why I keep running it; the caching server is excellent, VPN service is good, and file sharing is pretty easy to setup. Good thing I’m not afraid of the command line and can figure this out.

Setting up QoS on the Edge Router Lite

When I started using the EdgeRouter Lite, I knew that it was quite powerful and could handle pretty much anything I threw at it. Last week I had to send a very large file to a co-worker, so I put it in Dropbox and soon saw Dropbox use pretty much 100% of my upstream bandwidth; I tried limiting the upstream bandwidth in Dropbox, but the setting failed to do anything. Someone tried to call us on our Ooma and I had trouble with the call, so I stopped the Dropbox upload and turned on the Network Link Conditioner on my Mac to limit the upload.

That evening I started playing with Quality of Service (QoS) on the EdgeRouter Lite to see what I could do to prioritize the voice traffic and limit bandwidth while making calls. After much research over about a week, I finally managed to come up with a configuration that seems to work. There are lots of guides out there, but I didn’t quite understand them, so I’ll do what I can to explain what I’ve done here. If there are any errors or tips, feel free to let me know.

First up, I setup a DOWNLOAD traffic-policy to handle downstream bandwidth. I’m not sure that this is truly necessary as I’ve been unable to continuously max out my downstream connection and when Time Warner Cable upgrade to MAXX this November, it will become less of an issue.

I SSH’d into the box and did the following:

configure

set traffic-policy shaper DOWNLOAD bandwidth 60mbit

DOWNLOAD is an arbitrary label. 60mbit is my total downstream bandwidth.

set traffic policy shaper DOWNLOAD class 10 bandwidth 25%
set traffic policy shaper DOWNLOAD class 10 burst 2.5k
set traffic policy shaper DOWNLOAD class 10 ceiling 50%
set traffic policy shaper DOWNLOAD class 10 description RTP
set traffic policy shaper DOWNLOAD class 10 match VOIP-RTP ip dscp 46
set traffic policy shaper DOWNLOAD class 10 match VOIP-RTP-ipv6 ipv6 dscp 46
set traffic policy shaper DOWNLOAD class 10 priority 7
set traffic policy shaper DOWNLOAD class 10 queue-type fair-queue

This sets up a “rule” given this particular traffic a minimum of 25% of the total bandwidth and caps it at 50% of the total bandwidth. It matches on the VOIP RTP packet header. The docs say that the priority is 0 to 7 where 7 is the highest priority. Note that the docs say that for default, 7 is the lowest priority.

set traffic policy shaper DOWNLOAD class 20 bandwidth 5%
set traffic policy shaper DOWNLOAD class 20 burst 2.5k
set traffic policy shaper DOWNLOAD class 20 ceiling 50%
set traffic policy shaper DOWNLOAD class 20 description RTP
set traffic policy shaper DOWNLOAD class 20 match VOIP-SIP ip dscp 24
set traffic policy shaper DOWNLOAD class 20 match VOIP-SIP-ipv6 ipv6 dscp 24
set traffic policy shaper DOWNLOAD class 20 priority 7
set traffic policy shaper DOWNLOAD class 20 queue-type fair-queue

The bandwidth values are pretty arbitrary as VOIP should never use that much bandwidth on my network.

set traffic policy shaper DOWNLOAD class 30 bandwidth 10%
set traffic policy shaper DOWNLOAD class 30 burst 2.5k
set traffic policy shaper DOWNLOAD class 30 ceiling 20%
set traffic policy shaper DOWNLOAD class 30 description OOMA
set traffic policy shaper DOWNLOAD class 30 match OOMA-ADDRESS ip destination 10.0.1.102/32
set traffic policy shaper DOWNLOAD class 30 match OOMA-MAC there destination 00:00:00:00:00:00
set traffic policy shaper DOWNLOAD class 30 priority 7
set traffic policy shaper DOWNLOAD class 30 queue-type fair-queue

This rules gives my Ooma box guaranteed bandwidth; the matching is an ANY so either match will work. I really don’t need both matches, but I put it here to try things out.

set traffic policy shaper DOWNLOAD default bandwidth 65%
set traffic policy shaper DOWNLOAD default burst 15k
set traffic policy shaper DOWNLOAD default ceiling 100%
set traffic policy shaper DOWNLOAD default priority 7
set traffic policy shaper DOWNLOAD default queue-type fair-queue

This is for any traffic that doesn’t match the other rules. Note that the total bandwidth (not ceiling) should add up to 100%.

Something similar for UPLOAD:

set traffic-policy shaper UPLOAD bandwidth 6mbit
set traffic policy shaper UPLOAD class 10 bandwidth 25%
set traffic policy shaper UPLOAD class 10 burst 2.5k
set traffic policy shaper UPLOAD class 10 ceiling 50%
set traffic policy shaper UPLOAD class 10 description RTP
set traffic policy shaper UPLOAD class 10 match VOIP-RTP ip dscp 46
set traffic policy shaper UPLOAD class 10 match VOIP-RTP-ipv6 ipv6 dscp 46
set traffic policy shaper UPLOAD class 10 priority 7
set traffic policy shaper UPLOAD class 10 queue-type fair-queue
set traffic policy shaper UPLOAD class 20 bandwidth 5%
set traffic policy shaper UPLOAD class 20 burst 2.5k
set traffic policy shaper UPLOAD class 20 ceiling 50%
set traffic policy shaper UPLOAD class 20 description RTP
set traffic policy shaper UPLOAD class 20 match VOIP-SIP ip dscp 24
set traffic policy shaper UPLOAD class 20 match VOIP-SIP-ipv6 ipv6 dscp 24
set traffic policy shaper UPLOAD class 20 priority 7
set traffic policy shaper UPLOAD class 20 queue-type fair-queue
set traffic policy shaper UPLOAD class 30 bandwidth 10%
set traffic policy shaper UPLOAD class 30 burst 2.5k
set traffic policy shaper UPLOAD class 30 ceiling 20%
set traffic policy shaper UPLOAD class 30 description OOMA
set traffic policy shaper UPLOAD class 30 match OOMA-ADDRESS ip source 10.0.1.102/32
set traffic policy shaper UPLOAD class 30 match OOMA-MAC there source 00:00:00:00:00:00
set traffic policy shaper UPLOAD class 30 priority 7
set traffic policy shaper UPLOAD class 30 queue-type fair-queue
set traffic policy shaper UPLOAD default bandwidth 60%
set traffic policy shaper UPLOAD default burst 15k
set traffic policy shaper UPLOAD default ceiling 100%
set traffic policy shaper UPLOAD default priority 7
set traffic policy shaper UPLOAD default queue-type fair-queue

set interfaces ethernet eth1 redirect ifb1
set interfaces ethernet eth1 traffic-policy out DOWNLOAD
set interfaces input ifb1 traffic-policy out UPLOAD

commit
save

The last bit above tells eth1 (my LAN interface) to use the download policy (out means leaving the box). In order to get the upload to work, I’ve told the interface to redirect the traffic to the “input” interface and then applied the QoS policy.

One drawback to QoS is that with the 1.7 firmware, traffic analysis doesn’t work. Ubiquiti has said that they may add traffic analysis to QoS packets. In addition, the release notes indicate that at higher bandwidths, QoS bogs down the box and limits the bandwidth. So, when my cable modem gets upgraded, I may turn off the DOWNLOAD policy, but can still keep the UPLOAD policy which is where most of the problem lies.

After setting all of this up, I was able to run a download test and still maintain a voice call, so I succeeded in my mission. I’m sure something isn’t quite right and I’ll be tweaking this again when iOS 9 brings WiFi calling as I’m not sure that my policies will match the correct traffic.

Putting this all together, I got the following configuration:

traffic-policy {
    shaper DOWNLOAD {
        bandwidth 60mbit
        class 10 {
            bandwidth 25%
            burst 2.5k
            ceiling 50%
            description RTP
            match VOIP-RTP {
                ip {
                    dscp 46
                }
            }
            match VOIP-RTP-IPv6 {
                ipv6 {
                    dscp 46
                }
            }
            priority 7
            queue-type fair-queue
        }
        class 20 {
            bandwidth 5%
            burst 2.5k
            ceiling 50%
            description SIP
            match VOIP-SIP {
                ip {
                    dscp 24
                }
            }
            match VOIP-SIP-IPv6 {
                ipv6 {
                    dscp 24
                }
            }
            priority 7
            queue-type fair-queue
        }
        class 30 {
            bandwidth 5%
            burst 2.5k
            ceiling 10%
            description OOMA
            match OOMA-ADDRESS {
                ip {
                    destination {
                        address 10.0.1.102/32
                    }
                }
            }
            match OOMA-MAC {
                ether {
                    destination 00:00:00:00:00:00
                }
            }
            priority 7
            queue-type fair-queue
        }
        default {
            bandwidth 65%
            burst 15k
            ceiling 100%
            priority 7
            queue-type fair-queue
        }
    }
    shaper UPLOAD {
        bandwidth 6mbit
        class 10 {
            bandwidth 25%
            burst 2.5k
            ceiling 50%
            description RTP
            match VOIP-RTP {
                ip {
                    dscp 46
                }
            }
            match VOIP-RTP-IPv6 {
                ipv6 {
                    dscp 46
                }
            }
            priority 7
            queue-type fair-queue
        }
        class 20 {
            bandwidth 5%
            burst 2.5k
            ceiling 50%
            description SIP
            match VOIP-SIP {
                ip {
                    dscp 24
                }
            }
            match VOIP-SIP-IPv6 {
                ipv6 {
                    dscp 24
                }
            }
            priority 7
            queue-type fair-queue
        }
        class 30 {
            bandwidth 10%
            burst 2.5k
            ceiling 20%
            description OOMA
            match OOMA-ADDRESS {
                ip {
                    source {
                        address 10.0.1.102/32
                    }
                }
            }
            match OOMA-MAC {
                ether {
                    source 00:00:00:00:00:00
                }
            }
            priority 7
            queue-type fair-queue
        }
        default {
            bandwidth 60%
            burst 15k
            ceiling 100%
            priority 7
            queue-type fair-queue
        }
    }
}
interfaces {
    ethernet eth1 {
        address 10.0.1.1/24
        description Local
        duplex auto
        redirect ifb1
        speed auto
        traffic-policy {
            out DOWNLOAD
        }
    }
    input ifb1 {
        traffic-policy {
            out UPLOAD
        }
    }
}

A simple solution to poorly working remotes

When we remodeled our house, I ran HDMI cable from the TV through the wall to my equipment rack which is on the other side of the wall. At the time, we were using an Apple TV, so we were just using an iPhone over WiFi controlling the Apple TV. This worked well until I added a Roku 3 and a Fire TV to the mix. The Roku 3 uses a WiFi Direct remote and the Fire TV uses a Bluetooth remote.

In theory, the remotes should have no problem reaching about 10 feet through one wall. In practice, the remotes have been flakey. For the longest time, I’ve been blaming batteries and the remotes. I got so fed up last week that I decided to take another look. My server rack is in the lower level of our house whereas the TV is in the middle level (it’s a split level house) and the Roku and Fire TV have both been in the rack; this puts them slightly below the floor level of the middle level. This isn’t the best picture, but it will suffice.

Rack

I put zip ties around the Fire TV and Roku TV and mounted them on the wall next to the equipment rack; I put them as high as possible given the Ethernet cable I had which situated them about a foot above the rack. Moving the boxes out of the rack and raising them up slightly seems to have done wonders to the remotes. They are now very responsive and I haven’t reset the Roku remote in a week and haven’t banged them against the couch to get them to work!

While the solution is now obvious, I was so concerned about making my equipment rack nice and neat that I chose to ignore simple troubleshooting steps dealing with RF. Some people may be asking why I don’t just stick the boxes next to the TV. For starters, I don’t want to see the boxes as they would add clutter to our TV area. Second, I have the wall warts plugged into a power strip that turns off at night to conserve energy (I plug as much as possible into this strip and it automatically shuts down at around 11 pm and turns on around 6 am).

Knock on wood that things keep working; if I have problems again, I’ll get some longer Ethernet cables and move the boxes so that they’re higher up and on the backside of the TV wall.

Following directions (or how to fix a clogged dishwasher)

Ever since I can remember, I’ve used liquid or gel dishwashing liquid in my dishwashers. For some reason I thought that the liquid would dissolve better and would wash better. When we got our new dishwasher (a Bosch) 2 years ago, I didn’t really read the instruction manual and kept using liquid detergent (I do most of the dishes in our house as my wife does the cooking). Since getting the dishwasher, we’ve periodically had it give us an error (E22) and fail to completely drain.

I read about the error online and people seemed to indicate there was a problem with how the drain hose was hooked up. Ours was hooked up to an air gap and I adjusted the hoses several times which fixed the problem temporarily. After the last time I got the error, I decided to read the manual. The manual recommended using Finish Powerball Tablets. I thought this was a bit wasteful, but it sure looked convenient. I went to Costco, bought a box of these and started using them.

Since I started using them (knock on wood), we haven’t had a problem with the dishwasher. My theory is that the gel we were using didn’t completely dissolve and clogged up part of the drain or hose enough that the dishwasher thought it was completely clogged and shut off the cycle (the error always occurred at the end of the wash).

So the lesson here is “read the instructions”!

The end of my vacation rental nightmare

For the last 9 months or so, the property next to me has been used as a mini-hotel; some people call would call it a short term vacation rental, I simply call it a pain in my rear. The City of San Diego hasn’t budged on its misinterpretation of the municipal code (§131.0422) that prohibits visitor accommodations in residential zones and the owner has said that he can do whatever he wanted to with this property.

I’ve spent a lot of my free time amounting to hundreds of hours researching laws and anything I could find about short term vacation rentals. I looked for anything that would be on my side and uncovered trash regulations, but those weren’t enough to get the City to do anything. I like watching law dramas where lawyers seems to pull rabbits out of their hats to win cases; I was looking for my rabbit.

When we purchased our house, I knew that there were CC&Rs (covenants, conditions, and restrictions) relating to my property. Our neighborhood was developed in the 1960s and a homeowners association was no longer in existence (if one did exist). I read the CC&Rs many times looking for something that would give me legal recourse against the property owner next to me (neighbors are people that live near you and don’t just own property near you). It wasn’t until I had read the CC&Rs at least 10 times that I stumbled upon a few clauses that looked like they could apply. In addition, the CC&Rs appeared to automatically renew every 10 years and only expired if a majority of owners voted to terminate them.

I started talking to one of my neighbors that is an attorney that deals with real estate law and he seemed to think that while we could go after the City to enforce the code, my idea about using the CC&Rs could work. As much as I want to help other people, my primary goal was to get rid of the vacation rental next to me. I retained my neighbor as my attorney and he said the first step was to gather support from other neighbors; I got several of my neighbors to join and my attorney drafted a letter to the property owner claiming he violated the CC&Rs on a number of grounds.

The owner next door hired an attorney to represent him and basically wanted to settle without going to mediation or court. Without acknowledging that all of our claims were valid, he agreed to stop renting out his property as a vacation rental. Wow, I was stunned when my attorney told me this! Over the course of the next few weeks, we went back and forth hammering out the details of an agreement. As of now, the current owner is no longer going to rent it out for less than 30 days.

While I’ve managed to rid my neighborhood of this annoyance, my approach probably won’t work for many others. I had CC&Rs that were still enforceable, I had the determination to fight, and I was able to afford an attorney to represent me. I’m not an attorney, so if anyone else wants to pursue an action to rid their neighborhood of a vacation rental, please contact an attorney (I now have several on speed dial :-)).

My time with the Apple Watch

More than 3 years ago, I backed the Pebble watch on Kickstarter. My needs for the watch were quite simple; I wanted to be able to tell the time (the summer before that, the battery on my watch died twice and I got tired of replacing it), I wanted to get text message notifications on it and I wanted to be notified when the phone rang. The last 2 were important as I don’t always feel my phone vibrate when it is in my pocket. About 2.5 years ago, my Pebble arrived. Initially it didn’t do all that I wanted (from the list above), but with software updates and iOS updates, it completely met my needs.

Pebble added apps and I played with a few of them, but didn’t really use them. I’m likely an atypical user as I’m at my computer most of the day and when I’m away from my desk, I don’t spend a lot of time playing on my watch or phone (I don’t commute anywhere).

My only real complaint with the Pebble is that it got confused often and didn’t always give me notifications.

When Apple announced the Apple Watch, my main reason for wanting one was better integration with the iPhone. About a month and a half ago, mine arrived (black sport). Since then I’ve worn it everyday and have been pleased with it for my use. Like the Pebble, I use it for notifications and telling the time. In addition, the watch face I use shows upcoming appointments; that is very useful.

I’ve been quite pleased with Siri on it as I tell Siri to add things to my grocery list and set timers. I haven’t used it for Apple Pay, yet, but hope that more stores accept Apple Pay so I can use it. I don’t use it while running as I’m not sure I’d find it all that comfortable with all my sweat; also I don’t know how well I can get the sweat out of the watch.

In my use, the battery life is excellent (I charge it nightly and it usually has 60-70% battery remaining). I don’t really use the apps, but maybe with watchOS 2.0, that will change. For notifications, the watch performs well. As a watch, it is kind of mediocre; having to lift my wrist is strange and I’d like to just be able to glance at it and see the time. However, having to lift my wrist makes me check the time less often which may be a good thing.

Is the watch for everyone? Of course not. I love technology and the Apple Watch does everything I expected it to do and then some (like Siri), so I guess it was worth it. I knew what it was going to do before getting it and it performs those functions adequately. I know that there are people that hate the Apple Watch and those that love them. Will it be a mainstream product? I kind of doubt it, but that doesn’t matter to me as it does what I need.

Surge Protector on Coax – Bad idea?

It seems that most surge protectors that are a bit more expensive than the run of the mill ones have ports for coax, Ethernet and phone lines to provide surge protection for these items. I’ve always ignored the Ethernet and phone line ones as I use VOIP and it is unlikely that the VOIP box would generate a surge and same goes for Ethernet. However, when I setup my UPS almost 2 years ago, I figured that connecting the coax for the cable modem into the coax on the UPS would be a good idea in case a surge came in through the cable line, it wouldn’t blow up my cable modem.

As I’ve written about in the last few months, I’ve had some problems with my Internet cutting in and out sometimes and I basically came to the conclusion that it was the router. However, at the same time I replaced the router, I came across a post indicating that the coax surge suppressor will cause signal loss (any coax connection likely does this) and that depending on the suppressor, it might not pass all the frequencies needed for the cable modem. (From what I understand, the newer cable equipment can use higher frequencies. I don’t know enough about this to verify what I’ve read, unfortunately.)

I decided to just connect the coax directly to the cable modem (the coax into my cable modem has no splitters from the street; just a few connections would shouldn’t cause signal loss) and see what happened. Combined with my router change, my cable modem connection has been rock solid. Is all this a coincidence? Who knows, but the signal loss issue is quite possible (the cable tech said that the loss was acceptable, however) and what do I have to lose? If there is a surge through the cable line, I’ll lose the cable modem.

Food for thought if you’re having problems with your cable modem.

Air conditioning in coastal San Diego?

I’ve lived in a few places in San Diego over the last 20 years; 4 relatively close to the coast and 3 further inland. In that time, I had air conditioning in the places inland, but rarely used it. In fact, when we lived 20 minutes more inland than we do now, I don’t recall turning on our A/C for the first few years we lived there; we used ceiling fans and just dealt with the warmer days. So when we bought our house and remodeled it, adding A/C didn’t even cross my mind. We’re pretty close to the coast, and I figured we’d get a coastal breeze. Within the first year of living here, I knew that this was a bad decision. It turns out that last year was the hottest year on record in San Diego with 12 days over 90 degrees at the airport whereas there are normally 1-2 days over 90 degrees. I thought maybe I had become a wimp and couldn’t take the heat (well, that still may be true), but the facts confirmed that I wasn’t completely crazy.

So, at the beginning of this year, I had air conditioning installed. It was pretty painless as we already had duct work, a thermostat and a place to put a compressor. Since the installation, we’ve turned it on 3 times. Twice last week because the humidity was so high and one of the days, it rained which required us to close our windows (it was well over 80 degrees outside so closing up the house rapidly increased the house temperature). I really hate to use air conditioning as it is a dry air and it isn’t cheap, but with outside temperatures increasing, this was a wise decision. We still use ceiling fans as much as possible, but knowing that we have this backup just makes our house even more comfortable.