Posts

Posted on

Paperless Extractor

When I was designing ReceiptWallet, I was concerned about the database being corrupted and losing all my data. I decided that as a backup to the database, I’d store all the files as PDFs (later in life it would store other types of files, but for the most part people only stored PDFs) and would embed all the metadata as PDF keywords that could be recovered. The keywords in the PDFs also served as a transport mechanism where you could send a PDF from ReceiptWallet to someone else with ReceiptWallet and when imported, all the metadata would be filled out. In addition to being used as a backup of the data, I envisioned the day when I no longer used ReceiptWallet as I didn’t want to keep supporting it.

While this original idea seemed pretty solid, writing the keywords for the PDFs proved to be somewhat problematic. At the time, Apple’s PDFKit wasn’t completely reliable and could corrupt files when writing them out or crash. Reluctantly I added a switch to ReceiptWallet to turn off this feature (default was to have this on). So depending on the PDF, some PDFs may not have keyword data in them so metadata would basically be lost.

As I wrote earlier this year, that day has come. In my transition away from Paperless (formerly ReceiptWallet), I manually dragged out all files from one of my libraries into the file system and organized them into folders by categories. I placed the folder (named Document Library) in iCloud Drive allowing me to access all the files from anywhere including on my phone. At one point in the past, I started looking at an iPhone version of ReceiptWallet that would sync files; it didn’t go anywhere. Now I finally have that feature!

While I moved a document library over to the file system, I left all my libraries for receipts (one for each year) in Paperless and left them to deal with another day. A few weeks ago I was talking to my father about his transition away from Paperless (he likes to follow my lead sometimes) and I asked him if he had a tool to extract all his files, if he’d use it.

I spent a few hours that day working on a small Mac application that did just that. I had a chance to do a little SwiftUI and Swift concurrency. You drop a Paperless library onto the app’s main window, it walks the library (the library is a bundle which is just a folder which a file extension) looking for every PDF and then opens each file to find the date and merchant. It then creates folders by year and month for the files.

I’ve made the source to this app freely available. It comes with no warranty and I will not make a binary of it available; the tool has worked fairly well for my father and me, but may have some quirks (at one point, extra PDFs without metadata were added to the Paperless libraries).

This app was made without using any source to ReceiptWallet or Paperless; the directory structure of a library can be viewed in the Finder by control clicking the library and choosing Show Package Contents.

I have no information on the future of Paperless and I’m switching away from it as the file system handles my needs better than it did when I wrote ReceiptWallet over 16 years ago.

Posted on

Goodbye old friend, ReceiptWallet

About 10 years ago, I was searching for a receipt and couldn’t find it. I thought there could be a better solution and searched for an application for the Mac that would allow me to store receipts. Unfortunately, at the time I couldn’t find anything so I decided to write ReceiptWallet.

I made many decisions writing ReceiptWallet that turned out to be very forward thinking that would allow me to switch how I tracked receipts in the future. I decided on using PDFs for storing everything (later I allowed for storing zip files and some graphics formats) and storing the files in the file system in a file bundle instead of embedding them in a proprietary database. In addition to storing metadata in a database, I added it to the PDFs that could be extracted without running my software. This was problematic as sometimes the PDFs that were imported either didn’t accept being modified with the metadata or the PDFKit APIs would crash.

ReceiptWallet was quite successful for me, but other things (like being a father) made it hard for me to keep up with it as a side project. So I decided to sell it and it became Paperless. Over the years I’ve done some work to help them with the product.

At the end of last year I was looking at Paperless as I used it daily and started thinking how long it would be around. It hasn’t made it to native Apple Silicon, yet and the user interface is becoming very dated. After chatting with a friend that said he just uses Spotlight to find things, I started looking to see if I could recreate the main features that I used in Paperless in the Finder. I also looked at some other apps that were/are competitors to Paperless and didn’t quite find what I wanted. I used the software for two distinct types of filing; the first was for manuals and documents. The second was for receipts and taxes.

For the first type of filing, I had already started copying certain documents to iCloud Drive such as manuals, certifications, vaccine records, etc. so that I could access them when not at my computer (I had started work on an iPhone version of ReceiptWallet that synced, but didn’t make much progress on it as storing all those documents on the iPhone would quickly eat up the limited storage at the time). I went ahead and made a directory called "Document Library" and then put folders in it. I then transferred all the documents out of my Paperless document library into folders and sub folders in the filing system. It took a few hours, but I managed to classify all my documents. By placing the new "Document Library" on iCloud Drive, I immediately gained mobile access without a special application. I could have used Finder tags, but decided that the directories and searching would be more than adequate. One concern is backing up iCloud Drive as it doesn’t directly appear in the file system. Luckily it is there under ~/Library/Mobile Documents/com~apple~CloudDocs and can be backed up with Carbon Copy Cloner or other tools. Note, however, that if you use tags and backup files to network volumes, CCC by default won’t copy the tags.

Documents Folder

For the second type of filing, receipts, I looked at what I was doing in Paperless. I organized receipts by year and then added categories to them in the interface. In reality, the only reason to add the categories was in case I needed to find a receipt for tax purposes. So I setup a folder called Receipts, created a 2023 folder and then a folder for each month under it. Not strictly necessary, but it is easier for me to open a folder with maybe 25-30 receipts than one with 500. Now all I do is scan or save receipts to the right month and I’m done. I don’t worry about filling in all the fields that I did with Paperless such as category, payment method, or amount. Since the folder is in iCloud Drive, I could just snap a picture of the receipt from my phone, save it and be done. I don’t do that, however, as I like the better quality of a scan, my scanner software does OCR, and I still enter every transaction in Banktivity. I do apply tags to certain transactions that are business or tax related which is kind of handy and the tags are visible on my iPad.

Receipts Folder

Am I going to miss Paperless? It does seem like an end of an era for me, but by simplifying how I handle receipts and documents will hopefully lead to time savings in the future.

Posted on

Money Donation Machine – My son’s Eagle Scout Service Project

[As this article relates to my son’s Eagle Scout service project, I wrote most of it, but he had editorial discretion and added some content in various places. In the article, "I" refers to Scott, and "he" refers to Aiden, my son.]

About 9 years ago, my wife and I decided that we wanted to impart a sense of giving back to our son, so we decided to start volunteering with Feeding San Diego. We’ve been regular volunteers since, and when the opportunity arose to become trained as Team Leaders, we did so and our son, Aiden, at age 10 became the youngest team leader.

When it came time in August 2020 for Aiden to start planning his Eagle Scout service project, he wanted to do something for Feeding San Diego. He brainstormed with their marketing department, and they decided on a cash donation machine similar to ones at science museums and children’s museums, which suck the money up through a series of clear tubes. Aiden went back and forth with them over the course of many months to come up with a design and get it approved. While ironing out the design, he started figuring out the actual mechanics of how it would work. I was as hands off on the project as possible, but gave some suggestions here and there. While this donation machine won’t be handling million dollar donations, it will hopefully encourage people to donate, and be used as a talking piece in their lobby.

Finished project with graphics

He started his concept using an open source project he found, but it didn’t quite meet his needs. The first part was to be able to turn on a vacuum when some type of trigger (more on that later) was activated. I prototyped using a WiFi outlet and a WEMOS D1 Mini, but didn’t tell him how I did it just to give myself confidence that it could be done. It was a pretty simple concept where I setup the WEMOS as a WiFi access point and had the outlet connect to it; when a button was pressed, the WEMOS (basically an Arduino) sent an HTTP command to the outlet running Tasmota firmware. This concept was great, but in the end there was too much wireless interference when Aiden installed the project for it to actually work. Luckily Aiden found a UL listed wired switch called the Dataprobe iBoot-IO. This product was quite expensive, but turned out to be reliable and met our requirement of being UL listed.

After getting the outlet to trigger, the next piece was how should it trigger? Basically, when money is put in the tube, it breaks an infrared beam and sends a signal to the outlet. This was completely out of my scope of knowledge, so Aiden did some research, figured out how to wire an IR sensor and receiver. The schematic is below. With that out of the way, how was it going to be mounted? Aiden has become a pro at using Fusion 360 and designed a ring to house the electronics and fit around the tubing. As part of programming the sensors, Aiden wrote the Arduino code to turn off the outlet after a certain number of seconds to allow the money to go through the series of tubes.

IR sensor

Is the project done, yet? I kept asking myself that numerous times over the 10 months of the project! Feeding San Diego wanted to make the cabinet to house the vacuum, electronics and collect the money, so Aiden designed and provided the dimensions for the box (again using Fusion 360). Unfortunately, the box wasn’t made to those dimensions, but that turned out to be a good thing. The box had a compartment for the vacuum and then a top compartment where the money would go that had a window for people to see. Aiden’s assumption on the box is that the top part would be completely sealed such that the vacuum could be attached to the bottom and there would be enough suction to draw the money through the tubes. I wasn’t, however, convinced of this and suggested that Aiden have a backup plan. I gave him an idea of a flap on the end of the tube that would close when the vacuum was turned on (the vacuum would provide enough suction to close the flap). Unfortunately the vacuum didn’t do that, so Aiden expanded on the idea and designed a flap with a servo such that his program sent a command to the servo to close the flap when the vacuum was turned on and then opened the flap when the vacuum was turned off so that the money would just drop.

Suction flap

With that finished, Aiden was confident that everything would work when installed. A few days before he installed the project, we were able to pick up the box so that Aiden could prepare it by installing thick foam to deaden the sound of the vacuum. When install day came around, Aiden led a number of Scouts and adults from his troop on attaching it to the wall. At the end when it came time to turn on the machine, the wireless interference I mentioned before caused problems with it not being reliable. After trying to troubleshoot it, Aiden finally found the iBoot IO box and ordered it as that would be more reliable and he wouldn’t have to worry about it. Waiting for the part, however, was torturous for him.

iBoot IO Box

Once the part arrived, we went back to Feeding San Diego and he installed it. The electronics worked well and did what it was supposed to do. Unfortunately, there was another hiccup and that was that if the money wasn’t crumpled, it wouldn’t make it through the tubes. Back to Fusion 360 to design a "cap" to make the hole smaller and require people to crumple the money (signage has been added). Was that the final piece in making everything work? No. Turns out the vacuum was a bit underpowered, and the money wouldn’t always make it through even if it was crumpled. More research and a trip to Home Depot for a larger vacuum got him one step closer. Remember I mentioned the larger box before? The new vacuum was larger and some of the insulation he put in had to be cut out to accommodate it.

Was he done after the new vacuum? Well, almost. Aiden didn’t follow the KISS principle in his code which caused some false activations on the vacuum. I encouraged him to remove extra code and just turn on and off the outlet when the sensor was tripped; he had code in there that attempted to detect if someone kept triggering the vacuum and was playing around. That code, unfortunately, was a bit too complex for this application.

Does it work? Yes!

Aiden wanted me to write up his project and I’ve done my best to capture the big pieces.

If you have any questions about this, please don’t hesitate to contact me.

Complete Project Picture

Posted on

Website Monitoring using Home Assistant and Node-RED

Recently I was asked by a Scout unit to try to let them know when an event was available for signup. While there are sites that allow you to do this, I didn’t want to have to signup for another site so I decided to figure out how to get Home Assistant to notify me of changes. I’ve written in the past about using Node-RED with Home Assistant and while Home Assistant has made it easier to create automations without having to use YAML, I still find that Node-RED is a much cleaner and simpler way of creating and viewing automations.

So I started trying to figure out how to get Node-RED to check websites and notify me. Turns out the process was pretty easy.

  1. Add node-red-contrib-object-hash to Node-RED’s configuration.
  2. Tell Node-RED to do an HTTP GET on each site once an hour.
  3. Take the result of the HTTP GET and perform a hash on the results.
  4. Create a Home Assistant sensor using MQTT with the URL and the hash in it.
  5. Wait for a change in the hash and then notify me of the change (I use Slack right now).

It’s a pretty simple slow and the downsides are that there is a slim chance that a change would trigger a hash change (hash collisions are possible, but rare) and it doesn’t tell me what the change actually is.

I’m sure there are other ways to monitor sites, but I don’t have to rely on others for this. While the notification uses Slack, I can switch out the notification to any other method.

Picture of Node-RED Flow

You can download the flow here.

Posted on

Review: Ender 3 Direct Drive Upgrade Kit

I’ve been 3D printing for about 3 years now and anyone that has taken up this hobby knows that there is a lot of fiddling and tweaking to get the kind of prints you want. When I bought my Ender 3 Pro, I knew I wouldn’t be satisfied with the stock printer as I had read about so many nifty little upgrades. Over the years I’ve added things like an EZRStuder, a glass bed, a filament holder with bearings, a Capricon Bowden Tube, etc. With these upgrades, I’ve been pretty successful at printing PETG, PLA, and TPU filament. Since most of my prints these days are functional prints, I really haven’t tuned the printer for the best quality.

This summer I went to print some battery caps for my Ryobi batteries out of Sainsmart TPU like I had done many times and had the hardest time getting it to print well. I replaced the Bowden tube, used a glue stick to get the print to stick, changed settings in Cura, and various other things I could think of really to no avail. I was so frustrated that I ordered an Ender 3 Direct Drive Upgrade Kit and thought that it would solve all my problems. After reading a bit more about this and how it could lead to other problems due to the weight on the gantry, I cancelled my order and just lived with the mediocre prints.

Ryobi Battery Cap Ryobi Battery Cap

A few weeks ago after reading in the Reddit forums, I decided to give the upgrade kit another try. It arrived pretty quickly, but I had to wait to install and set it up. Installation was a breeze, but it turns out that I made enough changes to the printer that the prints (even with regular PLA filament) turned out poorly. I tried to tune my Cura settings, but that didn’t help. I went back and tightened the belts, the eccentric nuts, re-leveled the bed, changed the nozzle, reset the Z probe offset, and finally got some decent prints.

After all that work, I went ahead and started printing various pieces in PLA, TPU 95 and TPU 85. All of the pieces came out good enough for what I needed. I still can fine tune the Cura settings, but the direct drive has allowed me to print faster with fewer problems. Since it is cold in my garage where my printer is, I decided to forego the PEI bed I’ve been using and simply use a glass bed with a glue stick.

The direct drive upgrade has gotten me thinking about an Ender S1 Pro printer. However, it is really hard to justify that kind of money when I don’t print that often. The $30 I spent on the direct drive upgrade was a much better investment because a new printer would bring different issues and would still require me to fine tune various aspects of it.

Installed Direct Drive Installed Direct Drive

Pros

  • Inexpensive upgrade
  • Easy to install
  • Easier to print flexible filaments
  • Eliminates Bowden tube issues

Cons

  • Requires tuning the printer again
  • Extra weight on gantry could cause quality issues with prints
  • Still has a piece of tubing between the extruder and the hotend (could get clogged)

Summary

If you have an Ender 3 printer, this upgrade is a no brainer. Even if you don’t print flexible filaments, it is so much easier to print with PLA when the filament goes directly into the hot end. Of all the upgrades I’ve done to my printer, this may have given me the most bang for the buck. If I had gotten this before, I could have passed on the EZRStruder and the Capricorn tubing.

One caution if you do get this and that is that it may take some time to tune the printer back to the way it was. It took me about 5 hours to get prints working again.

3D Printer 3D Printer

Posted on

Review: LED Headlamp

I’ve written in the past about having a variety of flashlights and when I came across this headlamp, I had to give it a try. I actually stumbled upon it from a spammy ad for a similar product; given that there were a number of products that looked identical, I decided to purchase the cheapest with the idea that they all came from the same factory and I was probably correct.

Lately I’ve been walking the dog when it is dark (I’ve run out of time during the day and the dog needs a walk), so I was interested in what would make it easier to see at night. I found that other headlamps were uncomfortable and were very directional with a very narrow beam. While some people swear by headlamps for camping and other activities, I’ve avoided them as much as possible as I found a flashlight to be easier to light my way.

This headlamp differs from other headlamps in that the main light is a strip of LEDs across the front and weighs significantly less than other lights. I’ve used it on a few walks and was amazed at how bright it was and how much it lit the path. There is a gimmicky feature on it that lets you wave your hand on the side of the light to turn it on and off (kind of like wax on/wax off). Turns out it is actually useful! My walk isn’t in the complete dark, so I was easily able to turn the light off when not needed. However, the sensor is quite sensitive so if you’re using it in an enclosed space, it will likely turn on and off as you move.

I haven’t had a chance to see how long it lasts, so I can’t confirm the runtime. It does charge pretty quickly over USB-C. Well, sort of. I was intrigued about USB-C charging so I connected a USB-C to USB-C cable to 2 different chargers and it didn’t charge. The headlamp uses a USB-C connector, but doesn’t actually use USB-C (it comes with a USB-A to USB-C cable for charging). That’s a bit disappointing as I have to bring a different cable if I go camping.

Pros

  • Light weight
  • Wide beam

Cons

  • Doesn’t use real USB-C charging
  • Hand waving sensor is gimmicky

Summary

This inexpensive headlamp is well worth the $10 I paid for it. While I hope that it lasts, I have no idea the longevity of it. I’m excited to take it on my next camping trip. While I would have designed it differently by removing the side sensor, putting the battery in the back and getting rid of the light on the side, it is good enough for my needs.

Posted on

Relying on the cloud for home automation and monitoring

When I started getting into home automation 9 years ago, I decided to go with Z-Wave devices as it had the largest range of devices I could integrate into a system. Z-Wave communicates locally with a hub for automation and doesn’t rely on the cloud to work. At the time I made this choice, there weren’t many options let alone ones that relied on the cloud or apps to work. Fast forward a few years and there are a number of proprietary systems and apps that won’t work without a connection to the cloud. This, of course, causes problems when the Internet is down, the provider is down, or the provider has gone out of business. Luckily my setups have never relied on the cloud (they have had some integrations for notifications and such, but the basic automations have been local).

Part of my home automation is monitoring power usage as well as solar production. When I had my SunPower system installed two years ago, I had researched how to monitor the system locally. I’ve written about my system and how I monitor it. While SunPower has their own app, I’ve never relied on it because it isn’t integrated into the rest of my home automation setup. I was told by the installer that if there was ever a problem, they would be notified and would give me a call to come out and take a look. That shouldn’t great, but I never really trusted that would happen as one of the panels wasn’t activated when the system was turned on and because of my own monitoring, I alerted them to come out. I have no idea if they would have called me to fix the problem or I would have been out 1 panel of production.

If you follow the discussion of my article, you’ll see that people have been reporting that the SunPower app isn’t showing any data. I hadn’t noticed, but checked and saw that the app hasn’t recorded data for the last 1.5 months! Ouch. I don’t know how SunPower or the installer monitors the system when it hasn’t reported in, but I would have expected a call to fix the system. Luckily I don’t rely on the app to see production. My nice dashboard in Home Assistant tells me production and usage without talking to the cloud. If my system had failed 1.5 months ago, it would cost me a significant amount of money and I would have been quite annoyed that SunPower didn’t alert me.

I just used the app to change the device to connect via WiFi (it is still connected via wired Ethernet) and now the device says it is online. Will it work tomorrow when the sun is up? I don’t know. Do I care? Not really as long as my non-cloud based system keeps working.

Side note

This year we may have turned a corner with relying on the cloud for various home automations. The Matter standard aims to join various systems and have local control over devices. Will this work out? I can only hope.

Posted on

Eating my own words (again)

I’ve spent a little less than half my career self employed and the other half working for companies, so I’d like to think I know something about the pros and cons of both types of work. In the last two companies I worked, I said at the end of my tenure that I’d never work for anyone again and would always be self employed. That didn’t work out for a number of reasons causing me to eat my words.

Being self employed sounds great as I didn’t have a boss, no reviews, I could work my own schedule, and no one was looking over my shoulder. That was only partly true; as a contractor I always had to work on someone else’s schedule and meet deadlines. In addition, I was sometimes the only person doing my type of work meaning I had to handle all the support, deal with all the issues, and sort of be on call all the time. Also any time I took off, I didn’t get paid for meaning that it could be very costly (depending on the contract I could get paid for some vacation, but I always felt guilty about taking time off). Being self employed also means sending out invoices, hoping you get paid on time, stressing over raising rates, and crossing your fingers that your contract isn’t terminated without notice.

One of the things that I missed when I was self employed last was working with other developers doing the same type of work, i.e. iOS or Mac development. This made it hard to learn new things as there was no one around to push me or show me better ways. No one was looking over my code and since I’m not perfect, I likely kept making the same mistakes over and over again.

This past April I started a new job and now work on a team with another 14 iOS developers. I pretty much learn something new everyday and know that I’m not alone when it comes to figuring out problems or tackling the work. Of course, I do my best at work, but it is much easier to leave work at the end of the day.

As much as being self employed sounded great, this change has been good to me and has lowered my stress. It has also given me opportunities to learn new skills and hone my existing ones.

Those words about never working for someone else sure tasted good!

Posted on

Fixing Battery Drain in Kwikset Locks

Almost 10 years ago I installed Kwikset Z-Wave locks on our doors. They had been working well for years, but I noticed that the batteries drained faster than I had expected. I attributed this to buying refurbished or b stock locks and just tried to get batteries that lasted longer. Last December I decided to replace the locks with new ones that hopefully drained the batteries slower.

The locks appeared to drain slower initially, but I wasn’t get the year out of a set of batteries that I had read others could get; I could get a few months. After one of the updates to Home Assistant, I noticed in the Logbook that it recorded Z-Wave commands. In the list of commands were a bunch of "lock" commands sent to one of the locks; about once every 30 seconds at night.

Ah ha! It finally dawned on me. In my Node-RED automations, I had it setup to lock the garage door lock at night if the door was closed. This was repeated every 30 seconds. I had thought that if the lock was locked, nothing would happen. That was not the case as Home Assistant happily sent the command causing the lock to wake up every 30 seconds. The fix was quite simple: before sending the lock command check the state of the lock as Home Assistant stored this without having to poll the lock. With this simple change I stopped seeing the Z-Wave lock command and knock on wood, the batteries are lasting longer!

Posted on

Review: Ryobi Hobby Station (and a missed opportunity)

I love having the right tools for the job whether it is software tools or physical tools to tackle projects around the house. As I’ve mentioned before, I really like the Ryobi 18V One+ line of tools. I have a number of the tools and keep adding to my collection. While the tools may not be contractor grade, they are more than enough for my uses.

Last year they announced a number of new tools in their collection including a new Power Source that can charge batteries from USB-C as well as power USB-C devices. Unfortunately the product isn’t available and I check Ryobi’s website every few days to see if the "Coming Soon" button changes to "Buy Now".

A few weeks ago when I was checking the site, I noticed the Hobby Station which takes on of Ryobi’s rotary tools and turns it into a drill press and holds the tool making it easier to use. This was a little expensive for what it was, but I decided to order it. Even though I don’t own any of the rotary tools that Ryobi says works with it, I though that I might be able to use my P460 Rotary Tool Station with it if I screwed off the top housing. What I didn’t realize at the time is that the diameter of that housing and the diameter of the tools that are designed to work with the station are different.

The Hobby Station arrived and I quickly set it up. When I put the tool in the holder, I found my mistake where the tool was too small. However, that didn’t deter me. I screwed the tool in place and it gripped the rubber and seemed fairly secure. Perfect, I didn’t have to return it!

Hobby Station
Hobby Station with rotary tool

Later I thought about the main nut holding the tool in place digging into the rubber and didn’t like it. So I looked for a reducer bushing and thought I found an irrigation one at Home Depot, but realized that the threads probably wouldn’t match. Not deterred by this, I asked my son if he could design a part and I could 3D print it. After a number of test prints, we had a working bushing! While it isn’t an exact fit, the plastic is soft enough that the threads of the tools can dig into it and provide a fairly tight fit.

Reducer Bushing

Now that I had a working hobby station, what do I think? For starters, the drill press has limited utility. With my solution, there is a bit of wiggle in the tool and frankly, how often have I needed a drill press? The station does hold the tool and make it easier for me to just hold the working piece. The angle adjustment is somewhat useful as I can see myself using it facing straight down or at a 90 degree angle. I don’t know how much I’ll use the fence that comes with it or use the vacuum attachment.

Rotary tool in hobby station

Pros

  • Convenient way to hold a rotary tool
  • Flexible angles to hold tool
  • Appears well made

Cons

  • Only works with some rotary tools
  • Drill press has limited utility
  • Over priced

Summary
This tool is definitely over priced and has limited utility for many people. It isn’t a precision instrument which is needed for a drill press. It does seem like it is built solidly and will work for my needs; holding the rotary tool while I work on something is very useful for me.

Unfortunately I can’t recommend this to others; I will keep mine, but there are probably better solutions out there. If Ryobi had included a reducer bushing to fit their other rotary tools, it may have been a more compelling purchase and would definitely increase potential buyers.

Proudly powered by WordPress