Eating my words about OCR

I’ve said in the past that I thought OCR was pretty ridiculous for receipts as it isn’t 100% accurate and receipts have so many wacky formats. Well, after a lot of work and evaluation, I have managed to integrate the Tesseract OCR engine into ReceiptWallet. It is an open source engine that has OK results for the price. (I did investigate some commercial engines; there are 2 that are on the Mac, and the costs were outrageous and currently one doesn’t even work on PowerPC machines.) As Google continues to work on the engine, I’ll be integrating the new versions into ReceiptWallet.

So, how did I manage to get things to work and get OK results? How about I just say magic? :-). Turns out the hardest part is attempting to recognize merchant names; I cheated on this and simply recognize merchants that the user has already used. This works quite well as I find that I keep going to the same merchants, so while it won’t recognize a merchant the first time you visit it, it has half a chance on subsequent visits.

I’ve been using my ScanSnap for testing and it produces very clear images that the OCR engine seems to like. My DocketPORT, however, has had mixed results. I think I need to change the contrast/brightness settings for better results. I’ll leave this up to beta testers to try out.

My goal is to push out a beta this weekend and see what happens. There is no way that this will be 100% accurate, but it might help a little and then I get to say that it has OCR!

iPhone not syncing on dock (Solution)

I noticed that my iPhone didn’t launch iTunes when I dropped it in the dock even though I specified automatic sync. It dawned on me that in the past I saw an iTunes Helper app as a Login Item. Sure enough, this wasn’t in my Login Items. I dug into iTunes.app/Resources, launched iTunes Helper and also dragged it to Login Items. Now when I dock my iPhone, it syncs automatically. You’d think that Apple would make sure that the iTunes helper is always running, say on each launch of iTunes. Anyway, I have a solution to my problem.

Buggy QuickBooks

You’d think that an accounting package would be 100% reliable or as close as possible to that as people rely on them all the time. I just happened to be looking at some reports in QuickBooks for Mac and noticed a $4000 difference in a particular section (expenses) and checked the filters; they were identical. I unchecked some fields in one and rechecked them and presto, the difference went away. It’s a good thing I noticed it otherwise I would have paid myself more than I deserved which would have made a royal mess out of the accounting.

While I realize it is software and there are always bugs, it seems that the software costs so much that Intuit could afford a few more QA folks to bang on every aspect of the program.

I read the other day that QuickBooks for Mac 2008 would be coming next year. I always have to question, what more can be added? I don’t use the integrated online banking and my needs are simple. The only thing I want is a stable product; features don’t mean much for me. This is apparent as I still use Quicken 2005; all I want for that program is an Intel native version…that I’d pay for as I don’t like running PPC binaries on my MacBook Pro.

Tired of crappy HP Scanner software

Just about everyday, I get crash reports from customers using HP scanners. The latest one I got, the customer was quite angry at me because DocumentWallet kept crashing. He finally sent the crash log and look at what it contains:

Thread 0 Crashed:
0   HP All-in-One Scan DS          	0x058f1240 0x57a4000 + 1364544
1   HP All-in-One Scan DS          	0x057b84fc BroadcastMessage__12LBroadcasterFlPv + 108
2   HP All-in-One Scan DS          	0x057bc604 BroadcastValueMessage__8LControlFv + 52
3   HP All-in-One Scan DS          	0x057e8388 HotSpotResult__11LPushButtonFs + 24
4   HP All-in-One Scan DS          	0x057bcbc8 ClickSelf__8LControlFRC15SMouseDownEvent + 184
5   HP All-in-One Scan DS          	0x057c2d6c Click__5LPaneFR15SMouseDownEvent + 140
6   HP All-in-One Scan DS          	0x057c96ec Click__5LViewFR15SMouseDownEvent + 76
7   HP All-in-One Scan DS          	0x057c96ec Click__5LViewFR15SMouseDownEvent + 76
8   HP All-in-One Scan DS          	0x057bdc38 ClickInContent__13LGrafPortViewFRC11EventRecord + 232
9   HP All-in-One Scan DS          	0x05874bec DoMouseDown__15CHPGrafPortViewFR11EventRecord + 236
10  HP All-in-One Scan DS          	0x0584b4e0 ScanWindowEventHandler__FP25OpaqueEventHandlerCallRefP14OpaqueEventRefPv + 304

Yes, folks, that means that HP’s software crashed DocumentWallet. It’s hard to convince user’s that my app isn’t causing the problem, but that’s reality. Vendors make crappy Mac scanner software and should be ashamed. They never expect people to use the TWAIN drivers because they suck so much and apparently never get tested.

HP, are you listening? Please fix your TWAIN drivers so that they don’t crash app’s that make scanner calls using proper TWAIN interfaces.

UI has gone to the birds

A new app called Pixelmator has gone into private beta according to TUAW and while it looks like it could be a replacement for Photoshop Elements, the screenshots in their gallery make it look like the developers went overboard with the semi transparent look. I have no idea how the app will actually work, but when everything is semi transparent, it looks like it could be a mess. In this screenshot, this is a sheet upon on a window. It’s a bit difficult to see where the sheet ends and the window begins. I’ve seen this trend in other apps such as Disco and just don’t understand. While I’m not an old geezer, I can’t imagine how people who are my father’s age can use this as some people already have problems seeing the display.

Please developers, if you want to have this look, make it an option to turn it off.

Getting Mail.app to work the way I want

I use multiple email addresses for various reasons, but have all the mail come into one account. Mail.app handles this fine, but there is no way to link the signatures to the accounts. I seem to recall that there was a plugin to handle this awhile ago, but with Tiger mail, it wasn’t needed. My switch to IMAP seems to have confused Mail and now I have to manually switch the signatures to match the account.

Not being one to do manual things over and over again, I hacked up a Mail.app plugin that sets the signature based on the from name (the signatures are named the same as the from addresses). It didn’t take long and I’m not sure what I should do with this besides use it myself. Who knows what will happen in Leopard as this plugin uses undocumented APIs.

Am I such the exception that I have many email addresses and want 1 signature per address? I don’t need multiple actual email accounts (Mail.app handles the signatures with those), just multiple addresses for my multiple personalities.

Stop wrapping dmg files in zip archives!

I routinely download software that ships as a Mac disc image, but when uploaded to a web server is placed in a zip archive. The dmg file is already compressed, so the zip file doesn’t reduce the size that much and just adds one step to decompressing (when I decompress, I’m left with the zip file and dmg file; I then have to trash the zip file). My suspicion on why this is done is that the web server isn’t setup to send dmg files as binary and people get a mess in their browsers. So solution (for apache servers), is simple. In /etc/mime-types, add dmg to the octet-stream line like:

application/octet-stream        bin dms lha lzh exe class so dll img iso dmg

First look at OCR

A recent comment on my blog struck a nerve where the commenter said that OCR would basically put a competitor to ReceiptWallet above it. While I still don’t believe that OCR is all that useful for receipts (if there is one mistake when you’re generally only entering 3 small pieces of data, you’ve wasted time because you have to review each entry carefully), I took a look at an open source OCR package. While this code is a bit rusty, there has been some recent work on it. My first test was a Rite-Aid receipt where I was looking to see if it could read 3 pieces of data, the merchant name, date, and total. It failed on the merchant name because it was a graphic, however, it picked up the date and total in such a way that I could parse the data and grab what I needed. I then tried 2 other receipts, both from Costco and the results were completely miserable such that I couldn’t get anything from them. I’ll keep plugging away and testing to see if my results are better.

In addition, I put in a request for a quote for a commercial OCR engine. However, I suspect that it will be cost prohibitive. If it costs $5,000-$10,000 upfront plus a per copy licensing fee, I can’t afford that as it would completely wipe out any profit unless I significantly increased the cost of ReceiptWallet.

If anyone has more information on OCR engines for the Mac (commercial or open source), please let me know.

Useless application?

I downloaded and installed Little Snitch today to give it a spin as it was part of the bundle of software I bought it (which brings me to the question, why did I buy it?, but that’s another story). The program is well implemented, but just about every application makes outgoing network connections these days, so it is always popping up basically saying “xyz application wants to connect to abc server. Allow?” After click always allow more than a few dozen times, I finally disabled it (for now). I’m sure after I train it, it will be less annoying, but how can I really tell what applications are making legitimate requests and which are not? This seems like the little boy who cried wolf.

Speeding up Ruby

When I setup my store, I sued FastCGI to get acceptable performance which was fine as I don’t get many hits. Today I installed BrowseBack because it came as part of the software bundle I purchase through MacHeist/MacUpdate. As I hit my store a lot to check stats and such, BrowseBack kept loading the store which spiked the server load to 4 and required me to kill BrowseBack to get my server back to normal. While this isn’t normally a problem, it exposed a potential kink in my server’s armor. So I went looking for an alternative. I’ve read about using lighttpd by proxying requests from Apache. Setting it up was straightforward and seemed to work well, until a customer complained that he couldn’t purchase. I tried it myself (again) and it worked fine. I had someone else try again and it failed. Hmmm. After much tinkering, I figured out the problem, my store code required an https connection otherwise it redirected to https which was fine, except that coming from the proxy, it was always an http connection. Since I already do a redirect on my store so that any http://store.receiptwallet.com requests goes to https://store.receiptwallet.com, I can be assured that all requests are secure. So I commented that line out of my store and everything is working fine.

(On a side note, the reason I couldn’t see it myself is that it was checking for local requests and since I access my server using a private IP address range, it basically got flagged as local, so it never required the SSL connection.)