Getting Android's Aldiko to user Calibre's Webservice as a Catalog
posted on Fri, 04/08/2011 - 17:02 by theshadow
The Beginning
Alright, so I'm going to start from the beginning. A while ago I bought a Nook because I really wanted to be able to carry my books with me. At first I was just tossing them in to a My Books off of my ~ directory and had a script to sync them over to my Nook when I had it plugged in. After a while this became tedious and I started to look around for a better solution.
Read how I upgraded my e-book management after the jump.
Calibre To the Rescue!
So began my search of Google for a better solution. It didn't take too long before I stumbled across this open-source/cross-platform solution called Calibre. Like most FOSS projects I was skeptical about how well the software would work. I mean the site looked good and the screen shots looked great but after being in the FOSS world for long enough you know how the screen shots and reality can differ. But I said, "what the hell", and went for it.
After some initial setup hiccups related to package requirements I was off to importing my collection of e-Books. Let me tell you it was a dream. If I could compare Calibre to something I would say Calibre is to e-Books as iTunes is to music management (ignore the store bit, I know it sucks horribly). It had been a long while since I was impressed by a project so much that I felt the need to donate. Which I did shortly after.
Now for a while Calibre was the king of my e-Book world. I'd import new content in and then just connect up my Nook and it would handle syncing the content. And all was well in the world, at least until I started trying to use my awesome Nook for full colored PDFs. This is where my poor Nook just couldn't keep up. Sure it could throw up text in all it's beautiful glory but it would choke and sometimes even die on big PDFs. Eying that delicious looking Nook Color. For months I went back and forth trying to decide if I really wanted another e-Reader that only going to be a hardware upgrade. I mean sure the Nook had unbelievable battery life but was it worth another $500 just to get a minor boost for PDFs?
In the end I thought no and waited for the Xoom.
Xoom and Nook
First came the 3G Xoom model which sounded awesome but I really wasn't in the mood to get a 2-year contract and I really didn't want to pay for Verizon's service. No offense to any Verizon customers out there. So I waited, and waited and waited some more till the fateful day when the Xoom Wi-Fi game out. By this point I had Aldiko, Nook, and Kindle running on my phone and loving it, except for the screen size and the slowness so when the Xoom Wi-Fi hit store shelves I knew it was time to get one.
I mean seriously, now with a Xoom Wi-Fi I could just turn my phone in to a wireless access point and then connect the Xoom to the phone and viola! Anywhere I had 3G signal I had internet access for my Xoom. Then came the lull where I tried over and over again to get my library to import in to my Xoom and was just having all kinds of problems. Plus, what happens when I'm away and want a book I didn't load? Oy, the concerns abound. It was then that I remembered that Calibre had a built in web service, and though a small bit of finagling I could get it to run headless!
So I marched off and set it up on my VPS full speed ahead. It took about an hour but after the initial set up and some script magic to rsync my Calibre Library to my VPS I could now visit and access my books via any web interface! Which was great I could then open them on my tablet, download them and open them with aldiko and read to my hearts content.
Sadly though, aldiko didn't seem to import the books I downloaded in to the library. So opening the books over and over again by navigating to the Downloads folder was cumbersome and annoying.
Xoom, Aldiko, and Calibre
So back to the drawing board. After a little bit of research I found that Aldiko had a way to import and use personal catalogs. Which I followed up with a Google search for "Aldiko and Calibre" and low and behold others had used Calibre as a catalog! So off I went to set it all up and all excited as a kid on x-mas morning I added the catalog, entered the login and password and waited, and waited, and waited.... No error, no status updates only clue that something was amiss was that when I touched the catalog it opened the validation dialog again. Asking for the username and password.
Well, this was annoying and downright frustrating. Nothing appeared in my logs, nothing appeared anywhere. Just nothing happened. After digging around some more I discovered that the HTTP implementation in Android doesn't correctly handle the HTTP 401 message and that this has been a known bug since 2008!
Obviously this wasn't on Google's top to-do list to fix. Fine, so I figured that others had to have found a solution, they just had to. Back to Google I went for more answers (ironic, no?) after a while I found that others had used a reverse proxy to get around the issue. Simply, by using a man in the middle that correctly handled the HTTP 401 requests make all the requests you could use it in Aldiko since it looked all the same.
Down the Rabbit Hole of Apache Httpd and ProxyPass
Simple enough I figured. I'd go in and figure out how to set up Apache httpd to do it for me! Boy, how wrong I was. The documentation on the virtual hosts declarations were all well and documented, what wasn't was any clarity on when to use which declarations. Hours spent, even in meetings at work while sitting on my tablet, trying to get this to work.
Asking questions in #httpd on FreeNode going back and trying again. Honestly, in the end the changes necessary weren't that hard most of the hiccups came about either because I mistyped something or because I was misusing something. In the end and after much help from the guys in #httpd I got things going.
After the reverse proxy was up and going Aldiko propery identified the catalog and I was free to browse and download to my hearts content.
So, what was the point of this blog post? Well partially a rant, and partially to help those who follow. Of course if Google gets off their ass and fixes this bug then this won't be necessary. Till then, the following VHost declaration should help you out.
#this VHOST works for anything trying to connect on port 80
<VirtualHost *:80>
#define that this vhost works with the my sub-domain.
ServerName my.example.com
#I'm not 100% sure on this one but the docs say if you're using ProxyPass or it's kin this must be off.
ProxyRequests Off
#state that any resource request is allowed to be read and overridden, in Ubuntu http_proxy defaults to disabling it.
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
#declare that any request to my.example.com/ or there in are proxy'd to localhost:8080
#port 8080 is the default for the calibre web service.
ProxyPass / http://localhost:8080/
#This allows you to rewrite any links from localhost:8080 back to / so that it all works correctly
ProxyPassReverse / http://localhost:8080/
<Location / >
Order allow,deny
Allow from all
</Location>
</VirtualHost>
Tags: