Here’s an email I recently received from the URL-shortening service bit.ly, a popular Twitter-related service that I use to automatically shorten URLs for LGF-related tweets. The nice thing about bit.ly (and j.mp, it’s the same service) is that it doesn’t just shorten URLs; if you register an account, it also tracks some interesting statistics for any URLs you shorten. And they make these statistics available to outside developers through the bit.ly API; this is how our Retweet buttons are able to show the number of clicks on each shortened link.

But apparently, LGF has been hammering their API servers too hard.

Hi,

I’m writing in regards to your bit.ly API usage. You are making an unusually large number of shorten requests to our API (often over 50,000/hour), and it looks like you are doing this on page load (for the Twitter button on pages such as http://littlegreenfootballs.com/article/35599).

Normally it is best not to make /shorten requests on a page load, but instead to make a /shorten request to bit.ly when a user clicks a share button. You can find an example of that at code.google.com. We are increasing our application of rate limits to manage abusive API users, and we encourage you to make the above fix as soon as possible to ensure that your site continues to function normally. I’d be glad to discuss this further and/or answer any questions you have, and thanks for using bit.ly!

The problem was related to the John Resig Retweet button, which I wrote about in this Tech Note. When the LGF home page loaded, the Retweet button Javascript code called the bit.ly API’s “shorten” function to find out the shortened URL, followed by the “stats” function to get the number of clicks, for each post, every time the page was loaded. As you can imagine, with 80,000 to 120,000 pageviews a day, that’s a lot of API calls. No wonder they noticed it.

I’ve vastly reduced the number of API calls now with the latest code refactoring of the Retweet button. I stopped using John Resig’s Javascript code (no hard feelings, it’s great for most sites but not LGF), and wrote my own jQuery routines that use Ajax to call a server-side script when the page loads, passing it the necessary data on every Retweet button in the page in JSON format. The Ajax server calls the API’s “stats” function, then sends back (again in JSON) the number of clicks and some other data. The jQuery code then displays those numbers in the Retweet buttons.

The key to reducing our API calls is that the shorten function is only called once, not when the page loads but when a new entry is posted to the LGF blog. The shortened URL is used to automatically tweet about the new post on Twitter, then it’s saved along with the new entry data, to be reused endlessly as needed without ever calling the API again.

Previously, the Resig code would call the API’s “shorten” routine and the “stats” routine for every post. Now the PHP Ajax server uses the saved bit.ly (or j.mp) URL to call the API’s “stats” routine only, which should cut our API usage in half. And retrieving statistics should be a much less costly operation for the server than shortening a URL. Hopefully, this will bring us back within the bit.ly API usage limits.

And since this is probably Venusian to most LGF readers, consider it an open thread from here on out…