Amateur Topologist

Everything but topology.

Category: Computers

Writing Heist Splices for Snap

I’ve been doing a lot of web stuff lately; so far, it’s only been very simple HTML + CSS + JS (well, CoffeeScript, but who’s counting), but eventually I might move on to other, neater things. And in the process one of the things that I found is Twitter Bootstrap; it’s a bunch of CSS that makes it ‘easy’ to make professional-looking sites. I put ‘easy’ in quotes because the learning curve is definitely steeper than that of CSS itself, because you have to nest your DOM just so in order for the given CSS to apply right. But you get nice margins, good color themes, etc.

One of the things that you get when you use Bootstrap is a very nice-looking nav bar, which is actually just a very well-themed unordered list of links.

In particular, if you set the class of one of the list items to “active”, it ‘highlights’ it:

Note the highlight on 'Scalemate generator'

Now, I’m serving this site through Snap, the Haskell webdev framework; I’m doing this because one of my subsites needs to be able to upload files, and I figured I might as well use Haskell. And Snap comes with a templating language called Heist. So, given that I’m already using templating to insert the boilerplate jQuery/Bootstrap includes and the GitHub link/authorship at the bottom, wouldn’t it make sense to template the nav links?

It turns out that it’s fairly simple to do, as illustrated in NavLinks.hs. We start with imports; nothing too interesting here. We need Heist to access the templating engine and Text.XmlHtml to actually construct the nodes.

1
2
3
4
5
6
7
8
9
10
11
12
{-# LANGUAGE OverloadedStrings #-}

module NavLinks where

import qualified Data.ByteString as B
import qualified Data.Text as T
import           Data.Text.Encoding

import           Snap

import           Text.Templating.Heist
import qualified Text.XmlHtml as X

Next we declare the type:

17
navSplice :: MonadSnap m => [(B.ByteString, T.Text)] -> Splice m

The way a splice works is this: you construct a Splice in some monad Splice m, and you can lift monad actions in the m monad into Splice m. In this case, we can be in any monad that provides us with getRequest, since we want to get the URI of the request. Since getRequest :: MonadSnap m => Request, we can be in any monad that is an instance of MonadSnap. The argument to navSplice is the list of (path, title) tuples; for example, it might look like [("/about", "About"), ("/blog", "Blog")]. The path is a ByteString because the request gives us the URI as a ByteString, and the title is Text because Text.XmlHtml uses Text. Now, onto the actual declaration:

18
19
20
21
22
23
24
25
26
navSplice links = (:[]) . X.Element "ul" [("class", "nav")] <$> do
  currentURI <- lift $ rqURI <$> getRequest
  -- add the 'active' class if the href is a prefix of the current URI
  let li path
        | path `B.isPrefixOf` (currentURI `B.append` "/") =  X.Element "li" [("class", "active")]
        | otherwise = X.Element "li" []
  return $ map (\(path, title) -> li path [buildLink path title]) links
   -- build a link to the path with the given text
    where buildLink path title = X.Element "a" [("href", decodeUtf8 path)] [X.TextNode title]

Look at the do-block first. Line 19 just gets the current URI out. Lines 21-23 are interesting; what we’re doing here is declaring a function of type li :: B.ByteString -> [X.Node] -> X.Node. The way an HTML element is constructed in Text.XmlHtml is through the Element constructor, which has type T.Text -> [(T.Text, T.Text)] -> [X.Node] -> X.Node. The first Text is the tag; the array of Text tuples is the list of attributes, and the list of Nodes is the list of children. So, here we’re partially applying the tag (always “li”) and the attributes (which is either empty or class="active"). We apply the “active” class only if the path is a prefix of the current URI; we append a “/” to the end of the current URI to make sure that trailing slashes are ignored.

Next, look at line 26; buildLink just builds a link node looking like <a href="path">title</a>. X.TextNode is, obviously, the constructor for text Nodes.

On the last line of the do block, we map over the (path, title) tuples, build up a list of <li> elements, and return that list. Finally, on line 18 we construct an unordered list element with the “nav” class. Finally, we construct a single-element list using (:[]) (which some people call the monkey function), which just constructs a list with its argument inside. My actual code uses return instead, but I decided that this would be clearer.

Adding that splice to the default list of splices is as simple as adding

pages :: [(B.ByteString, T.Text)]
pages = [ ("/chargen/", "Character generator")
        , ("/logify/", "Chatlog formatter")
        , ("/scalemate/", "Scalemate generator")
        ]

to the toplevel of Site.hs and adding

addSplices [("nav", liftHeist $ navSplice pages)]

to the app initializer. Now <nav/> can be used in the template to automatically insert the nav link list and highlight the current page!

One small annoyance of this is that the generated HTML is all in one line, although since it’s not handwritten you probably won’t care so much, especially since Firebug and Chrome’s debugging tools will automatically do the indentation for you and let you collapse/expand child nodes at will.

Name Your Type Variables!

Haskell’s type system is commonly touted as one of the most powerful features of the language; it’s often said that if a program compiles, it’s pretty likely to be correct. And while there are always going to be errors your type system can’t catch (logic errors, off-by-one-errors, etc.), I’ve found that the type system helps ensure that your programs are put together correctly. If it typechecks, you’re probably not making a higher-level logic error; you might be trying to fit a red peg into a blue hole, but you’re not trying to fit a square peg into a round hole, so to speak.

The problem is that figuring out what the type variables in a polymorphic type (i.e., type like length :: [a] -> Int) mean. Sure, in simple cases it’s simple; you look at a type signature like ifM :: Monad m => m Bool -> m a -> m a -> m a and you know that m is going to be some monad and a is some arbitrary action. But take a look at validate :: Monad m => Form m i e v a -> Validator m e a -> Form m i e v a, from digestive-functors. What do m, i, e, v, and a stand for? It’s pretty clear that m is some monad, but what about the others? It turns out that:

  • i represents the input to the form, or its environment (i.e., the submitted parameters)
  • e represents the the error type that can be produced
  • v represents the ‘view’ of the form (how it’s represented to the user)
  • a represents the actual evaluation of a form (for example, in a registration form, it might be data Registration = Registration Text ByteString)

So my argument is that it might be better to describe it as Form m input err view result -> Validator m err result -> Form m input err view result; this way, it’s clearer exactly what each type variable corresponds to.

Again, I’m not saying that single-letter type variables should be completely banned; something like (>>=) :: (Monad m) => m a -> (a -> m b) -> m b is simple enough. The conventions that a and b stand for the to type variables in a generic function and that m means ‘some arbitrary monad’ are strong enough that it’s like using i in a for loop in Python or whatever.

(Full disclosure: yes, lots of my own Haskell code uses single-letter type variables in places where I shouldn’t. I do intend to fix this).

But if you’re trying to learn some new framework or module, imagine how much nicer it would be if the types that the Haddock gives you actually told you something about what it’s doing (especially given that most Haskell modules seem to be lightly documented outside of their Haddock documentation, lacking example use cases). Programmers outgrew using single-letter variable names decades ago; we shouldn’t make the same mistake in the type system. This is part of the reason why I’m making such slow progress learning the Snaplet framework in the dev version of snap; it’s hard for me to get an intuition as to what the types are when I keep seeing Handler b v a (b is the base app/snaplet, v is what’s being ‘focused on’, and a is what the Handler ‘evaluates to’; Handler b v has a Monad (and Functor, and MonadPlus, etc.) instance).

By the way, the Snaplet framework, though slightly complicated, does wind up making a good deal of sense once you get used to it; I plan on writing up a post about what the commonly-used types/type constructors 'mean', as well as a post about how to make a very simple login Snap app. And I think, ultimately, documentation is one of the things Haskell sorely needs; not just more documentation on what individual functions do, but how those functions combine into a larger whole. Imagine trying to learn Django just by looking at the module documentation without having the benefit of the tutorial! But I think that's a subject for another post; I'm trying to get this blog active again, after all.

Settings in Chrome Extension Content Scripts

Google Chrome, like most modern browsers, allows you to write extensions for it. And those extensions can inject JavaScript into pages that the end user views, so that you can write extensions to do things such as customize the interface of some site, theme it, etc. Simple extensions that don’t need to be configured by the user can get away with being simple Greasemonkey-like scripts that the user installs by navigating to and then forgets. The upside of this is that it’ll also work in Firefox, since Chrome’s native content script support is a strict subset of Greasemonkey’s (it doesn’t have @require or the ability to set/get data, among a few other things). The downside is that there’s no way to do any kind of updating short of your users manually installing a new version, and, more importantly, your extension cannot have any state or user configuration. In order to have stateful extensions, you have to go full-blown Chrome.

The basics of Chrome extension development are adequately covered by Google’s own Getting Started tutorial. The interesting part comes when you want your content scripts to be configurable by the user. For example, Tumblr Hate (which I’m the author of, and which has source that you can view here) adds ‘hate’ links to tumblr posts that allows the user to hide them. The user can configure the text of the ‘hate’ links, as well as what shows up when a post is ‘hated’ (i.e., hidden). Fortunately, Chrome has built-in support for Extension options pages, which have a localStorage object that you can use as a key/value store (although values are serialized to strings). Unfortunately, the localStorage object is not shared between your options page and your content scripts. I assume the reason for this is because of sandboxing, but it creates a problem: how do your extension’s content scripts get the settings?

Use chrome.extensions.sendRequest

You have your options page do whatever it does, saving settings to localStorage. You then create a ‘background’ page that installs a listener using chrome.extension.onEvent.addListener; the exact form of a request and how it should be responded to is obviously up to you, but here’s what I use: A request looks something like  {type: "get", name: ["settingName", "otherSettingName"]}, which is responded to with an object that looks something like {settingName: 1, otherSettingName: true}, or {type: "set", name: "settingName", value: "5"}, which sets the setting settingName to 5. You can even special-case get requests so that if the name is a single string, the response is just the value and not an object. If your extension does things other than inject content scripts, you’ll need other request types to do things like open tabs or whatever, but this is good enough for simply mimicing a key/value store. The code to do this is simple:

function(request, sender, sendResponse) {
    switch (request.type) {
    case "get":
        var data = {};
        if (request.name.constructor == Array) {
            for (var i = 0; i < request.name.length; i++) {
                data[request.name[i]] = localStorage[request.name[i]];
            sendResponse(data);
        } else {
            sendResponse(localStorage[request.name]);
        break;
    case "set":
        localStorage[request.name] = request.value
        break;
    default:
        console.log("invalid request " + request);
        break;
    }
}

The next issue, however, is trickier: chrome.extension.sendRequest is asynchronous. In particular, instead of returning the response, it accepts a callback that takes the response as an argument, and returns nothing useful. So what do you do?

Do your work in the callback

If you just use the callback to set some globals and then continue on, you have a race condition. Consider the following code:

var setting;
chrome.extension.sendRequest({type: "get", name: "settingName"}, function (resp) {
    setting = resp;
}
);
console.log(setting);

This will fail, because sendRequest is asynchronous, and sending a request and receiving the response very well might happen after the setting has been used! The correct thing to do here is this:

var setting;
chrome.extension.sendRequest({type: "get", name: "settingName"}, function (resp) {
    setting = resp;
    doStuff();
}
);

function doStuff() {
    console.log(setting);
}

Here, you can guarantee that doStuff won’t be called unless the setting has been properly set, so you can safely rely on its value.

Note that I didn’t pass setting as an argument to doStuff; I very well could have. But in the case of Tumblr Hate, for instance, I didn’t feel like threading the relevant variables through function calls, so I simply declared the variables at the top so they’d be global. (Well, no I didn’t, I set the response’s variables as properties of a global ‘root’ object, but that’s basically the same thing.)

Preserving Links Across Reorganization

Link rot is the scourge of the modern Internet. It’s incredibly frustrating to find a link to something that looks really interesting, or that looks like it might solve all of your problems, only to have the link be dead. Sometimes it’s because the content is just gone; there’s really no getting around that. But sometimes it’s because the site it was hosted on reorganized their structure in such a way that the data’s still there, but it can’t be reached at that old URL. If the site has a built-in search function, then you might be able to find it that way; you also might be able to use Google’s site:example.com domain restriction (or find it on another site that so graciously copied the entire text). But that still involves extra effort.

So what can you, as a site owner, do to minimize link rot? The answer is make sure that all your old URLs continue to function. Here’s an example: I recently moved all the files for this blog from /var/www to /var/www/blog, to keep my directory structure neat. I then mucked around in WordPress to tell it the new file location, set up /etc/apache2/sites-enabled/000-default properly, and everything was working fine. But the old incoming links were now broken! Fortunately, mod_rewrite exists and it’s fixable with a few simple rules in /var/www:

RewriteEngine On
RewriteBase /
RewriteRule ^((\d\d\d\d|tag|feed|category).+)$ /blog/$1 [R]
RewriteRule ^$ /blog [R]

The first two lines here are standard mod_rewrite boilerplate. The third and forth do the actual interesting rewriting: The regex will match any URL that starts with (after the amateurtopologist.com/) four digits or the words ‘tag’, ‘feed’, or ‘category’ and prepend /blog/ to them. So, for example, http://www.amateurtopologist.com/2010/12/01/some-useful-approximations/ will get rewritten into http://www.amateurtopologist.com/blog/2010/12/01/some-useful-approximations/; both of those URLs do work and will send you to the same post. The [R] indicates that Apache should serve a redirect instead of transparently rewriting it; I did this so that people would use the new URLs in case I ever have to actually break the old ones. Finally, the last line rewrites the empty request into /blog (again using redirect).

The net result here is that pretty much every single URL that used to work will still work, but I can now keep my blog and my other stuff in separate directories. A few URLs broke because I’m too lazy to fix them (search URLs and the about page), but nothing that people were likely to actually link to.

The Desktop of Theseus

I recently decided to upgrade my aging desktop. First on the list was a better CPU; I have a rather unfortunate stepping of the AMD Phenom 9600 that has a bug that reduces performance by 10-20%, with the only fix causing the system to become unstable. So I got a nice Athlon II, but the problem was that my (mostly stock) computer has an AM2 motherboard. So I had to replace that. Then I decided to get an ATX, since I don’t like the small size of mATX stuff. Which meant I needed a bigger case. And then I figured that 3GB is positively shameful for a desktop (or even a laptop) in 2011, so I got 4 GB RAM, which I’ll use along with the 2 1GB sticks I have already (and it’s a good thing that I did, because I forgot that DDR2 and DDR3 aren’t compatible). So the only things left from the current computer will be the power supply, the hard drive, the Blu-Ray drive, and the graphics card.

This naturally raises the question: is it the same computer any more? This is the paradox the title refers to, the so-called ship of Theseus, which had every plank replaced. It’s about one-third old parts, two-third new parts. And so I kind of want to say that it’s the same computer, because I’m moving over the hard drive, and therefore the data. But does that mean if the hard drive in a computer fails, and you replace it, it is now a different computer? That’s absurd. I don’t think that there’s any well-defined answers here; I didn’t reinstall Windows or anything and I kept the hostname and all the other interesting state. Me, personally, I view it as the same computer on some base level, but on a more intellectual level, I have no idea. I’d say it’s the same computer, though, because it serves the same ‘function’ and uses parts from the old one. If I just bought an entirely new stock computer and threw this one out, it’d be different. Similarly, when I replace the hard drive and graphics card, it’ll still be the same computer, even though it won’t share any parts with the original.

On a mostly unrelated note, assembling a computer is a very good way to at least learn how stuff is actually physically connected; some things draw power from the motherboard, some connect directly to the power supply, etc. It’s also cheaper than buying stock, and it means you’ll have an easier time upgrading parts when/if you decide to do so. The only downside is that you probably won’t get a free copy of Windows, and you can’t get a warranty. But your individual parts are probably warrantied, and if one of them fails it’s not too hard to determine what failed; clicking noise means a failed drive, memtest86 will tell you if your RAM is bad, etc. Plus it’s honestly more satisfying using a computer that you assembled.

Handling modifier keypresses in Cocoa

One of the features of Snow Leopard is the addGlobalMonitorForEventsMatchingMask:handler: method, which lets you handle events in other programs. This requires the user to have enabled accessibility in System Preferences, or for the program to be running as root and grant itself trusted status, but it lets you install a handler that will be called whenever any event that matches the relevant mask occurs outside of your application (for events inside your application, use addLocalMonitorForEventsMatchingMask:handler:. One of the things that you can capture is the pressing of a modifier key, such as Cmd or Option without a corresponding ‘real’ key. If your mask includes NSFlagsChanged, then any press or release of a modifier key will send your handler an NSEvent that you can call modifierFlags on to get an NSUInteger corresponding to which modifier keys were actually pressed. But there’s a problem here: if you hold down Command and repeatedly press the Shift key, your handler will be called for both the keypress and the key release. So if you’re counting the number of presses of each modifier, you’ll count the Command key again each time you press or release the Shift key, which is probably not what you want!

The solution is to use a block. A block is like a closure in other languages; we can use it to create a function that tracks a bit of state around with it. In this case, we want our flag handler function to track what modifier flags were present the last time it was called; if we compare the set of flags in the new invocation to those from the old invocation, we can determine what changed, and therefore what actually happened:

__block NSUInteger lastFlags = 0;
void (^flagTrigger)(NSEvent*) = Block_copy(^ (NSEvent *event) {
    NSUInteger flags = [event modifierFlags];
    NSUInteger masked = flags & ~lastFlags;
    lastFlags = flags;
    // flags now only includes keys that were actually pressed
    // handle those keys here
};

[NSEvent addGlobalMonitorForEventsMatchingMask:(NSFlagsChangedMask) handler:flagTrigger];

Here, we declare a lastFlags NSUInteger that we’ll use to store the previous set of pressed modifier flags; by declaring it using __block, we note that the block is allowed to modify it. We then create a block flagTrigger, whose type states that it takes an NSEvent* and returns void, and which handles the logic: when it’s called, it gets the modifiers in the event, and applies a mask consisting of just the flags that weren’t already set last time. We then store the flags variable into lastFlags for the next time, and then process the masked flags. Finally, once we’ve declared the block, we install it as the handler.

The code above is basically copied wholesale from a project I’m working on called KeyboardViz, specifically the Keyboard class which displays an onscreen keyboard that changes colors as you press keys. I’m using it to learn ObjC and Cocoa, so suggestions on coding style are welcome.

Facebook’s privacy settings: another look

Much has been made in the news recently about Facebook’s relative lack of privacy controls, and the degree to which they’re hidden and made unintuitive to use. Naturally, people have been speculating about why they do this. The intuitive answer, and the one that I’ve heard a lot of people claim, is that it allows them to sell data to advertisers; the reasoning goes that they can’t sell your data to third parties and make money off of you if your profile settings are all set to private. But as far as I know this isn’t the case; I’ve got all my Facebook data pretty locked down; the only information about me if you’re not my friend is my name and a picture of me, which I figure is enough to allow people I know to friend me while being certain that they’re getting the right me. Yet I still get ads that I know are targeted to me because they mention my college, my location, etc.

Instead, I suspect that the real money that Facebook makes off of people who don’t have their information private is off advertising impressions. A lot of people, when they want to find more about someone, will immediately check Facebook to see if they have a profile, and if they do, they’ll spend a while ‘stalking’ them on it. If the ‘stalkee’ has their information private and the stalker isn’t friends with them, then they have to send a friend request, which means that most of the time they’ll back off. But if the stalkee’s information is public, then the stalker can spend large amounts of time looking at their information. Which means large amounts of pageviews, and large amounts of advertisements being displayed to them, which means more money for Facebook. And I think that that’s one point that a lot of people miss when they write about Facebook; not only do they want you to put all of your information on there so they can sell it, they want you to put your information on there so that other people will spend time on their site looking at your information.

Managing Online Identity

Like anybody who’s even looked at a computer in the past few years, I have an ‘online identity’. I have accounts on Twitter, Facebook, Github, Wikipedia, Formspring, Steam, AIM, MSN, and more forums and IRC servers than I can honestly remember at this point. But obviously I don’t want all of those to be connected; I don’t want the people who know me on a forum I post on to be able to figure out my real name and where I live. Similarly, I don’t want my Twitter account to be associated with my real name, because I post things to it that I wouldn’t want an employer to see. But I’ve messed up; I’ve used this site in the past to host random images, and I can’t remove that reference because other people have quoted those posts. I’ve linked to this site on my Twitter account in the past, before I used my real name.

Even if I didn’t use my name here, and I changed my username on the site from ‘phurst’, you could still figure it out; I’ve linked to my Github account, and my commits have my real name in them. I could remove those posts, but then I couldn’t write about sizeable code projects that I’ve made, since I tend to code under my real name. Plus, having a site full of respectable content come up as the first Google result for your name that’s actually you is good.

I’m especially concerned given the interconnected nature of the Internet and the growing ability of search engines to discern semantic content from websites as opposed to simply statically indexing text; the power of a human-like intelligence parallel processing everything I’ve written under one name or another I can almost guarantee will connect most of my online identities together.

On the other hand, I’m not the only person with online secrets to hide, especially with growing Facebook use. I don’t have photos of me drinking while underage, or status updates about cheating on tests or stealing or anything. Maybe the growing all-seeing eye of the Internet won’t hurt me as bad in comparison to everybody else. For now, I’ll continue not linking anything to anything else unless I’d be fine with everybody ever knowing that the two are one and the same.

A Haskell Newbie’s Guide to Text.JSON

JSON parsing is practically required for any modern language to be able to interface with web-based applications; most of them offer JSON as a reply format, and the alternative (usually XML) can be cumbersome to work with. But Haskell and its strong type system seem like they’d be extremely ill-suited to parsing JSON; an object’s values can be arbitrary JSON objects, including different types in the same object, and arrays can contain different types of objects. How do you deal with this? Well, if you’re going to be parsing JSON, then you have to have some kind of format that you’re expecting; you know that you’ll get, say, an array of objects each of which has a specific key that has an integer as a value, and that integer is all you care about. Fortunately, Text.JSON exists, and once you get your head around how to use it, it’s simple. (Then again, so are many things in Haskell, but that doesn’t help me understand Arrows any better!)

So, let’s look at an example: getting a user’s public timeline from Twitter, and turning it into an array of Status values. We can ignore the process of actually sending Twitter the request and pulling the JSON out of the response, and for the sake of brevity we can ignore a bunch of the irrelevant data that Twitter returns, such as whether the tweet is a reply, the source, etc. Here’s the result of asking for the last 2 tweets from the official Twitter account, with irrelevant detail stripped:

[
    {
    "user": { "screen_name": "twitter" },
    "text": "Read this guest post from @FiresideInt on his experience in Haiti http:\/\/t.co\/mbMU56R"
    },
    {
    "user": { "screen_name": "twitter" },
    "text": "Do you use Twitter for a business, school, community group or another local organization? Follow @TwitterBusiness for tips and useful info!"
}
]

So, what we have here is a list of objects; each object has a user attribute whose screen_name is the actual username; the object’s text attribute then contains the actual text of the tweet. Let’s get to work.

Making the JSON value

It’s helpful to play around with whatever we’re trying to manipulate in ghci; so let’s load up the JSON into a String, json, and try to parse it:

Prelude Text.JSON> json <- readFile "/Users/phurst/json"
Prelude Text.JSON> decode json

<interactive>:1:0:
    Ambiguous type variable `a' in the constraint:
      `JSON a' arising from a use of `decode' at <interactive>:1:0-10
    Probable fix: add a type signature that fixes these type variable(s)

Why doesn’t this work? Well, decode has type (JSON a) => String -> Result a. So you can decode a string of JSON into anything in the JSON typeclass. Well, JSValue is in the JSON typeclass, so let’s try that:

Prelude Text.JSON> decode json :: Result JSValue
Ok (JSArray [JSObject (JSONObject {fromJSObject = [("user",JSObject (JSONObject {fromJSObject = [("screen_name",JSString (JSONString {fromJSString = "twitter"}))]})), (rest of line omitted)

Wordy. But we did get a successful parse (as indicated by the Ok); we then have a JSArray which contains the two JSONObjects. But in order to get at the things inside the JSArray, we’d have to manually remove the constructor via (\(JSArray x) -> x) or something. And then if we didn’t get an array (because we got an error!), we would get an unfriendly “Non-exhaustive patterns in lambda” exception. So, what do we do? Well, we’re trying to get a list of values, so let’s ask decode to give us one:

Prelude Text.JSON> decode json :: Result [JSValue]
Ok [JSObject (JSONObject {fromJSObject = [("user",JSObject (JSONObject {fromJSObject = [("screen_name",JSString (JSONString {fromJSString = "twitter"}))]})), (rest omitted)

Awesome! We have an array of JSObjects. But again, we’re still stuck inside that JSObject ‘wrapper’. This annoyed me for a while, until I realized that JSObject is both a data constructor in the JSValue type, and its own type! So we can ‘ask’ decode to give us a list of JSObjects:

Prelude Text.JSON> let decoded = decode json :: Result [JSObject JSValue]
Ok [JSONObject {fromJSObject = [("user",JSObject (JSONObject {fromJSObject = [("screen_name",JSString (JSONString {fromJSString = "twitter"}))]})), (rest omitted)

Now we’re in business. We have a list of JSObjects, which is as good as we’re going to get. Now we need to actually deal with getting data out of them. It’s a good idea to split the ‘parse an individual item’ logic off into its own function, which I’ll call makeStatus.

Writing makeStatus, dealing with nested objects

Now, we could call fromJSObject :: JSObject e -> [(String, e)], search through the pairs, and then deal with them manually. But that’d be messy, and we wouldn’t get error handling if for some reason Twitter mysteriously didn’t give us a “user” object. Instead, we should take advantage of the function valFromObj :: JSON a => String -> JSObject JSValue -> Result a, and the fact that Result is a monad. Together, these mean that we can write simple code:

data Status = Status { user :: String, text :: String }

makeStatus :: JSObject JSValue -> Result Status
makeStatus tweet = let (!) = flip valFromObj in do
    userObject <- tweet ! "user"
    user <- userObject ! "screen_name"
    text <- tweet ! "text"
    return Status {user = user, text = text}

Here, I’ve defined (!) for brevity’s sake, then used the Monad instance of Result to ‘chain together’ my calls to (!). At the end, I wrap the user and text into a Status value, then ‘return’ it back into the Result monad. Let’s see it:

Prelude Text.JSON> let tweet = (\(Ok x) -> x) decoded !! 0 -- just to get a non-monadic JSObject for now
Prelude Text.JSON> makeStatus tweet
Ok (Status {user = "twitter", text = "Read this guest post from @FiresideInt on his experience in Haiti http://t.co/mbMU56R"})

We’ve successfully parsed a status update into its components! If we cared, we could pull more information out of the original object, such as real name, avatar URLs, time of posting, etc. But first, we have a more interesting problem: how do we join these two together?

Combining decoding and parsing

Look at the type for our decoding function and for our parse function:

Prelude Text.JSON> :t \json -> decode json :: Result [JSObject JSValue]
\json -> decode json :: Result [JSObject JSValue]
  :: String -> Result [JSObject JSValue]
Prelude Text.JSON> :t makeStatus
makeStatus :: JSObject JSValue -> Result Status

Clearly, we’d like it if we could put those in one function with little effort. So, let’s abstract out the specifics: we have functions f :: a -> m [b] and g :: b -> m c. We want h :: a -> m [c]. It’s obvious we want to map g, but a normal map isn’t right, since we’d get a function of type [b] -> [m c]; we want the monad to be outside the list! So we use the monadic map, mapM g :: [b] -> m [c].

Now, we have our two functions, and we could use do-block syntax to combine them:

parseTimeline json = do
    decoded <- decode json :: Result [JSObject JSValue]
    mapM makeStatus decoded

But do-block syntax isn’t terribly ‘Haskell-y’, and there’s only two lines of it. Surely there’s a way to combine them! And if we use Hoogle and search for (a -> m b') -> (b' -> m c') -> (a -> m c') (using primes to represent lists), we find that (>=>) in Control.Monad does exactly what we want! So we can rewrite it as a one-liner:

parseTimeline :: String -> Result [Status]
parseTimeline = decode >=> mapM makeStatus

Two things to notice: first, the explicit type declaration for decode isn’t necessary. In fact, it wasn’t necessary in the above version either. This generally happens once you’ve actually written the processing part of your JSON handler; the type signature of the processor forces the types in the parser! Second, the function is now pointfree; there’s no point in including argument, so we might as well omit it.

Errors

Now, there’s one problem here that I haven’t addressed. While we do get error handling for ‘free’, since monadic handling of Result values will pass through errors, the errors are typically unhelpful; they only show what failed to parse. And if we failed to parse the initial array, for example, that probably means that Twitter gave us an error message instead, and we’d like to know what it says! Finally, our output is stuck in the Result monad until we get it out.

The solution here is to write an error handler combinator; it takes a processor and a JSON string, and tries to process it; if the result pattern-matches against Ok x, then we parsed successfully; if it doesn’t, then we parse it again looking for the error message and handle that according to however your program deals with errors.
For my part, I kind of cheat; I don’t use the JSON in the error combinator, I use the raw HTML response, and if the parse fails, I throw an exception according to to the access code returned. But I eventually do plan to actually grab the error message, which will simplify the control flow of the library and give me more specifics on what went wrong.


This entire post was inspired by my experiences writing askitter, a Haskell Twitter library using OAuth for authentication. Most of the ‘final’ code in here is copied straight from there, aside from various wrapping/unwrapping functions. It’s been very useful for learning lots of things; it’s been suggested that I ‘hide’ the fact that Twitter only gives you chunks of 20 tweets at a time by using enumerators; when/if I get around to that, I’ll write an explanation.

URL bar marquees!

Remember the heady days of the late 1990s, when every Geocities webpage would have a script that displayed a marquee in the status bar? Well, thanks to the magic of HTML5, you can revisit them with a twist: instead of marqueeing in the status bar, you can do it in the URL bar without the annoyance of having your history be clogged with a bunch of shit. How does it work? If you’ve ever browsed someone’s YouTube channel, you’ve probably noticed that the URL that you get in the address bar changes each time you click a video. But the page doesn’t refresh; this is because the part that’s changing is actually the anchor element, which comes after the #. This has the downside of potentially interacting poorly with your browser history or other things. HTML5 instead provides pushState and replaceState, which basically allow you to rewrite the displayed URL to any page on the same domain without actually forcing a page reload or whatever. They also allow you to carry around titles for the various states (different from the <title> part of an HTML document) and actual state data. Flickr uses this to great effect; if you go to, say, this photo and click the ‘This photo also appears in’ tabs, you’ll see the URL being rewritten without a page load. Note that as of now this only works in Firefox 4, Safari, and Chrome.

But that’s not what this post is about. It’s about marquees! If you use javascript, you can create a marquee in the URL bar. It has the disadvantage of everything being URL-encoded, so no fancy Unicode characters, or even spaces. It also means that unless your webserver is set up right, hitting refresh will take the user to a non-existant page. It’ll also annoy the ever-loving fuck out of your users since they won’t be able to type in the address bar before your code rewrites it. But who needs that anyway when you have glorious scrolling text?