Amateur Topologist

Everything but topology.

Writing Heist Splices for Snap

I’ve been doing a lot of web stuff lately; so far, it’s only been very simple HTML + CSS + JS (well, CoffeeScript, but who’s counting), but eventually I might move on to other, neater things. And in the process one of the things that I found is Twitter Bootstrap; it’s a bunch of CSS that makes it ‘easy’ to make professional-looking sites. I put ‘easy’ in quotes because the learning curve is definitely steeper than that of CSS itself, because you have to nest your DOM just so in order for the given CSS to apply right. But you get nice margins, good color themes, etc.

One of the things that you get when you use Bootstrap is a very nice-looking nav bar, which is actually just a very well-themed unordered list of links.

In particular, if you set the class of one of the list items to “active”, it ‘highlights’ it:

Note the highlight on 'Scalemate generator'

Now, I’m serving this site through Snap, the Haskell webdev framework; I’m doing this because one of my subsites needs to be able to upload files, and I figured I might as well use Haskell. And Snap comes with a templating language called Heist. So, given that I’m already using templating to insert the boilerplate jQuery/Bootstrap includes and the GitHub link/authorship at the bottom, wouldn’t it make sense to template the nav links?

It turns out that it’s fairly simple to do, as illustrated in NavLinks.hs. We start with imports; nothing too interesting here. We need Heist to access the templating engine and Text.XmlHtml to actually construct the nodes.

1
2
3
4
5
6
7
8
9
10
11
12
{-# LANGUAGE OverloadedStrings #-}

module NavLinks where

import qualified Data.ByteString as B
import qualified Data.Text as T
import           Data.Text.Encoding

import           Snap

import           Text.Templating.Heist
import qualified Text.XmlHtml as X

Next we declare the type:

17
navSplice :: MonadSnap m => [(B.ByteString, T.Text)] -> Splice m

The way a splice works is this: you construct a Splice in some monad Splice m, and you can lift monad actions in the m monad into Splice m. In this case, we can be in any monad that provides us with getRequest, since we want to get the URI of the request. Since getRequest :: MonadSnap m => Request, we can be in any monad that is an instance of MonadSnap. The argument to navSplice is the list of (path, title) tuples; for example, it might look like [("/about", "About"), ("/blog", "Blog")]. The path is a ByteString because the request gives us the URI as a ByteString, and the title is Text because Text.XmlHtml uses Text. Now, onto the actual declaration:

18
19
20
21
22
23
24
25
26
navSplice links = (:[]) . X.Element "ul" [("class", "nav")] <$> do
  currentURI <- lift $ rqURI <$> getRequest
  -- add the 'active' class if the href is a prefix of the current URI
  let li path
        | path `B.isPrefixOf` (currentURI `B.append` "/") =  X.Element "li" [("class", "active")]
        | otherwise = X.Element "li" []
  return $ map (\(path, title) -> li path [buildLink path title]) links
   -- build a link to the path with the given text
    where buildLink path title = X.Element "a" [("href", decodeUtf8 path)] [X.TextNode title]

Look at the do-block first. Line 19 just gets the current URI out. Lines 21-23 are interesting; what we’re doing here is declaring a function of type li :: B.ByteString -> [X.Node] -> X.Node. The way an HTML element is constructed in Text.XmlHtml is through the Element constructor, which has type T.Text -> [(T.Text, T.Text)] -> [X.Node] -> X.Node. The first Text is the tag; the array of Text tuples is the list of attributes, and the list of Nodes is the list of children. So, here we’re partially applying the tag (always “li”) and the attributes (which is either empty or class="active"). We apply the “active” class only if the path is a prefix of the current URI; we append a “/” to the end of the current URI to make sure that trailing slashes are ignored.

Next, look at line 26; buildLink just builds a link node looking like <a href="path">title</a>. X.TextNode is, obviously, the constructor for text Nodes.

On the last line of the do block, we map over the (path, title) tuples, build up a list of <li> elements, and return that list. Finally, on line 18 we construct an unordered list element with the “nav” class. Finally, we construct a single-element list using (:[]) (which some people call the monkey function), which just constructs a list with its argument inside. My actual code uses return instead, but I decided that this would be clearer.

Adding that splice to the default list of splices is as simple as adding

pages :: [(B.ByteString, T.Text)]
pages = [ ("/chargen/", "Character generator")
        , ("/logify/", "Chatlog formatter")
        , ("/scalemate/", "Scalemate generator")
        ]

to the toplevel of Site.hs and adding

addSplices [("nav", liftHeist $ navSplice pages)]

to the app initializer. Now <nav/> can be used in the template to automatically insert the nav link list and highlight the current page!

One small annoyance of this is that the generated HTML is all in one line, although since it’s not handwritten you probably won’t care so much, especially since Firebug and Chrome’s debugging tools will automatically do the indentation for you and let you collapse/expand child nodes at will.

Name Your Type Variables!

Haskell’s type system is commonly touted as one of the most powerful features of the language; it’s often said that if a program compiles, it’s pretty likely to be correct. And while there are always going to be errors your type system can’t catch (logic errors, off-by-one-errors, etc.), I’ve found that the type system helps ensure that your programs are put together correctly. If it typechecks, you’re probably not making a higher-level logic error; you might be trying to fit a red peg into a blue hole, but you’re not trying to fit a square peg into a round hole, so to speak.

The problem is that figuring out what the type variables in a polymorphic type (i.e., type like length :: [a] -> Int) mean. Sure, in simple cases it’s simple; you look at a type signature like ifM :: Monad m => m Bool -> m a -> m a -> m a and you know that m is going to be some monad and a is some arbitrary action. But take a look at validate :: Monad m => Form m i e v a -> Validator m e a -> Form m i e v a, from digestive-functors. What do m, i, e, v, and a stand for? It’s pretty clear that m is some monad, but what about the others? It turns out that:

  • i represents the input to the form, or its environment (i.e., the submitted parameters)
  • e represents the the error type that can be produced
  • v represents the ‘view’ of the form (how it’s represented to the user)
  • a represents the actual evaluation of a form (for example, in a registration form, it might be data Registration = Registration Text ByteString)

So my argument is that it might be better to describe it as Form m input err view result -> Validator m err result -> Form m input err view result; this way, it’s clearer exactly what each type variable corresponds to.

Again, I’m not saying that single-letter type variables should be completely banned; something like (>>=) :: (Monad m) => m a -> (a -> m b) -> m b is simple enough. The conventions that a and b stand for the to type variables in a generic function and that m means ‘some arbitrary monad’ are strong enough that it’s like using i in a for loop in Python or whatever.

(Full disclosure: yes, lots of my own Haskell code uses single-letter type variables in places where I shouldn’t. I do intend to fix this).

But if you’re trying to learn some new framework or module, imagine how much nicer it would be if the types that the Haddock gives you actually told you something about what it’s doing (especially given that most Haskell modules seem to be lightly documented outside of their Haddock documentation, lacking example use cases). Programmers outgrew using single-letter variable names decades ago; we shouldn’t make the same mistake in the type system. This is part of the reason why I’m making such slow progress learning the Snaplet framework in the dev version of snap; it’s hard for me to get an intuition as to what the types are when I keep seeing Handler b v a (b is the base app/snaplet, v is what’s being ‘focused on’, and a is what the Handler ‘evaluates to’; Handler b v has a Monad (and Functor, and MonadPlus, etc.) instance).

By the way, the Snaplet framework, though slightly complicated, does wind up making a good deal of sense once you get used to it; I plan on writing up a post about what the commonly-used types/type constructors 'mean', as well as a post about how to make a very simple login Snap app. And I think, ultimately, documentation is one of the things Haskell sorely needs; not just more documentation on what individual functions do, but how those functions combine into a larger whole. Imagine trying to learn Django just by looking at the module documentation without having the benefit of the tutorial! But I think that's a subject for another post; I'm trying to get this blog active again, after all.

Settings in Chrome Extension Content Scripts

Google Chrome, like most modern browsers, allows you to write extensions for it. And those extensions can inject JavaScript into pages that the end user views, so that you can write extensions to do things such as customize the interface of some site, theme it, etc. Simple extensions that don’t need to be configured by the user can get away with being simple Greasemonkey-like scripts that the user installs by navigating to and then forgets. The upside of this is that it’ll also work in Firefox, since Chrome’s native content script support is a strict subset of Greasemonkey’s (it doesn’t have @require or the ability to set/get data, among a few other things). The downside is that there’s no way to do any kind of updating short of your users manually installing a new version, and, more importantly, your extension cannot have any state or user configuration. In order to have stateful extensions, you have to go full-blown Chrome.

The basics of Chrome extension development are adequately covered by Google’s own Getting Started tutorial. The interesting part comes when you want your content scripts to be configurable by the user. For example, Tumblr Hate (which I’m the author of, and which has source that you can view here) adds ‘hate’ links to tumblr posts that allows the user to hide them. The user can configure the text of the ‘hate’ links, as well as what shows up when a post is ‘hated’ (i.e., hidden). Fortunately, Chrome has built-in support for Extension options pages, which have a localStorage object that you can use as a key/value store (although values are serialized to strings). Unfortunately, the localStorage object is not shared between your options page and your content scripts. I assume the reason for this is because of sandboxing, but it creates a problem: how do your extension’s content scripts get the settings?

Use chrome.extensions.sendRequest

You have your options page do whatever it does, saving settings to localStorage. You then create a ‘background’ page that installs a listener using chrome.extension.onEvent.addListener; the exact form of a request and how it should be responded to is obviously up to you, but here’s what I use: A request looks something like  {type: "get", name: ["settingName", "otherSettingName"]}, which is responded to with an object that looks something like {settingName: 1, otherSettingName: true}, or {type: "set", name: "settingName", value: "5"}, which sets the setting settingName to 5. You can even special-case get requests so that if the name is a single string, the response is just the value and not an object. If your extension does things other than inject content scripts, you’ll need other request types to do things like open tabs or whatever, but this is good enough for simply mimicing a key/value store. The code to do this is simple:

function(request, sender, sendResponse) {
    switch (request.type) {
    case "get":
        var data = {};
        if (request.name.constructor == Array) {
            for (var i = 0; i < request.name.length; i++) {
                data[request.name[i]] = localStorage[request.name[i]];
            sendResponse(data);
        } else {
            sendResponse(localStorage[request.name]);
        break;
    case "set":
        localStorage[request.name] = request.value
        break;
    default:
        console.log("invalid request " + request);
        break;
    }
}

The next issue, however, is trickier: chrome.extension.sendRequest is asynchronous. In particular, instead of returning the response, it accepts a callback that takes the response as an argument, and returns nothing useful. So what do you do?

Do your work in the callback

If you just use the callback to set some globals and then continue on, you have a race condition. Consider the following code:

var setting;
chrome.extension.sendRequest({type: "get", name: "settingName"}, function (resp) {
    setting = resp;
}
);
console.log(setting);

This will fail, because sendRequest is asynchronous, and sending a request and receiving the response very well might happen after the setting has been used! The correct thing to do here is this:

var setting;
chrome.extension.sendRequest({type: "get", name: "settingName"}, function (resp) {
    setting = resp;
    doStuff();
}
);

function doStuff() {
    console.log(setting);
}

Here, you can guarantee that doStuff won’t be called unless the setting has been properly set, so you can safely rely on its value.

Note that I didn’t pass setting as an argument to doStuff; I very well could have. But in the case of Tumblr Hate, for instance, I didn’t feel like threading the relevant variables through function calls, so I simply declared the variables at the top so they’d be global. (Well, no I didn’t, I set the response’s variables as properties of a global ‘root’ object, but that’s basically the same thing.)

Preserving Links Across Reorganization

Link rot is the scourge of the modern Internet. It’s incredibly frustrating to find a link to something that looks really interesting, or that looks like it might solve all of your problems, only to have the link be dead. Sometimes it’s because the content is just gone; there’s really no getting around that. But sometimes it’s because the site it was hosted on reorganized their structure in such a way that the data’s still there, but it can’t be reached at that old URL. If the site has a built-in search function, then you might be able to find it that way; you also might be able to use Google’s site:example.com domain restriction (or find it on another site that so graciously copied the entire text). But that still involves extra effort.

So what can you, as a site owner, do to minimize link rot? The answer is make sure that all your old URLs continue to function. Here’s an example: I recently moved all the files for this blog from /var/www to /var/www/blog, to keep my directory structure neat. I then mucked around in WordPress to tell it the new file location, set up /etc/apache2/sites-enabled/000-default properly, and everything was working fine. But the old incoming links were now broken! Fortunately, mod_rewrite exists and it’s fixable with a few simple rules in /var/www:

RewriteEngine On
RewriteBase /
RewriteRule ^((\d\d\d\d|tag|feed|category).+)$ /blog/$1 [R]
RewriteRule ^$ /blog [R]

The first two lines here are standard mod_rewrite boilerplate. The third and forth do the actual interesting rewriting: The regex will match any URL that starts with (after the amateurtopologist.com/) four digits or the words ‘tag’, ‘feed’, or ‘category’ and prepend /blog/ to them. So, for example, http://www.amateurtopologist.com/2010/12/01/some-useful-approximations/ will get rewritten into http://www.amateurtopologist.com/blog/2010/12/01/some-useful-approximations/; both of those URLs do work and will send you to the same post. The [R] indicates that Apache should serve a redirect instead of transparently rewriting it; I did this so that people would use the new URLs in case I ever have to actually break the old ones. Finally, the last line rewrites the empty request into /blog (again using redirect).

The net result here is that pretty much every single URL that used to work will still work, but I can now keep my blog and my other stuff in separate directories. A few URLs broke because I’m too lazy to fix them (search URLs and the about page), but nothing that people were likely to actually link to.

Off scripts and onto my own machine

For the past year or so, I’ve been hosting this blog on scripts, a free webhosting service provided by MIT’s Student Information Processing Board for MIT students and other account-holders. And it was good; I didn’t have any complaints with the performance and the reliability was excellent for what I paid for it ($0) and for what the maintainers were paid to maintain it ($0). But recently I managed to get together enough spare parts to assemble a server, and so I’ve decided to move off of scripts and onto my own hosting. The reliability might suffer a little; it’s just one machine sitting in my dorm room, not a cluster of machines in an actual machine room. But I feel like the experience of running my own webserver is a good one to have, and it’s not like this website is mission-critical to anything anyway. If all else fails I can always transfer it back.

 

Also, yes, I know I haven’t posted anything since late January. I’m going to try to fix that.

The Desktop of Theseus

I recently decided to upgrade my aging desktop. First on the list was a better CPU; I have a rather unfortunate stepping of the AMD Phenom 9600 that has a bug that reduces performance by 10-20%, with the only fix causing the system to become unstable. So I got a nice Athlon II, but the problem was that my (mostly stock) computer has an AM2 motherboard. So I had to replace that. Then I decided to get an ATX, since I don’t like the small size of mATX stuff. Which meant I needed a bigger case. And then I figured that 3GB is positively shameful for a desktop (or even a laptop) in 2011, so I got 4 GB RAM, which I’ll use along with the 2 1GB sticks I have already (and it’s a good thing that I did, because I forgot that DDR2 and DDR3 aren’t compatible). So the only things left from the current computer will be the power supply, the hard drive, the Blu-Ray drive, and the graphics card.

This naturally raises the question: is it the same computer any more? This is the paradox the title refers to, the so-called ship of Theseus, which had every plank replaced. It’s about one-third old parts, two-third new parts. And so I kind of want to say that it’s the same computer, because I’m moving over the hard drive, and therefore the data. But does that mean if the hard drive in a computer fails, and you replace it, it is now a different computer? That’s absurd. I don’t think that there’s any well-defined answers here; I didn’t reinstall Windows or anything and I kept the hostname and all the other interesting state. Me, personally, I view it as the same computer on some base level, but on a more intellectual level, I have no idea. I’d say it’s the same computer, though, because it serves the same ‘function’ and uses parts from the old one. If I just bought an entirely new stock computer and threw this one out, it’d be different. Similarly, when I replace the hard drive and graphics card, it’ll still be the same computer, even though it won’t share any parts with the original.

On a mostly unrelated note, assembling a computer is a very good way to at least learn how stuff is actually physically connected; some things draw power from the motherboard, some connect directly to the power supply, etc. It’s also cheaper than buying stock, and it means you’ll have an easier time upgrading parts when/if you decide to do so. The only downside is that you probably won’t get a free copy of Windows, and you can’t get a warranty. But your individual parts are probably warrantied, and if one of them fails it’s not too hard to determine what failed; clicking noise means a failed drive, memtest86 will tell you if your RAM is bad, etc. Plus it’s honestly more satisfying using a computer that you assembled.

Not all countably infinite sets are equal

This post was inspired by an interesting post I read called ‘Seemingly impossible functional programs‘, which is about how to write programs that search infinite spaces (elements of what I call F(N)) in finite time.

If you’re familiar at all with set theory and cardinal numbers, you know that 2^{|S|} > |S| for any set S, a result known as Cantor’s theorem. Now, one way to think of elements of 2^S is as sets of elements of S. But another way to think about them is functions over S that either return true or false; the function returns true just for the elements in the set. So 2^S is really the set of functions from S to \{0, 1\}.

But not every function is computable! The busy beaver is an example; there is no algorithm to compute the busy beaver function of a given integer n. So what happens if we only consider computable functions? Well, let F(S) be the set of computable functions over S. Then F(S) is always countable; a simple way to see this is to consider the set of valid programs in whatever methodology you’re using to describe your functions. Each of these programs is describable in finite characters, and there’s only a countable number of finite strings of characters from a finite alphabet.

Now, here’s the interesting part. In particular, this means that F(N) is countable, and so there is a bijection between F(N) and N. But can we compute the bijection? Suppose we could, and let f_i be the function corresponding to i. Then consider f' such that f'(x)=1-f_x(x). Then obviously f' \neq f_i, and so our enumeration has failed. So there is no computable bijection between F(N) and N!

Finally, consider F(F(N)). We can visualize each element of this set as a tree whose leaves have value 0 or 1; to compute g(f) where f \in F(N), at each node of depth d, we take the left path if f(d) is false, right path if it’s true. But then each tree has to be finite, because otherwise we could construct an f that would take that infinite path (proving that said f is computable is left as an exercise to the reader, unfortunately.). And there are only countably many finite binary trees, and the bijection here is computable, since there are finitely many trees of depth d for any given d, and we can just assign numbers to the various d-depth tree in whatever order we feel like. So F(F(N)) has a computable bijection with N, even though F(N) doesn’t despite being countable.

Handling modifier keypresses in Cocoa

One of the features of Snow Leopard is the addGlobalMonitorForEventsMatchingMask:handler: method, which lets you handle events in other programs. This requires the user to have enabled accessibility in System Preferences, or for the program to be running as root and grant itself trusted status, but it lets you install a handler that will be called whenever any event that matches the relevant mask occurs outside of your application (for events inside your application, use addLocalMonitorForEventsMatchingMask:handler:. One of the things that you can capture is the pressing of a modifier key, such as Cmd or Option without a corresponding ‘real’ key. If your mask includes NSFlagsChanged, then any press or release of a modifier key will send your handler an NSEvent that you can call modifierFlags on to get an NSUInteger corresponding to which modifier keys were actually pressed. But there’s a problem here: if you hold down Command and repeatedly press the Shift key, your handler will be called for both the keypress and the key release. So if you’re counting the number of presses of each modifier, you’ll count the Command key again each time you press or release the Shift key, which is probably not what you want!

The solution is to use a block. A block is like a closure in other languages; we can use it to create a function that tracks a bit of state around with it. In this case, we want our flag handler function to track what modifier flags were present the last time it was called; if we compare the set of flags in the new invocation to those from the old invocation, we can determine what changed, and therefore what actually happened:

__block NSUInteger lastFlags = 0;
void (^flagTrigger)(NSEvent*) = Block_copy(^ (NSEvent *event) {
    NSUInteger flags = [event modifierFlags];
    NSUInteger masked = flags & ~lastFlags;
    lastFlags = flags;
    // flags now only includes keys that were actually pressed
    // handle those keys here
};

[NSEvent addGlobalMonitorForEventsMatchingMask:(NSFlagsChangedMask) handler:flagTrigger];

Here, we declare a lastFlags NSUInteger that we’ll use to store the previous set of pressed modifier flags; by declaring it using __block, we note that the block is allowed to modify it. We then create a block flagTrigger, whose type states that it takes an NSEvent* and returns void, and which handles the logic: when it’s called, it gets the modifiers in the event, and applies a mask consisting of just the flags that weren’t already set last time. We then store the flags variable into lastFlags for the next time, and then process the masked flags. Finally, once we’ve declared the block, we install it as the handler.

The code above is basically copied wholesale from a project I’m working on called KeyboardViz, specifically the Keyboard class which displays an onscreen keyboard that changes colors as you press keys. I’m using it to learn ObjC and Cocoa, so suggestions on coding style are welcome.

The Future of Minecraft

Minecraft is a game that needs no introduction; the developer has made over a million dollars pre-tax from sales of a $13 game without any support from programming studios. The game has no predefined goals, no scoring system worth talking about; the only things that you can do are mine for resources, defend yourself from monsters, build, and explore. At over 600,000 copies apiece, the developer has made almost 7 million dollars by himself, and has recently started a company and hired some people to help work on it. The community is generally optimistic, as is the lead developer.

But where does Minecraft go from here? While the game is still selling at a decent pace, it’s inevitable that sales will taper off, if only because there’s only so many people in the world that are interested in playing it. And while the constant updates are enough to keep the sales going for the moment, it’s the nature of things that they’ll have to drop off eventually. Furthermore, Notch’s attitude towards fixing bugs has been criticized; the latest update caused a bug in the multiplayer mode where sometimes blocks would not be ‘mined’ even though the tool quality would degrade, causing mining to take longer. This should be a simple bugfix, as it was caused by a regression, but Notch has not pushed out an emergency update.

On the other hand, the fact that Minecraft is a Java game makes it amenable to reverse-engineering; there are dozens of mods that change everything from adding ambient occlusion to the lighting to changing the world generation mechanism to turning gold blocks into minecart boosters. Previously, Notch has stated that he “frowns upon” client-side mods; however, he’s since relented his stance, and he has stated that he intends to add a modding API.

In the time between when I started writing this post and when I finished it, Notch set up a bug tracker. This is, if nothing else, a great first step; it gives him a centralized place to look at bug reports/feature requests, and it makes the game look more professional. The fact that he intends to make updates optional, as opposed to forcibly pushing out potentially buggy versions, is also a good sign. Chrome can get away with it due to the fact that they have testers; Minecraft doesn’t (yet?).

So where do I think Minecraft is going to go? I have no clue. I’m somewhat pessimistic due to the diminishing sales problem as well as rumors that Notch is going to start work on another project, as well as the fear that he’s going to develop Toady One syndrome and start half-implementing a bunch of features. On the other hand, he has hired other people to work on the codebase, and it’s not like the game will completely die even if he stops work. So I’m cautiously optimistic.

Some useful approximations

As much as I hate to admit it, mathematicians tend to deal with approximations. A lot of times, formulas are just too complicated to deal with the full complicated formula, and you have to simplify it. So here’s some handy approximations, as well as about where they’re valid.

  • \log (1+x) \approx x, e^x \approx 1+x for x \ll 1
  • (1+x)^{n} \approx 1+nx for nx \ll 1, and in particular \sqrt{1+x} \approx 1+x/2 and (1+x)^{-1} \approx 1-x
  • \sin x \approx x, \cos x \approx 1 for x \ll 1
  • (1+\frac{a}{n})^{bn} \approx e^{ab} for n \gg a,b, especially for a,b < 1
  • n! \approx \sqrt{2 \pi n} \left(\frac{n}{e}\right)^n
  • H_n \approx \ln n + \gamma, where \gamma \approx 0.577 is the Euler-Mascheroni constant
  • \pi(n) \approx n / \ln n, where \pi(n) is the number of primes less than n
  • p(n) \approx n \ln n, where p(n) is the nth prime number.
  • 2^{10} \approx 10^3

What do you think every mathematician should know?