tumblilarius lukhnos

"Currying" Objective-C Methods

Wed, 08 Feb 2012 10:15:00 -0800

In this blog post, I’ll show you a way to “curry” Objective-C methods. I’ll first talk about the problem I’m trying to solve, what currying is, and how currying can help solve the problem. Then I’ll discuss how you can use blocks to achieve currying and the issue with that approach. Next, I’ll present a library that can easily “curry” Objective-C methods of your choice. You have probably noticed I put the term in quotes, and I’ll briefly talk about why the approach I present here is not the real currying in a strict sense. I’ll also discuss some other issues in introducing the notion of currying in an imperative, object-oriented programming language like Objective-C.

Please note that the library is written for study purposes, and I didn’t consider its practicality or performance. Nor is it intended for production use.

If you want to dive right into the code, here’s the source, along with some sample showing you how to use it.

Introduction

Let’s say we have a method in SomeClass:

    - (void)doThis:(id)a withThat:(id)b andThat:(id)c;

If we refer only to the method and want to emphasize its class membership, we can also write it as -[SomeClass doThis:withThat:andThat:], although that does not reveal the type information for the parameters and return value.

Then one day you find that, in your project, you have quite a few places that look like this:

    SomeClass *foo = [[SomeClass alloc] init];
    [foo doThis:a withThat:b andThat:c];
    [foo doThis:a withThat:b andThat:d];
    [foo doThis:a withThat:b andThat:e];
    [foo doThis:a withThat:b andThat:f];

The method takes three arguments, but we are always passing the same a and b here. Would it be nice if there’s some way to remember those, so that we only need to pass the argument that actually changes, namely, c, d, e, and f, like this:

    [someMagicFoo blahAndThat:c];
    [someMagicFoo blahAndThat:d];
    [someMagicFoo blahAndThat:e];
    [someMagicFoo blahAndThat:f];

It turns out that there are quite a few possible solutions to this problem. Before I discuss them, let me first introduce a concept called currying.

Currying

Let’s take the C function for example:

    int f(int x, int y)
    {
        return x + y;
    }

Then say you want to do these computations:

    int z;
    z = f(5, 10);
    z = f(5, 20);
    z = f(5, 30);

Again, we see that we’re always passing 5 for x. One way to save that effort is to write another function:

    int g(int y)
    {
        return f(5, y);
    }

    int z;
    z = g(10);
    z = g(20);
    z = g(30);

In other words, we want to “fix” the first argument of f and we only want to pass the second argument to get the function value. This is called “partial application”, as we only want to apply the function to a smaller set of arguments.

Wouldn’t it be nice if, instead of writing g ourselves, the compiler can help us write that, using the hypothetical construction below:

    int (*g)(int) = fix(f, 5);  // hypothetical construction
    int z;
    z = g(10);
    // ...

The construction above is actually not hypothetical, although it’s not available in the C language. The transformation that allows such construction is called currying.

In C, we declare the prototype of function f as int f(int, int), meaning it’s a function that has two integer parameters and returns an integer value. It can also be written formally as:

    f: int × int -> int

We can also write it as:

    f: (int, int) -> int

Which reads “f is a function that maps a pair of integers to an integer”. The two forms are equivalent, as n-tuples (a pair is a 2-tuple) can be used to represent set cross products.

Now let’s talk about currying with a concrete example. We can “curry a function”, in the sense that we pass a function to a machinery called curry, and it will give us a new function (think of it as someone who writes a new function for us).

So, if we curry f (i.e. passing f to curry), then we get a new function with the following type:

    curry f: int -> (int -> int)

That looks bizarre, so let me explain. The above line reads “curry f (the returned function of calling curry with f) is a function that, when passed an integer, returns a function that takes an integer and returns an integer”.

Sounds familiar?

It should! Remember the C example we just mentioned. Let’s suppose if the C language supports currying, and we can actually obtain a function from that:

    // we need this bizarre type to declare h
    typedef ...SomeBizarreType...;

    SomeBizarreType h = curry(f);

Then, if we call h(5), it will give us the function g that we want!

    int (*g)(int) = h(5);
    int z;
    z = g(10);
    z = g(20);
    z = g(30);

curry is actually a function that takes a form like this:

    curry: ((a, b) -> c) -> (a -> (b -> c))

It can be a bit cumbersome to read out the definition above. Here’s a way to see this: if you call curry(f), it returns function h to you. Then if you call h(5), it returns our good friend g. Here, f is the function of type (a, b) -> c, and h is (a -> (b -> c)), and g is (b -> c). Those parentheses are added to disambiguate, but rest assured that there are many people who can do without them.

This transformation technique is named after Haskell Curry (1900-1982), a mathematician (yes, the language is named after him), who, among other things, discovered a fixed-point combinator called Y combinator (yes, that firm is named after it), a construct which can express recursive functions.

Also note that curry is not confined to functions that have two parameters (i.e. have a pair of parameters):

    curry: ((t_0_, t_1, ..., t_{n-1}) -> t_n) ->
            (t_0 -> (t_1 -> ( ... -> (t_{n-1} -> t_n))))

And there’s uncurry that is the inverse of curry:

    uncurry: (t_0 -> (t_1 -> ( ... -> (t_{n-1} -> t_n)))) ->
            ((t_0_, t_1, ..., t_{n-1}) -> t_n)

Now that we know what currying is, let’s see how it can help us find a solution to the problem we have in the beginning.

Using Objective-C Blocks for Currying

It turns out that there isn’t really a way to write a curry function in C. The problem is that in C, functions are not first-class objects, meaning you can’t manipulate functions like you do data structures. Unlike, say, in Haskell, where curry is a function you can actually use.

Then, there’s a practical issue. Recall the example of f and g above. If we always want to fix the x parameter with a constant, then for each constant we need to write a new g. What if we don’t want to be confined to a constant?

    int g(int y)
    {
        // a is an argument we want to fix
        return f(a, y);
    }

Now the problem is that g has to “remember” what a is. You don’t want to use global variable for that:

    // suppose a is a global variable and a = 5 initially
    z = g(10);  // 15
    z = g(20);  // 20
    foo();      // a gets changed to 0
    z = g(10);  // ugh! it's 10 now

In the example above, we want to have a g that “remembers” to pass 5 for x to f. Using a global variable ruins that.

A construct to “remember” something happening in your program is called closure. A closure “remembers” the local states when it’s created.

In Objective-C, blocks are closures. We can actually use blocks to write the h function (compilable code here):

    typedef int (^int_to_int_t)(int);
    typedef int_to_int_t (^int_to_int_to_int_t)(int);

    int_to_int_to_int_t h = ^(int x) {
        int_to_int_t fx = ^(int y) {
            return f(x, y);  
        };

        return fx;
    };

    int_to_int_t g = h(5);

    int z;
    z = g(10);

So, when the block fx is created, it will “remember” the x argument that’s passed to h. You also noticed that we need to use some typedefs here.

I’m declaring h as a local block because I don’t want to dive into the block copying issue here, which is another complex matter that requires an entire blog post. Just remember that you should return an autoreleased copy of the block if you want to pass it around (yes, I’m essentially telling you that I’m doing it wrong here for the sake of brevity).

In fact, we can go one step further, then we’ll have something that’s very close to the curry function, by passing the function we want to curry using a function pointer (compilable code here):

    // typedefs for int->int, int->(int->int), etc.
    typedef int (^int_to_int_t)(int);
    typedef int_to_int_t (^int_to_int_to_int_t)(int);
    typedef int (*int_x_int_to_int_t)(int, int);
    typedef int_to_int_to_int_t (^curry_t)(int_x_int_to_int_t);

    curry_t curry = ^(int_x_int_to_int_t f) {
        int_to_int_to_int_t h = ^(int x) {
            int_to_int_t g = ^(int y) {
                return f(x, y);  
            };
            return g;        
        };
        return h;
    };

    int_to_int_to_int_t h = curry(f);
    int_to_int_t g = h(5);

    int z;
    z = g(10);

That’s a lot of declarations, not pretty, but it’s something very close to the real thing.

What about Objective-C Methods?

Now that we know what currying is, and how Objective-C blocks can give us some kind of “curried” functions, let’s see how they can help when it comes to Objective-C methods. Recall our first example:

    SomeClass *foo = [[SomeClass alloc] init];
    [foo doThis:a withThat:b andThat:c];
    [foo doThis:a withThat:b andThat:d];
    [foo doThis:a withThat:b andThat:e];
    [foo doThis:a withThat:b andThat:f];

If we want to fix foo, a, and b, we can actually write a simple block like this:

    void (^fooABAndThat)(id) = ^(id x) {
        [foo doThis:a withThat:b andThat:x];
    };

    fooABAndThat(c);
    fooABAndThat(d);
    // ...

If you want to have to flexibility to just fix a then fix b or some other arguments later, however, then you’ll end up with something as messy as the previous “curry” function we have. Namely, if you want to do something like this (provided we always fix foo, to make things simpler):

    ((fooDoThis(a))(b))(c);

Then you have to write something like:

    typedef void (^id_to_void)(id);
    typedef id_to_void (^id_to_id_to_void)(id);
    typedef id_to_id_to_void (^id_to_id_to_id_to_void)(id);
    id_to_id_to_id_to_void fooDoThis = ^(id x) {
        id_to_id_to_void withThat = ^(id y) {
            id_to_void andThat = ^(id z) {
                [foo doThis:x withThat:y andThat:z];
            };
            return andThat;
        };
        return withThat;
    };

    ((fooDoThis(a))(b))(c);

Apparently, that’s not very fun to write.

As I mentioned before, in C, functions are not first class objects. That you have to declare a type for everything doesn’t help, either (hence the messy typedefs above). Unless we have a fancy compiler or language extension that writes those declarations implicitly and automatically for us, this seems to be as good as it can get with blocks.

But wait. Objective-C is a dynamic language. Its runtime API allows us to create classes and methods on the fly. Before the introduction of blocks, a lot of language have actually been built by making use of that property. Is there an easier way to “curry” an Objective-C method, so that we can do something like this?

    [someMagicFoo blahAndThat:c];
    [someMagicFoo blahAndThat:d];
    [someMagicFoo blahAndThat:e];
    [someMagicFoo blahAndThat:f];

With ObjCurry, we can do that.

Using Objective-C Runtime API to “Curry” Methods

ObjCurry is an Objective-C library that allows you to “curry” a wide range of Objective-C methods. Once included in a project, you can do this:

    [SomeClass curry:@selector(doThis:withThat:andThat:)];

Then, you can do things like:

    id fooDoWithA = [foo doThis:a];
    id thenB1 = [foo withThat:b1];
    id thenB2 = [foo withThat:b2];

    // equivalent to [foo doThis:a withThat:b1 andThat:c]
    [thenB1 andThat:c];

    // equivalent to [foo doThis:a withThat:b1 andThat:c]
    [thenB2 andThat:c];

    id someMagicFoo = [[foo doThis:a] withThat:b];
    [someMagicFoo blahAndThat:c];
    [someMagicFoo blahAndThat:d];
    [someMagicFoo blahAndThat:e];
    [someMagicFoo blahAndThat:f];

And, of course, we can curry methods in a Foundation class. For example:

    [[NSMutableDictionary class] curry:@selector(setObject:forKey:)];

Then, say, we have a dictionary, and we want to be able to set the same value to different keys:

    NSMutableDictionary *dict = ...;    
    id dictSetBarWithKey = [dict setObject:@"bar"];
    [dictSetBarWithKey forKey:@"a"];
    [dictSetBarWithKey forKey:@"b"];
    [dictSetBarWithKey forKey:@"c"];

    // dict is now {"a":"bar", "b":"bar", "c":"bar"}

Under the Hood

ObjCurry uses Objective-C runtime API to dynamically create new classes and methods. In the previous example, after you call:

    [SomeClass curry:@selector(doThis:withThat:andThat:)];

A new instance method doThis: will be added to SomeClass. Then, when you call:

    id fooDoWithA = [foo doThis:a];

The returned object, fooDoWithA, is actually a proxy object whose class implements withThat:. That method, in turn, will return another proxy object, whose class (currently the same as the previous proxy object’s) implements andThat:. The last method in the chain, andThat:, will actually send the message doThis:withThat:andThat: to foo, along with the captured three arguments.

The implementation uses NSInvocation, which is the working horse for many Objective-C techniques often collectively called “higher-order messaging”, to capture the arguments along the way. Proxy objects are also often used in tandem with NSInvocation. One of the best example is NSUndoManager, which can give you a proxy object for undoing. Any message you send (i.e. and method invocation) to that proxy object is recorded but not executed. The real invocation is deferred until the user chooses to undo, then the recorded invocation is “played back” (i.e. the real method on the target object is called).

Because methods have to be typed, ObjCurry only supports a limited number of common types, such as id, char, (un)signed short/int/long/long long, and generic pointer (void *). For return types, there is one more supported: void.

It’s possible to add more types — see NSObject+Curry.m to see how to expand the type support. For example, you can add CGRect/NSRect support. The problem is that, unless deeper runtime API hack is found (perhaps some assembly required), for n argument types and m return types, we have to write nm* definitions. Even with the current macro usage it still won’t scale well. If you have better solution, I’d be very happy to hear from you.

Issues and Extensions

You probably noticed that when I say we want a way to “curry” Objective-C methods, I always put the term in quotes.

The reason is that ObjCurry is not the real thing. To understand why, we have to first know how is message sending done in Objective-C.

Let’s say we have an Objective-C instance method of class Foo that adds two NSNumber objects:

    - (NSNumber *)add:(NSNumber *)x to:(NSNumber *)y;

When we call:

    NSNumber *z = [foo add:x to:y];

The compiler actually turns that into a function call:

    z = objc_msgSend(foo, @selector(add:to:), x, y);

(The real code is of course optimized, but this is what the compiler would do from a high-level point of view.)

And the method -[Foo add:to:] is compiled as something like the following (again this is just a sketch). Remember that in Objective-C there is actually only one object type, hence the use of type id all over:

    id Foo_add_to(id obj, SEL selector, id x, id y)
    {
        // ...
    }

Let’s ignore the selector parameter to simplify the matter. Obviously, a method is like a function, but it needs to take an object parameter so that it knows which object to act on. In other words, our Foo_add_to function actually has the type:

    Foo_add_to: id × id × id -> id

Ideally, a curry function should give us something like:

    curry Foo_add_to: id -> (id -> (id -> id))

But because ObjCurry introduces proxy objects in the picture, what we actually have is more like this:

    "curried" Foo_add_to: (id × id) -> (id × id -> id)

That’s why ours is not a real thing in the strict sense.

Another issue that comes with using ObjCurry is that Objective-C compilers give you warnings on previously undeclared methods. This is a nuisance, although there is a workaround. In the main.m that demonstrates how the use ObjCurry, I put a few Objective-C categories that declares the methods as a result of currying. This silences the compiler warnings, but it can be a debugging hazard when you invoke a method that’s not actually created, since the compiler now trusts you that you have an implementation somewhere, even if not statically available at compile or link time.

In addition, there’s the issue of mutability. Although proxies behave like immutable objects (as each time a curried method is called, a new proxy object is created), the target object (i.e. the original object on which the original method is invoked) is not. So while the proxies create an illusion of closures, you are still dealing with an imperative language. So if you intend to try ObjCurry in a multithreaded system, you have been warned.

With the basics in place, it’s possible to extend the fun. For example, each proxy object can actually remember the parameters that are ever passed to them and the returned objects. So if the parameter doesn’t change, the proxy can always returned the “memorized” return value. This is known as “memoization” (some people say it’s just a fancy term for “caching”). Again, in an imperative language like Objective-C, there are all kinds of issues that come with this.

The use of proxy object also brings some overhead and will have some impact on performance in places such as a tight loop. NSInvocation is slow compared to the straightforward message sending, considering the overhead in creating invocation objects and all the house keeping efforts.

Finally, because this is not a feature supported by the compiler, debugging can be tricky if something happened inside the proxy object (especially in the last step that actually invokes the original method).

Related Works

A working paper written by Laurent Dami in 1991, “More Functional Reusability in C/C++/Objective-c [sic] with Curried Functions” is an oft-cited work regarding currying in Objective-C.
If you know Haskell, you probably have read the documentation on currying and partial application.
Wikipedia’s entry on currying also explains it in detail (and it also links to Dami’s paper).
Josh Weinberg has an experimental implementation of currying and uncurrying that is written in Objective-C++ and uses templates.
There are also some discussions on Stack Overflow (on currying in C++ and in C).
CocoaDev Wiki has collected some of the higher-order messaging techniques in Objective-C.
David Chisnall’s Advanced Flow Control for Objective-C is a nice summary of the “higher-order messaging” techniques and their applications. Among other things, it talks about the possibility of using proxy objects to achieve something like curried functions. It’s more than the entries on CocoaDev Wiki recent since it also discusses blocks.
Finally, Mike Ash discusses block proxying and building a memoizing block proxy using NSInvocation and blocks in great detail.

Conclusion

The idea of currying is actually very straightforward, but implementing it with a non-functional programming language incurs many issues, some of which I have discussed above. We have seen how Objective-C blocks can be used to curry functions, and I’ve also introduced the library ObjCurry to “curry” Objective-C methods using the Objective-C runtime API. As I said in the beginning, the project is more of a study and does not consider performance issues. Still, we have in effect created a small language extension of our own. There are still other issues with the implementation, which I’ve also briefly addressed. As Curry is an open source project, you can try it out yourself. And you know a way to generate automatically IMPs for non-id, non-primitive data types such as NSRect/CGRect, I’d be more happy to hear from you.

Krzysztof Kieślowski on Crafting an Experience

Sat, 31 Dec 2011 09:59:00 -0800

I recently re-watched the Three Colors trilogy by the late Polish director Krzysztof Kieślowski (1941-1996). I first watched the three films when I was in college. I think I understood more and saw more in the films than I did when I was younger. Growing up, having traveled, and some knowledge of the history and one of the languages spoken in the film (French) all helped. But even back then I was deeply touched by the films the first time I watched them.

I also appreciated more Kieślowski’s mastery of the film language.

The DVD edition of the trilogy comes with an extra feature called “Krzysztof Kieślowski’s Cinema Lesson”, in which the director talked about the making of one significant scenes in each of the trilogy. In particular, he talked about the filming process and the editing choices involved. This is the auteur dissecting his own work in details.

In Blue, for example, there’s a famous “sugar cube scene”, starting at 52′20″ and lasting about five seconds. Julie, the heroine, is found sitting in a café by Olivier. Julie has recently lost her husband and daughter in a car accident, and she tries hard to leave everything in her life behind. Olivier, who has worked with her husband on a song commissioned by the European Union, becomes a lover and puts an effort in finding her. Julie turns him down in the café. After he leaves, she plays with the sugar cube that comes with her coffee, while a flute player on the street is playing a number that sounds very similar to one of the motives in the commissioned (but yet finished) song.

Here’s Kieślowski talking about how he required the cube to be soaked in coffee in exactly five seconds, and the rationale behind it. Emphasis mine:

This is a sugar cube about to fall in the cup of coffee. What does this obsession with close-ups mean? Simply that we’re trying to show the heroine’s world from her point of view, to show that she sees these little things, things that are near her, by focusing on them, in order to demonstrate that the rest doesn’t matter to her.

It seems easy to film a sugar cube soaking up coffee, sucking it up and turning brown. […] We can start a stopwatch. It should take five and a half seconds or five seconds to be completely soaked. How to make sure that it only takes five seconds? Not so easy. Let’s take a regular sugar cube, like this one, and soak it in my coffee. I’ll start the stopwatch… Eight seconds. That’s three seconds too long. We had to prepare one that would be soaked in five seconds. We decided such a detail shouldn’t last more than five seconds. For half a day, my assistant tested all kinds of sugar cubes, […]

What do we care about a stupid sugar cube sucking up some stupid coffee? Nothing… unless we are, for a moment, in our heroine’s world. She dips a sugar cube in her coffee and focuses on it to reject the offer that the man who loves her just made her. […]

And when you ask me if I think about the viewer, about the viewer’s point of view — I go back to this stupid sugar cube — I try always to keep this in mind. We don’t do previews, test screenings. I believe an audience can take four and a half seconds of sugar soaking, but that eight and a half would be too much.

Cirque du Soleil and Radio Canada International

Sun, 13 Nov 2011 01:52:00 -0800

In the mid-1990s I owned a shortwave radio. It was one of the top-tier models made by Sangean, although I forgot the model number. That was before Internet radio came along, and you could still write to shortwave radio stations, with an attached international reply coupon, to request for their schedule and publications. So I wrote to Radio Canada International (RCI) to ask for their schedule and a booklet on how to install shortwave antennae. Sangean’s manual listed the addresses of the radio stations you could write to.

Because the city is in a basin, shortwave reception in Taipei wasn’t great, so an external antenna made sense. I bought one coupon from a stamp collecting store (that’s the one and only time I did that in my life), sent the letter, and a few weeks later, I received the schedule (which got updated every 6 months — because the atmospheric condition varies by season) and a guide to install various kinds of antennae. My father helped me install a long-wire antenna, the simplest type, on the roof of our apartment building (he also told me how shortwave radio was a restricted item in the 1950s in Taiwan, as the government didn’t want people to have too much access to the information of the outside world). We calculated the needed length and got the insulation right, dropped the connecting cable to my window, and connected to the radio’s external antenna socket. In addition to RCI, we were also able to listen to BBC World Service, Voice of America, Deutsche Welle, Radio Japan (NHK) and a few others with different reception qualities, a great supplement (my father would have said alternative) to ICRT, Taipei’s local English station.

RCI’s broadcast time in East Asia was short, 30 minutes in French and then 30 minutes in English, but the timing was perfect. The French broadcast started at 6 AM Taipei time, then followed by the English broadcast at 6:30. My Sangean radio had both a timer and tape recorder, so I let RCI wake me up every morning, then later on the bus I would play the English news and programs in my Aiwa cassette player. That was a great way to wake up and learn English.

Before a shortwave broadcast begins, the station plays a short sequence to identify itself, called “ident.” The ident of BBC World Service, “This is London calling,” is among the best known (check out this YouTube video, the version I heard is between 1:01 and 1:30). RCI, unsurprisingly, used the first bar of “O Canada,” the national anthem, followed by the bilingual announcement “Radio Canada International / Radio Canada International” (yes, one of them was in French…), and the sequence repeated itself two more times.

For me, one mystery about RCI’s broadcast, at least in East Asia, was that before the ident, they actually played three minutes of a song, in a language that was definitely neither French or English. I hummed the tune to many people, but no one I asked knew what it was. It was also strange, too, because not many shortwave stations did that, a full three minutes of a song before the ident.

I usually erased the tape on which I recorded each morning’s program, but one day I decided to save a tape for the opening song. A few years later I converted the tape into MP3, played the clip to a few more people, but I still hadn’t got any answer. And then I forgot the whole thing.

Just today, out of the blue, I realized I still kept the MP3 file, and now I could use services like Shazam to identify what the song was. And so I did. Turns out that the opening song was “Kumbalawe,” the opening song of a show by Cirque du Soleil. It totally made sense now, as they are Canadian (from Montréal). The song uses an invented language, though. Isn’t it amazing, that with a technology like music identification, we can find out the name of a song whose melody you never forget, after so many years?

On the other hand, I guess I could have written to RCI long ago to ask about the song. Still, it was a happy rediscovery of a song that made so many wonderful early mornings.

Permanently Convert WordPress Blogs to Static Pages

Mon, 07 Nov 2011 01:49:00 -0800

Since I started using Tumblr to host my blogs, I no longer used WordPress. It’s a great blog software, but even if you stopped writing blogs with it, you still need to run its software update from time to time for security. Because of that, plus other various personal reasons, I decided to convert all the WordPress blogs that I hosted on my own to static pages.

After some research online, the best solution I found for this task is Ammon Shepherd’s WPStatic utility. Shepherd wrote extensively about his project (usage, origin, more details). The project has been around for a while, and the version hosted on github works with the latest WordPress. Here’s a quick summary of what I do to “staticize” my blogs.

First, you need to know what WPStatic does:

It backs up your WordPress database (as a MySQL dump). It actually makes two backups: One before it makes the changes to your database, and one after.
It changes your blog’s permalink structure and close all the comments (for both posts and pages)
It runs wget to fetch every static pages linkable from the top page
It archives the MySQL dump, the static pages, and the files in wp-content into a .zip file

The script is only one file, called wpstatic. Just place the file in your blog root, then run chmod +x wpstatic to make it executable, and then run ./wpstatic -h for help. Some of the above actions, such as permalink structure modification, can be toggled off.

Then, here’s what I did:

I ran some of the backups on Mac OS X and some on Linux. If you run the script on Linux, you need to change nothing. If you run the script on Mac OS X, you need to do a global replace to change every occurrence of sed -r to sed -E (that’s because the sed that comes with Mac OS X comes from BSD).
Run ./wpstatic but don’t proceed with the wget backup (the actual page fetching). I ran this step to let the script close all the comments for me.
Now this step is important. By default, the script changes your blog’s permalink strcuture to something like http://example.com/year/month/day/post-name. This didn’t work for me since some of my blog posts contain CJK titles, and CJK filenames work badly (a tragedy compounded by both URL encoding and file systems). Plus I have been using the structure like http://example.com/archives/123 for many years, and Google searchs on my blogs point to URLs like that. So, after the script changed the permalink structure, I had to go back to the admin page again, first change the permalink back to /archives/%post_id%, then I added a / at the end. The final permalink structure became /archives/%post_id%/, and this is great, because this actually caused the pages to be saved into something like /archives/123/index.html.

Modifiy wp-includes/default-filters.php and comment out all actions that generate feed links. I turned off (commented out) these lines. You might want to remove more:

add_action('wp_head','feed_links',2);
add_action('wp_head','feed_links_extra',3);
add_action('wp_head','rsd_link');
add_action('wp_head','wlwmanifest_link');
add_action('wp_head','rel_canonical');
add_action('wp_head','wp_shortlink_wp_head',10,0);
add_action('template_redirect','wp_shortlink_header',11,0);

Modify your theme. In particular, remove the Meta links in the sidebar, the feed links in the header and the footer, and anything (such as the comment feed link in the single post page) that may generate RSS feeds, trackbacks or pingbacks — if you’re converting your blog into static pages, those data is probably going to be meaningless since you are not updating the pages anymore. Search engines will still love your static pages.
Run the script again, this time with wpstatic -p. You probably need to run a few times to make sure that it fetches no more than it should (i.e. no feeds, no trackback links, etc.)
If your blog has database prefix wp_, the generated static page directory will be called wp-static. Inside the script has packed the MySQL dump and other static contents to wp-content.zip. You want to back that file up immediately.
Backup your original blog directory for its scripts and settings.
Remove the wp-content.zip in the generated static folder. Move static folder out. Remove the WordPress folder and rename the static folder to your blog’s directory name.

Your mileage may vary. And most of important of all, backup everything before you perform any of the above conversion steps.

Tim Cook, CEO of Apple, at the company memorial of Steve Jobs: But there is one more thing he...

Mon, 24 Oct 2011 02:14:00 -0700

Tim Cook, CEO of Apple, at the company memorial of Steve Jobs:

But there is one more thing he leaves us. He leaves us with each other. Because without him, Apple would have died in the late-90s, and the vast majority of us would have never met.

The works of Apple have touched and will continue to touch many people’s lives. They have certainly touched, and changed, mine. I gave up on programming when I decided to major in the humanities. Many years later, I realized that I still liked software and cared about it. In 2004 I wanted to pick up the skills again, but found the landscape wide and unwieldy. I didn’t know where to start. In the summer of that year, I bought the first Mac that I truly owned — a PowerBook G4 — and found a starting point: an open source project on Traditional Chinese input methods that a couple of good friends and I started. Mac OS X always had a great UNIX foundation, and Mac was already the choice of many open source developers. Friends taught me important things I needed to catch up, like how C++ had evolved during the years I wasn’t paying attention. At the same time I realized that the native app development on Mac OS X had its roots in NeXTSTEP. Years ago, I was once shown the amazing NeXT cubes at an acquaintance’s lab when he was doing his grad study in Hsinchu, Taiwan, and I never dreamt that one day I’d be able to program with the operating system’s software frameworks. It was a joy, almost illuminating, to discover and learn that so many good things had never lost their way. With the PowerBook, I learned how to program once more, and I’ve got to know many great people and made many good friends.

I wrote to Steve Jobs once. That was when Snow Leopard first came out in the late summer of 2009, and there were some issues with the new Traditional Chinese fonts. Those issues made them unusable in formal settings. With friends’ help and encouragement, we compiled the screenshots, filed Radar, and I finally got to draft the email. I didn’t expect him to read it, but the next day, someone at Apple replied, asking if I could send back the attachments that I “emailed Mr. Jobs.” And the issues with the fonts were eventually resolved. It’s not the famous one-liner that people loved to post online, and it’s easy for us to attribute everything that a company does to one person. Still, that is important: there are people at Apple that care.

I appreciate that Apple shared the recording of its company event with the rest of the world. It was solemn but at times lighthearted, comforting and at the same time gracefully cheering. It gave me a way to say goodbye, even though I never met him.

Nobody is an island, and what a great loss it was. Norah Jones’s songs soothed, and Coldplay lifted us up. It is truly, in Apple’s own way, a celebration of Steve’s life.

Goodbye, Steve.

Thank you, Steve.

Flickr OAuth Support in ObjectiveFlickr

Mon, 10 Oct 2011 08:52:00 -0700

I’ve added Flickr OAuth support to ObjectiveFlickr over the weekend.

Flickr announced in June that they started requiring new apps to use the OAuth-based authentication process, and the previous authentication process was deprecated and will be eventually phased out in early 2012. I recommend developers who use ObjectiveFlickr check out the latest version to start the transition.

ObjectiveFlickr is hosted on github.

What’s Added to ObjectiveFlickr

The main class, OFFlickrAPIContext, has two new methods:

-fetchOAuthRequestTokenWithCallbackURL: makes a request to Flickr for a request token
-fetchOAuthAccessTokenWithRequestToken:verifier: makes a request to Flickr to exchange the user-authorized request token (see below) for an access token

Two new delegate methods are added to the OFFlickrAPIRequestDelegate protocol, which you need to implement:

-flickrAPIRequest:didObtainOAuthRequestToken:secret: handles the result of request token fetch
-flickrAPIRequest:didObtainOAuthAccessToken:secret: userFullName:userName:userName:userName: handles the result of access token fetch

The class OFFlickrAPIContext now also support properties OAuthToken and OAuthTokenSecret. It also has a new method -userAuthorizationURLWithRequestToken:requestedPermission: that returns the URL for user authorization, which you need to open in a new browser window.

A new sample app, OAuthTransitionMac, is also added to show how to use the new authentication API along with the old. It also shows you how to exchange the existing auth token (from the old API) for an OAuth-based token. OAuthTransitionMac uses ARC, so it also shows how to set up the project (since ObjectiveFlickr still uses manual retain/release).

The iPhone app, SnapAndRun, has also been updated to show how to use OAuth in your iOS apps. Many people have had difficulties in linking the ObjectiveFlickr library externally, particularly on the iOS Simulator. So I simply put everything in the project. I also realized that SnapAndRun is unnecessarily complicated for its purpose — this is one thing you may be able to help improve (see the section “ObjectiveFlickr Needs Your Contribution” below).

What You Need to Do

Read Flickr’s own document to get an overview of the new authentication process. A glimpse of the flow chart should suffice for knowing what needs to be done.
If your app doesn’t have callback URL support, this is the time to add it. Refer to SnapAndRun (which has snapnrun://) and OAuthTransitionMac (oatransdemo://) for how to add URL support. In essence, you need to handle the incoming URL in your app delegate. This applies to both Mac and iOS apps.
When your app requests the user to authenticate, and when the user agrees to proceed, make a call using -fetchOAuthRequestTokenWithCallbackURL: to obtain a request token. Pass your app’s callback URL (e.g. oatransdemo://callback) to the method.
Implement -flickrAPIRequest:didObtainOAuthRequestToken:secret:. This delegate method is called after the request token is successfully obtained, and you need to set your Flickr context’s OAuth and OAuthSecret properties to the returned token and secret. Setting those two properties is required so that the subsequent access token fetch request will be signed correctly.
If any error occurs, the delegate method -flickrAPIRequest:didFailWithError: will still be called. The NSError object passed will have the error code OFFlickrAPIRequestOAuthError, and the userInfo dictionary will have a key-value pair under OFFlickrAPIRequestOAuthErrorUserInfoKey, which is simply the error message that Flickr returns.
In the same delegate method, call -userAuthorizationURLWithRequestToken:requestedPermission: to obtain the user authorization URL. The permission will be something like OFFlickrWritePermission (see ObjectiveFlickr.h for possible values). Then, open the URL in a separate browser. Use -[UIApplication openURL:] on iOS, -[NSWorkspace openURL:] on Mac.
After the user authorized your app to use their data, your app will be activated again. Parse the URL’s query with the OFExtractOAuthCallback function supplied in OFUtilities.h. You should obtain the authorized request token and verifier now.
Obtain the actual access token with -fetchOAuthAccessTokenWithRequestToken:verifier:
Implement the delegate method -flickrAPIRequest:didObtainOAuthAccessToken:secret: userFullName:userName:userName:userName:. This handles the access token fetch, and set your Flickr context’s OAuth and OAuthSecret to the final access token/secret pair that is passed back to the delegate method. Again, any error occurred will be passed to the error handling delegate method.
Done! You can start making API calls and upload photos with your existing code base.

Implementation Details

Flickr’s OAuth uses HMAC-SHA1 signing, which I decided to implement inside ObjectiveFlickr without depending on any other library. At this point our app projects probably have too many dependencies, and I want the changes to OF and to the way you use it to be minimal.

I’ve moved the inline functions out of OFUtilities.h to a new OFUtilities.m. This makes projects that use ARC happier, as inline functions cannot make retain/release calls under ARC. A Base64 encoding routine written by Matt Gallagher is also embedded in OFUtilities.m and is not exposed to the outside (i.e. the functions are declared static). Again this is to minimize dependencies.

To minimize implicit state changes, I decided to require you to set your Flickr context’s OAuthToken and OAuthTokenSecret in your delegate methods. Having OFFlickrRequest do it for you implicit seems to be a bad idea architecturally.

The current version “burns in” Flickr’s auth service endpoints in the code. It may be a good idea to make them properties in OFFlickrContext. Flickr now also has HTTPS endpoints. I’ll be glad to merge your pull requests if you have tried them out and made changes to the framework.

The API calling routines in OFFlickrRequest now has two code paths. If both the OAuthToken and OAuthTokenSecret properties are present (i.e. not nil) in your OFFlickrContext object, then the request object uses the OAuth-based signing to make the call. Otherwise it falls back to use the traditional authToken. If you’re only interested in public data, there’s no need for the authentication.

The OAuth spec allows three different ways of passing the signature. To simplify, I didn’t use the Authorization HTTP header, but chose to put the signature as part of the HTTP POST body or HTTP GET query.

ObjectiveFlickr Needs Your Contribution

Many projects use ObjectiveFlickr, and I need your help.

Please let me know if ObjectiveFlickr works or breaks your project. I no longer work on projects that need to support OSes older than Mac OS X 10.6 or iOS 4.3. I tried to be careful, for example I don’t use fast enumeration (lest it break OS X 10.4 support), but there may be places I use properties instead of old-school getters and setters in the library (not samples — they now only target the latest Mac OS X/iOS). You can help expand the test coverage in this area.

Eventually support for older OS or SDKs will and should be dropped. So much happened during the past two years, and more will come! It is important the shed some skin in order to catch up with those new developments in the Mac/iOS development landscape.

OAuth requires the consumer of an API (your app) to sign their calls and it involves a lot of URI string escaping. Handling improperly and you might see strange errors when you set a fancy title to a photo. The chars that need to be escaped, in addition to the default charset, are defined in the kEscapeChars constant in ObjectiveFlickr.m. I tried a number of common cases, but again the test coverage is very small. This is another area that you may be able to help.

ObjectiveFlickr was first developed in 2006 and was rewritten in early 2009 during the iOS 2.2 days. And it shows: the SnapAndRun iPhone sample app is unnecessarily complicated, and many conventions are out of date. A sample app written from ground up should be able to demo how to use the library more succinctly. The library’s own Xcode project has also needed to be remade so that app projects can again use cross-project reference to ObjectiveFlickr.

Then there’s documentation and code comments. Or synchronous API calls (so that you can make calls in other threads easily) or even NSOperation/block-based request methods. A lot of things can be and need to be done in a framework project like ObjectiveFlickr. Your contribution will benefit the Flickr and Mac/iOS development community a lot.

Conclusion

It’s highly recommended that you start migrating to use Flickr’s OAuth-based authentication, and ObjectiveFlickr now has enough support for it. The work for the transition shouldn’t be too much, as you only need to change your authentication process. Hopefully ObjectiveFlickr continues to serve your apps well. Best of luck with the transition!

The Low-Hanging Fruits and Bumpy Ride into the Future

Fri, 30 Sep 2011 23:24:00 -0700

I bought Tyler Cowen’s book “The Great Stagnation” and finished it the same evening. I enjoyed it much like I did from reading books like “Day of Empire”, in that they shed light on understanding the world we live in and where we are going from that. Cowen’s book is entirely about the situation in the US, but much of his analysis should apply to many other places in the world. He also talked about the political implication of such situation, or the “great stagnation”, and his view on China’s catching up, in an interview with The Economist.

Cowen’s main point in the book is that we have exhausted what the technological advances of the past few decades have been able to bring us — hence the first chapter’s name, “The Low-Hanging Fruit We Ate.” New advances are harder to come by, as evidenced by the diminished return in ever-increasing R&D spending. The situation is worsened by two additional factors: The US education system is inadequate, especially the K-12, and the hubris in the financial system, based on the assumption that previous growth trend would continue, finally led to all kinds of troubles that we are still seeing today.

I’m not so sure about Cowen’s view on the Internet. He devotes a whole chapter on it arguing that, while the Internet has brought us many great stuff, it creates few jobs, and people may even spend less because of all the (mostly free) fun they get from the Internet. Certainly eBay (one of his low job creation examples) does not hire many people — not at a scale of the Big Cars — but what about the benefits of disintermediation, which reduces the friction in transaction and makes many sellers reachable to more potential buyers (indie music comes to mind). But Cowen only spends a page to say that Google and Facebook hire less than 30k people combined and doesn’t give us numbers to argue that it’s indeed “not saving the revenue-generating sector of the economy.” What’s worrying about such argument is that, with politicians fixated on job creation (and that’s certainly not confined to the US), when they start pointing fingers at the tech industry that they aren’t creating many jobs, we may start to wonder where this may lead us.

One interesting, if also somber, implication of “the great stagnation” is political polarization, which Cowen doesn’t mention in his book but talked about in the aforementioned interview. This seems a good perspective to understand the political dramas (sometimes ludicrously baffling to foreigners) in the US.

Cowen prescribes bitter remedies: “Raise the social status of scientists” — and this is not easy as it sounds. He warns us that the recession will be long, although Japan shows the US how to cope with the decline as a society to a large extent. There’s a twist to the theme here, because Cowen also warns us to be ready when more low-hanging fruit comes:

[…] because sometimes low-hanging fruit is dangerous. The last time the world had a major dose of low-hanging fruit, a few countries didn’t handle it very well, including the Axis powers, the Soviet Union, and Communist China, among others.

Without the new technologies of the time, the totalitarian mistakes of the twentieth century would not have been possible. […] The record-keeping techniques of mass bureaucracy were used to control and often kill other human beings en masse. Only after bitter experience did fascist ideas become less popular, and social and political norms subsequently evolved to protect electorates against the fascist temptation.

So expect a bumpy ride to the future. I think that’s a good advice to everyone who’s lived the optimistic 1990s. The ending words are worth quoting in full:

In the meantime, we need to be prepared for a recession that could last longer than we are used to. We need to be prepared for the possibility that the growth slowdown could continue once the immediate recession passes. Part of science is coming to terms with its limits. The rate of scientific progress will continue to be uneven, sometimes grossly so. Yet reason and science have never been more important: If nothing else, a more reasonable and more scientific understanding of our predicament can help us cope, both intellectually and emotionally.

Back to the hard problems.

Indeed.

Using OS X's Built-in ICU Library in Your Own Project

Sat, 11 Jun 2011 20:37:00 -0700

Unicode string is usually not difficult to handle¹. The tricky part comes when you need to have knowledge of Unicode on things like:

Turning strings like “Café” to upper case or lower case (the toupper() or tolower() in Standard C Library won’t work)
Stripping all punctuation symbols. This is not as easy as it seems. How do you also strip various European quotation marks as well as full-width ones used in East Asian text?

Some languages, like Java, already have such Unicode knowledge baked in. For C and C++, the standard library to use is IBM’s ICU.

You can either build ICU from its source code, or use tools like MacPorts to install it for you. But you may also wonder: Since OS X has great internationalization support, perhaps OS X also uses ICU?

It turns out that OS X does include a version of pre-built ICU library² placed in /usr/library/libicucore.dylib, but for various reasons — and I also assume it has much to do with the fact that ICU is written in C++, and library versioning with C++ is a pain — it does not include the headers for you to use.

Here’s how you can link against the built-in library:

Install a copy of the latest ICU
In your project file, set header path to include the installed ICU header (e.g. /opt/local/include)
Define U_DISABLE_RENAMING in your project (i.e. setting the compiler flag -DU_DISABLE_RENAMING=1)— this turns off ICU’s version renaming scheme so that you’ll be able to link against an older library (the one that comes with OS X in this case) with the latest header
Link against libicucore (with the linker flag -licucore)

As the name implies, it only contains the core ICU library, so it lacks quite a few things. For example, C++ methods such as UnicodeString::toUTF8, classes as StringPiece, StringByteSink are missing. A good way to check availability is to use the UNIX tool nm to dump libicucore.dylib. Your best bet is to stick to the C API. On the other hand, this saves the trouble of including your own copy of ICU.

Read Joel Spolsky’s excellent primer on Unicode if it still isn’t to you. ↩
Many of NS/CFString and NSCharacterSet’s internationalization features use ICU under the hood. I haven’t tested it, but I believe the tips provided here can also be used on iOS. ↩

Brad Cox's "Object-Oriented Programming" (1986)

Wed, 06 Apr 2011 16:10:00 -0700

A few weeks ago I had a chance to re-read this particular book by Brad Cox¹. It is also the first book that introduced the Objective-C language to the world.

It’s not a heavy book — at 274 pages including index, Cox has outlined his solution for the software reusability problem in a succinct language (yes, pun intended). The book spends the first two chapters on why reusability is essential to mitigate the complexity in building a large system, what Cox calls “software crisis”, and on how the object-oriented programming paradigm, which Cox describes using the analogy of integrated circuits (“Software ICs”), is in turn the way to achieve reusability.

In the following chapters, Cox describes the language that he designed to fulfill his vision of Software IC: Objective-C. A short introduction and evaluation of Smalltalk-80 is given, and comparisons with Ada and then C++ are also provided. Cox likes many aspects of Smalltalk-80: The language is designed for personal computing, with interaction and graphics in mind (remember this is the language that gives the birth to modern graphical user interface), and it has many advanced features in its time: virtual machines with persistent stores (so say “saving files” is an unnecessary feature), a language built on blocks (think lambda) and message passing mechanisms, an IDE, and so on. As for why Smalltalk-80 didn’t become mainstream, Cox’s explanation is that Smalltalk-80 is a language designed for research purposes. To be of practical value, a hybrid language that can work with existing infrastructure will have a better chance:

My own interest in hybrid object-oriented language developed from the dream that organizational computing and personal computing could meet each other halfway, via a bridge over which arbitrary data could flow with minimum programming intervention. The bridge is created by providing the organizational computer with its own object-oriented programming language so that data can be put into object form. Object passivation [persistence] machinery […] can automatically convert this data into a form that Smalltalk can read and vice versa, so that much of the information transfer could be done automatically. This, and C’s ability to access databases maintained on older languages like FORTRAN [Cox also mentions COBOL in previous paragraph], creates a bridge between data locked in the mainframe and the user by relying on the ability of Smalltalk-80 to build user interfaces rapidly.

For the rest of the book, Cox describes the features of Objective-C in detail with an example of using the language to build a visual graph manipulation tool. Some syntactical features are no longer found in today’s Objective-C (e.g. the = ClassName declaration, and the feature that class declaration and definition are one unified compilation unit, much like Java, instead of separate parts). In the first of the last two chapters, Cox outlines the architecture of an “iconic user interface” — the model-view-controller paradigm (MVC), which was also introduced by Smalltalk. The last chapter briefly discusses the runtime environment (such as garbage collection) and possibilities for other higher-level language features, including “virtual object memories”², concurrency (using coroutines), distributed systems and coordination systems³.

Overall, I find the book full of insights that still resonate, 25 years after its publication. It’s interesting to take a historical perspective on how much progress we have made since then. Objective-C had never been as popular as C++, a “competitor” in some senses, or as Java, some features of which are actually influenced by Objective-C (such as interfaces, or what Objective-C calls protocols). NeXTSTEP, the first platform that heavily used Objective-C as the main language, was never mainstream. It takes Mac OS X, and then iOS, to propel it to one of the most popular programming languages worldwide. And many ideas are still waiting for more refined realizations. Some, like garbage collection, finally become an option on desktop platform. Some, like concurrency, are still things that frustrate programmers, expert or novice alike. Perhaps, like what some who teach functional programming would like to show, we are past the apex of the object-oriented model, and a new paradigm shift awaits.

Cox, Brad J. Object-Oriented Programming: An Evolutionary Approach. Addison-Wesley, 1986. I’m referring to the first edition here. If you know the differences between its first and second editions, I’d appreciate it if you can write something about it. ↩
We can identify some of Cox’s description of such a “capability-based” addressing system in today’s uniform resource identifiers (URI). ↩
Although gcc has been supporting gc-capable Objective-C for years, it was not possible to use it in Mac applications due to lack of framework support. Apple eventually came up with its own “mix of curry” of concurrent, generational garbage collector, with runtime and full framework support, in 2006. Objective-C also has a feature called Distributed Objects, which is often used among Mac applications for inter-process communications. Library support for operation-based concurrency became part of the Mac OS X 10.5 framework, again since 2006. ↩

BugzKit, a FogBugz API Library for Objective-C

Wed, 02 Mar 2011 05:03:49 -0800

BugzKit, a FogBugz API Library for Objective-C

BugzKit is a FogBugz API library for Objective-C. I started developing BugzKit in mid-2009 at Lithoglyph as the foundation for LadyBugz, a FogBugz client for the Mac. Since the company has discontinued the development of the app, I think it’s time to release the library to the public as an open source project. I hope that by opening the library, it can help others develop new FogBugz clients, such as ones for the iOS platform.

The library is now available on GitHub and is released under the MIT License.

One important issue in designing an API library for FogBugz is data dependency. The case (ticket) data you fetch from the server is meaningless without the lists of projects, people, milestones, and so on. Therefore you want to fetch those lists before you can fetch the cases of a particular project or filter. Also, you want to fetch lists as well as cases in parallel, as there are many such lists and fetching them serially takes too long.

For example, in the BasicRequests sample app that comes with the library, we want to perform the following tasks:

Check the API version and availability
Login (FogBugz uses the term “log on”)
Fetch three lists (projects, people, milestones) in parallel
If all succeed, logout (“log off”)

You may notice that logout should be performed after Step 3. even if any task in that step fails: Since we have an auth token, we should logout. For the sake for simplicity, let’s stick to what the sample code implements.

Apparently, each step depends on its previous step. If there’s no API available (such as the service endpoint is down, or the URL is incorrect, etc.), you can’t login. If you don’t have the auth token obtained from the login, you can’t fetch the lists. If any of the steps fails, the program should cancel all other operations standing in line and exit. If we draw a dependency graph, it will look like this:

BugzKit solves the problem by separating request objects and request operations. Request objects encapsulates information required to make a request (URL, parameters, HTTP method name, etc.) whereas request operations are the actual drivers that perform the HTTP request. Once we have the operation layer, we can then make use of the operation queue framework in Foundation (NSOperation and NSOperationQueue are the two main classes). NSOperation has the better granularity (compared with threads) for handling operation dependency, cancellation, and convergence (e.g. two or more finished operations meeting each other). Extending the sample code to anything that works in the real world will take a lot more work, but I believe the library makes it at least manageable.

The API library covers most of the functions required to fetch lists and manipulate cases. It also handles attachments for you by automatically creating the temp files on demand and preparing an input stream so that you can send the byte stream as the POST request body (with the multipart MIME type). Many areas are not covered, though, such as wiki or time tracking. It should be easy to extend the library to support those FogBugz features.

The Hybrid Drive Upgrade

Sat, 12 Feb 2011 17:16:00 -0800

I replaced the old hard disk drive in my laptop as it showed signs of dying two weeks ago. There is a lot of talk about upgrading to SSD among the people I follow on Twitter. SSD is great, no doubt, but it’s still an expensive option.

Then I came upon this: a hybrid drive. It’s actually a traditional, spinning hard disk drive with 4 GB flash memory used as read cache.

I’m never a benchmarking person when it comes to hardware. I do have read the AnandTech review after having installed and used the new drive for a few days, and my experience agrees with the review’s findings. In some areas like boot-up speed, this drive does provide SSD-like performance. In other areas it’s just a fast spinning drive.

The unpleasant, unresponsive period after login is a thing of the past, and app launching is faster too.

It’s true that many Mac users seldom restart their machines, so a big selling point of this kind of hybrid drive, “it speeds up booting up”, seems moot. It’s true that the hybrid drive is more expensive than its traditional spinning cousins, at about 2x the price, but an SSD of the same capacity will cost about 6x-10x. So for a bit more money you get big speed improvement in one area and small improvements in other areas with this hybrid drive.

This drive does give me a small pleasant surprise: Improved virtual machine performance. For work reasons I usually keep 4-5 virtual machine images around, and I often have to fire up and then suspend some of them (I use VMware Fusion). There must be some complex interactions between the VM’s and the host’s file systems, and this hybrid drive seems to be of great help. Not only do my VMs start faster, but the long latency after resuming is also gone.

Improved virtual machine gives me a great incentive to virtualize more. Now I can compartmentalize many more things, like trying new apps, in separate VMs. In theory I could have already been doing this all along, but the slow start-up and resume time has put me off, and I seldom used VM other than for work. But this enables a whole new usage for me.

Overall I’m satisfied with the upgrade. I didn’t spend too much, and the marginal return is high. Those SSD upgrade tweets made me crave them but I was also wary of their pocket-burning tendencies. Now I won’t have to worry about them for at least quite a while.

"The storytelling that thrives for a long time in the milieu of work — the rural, the maritime, and..."

Tue, 02 Nov 2010 14:01:00 -0700

“

The storytelling that thrives for a long time in the milieu of work — the rural, the maritime, and the urban — is itself an artisan form of communication, as it were. It does not aim to convey the pure essence of the thing, like information or a report. It sinks the thing into the life of the storyteller, in order to bring it out of him again. Thus traces of the storyteller cling to the story the way the handprints of the potter cling to the clay vessel. Storytellers tend to begin their story with a presentation of the circumstances in which they themselves have learned what is to follow, unless they simply pass it off as their own experience.

…

Seen in this way, the storyteller joins the ranks of the teachers and sages. He has counsel — not for a few situations, as the proverb does, but for many, like the sage. For it is granted to him to reach back to a whole lifetime (a life, incidentally, that comprises not only his own experience but no little of the experience of others; what the storyteller knows from hearsay is added to his own). His gift is the ability to relate his life; his distinction, to be able to tell his entire life. The storyteller: he is the man who could let the wick of his life be consumed completely by the gentle flame of his story. This is the basis of the imcomparable aura about the storyteller […]. The storyteller is the figure in which the righteous man encounters himself.

”

- Walter Benjamin, “The Storyteller” (1936). English translation from the book “Illuminations”, New York, NY: Schocken, 1969.

People Who Don't Boast Their Working on Making the Change

Tue, 26 Oct 2010 18:31:00 -0700

I don’t have the DVD with me now, but in one of the bonus scenes of the film Helvetica, Erik Spiekermann, a German design titan, talks about how his satisfaction comes from the fact that he designed the signage of train stations that people rely on and pass by everyday without realizing the design was his work.

It’s an interesting way of seeing things that I find, again and again, in people who design. It doesn’t have to be visual design or typography: Think about designing a system, a software framework, a house, a mechanism, things that other parts rely on and people use.

To stretch a bit, hence: There are at least two kinds of people. One boasts how they are going to make the change. The other makes the change, and moves on.

LEMON and That parse.y File in SQLite Source

Fri, 22 Oct 2010 22:24:00 -0700

I had really only took a cursory glimpse of SQLite’s source, and I had always thought the parse.y file was a yacc/Bison parser source. I was wrong: It’s the parser source for LEMON, the parser generator written by SQLite’s author, D. Richard Hipp.

According to its web page:

The parser generated by Lemon is both re-entrant and thread-safe.
Lemon includes the concept of a non-terminal destructor, which makes it much easier to write a parser that does not leak memory.

SQLite is such an amazing example of source and tool self-reliance.

[A-Za-z0-9\s\n\.,]-Only Programming Language?

Thu, 21 Oct 2010 00:20:00 -0700

Someone must have already said this somewhere.

I did more note-taking, email and document drafting these days on my iPad. Most types of text I can do it fine on iPad’s on-screen keyboard, except one thing: code snippets. C and C-like languages use tons of symbol characters that often hide two level down on iPad’s keyboard, and it’s horrible if you need to write just a simply code snippet to get your idea across. The alternative could have been an external keyboard, but then the new MacBook Air now totally makes sense: Programming stills belongs to the veritable laptops.

Or does it? I stared at the on-screen keyboard and thought, maybe there is some programming language that can be written mostly using the Latin alphabet. Caveat: You don’t want to make the language too similar to human language. Many scripting languages tried to do that (let x be Window, call open, blah blah…). There seems to be some tricky boundary between real human language and programming language. Maybe the keywords should be in some dead language, like Latin, then¹?

If you want to be really picky, there should be a * in the title, after the square bracket (Tumblr’s Markdown escapes one of the characters if I type the expression in the body text, so I’m not going to do it). And, think about it, AZaz09 doesn’t look like a bad name for a language. ↩

And I still read essays sometimes

Wed, 06 Oct 2010 00:05:10 -0700

“Change or Perish” by Roger Cohen, via Lexington:

Before leggings, when there were letters, before texts and tweets, when there was time, before speed cameras, when you could speed, before graffiti management companies, when cities had souls, we managed just the same.

Before homogenization, when there was mystery, before aggregation, when the original had value, before digital, when there was vinyl, before Made in China, when there was Mao, before stress management, when there was romance, we had the impression we were doing all right.

The rhythm feels e. e. cummings-ish (“long enough and just so long / tomorrow will not be too late”). The tone sad, but beautiful.

Follow-Up on Objective-C Cyclic Retention in Block: Referencing Self

Sat, 02 Oct 2010 00:47:00 -0700

I’ve written on a pitfall of Objective-C blocks. Blocks implicitly retain the objects referenced inside them, and that can cause all kinds of problems.

Mark Dalrymple has written on the same topic, and his emphasis is on using self. He came up with a “lighter” solution without resorting to an NSValue object: Use the __block storage class.

Read his blog entry in full to understand the issue better. I highly suspect this issue will be with us for a long time. I don’t think GC will rescue us all: Apple’s GC is so buggy to the extent that it’s totally unusable in some applications, and it’s not available to iOS at least at the current version. The only hope to mitigate the problem is for the static analyzer to be able to warn the dangers ahead.

Lithoglyphics: LadyBugz 1.6.9: More Bug Fixes

Thu, 19 Aug 2010 05:22:33 -0700

Lithoglyphics: LadyBugz 1.6.9: More Bug Fixes:

We’ve just released LadyBugz 1.6.9. It turns out that 1.6.7 didn’t completely fix the crashing bugs, and from the crash reports we’ve received (thanks everyone for letting us know!), it concentrated in one specific area. We’ve revamped that part of the code, and version 1.6.9 should clear that issue now.

Another Mile to Go in the GC-back-to-NonGC Transition

Sun, 08 Aug 2010 00:25:00 -0700

Another Mile to Go in the GC-back-to-NonGC Transition:

From Lithoglyphics, the blog of my company:

We have just released LadyBugz 1.6.6, available from our website and through auto update.

After we released LadyBugz 1.6.1, we have received a number of bug reports that it crashed at various places. We realized that our garbage collection transition was not completely done, and there were loose ends that we overlooked. We take those issues seriously, and after a week of testing, we believe 1.6.6 should behave way better.

I have written previously why we chose to go back from garbage collection to manual memory management, and while it was absolutely necessary, I really wish we could have found the crashing bugs earlier with a broader range of testing data set. Quitting the perks of garbage collection is a hard process (not having to worry about complex object relationship and ownership is especially addictive), and hard-to-reproduce crashing bugs suck. We have learned a thing or two the hard way.

Cyclic Retention Pitfall in Objective-C Blocks

Tue, 03 Aug 2010 01:20:00 -0700

Update: Please Mark Dalrymple’s article for a more elegant solution. In short, use __block instead of the NSValue trick below.

Is it tempting to write the following code:

    NSBlockOperation *op = [[[NSBlockOperation alloc] init] autorelease];

    [op addExecutionBlock:^(void) {
        if (![op isCancelled]) {
            // run the block if the operation is not cancelled
        }
    }];

    [someOperationQueue addOperation:op];

If you don’t use garbage collection (gc), the snippet above will eat up memory, and the Leaks Instrument will not be able to discover it as a leak. Why?

In Objective-C, blocks behave like objects. In non-gc runtime, they have retain count, can be copied¹ and must be released when you want to discard them.

When a block is created, it implicitly retains every Objective-C object referenced in its scope. When the block is deallocated, it sends -release to all those retained objects. In Objective-C, those extra calls are done for you by the compiler. If you set breakpoints to -retain and -release, you’ll be able to observe this behavior as blocks are created or deallocated.

Usually you don’t need to worry about those things, and that’s how you can use blocks as good closures that many other modern languages have.

But here’s the problem: Because blocks only send -release when they are deallocated, not when their scope exits, it is possible for them to retain the objects which retain them. The code snippet above is a case in point. The NSBlockOperation object op retains an execution block, but then the block implicitly retains op. The result of such cyclic retention is that the retain count of op will never reach zero, and hence op will eat up a chuck of unreclaimable memory. And because op is retained by another object, it is not a lone wanderer that you forget to call -release, therefore the Leaks Instrument will never discover it as a leak.

A solution to this problem would be:

    NSBlockOperation *op = [[[NSBlockOperation alloc] init] autorelease];

    NSValue *weakOpValue = [NSValue valueWithNonretainedObject:op];
    [op addExecutionBlock:^(void) {
        if (![[weakOpValue nonretainedObjectValue] isCancelled]) {
            // run the block if the operation is not cancelled
        }
    }];

    [someOperationQueue addOperation:op];

The NSValue object used above is effectively a weak reference to op. This breaks the retention cycle, so the -dealloc of op will now be called, and the block will also be released.

Cyclic retention is not a problem for Objective-C runtime’s garbage collector, so the first code snippet will not be a problem if you use gc.

Another incomplete solution is for Apple to change the timing of calling -release in a block. Releasing retained objects when a block exits seems to be a reasonable expectation. But, given the nature of the use case above (an NSBlockOperation might never get executed), plus the fact that referenced objects must be retained before a block is entered, it’s not an encompassing solution, although it would make blocks behave more to such expectation.

One final note: Because of the potential risk of cyclic retention, you must be very careful if you use blocks within a block. This is one place where gc can liberate us from such a blocking issue (pun intended), but gc is not available on the iOS and has its issues on Mac, too.

But not retained. The reason is that when a block is first created, its scope refers to the values residing in the nearest outer scope on the stack. If you toss it around (i.e. passing it as an object), it must first make a (const) copy of those values onto the heap. If you just retain a block then pass it around, chances are it’s still referring to the stack. And you know what will happen when dangling stack references are passed around. ↩