Krzysztof Kieślowski on Crafting an Experience

I recently re-watched the Three Colors trilogy by the late Polish director Krzysztof Kieślowski (1941-1996). I first watched the three films when I was in college. I think I understood more and saw more in the films than I did when I was younger. Growing up, having traveled, and some knowledge of the history and one of the languages spoken in the film (French) all helped. But even back then I was deeply touched by the films the first time I watched them.

I also appreciated more Kieślowski’s mastery of the film language.

The DVD edition of the trilogy comes with an extra feature called “Krzysztof Kieślowski’s Cinema Lesson”, in which the director talked about the making of one significant scenes in each of the trilogy. In particular, he talked about the filming process and the editing choices involved. This is the auteur dissecting his own work in details.

In Blue, for example, there’s a famous “sugar cube scene”, starting at 52′20″ and lasting about five seconds. Julie, the heroine, is found sitting in a café by Olivier. Julie has recently lost her husband and daughter in a car accident, and she tries hard to leave everything in her life behind. Olivier, who has worked with her husband on a song commissioned by the European Union, becomes a lover and puts an effort in finding her. Julie turns him down in the café. After he leaves, she plays with the sugar cube that comes with her coffee, while a flute player on the street is playing a number that sounds very similar to one of the motives in the commissioned (but yet finished) song.

Here’s Kieślowski talking about how he required the cube to be soaked in coffee in exactly five seconds, and the rationale behind it. Emphasis mine:

This is a sugar cube about to fall in the cup of coffee. What does this obsession with close-ups mean? Simply that we’re trying to show the heroine’s world from her point of view, to show that she sees these little things, things that are near her, by focusing on them, in order to demonstrate that the rest doesn’t matter to her.

It seems easy to film a sugar cube soaking up coffee, sucking it up and turning brown. […] We can start a stopwatch. It should take five and a half seconds or five seconds to be completely soaked. How to make sure that it only takes five seconds? Not so easy. Let’s take a regular sugar cube, like this one, and soak it in my coffee. I’ll start the stopwatch… Eight seconds. That’s three seconds too long. We had to prepare one that would be soaked in five seconds. We decided such a detail shouldn’t last more than five seconds. For half a day, my assistant tested all kinds of sugar cubes, […]

What do we care about a stupid sugar cube sucking up some stupid coffee? Nothing… unless we are, for a moment, in our heroine’s world. She dips a sugar cube in her coffee and focuses on it to reject the offer that the man who loves her just made her. […]

And when you ask me if I think about the viewer, about the viewer’s point of view — I go back to this stupid sugar cube — I try always to keep this in mind. We don’t do previews, test screenings. I believe an audience can take four and a half seconds of sugar soaking, but that eight and a half would be too much.

Cirque du Soleil and Radio Canada International

In the mid-1990s I owned a shortwave radio. It was one of the top-tier models made by Sangean, although I forgot the model number. That was before Internet radio came along, and you could still write to shortwave radio stations, with an attached international reply coupon, to request for their schedule and publications. So I wrote to Radio Canada International (RCI) to ask for their schedule and a booklet on how to install shortwave antennae. Sangean’s manual listed the addresses of the radio stations you could write to.

Because the city is in a basin, shortwave reception in Taipei wasn’t great, so an external antenna made sense. I bought one coupon from a stamp collecting store (that’s the one and only time I did that in my life), sent the letter, and a few weeks later, I received the schedule (which got updated every 6 months — because the atmospheric condition varies by season) and a guide to install various kinds of antennae. My father helped me install a long-wire antenna, the simplest type, on the roof of our apartment building (he also told me how shortwave radio was a restricted item in the 1950s in Taiwan, as the government didn’t want people to have too much access to the information of the outside world). We calculated the needed length and got the insulation right, dropped the connecting cable to my window, and connected to the radio’s external antenna socket. In addition to RCI, we were also able to listen to BBC World Service, Voice of America, Deutsche Welle, Radio Japan (NHK) and a few others with different reception qualities, a great supplement (my father would have said alternative) to ICRT, Taipei’s local English station.

RCI’s broadcast time in East Asia was short, 30 minutes in French and then 30 minutes in English, but the timing was perfect. The French broadcast started at 6 AM Taipei time, then followed by the English broadcast at 6:30. My Sangean radio had both a timer and tape recorder, so I let RCI wake me up every morning, then later on the bus I would play the English news and programs in my Aiwa cassette player. That was a great way to wake up and learn English.

Before a shortwave broadcast begins, the station plays a short sequence to identify itself, called “ident.” The ident of BBC World Service, “This is London calling,” is among the best known (check out this YouTube video, the version I heard is between 1:01 and 1:30). RCI, unsurprisingly, used the first bar of “O Canada,” the national anthem, followed by the bilingual announcement “Radio Canada International / Radio Canada International” (yes, one of them was in French…), and the sequence repeated itself two more times.

For me, one mystery about RCI’s broadcast, at least in East Asia, was that before the ident, they actually played three minutes of a song, in a language that was definitely neither French or English. I hummed the tune to many people, but no one I asked knew what it was. It was also strange, too, because not many shortwave stations did that, a full three minutes of a song before the ident.

I usually erased the tape on which I recorded each morning’s program, but one day I decided to save a tape for the opening song. A few years later I converted the tape into MP3, played the clip to a few more people, but I still hadn’t got any answer. And then I forgot the whole thing.

Just today, out of the blue, I realized I still kept the MP3 file, and now I could use services like Shazam to identify what the song was. And so I did. Turns out that the opening song was “Kumbalawe,” the opening song of a show by Cirque du Soleil. It totally made sense now, as they are Canadian (from Montréal). The song uses an invented language, though. Isn’t it amazing, that with a technology like music identification, we can find out the name of a song whose melody you never forget, after so many years?

On the other hand, I guess I could have written to RCI long ago to ask about the song. Still, it was a happy rediscovery of a song that made so many wonderful early mornings.

Permanently Convert WordPress Blogs to Static Pages

Since I started using Tumblr to host my blogs, I no longer used WordPress. It’s a great blog software, but even if you stopped writing blogs with it, you still need to run its software update from time to time for security. Because of that, plus other various personal reasons, I decided to convert all the WordPress blogs that I hosted on my own to static pages.

After some research online, the best solution I found for this task is Ammon Shepherd’s WPStatic utility. Shepherd wrote extensively about his project (usage, origin, more details). The project has been around for a while, and the version hosted on github works with the latest WordPress. Here’s a quick summary of what I do to “staticize” my blogs.

First, you need to know what WPStatic does:

  • It backs up your WordPress database (as a MySQL dump). It actually makes two backups: One before it makes the changes to your database, and one after.
  • It changes your blog’s permalink structure and close all the comments (for both posts and pages)
  • It runs wget to fetch every static pages linkable from the top page
  • It archives the MySQL dump, the static pages, and the files in wp-content into a .zip file

The script is only one file, called wpstatic. Just place the file in your blog root, then run chmod +x wpstatic to make it executable, and then run ./wpstatic -h for help. Some of the above actions, such as permalink structure modification, can be toggled off.

Then, here’s what I did:

  1. I ran some of the backups on Mac OS X and some on Linux. If you run the script on Linux, you need to change nothing. If you run the script on Mac OS X, you need to do a global replace to change every occurrence of sed -r to sed -E (that’s because the sed that comes with Mac OS X comes from BSD).

  2. Run ./wpstatic but don’t proceed with the wget backup (the actual page fetching). I ran this step to let the script close all the comments for me.

  3. Now this step is important. By default, the script changes your blog’s permalink strcuture to something like http://example.com/year/month/day/post-name. This didn’t work for me since some of my blog posts contain CJK titles, and CJK filenames work badly (a tragedy compounded by both URL encoding and file systems). Plus I have been using the structure like http://example.com/archives/123 for many years, and Google searchs on my blogs point to URLs like that. So, after the script changed the permalink structure, I had to go back to the admin page again, first change the permalink back to /archives/%post_id%, then I added a / at the end. The final permalink structure became /archives/%post_id%/, and this is great, because this actually caused the pages to be saved into something like /archives/123/index.html.

  4. Modifiy wp-includes/default-filters.php and comment out all actions that generate feed links. I turned off (commented out) these lines. You might want to remove more:

    add_action('wp_head','feed_links',2);
add_action('wp_head','feed_links_extra',3);
add_action('wp_head','rsd_link');
add_action('wp_head','wlwmanifest_link');
add_action('wp_head','rel_canonical');
add_action('wp_head','wp_shortlink_wp_head',10,0);
add_action('template_redirect','wp_shortlink_header',11,0);

  5. Modify your theme. In particular, remove the Meta links in the sidebar, the feed links in the header and the footer, and anything (such as the comment feed link in the single post page) that may generate RSS feeds, trackbacks or pingbacks — if you’re converting your blog into static pages, those data is probably going to be meaningless since you are not updating the pages anymore. Search engines will still love your static pages.

  6. Run the script again, this time with wpstatic -p. You probably need to run a few times to make sure that it fetches no more than it should (i.e. no feeds, no trackback links, etc.)

  7. If your blog has database prefix wp_, the generated static page directory will be called wp-static. Inside the script has packed the MySQL dump and other static contents to wp-content.zip. You want to back that file up immediately.

  8. Backup your original blog directory for its scripts and settings.

  9. Remove the wp-content.zip in the generated static folder. Move static folder out. Remove the WordPress folder and rename the static folder to your blog’s directory name.

Your mileage may vary. And most of important of all, backup everything before you perform any of the above conversion steps.

Tim Cook, CEO of Apple, at the company memorial of Steve Jobs:

But there is one more thing he leaves us. He leaves us with each other. Because without him, Apple would have died in the late-90s, and the vast majority of us would have never met.

The works of Apple have touched and will continue to touch many people’s lives. They have certainly touched, and changed, mine. I gave up on programming when I decided to major in the humanities. Many years later, I realized that I still liked software and cared about it. In 2004 I wanted to pick up the skills again, but found the landscape wide and unwieldy. I didn’t know where to start. In the summer of that year, I bought the first Mac that I truly owned — a PowerBook G4 — and found a starting point: an open source project on Traditional Chinese input methods that a couple of good friends and I started. Mac OS X always had a great UNIX foundation, and Mac was already the choice of many open source developers. Friends taught me important things I needed to catch up, like how C++ had evolved during the years I wasn’t paying attention. At the same time I realized that the native app development on Mac OS X had its roots in NeXTSTEP. Years ago, I was once shown the amazing NeXT cubes at an acquaintance’s lab when he was doing his grad study in Hsinchu, Taiwan, and I never dreamt that one day I’d be able to program with the operating system’s software frameworks. It was a joy, almost illuminating, to discover and learn that so many good things had never lost their way. With the PowerBook, I learned how to program once more, and I’ve got to know many great people and made many good friends.

I wrote to Steve Jobs once. That was when Snow Leopard first came out in the late summer of 2009, and there were some issues with the new Traditional Chinese fonts. Those issues made them unusable in formal settings. With friends’ help and encouragement, we compiled the screenshots, filed Radar, and I finally got to draft the email. I didn’t expect him to read it, but the next day, someone at Apple replied, asking if I could send back the attachments that I “emailed Mr. Jobs.” And the issues with the fonts were eventually resolved. It’s not the famous one-liner that people loved to post online, and it’s easy for us to attribute everything that a company does to one person. Still, that is important: there are people at Apple that care.

I appreciate that Apple shared the recording of its company event with the rest of the world. It was solemn but at times lighthearted, comforting and at the same time gracefully cheering. It gave me a way to say goodbye, even though I never met him.

Nobody is an island, and what a great loss it was. Norah Jones’s songs soothed, and Coldplay lifted us up. It is truly, in Apple’s own way, a celebration of Steve’s life.

Goodbye, Steve.

Thank you, Steve.

Flickr OAuth Support in ObjectiveFlickr

I’ve added Flickr OAuth support to ObjectiveFlickr over the weekend.

Flickr announced in June that they started requiring new apps to use the OAuth-based authentication process, and the previous authentication process was deprecated and will be eventually phased out in early 2012. I recommend developers who use ObjectiveFlickr check out the latest version to start the transition.

ObjectiveFlickr is hosted on github.

What’s Added to ObjectiveFlickr

The main class, OFFlickrAPIContext, has two new methods:

  • -fetchOAuthRequestTokenWithCallbackURL: makes a request to Flickr for a request token
  • -fetchOAuthAccessTokenWithRequestToken:verifier: makes a request to Flickr to exchange the user-authorized request token (see below) for an access token

Two new delegate methods are added to the OFFlickrAPIRequestDelegate protocol, which you need to implement:

  • -flickrAPIRequest:didObtainOAuthRequestToken:secret: handles the result of request token fetch
  • -flickrAPIRequest:didObtainOAuthAccessToken:secret: userFullName:userName:userName:userName: handles the result of access token fetch

The class OFFlickrAPIContext now also support properties OAuthToken and OAuthTokenSecret. It also has a new method -userAuthorizationURLWithRequestToken:requestedPermission: that returns the URL for user authorization, which you need to open in a new browser window.

A new sample app, OAuthTransitionMac, is also added to show how to use the new authentication API along with the old. It also shows you how to exchange the existing auth token (from the old API) for an OAuth-based token. OAuthTransitionMac uses ARC, so it also shows how to set up the project (since ObjectiveFlickr still uses manual retain/release).

The iPhone app, SnapAndRun, has also been updated to show how to use OAuth in your iOS apps. Many people have had difficulties in linking the ObjectiveFlickr library externally, particularly on the iOS Simulator. So I simply put everything in the project. I also realized that SnapAndRun is unnecessarily complicated for its purpose — this is one thing you may be able to help improve (see the section “ObjectiveFlickr Needs Your Contribution” below).

What You Need to Do

  1. Read Flickr’s own document to get an overview of the new authentication process. A glimpse of the flow chart should suffice for knowing what needs to be done.

  2. If your app doesn’t have callback URL support, this is the time to add it. Refer to SnapAndRun (which has snapnrun://) and OAuthTransitionMac (oatransdemo://) for how to add URL support. In essence, you need to handle the incoming URL in your app delegate. This applies to both Mac and iOS apps.

  3. When your app requests the user to authenticate, and when the user agrees to proceed, make a call using -fetchOAuthRequestTokenWithCallbackURL: to obtain a request token. Pass your app’s callback URL (e.g. oatransdemo://callback) to the method.

  4. Implement -flickrAPIRequest:didObtainOAuthRequestToken:secret:. This delegate method is called after the request token is successfully obtained, and you need to set your Flickr context’s OAuth and OAuthSecret properties to the returned token and secret. Setting those two properties is required so that the subsequent access token fetch request will be signed correctly.

  5. If any error occurs, the delegate method -flickrAPIRequest:didFailWithError: will still be called. The NSError object passed will have the error code OFFlickrAPIRequestOAuthError, and the userInfo dictionary will have a key-value pair under OFFlickrAPIRequestOAuthErrorUserInfoKey, which is simply the error message that Flickr returns.

  6. In the same delegate method, call -userAuthorizationURLWithRequestToken:requestedPermission: to obtain the user authorization URL. The permission will be something like OFFlickrWritePermission (see ObjectiveFlickr.h for possible values). Then, open the URL in a separate browser. Use -[UIApplication openURL:] on iOS, -[NSWorkspace openURL:] on Mac.

  7. After the user authorized your app to use their data, your app will be activated again. Parse the URL’s query with the OFExtractOAuthCallback function supplied in OFUtilities.h. You should obtain the authorized request token and verifier now.

  8. Obtain the actual access token with -fetchOAuthAccessTokenWithRequestToken:verifier:

  9. Implement the delegate method -flickrAPIRequest:didObtainOAuthAccessToken:secret: userFullName:userName:userName:userName:. This handles the access token fetch, and set your Flickr context’s OAuth and OAuthSecret to the final access token/secret pair that is passed back to the delegate method. Again, any error occurred will be passed to the error handling delegate method.

  10. Done! You can start making API calls and upload photos with your existing code base.

Implementation Details

Flickr’s OAuth uses HMAC-SHA1 signing, which I decided to implement inside ObjectiveFlickr without depending on any other library. At this point our app projects probably have too many dependencies, and I want the changes to OF and to the way you use it to be minimal.

I’ve moved the inline functions out of OFUtilities.h to a new OFUtilities.m. This makes projects that use ARC happier, as inline functions cannot make retain/release calls under ARC. A Base64 encoding routine written by Matt Gallagher is also embedded in OFUtilities.m and is not exposed to the outside (i.e. the functions are declared static). Again this is to minimize dependencies.

To minimize implicit state changes, I decided to require you to set your Flickr context’s OAuthToken and OAuthTokenSecret in your delegate methods. Having OFFlickrRequest do it for you implicit seems to be a bad idea architecturally.

The current version “burns in” Flickr’s auth service endpoints in the code. It may be a good idea to make them properties in OFFlickrContext. Flickr now also has HTTPS endpoints. I’ll be glad to merge your pull requests if you have tried them out and made changes to the framework.

The API calling routines in OFFlickrRequest now has two code paths. If both the OAuthToken and OAuthTokenSecret properties are present (i.e. not nil) in your OFFlickrContext object, then the request object uses the OAuth-based signing to make the call. Otherwise it falls back to use the traditional authToken. If you’re only interested in public data, there’s no need for the authentication.

The OAuth spec allows three different ways of passing the signature. To simplify, I didn’t use the Authorization HTTP header, but chose to put the signature as part of the HTTP POST body or HTTP GET query.

ObjectiveFlickr Needs Your Contribution

Many projects use ObjectiveFlickr, and I need your help.

Please let me know if ObjectiveFlickr works or breaks your project. I no longer work on projects that need to support OSes older than Mac OS X 10.6 or iOS 4.3. I tried to be careful, for example I don’t use fast enumeration (lest it break OS X 10.4 support), but there may be places I use properties instead of old-school getters and setters in the library (not samples — they now only target the latest Mac OS X/iOS). You can help expand the test coverage in this area.

Eventually support for older OS or SDKs will and should be dropped. So much happened during the past two years, and more will come! It is important the shed some skin in order to catch up with those new developments in the Mac/iOS development landscape.

OAuth requires the consumer of an API (your app) to sign their calls and it involves a lot of URI string escaping. Handling improperly and you might see strange errors when you set a fancy title to a photo. The chars that need to be escaped, in addition to the default charset, are defined in the kEscapeChars constant in ObjectiveFlickr.m. I tried a number of common cases, but again the test coverage is very small. This is another area that you may be able to help.

ObjectiveFlickr was first developed in 2006 and was rewritten in early 2009 during the iOS 2.2 days. And it shows: the SnapAndRun iPhone sample app is unnecessarily complicated, and many conventions are out of date. A sample app written from ground up should be able to demo how to use the library more succinctly. The library’s own Xcode project has also needed to be remade so that app projects can again use cross-project reference to ObjectiveFlickr.

Then there’s documentation and code comments. Or synchronous API calls (so that you can make calls in other threads easily) or even NSOperation/block-based request methods. A lot of things can be and need to be done in a framework project like ObjectiveFlickr. Your contribution will benefit the Flickr and Mac/iOS development community a lot.

Conclusion

It’s highly recommended that you start migrating to use Flickr’s OAuth-based authentication, and ObjectiveFlickr now has enough support for it. The work for the transition shouldn’t be too much, as you only need to change your authentication process. Hopefully ObjectiveFlickr continues to serve your apps well. Best of luck with the transition!

The Low-Hanging Fruits and Bumpy Ride into the Future

I bought Tyler Cowen’s book “The Great Stagnation” and finished it the same evening. I enjoyed it much like I did from reading books like “Day of Empire”, in that they shed light on understanding the world we live in and where we are going from that. Cowen’s book is entirely about the situation in the US, but much of his analysis should apply to many other places in the world. He also talked about the political implication of such situation, or the “great stagnation”, and his view on China’s catching up, in an interview with The Economist.

Cowen’s main point in the book is that we have exhausted what the technological advances of the past few decades have been able to bring us — hence the first chapter’s name, “The Low-Hanging Fruit We Ate.” New advances are harder to come by, as evidenced by the diminished return in ever-increasing R&D spending. The situation is worsened by two additional factors: The US education system is inadequate, especially the K-12, and the hubris in the financial system, based on the assumption that previous growth trend would continue, finally led to all kinds of troubles that we are still seeing today.

I’m not so sure about Cowen’s view on the Internet. He devotes a whole chapter on it arguing that, while the Internet has brought us many great stuff, it creates few jobs, and people may even spend less because of all the (mostly free) fun they get from the Internet. Certainly eBay (one of his low job creation examples) does not hire many people — not at a scale of the Big Cars — but what about the benefits of disintermediation, which reduces the friction in transaction and makes many sellers reachable to more potential buyers (indie music comes to mind). But Cowen only spends a page to say that Google and Facebook hire less than 30k people combined and doesn’t give us numbers to argue that it’s indeed “not saving the revenue-generating sector of the economy.” What’s worrying about such argument is that, with politicians fixated on job creation (and that’s certainly not confined to the US), when they start pointing fingers at the tech industry that they aren’t creating many jobs, we may start to wonder where this may lead us.

One interesting, if also somber, implication of “the great stagnation” is political polarization, which Cowen doesn’t mention in his book but talked about in the aforementioned interview. This seems a good perspective to understand the political dramas (sometimes ludicrously baffling to foreigners) in the US.

Cowen prescribes bitter remedies: “Raise the social status of scientists” — and this is not easy as it sounds. He warns us that the recession will be long, although Japan shows the US how to cope with the decline as a society to a large extent. There’s a twist to the theme here, because Cowen also warns us to be ready when more low-hanging fruit comes:

[…] because sometimes low-hanging fruit is dangerous. The last time the world had a major dose of low-hanging fruit, a few countries didn’t handle it very well, including the Axis powers, the Soviet Union, and Communist China, among others.

Without the new technologies of the time, the totalitarian mistakes of the twentieth century would not have been possible. […] The record-keeping techniques of mass bureaucracy were used to control and often kill other human beings en masse. Only after bitter experience did fascist ideas become less popular, and social and political norms subsequently evolved to protect electorates against the fascist temptation.

So expect a bumpy ride to the future. I think that’s a good advice to everyone who’s lived the optimistic 1990s. The ending words are worth quoting in full:

In the meantime, we need to be prepared for a recession that could last longer than we are used to. We need to be prepared for the possibility that the growth slowdown could continue once the immediate recession passes. Part of science is coming to terms with its limits. The rate of scientific progress will continue to be uneven, sometimes grossly so. Yet reason and science have never been more important: If nothing else, a more reasonable and more scientific understanding of our predicament can help us cope, both intellectually and emotionally.

Back to the hard problems.

Indeed.

Using OS X’s Built-in ICU Library in Your Own Project

Unicode string is usually not difficult to handle1. The tricky part comes when you need to have knowledge of Unicode on things like:

  • Turning strings like “Café” to upper case or lower case (the toupper() or tolower() in Standard C Library won’t work)
  • Stripping all punctuation symbols. This is not as easy as it seems. How do you also strip various European quotation marks as well as full-width ones used in East Asian text?

Some languages, like Java, already have such Unicode knowledge baked in. For C and C++, the standard library to use is IBM’s ICU.

You can either build ICU from its source code, or use tools like MacPorts to install it for you. But you may also wonder: Since OS X has great internationalization support, perhaps OS X also uses ICU?

It turns out that OS X does include a version of pre-built ICU library2 placed in /usr/library/libicucore.dylib, but for various reasons — and I also assume it has much to do with the fact that ICU is written in C++, and library versioning with C++ is a pain — it does not include the headers for you to use.

Here’s how you can link against the built-in library:

  • Install a copy of the latest ICU
  • In your project file, set header path to include the installed ICU header (e.g. /opt/local/include)
  • Define U_DISABLE_RENAMING in your project (i.e. setting the compiler flag -DU_DISABLE_RENAMING=1)— this turns off ICU’s version renaming scheme so that you’ll be able to link against an older library (the one that comes with OS X in this case) with the latest header
  • Link against libicucore (with the linker flag -licucore)

As the name implies, it only contains the core ICU library, so it lacks quite a few things. For example, C++ methods such as UnicodeString::toUTF8, classes as StringPiece, StringByteSink are missing. A good way to check availability is to use the UNIX tool nm to dump libicucore.dylib. Your best bet is to stick to the C API. On the other hand, this saves the trouble of including your own copy of ICU.

  1. Read Joel Spolsky’s excellent primer on Unicode if it still isn’t to you. ↩

  2. Many of NS/CFString and NSCharacterSet’s internationalization features use ICU under the hood. I haven’t tested it, but I believe the tips provided here can also be used on iOS. ↩

Brad Cox’s “Object-Oriented Programming” (1986)

A few weeks ago I had a chance to re-read this particular book by Brad Cox1. It is also the first book that introduced the Objective-C language to the world.

It’s not a heavy book — at 274 pages including index, Cox has outlined his solution for the software reusability problem in a succinct language (yes, pun intended). The book spends the first two chapters on why reusability is essential to mitigate the complexity in building a large system, what Cox calls “software crisis”, and on how the object-oriented programming paradigm, which Cox describes using the analogy of integrated circuits (“Software ICs”), is in turn the way to achieve reusability.

In the following chapters, Cox describes the language that he designed to fulfill his vision of Software IC: Objective-C. A short introduction and evaluation of Smalltalk-80 is given, and comparisons with Ada and then C++ are also provided. Cox likes many aspects of Smalltalk-80: The language is designed for personal computing, with interaction and graphics in mind (remember this is the language that gives the birth to modern graphical user interface), and it has many advanced features in its time: virtual machines with persistent stores (so say “saving files” is an unnecessary feature), a language built on blocks (think lambda) and message passing mechanisms, an IDE, and so on. As for why Smalltalk-80 didn’t become mainstream, Cox’s explanation is that Smalltalk-80 is a language designed for research purposes. To be of practical value, a hybrid language that can work with existing infrastructure will have a better chance:

My own interest in hybrid object-oriented language developed from the dream that organizational computing and personal computing could meet each other halfway, via a bridge over which arbitrary data could flow with minimum programming intervention. The bridge is created by providing the organizational computer with its own object-oriented programming language so that data can be put into object form. Object passivation [persistence] machinery […] can automatically convert this data into a form that Smalltalk can read and vice versa, so that much of the information transfer could be done automatically. This, and C’s ability to access databases maintained on older languages like FORTRAN [Cox also mentions COBOL in previous paragraph], creates a bridge between data locked in the mainframe and the user by relying on the ability of Smalltalk-80 to build user interfaces rapidly.

For the rest of the book, Cox describes the features of Objective-C in detail with an example of using the language to build a visual graph manipulation tool. Some syntactical features are no longer found in today’s Objective-C (e.g. the = ClassName declaration, and the feature that class declaration and definition are one unified compilation unit, much like Java, instead of separate parts). In the first of the last two chapters, Cox outlines the architecture of an “iconic user interface” — the model-view-controller paradigm (MVC), which was also introduced by Smalltalk. The last chapter briefly discusses the runtime environment (such as garbage collection) and possibilities for other higher-level language features, including “virtual object memories”2, concurrency (using coroutines), distributed systems and coordination systems3.

Overall, I find the book full of insights that still resonate, 25 years after its publication. It’s interesting to take a historical perspective on how much progress we have made since then. Objective-C had never been as popular as C++, a “competitor” in some senses, or as Java, some features of which are actually influenced by Objective-C (such as interfaces, or what Objective-C calls protocols). NeXTSTEP, the first platform that heavily used Objective-C as the main language, was never mainstream. It takes Mac OS X, and then iOS, to propel it to one of the most popular programming languages worldwide. And many ideas are still waiting for more refined realizations. Some, like garbage collection, finally become an option on desktop platform. Some, like concurrency, are still things that frustrate programmers, expert or novice alike. Perhaps, like what some who teach functional programming would like to show, we are past the apex of the object-oriented model, and a new paradigm shift awaits.

  1. Cox, Brad J. Object-Oriented Programming: An Evolutionary Approach. Addison-Wesley, 1986. I’m referring to the first edition here. If you know the differences between its first and second editions, I’d appreciate it if you can write something about it. ↩

  2. We can identify some of Cox’s description of such a “capability-based” addressing system in today’s uniform resource identifiers (URI). ↩

  3. Although gcc has been supporting gc-capable Objective-C for years, it was not possible to use it in Mac applications due to lack of framework support. Apple eventually came up with its own “mix of curry” of concurrent, generational garbage collector, with runtime and full framework support, in 2006. Objective-C also has a feature called Distributed Objects, which is often used among Mac applications for inter-process communications. Library support for operation-based concurrency became part of the Mac OS X 10.5 framework, again since 2006. ↩

BugzKit, a FogBugz API Library for Objective-C

BugzKit, a FogBugz API Library for Objective-C

BugzKit is a FogBugz API library for Objective-C. I started developing BugzKit in mid-2009 at Lithoglyph as the foundation for LadyBugz, a FogBugz client for the Mac. Since the company has discontinued the development of the app, I think it’s time to release the library to the public as an open source project. I hope that by opening the library, it can help others develop new FogBugz clients, such as ones for the iOS platform.

The library is now available on GitHub and is released under the MIT License.

One important issue in designing an API library for FogBugz is data dependency. The case (ticket) data you fetch from the server is meaningless without the lists of projects, people, milestones, and so on. Therefore you want to fetch those lists before you can fetch the cases of a particular project or filter. Also, you want to fetch lists as well as cases in parallel, as there are many such lists and fetching them serially takes too long.

For example, in the BasicRequests sample app that comes with the library, we want to perform the following tasks:

  1. Check the API version and availability
  2. Login (FogBugz uses the term “log on”)
  3. Fetch three lists (projects, people, milestones) in parallel
  4. If all succeed, logout (“log off”)

You may notice that logout should be performed after Step 3. even if any task in that step fails: Since we have an auth token, we should logout. For the sake for simplicity, let’s stick to what the sample code implements.

Apparently, each step depends on its previous step. If there’s no API available (such as the service endpoint is down, or the URL is incorrect, etc.), you can’t login. If you don’t have the auth token obtained from the login, you can’t fetch the lists. If any of the steps fails, the program should cancel all other operations standing in line and exit. If we draw a dependency graph, it will look like this:

BugzKit solves the problem by separating request objects and request operations. Request objects encapsulates information required to make a request (URL, parameters, HTTP method name, etc.) whereas request operations are the actual drivers that perform the HTTP request. Once we have the operation layer, we can then make use of the operation queue framework in Foundation (NSOperation and NSOperationQueue are the two main classes). NSOperation has the better granularity (compared with threads) for handling operation dependency, cancellation, and convergence (e.g. two or more finished operations meeting each other). Extending the sample code to anything that works in the real world will take a lot more work, but I believe the library makes it at least manageable.

The API library covers most of the functions required to fetch lists and manipulate cases. It also handles attachments for you by automatically creating the temp files on demand and preparing an input stream so that you can send the byte stream as the POST request body (with the multipart MIME type). Many areas are not covered, though, such as wiki or time tracking. It should be easy to extend the library to support those FogBugz features.

The Hybrid Drive Upgrade

I replaced the old hard disk drive in my laptop as it showed signs of dying two weeks ago. There is a lot of talk about upgrading to SSD among the people I follow on Twitter. SSD is great, no doubt, but it’s still an expensive option.

Then I came upon this: a hybrid drive. It’s actually a traditional, spinning hard disk drive with 4 GB flash memory used as read cache.

I’m never a benchmarking person when it comes to hardware. I do have read the AnandTech review after having installed and used the new drive for a few days, and my experience agrees with the review’s findings. In some areas like boot-up speed, this drive does provide SSD-like performance. In other areas it’s just a fast spinning drive.

The unpleasant, unresponsive period after login is a thing of the past, and app launching is faster too.

It’s true that many Mac users seldom restart their machines, so a big selling point of this kind of hybrid drive, “it speeds up booting up”, seems moot. It’s true that the hybrid drive is more expensive than its traditional spinning cousins, at about 2x the price, but an SSD of the same capacity will cost about 6x-10x. So for a bit more money you get big speed improvement in one area and small improvements in other areas with this hybrid drive.

This drive does give me a small pleasant surprise: Improved virtual machine performance. For work reasons I usually keep 4-5 virtual machine images around, and I often have to fire up and then suspend some of them (I use VMware Fusion). There must be some complex interactions between the VM’s and the host’s file systems, and this hybrid drive seems to be of great help. Not only do my VMs start faster, but the long latency after resuming is also gone.

Improved virtual machine gives me a great incentive to virtualize more. Now I can compartmentalize many more things, like trying new apps, in separate VMs. In theory I could have already been doing this all along, but the slow start-up and resume time has put me off, and I seldom used VM other than for work. But this enables a whole new usage for me.

Overall I’m satisfied with the upgrade. I didn’t spend too much, and the marginal return is high. Those SSD upgrade tweets made me crave them but I was also wary of their pocket-burning tendencies. Now I won’t have to worry about them for at least quite a while.