Luke’s Blog

So I walked into one of the toilets in the engineering building at university today (wow!), and came across this piece of collaborative work:

And, of course, no university is complete without an Arts Degree Dispenser:

Ah…. the things students do between classes.

So, Google has released App Engine.

It is pretty interesting from what I have read so far, and you will find some of the common features that you find in most application servers, like automatic scaling or clustering, load balancing, persistent data stores, and, of course, dynamic web serving.

Here is a brief run-down of some of the features that caught my eye.

App Engine includes a very simple web application framework, webapp, and also the popular web application framework with a solid foundation, Django.

The datastore used by App Engine is NOT a relational database, which I thought was an interesting decision. It makes use of an SQL-like language, GQL. One of the advantages of the API that Google has provided is how you can construct queries. In many APIs, to create prepared statements, the developer is forced to use placeholders like this, for example:

select * from Users where name = ? and phone_number = ? and location = ?

And then, using whatever functions the implementation language provides, you pass in the values for name, phone_number and location, in that order. However, as queries increase in complexity and the number of placeholders required gets larger, it can introduce a lot of confusion, particularly when it comes to testing and debugging. Although the Google API allows developers to construct queries in this way, it also allows the use of named placeholders:

q = GqlQuery('select * from Users where name = :name and phone_number = :phone and location = :location',
    name='Luke',
    phone='12345678',
    location='Australia')

Which I think is much easier to read and debug. This is very similar to how you write queries with JPA, although Python’s keyword arguments make it look a little more elegant I think.

The datastore API also allows you to use keyword arguments and member functions of the Query class to do things like limit the number of results, order the results by a specific field and other common filtering techniques.

Another useful, time saving and, as far as I know, unique feature of App Engine is that it allows for integration of Google accounts, via the Users API. This means that, rather than your application having to keep a database of its own registered users, thus forcing the user to create yet another account for yet another system and forcing you to keep track of usernames and passwords for yet another application, the user can just login with their Google account details and start using your application straight away. The Users API also allows you to distinguish certain Google accounts as being administrators of the application.

Of course, no service like this can be free without a few constraints.

At the moment, during the “preview period”, there is a restriction that the free accounts can use no more than 500MB of storage, and have a limit of 5 million page views per month. You may also only create up to three applications. Non-free accounts are not available at the moment, so users cannot purchase extra resources yet.

Data queries may not return more than 1,000 results and will be pruned back down to 1,000 results if they exceed the limit. This could be problematic, depending on your application, but I believe that work-arounds are possible.

Something I was not able to find any information about was whether or not connection pooling is provided for database connections, or similar.

As a first release, Google App Engine seems to covers a lot of ground, but the only really different features that you will not find in most other application servers is the integration with Google accounts and the support of Python. The rest are common to many other application servers. However, with the features provided, the free hosting of applications, with certain constraints, and the support of a high-level, dynamically typed programming language, it looks like a fantastic platform for the rapid development of applications that require scalability, a persistent data store, user accounts and the high-level functionality of the Python programming language.

edit: A more in-depth article can be found here.

Almost a year ago I bought a MacBook Pro to replace my PC that was suffering from a broken motherboard and a blown power supply, caused by my brother when he attempted to find out what happened to the motherboard. Up until then I was using a small machine with 128 MB RAM and a little over 1 GHz CPU and FreeBSD 6.1 as the primary OS (using evilwm, in case anyone was interested). For my needs, this small machine was perfect, aside from how long it took to compile new versions of certain software that were not yet available via Fresh Ports (like OS X’s Darwin Ports, but for FreeBSD). However, until about a week ago I never had a dire need for a word processor, and I needed one quickly in order to write a report for the first assignment of my Enterprise Java subject.

I tried to download OpenOffice, because that is what I was using before I bought the MacBook Pro. However, the OpenOffice download for OS X, at that time, was not available, and there was no way I was going to fall into bad habits by installing Microsoft Office on OS X, disregarding the fact that I didn’t have MS Office for OS X on-hand anyway. And then I remembered that Google had an online word processor, Google Docs. And since I had recently moved all my email into a Gmail account, and had also started to use Google Calendar, I decided to have a go at yet another Google web app.

To start off with, I didn’t expect a whole lot since it still had “beta” attached to the icon. And the first thing I noticed was it did not work on Opera at all, which I prefer to use as my main web browser these days. The menus seemed like inanimate objects which made it impossible to even create a new document. So I switched to Safari. It worked on Safari.

The only other problem I had with Google Docs on Safari was that whenever I would write a heading, in 24pt size and bold font, that then became the default font settings for the rest of the document. And even if I started to write a new paragraph and had set the font back to 10pt with no bold, it would change the font back to 24pt bold text. So I got around this by first typing up the main content of the document, and then added any headings I needed when I was finished. Had this been a document of 10+ pages, then not being able to add headings as I worked through the document would have been frustrating and confusing.

Aside from that, it was very easy to write up the document. The interface is aesthetically pleasing and is set out like the majority of word processors, thus taking literally seconds to figure out how to do everything I needed to do. The other feature I liked was that I was able to export the document in a number of different formats (MS Word, HTML, OpenOffice, RTF, PDF etc…) Google Docs also allows you to share documents publicly, to allow for collaborative writing of documents.

In my opinion, Google Docs, as it is at the moment, is most suited to small-to-medium sized documents that do not require any relatively advanced formatting. I also think that it would be perfect for groups of students who have to collaboratively work on assignments.

All in all, Google Docs provides users with the basic functionality one would expect from a word processor, but with the added convenience of storing it online and allowing for collaborative writing, thus saving the user from having to carry their documents around on a USB flash drive, CD or, for the truly masochistic, floppy disk and also from having to email documents back and forth with other people. Without a doubt, I am definitely going to use it during the year for other assignments.

So I proved last year that I suck at regular blogging. But that was mainly due to the fact that after I got home from work, the absolute last thing I wanted to do was touch anything that even remotely resembled something to do with programming because I had just done eight hours of it. But hey, it’s worth another shot this year now that I’m back at uni for my final year. Only this time I’m not even going to try to do it regularly. Just whenever I do something new and sort of interesting.

I’ve spent the last few weeks playing with Ruby on Rails for an assignment, and will possibly be using it again for my final year project at university. At the moment that’s about it in the programming spectra. I still occasionally write some Haskell and some Python and am still planning to do some more work with Erlang. Whether or not I have time to actually do something interesting this year… who knows. But if I do, I’ll be sure to write about it.

Last week I was restoring an Oracle database from backups. The machine I was using had two partitions: a 10GB (primary) and a 20GB. The size of the backup set was just over 4.5GB. After installing Oracle 10g, the primary partition had about 4.9GB of free space; just enough space to house the backup set. But, when you restore a database from backups, you need another 4.5GB of space, which I didn’t have. Not having much experience with Oracle, I figured that I could just reinstall Oracle onto the larger partition and it would restore the datafiles onto that partition. I was wrong

Contained within Oracle an backup is a bunch of metadata (the spfile for the Oracle-savvy person), including which directory the datafiles are to be written to. However, this directory is not relative to where Oracle is installed. It is an absolute path. So even though I installed Oracle onto the larger partition, Oracle’s Recovery Manager still wanted to write the datafiles to the smaller partition and I had the problem of a lack of hard drive space again.

I thought to myself “if this were Linux I could just use a symlink to get past this stupid problem.” After Googling for “windows symbolic links”, and not expecting to get any meaningful results, I came across Junction. It was written by the people at Sysinternals (they have many great tools).

Junction creates symbolic links (they are called junctions in NTFS, not surprisingly) to directories only. Not files. But that was enough for my needs. Apparently this functionality has been around since NTFS 5.0 (Windows 2000 onwards) and is part of the Windows API, however it is a very undocumented and not well-known feature.

To get past the issue with Oracle, I created a junction between the folder Oracle needed to write to and a folder I created on the larger partition. Crisis over.

I had never heard of junctions before and thought such functionality would require some kind of cheap hack. So I thought it was cool enough to post.

Following on from my last post, a sixteen year-old student has cracked the government’s $84 million porn filter in half an hour. What a waste of money that was.

Considering getting through filters and hacked-up security policies was all we did in high-school, I’m not surprised, and am actually glad, someone this young was able to crack their crappy system.

Nothing will get in the way of a kid who wants to watch porn

Yes, the Australian Federal Government wants to waste $90 million of taxpayers money in an attempt to filter out porn to protect children. And they want to filter it at the ISP level! I have no problem with spending a further $40 million to help track internet predators. However, spending more than double that on porn filtering at the ISP level is a joke.

Of course this is an entirely feasible idea that can guarantee the safety of childrens’ precious minds when browsing the internet! And these new “software-based internet filters to install on their home computer” will be completely different from all of the previous software packages developed over the past decade and is a great way to spend $90 million of our money! Yes, they will succeed where everyone else failed.

And don’t forget, we have super high-tech image analysis software that tells us whether or not an image is of a pornographic nature and the studies on text analysis gives us the know-how to decide programatically whether a web site contains even the slightest amount of inappropriate sexual innuendo.

To slightly misquote one amused Australian on a completely unrelated matter: my initial response to this idea was one of voluble hilarity. I literally burst out laughing aloud.

Thankfully, the article from The Age was able to quote techinically savvy employees from various ISPs who were able to backup the fact that this idea is “unworkable”. What the hell was the government thinking?

About a month ago, I bought a MacBook Pro; the one with the 15 inch screen, 2.33GHz Intel Core 2 Duo processor, 2GB RAM and 120GB hard drive. So far, I’m pretty damn impressed considering this is my first proper Mac toy and is also the first time I’ve been able to properly experience OS X, although there are still some things that are taking time to get used to.

First off, knowing whether or not to press the function, ctrl, option or Apple key is a bit of a challenge. For example: I downloaded Opera to use as my primary web browser. However, I have to use Apple-T and Apple-W to open and close new tabs and Option-Tab and Option-Shift-Tab to move forwards and backwards through the tab listing. Contrast this with any other Linux, BSD or Windows system I use and I simply use the Ctrl key in place of the Apple and Option keys. The positioning of the Apple and Option keys also means that I now have to use my thumb as opposed to my little finger which I have used on all of the other keyboards I have on other machines. I know it’s only a small usability issue, however opening/closing/moving-between tabs in a web browser is probably the most common thing I do when sitting down at a machine, so it affects my web browsing a bit.

And finally, the only other problem I have had is with Boot Camp (so I can play games on Windows), although I guess I can’t complain given that I decided to use a beta version rather than wait for a full stable release, which I assume will be released along with Leopard later in the year. As far as the Windows XP install goes, there haven’t been any problems you wouldn’t get from any other Windows XP installation, however I have noticed some small bugs in OS X. For example, sometimes now when I have Quicktime open and I Apple-Tab between other applications and back to Quicktime, the other applications occasionally remain on top of the Quicktime window. The other thing I have noticed is when I empty my trash, there are still files in there, but they don’t exist; the icons for the files just haven’t been removed from the Trash. Simply clicking on the icons makes them disappear though.

On to the good stuff now.

Installing applications from .dmg image files is the simplest, most hassle-free process I have come across. Sometimes they even have a small screen with two icons; one representing the application you are installing and one representing your Applications folder and a big fat arrow pointing from the former to the latter; all you do is drag and drop. Very simple. And for those programs where an image is not available (for example, all of my favorite compilers and interpreters ), I have been able to use Darwin Ports, akin to FreeBSD’s Fresh Ports, to install a port of the program.

It has been hard to find an instant messaging client that satisfies my need of MSN’s childish nudge, wink and webcam features, however I’m willing to settle on Adium for now, which has a very nice interface. I use iChat for video conferencing.

I think it’s funny though. No matter what operating system I find myself using, a terminal and Emacs (or Aquamacs now) are the first programs I open and they stay open 95% of the time. I guess the hardcore Unix nerd inside never dies, no matter how pretty the interface.

And finally, Cocoa applications look fairly pretty too and they make me want to learn Objective C Maybe next year though. Erlang is keeping me busy for now

A long awaited O’Reilly book on Haskell is now in the making and when I saw the list of planned chapters for the book, I can’t say I wasn’t excited!

They are planning to include content of which I still, after the 18 months or so of using (and still learning) Haskell, have not looked into, specifically concurrency and parallelism, foreign-function interfacing, GUI programming with gtk2hs and web applications (presumably with HAppS). Of course, it never hurts to have a few chapters on monads and monad transformers although I was hoping to see a chapter on arrows and/or comonads, however given that the purpose of the book is to allow the reader to write real-world Haskell code as quickly as possible, I assume it was decided that these topics were too academic and not as real-world applicable as the rest of the book.

With the upcoming Erlang book from the Pragmatic Programmers on its way too, I think I have a lot of reading ahead of me

Oh, and an introductory article on Haskell from O’Reilly has popped up too. It’s not surprising that the article has surfaced around the same time the book was announced (a couple of days before actually)

Have a read of this.

I have read various blog posts and articles about Kathy Sierra’s troubles over the past month or so, and I cannot believe some of the crap she has had to deal with.

And some people wonder why there aren’t more women in IT…