TurboGears meets IMDb

May 3rd, 2007

I recently gave two guest-lectures at Nottingham University on web-programming using TurboGears. The first lecture was a general introduction to Model-View-Controller web-programming using TurboGears. The second lecture was more practical: I wanted to present a full mini-application and show the students how it was built. And I also wanted the application to be at least remotely usable and interesting – therefore I choosed to write a little movie collection web application as I thought everybody enjoys movies.

IMDb
The idea was to utilize movie summaries from the Internet Movie Database (IMDb). As you might know the IMDb is a very popular online database which covers about a million movies and more than two million people. It is one of the oldest collaborative online efforts (according to Wikipedia it launched 17.10.1990 and originated from the Usenet list rec.arts.movies of the late 1980’s) which has been acquired by Amazon in 1998.

IMDb data
What you might not know is that this supposingly closed source of information always had a free interface which you are allowed to use for private reasons. This interface consists of zipped text-files which can be downloaded from FTP servers. In the mid-nineties there was a very good programm for the Commodore Amiga computers called “Movie MUI” which used those files to built a local copy of the IMDb database and then allowed you to browse the data with a very nice graphical user interface. More recently, a Python module called IMDbPy became available. Since TurboGears is written in Python, it seemed obvious to try out IMDbPY from within TurboGears.

Required Funtionality
The idea was to be able to browse movies (using IMDb data), store your favourites and possibly annotate them. Therefore the webapp should provide the following functionality:

  • Search (fuzzy, showing matching titles)
  • Browse movie details
  • Store movie to favourites (and remove as well)
  • Add/edit notes

Webapp – Main screen
View larger

You can see a list of movies that I have compiled using the application itself. From this screen you can search the movie database or pick a movie from the collection to bring up the details, including your annotations.

Webapp – Search results
View larger

The above screenshot shows a search for “Silicon Valley Story”, which is the main part of the German title of a great 1999 TV movie originally called “Pirates of Silicon Valley”. The two interesting features to note here are:

  • You have a fuzzy search, also returning similar titles
  • You can search for non-english titles

Webapp – Movie details
View larger

This is the movie details screen. It is noteworthy that the search for the German title (which is still in the search box) has correctly led us to the movie “Pirates of Silicon Valley”. From here the user can go back to his search list, go to the main menu, store the movie to his collection or refine the searchterm.

Webapp – Store movie, edit note
View larger

You can add a note when storing the movie. Note that the movie will be stored under its original title with year of release, which is the IMDb naming convention.

Webapp – Movie in collection
View larger

The movie has been added to the collection and the custom note is display at the bottom.

Note on IMDbPY

I generated the local copy of the IMDb database on a 1.7 GHz Centrino laptop with 512 MB Ram, running a modified imdbpy2sql.py (from the IMDbPY package) with Psyco and outputting to SQLite with option autoCommit=False (?autoCommit=&). This was a very lengthy process which obviously has to be optimized in the future. It took over 11 hours to generate the indexed database on my laptop. The resulting file was just under 2 GB in size.

Here are the last lines of output from imdbpy2sql.py (look at the time spent between lines 2 and 3!):

”’[..] TOTAL TIME TO INSERT DATA: 46 minutes, 34 seconds
building database indexes (this may take a while)
# TIME createIndexes() : 646 min, 17 sec.
DONE! (in 692 minutes, 52 seconds)
”’

Summary
This was my first TurboGears project. The web application works as expected and I really enjoyed the development process. Feel free to contact me (via email or via the Xing profile provided in the sidebar) if you are interested in the sources.

First offspring of Lovecity

February 26th, 2007

26.02.2007, 12:20:
Venture, Tongue and View just created the first offspring of Lovecity. Their name is Silly. Welcome new citizen!

Lovecity up and running

February 22nd, 2007

A child is born: Lovecity, our location based mobile phone game, is up and running and participants are starting to play. Today we received about 250 SMS from players, including some from ourselves.

The last weeks have been a painful but highly rewarding time. I am extremely grateful to work with such a talented and enthusiastic team:

  • Capra and Michael do a great job on the website
  • Matt and Rachel provide the artistic input, polish things up and suffer from several nervous breakdowns (kidding!)
  • Robin and Christoph doing a tremendous job behind the curtains as lovely code monkeys

It’s been a blast. Thank you!

Visit the Lovecity website and join the game if you are from Derby, Leicester or Nottingham. It is live up to the 14th of march.

Getting to Grips with TurboGears

January 15th, 2007

I kept this log of my TurboGears installation process on my Windows XP Pro machine.

  • ~19:20: visited webpage “www.turbogears.org”
  • 19:25 – 19:28: installed running “tgsetup.py” with an existing Python 2.4.2
  • added c:\python24\scripts to the path
  • 19:31: ran “tg-admin info”
  • started writing this notes-file
  • 19:36: started to follow the “getting started” tutorial at http://docs.turbogears.org/1.0/Wiki20/Page1?action=show&redirect=1.0%2FGettingStarted
  • 19:37: “easy_install docutils” … distutils are working great!
  • 19:38: “easy_install pysqlite” is not working 🙁
    ”’
    No local packages or download links found for pysqlite
    error: Could not find suitable distribution for Requirement.parse(‘pysqlite’)
    ”’
    site mentioning such a problem on ubuntu: http://inre.dundeemt.com/2007-01-05/installing-pysqlite-in-a-workingenv-on-unbuntu-610/
    proposes:
    “easy_install -f http://initd.org/pub/software/pysqlite/releases/2.3/2.3.2/ pysqlite”
  • 19:42: that seems to have worked
  • 19:45: got IPython which was mentioned in the tutorial (http://ipython.scipy.org/dist/)
  • 19:48: “tg-admin quickstart” (output saved to quickstart_log.txt)
  • 19:51: ran webpage using “start-leiftest1” (output saved to server_log.txt)

Voila, just thirty minutes from first contact with webpage to a working own installation and I even had an unexpected problem to solve on the way. That seems pretty impressive to me – I like TurboGears!

Japanese (?) site talking about my Ubicomp paper

December 6th, 2006

I just stumbled across the japanese (?) Academic Ubiquitous InfoDB which clearly summarizes our Ubicomp 2006 paper. Cool, if only I could read those unfamiliar signs.

Out and about in the O.C.

September 18th, 2006

As part of the Exurban Noir workshop at the Ubicomp 2006 conference, I spent a day out in Orange County. Here is a description of what had been prepared for us:
Group 4: Private Investigations and boats

Guide: Rob W.

“Robert spent 30 years as a federal agent, training in Washington DC. He has been inside the White House at least six times, met several presidents, and got tossed out of Congress for asking too many questions. He once investigated and obtained a conviction of an Orange County congressman. He is a certified fraud examiner. He is now a private investigator based out of Newport Beach, and travels widely for work. He has been conducting investigations for a total of 35 years.

Your Mission

Rob will pick you up at the Marriott after the morning session of the workshop. He will take you for a tour of his office, elaborating on his day-to-day activities as a private investigator. On a normal Sunday afternoon, he’d probably be boating, so take the ferry across Newport Harbor. Visit some yachts. Before dinner, explore Fashion Island with him. He is very flexible about his itinerary, so if there’s anything you’re curious about, be sure to ask!”

Pictures of the day are online at the workshop blog. You can also follow the day by opening this KML for Google Earth (which has MP3 audio in it – works with Quicktime). Molly Steenson, one of my group mates during the workshop, wrote a detailed recap in her blog.
The workshop proceedings are here (PDF) – our position paper is called “The Digital City: Sex, Cams and Scams” and can be found on page 30.

Transcoding Audio

August 2nd, 2006

Ever ran into the problem of having an audio-file in one format but really needing it in another format? Then check BeSweet GUI, it’s awesome!

Encoding AVI to MPEG files

July 31st, 2006

Just a little note about TMPEGEnc 2.5x which is a fantastic, no non-sense video encoding program that just works. It can encode your (DV-)AVI files into MPEG. Options include PAL/NTSC, VideoCD/SVCD/DVD with or wihout Variable bitrate and all that stuff. The point here is that you can actually configure it down to the tiny details, but you don’t have to since it just works out of the box. The visual quality of the results is great, too, although a friend on mine argues that Cinemacraft files look better.

The free test-version gives you unlimited time to encode MPEG1 (VideoCD), but to use the MPEG2 (SVCD/DVD) capabilities for longer than a month you have to register the program for about 37$. Fair enough.

Doom9 DVD tutorials

July 31st, 2006

Similarly to the Digital Fact Book, the Doom9 page is a fantastic, more hands-on page full of tutorials and in-depth knowledge about video-enconding and transcoding. It has been online for years and is still updated frequently. Highly recommended!

Quantel’s Digital Fact Book

July 31st, 2006

Quantel’s Digital Fact Book is a valueable resource for all who are interested in digital video. I learnt about Quantel’s products while I was an intern in the VCC post-production in Hamburg. Some colleague recommended it to me back then and I was fortunate enough to be able to order a free printed version. I don’t think they provide you with free copies anymore, but you can grab the full thing as PDF from the above page.