Archive for the ‘Database’ Category

Building Equip2 DLL from the CVS

Friday, February 12th, 2010

Equip2 is a dataspace implementation that provides flexible access to programming language objects across a variety of platforms and programming languages. It is especially well suited as a higher level datastore for distributed applications.  Equip2 provides template-based data queries into the datastore and an integrated state/event model with change notification.

Equip2 is available on Sourceforge. There is a full paper from UbiComp 2007 entitled “Addressing Mobile Phone Diversity in Ubicomp Experience Development” that describes the reasoning behind the design of Equip2. Overall, Equip2 is really well documented, but even for a developer the available information can sometimes be a bit terse. This post is meant to supplement the existing installation guide and C++ build information.

Equip2 has been used in many of the ubicomp experiences that were developed at the MRL since about 2005. However, its learning curve is quite steep and it can actually be quite scary. This is especially true when getting in touch with its elaborate Java-Bytecode to C++ Sourcecode transcoding in the build process for the first time. Here are some notes from August 2007 that summarise my  experience with the build process.

[.. after CVS checkout and following the installation guide / C++ build information..]

Problems / actions:
cl.exe was not found
This is part of Visual Studio. Running the cl.exe from the vc7/bin folder didn’t work (mspdb71.dll not fount, etc.).
The solution is to take the VC command prompt from Start>Programs>Microsoft Visual Studio .NET 2003>Visual Studio .NET Tools>Visual Studio .NET 2003 Command Prompt
(as mentioned in equip2/doc/EQUIP2_CPP_Programming.html)

WTK was not installed and not mentioned in the build docs
Download and install the WTK 2.3, 2.5.1, or newer from Sun’s webpage

>> now it was possible to build /javatrans according to http://equip.sourceforge.net/javatrans/docs/Javatrans_introduction.html#Building_and_testing
“ant jar”, “cd targets/win32”, “buildlib-win32.bat”, “buildexample-win32.bat”

buildlog4j-win32.bat cannot find “java_lang_Object.h”
The file exists three times at the following locations below /equip2:

  • build.cpp/log4j.src.cpp/dependencies/java_lang_Object.h
  • build.cpp/src.cpp/dependencies/java_lang_Object.h
  • build.cpp/test.src.cpp/dependencies/java_lang_Object.h

The file also exists where it should be included from according the the batchfile at /javatrans/src.cpp/include

The cl.exe commandline sets the includedir in respect to %javatranshome% which is not set in the commandline-environment
(and also not mentioned in env.bat).
Solution:
‘set javatranshome=”D:\CVS-Dumas\Equator\javatrans\”‘
‘buildlog4j-win32.bat’
Will lead to the next problem, because javatranshome must be set to D:\CVS-Dumas\Equator\javatrans\
so:
‘set javatranshome=D:\CVS-Dumas\Equator\javatrans\’
‘buildlog4j-win32.bat’
works!

– ‘buildequip2-win32.bat’ now builds at first try

– ‘buildequip2test-win32.bat’ now works

Hope this helped someone.

Installing Turbogears meets IMDb from scratch

Monday, April 28th, 2008

I have given my “Turbogears meets IMDb” webprogramming and scripting lecture again and produced a little video for it. You can see me getting the project to run on a vanilla machine (Python 2.5 and easy_install already installed) in less than two minutes.

The video also shows a quick walk-through of the browser-based user interface, incl.:

  1. selecting a movie from existing collection and show movie info together with own notes
  2. search for a movie with fuzzy search-terms
  3. annotate and store a movie to collection

Note: the video is encoded as Flash and plays fine in IE7 but seems to have problems playing on my Firefox. So try IE if it doesn’t seem to play on your machine.

TurboGears meets IMDb

Thursday, May 3rd, 2007

I recently gave two guest-lectures at Nottingham University on web-programming using TurboGears. The first lecture was a general introduction to Model-View-Controller web-programming using TurboGears. The second lecture was more practical: I wanted to present a full mini-application and show the students how it was built. And I also wanted the application to be at least remotely usable and interesting – therefore I choosed to write a little movie collection web application as I thought everybody enjoys movies.

IMDb
The idea was to utilize movie summaries from the Internet Movie Database (IMDb). As you might know the IMDb is a very popular online database which covers about a million movies and more than two million people. It is one of the oldest collaborative online efforts (according to Wikipedia it launched 17.10.1990 and originated from the Usenet list rec.arts.movies of the late 1980’s) which has been acquired by Amazon in 1998.

IMDb data
What you might not know is that this supposingly closed source of information always had a free interface which you are allowed to use for private reasons. This interface consists of zipped text-files which can be downloaded from FTP servers. In the mid-nineties there was a very good programm for the Commodore Amiga computers called “Movie MUI” which used those files to built a local copy of the IMDb database and then allowed you to browse the data with a very nice graphical user interface. More recently, a Python module called IMDbPy became available. Since TurboGears is written in Python, it seemed obvious to try out IMDbPY from within TurboGears.

Required Funtionality
The idea was to be able to browse movies (using IMDb data), store your favourites and possibly annotate them. Therefore the webapp should provide the following functionality:

  • Search (fuzzy, showing matching titles)
  • Browse movie details
  • Store movie to favourites (and remove as well)
  • Add/edit notes

Webapp – Main screen
View larger

You can see a list of movies that I have compiled using the application itself. From this screen you can search the movie database or pick a movie from the collection to bring up the details, including your annotations.

Webapp – Search results
View larger

The above screenshot shows a search for “Silicon Valley Story”, which is the main part of the German title of a great 1999 TV movie originally called “Pirates of Silicon Valley”. The two interesting features to note here are:

  • You have a fuzzy search, also returning similar titles
  • You can search for non-english titles

Webapp – Movie details
View larger

This is the movie details screen. It is noteworthy that the search for the German title (which is still in the search box) has correctly led us to the movie “Pirates of Silicon Valley”. From here the user can go back to his search list, go to the main menu, store the movie to his collection or refine the searchterm.

Webapp – Store movie, edit note
View larger

You can add a note when storing the movie. Note that the movie will be stored under its original title with year of release, which is the IMDb naming convention.

Webapp – Movie in collection
View larger

The movie has been added to the collection and the custom note is display at the bottom.

Note on IMDbPY

I generated the local copy of the IMDb database on a 1.7 GHz Centrino laptop with 512 MB Ram, running a modified imdbpy2sql.py (from the IMDbPY package) with Psyco and outputting to SQLite with option autoCommit=False (?autoCommit=&). This was a very lengthy process which obviously has to be optimized in the future. It took over 11 hours to generate the indexed database on my laptop. The resulting file was just under 2 GB in size.

Here are the last lines of output from imdbpy2sql.py (look at the time spent between lines 2 and 3!):

”’[..] TOTAL TIME TO INSERT DATA: 46 minutes, 34 seconds
building database indexes (this may take a while)
# TIME createIndexes() : 646 min, 17 sec.
DONE! (in 692 minutes, 52 seconds)
”’

Summary
This was my first TurboGears project. The web application works as expected and I really enjoyed the development process. Feel free to contact me (via email or via the Xing profile provided in the sidebar) if you are interested in the sources.