Quantum GIS and Homebrew, Pt. 1

A few weeks ago I decided to give Homebrew as package manager a try and move some of my projects to Mac OS X as development platform. Until now I really liked the advantages of using Ubuntu as virtual machine, have development dependencies separated from my day-to-day system, safely try stuff out and be able to rollback to a snapshot if something goes terribly wrong. Sharing hardware resources is obviously the drawback of given setup, which basically made me think about alternatives.

Homebrew is great so far: very easy to setup, integrates well with Mac OS X, large repository and a great “Formula” system to add your own packages.

When it comes to GIS applications and libraries on Mac OS X, William Kyngesburye’s site is one of the best resources you’ll find out there. I used his’ downloads quite a bit. But with the move to Homebrew, I also had the intention to manage my GIS packages with it. Although Homebrew doesn’t provide user friendly double-click-to-install packages, like William does, its Terminal commands are simple enough:

$ brew install mypackage

Drawback of having Homebrew manage my GIS dependencies was, as I noticed immediately, that William’s Quantum GIS package didn’t work any longer. So I decided to build Quantum GIS on my own, from source, against dependencies managed by Homebrew. The entire build process is very well documented on QGIS’s wiki. Using Homebrew for getting all dependencies is as simple as:

$ brew install python --framework --universal
$ brew install postgis qt gdal geos proj \
sqlite libspatialite libiconv gsl pyqt qwt cmake

I edited the existing SIP-Formula slightly (added a “–universal” option) to meet QGIS’ requirements on Mac OS X.

$ brew install sip --universal

Next step is to configure and make QGIS. Some paths and libs weren’t found automatically by cmake, so I had to specify them:

$ cmake -D CMAKE_INSTALL_PREFIX=~/Applications \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_BUILD_TYPE=MinSizeRel \
-D WITH_INTERNAL_SPATIALITE=TRUE \
-D QWT_LIBRARY=/usr/local/lib/libqwt.dylib \
-D QWT_INCLUDE_DIR=/usr/local/include \
-D GEOS_INCLUDE_DIR=/usr/local/include \
-D GEOS_LIBRARY=/usr/local/lib/libgeos_c.dylib \
-D GDAL_INCLUDE_DIR=/usr/local/include \
-D GDAL_LIBRARY=/usr/local/lib/libgdal.dylib \
-D GSL_INCLUDE_DIR=/usr/local/include \
-D GSL_LIBRARIES=/usr/local/lib/libgsl.dylib \
-D POSTGRES_INCLUDE_DIR=/usr/local/include \
-D POSTGRES_LIBRARY=/usr/local/lib/libpq.dylib \
-D PYTHON_EXECUTABLE=/usr/local/bin/python2.7 \
..
$ make
$ make install

…produces the QGIS.app in your ~/Applications folder.

The Homebrew Formula for QGIS that I’ve started, doesn’t make a successful QGIS build yet. It fails somewhere at

Linking CXX shared library libqgis_analysis.dylib
[ 33%] Built target qgis_analysis
make: *** [all] Error 2

…although Homebrew is basically building against the same dependencies as my “manual” steps did – working on it.

So far so good. However, the bad news, maybe just in my case, is that QGIS seems to be linked to Mac OS 10.6 original Python version 2.6.1 and throws following error at startup:

Couldn’t load SIP module. (note: SIP is installed in /usr/local/bin)
Python support will be disabled.

Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named sip

Python version:
2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)]

Major downer here is the second line, “disabled Python”, which means most plugins won’t work. This is major since one of Quantum GIS’ strength is its plugin-architecture.

To be continued…

Snowboard traces in Vail

We had a blast last week visiting friends in Denver, Colorado. For three days we went up to Vail, which is one of the best skiing regions I’ve ever been too – great slopes and awesome back country terrain!

All mine

The tiny green geonerd sitting on my shoulder made me turn my GPS on while being out on the mountain. Carrying the GPS receiver in the pocket of my jacket worked surprisingly well and resulted in about 160km of neat slope and lift traces. Here they are, shown as KML overlay on a Google Map…

…or here, uploaded to OpenStreetMap, waiting to be mapped and tagged for OpenPisteMap.

Distortions

An interesting piece – Medusa, 2001, by William Kentridge – that we recently saw here at MassArt, reminding me of map projections and how distorted and often misunderstood the pictures are that we are drawing of our planet.

R visualizations

New Year’s resolution: harvest some knowledge from those colleagues who are digging through R. Apparently it’s not only an excellent tool for number crunching, but also can be used for neat geographic data visualizations.

Visualizing Facebook Friends: Eye Candy in R

Facebook visualization using R

Open Source for Non-Profits

There is one question that won’t stop spinning in my head recently:

You decide to open-source your software development project, but at the same time limit the Open Source characater to non-profit use only. The idea is to have a second commercial side of the project, that cross-subsidies the non-profit branch through for-profit licensing.

The main difference compared to other popular existing dual-licensing schemas like Ext JS or MySQL is, that the Open Source license and character only applies to non-profit use. Imagine “GPLed for Non-Profits”.

I don’t seem to be able to find a best-practice or successful implementation of such a license schema.

Why is that?

Open Source is around long enough. It is applied in academia, large non-profit foundations and global organizations and the developer community seems to often support the philanthropic cause.

Topology rules with GeoDjango

I’m still very new to GeoDjango and the Django world in general, but I have to say that, based on what I’ve seen so far, I’m very impressed by how easy, fast and clean you can build powerful geographic web applications with it.

Django 1.2 is the current version and one thing that caught my eye in the release notes was Model validation. Among other benefits, model validation allows you to “perform custom validation on your Model” everytime a user wants to save data through a ModelForm. Translated to GeoDjango, that basically means that one can access the unleashed power of PostGIS (or the functionality of your supported geo-database of choice) to validate geometries and provide feedback for users through a nice web interface. Best part of it: it’s all done only by a few lines of additional code in your data model.

I’m not going into much detail how Django’s model validation works, you should check out the documentation if you’re interested, but here’s a quick example of creating something like a simple topology rule with GeoDjango:

Let’s say I have users entering and modifying point data and need to make sure that each point falls inside a polygon from another layer.

# models.py
from django.contrib.gis.db import models
from django.core.exceptions import ValidationError

class Poly(models.Model):
	geometry = models.PolygonField()
	objects = models.GeoManager()

class Point(models.Model):
	geometry = models.PointField()
	objects = models.GeoManager()

	# simple topology rule
	def clean(self):
		# find Polys that contain the Point
		pq = Poly.objects.filter(geometry__contains=self.geometry)
		if not pq: # no Poly has been found
			raise ValidationError('Point is not within a Poly.')

That’s it. You only need to add a custom clean() method and put your validation rules there. Those few lines of additional code check if the new point falls inside a polygon and tell the user if it’s not through the Django admin interface.

GeoDjango Model Validation

I’ve posted the project on bitbucket if you want to give it a shot. Add a little more PostGIS extravaganza and you can have a pretty sophisticated set of topology rules in your web application, bundled with the great capabilities Django apps offer.

Favorite ArcGIS 10 feature so far…

ArcGIS 10 can HAZ_GDAL
Just finished updating my machine to ArcGIS 10. Best part so far is clearly the improved Python integration, instead of VBA.

GeoDjango on a Windows server

Deploying GeoDjango projects on a Windows server is obviously possible, but not necessarily the most seamless process one can think of. To get started, there are some great tutorials – at the django project site and here, with some more information about the correct PATH settings – available, that guide you step-by-step through a GeoDjango installation.

Below are some things that I did in addition, did differently, or simply couldn’t do because I never got them to work.

The basics: Django and IIS

IIS is was the web server of choice on the Windows machine in question. Serving Django through IIS is of course doable, there is even a dedicated page on the Django project about it. However, I totally failed in getting Django to work well with IIS. It seemed painfull and eventually made me switch the entire web server to Apache. Django integration with WSGI under Apache is a piece of cake compared to PyISAPIe and IIS if you ask me, up and running in less than 10 minutes.

Virtual Environments

Virtual Environments are great when developing with Python. They give you separated Python installations for every project you work on and avoid interferences with dependencies and installed packages across your projects. Virtual environments is something I’ll really miss on Windows, because I never got it to work correctly. Virtualenv basically creates the correct folder structure and activates a Virtual Environment to work in, but for some reason dependencies in my projects were attempting to access the main Python installation. Which obviously defeats the purpose of Virtual Environments.
What I ended up doing was, using one Django installation and being extremely careful about what goes there. Basically I have to manually validate, check and test that new packages for new projects don’t mess with existing once. Painful, I tell ya.

win-psycopg headache

I ran into serious troubles with psycopg 2.2.1 (the PostgreSQL adapter for Python). My project wouldn’t connect to the PostgreSQL database server at all. I found this entry at stackoverflow, which describes exactly the same issue. The posted solution worked for me too: use psycopg 2.0.14 instead of 2.2.1, no problems so far.

Multiple Django projects in one Apache/WSGI instance

This was the most annoying issue: turned out that only one project at a time was working on the server. That might be an option for the iPhone, but it clearly was unacceptable in my case. The problem caused by the “Python for Windows extensions” is documented in this thread, and my solution, similar to the one posted at the end of the thread, was to install pywin32-212 instead of pywin32-214.

Lessons learned

If you have a choice, get a Linux box to deploy GeoDjango. Anything but .NET seems painful to serve on Windows.

About scales

The Metric System is one of the things you’ll start missing most after a transatlantic move. Distances are no longer distances, voluminas and weights are messed up, soccer-fields and hectares don’t mean anything to anybody in this world. Smartphones and little converter apps are the only things that give you some clue about what’s going on.

Now, over a year after my move over the ocean, I still catch myself responding to a “400ft on the left hand side” with a silent “whatever”, and trigger a recalculation-process to the system I’m familiar with. Seriously, given the fact that there is absolutely no easy way to master feet, miles, yards, acres, sq mi, ounces (volumina and weight in one measure, for Christ’s sake!), pounds, pints, cups, gallons, etc. I started asking myself, how did this country even get so far?

The one scale I started appreciating however, is Fahrenheit. In terms of human readability and usability, I think Fahrenheit is a great scale. First you need to know that people refer to temperature here with “it’s going to be in the 60ies”, meaning the temperature will be between 60F to 69F. 9 deegree variation seems a lot if you’re used to Celsius, but it’s not that big of a deal in Fahrenheit. I’ve learned that the term “in the 60ies” gives you very precise information about how to prepare for the temperature. Much more than I’ve ever learned to correctly read the Celsius scale. Let me illustrate this with my personal Fahrenheit scale, that I’ve sort of developed over the last year:

  • everything < 50F: freezing, winter coat weather, below 40F thick winter coat
  • 50ies: cold, don’t leave the house without jacket
  • 60ies: chilly, you might want to bring a light jacket or some long sleeve shirt
  • 70ies: nice warm, t-shirt, maybe shorts, I still prefer long trousers
  • 80ies: hot, clearly shorts conditions
  • > 90F: sweat, don’t forget your water bottle

See?

In Celsius the same information would be much more granular and not as easy to memorize. Usability of Fahrenheit is way superior to Celsius’. Despite the insanity of all the other scales, I really came to like Fahrenheit a lot here.

Boston mapping folks!

… join the

Boston OSGeo Enthusiasts Meetup

next Tuesday, May 18th, 6:30-8pm

at the Bocoup Loft (319 A St, Boston, MA 02210)


View Larger Map

… and meet other Open Source mapping software users and developers, share experiences, learn about tools, discuss projects and developments.

To RSVP, please just add your name to the Doodle.

For more information and to participate in planning the meetup, please join the mailing list and follow the May meetup thread.

See you there!