Latest inklings

I can never remember this.

A Python Love Story: Virtualenv and Hudson

I’m going to take a look at using Jenkins for CI at work.

Irish political parties: how complex are things?

Contrary to what seems to be opinion elsewhere, the Republic of Ireland has a pretty straightforward set of political parties these days. Here’s a breakdown for the interested and bored. Links will be to the Wikipedia pages on the parties because I don’t endorse any of them.

Fianna Fáil (FF)

These days, Fianna Fáil is your bog-average vaguely centre-right, populist/big-tent party. Nominally republican, they haven’t been ideologically so since the late 1980s in any meaningful sense. They see themselves as the natural party of government, and their ideological rhetoric shifts so as to maintain that status.

They can be said to be slightly to the left of Fine Gael, but being a big tent and given their tendency to shift their rhetoric to suit the current situation, it’s almost meaningless to ascribe any particular position to them.

Fine Gael (FG)

The other centre-right party in the country. They differ from Fianna Fáil in that they’re a christian democratic and liberal conservative party, thus are easier to pin down ideologically, though they did flirt with a very mild form of social democracy in the 1980s.

The only interesting twist to all this is that due to a century-old grudge between FF and FG dating back to the foundation of the state, neither party will contemplate going into coalition with one another. Thus rather than having a single conservative party with two separate wings, one populist, the other christian democratic, we have two parties filling those roles. Aside from a lingering hate for FF, FG haven’t been anything other than nominally republican for decades. Even the ‘Christian’ bit is moderating these days.

Labour

A bog-average centre-left social democratic party.

They tend to end up in government to make up the numbers with either FF or FG. They’d likely be bigger if it weren’t for FF’s big-tent approach to organisation which keeps people whose ideological home might be more naturally Labour than a centre-right party like FF.

Greens

A bog-average centre-left progressive green party. About the only remarkable thing about them is that they’re slightly eurosceptic.

They were recently blown out of the water after being part of a disastrous coalition government with FF and the Progressive Democrats, a free-market liberal party that completely imploded over the course of 2008 and 2009. They ended up losing support due to a perception that they were too willing to bend to FF’s natural populism.

Sinn Féin

The only meaningfully nationalist/republican party in the state. They’re a secular democratic socialist party with a Trotskyist bent that’s giving way to pragmatism where necessary. I’m not sure I’d describe them as having ever really been sectarian, at least not under Gerry Adams anyway: any Catholic chauvinism was more of a fringe within them compared to their republican and democratic socialist core.

And that’s it: aside from Sinn Féin (who are turning into just another left-wing party anyway) and lingering political animosity between FF and FG. There’s really very little that can be said to be odd the structure of Irish politics.

∞

Know Thy Complexities!

Good algorithmic complexity reference!

∞

FreeNAS 8 on Intel SS4200-E

Getting one, putting the other on it.

∞

5 useful tips for a better commit message

Mainly git-focused, but most of the tips are generally applicable.

∞

Study: A Simple Surgery Checklist Saves Lives

From a few years ago, but still relevant.

∞

SimpleParse

Python parser generator that supports EBNF.

∞

Creating a simple LDAP application

ldaptor looks like it would be a useful LDAP server to development purposes. This post outlines how to hack together a simple server with it.

∞

How to change structural objectClass in OpenLDAP

This only works as of OpenLDAP 2.4.*, and only properly since OpenLDAP 2.4.27.

∞

Using Sphinx to Write Technical Books

∞

Tips for Using Mutt

∞

Fixing E.T. The Extra-Terrestrial for the Atari 2600

∞

Learn Vimscript the Hard Way

I’ve always intended on actually learning Vimscript properly. This looks like a good way of doing it.

∞

HTTPretty

HTTP client mocking tool for Python.

∞

The OAuth Bible

Nice overview of OAuth and its workflows.

∞

Setting up ReviewBoard without mod_wsgi

Yesterday I set up Review Board at work to see if it might be a better way to conduct code reviews than RhodeCode, which has code review functionality, but it’s never worked particularly well for us.

Owing problems we’ve had in the past¹, we rarely use mod_wsgi these days and prefer run our applications as standalone daemons managed by Supervisor that listen on the loopback interface, with Apache² acting as a reverse proxy using mod_proxy.

First thing I did was create a virtual environment for the application to run in:

# mkdir -p /opt/reviewboard
# cd /opt/reviewboard
# virtualenv --no-site-packages env

The server in question is running Debian Squeeze, which makes it necessary to use the --no-site-packages flag with virtualenv.

Review Board requires easy_install, so pip is a no-go:

# env/bin/easy_install ReviewBoard

It’s advisable to have a memcached instance running, so that was installed from APT.

As we authenticate off of a central LDAP server, I installed the python-ldap from PyPI into the virtual environment with pip:

# env/bin/pip install python-ldap

As we’ll need a WSGI server to run it under, I chose to use waitress along with a command line runner I use called waitress-serve³ to make starting instances of waitress on the command line easier:

# env/bin/pip install waitress
# env/bin/pip install https://github.com/kgaughan/waitress-serve/archive/master.zip

With that done, it’s time to generate the site with the rb-site command:

./env/bin/rb-site install site

After asking a few questions, that will create a directory called /opt/reviewboard/site to contain the site assets.

Out of the box, reviewboard doesn’t have a way to run standalone. To get around that, I had it generate a mod_wsgi site, given that was the closest option, and took a look at the .wsgi file generated. It looked like this:

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = "reviewboard.settings"
os.environ['PYTHON_EGG_CACHE'] = "/opt/reviewboard/site/tmp/egg_cache"
os.environ['HOME'] = "/opt/reviewboard/site/data"
os.environ['PYTHONPATH'] = '/opt/reviewboard/site/conf'

sys.path = ['/opt/reviewboard/site/conf'] + sys.path

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

As we’ll be running this as a standalone server to be managed by Supervisor, I used this as the basis of the Supervisor configuration block for the application:

[program:reviewboard]
command=/opt/reviewboard/env/bin/waitress-serve
  --expose-tracebacks
  --url-scheme=https
  --host=127.0.0.1
  --port=8005
  --call
  django.core.wsgi:get_wsgi_application
user=nobody
stdout_logfile=/opt/reviewboard/site/logs/%(program_name)s.out.log
stderr_logfile=/opt/reviewboard/site/logs/%(program_name)s.err.log
environment=
  DJANGO_SETTINGS_MODULE="reviewboard.settings",
  PYTHON_EGG_CACHE="/opt/reviewboard/site/tmp/egg_cache",
  HOME="/opt/reviewboard/site/data",
  PYTHONPATH="/opt/reviewboard/site/conf"

Normally, waitress-serve expects a WSGI application object to be specified, but as none is created by default in a Django application, I used its --call flag so that a callable could be specified to call to get an application object.

With that set up, it’s time to set up Apache:

<VirtualHost *:80>
    ServerName reviewboard.example.com

    RewriteEngine on
    RewriteCond %{HTTPS} !^on$ [NC]
    RewriteRule . https://%{HTTP_HOST}%{REQUEST_URI} [L]
</VirtualHost>

<VirtualHost *:443>
    ServerName reviewboard.example.com

    DocumentRoot /opt/reviewboard/site/htdocs
    ProxyPass /static !
    ProxyPass /media !
    ProxyPass /errordocs !

    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:8005/
    ProxyPassReverse / http://127.0.0.1:8005/
    SetEnvIf X-Url-Scheme https HTTPS=1
    RequestHeader Set X-Forwarded-Proto https

    SSLEngine On
    SSLCertificateFile /path/to/certificate.crt
    SSLCertificateKeyFile /path/to/certificate.key
</VirtualHost>

The ProxyPass directives just following DocumentRoot ensure that any static content is handled by Apache. The block that follows that passes any other traffic through to the server we’ve bound to the loopback interface.

With all that in place, tell Supervisor and Apache to reload their configuration:

# supervisorctl update
# service apache2 graceful

And bingo! It should be working!

Caveat regarding LDAP authentication

When I set up LDAP authentication, I noticed it wasn’t working and there were some bizarre errors in the log:

WARNING:root:LDAP error: {'info': '(unknown error code)', 'desc': "Can't contact LDAP server"}

After some hunting and experimenting, I discovered that the problem was that python-ldap was doing certificate checking. To get around that, I opened up the reviewboard.accounts.backends module, found the LDAPBackend class, and in the authenticate() function, inserted the following statement following the import ldap statement:

ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_ALLOW)

And it was able to authenticate off of LDAP.

YMMV: mod_wsgi is still a solid piece of software, but it just didn’t suit our deployment scenarios. ↩
We may switch to Nginx for this for lower overhead, but it’s not a priority. ↩
I’ll either be submitting it as part of a pull request to be included in waitress itself, or package it up myself and submit it to PyPI, as soon as I make it a bit more user-friendly. ↩

∞

password-store

Simple password manager using gpg and ordinary unix directories.

Repo is here.

∞

IETF Tools

∞

Skeleton

Mobile-friendly CSS framework. Source is on Github.

∞

pathod and pathoc

Pathological HTTP server and client designed for fuzz testing HTTP clients and servers.

∞

Alembic

Database migrations for SQLAlchemy.

∞

wireframe.cc

Wireframing tool.

[It’s just occurred to me that when I’m coding up this site’s replacement, I should really include something for tagging.]

∞

Firefox and Chromium plugins I'm using

Since I’m doing a tab clearout of my browser in work, I’d might as well note the plugin’s I’m using:

Work (Debian 7.0)

Firefox/Iceweasel 10.0.12

Bookmark Current Tab Set
Bookmark All - not sure why both this and the above are installed, so must see which I’m actually using. (Removed, 2013-04-02)
Duplicate Bookmark Detector
British English Dictionary
Download Statusbar
Firebug
Flashblock - ads are one thing, but Flash ads are anathema.
Linky (Removed, 2013-04-02)
Live HTTP Headers - has functionality that ought to be in Firebug, but isn’t.
MemChaser - haven’t had need of this for quite some time, so may get rid of it. (Removed, 2013-04-02)
pwgen - for when I’m too lazy to open up a terminal. (Removed, 2013-04-02)
Saved Password Editor - makes certain applications I use at work much less of a pain to deal with.
Web Developer Toolbar - given how little web development I do these days (it’s almost completely backend stuff), you’d think this’d be nearly useless to me, but it’s sometimes bypassing nonsense that some sites inflict on their readers.

∞

Song of GitHub

Let me sing you the song of my contributions.

I haven’t listened to what mine sounds like, but it’s here.

∞

GeoNames

The GeoNames geographical database covers all countries and contains over eight million placenames that are available for download free of charge.

Commercial access costs are pretty reasonable too.

∞

Sherlockcat and the Case of the Poopy Pawprints, a Book in Two Volumes

Volume I opens in 221B Dodger Street, with Sherlockcat and Dr Watson preparing for dinner, when Ms Lawson, his landlady and sometime housekeeper, announces that Inspector Lestrade has asked for his assistance concering the appearance of poopy pawprints around London.

As Sherlockcat and Watson investigate, the evidence begins to point the blame at Sherlockcat himeself. However, Sherlockcat discovers that the actual perpetrator is none other than Professor Maowriarty!

With no solid evidence of Maowriarty’s guilt and the police closing in on him, Sherlockcat is forced to flee until he can find the evidence to clear his name.

Volume II opens with Dr Watson, who has been attempting to track down Sherlockcat ever since he absconded, discovering Sherlockcat in a catnip den in Shanghai. Fearing the worst for his friend, he attempts to intervene, but Sherlockcat convinces him that he is, in fact, only in the den as part of his investigation of Maowriarty, the actual cat in the mirror in the first book.

Sherlockcat discovers that the events of the first book were simply a rouse to make sure that Sherlockcat, as the only intellect Maowriarty considered his equal, was discredited and thus could not intervene in his true master plan: the cornering of the world yarn trade.

After barely escaping Shanghai with their lives, Sherlockcat and Watson track Maowriarty to Austria, where Sherlockcat confronts Maowriarty in a castle in the Austrian Alps. In the ensuing fight, both Sherlockcat and Maowriarty tumble over a balcony in to a raging waterfall, falling to their doom.

Or did they? Has Maowriarty been defeated? Will Sherlockcat return? Find out in the next book: “The Pups of the Baskervilles”.

∞

BTW, this whole circus reminded me of Allen Short’s excellent lightning talk from PyCon 2010: Big Brother’s Design Rules (skip to 17:30). To summarize Allen’s pithy maxims:

War is Peace: assume you are at war, all input is an attack, and then you can be at peace.

Slavery is Freedom: the more you constrain your code’s behavior, the more freedom you have to act. The smaller your interface, the smaller your attack surface.

Ignorance is Strength: the less your code knows about, the fewer things it can break. This is the principle of least authority.

∞