Handling concurrent update-apt-xapian-index runs

So, update-apt-xapian-index takes a bit of a long time to run. When you install the package, if the index isn't there then postinst will rebuild it in background, good.

Now, suppose you have an advanced package manager that after installing apt-xapian-index, detects that it's installed, detects that there is no database and runs an update with a nice GUI feedback. If this happened while the update is running in the background, the usual thing to do would be to abort the update; however, the package manager wouldn't be able to know when the indexing has finished and it can enable the advanced features.

Let's not do the usual thing, then:

# `progress' is the object that does progress reporting.  Commandline options
# can choose a silent one, a machine readable one or a user readable one.

# Lock the session so that we prevent concurrent updates
lockfd = os.open(XAPIANDBLOCK, os.O_RDWR | os.O_CREAT)
lockpyfd = os.fdopen(lockfd)
try:
    fcntl.lockf(lockpyfd, fcntl.LOCK_EX | fcntl.LOCK_NB)

    # `ServerProgress` is a proxy progress reporter that listens for
    # connections on a Unix socket, and broadcasts progress status updates both
    # to the local progress reporter and to whoever connects to the socket.
    progress = ServerProgress(progress)
except IOError, e:
    if e.errno == errno.EACCES or e.errno == errno.EAGAIN:
        progress.notice("Another update is already running: showing its progress.")
        # `ClientProgress' is another proxy class that connects to the socket
    # forwards progress updates to whatever the user has chosen as the
    # local progress indicator.
        childProgress = ClientProgress(progress)
        childProgress.loop()
        sys.exit(0)
    else:
        raise

What happens here is that if the upload is in progress, the second update-apt-xapian-index will not be able to lock a lockfile. But instead of exiting, it connects to the other indexer via a unix socket, and shows the progress just as if it were the one doing the indexing.

So, now, suppose I run update-apt-xapian-index:

# update-apt-xapian-index
Reading Debtags database: done.
Rebuilding Xapian index... 20%

And suppose I run another one:

# update-apt-xapian-index
Another update is already running: showing its progress.
Rebuilding Xapian index... 20%

And suppose I run yet another one:

# update-apt-xapian-index --batch-mode
notice: Another update is already running: showing its progress.
begin: Rebuilding Xapian index
progress: 19/100
progress: 20/100

And suppose I run yet another one (running with -q shows nothing, but it will wait until the master indexer has finished):

# update-apt-xapian-index -q

This way, the advanced package manager (adept 3 does this) will show a progress indicator and correctly let the user know when indexing has finished... only, if an update is already running, the progress indicator will not start at 0% because the update was already in progress, and the user will think: "wow, I've been lucky!".

The server side of the socket only uses nonblocking I/O, so there's no risk of slave indexers causing the master indexer to hang.

It's been simple to implement, and brillant. It is in apt-xapian-index 0.15, which I have just uploaded to sid. Shame it's a bit late for lenny.