Tag pdo

Latest posts for tag pdo

2018-08-14 15:08:02+02:00

DebConf 18

This is a quick recap of what happened during my DebConf 18.

24 July:

25 July:

26 July:

27 July:

28 July:

29 July:

30 July:

31 July:

01 August:

02 August:

03 August:

04 August:

debian eng pdo
2018-08-03 11:26:51+02:00

Multiple people

These are the notes from my DebConf18 talk

Slides are available in pdf and odp.


Starting from Debian, I have been for a long time part of various groups where diversity is accepted and valued, and it has been an invaluable supply of inspiration, allowing my identity to grow with unexpected freedom.

During the last year, I have been thinking passionately about things such as diversity, gender identity, sexual orientation, neurodiversity, and preserving identity in a group.

I would like to share some of those thoughts, and some of that passion.

Multiple people

"Debian is a relationship between multiple people", it (I) says at the entrance.

I grew up in a small town, being different was not allowed. Difference attracted "education", well-meaning concern, bullying.

Everyone knew everyone, there was no escape to a more fitting bubble, there was very little choice of bubbles.

I had to learn from a very young age the skill of making myself accepted by my group of peers.

It was an essential survival strategy.

Not being a part of the one group meant becoming a dangerous way of defining the identity of the group: "we are not like him". And one would face the consequences.

"Debian is a relationship between multiple people", it (I) says at the entrance.

Debian was one of the first opportunities for me to directly experience that.

Where I could begin to exist

Where I could experience the value and pleasure of diversity.

Including mine.

I am extremely grateful for that.

I love that.

This talk is also a big thank you to you all for that.

"Debian is a relationship between multiple people", it (I) says at the entrance.

Multiple people does not mean being all the same kind of person, all doing all the same kind of things.

Each of us is a different individual, each of us brings their unique creativity to Debian.

Classifying people

How would you describe a person?

There are binary definitions:

Labels: (like package sections)

Spectra: (like debtags)

We classify packages better than we classify people.

Identity / spectrums

I'm going to show a few examples of spectra; I chose them not because they are more or less important than others, but because they have recently been particularly relevant to me, and it's easier for me to talk about them.

If you wonder where you are in each spectrum, know that every place is ok.

Think about who you are, not about who you should be.

Gender identity

My non binary awareness began with d-w and gender neutral documentation.

Sexual orientation


table of sexual preference prefixes combinations


I'll introduce neurodiversity by introducing allism

An allistic person learns subconsciously that ey is dependent on others for eir emotional experience. Consequently, ey tends to develop the habit of manipulating the form and content of social interactions in order to elicit from others expressions of emotion that ey will find pleasing when incorporated into eir mind.


The more I reason about this (and I reasoned about this a lot, before, during and after therapy), the more I consider it a very rational adaptation, derived from a very clear message I received since I was a small child: nobody cared whom I was, and to be accepted socially I needed to act a social part, which changed from group to group. Therefore, socially, I was wrong, and I had to work to deserve the right to exist.

What one usually sees of me in large groups or when out of comfort zone, is a social mask of me.

This paper is also interesting: analyzing tweets of people and their social circle, they got to the point of being able to predict what a person will write by running statistics only on what their friends are writing.

Is it measuring identity or social conformance?

Discussion about the autism spectrum tends to get very medical very fast, and the DSM gets often criticised for a tendency of turning diversity into mental health issues.

I stick to my experience from a different end of the spectrum, and there are various resources online to explore if you are interested in more.

Other spectra

I hope you get the idea about spectrum and identity.

There are many more, those were at the top of my head because of my recent experiences.

Skin color, age, wealth, disability, language proficiency, ...

How to deal with diversity

How to deal with my diversity

Let's all assume for a moment that each and every single one of us is ok.

I am ok.

You are ok.

You've been ok since the moment you were born.

Being born is all you need to deserve to exist.

You are ok, and you will always be ok.

Like every single person alive.

I'm ok.

You're ok.

We're all ok.

Hold on to that thought for the next 5 minutes. Hold onto it for the rest of your life.

Ok. A lot of problems are now solved. A lot of energy is not wasted anymore. What next?

Get to know myself


Get in touch with my feelings, get to know my needs.

Here's a simple algorithm to get to know your feelings and needs:

  1. If you are happy, take this phrase: I feel … because my need of … is being met
  2. If you are not happy, take this phrase: I feel … because my need of … is not being met
  3. Fill the first space with one of the words from here
  4. Fill the second space with one of the words from here
  5. Done!

To know more about Non-Violent Communication, I suggest this video

This other video I also liked.

Forget absolute truths, center on my own experience. Have a look here for more details.

Learn to communicate and express myself

Communicating/being oneself

Find out where to be myself

Look for safe spaces where I can activate parts of myself

Learn to protect myself

I will make mistakes acting as myself:

Learn to know my boundaries

Learn to recognise when they are being crossed


Use my anger to protect my integrity. I do not need to hurt others to protect myself

How to deal with the diversity of others

Diversity is a good thing

Once I realised I can be ok in my diversity, it was easier to appreciate the diversity of others

Opening to others doesn't need to sacrifice oneself.

Curiosity is a good default.

Do not assume. Assume better and I'll be disappointed. Assume worse and I'll miss good interactions

When facing the new and unsettling, use curiosity if I have the energy, or be aware that I don't, and take a step back

The goal of the game is to affirm all identities, especially oneself.

Love freely.

Expect nothing.

Liberate myself from imagined expectations.


What is not acceptable


The paradox of tolerance, as a comic strip

Less well known is the paradox of tolerance: Unlimited tolerance must lead to the disappearance of tolerance. If we extend unlimited tolerance even to those who are intolerant, if we are not prepared to defend a tolerant society against the onslaught of the intolerant, then the tolerant will be destroyed, and tolerance with them. — In this formulation, I do not imply, for instance, that we should always suppress the utterance of intolerant philosophies; as long as we can counter them by rational argument and keep them in check by public opinion, suppression would certainly be unwise. But we should claim the right to suppress them if necessary even by force; for it may easily turn out that they are not prepared to meet us on the level of rational argument, but begin by denouncing all argument; they may forbid their followers to listen to rational argument, because it is deceptive, and teach them to answer arguments by the use of their fists or pistols. We should therefore claim, in the name of tolerance, the right not to tolerate the intolerant.

Use diversity for growth

Identifying where I am gives me more awareness about myself.

Identifying where I am shows me steps I might be interested in making.

Identity can change, evolve, move I like the idea of talking about my identity in the past tense

Diversity as empowerment, spectrum as empowerment

Take control of your narrative: what is your narrative? Do you like it? Does it tell you now what you're going to like next year, or in 5 years? Is it a problem if it does?

Conceptual space is not limited. Allocating mental space for new diversity doesn't reduce one's own mental space, but it expands it

Is someone trying to control your narrative? gaslighting, negging, patronising.

Debian and diversity

Impostor syndrome

Entering a new group: impostor syndrome. Am I good enough for this group?

Expectations, perceived expectations, perceived changes in perceived identity, perceived requirements on identity

I worked some months with a therapist to deal with that, to, it turned out, learn to give up the need to work to belong.

In the end, it was all there in the Diversity Statement:

No matter how I identify myself or how others perceive me: I am welcome, as long as I interact constructively with my community.

Ability of the group to grow, evolve, change, adapt, create

And here, according to Trout, was the reason human beings could not reject ideas because they were bad: “Ideas on Earth were badges of friendship or enmity. Their content did not matter. Friends agreed with friends, in order to express friendliness. Enemies disagreed with enemies, in order to express enmity.

“The ideas Earthlings held didn’t matter for hundreds of thousands of years, since they couldn’t do much about them anyway. Ideas might as well be badges as anything.

(Kurt Vonnegut, "Breakfast of Champions", 1973)

Keep one's identity in Debian

If your identity is your identity, and the group changes, it actually extends, because you keep being who you are.

If your identity is a function of the group identity, you become a control freak for where the group is going.

When people define their identity in terms of belonging to a group, that group cannot change anymore, because if it does, it faces resistance from its members, that will see their own perceived identity under threat.

The threat is that rituals, or practices, that validated my existance, that previously used to work, cease to function. systemd?

Free software

Us, and our users, we are a diverse ecosystem

Free Software is a diverse ecosystem

Free software can be a spectrum (free hardware, free firmware, free software, free javascript in browsers...)


Debian exists, and can move in a diverse and constantly changing upstream ecosystem

Vision / non limiting the future of Debian (if your narrative tells you what you're going to like next year, you might have a problem) (but before next year I'd like to get to a point that I can cope with X)

Debian doesn't need to be what people need to define their own identity, but it is defined by the relationship between different, diverse, evolving people

Appreciate diversity, because there's always something you don't know / don't understand, and more in the future.

Nobody can know all of Debian now, and in the future, if we're successful, we're going to get even bigger and more complex.

We're technically complex and diverse, we're socially complex and diverse. We got to learn to deal with with that.

Because we're awesome. We got to learn to deal with with that.

Ode to the diversity statement


debconf debian eng life pdo talk
2018-07-26 14:36:30+02:00


This work has been brought to you by the wonderful DebCamp.

I needed to reproduce a build issue on an i386 architecture, so I started going through the instructions for finding a porterbox and setting up a chroot.

And then I though, this is long and boring. A program could do that.

So I created a program to do that:

$ debug-on-porterbox  --help
usage: debug-on-porterbox [-h] [--verbose] [--debug] [--cleanup] [--git]
                          [--reuse] [--dist DIST] [--host HOST]
                          arch [package]

set up a build environment to debug a package on a porterbox

positional arguments:
  arch           architecture name
  package        package name

optional arguments:
  -h, --help     show this help message and exit
  --verbose, -v  verbose output
  --debug        debug output
  --cleanup      cleanup a previous build, removing porterbox data and git
  --git          setup a git clone of the current branch
  --reuse        reuse an existing session
  --dist DIST    distribution (default: sid)
  --host HOST    hostname to use (autodetected by default)

On a source directory, you can run debug-on-porterbox i386 and it will:

The only thing left for you to do is to log into the machine debug-on-porterbox tells you, run the command porterbox tells you to enter the chroot, and debug away.

At the end you can clean everything up, including the remote chroot and the git remote in the local repo, with: debug-on-porterbox [--git] --cleanup i386

The code is on Salsa: have fun!

debian eng pdo sw
2018-06-13 13:43:09+02:00

Progress bar for file descriptors

I ran gzip on an 80Gb file, it's processing, but who knows how much it has done yet, and when it will end? I wish gzip had a progressbar. Or MySQL. Or…

Ok. Now every program that reads a file sequentially can have a progressbar:



Print progress indicators for programs that read files sequentially.

fdprogress monitors file descriptor offsets and prints progressbars comparing them to file sizes.

Pattern can be any glob expression.

usage: fdprogress [-h] [--verbose] [--debug] [--pid PID] [pattern]

show progress from file descriptor offsets

positional arguments:
  pattern            file name to monitor

optional arguments:
  -h, --help         show this help message and exit
  --verbose, -v      verbose output
  --debug            debug output
  --pid PID, -p PID  PID of process to monitor


pv has a --watchfd option that does most of what fdprogress is trying to do: use that instead.


fivi also exists, with specific features to show progressbars for filter commands.

debian eng pdo sw
2018-05-15 14:06:06+02:00

Starting user software in X

There are currently many ways of starting software when a user session starts.

This is an attempt to collect a list of pointers to piece the big picture together. It's partial and some parts might be imprecise or incorrect, but it's a start, and I'm happy to keep it updated if I receive corrections.


man xsession

systemd --user

dbus activation

X session manager

xdg autostart

Other startup notes


To connect to an X server, a client needs to send a token from ~/.Xauthority, which proves that they can read the user's provate data.

~/.Xauthority contains a token generated by display manager and communicated to X at startup.

To view its contents, use xauth -i -f ~/.Xauthority list

debian eng pdo sw
2018-04-19 23:04:02+02:00

Detect a UEFI partition

Today I had to implement a check to see if a disk contains a UEFI ESP partition.

Here it is, it also works on disk image files instead of devices:

def get_uefi_partition(self, disk_dev):
    Return the partition device of the UEFI ESP partition for the device in

    Returns None if disk_dev contains no UEFI ESP partition.
    import parted
    pdev = parted.getDevice(disk_dev)
    pdisk = parted.newDisk(pdev)
    if pdisk.type != "gpt":
        log.error("device %s has partition table type %s instead of gpt", disk_dev, pdisk.type)
        return None
    for part in pdisk.partitions:
        if part.getFlag(18):
            log.info("Found ESP partition in %s", part.path)
            return part.path
    log.info("No ESP partition found in %s", disk_dev)
    return None
debian eng pdo sw
2018-04-13 01:39:44+02:00

ansible nspawn connection plugin

I have been playing with system images using ansible and chroots, and I figured that using systemd-nspawn to handle the chroots would make things nice, giving ansible commands the benefit of a running system.

There has been an attempt which was rejected.

Here is my attempt. It does boot the machine then run commands inside it, and it works nicely. The only thing I missed is a way of shutting down the machine at the end, since ansible seems to call close() at the end of each command, and I do not know enough ansible internals to do this right.

I hope this can serve as inspiration for something that works well.

# Based on chroot.py (c) 2013, Maykel Moya <mmoya@speedyrails.com>
# Based on chroot.py (c) 2015, Toshio Kuratomi <tkuratomi@ansible.com>
# (c) 2018, Enrico Zini <enrico@debian.org>
# This is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

import distutils.spawn
import os
import os.path
import pipes
import subprocess
import time
import hashlib

from ansible import constants as C
from ansible.errors import AnsibleError
from ansible.plugins.connection import ConnectionBase, BUFSIZE
from ansible.module_utils.basic import is_executable

    from __main__ import display
except ImportError:
    from ansible.utils.display import Display
    display = Display()

class Connection(ConnectionBase):
    ''' Local chroot based connections '''

    transport = 'schroot'
    has_pipelining = True
    # su currently has an undiagnosed issue with calculating the file
    # checksums (so copy, for instance, doesn't work right)
    # Have to look into that before re-enabling this
    become_methods = frozenset(C.BECOME_METHODS).difference(('su',))

    def __init__(self, play_context, new_stdin, *args, **kwargs):
        super(Connection, self).__init__(play_context, new_stdin, *args, **kwargs)

        self.chroot = self._play_context.remote_addr
        # We need short and fast rather than secure
        m = hashlib.sha1()
        self.machine_name = "ansible-" + m.hexdigest()

        if os.geteuid() != 0:
            raise AnsibleError("nspawn connection requires running as root")

        # we're running as root on the local system so do some
        # trivial checks for ensuring 'host' is actually a chroot'able dir
        if not os.path.isdir(self.chroot):
            raise AnsibleError("%s is not a directory" % self.chroot)

        chrootsh = os.path.join(self.chroot, 'bin/sh')
        # Want to check for a usable bourne shell inside the chroot.
        # is_executable() == True is sufficient.  For symlinks it
        # gets really complicated really fast.  So we punt on finding that
        # out.  As long as it's a symlink we assume that it will work
        if not (is_executable(chrootsh) or (os.path.lexists(chrootsh) and os.path.islink(chrootsh))):
            raise AnsibleError("%s does not look like a chrootable dir (/bin/sh missing)" % self.chroot)

        self.nspawn_cmd = distutils.spawn.find_executable('systemd-nspawn')
        if not self.nspawn_cmd:
            raise AnsibleError("systemd-nspawn command not found in PATH")
        self.machinectl_cmd = distutils.spawn.find_executable('machinectl')
        if not self.machinectl_cmd:
            raise AnsibleError("machinectl command not found in PATH")
        self.run_cmd = distutils.spawn.find_executable('systemd-run')
        if not self.run_cmd:
            raise AnsibleError("systemd-run command not found in PATH")

        existing = subprocess.call([self.machinectl_cmd, "show", self.machine_name], stdout=open("/dev/null", "wb"))
        self.machine_exists = existing == 0

    def set_host_overrides(self, host, hostvars=None):
        super(Connection, self).set_host_overrides(host, hostvars)

    def _connect(self):
        ''' connect to the chroot; nothing to do here '''
        super(Connection, self)._connect()
        if not self._connected:
            if not self.machine_exists:
                display.vvv("Starting nspawn machine", host=self.chroot)
                self.chroot_proc = subprocess.Popen([self.nspawn_cmd, "-D", self.chroot, "-M", self.machine_name, "--register=yes", "--boot"], stdout=open("/dev/null", "w"))
                self.chroot_proc = None
                display.vvv("Reusing nspawn machine", host=self.chroot)
            self._connected = True

    def _local_run_cmd(self, cmd, stdin=None):
        display.vvv(" -exec %s" % repr(cmd), host=self.chroot)
        display.vvv(" -  or %s" % " ".join(pipes.quote(x) for x in cmd), host=self.chroot)
        p = subprocess.Popen(cmd, shell=False, stdin=subprocess.PIPE,
                             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, stderr = p.communicate(stdin)
        display.vvv(" - got %d" % p.returncode, host=self.chroot)
        display.vvv(" - out %s" % repr(stdout), host=self.chroot)
        display.vvv(" - err %s" % repr(stderr), host=self.chroot)
        return p.returncode, stdout, stderr

    def _systemd_run_cmd(self, cmd, stdin=None):
        local_cmd = [self.run_cmd, "-M", self.machine_name, "-q", "--pipe", "--wait", "-E", "HOME=/root", "-E", "USER=root", "-E", "LOGNAME=root"] + cmd
        local_cmd = [x.encode("utf8") if isinstance(x, unicode) else x for x in local_cmd]
        return self._local_run_cmd(local_cmd, stdin=stdin)

    def exec_command(self, cmd, in_data=None, sudoable=False):
        ''' run a command on the chroot '''
        super(Connection, self).exec_command(cmd, in_data=in_data, sudoable=sudoable)

        display.vvv("cmd: %s" % repr(cmd), host=self.chroot)
        return self._systemd_run_cmd(["/bin/sh", "-c", cmd], stdin=in_data)

    def _prefix_login_path(self, remote_path):
        ''' Make sure that we put files into a standard path

            If a path is relative, then we need to choose where to put it.
            ssh chooses $HOME but we aren't guaranteed that a home dir will
            exist in any given chroot.  So for now we're choosing "/" instead.
            This also happens to be the former default.

            Can revisit using $HOME instead if it's a problem
        if not remote_path.startswith(os.path.sep):
            remote_path = os.path.join(os.path.sep, remote_path)
        return os.path.normpath(remote_path)

    def put_file(self, in_path, out_path):
        ''' transfer a file from local to chroot '''
        super(Connection, self).put_file(in_path, out_path)
        display.vvv("PUT %s TO %s" % (in_path, out_path), host=self.chroot)

        out_path = pipes.quote(self._prefix_login_path(out_path))
        p = subprocess.Popen([self.machinectl_cmd, "-q", "copy-to", self.machine_name, in_path, out_path], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, stderr = p.communicate()
        if p.returncode != 0:
            raise AnsibleError("failed to transfer file %s to %s:\n%s\n%s" % (in_path, out_path, stdout, stderr))

    def fetch_file(self, in_path, out_path):
        ''' fetch a file from chroot to local '''
        super(Connection, self).fetch_file(in_path, out_path)
        display.vvv("FETCH %s TO %s" % (in_path, out_path), host=self.chroot)

        in_path = pipes.quote(self._prefix_login_path(in_path))
        p = subprocess.Popen([self.machinectl_cmd, "-q", "copy-from", self.machine_name, in_path, out_path], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, stderr = p.communicate()
        if p.returncode != 0:
            raise AnsibleError("failed to transfer file %s from %s:\n%s\n%s" % (out_path, in_path, stdout, stderr))

    def close(self):
        super(Connection, self).close()

# FIXME: how can we power off the machine? close and __del__ seem to be called after each command
#    def __del__(self):
#        ''' terminate the connection; nothing to do here '''
#        # super(Connection, self).close()
#        display.vvv("CLOSE", host=self.chroot)
#        if self._connected:
#            p, stdout, stderr = self._local_run_cmd([self.machinectl_cmd, "poweroff", self.machine_name])
#            if p == 0 and self.chroot_proc:
#                self.chroot_proc.wait()
#            self._connected = False
ansible debian eng pdo sw systemd
2018-04-04 09:48:03+02:00

Command line arguments are code

A story of yak shaving

I wanted to see the full pathname of the currently edited file in neovim, I found out it's ctrl+G.

But wouldn't it be nice to always see it?

sudo apt install vim-airline vim-airline-themes fonts-powerline
vim-addon-manager install vim-airline vim-airline-themes
echo "let g:airline_powerline_fonts = 1" >> ~/.vimrc

I recall one could also see the current git branch?

sudo apt install vim-fugitive
vim-addon-manager install vim-fugitive

Ooh, I remember I used to have pycodestyle highlighting in my code?

sudo apt install vim-syntastic
vim-addon-manager install vim-syntastic

A story of horror

Great! Uhm, wait, I recall syntastic had some pretty stupid checkers one needed to be careful of. Ah yes, it looks like I can whitelist them:

let g:syntastic_mode_map = { 'mode': 'active',
                           \ 'active_filetypes': ['python', 'cpp'],
                           \ 'passive_filetypes': [] }
let g:syntastic_python_checkers = ['flake8', 'python']
let g:syntastic_python_python_exec = '/usr/bin/python3'
let g:syntastic_cpp_checkers = ['gcc', 'clang-tidy']

Note, when I say "stupid", I mean something focusing way more on what can be done, rather than on what should be done.

I appreciate that a syntastic checker, that sends all of your current file to a remote website over http every time you open it or save it, can be written. I believe it should not be written, or at least not distributed.

Ok, now, how do I pass extra include dirs to clang-tidy? Ah, there is a config file system.

How does it work exactly?


Lesson learned

Command line options should be treated as code, not as data.

Any configuration system that allows to inject arbitrary command line options to commands that get executed, should be treated as a configuration system that can inject arbitrary code into your own application. It can be a powerful tool, but it needs to be carefully secured against all sorts of exploits.

debian eng pdo sw
2018-02-25 11:24:22+01:00

Automatic deploy from gitlab/salsa CI

At SnowCamp I migrated Front Desk-related repositories to Salsa gitlab and worked on setting up Continuous Integration for the web applications I maintain in Debian.

The result is a reusable Django app that integrates with gitlab's webhooks

It is currently working for https://contributors.debian.org and I'll soon reuse it for https://nm.debian.org and https://debtags.debian.org.

The only setup needed on DSA side is to enable systemd linger on the deploy user.

The CI/deploy workflow is this:

And manage.py deploy does this:

For more details, see the app's README.md

I find it wonderful that we got to a stage where we can have this in Debian, and I am very grateful to all the work that has been done and is being done in setting up and maintaining Salsa.

debian eng pdo salsa sw
2018-02-08 10:26:06+01:00

Gnome without chrome-gnome-shell

New laptop, has a touchscreen, can be folded into a tablet, I heard gnome-shell would be a good choice of desktop environment, and I managed to tweak it enough that I can reuse existing habits.

I have a big problem, however, with how it encourages one to download random extensions off the internet and run them as part of the whole desktop environment. I have an even bigger problem with gnome-core having a hard dependency on chrome-gnome-shell, a plugin which cannot be disabled without root editing files in /etc, which exposes parts of my destktop environment to websites.

Visit this site and it will know which extensions you have installed, and it will be able to install more. I do not trust that, I do not need that, I do not want that. I am horrified by the idea of that.

I made a workaround.

How can one do the same for firefox?


chrome-gnome-shell is a hard dependency of gnome-core, and it installs a browser plugin that one may not want, and mandates its use by system-wide chrome policies.

I consider having chrome-gnome-shell an unneeded increase of the attack surface of my system, in exchange for the dubious privilege of being able to download and execute, as my main user, random unreviewed code.

This package satifies the chrome-gnome-shell dependency, but installs nothing.

Note that after installing this package you need to purge chrome-gnome-shell if it was previously installed, to have it remove its chromium policy files in /etc/chromium


apt install equivs
equivs-build contain-gnome-shell
sudo dpkg -i contain-gnome-shell_1.0_all.deb
sudo dpkg --purge chrome-gnome-shell
debian eng gnome pdo rant sw