Tag eng

Latest posts for tag eng

2017-09-26 00:00:00+02:00

Systemd service units

These are the notes of a training course on systemd I gave as part of my work with Truelite.

.mount and .swap units

Describe mount points similarly as what /etc/fstab, but with more functionality and integration with the dependency system.

It is possible to define, for example, a filesystem that should be mounted only when the network is available and a given service has successfully started, and a service that should be started only after a given filesystem has been successfully mounted.

At boot, systemd uses systemd-fstab-generator to generate mount and swap units from /etc/fstab, so that the usual fstab configuration file can still be used to configure the file system layout.

See man systemd.mount, and man systemd.swap.

See systemctl --all -t mount and systemctl --all -t swap for examples.

debian eng pdo sw systemd-truelite
2017-09-25 00:00:00+02:00

Systemd service units

These are the notes of a training course on systemd I gave as part of my work with Truelite.

.service units

Describe how to start and stop system services, like daemons.

Services are described in a [Service] section. The Type= configuration describes how the service wants to be brought up:

There are a lot more configuration options to fine-tune how the program should be managed, to limit its resource access or capabilities to harden the system security, to run setup/cleanup scripts before or after it started, and after it gets stopped, to control what signals to send to ask for reload or quit, and quite a lot more.

See: man systemd.service, man systemd.exec, man systemd.resource-control, and man systemd.kill.

See systemctl --all -t service for examples.

debian eng pdo sw systemd-truelite
2017-09-24 00:00:00+02:00

Systemd unit files

These are the notes of a training course on systemd I gave as part of my work with Truelite.

Writing .unit files

For reference, the global index with all .unit file directives is at man systemd.directives.

All unit files have a [Unit] section with documentation and dependencies. See man systemd.unit for documentation.

It is worth having a look at existing units to see what they are like. Use systemctl --all -t unittype for a list, and systemctl cat unitname to see its content wherever it is installed.

For example: systemctl cat graphical.target. Note that systemctl cat adds a line of comment at the top so one can see where the unit file is installed.

Most unit files also have an [Install] section (also documented in man systemd.unit) that controls what happens when enabling or disabling the unit.

See also:

.target units

.target units only contain [Unit] and [Install] sections, and can be used to give a name to a given set of dependencies.

For example, one could create a remote-maintenance.target unit, that when brought up activates, via dependencies, a set of services, mounts, network sockets, and so on.

See man systemd.target

See systemctl --all -t target for examples.

special units

man systemd.special has a list of units names that have a standard use associated to them.

For example, ctrl-alt-del.target is a unit that is started whenever Control+Alt+Del is pressed on the console. By default it is symlinked to reboot.target, and you can provide your own version in /etc/systemd/system/ to perform another action when Control+Alt+Del is pressed.

User units

systemd can also be used to manage services on a user session, starting them at login and stopping them at logout.

Add --user to the normal systemd commands to have them work with the current user's session instead of the general system.

See systemd/User in the Arch Wiki for a good description of what it can do.

debian eng pdo sw systemd-truelite
2017-09-23 00:00:00+02:00

Systemd on the command line

These are the notes of a training course on systemd I gave as part of my work with Truelite.

Exploring the state of a system

Start and stop services

Similar to the System V service command, systemctl provides commands to start/stop/restart/reload units or services:

Changing global system state

systemctl has halt, poweroff, reboot, suspend, hibernate, and hybrid-sleep commands to tell systemd to reboot, power off, suspend and so on. kexec and switch-root also work.

The rescue and emergency commands switch the system to rescue and emergency mode (see man systemd.special. systemctl default switches to the default mode, which also happens when exiting the rescue or emergency shell.

Run services at boot

systemd does not implement runlevels, and services start at boot based on their dependencies.

To start a service at boot, you add to its .service file a WantedBy= dependency on a well-known .target unit.

At boot, systemd brings up the whole chain of dependency started from a default unit, and that will eventually activate also your service.

See systemctl get-default for what unit is currently the default in your system. You can change it via the systemd.unit= kernel command line, so you can configure multiple entries in the boot loader that boot the system running different services. For example systemd.unit=rescue.target for a rescue mode, systemd.unit=multi-user.target for a non-graphical mode, or add your own .target file to implement new system modes.

See systemctl list-units -t target --all for a list of all currently available targets in your system.

Notes: systemctl start activates a unit right now, but does not automatically enable it at boot systemctl enable enables a unit at boot, but does not automatically start it right now * a disabled unit can still be activated if another unit depends on it

To disable a unit so that it will never get started even if another unit depends on it, use systemctl mask unitname. Use systemctl unmask unitname to undo the masking.

Reloading / restarting systemd

systemctl daemon-reload tells systemd to reload its configuration.

systemctl daemon-reexec tells systemd to restart iself.

debian eng pdo sw systemd-truelite
2017-09-22 00:00:00+02:00

Systemd Truelite course

These are the notes of a training course on systemd I gave as part of my work with Truelite.

There is quite a lot of material, so I split them into a series of posts, running once a day for the next 9 days.

Units

Everything managed by systemd is called a unit (see man systemd.unit), and each unit is described by a configuration in ini-style format.

For example, this unit continuously plays an alarm sound when the system is in emergency or rescue mode:

[Unit]
Description=Beeps when in emergency or rescue mode
DefaultDependencies=false
StopWhenUnneeded=true

[Install]
WantedBy=emergency.target rescue.target

[Service]
Type=simple
ExecStart=/bin/sh -ec 'while true; do /usr/bin/aplay -q /tmp/beep.wav; sleep 2; done'

Units can be described by configuration files, which have different extensions based on what kind of thing they describe:

System unit files can be installed in:

Unit files in /etc/ override unit files in /lib/. Note that while Debian uses /lib/, other distributions may use /usr/lib/ instead.

If there is a directory with the same name as the unit file plus a .d suffix, any file *.conf it contains is parsed after the unit, and can be used to add or override configuration options.

For example:

Similarly, a unitname.wants/ or unitname.requires/ directory can be used to extend Wants= and Requires= dependencies on other units, by placing symlinks to other units in them.

See also:

debian eng pdo sw systemd-truelite
2017-08-13 16:26:40+02:00

Consensually doing things together?

On 2017-08-06 I have a talk at DebConf17 in Montreal titled "Consensually doing things together?" (video). Here are the talk notes.

Abstract

At DebConf Heidelberg I talked about how Free Software has a lot to do about consensually doing things together. Is that always true, at least in Debian?

I’d like to explore what motivates one to start a project and what motivates one to keep maintaining it. What are the energy levels required to manage bits of Debian as the project keeps growing. How easy it is to say no. Whether we have roles in Debian that require irreplaceable heroes to keep them going. What could be done to make life easier for heroes, easy enough that mere mortals can help, or take their place.

Unhappy is the community that needs heroes, and unhappy is the community that needs martyrs.

I’d like to try and make sure that now, or in the very near future, Debian is not such an unhappy community.

Consensually doing things together

I gave a talk in Heidelberg.

Valhalla made stickers

Debian France distributed many of them.

There's one on my laptop.

Which reminds me of what we ought to be doing.

Of what we have a chance to do, if we play our cards right.

I'm going to talk about relationships. Consensual relationships.

Relationships in short.

Nonconsensual relationships are usually called abuse.

I like to see Debian as a relationship between multiple people.

And I'd like it to be a consensual one.

I'd like it not to be abuse.

Consent

From wikpedia:

In Canada "consent means…the voluntary agreement of the complainant to engage in sexual activity" without abuse or exploitation of "trust, power or authority", coercion or threats.[7] Consent can also be revoked at any moment.[8]»

There are 3 pillars often included in the description of sexual consent, or "the way we let others know what we're up for, be it a good-night kiss or the moments leading up to sex."

They are:

  • Knowing exactly what and how much I'm agreeing to
  • Expressing my intent to participate
  • Deciding freely and voluntarily to participate[20]

Saying "I've decided I won't do laundry anymore" when the other partner is tired, or busy doing things.

Is different than saying "I've decided I won't do laundry anymore" when the other partner has a chance to say "why? tell me more" and take part in negotiation.

Resources:

Relationships

Debian is the Universal Operating System.

Debian is made and maintained by people.

The long term health of debian is a consequence of the long term health of the relationship between Debian contributors.

Debian doesn't need to be technically perfect, it needs to be socially healthy.

Technical problems can be fixed by a healty community.

graph showing relationship between avoidance, accomodation, compromise, competition, collaboration

The Thomas-Kilmann Conflict Mode Instrument: source png.

Motivations

Quick poll:

What are your motivations to be in a relationship?

Which of those motivations are healthy/unhealthy?

"Galadriel" (noun, by Francesca Ciceri): a task you have to do otherwise Sauron takes over Middle Earth

See: http://blog.zouish.org/nonupdd/#/22/1

What motivates me to start a project or pick one up?

What motivates me to keep maintaning a project?

What motivates you?

What's an example of a sustainable motivation?

Is it really all consensual in Debian?

Energy

Energy that thing which is measured in spoons. The metaphore comes from people suffering with chronic health issues:

"Spoons" are a visual representation used as a unit of measure used to quantify how much energy a person has throughout a given day. Each activity requires a given number of spoons, which will only be replaced as the person "recharges" through rest. A person who runs out of spoons has no choice but to rest until their spoons are replenished.

For example, in Debian, I could spend:

What is one person capable of doing?

Have reasonable expectations, on others:

Have reasonable expectations, on yourself:

Debian is a shared responsibility

When spoons are limited, what takes more energy tends not to get done

As the project grows, project-wide tasks become harder

Are they still humanly achievable?

I don't want Debian to have positions that require hero-types to fill them

Dictatorship of who has more spoons:

Perfectionism

You are in a relationship that is just perfect. All your friends look up to you. You give people relationship advice. You are safe in knowing that You Are Doing It Right.

Then one day you have an argument in public.

You don't just have to deal with the argument, but also with your reputation and self-perception shattering.

One things I hate about Debian: consistent technical excellence.

I don't want to be required to always be right.

One of my favourite moments in the history of Debian is the openssl bug

Debian doesn't need to be technically perfect, it needs to be socially healthy, technical problems can be fixed.

I want to remove perfectionism from Debian: if we discover we've been wrong all the time in something important, it's not the end of Debian, it's the beginning of an improved Debian.

Too good to be true

There comes a point in most people's dating experience where one learns that when some things feel too good to be true, they might indeed be.

There are people who cannot say no:

There are people who cannot take a no:

Note the diversity statement: it's not a problem to have one of those (and many other) tendencies, as long as one manages to keep interacting constructively with the rest of the community

Also, it is important to be aware of these patterns, to be able to compensate for one's own tendencies. What happens when an avoidant person meets a narcissistic person, and they are both unaware of the risks?

Resources:

Note: there are problems with the way these resources are framed:

Red flag / green flag

http://pervocracy.blogspot.ca/2012/07/green-flags.html

Ask for examples of red/green flags in Debian.

Green flags:

Red flags:

Apologies / Dealing with issues

I don't see the usefulness of apologies that are about accepting blame, or making a person stop complaining.

I see apologies as opportunities to understand the problem I caused, help fix it, and possibly find ways of avoiding causing that problem again in the future.

A Better Way to Say Sorry lists a 4 step process, which is basically what we do when in bug reports already:

1, Try to understand and reproduce the exact problem the person had. 2. Try to find the cause of the issue. 3. Try to find a solution for the issue. 4. Verify with the reporter that the solution does indeed fix the issue.

This is just to say

My software ate
the files
that where in
your home directory

and which
you were probably
needing
for work

Forgive me
it was so quick to write
without tests
and it worked so well for me

(inspired by a 1934 poem by William Carlos Williams)

Don't be afraid to fail

Don't be afraid to fail or drop the ball.

I think that anything that has a label attached of "if you don't do it, nobody will", shouldn't fall on anybody's shoulders and should be shared no matter what.

Shared or dropped.

Share the responsibility for a healthy relationship

Don't expect that the more experienced mates will take care of everything.

In a project with active people counted by the thousand, it's unlikely that harassment isn't happening. Is anyone writing anti-harassment? Do we have stats? Is having an email address and a CoC giving us a false sense of security?

When you get involved in a new community, such as Debian, find out early where, if that happens, you can find support, understanding, and help to make it stop.

If you cannot find any, or if the only thing you can find is people who say "it never happens here", consider whether you really want to be in that community.

(from http://www.enricozini.org/blog/2016/debian/you-ll-thank-me-later/)

There are some nice people in the world. I mean nice people, the sort I couldn’t describe myself as. People who are friends with everyone, who are somehow never involved in any argument, who seem content to spend their time drawing pictures of bumblebees on flowers that make everyone happy.

Those people are great to have around. You want to hold onto them as much as you can.

But people only have so much tolerance for jerkiness, and really nice people often have less tolerance than the rest of us.

The trouble with not ejecting a jerk — whether their shenanigans are deliberate or incidental — is that you allow the average jerkiness of the community to rise slightly. The higher it goes, the more likely it is that those really nice people will come around less often, or stop coming around at all. That, in turn, makes the average jerkiness rise even more, which teaches the original jerk that their behavior is acceptable and makes your community more appealing to other jerks. Meanwhile, more people at the nice end of the scale are drifting away.

(from https://eev.ee/blog/2016/07/22/on-a-technicality/)

Give people freedom

If someone tries something in Debian, try to acknowledge and accept their work.

You can give feedback on what they are doing, and try not to stand in their way, unless what they are doing is actually hurting you. In that case, try to collaborate, so that you all can get what you need.

It's ok if you don't like everything that they are doing.

I personally don't care if people tell me I'm good when I do something, I perceive it a bit like "good boy" or "good dog". I rather prefer if people show an interest, say "that looks useful" or "how does it work?" or "what do you need to deploy this?"

Acknowledge that I've done something. I don't care if it's especially liked, give me the freedom to keep doing it.

Don't give me rewards, give me space and dignity.

Rather than feeding my ego, feed by freedom, and feed my possibility to create.

debian eng life pdo
2017-06-15 09:37:59+02:00

5 years of Debian Diversity Statement

The Debian Project welcomes and encourages participation by everyone.

No matter how you identify yourself or how others perceive you: we welcome you. We welcome contributions from everyone as long as they interact constructively with our community.

While much of the work for our project is technical in nature, we value and encourage contributions from those with expertise in other areas, and welcome them into our community.

The Debian Diversity Statement has recently turned 5 years old, and I still find it the best diversity statement I know of, one of the most welcoming texts I've seen, and the result of one of the best project-wide mailing list discussions I can remember.

debian eng pdo
2017-05-31 21:00:00+02:00

Today I Learnt

Build a system that can install GRUB2 on UEFI and on legacy systems

grub-efi-amd64 and grub-pc are not coinstallable. It turns out however that they do not contain GRUB, but the machinery to keep GRUB configuration up to date on the current system. If I want to be able to install GRUB on other systems, I can use the -bin packages:

apt install grub-common grub2-common grub-efi-amd64-bin grub-pc-bin

That gave me a grub-install command that worked on both kinds of systems.

GRUB configuration on a UEFI system

An old GRUB configuration on a UEFI system gave me this:

error: no suitable mode found
Booting blind

which boots on a blank screen until the kernel reinitialises the video hardware.

The Arch Linux Wiki has excellent documentation for this case, and here's the resulting UEFI GRUB snippet:

insmod efi_gop
insmod efi_uga
insmod font
if loadfont ${prefix}/fonts/unicode.pf2
then
    insmod gfxterm
    set gfxmode=auto
    set gfxpayload=keep
    terminal_output gfxterm
fi

# Follow with the usual GRUB menu entries…

Use an unsigned local APT repository for testing/development purposes

I found out today that one can have options in square brackets in sources.list:

# In /etc/apt/sources.list.d/local-devel.list
deb [trusted=yes] http://localhost:1234/debian jessie main

pabs on IRC also mentioned local-apt-repository but I haven't tried it.

Booting Jessie Debian Live with a kernel from jessie-backports

This requires working around #844749 and 844749.

In hooks/9000-fix-bugs.chroot I ended up having this:

# Workaround per https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844749
if ! grep -q ^nls_ascii /etc/initramfs-tools/modules
then
        echo "nls_ascii" >> /etc/initramfs-tools/modules
fi

# Workaround per https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844749
if ! grep -q ^overlay /etc/initramfs-tools/modules
then
        echo "overlay" >> /etc/initramfs-tools/modules
fi

Using a custom kernel in Jessie Debian Live

How do I have live-build pick a custom kernel package instead of the default one?

  1. lb config --linux-packages linux-image-$SOMETHING
  2. Use equivs to build a linux-image-$SOMETHING-$ARCH package that depends on the kernel that you built.
debian eng pdo
2017-05-31 20:29:08+02:00

Debian Jessie Live on UEFI part 2

A refinement on my previous attempt.

This is how to configure a Jessie live-build environment to boot on UEFI systems, and get a USB key image that works:

# Build a FAT image instead of an ISO image...
lb config -b hdd

# ...and work around #773833
echo "/usr/lib/syslinux/mbr/*.bin /usr/lib/syslinux/" > hooks/9000-fix-773833.chroot

# Get EFI Shell from https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/
curl -o binary/efi/boot/Bootx64.efi https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/Shell.efi

# Configure the EFI shell to boot the live setup
echo 'live\vmlinuz initrd=live\initrd.img append boot=live components' > binary/startup.nsh

Rationale: UEFI understants FAT filesystems, and would run EFI binaries placed under efi/boot.

For a hard drive, it only considers a FAT filesystem on a GPT partition marked with a special UUID, so that it doesn't get confused with other FAT filesystems that are on disk.

For a USB key, it seems that most hardware will happily look for efi/boot even if the partition table is the old MBR kind.

live-build can build a FAT image for USB keys, losing the ability to boot on CDROMs and DVDs. Since I don't need that ability, I can use -b hdd to get the live system packaged inside a container that UEFI hardware can understand (FAT).

At that point, enabling UEFI boot on a Live Debian Jessie is just a matter of adding an efi/boot/Bootx64.efi binary that is able to load the kernel and initrd in memory, and blow life into them.

debian eng pdo sw
2017-05-29 20:36:40+02:00

Egg-walking with qemu-nbd and kpartx

I wanted to retrieve a file from a VirtualBox VDI image for this blog post.

I followed these instructions and ended up here:

Once having used nbd0, only rebooting the system makes it possible to mount another image ... a little bit unpractical.

What happened was this:

# modprobe nbd  # NOO! Don't *EVER* do that!
# qemu-nbd -c /dev/nbd0 file.vdi
# kpartx -d /dev/nbd0
# mount /dev/nbd0… EHI! Where's /dev/nbdpp1 ??
# qemu-nbd -d /dev/nbd0
# rmmod nbd
rmmod: ERROR: Module nbd is in use
# kpartx -d /dev/nbd0
read error, sector 0
llseek error
llseek error
llseek error
# rmmod nbd
rmmod: ERROR: Module nbd is in use
# WHAT THE…

It turns out it's really modprobe nbd max_part=16, otherwise max_part defaults to, uhm, zero? really? and kpartx cannot create device mappings because there are not enough (as in, not even a single one) partition devices available.

At this point, however, kpartx did create some mappings connected to, uhm, probably Ancient Beings from beyond spacetime, and because of those the device is in use and cannot be removed, and unmapping doesn't work either because the Ancient Beings from beyond spacetime are keeping the device busy by feeding on it.

I energized the pentacle and tried a desperate ritual of banishment:

# # Reconnect nbd0 to the vdi file to Restore the Balance
# qemu-nbd --verbose -c /dev/nbd0 file.vdi
# # This works now
# kpartx -vd /dev/nbd0
del devmap : nbd0p5
del devmap : nbd0p2
del devmap : nbd0p1
# # This too, the Ancient Beings lie asleep yet again
# modprobe nbd -r

At this point I managed to get my file, almost:

# modprobe nbd max_part=16
# qemu-nbd --verbose -c /dev/nbd0 file.vdi
NBD device /dev/nbd0 is now connected to file.vdi
# kpartx -va /dev/nbd0
add map nbd0p1 (254:12): 0 60260352 linear 43:0 2048
add map nbd0p2 (254:13): 0 2 linear 43:0 60264446
add map nbd0p5 (254:14): 0 2648064 linear 43:0 60264448
# mount /dev/nbd0p1 /mnt
mount: /dev/nbd0p1 is already mounted or /mnt busy
# # WHAT NOW?!
# lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
…
nbd0                                        43:0    0    30G  0 disk
├─nbd0p1                                    43:1    0  28.8G  0 part
├─nbd0p2                                    43:2    0     1K  0 part
├─nbd0p5                                    43:5    0   1.3G  0 part
├─nbd0p1                                   254:12   0  28.8G  0 part
├─nbd0p2                                   254:13   0     1K  0 part
└─nbd0p5                                   254:14   0   1.3G  0 part
# # WHAAAT?!!
# kpartx -vd /dev/nbd0
del devmap : nbd0p5
del devmap : nbd0p2
del devmap : nbd0p1
# lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
…
nbd0                                        43:0    0    30G  0 disk
├─nbd0p1                                    43:1    0  28.8G  0 part
├─nbd0p2                                    43:2    0     1K  0 part
└─nbd0p5                                    43:5    0   1.3G  0 part
# mount /dev/nbd0p1 /mnt
# # I got my file, my preciouss file!
# umount /mnt
# kpartx -vd /dev/nbd0
# qemu-nbd -d /dev/nbd0
# rmmod nbd
# # sit in a corner hugging my precious file and sobbing quietly

As can be seen from the multiple exclamation marks, those Ancient Beings from beyond spacetime did manage to have a bite on my sanity after all.

debian eng pdo sw