Tag sw

Latest posts for tag sw

2017-05-29 20:36:40+02:00

Egg-walking with qemu-nbd and kpartx

I wanted to retrieve a file from a VirtualBox VDI image for this blog post.

I followed these instructions and ended up here:

Once having used nbd0, only rebooting the system makes it possible to mount another image ... a little bit unpractical.

What happened was this:

# modprobe nbd
# qemu-nbd -c /dev/nbd0 file.vdi
# kpartx -d /dev/nbd0
# mount /dev/nbd0… EHI! Where's /dev/nbdpp1 ??
# qemu-nbd -d /dev/nbd0
# rmmod nbd
rmmod: ERROR: Module nbd is in use
# kpartx -d /dev/nbd0
read error, sector 0
llseek error
llseek error
llseek error
# rmmod nbd
rmmod: ERROR: Module nbd is in use
# WHAT THE…

It turns out it's really modprobe nbd max_part=16, otherwise max_part defaults to, uhm, zero? really? and kpartx cannot create device mappings because there are not enough (as in, not even a single one) partition devices available.

At this point, however, kpartx did create some mappings connected to, uhm, probably Ancient Beings from beyond spacetime, and because of those the device is in use and cannot be removed, and unmapping doesn't work either because the Ancient Beings from beyond spacetime are keeping the device busy by feeding on it.

I energized the pentacle and tried a desperate ritual of banishment:

# # Reconnect nbd0 to the vdi file to Restore the Balance
# qemu-nbd --verbose -c /dev/nbd0 file.vdi
# # This works now
# kpartx -vd /dev/nbd0
del devmap : nbd0p5
del devmap : nbd0p2
del devmap : nbd0p1
# # This too, the Ancient Beings lie asleep yet again
# modprobe nbd -r

At this point I managed to get my file, almost:

# modprobe nbd max_part=16
# qemu-nbd --verbose -c /dev/nbd0 file.vdi
NBD device /dev/nbd0 is now connected to file.vdi
# kpartx -va /dev/nbd0
add map nbd0p1 (254:12): 0 60260352 linear 43:0 2048
add map nbd0p2 (254:13): 0 2 linear 43:0 60264446
add map nbd0p5 (254:14): 0 2648064 linear 43:0 60264448
# mount /dev/nbd0p1 /mnt
mount: /dev/nbd0p1 is already mounted or /mnt busy
# # WHAT NOW?!
# lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
…
nbd0                                        43:0    0    30G  0 disk
├─nbd0p1                                    43:1    0  28.8G  0 part
├─nbd0p2                                    43:2    0     1K  0 part
├─nbd0p5                                    43:5    0   1.3G  0 part
├─nbd0p1                                   254:12   0  28.8G  0 part
├─nbd0p2                                   254:13   0     1K  0 part
└─nbd0p5                                   254:14   0   1.3G  0 part
# # WHAAAT?!!
# kpartx -vd /dev/nbd0
del devmap : nbd0p5
del devmap : nbd0p2
del devmap : nbd0p1
# lsblk
NAME                                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
…
nbd0                                        43:0    0    30G  0 disk
├─nbd0p1                                    43:1    0  28.8G  0 part
├─nbd0p2                                    43:2    0     1K  0 part
└─nbd0p5                                    43:5    0   1.3G  0 part
# mount /dev/nbd0p1 /mnt
# # I got my file, my preciouss file!
# umount /mnt
# kpartx -vd /dev/nbd0
# qemu-nbd -d /dev/nbd0
# rmmod nbd
# # sit in a corner hugging my precious file and sobbing quietly

As can be seen from the multiple exclamation marks, those Ancient Beings from beyond spacetime did manage to have a bite on my sanity after all.

debian eng pdo sw
2017-05-29 20:12:46+02:00

Jessie live on UEFI systems

According to the Debian Wiki, you can't boot a Debian Live based on Jessie on a UEFI system:

UEFI support in live images At this point, UEFI support exists only in Debian's installation images. The accompanying live images do not have support for UEFI boot, as the live-build software used to generate them still does not include it. Hopefully the debian-live developers will add this important feature soon.

Some people really needed it, though, so I kept looking.

Here's a script that takes a Jessie Debian Live .iso file and the device name for a USB pendrive, and gives you a pendrive that boots on UEFI:

#!/bin/sh
# License: do what you want but it's not my fault, I told you not to.

sh -ue

ISO=${1:?"Usage: $0 file.iso usbdev"}
DEV=${2:?"Usage: $0 file.iso usbdev"}

parted -s $DEV mklabel gpt mkpart primary fat32 1 100%
mkfs.vfat ${DEV}1
mount ${DEV}1 /mnt

bsdtar -C /mnt -xf $ISO

mkdir -p /mnt/efi/boot
# Shell.efi comes from https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/
cp Shell.efi /mnt/efi/boot/Bootx64.efi
echo 'live\vmlinuz initrd=live\initrd.img append boot=live components' > /mnt/startup.nsh

umount /mnt

Only use it if you really need it, though: Stretch will support this out of the box, and it's coming soon.

debian eng pdo sw
2017-04-22 20:48:43+02:00

Splitting a git-annex repository

I have a git annex repo for all my media that has grown to 57866 files and git operations are getting slow, especially on external spinning hard drives, so I decided to split it into separate repositories.

This is how I did it, with some help from #git-annex. Suppose the old big repo is at ~/oldrepo:

# Create a new repo for photos only
mkdir ~/photos
cd photos
git init
git annex init laptop

# Hardlink all the annexed data from the old repo
cp -rl ~/oldrepo/.git/annex/objects .git/annex/

# Regenerate the git annex metadata
git annex fsck --fast

# Also split the repo on the usb key
cd /media/usbkey
git clone ~/photos
cd photos
git annex init usbkey
cp -rl ../oldrepo/.git/annex/objects .git/annex/
git annex fsck --fast

# Connect the annexes as remotes of each other
git remote add laptop ~/photos
cd ~/photos
git remote add usbkey /media/usbkey

At this point, I went through all repos doing standard cleanup:

# Remove unneeded hard links
git annex unused
git annex dropunused --force 1-12345

# Sync
git annex sync

To make sure nothing is missing, I used git annex find --not --in=here to see if, for example, the usbkey that should have everything could be missing some thing.

Update: Antoine Beaupré pointed me to this tip about Repositories with large number of files which I will try next time one of my repositories grows enough to hit a performance issue.

debian eng gitannex pdo sw
2017-04-09 20:54:06+02:00

Ansible config for my stereo

I bought a Raspberry Pi 2 and its case. I could not reuse the existing SD card because it wants a MicroSD.

A wise person once told me:

First you do it, then you document it, then you automate it.

I had done the first two, and now I've redone the whole setup with ansible, here: stereo.tar.xz.

Hifi with a Raspberry Pi 2 and its case

debian eng hw pdo raspi-hifi sw
2017-04-03 11:15:38+02:00

Free Software on my phone

I try to run my phone on Free Software as much as I can.

I recently switched to LineageOS. I took it as an opportunity to do a full factory wipe and reinstall, to simulate a disaster recovery.

Here's a summary of the basic software I use:

debian eng pdo phone sw
2017-04-02 00:10:00+02:00

Stereo remote control

Wouldn't it be nice if I could use the hifi remote control to control mpd?

It turns out many wishes can come true when one has a GPIO board.

A friend of mine had a pile of IR receiver components in his stash and gave me one. It is labeled "38A 1424A", and the closest matching datasheet I found is this one.

I wired the receiver with the control pin on GPIO port 24, and set up lirc by following roughly this guide.

Hifi with shutdown button and IR receiver

Enable lirc_rpi support

I had to add these lines to /boot/config.txt to enable lirc_rpi support:

dtoverlay=lirc-rpi,gpio_in_pin=24,gpio_out_pin=22
dtparam=gpio_in_pull=up

At first I had missed configuration of the internal pull up resistor, and reception worked but was very, very poor.

Then reboot.

Install and configure lirc

apt install lirc

I added these lines to /etc/lirc/hardware.conf:

DRIVER="default"
DEVICE="/dev/lirc0"
MODULES="lirc_rpi"

Stopped lircd:

systemctl stop lirc

Tested that the receiver worked:

mode2 -d /dev/lirc0

Downloaded remote control codes for my hifi and put them in /etc/lirc/lircd.conf.

Started lircd

systemctl start lirc

Tested that lirc could parse commands from my remote control:

$ irw
0000400405506035 00 CD_PAUSE RAK-SC304W
0000400405506035 01 CD_PAUSE RAK-SC304W
0000400405506035 02 CD_PAUSE RAK-SC304W
0000400405505005 00 CD_PLAY RAK-SC304W
0000400405505005 01 CD_PLAY RAK-SC304W

Interface lirc with mpd

I made this simple lirc program and saved it in ~pi/.lircrc:

begin
     prog = irexec
     button = CD_NEXT
     config = mpc next
end

begin
     prog = irexec
     button = TAPE_FWD
     config = mpc next
end

begin
     prog = irexec
     button = TAPE_REW
     config = mpc prev
end

begin
     prog = irexec
     button = CD_PREV
     config = mpc prev
end

begin
     prog = irexec
     button = TAPE_PAUSE
     config = mpc pause
end

begin
     prog = irexec
     button = CD_PAUSE
     config = mpc pause
end

begin
     prog = irexec
     button = CD_PLAY
     config = mpc toggle
end

begin
     prog = debug
     button = TAPE_PLAY_RIGHT
     config = mpc toggle
end

Then wrote a systemd unit file to start irexec and saved it as /etc/systemd/system/mpd-irexec.service:

[Unit]
Description=Control mpd via lirc remote control
After=lirc mpd

[Service]
Type=simple
ExecStart=/usr/bin/irexec
Restart=always
User=pi
WorkingDirectory=~

[Install]
WantedBy=multi-user.target

Then systemctl start mpd-irexec to start irexec, and systemctl enable mpd-irexec to start irexec at boot.

Profit!

All of this was done by me, with almost no electronics training, following online tutorials for the hardware parts.

To connect components I used a breadboard and female-male jumper leads, so I didn't have to solder, for which I have very little practice.

Now the Raspberry Pi is so much a part of my hifi component that I can even control it with the hifi remote control.

Given that I disconnected the CD and tape players, there are now at least 16 free buttons on the remote control that I can script however I like.

Raspberry Pi closeup

debian eng hw pdo raspi-hifi sw
2017-04-01 00:10:00+02:00

Shutdown button for my Raspberry Pi

My Raspberry Pi hifi component setup lacked a way to cleanly shutdown the system without ssh.

I wished the Raspberry Pi had a button so I can tell it to shutdown.

I added one to the GPIO connector.

It turns out many wishes can come true when one has a GPIO board.

This is the /usr/local/bin/stereo-gpio script that reacts to the button press and triggers a shutdown:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
#!/usr/bin/python3
# http://razzpisampler.oreilly.com/ch07.html
# http://web.archive.org/web/20160305001215/http://razzpisampler.oreilly.com/ch07.html
from RPi import GPIO
import time
import subprocess

def on_button(pin):
    print("Button pressed", pin)

GPIO.setmode(GPIO.BCM)

GPIO.setup(18, GPIO.IN, pull_up_down=GPIO.PUD_UP)

while True:
    pin = GPIO.wait_for_edge(18, GPIO.FALLING)
    if pin == 18:
        subprocess.check_call(["/bin/systemctl", "halt"])

This is the /etc/systemd/system/stereo-gpio.service systemd unit file that runs the script as a daemon:

[Unit]
Description=Stereo GPIO manager

[Service]
Type=simple
ExecStart=/usr/local/bin/stereo-gpio
Restart=always

[Install]
WantedBy=multi-user.target

Then systemctl start stereo-gpio to start the script, and systemctl enable stereo-gpio to start the script at boot.

debian eng hw pdo raspi-hifi sw
2017-03-31 09:06:22+02:00

Raspberry Pi as a Hi-Fi component

I have a 25 years old Technics hifi system that still works fine, and I gave it a new life by replacing the CD player and cassette player modules with a Raspberry Pi.

Technics hifi with a Raspberry Pi attached

Connection

Each component of the hifi has a mains input and a mains plug that is used to power the next component. The element where the main power lead goes in is the radio component, which has a remote control receiver, a watch and a timer, and will power on the rest of the system when turned on by its power button, the remote control, or the alarm function.

I disconnected the cassette and cd player modules, and plugged the Raspberry Pi phone charger/power supply in the free plug behind the amplifier module, at the end of the (now very short) power lead chain.

I also connected the audio output of the Raspberry Pi to the CD input of my stereo. The advantage of CD over AUX is that the remote control buttons for switching audio sources don't cover the AUX inputs.

With alsamixer I adjusted the output volume to match that of the radio component, so that I can switch between the two without surprising jumps in volume. I used alsactl store to save the mixer levels.

Now when I turn the hifi on I also turn the Raspberry Pi on, and when I turn the hifi off, I also cut power from the Raspberry Pi.

Operating system

Operating system install instructions:

  1. I downloaded a Raspbian Jessie Lite image
  2. I put it on an SD card
  3. I created an empty ssh file on the boot partition
  4. I put the SD card on the Raspberry Pi and turned on the stereo.
  5. ssh pi@raspberrypi password raspberry
  6. sudo raspi-config to change the hostname, the password, and to enlarge the root partition to include all the rest of the space available in the SD card.

Music Player Daemon

This is the set up of the music player part, with mpd.

apt install mpd

The configuration file is /etc/mpd.conf. The changes I made are:

Make mpd accessible from my local network:

bind_to_address         "any"

Make mpd discoverable:

zeroconf_enabled                "yes"
zeroconf_name                   "stereo"

Allow anyone who visits me to control the playlist, and only me to run admin functions:

password                        "SECRET@read,add,control,admin"
default_permissions             "read,add,control"

At my first try, mpd hung when changing songs. I had to disable dmix by uncommenting the device option in the audio_output configuration. use_mmap is cargo-culted from the archlinux wiki.

audio_output {
        type            "alsa"
        name            "My ALSA Device"
        device          "hw:0,0"        # optional
        use_mmap        "yes"
}

If at some point I'll decide to use other audio software on the system, I'll probably want to play via pulseaudio.

Sending music to the stereo

I made a little script to sync the music directory on my laptop with /var/lib/mpd/music:

#!/bin/sh

rsync -avz --filter=". sync-stereo.filter" --copy-links --prune-empty-dirs --delete ./ pi@stereo:/var/lib/mpd/music

ssh pi@stereo "chmod u=rwX,go=rX -R /var/lib/mpd/music"

It uses this sync-stereo.filter rules file for rsync:

hide /_archive
include */
include **.mp3
hide *

mpd clients

mpc

$ mpc -h stereo status
UltraCat - Unexpected Little Happenings
[playing] #15/22   0:03/4:06 (1%)
volume: 80%   repeat: off   random: on    single: off   consume: off

M.A.L.P.

On my phone I installed M.A.L.P. and now I have a remote control for mpd.

In its settings, I made a profile for home where I just had to set the hostname for the stereo and the admin password.

Cantata

On my laptop I installed cantata, set the hostname and password in the preferences, and had the client ready.

Profit!

Now I can take the remote control of my hi-fi, turn it on, and after a while mpd will resume playing the song that was playing when I last shut it down.

I also have realtime player status on my phone and on my laptop, and can control music from either at any time. Friends who visit me can do that as well.

Everything was rather straightforward, well documented and easy to replicate. The hardware is cheap and very easy to come by.

debian eng hw pdo raspi-hifi sw
2017-03-16 12:01:00+01:00

Django signing signs, does not encrypt

As is says in the documentation. django.core.signing signs, and does not encyrpt.

Even though signing.dumps creates obscure-looking tokens, they are not encrypted, and here's a proof:

>>> from django.core import signing
>>> a = signing.dumps({"action":"set-password", "username": "enrico", "password": "SECRET"})
>>> from django.utils.encoding import force_bytes
>>> print(signing.b64_decode(force_bytes(a.split(":",1)[0])))
b'{"action":"set-password","password":"SECRET","username":"enrico"}'

I'm writing it down so one day I won't be tempted to think otherwise.

debian django eng pdo sw
2017-02-22 14:10:58+01:00

staticsite news: github mode and post series

GitHub mode

Tobias Gruetzmacher implemented GitHub mode for staticsite.

Although GitHub now has a similar site rendering mode, it doesn't give you a live preview: if you run ssite serve on a GitHub project you will get a live preview of README.md and the project documentation.

Post series

I have added support for post series, that allow you to easily interlink posts with previous/next links.

You can see it in action on links and on An Italian song a day, an ongoing series that is currently each day posting a link to an Italian song.

eng pdo ssite sw