Latest posts for tag eng
This looks straightforward and is far from it. I expect tool support will improve in the future. Meanwhile, this blog post serves as a step by step explanation for what is going on in code that I'm about to push to my team.
Let's take this relatively straightforward python code. It has a function printing an int, and a decorator that makes it argument optional, taking it from a global default if missing:
from unittest import mock
default = 42
def with_default(f):
def wrapped(self, value=None):
if value is None:
value = default
return f(self, value)
return wrapped
class Fiddle:
@with_default
def print(self, value):
print("Answer:", value)
fiddle = Fiddle()
fiddle.print(12)
fiddle.print()
def mocked(self, value=None):
print("Mocked answer:", value)
with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked):
fiddle.print(12)
fiddle.print()
It works nicely as expected:
$ python3 test0.py
Answer: 12
Answer: 42
Mocked answer: 12
Mocked answer: None
It lacks functools.wraps and typing, though. Let's add them.
Adding functools.wraps
Adding a simple @functools.wraps, mock unexpectedly stops working:
# python3 test1.py
Answer: 12
Answer: 42
Mocked answer: 12
Traceback (most recent call last):
File "/home/enrico/lavori/freexian/tt/test1.py", line 42, in <module>
fiddle.print()
File "<string>", line 2, in print
File "/usr/lib/python3.11/unittest/mock.py", line 186, in checksig
sig.bind(*args, **kwargs)
File "/usr/lib/python3.11/inspect.py", line 3211, in bind
return self._bind(args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/inspect.py", line 3126, in _bind
raise TypeError(msg) from None
TypeError: missing a required argument: 'value'
This is the new code, with explanations and a fix:
# Introduce functools
import functools
from unittest import mock
default = 42
def with_default(f):
@functools.wraps(f)
def wrapped(self, value=None):
if value is None:
value = default
return f(self, value)
# Fix:
# del wrapped.__wrapped__
return wrapped
class Fiddle:
@with_default
def print(self, value):
assert value is not None
print("Answer:", value)
fiddle = Fiddle()
fiddle.print(12)
fiddle.print()
def mocked(self, value=None):
print("Mocked answer:", value)
with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked):
fiddle.print(12)
# mock's autospec uses inspect.getsignature, which follows __wrapped__ set
# by functools.wraps, which points to a wrong signature: the idea that
# value is optional is now lost
fiddle.print()
Adding typing
For simplicity, from now on let's change Fiddle.print to match its wrapped signature:
# Give up with making value not optional, to simplify things :(
def print(self, value: int | None = None) -> None:
assert value is not None
print("Answer:", value)
Typing with ParamSpec
# Introduce typing, try with ParamSpec
import functools
from typing import TYPE_CHECKING, ParamSpec, Callable
from unittest import mock
default = 42
P = ParamSpec("P")
def with_default(f: Callable[P, None]) -> Callable[P, None]:
# Using ParamSpec we forward arguments, but we cannot use them!
@functools.wraps(f)
def wrapped(self, value: int | None = None) -> None:
if value is None:
value = default
return f(self, value)
return wrapped
class Fiddle:
@with_default
def print(self, value: int | None = None) -> None:
assert value is not None
print("Answer:", value)
mypy complains inside the wrapper, because while we forward arguments we don't
constrain them, so we can't be sure there is a value in there:
test2.py:17: error: Argument 2 has incompatible type "int"; expected "P.args" [arg-type]
test2.py:19: error: Incompatible return value type (got "_Wrapped[P, None, [Any, int | None], None]", expected "Callable[P, None]") [return-value]
test2.py:19: note: "_Wrapped[P, None, [Any, int | None], None].__call__" has type "Callable[[Arg(Any, 'self'), DefaultArg(int | None, 'value')], None]"
Typing with Callable
We can use explicit Callable argument lists:
# Introduce typing, try with Callable
import functools
from typing import TYPE_CHECKING, Callable, TypeVar
from unittest import mock
default = 42
A = TypeVar("A")
# Callable cannot represent the fact that the argument is optional, so now mypy
# complains if we try to omit it
def with_default(f: Callable[[A, int | None], None]) -> Callable[[A, int | None], None]:
@functools.wraps(f)
def wrapped(self: A, value: int | None = None) -> None:
if value is None:
value = default
return f(self, value)
return wrapped
class Fiddle:
@with_default
def print(self, value: int | None = None) -> None:
assert value is not None
print("Answer:", value)
if TYPE_CHECKING:
reveal_type(Fiddle.print)
fiddle = Fiddle()
fiddle.print(12)
# !! Too few arguments for "print" of "Fiddle" [call-arg]
fiddle.print()
def mocked(self, value=None):
print("Mocked answer:", value)
with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked):
fiddle.print(12)
fiddle.print()
Now mypy complains when we try to omit the optional argument, because Callable cannot represent optional arguments:
test3.py:32: note: Revealed type is "def (test3.Fiddle, Union[builtins.int, None])"
test3.py:37: error: Too few arguments for "print" of "Fiddle" [call-arg]
test3.py:46: error: Too few arguments for "print" of "Fiddle" [call-arg]
typing's documentation says:
Callable cannot express complex signatures such as functions that take a variadic number of arguments, overloaded functions, or functions that have keyword-only parameters. However, these signatures can be expressed by defining a Protocol class with a call() method:
Let's do that!
Typing with Protocol, take 1
# Introduce typing, try with Protocol
import functools
from typing import TYPE_CHECKING, Protocol, TypeVar, Generic, cast
from unittest import mock
default = 42
A = TypeVar("A", contravariant=True)
class Printer(Protocol, Generic[A]):
def __call__(_, self: A, value: int | None = None) -> None:
...
def with_default(f: Printer[A]) -> Printer[A]:
@functools.wraps(f)
def wrapped(self: A, value: int | None = None) -> None:
if value is None:
value = default
return f(self, value)
return cast(Printer, wrapped)
class Fiddle:
# function has a __get__ method to generated bound versions of itself
# the Printer protocol does not define it, so mypy is now unable to type
# the bound method correctly
@with_default
def print(self, value: int | None = None) -> None:
assert value is not None
print("Answer:", value)
if TYPE_CHECKING:
reveal_type(Fiddle.print)
fiddle = Fiddle()
# !! Argument 1 to "__call__" of "Printer" has incompatible type "int"; expected "Fiddle"
fiddle.print(12)
fiddle.print()
def mocked(self, value=None):
print("Mocked answer:", value)
with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked):
fiddle.print(12)
fiddle.print()
New mypy complaints:
test4.py:41: error: Argument 1 to "__call__" of "Printer" has incompatible type "int"; expected "Fiddle" [arg-type]
test4.py:42: error: Missing positional argument "self" in call to "__call__" of "Printer" [call-arg]
test4.py:50: error: Argument 1 to "__call__" of "Printer" has incompatible type "int"; expected "Fiddle" [arg-type]
test4.py:51: error: Missing positional argument "self" in call to "__call__" of "Printer" [call-arg]
What happens with class methods, is that the function object has a __get__
method that generates a bound versions of itself. Our Printer protocol does not
define it, so mypy is now unable to type the bound method correctly.
Typing with Protocol, take 2
So... we add the function descriptor methos to our Protocol!
A lot of this is taken from this discussion.
# Introduce typing, try with Protocol, harder!
import functools
from typing import TYPE_CHECKING, Protocol, TypeVar, Generic, cast, overload, Union
from unittest import mock
default = 42
A = TypeVar("A", contravariant=True)
# We now produce typing for the whole function descriptor protocol
#
# See https://github.com/python/typing/discussions/1040
class BoundPrinter(Protocol):
"""Protocol typing for bound printer methods."""
def __call__(_, value: int | None = None) -> None:
"""Bound signature."""
class Printer(Protocol, Generic[A]):
"""Protocol typing for printer methods."""
# noqa annotations are overrides for flake8 being confused, giving either D418:
# Function/ Method decorated with @overload shouldn't contain a docstring
# or D105:
# Missing docstring in magic method
#
# F841 is for vulture being confused:
# unused variable 'objtype' (100% confidence)
@overload
def __get__( # noqa: D105
self, obj: A, objtype: type[A] | None = None # noqa: F841
) -> BoundPrinter:
...
@overload
def __get__( # noqa: D105
self, obj: None, objtype: type[A] | None = None # noqa: F841
) -> "Printer[A]":
...
def __get__(
self, obj: A | None, objtype: type[A] | None = None # noqa: F841
) -> Union[BoundPrinter, "Printer[A]"]:
"""Implement function descriptor protocol for class methods."""
def __call__(_, self: A, value: int | None = None) -> None:
"""Unbound signature."""
def with_default(f: Printer[A]) -> Printer[A]:
@functools.wraps(f)
def wrapped(self: A, value: int | None = None) -> None:
if value is None:
value = default
return f(self, value)
return cast(Printer, wrapped)
class Fiddle:
# function has a __get__ method to generated bound versions of itself
# the Printer protocol does not define it, so mypy is now unable to type
# the bound method correctly
@with_default
def print(self, value: int | None = None) -> None:
assert value is not None
print("Answer:", value)
fiddle = Fiddle()
fiddle.print(12)
fiddle.print()
def mocked(self, value=None):
print("Mocked answer:", value)
with mock.patch.object(Fiddle, "print", autospec=True, side_effect=mocked):
fiddle.print(12)
fiddle.print()
It works! It's typed! And mypy is happy!
Suppose you have a meson project like this:
meson.build:
project('example', 'cpp', version: '1.0', license : '…', default_options: ['warning_level=everything', 'cpp_std=c++17'])
subdir('example')
example/meson.build:
test_example = executable('example-test', ['main.cc'])
example/string.h:
/* This file intentionally left empty */
example/main.cc:
#include <cstring>
int main(int argc,const char* argv[])
{
std::string foo("foo");
return 0;
}
This builds fine with autotools and cmake, but not meson:
$ meson setup builddir
The Meson build system
Version: 1.0.1
Source dir: /home/enrico/dev/deb/wobble-repr
Build dir: /home/enrico/dev/deb/wobble-repr/builddir
Build type: native build
Project name: example
Project version: 1.0
C++ compiler for the host machine: ccache c++ (gcc 12.2.0 "c++ (Debian 12.2.0-14) 12.2.0")
C++ linker for the host machine: c++ ld.bfd 2.40
Host machine cpu family: x86_64
Host machine cpu: x86_64
Build targets in project: 1
Found ninja-1.11.1 at /usr/bin/ninja
$ ninja -C builddir
ninja: Entering directory `builddir'
[1/2] Compiling C++ object example/example-test.p/main.cc.o
FAILED: example/example-test.p/main.cc.o
ccache c++ -Iexample/example-test.p -Iexample -I../example -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -Wcast-qual -Wconversion -Wfloat-equal -Wformat=2 -Winline -Wmissing-declarations -Wredundant-decls -Wshadow -Wundef -Wuninitialized -Wwrite-strings -Wdisabled-optimization -Wpacked -Wpadded -Wmultichar -Wswitch-default -Wswitch-enum -Wunused-macros -Wmissing-include-dirs -Wunsafe-loop-optimizations -Wstack-protector -Wstrict-overflow=5 -Warray-bounds=2 -Wlogical-op -Wstrict-aliasing=3 -Wvla -Wdouble-promotion -Wsuggest-attribute=const -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wtrampolines -Wvector-operation-performance -Wsuggest-attribute=format -Wdate-time -Wformat-signedness -Wnormalized=nfc -Wduplicated-cond -Wnull-dereference -Wshift-negative-value -Wshift-overflow=2 -Wunused-const-variable=2 -Walloca -Walloc-zero -Wformat-overflow=2 -Wformat-truncation=2 -Wstringop-overflow=3 -Wduplicated-branches -Wattribute-alias=2 -Wcast-align=strict -Wsuggest-attribute=cold -Wsuggest-attribute=malloc -Wanalyzer-too-complex -Warith-conversion -Wbidi-chars=ucn -Wopenacc-parallelism -Wtrivial-auto-var-init -Wctor-dtor-privacy -Weffc++ -Wnon-virtual-dtor -Wold-style-cast -Woverloaded-virtual -Wsign-promo -Wstrict-null-sentinel -Wnoexcept -Wzero-as-null-pointer-constant -Wabi-tag -Wuseless-cast -Wconditionally-supported -Wsuggest-final-methods -Wsuggest-final-types -Wsuggest-override -Wmultiple-inheritance -Wplacement-new=2 -Wvirtual-inheritance -Waligned-new=all -Wnoexcept-type -Wregister -Wcatch-value=3 -Wextra-semi -Wdeprecated-copy-dtor -Wredundant-move -Wcomma-subscript -Wmismatched-tags -Wredundant-tags -Wvolatile -Wdeprecated-enum-enum-conversion -Wdeprecated-enum-float-conversion -Winvalid-imported-macros -std=c++17 -O0 -g -MD -MQ example/example-test.p/main.cc.o -MF example/example-test.p/main.cc.o.d -o example/example-test.p/main.cc.o -c ../example/main.cc
In file included from ../example/main.cc:1:
/usr/include/c++/12/cstring:77:11: error: ‘memchr’ has not been declared in ‘::’
77 | using ::memchr;
| ^~~~~~
/usr/include/c++/12/cstring:78:11: error: ‘memcmp’ has not been declared in ‘::’
78 | using ::memcmp;
| ^~~~~~
/usr/include/c++/12/cstring:79:11: error: ‘memcpy’ has not been declared in ‘::’
79 | using ::memcpy;
| ^~~~~~
/usr/include/c++/12/cstring:80:11: error: ‘memmove’ has not been declared in ‘::’
80 | using ::memmove;
| ^~~~~~~
…
It turns out that meson adds the current directory to the include path by default:
Another thing to note is that
include_directoriesadds both the source directory and corresponding build directory to include path, so you don't have to care.
It seems that I have to care after all.
Thankfully there is an implicit_include_directories setting
that can turn this off if needed.
Its documentation is not as easy to find as I'd like (kudos to Kangie on IRC), and hopefully this blog post will make it easier for me to find it in the future.
Abstract
Debusine manages scheduling and distribution of Debian-related tasks (package build, lintian analysis, autopkgtest runs, etc.) to distributed worker machines. It is being developed by Freexian with the intention of giving people access to a range of pre-configured tools and workflows running on remote hardware.
Freexian obtained STF funding for a substantial set of Debusine milestones, so development is happening on a clear schedule. We can present where we are and, we're going to be, and what we hope to bring to Debian with this work.
Abstract
Although Debian has just turned 30, in my experience it has not yet fully turned adult: we sometimes squabble like boys in puberty, like children we assume that someone takes care of paying the bills and bringing out the trash, we procrastinate on our responsibilities and hope nobody notices.
At the same time, we cannot assume that people have the energy and motivation to do what is needed to keep the house clean and the boat afloat: Debian is based on people volunteering, and people have diverse and changing reasons to be with us, and private lives, loved ones and families, bills to be paid.
I want to start figuring out how to address practical issues around the sustainability of the Debian community, in a way that fits the needs and peculiarities of the Debian community.
The end does not justify the means: really, the means define what the end will be. I want to talk about the means: how to be sustainable, how to be interesting, how to be fun, how to have a community worth caring for, how to last for centuries
Debian: when you're more likely to get a virus than your laptop
Uhm, salsa is not resolving:
$ git fetch
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
fatal: Could not read from remote repository.
$ ping salsa.debian.org
ping: salsa.debian.org: Name or service not known
But... it is?
$ host salsa.debian.org
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 mailly.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
salsa.debian.org mail is handled by 10 muffat.debian.org.
It really is resolving correctly at each step:
$ cat /etc/resolv.conf
# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
# [...]
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
# [...]
nameserver 127.0.0.53
options edns0 trust-ad
search fritz.box
$ host salsa.debian.org 127.0.0.53
Using domain server:
Name: 127.0.0.53
Address: 127.0.0.53#53
Aliases:
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 mailly.debian.org.
salsa.debian.org mail is handled by 10 muffat.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
# resolvectl status
Global
Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Link 3 (wlp108s0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.178.1
DNS Servers: 192.168.178.1 fd00::3e37:12ff:fe99:2301 2a01:b600:6fed:1:3e37:12ff:fe99:2301
DNS Domain: fritz.box
Link 4 (virbr0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 9 (enxace2d39ce693)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.178.1
DNS Servers: 192.168.178.1 fd00::3e37:12ff:fe99:2301 2a01:b600:6fed:1:3e37:12ff:fe99:2301
DNS Domain: fritz.box
$ host salsa.debian.org 192.168.178.1
Using domain server:
Name: 192.168.178.1
Address: 192.168.178.1#53
Aliases:
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 muffat.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
salsa.debian.org mail is handled by 10 mailly.debian.org.
$ host salsa.debian.org fd00::3e37:12ff:fe99:2301 2a01:b600:6fed:1:3e37:12ff:fe99:2301
Using domain server:
Name: fd00::3e37:12ff:fe99:2301
Address: fd00::3e37:12ff:fe99:2301#53
Aliases:
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 muffat.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
salsa.debian.org mail is handled by 10 mailly.debian.org.
Could it be caching?
# systemctl restart systemd-resolved
$ dpkg -s nscd
dpkg-query: package 'nscd' is not installed and no information is available
$ git fetch
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
fatal: Could not read from remote repository.
Could it be something in ssh's config?
$ grep salsa ~/.ssh/config
$ ssh git@salsa.debian.org
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
Something weird with ssh's control sockets?
$ strace -fo /tmp/zz ssh git@salsa.debian.org
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
enrico@ploma:~/lavori/legal/legal$ grep salsa /tmp/zz
393990 execve("/usr/bin/ssh", ["ssh", "git@salsa.debian.org"], 0x7ffffcfe42d8 /* 54 vars */) = 0
393990 connect(3, {sa_family=AF_UNIX, sun_path="/home/enrico/.ssh/sock/git@salsa.debian.org:22"}, 110) = -1 ENOENT (No such file or directory)
$ strace -fo /tmp/zz1 ssh -S none git@salsa.debian.org
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
$ grep salsa /tmp/zz1
394069 execve("/usr/bin/ssh", ["ssh", "-S", "none", "git@salsa.debian.org"], 0x7ffd36cbfde8 /* 54 vars */) = 0
How is ssh trying to resolve salsa.debian.org?
393990 socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
393990 connect(3, {sa_family=AF_UNIX, sun_path="/run/systemd/resolve/io.systemd.Resolve"}, 42) = 0
393990 sendto(3, "{\"method\":\"io.systemd.Resolve.Re"..., 99, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 99
393990 mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4fc71ca000
393990 recvfrom(3, 0x7f4fc71ca010, 135152, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
393990 ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=119, tv_nsec=999917000}, NULL, 8) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=119, tv_nsec=998915689})
393990 recvfrom(3, "{\"error\":\"io.systemd.System\",\"pa"..., 135152, MSG_DONTWAIT, NULL, NULL) = 56
393990 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
393990 close(3) = 0
393990 munmap(0x7f4fc71ca000, 135168) = 0
393990 getpid() = 393990
393990 write(2, "ssh: Could not resolve hostname "..., 77) = 77
Something weird with resolved?
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
Let's try disrupting what ssh is trying and failing:
# mv /run/systemd/resolve/io.systemd.Resolve /run/systemd/resolve/io.systemd.Resolve.backup
$ strace -o /tmp/zz2 ssh -S none -vv git@salsa.debian.org
OpenSSH_9.2p1 Debian-2, OpenSSL 3.0.9 30 May 2023
debug1: Reading configuration data /home/enrico/.ssh/config
debug1: /home/enrico/.ssh/config line 1: Applying options for *
debug1: /home/enrico/.ssh/config line 228: Applying options for *.debian.org
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolving "salsa.debian.org" port 22
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
$ tail /tmp/zz2
394748 prctl(PR_CAPBSET_READ, 0x29 /* CAP_??? */) = -1 EINVAL (Invalid argument)
394748 munmap(0x7f27af5ef000, 164622) = 0
394748 rt_sigprocmask(SIG_BLOCK, [HUP USR1 USR2 PIPE ALRM CHLD TSTP URG VTALRM PROF WINCH IO], [], 8) = 0
394748 futex(0x7f27ae5feaec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
394748 openat(AT_FDCWD, "/run/systemd/machines/salsa.debian.org", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
394748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
394748 getpid() = 394748
394748 write(2, "ssh: Could not resolve hostname "..., 77) = 77
394748 exit_group(255) = ?
394748 +++ exited with 255 +++
$ machinectl list
No machines.
# resolvectl flush-caches
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
# resolvectl reset-statistics
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
# resolvectl reset-server-features
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
# resolvectl monitor
→ Q: salsa.debian.org IN A
→ Q: salsa.debian.org IN AAAA
← S: EINVAL
← A: debian.org IN NS sec2.rcode0.net
← A: debian.org IN NS sec1.rcode0.net
← A: debian.org IN NS nsp.dnsnode.net
← A: salsa.debian.org IN A 209.87.16.44
← A: debian.org IN NS dns4.easydns.info
I guess I won't be using salsa today, and I wish I understood why.
Update: as soon as I pushed this post to my blog (via ssh) salsa started resolving again.
Gtk4 has interesting ways of splitting models and views. One that I didn't find very well documented, especially for Python bindings, is a set of radio buttons backed by a common model.
The idea is to define an action that takes a string as a state. Each radio button is assigned a string matching one of the possible states, and when the state of the backend action is changed, the radio buttons are automatically updated.
All the examples below use a string for a value type, but anything can be used
that fits into a GLib.Variant.
The model
This defines the action. Note that enables all the usual declarative ways of a status change:
mode = Gio.SimpleAction.new_stateful(
name="mode-selection",
parameter_type=GLib.VariantType("s"),
state=GLib.Variant.new_string(""))
gtk_app.add_action(self.mode)
The view
def add_radio(model: Gio.SimpleAction, id: str, label: str):
button = Gtk.CheckButton(label=label)
# Tell this button to activate when the model has the given value
button.set_action_target_value(GLib.Variant.new_string(id))
# Build the name under which the action is registesred, plus the state
# value controlled by this button: clicking the button will set this state
detailed_name = Gio.Action.print_detailed_name(
"app." + model.get_name(),
GLib.Variant.new_string(id))
button.set_detailed_action_name(detailed_name)
# If the model has no current value set, this sets the first radio button
# as selected
if not model.get_state().get_string():
model.set_state(GLib.Variant.new_string(id))
Accessing the model
To read the currently selected value:
current = model.get_state().get_string()
To set the currently selected value:
model.set_state(GLib.Variant.new_string(id))
I acquired some unusual input devices to experiment with, like a CNC control panel and a bluetooth pedal page turner.
These identify and behave like a keyboard, sending nice and simple keystrokes, and can be accessed with no drivers or other special software. However, their keystrokes appear together with keystrokes from normal keyboards, which is the expected default when plugging in a keyboard, but not what I want in this case.
I'd also like them to be readable via evdev and accessible by my own user.
Here's the udev rule I cooked up to handle this use case:
# Handle the CNC control panel
SUBSYSTEM=="input", ENV{ID_VENDOR}=="04d9", ENV{ID_MODEL}=="1203", \
OWNER="enrico", ENV{ID_INPUT}=""
# Handle the Bluetooth page turner
SUBSYSTEM=="input", ENV{ID_BUS}=="bluetooth", ENV{LIBINPUT_DEVICE_GROUP}=="*/…mac…", ENV{ID_INPUT_KEYBOARD}="1" \
OWNER="enrico", ENV{ID_INPUT}="", SYMLINK+="input/by-id/bluetooth-…mac…-kbd"
SUBSYSTEM=="input", ENV{ID_BUS}=="bluetooth", ENV{LIBINPUT_DEVICE_GROUP}=="*/…mac…", ENV{ID_INPUT_TABLET}="1" \
OWNER="enrico", ENV{ID_INPUT}="", SYMLINK+="input/by-id/bluetooth-…mac…-tablet"
The bluetooth device didn't have standard rules to create /dev/input/by-id/
symlinks so I added them. In my own code, I watch /dev/input/by-id with
inotify to handle when devices appear or disappear.
I used udevadm info /dev/input/event… to see what I could use to identify the
device.
The Static device configuration via udev page of libinput's documentation has documentation on the various elements specific to the input subsystem
Grepping rule files in /usr/lib/udev/rules.d was useful to see syntax
examples.
udevadm test /dev/input/event… was invaluable for syntax checking and testing
my rule file while working on it.
Finally, this is an extract of a quick prototype Python code to read keys from the CNC control panel:
import libevdev
KEY_MAP = {
libevdev.EV_KEY.KEY_GRAVE: "EMERGENCY",
# InputEvent(EV_KEY, KEY_LEFTALT, 1)
libevdev.EV_KEY.KEY_R: "CYCLE START",
libevdev.EV_KEY.KEY_F5: "SPINDLE ON/OFF",
# InputEvent(EV_KEY, KEY_RIGHTCTRL, 1)
libevdev.EV_KEY.KEY_W: "REDO",
# InputEvent(EV_KEY, KEY_LEFTALT, 1)
libevdev.EV_KEY.KEY_N: "SINGLE STEP",
# InputEvent(EV_KEY, KEY_LEFTCTRL, 1)
libevdev.EV_KEY.KEY_O: "ORIGIN POINT",
libevdev.EV_KEY.KEY_ESC: "STOP",
libevdev.EV_KEY.KEY_KPPLUS: "SPEED UP",
libevdev.EV_KEY.KEY_KPMINUS: "SLOW DOWN",
libevdev.EV_KEY.KEY_F11: "F+",
libevdev.EV_KEY.KEY_F10: "F-",
libevdev.EV_KEY.KEY_RIGHTBRACE: "J+",
libevdev.EV_KEY.KEY_LEFTBRACE: "J-",
libevdev.EV_KEY.KEY_UP: "+Y",
libevdev.EV_KEY.KEY_DOWN: "-Y",
libevdev.EV_KEY.KEY_LEFT: "-X",
libevdev.EV_KEY.KEY_RIGHT: "+X",
libevdev.EV_KEY.KEY_KP7: "+A",
libevdev.EV_KEY.KEY_Q: "-A",
libevdev.EV_KEY.KEY_PAGEDOWN: "-Z",
libevdev.EV_KEY.KEY_PAGEUP: "+Z",
}
class KeyReader:
def __init__(self, path: str):
self.path = path
self.fd: IO[bytes] | None = None
self.device: libevdev.Device | None = None
def __enter__(self):
self.fd = open(self.path, "rb")
self.device = libevdev.Device(self.fd)
return self
def __exit__(self, exc_type, exc, tb):
self.device = None
self.fd.close()
self.fd = None
def events(self) -> Iterator[dict[str, Any]]:
for e in self.device.events():
if e.type == libevdev.EV_KEY:
if (val := KEY_MAP.get(e.code)):
yield {
"name": val,
"value": e.value,
"sec": e.sec,
"usec": e.usec,
}
Edited: added rules to handle the Bluetooth page turner
- str.endswith() can take a tuple of possible endings instead of a single string
About JACK and Debian
- There are 3 JACK implementations: jackd1, jackd2, pipewire-jack.
- jackd1 is mostly superseded in favour of jackd2, and as far as I understand, can be ignored
- pipewire-jack integrates well with pipewire and the rest of the Linux audio world
- jackd2 is the native JACK server. When started it handles the sound card directly, and will steal it from pipewire. Non-JACK audio applications will likely cease to see the sound card until JACK is stopped and wireplumber is restarted. Pipewire should be able to keep working as a JACK client but I haven't gone down that route yet
- pipewire-jack mostly works. At some point I experienced glitches in complex JACK apps like giada or ardour that went away after switching to jackd2. I have not investigated further into the glitches
- So: try things with pw-jack. If you see odd glitches, try without pw-jack to use the native jackd2. Keep in mind, if you do so, that you will lose standard pipewire until you stop jackd2 and restart wireplumber.
I have Python code for reading a heart rate monitor.
I have Python code to generate MIDI events.
Could I resist putting them together? Clearly not.
Here's Jack Of Hearts, a JACK MIDI drum loop generator that uses the heart rate for BPM, and an improvised way to compute heart rate increase/decrease to add variations in the drum pattern.
It's very simple minded and silly. To me it was a fun way of putting unrelated things together, and Python worked very well for it.