Write Raspbian image to the SD card

This is part of a series of posts on the design and technical steps of creating Himblick, a digital signage box based on the Raspberry Pi 4.

We have a SD card device with all partitions unmounted, we now need to write a Raspbian Lite image to it.

Since writing an image can take a long time, it's good to have a progressbar, and ETA estimation, during the process. This is sometimes tricky, as Linux caching by default makes everything look like it's progressing very quickly, to then get stuck doing the actual work the first time fdatasync is called.

To get a more useful progress indication we skipped using shutils.copyfileobj, but did our own read/write loop:

    def write_image(self, dev: Dict[str, Any]):
        """
        Write the base image to the SD card
        """
        backing_store = bytearray(16 * 1024 * 1024)
        copy_buffer = memoryview(backing_store)
        pbar = make_progressbar(maxval=os.path.getsize(self.settings.BASE_IMAGE))
        total_read = 0
        with open(self.settings.BASE_IMAGE, "rb") as fdin:
            with open(dev["path"], "wb") as fdout:
                pbar.start()
                while True:
                    bytes_read = fdin.readinto(copy_buffer)
                    if not bytes_read:
                        break
                    total_read += bytes_read
                    pbar.update(total_read)
                    fdout.write(copy_buffer[:bytes_read])
                    fdout.flush()
                    os.fdatasync(fdout.fileno())
                pbar.finish()

And here's make_progressbar:

class NullProgressBar:
    def update(self, val):
        pass

    def finish(self):
        pass

    def __call__(self, val):
        return val


def make_progressbar(maxval=None):
    if progressbar is None:
        log.warn("install python3-progressbar for a fancier progressbar")
        return NullProgressBar()

    if not os.isatty(sys.stdout.fileno()):
        return NullProgressBar()

    if maxval is None:
        # TODO: not yet implemented
        return NullProgressBar()
    else:
        return progressbar.ProgressBar(maxval=maxval, widgets=[
            progressbar.Timer(), " ",
            progressbar.Bar(), " ",
            progressbar.SimpleProgress(), " ",
            progressbar.FileTransferSpeed(), " ",
            progressbar.Percentage(), " ",
            progressbar.AdaptiveETA(),
        ])