First we’ll see how it all works, then I’ll describe an easy implementation of a build script used in my project 👉

Building AppImages

The modern ecosystem of AppImage tooling provides an essential tool, appimagetool, that can be included in a build pipeline to produce an AppImage file from a basic directory structure.

Getting the tool

Downloading appimagetool for your architecture is easy. Here’s how you can do it using curl:

ARCH="x86_64"
APPIMAGETOOL="appimagetool-$ARCH.AppImage"

curl -L -o $APPIMAGETOOL https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-$ARCH.AppImage
chmod +x $APPIMAGETOOL

Setting up the app directory

Now the tool can be used to create an AppImage file from an app directory. Setting up this directory is easy due to its minimal structure:

appdir/
    AppRun
    myapp.desktop
    myapp.png

The app directory only requires three files:

• An executable file named AppRun

  • This can be your application binary itself or a script that performs set up before executing your binary

• A .desktop file in the Desktop Entry specification format, used by desktop environments on Linux to launch applications

• An icon PNG file

Let’s explore the first two in more detail.

AppRun file

This file is the entry point into your package executed by the AppImage package structure. There are two approaches to providing this file:

1. Use a Bash script to perform setup before executing your binary:

#!/bin/sh

cd "$(dirname "$0")"
# Setup here
exec ./myapp

This file needs to be made executable:

chmod +x appdir/AppRun

Next, simply move your binary into the directory so it will be executed by the script:

mv myapp appdir/myapp

2. The other option is to rename your binary executable to AppRun and have it executed directly:

mv myapp appdir/AppRun

Desktop entry file

Desktop entry files have an INI-style format that looks like this:

[Desktop Entry]
Type=Application
Name=MyApp
Comment=The next big thing
Icon=myapp
Categories=Utility

The key-value pairings are self-explanatory. The important things to note are:

• The Icon value must match the filename of your icon, i.e. myapp refers to myapp.png in the directory

• appimagetool requires the Categories key to be present. See a list of valid categories here

Check out this tutorial for a more in-depth look at these files.

Building the AppImage

With the tool downloaded and the app directory structure in place, producing an AppImage file requires only one command:

./$APPIMAGETOOL appdir

If the process succeeds, you will find the AppImage file in the current directory. The final directory set up would look like this (assuming a x86_64 architecture):

myapp-project/
    appdir/
        AppRun
        myapp.desktop
        myapp.png
    appimagetool-x86_64.AppImage
    MyApp-x86_64.AppImage

Finally, make the AppImage file executable and run it:

chmod +x MyApp-x86_64.AppImage
./MyApp-x86_64.AppImage

Makefile implementation

Makefiles may seem archaic to some, but I find they’re perfectly suited for declaring simple build pipelines like the one above. Here’s a Makefile I created for the Wails project:

ARCH ?= x86_64

BIN = build/bin/brainstack
APPIMAGETOOL = build/appimagetool-$(ARCH).AppImage
APPIMAGE = Brainstack-$(ARCH).AppImage

$(BIN):
    wails build

appimage: build/$(APPIMAGE)

$(APPIMAGETOOL):
    curl -L -o $(APPIMAGETOOL) https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-$(ARCH).AppImage
    chmod +x $(APPIMAGETOOL)

build/$(APPIMAGE): $(BIN) $(APPIMAGETOOL)
    cp $(BIN) appdir/AppRun
    ./$(APPIMAGETOOL) appdir
    chmod +x $(APPIMAGE)
    mv $(APPIMAGE) build

.PHONY: appimage

Makefiles follow a simple paradigm. Rules with targets and prerequisites are declared like this:

targets: prerequisites
    command

Let’s walk through what happens when we run the appimage rule found in the above Makefile:

make appimage

The first thing to note is that the target of this rule is phony, as declared at the bottom of the Makefile. This means the target doesn’t represent a real file, so it will always run when called from the command-line like above. This is useful for high-level rules intended to be run by hand or from another script. When the appimage rule runs, it checks its prerequisites to find that build/$(APPIMAGE) doesn’t exist, so it in turn runs the matching rule:

build/$(APPIMAGE): $(BIN) $(APPIMAGETOOL)
    cp $(BIN) appdir/AppRun
    ./$(APPIMAGETOOL) appdir
    chmod +x $(APPIMAGE)
    mv $(APPIMAGE) build

Because this rule depends on the binary built by Wails and the downloaded tool, those rules will be triggered to produce the requisite files. Lastly, the rule’s commands are executed, which copy the application binary into the app directory, run the appimagetool to produce an AppImage file, make the file executable, and move the file into the build directory to keep the project directory clean. It’s that easy!

You can learn more about Makefiles here.

Conclusion

Packaging software as AppImages is one of the easiest options available to us on modern Linux desktops. The simple build pipeline can be integrated into CI systems such as GitHub Actions to build and publish standalone AppImage files automatically when you release your software!

Check back for more articles on this topic as I explore it further.

In this article I will explore my solution to synchronize text file data between devices and additionally back this data up to Google Drive. While this article focuses on Logseq as an example, the information presented here is easily adaptable to general synchronization and backup requirements.

My open-source utilities of choice for this purpose are:

• Syncthing, a "continuous file synchronization program"

• rclone, a "command-line program to manage files on cloud storage"

My integrated solution involves the synchronization of data from two machines to a local server where cloud backups are performed. All of my devices run Linux, but this setup should be adaptable to other operating systems.

Let's take a look at how to set this up! 🛠️

Synchronizing with Syncthing

I suggest you begin with Syncthing's thorough Getting Started guide. Here you will find installation instructions as well as introductions to its specific terminology including "local" and "remote" devices.

For my setup, I chose to sync the entire ~/Documents directory, home to Logseq's data folder. The following screenshots are from the "local" device (my local home server) hosting a Syncthing instance in a container - this is why the folder path exists inside the /config directory.

I share this directory between two devices with the hostnames intent and sol.

This approach means the two devices don't need to be powered on simultaneously to sync data between them. The local server acts as an "always on" intermediary.

For more information about running Syncthing in a container on a home server, check out my blog post about using Ansible for precisely this purpose!

Backing up with rclone

While Syncthing ensures all devices on the local network have access to the same Logseq data, it's important to back this data up outside the network. I chose to use Google Drive for this purpose as I find it an easy-to-use option with generous free storage space. The featureful rclone tool interfaces with many other cloud storage options as well, so feel free to use any of its supported providers.

Setting up rclone

First, install rclone using these instructions.

Next, initialize a new remote using rclone's interactive setup processing, following the instructions provided in its excellent Google Drive documentation.

Now, you can backup the synchronized Documents directory using this rclone command (assuming Syncthing is running in a container with a volume named syncthing-config, using a remote named "remote"):

rclone sync ~/.local/share/containers/storage/volumes/syncthing-config/_data/Documents remote:Backup/Documents

The ~/.local/share/containers/... path is where container volumes are stored by podman, an alternative to Docker.

If you're synchronizing the entire Documents directory but you're not running Syncthing in a container, you can back up the Documents directory directly:

rclone sync ~/Documents remote:Backup/Documents

Backups reside in the Backup directory at the top level of my Google Drive. Here you can see the "brain" graph inside the Documents/logseq directory backed up as intended.

However, this required manually running the rclone command-line tool. Let's automate this process using systemd timers!

Scheduling backups

First, create a new systemd service file ~/.config/systemd/user/rclone-sync.service containing this content (adapting the rclone command for your specific setup):

[Unit]
Description=rclone backup

[Service]
Type=oneshot
ExecStart=/usr/bin/rclone sync ~/.local/share/containers/storage/volumes/syncthing-config/_data/Documents remote:Backup/Documents

Next, create a systemd timer file ~/.config/systemd/user/rclone-sync.timer. Note that the filename (minus the .timer extension) must match the service name, i.e. rclone-sync. The file will look like this:

[Unit]
Description=rclone hourly backup

[Timer]
OnBootSec=1h
OnUnitActiveSec=1h

[Install]
WantedBy=timers.target

Here, the OnBootSec= and OnUnitActiveSec= options instruct systemd to run this timer an hour after the machine boots, then an hour after the last activation, so this timer will run continuously every hour. Should you prefer an alternative backup schedule, see this documentation for information about the available time specifications.

Finally, enable the timer with this command:

systemctl --user enable rclone-sync.timer

You can observe the status of timers using this command:

systemctl --user list-timers

Conclusion

The availability of open-source synchronization and backup tools allows anyone to create custom solutions for their given use case. With the setup explored here, Logseq data is automatically synchronized between devices through a home server where this data is backed up to the cloud so that your knowledge base is readily available on your network and securely stored outside of it for redundancy and peace of mind.

Embrace the synchronicity! 🥳

It allows the use of extensions to modify its behavior, which some contest as hacks, saying they monkey-patch the live shell engine, run unchecked by discrete APIs or permissions, and constantly break after updates due to their scopeless nature.

In response, I offer this sage wisdom: Let us not forgo what is for what could be. Instead, let's get hacking 🔧

First steps

GNOME Shell includes a handy tool for scaffolding a new extension. Get to it with haste using the following command:

gnome-extensions create --interactive

You'll be walked through a few prompts to initialize an extension project and the main extension.js file should be automatically opened in your editor. If it's not, you can find the extension in the ~/.local/share/gnome-shell/extensions directory with the UUID given in the interactive creation process.

You can find more information about this process here.

The framework

GNOME Shell extensions are written in JavaScript and the extension.js file is home to your extension's code.

The file is populated with class-based scaffolding like this:

import { Extension } from 'resource:///org/gnome/shell/extensions/extension.js';

export default class MyExtension extends Extension {
    constructor(metadata) {
        super(metadata);
    }

    enable() {
        this._settings = this.getSettings();
    }

    disable() {
        this._settings = null;
    }
}

This simple framework determines the lifecycle of your extension:

• Extending the Extension class allows access to functions such as this.getSettings()

• The constructor method is a standard JavaScript feature used to initialize an instance of a class

  • The parent class, Extension, must also be constructed using super(metadata)

• The enable and disable class methods are called when your extension is enabled and disabled in the GNOME Extensions app, respectively

  • enable is responsible for integrating into GNOME Shell to establish your extension's functionality

  • disable is responsible for removing this integration so your extension no longer interferes

For more information about the changes to module imports made in GNOME Shell 45, see this guide to porting extensions.

What's in an extension?

As mentioned before, GNOME Shell extensions are essentially scopeless—in other words, they have boundless capabilities. The Guide to JavaScript for Gnome cites that extensions can:

• Use Mutter to control displays, workspaces and windows

• Use Clutter & St to create new UI elements

• Use JavaScript modules to create new UI elements

• Use any other library supporting GObject-Introspection

• Access and modify any internal GNOME Shell code

Good luck with all that! My experience creating extensions is much more limited, having built an extension that only performs basic window management. Let's look into it as a narrow example of extension development.

An extension case study

My extension is called Fifty Fifty and its code is available here. Its primary function is to resize two windows to fit side-by-side. This is easily accomplished with the following code:

function placeWindow(window, r) {
  window.unmaximize(Meta.MaximizeFlags.HORIZONTAL)
  window.unmaximize(Meta.MaximizeFlags.VERTICAL)
  window.move_resize_frame(false, r.x, r.y, r.width, r.height)
}

The complexity comes from determining which windows to tile. This depends on the history of focused windows on a workspace, which (as far as I can tell) is not stored in a manner accessible to extensions. The bulk of the extension's code is related to connecting and disconnecting signals to track and record the focus history.

This occurs in the extension's enable method, like this:

enable() {
    this.globalSignals[0] = global.workspace_manager.connect(
      "workspace-added",
      (_, index) => this._onWorkspaceAdded(index)
    )
    this.globalSignals[1] = global.workspace_manager.connect(
      "workspace-removed",
      (_, index) => this._onWorkspaceRemoved(index)
    )

    for (let i = 0; i < global.workspace_manager.n_workspaces; i++) {
      this._onWorkspaceAdded(i)
    }

    // We'll look at the remaining code later...
}

The signal connection IDs must be saved to remove them in the disable method.

In essence, this code sets up signals to react to the addition and removal of workspaces, which triggers the connection and disconnection of more signals responsible for reacting to the addition and removal of windows to this workspace.

Here's the _onWorkspaceAdded method:

_onWorkspaceAdded(index) {
    const workspace = global.workspace_manager
        .get_workspace_by_index(index)
    const data = { focusHistory: [], signals: [] }

    data.signals[0] = 
        workspace.connect("window-added", (_, window) =>
        this._onWindowAdded(window)
    )
    data.signals[1] = 
        workspace.connect("window-removed", (_, window) =>
        this._onWindowRemoved(window)
    )
    this.workspaces.set(workspace, data)

    workspace.list_windows()
        .forEach((window) => this._onWindowAdded(window))
}

Again, the signal IDs are stored in the data.signals array to be later disconnected in the _onWorkspaceRemove method.

Ultimately, it's the _onWindowAdded method that establishes the focus history tracking:

_onWindowAdded(window) {
    if (!window || window.get_window_type() !== Meta.WindowType.NORMAL)
        return

    this.windowSignals.set(
      window,
      window.connect("focus", () => this.updateFocusHistory(window))
    )

    if (window.has_focus()) this.updateFocusHistory(window)
}

Now, all of this orchestration would be useless without some means of activating the fifty-fifty tiling.

This is accomplished by defining a keybinding in the aforementioned enable method:

Main.wm.addKeybinding("toggle", settings, flag, mode, () => {
    const workspace = global.workspace_manager.get_active_workspace()
    const windows = this.workspaces
      .get(workspace)
      .focusHistory.slice(0, 2)
      .reverse()

    if (windows.length === 1) {
      placeWindow(windows[0], generateRects(windows[0], 0).workspace)
      return
    }

    let cachedRects = []
    let i = 0

    const fullscreen = windows.reduce((state, window) => {
      const r = generateRects(window, i)
      cachedRects[i++] = r
      return state && rectsEqual(window.get_frame_rect(), r.window)
    }, true)

    i = 0
    windows.forEach((window) => {
      const r = cachedRects[i++]
      placeWindow(window, fullscreen ? r.workspace : r.window)
      window.raise()
    })
})

The logic here gives this extension some dynamic functionality depending on the tiled and focused state of the two most recently focused windows:

• If at least one of the two most recently focused windows is not tiled, tile both with the focused window on the left.

• If the focused window is tiled on the right, swap it with the window on the left.

• If the focused window is tiled on the left, maximize the two most recently focused windows.

• If only one window has focus history, maximize it.

Schema files

The above keybinding is made accessible to GNOME Shell via a GSettings schema file.

Create such a file in the schemas directory of your extension project:

mkdir schemas
touch schemas/org.gnome.shell.extensions.example.gschema.xml

In the above filename, example should reflect the UUID specified at extension initialization. In the case of my extension, the UUID is fiftyfifty@jcrd.github.io so the schema filename is org.gnome.shell.extensions.fiftyfifty.gschema.xml.

The schema itself looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<schemalist>
    <schema id="org.gnome.shell.extensions.fiftyfifty"
            path="/org/gnome/shell/extensions/fiftyfifty/">
        <key type="as" name="toggle">
            <default><![CDATA[['<Super>f']]]></default>
            <summary>Toggle split</summary>
        </key>
    </schema>
</schemalist>

As you can see, the <Super>f keybinding is assigned to the toggle function.

For GNOME Shell to use the schema, it must first be compiled with this command:

glib-compile-schemas schemas/

Details about these schemas and changing them via a preferences window can be found here.

Using extensions

Now, with the extension in a functional state, there are two approaches to testing it:

• Start a nested instance of GNOME Shell with this command:

    dbus-run-session -- gnome-shell --nested --wayland
  

• Restart your active instance of GNOME Shell by logging out and back in

After either of these steps, the extension should be registered by GNOME Shell and can be enabled with the following:

gnome-extensions enable fiftyfifty@jcrd.github.io

There we go. Not even the sky's the limit when it comes to extension functionality, so don't be afraid to shoot for the moon and hack away!

What's next?

The Extensions section of the Guide to JavaScript for Gnome is an invaluable resource for extension development, containing much of this information and more.

If you build something useful, submit your extension for distribution on extensions.gnome.org after checking the review guidelines.

Explore the gnome-shell-extension topic on GitHub for some inspiration 🍀

As a follow up to my article on reproducible distro configuration, let's explore how to set up a home server running Linux using Ansible, plus my method of deploying self-hosted applications via containers for the ultimate combination of stability, flexibility, and modernity!

And as a bonus, we'll take a look at my server landing page software designed for precisely this use case.

Prerequisites

• Home server hardware

• Ansible installed on your workstation and familiarity with its use

The OS

Let's start with the operating system. Many Linux distributions can adequately fill this role:

• Debian

• Ubuntu Server

• OpenSUSE Leap

• Fedora Server

• Any of the RHEL clones such as Rocky Linux or AlmaLinux

I use AlmaLinux for a number of reasons:

• It's built on modern software

  • dnf, a powerful RPM package manager

  • firewalld, a firewall service supported by Ansible out of the box

  • podman, the easy-to-use container engine we'll use to run our apps

• It's robust and stable, requiring minimal updates

• Access to the Extra Packages for Enterprise Linux (EPEL) repository means lots of available software

  • Additionally, Copr can build against this repository so packaging your own software is easy

Documentation for your distribution of choice should walk you through the download and installation process.

Make sure your server is accessible via ssh!

Going forward, this article will assume the use of an RPM-based distro with the aforementioned software.

The set up

Rather than manually installing and configuring software on my home server, I prefer to automate this process using Ansible. This means my Ansible playbook is the single source of truth regarding the home server's configuration and it can be easily reproduced as necessary.

Here's my localserver.yml playbook file:

- hosts: localserver
  pre_tasks:
    - name: Get host IP
      shell: hostname -I | awk '{print $1}'
      register: hostip_cmd
  vars:
    hostip: "{{ hostip_cmd.stdout }}"
    homedir: "{{ lookup('env', 'HOME') }}"
    configdir: "{{ homedir }}/.config"
    mediadir: "{{ homedir }}/media"
    easymodeconfig: "{{ homedir }}/.easymode-config"
  roles:
    - base
    - dotfiles
    - openvpn_client
    - jellyfin
    - qbittorrent
    - rclone
    - syncthing
    - easymode

As you can see, many roles are involved in the set up of my server. We will look into these shortly.

In order to run such a playbook, first configure Ansible's /etc/ansible/hosts file like so:

[localserver]
192.168.1.111 # Your server's local IP address

Then the following command can be used:

ansible-playbook --ask-pass --ask-become-pass localserver.yml

It will prompt for the server's password, connect via ssh, and proceed to run the playbook's roles.

Now, let's see how these roles make use of podman for deploying containerized applications!

The apps

Before going any further, let's answer the question: why use containers?

The primary rationale is this:

• Any software designed to be self-hosted likely offers a containerized version or has been packaged as such by a third party for easy installation

• These user-facing applications needn't follow your distro's release cadence or even be packaged by said distro

Let's look at syncthing as an example. The LinuxServer community offers a containerized image on Docker Hub 👏

Keep in mind: Docker images comply with the OCI Container standard and are fully compatible with podman.

Installing this software with Ansible is simple enough—it requires only a few tasks and files.

I keep all the necessary tasks in a main.yml tasks file:

- name: Check existence of container
  command: podman container exists syncthing
  register: container_exists
  failed_when: container_exists.rc > 1

- name: Create container
  command: >
    podman create --name=syncthing
    --userns=""
    -e PUID=0
    -e PGID=0
    -p 8384:8384
    -p 22000:22000/tcp
    -p 22000:22000/udp
    -p 21027:21027/udp
    -v syncthing-config:/config
    docker.io/linuxserver/syncthing
  when: container_exists.rc == 1

- name: Configure firewalld
  firewalld:
    service: "{{ item }}"
    state: enabled
    immediate: true
    permanent: true
  loop:
    - syncthing
    - syncthing-gui
  become: true

- name: Install systemd service
  copy:
    src: syncthing.service
    dest: '{{ configdir }}/systemd/user/syncthing.service'
    mode: 0644

- name: Enable systemd service
  systemd:
    name: syncthing.service
    scope: user
    state: started
    enabled: true
    daemon_reload: true

- name: Install easymode config
  copy:
    src: syncthing.yml
    dest: "{{ easymodeconfig }}/"

These tasks are fairly self-documenting, but let's go through them in order nonetheless:

1. First, check if the container already exists using podman

2. If it doesn't yet exist, create it using podman

   • You can typically find the appropriate container creation flags on the app's Docker Hub page

3. Configure firewalld to open syncthing's ports

   • firewalld comes with "services" that define the ports used by common software such as syncthing!

4. Install and enable the systemd service (we'll look into this next)

5. Install the accompanying easymode configuration file

   • easymode is a server landing page of my own creation—stick around to the end for more info!

Because podman doesn't operate as a daemon in the manner of docker, we need a syncthing.service file to run our container at system startup:

[Unit]
Description=Syncthing podman container

[Service]
ExecStart=/usr/bin/podman start -a syncthing
ExecStop=/usr/bin/podman stop -t 10 syncthing
Restart=on-failure

[Install]
WantedBy=default.target

The location of these files must follow Ansible's role directory structure:

roles/
    syncthing/
        files/
            syncthing.service
            syncthing.yml # easymode config file
        tasks/
            main.yml

And that's all it takes! Repeat as needed to install the containerized apps of your choosing 🤩

The landing page

I find it useful to set up a server landing page so I don't need to memorize the ports used by the various web interfaces of running apps.

There are a few popular projects in this domain:

• Organizr

• Heimdall

I used Heimdall for a time, and it works well, but I wanted a landing page app with a setup procedure more conducive to automation. Thus, easymode was born! It's designed to be configured by automation systems such as Ansible.

The configuration file for syncthing is this simple:

name: syncthing
url: 8384
icon: arcticons:syncthing

The file is installed to easymode's configuration directory; the files here become entries in the generated landing page.

You may have noticed the server's IP address is stored as a variable when running my Ansible playbook. This is used by easymode to construct the correct URL given the above port!

Here's the task responsible for creating easymode's container:

- name: Create container
  command: >
    podman create --name=easymode
    -p 80:80
    -e EASYMODE_HOSTNAME={{ ansible_hostname }}
    -e EASYMODE_IP={{ hostip }}
    -v {{ easymodeconfig }}:/config:z
    docker.io/supplantr/easymode

The rest of the role closely resembles the syncthing role we explored above.

Finally, navigate to your home server's IP address and enjoy!

There are a few naive approaches to making this calculation:

1. Convert the image to greyscale and use either the average or the root mean square pixel brightness.

2. Calculate perceived brightness using the pixel average.

The latter uses magic numbers found in this article about the HSP color model:

$$brightness = sqrt( .299 R^2 + .587 G^2 + .114 B^2 )$$

This Stack Overflow answer provides some alternatives and their implementations.

I chose the first option, so let's see what the full implementation looks like in Python:

import io
import subprocess

from PIL import Image, ImageStat


def get_brightness(path):
        # Get a single frame from the webcam device using ffmpeg,
        # sending the image bytes to standard output to be captured.
        ps = subprocess.run(
            f"ffmpeg -i {path} -vframes 1 -f image2pipe -".split(),
            capture_output=True,
        )
        # Convert the image bytes into a BytesIO object and open it
        # as a Pillow Image.
        im = Image.open(io.BytesIO(ps.stdout))
        # Convert the image to greyscale.
        im = im.convert("L")
        # Return the root mean square (rms) brightness value.
        return ImageStat.Stat(im).rms[0]

get_brightness("/dev/video0")

The Python Pillow library makes this easy, so what's the problem? 🤔

The first problem arose immediately: on my laptop, a 7th generation ThinkPad X1 Carbon, the webcam doesn't seem to fully calibrate itself to ambient light within the time it takes to capture a single frame, so this calculation would yield extremely low values.

I worked around this by "waking up" the webcam using this function:

def wakeup(path):
        subprocess.run(
            f"ffmpeg -i {path} -vframes 10 -f image2pipe -".split(),
            stdout=subprocess.DEVNULL,
            stderr=subprocess.DEVNULL,
        )

I chose 10 frames arbitrarily. These frames are discarded by directing ffmpeg's output to the null device. Afterward, the above get_brightness function returns a value that much better reflects the ambient brightness level.

Indoors, at least. Office? No problem. Coffee shop? Should be fine. Outside, lit by the sun from the front, against a backdrop of dark-leaved trees? In this case, the calculation is wildly undervalued—which makes sense: most of the image is dark.

How can this be worked around? It's possible to detect the user's face in this image and calculate the brightness of that area of the image alone, which could work in this specific case, but consider another scenario: in a dark room, a very bright laptop screen illuminates its user's face. These captured images would likely produce similar pixel brightness values, yet the ambient light levels are completely opposite! Clearly, these methods utilizing averages are too dependent on an image's background composition.

Ultimately, I don't think there's a reliable and consistent method of correctly detecting ambient light level based on a webcam image, but the resulting code might still be useful in some situations.

If you have any insights, please share them!

Let's explore the hardware, software, and terrarium setup that bring this world to life! 🌍

The hardware

This project uses several hardware components:

• An Arduino board capable of running CircuitPython (Adafruit QT Py RP2040)

• A compatible real-time clock (Adafruit PCF8523)

  • This will also require an appropriate battery

• A means of connecting them (STEMMA QT cable)

• A NeoPixel LED ring (NeoPixel Ring 24 x 5050)

• A means of connecting that to the Arduino board

  • I wanted a way to detach the LED ring from the rest of the assembly so I used a pair of matching Molex PicoBlade-compatible 3-pin cables (such as these)

• An external power supply for the LEDs

• A "project box" capable of holding these components

I power both the LED ring and Arduino board via a 5V 2A power supply with a DC power adapter. This requires additional components:

• A 1000 µF capacitor used across the + and - terminals of the DC adapter, as noted here:

When using a DC power supply, or an especially large battery, we recommend adding a large capacitor (100 to 1000 µF, 6.3V or higher) across the + and – terminals.

• A diode (such as the common 1N4007 rectifier diode) between the 5V pin on your board and the power supply, as noted here:

5V - This is 5v out from the USB port. You can also use this as a voltage input but you must have some sort of diode (schottky, signal, power, really anything) between your external power source and this pin with anode to battery, cathode to 5V pin.

Solder the LED ring input (labeled "DIN" or "DI") to the digital pin 5 (labeled "SCL") on your board, and run the power and ground wires to their appropriate DC adapter terminals. Power the Arduino board with a wire to ground and the diode mentioned above. Make sure the capacitor is connected as required. Lastly, hook up the real-time clock using the STEMMA QT cable. My finished assembly looks like this:

To prepare the project box, I drilled out a hole for the female DC connector and sawed a slot on the opposite side for the LED ring wires:

Fit the assembly into the project box:

The software

This process assumes use of a Linux system and basic experience working with Arduino boards and the command line.

The software controlling the artificial sun is written in CircuitPython. Follow this guide to set it up on a QT Py RP2040 board.

Make sure you install the library bundle, then obtain the source code:

git clone https://github.com/jcrd/artificial-sun

Assuming the name of your device is CIRCUITPY, you can use an included update script to set the correct current time of your real-time clock and install the code file:

cd artificial-sun
./update.sh

Your sun should now be radiating light (and a little warmth for good measure 🌞).

How does it work?

The main logic of the code is in the linear interpolation of a small set of data relating time-of-day to LED light brightness and color.

light_data = [
    # Time, (Brightness, (Red, Green, Blue, White color))
    (0.0, (0.0, (0, 0, 0, 0))),
    (7.0, (0.1, (244, 254, 169, 0))),
    (8.0, (0.2, (243, 254, 174, 0))),
    (13.0, (1.0, (234, 254, 202, 0))),
    (18.0, (0.2, (243, 254, 174, 0))),
    (19.0, (0.1, (244, 254, 169, 0))),
    (24.0, (0.0, (0, 0, 0, 0))),
]

Its meant to be an approximation of sunrise, midday, and sunset. I expect the data to change as I continue experimenting with it over time.

The lerp function itself is straightforward:

def lerp(begin, end, t):
    return begin + t * (end - begin)

It calculates a position between the begin and end values given a percentage t. For example: let's start at 0 and end at 20; a t value of 0.4 is the value 8.

The function used to interpolate between the brightness and color values associated with each time-of-day set is a bit more complex:

def get_light_lerp(daytime):
    pt, (pb, pc) = (0, (0, (0, 0, 0, 0)))

    for t, (b, c) in light_data:
        if t == daytime:
            print("{}: {} @ {}".format(t, c, b))
            return (b, c)
        if t < daytime:
            pt, pb, pc = t, b, c
            continue
        lt = (daytime - pt) / (t - pt)
        lb = lerp(pb, b, lt)
        lc = tuple(int(lerp(pc[i], v, lt)) for i, v in enumerate(c))
        print("{} -> {} ({}): {} @ {}".format(pt, t, lt, lc, lb))
        return (lb, lc)

Let's walk through the logic imagining its 11 o'clock:

• If the current time matches a set exactly, return the associated data.

• Otherwise, loop through the data recording each set with a time less than 11.

• When a time greater than 11 is encountered, the last recorded set will be the beginning of the lerp function, while the current set is the end.

  • Given the above data, the beginning and end sets will be at 8 and 13 o'clock respectively.

• With this, calculate the percentage that describes 11 o'clock:

$$(11 - 8) / (13 - 8) = 0.6$$

• Now the brightness and color values can be individually interpolated between the two sets using the percentage! 🤯

The terrarium

The first step of building a terrarium is to select the enclosure itself. I used a generic pentagonal "geometric" terrarium, readily available online and in local hobby stores. The design of your enclosure will also determine if your terrarium is "open" or "closed":

• An open terrarium allows air to circulate throughout, meaning moisture will evaporate;

• A closed terrarium will have a lid of some sort (think: mason jar), meaning moisture will be sealed inside.

This critical difference determines the plants most suitable for your terrarium. I recommend you check out one of the many online guides that go into detail about which plants to select and the ideal makeup of your substrate. As my enclosure is open, I collected the appropriate substrate materials, picked out some succulents, and got to work assembling my world. 🌵

Next, it was time to give it light by attaching the LED ring. There are many ways to go about this. After some failed experimentation, I settled on using corkboard to construct a "lid" for the open face of my enclosure made of two layers glued together with a hole the size of the inner diameter of the LED ring cut out of each so that air is still able to circulate. This only required the use of scissors, a precision knife, and a hot glue gun. Finally, I pushed pins through the LED ring's PCB holes to fix it to the underside of the corkboard assembly so that it can be easily detached if necessary.

Here's the finished product:

For more pizzazz, you might consider building something similar out of wood!

Finally, I cut a piece of adhesive hook and loop tape (commonly known by the brand name Velcro) to fit the project box which I attached to the back of the terrarium:

Supply power to the DC adapter and step back to revel in the miniature majesty!

Check out this brief demo video for the full effect:

Writing SPEC files

RPM packages are produced using SPEC files as input, which specify the build and installation processes of some software in addition to metadata such as its name and version. Let's look at Clight as a case study. Here's its SPEC file:

Given such a SPEC file, it's possible to generate an RPM package using the rpmbuild tool, and this is a valid way of going about it. However, I use toolboxcutter because it builds packages in a podman container, meaning:

• Build dependencies won't litter your base system

• You won't miss a dependency specification that is coincidentally already installed

I wrote about using toolboxcutter for this purpose here!

The general procedure begins like this:

git clone https://github.com/FedeDP/Clight
cd Clight
tb init rpkg-toolbox

This tb command seen above is toolboxcutter, and calling init will create and open a Dockerfile.toolbox file in this repo's directory, built from rpkg-toolbox, which is a local podman image with rpkg and other packages installed. This is its Dockerfile:

FROM registry.fedoraproject.org/fedora-toolbox:37

RUN dnf install -y neovim
RUN dnf install -y rpkg
RUN dnf install -y rpmdevtools
RUN dnf install -y zsh

RUN ln -s /usr/bin/nvim /usr/local/bin/vim

CMD /usr/bin/zsh

toolboxcutter will eventually build Clight in the container specified by the Dockerfile in its directory, so let's add the required build dependencies:

FROM rpkg-toolbox

RUN dnf install -y cmake
RUN dnf install -y g++
RUN dnf install -y systemd-devel
RUN dnf install -y popt-devel
RUN dnf install -y libconfig-devel
RUN dnf install -y gsl-devel
RUN dnf install -y dbus-devel
RUN dnf copr enable -y jcrd/libmodule
RUN dnf install -y libmodule

Thankfully, Clight's dependencies are recorded in its documentation. For other software, some trial and error may be necessary to determine what's required. The next step is to specify these dependencies in the rpkg SPEC template file. rpkg's documentation describes how to get started with such a template.

In the directory containing the to-be-packaged software's source, let's create a subdirectory and the rpkg SPEC template file:

mkdir spec
touch spec/clight.rpkg.spec

Now, edit the created file, filling in the SPEC template details. Clight's looks like this:

As you can see, it resembles the final SPEC file but contains some rpkg macros enclosed in {{{ }}}. These allow the production of an RPM package directly from the source repository where this template resides. The installed files listed under the %files section may not be obvious without first attempting to build the package.

From outside the toolbox container, run this command:

tb rpkg-install

Without the proper installed files list, you will receive an error such as:

error: Installed (but unpackaged) file(s) found:
   /etc/default/clight.conf
   /usr/bin/clight
   /usr/etc/xdg/autostart/clight.desktop
   /usr/include/clight/public.h
   /usr/share/applications/clightc.desktop
   /usr/share/bash-completion/completions/clight
   /usr/share/clight/inhibit_bl.skel
   /usr/share/clight/nightmode.skel
   /usr/share/clight/synctemp_bumblebee.skel
   /usr/share/dbus-1/services/org.clight.clight.service
   /usr/share/icons/hicolor/scalable/apps/clight.svg
   /usr/share/man/man1/clight.1.gz
   /usr/share/zsh/site-functions/_clight

You can fill in the %files section with these paths using the appropriate RPM macros as seen above. Rerunning toolboxcutter's rpkg-install command should now produce and install an RPM package.

Finally, convert the rpkg SPEC template into a standalone spec file by replacing rpkg's {{{ }}} macros with their concrete values. rpkg can handle this automatically to an extent (with rpkg spec), with the caveat being: the package Source will be directly derived from the software's hosted source code repository, whereas you probably want to specify a source code archive associated with a released version of this software.

Compare the aforementioned SPEC file with the above rpkg SPEC template if you need help determining which values belong where. Once this is complete, you have a working SPEC file capable of producing an installable RPM package! 💪

Building distributable RPM packages

With the SPEC file in hand, building a distributable RPM package is easy using Copr. It's similar in theory to Ubuntu's ppas. You can find more information about the Copr build system here. An alternative might be OBS (Open Build Service) but I have not yet used it myself.

This tutorial illustrates the process of creating a new Copr project and initiating a build. However, instead of specifying SRPM files by URL, you can provide the URL of a SPEC file. I host my SPEC files on gist.github.com for revision tracking and ease of editing. Provide the URL to the raw .spec file, start the build, and if everything goes according to plan, you'll see that it succeeded! Check out Clight's project to see what this looks like.

Now, on your local machine, enable your newly created Copr repository and install the RPM package:

dnf copr enable jcrd/clight
dnf install Clight

Voilà! This package is now distributable: it can be installed on any system for which it was built by enabling its Copr repository. 👏

This pairs especially well with reproducible distro configurations!

But what happens when a reinstall is required, perhaps due to data corruption (?!), or simply desired? If you were thinking "I'll painstakingly recall the additional software I installed and custom configuration and manually set up the same state!" let's think again. We can encode the custom state of our post-installation OS to automate its recreation, thus satisfying our objective: reproducibility! 🤯

Reproducible OSes?

Some modern distros offer a mechanism to reproduce immutable installations, which addresses the software installation concerns. See:

• NixOS

• Fedora Silverblue

However, it may not address the custom configuration that often lives outside the jurisdiction of system-level package managers; think ~/.config and friends.

A solution explored below could be used in conjunction with the declarative, immutable architecture to set up fully reproducible systems, and this is a fine approach should you use such a novel OS.

But, I don't, as I use my software projects that are reinstalled as changed and I have yet to explore how this workflow could be easily adapted to an immutable system.

So, what might be some alternative approaches?

Ansible and friends?

How about existing "infrastructure automation" software such as Ansible or Puppet?

For a long time, I used Ansible and this worked well enough. But, as I didn't use it frequently outside of this use case, and my configuration remained relatively stable, I found myself often needing to reference the docs when I had to make changes; an unacceptably unwieldy process. Additionally, the forced directory structure and method of injecting variables became cumbersome. Essentially, Ansible was overly-complex for my use case, but it gets credit for its declarative configuration, which will make an appearance in my solution.

It is worth noting that while using Ansible I was installing an X11 window manager and associated configuration, significantly complicating the typical post-installation state. When I switched to GNOME Shell, much of the Ansible config could be eliminated, which only made the aforementioned shortcomings all the more obvious.

Check out my obsolete Ansible-based setup to see what this looked like in practice.

Custom scripts!

Why not write custom shell scripts? One could say: because then managing reproducible configuration becomes a software project all of its own. The antidote here is declarative configs, like Ansible's YAML-based architecture. However, one could retort: this simply moves the complexity from the user-facing configuration to the business logic of the software, which we will have to write if creating a custom solution! Fear not, in my case this code is as simple as:

dnf() {
    sudo dnf -q -y $@
}

enable_coprs() {
    echo '--- Enabling coprs ---'
    while read -r copr; do
        dnf copr enable "$copr"
    done < "$WORKDIR"/coprs.txt
}

install_packages() {
    echo '--- Installing packages ---'
    while read -r package; do
        dnf install "$package"
    done < "$WORKDIR"/packages.txt
}

The $WORKDIR variable allows these "library" functions to be applied to any directory in the project's root (facilitating modular configuration, like Ansible's roles), and the only structure enforced is the naming of the plain-text declaration files containing repositories to enable and software packages to install. Additional functions can be written as needed. No more documentation to reference!

A main configure script employs these library functions as such:

#!/bin/sh

# Set the chassis type (desktop, laptop) to include hardware-specific configuration.
export CHASSIS="$(hostnamectl | awk '$1 == "Chassis:" {print $2}')"
export WORKDIR=.

# Source the library functions.
source lib/dnf.sh

# This function clones and configures dotfile repositories.
configure() {
    [ ! -e "$2" ] && git clone https://github.com/jcrd/"$1" "$2"
    pushd "$2"
    ./configure
    popd
}

# Configure dotfiles.
configure zsh-config ~/.config/zsh

# Enable copr repos with a library function.
enable_coprs

# Install packages with another library function!
install_packages

# Custom state is easy to recreate: just include the shell commands
# you already know and love!
# Set user's shell to zsh. 😎
sudo usermod --shell /usr/bin/zsh "$USER"

# Chassis-specific configuration, like Ansible's roles.
export WORKDIR=chassis/"$CHASSIS"
if [ -e "$WORKDIR" ]; then
    ./"$WORKDIR"/configure
fi

See my live configuration for the whole thing in action, complete with more library functions and the above desktop role!

Now, after a fresh installation of Fedora, reproducing my configuration is as easy as:

git clone https://github.com/jcrd/fedora-workstation-config
cd fedora-workstation-config
./configure

Say goodbye to complexity and hello to easily reproducible Linux installations for maximum automation ⚙️ and peace of mind 🙏!

I found myself repeating the process of checking out the "production" branch, merging the development branch, and pushing—essentially "shipping" stable code—so I created a git alias. Let's check it out!

git aliases

git aliases allow you to specify alternative names for commands. For example, you can use an abbreviation for the checkout command:

git config --global alias.co checkout
git co

Another handy feature is the ability to create an alias to a separate executable; this is done by prefacing the command with ! on the command-line using git config:

git config --global alias.ship '!git-ship'

Remember to include the single quotes as ! is a special history expansion character in popular shells such as bash and zsh. Also, be sure this executable exists in your shell's PATH.

git-ship

My alias points to a separate executable shell script that chains together multiple git commands:

#!/bin/sh

prod=${GIT_SHIP_PROD:-master}
dev=${GIT_SHIP_DEV:-devel}

git checkout $prod && git merge $dev && git push && git checkout $dev

These commands perform the following actions:

1. check out the production branch;

2. merge the development branch into production;

3. push the production branch to the default remote;

4. and check out the development branch to prepare for new commits.

The targeted branches can be changed using environment variables.

In the directory of a given project, I "finalize" development commits and "ship" them to the remote repository with:

git ship

And it couldn't be easier!

The toolbx project was originally named toolbox and the command reflects this.

Typical usage might look like this:

[user@hostname ~]$ toolbox create
Created container: fedora-toolbox-36
Enter with: toolbox enter
[user@hostname ~]$ toolbox enter

While this may provide the foundation of a compelling development workflow, I felt something was missing. From my perspective, a development environment is specific to the project being developed.

So, you could use toolbox like this:

[user@hostname ~]$ cd my-project
[user@hostname my-project]$ toolbox create my-project-toolbox
[user@hostname my-project]$ toolbox enter my-project-toolbox

And proceed to install dev. dependencies in your container, but I want a toolbox that can be created and recreated on-demand, from a Dockerfile, for example. It then makes sense to commit this dev. environment manifest to your project so that your environment's specifications follow your project's development. Ultimately, what I sought was a means to more easily manage per-project toolboxes, which prompted the creation of toolboxcutter.

toolboxcutter

toolboxcutter (invoked as tb) is a script that manages the lifecycle of per-project development containers using Dockerfiles.

It began life as a zsh function, but as its feature set expanded, I realized it had become a project of its own, so I ported it to a standalone bash script.

It's designed to be easy to use. Within a directory containing a toolbox-based Dockerfile, simply run tb and your toolbox will be entered, creating the container if necessary.

Install it on Fedora with these commands:

dnf copr enable jcrd/toolboxcutter
dnf install toolboxcutter

A complete overview of its functionality is given in its command-line usage message:

usage: tb [command]

With no command, enter toolbox.

commands:
  init IMAGE      Initialize Dockerfile based on IMAGE
  create [IMAGE]  Create container (from IMAGE if provided)
  recreate        Remove and recreate container
  build           Build image
    options:
      -n NAME       Name of image
      -c            Build without cache
  rebuild         Remove container and rebuild image
    options:
      -n NAME       Name of image
      -c            Build without cache
  stop            Stop container
  rm              Remove container
  rmi             Remove image
  rpkg            Build rpm via rpkg
    options:
      -n NAME       rpkg spec template name
      -e EXT        rpkg spec template extension
  rpkg-install    Build and install rpm via rpkg
    options:
      -n NAME       rpkg spec template name
      -e EXT        rpkg spec template extension
      -r NAME       Name of produced rpm to install
  run COMMAND     Run COMMAND in toolbox
  version         Show version

As tb is largely a wrapper around toolbox, most of its commands translate directly to their toolbox equivalents. These are supplemented with image and container management commands that would otherwise require invoking podman.

Building RPM packages

Unique to toolboxcutter is its ability to build RPM packages from rpkg SPEC template files. This feature has been instrumental in the development, testing, and dogfooding of my own projects while using Fedora. I believe it elevates tb to a truly powerful developer tool for these reasons:

• Build dependencies need only exist in your project's container, minimizing pollution of your workstation's environment;

• Most importantly, the development-packaging cycle is combined into a single fluid process, providing interactivity benefits similar to that of a read-eval-print loop.

This functionality depends on the availability of rpkg in the toolbox container itself, so your project's Dockerfile should install it. This is the Dockerfile for the base image I use:

FROM registry.fedoraproject.org/fedora-toolbox:36

RUN dnf install -y git-subtree
RUN dnf install -y make
RUN dnf install -y neovim
RUN dnf install -y rpkg # Install rpkg
RUN dnf install -y rpmdevtools
RUN dnf install -y zsh

CMD /usr/bin/zsh

Any images or containers derived from this can be used to build RPM packages provided a SPEC template. By default, tb looks for this file at spec/*.rpkg.spec in your project's root directory.

Here is toolboxcutter's own SPEC template file:

Documentation about the format of these templates is sparse and much of my knowledge was gleaned from existing files. Nevertheless—perhaps with a bit of trial and error—it is possible to run tb rpkg-install in your project's directory to produce an installable RPM package with your latest changes!

D-Bus is an inter-process communication mechanism—a medium for local communication between processes running on the same host.

Essentially, clients connect to the system- or session-level bus where they can then communicate with associated services by calling their methods or listening to their signals, which looks something like:

This is the approach I used with sessiond and one I decided to implement in my monitor brightness controller, lighten—written in Python.

First, let's address why D-Bus is an appropriate choice for such systems:

• Using a standardized, full-featured protocol means we don't have to reinvent a client-server communication paradigm.

• Its ubiquity on the Linux desktop means access to a greater ecosystem of tools, language bindings, and learning resources.

Now, let's go into detail about the D-Bus architecture and explore the how of building a service in Python!

D-Bus architecture

There are typically at least two D-Bus buses running in a given Linux environment. One operates at the system-level, independent of logged-in users. This is where system services, such as systemd and systemd-logind interact. Additionally, each user gets their own bus, where session-level services communicate.

When a service connects to one of these buses, it registers using a well-known name. This is the address by which other services or clients on the bus refer to it. These names conventionally look like reversed domain names. In the case of lighten, this name is: com.github.jcrd.lighten.

Services expose communication endpoints called objects, identified by an object path like /com/github/jcrd/lighten. These objects implement interfaces, where methods and signals are defined. lighten has two such interfaces:

• com.github.jcrd.lighten.Backlight, with methods for interacting with the monitor's backlight;

• com.github.jcrd.lighten.Sensor, with a method to get sensor data.

Each of these interfaces is implemented by a single object, but this isn't always the case. For example, sessiond has an interface for audio sinks, with an object created for each existing audio device.

Interfaces define methods with signatures denoting the type of in and out parameters. These parameters can be thought of as arguments and return values, respectively. Supported types include:

• integers

• booleans

• strings

• arrays

• dictionaries

See this document for a complete list.

Methods are 1:1 modes of communication, returning data to the requesting client.

Interfaces also define signals with type signatures. Signals are 1:n modes of communication, publishing data to all subscribed clients.

This overview grants an understanding of the flow of data through the D-Bus architecture sufficient to create our own service. Let's go!

Python implementation

There are numerous Python libraries for building D-Bus services. Perhaps historically the most popular, dbus-python now describes itself as a legacy API and advises the use of alternatives.

I think the best alternative is GDBus, the D-Bus subsystem integrated into GLib. I used the C library in sessiond. It's usable in Python via PyGObject which also provides access to much of GLib itself.

The biggest hurdle was the lack of Python-specific documentation. Thankfully, I found this discussion, wherein user J Arun Mani (thanks!) provides working examples of both client and server code.

Follow this tutorial to install PyGObject and let's get to work!

To build out the D-Bus service in Python, first import the relevant libraries:

from gi.repository import Gio, GLib

Next, use inline XML to specify the service's interfaces:

xml = f"""
<node>
  <interface name='com.github.jcrd.lighten.Backlight'>
      <method name='SetBrightness'>
          <arg name='value' type='u' direction='in'/>
          <arg name='success' type='b' direction='out'/>
      </method>
      <method name='AddBrightness'>
          <arg name='value' type='i' direction='in'/>
          <arg name='success' type='b' direction='out'/>
      </method>
      <method name='RestoreBrightness'>
          <arg name='success' type='b' direction='out'/>
      </method>
      <method name='GetBrightness'>
          <arg name='value' type='i' direction='out'/>
      </method>
  </interface>
</node>
"""

This page provides more information about the D-Bus Introspection XML.

Every interface needs an accompanying interface handler function to dispatch Python code based on the called D-Bus method. The handler's function signature is:

def handler(self, conn, sender, path, iname, method, params, invo):

The useful arguments are:

• method: the name of the called method;

• params: the parameters to the called method;

• invo: the invocation object used to return values.

params must be converted to Python types and accessed as an array. Get the first D-Bus method parameter with:

params.unpack()[0]

Return values must be provided to invo as a GLib variant:

invo.return_value(GLib.Variant("(b)", (True,)))

The D-Bus signature, (b), is always a structure in this use case, so parentheses are required. Refer to the Signature Encoding table here to determine the appropriate type character.

To wire everything together, three different bus handler functions can be defined. These are called when:

1. the service connects to its bus;

2. the service is assigned its name;

3. the service loses its name.

There are some nuances in the operation of these handlers. The first two serve nearly the same purpose, but it could be the case that bus connection succeeds yet a service with the same name already exists, so the second handler might not be called. It is important to note that a service can replace one with the same name provided the proper initialization flag. It's in this case that the replaced service's third handler is called. Disconnection from the bus itself would also trigger it.

Discretion is advised in choosing which to fully implement. For sessiond, which must exhibit robust behavior, I implemented handlers 2 and 3, forgoing 1 entirely. However, for lighten, I only used the first!

The general approach is to register objects when the service is connected and available, and, if appropriate, tear them down should the service become unavailable.

I recommend encapsulating all the service logic in a Python class, like so:

class Service:
    def __init__(self):
        # Parse the XML interfaces:
        self.node = Gio.DBusNodeInfo.new_for_xml(xml)
        # Reference the GLib main loop:
        self.loop = GLib.MainLoop()

        # Connect to a bus:
        self.owner_id = Gio.bus_own_name(
            # Specify connection to the session bus:
            Gio.BusType.SESSION,
            # Set the well-known name:
            "com.github.jcrd.lighten",
            # Provide any flags
            # (for example, to allow replacement):
            Gio.BusNameOwnerFlags.NONE,
            # Provide handler 1 (defined below):
            self.on_bus_acquired,
            # Provide handler 2:
            None,
            # Provide handler 3:
            None,
        )

    def __del__(self):
        # Disconnect when the class is destroyed:
        Gio.bus_unown_name(self.owner_id)

    # Define handler 1:
    def on_bus_acquired(self, conn, name):
        # Register an object:
        conn.register_object(
            # Set the object path:
            "/com/github/jcrd/lighten",
            # Specify the interface via index
            # (as defined above in the XML):
            self.node.interfaces[0],
            # Provide the interface handler function:
            self.on_handle_backlight,
            None,
            None,
        )

  # Define the interface handler function
  # (abbreviated here with ...):
  def on_handle_backlight(self, conn, sender, path, iname, method, params, invo):
    if method == "SetBrightness":
        v = params.unpack()[0]
        r = ...(v)
        invo.return_value(GLib.Variant("(b)", (r,)))
    elif method == "AddBrightness":
        ...

Now, instantiate the class and run the D-Bus service with:

Service().loop.run()

Finally, use a tool such as d-feet to admire the fruits of your labor!

I was already changing my monitor's brightness via its DDC/CI interface using ddcutil on Fedora, so I figured this could be achieved. I set out to build a software monitor brightness controller in Python that uses data from an Arduino-powered light sensor to correlate ambient light and backlight brightness. Now, as the system regularly reads the sensor data, my monitor will automagically change its brightness to reflect the room's level of ambient light—I call it: lighten. 💡

Here I will share how to go about setting up the hardware and software for this project, how it works, and what I learned along the way.

Setup

The hardware

For any of this to work, a ddcutil-controllable monitor must be used.

The hardware side of this project is composed of an Arduino board, a light sensor, and a means of connecting them.

I chose these components:

• Adafruit QT Py ESP32-S2

  • This board has built-in support for the TinyUSB library

• Adafruit TSL2591 Light Sensor

  • This sensor is precise, allowing for exact lux calculations

• STEMMA QT Cable

  • This cable is super convenient!

Simply connect the QT Py board to the light sensor with the STEMMA QT cable and position them behind your monitor with masking tape!

Behold the beautiful hack:

Remember to keep the light sensor itself exposed:

Power and receive data from the Arduino with a USB-C to USB cable plugged into your computer.

The software

The Arduino code to read the light sensor data and the system-side controller are open-source.

This process requires a working Arduino IDE, basic knowledge of uploading sketches, and experience with the Linux command line.

Arduino

The Arduino code can be found here: arduino-lighten. It requires the following libraries:

• Adafruit TSL2591 library

• Adafruit Unified Sensor

• Adafruit TinyUSB (The board used should have built-in support for this library)

This tutorial explains how to set up the first two.

The TinyUSB library can be installed similarly:

In the Arduino IDE menus, go to Sketch -> Include Library -> Manage Libraries, then search for and install Adafruit TinyUSB.

Now clone the arduino-lighten repo:

git clone https://github.com/jcrd/arduino-lighten.git

Open the arduino-lighten.ino file in the Arduino IDE and upload it to the board.

Next, appropriate udev rules must be configured to access the HID device.

Create a .rules file in /etc/udev/rules.d that looks like:

SUBSYSTEM=="usb", ATTR{idVendor}=="239a", ATTR{idProduct}=="8111", MODE="0660", GROUP="plugdev", TAG+="uaccess", TAG+="udev-acl"
KERNEL=="hidraw*", ATTRS{idVendor}=="239a", ATTRS{idProduct}=="8111", MODE="0660", GROUP="plugdev", TAG+="uaccess", TAG+="udev-acl"

where the idVendor and idProduct attributes reflect the values found using lsusb:

 > lsusb
 Bus 005 Device 002: ID 239a:8111 Adafruit QT Py ESP32-S2

Here the vendor ID is 239a and the product ID is 8111.

See this section of the hidapi Python library documentation for more information.

lighten

lighten runs on your Linux computer and controls your monitor's brightness with ddcutil.

This tool requires read/write access to /dev/i2c video card devices. In order to use it without root permissions:

1. Add user to i2c group:

    sudo usermod <user-name> -aG i2c
   

2. Copy ddcutil's udev rule into place:

    sudo cp /usr/share/ddcutil/data/45-ddcutil-i2c.rules /etc/udev/rules.d
   

3. Reload and trigger the new rule:

    sudo udevadm control --reload
    sudo udevadm trigger
   

See this document for more information.

Now, let's set up lighten. It's currently available as an RPM package on Fedora, but it should be compatible with any Linux distro if installed from source!

1. Install with copr:

    dnf copr enable jcrd/lighten
    dnf install lighten
   

2. lighten requires the product and vendor ID of the Arduino HID device. These are the same values we found previously using lsusb, i.e. 239a and 8111. Create a new file at ~/.config/lighten/lightend.conf with this content:

    [sensor]
    vendor_id=239a
    product_id=8111
   

3. Enable the daemon's systemd service:

    systemctl --user enable --now lightend
   

   If the command above succeeds, everything should be operational!

How it works

lighten makes no assumptions about what monitor brightness value corresponds to an ambient light reading. The daemon, lightend, runs in the background and records manual changes to monitor brightness made with the client, lighten, until it's able to guess which values are appropriate.

lighten is used to adjust monitor brightness like this:

lighten set - 10 # decrease brightness by 10
lighten set + 20 # increase brightness by 20
lighten set = 100 # set brightness to max

Over time, lighten builds up a database of the ideal monitor brightness in relation to the ambient light level based on your adjustments, and restores brightness:

• when ambient light changes significantly

• on demand

• at startup

• upon wakeup from sleep

• at regular intervals as time passes

Update: If appropriate, lighten can set the monitor brightness to the value detected by the sensor, so that a sensor value of 100 corresponds to 100 monitor brightness. This is enabled by setting the normalize_mode configuration option to true.

What I learned

I learned how to implement and interface with an HID device using TinyUSB and python-hid after trying and failing to maintain a serial connection to the Arduino board upon the computer waking up from suspension.

I also learned how to use GLib via PyGObject to run a main loop with custom GSources alongside a DBus server all in Python. I found only scattered documentation about this, so I wrote about building a D-Bus service in Python here!

The code

This implementation utilizes three files: the main file, and two logger implementation files with build constraints.

The main file includes the interface definition and a global variable:

package main

type Logger interface {
    // format string, and objects to format
    Log(string, ...interface{})
}

var logger Logger

One of the logger implementations, the dummy, satisfies this interface with methods that do nothing:

//go:build !debug

package main

type DummyLogger struct {}

func (dl DummyLogger) log(format string, v ...interface{}) {}

func init() {
    logger = DummyLogger{}
}

The //go:build !debug build constraint means that it will only be included in the production build when the debug tag is absent.

The init function will be called at runtime, setting the global logger variable for use in the rest of the program.

The debug logger is implemented in the same way, using the //go:build debug build constraint:

//go:build debug

package main

import (
 "log"
)

type DebugLogger struct {}

func (dl *DebugLogger) log(format string, v ...interface{}) {
    log.Printf(format, v...)
}

func init() {
 logger = &DebugLogger{}
}

The build

Specifying tags at build time is simple:

go build -tags debug .

This can be integrated into a Makefile, for example:

main:
    go build .

debug:
    go build -tags debug .

.PHONY: debug

Now, logging does nothing unless the debug tag is specified!

• downloads/installs packages to user-specific or system-wide locations

• configurable dependency resolution

• builds locally define rocks (Lua modules packaged by LuaRocks)

• queries information about the active LuaRocks configuration

In these ways, LuaRocks is functionally similar to the package managers of other dynamic languages such as pip for Python and npm for JavaScript.

Where these features fall short is in managing the dependencies of multiple Lua projects—each with a potentially different version of Lua. Being that Lua is an embedded language, this is in fact common, and further exacerbated by the popularity of both the reference Lua implementation and luajit.

Problem

I encountered this issue during the development of an experimental window manager, dovetail. It is based on the awesome window manager framework, which can be built with Lua versions 5.1-5.3 or luajit. It also depends on a few Lua packages, and therein lies the problem. It quickly became an overwhelming task to unify the range of Lua versions and implementations, 3rd party packages, varying installation paths, and integration with an embedded interpreter.

Solution

After consideration of my options, I decided the easiest solution was to vendor all dependency packages. I set up Makefile rules to download the version of packages specified in a version-controlled file to a specific Lua rocks tree where the package’s files are extracted and moved into a lua_modules directory in the root of the project. During installation, these modules are copied to a system-wide shared data directory and referenced by awesome’s Lua interpreter.

When I required this functionality outside of the dovetail project, I re-implemented these Makefile rules as a shell script called luarocket. It is meant to be included directly in a project and utilized as part of its build process.

This is an excerpt from the time command's man page written in troff:

.TH TIME 1 2019-03-06 "" "Linux User's Manual"
.SH NAME
time \- time a simple command or give resource usage
.SH SYNOPSIS
.B time \c
.RI [ options ] " command " [ arguments... ]
.SH DESCRIPTION
The
.B time
command runs the specified program
.I command
with the given arguments.

From my perspective, an alternative is necessary. The markup syntax is cryptic and what would be inline markup in other formats requires a newline in troff.

There are numerous means of generating man pages from assorted markup formats. For example, using:

• reStructured text with rst2man,

• AsciiDoc format with AsciiDoc itself,

• Markdown with ronn or pandoc,

• or Pod with pod2man

to name a few.

Admittedly, I'd enjoy writing man pages in Markdown, but in approaching the question of an appropriate build pipeline, I decided early on that I'd like to avoid any external dependencies. This eliminates most options, but the pod2man command is available by default in the Linux distributions I've used. With this in mind, Perl's Pod, or Plain Old Documentation, is the top contender.

Here's an example of a Pod-formatted man page from my iniq project:

=head1 NAME

iniq - INI file reader

=head1 SYNOPSIS

B<iniq> [options] [FILE]

With no FILE, read standard input.

=head1 DESCRIPTION

iniq is a simple INI file reader for the command line. It queries an INI file
based on the path <I<section>><I<separator>><I<key>> and allows use of custom
separators in the file and formatting of the output. Sections inherit keys from
a special DEFAULT section unless the I<-D> flag is used. See below for examples.

=head1 OPTIONS

=over

=item B<-h>

Show help message.

Compared to troff, this is both easier to write and to read, which are important qualities in the maintenance of documentation.

Converting .pod files to man pages

Producing a man page from a .pod file is straightforward using the pod2man command. In most of my projects, this step is part of a Makefile.

In the case of iniq, these Makefile rules build and install its man page:

VERSIONCMD = git describe --dirty --tags --always 2> /dev/null
VERSION := $(shell $(VERSIONCMD) || cat VERSION)

PREFIX ?= /usr/local
MANPREFIX ?= $(PREFIX)/share/man
MANPAGE = iniq.1

$(MANPAGE): man/$(MANPAGE).pod
 pod2man -n=iniq -c=iniq -s=1 -r=$(VERSION) $< $(MANPAGE)

install: $(MANPAGE)
 mkdir -p $(DESTDIR)$(MANPREFIX)/man1
 cp -p $(MANPAGE) $(DESTDIR)$(MANPREFIX)/man1

The conversion command:

pod2man -n=$name -c=$header -s=$section -r=$footer $input $output

• takes the input .pod file as its first positional argument,

• and the output file name as its second;

• the -n flag sets the man page name,

• the -c flag sets the page header,

• the -s flag sets the section, which should match the output file's extension,

• the -r flag sets the page footer, which in my projects is always the version.

More information

Descriptions of pod2man's various options can be found here or in its man page using man pod2man; and with that we've come full circle!

Abstract debouncing

In the case of sessiond, only the time an input event occurred is relevant, so an abstract implementation of debouncing might look like this:

debounce_interval = 1

# Called when time given to `schedule_event_processing` is reached.
def process_event():
    if current_time() - last_event_time >= debounce_interval:
        process()

while true:
    if event:
        last_event_time = current_time()
        schedule_event_processing(process_event,
          last_event_time + debounce_interval)

When an event occurs, the current time is recorded as last_event_time. There should be no response to the event until at least debounce_interval has elapsed, so event processing is scheduled for a time in the future equal to the current time plus the debounce interval. If another event occurs before this time has passed, an additional response is scheduled, and last_event_time is updated.

In the processing function, it is important to compare the time that has elapsed since the last event to debounce_interval to ensure additional events were not generated after this response was scheduled. This way, event processing happens only when debounce_interval time has passed since the last event was received—the essence of debouncing.

Custom GSource implementation

This method of debouncing can be implemented with a custom GSource's check and dispatch functions using g_source_set_ready_time.

First, define a custom GSource by declaring a struct containing a GSource:

typedef struct {
    GSource source;
    gpointer fd;
    gint64 last_event_time;
} InputSource;

Next, implement the check function, which determines if the source is ready to be dispatched:

#define DEBOUNCE_US (1 * 1000000)

gboolean
inputsource_check(GSource *source)
{
    InputSource *self = (InputSource *)source;
    GIOCondition revents = g_source_query_unix_fd(source, self->fd);

    if (!(revents & G_IO_IN))
        return FALSE;

    // Process events...

    self->last_event_time = g_get_monotonic_time();
    g_source_set_ready_time(source, self->last_event_time + DEBOUNCE_US);
}

In this function, a file descriptor is queried to determine if any events are pending. If there are no events to be handled, the function returns FALSE to signify there is no need to call the dispatch function. Otherwise, pending events should be processed accordingly. Finally, the current monotonic time—being that of the last event(s)—is recorded and the source is instructed to dispatch in DEBOUNCE_US microseconds with g_source_set_ready_time.

Now, implement the dispatch function, which is responsible for calling the callback function provided to this GSource at creation time:

gboolean
inputsource_dispatch(GSource *source, GSourceFunc func, gpointer user_data)
{
    XSource *self = (XSource *)source;

    if (g_get_monotonic_time() - self->last_event_time >= DEBOUNCE_US) {
        g_source_set_ready_time(source, -1);
        return func(user_data);
    }

    return G_SOURCE_CONTINUE;
}

The logic here is as follows: this function is called at the monotonic time given to g_source_set_ready_time in the check function above, so we know at least DEBOUNCE_US time has passed since the handling of those events, but additional events may have been received in the meantime, reflected by an updated self->last_event_time. If at least DEBOUNCE_US time has elapsed since the last event, we call the user-provided callback function, and g_source_set_ready_time(source, -1) is used to stop future dispatching of this source until the check function detects pending events. This is necessary because g_source_set_ready_time will cause the source to be continuously dispatched if the time it was last given is in the past, which will inevitably be the case.

Finally, create the GSourceFuncs struct and initialize the custom GSource:

GSourceFuncs inputsource_funcs = {
    NULL, // prepare function
    inputsource_check,
    inputsource_dispatch,
    NULL, // finalize function
    NULL,
    NULL,
};

GSource *source = g_source_new(&inputsource_funcs, sizeof(InputSource));
InputSource *self = (InputSource *)source;

self->last_event_time = 0;

// Add the file descriptor.
// self->fd = g_source_add_unix_fd(source, ..., G_IO_IN);

// Set callback.
// g_source_set_callback(source, ...);

// Attach source to context.
// g_source_attach(source, ...);

More information

This tutorial describes in greater detail the check, dispatch, and other functions that control the behavior of a custom GSource.

For a complete working example, refer to sessiond's debouncing implementation in xsource.c.

The most noticeable omission was proper session management, which facilitates, for instance: locking the screen or suspending the system after a period of inactivity. This was not a dealbreaker on a desktop computer, but using a laptop without these features felt like a dysfunctional hack.

It would be quite a few more years of exploring the universe of tiling window managers before deciding to address this issue myself, and with the broad adoption of systemd by the larger Linux ecosystem, it was easier than ever. Enter: sessiond, a standalone session manager for Linux.

What does it do?

sessiond is a daemon for systemd-based Linux systems that interfaces with systemd-logind to provide the missing session management features to X11 window managers. Its primary responsibility is to monitor keyboard and mouse activity to determine when a session has become idle and to then act accordingly. It is capable of:

• locking the screen when idle and before suspending the system

• dimming the screen's backlight when idle

• triggering systemd targets for use by the window manager or end user

• optionally managing DPMS settings

• controlling keyboard and monitor backlight brightness

• controlling audio sink volume and mute state

The audio sink interface is new in version 0.6.0.

It also provides a D-Bus service so that it may be integrated with modern window managers. For example, a window manager can prevent idling when a media player is open by interacting with the D-Bus methods.

It is designed to be zero-configuration, providing sensible defaults, but allows configuration if needed. See here for additional details.

How do I use it?

sessiond requires a Linux system utilizing systemd-logind. It may be possible to use elogind but this has not been tested.

This brief tutorial assumes basic knowledge of systemd-based Linux systems and the command-line.

Installation

Currently, sessiond RPM packages are built for Fedora via copr and installable with the following commands:

dnf copr enable jcrd/sessiond
dnf install sessiond

If you're using Arch Linux, sessiond is available via the AUR (thanks to Denis Oster)!

Use your preferred AUR helper or install it with:

git clone https://aur.archlinux.org/sessiond.git
cd sessiond
makepkg -si

I would like to see packages for other major distros in the future. Until sessiond achieves world domination, it is recommended to build from source by following these instructions.

Setting up your window manager

The intended way to use sessiond with your window manager of choice is to create a custom systemd service in the ~/.config/systemd/user directory. For example, below is a awesome.service file that runs the Awesome window manager:

[Unit]
Description=Awesome window manager
Requires=sessiond-session.target
After=sessiond.service
PartOf=graphical-session.target

[Service]
ExecStart=/usr/bin/awesome

[Install]
Alias=window-manager.service

The options in the [Unit] section ensures your window manager is only running alongside the sessiond daemon. The Alias= option in the [Install] section lets sessiond know this service is the window manager so the session will be stopped when it exits.

Next, enable the window manager service with systemctl --user enable awesome.service.

Now, select the sessiond session entry via your display manager or set it as the default in its configuration file. For example, if using lightdm, set user-session=sessiond in /etc/lightdm/lightdm.conf.

Locking the session

sessiond wouldn't be of much use without a means of locking the screen. Create a service for your screen locker of choice in ~/.config/systemd/user. For example, here is a i3lock.service that runs i3lock as the screen locker:

[Unit]
Description=Lock X session with i3lock
PartOf=graphical-session.target

[Service]
ExecStart=/usr/bin/i3lock
ExecStopPost=/usr/bin/sessionctl unlock

[Install]
WantedBy=graphical-lock.target

The sessionctl unlock command in ExecStopPost under the [Service] section notifies sessiond that the session has been unlocked when the service exits. Enable with systemctl --user enable i3lock so it's started upon triggering the graphical-lock target, which by default occurs when the session becomes inactive and before suspending the system.

Other services

Additional services (e.g. a compositor) can be started with the session by creating systemd service files in ~/.config/systemd/user containing:

[Unit]
PartOf=graphical-session.target

and:

[Install]
WantedBy=graphical-session.target

Enable such services with systemctl --user enable <service>.

Inhibiting idling

It's often desirable to prevent the session from idling while, for instance, watching a video. sessiond-inhibit comes to the rescue:

sessiond-inhibit -y 'movie night' mpv ...

This can be automated if your window manager supports client rules and some level of scripting.

More information

For more information about sessiond, visit its website.

You should now be equipped with sufficient knowledge to go forth and manage your sessions!