recycled math

"Containerize" individual functions in Rust with extrasafe

2024-04-04T00:00:00-04:00

Extrasafe now has namespace support

Today I released a new version of extrasafe which has support for using Linux's unprivileged namespaces feature to create browser-style content processes. Namespaces are perhaps more famously used in container runtimes, which is why I used them in my clickbait title, but the main inspiration and use-case for extrasafe's Isolate is closer to a browser's content process.

Extrasafe is a Linux security toolkit for Rust that makes it simple to use kernel security features directly inside your Rust code. Extrasafe already has support for seccomp and Landlock, which are also used in browsers, containers, and other kinds of isolation tools like firejail and bubblewrap.

The Isolate feature I'm releasing today lets you run individual functions inside unprivileged user namespaces.

Overview

Fundamentally extrasafe's namespace support works mostly the same as a typical container runtime:

At some point in your code, call Isolate::run
Create a memfd and copy /proc/self/exe into it (this protects the original executable from being modified)
Use a standard std::process::Command to exec /proc/self/fd/<memfd> with an argv[0] that indicates to extrasafe we're a content process
Back in main, hit the Isolate::main_hook call
Allocate a new stack
Clone with CLONE_NEWUSER, CLONE_NEWNS, etc flags to enter a new namespace (often fork + the unshare syscall are used instead)
- the CLONE_PIDFD flag is also used
Map the current user into the namespace as root
Mount a new tmpfs and bindmount anything the application needs into it
Call the pivot_root to switch to the filesystem set up previously
Unmount the old root mount
Clear all fds except for stdout and stderr
Run the user-provided function and exit when it's done
In the process spawned by Command, wait on the pidfd provided by clone.
In the original process, wait on the new process with the Command::output method to gather any output

We exec (see discussion below for why we have to exec at all) prior to clone/pivot_root because otherwise the linker may not be able to find the necessary .so files to link to if the program is dynamically linked, even if we kept a fd with the contents of /proc/self/exe around. This sequence of operations requires that we have something like Isolate::main_hook to do the setup, and adds an extra point of failure in the sense that a user might not call it. It's not really an issue because in that case the program wouldn't make it to the code to be isolated anyway, but it makes the overall experience more complicated.

Also, one random fact I discovered while working on this feacture that doesn't seem to be documented anywhere besides a random git commit message in systemd: In order to mount a proc or sysfs filesystem inside a user namespace, you must already have a proc (or sysfs) filesystem mounted in your tree somewhere.

Use-cases

At my previous job we had a separate internal service for converting and transforming image data because of issues with CVEs in imagemagick and similar image-parsing libraries. While it may make sense from a deployment and scaling perspective to do so regardless (because doing image ops is more computationally intensive than serving web requests, for example), using something like the equivalent of a browser's "content process" to isolate CVE-prone libraries could be an operational and performance (esp. latency) win in some cases.

Other similar tasks you might want to use an extrasafe Isolate for are things like: running ffmpeg, calling into closed-source binary libraries (another thing from my last job), and possibly GPU stuff - in particular I'd like to write a gpu example based on chromium/firefox's content processes.

Design choices

Like the rest of extrasafe, the design choices were made to make the implementation as simple as possible and the features simple to use and hard to misuse, not to provide all features of the underlying security tools.

Clone vs fork

Using libc's clone wrapper rather than fork+unshare means that you get to start with a completely clean stack. Using fork makes it easier to pass data from prior to clone to the new process because you have all the existing stack variables, but it's not that hard to shove everything into a Box (which puts the data on the heap, which is copied to the new process), and then use libc's clone wrapper data pointer to access it again after the clone.

When to exec

The biggest design choice that I wasn't sure about was to exec "first" (before entering the user namespace) or exec "last" (after doing all the setup but before pivot_mount), and the related issue of closing stdout/stderr or not. In the end I decided that re-using the std lib's Command made the implementation a lot simpler, and I don't think the tradeoffs are that bad. The biggest issue is in the setup code: execing first means the Isolate's config/setup data has to exist without the context of the code when Isolate::run is called, whereas execing "last" (i.e. right before we pivot_root) means that we can pass the Isolate's configuration data in memory from the point of starting the isolate all the way to after we're in the namespace.

With the current design, it forces you to come up with all of the configuration data at the start of the program, without any context of the rest of the program. This is nice because you have less to reason about in terms of state at startup inside the isolate, but can make usage more confusing. For example, if you have a config file that gets passed in the command line which contains a directory you want to bindmount into an isolate, you must first parse the CLI args, read the config file, get the directory, run the isolate, passing the directory as an environment variable to be used at isolate setup.

However, the isolate code that uses that environment variable will go before the above CLI/config parsing code in main (because the associated cli args aren't there in the isolate process). However, since the environment variable doesn't exist in the original process, the setup code must be gated on whether we're inside the Isolate already or not. That is why the second argument to Isolate::main_hook is a function: it only gets called and does setup if main_hook first detects we're in an Isolate.

This decision is one I would reconsider if someone came up with a use-case that the current design doesn't work well with.

Relatedly, there's no way to communicate between the parent and worker by default, which is also something I went back and forth with. In the end it's easy enough to set up a unix socket for communication, and pipes can be complicated (they can clog!). There's an example of using a unix socket for communication in examples/isolate_tests.rs.

Filesystem consideration

By default the isolate creates a new tmpfs on top of a temporary directory created by mkdtemp, but I also considered just having the user provide a directory. I thought people might just give / though which kind of defeats the purpose - if you want a full OS container, just use docker or podman or whatever.

Not execing at all?

This isn't really a design choice since I don't think there's an alternative but it would really be nice if we didn't have to exec and do the main hook thing, but execing:

Lets us protect the original /proc/self/exe on disk via the memfd trick
"Cleans" the memory space so no data in memory from prior to entering the isolate is available to the worker
env and argv can be sanitized (env from program start can be accessed at /proc/self/environ even if you modify it in-process)
Lets us take advantage of std Command's code that replaces stdout/stderr with pipes and reads them to strings rather than have to write it ourselves.

so I don't think there's a way around it.

Network

I considered not even including the option to not create a new network namespace because in order to do HTTP, you most likely need to bindmount at the very least /etc/resolv.conf for DNS, and depending on your openssl situation /etc/ssl and similar directories (I spent a couple hours unsure why statx was failing on /etc/ssl/cert.pem when it was bindmounted via /etc/ssl/ but not when I mounted the whole /etc/ until I realized it was a symlink to a file in /etc/ca-certificates/). Using reqwest with the rusttls-tls feature makes it simpler because it bundles the certificates, but there can still be issues with DNS depending on how it's configured.

Ultimately the reason I kept the option is because even with the above DNS/SSL issues, it could be useful to isolate something and have it speak with a daemon that listens on a local IP port - not everything speaks unix sockets.

Testing

Due to the way cargo's test runners are set up, I couldn't figure out a way to get Isolates to work with cargo test. So examples/isolate_test.rs just manually asserts and doesn't have a pretty output format, and is run separately in CI from cargo test.

Isolates also make coverage collection more difficult/impossible, but code coverage was already hard for extrasafe. Currently I'm using llvm-cov but it might be worth trying out tarpaulin again, which is ptrace-based.

Future work

Isolates

A proc wrapper macro for main instead of Isolate::main_hook would be nice.

Also, instead of using strings to link Isolate startup in run and usage in main_hook, it would be nice to use some kind of unique token manufacturing technique with some type system trickery. Initially I prototyped a hack where you could pass either a string or a unit struct using a std hasher and the hashes of the object itself and its type id, but I wanted to just release the feature already so I stayed with String ids for the time being, since you need the string for argv[0] regardless.

I'd also like to add a mount_proc option that bindmounts the parent proc inside the isolate, mounts a new proc, and then unmounts the original proc. This is already possible so I didn't think I needed to include it in the original release, but it would be nice as an option. There's a demo of this in examples/isolate_test.rs - as mentioned previously, you can't mount a proc fs without having one already mounted, so there's a bit more setup than just calling mount.

I don't know how hard it would be to add support for bridging just the loopback interface so that programs that don't want to have external network access but would prefer to communicate with the Isolate via TCP or UDP, but it would be nice to have. It feels like it would be a lot of work.

Issues

For some reason strace doesn't like the tests in examples/isolate_test.rs that panic in the isolates. They complete normally when run outside of strace, but if you run strace on the test binary, the isolate process segfaults in some kind of loop after recieving a SIGABRT via tgkill, segfaulting, and then resetting(?) the segfault signal handler.

Other security features

The last remaining major security feature to add to extrasafe is capabilities. After entering the isolate, by default the user has the CAP_SYS_ADMIN capability inside the new namespace (it has all capabilities) which can be useful, but we'd also like to be able to drop it if we don't need it.

cgroups support is also a possibility but for the current use-case I'm not sure it wouldn't be better to configure it external to the program (e.g. via a systemd unit file). cgroups does allow you to, for example, set limits on processes which might help prevent DOS attacks, but you can also just use seccomp to stop the program from forking at all once you're in the isolate.

It would also be nice to document how to use bindmounts etc to access the gpu, as mentioned above, without just bindmounting the entire sys and proc directories from the host. I'm sure there's docker images from nvidia or someone that would be useful to take a peek at.

Thanks for reading

If you've read this far, I am looking for a new job. If you'd like to talk, please reach out to me via email at my first name @ this domain

Yet another E-Ink weather display - but with Rust!

2023-09-24T00:00:00-04:00

I made one of those e-ink weather displays that you see on tech blogs and hacker news sometimes. It can display the current weather, the temperature and precipitation forecast for the rest of the week, and my current and upcoming tasks from Todoist. It's powered by a couple NiMH AA batteries and lasts at least a couple months on a single charge, if not more.

The code can be found on my github.

Hardware

E-ink display

The screen is a 7.5", 800 by 480 pixel, 3-color e-ink display that I purchased from waveshare. It does black, white, and red, which is slightly unusual for e-ink and increases the price and lowers the refresh rate, but looks really nice. I believe it works (skipping over how e-ink works in general) by first pushing the black particles to the front, and then pushing the red particles, which are maybe smaller or lighter or less dense, further in front of the black ones.

Microcontroller

I bought a pre-assembled ESP32 board, also from waveshare, which contains the FPC connector already assembled and connected to the ESP32 module's SPI pins.

I desoldered the on-board LEDs to increase battery life, since the power LED was always on, and the other led seemed to be slightly receiving/drawing power during sleep. I was able to turn it off in software by configuring an internal pulldown to stay on during sleep but I wasn't sure if that would also draw some current so I just removed it since I wasn't using it.

Software

This project consists of two different Rust projects.

There's the code that runs on the esp32 board connected to the display, and then there's also code running on a server that both gathers the data and renders it to a file. Originally I was going to do everything on-device, but since the display is battery-powered I thought it would be more efficient to download a single file and display it. So all the API requests, graph drawing, and text rendering operations happen on the server, and the display board effectively acts as a dumb terminal which just displays the data it receives.

On-device software

ESP32 overview

For the "firmware" running on the esp32 board, I'm using esp-idf with the standard library (i.e. not no-std) via the esp-rs project.

Setup is a tiny bit complicated, but the esp-rs book is a great guide and explains everything fairly clearly. Hopefully one day it will be as simple as "edit your runner in .cargo/config.toml to use espflash and then just cargo run" although I think for RISC-V boards with no-std it might be pretty close already. I added a Justfile that exports the libclang and esp toolchain environment variables internally so that I can just build and just run.

The esp32-std embedded ecosystem is comprised of several different crates, including:

esp-idf-sys, which contains bindgen bindings to the esp-idf C API
esp-idf-hal, which contains higher-level, type-safe wrappers and drivers for hardware like GPIO and SPI
esp-idf-svc, which contains implementations and wrappers for system services like Wifi and storage

Additionally, the above crates use and implement traits from the following embedded-rust ecosystem crates:

and several other setup utility binaries like espup, espflash, and embuild, in addition to a fork of the rust compiler for the Xtensa/ESP32 architecture (there are also RISC-V ESP32 processors which don't require the forked compiler and llvm).

I mostly read example code, the various projects' mentioned above documentation, and the esp-idf documentation to figure out how to put everything together. In particular this repo is fairly extensive: https://github.com/ivmarkov/rust-esp32-std-demo

It can be somewhat confusing to figure out which types you need from which crates to e.g. turn on wifi, but this isn't unique to esp32 or embedded rust. Libraries will take traits from embedded-hal/svc as impl parameters and it can sometimes be difficult to figure out how to instantiate concrete versions of those types. Typically the sample code is useful in those cases to get things started.

Overall, the code is pretty simple.

We "gather" the peripherals we need - spi, the modem for wifi, and the gpio pin used to wake up via button press
Turn on the wifi
Get the display data from the server
Turn off the wifi
Send the data to the display and wait until we expect it's done
Tell the e-ink display to go to sleep
Tell the device to sleep for 90 minutes or until the button is pressed

The URL to request the data from and the wifi SSID and password are just constants in a src/config.rs file which isn't checked in to git. This gets baked into the final binary.

Getting the data

To request the display data from the server, we use the esp_idf_svc::http and embedded_svc::http modules.

There really isn't too much to the code - it's a pretty standard http request. The only thing of note is that we explicitly check that we're getting the right size file back from the server to fit the display.

Waveshare e-ink driver

To send the image data to the display, we use the epd-waveshare crate, but because the last release was published to crates.io 2 years ago, we have to use the git repo url directly in our Cargo.toml file. My display uses the "epd7in5b_v2" driver. The sticker on the back says v3 but everything seems to work.

Other than that everything just works - you set up the SPI device, pass it to the library and give it the image data. It doesn't seem to wait for the display to actually finish updating, so we wait for about 20 seconds before sending a sleep signal to the display, which lowers its power consumption. Since it's an e-ink display, the image continues to be displayed even during sleep, of course.

Aside: The memory layout for the display data is just the raw bits bitpacked into bytes. The red pixels are handled by simply having two separate buffers packed next to each other. This is inefficient in terms of space - we could instead do a variable length encoding scheme where say 0 is white, 10 is black and 11 is red. However, this would make it significantly slower to do standard drawing operations because individual pixel access becomes O(n).

Rendering software

The rendering code was by far the most interesting. Everything is drawn with the imageproc crate, which uses the image crate and the rusttype crate for fonts. Additionally, it uses the same epd_waveshare crate and also the embedded_graphics crate to pack the image buffer into the format used by the display driver.

So again, the strategy here is to draw the image into a generic image buffer, and then "render" it into the driver display library's buffer type as if the code were running directly on the device. Then as above the esp32 downloads the file and sends it directly to the display without having to do any extra work.

Gathering the data

I'm using weather.gov's free API for the weather data, which requires just a couple HTTP requests to get the current weather and the forecast. To figure out the station and gridpoint parameters for the requests, you'll need to make a couple requests manually to some other endpoints that take GPS coordinates and return the nearest stations and gridpoint.

The todo list data is just a single API request with a filter of -24h to +48h on the due date.

Drawing and measuring text

While imageproc has a built-in function for drawing text, I had to write several variations of wrappers to get different alignments. The measure_text function I'm pretty sure was taken from example code somewhere in rusttype or imageproc.

Drawing the graphs

Imageproc's line drawing methods are just basic Bresenham and have no thickness parameter, so to get thick lines I just drew them with the start and end points offset vertically by a pixel. Note that this obviously does not work for perfectly straight lines and mathematically you'd want to offset by the vector perpendicular to the original line, but since the graph is mostly horizontal it works fine for this case.

Regarding the precipitation graph, it was a fair amount of trial and error to get it looking the way I imagined it. The code is a little obscure and I'm pretty sure it can be simplified. The idea is that we just split the graph into 6x6 squares, turn on some pixels on the diagonal, and then offset the squares themselves. The last part is the part that I think could be simplified (or at least better explained) and is expressed in the xm variable in the code.

I really love how the thin lines in the precipitation graph really make it look blue even though it's the same black as the rest of the image.

Dithering and "rendering" to the display buffer

Since the display can only handle 3 colors but the text is rendered with anti-aliasing (and I couldn't find an easy way to not anti-alias)(EDIT: I did end up finding a way to not anti-alias by rendering the fonts myself, basically, so this section is not correct anymore. See the code here) I ended up using the image crate's dithering functionality with just a manually-tuned cutoff point that looked good with the font and the display. Fortunately e-ink displays are a bit fuzzy due to their construction, so combined with the curvy font I chose it's hard to see any artifacts without looking very closely.

After the final image is dithered to white/black/red, we use the same exact crate that's used in the firmware, but this time to write the buffer instead of read it. All we have to do is enumerate over the pixels in the image and call set_pixel inside the buffer, and the crate takes care of doing the bitpacking for us. It could be done more efficiently, but since we're not running on the device it doesn't matter as much.

Running the rendering software

Unlike the firmware, the rendering software takes an actual json-formatted config file as a cli parameter, along with the location of the output file.

The rendering binary gets run every hour via a systemd user service and timer file which can be found in the scripts/ directory. In my case, the output file gets put into a directory which nginx is configured to serve static files from.

Extrasafe

I used my own extrasafe crate in the rendering software on the server. It allows you to restrict your software's syscall usage to a subset of your choosing via seccomp. We start one thread to make the HTTP requests to all the APIs, then pass that raw JSON data to another thread with even less privileges to do the parsing, and finally pass that back to the main thread, where we then do our final restriction that only allows us to write to the output file. I think ideally we would first spawn the threads, wait until the original thread is restricted, and then run the other threads, but with more than one worker thread in the sequence it becomes difficult to organize everything manually.

I'm currently working on improvements to extrasafe to allow the use of Landlock and also maybe a helper function for the unshare syscall. In particular I'd like to restrict the network thread's filesystem access to only the necessary files for DNS and SSL, rather than all files.

The Case

I learned how to use CAD software for this project and it was simpler than I thought it would be.

First, you select a plane to draw on, which can be either a plane along two major axes, or a plane formed by part of your model that you've drawn alreday. Then you can draw 2D shapes on that plane with standard 2D curve tools like lines, bezier curves, conics, etc. Once you've got the parts of your shape in place, you then need to constrain the shape. Here "constrain" means to lock in the position, dimension, and other parameters such as angles, radii, control point position, which you do with operations like "these two points are concurrent", "these two points are symmetric about this line", "these two lines are perpendicular", "the length of this line is 5mm", "the angle between these two lines is 30 degrees". Finally, back in "3D mode", you can use tools to extrude, cut out pockets, revolve, or otherwise 3d-ify the 2d sketches you drew.

Additionally, you can make a spreadsheet or mapping of labeled dimensions and use them in the 2d sketches or extrusions in place of using specific dimensions like "5mm" and the model will automatically be updated when you change the values. Overall it's actually pretty fun, although maybe a bit tedious.

I tried out both freecad and onshape, and freecad seemed a bit too easy to get into a buggy state but I otherwise liked the constraint interface. In particular one tiny thing I liked about freecad was that the sketch turned a very visible green when it's fully constrained. Onshape's blue/black scheme is pretty low contrast and can be hard to see, especially when using flux or redshift. Maybe I just missed an external indicator that said whether the sketch was fully constrained or not. The built-in variables table in onshape was simpler than freecad, where you have to use spreadsheet workbench or a separate plugin, and even then I'm pretty sure you have to type spreadsheet.<name> or dd.dd<name> whenever you want to use a variable inside a sketch. I also didn't see a way in onshape to see a list of constraints like you can see in freecad.

I had the case printed by JLCPCB in white resin via SLA for about $20 - $30 USD shipped and it was both really easy and it came out pretty clean. I had to do a second revision because I messed up with the bezel size (accidentally made it symmetric rather than the bottom having a larger lip) and the second order came out just as good or better than the first. Here's the oncad project.

I couldn't find any cheap 3D printing services in the US to do the print for me - I guess they either get outcompeted on labor costs by China or there isn't demand for them on a hobbyist level because hobbyists can buy their own or use a makerspace's printer?

The case is mounted on the wall with regular Command picture hanging strips - I designed pads on the backside of the case specifically for this purpose.

Things I would like to improve

Case snap-fit for pcb, maybe for display as well/or just supports
- I'm not really sure what's possible for resin in terms of both printability and flexibility / thin parts
Custom PCB with a lower quiescient current voltage regulator for improved battery life? Plus on-board battery holder.
The weather.gov API is occasionally flakey or provides bad JSON, maybe add a retry option via the systemd unit file
Add some code to note if the image from the server hasn't been updated (either via comparing to an RTC or storing the last successful update time in flash) and then maybe draw a red dot or line in the top corner or somewhere. This would be equivalent to just setting one or a couple bytes to all 1s in the binary, so it wouldn't be very difficult.
Minor issues with graph data
Make daily high/lows red and smaller
Show daily high/lows for current day
Measure current draw accurately
Current ranger / ucurrent gold / borrow one from somewhere?
Better test the renderer
Set up some kind of testing with qemu for the firmware

Conclusions

It's really easy and fun to get started writing code for microcontrollers in Rust! CAD is also pretty fun.

Thanks to Neil Chen and Stan Zhang for reviewing a draft of this post

Update 2023-10-13: Thanks to Hacker News commentors doodlebugging and goosinmouse for pointing out some issues with the temperature scale and min/max values that I believe I've fixed.

How to pair program like it's the 1980s (with GNU Screen)

2021-07-26T00:00:00-04:00

What we're doing

Suppose you want to pair program with someone remotely. You could use one of several online in-browser services or IDE plugins, but if your environment is special or you need access to some resources behind a VPN, these solutions may not work for you.

Screen is a tool that allows you to run multiple terminals in one session. It's particularly useful on servers, where you can leave your session running after disconnecting in the same way that when you put your laptop to sleep, all your windows are still there when you open it back up. Tmux is a similar (but released 20 years later and so makes a less interesting title) program that's probably more popular, and also supports multiuser access but with less sophisticated access controls.

We're going to use screen's multiuser feature to share the same screen session across two ssh sessions, letting two people interact remotely with the same shell at the same time. You could pair program by having each person type alternating letters if you really wanted.

Why

Why not? Maybe you don't like screensharing because your Spotify playlist is full of Taylor Swift, or your OnlyFans notifications will give away your secret NASCAR hobby. Maybe you're actually swapping who's typing frequently. Maybe you can't program without your One True Vim Configuration.

Prerequisites setup

To make this work you need to have access to a shared computer that you can ssh into. You can set this up with a single user (requires less configuration but not always ideal or possible, especially in corporate environments), two existing users if you're on a corporate or educational network and you already have shared access to several machines, or you can make an extra "guest" user and generate ssh keys per-guest, giving you the ability to revoke access easily.

In the first case (one user), you need your friend to generate a new ssh key via something like ssh-keygen -t ecdsa -C "pair programming key $(date +%F)", have them send you the public key, and then add it to your authorized_keys file. Then all they need to do is ssh to your machine (on the same user as you) and use screen -x to attach to your screen session, without doing the extra setup for multiuser below.

With a second guest user, you need to set up that user on your machine, have your friend generate the ssh key as above, and then add the key to the guest user's authorized_keys file.

Additionally, there is information on the internet about SELinux needing to be enabled but that doesn't seem to be the case on my desktop or server that I tested on. The screen binary also must be suid root but that was already the case on my server and desktop as well. I did encounter a non-suid-root screen binary on a work computer, but all you need to do to resolve that is sudo chmod u+s `which screen` && sudo chmod 755 /var/run/screen.

Screen basics

The most basic usage of screen is just to type screen, which creates a new session. You can then use C-a c to create a new terminal, and C-a a to switch back to the previous one. You can use C-a n to go to the next in order and C-a p to go to the previous.

C-a d detaches the current screen, leaving all your terminals running and ready to be re-opened later. Note that if you put your laptop to sleep or shut your computer down, they will of course also pause or be terminated - it's not magic.

To resume a session, from a terminal we can run screen -r.

I typically run screen -Udr to resume a session, where the -U enables Unicode, and using -dr instead of just -r will detach other currently attached terminals. This is useful if you were attached via a laptop and want to attach from a different computer, or simply if you're lazy and don't want to find whatever other terminal is attached.

Execution

There are a few parts:

We need to give our screen session a name so it's easy to join
We need to enable multiuser mode so other users can join
We need to give the specific user we want access to the session
The other user needs to attach to the shared session

Incidentally, the third point is why it may be easier to make a guest account for pair programming on your machine, because then you can give that guest account access permanently as we'll see later.

1. Naming a session

When starting a screen session, we can name it with the -S <sessionname> argument. Combined with the unicode flag above, it would look like screen -US shared, for example.

2. Enable multiuser mode

To enable multiuser mode, type C-a :multiuser on. Once you hit : after C-a, it will appear in the bottom left where the status messages appear, and as you type the rest it will show up there.

3. Give the other user access to the session

To give the other user access to our session, type C-a :acladd <username> where <username> is the name of the user's login on the machine, e.g. guest or harrystern.

4. Attaching to the shared session

Once your friend has sshed to your server, all they need to do to join your session is use the -x <user>/<sessionname> flag instead of -r. So as before with the -U flag, the full command would look something like screen -Ux harrystern/shared.

They should now see whatever you see on your terminal, and vice-versa. Anything you or they type in the same terminal will be shared, but you can open separate terminals with C-a c and view and use them separately. That is, the terminals themselves are shared but displaying them are not - you can view and type in one while your friend works in another, and switch between them freely. You just need to be careful about not accidentally typing when you're in the same terminal as someone else when they're working.

Using `.screenrc` for streamlined shared access

In order to make this process easier, you can put some commands in a .screenrc file so that you don't have to do the permissions setup each time. Depending on your security requirements and level of trust, this may or may not be viable in your personal situation. e.g. I would not necessarily do it on a shared university machine, but I don't see an issue doing it on a secure corporate cloud machine where only other developers may have ssh access.

You can just open ~/.screenrc with your favorite text editor and add the lines multiuser on and acladd <username> as we did in steps 2 and 3 above. Then all your screen sessions will be multiuser-enabled and available to join by anyone whom you've given access to via acladd.

Using .screenrc to manage access also makes it easier to use screen's advanced multiuser acl features, like enabling read-only access by using the -w flag on the acladd command. See the man page for more information.

Summary

Set up ssh access for your friend, and then you run:

screen -US example_name
C-a :multiuser on
C-a :acladd friends_username

Then your friend sshs in and runs

screen -Ux your_username/example_name

Credits

I used this page to remind myself how to do this, and really the reason I wrote this post is so that I don't have to find that page in the future when I forget.

How to format your entire codebase without introducing backdoors

2021-05-14T00:00:00-04:00

When working on legacy code with many developers, it's pretty common for someone to suggest running a code formatter on the whole codebase. Even small code formatting changes often get called out in code review for making the code harder to review. Why don't we automatically format our code?

The fundamental problem is that it's virtually impossible to review the changes since they're likely to hit every file in your codebase. It would be very easy to introduce, for example, a hidden backdoor, and you also can't verify if, due to configuration differences, the formatting simply isn't as agreed upon without further systems in place.

So how do we take a codebase without consistent style and make it pretty?

Step 0: Make a decision as a team to format the code

If you're working as a team (if you're the sole owner of your code presumably you can trust yourself to not introduce backdoors) you need to agree on what code style and formatter you're going to use. Try not to bikeshed too much and use the defaults your particular tool has.

Step 1: Make sure that code formatting is required on all future commits.

You must have an automated step in your review process or build pipeline to check that the code is formatted properly. If you don't do this, you'll eventually end up where you started: inconsistent formatting, unreadable test names, and needless clashes over style in code review.

You may be able to enable this prior to actually formatting the rest of the codebase if your tool is able to process only diffs, but if not you will need to enable it immediately after committing the formatted code.

In fact, you may be able to stop here! If you trust your CI system, it may be sufficient to post the code review and have your reviewers note that it passed the formatting check in your pipeline. If it isn't possible to do this or you would like to have more confidence, keep reading.

Step 2: Document the steps you will take to format the code

We want to write this document for two reasons: One, so that when new developers join your team they can set up their environment to use the same formatting, and two, so that the actions performed during step 3 below are clear to everyone.

This may be as simple as writing "run command X in the root of the repository" in your project's README or adding a script to your existing build process. Be sure to include any configuration files required for the formatter! Common IDE configurations are also nice. This step should be pretty easy, because it should be the same as your automated system's setup.

Step 3: The Formatting Ceremony

This is the interesting part.

Similar to a key-signing party or the Root DNS key signing ceremony, decide as a team who you trust to make the changes, have them gather in a room (or use pre-shared gpg keys to sign the commits if you are fully remote), and have them all run the same steps to format the code.

Then, by sharing the code (pushing separate branches to a shared repository, transferring via USB drive, or carrier pigeon), have each member diff their formatted code against the others' and check that there are no deviations. If you're remote, make sure to check the signatures on each commit with git log --show-signature.

If everything checks out, pick someone at random to send out the pull request and have the other ceremony participants sign off after checking the commit id (and signature) matches the ones checked locally, hopefully in addition to the newly-added automated formatting check.

Remaining issues and alternative solutions

The biggest issue with a large formatting commit is that the output of git blame and to a lesser extent git log are obscured. Because your formatter will very likely touch a large percentage of the lines in your codebase, running git blame will show you the formatting commit as the most recent commit, instead of actual code changes, on most lines.

There's a good article describing a relatively recent feature of git which lets you configure a file (or command line parameter) listing commits that git blame should ignore. As noted in the article, most tools, like GitHub and GitLab, do not support this feature yet.

Also, there may be a way to streamline the verification process if you're meeting physically in the same place, by doing some kind of variant of the Zimmermann-Sassaman key-signing protocol used at key-signing parties.

An alternative and gradual but more annoying solution

There is an alternate, but more annoying way to make these changes without a single big commit, and without obscuring the git blame.

You can run the code formatter per-diff, only on the code that has been changed. As your codebase is worked on, it will gradually be formatted. This can work if it's constantly under change, you have a lot of large commits and rewrites, or your codebase is mostly pretty clean and you just want to enforce the standard for new code only. There are cases where this will not work, if for example you want to change indentation style in Python or if your formatting style requires a specific method/class/variable naming convention and your commits do not touch all uses of the name.

The main downside of this method is that your codebase becomes partially-formatted, which may make it more unreadable than it was before.

The other downside of doing the formatting partially is that if you want to make this process automated, your formatter has to be able to work on diffs, which is not a particularly common feature. You may have to format the entire file and then use features of git add to only add the specific lines you want, which is annoying.

Summary

Agree to require code formatting on all commits
Make it automated
Gather trusted people in a room, or have them exchange gpg keys over a trusted channel if remote
Everyone runs the same steps to format the code, exchanges the resulting commits, and verifies that
The commits are all the same
The signatures match (if working remotely)
Randomly choose someone to push the code for review
Everyone again verifies the code in the review matches their own diff, and approves the code
Enable a formatting check in your CI pipeline / code review tool
Use git config blame.ignoreRevsFile with a file that contains the formatting commit id to ignore it in git blame

I honestly don't know how many projects this is actually useful for: it's mostly written to try to convince my coworkers to do it for our codebase. I think there would be objections to doing this on an even mildly popular open source project.

I'd love to hear about alternate methods to do this, or if you've successfully done something similar in your own codebase as a datapoint for implementing it at work.

Thank you Victor, Stan, and Pasha for reviewing drafts of this article.

How to make daily backups with rdiff-backup and systemd

2014-09-12T00:00:00-04:00

The other day I was talking to one of my friends and he mentioned how he lost a hard drive recently and consequently all of his stuff. I realized that, although I had bought a backup hard drive a little while ago, I hadn't done more than a couple backups with it when I felt like it. So I decided that I would finally set up regular backups of my home directory.

I'd looked briefly at backup solutions before and thought that rdiff-backup seemed like a good choice. I also want to do snapshots at some point either with straight-up rsync or rsnapshot, but for now just the differential backups are probably fine. So I chose rdiff-backup.

One easily-solved problem I encountered was that I have large data files that I don't have space to back up and that aren't super necessary to back up in the first place. However, I wanted to keep track of the files. So I'm using rdiff-backup --exclude to exclude those directories, and then writing the output of ls -lLR in those directories to a file to keep track of what's in them. Since the -l option tells us the modified times, I didn't think it was necessary to write the listing files out into my actual home directory and then have that be part of the backup. EDIT: On second thought, it actually is necessary to do so. If, for example, the entire directory we're saving the listing of was deleted, and then the backup script ran again, the listing would be overwritten with nothing and it would be pointless. So we need to do the listing before running the background and it needs to output to a directory that is going to be backed up.

I combined these two steps into a python script that prints some information about what it's doing and then runs the commands. I always prefer python scripts to bash because you don't have to worry about escaping, or portability, or unreadability, or any of the other problems one encounters with shell scripting.

The other half of the backup problem is actually running the backups. Putting a backup command in your crontab isn't the best idea for a desktop or laptop because it might be scheduled to run while your computer is off, resulting in a missed backup. Anacron is traditionally the solution to this: it runs cron jobs on a hourly/daily/monthly/whatever basis and, more importantly, will run jobs that were missed while the computer was off.

However, I knew that systemd somewhat-recently gained support for doing cron-like things, and I knew that if I used systemd and wrote a unit file to run the backups, I could get the output easily and nicely into systemd's journal, making it easy to check that backups are running. If I put it in cron or anacron, which on my system are both covered by cronie, the output from my backup script would be mixed together with the other cronjobs' outputs. (I think.)

So I wrote a simple .service unit file that runs the python script I wrote as a oneshot, and then wrote a .timer file which tells systemd to run the corresponding .service file with the same name daily (OnCalendar=daily) and to run it ASAP if we missed the last time it was supposed to run (Persistent=true). The service file also has some options to set the nice value to 19 (the lowest priority) and lower the IO priority. (Nice=19, IOSchedulingClass=2, IOSchedulingPriority=7) These were mostly copied from this page on the arch linux wiki. Make sure to delete the in-line comments; they will cause errors if you try to actually use them as they are.

I looked into using systemd user sessions to run these, but it seemed a bit complicated to set up and I wasn't entirely sure what the benefit was. Furthermore, I might use this script to do backups of /etc or something in the future, which would mean I wouldn't use a user session anyway. So instead I have the unit run as my own user (User=username) and copied the .service and .timer files into /etc/systemd/system/. I copied the python script that does the backups to /usr/local/bin/backup-scripts/, though you could just leave it in /usr/local/bin/ just the same. The last step to make everything work is to enable the timer unit with something like sudo systemctl enable my-backup.timer.

It's probably irrational, but I feel a bit nervous making my script and unit files public even though the worst/best thing that would probably happen would be someone pointing out a way to make them better. Email me or something if you want to see them, I guess? that feels really lame.

Convolution

2014-06-09T00:00:00-04:00

So I'm reading the chapter "Signals and Systems" in PDIS, and since I took Professor Goodman's DSP course a lot of the material is familiar to me, but presented in a different way or context. It's very neat to see things presented in a slightly unfamiliar way and then realize "oh, I know this." It's somehow validating your knowledge or something and is gratifying in a weird but nice way.

In particular, this sentence really clicked for me in sort of mapping different intuitions to the same underlying idea:

Convolution is important because it tells us how to use a system's impulse response to find the output of the system to a given input.

(at this point I continued reading)

The exposition in this book is really quite good. Following from the previous statement, we determined that $e^{\omega t}$ is an eigenfunction of LTI systems by taking the convolution with the system's impulse response $h(x)$. (this sounds like a simple statement, or at least a succinct one, but it takes a bit of time to understand) Its eigenvalue is the frequency response $\int h(\tau)e^{-\omega\tau}d\tau$.

Then we have:

This fact reveals that one of the easiest types of functions to study with respect to LTI systems are the complex exponentials, since they pass through such systems unchanged except for complex scaling. If we can represent an input signal as a sum of these functions, then we can find the response of the system to each exponential individually, and then sum the responses together. The Fourier series and transform provide precisely the tools that decompose a signal into a sum of exponentials.

This seems like a very different way to arrive at Fourier series and transform than "let's decompose a function as a sum of sine and cosines because that sounds like fun". Basically, it gives us a reason why we might want to do such a thing a priori.

Principles of Digital Image Synthesis

2014-06-04T00:00:00-04:00

So one of the things I'm doing this summer is reading Principles of Digital Image Synthesis by Andrew Glassner. It's from 1995 which makes it almost 20 years old, but the fundamental mathematics don't change and it's quite comprehensive. I found it released freely here

It starts with "The Human Visual System and Color" for the first 100ish pages, covering "The Human Visual System", "Color Spaces", and "Displays". While I thought the chapter on how the eye works was interesting, I didn't really pay close attention to it, so that I feel like while I have a better understanding of how eyesight works than before, I don't think I could really explain it to someone else.

Color spaces was too abstract for something that I thought should have been more visceral considering that it's color, but I think the main point of the chapter is that saying "it's just color, how difficult can it be?" is a very big mistake. There are problems with how a human perceives color, the unintuitiveness of the CIE XYZ color space, and more importantly the gamut that your monitor or printer is able to render. The chapter on color spaces also talks about the problem of perceptual uniformity; that is, in the XYZ color space, changes of equal distance are not equally perceptible.

The ellipses shown were reported to be of constant color in an experiment, and they may or may not on your computer because the ellipses are enlarged and your computer may display the colors differently, so that there is a perceivable difference.

This is the part where I sort of stopped trying to understand 100% of what was going on. To fix this problem of perceptual (non-)uniformity, the CIE defined other spaces, L*a*b and L*u*v, which are transformations of the XYZ color space. I understand that the point is to make the spaces perceptually linear, but I don't really see what the formulas are doing or have any intuition for them other than the description that you can imagine a cylinder with the vertical axis being lightness and the angle and distance from the center being hue and saturation or whatever. Maybe if the pictures in the pdf were in color they would make sense. We actually have a copy of the book in the libraries at Rutgers; I should really just check the first volume out or something.

The chapter on Displays was interesting but mostly about CRTs, which, obviously, aren't used very much after 20 years, so I don't really have much to say about it. There was material about the RGB color space and gamut mapping but I feel like I don't need to understand that 100% right now so I will continue on.

I wrote surprisingly more than I thought I would for what amounts to a summary of a cursory reading of the (if I may) more boring parts of a textbook.

recycled math

"Containerize" individual functions in Rust with extrasafe

Extrasafe now has namespace support

Overview

Use-cases

Design choices

Clone vs fork

When to exec

Filesystem consideration

Not execing at all?

Network

Testing

Future work

Isolates

Issues

Other security features

Thanks for reading

Yet another E-Ink weather display - but with Rust!

Hardware

E-ink display

Microcontroller

Software

On-device software

ESP32 overview

Getting the data

Waveshare e-ink driver

Rendering software

Gathering the data

Drawing and measuring text

Drawing the graphs

Dithering and "rendering" to the display buffer

Running the rendering software

Extrasafe

The Case

Things I would like to improve

Conclusions

How to pair program like it's the 1980s (with GNU Screen)

What we're doing

Why

Prerequisites setup

Screen basics

Execution

1. Naming a session

2. Enable multiuser mode

3. Give the other user access to the session

4. Attaching to the shared session

Using .screenrc for streamlined shared access

Summary

Credits

How to format your entire codebase without introducing backdoors

Step 0: Make a decision as a team to format the code

Step 1: Make sure that code formatting is required on all future commits.

Step 2: Document the steps you will take to format the code

Step 3: The Formatting Ceremony

Remaining issues and alternative solutions

An alternative and gradual but more annoying solution

Summary

How to make daily backups with rdiff-backup and systemd

Convolution

Principles of Digital Image Synthesis

Using `.screenrc` for streamlined shared access