Hang Out With J. B. Rainsberger

Why My Webcam Image Is So Damn Red!

Wed, 29 Apr 2026 00:00:00 +0000

After a brief absence from home, I had a Zoom meeting, so I turned on my webcam and, much to my surprise, everything looked shifted far towards the red! I didn’t find anything in my OBS Studio settings nor in Zoom, but I was late for the meeting, so I had to switch back to my crappy internal webcam until I could sit down to figure out just what happened.

Just What Happened

I used guvcview to look at the settings of the camera. Fortunately, I have another camera, so I could do some basic A/B testing to isolate the problem. The other camera looked fine. The difference lay in a single setting: Saturation. On the “faulty” camera, it was set to 100; on the “working” camera, it was set to 50.

I don’t know how that setting was changed. The problem is solved!

A Neat Feature

The utility guvcview has the feature to save a profile to a plain text file, so I could save the settings on both cameras, then compare them with meld. Both cameras have the same settings now, so presumably they behave the same now. Great! I can even put these profiles into a version control repository as a baseline for the next time my camera does something strange.

Details

The cameras involved are Anker PowerConf C200 cameras and I experienced this problem on Pop!_OS 24.04. I also found a project that threatens to have CLI tools for PowerConf cameras: https://github.com/erans/anker-powerconf-c200-linux-tools. This project bears examining, but I haven’t done that yet.

Byobu Session Name in Terminal Window Title

Tue, 17 Mar 2026 00:00:00 +0000

I started using multiple byobu windows to organize my projects, but in the process, something annoying happened: I began to lose track of which window was which. When switching windows, I couldn’t tell from the window title which byobu window to switch to.

“Uhh… which one?”

Fortunately, I use tmux as the terminal back-end for byobu, so I some very elementary tmux configuration was enable to solve the problem.

Turn Titles On in `tmux`

In a tmux window:

Press F12.
Type :set -g set-titles on.
Type :set -g set-titles-string "#{session_name} @ #{host_short}:#{pane_current_path}".

When you reopen the terminal window, you’ll see that the byobu session name is now at the beginning of the terminal window title, making it easier to see when switching windows.

“A little busy, but good enough for now.”

Remember These Settings Forever

Edit the .tmux.conf configuration file. I found mine at $HOME/.config/byobu/.tmux.conf. Add these lines:

set -g set-titles on
set -g set-titles-string "#{session_name} @ #{host_short}:#{pane_current_path}"

When you reopen the terminal window, you’ll see that these settings have become yours to keep forever.

Viewing HEIC (HEIF) Images on Pop! OS 22.04

Tue, 14 Oct 2025 00:00:00 +0000

Today I was blindsided by the existence of HEIC images, because I don’t often receive images from iPhone/iPad users. Since Pop!_OS is still running Ubuntu 22.04, I not only didn’t have the library I needed to view these images, but when I found the library I needed, the standard package manager had an out-of-date version for the images I was trying to view. It took an hour of stumbling around to track down a solution.

The Solution

Install the latest version of the libheif package from the Ubuntu PPA.

$ sudo add-apt-repository ppa:strukturag/libheif
$ sudo nala install libheif1 heif-gdk-pixbuf heif-thumbnailer

The Symptom

I couldn’t open the HEIC images using eog, but I could open them using GIMP. I wanted to convert the images in bulk to JPG, so I didn’t stop at using GIMP to convert the images one by one using the UI.

Attempt 1

I tried to install heif-gdk-pixbuf, but then neither magick nor eog could read the image. I got this error message:

Could not read HEIF/AVIF file: Invalid input: Unspecified: Metadata not correctly assigned to image

This is how I learned that the version (1.12) of libheif that ships (as of the date of writing this) in the Ubuntu package manager can’t read HEIC images taken on iOS 18. I needed to install the latest version (1.19.8) from the Ubuntu PPA listed in the solution.

Now I can receive images from my friends who use iOS 18 and, so far, all the image tools I use can read the images without issue.

Video Gradually and Increasingly Falling Behind: Don't Use Buffering

Wed, 16 Jul 2025 00:00:00 +0000

I noticed in early 2025 that, when I used my USB external webcam together with OBS Studio for video sessions, that the video signal was falling gradually and increasingly behind my audio signal and, indeed, real time. Sometimes it took 10 minutes, sometimes it took an hour, but eventually, my video would become so far out of sync with my audio that it became too distracting to continue. Restarting OBS Studio brought the camera back in sync with my microphone.

The problem lay in buffering, so I turned it off, and now all seems well.

In OBS Studio, open the Properties for your V4L2 Video Source.
Look for the option Use buffering and unselect it. In OBS Studio 31, this is a checkbox, so uncheck it.

I don’t know whether you need to restart OBS Studio for this change to take effect, but you might.

With buffering off, my webcam has not (yet) fallen behind noticeably, even after over 90 minutes of continuous use on my laptop. If the symptom remains, it is much much much less noticeable.

A Fix?

The nice folks in the OBS Studio community are working on a fix as of July 25, 2025.

Key References

Some Details

First, the ingredients that led to the symptom:

OBS Studio 31.0.4 running on Flatpak
Anker PowerConf C200 external USB webcam connected by USB-A
Røde lapel microphone connected to an audio-to-USB adapter connected to an external (powered) USB hub connected by USB-A
The Zoom client for Linux
Pop!_OS 22.04

The symptom: the video signal echoed through OBS Studio and transmitted over Zoom gradually and increasingly fell behind real time, although my audio signal stayed very close (close enough!) to real time for me not to notice the difference. Video fell more and more out of sync with audio (and real time) without a clear pattern for when it would happen nor how much. I only knew that it happened increasingly, so that choosing a fixed audio offset would not solve the problem.

With all these variables, you can imagine how many permutations I had to test to gather information. Sometimes I could run my camera for 30 minutes and not notice a difference and sometimes I could run my camera 10 minutes and start seeing my video fall noticeably behind. As I was thinking about what could be happening, I was ruling ingredients out, such as Flatpak, the USB hub, the microphone, the webcam (my laptop’s internal webcam seemed fine), the Zoom client, the Zoom web app on Chrome, the current version of OBS Studio… it took weeks to gather enough information to be able to ask for help in a meaningful way.

Why not just use my internal webcam? Because on it I look pink like a lobster… part of the time. I like this external USB webcam precisely because it provides a nicer and more consistent picture.

Finally, I managed to point the finger quite strongly at OBS Studio, which led me to ask on the forum. It took nearly two weeks, but someone replied not only with a helpful idea, but with a very likely explanation for all the behavior I was seeing: OBS Studio was buffering the frames and insisting on displaying “out of date” frames rather than jumping forward to the front of the queue. When the computer was under heavier load, the frame queue kept expanding and OBS Studio kept insisting on displaying frames in sequence, rather than jumping forward. The issue seems to revolve around the difficult question of which timestamps to trust coming from which clocks.

Very well, then: turn buffering off, which results in perhaps dropping more frames, but eliminates the problem of never catching up to the front of the frame queue.

Presumably this buffering also affects my laptop’s internal webcam, but I presume that external webcams connected by an external USB port are more likely to be affected by variations in the computer’s workload. I suppose if I kept OBS Studio open for 24 hours and kept my laptop busy from time to time, even a buffered internal webcam video source might fall behind real time.

So there it is. I don’t know whether I will ever regret turning buffering off, but at least for now, my voice and my face are reunited, which is plenty good enough for today.

Upgrading Discourse When You Totally Have Enough Disk Space

Tue, 21 May 2024 00:00:00 +0000

I run some forums with Discourse and I upgrade them. Sometimes, when I upgrade them, I end up in an infinite loop of sadness. There is an easy way out that doesn’t require more disk space, because sometimes, you know, you already have enough disk space.

The Symptom

Try to upgrade and you get a message telling you to free up disk space, but when you try to free up disk space by deleting unused containers, and then you have something like 7.5 GiB free, you get that message again. But… I have enough disk space!

A Workaround

According to this post at the Discourse Discourse, you can tell the upgrade script not to check for free space, but just go ahead. And it often works!

# ./launcher rebuild app --skip-prereqs

If this doesn’t work, then you actually need to free some disk space or increase the disk capacity.

Why?!

You have 7.1GB free before downloading the discourse container image but less than 5gb after downloading it when we perform the check.

Now you know.

unWise Arithmetic

Mon, 08 Apr 2024 00:00:00 +0000

My company and I bank with Wise and today, while doing a routine check of exchange rates, Wise’s UI scared the hell out of me. Fortunately, they were able to explain why it looked like their web site was bothering to throwing away the paltry sum of $10 on a currency conversion. They weren’t. It’s fine; it’s just a questionable user experience.

Please note: I have notified Wise customer support about this. Maybe they’ll do something to make it better. I’m guessing that they’ve had non-trivial internal discussion about the tension between a cleaner UI and a more-transparent UI. The ball is firmly in their court now.

The Symptom

I visited Wise’s “Convert Money” feature to see what they would give me in CAD for all the USD in my account. Their UI helpfully shows the conversion like this:

USD amount
- our fees
x exchange rate
= the CAD we would deposit into your account

I really like how clearly they present their calculation. There was only one problem: the CAD amount didn’t match the calculation. It was a difference of about $10, which is not a large amount, but is enough to look like a hidden fee, rather than a computation error. Wise’s brand is fee transparency as well as keeping fees low.

The Impact

Was there some hidden fee I didn’t know about?! Did Wise forget how to do arithmetic?! I expect them to get it right to the penny. Every. Single Time. It’s what I pay every bank for. Whatever the explanation, there is some part of my psyche that now trusts them less. I can’t be the only person.

The Problem

Wise reports the exchange rate rounded to four decimal places, however, they apply a more-precise exchange rate. When the amount to convert is large enough, a difference of 0.00005 becomes noticeable, and when they round up, it looks like I’m not getting all my money, even though I am.

The Solution

I told Wise customer support what psychological effect this experience had on me, notably on my perception of their brand. Now it’s on me to remember this for the future as well as on them to decide whether a slightly cleaner UI is worth scaring a segment of their customer base. It has become a business decision for them. I did my duty and reported the issue; I also did my best to present my opinion without undue drama.

And I believe my heart rate is all the way back to normal.

strptime: a Beginner's Mistake

Thu, 14 Mar 2024 00:00:00 +0000

I’m using jq to process CSV files to use in a Plain Text Accounting environment, specifically using hledger. I mention all this, not because it’s salient, but purely to make it easier for you to find this article by a web search.

The Symptom

$ jq -n '"8/4/2023" | strptime("%D")'
jq: error (at <unknown>): date "8/4/2023" does not match format "%D"

$ jq -n '"8/4/2023" | strptime("%m/%d/%y")'
jq: error (at <unknown>): date "8/4/2023" does not match format "%m/%d/%y"

WAT?!

The Problem

The strptime pattern %y matches the string 23, but not the string 2023. It’s pretty clear in the man page, if a little unexpected.

%y     The  year  within  century  (0–99).   When a century is not otherwise specified, values in the range 69–99 refer to years in the
      twentieth century (1969–1999); values in the range 00–68 refer to years in the twenty-first century (2000–2068).

%Y     The year, including century (for example, 1991).

The Solution

$ jq -n '"8/4/2023" | strptime("%m/%d/%Y")'
[
  2023,
  7,
  4,
  0,
  0,
  0,
  5,
  215
]

Remove a Password from a PDF Document on Linux the Super-Easy Way

Mon, 19 Feb 2024 00:00:00 +0000

My accountants sensibly send me PDFs of returns and authorization forms with password protection enabled. Sadly, this makes it difficult for me to add text and sign the document digitally, at least with my preferred PDF editor on Linux, Master PDF Editor. Every year I fail to remember how to work around this problem, so I’m writing this article, hoping that my web search will find it next year.

The Symptom

My PDF document is password protected, so I can’t do anything with it, including adding text and signing it digitally.

The Problem

Web searches for this topic return many many many irrelevant articles. It’s worse than searching the web for recipes.

The Solution

Use evince to print a new copy of the PDF document to file. This removes the passwodrd protection by default.

The Mysterious Extra Quotes that `jc` Put In My JSON

Tue, 06 Feb 2024 00:00:00 +0000

The Symptom

I try to use jc to convert a CSV file to JSON for further processing with jq. When I try to process the JSON document with jq, it fails, unable to find a value for the first cell of the first row.

When I look at the JSON document, the JSON object key corresponding the first column name in the CSV document has extra quotes around it: instead of "Date", the key is "\"Date\"". Only this key has been changed; the others are exactly as I expect them.

It seems that jc has put extraneous quotes (or extra quotation marks¹ ) around a CSV column header name.

The Cause

The CSV file has a Byte Order Mark (BOM) at the beginning, which is unprintable, so I didn’t notice it. These unprintable characters become part of the name of the first CSV column, which then becomes a key in the resulting JSON objects that jc --csv parses.

Since "\"Date\"" is not equal to "Date", chaos reigns.

The Solution

Upgrade jc to at least version 1.22.2, at least according to this issue. I was using… 1.20.4. How embarrassing.

$ pip3 install jc
$ jc --version
jc version:  1.25.0
python interpreter version:  3.11.2
python path:  /home/jbrains/ThirdParty/python-3.11.2/bin/python3

https://github.com/kellyjonbrazil/jc
© 2019-2024 Kelly Brazil

The Gory Details

I reproduce these details in order to give the next poor soul a chance to make it here by reading a single web page. Here are the steps that led me here.

I downloaded a CSV file of transactions from PayPal. (I use them against my will.)

I tried to process the CSV file with jc --csv | jq '.["Date"] | parse_paypal_date', because I’ve written a jq function to parse PayPal’s annoying date format of dd/mm/yyyy. (If I got to pass one law, it’d require ISO 8601 for all dates all the time everywhere.)

My processing command failed with

jq: error (at <stdin>:1): strptime/1 requires string inputs and arguments

I checked the CSV file and I saw a column named Date with obvious string values, starting with "02/11/2022", which is, of course, 2022-11-02 to people with taste.

After a few iterations of disbelief and confusion, I checked the JSON document.

[
  {
    "\"Date\"": "02/11/2022",

Extra quotes?!

And, BTW, when I paste the JSON output into Typora to write this article, Typora clearly shows me that there’s an unprintable character at the beginning of that key.

I’ve never seen this before, so let me check another, very similar CSV file from a different source. Was this always broken?!

Nope. Everything looks good from another CSV document that also has "Date" as the first column name.

[
  {
    "Date": "11/01/2022",

Spot the Difference!

At least now I can compare these two CSV documents and ignore the rest of the world. That makes diagnosis much simpler.

Since I can’t see a difference in the first column name between these two files, there must be an unprintable difference. How do I see the unprintable characters in a text file on the typical Linux distribution?

Today I Learned about cat -A.

$ cat -A /path/to/annoying/paypal.csv | head -n 1    
M-oM-;M-?"Date","Time","TimeZone","Name","Type","Status","Currency","Gross","Fee","Net","From Email Address","To Email Address","Transaction ID","Shipping Address","Address Status","Item Title","Item ID","Shipping and Handling Amount","Insurance Amount","Sales Tax","Option 1 Name","Option 1 Value","Option 2 Name","Option 2 Value","Reference Txn ID","Invoice Number","Custom Number","Quantity","Receipt ID","Balance","Address Line 1","Address Line 2/District/Neighborhood","Town/City","State/Province/Region/County/Territory/Prefecture/Republic","Zip/Postal Code","Country","Contact Phone Number","Subject","Note","Country Code","Balance Impact"$

TIL: Byte Order Marks

I inhaled quickly, then exhaled slowly.

I searched the web for "M-oM-;M-?" and found a non-trivial number of resuilts, which led me to learn about Byte Order Marks (BOM), thanks first to a Stack Overflow answer.

I don’t know much about Byte Order Marks, except that now I have better search keywords.

I searched the web for remove "BOM" from csv file and found a non-trivial number of results, including the authoritative-sounding “CSV and BOM character”. Unfortunately, I’m not writing PHP code using LeagueCSV, so that helps me a bit less to work around the problem. Even so, it gives me confidence that there exists some body of prior art on this subject, so I don’t need to figure this all out on my own.

I Can’t Be the First!

I went back to jc, hoping to find an issue about this, before I went charging off on my own to ~~fix~~ work around the problem. And that’s when I found this issue. And that’s when I discovered that I’m using an outdated version of jc and that upgrading might fix the problem.

And that’s when I wrote all this, while it was fresh in my mind.

I add synonyms here, not for clarity, but to attract more hits in web searches.↩︎

Ducking in Screenflow

Sat, 26 Aug 2023 00:00:00 +0000

The Problem

I don’t remember how to do ducking correctly in Screenflow. I can’t figure it out by just doing what I think makes sense. Everyone wants to provide help as a video instead of as searchable text. I’m impatient.

The Solution

Leave the music bed track, the one under the voiceover, alone. Enable ducking on the voiceover track. Adjust until happy.

LibreOffice Calc, Headers, Footers, and Every Page!

Wed, 17 May 2023 00:00:00 +0000

I prepare expense reports using LibreOffice Calc, because nothing seems to handle multiple currencies the way my company wants to do it. The details don’t matter, but it’s enough to know that I print a workbook to PDF as part of my workflow. This workbook has many worksheets. I want headers and footers on each page in order to easily see which document I’m looking at.

Suddenly, without warning, I started to see out-of-date headers on page 2 of my document. Several minutes of checking page styles and header settings didn’t change anything. I couldn’t find the magic keywords to search the web to find the problem. And then, finally, I noticed the issue: a new feature in LibreOffice 7.

The Problem

Somebody wanted to be able to use different headers on the first page from the headers on remaining pages within a single sheet page style. They added that feature. They even made it possible to keep the previous behavior of using the same headers on every page. Sadly, they didn’t make the old behavior the default choice, so it changed without my consent.

Moreover, they named the option in a way that makes sense to programmers, but not to normal, everyday humans. I didn’t notice it even after looking directly at it a handful of times.

The Solution

Open the Styles panel and locate your page style.
Modify….
Choose the tab Header.
Locate the new option “Same content on first page”. It’s probably disabled, so enable it!
Press Edit…. Notice that now there only one tab for the header.

LibreOffice Calc will now use this one header on every page of your PDF document.

If you’d like to see the difference, disable the option Same content on first page, then edit the headers again. Now notice that there are two settings tabs: one for the first page and one for the rest of the pages.

Now check your Footer settings.

A UX Suggestion For Everyone

When you add a new feature with configuration settings, give those settings default values that match the previous behavior, so that you don’t confuse users. Thanks.

How Pandoc and Ranger Froze My Laptop

Thu, 20 Apr 2023 00:00:00 +0000

I use ranger to manage files from my shell, because it gives me vim-style navigation¹ and search features that help me navigate my file system very quickly.

I use LibreOffice for all my spreadsheet needs because, well, Numbers.app is not available on Linux.

I use pandoc for a variety of reasons, including converting blog posts into PDFs.

When we put those three together, however, my laptop fan started blowing at top speed and wouldn’t stop. My laptop froze—or at least went very very slowly—and I had to reboot a few times and explore a little in order to discover the problem.

The Problem

ranger can create previews of files as you navigate through the file system.
ranger uses pandoc to create previews of certain kinds of files, rendering the text from some kinds of non-text files. This makes it possible to preview a PDF, for example, by extracting the text instead of rendering a 1/4-size image of the file that only very young eyes could possibly read.
pandoc (at least version 2.18, which I have installed) becomes stuck when it tries to process an Open Document Spreadsheet (*.ods). According to the project’s lead maintainer, pandoc doesn’t support that format so whatever happens, happens.
When I navigate to a directory in ranger and that directory contains Open Document Spreadsheet files… pandoc tries very hard to render the spreadsheet, leaks memory, spins forever at top speed, and begins eating CPUs.

Users of pandoc should not expect this problem to be resolved any time soon. Garbage in, garbage out, it seems. And yes: a binary file is garbage to a Markdown parser.

The Solution

Install a version of ranger that doesn’t try to render previews for Open Document Spreadsheet files. Sadly, ranger doesn’t release new versions very often, which makes this slightly more complicated than installing the latest version from your Linux distribution’s package manager.

It appears that ranger fixed the problem in commit 9da2b4c79323a3f34da8d54198364d549a29ff00. I built ranger from source with the latest version of the main branch of the project as of 2023-04-20. This version worked for me. Building from source didn’t seem intimidating to me and it just worked the first time.

I hope you have a similarly smooth experience fixing the problem.

I would prefer kakoune-style navigation, but I can settle for vim.↩︎

Installing hledger

Tue, 31 Jan 2023 00:00:00 +0000

I became interested in Plain Text Accounting and decided to install hledger to see how to start using it. I failed to install it the first time due to this message:

$ stack install --resolver=lts-19 hledger-lib-1.28 hledger-1.28 hledger-ui-1.28 hledger-web-1.28
Selected resolver: lts-19.33

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for hledger-ui-1.28:
    brick-0.68.1 from stack configuration does not match >=1.5  (latest matching version is 1.6)
    fsnotify-0.3.0.1 from stack configuration does not match >=0.4 && <0.5  (latest matching version is 0.4.1.0)
needed since hledger-ui is a build target.

Some different approaches to resolving this:

  * Set 'allow-newer: true' in /home/jbrains/.stack/config.yaml to ignore all version constraints and build anyway.

  * Recommended action: try adding the following to your extra-deps in /home/jbrains/.stack/global-project/stack.yaml:

- brick-1.6@sha256:4a94b18be68d2d27da9e84eff7f8b4dc8bfa3a40d28d536fb737828d73a0e3eb,18899
- fsnotify-0.4.1.0@sha256:44540beabea36aeeef930aa4d5f28091d431904bc9923b6ac4d358831c651235,2854

Plan construction failed.

Since I don’t know ghcup at all, I was stuck. I managed to piece together an answer: specify the additional packages in the command, so that stack would install them.

$ stack install --resolver=lts-20.8 hledger-lib-1.28 hledger-1.28 hledger-ui-1.28 hledger-web-1.28 brick-1.5 fsnotify-0.4.0.0

This installed hledger and now I have a new task to spend a few hours figuring out what I can do with it.

Notice the version numbers of the various packages, which don’t exactly match the installation documentation nor the “Recommended action” in the installation failure message.

How I Knew…

I couldn’t figure out how to specify the additional packages simply. I didn’t want to change a magic global configuration file, so I inferred that there must be a way to specify the extra packages as part of the stack install command.

I read the section of the install documentation entitled “Build with hledger-install”, which directed me to read the installation script. When I did this, I noticed some helpful-looking package version numbers.

# the oldest version of stack that might possibly work: perhaps 2.5.1
STACK_MIN_VERSION=2.5.1

# stackage snapshot to use when installing with stack.
# You can try specifying a different stackage version here, or
# commenting out this line to use your current global resolver,
# to avoid unnecessary building.
STACK_RESOLVER="--resolver=lts-20.8"

# Dependencies we require that aren't in the above stackage snapshot.
# (Also requested when using cabal, but that's harmless.)
STACK_EXTRA_DEPS="\
brick-1.5 \
fsnotify-0.4.0.0 \
"

This suggested to me that I ought to try a different resolver version (lts-20.8 instead of lts-19) and these particular package versions for brick and fsnotify.

After reading through the installation script, I realized—and felt mildly stupid about this—that I needed merely to specify the additional packages with their version numbers as packages to install. (I had kept looking for some “extra dependencies” command-line argument and couldn’t find it!) That led me to the winning command.

As of today, these versions of the various tools worked for me on Pop!_OS 22.04.

ghc 9.2.5
stack 2.9.1
- resolver lts-20.8
hledger-* 1.28
brick-1.5
fsnotify-0.4.0.0

I have no idea how long this information will remain relevant and accurate.

Pop! OS Won't Mount Any External USB Disk

Sat, 24 Dec 2022 00:00:00 +0000

I use Pop!_OS 22.04 and today I encountered a nasty surprise: I couldn’t mount the portable USB disk that I use to back up my Bitwarden vault, which contains passwords and secure notes.¹ Fortunately, it only took a few seconds of web searching to find someone who encountered the same problem.

Solution

Reboot, then mounting the USB disks worked as expected.

Why?

During an operating system package upgrade, I evidently upgraded the Linux kernel from version 5.x to version 6.x. This seems to have caused the usb_storage kernel module to stop working. This meant that I couldn’t mount any external USB disk, no matter which port (USB-C or USB-A). I infer that this kernel module changed in a non-backwards-compatible way and reloading it seems to have solved the problem.

I did not try reloading the kernel module with modprobe, so maybe that would have worked.

How Did We Get Here?

I connected a USB disk (call this “Disk A”) to my System76 Lemur Pro’s USB-C port. It did not mount.

I connected Disk A into my Galaxy Tab. It mounted. I inferred that Disk A works.

I found another USB disk (call this “Disk B”) and connected it to one of my Lemur Pro’s USB-A ports. It did not mount. I inferred that this is a software problem instead of a hardware problem, because what are the chances that both ports are dead at once?

I searched the web: lemur pro usb-c port doesn't work. I found a comment suggesting to check dmesg for related log messages.

$ sudo dmesg | rga -i "usb"
[...a bunch of stuff...]
failed to validate module [usb_storage] BTF: -22
[...a bunch of stuff...]

That seemed like a sign of trouble, so I searched the web: "failed to validate module" usb_storage. I found https://forum.proxmox.com/threads/usb-storage-can-not-be-mounted.100480/. This is how I learned about journalctl -f for monitoring the system logs in real time. I connnected and disconnected the USB disks and watched the messages in real time, confirming the symptom I’d seen. The person who asked the question in this thread reported

I tested again yesterday after a restart. Now everything is working as expected.

I didn’t love trying a magic solution, but I happened to use the command uname -r and noticed that I’d upgraded the kernel to a new major version, namely 6.0.x. That gave me the idea that maybe a kernel module had breaking changes and needed to be reloaded, then suddenly “reboot” didn’t sound like a magic solution any more, but rather made perfect sense to me.

I rebooted. Now all my USB disks mount as expected.

Yes, I deleted my Lastpass account after the breach reported in late December 2022. Thank you for your concern. I hope my master password was strong enough.↩︎

Why My Spreadsheet File is So Big

Thu, 22 Dec 2022 00:00:00 +0000

Today I noticed that a spreadsheet I use had a file size significantly higher than I’d expect: 2 worksheets, maybe 100 rows, but somehow 31.2 MB. (I’m using LibreOffice.)

Fortunately, it didn’t take long for me to find the problem: embedded fonts.

Solution

Open the file in LibreOffice Calc.
File > Properties….
Go to the Font tab.
Uncheck the various settings for embedding fonts in the document.
Save the file and watch the file size decrease.

How I Got Here

I installed 7-Zip and listed the contents of the spreadsheet file (7zz l $FILENAME). I noticed font files. I inferred that LibreOffice had embedded the font files in the spreadsheet file. I searched the web and learned how to stop doing that.

What’s odd is that I don’t remember embedding fonts in this document, so maybe I did it by accident. I also couldn’t find any kind of sticky preference in LibreOffice that would embed fonts in new documents by default. This means that I have no idea how this happened; I presume that information is lost to time.

When Your Site Can't Find Its Discourse Comments

Sun, 06 Nov 2022 00:00:00 +0000

It’s November 2022 and I insist on having comments on my professional blogs, even though nobody really cares. In 2021 I began migrating comments away from Disqus (which I dislike for various reasons) towards Discourse. With Discourse, I embed a topic into each blog post, so that each post corresponds to a Discourse topic. Discourse creates a topic for each post and the comment thread of the topic becomes the comment thread of the post. Nice.

Symptom

Today I was rudely greeted with an HTTP/404 response in the spot on a blog post where comments should appear. In my browser, that showed up as a sad face.

Problem

Discourse did not allow embedding from the domain I was visiting.

Solution

Tell Discourse to allow embedding from all the domains that you use for your site/blog/whatever.

As of November 2022, you’ll find the setting in Discourse under Admin > Customize > Embedding. If you need more details, search the Discourse Forum about Discourse, just as I do every time I need to look something up about Discourse.

Why the Problem Was a Problem

My programming-focused blog can be found at https://blog.thecodewhisperer.com, but also at https://www.thecodewhisperer.com and even at https://thecodewhisperer.com. I need to do one of two things:

Change the HTTP redirect response code to forward the requester (change the address in the browser’s address bar) on to the canonical subdomain blog. or
Add all these domains to the “allow list” for embedding (with the same settinngs).

I chose the latter, even though I don’t love the duplication. I would be thrilled if Discourse added the feature of applying the same “embed allow list” settings to a colletion of domains, but I don’t have the energy to do that today.

Enjoy!

Fix the Automatic Time Zone on Pop! OS

Mon, 22 Aug 2022 00:00:00 +0000

Somewhere along the way, my System 76 laptop stopped automatically updating the time zone based on its location. This annoyed me for a few months, but only today did I finally take some time to explore what was happening and how to fix it.

The Short Version

It seems to matter the sequence of enabling the two relevant settings: Location Services and Automatic Time Zone.

Turn both Location Services and Automatic Time Zone off.
Breathe for 10 seconds.
First, turn Location Services on.
Breathe for 10 seconds.
Next, turn Automatic Time Zone on.

This changed my time zone to the one I find myself in right now, which is not my home time zone. Problem solved.

How I Got Here

Somewhere along the way I turned off Location Services in Pop!_OS 22.04 settings, presumably for privacy reasons. Unfortunately, I had also turned on the Automatic Time Zone setting, not knowing that it needed Location Services to do its work. There was no way to get the correct location in order to find the correct current time zone. I consider this a UI defect, but at least I can make sense of it.

Sadly, when I turned Location Services on, nothing changed. I looked for ways to force the Time Synchronization service to, you know, synchronize the time, but nothing worked. I learned a little about timedatectl along the way. The system time never changed, but neither did I find any error messages in the systemd journal for the systemd-timesyncd service. I couldn’t find any sign that anything was wrong. I infer now that the operating system was trying to ask for the current time zone, didn’t have access to Location Services, and quietly failed to even ask the question. As a result, my laptop remained stuck in Atlantic Daylight Time when I was clearly sitting in Eastern Daylight Time.

And then I thought of something.

My computer programmer mind told me that there was probably some silly dependency between the services, so it probably mattered in which sequence I turned these two services on.

Bingo.

Do you know how I could have reliably discovered this problem, rather than guessing and getting lucky? Please tell me; otherwise, this seems like two defects to me:

the settings UI doesn’t signal the dependency between these two services
the “synchronize time” command doesn’t signal that since Location Services is disabled, it doesn’t know which current location to use to ask for the time zone

Remap Keys on System76 Less Painfully

Tue, 26 Jul 2022 00:00:00 +0000

If you search the web for “remap keys ubuntu”, you’ll find a dizzying collection of contradictory articles on the subject. This is one of those simple questions that seems to have a complicated answer, because of the long legacy of remapping keys on various distributions of Linux over the decades. I found many many ways to do it, not all equivalent, and with a complicated variety of consequences, most of which I do not understand yet. This makes it easy to find answers, but difficult to find answers that fit you well. It seems like one needs to get a Ph. D. in Linux to find an answer that you’ll feel comfortable with.

Of course, if you want to learn more about the boundaries between the operating system and your desktop environment, remapping keys provides an effective way to learn some part of that. I do not—at least not today.

And if you’re like me, you’ll forget about how you remapped your keys, then you’ll wake up one morning after a software update, your keyboard layout will have been reset, and you won’t remember how to fix it.

My Primary Goal

I wanted a solution that allowed me to put my keyboard configuration in version control, so that I knew exactly how to reapply my keyboard mapping any time it changed on me. I wanted the ability to do this quickly and easily.

Rather than wade through all the options available to me, I asked System 76 support what they recommended to me, and I really like their recommendation, at least for now, as of late 2022.

My Current Solution

I use the package system76-keyboard-configurator, which provides two things that I like:

a GUI to remap the keys, so that I can see what I’m doing
an import/export feature, so that I can store my configuration in a text file (JSON format) that I can put in version control (git).

The whole exercise took about 1 minute.

This software changes the keymap at the firmware level, which I infer means that these changes apply before any changes I’ve made in Gnome, whether using the keyboard settings or Gnome Tweaks. I’ll keep an eye out for whether this causes any problems.

If anything behaves strangely, I’ll probably prefer to move the keyboard mappings out of Gnome and into the firmware. For example, I want my Caps Lock key to behave like Control all the time, and not only when I’m running Gnome.

Of course, I’ll need to look for a different solution when I’m working on a Linux machine that doesn’t come from System76, but the only reason I need this is that my Lemur Pro 10 laptop keyboard has a Prt Sc key and no Insert key, but I am used to using the keystroke Alt+Insert quite often in IntelliJ IDEA. In an emergency, I can always change the keystroke in IDEA; in general, I don’t need Prt Sc, because I use a screenshot tool such as Spectacle.

The Steps

$ sudo apt-get install system76-keyboard-configurator
$ system76-keyboard-configurator

This launches the GUI.

Press “Configure Keyboard”.
Remap the keys, as you like.
From the menu, choose “Export Layout”.

This produces a JSON format text file that you can store anywhere you like. I put mine in my dotfiles repository, even though it’s not really a dotfile.

As a test, I used the application to reset my keymap, then import my custom layout. This worked great!

References

https://github.com/pop-os/keyboard-configurator I installed the release published into apt, but it looks pretty easy to run this application directly from source, if you prefer.

“Configuring your new Launch keyboard”. I didn’t figure this out myself, mostly because this article doesn’t make it easy to notice that I can use the Keyboard Configurator on my laptop’s keyboard. When I skimmed this article, I assumed that one would use this tool only with a Launch keyboard, which I don’t have.

Fixing Your Virtual Camera On Microsoft Teams for Linux

Tue, 26 Apr 2022 00:00:00 +0000

I host client meetings, I like OBS Studio, and I run Linux. Sometimes, the pieces don’t talk together nicely. This is the case when my client prefers to use Microsoft Teams. (I typically use Zoom. I know the risks. We are where we are.)

When I tried to use my OBS Studio Virtual Camera device in Microsoft Teams on Linux, I got only a black screen/empty image. No video. (Audio worked fine.)

After some digging, I discovered that this is a known limitation of Microsoft Teams for Linux.

The Workaround

In OBS Studio, set the overall video output resolution to 720p. That’s it. You might need to restart both OBS Studio and Teams.

The Consequences

You will need to resize the various windows and backgrounds you use in your scenes. It’s probably wise to have two OBS Studio configuration directories, so that you can switch back and forth. I’m doing that and it helps.

Exclude Big Files From Your restic Backup

Fri, 04 Feb 2022 00:00:00 +0000

I use restic for backups on Linux. After a cappuccino scare, I turned my attention back to automating a local backup of my everyday laptop. After a few days, I noticed that each quasi-daily backup was adding 200-500 MB of data, which seems excessive. I presumed that I should exclude more files from this automated backup, but I didn’t know which ones. I guessed that I should look for big files or big directories. But how do I get that information?

It turns out to be a combination of restic, stat, and awk.

A Solution

Please note that this solution works on zsh and apparently not on bash. I don’t know enough details to rewrite this in bash, so I can’t help you do that.

Also please note that I use rga, but regular grep would suffice.

restic $BACKUP_CONNECTION_DETAILS \
    diff $BEFORE_SNAPSHOT_ID $AFTER_SNAPSHOT_ID \
    | rga "^(?:M|\+)" | awk '{print $2}' \
    | while read f; do; (stat -t $f | awk '{print $2 , $1}'); done \
    2> /dev/null \
    | sort -n

Your $BACKUP_CONNECTION_DETAILS includes your repository address and password file.
The snapshot IDs are the ones to compare. I compared the most recent two, but just as a start.
stat -t reports files it can’t find to stderr, so I remove them. It shouldn’t be needed, but it’s there just in case.

This command lists the files modified and added to the backup, along with their sizes, sorted with largest files at the bottom. It gave me some clear ideas which directories to exclude from this backup.

The most recent backup added only 8 MB of data to the repository. That’s more like it.

What did I exclude? $HOME/.config/Slack and $HOME/.local/share/flatpak. I’m sure I should exclude more, but I don’t need to figure that out right now. If this becomes a bottleneck again, I’ll look into it.

I will also eventually forget old snapshots and prune the repository, which will remove the large files that I should never have bothered backing up in the first place. There is no rush.

Get Push Notifications for Rate Alerts at wise.com

Thu, 27 Jan 2022 00:00:00 +0000

Wise (aka Transferwise) offers push notifications for foreign currency exchange rate alerts, but I had trouble creating them. I could easily create email alerts, but I wanted push notifications as well. Although it’s possible, it’s not obvious.

The Solution

When you create a rate alert, subscribe with an email address associated with a Wise account. When you do this, the final confirmation page will offer you options for email and push notifications. If you use an email address not associated with a Wise account, you will only see confirmation that you’ll receive rate alerts by email with no evidence that receiving push notifications was ever possible.

As of January 2022, here is the complete flow.

Visit https://wi.se/rate_tracker.
Create a rate alert and subscribe by email using the exact email address associated with your Wise account. (I haven’t tried email addresses with + in them, but I assume they won’t match, even though that would be helpful.)
Wait to receive the email to activate your rate alert by confirming your email address. Visit the link in the email to confirm your email address.

This last page includes checkboxes for both email and push notification, as well as a button to unsubscribe.

Why Did This Confuse Me?

I wanted to receive rate alerts by push notification. Their Rate Tracker page claimed it was possible to do this by downloading their app. I already had their app installed on my phone, so I looked for the feature in the app. As of January 2022, it isn’t in the app. This was the first point of confusion.

I returned to the Rate Tracker page, where it suggested to “Join Wise” in order to get push notifications. I already have a Wise account, so I tried logging in, believing that they would serve a different Rate Tracker page to logged-in users than to anonymous browsers. This changed nothing. This was the second point of confusion.

I decided to try again by subscribing by email, in case that offered a clue about what was happening. I chose an email address that makes it convenient to filter incoming messages, because that lets me add a filter that would give me an email notification. This was not the email address associated with my Wise account, because nothing on any of these pages suggested that that was needed. Nowhere did I see anything in these repeated attempts that even hinted at being able to get push notifications. This was the third point of confusion.

I wanted to receive rate alerts by push notification, but I had to subscribe by email first. This is just not how I was thinking at the moment. I didn’t want email alerts; I wanted push notifications in the app. There were no words on any page that I encountered that promised push notifications at the end of the “subscribe by email” flow, so I didn’t know that I needed to do that, let alone subscribe to a rate alert with exactly the One True email address that Wise was expecting me to use.

I have asked Wise to improve the flow, which would make this article obsolete.

Stop Using My Microphone As a Speaker!

Thu, 20 Jan 2022 00:00:00 +0000

When I connect my microphone over USB to my Pop!_OS machine, the OS insists on choosing the microphone both as input device (yes!) and output device (why?!?!). This forces me to switch the output device to my headphones, earbuds, or even merely the built-in speakers. Yes, there are ways to make that easier, but I’d rather not need to do it at all.

A Solution

Use pavucontrol (PulseAudio Volume Control) to configure the device as only an input device.

You might not even have this installed, so you might need to install it.

$ sudo apt-get install pavucontrol
[Evidence that the OS is installing it...]

Run pavucontrol and go to the Configuration tab.
Connect the microphone. See the microphone show up on this tab. It’s probably using the “Analog Stereo Duplex” Profile.
Select the “Analog Stereo Input” Profile.

That’s it! Now, when you connect the microphone next time, only the “Input Device” changes. The “Output Device” stays as it was. Victory!

Install Microsoft Office 365 Inside Virtualbox

Sun, 29 Aug 2021 00:00:00 +0000

First things first: I did not want to do this at all. I hadn’t touched Windows at all since Windows 7 and for good reasons. This experience merely affirmed that decision, but I digress.

The Symptom

When I tried to install Microsoft Office 365 applications in Windows 10 within Virtualbox, the installer got stuck at “We’re getting things ready…”. It got stuck at not always exactly the same spot. To recover, I would have to kill Virtualbox and try again.

A Solution

I managed to get past this by doing the following:

Download the offline installer for Microsoft Office 365. (Maybe this isn’t necessary, but it worked.) This is shipped as a disk image. You’ll find this under “Other options” when you download the installer.
Mount the disk image.
Browse inside the disk image using File Explorer.
Ignore the installer program (called “Setup”) at the root of the disk image. (It’s a trap.)
Enter the folder called “Office”. There are two installers in this folder: a 32-bit one and a 64-bit one.
Do not yet run the 64-bit installer by merely double-clicking on it. It’s called “System64”. Instead, run this installer in “Windows 8 Compatibility Mode” and “Run as administrator”. You’ll find Compatibility Mode inside the installer’s Properties. You’ll find “Run as administrator” by right-clicking the installer.

This seemed to work.

Why?!

Due to various accidents of history, I have a QuickBooks backup file. Microsoft Excel alleges that it can import the backup file so that I can work with the data. Why QuickBooks? Because I ran Windows in 2001 when I opened my company and that’s what my then-bookkeeper used.

References

https://ilmubersama.com/2019/07/29/office-365-installing-stuck-at-were-getting-things-ready/. It’s in Indonesian, but it gave me the hint of running the installer in Windows 8 Compatbility Mode and as Administrator.

Your git log output as CSV

Wed, 11 Aug 2021 00:00:00 +0000

I use git to track cash expenses. (Don’t you?)

No, really. I track cash expenses using my phone by adding empty commits to a git repository, then pushing. (That’s my “sync” feature.) I still haven’t found a better way to automatically timestamp a bunch of simple one-line text entries that I can quickly sync off site. I use git with Gitlab.

When I want to collect that information to process it, I need the output in a more convenient format than the default git log format. For that, I combine git log with jq to make magic happen.

$ git log master...offsite-backup/master --format='{"date": "%aI","transaction": "%s"}' | jq --slurp '. | sort_by(.date)' | jq -r '. [] | [.date, .transaction] | @csv' > ~/Downloads/cash-transactions.csv

I can make that easier to read.

$ git log master...offsite-backup/master \
    --format='{"date": "%aI","transaction": "%s"}' \
    | jq --slurp '. | sort_by(.date)' \
    | jq -r '. [] | [.date, .transaction] | @csv' \
    > $HOME/Downloads/cash-transactions.csv

In case not all this feels familiar to you yet:

Give me all the commit comments from my offsite backup. Before this, I did git fetch offsite-backup to download the transactions.
Format the commits as a stream of JSON objects (in preparation for jq to process them), with the date in ISO 8601 format (is there any other format?) and using the entire commit comment as the transaction’s description.
Slurp the stream of items into a JSON array, then sort them by date (ascending, by default). Maybe I could have asked git log to sort the commits in ascending order by commit date, but I didn’t try. That would eliminate a step.
Grab the JSON output again, this time in “raw” mode, so that I can format it as CSV. (I don’t know why yet; I’m just doing it.) First, map the JSON array back into a stream of rows, then map each object into an array of values, then format as CSV. In order to format as CSV, jq wants a stream of rows, each row as an array of values.
Pipe the output to a CSV file so that I can more-easily import that into Libreoffice Calc.

There you have a very tiny introduction to jq! I find it very handy and it helps me do things that I used to do by writing little Ruby programs. It’s my go-to tool for processing structured data and I love it especially for how it turns unstructured output from standard bash tools into structured data more suitable for complicated processing.

So… go learn about jq! Set aside an hour. You’ll be glad you did.

Roll back a branch at Github

Thu, 01 Jul 2021 00:00:00 +0000

This is one of those things that’s obvious once you see it, but perhaps not obvious when you’re looking for it.

Let’s say you find yourself in an environment where you don’t have easy access to a shell, such as working on a project using Code Sandbox, Codepen, or some other in-the-browser cloud-based Javascript programming environment. It’s common in these environments to be able to commit changes to a Git repository—indeed, that might be the only way to publish and test changes to your project.

Now imagine you’ve spent two hours trying to figure out why something’s not working and you’ve given up for today. Now you have 38 extra commits that you’d like to delete, but you only have access to your programming environment and https://github.com. What do you do?

If you had a shell, you’d type something like git reset --hard HEAD~38 and be done with it, but you can’t do that in Code Sandbox nor at Github. What now?!

A Solution

Refactor your Git branches at Github. No, really.

Visit Github, specifically the Branches in your project.
For safety, create a branch level with your current working branch (which I’ll call main). I’ll call the newly-created branch old-main-delete-me.
If main is the default branch, and it often is, then change the default branch to old-main-delete-me.
Now that main is not the default branch, delete branch main.
Return to your project’s code, then visit the Commits.
Select the commit you’d like to roll back to, which I’ll call the “target commit”. In our example, that’s 38 commits before old-main-delete-me.
Create a branch called main at the target commit.
Change the default branch back to main. Now is a good time to verify that main correctly points to the target commit.
Delete the branch old-main-delete-me. Wait for Github to throw away your abandoned commits, whenever it gets around to doing garbage collection in your repository.

And now main points 38 commits earlier than it used to. Enjoy!

Change the Default Font in Master PDF Editor

Thu, 01 Apr 2021 00:00:00 +0000

If you use Master PDF Editor on Linux, then don’t use Noto Sans as the font for Typewriter comments/annotations, because the font will sometimes render incorrectly. This isn’t just a rendering problem in the viewer, but a font problem in the document itself. (I see the same disfigured text when I view the document in Zathura.) Changing the default font is not obvious from the Settings UI, so do this:

Open Tools > Settings.
From the menu of option categories, choose Comments.
In the unnamed drop-down box, choose Typewriter.
Under the settings for Text, choose a different font. I like “Inter”, but that font just happens to work.

You know that you have this problem if you enter a Typewriter comment that contains the character sequence “fi”. I infer that there is some font ligature rendering problem with the default font Noto Sans in Master PDF Editor.

The Details

When I switched from Mac OS to Pop!_OS, I worried about how I would replace the venerable PDF Pen. I love PDF Pen. Since PDFs lie at the center of my company’s record-keeping system, I needed a solid and quick-to-use alternative. When I learned about Master PDF Editor, I felt encouraged. It seemed to have all the features I needed and I learned fairly quickly how to use it. It has the typical user experience quirks that one expects from Linux software, but it performs very well and has done everything I’ve needed it to do.

Imagine my chagrin recently, when I noticed a font rendering problem when I added a text annotation to a document. It took me a few weeks to notice a pattern: it happened whenever I wrote a word that contained the sequence “fi”. Ah, yes! I presume that this is a font ligature problem. (I don’t know that it’s a font ligature problem, but it seems like a reasonable explanation.) After a few months of noticing this problem and working around it (by restricting myself to words without “fi” in them!), I finally figured out that this problem lies in the default font that Master PDF Editor uses for such annotations: “Noto Sans”. And maybe in other fonts. I haven’t tested extensively.

It’s important to know this: I don’t know fonts. I am no expert in fonts. I don’t know exactly where the problem lies. I don’t know whether to blame Master PDF Editor or the font or some strange interaction between the two. I only know this: when I type a word containing “fi” in a Typewriter comment in Master PDF Editor using the font Noto Sans, I get something that looks like this:

Those alternating “fi” strings should all look the same and, you know, sensible. And this isn’t just a rendering problem when I view the document in Master PDF Editor, but the problem lies in the document. When I open that document in another PDF viewer, I see the same ugly text.

I have notified the nice folks who produce Master PDF Editor and they have told me that they intend to fix the problem, likely by choosing a different default font for Typewriter comments. I also asked them to improve the Settings UI to add some kind of label or explanation for that unnamed drop-down box.

Do you understand what’s going on here? Do you have the energy to figure it out? If you do, then feel free to let me know. I’m not burning to understand it, but I would probably find an explanation amusing.

Linux Sound and USB Microphones

Wed, 10 Mar 2021 00:00:00 +0000

A few weeks ago, I was minding my own business, furiously trying to get ready for a Zoom meeting. I was moving USB sound devices around when, suddenly, without warning, Pop!_OS no longer detected my USB microphone. No sound.

Great.

Unplug from the USB hub, then plug into the computer directly. No. Unplug from the left-side USB port and plug into the right-side USB port. No. Plug into each and every port in the USB hub. No.

Great.

Reboot.

Plug in the microphone directly into the computer on the inconvenient-for-my-setup right-side USB port.

Yes! Sound!

I had my meeting over Zoom. All was well.

But what had happened?!

Recreating the Problem

Recreating the problem took a couple of weeks. I noticed this problem intermittently. My favorite! Eventually I looked straight at the error message in syslog.

D: [pulseaudio] module-udev-detect.c: /dev/snd/controlC1 is accessible: yes                                                             
D: [pulseaudio] module-udev-detect.c: /devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/sound/card1 is busy: no                         
D: [pulseaudio] module-udev-detect.c: Loading module-alsa-card with arguments 'device_id="1" name="usb-Blue_Microphones_Yeti_Stereo_Microphone-00" card_name="alsa_card.usb-Blue_Microphones_Yeti_Stereo_Microphone-00" namereg_fail=false tsched=yes fixed_latency_range=no ignore_dB=no deferred_volume=yes use_ucm=yes avoid_resampling=no card_properties="module-udev-detect.discovered=1"'                        
D: [pulseaudio] reserve-wrap.c: Device 'Audio1' already locked.
E: [pulseaudio] module.c: Failed to load module "module-alsa-card" (argument: "device_id="1" name="usb-Blue_Microphones_Yeti_Stereo_Microphone-00" card_name="alsa_card.usb-Blue_Microphones_Yeti_Stereo_Microphone-00" namereg_fail=false tsched=yes fixed_latency_range=no ignore_dB=no deferred_volume=yes use_ucm=yes avoid_resampling=no card_properties="module-udev-detect.discovered=1""): initialization failed.
I: [pulseaudio] module-udev-detect.c: Card /devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7:1.0/sound/card1 (alsa_card.usb-Blue_Microphones_Yeti_Stereo_Microphone-00) failed to load module.
I: [pulseaudio] client.c: Created 3 "Native client (UNIX socket client)"

OK. “Device X already locked.” This looks like a problem. “Failed to load module.” This looks like a problem. Fine.

Search the web.

Nothing useful.

OK.

After a few days, I could reproduce the problem reliably and I could work around the problem by rebooting. I didn’t even care about fixing the problem so much as finding a workaround that didn’t require rebooting. I wasn’t asking much.

Ask the Right Question

I asked on askubuntu, but they told me that I was in the wrong place, because I have Pop!_OS, which is essentially Ubuntu, but not exactly Ubuntu. Fine. I moved my question to a better place.

Here is the question: https://unix.stackexchange.com/q/635813/256763

And fortunately, beautifully, I received a useful answer one day later.

Stop Pipewire. Restart pulseaudio. Gather wood. Check the traps.

$ systemctl --user stop pipewire.service
$ systemctl --user restart pulseaudio

According to Paul Medynski:

It appears to be a race between pipewire and pulseaudio. I don’t know anything about pipewire, but it sometimes seems to grab a lock on the USB audio device before pulseaudio can access it.

Fine.

$ systemctl --user disable pipewire.service
$ systemctl --user disable pipewire.socket
$ systemctl --user restart pulseaudio

And now all is well. I just tested by unplugging then plugging in my Blue Yeti microphone 6 times in 30 seconds. All is well.

I have sound.

Thank you, Paul. I appreciate you for taking the time to answer my question.

A Puzzle about HTML: <details> and flexbox

Mon, 01 Mar 2021 00:00:00 +0000

This is a puzzle, rather than a solution. I publish this here in hopes that a web search engine crawls this article, someone finds it, knows the answer, and takes a moment to tell me. Will you please be that person?

Problem

I have recently become enamored by the HTML 5 <details> element. I like that it allows me to elide certain details on a page until the reader wants to read them. I use it to make my pages more easily skimmed, hoping that that confers some benefit to the reader. I’m trying to be nice and hoping that it helps convert some readers into customers. (I have to feed my family.)

The structure of a <details> element is this:

<details>
  <summary>A summary that readers always see.</summary>
  [... various HTML elements that the reader doesn't see until they click on the summary, such as <p>s ...]
</details>

As you can imagine, I think of the content inside the <details> element as text like any other on the page, so I’d like to style it similarly. Routinely, I use Flexbox’s gap property to create interparagraph vertical spacing throughout the document. This works as I’d expect in general, but not inside a <details> element.

If I style the <details> element as flex, then the gap property doesn’t apply to the children of this element. Instead, I have to do this:

<details>
  <summary>A summary that readers always see.</summary>
  <section class="the-details">
      [... various HTML elements that the reader doesn't see until they click on the summary, such as <p>s ...]
  </section>
</details>

If I style section.the-details as flex, then the gap property applies to its children as I’d expect.

Why?

I’ve provided an example with Codepen that illustrates the problem.

If you know why this happens, then please tell me. Scroll down to see various ways to contact me.

Many thanks.

I Want To Listen With My Headphones, Not My Microphone!

Tue, 23 Feb 2021 00:00:00 +0000

I love my Blue Yeti Microphone, but it creates problems for me when I connect my audio devices to my laptop for video conferencing. I often have to fiddle with the Linux sound settings after I connect everything in order to fix the interesting assumptions that Pop!_OS has made about which audio devices perform which jobs.

If I don’t pay attention, then I can’t hear anything through my Bluetooth headset, which annoys me at the best of times, but embarrasses me (just a little) when my client sees me fumbling at the beginning of a session.

Solution

Remember that the last connected device wins. This means connecting the devices in this sequence:

Blue Yeti Microphone
Bluetooth headset (SX-991, in case you wondered)

That’s it. If I connect them in the other sequence, then I get no audio through my Bluetooth headset.

Explanation

The Blue Yeti Microphone also acts as an output device (speaker, headphones), because it contains a built-in monitor for the microphone. I can connect wired headphones to the microphone to hear how I sound through the microphone while I’m speaking. I don’t use this feature, but I have it available. Accordingly, the Blue Yeti Microphone advertises itself to the operating system as both an input and output device, so Pop!_OS chooses it as an output device when I connect the microphone by USB.

Worse, sadly, Linux doesn’t always detect the microphone correctly, leading to other problems. Evidently, audio on Linux is hard. I accept it as it is and I forgive it.

How To Resize A Comment Box In LibreOffice Calc

Mon, 08 Feb 2021 00:00:00 +0000

I did not expect this to stymie me today.

Sorry, your browser doesn’t support embedded videos.

No matter how much I try to drag, I can’t resize the comment. What?!

Solution

Show the comment. Really. After I show the comment, I can resize it. I can’t resize the comment while I’m editing it, but I can resize the comment while I’m merely looking it.

Annoying. I’m going to forget this. When I’m editing the comment, the icon suggests to me that I will be able to resize the comment box.

As usual, I published this so that I would find it when later I have the same problem.

Reference

OpenOffice forum, https://forum.openoffice.org/en/forum/viewtopic.php?f=9&t=63642. It looks like LibreOffice has the same behavior.

Print Sensibly to PDF in Firefox

Fri, 08 Jan 2021 00:00:00 +0000

I use HTML and CSS as my word processor. No, really. I started doing this about two years ago and I really like the simplicity of writing in Markdown, processing that as HTML, then producing documents that I can distribute either as web pages or as PDF. I could even dump them into a Leanpub book if I wanted to. I compose in Typora, do power editing in Kakoune, use Jekyll and pandoc for production, then Firefox to preview. Finally, I can use Firefox to print to PDF. I spent hours crafting a print CSS stylesheet that I can feel good about.

Today, however, I ran into trouble: the header were suddenly flush with the edge of the page. This image illustrates what I wanted compared to what I got.

My “word processor” produces the document, but Firefox adds the header and footer. Setting the margins for the document was easy. Setting the margins for the header was… not obvious.

No options in the Print Preview. No options in the Print… dialog. Nothing. Srsly!

The Solution

Evidently, one must change configuration settings deep in the bowels of Firefox. You can read more at https://support.mozilla.org/en-US/questions/1061569.

In Firefox, visit about:config.
Accept the risk of changing the complicated and scary configuration settings.
Search for preferences matching the text print.Mozilla_Save_to_PDF, since Mozilla recently (in mid-January 2021) changed the name of this device from “Print to File” to “Mozilla Save to PDF”, at least on Linux.
Look for four configuration settings whose names match the pattern print.Mozilla_Save_to_PDF.print_edge_<where>, for the four areas of the page: bottom, left, right, top.
Set values for each of these to match the desired margins. (See the next section for details.)
Close any print preview and open it again. Yay!

The Print Edge Margin Units

If your paper measurement system is Colonial/Imperial/Archaic, such as “Letter (8.5 inches by 11 inches)”, then enter a margin as a multiple of 0.01 inch. For example, for a 0.5-inch margin, enter 50, and for a 0.25-inch margin, enter 25.

If you measure paper using Modern/Sensible/Metric designations, such as “A4”, then enter a margin in millimetres, like a functioning adult.

I live in Canada. Sadly, we are Colonial. Hundredths of an inch it is.

I chose left and right margins of 50/100 inch as well as top and bottom margins of 25/100 inch. Excellent! You can see the great results in the image near the beginning of this article.

Now I simply have to remember to check this if I ever need to print an article onto physical sheets of paper.

Changes in Mid-January 2021

Recently I noticed that my header and footer moved back to the edge of the page. This happened at the same time that the print dialog changed, at least on Linux. I hunted through the configuration properties and found the new properties print.print_edge_bottom (and so on for the four edges of the page) which seem to do nothing, because when I changed those values, nothing changed.

I did, however, after a while, notice another batch of “print edge” configuration settings named print.print_Mozilla_Save_to_PDF.print_edge_bottom and so on. When I changed these values, then opened a new print preview, the header and footer moved to the expected spot on the page. Success!

Since I feel lucky, I reset the properties print.print_edge_* to their default values and deleted the properties print.printer_Print_to_File.print_edge_*, since these last ones seem now obsolete. I will update this article if, in the future, these settings cause any problems.

Rename a Bunch of Files

Mon, 13 Jul 2020 00:00:00 +0000

A reader asked me simply “How to rename a bunch of files?” but left a clearly fake email address, so I couldn’t reply directly to them. Instead, I’ll reply to you, dear readers.

One can rename a bunch of files in myriad ways, using plain bash scripts, software like A Better Finder Rename (Mac OS), and tools like zmv. Most people satisfice here: they learn one way to do it that works well enough that they can perform effortlessly enough, then they just use it. I learned another way to do it in the past year and this has become my current favorite: using the bulkrename command in ranger, a file manager that feels like vim.

My Solution

Open ranger on a folder.
Mark the files to rename.
Enter the command :bulkrename. This opens an editor with each file name on a separate line.
Edit the file. The first line becomes the new name of the first file, the second line becomes the new name of the second file, and so on.
Save the file and exit the editor. This opens and editor with a summary of the rename commands that correspond to the changes I made to the filenames.
Save the file and exit the editor once more. This renames the files.

This allows me to bring the full power of my editor to renaming files: creating sequence numbers, matching regular expressions, whatever I need. I can even use some general algorithm to rename 90% of the files then patch the other 10% by hand. I love the flexibility of it and you might, too.

Enjoy!

Would You Rather Watch?

TuneIn Radio Stops Streaming After a Minute Or So

Sun, 23 Feb 2020 00:00:00 +0000

I had to convince my Android phone that it had permission to let TuneIn Radio continue to stream music even after my screen went dark from inactivity. I don’t know what changed recently that caused me to need to do this, but I needed to do it.

My Solution

Right now, my Galaxy Note 8 runs Android 9, so these instructions might have become out of date by the time you read this. I hope they still help you.

Open Android settings.
Choose Apps.
Choose the app TuneIn Radio.
In the Usage section, choose Battery.
Notice that the option Allow background activity is disabled. Enable it.

That’s it! Now TuneIn Radio continues to run in the background, even when I lock my device.

Linux, snap, and Mysterious File Permission Errors

Wed, 30 Oct 2019 11:03:00 +0000

I figured this out, but it took me about 2 hours to think of it. I hope that by writing this, you will not need 2 hours to figure it out.

The Symptom

I installed a snap package. I tried to run it. I saw this:

PermissionError: [Errno 13] Permission denied: '/path/to/the/file/in/question.blah'

File permission error. Fine!

$ chmod -R 777 /path/to/the/file/**

Try again… and Permission denied again.

I checked the ownership and permissions of every directory in the path of this file. The correct user owned them all; they all allowed read, write, and execute. (Of course, I kept track of all the directories and files for which I did this, because I wanted to undo all that once I figured out what was happening!)

The Problem

I had to give the snap package explicit authority to touch the files on my removable media devices, because…

I have two physical drives in my laptop: one for booting and the other for extra storage.
I have symbolic links from my boot drive (where /home is located) to most of my data (on /mnt/extra-space), including the files that this snap package was trying to operate on.
I installed the tool in question as a snap instead of as a conventional Linux package (or whatever one correctly calls a “non-snap” package).

Even though all the file permissions looked correct on zsh/bash, the snap was not allowed to touch those files, and the resulting Permission denied error looks the same in both cases. Great!

The Solution

$ sudo snap connect $SNAP_NAME:removable-media

I read this as “Please allow the snap named $SNAP_NAME to talk to removable-media”. I consider the word “connect” as out of place here. If I’d known the technical terms involved, then I’d have had better keywords to search for and I’d have spent less than 2 hours figuring all this out.

Why `sudo`?

This is because you need to prove you have system snap modification rights on your machine. You can put your user in the ultra restricted circle of snap administrators on your system by logging in.

For this, just create a snap account if you don’t have one already on https://login.ubuntu.com/. Take the time to do this, we’ll wait!

—“Advanced snap usage”, a tutorial from Ubuntu.

No, thanks. This feels weird to me: authenticating with some server outside my machine in order to establish that I have permission to do something on my machine. Surely I should be able to give myself the necessary authorization without creating an account on some web site somewhere!

`devmode`! I wouldn’t.

You could install the snap in devmode and move on with your life, but it always makes me nervous when I give a tool more permission than it needs, particularly when I don’t really know what I’m doing. Unintended consequences and all that.

An Explanation

By default, snap confines its packages to only touch your $HOME directory. The snap packages operate by default in a sandbox environment designed to limit damage to your system. I really like this! Since I want this particular snap to write to a directory on my secondary drive, which is mounted somewhere on /mnt, I need to give that snap explicit permission to connect to the removable-media slot.

If you’re not sure which permissions/connections your snap has, then list them this way:

$ snap connections $SNAP_NAME

The snap environment connects to some slots automatically, then marks as manual the connections that you create yourself. Clear enough.

I hope this helps you get past your immediate obstacle.

References

Read more at https://ubuntu.com/blog/a-guide-to-snap-permissions-and-interfaces if you have the combination of interest and energy. This guide is slightly out of date already (interfaces have been renamed as connections), but I understood enough.

I found a tutorial that explains more about connections at https://tutorials.ubuntu.com/tutorial/advanced-snap-usage, which I’m reading now. I seem to understand what it tries to explain to me, which makes me feel encouraged. Sadly, this tutorial also refers to interfaces rather than the modern name connections, but the information seems accurate, at least as far as I can tell as a novice.

Linux, Firefox, Google Hangouts, and 'Camera Not Found'

Thu, 20 Jun 2019 16:30:00 +0000

I tried to join a Google Hangouts meeting and my laptop’s webcam didn’t turn on. The settings in Hangouts didn’t let me choose a camera, instead reading “Camera Not Found”. This happened in Firefox, so I tried Chrome, with the same result.

Was the problem with Firefox? Chrome? Linux? Hardware?!

Fortunately, I found a straightforward solution: I had accidentally deactivated my webcam. Stupid me. Press Fn+F10 to reactivate it.

On my laptop’s keyboard, if I press Fn+F10, this toggles the webcam on/off. I presume that this makes it easy to cut the webcam very quickly. I like the feature, but I had never paid attention to it, and since F10 sits beside F9, and F9 is “increase brightness”, I probably accidentally grazed F10 and got no visible feedback that I had done it, and then I’d forgot about it until I tried to use the camera.

I looked around for commands to help me detect that I had pressed Fn+F10, but I couldn’t find any. Please tweet me if you know how to do that! Not modprobe [-r] uvcvideo, which seems to provide another layer of activate/deactivate. If I press Fn+F10, then lsmod still lists the webcam device, so that doesn’t help me. Maybe the Fn+F10 key combination is a hardware switch in the laptop somewhere? I’d like to learn more.

Copy/Paste, Byobu, Scrollback Mode

Mon, 17 Jun 2019 00:00:00 +0000

I hope that you find this article when you have the same trouble that I had figuring out how to copy text to the system clipboard on Linux in Byobu’s scrollback mode.

What To Do

Enter Scrollback mode by pressing F7.
Scroll until you can see the text that you’d like to copy.
Turn Mouse mode to on by pressing Alt+F12. You’ll see “Mouse: ON”
Hold Shift and use the mouse/trackpad to select the text that you’d like to copy. You’ll see the selected text highlighted. Press Shift+Ctrl+c to copy. If you have a system clipboard viewer, then verify that the selected text has reached the clipboard.
Turn Mouse mode back to off by pressing Alt+F12 again. You’ll see “Mouse: OFF”
Leave Scrollback mode by pressing Ctrl+c.

Yay!

Holding Shift while selecting the text seems to be the critical part. Without Shift, Byobu (screen?) pastes the text immediately then discards it, rather than pasting it into the system clipboard. You can imagine how frustrating that was for me.

Reference

Stack Overflow, “Straightforward way of copying text from byobu to system clipboard?”. Thank you to user gcuendet. This told me that we need Shift when selecting the text to copy.

Use a Fujitsu ScanSnap Scanner With Linux

Thu, 17 Jan 2019 00:00:00 +0000

I used to rely heavily for my Fujitsu ScanSnap S510M scanner, but by 2023, I almost never use it. I use scanning apps on my Android phone to scan documents, and since I don’t scan long documents (more than a few pages), this suffices for my needs. As a result, I was surprised when I received an email from a concerned reader who told me that they couldn’t follow these instructions to install XSane. I hadn’t used XSane for about two years, but I remembered using it and it worked fine.

What Happened?

I don’t know. It appears that XSane has entered some kind of in-between state as a project.

If you don’t know—and I barely know—XSane provides a graphical application for using the sane libraries to drive scanners, such as the S510M. In typical programmer style, the community refers to these applications as “front ends”, so if you see that term, simply substitute the term “applications” (or even “apps”) that typical, everyday people would tend to understand. It’s enough to know that computers use sane to talk to your scanner and that applications such as XSane make it easier for humans to use sane.

Somewhere along the way, XSane’s web site began to refer to a German-language site at ubuntuusers.de’s Wiki. Fortunately, I read just enough German not to be frightened by this, and when I scrolled down to the bottom of the page, I found a link to an English-language snapshot of the documentation along with the simple comment “The original site is no longer online”. I also found a link to the source code, which shows no new commits for approximately 1 year. From what I’ve been able to understand so far, XSane continues to work, but there are no more friendly instructions for installing it.

Maybe this means that the following instructions are the only English-friendly ones on the web right now. How nice! And as far as I can tell, they still work, so if you try them, please let me know if you notice anything that I need to change in what follows.

The biggest difference in my own practice is that I use nala instead of apt as my day-to-day package manager on Pop!__OS; otherwise, if I reinstalled XSane now, I’d expect it to go something like…

The Original Instructions from 2019

I love my Fujitsu ScanSnap S510M scanner, mostly because it just always works. It almost never jams. I find it very easy to operate. It makes scanning longer documents very easy. Even though I always have the backup option of photographing pages and processing them with Mobile Doc Scanner for Android, I still occasionally receive full-page mail and the ScanSnap scanner makes processing those pages really easy.

I originally bought the S510M to work with my MacOS machines, but now that I’m trying to move entirely off MacOS, I’d like to make my scanner work on Linux. After some light web searching, this seems possible with some combination of the built-in SANE scanning software and with a custom package called VueScan (not FOSS). How do we get these things to work?

Software Included In Pop!_OS/Ubuntu

I start at https://help.ubuntu.com/community/Scanners and look at the scanners that SANE (Scanner Access Now Easy) supports. According to http://www.sane-project.org/sane-mfgs.html, SANE supports the Fujitsu ScanSnap S510M “completely” (I don’t yet know how they’re judging “completeness” here, but it sounds promising). This table provides some information that I will need later, so I make a note of it:

USB ID: 0x04c5/0x116f
Backend: fujitsu (133)

Now, how do I try this?

Basic Setup

I follow the instructions at https://help.ubuntu.com/community/sane and do the following.

Interlude: Something is Wrong as of June 2023

The instructions above don’t work, because they suggest installing the package libsane-extras, which is not found in the standard Ubuntu/Pop!_OS package repository. I’ve posted a question at the Unix & Linux StackExchange, hoping for a useful suggestion. Maybe you’ll find a useful answer by now!

Returning to Basic Setup

[As of June 2023, this command fails, because the package libsane-extras is not found. I don’t know how to fix this problem. Good luck.]

$ sudo apt-get install sane sane-utils libsane-extras xsane

A Detour Into Gimp Configuration Files

When installing these packages, apt asks me about changing some configuration files related to Gimp. Since I don’t qualify as a Gimp power user, I probably don’t care too much about the contents of these configuration files, but I might learn something by reviewing these changes, so I look at them one by one. This also gives me a chance to learn a little bit more about the details of installing packages with apt.

First, comes the configuration file /etc/gimp/2.0/controllerrc. All the changes in this file appear to relate to keyboard shortcuts that I don’t rely on, so I feel safe in accepting the changes. I have no muscle memory with Gimp to interfere with.

`/etc/gimp/2.0/controllerrc`

< represents the configuration file “as is” and > represents the proposed changes to the configuration file. I’m no expert in Lisp, but even I can read this.

8,17c8,17
<         (map "scroll-up-alt"                 "tools-value-1-increase-skip")
<         (map "scroll-down-alt"               "tools-value-1-decrease-skip")
<         (map "scroll-up-control-alt"         "context-gradient-select-next")
<         (map "scroll-down-control-alt"       "context-gradient-select-previous")
<         (map "scroll-up-shift-alt"           "context-pattern-select-next")
<         (map "scroll-down-shift-alt"         "context-pattern-select-previous")
<         (map "scroll-up-shift-control"       "context-brush-select-next")
<         (map "scroll-down-shift-control"     "context-brush-select-previous")
<         (map "scroll-up-shift-control-alt"   "context-font-select-next")
<         (map "scroll-down-shift-control-alt" "context-font-select-previous")))
---
>         (map "scroll-up-alt"                 "tools-opacity-increase")
>         (map "scroll-down-alt"               "tools-opacity-decrease")
>         (map "scroll-up-shift-primary"       "tools-aspect-increase")
>         (map "scroll-down-shift-primary"     "tools-aspect-decrease")
>         (map "scroll-up-shift-alt"           "tools-angle-increase")
>         (map "scroll-down-shift-alt"         "tools-angle-decrease")
>         (map "scroll-up-primary-alt"         "tools-size-increase")
>         (map "scroll-down-primary-alt"       "tools-size-decrease")
>         (map "scroll-up-shift-primary-alt"   "tools-spacing-increase")
>         (map "scroll-down-shift-primary-alt" "tools-spacing-decrease")))
28,35c28,35
<         (map "cursor-up-control"    "view-scroll-top-border")
<         (map "cursor-down-control"  "view-scroll-bottom-border")
<         (map "cursor-left-control"  "view-scroll-left-border")
<         (map "cursor-right-control" "view-scroll-right-border")
<         (map "cursor-up-alt"        "tools-value-1-increase-skip")
<         (map "cursor-down-alt"      "tools-value-1-decrease-skip")
<         (map "cursor-left-alt"      "tools-value-1-decrease")
<         (map "cursor-right-alt"     "tools-value-1-increase")))
---
>         (map "cursor-up-primary"    "view-scroll-top-border")
>         (map "cursor-down-primary"  "view-scroll-bottom-border")
>         (map "cursor-left-primary"  "view-scroll-left-border")
>         (map "cursor-right-primary" "view-scroll-right-border")
>         (map "cursor-up-alt"        "tools-size-increase-skip")
>         (map "cursor-down-alt"      "tools-size-decrease-skip")
>         (map "cursor-left-alt"      "tools-size-decrease")
>         (map "cursor-right-alt"     "tools-size-increase")))

Back to my terminal session:

Configuration file '/etc/gimp/2.0/controllerrc'
 ==> File on system created by you or by a script.
 ==> File also in package provided by package maintainer.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** controllerrc (Y/I/N/O/D/Z) [default=N] ? Y

Now the same for /etc/gimp/2.0/gimprc. I see a large number of changes, but every single one involves a comment, so I see no harm in accepting the new version, so I do that by answering “Y” again.

Next, /etc/gimp/2.0/sessionrc. By reading the preamble comment, I feel comfortable just accepting the new version, so I do that by answering “Y” again.

`/etc/gimp/2.0/sessionrc`

# GIMP sessionrc
# 
# This file takes session-specific info (that is info, you want to keep
# between two GIMP sessions). You are not supposed to edit it manually, but
# of course you can do. The sessionrc will be entirely rewritten every time
# you quit GIMP. If this file isn't found, defaults are used.
[... session settings that you don't need to see here...]

Next, /etc/gimp/2.0/templaterc. This appears to have a list of templates of some kind. Since I don’t use Gimp templates (yet?), I see no harm in accepting the new version.

This appears to conclude the configuration-file surgery portion of this installation.

Back To Installing SANE

When installing completes, I see evidence that apt has installed all the packages, as well as that etckeeper has committed and pushed the changes to /etc. Unfortunately, then I see this:

W: Operation was interrupted before it could finish

After reading a few Stack Overflow answers and forum posts, W means “warning” and the message seems likely harmless. Just to check, I re-install the packages, and this operates without any warnings. I wonder whether, somehow, my exploring the configuration file differences resulted in this warning. I suppose I’ll never know.

All this merely completed step 1 at https://help.ubuntu.com/community/sane.

On to step 2: I connect my scanner by USB, then launch XSane, and XSane detects the scanner as “ScanSnap S510M”! This might just work!

Scanning A Page

I put a page into the scanner, then press the button Acquire preview on XSane. I haven’t changed any configuration settings yet. XSane scans the page and I see a preview image, which doesn’t look great, but shows evidence that I can scan a page. Unfortunately, when I try to press the Scan button on the main XSane window, I get an error: “Failed to start scanner: Invalid argument”.

So I can acquire a preview image, but I can’t scan a page “for real”? What does that even mean?!

When In Doubt, Try It Again

I put the page back into the scanner, then pressed Acquire preview again. After that, I pressed Scan and got a different error telling me that the document feeder was out of pages, so I put the page back into the scanner, then pressed Scan, then I get a scanned document as PDF!

Well… sort of. When I open the PDF, it looks awful. It definitely doesn’t look like the crisp, sharp scans that I’d grown accustomed to when scanning a page on MacOS. Clearly, I have change some settings in order to improve the scan quality, but of course, I don’t know which settings to change, so I get to learn more about scanning quality than I ever wanted to know.

Changing Scanning Settings

I see that the scan resolution is set to 50. I assume that’s 50 dots per inch (DPI). That seems really low. I look at the resulting PDF file, which has a size of 82 kB, and these two observations together seem to match. Let me try 300 DPI, which seems like overkill, but at least would show me a stark difference between the two scans, if it behaved as I expect. I set the scan resolution to 300, then scan again.

When I scan the page at 300 DPI, I notice two things:

XSane shows me “2550*3300*8 (8.0 MB)”, which I interpret as 2550 pixels by 3300 pixels by 8 bits of color for a total page size of about 8.0 MB. This sounds like overkill and should look much sharper.
When I scan the page, it takes a lot longer to scan. This seems to correspond to the ScanSnap Manager’s various scanning settings of “Better, Best, Fastest” and so on. This setting seems to correspond to something like “Better” or even “Best”.

The resulting PDF has a size of 2.2 MB, and not 8.0 MB, but that seems like it would look sharper. When I open it in my PDF viewer, it indeed looks very sharp! I imagine that I could now just resample it to lower resolution if I wanted to. Perhaps it has become time to learn how to use Gimp “for real”.

Save The Settings!

Before I play around with the various scan settings to find what I really want, I save the device settings so that I don’t lose them, using the XSane menu Preferences > Settings (Ctrl+P).

Now I can safely play around with the scanning settings and save them as various profiles. Nice!

Fixed: I Couldn't Open A Terminal Window!

Thu, 17 Jan 2019 00:00:00 +0000

I just had quite a nice scare: I was installing some packages on Pop!_OS 18.10 (Ubuntu), and I unknowingly caused my trackpad to stop working and I couldn’t even open a Terminal window to see what was happening.

The Symptoms

I had been installing some packages with apt. At one point, when I tried to take the next step, I tried to open a new window in Byobu Terminal. I pressed F2 and nothing happened. I tried again, and nothing happened. I couldn’t open a new window. I tried Alt+F2 and Ctrl+F2, but neither of these key combinations did anything. I closed my Byobu Terminal session and tried to open a new one, but launching Byobu did nothing. No window opened.

Oops.

Next, I tried launching the regular Terminal application. Again, no window. Nothing. I tried XTerm. Nothing. I couldn’t open a Terminal session at all! How the hell was I going to even get information to help me diagnose the problem?

I tried logging out, then logging in again. I couldn’t open any Terminal windows and, worse, I couldn’t use my trackpad! My trackpad didn’t move my cursor at all. This would make figuring out the problem very difficult.

I tried restarting the computer. No change: I couldn’t open a Terminal window and I couldn’t use my trackpad.

Getting Information

I couldn’t see any output from trying to start a Terminal session, so I had no idea how to even guess at the symptoms, let alone the cause. How could I start a shell and see the result of trying to start one? And how do I get around my windowing environment without a trackpad?

Fortunately, I had a USB mouse nearby, so I plugged that in and it just worked. This meant that I could at least browse the web on this computer to figure out what else to do. I searched the web and tried using Alt+F2 to run an arbitrary command (in Gnrome 3), so I ran gnome-terminal. This didn’t open a Terminal session and it showed me no output, so I didn’t learn anything about the problem.

A Trick

In desperation, I launched GVim, then tried using the :! command to run a shell. This finally gave me a clue:

error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

Aha! apt fooled me.

The Trouble With `autoremove`

I had been trying to install some printer-related packages, and in the process of uninstalling them (they didn’t do what I needed), apt told me that it could autoremove these packages:

libncursesw5
libtinfo5

Since I didn’t know any better, I executed apt autoremove, which removed these packages, and my problems started there.

Thank You, `etckeeper`

Fortunately, etckeeper made it relatively easy both to recover and to document what went wrong. I could see a commit from when I removed the printer-related packages that I’d installed, and then a second commit from executing apt autoremove. This helped me connect the error message above with the packages that I’d removed. This gave me the idea that I could simply reinstall those packages and return everything to normal.

Opening A Shell… Any Shell

I couldn’t open my normal shell (zsh) to fix the problem, but could I open any shell? I opened the Preferences for the Terminal application, then cloned my Pop! profile in order to change the shell command that the terminal runs to /bin/bash. I set the new profile—I called it “Recovery”—as the default profile for the Terminal application, then launched it. Yes! I got a bash shell. Now I could try my idea for fixing the problem.

Even Better!

After reading https://askubuntu.com/a/809058/839487 I saw that I could have started gnome-terminal using bash instead of my shell by pressing Alt+F2 then running the command gnome-terminal -x bash. I hope I can remember this next time!

Fixing The Problem

I fixed the problem (some of it—keep reading) by reinstalling the packages that apt autoremove had removed.

$ sudo apt-get install libncursesw5 libtinfo5
[Evidence of installing various packages...]
$ sudo apt list --upgradable
[Nothing. Good.]
$ zsh
[The zsh prompt.]

I could open a shell again. Maybe everything had returned to normal. Before I did anything else, I documented what I learned where I might one day stumble upon it again: in the git log for /etc.

Documenting The Problem

I amended the relevant etckeeper commits, adding detail about what happened, as well as making it easier for the eye to notice that something interesting had happened. First, I decorated the one-line commit messages.

$ cd /etc
$ sudo git log --graph --all --oneline
* c2efaa2 (HEAD) [FIX] committing changes in /etc after apt run
* 8866206 [MISTAKE] committing changes in /etc after apt run
| * 4c984ec (offsite-backup/master, master) committing changes in /etc after apt run
| * 984c8eb committing changes in /etc after apt run
|/  
* 2945f32 committing changes in /etc after apt run
* 81410e8 saving uncommitted changes in /etc prior to apt run

The last two commits contained the mistake (apt autoremove and the fix apt-get install...). I added simple text tags to make it easier to spot them. I also added some detail to the log messages.

$ sudo git log --all
commit c2efaa27381fa9a3afcb5426c44a7577a6cc01de (HEAD)
Author: J. B. Rainsberger <me@jbrains.ca>
Date:   Thu Jan 17 10:33:36 2019 -0400

    [FIX] committing changes in /etc after apt run
    
    Reinstalling these packages fixes the previous action
    `apt autoremove`.
    
    Package changes:
    +libncursesw5 6.1+20180210-4ubuntu1 amd64
    +libtinfo5 6.1+20180210-4ubuntu1 amd64

commit 8866206fa2b6ff42a10d0d05e83da2eefaf44424
Author: J. B. Rainsberger <me@jbrains.ca>
Date:   Thu Jan 17 10:16:53 2019 -0400

    [MISTAKE] committing changes in /etc after apt run
    
    `apt autoremove` removed these packages, but after I did that,
    I could no longer open my shell (zsh), even though I could open
    bash. Trying to launch zsh resulting in an error, unable to
    find libtinfo.so.5.
    
    I don't know why this happened, but reinstalling these two
    packages seemed to fix the problem.
    
    Package changes:
    -libncursesw5 6.1+20180210-4ubuntu1 amd64
    -libtinfo5 6.1+20180210-4ubuntu1 amd64

I had documented my adventures for myself and could safely breathe, then begin describing my adventures for you, in case you should ever get yourself into a similar situation.

Verifying The Fix

I could start my shell again, but my trackpad still didn’t work, so I got all this information out of my head, then I tried to see whether my trackpad would work again, first by logging out, then by rebooting in case logging it didn’t suffice.

Curiously, when I logged out, my trackpad started to work again, but when I logged in, it stopped working. Moreover, each time I tried this, the cursor jumped to the same spot near the bottom-right corner of my display. I didn’t know what that meant yet, but it made me think that I had some configuration setting somewhere that needed to change. I hoped that rebooting would fix the problem, so I rebooted.

Sadly, that didn’t fix the entire problem. Good news: I can start my shell and Byobu. Bad news: my trackpad only works when I’m logged out. This points pretty clearly to a user-level configuration problem, which I can probably diagnose (and maybe fix) by looking at the changes to /etc or to my dotfiles when I reinstalled these packages. Fortunately, in the meantime, a USB mouse works, so I don’t have to fix this problem right now, even though I’ll want to fix it very soon.

Post Script: Fixing the Touchpad

I guess Linux calls it a “touchpad” and not a “trackpad”. Old habits die hard.

I don’t know how it happened, but somehow, in the middle of removing and reinstalling packages, the “on/off” toggle for the touchpad in Gnome switched to “off”. That’s all. I switched it back “on” through the Mouse & Touchpad settings, and the touchpad just started working again. Yay.

rvm and byobu

Sun, 11 Nov 2018 00:00:00 +0000

I use rvm and I like byobu terminal, so I want to use them together. Today, when I tried to write an article at https://blog.thecodewhisperer.com, I saw this:

$ rvm use 2.4.3

RVM is not a function, selecting rubies with 'rvm use ...' will not work.
  
You need to change your terminal emulator preferences to allow login shell.
Sometimes it is required to use `/bin/bash --login` as the command.
Please visit https://rvm.io/integration/gnome-terminal/ for a example.

I run Gnome 3 on Pop!_OS 18, so I visited https://rvm.io/integration/gnome-terminal and followed the instructions there. I opened the Preferences for my terminal application and verified that “Run command as a login shell” is enabled.

That did not seem to help.

Workaround

# In a byobu session...
# Load the script your own self!
$ source $HOME/.rvm/scripts/rvm
$ rvm use 2.4.3
Using /home/jbrains/.rvm/gems/ruby-2.4.3

Solution?

I don’t know. I suppose I get to learn about various profile scripts and how the shell loads them. I assume that one of the various profile scripts doesn’t load (“source”) the rvm script, but it should. If you know how to figure that out, then please use the contact links below to tell me or tweet a mention to be about it. I’d appreciate it.

Using git When the Firewall Says, "No"

Thu, 25 Oct 2018 00:00:00 +0000

If you find yourself on a network that blocks SSH (port 22), but you want to push some code to your favorite hosted git repository, you can probably do it. Github, Gitlab, and Bitbucket all provide an alternative git server so that you can connect over HTTPS (port 443), which such networks can’t really afford to block.

Good news: you can use an alternative SSH configuration to connect without adding different remotes to your local repository. Bad news: every tutorial I read suggests changing your global/per-user SSH configuration settings, usually in $HOME/.ssh/config. How likely do you think you’ll remember to change this back when you return to a nicer network? Exactly.

I think like a programmer. I prefer to configure systems as narrowly as possible: at the user level instead of system-wide; for an individual session instead of at the user level; for an individual request instead of for the entire session. You get the idea. I know that I can tell ssh which configuration file to use for each request. I would like to do that for git.

I can almost do this, otherwise I wouldn’t write this article.

Using `git` with an alternate SSH configuration

I created an alternative SSH configuration file, then taught git how to use that configuration file when connecting to SSH. This way, I don’t have to change the configuration for all SSH sessions by my user, but only the ones to the git hosting services’ servers.

Step by Step Instructions

Look up the HTTPS connection information from your git repository hosting service. I had to search for while, but I found that information for gitlab.com and you’ll find it in the references below.
Test that you can connect to their alternate server with ssh -T -p 443 $server_name. For Gitlab, I use altssh.gitlab.com as the server name. If you can’t connect, then I imagine you have a bigger problem and these instructions won’t help you.
Create an alternative SSH configuration file at $HOME/.ssh/config-blocked-ssh to use in situations like this.
Paste into config-blocked-ssh the connection information from your git repository hosting service. (See the References section.)
Now try to connect with ssh -T @server_name. With luck, your git repository hosting service will tell you that you connected, even if they don’t give you shell access.
In a terminal session, tell git to use a different SSH command with GIT_SSH_COMMAND="ssh -F $HOME/.ssh/config-blocked-ssh".
Now try pushing or showing a specific remote. That should just work!

Now, I can move this command into a script with a nice name that I’ll remember for the next time I find myself in this situation.

References

Stack Overflow, “Tell git which SSH config file to use”.

Achilleas Pipinellis, “Gitlab.com Now Supports an Alternate git+ssh Port”. The connection information for Gitlab. Your git repository hosting service likely offers something similar.

Installing amp, a text editor

Fri, 21 Sep 2018 00:00:00 +0000

Recently I learned about amp, a text editor that might lure me away from vim. Intrigued by the idea, I tried to install it. I really tried, but I couldn’t manage it. I learned that I had some kind of dependency problem that I could solve by using the most recent tools for the Rust programming language.

I don’t want to learn Rust (yet), but I’d like to try amp as a potential new go-to text editor. I wasn’t in the mood to learn enough about Rust tools to resolve the problem myself, so I posted an issue to github.

Fortunately, nice people gave me reasonable-sounding instructions, so I’m trying them. (Spoiler: they worked on my machine. Read on.)

The Sitch

I’m on Pop!_OS 18.04, which is mostly Ubuntu 18.04. I tried to follow these instructions, and I ended up with a compile error that seems to relate to a dependency version mismatch with libssh2-sys-0.2.11. And I literally have no idea what that signifies. I understand the individual words, but not the significance of the sentence as a whole.

According to this Rust-related issue, this might signal an out-of-date version of Rust. I assume that whatever version of Rust I have came when I installed cargo and that puts me out of rocks. I just started using cargo minutes earlier, so I don’t know how it installs a Rust compiler nor how to change it nor whether somehow, magically, at some point in the past I accidentally installed some out-of-date version of the Rust development environment….

Evidently, since I’d tried to install cargo through the Ubuntu package manager apt, I probably got an out-of-date version, which led me to this dependency mismatch. I suppose this illustrates the down side of using package managers with rapidly-evolving software. Since then, I have removed cargo from my system in the hopes of returning it to its previous state. With luck, I have nothing hanging around that will interfere with installing amp in the future.

Some Hope

I have gleaned from the comments on my amp issue the following instructions:

Install a stable version of Rust using rustup.
Install cargo, and presumably it will use the stable version of Rust that I would have just installed.
Run cargo install --git https://github.com/jmacdonald/amp, which I’d tried earlier, and which led me to this mess and to writing these words.

So let me try.

Let Me Try

I follow the instructions at https://www.rust-lang.org/en-US/install.html to install rustup, which seems to be like sdkman in the Java world, rvm for Ruby, and so on.

I see a version of Rust from literally 8 days ago. I feel very bleeding-edge.

$ curl https://sh.rustup.rs -sSf | sh
[Many instructions, some of which provide evidence of installing up-to-date rust and cargo.]

So I seem to get two for the price of one. I will need to move $HOME/.cargo to where I store my dotfiles in a git repository and then stow it. No problem. I will also need to commit changes to my shell profile scripts, so before I continue, let me check that my dotfiles repository is clean.

# In another shell
$ cd $DOTFILES_PROJECT_ROOT
$ git status -s
[No uncommitted changes.]

Since my dotfiles are clean, I can continue. I can uninstall any time with rustup self uninstall, so I note that command here in this article so that I might remember where to look for it later.

I continue with the installation using the default options, because—and I can’t stress this enough—I don’t know what I’m doing.

Back in the first shell window, where I’m installing rustup, I choose Proceed with installation (default) because it is the default, then I see evidence of installing something.

Rust is installed now. Great!

Before I go anywhere and do anything, I commit changes to my dotfiles.

$ cd $DOTFILES_PROJECT_ROOT
$ git status -s
[I see changes to .profile and .bash_profile, but strangely not .zprofile, so I guess I don't have one. I still don't understand the difference between the profile files and the env files and the rc files.]
$ git add -A
$ git commit -m "Installed Rust and Cargo with rustup."
$ git push offsite-backup
$ popd
# I have `cd` aliased to `pushd`, so every `cd` is just a `pushd`. I like it.

Next, I add cargo to my system path.

$ source $HOME/.cargo/env
$ env | grep -i cargo
PATH=/home/jbrains/.cargo/bin:[the rest of my PATH variable]

Next, I move .cargo to where I track my dotfiles, then stow it.

$ mkdir -p $DOTFILES_PROJECT_ROOT/system76-laptop/cargo
$ mv $HOME/.cargo $DOTFILES_PROJECT_ROOT/system76-laptop/cargo
$ stow -t $HOME -d $DOTFILES_PROJECT_ROOT/system76-laptop cargo
$ ls -al $HOME | grep cargo
[Evidence that $HOME/.cargo points to the expected place in my dotfiles repository.]

This requires committing new changes to my dotfiles.

$ cd $DOTFILES_PROJECT_ROOT
$ git add -n .
[I see lots of stuff that probably shouldn't be in git, because I can download it.]
$ cd system76-laptop/cargo/.cargo
$ vim .gitignore
# In .gitignore
# Stuff I can download any time
registry
bin
# A nested git repository that I'm not going to track
git
$ git status -s
add 'system76-laptop/cargo/.cargo/.crates.toml'
add 'system76-laptop/cargo/.cargo/.gitignore'
add 'system76-laptop/cargo/.cargo/env'
[MUCH better. That all looks like pure configuration files.]
$ git add -A
$ git commit -m "We can now stow Cargo."
$ git push offsite-backup
$ popd
# I have `cd` aliased to `pushdga`, so every `cd` is just a `pushd`. I like it.
$ ls -al $HOME | grep cargo
lrwxrwxrwx  1 jbrains jbrains     70 Sep 21 14:49 .cargo -> $DOTFILES_PROJECT_ROOT/system76-laptop/cargo/.cargo

Now I can try to do something with cargo.

Let’s Do Something With `cargo`

In principle, I should now be able to install amp with cargo. Let’s see.

$ cargo install --git https://github.com/jmacdonald/amp
[...]
   Compiling gag v0.1.10
   Compiling num-traits v0.1.43
   Compiling thread-id v2.0.0
error: failed to run custom build command for `onig_sys v68.0.1`
process didn't exit successfully: `/tmp/cargo-installES1HqJ/release/build/onig_sys-4b9d7f80a3b9005f/build-script-build` (exit code: 101)
--- stdout
running: "cmake" "/mnt/extra-space/Workspaces/dotfiles/system76-laptop/cargo/.cargo/registry/src/github.com-1ecc6299db9ec823/onig_sys-68.0.1/oniguruma" "-DBUILD_SHARED_LIBS=OFF" "-DCMAKE_INSTALL_PREFIX=/tmp/cargo-installES1HqJ/release/build/onig_sys-849a1e77a747337f/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_BUILD_TYPE=Release"

--- stderr
thread 'main' panicked at '
failed to execute command: No such file or directory (os error 2)
is `cmake` not installed?

build script failed, must exit now', /home/jbrains/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.31/src/lib.rs:643:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

warning: build failed, waiting for other jobs to finish...
error: failed to compile `amp v0.5.0 (https://github.com/jmacdonald/amp#07e6c192)`, intermediate artifacts can be found at `/tmp/cargo-installES1HqJ`

Caused by:
  build failed

Nope.

Interesting… “is cmake not installed?” Let’s see.

$ cmake
zsh: command not found: cmake

Fine. How do I install it, then?

Oh, boy. According to https://www.pyimagesearch.com/2018/05/28/ubuntu-18-04-how-to-install-opencv/, this will take some work.

Install `cmake`

$ python3 --version
Python 3.6.5
[OK.]
$ sudo apt-get update
$ sudo apt-get upgrade
[Installing a handful of libraries, including something related to python. Maybe that's good news?]
$ sudo apt-get install build-essential cmake unzip pkg-config
[Indeed, cmake was not installed. Now it is installing.]

That wasn’t so bad! Let’s see whether that fixes the entire problem.

$ cargo install --git https://github.com/jmacdonald/amp

This runs for a few minutes. And then it stops. And it appears to work!

Run `amp`

$ amp
[Something that looks like a text editor and an empty buffer. Yay!]

All right! Now… how exactly does one use amp?

Moving to Linux: Evernote

Mon, 17 Sep 2018 00:00:00 +0000

I store a lot of stuff in Evernote. Too much. My Evernote library has become a graveyard. Sadly, Evernote doesn’t produce a Linux client, which leads to one severe user experience problem: deleting notes.

On a desktop client, one can select many notes then delete them with a single command. One can’t do this on Android nor the web client.

On Android, one can select 20 notes at a time—I can’t begin to imagine the reason for this limitation—which doesn’t suffice when I have 210 notes to delete. Why not at least let me delete all the notes that match the current search criteria?!

On the web, one can delete a single note at a time by clicking multiple times. No.

What could possibly help here?

What Could Possibly Help Here

I search the web for delete all notes evernote linux. This leads me to http://www.linuxandubuntu.com/home/3-best-evernote-alternatives-with-official-linux-clients, which suggests NixNote. NixNote threatens to synchronize with Evernote, and this threatens to meet my immediate needs.

I learned about two versions of NixNote, and as of 2018-09-17, when I write these words, I recommend that you try the NixNote fork maintained by Github user robert7. https://github.com/robert7/nixnote2/ There, you’ll find an AppImage release. Read https://github.com/robert7/nixnote2/wiki/HowTo---Run-AppImage to install the AppImage.

The Log

I strongly recommend monitoring the NixNote logs to get an idea what’s going on, especially when you synchronize notes for the first time. I know that I prefer to see what’s happening in some detail when I use software for the first time, since I typically don’t trust it to work at all.

I found logs in $HOME/.nixnote/logs-*, where it looks like the directory name reflects some kind of account number internal to NixNote. My NixNote account (not the same as my Everote account) appears to be number 2, so that means that I’ll find the logs in the subdirectory logs-2.

$ cd $HOME/.nixnote/logs-2
$ tail -f messages.log

This helps me see that NixNote is making progress on long-running jobs. Once it finishes synchronizing with Evernote the first time, NixNote becomes much more responsive and behaves the way one would expect such an application to behave.

Synchronize

In order to connect NixNote to Evernote, I learn that I need to synchronize notes.

Tools > Synchronize (F9).
Log in to Evernote in order to authorize NixNote to talk to Evernote.
Wait. Monitor the logs.

Sadly, apart from a spinning icon, NixNote doesn’t appear to report much in the way of status. At least, I don’t notice it right away. (If one can’t find the feature, then it doesn’t exist.) I learn that I can also check on how many notes NixNote has processed with Tools > Database Status. This seems to update the number of notes that NixNote has synchronized, but in batches of 50.

After a while, I notice a status update in the bottom-left corner. I don’t know why it sometimes appears and sometimes disappears, but it reports progress as “Download 12% complete”. You can follow an open issue at https://github.com/baumgarr/nixnote2/issues/457. (It’s not clear to me whether I should have opened that issue at baumgarr’s version or robert7’s version, but robert7 has commented on it, so I think it’ll be fine.)

Backup and Restore

I want to keep my data safe, so I always look for some kind of export or backup feature. NixNote has it.

File > Backup Database. Then choose a directory and a file name for your NixNote Export file, of type .nnex. Prepare yourself for a large file: mine was 7.6 GiB.

CLI?

NixNote appears to have some kind of command-line query interface, which I haven’t explored yet. It gives me the idea that I could do some interesting things with my notes. I don’t yet know how to use the command-line interface with the AppImage version of the application, so if you know how to do that, then please let me know.

Move Your Data Directory

If you’d like to store your NixNote data elsewhere, then can do that. https://github.com/robert7/nixnote2/wiki/Relocation-of-config-and-data-directory

I Deleted 3000 Notes

I managed to delete over 3000 notes and several old notebooks. I had to go to the web application to empty the trash, but I don’t mind doing that every so often.

What If You Installed NixNote 2 and not 2.1?

When I first started, I installed NixNote from my package manager. Fortunately, if you did the same, then you can move from it to the more updated AppImage version relatively easily.

First, I had found the package and installed it.

$ pushd /etc; sudo git status -s
[Evidence of no uncommitted changes in /etc]
$ popd

$ sudo apt-cache search nixnote
nixnote2 - Open Source Evernote client
$ sudo apt-get install nixnote2
[Evidence of installation.]

Next, I’d launched NixNote and synchronized notes.

Tools > Synchronize (F9).
Log in to Evernote in order to authorize NixNote to talk to Evernote.
Wait.

After moving notes around and generally exploring what NixNote could do, I migrated it to NixNote 2.1.

Migrating To NixNote 2.1

In NixNote 2 (installed by apt), backup the database, just to be safe.
Download the NixNote 2.1 (the AppImage version).
Move NixNote 2.1 to an Applications directory, which I call $APPLICATIONS.
Create a .desktop file in $HOME/.local/share/applications to launch the AppImage. See https://stackoverflow.com/questions/12964512/creating-an-application-launcher-for-gnome-3-in-ubuntu for details.
Launch NixNote 2.1. Verify that I can see my notes and synchronize them.
$ sudo apt-get remove nixnote2 && sudo apt-get autoremove to uninstall NixNote 2.

Win!

Add a Personal Payee Your Own Damn Self

Wed, 29 Aug 2018 00:00:00 +0000

As of October 2022, these instructions no longer work. Sorry about that, folks. Maybe someone out there knows how to fix them? Please tell me.

I have bank accounts with TD Canada Trust and I occasionally need to add a “personal payee”. One used to need to call them in order to do this, but now it appears that you can do this yourself using online banking as long as you know how. This might save you some effort, especially if you dislike banking over the phone.

Adding a Personal Payee

These instructions were correct as of August 2018.

Ask your intended recipient for their TD Canada Trust banking information: transit number and account number.
Log in to TD Canada Trust online banking.
Under “Pay Bills”, look for “Pay Canadian Bills” and then select “Add Payee”.
Choose the right Access Card, in case like me, you have both personal and business access cards.
Search for a “Payee Name” of “personal payee”. This means that you’re adding an individual instead of a known vendor/company like your mobile phone provider. (I know; what a hack!)
Don’t be put off by the menacing-looking “Account Number” field. I’ve explained how to format the account number below.
Enter a human-friendly name in the “My Payee Name” field.
Press “Next”, then confirm what you’ve entered and save it.

Now you can check the list of payees for the one that you’ve just added. If you see a mistake, then press “Edit” to fix it. Now you can send this person money overnight and not pay Interac e-mail Transfer fees.

Formatting the Account Number

Your recipient will probably give you the following information:

Account Holder Name: Arbitrary Human
Transit Number: 12345
Institution Number: 004
Account Number: 9876543

Use the Institution Number unchanged as the “Bank Code”. It’s 004 because that represents TD Canada Trust.
Drop the last digit of the Transit Number and add a 0 to the front, so in this example, that becomes 01234.
Use the Account Number unchanged.
Pick a “nickname”, which is up to 5 characters long and must start with a letter. Choose only letters and numbers. I chose HUMAN for this example.

Now put these four bits of text together and your personal payee “Account Number” is 004012349876543HUMAN.

If your recipient gives you an 11-digit account number, then use only the last 7 digits. For example, 0667 refers to a small business bank account (or at least it once did).

Why Not Just Use Interac E-mail Transfer?

I like Interac e-mail transfers in some ways, but a TD Canada Trust personal payee offers a few advantages:

No need to use a password with the transfer, since some people manage to stumble with passwords, especially if you send them money more than once.
No extra fees to pay nor to keep track of. I care more about the cost of the record-keeping than the cost of the transfer itself.
Higher transaction limits. I can send CAD 5000 per day with no limit over a 7-day nor 3-day period.

Of course, if you know something that I don’t about TD Canada Trust direct transfers and you want to share it with me, then contact me through the various means on this page, because I’d prefer not know than not to know.

No, This Album Is Not A Compilation

Tue, 17 Jul 2018 00:00:00 +0000

In the process of switching from Mac OS to Pop!_OS, I liberated my music from iTunes, and then sought a music organizer for Linux. I am in the process of trying Clementine. I found it easy to import my music, but Clementine has listed some albums under “Various Artists” that I didn’t expect to see there. I know that a single artist recorded these albums, so I want them to show up where I expect to see them.

Moving the albums into different directories didn’t change how Clementine organizes the music; Clementine uses metadata in the music files themselves to organize them by album and artist. I used Kid3 Audio Tagger to change the metadata of the affected music files. Specifically, I used the command-line interface so that I could fix all my music with a single command.

Visit https://kid3.sourceforge.io/#download for installation instructions. I installed the package named kid3-cli.

I did this:

Move the real compilations to another directory somewhere out of the way, leaving a directory tree with only music that I want to mark as “not a compilation”.
Go to the root of this directory tree with cd.
Run this command.

$ find . -type f  -exec kid3-cli -c "set Compilation 0" {} \;

This sets the “Compilation” tag in each file to false.

Next, I rescanned the music library within Clementine and the albums appeared where I expected them to appear. This made me happy.

I then moved the compilations music directory back to where I store the rest of the music, and then rescanned the music library within Clementine once more. Of course, I could have figured out the find magic to exclude that one directory, but I didn’t feel like it. If I have to do this more than once again, then I’ll figure out then how to do that.

The Real Reason The WiFi Thermostats Didn't Work

Tue, 03 Jul 2018 00:00:00 +0000

Ignore everything I wrote at https://jb.rainsberger.ca/permalink/honeywell-wifi-thermostats-not-connecting-to-wifi-network about the failures with my WiFi thermostats. I got that wrong. I have it right now, and I have my wife, her Kobo, and her determination to thank for it.

Summary

The Actiontec R3000 modem/router combination device appears to have a limit of 11 WiFi clients connected at once. This will cause devices to mysteriously stop joining the network. We worked around the problem by connecting a wireless access point over a wire (Ethernet) to the Actiontec router, then connecting our devices to the wireless access point. This reduces the number of WiFi clients on the Actiontec router and the problem disappears.

No, I don’t know why the Actiontec R3000 has this limit. I find it silly. I don’t want to know why, because knowing why will only make me feel more sad.

The Details

It seemed to start about 6 weeks ago when the WiFi thermostats fell off the WiFi network at home and couldn’t rejoin it. I spent about a week exchanging email with Honeywell support, trying various silly things, watching them work intermittently, and then giving up by turning the thermostats’ WiFi radios off. Before giving up, I wrote “Honeywell WiFi Thermostats Not Connecting To WiFi Network”, prematurely concluding that if I reserved IP addresses for the thermostats, then they would stay on the network. This did, after all, seem to work… for a while. The next day, the thermostats fell off the network, I flipped a table, turned their WiFi radios off, and then resigned myself to owning programmable-but-no-longer-WiFi thermostats. It wouldn’t become a problem until the winter, anyway. Then, suddenly, Sarah’s Kobo e-reader wouldn’t join the network either.

Srsly?!

Good news: this pointed to the network, not the devices, as the problem. Bad news: I didn’t want to diagnose the problem. Good news: Sarah somehow likes to look things up, dig through the details, and suggest possible causes. I got to keep working on my tasks while she occasionally tossed out a possible cause. After a few iterations of this, it sounded like she struck gold.

Sarah found a discussion thread at digitalhome.ca in which one participant mentioned that they’d learned from our ISP that the Actiontec R3000 could only handle 11 WiFi clients. The whole thing sounded bizarre to me, but plausible, so I looked at our router’s administrative page and sure enough, I saw 11 connected clients. Sarah was on to something.

I knew that we had a very long Ethernet cable, which we could use to connect our signal range extender to the Actiontec router, which should reduce the number of WiFi-connected clients. We tried that, and at first we were crestfallen: the Actiontec router reported WiFi clients connected to the signal range extender as “WiFi connected clients”. But… there were now 14! And the Kobos could connect to the network!

We rebooted the Actiontec router, then saw only a few WiFi clients directly connected to it, like our television PVR. The Kobos connected without any difficulty. Every other WiFi device in the house seemed to be working, all connected to the signal range extender rather than the Actiontec router. So we tried the WiFi thermostats.

It took about 2 minutes, but they connected. I turned their WiFi radios back on and one of them connected automatically while the other restarted in WiFi Setup mode, but I connected it to the signal range extender’s network without incident.

Success!

The Network Setup

We now have an Actiontec R3000 modem/router connected directly to our FibreOp box. It runs a WiFi network that we essentially don’t use. We currently have a Netgear EX7000 WiFi signal range extender connected by Ethernet to the Actiontec R3000, and this signal range extender acts as a WiFi router for the house. We could certainly replace this signal range extender with a plain WiFi router and that would just work, since it now connects to the main network by a wire rather than WiFi.

WiFi speeds throughout the house have at least doubled.

We have even connected the television PVR to the Netgear router over Ethernet, so literally nothing in the house uses the Actiontec WiFi network at the moment. This would make it easy to have a guest network separate from the network we use.

Excellent! We did it!

If your ISP forces an Actiontec R3000 modem/router on you, then I strongly recommend that you connect another router to the R3000 and then connect your WiFi devices to the second router, in order to avoid this 11-device limit. I didn’t need any specialized networking knowledge to make this work. I simply connected my second router to the Actiontec by Ethernet, then ran the second router as an access point. Easy. Even I knew enough to do it without reading any online tutorial nor calling any networking-savvy friends.

Enjoy.

Honeywell WiFi Thermostats Not Connecting To WiFi Network

Thu, 17 May 2018 00:00:00 +0000

Against my better judgment, I purchased two Honeywell RTH8580WF WiFi thermostats several years ago. I did this only after verifying that the thermostats work as expected, even when not connected to the network. One can program schedules into them and they control the heating system, even when they can’t connect to the network. This gave me the confidence to try them. In the worst case, they become annoying-to-program-but-at-least-programmable thermostats.

Do not read any further. Read the next article for the real reason why these thermostats stopped working, and then come back here for a laugh. No, really.

The devices worked just fine for several years. I could control them over the internet. I could program them relatively easily. I could even turn the heat on just before embarking on the 1-hour-long trip from the airport to home after a winter trip. I could turn the heat on from bed if I happened to wake up a little early (or a little late). I decided that I like the devices.

And then they fell off the network and wouldn’t climb back on. For days. Ugh.

I didn’t think I’d changed anything—at least not recently. Yes, we changed routers, but several weeks prior. Suddenly, and with no recourse to useful diagnostic messages, neither thermostat could get an IP address from the router’s DHCP service. I contacted Honeywell support and, as I expected, they didn’t tell me anything beyond what I’d already read in the user manual. Turn WiFi off, then on. Enter WiFi setup mode. Nothing helped.

And then they started working again… for a few days. And then they stopped again. Oy. This illustrates exactly why we shouldn’t put everything on the network. This illustrates in particular why we should never put anything on the network that doesn’t also work offline!

And then it hit me: if these devices can’t get IP address reliably, then let’s cut some steps out. Specifically, let’s just ask the router to give them the same IP address every time. Maybe that would help.

I do not know enough about networks to call myself anything more than an Advanced Beginner, so if you know more and can suggest something, then please contact me and tell me what I should have done. Thanks.

Fortunately, the thermostats display their MAC addresses on a little sticker, so I logged on to the router and asked it to reserve fixed IP addresses for the two WiFi thermostats using their MAC addresses. I had to change the first three bytes of the MAC addresses to 02:0f:b5, because of virtual MAC addresses, so I even learned something in the process. Something I didn’t care to learn, but that’s sometimes how it goes.

Now—for the moment, at least—the thermostats have IP addresses and I feel happy about that. I hope it stays that way.

If you have these thermostats, then I recommend that you consider configuring your router to reserve fixed IP addresses for them, instead of negotiating new ones every time. This might save you some headache, especially in the winter, when you actually need these things to work.

How To Exit Full-Screen Mode from Screen Sharing on MacOS

Fri, 05 Jan 2018 15:58:00 +0000

I use Screen Sharing on MacOS to connect to my Mac Mini machines. I like to use full-screen mode. I usually prefer to use Jollys Fast VNC, but I haven’t installed it yet on my newest Mac Mini, so I used the built-in Screen Sharing. Sadly, this decision led to learning something that one shouldn’t need to learn.

How does one exit full-screen mode on MacOS’s built-in Screen Sharing when one uses a mouse instead of a trackpad, and therefore can’t swipe? I couldn’t do this with the keyboard shortcuts that I knew (Esc or Cmd+Ctrl+F). I had to search the web.

For Screen Sharing, in particular, to exit full-screen mode, move the cursor up to the top of the screen and then wait several seconds for the Screen Sharing menu to appear. You could have given me ten years and it wouldn’t have occurred to me to try this; instead, I moved the cursor up to the top of the screen, I didn’t seen anything happen, and then I moved the cursor wildly around the screen. Now I know.

And now you know. Enjoy.

Reference

Apple Support, “How do I exit full screen mode in Screen Sharing?”

Moving Screenflow Markers Without Disturbing Your Clips

Wed, 12 Jul 2017 20:19:00 +0000

I produced over 2 hours of video, complete with chapter markers to export into Subler as an index of the key moments of the video. Just before I went to export the final version for upload, I realized that I’d forgot to add transitions between the various clips. Leaving it this way would have looked terrible, but dragging clips around to add transitions doesn’t drag the markers with them unless you create all the markers on the clips themselves. I didn’t, because if I did, then they wouldn’t show up as chapter markers for the index.

Sigh.

Am I really going to have to move the clips, then remember to move every single marker on its own?! 12 frames for each transition? Really?! There was no way I would get this right.

Then it hit me.

A Brilliant Workaround

I wondered what would happen if I locked all the clips, then used Ripple Delete on a section of the video between markers—would it move the markers without disturbing the clips?

It did.

This became the workflow:

Go to the first transition point, where two clips meet.
Unlock all the clips, then drag all the later clips (as a big group) over the earlier clip to create the transition. I usually drag it 12 frames and use the “Cross Dissolve” transition. (I choose all the clips after the transition point to avoid creating a gap later in the timeline.)
Lock all the clips.
Create In and Out Points over the 12-frame transition.
Ripple Delete. Watch that the markers move, but the clips remain untouched.
Breathe.
Repeat all this for the remaining transition points, slowly and surely.

What could have become hours of painstaking work suddenly became much less stressful. Of course, for a 2-hour-long video, it still look a long time, but at least I knew what I was doing and felt confident about it.

Yay!

Extracting Chapter Markers Quickly from Screenflow

Sun, 09 Jul 2017 17:46:00 +0000

I use Screenflow 6 (as of this writing) to edit and produce videos. Some of those videos are quite long, and so I use the markers feature of Screenflow in order to allow viewers to skip ahead to various important points in the video, known as chapter markers or chapters. I provide notes along side these long videos to help the viewer find relevant information inside them, and I like to include in those notes an index of the video, which I produce from the chapter markers using a tool for Mac OS called Subler. Sadly, in order to export the chapter markers from Screenflow, I have to do two things:

Export the video.
Remember to check the damn checkbox for “include chapter markers”.

Exporting the video can take a long time. Recently, I waited about an hour to export a 2-hour-long video. I don’t object to how long it takes to export video, but I mind that I have to export the entire video in order to export the chapter markers. There has to be a better way.

I found one. I export the video without the video.

When I export the video from Screenflow, I export it in .mp4 format, but I exclude the video track from the video, exporting only the audio. I tried exporting nothing, but this seemed to cause Screenflow to do nothing and I needed to kill Screenflow and restart it. (I can hear the meeting now: “No-one’s ever going to have a legitimate reason to export a video with neither the video track nor the audio track!”) When I tried again, I exported in .mp4 format, and then pressed Customize..., which allowed me to select only the audio track. This time, exporting the 2-hour-long video took only about 3 minutes. Much better.

So this becomes my workflow:

Edit the video, setting chapter markers throughout.
Group the entire video into a single, large Nested Clip.
Add the intro and outro as Nested Clips, so that I have the marker times right.
Add a marker for the first frame of the video, as suggested by http://telestream.force.com/kb/articles/Knowledge_Article/Chapter-Markers.
Export the video in .mp4 format, exporting only the audio track.
Open the exported “video” in Subler.
From Subler, export the chapter markers as a text file.
Process the text file to turn the output format into something a human can read.

Enjoy.

Of course, since there exists a workaround, Telestream will never implement the feature to export only the chapter markers from a video, even though it means that you will almost certainly waste some non-trivial amount of time before discovering this article and learning what to do. Shame.

Rescuing PDF Clerk Pro Files Made Easy

Sat, 10 Jun 2017 13:50:00 +0000

The Short Version

Your .pdfp files are just bundles containing the contents as a single PDF file and some metadata, so it doesn’t matter that PDF Clerk Pro no longer runs on current versions of MacOS.

The Details

I depend on PDFs to run my business: expense receipts, sending proposals to clients, contracts, and so on. I rely on tools that slice and dice PDFs, perform OCR, make annotations easy… on MacOS I used to use PDFClerk — so much so that I bought the Pro version. I used it for years, especially to wrangle expense receipts into expense reports. Eventually, I discovered PDFPen, which I now use every day, and which I love. (If you buy PDFPen, please tell them that I said “Hi!” It’s one of a handful of tools that I happily upgrade.) I left PDFClerk behind (sorry!) and, while I wasn’t looking, PDFClerk faded into the background. I preferred PDFPen in part because it didn’t use a specialized file format for its contents, but rather operated directly on PDFs.

Fast forward to today, when I rudely discovered that MacOS 10.12 (Sierra) can’t open PDFClerk Pro 3.12. If you haven’t opened PDFClerk in a while, then I suggest trying it now, especially if you’re still on an older version of MacOS 10. I had a moment of panic. What will I do?! Who has MacOS 10.6 lying around on an old backup drive or something?

No worry. The .pdfp file is just a package that wraps its contents as a single PDF file and adds a metadata file (main.clerkdata) and a QuickLook file. In Finder, right-click the .pdfp package, choose Show Package Contents, and you’ll find the .pdf file, which you can rescue. You’ll lose the PDFClerk metadata, but I don’t even remember what was in there, so I don’t mind losing that metadata.

So if you used PDFClerk, there’s no need to fear no longer being able to use it: rescue the PDFs inside your .pdfp files, use PDFPen, and all is well again. Enjoy.

unZettling, or How I Discovered That Credit Card Companies Turn Cash Advances Into Absurd Uncertainty

Tue, 14 Feb 2017 15:15:00 +0000

If you use an iZettle device to purchase something by credit card in Sweden, then check your credit card statement immediately, because your credit card company might treat the purchase as a cash advance and additional fees and interest might apply.

Sigh.

I learned this the hard way traveling in Sweden recently. Fortunately, the customer service representative at my credit card company agreed to reverse the cash advance-related charges; however, this highlights two problems: that iZettle might report a purchase as a cash advance and that once you withdraw cash on a credit card, you can’t reasonably know how much to pay in interest. In my case, the purchase was somehow treated as a Swish mobile payment, and I have no way to know how this happened: did the vendor configure something incorrectly, report a sale incorrectly, did some automated system record the payment method incorrectly? I would expect that nobody will ever know how this happened.

The iZettle Problem

I want to support iZettle. I love not having to use cash in foreign countries, where foreign bank machine withdrawals carry uncertain exchange rates and unpredictable withdrawal fees. I also love the typical benefits of using a credit card: payment grace period, automatically tracking transactions, even cash-bank rewards (in the case of this particular credit card). I do not, however, wish to pay the price of a cash advance on a credit card for these benefits.

I have asked iZettle through their web contact form for information regarding this problem. I made at least a dozen purchases around Stockholm using iZettle devices and only one—at a Christmas market vendor for hot chocolate—was reported as a cash advance. All the other purchases were reported as purchases and handled as one would expect of a typical (North American) credit card account. As of this writing, I don’t know whether iZettle will have a useful answer for me. I intend to add details to this article as they find me.

I have a Square reader and make regular purchases at vendors who use them. I have never seen those purchases treated as cash advances. Is it possible that customers of Square-wielding vendors have experienced this problem, too? I’d like to know.

The Cash Advance Problem

I will now describe my experience with an MBNA Mastercard in Canada, although I imagine that others have had similar experiences, at least in North America.

Once I withdrew cash from a bank machine on my credit card account, the bank began charging interest immediately. This differs from purchases. On purchases, the credit card company gives me approximately 21 days to pay for my purchases before charging interest on those purchases, known as the grace period. Your credit card statement probably describes at least the highlights of your grace period. This makes it possible to review my purchases and resolve disputes before needing to pay.

Since the grace period covers purchases but not cash advances, then during the grace period, interest accumulates on the cash advance. When MBNA Mastercard publishes my statement and claims “You owe $2,750”, this amount is wrong by the time I see the statement. This means that if I pay $2,750 on the due date that they publish on the statement, then I am still charged an uncertain amount of interest unless, by chance, I know exactly how they are calculating the interest, which formula they do not publish. Yes, there are standard formulas for compound interest, but the best I can do is overpay the balance due and hope that I overpay by enough to cover the interest charges.

Worse: credit card companies apply payments to purchases first, and then to cash advances. This means that I can’t just pay off the cash advance first, then pay off the purchases after the grace period. This leaves me a Morton’s Fork:

Overpay the statement balance before the statement date (the “closing date”), in the hopes that I correctly predict which purchases will be posted to the account before the credit card company publishes the statement, or
Wait until the statement is published, then overpay the statement balance as soon as possible—certainly before the due date—in the hopes that I correctly calculate the interest that has accumulated on the cash advance.

Either way, this creates cash flow pressure: I have to pay my entire credit card balance 2-3 weeks before the vast majority of that debt becomes due in order to avoid an unpredictable amount of interest.

Big Deal? It Depends.

Right now I can treat this as a minor inconvenience. The interest charge is less than $1 per month and I have enough slack in my cash flow to pay the statement balance early if I wanted to avoid the interest charge. After about 2 months, the outstanding interest on the cash advance falls below $0.005 and rounds down to $0.00. This costs me more in effort and energy than in money: tracking the charges, understanding what happened, recording the details in order to keep track of my personal financial situation. It’s like when your accountant has to spend 15 minutes understanding why a transaction is $4 out of balance: a significant waste of energy.

For someone with no slack in their cash flow who is forced by their circumstances to rely on cash advances from a credit card account, however, this could create an endless loop. They can’t overpay their balance due; they have to pay the posted minimum payment; however, the posted minimum payment is wrong by the time the credit card statement arrives by postal mail or email. This creates significant risk (and problems) for some people. I feel very bad for them.

How to defend oneself? One has to understand what’s happening so that one can decide how to handle it. I hope that this article helps at least one person improve their understanding of day-to-day, practice personal finances.

Back Up an SD Card from Android

Wed, 19 Oct 2016 00:00:00 +0000

I wanted to replace the 64 GB SD card in my Galaxy Note 4 with a 128 GB SD card. I assumed that I could simply take the smaller SD card out of my phone, connect it by USB (with a USB-based SD card reader) to my MacBook, copy the contents to the larger SD card, insert the larger SD card into my phone, and then go on my merry way.

Of course not.

When I inserted the smaller SD card into my MacBook, it told me that it could not read the SD card. I assume it is a formatting issue, so I guessed that I needed to transfer files by USB from the phone. I connected the phone by USB and launched Android File Transfer, but sadly, Android File Transfer is terrible. When I tried to copy a large amount of data at once — a single folder tree with 15 GB of audio in it — Android File Transfer froze up. I couldn’t tell the difference between “slow” and “stuck”. It seemed to handle smaller batches of data just fine, but I refuse to babysit these tools. The devices work for me.

Searching the web, I found a suggestion to use FTP to send the files from the phone to my MacBook. This meant installing AndFTP on the phone and launching ftpd on the MacBook. I tried that. Sadly, FTP is not a great protocol for “make these folders the same”. At best, it can “overwrite/resume”. When my phone didn’t copy all 58 GB of data in one session, I couldn’t tell how much it had copied and which files I needed to recopy. Moreover, it sent files at approximately 4 MB/s, which means that it takes about 4-5 hours to try to copy everything. This appears to be just long enough for copying to fail.

Srsly.

As a last-ditch effort, I tried installing adb so that I could copy files with a command-line tool, which is more likely to work and, more importantly, easier to script if I need to copy files in batches. Naturally, the Android world has decided to “make things easier” by repackaging the Android development tools inside Android Studio, thereby making it harder to learn how to use them as simple command-line tools.

After I posted this article, Edu Rodríguez showed me where to find the Android command-line tools. This led me to discover that I had a very old version of the Android command-line tools installed at /opt, which I suppose I installed some time in 2013. You might have the same thing! I had to remove files from /opt/android-sdk-macosx and patch my .zshrc to remove a PATH reference to this old Android SDK. After I did all this, I realized “Homebrew must package this somewhere…” and sure enough:

$ brew install android-sdk
$ mvim ~/.zshrc
<set ANDROID_HOME environment variable, per Homebrew's post-installation instructions>

Gracias, Edu!

It turns out that, at the time, I ignorantly went “the long way”, so I ended up doing all this:

Launch Android Studio, since I had installed it previously in a desperate attempt to hack on some Android project about a year ago. (How do you think it went?)
Notice that Android Studio is out of date, so update it, then restart it.
Notice that Android Development tools are out of date, so update them, then restart Android Studio.
Try to launch adb from a command line, and notice that the Android Development tools are not on the system path.
Figure out where Android Studio installed the development tools, so that I can try putting that directory on the system path.

Finally, this seemed to help. I could launch adb from the command line.

$ export PATH=$HOME/Library/Android/sdk/platform-tools:$PATH
$ which adb
/Users/jbrains/Library/Android/sdk/platform-tools/adb
$ adb
<adb usage guide>

Yay! Now… how do I actually copy stuff?!

I assume adding $HOME/Library/Android/sdk/platform-tools to my system path now conflicts with installing the Android SDK with Homebrew. Fortunately, I never changed my system PATH, because I wasn’t sure whether this environment was stable — I only knew that “it works for now”, so I only changed it in my current bash session.

Even so, just to be sure, I launched android sdk and uninstalled the Platform Tools to make sure that I didn’t have an old, incompatible, partially-installed version lying around my file system somewhere. If I did, then I’d like to discover that now… and fortunately, I didn’t!

This does mean, however, that as of 2016-10-22, when I write these words, you should probably use Homebrew to install android-sdk, and then launch android sdk to install the Platform Tools. As Homebrew warns, the formulae for android-sdk and android-platform-tools conflict, so don’t try to install both together.

$ brew uninstall android-platform-tools
<maybe Homebrew says, "No such keg", which is fine>
$ brew uninstall android-sdk
<maybe Homebrew says, "No such keg", which is fine>
$ ls -l /usr/local/bin | grep android
<if there are symlinks to an android-related Homebrew keg, then remove them>
$ brew install android-sdk
<maybe Homebrew suggests setting ANDROID_HOME, in which case, do it>
$ android sdk
<Use the Android SDK UI to install the platform tools>
$ which adb
/usr/local/bin/adb

Muuuuuuuch better.

Wait.

$ ls -l /usr/local/bin/adb
[...] /usr/local/bin/adb -> ../Cellar/android-sdk/24.4.1_1/bin/adb

So… which version of adb do I have again? I thought I used the Android SDK Manager to install Platform Tools version 25! By coincidence, Platform Tools 24.4.1_1 and Platform Tools 25 have the same version of adb, but now I’m back to not knowing what is installed on my machine.

What am I missing? When I use Android SDK Manager to install Platform Tools, then I expect it to make those Platform Tools the default choice for my environment. Am I wrong? Should I have to add /usr/local/opt/android-sdk/platform-tools to my system path as a separate step? Please tell me what I should expect.

Copying Files from Android Phone With `adb`

I assume that I need to “start a debugging session” with adb by “mounting” my phone as a device, then I can use adb pull to pull the relevant files from my phone.

First, however, I need to enabling USB debugging mode on my phone. Fortunately, I have done this before. Doing this seems to depend on the device and Android version, so I recommend that you search the web to learn how to enable USB debugging mode on your device. For example, I searched for enable usb debugging mode samsung "galaxy note 4" and found http://www.solvemix.com/index.php/android-smartphones/315-samsung-galaxy-note-4-how-to-activate-usb-debugging-solved.

So far, so good. To recap:

Enable USB debugging mode on the phone.
Unlock the phone.
Connect the phone by USB.
On the MacBook, dismiss and close Android File Transfer, because it is useless.
On the phone, allow the MTP connection.
On the MacBook, dismiss and close Android File Transfer again, because it can’t take a hint.
Check that adb can find the phone.

$ adb devices
List of devices attached
<meaningless hex number>    device

I assume that device is my phone, because what else could it be?

Now, can I pull a single file? First, let me poke around the phone.

$ adb shell
shell@trlecan:/ $ cd sdcard
shell@trltecan:/sdcard $ ls
<a listing of the files on my SD card>

Yay! I want to pull the directory tree /sdcard from my phone to my MacBook. May I?!

First, let me pull a single file.

$ adb pull /sdcard/temporary_contacts.txt ~/Desktop
[100%] /sdcard/temporary_contacts.txt
$ ls ~/Desktop | grep con
temporary_contacts.txt

Success! Now, clean up before moving on, then pull a directory of reasonable size.

$ rm ~/Desktop/temporary_contacts.txt
$ ls ~/Desktop | grep con
<nothing>
$ mkdir -p ~/Desktop/pull-directory && adb pull /sdcard/scanner ~/Desktop/pull-directory
<an actually useful progress report showing something that suggests that the computer is copying files to my Desktop!>

I stopped this job after it was 20% finished, because I felt convinced adb pull would do what I want it to do. It copied about 170 MB in what seemed like less than 1 minute. At 170 MB/minute, a full copy of 58 GB would take about 6 hours. I hope it’s faster than that.

Since I don’t know how long it will take to pull the entire 58 GB on the card, I start with a big-but-not-huge directory and time the pull operation, so that I know how long to expect the whole backup to take.

# Clean up before moving on
$ rm -fr ~/Desktop/pull-directory
$ mkdir -p ~/Desktop/pull-directory && time adb pull /sdcard/scanner ~/Desktop/pull-directory

Even though I’m using the time tool, I start a stopwatch on the side, just in case time somehow fails. Now I wait.

Woohoo! time worked and alleges that copying that directory took 4 minutes, 35 seconds, copying 1018 MB of data. My arithmetic says 3.70 MB/s, so 58 GB would take 16052 seconds or 4.45 hours to copy. I can live with it, as long as it actually works.

And… of course, the directory called /sdcard on my phone is in fact not the root of the SD card storage on my phone, because that would make sense and be helpful and the universe is not that merciful. How did I discover this? I tried to check the size of /sdcard and it reported something entirely unexpected.

$ adb shell
shell@trltecan:/data $ du -h /sdcard/ | tail -n 1
4.9G    /sdcard/

Uh… what?! I expect something closer to 58 GB, so why is there only 4.9 GB of stuff here?! Sigh. What is this directory and why is it on my phone? Is it an alias? (Later I found out that it is indeed an alias. Fine.)

Fine. Where is the SD card on my phone, O great Android?!

shell@trltecan:/data $ df
<a bunch of Permission denied messages>
Filesystem                       Size     Used     Free   Blksize
<a bunch of irrelevant file systems>
/storage/7C2F-B7C8              59.7G    52.3G     7.5G    128.0K
<a bunch of irrelevant file systems>

Aha! 52.3 GB used out of 59.7 GB. That must be it. I’m so glad that I wasted 5 minutes copying files from the wrong place onto my phone! At least now I can test again, in case my measurement of 3.70 MB/s changes due to copying from the SD card instead of my phone’s built-in storage. Let’s try that again.

# Clean up before moving on
$ rm -fr ~/Desktop/pull-directory
$ mkdir -p ~/Desktop/pull-directory && time adb pull /storage/7C2F-B7C8/scanner ~/Desktop/pull-directory

The directory /storage/7C2F-B7C8/scanner has 840 MB of data, which required 3 minutes, 25 seconds, for an average of 4.10 MB/s. Encouraging! At that rate, 52.3 GB would require about 13062 seconds or approximately 3.5-4 hours to copy. That would be nice!

Finally, I think, I can just do it.

# Clean up before moving on
$ rm -fr ~/Desktop/pull-directory ~/Desktop/BARNEY-SD
$ mkdir -p ~/Desktop/BARNEY-SD && time adb pull /storage/7C2F-B7C8/ ~/Desktop/BARNEY-SD

Now, we wait…

After We Wait…

It took 3 hours, 47 minutes and it seemed to work! The transfer ran at only about 4 MB/s, but at least I could more easily see progress and there’s no GUI to get in the way. Now I need to wait an hour or so to copy these files to my larger SD card so that I can install it in the phone. Now that I know how all this works, I can do this more easily next time.

Summary

Use adb to pull files from the SD card onto a new storage device, which could be the new SD card mounted on the computer.

Enable USB debugging mode on the Android device.
Install the Android Development tools, and specifically adb.
Find the mount path on the device for the SD card using df.
adb pull <mount path on Android SD card> <destination drive>, then wait several hours.

It sounds so simple, once you’ve actually learned how to do it.

But Wait! There’s More!

There’s no globbing in adb pull! If you want to copy a subset of the files in a directory, then you need to resort to lower-level trickery.

$ adb shell ls $1 | tr -s "\r\n" "\0" | xargs -0 -n1 adb pull

Here, $1 is your pattern for matching files, such as /storage/emulated/0/DCIM/Camera/20161207* to match all photos taken with your camera on December 7, 2016.

Thanks to http://stackoverflow.com/questions/11074671/adb-pull-multiple-files#11250068 and https://github.com/sschuberth/dev-scripts/blob/master/android/pull_all.sh for their help!

A Little Tip for Using Typora with Pandoc

Sun, 09 Oct 2016 15:50:00 +0000

I have only been using Typora for about a day, but I already love it. If you do any serious editing with Markdown, then I strongly recommend that you try it. I especially recommend that you try it now, while it remains free, and that you subsequently give the Typora production team money when they ask you for it. I recommend giving them at least 50% more than they ask for; I like it that much.

(No, I have no affiliation with the production team, but if they launch an affiliate program, then I will join it.)

Typora offers a hybrid WYSIWYG approach to editing: as you type Markdown syntax, you see Markdown syntax; and then as you type past it, it becomes rendered as HTML. I love it. You just have to try it. I’ve never used any Markdown editor that so wonderfully combines the experience of real-time preview with the transparency of a “source mode”, for when you absolutely need to tweak the Markdown source to get it just right. I love it.

Of course, the only thing better than a perfect tool is a tool that is perfect except for one small flaw that we can pick at all night. I have only found one in Typora, so far, and it has to do with nested lists and the Markdown specification.

After Installing Typora, Do This!

Once you download Typora and install it, open the Preferences, and then look for the option “Indent Size on Saving”. If the default is not 4 spaces, then change it to 4 spaces. If you don’t, then your nested lists will not always look right, depending on how you process your Markdown. I ran into this problem when I published an article on one of my other blogs. I had composed it in Typora, then processed it with pandoc in order to publish it. When I published it, my nested lists were not so nested. I managed to track the problem down to this: the Markdown specification requires 4 spaces to nest lists within lists, and although many processors treat this with leniency (that worked so well for browsers and HTML, so why not?!), they ought not to do that. In particular, pandoc does not like you using only 2 spaces to nest lists. Don’t do it.

Since I changed the default setting in my Typora installation, I can nest lists with total confidence that the lists will look right even when I process the final Markdown with pandoc for publication. Yay!

References

https://github.com/jgm/pandoc/issues/137 The Github issue for pandoc that discusses this very point, and the reason I assume that Typora, not pandoc needs to change.

https://twitter.com/jbrains/status/785144020683067392 Me asking the very nice Typora people to consider changing the default setting since, after all, they are currently not respecting the Markdown specification.

LibreOffice Calc Won't Print This Sheet!

Wed, 08 Jun 2016 00:00:00 +0000

I love LibreOffice and OpenOffice both for helping me escape from the clutches of Microsoft Office. I have donated to LibreOffice as a measure of gratitude for helping me run my business and indeed my life. Sadly, no technology product seems impervious to silly problems, and I encountered one today with LibreOffice that made me scream in frustration.

The Symptom

I have a workbook with 7 worksheets. I wanted to print only the 6th worksheet. I did this:

Deselect all the worksheets.
Select only the 6th worksheet.
Press Cmd+P for “Print…”.

The miniature print preview dialog did not show me the sheet that I want to print, but instead, the last sheet, quite arbitrarily. I tried selecting all the available options for choosing which parts of the workbook to print (“Select all sheets”, “Print only current selection”…), but I could only print the last sheet. I felt like this print dialog was mocking me.

The Fix

Clear all the print ranges, because some arbitrary sequence of commands from the past that you didn’t notice managed to set the print range on some of your sheets to “none” or “empty”. Obviously.

Some Details

I have no idea why any print range was set on any sheet in this workbook at any point. Sadly, none of that matters; it suffices to know that the print range for a given worksheet filters what you can choose to print from the print dialog, so if you can’t print what you want to print, then you should check the print ranges.

I did this:

Select all the sheets.
Choose Format > Print Ranges > Clear. (If “Clear” is not available to select, then you have a different problem, and I don’t know what that is.)
Deselect all the sheets.
Try printing again.

Success!

Reference

“Calc Will Not Print All Sheets in spreadsheet”. A forum discussion that gave me the idea to check for unexpectedly-set print ranges. Thank you, William Whalley, for noticing the cause of the unexpected behavior.

I Can't Boot From My External USB Drive!

Fri, 06 May 2016 00:00:00 +0000

I take backups pretty seriously. It’s the primary reason I switched to Mac: once I learned that I could boot from an external backup drive I wanted that security. Today, when my shiny new external USB 3.0 drives arrived in the mail, I immediately started a backup, but was shocked when I couldn’t boot from the backup! Well… somewhat. I eventually got there, and it wasn’t obvious to me, so I wanted to make sure that you knew what to do.

The Symptom

I did this:

Power down my MacBook Air.
Connect the external drive to the MacBook Air by USB.
Hold down the Alt/Option key while pressing the Power button to turn the computer on.
Wait for the boot menu to appear and list both my internal storage and the external drive as bootable disks.

I waited and waited and waited, but the external drive never appeared in the boot menu, so I couldn’t boot from it.

The Fix

While in the boot menu, disconnect the drive from the MacBook Air, wait a few seconds, then reconnect it. The drive should start to spin and after a few seconds, it should appear in the boot menu. Now you can boot.

Some Details

I booted successfully from the external drive using the Startup Disk section of System Preferences, so I guessed that SuperDuper! had correctly made the backup bootable.

Of course, if I have to first boot from my internal storage in order to subsequently boot from the external drive, then that defeats the purpose of a bootable backup. I need to boot from the external drive precisely when my internal storage fails. (I did this once with my trusty Powerbook G4, whose internal hard disk died. I plugged in my external drive, rebooted, and kept working. Sure, the machine was slow, but it worked for the few days I needed before I could get to a repair shop. Can Windows do this yet?)

I noticed that, at the boot menu, the computer was clearly sending enough power to the external drive, but that the activity light was not flashing with disk activity. With nothing left to lose, my wife disconnected the drive and reconnected it. Suddenly the disk activity light started flashing and, within a few second, the drive appeared in the boot menu. Success!

Just another little stupid trick one needs to have up one’s sleeve to deal with modern technology.

I Can't Find Stuff In My Markdown Files!

Tue, 26 Jan 2016 00:00:00 +0000

As I wrote previously in “Writing With Pleasure Using Markdown and Pandoc”, I use Markdown a lot for writing. I don’t use it just for writing articles, but also for writing outlines and making notes. If I draft it using plain text, then I usually use Markdown as the format.

Imagine my surprise this morning when I tried to search for an outline I’d drafted of a conference talk and couldn’t find it. I couldn’t even find it by file name. Panic!

The Symptoms

Today, I was flipping through some paper notes, trying to clean them up by digitizing what I needed to save, adding tasks for things I needed to do with them, then shredding them for recycling. I came across handwritten notes for a conference talk I plan to record soon. I felt certain that I’d already transcribed the notes into a plain text file—in Markdown format, of course—so I popped open Spotlight to search for the text in my head along with the operator kind:markdown, meaning “search in Markdown files”.

Spotlight found nothing. Weird.

I looked around my computer’s file system for where I thought I filed the outline document and I found it in a file called 60-Minute Talk Sketch.markdown. I opened it and found the exact line of text I’d searched for with Spotlight. Why didn’t Spotlight find it? I tried one more thing.

I love Alfred for moving around my computer’s file system. I won’t even try to write about how much time it saves me. I hate using other people’s computers for more than a few minutes unless they have Alfred installed. I sometimes worry that it slowly makes me increasingly unable to use a typical Mac. I digress.

Since Alfred uses data from Spotlight to find files, I opened Alfred to search for my conference talk outline. I couldn’t even find the file by name! Super weird.

I asked Alfred to find some other files for me… it found them without trouble. Just not Markdown files.

Ugh. I guess I have to “fix” something….

Spotlight and Markdown

Fortunately, others have encountered this problem before I, and have written about how they solved it. It appears that, out of the box, Mac OS X has no idea what a Markdown file is, so we have to tell it. This makes sense to me, so while it annoys me, and least it doesn’t surprise me. We need to tell Spotlight about the Markdown file format and in order to be able to search inside Markdown files, we can tell Spotlight, “Treat them like Rich Text files.” That sounds reasonable to me.

The Steps I Used to Solve the Problem

An overview of the steps:

Download and install an “importer” that instructs Mac OS X to index Markdown files as though they were just another kind of Rich Text.
Ask Spotlight to reindex everything.
Wait several minutes and try again.

Now, the details.

The Detailed Steps

Read “A Yosemite Markdown Spotlight Importer” for details.

Click here to download the importer.
Uncompress the importer.
Copy the file Markdown.mdimporter to /Library/Spotlight.
Open Terminal and tell Spotlight to reindex the entire local hard disk. sudo mdutil -E /
Check that Spotlight is using the new importer. mdimport -L should match at least one line of output, showing the location to which you copied Markdown.mdimporter.
Wait several minutes. I’ll have a tea sounds nice.¹

2016-01-26 12:01:21.177 mdimport[27566:3379462] Paths: id(501) (
    [...some irrelevant importers...]
    "/Library/Spotlight/Markdown.mdimporter",
    [...and so on...]
)

Some instructions tell you to copy the importer file to $HOME/Library/Spotlight, meaning at the scope of the current logged-in user. Since some people reported that that didn’t work for them, and I wasn’t in the mood to mess around, I just copied it to /Library/Spotlight, meaning at the scope of all users. If you want to experiment with copying this to $HOME/Library/Spotlight, then feel free. It might also work.

This worked for me. I hope it helps you.

Epilogue

Someone asked about this in an Apple discussion board thread from 2008, but there seems no way to tell that person about the solution. This makes me sad.

References

Hilton Lipschitz, “A Simple Markdown Spotlight Importer”.

Hilton Lipschitz, “A Yosemite Markdown Spotlight Importer”.

Stack Overflow, “How can I make spotlight index markdown files?”. More details on the solution, how it works, and alternative approaches.

Anthony Craig, “How to Force Spotlight to Index Markdown Files in El Capitan”.

Brett Terpstra, “Fixing Spotlight indexing of Markdown content”. All roads appear to lead to this article.

If you don’t get the reference, then I can’t help you.↩︎

Handling Urgent Email With Todoist

Wed, 20 Jan 2016 22:00:00 +0000

I use Todoist to track my work. You might call them “projects” or “next actions” or “to do”s, but I try to call it simply “work”, if for no other reason than to make it easier to talk about. I strongly prefer to track work in one place, and I’ve chosen Todoist for that. I know that some of you use email to manage your tasks, and that doesn’t work for me, for a few reasons.

I sometimes don’t open email for several weeks.
I really hate looking in multiple places to remind myself what I should be doing next.
When I open email to look for work to do, I become easily distracted by the new email to process, so I end up doing that instead.

Yes, perhaps I should not be so weak-minded, but managing one’s work well involves creating systems to tolerate one’s weakness, rather than work against them. Suffice it to say that I don’t want to track significant work in email; I want to see it in Todoist or, in some cases, directly on my calendar.

The Manual System

When I process my email inbox, I want to quickly assess which emails to delete immediately, file for future reference, answer someday, answer soon, or answer now. I always prefer to delete the message immediately or file it for future reference (notices, bills that I automatically pay, and so on), but when I have to answer the message, I apply the Two-Minute Rule. If I can answer it in two minutes, then I answer it as soon as I’ve sorted all the incoming mail. If it takes longer, then I forward it to Todoist, and fortunately there exists a nice plugin for Mailplane that lets me add a task to Todoist directly. If I prefer, I can also forward the email to a Todoist project and it will show up as an action in that project. As I process my email inbox, then, I decide quickly which of these to do for each email, in order of preference:

Delete it.
Send it to Evernote, because I’m likely going to need to refer it.
Archive it, because I might need to refer it.
Label it “Answer Eventually” and rely on my standing Todoist task every week that says, “Respond to some ‘Answer Eventually’ emails.”
Label it “Answer Soon” and send it to Todoist as a specific task in the project “Urgent Email”.
Label it “Answer Now” and come back to it once I’ve processed the rest of the inbox.

The emails that end up as specific Todoist tasks, rather than simply going into the “Answer Eventually” pile (also known as “When I Get Around To It”), expose a flaw in the system, at least when I start trying to automate it a little.

The Trouble with GMail and Todoist

Members of The jbrains Experience, my online learning and consulting circle, can ask me a question by posting a comment in a comment system. This sends me an email and that’s how I know that a question is waiting for me to answer it. Since I don’t look at email every day, and since those emails fall into the “Answer Soon” category, and since those emails follow a pattern (same sender, same words in the subject), I end up creating very similar Todoist tasks in the same project to remind me to answer these questions. After doing this five or six times, I wanted to set up a GMail filter to automatically forward the email to my “Urgent Email” project in Todoist. Sadly, I couldn’t.

To forward the emails with GMail, one must verify the email address. GMail sends it a message, you read it and click a link, then GMail knows that you own that email address and consent to receive forwarded emails to it. Sadly, although Todoist provides an email address for every project, one cannot receive those emails outside Todoist in order to click GMail’s verification link. This means that one cannot register a Todoist project email address in GMail as a trusted email address to which to forward email.

Mailgun To the Rescue

I have a Mailgun account and that gives me a solution to the problem. In Mailgun I create a Route that forwards emails from a middleman address I control to the Todoist project’s email address. Since I can read emails sent to this middleman address, I can verify that I control that address to GMail, then add a GMail filter that forwards the “I have a question” emails to the middleman. Mailgun then intercepts the email and forwards a copy to Todoist, which creates a task in the appropriate project. Problem solved.

Using Mailgun as a middleman between GMail and Todoist

You don’t have to use Mailgun in particular. You simply need an email address connected to a mailbox so that you can receive GMail’s address verification email and a service that can forward emails received at that address (or copies of them) to another email address (the Todoist project).

An Example Would Be Handy…

Suppose you run a consulting circle like I do and you want to automatically create a Todoist task every time one of your members asks you a question by email. In order to make this easier, you’ve set up a question form so that questions always arrive from the same email address and have some magic string in the subject line that an email filter can easily match. Here are the steps to follow to create the email-to-task pipeline.

Choose a Todoist project, then find the email address associated with it.¹ I’ll call it your-todoist-project@todoist.com here.
Invent an email address on a domain that you have registered with Mailgun. I’ll call it answer-consulting-circle-email-inbox@mydomain.com. (Of course, I don’t have mydomain.com, but I have to watch for spam.)
Tell GMail that you’d like to be able to forward emails to answer-consulting-circle-email-inbox@mydomain.com.² Complete GMail’s verification instructions for the new forwarding email address.
Create a GMail filter that matches the “somebody in my consulting circle asked me a question” emails and choose “forward to answer-consulting-circle-email-inbox@mydomain.com” as one of the actions for that filter.

That’s it! Enjoy.

References

David Allen, Getting Things Done. A system for managing work based on the principle of building a trusted system for figuring out what to do next so that your mind can focus on each task as you work on them.

J. B. Rainsberger, “Getting Started with Getting Things Done”. If you don’t have energy to read a book, then you can probably read four pages to try out the basics of the system and see whether it’s worth learning more.

J. B. Rainsberger, “The Two-Minute Rule”. If you really don’t have the energy even to put the basics of Getting Things Done into action, you can certainly benefit from the Two-Minute Rule, which will help you avoid the day getting away from you, especially when trying to deal with email.

Todoist. My primary “projects” database. I choose a projects database based mostly on how easily I can get tasks into it, and Todoist makes that quite easy.

Mailgun. I use Mailgun primarily for flexible email forwarding and sorting, as well as for creating email aliases whenever I need them.

Mailplane. If you use GMail on Mac OS, then you will probably enjoy Mailplane.

You can click https://support.todoist.com/hc/en-us/articles/205749772 to see how to do this. By the time you read this, Todoist might have changed.↩︎
You can click https://support.google.com/mail/answer/10957?hl=en to see how to do this. By the time you read this, GMail might have changed.↩︎

Writing with Pleasure Using Markdown and pandoc

Sun, 13 Dec 2015 18:24:00 +0000

I write a lot, and I never know where or when my ideas will strike. In order to at least get things out of my head, I like to write in a format that I can use from just about anywhere, so I rely on Markdown. It’s plain text, which means that I can write it on my phone, my tablet, my laptop, with or without an internet connection. It’s also lightweight HTML, which means that I can add some structure to my writing, like lists, headings, and links, without distracting myself from what I’m trying to write.

A friend, Mike Bowler, once told me that he hated writing in a word processor, because the moment he started formatting what he was writing, he stopped writing and became fixated on the formatting. I found the same thing when I worked in Word, Pages, and other full-fledged word processors. I even have this problem in HTML, but I have to get the formatting right. I’ve noticed that this problem mostly goes away when I write in Markdown. Even when I do a little formatting, I can do it quickly, unobtrusively, and then get back to the thing I’m actually writing. This makes my job much easier.

I have also found it easier to write in Markdown without worrying about where I’m going to publish what I’m writing, in part due to pandoc, the Swiss Army knife of text processing. Although pandoc is not suitable for all users, I think that even non-technical users can enjoy the benefits of writing whatever, whenever, without worrying about producing “the right format” for how you will publish your writing. With pandoc, I can write in Markdown, then decide later whether I need RTF, HTML, TeX, or any of a variety of formats. I can decide later when to publish an article as a blog post, a whitepaper, or even as a chapter in a book! The combination of Markdown and pandoc gives me options.

The Simplest Case: Publishing to a Blog

If you publish to a blog, then you probably already have a system in place for publishing new posts. Whatever you do, working with Markdown and pandoc will feel similar, and will probably not appear to give you much benefit over your current system. I felt the same way, which explains why I started using pandoc only relatively recently. My old system worked well enough, which both means that there wasn’t much reason to change, but the similarity of the new system also makes it relatively easy to change.

I tend to compose blog posts at my laptop, where I mostly use Atom. (I work in vim, too, but I remain at best an intermediate vim user.) I use an Atom plugin to let me preview my Markdown documents as HTML. The plugin, called markdown-preview-plus, lets me use pandoc as my Markdown processor, meaning that I get the same results in the preview window as when I publish to my jekyll blog, which also uses pandoc to convert Markdown to HTML. I don’t have to worry about formatting something one way while writing, then getting something unexpected when I publish, which normally causes me to scramble and fix something at the last moment. When I publish something, it just looks right.

Since I travel quite extensively to work, I also find myself in airports and coffee shops when I have the urge to write something. In this situation, if I find inconvenient to pull out my laptop—or if its battery is dead—then I can pull out my Logitech K480 keyboard and Samsung Galaxy NotePro and write, using an app called JotterPad. This app lets me compose in Markdown, get a quick preview (not with pandoc, but it’s good enough), then move the composition onto my laptop (directly or through Dropbox), where I can give it a final check before publishing it. This just works, and I feel no extra friction when I want to write using my tablet compared to using my “full-fledged” system on the laptop. I can write anywhere and get consistent results, which makes it easier for me to just write.

An Interesting Case: Rescuing Old Articles

I’m in the process of moving away from Wordpress towards jekyll for my blog at blog.jbrains.ca. This means moving a bunch of articles from older formats into my current system. I dreaded the prospect of doing this, because it meant reverse-engineering the Markdown source for 100 HTML articles. Bleh.

Nope. I didn’t have to worry, thanks to pandoc!

The first wonderful discovery I made about pandoc was that it can convert HTML back to Markdown with at least 95% accuracy. Yes, it misses things like embedded video, but it does 95% of this terribly tedious work, and for that, I love it. (I love it as much as one can love software, anyway.) Now, when I have 30 free minutes, I can rescue a few old blog posts, by doing this:

Log in to my Wordpress blog. Edit an article.
Load my new blog project into Atom and start a new document.
Copy the blog post’s title and publication date into the new document in Atom.
Add a category or two to the new document in Atom.
Select the entire Wordpress blog post content and copy it.
Run this command: pandoc --no-wrap --read=html --write=markdown <(pbpaste) | pbcopy, which replaces the HTML on my clipboard with (almost) equivalent Markdown.
Paste the Markdown on my clipboard into the document in Atom.
Save, preview, publish.
Redirect the old blog URL to the new one.

Granted, you need some technical chops to do the last step, and the pandoc command looks intimidating, but at least it’s exactly the same command every time, so you can just copy/paste it in your Terminal/Command Prompt. (If you work on Windows, then find a power user friend to teach you the exact command. I don’t do Windows, so I don’t know it.)

The important thing is this: pandoc is helping me finally, after years, get away from Wordpress. If you have a love/hate relationship with Wordpress, then you understand why I might want to get away from it. So far, I’m still on my honeymoon with jekyll. :)

Whatever You Want, `pandoc` Can Do It

I really hate most modern software, because it lets me down constantly, but not pandoc. I would even recommend it to non-technical users. Most non-technical users have two primary problems with software:

Installing it.
Dealing with problems when it fails, because every failure looks equally disastrous. It either works or it doesn’t.

Granted, installing pandoc isn’t pretty to non-technical users, even though highly-technical users probably love how simple it is. If you need someone to install pandoc for you, then I think it’s worth the effort. One thing you won’t have to worry about: pandoc is solid. It just works. Once you learn how to ask it to process a document, it just does it, well and quickly. You won’t have to fiddle with it. It just works.

This makes me trust pandoc. I don’t trust most software any more, so when I find something that I can trust, I really enjoy it. Now, if I need to convert some text from one format to another, I feel confident that pandoc will do it for me. This means that I don’t have to worry about “Will I be able to republish these blog posts into a book?” It’ll just work. I can take 20 blog posts, paste some connective tissue around them, and then publish it as a Leanpub book and charge $7. No, it’s not the same as publishing to amazon, but it’s really good!

The Bottom Line

Markdown lets me focus on the writing. pandoc gives me confidence that I can publish it anywhere, anytime, with minimal effort. I’ve written 50,000 words since I started using this pair of tools. I love it.

If you want to get started, read on.

The First Detail: Running `pandoc`

Once you have installed it, running pandoc can be as simple as this:

$ pandoc --read=markdown --write=html -o results.html my_awesome_post.markdown

If you don’t specify -o, then you pandoc just spits out the resulting HTML, so you can do it with whatever you want.

Of course, pandoc has many, many options. You’ll have to ask it for all the details: pandoc --help.

The Gory Details: Setting up Atom with `pandoc`

Install both Atom and pandoc, then read on.

Install`markdown-preview-plus`

Press Cmd+Shift+P, then start typing “Install packages”:

Search for markdown-preview-plus, then press Install.

After the package installs, choose Settings.

You might find it useful to change two settings:

Pandoc Options: Path. Set this to wherever you installed pandoc. In my case, $HOME/.cabal/bin/pandoc. If you don’t know, then open a Terminal window/command prompt and type which pandoc.
Pandoc Options: Commandline Arguments. I like to have my blog’s CSS available to the preview, so I choose --css /Users/jbrains/Workspaces/jekyll-jbrains.ca/php/css/main.css. Great, no?

Now you can close the Settings tab, open any file containing Markdown, and press Ctrl+Shift+M to get the preview window. If, for some reason, that doesn’t work, then do this from Atom’s menu: Packages > Markdown Preview Plus > Toggle Preview.

Now you can use pandoc from within Atom.

More Gory Details: Using `pandoc` with `jekyll`

Unless you speak bash, you should probably skip this section.

To use pandoc with jekyll requires creating a “relocatable binary”. Fortunately, the installation instructions for pandoc make that relatively clear. Once you have a relocatable binary, you can include it in your blog and make it available to your blog’s “production environment”—or server, if you prefer. I publish to OpenShift.

Ship `pandoc` with your `jekyll` blog

Since I want to schedule future posts to be published, I want my production environment (OpenShift) to rebuild my blog every so often. If you plan to publish each post as you go, then you don’t have to worry about this part at all: instead, just run jekyll build whenever you want to republish. This means that I need a way to rebuild my blog in production, then a way to schedule the “rebuild” command.

Tell `jekyll` to use `pandoc`

I added the following to my configuration file (the default name is _config.yml).

# Bundle pandoc for production. See .openshift/action_hooks/deploy
markdown: pandoc
pandoc:
  format: html5
  extensions: [smart, mathjax]

Rebuild!

I put pandoc in the folder libs at the root of my blog project. On OpenShift, this corresponds to $OPENSHIFT_REPO_DIR/libs/, so that I know how to use pandoc to build my site on OpenShift.

I created an action hook script at .openshift/action_hooks that rebuilds my blog using jekyll.

#!/bin/bash

echo "$(date) Regenerating blog." >> $OPENSHIFT_LOG_DIR/regenerating-blog.log
export LC_CTYPE=en_CA.UTF-8
export LANG=en_CA.UTF-8
export LD_LIBRARY_PATH=/opt/rh/mysql55/root/usr/lib64:/opt/rh/ror40/root/usr/lib64:/opt/rh/ruby200/root/usr/lib64
export PATH=$OPENSHIFT_REPO_DIR/libs:~/.gem/bin:/opt/rh/ruby200/root/usr/bin:$PATH
gem install bundle
cd $OPENSHIFT_REPO_DIR && bundle install && bundle exec jekyll build --config _config.yml

I found out the hard way that I have to set the locale explicitly by setting LC_CTYPE and LANG. I did not enjoy that little surprise.

I put $OPENSHIFT_REPO_DIR/libs/ on the system path so that bundle exec jekyll would find pandoc and be able to run it.

Rebuild Every Hour!

I added the Cron cartridge to my OpenShift application and added this little script to .openshift/cron/hourly

#!/bin/bash
echo "$(date) Regenerating blog." >> $OPENSHIFT_LOG_DIR/regenerating-blog.log
cd $OPENSHIFT_REPO_DIR && bundle install && bundle exec jekyll build

I now realize that I should change the deploy script to just invoke this one, but my bash skills aren’t excellent, so I’ll wait until I need to learn how to do that, then I’ll do it.

Now, I can write posts with a date in the future and my production environment will publish them within an hour of the date I specify. Nice, no?

References

John Gruber, “Markdown: Syntax”. A complete reference of the basics of Markdown. You’ll probably need to use this for a few weeks while you get used to how to write in Markdown.

Adam Pritchard, “Markdown Cheatsheet”. You can never have too many good references for writing in Markdown.

pandoc.org. Get started here with pandoc. The installation page looks pretty straightforward, especially for Windows.

atom.io. A fine text editor, especially if you speak Javascript and like to hack, but even if you don’t.

Markdown Preview Plus for Atom. The plugin I use to get a WYSIWYGish view of what I’m composing in Atom.

leanpub.com. When you want to publish what you’ve written and charge money, Leanpub provides a low-friction way to make that happen.

jekyllrb.com. When you want to publish what you’ve written to the web, Jekyll provides a very simple way to make that happen, and without Wordpress.

blog.jbrains.ca. A blog I publish with jekyll that uses pandoc to produce HTML from my posts composed in Markdown.

Hang Out With J. B. Rainsberger

Why My Webcam Image Is So Damn Red!

Just What Happened

A Neat Feature

Details

Byobu Session Name in Terminal Window Title

Turn Titles On in tmux

Remember These Settings Forever

Further Reading

Viewing HEIC (HEIF) Images on Pop! OS 22.04

The Solution

The Symptom

Attempt 1

Video Gradually and Increasingly Falling Behind: Don't Use Buffering

A Fix?

Key References

Some Details

Upgrading Discourse When You Totally Have Enough Disk Space

The Symptom

A Workaround

Why?!

unWise Arithmetic

The Symptom

The Impact

The Problem

The Solution

strptime: a Beginner's Mistake

The Symptom

The Problem

The Solution

Remove a Password from a PDF Document on Linux the Super-Easy Way

The Symptom

The Problem

The Solution

The Mysterious Extra Quotes that `jc` Put In My JSON

The Symptom

The Cause

The Solution

The Gory Details

Spot the Difference!

TIL: Byte Order Marks

I Can’t Be the First!

Ducking in Screenflow

The Problem

The Solution

LibreOffice Calc, Headers, Footers, and Every Page!

The Problem

The Solution

A UX Suggestion For Everyone

How Pandoc and Ranger Froze My Laptop

The Problem

The Solution

Installing hledger

How I Knew…

Pop! OS Won't Mount Any External USB Disk

Solution

Why?

How Did We Get Here?

Why My Spreadsheet File is So Big

Solution

How I Got Here

When Your Site Can't Find Its Discourse Comments

Symptom

Problem

Solution

Why the Problem Was a Problem

Fix the Automatic Time Zone on Pop! OS

The Short Version

How I Got Here

Remap Keys on System76 Less Painfully

My Primary Goal

My Current Solution

The Steps

References

Fixing Your Virtual Camera On Microsoft Teams for Linux

The Workaround

The Consequences

Exclude Big Files From Your restic Backup

A Solution

Get Push Notifications for Rate Alerts at wise.com

Turn Titles On in `tmux`

Why `sudo`?

`devmode`! I wouldn’t.

`/etc/gimp/2.0/controllerrc`

`/etc/gimp/2.0/sessionrc`

The Trouble With `autoremove`

Thank You, `etckeeper`