Musings on configuration of software

[tags, Vim, Vi, Linux, Unix, Debian, configuration, software]

Preamble: I just stumbled on what appears to be a completed-but-unpublished text, dated “Jun 11 2017” in my file system. As it is fairly lengthy, I have decided to publish it retroactively. Except for a spell-check, the text follows “as is”, including outdated time references, in the hope that the text actually was in order. Note that improvements to e.g. Vim might have been made in the almost four years since the text was written.

Over the last few days, I have been puzzled as to why my experimental install of Debian Stretch did not handle copy-and-paste using the “primary selection” within Xterm windows correctly.*

*Do not worry: Understanding that sentence is not a pre-requisite for understanding this post. Being interested in issues like software development could well be, however. A complete understanding will require a background with Unix-like environments, but much of the discussion should be clear without it.

After many twists and turns, I found out that the true problem was not related to Xterm—but to Vim (my favorite editor): The new Debian came with a new version (8) of Vim and the behavior of the default settings had changed considerably. More to the point, changed in a very ill-advised manner.*

*However, I stress that some degree of change is not automatically a bad thing, especially considering that some of the previous default behaviors had proven suboptimal and that a major version increase of Vim actually corresponds to a major release—unlike e.g. Firefox and its versioning idiocy. What is changed and how, that is where I have objections.

Among the problems:

1. Many of the choices are disputable, including what I stumbled on: The activation of the mouse in terminals (including Xterm). Any application that activates the mouse in a terminal automatically breaks the terminals own mouse-handling, with the implication that this should only be done when there is a considerable benefit from using a mouse. To boot, if the benefit is so large, the developers really should consider whether their application is compatible with a terminal in the first place. There are cases when even a terminal based application needs a very point-and-click interface, e.g. w3m*—but Vim does not have a point-and-click interface! By and large, Vim is keyboard driven and the mouse brings marginal or no benefits to those who can handle Vim by keyboard—and those who use Vim in a terminal should be assumed to belong to this group! In fact, Vim is the proverbial keyboard-driven application! A more sensible approach would have been to leave the mouse on in the GUI versions only.

   *w3m is a terminal-based browser, handy for instance for viewing HTML-based admin interfaces when logged in on a remote server. Because web pages are almost always written with the expectation of a point-and-click interface, even browsers with great keyboard support are handicapped without a mouse.

   Another example is the blanket use of the non-Vi-compatible mode*: A better solution would have been to follow the (well-established) convention of being compatible when called with the “old” name (here “vi”) and not when being called with the “new” (here “vim”), analogous to the behavior of e.g. Bash when called as “bash” and when called as “sh”.

   *Vim was created as an improved version of Vi, an older and extremely popular editor. Like many other (non-commercial) tools in the same situation, it tries to make life easier on users and admins by additionally allowing it self to be used as a drop-in replacement, thereby removing the need to have both it self and the original installed. Being a drop-in replacement, however, requires a high degree of backwards compatibility. On the other hand, this backwards compatibility would restrict the improved parts severely, were they active all the time. The solution is a separate compatibility mode.

   Yet another the remapping of “Q” (switch to ex mode) to do the formatting previously done with “gq”. Not only is this a highly unexpected break with old functionality, but it is also pointless: Users still need to hit the same number of keys whether they type “gq” or “Q” (the shift key and the “q” key). If the change at least had had a mnemonic advantage … An added complication is that messing with the keyboard short-cuts in Vim can have unexpected consequences in more advanced mappings made by the user: The conceptual binding between the key pressed and the action taken is so close and hard that it is quite common for users to just build their mappings as sequences of keys.* If the binding is changed, these mappings, some of which might have been used without fail for decades, could suddenly break.

   *It can be disputed whether this is a good idea, but fact remains that this is how it is mostly done. An interesting twist is that building mappings with “noremap” (i.e. simulate-a-set-of-manual-keystrokes-but-pretend-that-there-are-no-other-mappings) instead of “map” (i.e. simulate-a-set-of-manual-keystrokes) helps with isolating from such changes, but that is contingent on this change being done through a map. As it happens, it is done through a map, but that has the disadvantage that a user used to formatting with “Q” would now be forced to use “map” over “noremap” in order for his mappings to match his expected behavior… In most cases, the map/remap differentiation is not very helpful or practical, and I have always found it better to try to simply make sure that the keys of the “original” key–action-bindings are not used to identify mappings. (Disclaimer: To avoid confusion, the preceding is over-simplified. I have, for instance, skipped over topics like the mappings restricted to a certain mode and the possibility to use a “mapleader”.)

   A closer analysis would likely show similar problems with other entries and for any given entry at least some people will object to it. (Admittedly, to some degree, the argument holds that we cannot make everyone happy. However, this argument is not valid when it comes to supporting a mere matter of taste, and quite a few of the entries appear to be less centered on activating new features or objectively improving usability, and more on just picking the arbitrary preference of the file’s author—which is a lousy basis for defaults forced onto the rest of the world.)

   On the other hand, there is at least one point where Vim’s behavior is idiotic* for reasons of backwards compatibility, but where a change was not made… The “Y” action still “yanks” (copies) a whole line, instead of the line from the cursor column to the end of the line—as would have been expected, noting that “D” deletes from the cursor column to the end of the line, that “Y” behaves in this manner in the “vi mode” of e.g. Bash, and that there is another command, “yy” that already yanks the entire line (analogous to “dd”, which deletes the entire line). In fact, Vim’s behavior here has worsened over the years: There used to be an official setting to control this behavior; now, users have to define a “mapping” as a workaround.** Note that while there is a superficial similarity to the “Q”/“gq” issue, the cases are not analogous: Firstly, many users will instinctively type “Y” and expect the entire line to be yanked (because they use it that way in Bash, because they do what comes naturally, or because they are used to another instance of Vim, where they have corrected the problem); in contrast, the change of “Q” goes against instinct and expectation. Secondly, “gq” is typed with roughly the same effort and time consumption as “Q” (possibly even less than “Q”); in contrast, the recommended work-around for “Y” is the construct “y$”, which takes three key strokes and includes one of the more hard-to-reach characters***. The risk of breaking a mapping would remain, admittedly, but a prudent mapper would have written his mappings with “yy” instead of “Y” to begin with (and very many would already have re-mapped “Y” to “y$”).

   *To which could be added a number of options and behaviors where the default is likely to clash with the personal preference of the average user, typical programming guide-lines, or similar—without reaching the point of idiocy. For instance, in almost every project I have worked, the tab character has been banned and the standard indentation has been four spaces. Vim still adheres to the more 1970’s convention of using tabs for indentation and showing tabs as eight (!) spaces. (Eight not only being considerably rarer than four, but also likely rarer than two, possibly even three, today.)

   **In all fairness, it could be argued that creating a mapping in Vim is so easy that the official setting was redundant. However, through the existence of the setting, the developers at least acknowledged that there was a problem with the default behavior.

   ***y, shift, 4, assuming a standard QWERTY-keyboard. Compared to “Y” (shift y) this might take twice as long. Note that due to the keyboard-driven approach of Vim, this is an important concern, unlike in e.g. Microsoft Word.

2. defaults.vim gives the impression of being a set of application defaults, but in reality it comes closer to being a default ~/.vim/vimrc: It only applies when ~/.vim/vimrc is not present, forcing Vim’s documentation to present the suggestion to alter ~/.vim/vimrc to load it anyway*. Here the developers clearly have not made up their minds** what defaults.vim should actually do… In fact, it would have been better to never execute it automatically and instead giving the option to manually include it in ~/.vim/vimrc or /etc/vim/vimrc (with the added benefit that no code change would be necessary and that the initialization process would be simpler). A further snag is that based on names and naming conventions, the role as a default*** ~/.vim/vimrc is already filled by /etc/vim/vimrc, causing confusion and inconsistencies.

   *My recommendation is the exact opposite: Put anything from defaults.vim that you find sensible in your ~/.vim/vimrc and then forget that it ever existed. This way you are better insulated from later arbitrary alterations. Alternatively, an admin, especially when the sole physical user, might want to expand /etc/vim/vimrc and then remove/blank defaults.vim entirely. (But see below.)

   **This is almost always a bad idea in software development: The clearer the intent has been established and the fewer ifs-and-buts are made, the better things tend to work out. Software should be predictable and consistent. Failing to pay attention to this is one of the greatest causes of poor usability and high maintenance/extension costs that there is.

   ***Whether “default” is the right word can be disputed, since /etc/vim/vimrc is executed even when ~/.vim/vimrc is present. However, since the developers obviously (cf. above) want defaults.vim to be used by everyone, even those having a ~/.vim/vimrc, the difference is minimal. As an aside, system vs. user-specific config files are handled inconsistently for different tools, with many (e.g. Vim and Bash) always running both, and some (e.g. Xpdf) skipping the system file when the user-specific file is present (just like defaults.vim).

3. If an admin alters defaults.vim (e.g. to blank it, cf. above) chances are that the changes will at some point become overwritten or ineffectual, at the latest when Vim 8.1 appears*. A better approach would have allowed for some way to preserve overrides, e.g. the one used by Debian** to handle /etc/vim/vimrc: This file (which could be altered by e.g. a change in Debian’s handling of Vim) calls /etc/vim/vimrc.local, which is guaranteed to be left unaltered by Debian, and here an admin can safely put settings.

   *As can be deduced from the path on my system, which contains the version of Vim. Implication: 8.1, 8.2, 9.1, …, will have different files from 8.0, and changes to the 8.0 version will not affect other versions.

   **Whether a distribution should actually provide any settings for a tool like Vim is disputable (cf. elsewhere)—an originally blank /etc/vim/vimrc that is never altered by Debian might have been even better. However, the mechanism is still an improvement (and there are other cases where distribution settings are more legitimate).

4. The running order of defaults.vim and /etc/vim/vimrc is unexpected and suboptimal. The natural thing to do would be to run defaults.vim first, allowing /etc/vim/vimrc to override it. Unfortunately, defaults.vim’s role as a ~/.vim/vimrc stand-in applies with regard to execution order too. This implies both that upgrading to Vim 8 will break many deliberate global configurations that have worked for years and that admins will now have to alter two different files to set global defaults. Worse, since defaults.vim only (or so it is claimed) triggers when no ~/.vim/vimrc is present, the admins may never even notice the need…
5. Default settings intended for every single installation of a program, not just the local system, belong in the code, not in a config file. This might seem counter-intuitive, with the extra config file being a better abstraction and making alterations easier. However, there are several reasons why this must be so, including at least: The code must contain defaults anyway, so that the application can run in a well-defined manner if the configuration is missing, not readable, corrupt, whatnot; if we also have defaults in a file, there are now two alleged and potentially contradictory sets of defaults… (In the specific case of Vim, where the use of the file is conditional, this applies doubly. Notably, it is reasonably safe to conclude the two sets of defaults are contradictory, because otherwise defaults.vim would be almost pointless.) As implied by the previous, the config file is a potential, unnecessary, source of complications—and not limited to those already given. (What, e.g., if an admin alters this file instead of the intended system global configuration?) With a config file, there is a considerable risk that, deliberately or accidentally, the default settings will be altered* arbitrarily from release to release, forcing users and admins to use a different set of overrides, because they agree with the default on system A but not on system B.

   *There is no law against altering code-based defaults, but both the psychological and the practical obstacles tend to be higher, even considerably higher. Altering a config setting is done in virtually no time at all; recompiling the code of a larger application can take several minutes, even many minutes, in extreme cases hours, depending on the application and the machine doing the compilation. Alternatively, from a different point of view, developers tend to be more responsible where code changes are concerned.

6. At least with regard to the “man page”, the change is poorly documented. In fact, skimming through it, I can see no mention at all. defaults.vim is definitely not mentioned in the “FILES” section and what happens when the “-u” and “-U” flags are used is potentially* undefined.

   *The claim “All the other initializations are skipped” (resp. “All the other GUI initializations are skipped”) is made. However, with defaults.vim finding no mention on the page and with it’s inconsistent role, I would not trust the claim without doing a practical experiment. To boot, the Vim-internal help page claims “Most other initializations are skipped” and also makes no mention of defaults.vim… (Cf. “:help -u”.)

7. Unlike e.g. ~/.vim/vimrc there is no obvious way to suppress defaults.vim on the command line. Instead, ‘set the “skip_defaults_vim” variable’ is the advice given. (Cf. “:help skip_defaults_vim”.) This normally assumes that Vim is already running and that it is either already to late or that the statement is in another config file (and, obviously, before defaults.vim is executed). An obscure workaround along the lines of “vim –cmd skip_defaults_vim=1” (“let skip_defaults_vim=1”? “set skip_defaults_vim=1”?) might possibly achieve that goal, but compared to e.g. “vim -u” this is a hard-to-remember chore of a call, which is by no means obvious from the man page.

The “skeleton files” are a particularly misguided complication: Through this mechanism, configuration files with default content are automatically created in a user home directory upon its creation—for no good reason. Config settings that are supposed* to be valid globally belong in the global config files (e.g. /etc/bash.bashrc), not in the user ones (e.g. ~/.bashrc). Putting them in user files creates an immense duplication and maintenance problem—to the point that a global configuration file must still be present** and most likely duplicating almost everything in the skeleton file… If not, then users will have different defaults depending on when they were created. At the same time, users with many accounts have additional problems in putting everything in order, because there is one extra thing that they have to check. In common constellations like mine (viz. a single physical user of a system with many user accounts) the skeleton files are a positive hindrance, because if it had not been for the skeleton files, changing the global config would have been enough and there would never have been a user config. The automatically created user config has to be deleted for each and every individual user in order to ensure that I have the settings I want***.

*In the sense of a default. Only very, very rarely is it legitimate to enforce settings in such a manner that users cannot override them—for that matter, only very, very rarely is it even technically feasible.

**Although there may be rare instances when it is empty (and when it may or may not be removable for the time being). If that is the case, chances are that there was no reasonable content to put in the skeleton file either…

***Or the skeleton files deleted in advance, which is very easy to forget and many will not even be aware of them before a number of accounts have already been created. (For instance, yours truly when he set up his first system.)

Generally, the settings that Linux distributions (e.g. Debian) try to shove down the users’ throats are often ill advised or even dangerous (e.g. a umask that lets everyone read the files of everyone else) and rarely anything better than the personal (!) preferences of someone. It is highly disputable whether a distribution should make such settings at all. As a rule there should be three parties involved—and the distribution is not one of them: The individual user, the entity running the current system (which often coincides with the individual user), and the maker of the specific piece of software. Of these, even the second usually has little or no reason to intervene where user tools like Vim and Bash are concerned, leaving us with what comes close to a two-fold partition: The user and the software maker.

Advertisement