Michael Eriksson's Blog

A Swede in Germany

Posts Tagged ‘software

Tax filings for 2020 / The German IRS and Elster (again)

leave a comment »

And again fucking, unusable Elster!

Among the problems encountered:

  1. I began the process in (likely) July, by creating the needed documents and making some preliminary entries. With one thing and another, the rest of the job had to wait, which should have be no problem in light of a COVID-related and blanket three-month extension of the deadlines.

    But no: A few months later, I received emails that some of these documents would now be automatically deleted by Elster, because they had gone unedited for too long. I wrote back and forbade this deletion, while pointing out that this was an inexcusable act of user hostility. (Even by the standards of Elster and the German “IRS”.) I note that there is no advantage to such a deletion, but potentially enormous disadvantages.

    They were deleted nevertheless.

  2. The field for messages to the IRS still (!) does not take line-breaks.
  3. That I had added such a message brought Elster into a destructive loop, where (the German version of) “check document” led to a semi-error page that pointed out that I had left such a message (and why?!?!), which repeated again and again on subsequent attempts. The document was still sendable, but this broke the apparently preferred-by-the-IRS workflow of check-and-send-from-the-results-page. (Cf. an older text for these absurdities.)
  4. The “check document” for the main document originally failed on the claim that I needed to indicate whether I had received COVID support—even when I had not. There was no obvious field for this anywhere, there was no indication of help on how to do this, and only an internet search revealed that I needed to add an entire new attachment to the document, which then contained two fields, one for yes/no on whether I had received help, and another for the amount for those who had.
  5. Generally, “check document” is extraordinarily incompetent at indicating where an error (real or imagined by Elster) is located and makes odd jumps. (And there is not or only rarely a visual indication which fields are mandatory in advance.) For instance, in the EÜR document, there are two fields that seemed irrelevant to me, but where “check document” insisted on an entry. I made one entry (indicating 0) and clicked the confirmation button for that entry. Now, I obviously wanted to continue with the second field, which was immediately below the first. But no: Elster took me back to “check document”, forcing me to go back and find the relevant field again.
  6. Did I mention that mandatory fields are usually not marked as mandatory? (Yes, I did.)
  7. I copied a calculated-by-Elster value from one document to another (and why is this not handled automatically?!?), because this value was needed as an input in the second document. The output value contained both a thousand separator and decimal places (and a decimal separator). The input field required a value in whole Euro (no decimal places) and could not cope with the thousand separator, giving me two separate error messages.
  8. A great help in filling out the EÜR could have been pre-filled fields based on last year (which works well with the other documents, where the advantage is lesser), so that I could e.g. see where I had put postage and where train rides and where this-and-that. Specifically in the EÜR, this does not seem to work, however, as only trivial fields (like name and identifiers) are filled out. Then it is down to guesswork or Internet searches to find the right fields.

And to this a few things I might have forgotten, the great many problems discussed in earlier entries, the incomprehensible German tax system, …

Fucking amateurs!

Written by michaeleriksson

October 29, 2021 at 10:35 am

Posted in Uncategorized

Tagged with , , , ,

Tax filings for 2019 / The German IRS and Elster (again)

with one comment

Earlier today, I filed my (German) taxes for 2019—and, for once, with a few days to spare. This through a combination of a general increase in the last date for filing (July 31st; previously, Mai 31st), my less stressful workload, and the fact that I had less positions to file.* It still cost me several hours distributed over two days, to get everything in order and to use ElsterOnline, that utter bullshit tool that the German “IRS” has forced down the throat of the users.

*I spent half of 2019 on a sabbatical and then switched from IT consulting to writing novels, with no bills issued, no income, and much less costs for e.g. hotels and travel than in previous years.

I am not going to give a complete overview of Elster, as I have discussed it repeatedly in the past.* However, a few new (?) observations:

*Search for “Elster”, “IRS”, and/or “Finanzamt”.

  1. There is a new free-text field where the user can add a message to the IRS within the filing—finally: this has been years overdue.

    But: It is not possible to add line-breaks in this message. This repeats an inexcusable error, hostile to both the user and the IRS staff that later works with the filings, which was previously present in the specialized form for sending (external) messages. In that separate form, this error has been fixed—but it is still repeated here. Absolutely incredible!

  2. On several occasions, I tried to run my mouse over a field with outdated values to mark the contents, hit backspace, and then enter the new data. This was simply not possible, which is absurd for a functionality that works out of the box with a regular HTML form—unless somehow sabotaged, be it out of incompetence or malice. Instead, I had to click into the field, right-arrow until I was at the end, and then hit backspace until the field was empty.* This is the more annoying as the form based input and the structure of the forms more-or-less forced use of a mouse for tasks like navigating. This way, the user is forced to constantly switch between keyboard and mouse in a manner that goes too far. (And does it for no legitimate reason.)

    *In my impression, I was always, by force, put at the beginning of the field, with no ability to “click” to another position; however, I did not verify whether this was true. It is also conceivable that I could have “deleted” my way backwards with the “delete key”, but it is awkwardly placed and requires a simultaneous “shift” on my notebook, so that would have been more work—if it actually did work at all …

  3. One of the forms had two (times three; cf. below) fields for the Steuernummer*, one in a “cover” part, one in a content part. The latter was correctly imported from last years forms; the former had to be entered manually. WTF!

    *An identification number used by the IRS.

    To make matters worse, the forms insist on dividing the Steuernummer into three parts, each with a field of its own, which implies that a simple one-step copy-and-paste is not possible. To copy it within a form, with no additional sources, the tax payer then has to go forward one (or more?) page(s?), copy part one, go back to the original page, paste, go forward again, copy part two, etc., until all three parts are filled. (Personally, I committed parts one and two to memory and copied just the third part, trading a slight risk of errors for a reduced work load.)

  4. The data import from my previous filing was not complete. At least the VAT portion likely had not one single data field filled, which makes the new filing harder: there are a great number of obscure, poorly named, and poorly explained fields, and having the ability to just look at the old fields makes it much easier to identify which to use this year. Moreover, when I cannot rely on the old fields being pre-filled (if with outdated values), I do not just have to identify the correct fields, I also have to go through the sum of all fields on a just-in-case basis.
  5. While data import from the old filing was possible, I had no way of actually looking at the old filing, e.g. for comparisons per the previous item. For some reason, likely an arbitrary, unnecessary, and destructive time limit, they cannot be opened.

    And, no, there appears to be no way to save them locally in a reasonable format. (Something that I tried with my new filings, and likely last year too.) The only possibility to download the data, short of taking screenshots or saving countless individual HTML files, is a “save as PDF” functionality. This is sub-optimal and limiting to begin with, but, worse, this does not work at all on my computer (for reasons unknown). Odd: This should be a trivial task if implemented correctly: generate the PDF file server-side and then just let the browser download it. Possibly, the idiots are actually stupid enough to try generation client side, which is a recipe for unknown errors.* If it is server side and they still have bungled it, well, that is even worse.**

    *No, it cannot be justified by data protection. Such concerns are often legitimate, but here we had no data that was not already present and (at least somewhat) permanently stored server-side.

    **Software errors happen even to competent developers, but here we have a system that has been handling the 2019 taxes for almost seven months and is now in a high intensity phase. Not having fixed the problem by now, or having introduced it in the last few days, would be horrifyingly negligent. I also note that there was no error message of any kind, which would have been a must, had there been e.g. a temporary back-end problem, say, due to a temporary overload or system failure.

  6. Two fields were mandatory despite my having no value to provide (regarding transfer of assets from the private to the business sphere and vice versa). Here I had to add two entries of “0’, for no good reason. And, no, I could not just pick an existing field and enter the value “0”: these fields (in a wide sense) contained lists of fields, where each entry had to be manually added. Presumably, the IRS expects a detailed and enumerated list of each individual asset transferred, and it would then make sense to allow an empty list when no transfer has taken place. (This is indeed the case with other “list fields”.) But, no, an empty list was not allowed, and to signify that I had transferred no assets, I had to create two single list entries with the value “0” and an additional dummy “reason” (“name”, “details”, or whatnot).

    This is a “Software Development 101” mistake.

    (I have no recollection of this problem from prior years.)

I can only reiterate my yearly observation that this tool moves on a level of incompetence that is mind-numbing, including obviously faulty behavior, a complete disregard for established conventions, an extremely confused (and confusing) user interface, etc. As a former software developer, it boggles my mind that this type of shit can be made by (alleged) professionals—and while wasting tax payers money. Yes, I know, the government and incompetence, but still …

Written by michaeleriksson

July 29, 2020 at 6:34 pm

Posted in Uncategorized

Tagged with , , , ,

Undue checks of values

leave a comment »

A common annoyance with poor software is undue intolerance against values that are, in some sense, faulty. (And, no, this is not a post about the political Left …)

Checks for correctness and consistency can be a great aid, as can automatic warnings of errors. However, often, the baby is thrown out with the bath water.

Consider e.g. Alpine, an email client that I use extensively: It has a field in the configuration to specify the default sender address. Here I have simply specified “@” and my domain because I use a great number* of different user names for different tasks (mostly to reduce the damage when one address falls victim to spammers). The idea is that I have this string pre-filled in the “From” field and then just need to add the right user name.

*Too many for a solution using e.g. Alpine’s role system to be a good alternative.

But what happens? If I begin to compose an email, the “From” field is just filled* with INVALID_ADDRESS@”.SYNTAX-ERROR.” (quote signs present in the original), presumably to indicate its dissatisfaction with the missing user name. The actual value entered by me is neither visible nor retrievable and there is no reasonable world in which this is a good reaction. A check when the user attempts to send, by all means, but not when a default value is retrieved or entered. If there are objections to the default value, they should be uttered when and where the default value is configured;** however, here such objections are not reasonable, as use cases like the above are quite common.

*The actual field. Contrast this with keeping the “faulty” value and displaying a warning message next to it. (Which also would have been acceptable.) Writing this, I begin to suspect that this is not so much a deliberate choice as poor programming, that there is an internal consistency check when retrieving the value, that this check gives an unnecessary error message, and that the error message is blindly taken over as the value.

**This is not the case with Alpine. The explanation might be that the the entry mask deliberately has a tolerance that is later arbitrarily removed, or that this config value is part of a larger string, which is not parsed or verified at the time of entry.

The result is that I have to delete the error message, write the user name, and copy the remainder of the address from elsewhere, i.e. one step more than without this configuration and two steps more than if it had worked reasonably. Time to remove it …

Of course, these extra steps occasionally lead to errors. For instance, when I use post-by-email with WordPress, I usually just “reply” to the last post, switch out Subject and Body, and re-enter* the email address. But today, with the three steps needed for the email address, I forgot the Subject and published a text under the same title as the previous (entirely unrelated) text …

*No, Alpine is not smart enough to handle replies to own messages correctly, i.e. that the old address is kept. Instead, the configured one is used (if present, else the field is, probably, empty).

Other examples include e.g. applications that prevent any entry of faulty information, even without saving*, e.g. that a numerical value using a decimal point is not allowed in a German application expecting a decimal comma. Then, instead of copying a (read-only) value from a PDF file or output from a calculator into the field, changing the point to a comma, and then continuing, the user is forced to copy the value, paste it in an editor, edit the point to a comma, re-copy it, and then paste it in the field.** Or consider fields that allow entry of most, but not all, legal values or makes normally optional parts mandatory.***

*In many cases, even the saving of faulty values can be beneficial, e.g. that a numeric field can be filled with a “TODO”, and that the application merely gives a warning that the input is faulty. However, this is not always trivial and rarely worth the benefit, as it might require switching a numeric internal data type to a string data type or similar.

**Yes, this could be solved e.g. by some type of keyboard macro, but it is not a sufficiently common scenario to be worth the trouble—in stark contrast to writing a better functioning field that e.g. allows entry of any value and just shows a warning message or allows entry but not saving.

***I do not remember any of the specific cases off the top of my head, but consider, again, email addresses: These can be quite complicated, and e.g. a simplistic name-plus-@-plus-domain parser would disqualify many legitimate versions. Vice versa, an idiotic tool could make the idiotic display name idiotically mandatory.

Written by michaeleriksson

July 26, 2020 at 2:36 pm

Follow-up: Further Firefox screw-ups

leave a comment »

Since my original text, I have read some of the comments on the main Mozilla* page dealing with this issue.

*Mozilla develops Firefox. For convenience, my earlier text just spoke in terms of Firefox.

These comments show how dire the situation is—to the point that Firefox might disqualify it self as a serious browser candidate:

  1. There are many users who have been very hard hit. One commenter mentions how his password manager* with (IIRC) roughly 150 passwords has been disabled, which might be even worse than the NoScript issue. It is easy to imagine a user being cut off from email, blogging, social media, …, through such an issue. Worse: If this happens in a commercial setting, an entire business could be temporarily crippled.

    *However, I would advise against using an in-browser password manager (at least, where important passwords are concerned). This for reasons like the above, the greater risk of hacking, problems that can ensue when switching computers or trying to run several browsers in parallel, whatnot.

  2. The attempts by Mozilla to fix the issue appear to be slow and have not been met with enthusiasm.
  3. Mozilla’s preferred work-around, awaiting a proper fix, is to enable “studies”.

    This work-around has the side-effect of allowing Mozilla to run various spy-on-the-user functionality that many users have disabled for very good reasons—and that more-or-less everyone else should have disabled. This, obviously, amounts to Mozilla screwing up and then gaining an unfair advantage over its users through the screw-up…

    Further, this work-around can take up to six (!) hours to take effect, without an additional workaround (specifically, manual manipulation of the “app.normandy.run_interval_seconds” key). Mozilla’s stance: Wait, without attempting further work-arounds. Depending on timing, however, six hours can amount to an entire day lost, including for some who need the Internet extensively for professional reasons.

    Further, it is not even available on all Firefox instances, including those that use or are based upon the ESR*.

    *An older version with long-term support that is suitable for those in need of greater stability and/or who develop off-shot browsers, e.g. the Tor Browser.

    Further, some users who believe that it should work in their browsers report that it does not. (I have not kept tabs on the details and could be wrong, but I am under the impression that some of them were on the latest version—and, thus, correct in this estimate. There are some murmurings about some other key that might need to be manipulated, but, again, I have not kept tabs on the details.)

From a Tor-Browser perspective, there is an additional* complication through NoScript being used by the Tor Browser internally to implement some security features. The disabling of NoScript implies e.g. that the “security slider” will be highly misleading or malfunctioning. As some mention, such errors could cost someone his freedom or even life…** This, obviously, points to issues with the Tor Browser, including that it has chosen a dangerous path to implement security (dependent on the efforts of third parties) and that it has failed*** to protect it self against the risk of this type of deactivation.

*Which I had not realized when writing the first text, but which is clear from the page I linked to.

**Tor Browser is used by many dissidents in hostile regimes—not just regular surfers who value anonymity.

***In my understanding, such a protection and a protection mechanism is already present for some other plug-ins that come installed with the default Tor Browser, including “HTTPS Everywhere”. Correspondingly, an awareness of the possibility must have been present.

Written by michaeleriksson

May 6, 2019 at 3:04 pm

Further Firefox screw-ups

with one comment

And Firefox does it again:

A few days ago, my Firefox* suddenly claimed that the NoScript-plugin had been deactivated—and left me no means to reactivate it. There was precious little to be found on the topic on the Internet (at the time, cf. below), but I did find the tip that setting the “xpinstall.signatures.required” key to “false” might solve the problem. It did—but at an increased security risk** and after I had wasted a fair amount of time.

*The modified Tor Browser to be specific; however, the problems all originate in or surrounding the vanilla Firefox. Indeed, in the vanilla Firefox I might have been worse off, because the discussed key might not function…

**This key relates to signing and verification of plugins. Setting it to false could allow the installation of malware-plugins.

Today, it happened again in another browser installation*. Going back on the Internet to re-find the key to change, I found many more relevant seeming hits, e.g. [1] and links on that page. Apparently, the Firefox developers have screwed up severely, causing perfectly legitimate, signed, and previously verified plugins to be marked as non-verifiable during the last few days… (I have not looked into the exact details.)

*I have several different installations for different purposes.

However, this screw-up is not the main problem here (bad, yes; but not the end of the world—shit happens). Far more problematic—and further proof of a user-despising attitude:

  1. The plugin was deactivated without querying the user. Correct behavior would be to inform the user and request his decision as to what should be done with the plugin.
  2. There was no non-trivial and well-documented way to re-activate the plugin. However, such a way should have been present, e.g. through a “re-activate” button in the plugin view—if need be, with a big warning sign and a “Are you really sure?” query.
  3. An already installed plugin, which was previously deemed safe, was de-activated without the plugin it self having changed. Normally, such judgment should only be passed during the original installation.* On the outside, it might be sensible to allow a manual override by the developers due to new information, e.g. in that something that was previously considered secure and friendly has since proved dangerous or hostile. This could take the shape of e.g. (depending on the feature/software/whatnot under discussion) a manual key revocation or a manual blacklisting.

    *For this type of check. Other checks, e.g. virus scans, might legitimately allow for later re-evaluation. There might also be other types of files, installations, programs, whatnot that might legitimately be treated differently (but no obvious example occur to me, off the top of my head).

  4. The deactivations took place during on-going browser sessions and (at least, the first time) the notification of deactivation was belated: The first sign that something was wrong was that pages behaved differently than they should; the notification came a little later. This opens security and other risks; e.g. with NoScript,* that the user visits an untrusted or unknown site believing that JavaScript is off, while it actually is on—which is a much, much greater security risk than that posed by an already installed plugin. To boot, NoScript comes with quite a few security protections other than JavaScript on/off, e.g. relating to “click jacking”—these, too, are disabled with the plugin.

    *It is hard to give general examples, because the exact consequences vary from plugin to plugin.

  5. This could only happen because Firefox makes connections behind the user’s back, giving him no say and no transparency. (In particular, I have my browsers set to manual updates only. If this had been a side-effect of a user-allowed security update, it would have been a little less problematic.) No application, browser or other, should make such connections without having informed the user and having received his permission. This for a number of reasons, including the principle of having the user in control, the risks to the users privacy, the added amount of data (which can still be an issue on e.g. a smart-phone), the possibility that the application misbehaves or malfunctions when no Internet connection is present, ditto when a company goes bankrupt/turns off a server/is blocked by an ISP, …

    (Unfortunately, very many other software-makers also do make such connections.)

Written by michaeleriksson

May 6, 2019 at 4:25 am

Follow-up: The misadventures of a prospective traveler

with one comment

Recently, I had great problems booking an airplane ticket to Sweden, ultimately resorting to using a travel agency, which required both an unnecessary fee and a trip on foot.

For my return to Germany, my seasoned-traveler father booked the ticket from his computer.* While this worked in one go, the service that he ended up using (“supersavertravel”) was abysmal: The entire interface seemed geared at one thing and thing only—to coerce the user into buying expensive additional services that he did not need. This to the point that it was necessary to explicitly decline these many services and to do so individually—no, I do not want a hotel; no, I do not want extra insurance; no I do not want a rental car; no, …; no, …; no, …; no, …; no, …; no, …; no, …; no, …; no, …; … I even seem to recall (but could be wrong) that there was an additional query after submit along the lines of “You have not chosen this-or-that. Are you really sure that this is deliberate?”… Utterly inexcusable was the checkbox to decline spam: Where more main-stream businesses use a checked checkbox to imply “I consent to be spammed”, here the user needed to check the checkbox to decline spam…** The confirmation email, unsurprisingly, contained much more advertising and attempts to bring unneeded services to my attention than it did confirmation…***

*I had left my own computer in Germany in order to travel lightly; and only bought a one-way ticket to Sweden, because I did not know how long I needed to stay.

**Implying that the main idea almost certainly was to trick users into making the wrong choice.

***This in stark contrast to EuroWings below, where the confirmation email was informative, to the point, and did not even abuse HTML for the email text. (Portions of [1] contain some discussion of why HTML has no place in emails.)

A second trip turned out to be needed.* I tried EuroWings again, and this time everything actually worked.** However:

*My mother’s old house is being sold, and the time needed to sort through my own old books and whatnot turned out to be much longer than I originally thought.

**Contrast this with the original text. This time, I made sure to pay by credit card (3D-secure was not needed) instead of invoice. I do not know whether the old issue was a temporary server-side problem, a problem with a workflow somewhere, or whether there is some problem relating to invoices that I now ducked. (Regarding workflow: In my experience, most QA checks tend to run through fairly straight-forward scenarios, meaning that a scenario that involves the user e.g. going back to a previous step, responding to a validation error, actually reading the T-&-C’s, whatnot, is often left untested. These scenarios, however, are disproportionately likely to cause errors—especially when Ajax and other “state sensitive” technologies are used.)

  1. EuroWings too tried to advertise additional services, if far fewer, in a manner that detracted considerably from usability and prolonged the process unnecessarily. Unlike with “supersavertravel”, they were all opt-in, but it would be so much better if they were all collectively moved to a separate and skippable step, especially since they will only ever be interesting for a small minority of the customers. (Be it because they have no need, already have made other arrangements, would lose points with some program by booking/buying somewhere else, …)

    Hotels are a potentially odd area. In the specific context of a flight, admittedly, I can see many cases where it would be helpful to “co-book” a hotel. However, hotels are offered more-or-less everywhere, including for e.g. train-travel. In most of these cases, booking a hotel together with the means of travel turns the reasonable workflow on its head: It is usually the hotel, not the means of travel, that is the bottle-neck, and a reasonable workflow would then involve finding and booking a hotel first and finding means of travel second.

  2. Integrating a please-do-not-spam-me checkbox in the main pages would be trivial. Nevertheless, declining spam is only possible through visiting a separate page. On this page, moreover, the email address has to be added redundantly and manually, and it could be (depending on internals and the exact steps used by the customer) that the spam rejection only takes effect after the fact, e.g. in that the one click somewhere activates an unethical implicit consent to spam, while the other page only revokes this consent. This would leave a window of abuse open.

    Frankly, this is so common that legal measures are necessary: It must by law be forbidden both to use implicit consents and to require explicit rejections for any use of personal data (in general, but the more so for email data) that is not central to the process for which the data was provided. This, notably, from the customers perspective—not the data collector’s. (For instance, the data collector might see sending a news letter with advertising as a central part and having to send a confirmation email as an annoying negative, but for the customer it is the other way around.)

  3. There are potentially redundant entries for email, including one for the actual transaction and one for please-notify-me-in-case-of-delays. It would be better to keep them as one per default (if in doubt by automatically filling the one with the other and allowing a manual change). Further, the entries are likely made in the wrong order for most users, with a non-mandatory entry of please-notify-me-in-case-of-delays on one page and the mandatory actual transaction address on a later page. Further, the former came with a pop-up upon submit that urged me to fill in this non-mandatory field anyway—which seems more like fishing for email addresses than an attempt to provide a service.

    Why had I left the email address out? Well, I knew from my earlier attempts* that if I did provide an email address for notifications, then I would also be forced to provide a cell-phone** number—absolutely idiotic.

    *The attempts in general are described in the original text, but details like the above were left out.

    **Note that I currently do not even have a cell-phone. Also note that cell-phones too can be abused for spam (through SMS).

Written by michaeleriksson

February 25, 2019 at 1:08 am

The misadventures of a prospective traveler

with 3 comments

Three issues that have collaborated to drive me nuts today:

  1. I had promised my father and step-father to try to come to Sweden over Christmas or New Year’s.

    For this purpose, I had already made several attempts to find suitable tickets with the Tor Browser (relayed through Tor or directly over my non-torified Internet connection, even with JavaScript enabled). This had proved very annoying and unproductive. For instance: The Lufthansa site simply does not load at all, it just hangs with a perpetually waiting tab. The SAS site loads, appears to work, allows me to fill in all criteria—and then does nothing when I try to submit the search. Ditto EuroWings. A meta-search that I attempted to use was hopelessly slow, insisted on (with every single search) re-including flights with a change of planes,* and also insisted on re-ex(!)cluding potentially cheaper late-night** flights. At least one site interrupted my search by having a JavaScript pop-up demand that I take a survey to improve its usability…*** Almost invariably, these sites had various annoying blend-ins/-outs, animations, overly large images, poorly structured pages, …

    *Which roughly doubles the travel time between Düsseldorf and Stockholm (that have the most suitable airports) and is highly unwanted by me.

    **A smaller negative than changing flights…

    ***First tip: Do not molest customers with such pop-ups!

    Having given up and postponed the search twice already, and now having the 28th of December, I decided to install a brand-new vanilla Firefox, in the hope that at least the SAS/EuroWings problems would be explained by some version incompatibility with either the Tor Browser (as a Firefox derivative) or my version being too low.*

    *Many web-sites, in the year 2018!, still fail to make browser-agnostic implementations, insist on very recent versions of browsers, and similar—usually with no indication to the user that they do. (And visiting an eCommerce web-site without JavaScript on is more-or-less bound to fail.)

    First attempt, SAS: Everything seemed to work, but the web-site was now even more visually annoying than before. The choosing of dates, for some reason, worked in a different manner than before and was highly counter-intuitive. Seeing that there also (not entirely unsurprisingly) were no good flights prior to the New Year and that prices were unnecessarily high, I decided to look elsewhere first.

    Second attempt, Lufthansa: Still did not load…

    Third attempt, EuroWings: To my great and positive surprise, everything appeared to work perfectly, showing me timely and much-cheaper-than-SAS flights. Things kept working as I began my purchase, entering name, address, and whatnot—and I even found the alternative to pay by invoice!* Alas: As I tried to confirm the last step, I was met with an uninformative error message and the request to start again from the very beginning. There was, in particular, no mention of factors like the last few seats on that flight having been suddenly snatched by someone else, or invoice payment not being possible on so short notice**. Before starting over from the beginning, I gave just the last page a second try. I re-entered some (inexplicably deleted) information and re-submitted. Same error message—but now followed by an intrusive pop-up suggesting that I start a chat with someone… I clicked the dismiss button—but, instead of disappearing, the pop-up did a weird and time-consuming animation and kept blocking a significant part of the page even after the animation was finished. At this point, I had had enough, closed my browser, and decided to find other means—which will likely amount to going to a physical travel agency and visiting at some point after the New Year…

    *Thereby removing one of my last doubts, namely the risk that I would be forced to pay with a combination of credit card and 3D-secure—which I (a) have never attempted with my new bank, (b) fear would involve the idiotic use of SMS (I do not currently have a cell provider), (c) had found to simply not work at all with my previous bank (the to-be-avoided Norisbank).

    **For which I would have had some sympathies–but, in that case, invoice payment should never have been offered in the first place.

    These events are the more annoying, seeing that there actually was a time when it was reasonably easy to handle tasks like these over the Internet—it really is not that hard to implement a decent search–choose–pay UI. However, year for year, the usability of various Internet shops and whatnots grows worse and worse, and appears to make more and more specific demands on the browsers. Much of this goes back to the obsession with Ajax. Credit-card payments are also not what they used to be, being much more laborious and likely to fail than in the days before 3D-secure and similar technologies. Worse, from the customers point of view, they likely lead to a net loss of security, whereas the stores and involved payment entities see the gains.* Then, if not relevant above, we have the inexcusably poor efforts of various delivery services, notably DHL, which often make it less of a fuzz to go to a store and pick up a purchase in person…

    *For the customer, the risk that someone will manage to fake a payment is reduced, but if someone does, he has very few options to prove that he was the victim of fraud. Without 3D-secure, the burden of proof was on the other party, and the customer had very little risk at all (short of additional work). The merchants and credit-card acquirers, on the other hand, can have large costs and losses when a fraudulent purchase is followed by a charge-back—and 3D-secure helps them, not the customer, by reducing this risk.

  2. Installing and setting up Firefox proved to be a PITA. Apart from the issues of the next item, I note that any version of Firefox has tended to come with very poor default settings, including default UI behavior; and that the “new” Firefox is highly reduced compared to the “old”.* After installation and prior to my attempts at finding tickets, I spent at least five minutes going through and correcting settings—that were then, obviously, only even valid for that one user account**…

    *The changes would be enough for a long own text. For now, I will just note that (a) that the GUI-configurable settings have been reduced to a fraction of their previous scope, (b) the general attitude described in e.g. [1] is continued.

    **I have a number of user accounts for different purposes, in order to reduce the risk of and damage from security breaches and whatnots. This includes separate accounts for eCommerce (the current), my professional activities, ordinary surfing, porn surfing, and WordPress.

    To boot, a new dependency was installed: libstartup-notification0. I did some brief searching as to what this is, and it appears to be just a way for an application to change the shape of the cursor during startup… (Beware that my information might be complete.) Firstly, why would I want the cursor to change?!? Secondly, even if this was seen as beneficial, it certainly is not reason enough to add yet another dependency—there already are too many useless dependencies, many of them recursive (also see portions of a text linked below).

  3. The idiotic Debian “alternatives” systems and the “desktop nonsense”.

    Disclaimer: Some familiarity with Debian or similar systems might be needed in order to understand the below.

    When a Debian user installs an application, e.g. Firefox, /usr/bin/firefox (or whatever applies) does not contain the Firefox binary—nor even a link to the Firefox binary. Instead, it links to an entry in /etc/applications, which in turn links to the actual binary (unless a certain setup has even more indirections involved). To boot, this system is administered by a poorly thought-through tool (update-alternatives) and/or configuration; to boot, it is vulnerable to applications arbitrarily overriding the status quo, as well as adding pseudo-applications (e.g. x-www-browser) that at least I simply do not want polluting my system.

    In fact, these pseudo-applications are likely the reason why this system was added in the first place—because e.g. x-www-browser can be “provided” by a thousand-and-one different real applications, it would be highly complicated to work with straight links, let alone binaries (especially when one of the “providers” is removed). For real applications, there is a much better way to solve such problems—namely, to just link e.g. /usr/bin/firefox directly to the usually sole instance of Firefox present and give the user an explicit choice of the “default” Firefox every time a new Firefox version was installed or an old removed.

    Why do I not want these pseudo-applications? Firstly, they bring me and most reasonable users, at best, a very minor benefit (for which they bring the cost of the indirections and the greater effort needed when looking for something). Secondly, the “providers” are usually sufficiently different that unexpected effects can occur.* Thirdly, they are often used by other applications in a manner that is highly unwanted: For instance, one of the alleged main benefits of x-www-browser is that any other application, e.g. an email reader, should have an easy way to open an HTML document, without having to bother to check what browsers are installed—but I absolutely, positively, and categorically do not want my email reader to even try this. In a saner world, this would be something configurable in the email reader (and only there), and those who want this endangerment can configure it, while those who do not want it simply do not configure it. By having x-www-browser, the user no longer has such control. Worse: Since the real application behind x-www-browser can change without his doing (be it due to presumptuous applications or an administrator with different preferences), the effects can be very, very different from the expected—e.g. that a known browser with JavaScript, images, and Internet access disabled (appropriate for reading e.g. HTML emails) is replaced with an unknown browser with everything enabled. (Which, in combination with email, could lead to e.g. a security intrusion, leaking of data to a hostile party, activation of unethical tracking mechanisms of who-read-an-email-when, and similar.)

    *For instance, there are many highly specific tool families, e.g. awk, whose members will superficially appear to be and behave identically (much unlike e.g. Firefox and Chrome/Chromium, as x-www-browser candidates), but will have subtle differences that can lead to a failed execution or a different-than-expected result in certain circumstances. Such problems, especially when undetected, can have very serious consequences. It is then much better for the user to, depending on circumstances, pick the specific awk-version he needs by explicit call (=> the alternatives system is not needed), make sure (for a one-user system) that he only ever has one instance installed and use the generic “awk”-name (=> the alternatives system is not needed), or restrict himself to only the common base of identical features. In the last case, the alternatives system would have some justification—however, it would place a very high burden on the user in terms of not making mistakes, might still fail due to undocumented differences or bugs, and is vulnerable to other differences, e.g. regarding performance. Obviously, this would also reduce the available capabilities of the tool in question—in many cases, quite severely.

    Similar remarks concern the “desktop nonsense” (which would deserve a long text of its own; a partial treatment is present). In this particular case, there are at least two* further mechanisms (/usr/share/applications, /usr/lib/mime/packages/) that cause similar problems, including allowing e.g. email readers to launch things that they should not launch. I have used the tool chattr to forbid additions to these two directories; however, due to the incompetence of the apt implementers and/or package builders, this is only a partial help: Despite these entries being unimportant for the actual functioning of the system/the installed application, the chattr-setting leads to a hard error from the apt-tools. I know have to “de-chattr” the directories, re-attempt the install, manually delete the added files, and “re-chattr” the directories … Effectively, I do not prevent the directories from being polluted—instead I trade an increased work-load for the benefit of knowing when I have to manually clean them up after pollution.

    *Proof-reading, I suspect that /usr/lib/mime/packages is not strictly desktop related, and might better have been treated as a third area. In the big picture, this does not matter. (And I do not have the energy to sort out “what is what” at the moment.)

Written by michaeleriksson

December 28, 2018 at 6:43 pm

WordPress at it again: Backups and security through obscurity

leave a comment »

The stream of outrageous incompetence by WordPress continues…

For the first time in half an eternity,* I decided to download a backup of my WordPress blog. In the past, this has resulted in (most likely) a zip-file being offered for saving. Today, however, I was met with a message that a link to this zip-file would be sent to my email account… The link, in turn, was valid for a full seven days, downloadable by any arbitrary Internet user, and protected only by (what I hope was) a random sequence of characters added to the file name. This is not only highly user unfriendly—it is also a great example of idiots relying on “security through obscurity”: It is true that no-one who does not know the random part of the file name (obscurity) will be able to download the file (“security”). However, with the state of email security, a great number of hostiles** would have had the opportunity to grab the email contents and find the link. To boot, this approach opens the door for simple errors or oversights by WordPress to open an unnecessary security hole, e.g. if a list with the current such links is similarly weakly protected… Other risks might exist, e.g. that it might be easier for a family member or visitor to get hold of the email/link than access to the WordPress account.*** In contrast, with the old system, the backup was transient and protected by the normal user-account controls—and if those are breached, it does not matter how backups are handled…

*This is not as bad as it sounds: I write all posts offline, with separate backups, in the first place; there are not that many comments; and I intended to leave WordPress at some point anyhow. Correspondingly, little data would actually be lost, if something bad happened.

**Notably, not necessarily parties hostile towards the individual blogger. More likely, it would be someone hostile towards WordPress or who sees WordPress as an easy source of data. Such a hostile would then watch the outgoing traffic from the WordPress mail-servers, grab all the links it could find, and then simply download everything. And, yes, many blogs will contain contents that are not intended for public viewing, including private blogs, blogs restricted to a smaller circle, and public blogs with unpublished drafts.

***Anonymous bloggers are not necessarily known to even a closer circle and even those who are might have contents not yet suitable for viewing by others. (This need not even relate to something truly secret, which would be foolish in the extreme to put on WordPress in any manner, but could include e.g. a draft of a post dealing with an upcoming proposal, surprise party, whatnot.)

If we consider only the delay, there might be see some justification to accomodate extremely large blogs, where there is at least a possibility that the time needed* for the creation of the backup might be too large for normal in-browser interaction. However, if so, the correct solution would be to present the download only within the account it self. Indeed, even if we assume that this type of linking was acceptable (it is not), the procedure is highly suboptimal: The link should have been presented in the confirmation page, not sent per email;** the availability time should have been far shorter (a day?); and the contents should have been deleted or otherwise made unavailable upon download (if something goes wrong, the user can always create a new backup).

*I received the email almost instantaneously, implying that my backup would have had to be at least one, more likely several, orders of magnitude slower than it actually was before this concern became legitimate.

**Or the contents behind an emailed link be password protected, with the password displayed in the confirmation page; or the contents only being served after a successful WordPress login.

Written by michaeleriksson

September 28, 2018 at 7:36 pm

Problems with adduser

leave a comment »

Doing some light administration, I stumbled upon a few idiocies in the Linux tool “adduser”* and the associated config-file /etc/adduser.conf on my Debian** system.

*Used, unsurprisingly, to add new users to the system.

**Who is to blame for what is not clear. As I have grown increasingly aware over the last few years, the Debian developers make quite a few poor decisions and changes to various tools and defaults that do not correspond to the wishes of the original tool-makers or that violate common sense and/or established “best practices”.

  1. This tool is the source of the “skeleton” files* added during user creation (well, I know that already) and contains a config variable for the corresponding location (SKEL)—but provides no obvious way of turning this idiocy of entirely. (Although a secondary config variable for blacklisting some files, SKEL_IGNORE_REGEX, can probably be (ab-)used for this purpose; the better way is likely to just keep the directory empty.)

    *Presumably so called because they provide a metaphorical skeleton for the users home directory. Note that there are other mechanisms that create unwanted contents in home directories. (One example is discussed in an earlier post.)

    Why is this an idiocy? Well, while their might be some acceptable use case for this, the typical use is to fill the respective users home directory with certain default configurations. This, however, simply does not make sense: Configuration settings should either be common, in which case they belong in global files, not in individual per-user copies; or they should be individual, and then the individual user should make the corresponding settings manually. In neither case does it make sense to copy files automatically.

    Indeed, in the former case, the users and administrators are put out, because (a) the skeleton files must be kept synchronized with the global configuration files (to boot in an unexpected and counter-intuitive manner), (b) the users get a snap-shot of the configuration—the configuration as it was at the time of user creation, without any changes that were made later.

    In the latter case, a user who wants his own config file can simply copy the global configuration file manually should he at all want it.*

    *Experienced users tend to not want anyone elses preferences in their config files, and have often made too extensive changes or are set on a certain set of values they have used for years, implying that they will likely not want the global files to begin with.

    Now, one comparatively rare case where skeleton files could make sense, is when setting up several sets of users that have different characteristics (e.g. administrators, Java developers, C developers)—but that will not work with this mechanism, because the skeleton files are common for all users. In order to get this to work, one would have to provide an entire new config file or play around with command-line settings—and then it is easier to just create the users without skeleton files and then copy the right files to the right users in a secondary step.

    As an aside, I do not recommend the sometime strategy of having a user-specific config file call a global config file to include settings from there (as might have been a workaround in the case of skeleton files): This tends to lead to confusion and unexpected effects, especially when the same user-specific config file is used on several systems (e.g. a work and a home computer), when global config changes, or when something is done in an inconsistent order. Instead, I recommend the individual user to either use only the global file or only his own.

  2. When the home directory is created, the typical access-control defaults (“umask”) are ignored* in favor of the config variable DIR_MODE—and this variable has the idiotic and inexcusable value 0755**. In other words, home directories are created in such a manner that everyone can read*** the directory per default. It is true that this will not give the rights to read the contents of files****, but being able to see file names can be bad enough in it self: Consider e.g. if the wrong person sees names like “resignation_letter”, “proposal_plan”, “porn”, “how to make a bomb”, …

    *Such duplication of responsibility makes it harder to keep security tight, especially since the admins simply cannot know about all such loopholes and complications—if in doubt because they change over time or can vary from Unix-like system to Unix-like system.

    **The best default value is 0700, i.e. “only the owner can read”; in some cases, 0750, i.e. “owner and group members can read”, might be an acceptable alternative.

    ***To be more specific, list the directory contents and navigate the directory. (Or something very close to this: The semantic of these values with regard to directories are a bit confusing.)

    ****Files (and sub-directories) have their own access rights that do respect the value of the umask (at their respective creation).

  3. The format of a user name underlies restrictions by the configuration variable NAME_REGEX. Unfortunately, this variable only appears to add restrictions. Quoting the documentation:

    adduser and addgroup enforce conformity to IEEE Std 1003.1-2001, which allows only the following characters to appear in group and user names: letters, digits, underscores, periods, at signs (@) and dashes. The name may no start with a dash. The “$” sign is allowed at the end of usernames (to conform to samba).

    An additional check can be adjusted via the configuration parameter NAME_REGEX to enforce a local policy.

    This is unacceptable: Unix-like systems typically accept almost any character in a username, and what name schemes or restrictions are applied should be a local decision—not that of some toolmaker.

    For reasons of interoperability through a “least common divisor”, it makes great sense to apply some set of restrictions per default; however, these restrictions must be overrideable and should have been integrated in NAME_REGEX (or a second, parallel variable).

    As an aside, I am quite surprised that “@” is allowed per default, seeing that this character is often used to connect a user with e.g. a domain or server name (as with email addresses). When the user name, it self, can contain an “@” it becomes impossible to tell for certain whether “X@Y.Z” is a user name (“X@Y.Z”) or whether it is a user name (“X”) combined with a server or domain (“Y.Z”). In the spirit of the aforementioned “least common divisor”, I would not only have expected this to be forbidden—but to be one of the first and most obvious things to be forbidden. (I would speculate that there is some legacy issue that requires that “@” remains allowed.)

Written by michaeleriksson

July 3, 2018 at 4:44 am

XDG, lack of respect for users, and bad design

with 2 comments

Normally and historically, using Linux has been much more pleasant than using MS-Windows, at least for those who stay away from KDE, Gnome, et co. Unfortunately, there has long been a negative trend towards worse usability, greater disregard for the user’s right to control his own computer, etc. (See e.g. [1] or [2] for similar discussions.)

Today, after doing some experiments with various X* and WM setups, I found something utterly inexcusable, a truly Microsoftian disregard for the user’s interests:

*By which I refer to the window system named “X”—unlike many other instances where I have used it as a placeholder.

Somehow, a tool xdg-user-dir had been activated for the first time during my experiments (whether this tools was installed per default or has snuck its way in during my experiments, I do not know)—and promptly created a slew of directories in one of my user accounts. There was no inquiry whether I wanted them created, they were just silently created. To boot, they were exactly the type of directories that I do not want and that I have always deliberately deleted when they were present on installation: Random directories like “Desktop”, “Pictures”, “Music” bring me no value* whatsoever and do not in any way, shape, or form match my computer use—starting with the fact that I use different user accounts for different tasks and order my files and whatnot according to entirely different criteria**. They clutter my file system—nothing more, nothing less.

*Note that, unlike with MS Windows, these are not necessary for the OS not to throw a fit. (Although it is conceivable that one of the worse desktop environments would—but then I do not use these!)

**The exact criteria vary, but the most common is “by topic”, rather than “by type”. For instance, a video and an eBook on the same topic would land in the same directory; two eBooks on different topics would land in different directories.

Having a look at the documentation, the functioning of this tool appears to be fundamentally flawed, even apart from its presumptuousness: In my reading, the directories would simply have been created again, and again, and again, had I just deleted them. This too is inexcusable—a manually deleted directory should stay deleted unless there are very strong reasons speaking to the contrary*/**. Now, I have the computer knowledge and sufficient drive that I actually could research and solve this issue, through finding out what idiotic program had added the directories and make sure that a repetition did not take place (and do so before my other user accounts were similarly molested)—but this is still a chunk of my time that I lost for absolutely no good reason, just because some idiot somewhere decided that he knew better than I did what directories I should want. For many others, this would not have been an option—they would have deleted the directories, seen them recreated a while later, deleted them again, …, until they either gave up with a polluted user account or screamed in frustration.

*Such reasons would typically involve a directory or file that is needed for practical operations while not being intrusive to the user (almost always implying a “dot” or “hidden” file). Even here, however, querying the user for permission will often be the right thing to do. To boot, it is rarely the case that an application actually needs to create anything in the user account, when not explicitly told to do so (e.g. through a “save” action by the user). Off the top of my head, I can only think of history files. For instance, creating a config file is only needed when the user actually changes something from the default config (and if he wants his config persistent, he should expect a file creation); before that it might be a convenience for the application, but nothing more. Temporary files should be created in the temp-directory, which is in a central place (e.g. /tmp) per default (and should the user have changed it, there is an obvious implicit consent to creation). Caching is a nice-to-have and would also often take place in a central location. Indeed, caching is an area where I would call for great caution and user consent both because of the potential privacy issues involved and because caching can take up quite a bit of storage space that the user might not be aware of. Finally, should the user wish to save something, it is up to him where to save it—he must not be restricted to an application specific directory. All-in-all, most cases where an application “needs” a directory or file will be pseudo-needs and signs of poor design.

**Looking specifically at the directories from above, I note that they are not hidden—on the contrary, they are extremely visible. Further, that they almost certainly are intended for one or both of two purposes: Firstly, to prescribe the user where he should put his this-and-that, something which is entirely unacceptable and amateurish—this is, remains, and must be the user’s own decision. Secondly, to ensure that various applications can rely on these directories being present, which is also entirely unacceptable and amateurish: Applications should not make such assumptions and should be capable of doing their job irrespective of the presence of such directories. On the outside, they can inquire whether a missing directory should be created—and if the offer is turned down, the applications still need to work. If they, unwisely, rely on the existence of, say, a picture directory at all, it should also be something configurable. They must not assume that it is called “Pictures”, because the user might prefer to use “Images” and already have other tools using that directory; similarly, they must not assume that the directory, irrespective of name, is in a given position in the file system, because the user might prefer e.g. “~/Media/Pictures” over “~/Pictures”; he might even have put all his pictures on a USB-drive or a server, potentially resulting in entirely different paths.

Looking up XDG on the web gives a negative impression. For instance, the Wikipedia page has a list of hosted packages/projects, some of which are among my least favorite, because they are e.g. redundant, do more harm than good, or replace superior solutions, including D-Bus, PulseAudio, systemd, and Poppler*. To boot, they often violate the design principles that used to make Unix and its derivatives great. Some others leave me ambivalent, notably Wayland: Even assuming that X needs replacement or improvement**, Wayland seems to be the wrong way to go through reducing both flexibility and separation of concerns.***

*At least with regard to how it has screwed up xpdf.

**It quite probably does, about three decades (!) after the last major specification release. However, in my own use, I cannot think of anything major that bothers me.

***With the reservation that I have not read anything on Wayland in years and am not aware of the latest state: Because I am not a Ubuntu user, I have not been forced to a practical exposure.

Looking at the “Stated aims” further down the page, most are good or neutral (with some reservations for interpretation); however, “Promote X desktops and X desktop standards to application authors, both commercial and volunteer;” is horribly wrong: Promoting such standards to desktop authors is OK, but not to application authors. Doing the latter leads to unnecessary dependencies, creates negative side-effects (like the unwanted directories above), and risks us landing in a situation where the system might need a desktop to function—not just a window manager or just a terminal.

For instance, I have now tried to uninstall everything with “xdg” in its name. This has left me with “xdg-utils” irremovable: If I do uninstall it, a number of other packages will go with it, including various TeX and LaTeX tools that have nothing to do with a desktop. In fact, there is a fair chance that they are all strictly command-line tools…

I have also searched for instances of “freedesktop” (a newer name, cf. Wikipedia). Trying to uninstall “hicolor-icon-theme” (admittedly not something likely to be a source of problems) leads to a request to also uninstall several dozen packages, many (all?) of which should reasonably still work without an external icon theme. By all means, if icons can be re-used between applications, try to do so; however, there must be sufficient basic “iconity” present for a good program to work anyway—or the programs must work sufficiently well without icons to begin with. Indeed, several of these are either command-line tools (that should not rely on icons in the first place) or make no or only minimal use of icons (e.g. pqiv and uzbl).

Worse, chances are that a considerable portion of these tools only have an indirect dependency: They do not necessarily need “hicolor-icon-theme” (or “xdg-utils”). Instead they rely on something else, e.g. a library that has this dependency. Here we can clearly see the danger of having too many dependencies (and writing too large libraries)—tools that do not need a certain functionality, library, whatnot, still need to have it installed in order to function. This leads to system bloat, greater security risks, and quite possibly diminished performance. Unfortunately, for every year that goes by, this problem appears to grow worse and worse.

Forcing XDG functionality and whatnot into applications that do not actually need them is a bad thing.

(Similarly, a great deal of my skepticism against D-Bus arises from the fact that it is “needed” by every second application, but is rarely actually used for something sensible. In the vast majority of cases, the application would have been just as good without D-Bus, but it still has a dependency and it still presumes to start an unwanted D-Bus at will—which it typically does not clean up afterwards…)

Written by michaeleriksson

May 8, 2018 at 1:31 pm