Some thoughts on poor media types (formerly MIME types)

Disclaimer: The following arose as an excursion to another text (likely the next one to be published), but got out of hand in scope and was quite off-topic. I move the contents to a separate text, but the reader should not expect a higher quality than for a typical excursion.

Use of poor media types is common, especially on the Internet, and especially through implicitly claiming that text files are binary files and/or that human readable files are actually only machine readable. For instance, it is common that a web-server sends a file with a blanket “application/octet-stream”* (or similar), because no explicit media type has been configured (usually, but not necessarily, based on file extension) even for comparatively common formats. In a next step, a browser sees the media type, takes it a face value, refuses to display the file, and only offers the option of saving the file to disk—and this even when it would have been perfectly capable of display! The problem is so common that it might be better for a browser to defy the protocol specifications and ignore most media types in favor of file extensions… (And I say this as someone who believes strongly in protocol conformance—especially in light of my experiences with web development and the horrors of Internet Explorer in the late 1990s and early 2000s.)

*Approximate implication: “This is a (binary?) file, but I have no clue what it is.”

A particular idiocy is the mis-standardization of various human-readable files and/or files often edited using text editors as “application” over “text”, as with e.g. “application/sql” for SQL files vs. the pre-standard versions of “text/sql” and “text/x-sql”. Also note how we e.g. have a sensible standard of “text/html”, likely because the entry is old enough, but an insensible “application/xml”. Similarly, we have a sensible “text/csv” but an insensible “application/json”. (See below for more on “application”.)

The typical effect when e.g. trying to view an SQL file on the web, from an email attachment, or similar, is either a “save to disk” or, worse, an attempt to execute the file!* But SQL is not an executable format and the file simply is not an executable; and even if the media type was locally associated with a certain application that can run SQL code, doing so would be idiotic. For SQL to run, it needs to be run against a database and against the right database. (And this database might require a password; and whatnot.) The chance that this will work** for a random file found on the Internet is extraordinarily small, and, on the off-chance that it does, the results could be highly negative***. Certainly, no such execution should be attempted without prior manual inspection, but if e.g. a browser refuses to show the file, and tries an execution instead, how is the user to perform this inspection? Even within, say, a small team working on the same project, the execution might fail, because local versions of the same database schema might differ in detail or the execution depend on certain data being present in certain tables—and it is not a given that execution is wanted, be it at all or right now, even within the same project.

*Note that is only indirectly an effect of “application”, and depends on what a browser, an OS, or a whatnot decides to do based on the media type. Specifically, “application” seems to imply more “is intended to be read by an application” than “is an application” or “is an executable”. (Which makes the name poorly chosen. Of course, even this interpretation is only a half-truth, as demonstrated by e.g. “image/png”. Also see an excursion.)

**What if the file contains commands to connect to a database on the Internet, as opposed to locally and as opposed to having no connection information? Well, for that to work, the application that performs the execution must still be able to perform the correct connection, for which there is not even remotely a guarantee, it must be able to interpret and/or relate the SQL commands correctly, for which there is not even remotely a guarantee, the database must be reachable through various firewalls, for which there is not even remotely a guarantee, etc. (Note that various SQL dialects and DBMSs can show quite large differences.) And, again, there is no guarantee that the user actually wants to perform an execution.

***Assume, say, that the contents are malicious and consist of statements to delete entries from tables or dropping the tables altogether.

Indeed, the most common reason for looking at SQL files on the Internet or per email is to see an example of how to do something—not to automatically try to have the file do it.

If this “application” angle is needed, it would be much better to have a hierarchy of three levels instead of two, with a top-level division into “text” and “binary” (or some other category names with the same implication),* a mid-level division into categories of types (e.g. “application”, “image”), and a bottom-level division into specific types (e.g. “sql”, “html”). For that matter, it might be best to use three levels, but still scrap “application” in favor of less waste-basket-y categories.

*And, maybe, “multipart”, which is a special case in the current hierarchy. This is one example of where there is a historical influence, as the idea of (then) MIME types arose in the context of emails, that is probably harmful in the modern world, where (now) media types are used in a great many new contexts. Whether “multipart” should be kept at the top-level or turned into a mid-level entry (or, maybe, disappear in favor of something altogether different) is open to debate.

For instance, looking at the previous examples, we would see transformations like:*

*The first suggestion is what comes naturally based on the current system; any further suggestion is a more fine-grained alternative, to which I stress that these are just off-the-top-of-my-head suggestions and that a deeper investigation might lead to considerable revision.

application/sql -> text/application/sql or e.g. text/script/sql, text/code/sql

text/html -> text/application/html or e.g. text/markup/html, text/document/html

application/xml -> text/application/xml or e.g. text/markup/xml

text/csv -> text/application/csv or e.g. text/data/csv

application/json -> text/application/json or e.g. text/data/json

This joined by e.g.:

image/png -> binary/image/png

application/pdf -> binary/application/pdf or e.g. binary/document/pdf

Outside existing types, we might also add e.g. “text/image/ascii” for ascii art and “text/document/plain” for a document in plain text.*

*Oddly, I have not found an existing entry for plain text in the official list. There is one entry “text/vnd.ascii-art”, which presumably is related to ascii art, but “vnd” is VeNDor specific and not standardized in the way that the above examples are.

On pain of death, any application that can display text, e.g. a browser, should be obliged to display, or offer to display, any file with a top-level of “text” as text.

Excursion on the existing and failing text–application division:
I have not studied the history of this issue, but since hardly any text beyond plain text is intended solely for human consumption,* the idea of including text formats under “application” effectively makes “text” redundant, except for legacy entries. The end result then, is that we have a division into some formats, whether binary or text, that are “application” and various formats that are e.g. “image”, which makes precious little sense. (In a next step questions arise like, “Why should ‘image/png’ not be ‘application/png’, when PDFs are sorted under ‘application/pdf’?” resp. “Why is it ‘application/pdf’ and not ‘document/pdf’?”. Outside of some specific contexts, e.g. an email client, this division is one or both of artificial and inconsistent.)

*There is some differentiation in what proportions of the respective consumptions are by humans and by machines, but trying this road will lead to more conflict and more harm than benefits. Moreover, there is a long-term drift towards previously mostly edited-as-text formats increasingly being edited in some other manner, in conjuncture with the dumbing down of computer use, software development, and society in general. For example, many markup languages were developed to be edited as text, but the assumption has increasingly become that they should be edited with WYSIWYG editors, point-and-click interfaces, or similar, because users are assumed to be idiots. As a result, if relative proportion of use is the criterion, classifications would have to be continually revised…

Excursion on more complex examples of media types:
More complex examples than the above exist, including those specifying a character set or a double entry (e.g. “application/xhtml+xml”). I have not done the leg work to see how many such variations might be present and how they might fit in a three-level division, but I suspect that most or all them can be handled in direct analogy with the current system.

Advertisement