The Declarative/Imperative Language Rift

The recent round of debates around systemd have gotten me thinking again about the declarative/imperative language rift, and ways to bridge it.

Declarative languages start from a constrained set of use cases, compiling a constrained solution through a set of reasonably intelligent heuristics, to provide a reasonably efficient implementation. Contrast e.g., Python with SQL.

A well designed declarative language makes life much much easier for the 90% of use cases that fit within its pattern:

• It tends to be easier to write simpler code with more obvious intent.
• Language features and capabilities tend to be narrower and thus far more discoverable.
• It's easier for a compiler/runtime to discover specification problems early.

A few examples of (imo) successes:

• For the common case of building simple database UIs, ruby on rails does a great job of abstracting state.
• SQL engines generally do a great job of giving 90% of available performance, 90% of the time.

Unfortunately, declarative languages tend to be much much much harder to work with once you fall out of the 90% case.

• Rails code becomes a nightmare of spaghetti if you ever want to do something more than the standard CRUD pattern.
• When SQL fails, it fails hard. Any logic that involves a state machine is going to take you forever to implement and be very slow, and if the optimizer heuristics fail... you're in for a world of pain when it comes to coaxing them back into line.

The main problem that I see in these settings is that there's not generally a way to gracefully degrade from declarative to imperative logic. For example SQL engines generally make it hard for me to provide a piece of imperative code for just one join. Specifically:

• Declarative languages rarely provide an easy way to override their built-in heuristics.
• ... and when the declarative language does provide one, it is often brittle as it requires breaking through several layers of abstraction (e.g., see Souffle's .plan directives)
• There's often no easy way to gracefully degrade from declarative mode to a comparable imperative mode.
• ... and when there is, it's often uni-directional.

That said, there are a few success stories that come to mind. The Windows Forms API for one. I spent a summer interning at Microsoft and got to know this API fairlly well. Notably, the API uses a graphical editor that allows you to point and click your way into most simple interfaces. This is great for discoverability, and a fantastic starting point when I was just dinking around with the UI. What was really neat, and why I consider this a success story for declarative langs was that the editor's canonical representation (at least as far as I could tell) was the source code calling into the forms API. You could edit the code, and changes would be reflected when you re-opened the graphical editor. Obviously, once you got too far out of the editor's comfort zone, things would start to get weird... but it was still a great way to get started.

The rake build system is a nice counterpoint to rails, as it is basically a library on top of ruby with a thin runtime wrapper. Ultimately, you're just writing regular ruby code, but the library does a good job of abstracting out common patterns in build systems (dependency tracking, freshness tests, etc...).

LINQ is another case of what I would consider a success story; It allows SQL queries to be embedded in regular program logic. Logic fragments that you want to leave to the optimizer can be left to the optimizer, while there is relatively little barrier to implementing logic fragments in an imperative style (e.g., state machines).

The common pattern in all of these cases is that the declarative language is embedded in an imperative one. This forces the declarative language to conform (at least at the API boundaries) to the conventions of the imperative language, which in turn provides a graceful way to fall back to imperative land in cases where it's helpful.

The issue driving all of this is systemd. I actually like systemd's unit files. I like the fact that, as a programming environment, it makes common stuff easy (e.g., standardized logging), and establishes strict standards for program behavior (e.g., there is a single source of truth for things like "is this daemon running?"). However, I absolutely abhor its discoverability: Documentation on unit files is scarce, and the byzantine interfaces presented by systemctl and journalctl are crimes against newbies trying to figure out their systems (journalctl -afu is... aptly described by the last two flags). I hate how going outside its strictures (e.g., integrating with tools that already have logging infrastructures) breaks a lot of things. I hate how it is a monolithic dependency that needs to be imported in its entirety if an app touches even a little bit of it.

Ultimately, I'm trying to square the idea of unit files and strict conventions on things like logging with the more open environment presented by init.d.

• Something like a unit_runner could act as a shell to provide an rc.d style API to standard unit files.
• A unit runner shell would also make it very easy to extend rc.d with new conventions for e.g., tracking service state, since standard unit files would automatically support them.
• Maybe one could provide a log-fs, mounted at /var/log that acts as a normal filesystem, but provides fun features like log redirection, log rotation, etc...

My point here is that UNIX already provides a set of conventions, and many of the QoL improvements of systemd could, indeed, be integrated into those conventions. It would just need a bit of care.