Binaries are temporary; But the Source *should* be eternal

All men must die.

Nine (rings) for mortal men, doomed to die.

Everyone knows how to make another human, but the secrets to making a P94 plasma rifle are all but lost.

At the workshops related to release engineering at Flock, there was much discussion over "build artifacts." Build artifacts usually means various types of binaries (RPM packages, QCOW images, etc.)

My paradox is this:

• As an open source developer, I primarily care about the source code. I do not care about the binaries unless a user reports a bug.
• As a system administrator at work, I primarily care about the binaries. I do not care about the source code unless I run into a bug.

Perhaps even more paradoxical is this:

• At most companies, the value of each binary expires once a security update is released.
• At some companies, the entire "system" goes through a regulatory certification process, during which the "vendor's" binaries must have been recently certified. And yet the system must continually run for decades with as little as 5 minutes of downtime per year.

That last bullet point means "no patching for decades."

And by "system", they usually mean "multiple computers."

Don't believe me? Read that link. It goes on to explain:

Some organisations (with Industrial Control Systems (ICS)) we have spoken to about moving off Windows XP because it is no longer supported by Microsoft have admitted they are still running even earlier versions.

Organisations are often unwilling to update any systems, not only for cost reasons, but also because any change requires them to re-certify the whole system to comply with industry regulations.

I shared that article with my friend who is an open source contributor and business owner. He is familiar with customers needing to run unpatched Windows systems for years due to his business. (We agreed that air gapping is the best security approach for ICS.)

At home, I just finished migrating from Ubuntu to the Red Hat family. I migrated for technical reasons. Red Hat is known for selling its certified binaries. Yet Red Hat follows the model "upstream first," I've heard that argument before; it sounded like fluff. But I learned the value of it at Flock.

Here is what I've concluded:

The value of binaries is temporary.  

The value of the upstream source code is eternal.*

Bugs, especially security bugs, should be fixed promptly. And all the "vendor" testing that is done on binaries is invalidated by those updates. And the testing that a customer does on their systems is often invalidated by those updates. So both have to repeat their tests for each update; security related or not.

In my industry, we operate according to the principle "no change is too small to not re-test the entire system." Thus systems are shipped with known bugs, which are condoned by Review Boards, because they cannot repeat all the tests within cost and schedule.

And the value of the certifications of those binaries is also temporary. Even if "updates" are continually re-certified, the criteria for certification change over time. RHEL5 will never be compliant with LSB 5.0. And the binaries for Fedora 24 will never go through the LSB certification process under Red Hat's current business model.

But the source code changes to make RHEL LSB-compliant is upstreamed 1st in Fedora. Fedora 24 (which may or may not be the basis for RHEL8; I can only speculate) will at least be as close as possible to LSB 5.0 compliant. Perhaps it will be LSB 5.0 compliant in a technical sense, but not in a certification ($$$) sense.

Imagine hypothetically that Red Hat, a (mortal) company, were to fail. The community or another company could continue working on Fedora's source code and get it (or a downstream distro) certified.

* But if the source code is encumbered with a horrible CLA; a CLA that prevents others from contributing, then that will never happen. The value of that source code is temporary, just like the value of the "free" binaries.

Humans are born, live, and die. And before they die, they may become "legacy." These are 2 facts that Buddhism emphasizes. But their DNA, and their culture, lives on. And both our DNA and our culture are free to evolve; to adapt to the future. Even after the "build artifact" returns to the earth.

Speaking of which, I need to write another blog post. It will have a title like "All open source projects are mortal, just like open source companies." My blog, just like open source software, is never finished. But I need to release this piece of "content" now.

-Mike