Governance of Workflows
Why agile fails, and how category theory can help
Dr Charles T. Gray, Datapunk 
Pitfalls of getting tool locked
No matter how advanced the tooling is, mathematicians solve conceptual problems at the blackboard. Why?
- It forces minimal representation of the problem.
- Tools don’t constrain the solution.
Once the concept is clear, then we instantiate it in tools that serve that structure.
The problem with agile
Agile workflows and tools like JIRA often miss the mark and drift into tech theatre. Developers have pretty great ways to break tasks down to computationally isolated tasks, but the way of piecing them back together is doomed to fail because too often there is an assumption the project plan will remain stable.
However, development is highly chaotic; unexpected blockers as well as new solutions appear.
Think of agile like this.
Leadership: Mirror, mirror on the wall,
who has the FAIRest vision of them all?
Product managers smash the mirror into pieces.
Developers smash those pieces into shards.
Different developers paint the shards in
tiny ways to make the vision FAIR.
Developers reassemble shards into pieces.
Product managers reassemble those pieces
into the mirror painted to leadership's vision.
Leadership: Mirror, mirror on the wall, forget FAIR,
I can't see my fucking face at all.
Without governance of the workflows, epistemic drift becomes inevitable (draft).
Kanbans are for working, not reporting
Sadly, I’ve seen far too many managers attempt to govern from the workflow level, without any consideration of the core priorities. This locks the team in a myopic dystopia: always working toward short-term goals, yet never achieving anything of substance.
This is where systems of human intelligence governance, such as Getting Things Done (Allen 2015) provide models of structured intelligence governance. Just as GTD promotes an annual review, technology workflows benefit from first assessing core priorities conceptually – before turning to tools.
Governance of core domains
A critical failing in tech management is sufficient reporting to both leadership and developers. In my experience, managers set a vague goal and do little else but ask when it will get done, e.g., an epic with only the title filled in, “KPIs - due december”.
Too often this middle layer
- fail to correct the epistemic drift of development chaos by intervening in key blockers
- and also misrespresent to leadership the status of the project.
In their defense, I don’t believe there are good enough tools for them to do this yet, and I aim to contribute to mitigating that. We need better methodologies to ask how well our governing mechanisms (Figure 4) serve our core domains (Figure 1).
Far less defensible, however, is the norm I’ve seen from middle management: blame and dispose of developers when their vague fantasies about AI and machine learning fail to eventuate. Unfortunately, discrimination and exploitation of developers is the solution that is usually chosen by middle management.
Developers are not to blame for management lacking tools of governance, yet developers are the ones who bear responsibility.
Workflows should not be tool locked
Ways of working are critical to governance, otherwise developers will revert to what feels safe and comfortable but without the context managers have from leadership. Developers need workflows that facilitate deep work (Newport 2016) where they focus on a single task.
Task management tools are great, but often try too hard to be all the things. Better to go to the blackboard and inspect the combination of workflows, from stand up, to kanbans, or whatever.
Category theory lets us ask:
Do our workflows serve our priorities, or merely reinforce our tools?
With a diagrammatic model, we can answer that question formally.
Category theory is the mathematics of structure and transformation. Here, it becomes a language for reasoning about tasks, tools, and goals.
References
Allen, David. 2015. Getting Things Done: The Art of Stress-Free Productivity. Penguin.
Newport, Cal. 2016. Deep Work: Rules for Focused Success in a Distracted World. Hachette UK.