Polygon
Working title for a decentralized heterogenous project management & communication technology implemented by a Best of Breed Suite of open-source tools based on Stackspin. For background thoughts on the concept see https://github.com/xeruf/nodal.
Vision: Empower cross-disciplinary teams of diverse technical affinity to collaborate, each in the tool that fits their workflow.
Analogies
- Like https://www.make.com but FOSS and connected by default (in-sync rather than out-of-sync tools by default) -> central data rather than syncing databases
- Polygon is to productivity what Matrix is to messaging - and the Polygon Suite is a serviced FOSS product like https://www.beeper.com
- The protocol develops by consensus with arbitrary key-values allowed for extensibility like in https://wiki.openstreetmap.org/wiki/Tagging
Exemplary Pitch
What if...
- the discussion thread on Discourse is also a topic in Zulip is also an issue thread on Forgejo is also a comment thread in Vikunja?
- the task description in Vikunja is also a wiki entry is also a markdown file in Nextcloud (synced to your device for offline editing) is also a HedgeDoc document for collaborative editing and always available to export as a professional TeX PDF to update teams and customers?
- the calendar entry is also a task is also a drop in the Gantt chart is also a Kanban card is also tracked in your time tracking?
Would that not be convenient?
Over and over again I see people building specialized, unnecessarily complex, artificially restricted often proprietary tools. We are wasting so much potential by not acting on these and many more possible connections.
We need simple, generic, reusable backends that do one thing well. Artificial restrictions and structures as often used in hierarchies for notes and tasks are only useful fur humans, neither for developers nor for the machine, so only the human interface should be concerned with them. This also empowers advanced users to break free from the restrictions and develop their own flow, without needing to switch gears.
Evidently, we need a single source of truth. We already have this concept in identity management and even design and project management to some extent, but not for application data. SOLID is an interesting project for this, but I think we need to go more low-level: Everything can be represented as a node in a graph database, with tags, taxonomies and hierarchies all represented through connections.
Each node has a state - a wiki entry is TODO when it is empty, a task on a Kanban board when it is not in the Done column, a calendar entry might need preparations as well as postprocessing. Depending on its context, a node might have different attributes - timestamps, description, and plenty of connections.
Emacs Org mode seems to be the only ecosystem so far able to model this: A heading can have arbitrary properties, link to other headings by id, contain a description and a freely definable state and has endless possibilities for integration through elisp. But we need tooling that is not only for geeks.
Aspects
The working title Polygon has referred to various parts of the project in the past, so here is an overview:
- nodal is the draft of the protocol that should drive this in the future, currently planned to be built on nostr
- Stackspout is our extension of Stackspin to provide initial integration of various tools including Vikunja through existing tools and protocols (SCIM, n8n, LDAP)
- Polygon refers to the canonical suite implementing an out-of-the box experience with various tools kept in sync by nodal
The USP of the nodal Polygon is ultimate flexibility and sustainability. It is not an application or another platform with its own extension ecosystem with a core rotting over time (like Nextcloud and most ERP tools). Because such a system inevitably ties the extensions to implementation details of the core and encourages the core to grow, resulting in growing switching costs and a single point of failure as cessation of core development breaks the whole ecosystem.
Instead the UNIX command line serves as a model. The beauty of combining multiple simple tools through piping into a powerful command is striking, and the flexibility is unparalleled. Any part of the pipe can be exchanged within seconds if appropriate. So it is time to bring this concept to multi-user server-client applications of the 21st century, allowing you to unite your favorite productivity, communication and more tools into one mighty Polygon through Nodal without depending on any single one of them, allowing you to replace or remove parts at your convenience.
Practical Approaches
- develop a proof of concept nostr app, implementing a custom task type
- kubernetes stackspout
Modestly accurate ChatGPT Summary
The current landscape of task management and productivity tools is inadequate because they either lack good communication features or fall short in task management capabilities. This results in the use of multiple tools that are not synchronized, tools being used for unintended purposes, or compromises made with tools that can do a bit of everything. There is a need for a framework that addresses these limitations.
The new protocol, called Nodal, is built on top of the nostr protocol, which handles communication end-to-end using asymmetric key cryptography for identity verification. Nodal uses a graph structure of nodes as its foundation. It allows for the connection of various components without a central dependency.
The new protocol and interface aim to address the limitations of existing tools by providing a platform-agnostic approach and allowing for the integration of best-of-breed tools. The principles of the framework include being open-source, federated, offline-first with conflict resolution mechanisms, extensible, and immutable.
Polygon refers to the suite of tools implementing the Nodal protocol. The goal of Polygon is to empower cross-disciplinary teams to collaborate using tools that fit their workflow. The focus is on ultimate flexibility and sustainability, similar to the UNIX command line concept, where different tools can be combined and exchanged easily. The new interface built on top of Nodal can provide productivity tools for specific target groups. One example is task progress, where tasks are sliced into subtasks that can be completed in one day or a single work session. The completion of these subtasks can be used to display reliable progress bars and update GANTT charts dynamically.
There are challenges to overcome, such as creating a central navigation system for the different applications and ensuring consistent user logins and customer data across the ecosystem of applications. Options like OpenID Connect, LDAP, SCIM, or a messaging system for app synchronization are being considered for this purpose.
Overall, the Nodal framework and Polygon suite aim to provide a more comprehensive and integrated solution for task management and communication by addressing the limitations of existing tools and promoting interoperability between different productivity tools.
State of Stackspout (2022)
Stackspin is powering our company, with Nextcloud, OnlyOffice, Zulip and more tied in with Single Sign-On. Janek has worked himself into the Flux-Helm-Kubernetes-Docker stack and is adding and configuring apps such as Vikunja, InvoiceNinja and Gitea. All of this happens open-source on the GitLab instance of greenhost, this is our part: https://open.greenhost.net/xeruf/stackspout
How Flux works in Stackspin and Stackspout
Each code snippet in the text is a Kubernetes resource kind.
Upon installation,
a Flux GitRepository
is registered with the cluster,
which regularly pulls the latest resource definitions from git,
along with a Flux Kustomization
pointing at a base folder.
This folder contains further Flux Kustomizations,
each deploying resources from a specific folder in the gitrepository.
In Stackspout, the stackspout-apps Kustomization
automatically installs the additional application Kustomizations,
while in Stackspin these are added via a script or the dashboard.
For each application,
there is an add-APP Kustomization
,
which loads an APP-secrets and APP Kustomization
,
the former generating passwords and OpenID Connect secrets
for use by the APP.
These are then substituted by the APP Kustomization
into the ConfigMap
for the application
along with cluster-wide information from stackspin-cluster-variables
such as the domain and IP all services are provided under,
as well as login details for services to send E-Mails.
For each app,
there is a HelmRelease
pulling a specific version of a HelmChart
from a separately defined HelmRepository
.
The HelmChart
defines how to deploy an application,
similar to a docker-compose file but customizable through a ConfigMap
,
which depending on the chart leads to deployment of additional services,
creates config files, defines environment variables
and storage and backup mechanisms.
The HelmRelease
then deploys the resources according to the HelmChart
with ConfigMap
values.
This usually creates a few auxiliary resources like ConfigMaps and Secrets,
but the interesting part are the services:
-
stateless services are deployed via
Deployments
, which can be scaled anytime to a desired number of replicas -
stateful services are deployed via
StatefulSets
and get an own storage per pod For more see for example https://www.airplane.dev/blog/statefulset-vs-deployment
These then create and watch Pods
,
which usually contain a single docker Container
running the actual service component.
Note that one application often consists of multiple components
for example a StatefulSet
for the database
and a Deployment
for the frontend.
If the Pods
store data,
they usually bind to a previously reserved PersistentVolumeClaim
,
creating a PersistentVolume
which in Stackspin is a local folder under /var/lib/local-storage/Stackspin
.
Some quirks to note:
- Flux Kustomizations recursively deploy resources from referenced folders - if you want to limit the deployed resources, put a k8s kustomization.yaml into the folder. Note the difference in the apiVersion as both are of kind Kustomization!
- Modifications to resources on the cluster managed by flux will be reverted regularly upon reconciliation - nice for quick testing, but make sure to commit proper changes to the repository.
- If a Kustomization or HelmRelease gets stuck, it is sometimes helpful to suspend and resume it.
Challenges
Listed below are the big design challenges to be overcome to make this concept really viable.
Navigation System
All the different apps run on different URLs, able to be used independently, which in itself is quite handy. But the novice user is often better off with a central navigation, like GSuite and Microsoft Online and the likes have.
Currently we mimick this with iframes within a custom nextcloud navigation. But unfortunately, iframes do not preserve state, so reloading the page or sharing a URL leads back to the root of the iframed app.
One approach would be if the dashboard had access to the iframed app, so it could forward the URL parameters.
Single Truth
How can we ensure consistent user logins and customer data across an ecosystem of different applications? OpenID Connect, LDAP, SCIM, or rather a messaging system for the apps to update each other? Each has its pros and cons.
With a single point of truth like LDAP, the services become dependent on the central database, and account changes become more complicated - Zammad for instance overwrites any changes done to accounts synced from LDAP.
With a messaging system, one can never sure both services are actually in sync, but it would fit the distributed nature better and allow to use any API rather than requiring a specific protocol.
Existing Solutions
- Stackspin with Nextcloud with external apps as Hub
- We contributed to a decent overview of open suites in https://docs.stackspin.net/en/v2/reference/comparable_projects.html
- Event Streaming: https://www.confluent.io https://kafka.apache.org
- Similar HR Integration Platform: https://www.kombo.dev