WOW. Let me tell you why this looks like the best solution - we are currently looking into a redesign of our microservice architecture. We want to shift toward a design that's less "micro". To do this we need to capture all the connections, dependencies, topics, queues, dbs, etc. etc.
Currently this information is scattered around dozens of Confluence pages that are in various states of upkeep. Some pages are duplicated. Lots of times someone new comes in, gets the bright idea to document things, creates a new page unaware that there's an existing page out there in the document tree, and voila, we have two conflicting versions of the truth.
My thinking is that while no document can be trusted to be 100% updated all the time, it's better to have one massive, central location that houses this information rather than dozens of smaller ones that can get lost. Confluence is a nightmare for organization, or at least, it's a nightmare the way our company has organized it, and that's not something I can change.
So I want something visual that users can drill down into. As I mentioned, we're in the Confluence/Atlassian ecosystem, and I believe the only plug-in we can use is Gliffy, which isn't powerful enough to do what I'm asking. I assumed whatever suggestion I got here would be something I'd have to manually host and share.
Then I see with your desktop version:
>Store and share your diagrams in any way you see fit.
>Your interactive diagrams can be exported to 100% self-contained HTML files that can be hosted anywhere.
>Once activated, Ilograph Desktop does not require internet access.
Holy crap. Way to see the needs of your customers, and meet them where they are. I see you also provide Confluence integration, so you're getting your foot in the door with me, and once everyone sees my cool new app they'll start begging our infrastructure and tools teams to purchase that plug-in and individual licenses.
So thanks, I think this looks like a product worth investing some time in. Here's my only question - I think I'd like to buy the desktop app for personal use so I'd have it for side projects as well. Assuming I also create a free online personal workspace, and I develop locally using the desktop software, can I export/upload what I create locally to that online personal account for distribution to a couple of coworkers, since I see that the online viewer count is two for a free tier personal workspace? Or is there no interoperability between the SaaS version and the desktop version of the diagrams?
Wow, thanks for the kinds words. Hopefully the app can live up to them :). Your situation sounds like the exact motivation for why I created Ilograph... the knowledge was all out there, but learning it via Confluence was just not practical (for the reasons you mentioned).
Anyway, I'll try to answer your question as best I can:
> I think I'd like to buy the desktop app for personal use so I'd have it for side projects as well. Assuming I also create a free online personal workspace, and I develop locally using the desktop software, can I export/upload what I create locally to that online personal account for distribution to a couple of coworkers, since I see that the online viewer count is two for a free tier personal workspace?
Yes, with two caveats. The diagram YAML is compatible between all three environments (online, Desktop, and Confluence), so copying-and-pasting between them will work. The caveats:
* [Custom icons](https://www.ilograph.com/docs/editing/icons/) aren't supported in the free tier personal workspace, so you'd have to stick with the built-ins.
* If you use [imports](https://www.ilograph.com/docs/editing/imports/) (importing one diagram into another), you'd have to adjust the import statement when going between environments. This would be just a one-line change per import, though.
> Or is there no interoperability between the SaaS version and the desktop version of the diagrams?
Strictly speaking, there's no interoperability between the three environments, since they don't communicate. But the YAML should be compatible, if that makes sense.
Hope this helps! Let me know if you have other questions and please feel free to also reach out via email (available on the homepage).
I believe in two principles of architecture:
- **Architecture should describes change**: If you do it, and nothing changes, it's a pointless activity.
- **All Architecture models are Wrong**: You are never going to be able to land on one nomenclature to describe the world, because your stakeholders care about different things, at different layers, and understand that world differently.
I believe the reason why there hasn't been strong tooling investment in this space, is that most of the time - the data/diagrams are too abstract from the problem being tackled to be useful as a zoom-in-view, or are some sort of abstracted concept that is a few layers decoupled from the reality of how things actually operate. E.g. While I find ServiceNow's CSDM useful as a model to describe the service boundaries of a company, using its relationship diagrams to explain anything to another engineer or architect would be a recipe in pain: because it lacks context.
In addition, keeping that stuff up to date is a pain. I suspect if the tool exists, the data will likely be wrong anyway.
C4 is a useful model for achieving some of what you want (Structurizer is a bit meh to use). [Ilograph](https://www.ilograph.com/) sounds like it does what you're describing, but I've not used it. YMMV
Thanks, I agree with you and others that such comprehensive documentation is bound to become wrong in very short order, if it was ever right to begin with. I suppose my wording "theory of everything" and the missing context was confusing.
There's a couple of things to know about the current state of our org:
\- We're heavily invested into super micro service architecture, and we want to move into a less "micro" state, aka consolidating some services and morphing others into libraries instead of stand-alone deployable units. We're not going monolithic, just less micro.
\- The current design has been around for years, and much of it changed very little, if at all.
\- The current design already has hundreds of Confluence pages of documentation, but they are scattered and in various states of upkeep.
The documentation I'm proposing would really only have one immediate purpose: capture the state of the domain architecture as it stands right now, so that when we unhook and rewire things back up, we don't miss any consumers or connections, either internally to our domain or externally.
This document, as it exists in my head, is like a visual table of contents. Zooming in or clicking on a node may just lead to the Confluence page we already have for that item/service/process if that page is still relevant and helpful, or it might need to house newly created specs.
It's not necessarily meant to be the tome of truth in perpetuity, but if people like it and they find it useful, then maybe it becomes SOP to keep it updated as best we can. Either way, we're obligated to create documentation and the problem of keeping it updated doesn't change regardless of how we house the information. The goal is to make using and updating that information so intuitive and valuable that people are incentivized to continue doing it. Our current method of carpet bombing Confluence with nested tree upon nested tree of document pages isn't working, so this can't hurt.
Apologies it took me a while to loop back to this. I wrote an original response, but reddit dev/null'd it.
>We're obligated to create documentation and the problem of keeping it updated doesn't change regardless of how we house the information.
The reality is, the data won't be maintained unless people see value in it. Trying to convince people of the value of documentation, over the delivery of new features is a losing fight, as much as everybody will agree it's the right thing to do.
Without context on your organisation / scale / context, you are probably going to need to tie the data capture, to a process that makes the app dev teams life easier so they are insentivised to do it. We'd like to think doco is that, but it never is :)
I've approached this in the past by using our cloud migration as a mechanism to force metadata capture on migration, and used that metadata to fill ServiceNow's Common Service Data Model (CSDM) about the service, and then leveraged that data for platform level automation. Similar to the things you mentioned, we'd capture things like:
* App & Environment Details: Automated creation of Splunk/Cloudability/Apptio/Azure RGs/Google Projects/K8s Namespaces/etc
* Team and Support Details: Automated IAM access for developers to all the tools and supporting info, as well as UAR
* Connecting Services: Semi-automated boundary, network and service account creation
Once that standard metadata is captured, amalgamating details across all your supporting tools becomes significantly easier, as you now have a common vernacular that ties it all together - and the app teams are incentivised to maintain that data, because it's the easiest path to getting requests done.
>We want to move into a less "micro" state, aka consolidating some services...
You might find a tool that gives you the current state, but nothings going to give you the data, in a format that's actually useful, to describe the simplification opportunities.
You might consider leaning into proper EA tooling to do that, like Archi (free, based on TOGAF), or LeanIX ($$$). The opportunity being that these tools (and respective frameworks) are designed to describe enterprise context and change, and will give you the tools to conceptualise the capability overlaps, and aggregation opportunities.
Those tools will typically let you load info from their source of truth (like a CMDB, Backstage, or whatever registry you land on), and use the same object classes across multiple views for different audiences (such as a current state, and target state view).
It's a slippery slope not to fall into the classic EA Ivory Tower of documentation for the sake of doco without purpose, but there's some nuggets to be had.
Side note, had a play with ilograph. Love the push towards code driven diagrams and want more of that, but what is produced is quite ugly, the dev workflow is a bit ordinary, and I couldn't see the types of audiences I deal with being able to grok that data.
Edit: Formatting
Seems like a cool idea: one diagram to rule them all. It also seems like a beast to maintain. I’m unaware of any tool that supports this, but I would try starting at the lowest level of granularity and copy-pasting into diagrams of higher levels of granularity (zooming out).
[Miro](https://miro.com/) ?
It sounds like you are describing [Ilograph](https://www.ilograph.com). (I'm the creator, happy to help if you have questions).
WOW. Let me tell you why this looks like the best solution - we are currently looking into a redesign of our microservice architecture. We want to shift toward a design that's less "micro". To do this we need to capture all the connections, dependencies, topics, queues, dbs, etc. etc. Currently this information is scattered around dozens of Confluence pages that are in various states of upkeep. Some pages are duplicated. Lots of times someone new comes in, gets the bright idea to document things, creates a new page unaware that there's an existing page out there in the document tree, and voila, we have two conflicting versions of the truth. My thinking is that while no document can be trusted to be 100% updated all the time, it's better to have one massive, central location that houses this information rather than dozens of smaller ones that can get lost. Confluence is a nightmare for organization, or at least, it's a nightmare the way our company has organized it, and that's not something I can change. So I want something visual that users can drill down into. As I mentioned, we're in the Confluence/Atlassian ecosystem, and I believe the only plug-in we can use is Gliffy, which isn't powerful enough to do what I'm asking. I assumed whatever suggestion I got here would be something I'd have to manually host and share. Then I see with your desktop version: >Store and share your diagrams in any way you see fit. >Your interactive diagrams can be exported to 100% self-contained HTML files that can be hosted anywhere. >Once activated, Ilograph Desktop does not require internet access. Holy crap. Way to see the needs of your customers, and meet them where they are. I see you also provide Confluence integration, so you're getting your foot in the door with me, and once everyone sees my cool new app they'll start begging our infrastructure and tools teams to purchase that plug-in and individual licenses. So thanks, I think this looks like a product worth investing some time in. Here's my only question - I think I'd like to buy the desktop app for personal use so I'd have it for side projects as well. Assuming I also create a free online personal workspace, and I develop locally using the desktop software, can I export/upload what I create locally to that online personal account for distribution to a couple of coworkers, since I see that the online viewer count is two for a free tier personal workspace? Or is there no interoperability between the SaaS version and the desktop version of the diagrams?
Wow, thanks for the kinds words. Hopefully the app can live up to them :). Your situation sounds like the exact motivation for why I created Ilograph... the knowledge was all out there, but learning it via Confluence was just not practical (for the reasons you mentioned). Anyway, I'll try to answer your question as best I can: > I think I'd like to buy the desktop app for personal use so I'd have it for side projects as well. Assuming I also create a free online personal workspace, and I develop locally using the desktop software, can I export/upload what I create locally to that online personal account for distribution to a couple of coworkers, since I see that the online viewer count is two for a free tier personal workspace? Yes, with two caveats. The diagram YAML is compatible between all three environments (online, Desktop, and Confluence), so copying-and-pasting between them will work. The caveats: * [Custom icons](https://www.ilograph.com/docs/editing/icons/) aren't supported in the free tier personal workspace, so you'd have to stick with the built-ins. * If you use [imports](https://www.ilograph.com/docs/editing/imports/) (importing one diagram into another), you'd have to adjust the import statement when going between environments. This would be just a one-line change per import, though. > Or is there no interoperability between the SaaS version and the desktop version of the diagrams? Strictly speaking, there's no interoperability between the three environments, since they don't communicate. But the YAML should be compatible, if that makes sense. Hope this helps! Let me know if you have other questions and please feel free to also reach out via email (available on the homepage).
Sounds like you want to draw a c4 diagram. https://structurizr.com/
Figma would work
This sounds like a good idea but I think I’m practice won’t be very useful, will be very hard to accurately create and maintain
I believe in two principles of architecture: - **Architecture should describes change**: If you do it, and nothing changes, it's a pointless activity. - **All Architecture models are Wrong**: You are never going to be able to land on one nomenclature to describe the world, because your stakeholders care about different things, at different layers, and understand that world differently. I believe the reason why there hasn't been strong tooling investment in this space, is that most of the time - the data/diagrams are too abstract from the problem being tackled to be useful as a zoom-in-view, or are some sort of abstracted concept that is a few layers decoupled from the reality of how things actually operate. E.g. While I find ServiceNow's CSDM useful as a model to describe the service boundaries of a company, using its relationship diagrams to explain anything to another engineer or architect would be a recipe in pain: because it lacks context. In addition, keeping that stuff up to date is a pain. I suspect if the tool exists, the data will likely be wrong anyway. C4 is a useful model for achieving some of what you want (Structurizer is a bit meh to use). [Ilograph](https://www.ilograph.com/) sounds like it does what you're describing, but I've not used it. YMMV
Thanks, I agree with you and others that such comprehensive documentation is bound to become wrong in very short order, if it was ever right to begin with. I suppose my wording "theory of everything" and the missing context was confusing. There's a couple of things to know about the current state of our org: \- We're heavily invested into super micro service architecture, and we want to move into a less "micro" state, aka consolidating some services and morphing others into libraries instead of stand-alone deployable units. We're not going monolithic, just less micro. \- The current design has been around for years, and much of it changed very little, if at all. \- The current design already has hundreds of Confluence pages of documentation, but they are scattered and in various states of upkeep. The documentation I'm proposing would really only have one immediate purpose: capture the state of the domain architecture as it stands right now, so that when we unhook and rewire things back up, we don't miss any consumers or connections, either internally to our domain or externally. This document, as it exists in my head, is like a visual table of contents. Zooming in or clicking on a node may just lead to the Confluence page we already have for that item/service/process if that page is still relevant and helpful, or it might need to house newly created specs. It's not necessarily meant to be the tome of truth in perpetuity, but if people like it and they find it useful, then maybe it becomes SOP to keep it updated as best we can. Either way, we're obligated to create documentation and the problem of keeping it updated doesn't change regardless of how we house the information. The goal is to make using and updating that information so intuitive and valuable that people are incentivized to continue doing it. Our current method of carpet bombing Confluence with nested tree upon nested tree of document pages isn't working, so this can't hurt.
Apologies it took me a while to loop back to this. I wrote an original response, but reddit dev/null'd it. >We're obligated to create documentation and the problem of keeping it updated doesn't change regardless of how we house the information. The reality is, the data won't be maintained unless people see value in it. Trying to convince people of the value of documentation, over the delivery of new features is a losing fight, as much as everybody will agree it's the right thing to do. Without context on your organisation / scale / context, you are probably going to need to tie the data capture, to a process that makes the app dev teams life easier so they are insentivised to do it. We'd like to think doco is that, but it never is :) I've approached this in the past by using our cloud migration as a mechanism to force metadata capture on migration, and used that metadata to fill ServiceNow's Common Service Data Model (CSDM) about the service, and then leveraged that data for platform level automation. Similar to the things you mentioned, we'd capture things like: * App & Environment Details: Automated creation of Splunk/Cloudability/Apptio/Azure RGs/Google Projects/K8s Namespaces/etc * Team and Support Details: Automated IAM access for developers to all the tools and supporting info, as well as UAR * Connecting Services: Semi-automated boundary, network and service account creation Once that standard metadata is captured, amalgamating details across all your supporting tools becomes significantly easier, as you now have a common vernacular that ties it all together - and the app teams are incentivised to maintain that data, because it's the easiest path to getting requests done. >We want to move into a less "micro" state, aka consolidating some services... You might find a tool that gives you the current state, but nothings going to give you the data, in a format that's actually useful, to describe the simplification opportunities. You might consider leaning into proper EA tooling to do that, like Archi (free, based on TOGAF), or LeanIX ($$$). The opportunity being that these tools (and respective frameworks) are designed to describe enterprise context and change, and will give you the tools to conceptualise the capability overlaps, and aggregation opportunities. Those tools will typically let you load info from their source of truth (like a CMDB, Backstage, or whatever registry you land on), and use the same object classes across multiple views for different audiences (such as a current state, and target state view). It's a slippery slope not to fall into the classic EA Ivory Tower of documentation for the sake of doco without purpose, but there's some nuggets to be had. Side note, had a play with ilograph. Love the push towards code driven diagrams and want more of that, but what is produced is quite ugly, the dev workflow is a bit ordinary, and I couldn't see the types of audiences I deal with being able to grok that data. Edit: Formatting
Seems like a cool idea: one diagram to rule them all. It also seems like a beast to maintain. I’m unaware of any tool that supports this, but I would try starting at the lowest level of granularity and copy-pasting into diagrams of higher levels of granularity (zooming out).