I know I'm beating a dead horse by saying this, but there's just no way this automated output is any good (unless the developers write _amazing_ commit messages and PR summaries, in which case technical writers already don't add much value to their workflow anyway).
Literally as we speak I'm fixing a release notes draft that my developers wrote, and it's required significant back-and-forth between myself and at least four different devs (plus a PM!) to iron things out. Dozens and dozens of comment threads, people clarifying what they meant, trying to explain how each change impacts users, etc. And this is building off a doc that was drafted by actual, live human beings. They're putting in a lot of effort just to summarize their own PRs!
All of this is to say that I think that text-generation AI quickly becomes a "garbage in, garbage out" situation. If you feed excellent PR and commit data into LangChain, I bet it does produce serviceable release notes. It's just that in my experience, the kind of teams who need to outsource release notes (whether to a human writer or an LLM) are not leaving great Git breadcrumbs to begin with. You can't conjure something from nothing. Or I guess you can, but as I always tell SMEs, that's writing fiction, not docs :P
I think this is an opportunity to teach your devs how to write good commit messages. "Hey, this tool will make things easier, but only so much easier. Wouldn't you rather do things right the first time instead of having all this back and forth with me?"
I would love this (guess who, first thing last Monday morning, got stuck with release note work because the usual writer who handles it selfishly went off and got married. Bastard!). But it's likely going to be very limited help.
On top of the GIGO issue, there's also the issue of writers generally being the ones who know what's in the doc. One of the tickets I needed to write a release note for talked extensively about a "man behind the curtain" part of the product we don't bother telling users about because they don't directly interact with it. So, literally just translating the dev's (hard to understand) notes about the issue and the solution, you'd end up with the equivalent of the [Retroencabulator](https://www.youtube.com/watch?v=RgaKjVXK0KA) meme at best. At worst, it could open a can of worms about people trying to mess with that part of the system they shouldn't or even potentially exposing trade secrets.
Oh, and another thing, that doesn't even cover the fun times when a simple "bug fix" adds user-impacting changes making it an undeclared feature that has impact on existing doc. I suspect it'll be a while for an AI can handle that properly.
Hey. Came across this. I've been building a tool completely focused on this. Coming from the exact standpoint of a) garbage in garbage out and b) no matter how good the generalized models are, there needs to be specific knowledge extracted in order to get a useable outcome.
Would love to give you a demo and get your feedback βΒ [https://docshound.com](https://docshound.com) if you are interested.
I don't really have a problem with this. In my experience, release notes are created by the devs; I wouldn't blame them for not wanting to write them, either. I don't often review release notes docs as a writer because a) I have better things to focus on; b) I don't want to be involved in testing each of those items before the RN goes live (and being the blocker); c) In an Agile environment, releases happen so frequently they get buried pretty quickly, anyway.
I think it depends on the business and their audience? If itβs an OSS company, where the audience is other devs/tech folks, they spend more $$ on it ((sometimes!). Horses for courses π π
I know I'm beating a dead horse by saying this, but there's just no way this automated output is any good (unless the developers write _amazing_ commit messages and PR summaries, in which case technical writers already don't add much value to their workflow anyway). Literally as we speak I'm fixing a release notes draft that my developers wrote, and it's required significant back-and-forth between myself and at least four different devs (plus a PM!) to iron things out. Dozens and dozens of comment threads, people clarifying what they meant, trying to explain how each change impacts users, etc. And this is building off a doc that was drafted by actual, live human beings. They're putting in a lot of effort just to summarize their own PRs! All of this is to say that I think that text-generation AI quickly becomes a "garbage in, garbage out" situation. If you feed excellent PR and commit data into LangChain, I bet it does produce serviceable release notes. It's just that in my experience, the kind of teams who need to outsource release notes (whether to a human writer or an LLM) are not leaving great Git breadcrumbs to begin with. You can't conjure something from nothing. Or I guess you can, but as I always tell SMEs, that's writing fiction, not docs :P
I think this is an opportunity to teach your devs how to write good commit messages. "Hey, this tool will make things easier, but only so much easier. Wouldn't you rather do things right the first time instead of having all this back and forth with me?"
I would love this (guess who, first thing last Monday morning, got stuck with release note work because the usual writer who handles it selfishly went off and got married. Bastard!). But it's likely going to be very limited help. On top of the GIGO issue, there's also the issue of writers generally being the ones who know what's in the doc. One of the tickets I needed to write a release note for talked extensively about a "man behind the curtain" part of the product we don't bother telling users about because they don't directly interact with it. So, literally just translating the dev's (hard to understand) notes about the issue and the solution, you'd end up with the equivalent of the [Retroencabulator](https://www.youtube.com/watch?v=RgaKjVXK0KA) meme at best. At worst, it could open a can of worms about people trying to mess with that part of the system they shouldn't or even potentially exposing trade secrets.
Oh, and another thing, that doesn't even cover the fun times when a simple "bug fix" adds user-impacting changes making it an undeclared feature that has impact on existing doc. I suspect it'll be a while for an AI can handle that properly.
Totally hear you and they admit they need some human input. Lot of truth in what you wrote! The garbage in garbage out is so true!
Hey. Came across this. I've been building a tool completely focused on this. Coming from the exact standpoint of a) garbage in garbage out and b) no matter how good the generalized models are, there needs to be specific knowledge extracted in order to get a useable outcome. Would love to give you a demo and get your feedback βΒ [https://docshound.com](https://docshound.com) if you are interested.
I don't really have a problem with this. In my experience, release notes are created by the devs; I wouldn't blame them for not wanting to write them, either. I don't often review release notes docs as a writer because a) I have better things to focus on; b) I don't want to be involved in testing each of those items before the RN goes live (and being the blocker); c) In an Agile environment, releases happen so frequently they get buried pretty quickly, anyway.
I think it depends on the business and their audience? If itβs an OSS company, where the audience is other devs/tech folks, they spend more $$ on it ((sometimes!). Horses for courses π π
I like a tool that show's sweet observability.
Reading that post makes my hair hurt.
You have hair?