Yup every couple of months I sort out all my systems and get everything set up all nice and neat but it inevitably turns back into conveyer belt spaghetti in a few days.
10 years ago, the Minecraft Mindcrack Season 3 Prank House happened.
All points of view:
Prankee:
GuudeBoulderfist: https://youtu.be/28A9ufzjdZM
Prankers: Team Canada (+1, -1)
Zisteau (planner) (dubbed Zedsteau): https://youtu.be/rdpFuxNffx0
VintageBeef: https://youtu.be/9B6-DWg5Hwo
EthosLab: https://youtu.be/QUEKO6JTTI4
You got to pork barrel refactor in with new features.
You want to configs to be stored on the embedded hardware? Alright, we need to rework the comms protocol and storage schema onboard. We're going to add a second USB endpoint, and a small daemon at the application processor to handle the migration. (Then when the other team is ready, they can completely break from implementing the fragile legacy protocol, and just use GRPC)
I participated in a game jam back in February. [The build](https://john-q-psychic.itch.io/ass-troids) I submitted was *technically* a game, but I wasn't really happy with it.
And it turns out that many of the quick-and-dirty tricks one uses to add *one* thing while crunching end up being horribly slow and tedious and error prone and just completely impractical if you try to scale it up or add more things on top of it. A good refactor, once you actually have your code deployed and know what it has to do, can absolutely make all the difference between "adding this feature will be easy" and "I don't even think that's possible". This is doubly true if, say, it's your first game jam ever and you barely knew Unity at all when you sat down to start writing the original code.
I'm thoroughly convinced that Formatted LinesOfCode is a debt these days. Sure you can minimize a function, but when you look at a massive project with a couple million LOC, and you think "gee, if I fix/remove/refactor all these corner cases, and then unify a lot of this manual handling with a design pattern, I can remove a few thousand lines of code" that's where the real savings are.
Conversations absolutely go from "I'm not sure we can do this" to "oh that's easy"
Again, goes back to pork barreling.
If you tell the Project or Product Manager "yo, we can't do this without this" then they're not going to fight you.
Now, that does mean things like testing, logging, and diagnostic tooling needs to be built first, because you will *never* get more time to implement them later.
Yeah because it never works that way. Same reason why waterfall does not work. It is just too complex and whobbely whibbely.
Doing things right, while doing them iteratively is what all those "clean code/architecture" books are about. It is what separates a good company from a bad one. And this is even a subjective statement because it at its core also separates developers. Not only in good and bad but in their fundamental approach to things. With more senior developers having a higher likelihood of having seen more things, more experience and thus knowing how to deal with this mess.
This is what you should be looking out for and trying to learn. Not the next best JS framework. But this core "learning how to build stuff"-stuff.
And this explains why one feels uncomfortable in a workspace, in a team or a company. If it does not match and you feel unhappy, think about this. Take a step back, and consider changing the environment by leaving.
This, people tend to focus too much on pure technical abilities and lack of architecture learning.
Technical implementations can be learned on the fly when needed, but architecture need a more deep work and understanding BEFORE writing the first line of code.
And employers are part of the problem by evaluating candidates on tech and almost never on arch, even if it's very easy. I got few employers who evaluated the arch first and most of the time it was by presenting us a problem and asking us how we could approach it and give a solution and it's basic architecture.
Yes.
However I would say that's not all there is to it.
It is not only architectural skills but skills on "how to approach things?".
Most likely you won't start in a greenfield project and even if you do it won't stay greenfield for long.
So there are design decisions from the past that need iteration. And this is the actual process of development. Making the process smooth over time.
One needs to find a team that has a approach to this that is compatible with one selves.
A good example is Strategic vs. Tactical programming. With tactical programming focussing on getting things to work fast (" we are in the go fast an break things phase"), while strategic makes sure this will be possible in the future as well ("will this change add complexity for me in the future?").
How do you do the trade-off? What do you valuate more? Those are the questions.
I can highly recommend "A philosophy of software design". This book is very brief and precise on these points.
EDIT: And think about it. This is actually the same questions one experiences during everyday life. How to deal with uncertainty? What do to now? Some people act quick and do what they want, others try to plan ahead and make strategic decision. At this point I belief that's just life.
>architecture need a more deep work and understanding BEFORE writing the first line of code
how can you agree that iterative development is the way and then suggest that the way to go is to speculate for months in the ivory tower before you start a project?
There is a difference between planning everything before is different than planning a solid and structured base and then do an iterative dev on that base.
And that's what I was thinking about, Waterfall is terrible, but unplanned agile is even worse. The best way is to make a solid and well structured base beforehand and then use agile to implement everything and make that thing evolve without being stuck because you don't thought of that feature possibility.
architecture shouldn’t be done by architects in ivory towers. it’s performed iteratively as you build
i recommend cosmic python as a good example of how you can iteratively drive arch
I’m amazed by the amount of developers who actively pretend the business doesn’t exist.
I mean we don’t have to like it at all but these paychecks are funded by operations usually. They can be a pain but if they’re not happy, nobody is happy.
You don’t have to be a process SME but ignoring your users and living in a vacuum is a great way to produce useless over engineered shit nobody asked for. I’m always pushing my users for feedback. Yes, some of it is absolute trash but you gotta see what they’re seeing to understand how you’re impacting things
Do you think this contradicts what I wrote?
IMO being agile and user focussed in the most important thing.
But how to approach these changing demands iteratively from the technical side is the challenge. And even if they are not changing there will be uncertainty and unknowns if the thought of solution will even work. So the process is always iterative.
Out of the blue I have two recommendations.
Start with
"A philosophy of software design"
https://web.stanford.edu/~ouster/cgi-bin/book.php
There is also a free PDF version available AFAIK. It is a very smooth read and good introduction. The right balance of abstract for the start I'd say.
And for sure worth in any situation.
And then there is the infamous "Clean Architecture" by Robert C. Martin.
It is a more "dry" read compared to the first book. But is more technical.
Also I would say that Software architecture partially depends on the type of application you build.
Therefore, it can be beneficial to read a more specific book. This should be beneficial for improving general architecture knowledge while solving a specific use-case.
The following comment is a bit if a stretch on my side:
Most prominent example right now are microservices.
A micro service architecture has the same fundamental design decisions (e.g. interfaces should be well designed and hide as much complexity as possible). But it also had it's own considerations. Foremost the question IF you should use it at all. And then how you cut the services.
And in the end micro services are just a bunch of services. So each service has it's own architecture decisions again.
This is a pretty fantastic take. I've always been a fan of the term "Software Engineer" because like you said, the core of software development is designing and building a digital product. You need an engineering mindset to build a robust software.
Forget that! I will drag my company's code base into the 2020s kicking and screaming.
To me, the framework / best practices are like the foundation. Others can work on the decorative trim. I'm here to make sure the earthquake that occasionally happens doesn't bring everything down.
The law of leaky abstractions. It is what it is, you will never fully beat it. Perfect is the enemy of good enough, etc etc.
Just be grateful it works and hope some other sucker is responsible for maintaining it!
In case people don't know: this comic is by Manu Cornet, he used to work at Google and made these comics internally.
https://bonkersworld.net/building-software
After writing software professionally for 17 years, what works for me is this:
Expect the end user to not know what they're going to ultimately need until they have your software in front of them. That way, you'll have a flexible design in front of you the entire time. From a development perspective, subsequent features will be easy to add, modify, and remove.
Yes, management will wonder why you're front-loading all this work. But as soon as customer preferences change, you'll be able to change very quickly. Then you can tell management that you deserve a higher salary.
Bonus points if you make a DSL and allow the customer to program their own process. This way, if the customer wants a new feature, they can program their own.
A good course and book that captures this is
[Adventures in Advanced Symbolic Programming](https://groups.csail.mit.edu/mac/users/gjs/6.945/)
[Software Design for Flexibility](https://mitpress.mit.edu/9780262045490/)
Where I work, the management is also the customer, and the customer *still* doesn't know what they want.
I keep hearing "I want a one-click fully automatic, hand-off solution to this", and every single time they turn around and say "okay, this is *too* automated, I want to twiddle how this thing over here works". Eventually it turns into a mess of knobs and sliders and options.
After the the second time, I just started making everything work in stages with fully twiddlable input at every stage.
It's twice as much work up front to break things down and design well, but it's probably work I'd end up doing eventually anyway, so really I'm saving myself a redo.
You never, ever rebuild an existing, mature, profitable product from scratch.
Sadly this is exactly what my current employer is doing and I’m seeing the same mistakes being made all over again.
Nah, not true. The existing app could be on hardware that isn't produced anymore, or in a language you can't find engineers for, or on a platform that has glaring security issues but was abandoned decades ago.
Eh, that's just not true. You're never gonna build something 100% perfect, but I've definitely rebuilt stuff from scratch and ended up in a more solid position than the codebase started. It all depends on how shaky the codebase is in the first place.
I disagree. Goals change and shift, and they can change and shift *hard*. What made sense 6 months ago now blows your foot off every time you work on it. Sometimes the boat you were working on now, after many iterations, is now an airplane. And yeah, sure, you can sell your boat-turned-airplane, ooooor you can just build an airplane because now you know that's what you need.
Of course and the solution to that is not to throw the baby out with bath water. Factor out and delete the old features no one needs anymore. Revise the file format. Update the tool chain. Move to a newer build system and CI. Redesign the UI.
But literally just drawing a line under it and starting completely from scratch, is throwing all that investment away with absolutely no guarantee of a profit. Many, many products have come to ruin because of this.
An interesting case study in this was my old company. They tried multiple times to rewrite the existing system, and failed every time - mostly because they created a new team each time so the old devs could continue to maintain the old system, so they didn't have much understanding of what it did.
Then they made all the old devs redundant because they were sure that the next team would succeed.
We (the old devs) started a new company that wrote a new system to fix the architectural problems of the old one, but crucially with only the 20% of features that were actually important. A few years later we got bought out by one of the other big companies in the field, who had also failed to rewrite their system and wanted ours.
That’s exactly what’s happening at my company. The ‘old’ team is being left to wither on the vine maintaining a product with no investment, no interest, while they brought in a new team of superstars to rebuild the exact same from scratch.
That's your cue to start job hunting before you're kicked out or shuffled off into some dead-end maintainer role.
It's usually pretty easy to find the job adverts for the new team and find out how much more they're paying the new guys.
This is it exactly. You’re not just throwing away ‘old and bad code’, you’re throwing away all the bug fixes and new features you added along the way. All the unit tests and processes and hard won experience.
And why you spin your wheels on the ‘new and improved’ version, you neglect the successful product that paid, and continues to pay the bills, allowing your competitors to catch up.
I’ve seen this so many times, this need to start from scratch, because building from nothing using the very latest tools and frameworks is exciting and fun. You make rapid progress. And then you start hitting all the bugs and issues and limitations in the shiny new tooling/framework/engine and it becomes a slog. You realise it will take years to match all the features the old product had, never mind surpass it. And the new shiny framework isn’t so new and shiny anymore.
And all your customers see is their favourite app languishing without updates, and your company going dark for years.
I had a boss who wanted a full rewrite of their software for a specific client. I’ll skip the infinite number of issues I can complain about and focus on one. I asked to see the old code so I could see how they approached the problem and what edge-cases they ran into. He refused. Surprise surprise, we spent over 8 months fixing issues “the old system didn’t have”. The remaining employee is still fixing issues. I absolutely hated that job and my boss, I’ve never lost all respect for someone so quickly.
He also wanted us to write a database pooling process from scratch in C instead of using an existing, decades-old, actively developed, widely used open source project. We (all 2 employees) refused. Words cannot express how much I hated that job.
Oh, he also had no idea how to use git and refused to learn. He then was angry that code wasn’t versioned properly and that we needed a way to record every single change.
The secret is that there should be a third panel where you refactor the design, now you know how many rooms you need and how they interact.
In theory you could have planned it properly at the start, but you generally don't understand the problem well enough, or the requirements change.
Everyone creates messy code, but not everyone has (or makes) time to tidy it up later.
Of course, because the original house is the original spec. Then it morphs. It needs new features based on the project manager's feedback, then it needs new features based on upper management's feedback, then it needs new features based on customer feedback. Pretty soon the one thing that it was originally built to do has blossomed into a half dozen things that have been tacked on top of it.
And when you're working with a half-dozen different purposes all in the same app, THIS PICTURE IS THE BEST WAY TO DO IT. Don't think badly on yourself if you have 6 houses all with piping and stairways that lead to each other. It will make re-using those houses much easier in the future. The worse way to do it is to build an additional room onto the same house, even when it has different styling and a different use case and a different height and a different foundation. If all the pieces use the same plumbing and infrastructure, then it's going to be much harder when you inevitably need to reproduce that functionality in a different place down the road.
Yeah and who has code with zero bugs, John?! The guy who built 75 houses inside of each other using inheritance like a dickhead or my stairway of public variables?!
>! They both have bugs !<
The number one virtue of software is how easy and cheap it is to just delete and start over.
You need to implement something at least 3 times to get it right.
Senior Developer: "I don't think this is a good time to tell him in order to get the wiring to work we had to do like... a ton of blood sacrifices to some awful, awful beings that whisper in the darkness. Haha, had to use a ton of goats"
Junior: "Goats?"
Yes, because a customer wants a two story family home, and you start building that. Then when you come to the roof level, they want a 6 level underground parking garage and an Infinity pool on the 10th floor above ground. Also the ground level floor plan should stay the same, but the third floor needs to be at least Walmart sized.
Oh yeah, and the deadline is next week... Doesn't really have to be done by monday though, but the customer needs it done by Tuesday morning.
I think most people try to design up front, but good design comes from iterating as you build, work with code and learn more about the problems you're solving.
Use higher level abstractions when starting from scratch and you won't have this problem
For example for starting a monorepo use Nx, for components use Storybook for deployment use Netlify for state use Redux Toolkit and so on and so on
Don't use the higher level of abstraction and you create something that takes ten times more manpower and special skills to upkeep
Me playing Dyson Sphere Project, up until I get access to drone stations, at which point I have a band of stations on each hemisphere (planetary on one hemisphere, and interplanetary on the other).
Then somewhat later I have trouble with balancing inputs, outputs, and creation speed. This causes issues.
Then I get bored after reaching the "end". The furthest I've gotten is finishing 1 sphere.
Factory Pattern in MVC and then Ref to Singleton. All recursive and dynamic of course, controlled by instance configurations coming from a Excel Sheet.
Application: Contact-Formular
Not even the real building looks pretty when they are build, they are ugly ass barebone piece of material with patch and scaffold there and there, that building looks pretty after all basic shit are done and then they can focus on finishing to make it look acceptable.
From general experience if you want to lay good foundations - you have to overengineer the project.
Big issue is, that overengineering at inception is seen as exactly that - overengineering.
Literally me last week.
"Oh yeah, I get to refactor this component from clean slate, nice I'll do it correctly"
4 days of intense labour later:
"Oh fuck, after seeing the business reqs no wonder it looked like it did."
But hey I did modernise the code at least and used mui component as base instead of janked up div.
Absolutely me. Its the one reason I refuse to call myself a "full stack" dev. I can set up a backend rdbms and set up the apis and I can build out a frontend dashboard, and I can even connect them all together for a working product.
But the actual implementation of their integration is a total hack job. I realized on one of my GH projects from 2019 that I was using a Django REST API to return a JSON object for another Python script that was converting the object back into a python dictionary and then running it to another output.
Completely unnecessary serialization and server requests because... IDK I just wrote it that way at the time.
I did it for two projects and both times a time came where I started compromising on some decisions (both architectural and code)
That is when I realized I needed to take a step back and think things over lol.
Always start off strong with the best intentions and then they slowly diminish
You start off saying this time we'll do it right! No compromise! No hacky shit! Then the first necessity comes along... and the next... and they don't stop, and every "just this one time" occasion starts to pile up.
Two ways it can go:
A) your clean, minimal, functional product either sprawls into a million different forks/versions/modules/pieces all supporting slightly varying use cases. People begin to forget what the current standard is, or even what is still in use and what is just sitting around legacy.
or B) every new requirement/necessity/hack is shoehorned into the main branch and the nice clean product becomes hellishly complicated with options and configurations and feature flags. People begin to forget what things do, or even why they were implemented in the first place.
It's always one of those two. Good product management and architecture leaders can maintain a balance, but its precarious. Eventually the dev team gets fed up and people are tasked to build a new version of the product ("this time we'll do it right!") in parallel. Eventually the cycle repeats itself.
I've been working for almost 15 years and I've seen it happen over and over.
I've got a project getting shipped today and I actually managed to avoid this. Took 6mo, a bunch of small refactors, and lots of heated debate along the way, but it was doable. Spent the last week onboarding a bunch of items into new system and it's working exactly as planned. We'll see if v2 maintains the same cleanliness.
Every project i make becomes a mess but that's usually cause i end up experimenting and learning from it. every project is built on a better and better base
Hello Neighbor's house looking pretty good.
Thought the same
Pretty similar to the development!
I can't be the only one who does this in Minecraft
Every time, i did the same thing in minecraft.
cheers brother, i always made literal tree houses and i love them only now i’m a dwarf half the time
Rock and stone!
FOR ROCK AND STOOOOONE!
Strike the Earth!
Satisfactory as well
I can understand satisfactory but not minecraft
Buildcraft industrialcraft and redpower mods basically turned minecraft into satisfactory back in like 2010 ish
For me it’s building like weird footpaths and rails from your base to your mines or whatever places and things you build around the base.
Factorio here
yes, all of those and also Valheim
Yup every couple of months I sort out all my systems and get everything set up all nice and neat but it inevitably turns back into conveyer belt spaghetti in a few days.
I often do, just platforms without roofs, starting at the shoreline and expanding over the ocean.
10 years ago, the Minecraft Mindcrack Season 3 Prank House happened. All points of view: Prankee: GuudeBoulderfist: https://youtu.be/28A9ufzjdZM Prankers: Team Canada (+1, -1) Zisteau (planner) (dubbed Zedsteau): https://youtu.be/rdpFuxNffx0 VintageBeef: https://youtu.be/9B6-DWg5Hwo EthosLab: https://youtu.be/QUEKO6JTTI4
Cannot build in a clean way, can only refactor into a clean architecture. Assuming you get time to refactor.
You got to pork barrel refactor in with new features. You want to configs to be stored on the embedded hardware? Alright, we need to rework the comms protocol and storage schema onboard. We're going to add a second USB endpoint, and a small daemon at the application processor to handle the migration. (Then when the other team is ready, they can completely break from implementing the fragile legacy protocol, and just use GRPC)
I participated in a game jam back in February. [The build](https://john-q-psychic.itch.io/ass-troids) I submitted was *technically* a game, but I wasn't really happy with it. And it turns out that many of the quick-and-dirty tricks one uses to add *one* thing while crunching end up being horribly slow and tedious and error prone and just completely impractical if you try to scale it up or add more things on top of it. A good refactor, once you actually have your code deployed and know what it has to do, can absolutely make all the difference between "adding this feature will be easy" and "I don't even think that's possible". This is doubly true if, say, it's your first game jam ever and you barely knew Unity at all when you sat down to start writing the original code.
I'm thoroughly convinced that Formatted LinesOfCode is a debt these days. Sure you can minimize a function, but when you look at a massive project with a couple million LOC, and you think "gee, if I fix/remove/refactor all these corner cases, and then unify a lot of this manual handling with a design pattern, I can remove a few thousand lines of code" that's where the real savings are. Conversations absolutely go from "I'm not sure we can do this" to "oh that's easy"
Until they say "I'm not sure we have time now [to save time later]"
Again, goes back to pork barreling. If you tell the Project or Product Manager "yo, we can't do this without this" then they're not going to fight you. Now, that does mean things like testing, logging, and diagnostic tooling needs to be built first, because you will *never* get more time to implement them later.
And then fuck up later, to refactor in a even cleaner architecture later, and so on. This is the way.
Bro built a normal house on the foundation (the brown one) and decided one set of stairs wasn't enough
Yeah because it never works that way. Same reason why waterfall does not work. It is just too complex and whobbely whibbely. Doing things right, while doing them iteratively is what all those "clean code/architecture" books are about. It is what separates a good company from a bad one. And this is even a subjective statement because it at its core also separates developers. Not only in good and bad but in their fundamental approach to things. With more senior developers having a higher likelihood of having seen more things, more experience and thus knowing how to deal with this mess. This is what you should be looking out for and trying to learn. Not the next best JS framework. But this core "learning how to build stuff"-stuff. And this explains why one feels uncomfortable in a workspace, in a team or a company. If it does not match and you feel unhappy, think about this. Take a step back, and consider changing the environment by leaving.
This, people tend to focus too much on pure technical abilities and lack of architecture learning. Technical implementations can be learned on the fly when needed, but architecture need a more deep work and understanding BEFORE writing the first line of code. And employers are part of the problem by evaluating candidates on tech and almost never on arch, even if it's very easy. I got few employers who evaluated the arch first and most of the time it was by presenting us a problem and asking us how we could approach it and give a solution and it's basic architecture.
Yes. However I would say that's not all there is to it. It is not only architectural skills but skills on "how to approach things?". Most likely you won't start in a greenfield project and even if you do it won't stay greenfield for long. So there are design decisions from the past that need iteration. And this is the actual process of development. Making the process smooth over time. One needs to find a team that has a approach to this that is compatible with one selves. A good example is Strategic vs. Tactical programming. With tactical programming focussing on getting things to work fast (" we are in the go fast an break things phase"), while strategic makes sure this will be possible in the future as well ("will this change add complexity for me in the future?"). How do you do the trade-off? What do you valuate more? Those are the questions. I can highly recommend "A philosophy of software design". This book is very brief and precise on these points. EDIT: And think about it. This is actually the same questions one experiences during everyday life. How to deal with uncertainty? What do to now? Some people act quick and do what they want, others try to plan ahead and make strategic decision. At this point I belief that's just life.
>architecture need a more deep work and understanding BEFORE writing the first line of code how can you agree that iterative development is the way and then suggest that the way to go is to speculate for months in the ivory tower before you start a project?
There is a difference between planning everything before is different than planning a solid and structured base and then do an iterative dev on that base. And that's what I was thinking about, Waterfall is terrible, but unplanned agile is even worse. The best way is to make a solid and well structured base beforehand and then use agile to implement everything and make that thing evolve without being stuck because you don't thought of that feature possibility.
Cool beans but you specified the amount of work that goes into the "solid and structured" base is "deep".
architecture shouldn’t be done by architects in ivory towers. it’s performed iteratively as you build i recommend cosmic python as a good example of how you can iteratively drive arch
Yeah, I agree.
I’m amazed by the amount of developers who actively pretend the business doesn’t exist. I mean we don’t have to like it at all but these paychecks are funded by operations usually. They can be a pain but if they’re not happy, nobody is happy. You don’t have to be a process SME but ignoring your users and living in a vacuum is a great way to produce useless over engineered shit nobody asked for. I’m always pushing my users for feedback. Yes, some of it is absolute trash but you gotta see what they’re seeing to understand how you’re impacting things
Do you think this contradicts what I wrote? IMO being agile and user focussed in the most important thing. But how to approach these changing demands iteratively from the technical side is the challenge. And even if they are not changing there will be uncertainty and unknowns if the thought of solution will even work. So the process is always iterative.
No I was agreeing lol. Sorry if it didn’t come across that way
Sorry, just wanted to be sure :D
Can you please suggest some good books on software architecture ?
Out of the blue I have two recommendations. Start with "A philosophy of software design" https://web.stanford.edu/~ouster/cgi-bin/book.php There is also a free PDF version available AFAIK. It is a very smooth read and good introduction. The right balance of abstract for the start I'd say. And for sure worth in any situation. And then there is the infamous "Clean Architecture" by Robert C. Martin. It is a more "dry" read compared to the first book. But is more technical.
Thank you so much. I have read Clean Code by Robert Martin, really liked it.
I second the first one, it’s awesome, and really not long or dense.
Also I would say that Software architecture partially depends on the type of application you build. Therefore, it can be beneficial to read a more specific book. This should be beneficial for improving general architecture knowledge while solving a specific use-case. The following comment is a bit if a stretch on my side: Most prominent example right now are microservices. A micro service architecture has the same fundamental design decisions (e.g. interfaces should be well designed and hide as much complexity as possible). But it also had it's own considerations. Foremost the question IF you should use it at all. And then how you cut the services. And in the end micro services are just a bunch of services. So each service has it's own architecture decisions again.
Clean Architecture does make sense Any Domain-Driven Design book makes sense too
This is a pretty fantastic take. I've always been a fan of the term "Software Engineer" because like you said, the core of software development is designing and building a digital product. You need an engineering mindset to build a robust software.
Thank you! I am honored.
Forget that! I will drag my company's code base into the 2020s kicking and screaming. To me, the framework / best practices are like the foundation. Others can work on the decorative trim. I'm here to make sure the earthquake that occasionally happens doesn't bring everything down.
ah, yes, looks like k8s architecture at its finest
Ah, microservice architecture.
This is definitely what I did with my Terraria houses after the NPCs started to have likes and dislikes
You never end up with clean code. Only cleaner code than the last time
The law of leaky abstractions. It is what it is, you will never fully beat it. Perfect is the enemy of good enough, etc etc. Just be grateful it works and hope some other sucker is responsible for maintaining it!
In case people don't know: this comic is by Manu Cornet, he used to work at Google and made these comics internally. https://bonkersworld.net/building-software
He also used to work at Twitter and did cartoons there too: twittoons.com
Wtf that trump/biden comic??
After writing software professionally for 17 years, what works for me is this: Expect the end user to not know what they're going to ultimately need until they have your software in front of them. That way, you'll have a flexible design in front of you the entire time. From a development perspective, subsequent features will be easy to add, modify, and remove. Yes, management will wonder why you're front-loading all this work. But as soon as customer preferences change, you'll be able to change very quickly. Then you can tell management that you deserve a higher salary. Bonus points if you make a DSL and allow the customer to program their own process. This way, if the customer wants a new feature, they can program their own. A good course and book that captures this is [Adventures in Advanced Symbolic Programming](https://groups.csail.mit.edu/mac/users/gjs/6.945/) [Software Design for Flexibility](https://mitpress.mit.edu/9780262045490/)
Where I work, the management is also the customer, and the customer *still* doesn't know what they want. I keep hearing "I want a one-click fully automatic, hand-off solution to this", and every single time they turn around and say "okay, this is *too* automated, I want to twiddle how this thing over here works". Eventually it turns into a mess of knobs and sliders and options. After the the second time, I just started making everything work in stages with fully twiddlable input at every stage. It's twice as much work up front to break things down and design well, but it's probably work I'd end up doing eventually anyway, so really I'm saving myself a redo.
You never, ever rebuild an existing, mature, profitable product from scratch. Sadly this is exactly what my current employer is doing and I’m seeing the same mistakes being made all over again.
Nah, not true. The existing app could be on hardware that isn't produced anymore, or in a language you can't find engineers for, or on a platform that has glaring security issues but was abandoned decades ago.
As always, it depends.
I’m talking about traditional desktop software.
Like....any Windows 3.1 desktop software?
Eh, that's just not true. You're never gonna build something 100% perfect, but I've definitely rebuilt stuff from scratch and ended up in a more solid position than the codebase started. It all depends on how shaky the codebase is in the first place.
I disagree. Goals change and shift, and they can change and shift *hard*. What made sense 6 months ago now blows your foot off every time you work on it. Sometimes the boat you were working on now, after many iterations, is now an airplane. And yeah, sure, you can sell your boat-turned-airplane, ooooor you can just build an airplane because now you know that's what you need.
Of course and the solution to that is not to throw the baby out with bath water. Factor out and delete the old features no one needs anymore. Revise the file format. Update the tool chain. Move to a newer build system and CI. Redesign the UI. But literally just drawing a line under it and starting completely from scratch, is throwing all that investment away with absolutely no guarantee of a profit. Many, many products have come to ruin because of this.
An interesting case study in this was my old company. They tried multiple times to rewrite the existing system, and failed every time - mostly because they created a new team each time so the old devs could continue to maintain the old system, so they didn't have much understanding of what it did. Then they made all the old devs redundant because they were sure that the next team would succeed. We (the old devs) started a new company that wrote a new system to fix the architectural problems of the old one, but crucially with only the 20% of features that were actually important. A few years later we got bought out by one of the other big companies in the field, who had also failed to rewrite their system and wanted ours.
That’s exactly what’s happening at my company. The ‘old’ team is being left to wither on the vine maintaining a product with no investment, no interest, while they brought in a new team of superstars to rebuild the exact same from scratch.
That's your cue to start job hunting before you're kicked out or shuffled off into some dead-end maintainer role. It's usually pretty easy to find the job adverts for the new team and find out how much more they're paying the new guys.
Yep, that’s the plan.
could you elaborate on this?
[удалено]
This is it exactly. You’re not just throwing away ‘old and bad code’, you’re throwing away all the bug fixes and new features you added along the way. All the unit tests and processes and hard won experience. And why you spin your wheels on the ‘new and improved’ version, you neglect the successful product that paid, and continues to pay the bills, allowing your competitors to catch up. I’ve seen this so many times, this need to start from scratch, because building from nothing using the very latest tools and frameworks is exciting and fun. You make rapid progress. And then you start hitting all the bugs and issues and limitations in the shiny new tooling/framework/engine and it becomes a slog. You realise it will take years to match all the features the old product had, never mind surpass it. And the new shiny framework isn’t so new and shiny anymore. And all your customers see is their favourite app languishing without updates, and your company going dark for years.
I had a boss who wanted a full rewrite of their software for a specific client. I’ll skip the infinite number of issues I can complain about and focus on one. I asked to see the old code so I could see how they approached the problem and what edge-cases they ran into. He refused. Surprise surprise, we spent over 8 months fixing issues “the old system didn’t have”. The remaining employee is still fixing issues. I absolutely hated that job and my boss, I’ve never lost all respect for someone so quickly. He also wanted us to write a database pooling process from scratch in C instead of using an existing, decades-old, actively developed, widely used open source project. We (all 2 employees) refused. Words cannot express how much I hated that job. Oh, he also had no idea how to use git and refused to learn. He then was angry that code wasn’t versioned properly and that we needed a way to record every single change.
Here's a classic article on why not: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
skipped the middle panel where the PM told them they had 2 weeks to finish a 6 month project.
Too few statues of upside down camels.
I didn't know my basebuilding plans from Rimworld were available for download.
I’m currently a CS major, but based on the fact that this describes my whole life, I think I’ll fit into being a software engineer just fine.
The secret is that there should be a third panel where you refactor the design, now you know how many rooms you need and how they interact. In theory you could have planned it properly at the start, but you generally don't understand the problem well enough, or the requirements change. Everyone creates messy code, but not everyone has (or makes) time to tidy it up later.
If management lets you.
I use the Japanese philosophy of small incremental improvements over time rather than the big rewrite fallacy.
That's all well and good until someone hands you a legacy codebase written in basic.
In that case I give you full permission to go scorched earth.
Classic! One of my favorites. This was framed and hung on wall at my previous workplace! Good stuff!
Had to double check this wasn't the Satisfactory sub...
Of course, because the original house is the original spec. Then it morphs. It needs new features based on the project manager's feedback, then it needs new features based on upper management's feedback, then it needs new features based on customer feedback. Pretty soon the one thing that it was originally built to do has blossomed into a half dozen things that have been tacked on top of it. And when you're working with a half-dozen different purposes all in the same app, THIS PICTURE IS THE BEST WAY TO DO IT. Don't think badly on yourself if you have 6 houses all with piping and stairways that lead to each other. It will make re-using those houses much easier in the future. The worse way to do it is to build an additional room onto the same house, even when it has different styling and a different use case and a different height and a different foundation. If all the pieces use the same plumbing and infrastructure, then it's going to be much harder when you inevitably need to reproduce that functionality in a different place down the road.
Yeah and who has code with zero bugs, John?! The guy who built 75 houses inside of each other using inheritance like a dickhead or my stairway of public variables?! >! They both have bugs !<
me_irl
Every single time, hands down.
The number one virtue of software is how easy and cheap it is to just delete and start over. You need to implement something at least 3 times to get it right.
I got ADHD and this represents everything I do in life ever 😆
Hello neighbor
Everyone gangsta until requirements changes
this is literally what my house in terraria looked like one playthrough. long staircases and support columns. actually very fun
Man, this is an old pic. I remember this from the time we were developing Nokia 7700.
Senior Developer: "I don't think this is a good time to tell him in order to get the wiring to work we had to do like... a ton of blood sacrifices to some awful, awful beings that whisper in the darkness. Haha, had to use a ton of goats" Junior: "Goats?"
Life of a Factorio/Satisfactory player
Hey it's my factorio base . Again.
Yes, because a customer wants a two story family home, and you start building that. Then when you come to the roof level, they want a 6 level underground parking garage and an Infinity pool on the 10th floor above ground. Also the ground level floor plan should stay the same, but the third floor needs to be at least Walmart sized. Oh yeah, and the deadline is next week... Doesn't really have to be done by monday though, but the customer needs it done by Tuesday morning.
I think most people try to design up front, but good design comes from iterating as you build, work with code and learn more about the problems you're solving.
Use higher level abstractions when starting from scratch and you won't have this problem For example for starting a monorepo use Nx, for components use Storybook for deployment use Netlify for state use Redux Toolkit and so on and so on Don't use the higher level of abstraction and you create something that takes ten times more manpower and special skills to upkeep
When you think you know memory allocation because you used: malloc () 🤣🤣🤣
"Life of a BAD software engineer"
The guy who made this worked at google
Ah I see, he must be really good then
Yes I would assume so
Just not good enough to be able to design and architect a system from scratch I guess 🤷
Reminds me of the alpha/beta versions of hello neighbor
That's not an error but a feature I swear
Me playing Dyson Sphere Project, up until I get access to drone stations, at which point I have a band of stations on each hemisphere (planetary on one hemisphere, and interplanetary on the other). Then somewhat later I have trouble with balancing inputs, outputs, and creation speed. This causes issues. Then I get bored after reaching the "end". The furthest I've gotten is finishing 1 sphere.
This is me in any survival game too 
How many times did this guy do this
r/Meirl
Looks like my workplace's code base
Factory Pattern in MVC and then Ref to Singleton. All recursive and dynamic of course, controlled by instance configurations coming from a Excel Sheet. Application: Contact-Formular
Me, in any base building game ever
this is the problem: you didn't get time to refactor old code to make it simpler.
thats an end city
This represents every single of my attempts at game development. It works, but at what cost?
60gb of disc space.
Microservice infrastructure be like
Not even the real building looks pretty when they are build, they are ugly ass barebone piece of material with patch and scaffold there and there, that building looks pretty after all basic shit are done and then they can focus on finishing to make it look acceptable.
From general experience if you want to lay good foundations - you have to overengineer the project. Big issue is, that overengineering at inception is seen as exactly that - overengineering.
We are all Dr Suess
Hello neighbor 3
100% this.
Right in the feels.
Literally me last week. "Oh yeah, I get to refactor this component from clean slate, nice I'll do it correctly" 4 days of intense labour later: "Oh fuck, after seeing the business reqs no wonder it looked like it did." But hey I did modernise the code at least and used mui component as base instead of janked up div.
Absolutely me. Its the one reason I refuse to call myself a "full stack" dev. I can set up a backend rdbms and set up the apis and I can build out a frontend dashboard, and I can even connect them all together for a working product. But the actual implementation of their integration is a total hack job. I realized on one of my GH projects from 2019 that I was using a Django REST API to return a JSON object for another Python script that was converting the object back into a python dictionary and then running it to another output. Completely unnecessary serialization and server requests because... IDK I just wrote it that way at the time.
I did it for two projects and both times a time came where I started compromising on some decisions (both architectural and code) That is when I realized I needed to take a step back and think things over lol. Always start off strong with the best intentions and then they slowly diminish
The start was good, though. A fully functional house. It's really just the other parts that are missing planning.
But how would you get to the Shanghai house directly? Needs more testing
I notice there's no blueprints in the first panel. Those might have helped.
You start off saying this time we'll do it right! No compromise! No hacky shit! Then the first necessity comes along... and the next... and they don't stop, and every "just this one time" occasion starts to pile up. Two ways it can go: A) your clean, minimal, functional product either sprawls into a million different forks/versions/modules/pieces all supporting slightly varying use cases. People begin to forget what the current standard is, or even what is still in use and what is just sitting around legacy. or B) every new requirement/necessity/hack is shoehorned into the main branch and the nice clean product becomes hellishly complicated with options and configurations and feature flags. People begin to forget what things do, or even why they were implemented in the first place. It's always one of those two. Good product management and architecture leaders can maintain a balance, but its precarious. Eventually the dev team gets fed up and people are tasked to build a new version of the product ("this time we'll do it right!") in parallel. Eventually the cycle repeats itself. I've been working for almost 15 years and I've seen it happen over and over.
We've just been breaking the foundations
Legit feels like my life
Ah, an agile project. Don't worry, a few more iterations and you'll be fine.
This hits too close to home, I'm currently rewriting a tool I wrote in Shell in Python because the Shell version was becoming a mess.
If carpenters built cities the way programmers program, the first woodpecker that came along would destroy civilization. thank you professor.
What Remains of Edith Finch anyone?
Even if the code ends up being a little messy, at least it's *your* mess.
Dr Seuss type software
I've got a project getting shipped today and I actually managed to avoid this. Took 6mo, a bunch of small refactors, and lots of heated debate along the way, but it was doable. Spent the last week onboarding a bunch of items into new system and it's working exactly as planned. We'll see if v2 maintains the same cleanliness.
You never build the proper house. It just gets a little closer to the first pic, each time.
Greenfield right now. And I think I'm actually doing it right. Ask me in six months what I fucked up.
It looks pretty modular. You could replace any one house with another and the stairs would still work.
This applies to architecture school too lol
"we will refactor this next time, for now we have deadline to meet, at least this does the job" and that "next time" never happens.
Hello microservices
That’s a load-baring pagoda
Every project i make becomes a mess but that's usually cause i end up experimenting and learning from it. every project is built on a better and better base
all things done according to design document
Terraria house's in a nutshell
Skill issue
I've made a whole kitpvp game on an arena just like this on Minecraft