There is no single right answer to any non-trivial problem in software engineering. So, if multiple correct solutions exist, how do you decide which is best? It's difficult to determine which is best because "best" is highly subjective and deeply personal — your opinion is formed from your individual collection of experiences, strengths & weaknesses, and values on related aspects like simplicity, maintainability, & scalability.
It's these internal values that make all of this so tricky. Imagine a spectrum with engineering excellence at one end and business needs at the other. Both elements are required for a project to be successful, and operating at either extreme can be detrimental to the other. As an example, making too many quick-twitch fixes to address urgent business needs can have significant long-term impact on the quality of the code base or system maintainability; conversely, focusing too deeply on engineering excellence can lead to over-investment in areas or competitive disadvantages from being slow to market.
Understanding this spectrum — and having awareness of where you and your colleagues lie on it — can help your team to be more pragmatic.
Awareness of this spectrum alone isn't going to do you any favors in resolving conflict from perceived disconnects between you and co-workers, though. I've found myself in design/requirements stalemates many times, and I've used the spectrum as a way to visualize my frustration.
"You see, I live over here on one end of the spectrum," I'd say, "and my colleague operates here, at the other end. We can't agree on scope, and we aren't getting started or making any progress as a result."
The problem with the visualization as a tool for conflict resolution is those pesky personal values. Neither of us thinks we're advocating for a solution that would be in the unhealthy extremes of the spectrum. The person in the engineering excellence camp just believes that business value is generated by following all the best engineering principles and creating scalable, high-performing, resilient applications whereas business needs nation wants quick delivery and maximum responsiveness to meet the ever-changing needs of its customers.
So, how do you find compromise when the source of conflict is so visceral?
Let's see if we can steal a page from the Goldilocks playbook. She's got a knack for identifying the undesirable ends of a spectrum before settling into a satisfying sweet spot. If you and your team or colleague(s) can't agree on the scope of a solution, can you agree on what it shouldn't be?
What's a reasonable solution that everybody agrees is over-engineered, and what's the fastest, but perhaps short-sighted, thing you could do? What's the effort required for each approach, and what are the risks or consequences?
Just to be clear, I'm not suggesting to simply compare different proposals by plotting them on the spectrum— that probably won't get you anywhere. Instead, work collaboratively to come up with bad solutions that lean too far in both directions. Find agreement by identifying undesirable characteristics of these options in the unhealthy parts of the spectrum.
Still not able to find compromise? It's probably time to bring in a 3rd party, preferably a stakeholder. Show them your spectrum and explain the tradeoffs that exist at the opposite ends, then present the "real" options that are on the table and allow the stakeholder to decide.
The whole activity is an exercise in pragmatism. How can two parties with equal but conflicting opinions find common ground? The key is to calibrate and remove as much subjectivity as you can. By acknowledging the necessity of both aspects — engineering excellence and needs of the business — and agreeing on the "bounds" of the spectrum, you create a framework for identifying the region for compromise. That's your sweet spot. That's likely where your "best" solution should be.