Your development team can build almost anything. Machine learning models that automate complex tasks, algorithms that optimize operations, systems that scale beyond human capacity. The technology exists. The question most teams skip is more important than the capabilities they’re chasing.
Should we build this?
Coming up, we’ll explore why asking “should we” before “can we” separates responsible AI development from reckless innovation. You’ll discover the hidden costs of building without ethics, the critical role diverse teams play in algorithm design, and the practical framework for developing AI that serves humanity instead of just maximizing profit.
The Facial Recognition Problem
Cordell Carter, Executive Director of the Aspen Institute Socrates Program, travels internationally frequently. For years, he encountered the same frustrating problem at global entry checkpoints.
The facial recognition systems didn’t work on him. As someone with heavily melanated skin, he’d stand at the kiosk while the technology failed repeatedly. This continued until a major lawsuit forced changes to the algorithm design.
I travel internationally a lot as a dark-complected person. The facial recognition wasn’t working on global entry. It just never worked for me. That changed after a big lawsuit a couple years ago. Now no problem. They just had to adjust the design of the algorithm inside the machine to pick up pigmented skin, people that are heavily melanated.
Carter points out the obvious question: “You can imagine how frustrating that is when the vast majority of the world’s travelers look like me. So if they look like me, who designed it? People who didn’t look like me. Who are you building it for exactly?”
This story reveals the fundamental problem with how most organizations approach AI development. They optimize for technical capability without considering who benefits, who gets excluded, and whether the solution serves humanity or just serves convenience for a narrow demographic.
The Innovation Trap
Most technology development follows a predictable pattern. Engineers identify what’s technically possible. Product teams evaluate market opportunity. Leadership approves based on potential returns. The system gets built, deployed, and scaled.
The ethical implications get addressed later, if at all. By then, the technology has created dependencies, established market positions, and generated revenue streams that make course correction difficult.
Carter describes this as focusing solely on the “can we” question. Organizations celebrate innovation for its own sake, measuring success by technical achievement rather than human impact.
The problem compounds when you consider the pace of AI advancement. Technologies that seemed theoretical five years ago are now embedded in critical systems. Algorithms make hiring decisions, approve loans, diagnose medical conditions, and determine prison sentences. The stakes are no longer abstract.
When you build without asking “should we,” you create several predictable problems.
- Bias gets coded into systems. The facial recognition failure wasn’t an accident. It was the inevitable result of homogeneous teams building for users who looked like them.
- Unintended consequences multiply. AI optimizes for the objectives you specify, often in ways you didn’t anticipate and wouldn’t choose if you thought through the implications.
- Displacement happens without remedy. Automation eliminates jobs faster than new opportunities emerge, leaving entire communities economically stranded.
- Power concentrates. The organizations that control AI capabilities gain disproportionate influence over systems that affect everyone.
These aren’t hypothetical risks. They’re current realities playing out across industries and geographies.
What “Should We” Actually Means
Asking “should we” before “can we” requires a different framework for evaluating technology development. You need criteria beyond technical feasibility and market opportunity.
Carter offers clear guidance on what this evaluation should prioritize:
What are the things that truly make our life better? We should be using AI to tackle the world’s problems, not to make more money. We need humans to make money so they can buy things. That’s important for our economy. But we only have one use case: how can I get more profit per hour for the next widget being made? At some point you run out of people to purchase the things you saved all this money building.
This perspective reframes the entire development process. Instead of asking what you can automate to reduce costs, ask what problems genuinely need solving. Instead of optimizing for shareholder returns, consider the broader economic and social implications.
The “should we” question examines several dimensions.
- Does this make humanity better? Not just more efficient or more profitable, but genuinely better off. Does it solve a problem that matters? Does it improve wellbeing in measurable ways?
- Who benefits and who bears the cost? Every technology creates winners and losers. Make those trade-offs explicit before you build, when you still have the power to choose differently.
- What happens to displaced workers? If your AI eliminates jobs, what responsibility do you have for the people affected? Saying “that’s creative destruction” sounds reasonable until you’re the one being destroyed.
- How does this shift power dynamics? Does your technology concentrate control or distribute opportunity? Does it empower users or make them more dependent?
- What could go wrong? Run scenarios where your AI gets misused, hacked, or produces unintended outcomes. If the downside scenarios are catastrophic, you probably shouldn’t build it regardless of the upside potential.
The Diversity Imperative in Algorithm Design
The facial recognition failure illustrates a broader principle. Homogeneous teams build biased systems. They don’t intend to exclude, they simply design for what they know and who they are.
Carter emphasizes that solving this requires more than demographic diversity:
This is a call for more diversity when it comes to algorithm design. I’m talking about experiential diversity, ideological diversity, geographic diversity. People that have seen the world from different places, they need to be part of the design team. Because if you design it right, you won’t have built-in bias.
Diverse design teams catch problems homogeneous teams miss. Someone who has experienced discrimination notices when an algorithm might perpetuate it. Someone from a different economic background questions assumptions about access and affordability. Someone with disabilities identifies usability barriers before they ship.
This goes beyond avoiding harm. Diverse perspectives produce better solutions to complex problems. When you’re designing AI to address climate change, poverty, healthcare access, or education inequality, you need people at the table who understand those problems from lived experience.
Building diverse technical teams requires intentional effort. The current pipeline produces homogeneous talent pools. Waiting for the pipeline to fix itself means waiting decades while continuing to build biased systems.
You need several parallel strategies.
- Recruit from non-traditional sources. Look beyond elite universities and conventional backgrounds. Talent exists in unexpected places when you’re willing to train for specific technical skills.
- Create pathways for career changers. People with domain expertise in healthcare, education, social services, or other fields bring valuable perspectives when they transition into technical roles.
- Partner with diverse communities. Involve the people affected by your technology in the design process, not just as users but as co-creators.
- Examine your hiring criteria. Many requirements filter for privilege rather than capability. Question whether each criterion actually predicts performance or just selects for people who have had certain opportunities.
- Build inclusive cultures. Diverse hiring fails if people leave because they don’t belong. The belonging principles from workplace culture apply equally to technical teams.
The Global Labor Question
Carter raises a concern most AI developers haven’t seriously considered. Thirty years ago, China developed into an economic powerhouse partly through millions of low-wage, low-skill manufacturing jobs. People could work, earn income, and build lives.
The African continent faces a different reality. Over the next 15 to 20 years, an additional 500 million people between the ages of 16 and 35 will enter the workforce. That represents enormous human potential. It also represents a massive challenge.
The low-skill jobs that employed previous generations won’t exist. AI and automation have eliminated them. So what happens when you can’t employ half a billion people who want to work?
What happens when you can’t employ people who want to be employed? Bad things happen. We’ve seen in the US when NAFTA comes along and low-cost shipping makes the Midwest factories close. Those jobs went south. We didn’t have a great solution for those displaced workers. We basically said that’s the cost of development, this is called creative destruction. Well, that’s not very pleasant to hear if you’re the one being destroyed.
This isn’t just an African problem. It’s a global preview of what happens when innovation outpaces our ability to create opportunity. The same dynamic plays out in manufacturing communities, retail sectors, transportation industries, and increasingly in knowledge work.
The “should we” question must include: what happens to the humans displaced by this technology? If your answer is “not our problem,” you’re contributing to instability that eventually affects everyone, including your business.
Responsible AI development considers the labor implications before deployment, not after. This might mean:
- Phasing implementation to allow workforce adaptation instead of sudden displacement.
- Investing in retraining for affected workers, treating it as a cost of doing business rather than someone else’s responsibility.
- Choosing different optimization targets that augment human capability rather than replace it entirely.
- Supporting policy solutions that create safety nets and transition pathways for displaced workers.
These approaches cost more in the short term. They also prevent the social instability that disrupts markets and undermines the conditions your business needs to thrive.
Building AI for Human Problems
Carter suggests redirecting AI development toward problems that genuinely matter for humanity. Climate change. Environmental degradation. Healthcare access. Education quality. Food security.
These challenges exceed human cognitive capacity to solve through traditional means. AI could model scenarios, identify solutions, and optimize resource allocation in ways our brains alone cannot. This represents the technology’s highest use.
Think about supercomputing power and all the different possibilities that can actually get you to a solution set. That’s the perfect use for AI. There may be 15,000 possible scenarios to solve this environmental issue. Humans can’t do that fast. But an AI-powered machine could. Let them quit, sick it on these big intractable issues that are politically challenging to solve because humans are involved. Let’s take the humans out of tough things.
The shift from “how do we maximize profit” to “what problems need solving” changes everything about your development priorities. You evaluate opportunities based on impact rather than returns. You measure success by outcomes for humanity rather than shareholder value.
This doesn’t mean abandoning profitability. Markets exist for solutions to genuine problems. The companies that solve climate challenges, improve healthcare outcomes, or expand educational access will generate significant returns. The difference is leading with purpose and letting profit follow rather than optimizing solely for financial metrics.
Several questions guide this reorientation.
- What capabilities does AI have that humans lack? Use technology for what it does better than we do: processing vast datasets, modeling complex systems, identifying patterns across massive information sets.
- What problems cause the most suffering? Direct your innovation toward reducing harm rather than maximizing convenience for people who already have abundant options.
- Where does human judgment need to remain? Some decisions should never be fully automated, even if technically possible. Identify where human wisdom, empathy, and context matter more than optimization.
- How do we ensure equitable access? Building solutions that only wealthy individuals or nations can afford doesn’t solve global problems. Design for accessibility from the start.
The Practical Framework
Implementing “should we before can we” requires changing how your organization evaluates and approves AI development projects. You need new questions in your decision-making process.
Stage 1: Problem Definition
Before any technical work begins, articulate the problem you’re solving and why it matters. Generic answers like “increase efficiency” or “reduce costs” don’t suffice. Get specific about human impact.
Who experiences this problem? How does it affect their lives? What happens if it remains unsolved? Who else has tried to address it and what can we learn from their attempts?
This stage filters out projects that exist primarily because the technology is cool or the market opportunity looks attractive without clear human benefit.
Stage 2: Impact Assessment
Map both intended and potential unintended consequences. Who benefits from this solution? Who might be harmed? What systemic effects could emerge at scale?
Include diverse perspectives in this assessment. People with different backgrounds will identify risks and opportunities that homogeneous teams overlook.
Stage 3: Alternative Evaluation
Just because you can use AI doesn’t mean you should. Sometimes simpler solutions work better. Sometimes human judgment should remain central. Sometimes the problem needs policy changes rather than technical fixes.
Evaluate whether AI represents the best approach or simply the approach your organization knows how to implement.
Stage 4: Ethical Design Principles
If you proceed with development, establish clear principles that govern design choices. These might include transparency about how the system works, user control over their data, mechanisms for appeals or corrections, and regular audits for bias.
Document these principles and make them non-negotiable. When technical teams face trade-offs, they need clear guidance about which values take priority.
Stage 5: Ongoing Monitoring
Ethical AI development doesn’t end at deployment. Systems behave differently at scale than in controlled testing. Usage patterns reveal problems you didn’t anticipate. Context changes in ways that make yesterday’s ethical solution today’s harmful tool.
Build monitoring and adjustment into your development cycle. Assign responsibility for tracking outcomes and making corrections. Create channels for users to report problems and ensure those reports reach decision-makers.
What This Means for Your Business
Most business leaders view ethical AI development as a compliance burden or reputation management. This perspective misses the strategic advantage.
Organizations that ask “should we” before “can we” build better products. They avoid costly mistakes, reduce exposure to regulatory risk, and create solutions that customers trust.
When you involve diverse teams in design, you reach broader markets. When you consider labor implications, you avoid the backlash that disrupts operations. When you optimize for human benefit rather than narrow metrics, you build sustainable businesses rather than extractive ones.
The companies that will dominate the next decade won’t be those with the most advanced AI capabilities. They’ll be the ones that deploy those capabilities in service of genuine human needs, designed with input from diverse perspectives, implemented with consideration for social impact.
The choice isn’t between ethics and profits. It’s between short-term extraction and long-term value creation. Between building AI that serves a narrow slice of humanity and building AI that serves all of us.
Moving Forward
The pace of AI development will only accelerate. The gap between “can we” and “should we” will widen unless organizations intentionally close it.
You have a choice in how your company participates in this transformation. You can chase technical capability without considering implications, celebrating innovation while ignoring impact. Or you can lead with the harder question, building AI that makes humanity genuinely better off.
This requires structural changes. New evaluation frameworks that assess projects on human impact, not just technical feasibility or market opportunity. Diverse teams that bring multiple perspectives to design decisions. Ethical principles that guide development choices when trade-offs emerge. Ongoing monitoring that catches problems before they compound.
The organizations that make these changes now will define what responsible AI development means. They’ll set standards that others follow. They’ll build trust that translates to market advantage.
The question facing your business is simple. When you have the technical capability to build something, will you also have the wisdom to ask whether you should?
If you’re ready to develop an ethical AI framework that protects your business while serving humanity, book a complimentary strategy session with one of our executive coaches. We’ll help you build the decision-making systems that ensure your innovation creates value for everyone it touches.
a
Not sure if you're the bottleneck? Find out in 2 minutes →







