Artificial intelligence (AI) adoption is accelerating across industries, as businesses seek to leverage technologies like machine learning to derive data-driven insights, automate processes and improve decision making. However, while AI holds immense promise, successfully implementing it comes with significant challenges. In fact, according to Gartner, 85% of AI projects risk failure by 2022.
To avoid failure and maximize value, organizations must navigate the following key AI challenges:
1. Data Quality and Availability
High-quality, relevant data is the lifeblood of effective AI. As an expert in web scraping and data extraction, I‘ve seen firsthand how flawed data leads to faulty AI outcomes. Without sufficient useful data to train on, algorithms won‘t function properly. Unfortunately, acquiring comprehensive, clean data at scale remains difficult for most companies.
Common data challenges include:
-
Insufficient training data – Machine learning models need vast amounts of quality examples to learn from. Collecting or purchasing these large datasets carries high costs in time, labor and money. I‘ve seen clients struggle to obtain the volumes of labeled data needed for accurate computer vision and NLP models.
-
Poor data quality – If data is incomplete, duplicative, inaccurate or outdated, it will lead to poor model performance. In fact, inaccurate data costs US businesses $3.1 trillion per year according to IBM. Cleansing and preparing data requires substantial human effort and technology.
-
Data silos – Relevant data often resides in isolated internal silos across teams and incompatible systems. Consolidating it into a usable form is cumbersome and expensive without modern data pipelines. At one client, we found Salesforce and service databases had conflicting customer data that led to incorrect churn predictions.
-
Legal/privacy restrictions – Regulations like GDPR place necessary limits on how personal data can be used, hindering access to potentially useful data. Navigating data privacy compliance adds complexity for data scientists.
"We spent over 12 months just aggregating and cleansing data across regional databases before we could even begin training account scoring models. Data engineering took up 80% of the total AI project timeline." – Banking VP of AI from experience
Solutions:
-
Build high-quality training datasets – Allocate resources for data collection, labeling, cleansing, consolidation and compliance. Leverage techniques like crowdsourcing.
-
Use transfer learning – Leverage pretrained models that require less organization-specific data for fine tuning.
-
Adopt data governance – Develop frameworks for sourcing, managing and sharing data. Track data lineages.
-
Anonymize data – Remove personally identifiable information to enable broader usage per privacy regulations.
-
Start small, scale up – Test models on smaller samples, then expand scope as capabilities improve.
2. Algorithm Bias
Since AI systems learn from data, they naturally risk perpetuating any human biases or discrimination present in that data, leading to unfair and unethical decisions if unchecked. Models can also become black boxes lacking accountability.
I‘ve seen examples like:
-
Hiring algorithms that disadvantaged women after training on historically male-dominated resume data.
-
Loan approval models that deny certain ethnic groups due to biased historical lending data.
-
Compas algorithm that was more likely to falsely flag black defendants as higher risk based on racial correlations in the data.
"We found ourselves in an AI ethics conundrum the first time we deployed an ML recommendation model. The algorithm quickly became a black box reinforcing biases in our user data." – Leading Streaming Service Engineer
Solutions:
-
Perform bias testing using A/B testing with diverse sample groups. Check for unfair impacts across gender, race, etc.
-
Use techniques like adversarial debiasing and differential privacy to protect sensitive attributes.
-
Increase transparency through explainability methods like LIME that produce reports summarizing model logic.
-
Establish human-in-the-loop review before executing model decisions when confidence scores are low.
-
Appoint oversight teams to regularly audit algorithms for issues like data or algorithmic bias.
3. Integration with Existing Systems
To be useful at scale, AI systems must integrate smoothly with existing business solutions, workflows and data infrastructure. This integration is often extremely challenging.
Common integration hurdles include:
-
API limitations of legacy systems that restrict connectivity.
-
Data bottlenecks from pipelines unable to feed real-time data to predictive models.
-
Deployment friction from lack of containers or orchestration to ship models to production.
-
Process misalignment when predictions fail to align with downstream workflows.
"The vast majority of AI projects fail from integration challenges across complex legacy IT systems, not because the core technology doesn‘t work." – McKinsey Partner Viktor Schoner
Solutions:
-
Modernize incrementally – Replace limiting legacy systems piecemeal if full modernization is infeasible.
-
Build adaptable APIs – Develop APIs and microservices to better connect AI components to core systems.
-
Engineer seamless data pipelines – Ensure timely flow of quality data between systems. Add middleware to ease integration.
-
Containerize models – Use containers like Docker to encapsulate models and dependencies for smooth deployment.
-
Incorporate human review – Enable human-in-the-loop monitoring of model decisions to check alignment with business processes before retraining.
4. Talent Shortage
The surging corporate demand for AI has sparked a severe global talent shortage across roles like researchers, data scientists, ML engineers and DevOps.
Key talent challenges include:
-
Education gaps – Universities are not producing enough graduates with the ideal blend of AI software engineering and statistical/math skills. Masters and PhD talent is limited.
-
Deep expertise required – Mastery of both complex math and multifaceted software tooling is difficult to attain and in short supply. Most have only partial subsets of the required skills.
-
Competition from tech giants – Companies like Google pay astronomical sums for top of market talent, luring them away.
"There are fewer than 10,000 people globally with the full skillsets necessary to deliver meaningful AI across the pipeline." – Carnegie Mellon AI Professor Dr. Oren Etzioni
Solutions:
-
Reskill employees – Invest in internal education programs to reskill employees on AI tools and techniques. Build training data science masters programs.
-
Acquire startups – Obtain coveted experienced AI talent and prebuilt models by acquiring promising emerging startups.
-
Offer remote work – Expand talent pool geography by offering remote work options.
-
Provide incentives – Offer equity, research budgets and impactful work to attract and retain talent.
-
Partnerships – Partner with universities to shape curricula and source new graduates. Fund research.
5. Interpretability and Explainability
Many advanced AI techniques behave like "black boxes", making their internal logic hard to interpret. This lack of model explainability creates trust issues for users and challenges for auditors.
I‘ve seen problematic examples like:
-
Fraud AIs flagging legitimate user transactions without justification, hurting customer experience.
-
Recruiting algorithms inconsistent candidate rankings without transparency into weighting.
-
Chatbots giving nonsensical responses based on confusions within their natural language models.
"Any AI system that substantially impacts consumers should provide explanations regarding its significant recommendations or decisions." – Timnit Gebru, Lead of Ethical AI at Google
Solutions:
-
Implement inherently interpretable models like decision trees when possible.
-
Build explainability into complex model architectures directly.
-
Add explainability wrappers like LIME to complex models to attempt explaining their logic after the fact.
-
Log model confidence scores and flag predictions with low certainty for human review.
-
Select and deploy human-in-the-loop review for high-risk model categories with low explainability.
6. Cybersecurity Risks
Like any technology, AI can carry cybersecurity risks if not thoughtfully safeguarded. Attackers could steal AI data or algorithms, manipulate training data to degrade models, or evade AI defenses through adversarial techniques.
Recent examples include:
-
Fraudsters using synthesized audio to trick voice recognition and authentication AIs.
-
Adversarial sample images designed to fool computer vision classifiers at test time.
-
Stolen Tesla Autopilot source code which later leaked online.
"Applying AI introduces new attack surfaces and vulnerabilities that cyber defenders must stay ahead of through AI security best practices." – Microsoft CTO Tim O‘Brien
Solutions:
-
Employ AI itself to detect adversarial content and activity patterns.
-
Perform continuous security penetration testing using simulated attacks.
-
Anonymize and encrypt sensitive training data sets.
-
Compartmentalize access controls around AI decision systems.
-
Build algorithms to be robust and resistant to data and model manipulation.
-
Continuously patch training pipelines and models as new threats emerge.
7. Legal and Ethical Concerns
As AI takes on greater autonomous roles across finance, healthcare, transportation, and more, thorny legal and ethical debates arise around liability, privacy, bias, transparency and job loss. The lack of regulatory standards creates uncertainty.
Key concerns include:
-
Liability attribution if unpredictable AI systems cause harm.
-
Data privacy as more personal data is collected, analyzed and shared.
-
Bias and discrimination challenges as discussed previously.
-
Job losses from automation and its destabilizing impact on individuals and society.
"Until updated regulations address context-specific uses of AI, we are navigating a grey zone regarding the technology‘s legal, ethical and societal downsides." – Professor Effy Vayena, Swiss Federal Institute of Technology
Solutions:
-
Closely track emerging localized regulations applicable to AI systems under development. Seek legal counsel.
-
Proactively apply rigorous ethical principles and oversight processes to algorithm design, training data and application.
-
Pursue diverse internal and external input to identify potential pitfalls early – don‘t silo AI development.
-
Implement strong explainability measures to demonstrate due diligence across development and use.
-
Limit autonomous automation in high-risk scenarios until uncertainties are addressed through law and ethics bodies.
While AI promises immense opportunity, thoughtfully navigating its multifaceted challenges remains critical to project success and beneficial adoption. By taking a holistic view spanning data, algorithms, integration, talent, transparency, security and ethics, businesses can overcome pitfalls and unlock AI‘s full potential for their organizations and customers.
To learn more about successfully implementing AI, please contact me any time. I would be happy to discuss your organization‘s unique AI opportunities and challenges. My team of seasoned AI practitioners can help assess your AI readiness, create a tailored AI strategy, help select optimal solutions and vendors, and avoid the many pitfalls.