Data science talent is scarce, yet companies want to rely on analytics to stay competitive. I‘ve seen this dilemma firsthand through my decade of experience in the field. In this post, we‘ll explore how empowering employee "citizen data scientists" can help organizations extract maximum value from their data.
The Widening Gap Between Data Science Supply and Demand
The demand for data science skills has rapidly outpaced supply over the past decade. Just look at how data science job postings on LinkedIn have absolutely skyrocketed globally since 2012:
Data science job postings have risen over 650% since 2012. Source: McKinsey
And the situation is even more dire in the United States, where there are over 3 data science job openings for every qualified candidate as of 2020:
For every 1 qualified data science candidate in the US, there are 3.5 job openings. Source: QuantHub
While demand has exploded, the supply of formally trained data scientists simply hasn‘t kept pace. Top candidates can command over $120,000 per year on average in the US according to the Bureau of Labor Statistics. This crisis isn‘t going away anytime soon.
Domain Expertise is Crucial for Impactful Models
While data science is based on sophisticated quantitative skills, real-world business acumen is equally important. The most technically sound predictive models will fail if they don‘t account for organizational needs and processes.
This is why leading companies like LinkedIn, Airbnb, and CapitalOne have had success with citizen data science programs. By enabling non-technical employees to generate insights using smart tools, they benefit from these "amateurs‘" holistic view of the business.
For example, LinkedIn used citizen data science when developing their automated Lead Recommender system for sales leads. The model considers factors like member skills, company growth trends, and past sales success. This level of nuance would have been impossible without product managers collaborating closely with data teams.
Powerful New Tools Have Democratized Data Science
The rise of citizen data science has been enabled by rapid advances in analytics platforms over the past 5 years:
-
AutoML tools like Google Cloud AutoML and DataRobot automate complex, manual tasks like data cleaning, feature engineering, model selection and hyperparameter tuning. This allows non-experts to efficiently build and compare models.
-
Augmented analytics products from companies like Qlik, Tableau, and Microsoft embed ML directly into reports and dashboards. Users can get insights through natural language queries without coding.
-
Low-code solutions let citizen data scientists easily deploy models into production with drag-and-drop interfaces, instead of extensive manual programming. Mendix, Appian, and OutSystems are leaders in this space.
Democratizing the most tedious and technical aspects of data science has made it much more accessible to a wider range of employees within organizations.
4 Best Practices for Successful Implementations
While powerful tools have enabled citizen data science, organizations still need to employ best practices to ensure success:
1. Create Collaborative Workspaces
It‘s crucial to foster collaboration between citizen data scientists and more technical teams like data engineers. Using tools like Azure Machine Learning, DataBricks and ModelOps platforms allows them to iterate together.
2. Implement Training Programs
Many employees may not fully grasp concepts like statistical significance, data ethics, and algorithmic bias without proper training. Education helps them avoid critical mistakes.
3. Classify Datasets Appropriately
Make sure to classify datasets based on intended usage and restrict access where required. Not all data can be made available to every employee due to security and compliance risks.
4. Build Out Testing Sandboxes
Sandboxed environments that mirror production systems but contain synthetic data are invaluable for safely testing models before deployment. This avoids disruptions.
Turn Your Employees Into Data Heroes
While finding formally trained data scientists remains challenging, empowering citizen data scientists allows practically any organization to tap into the richness of their own internal data.
With the right tools, environment, and training, non-technical employees can generate tremendous value – while also upholding ethics and governance standards. The strategies above can help guide you through the process successfully.
Looking to implement a citizen data science program? Reach out for help structuring your approach.