In an earlier post "Data science skills for today", we explored the current state of skills and capabilities we expect from an individual operating in a Data Science role.
In this post, we explore several aspects of the Data Science skill-set that need further development in the coming years for individuals to grow and retain their relevance to stakeholders in the next ten years.
The future of Data Science
"Data is a previous think that will last longer than the systems themselves", Tim Berners Lee (the inventor of the World Wide Web
The goal of data science is and will remain focused on extracting business-focused insights from data within the organisation and the market it is servicing. Gartner presented the increasing value that can be derived in their Analytic Value Escalator graph:
"In the next 10 years, Data Science and Software will do more for the Medicines than all of the Biological Sciences together", Vinod Kholsa, Co-Founder of Sun Microsystems.
The future of Data Science is bright. Our dependency on cognitive intelligence and big data will grow. By all means, we can observe how the advancement in Automated Machine Learning (AML) is the key contributor to the progress of the data science remit mainly driven by the ongoing need to:
- Advance Personalisation of communications and engagements.
- Provide better search and recommendations.
- Enable access to actionable information using no-code techniques to stakeholders across the entire organisation.
- Support Quantum computing applications involved in heavy-research industries such as defence, healthcare and pharma etc.
AML acting as a catalyst to innovation is visible from how leading businesses such as Microsoft and Amazon put great efforts in making AML solutions and compute power readily accessible to the mass markets.
Tooling
Today, Data Scientists use a variety of languages and tools such as SQL, Hadoop, Python, R, Java, Hive, Tableau, PowerBI, DataRobot and TensorFlow to:
- Prepare structured and unstructured data for processing.
- Extract, align and clean.
- Statistically analyse the data to locate patterns or in preparation to input into a machine learning process.
While the capabilities and tooling of preference of Data Scientists will depend on the environment and teams they are operating within, we can quickly expect that these tools will continue to progress and advance in both simplicity and feature capabilities in leaps and bounds.
This advancement in tooling capability is already visible in the increasing popularity of no-code platforms and machine learning solutions such as Data Robot, Rapid Miner and Aleryx.
We are now also witnessing the rise of verticalised industry solutions such as Palantir Foundry for Crypto.
Moving forward
This continuous advancement in accessible and usable tooling and compute power will enable an increasingly broader audience to operate on the data, effectively lowering people's barriers to performing operations associated with data scientist roles.
The ease of access to knowledge and information makes the skill and know-how to implement systems transferrable to software development resources. You can easily search for and find tutorials on:
- X steps to implement a data science model
- X methods to optimise python code for data science
While this can be thought of as a negative, I see it as a positive. It supports the democratisation of capabilities across specialisations and people skills and enables people to focus on what they do best.
What does this mean?
The success of a Data Scientist is not defined by the tools and the complexity of code they can manage. Their success is determined by their ability to apply their knowledge to identify patterns that solve real-world problems. This focus on using knowledge also means that they need to become experts about the business practices, trends, issues, and opportunities of the industries they support.
This focus, in turn, allows role specialisations to evolve in line with the way that leverages the individual strengths and weaknesses of people's areas of focus:
- Data Scientists need to remain experts at working with the business stakeholders to identify real-world problems and develop investment cases aligned to business purposes.
- Product Managers work with stakeholders to create roadmaps of works that, when shipped, unlock the value desired.
- Development resources specialise in using the proper tooling to create and ship the required capabilities to quality, on time and to specification.
In Summary
In an earlier post "Data science skills for today", we explored the current state of skills and capabilities we expect from an individual operating in a Data Science role.
As the tools of the trade for handling technical aspects will become increasingly available, accessible, and usable by a broader group of technical people, our understanding of the field and expectations from the Data Scientist role will evolve.
The pressing hiring demand has shifted to problem solvers and critical thinkers who understand the business, the respective industry as well as its stakeholders. No longer will the ability to navigate a couple of software packages or regurgitate a few lines of code suffice, nor will a data science practitioner be defined by the ability to code.
Wrapping up
In this post, we explored how the explosion of accessible compute power and near-native ability to handle big data enabled us to apply data science techniques to address an unprecedented range of significant problems. This ability unleashed a new level of value realisation that organisations increasingly appreciate. As a result, we can easily see the demand for such capabilities to be exponentially higher in the future. This demand will require even higher levels of focus and true understanding of business circumstances and framing opportunities in business-speak to truly matter!.
With this in mind, the role of a data scientist will evolve to:
- Reflect on their area of expertise rather than the tools they use to implement it.
- Focus on business specialisation within the vertical industry(ies) they are supporting.
- Become a subject matter expert within the vertical industry, sensitive to identifying and articulating real-world problems and helping business leaders understand the realms of possibility in unlocking value and opportunity.
- Engage cross-functionally to understand individual circumstances such as those needed by compliance managers and operators and the business leadership needs.
- Work with the business to unlock investment prioritisations aligned to the business purpose.
- Demonstrate value and measure success by collaborating and reliably demonstrating the realised value of the workstreams shipped by the delivery teams.
