When I started Mango Solutions back in 2002 (long before we became part of the Ascent family) the wider world of data and analytics was mostly reactive, with workflows conducted by individuals who produced reports as ‘one time’ outputs.
While data professionals wrote code, it could largely be considered a by-product of what they did. The advent of data science , together with the increasing need for just-in-time intelligence, has driven more proactive analytic workflows underpinned by open source technologies such as Python.
Working at the forefront of data science, Mango understands the vital role of technology to allow data to be transformed into wisdom in a repeatable way and deployed to business users at the right time to support informed decision making. There is a clear learning here for modern technology initiatives:
“Every data project is a software project, and every software project is a data project.”
To realise business value, it is vital that we balance both data and software elements of technical projects around a common and clear purpose.
Every data project is a software project.
Back in 2012, Josh Wills described a data scientist as someone who is “better at statistics than any software engineer and better at software engineering than any statistician”. While modern data science incorporates a broader range of analytic approaches than statistical modelling alone, Josh’s description of data science at the intersection of analytics and software engineering still holds today.
The changing role of data and analytics from a reactive practice to a strategic approach has driven the need for advanced analytics to be combined effectively with software engineering. If analytics is now an always-on capability, we need to codify the intelligence in systems that can be properly deployed and scaled within a business.
A ‘local’ alternative is just not practical – you can’t become a true data-driven business if analytics is run by experts on their laptops. We can’t stop making intelligent decisions if a data scientist is on leave. If a consumer purchases a product on Amazon, they will not wait hours or days until a statistician crunches the data to come up with other recommended products.
To positively impact a business with data, an end-to-end analytic workflow needs to be implemented using software engineering approaches. This encompasses everything from the creation of data pipelines, the deployment of models, and the creation of user interfaces and applications that can convey insight in the right way, linked directly to operational systems to action and process outcomes.
Every software project is a data project.
Increasingly digitalisation and regulation have driven more focus on requirements regarding the role of data in software systems. We can consider 3 types of requirement regarding the treatment of data:
User – requirements relating to users and preferences to provide a more personalised experience
Governance – requirements relating to the way in which data is managed in a secure fashion to confirm with data regulations and protect confidential data
Provenance – requirements relating to historical system actions to provide an audit trail, or to enable rollout back to, or understanding of, previous actions
Beyond this, the most important consideration in the design of modern systems is the ability to leverage advances in data and analytics to create richer, more useful experiences and applications. A growing understanding of the possibilities offered by analytics allows us to strive to ask better questions – to build software tools that are truly aligned to a users’ objectives.
For example, imagine we are building a software application to be used by call centre staff when speaking with customers. Traditionally, we may have built a system that combined data from various sources to give the user a single view of the customer. Perhaps this included data on previous orders, previous interactions, demographic data etc.
With data science, we could extend the functionality for the user – perhaps to include an understanding of likely customer churn linked to suggested retention actions, or a suggested ‘next best offer’ for the customer, or suggestions around the ways in which to talk to the user. Perhaps when the customer calls the call centre they can be allocated to exactly the right person to talk to, as opposed to being randomly allocated to the next available agent.
The use of data and analytics in software can have a transformative effect on the quality and usefulness of our software systems.
Helping customers build capabilities at the intersection of data and software is the most effective way to unlock value in an increasingly digital economy. Technology businesses like ours who want to be part of that customer journey need to be ambidextrous in their approach to data and software, agile in their execution and above all empathetic to each customer’s unique context.