Blog - 17 May 2022
8 minute read

Does your company have the right technology platform for data transformation?

Blog

Does your company have the right technology platform for a successful data transformation? At first glance, this seems like an innocent enough question. Surely if you have a modern data platform comprising best of breed technologies, then the answer is a resounding ‘yes’.

data transformation, digital transformation, technology platform, data technologies, data engineers, data capability, data engineering, data governance

True

True

False

2022-05-16T23:00:00Z

Does your company have the right technology platform for data transformation?

In an evolving technology landscape, data transformation is a complex undertaking. Principal Consultant Jon Stafford puts tech in its place, highlighting the importance of cross-discipline working and the value of data citizenship.

article

Data & AI

Data conversation 3 image

//images.ctfassets.net/k26sw1bgepr3/6QeZOKPdhGpp44LOJpyvQn/6ffe24655aa06f0f07f6fb973259506d/iStock-1284284200__1_.jpg

Does your company have the right technology platform for a successful data transformation? At first glance, this seems like an innocent enough question. Surely if you have a modern data platform comprising best of breed technologies, then the answer is a resounding ‘yes’.

In actual fact the answer isn’t quite that simple. Technology is just a tool after all, and for any tool to provide value, you need the skills to use it, and it needs to be right tool for the job (which means that you also have to know what that job is). Having a DEWALT DW729KN does not make someone a master carpenter, and conversely, an experienced carpenter with the right tools and skills still needs to know what to make before they can create something valuable.

It’s the same story with data transformation – before you even start thinking about what technology platform you need, you must make sure that you understand the job that platform has to do. Ideally, you need a data strategy which first sets out the roadmap for how data can help achieve your goals and objectives, and then describes the role that technology needs to play. At a minimum, you need a set of business goals and objectives as your starting point. In any event, you should avoid driving your data transformation journey from a pure technology angle – there are too many examples of organisations who have implemented excellent data platforms without a clear view of the role the data needs to play. The result is a delayed – or worse, a failed – data transformation journey, with no value to show for the effort put in.

Once you have worked out what you want to achieve with data and why you want to achieve it, you are finally ready to start thinking about technology. But how do you pick the right platform to make sure you succeed? In the third of Ascent’s Data Conversation webinar series, I spoke to Alex James, Ascent’s Chief Technology Officer, to look at the key do’s and don’ts in this area, and signpost the path to success.

Where to start: those all-important initial decisions.

The initial decisions you need to make revolve around the key points in the data lifecycle. What technologies are you going to use to acquire data, transform and store it, analyse it and finally serve up information and insight derived from that analysis?

In the last few years, there has been a move towards cloud-based data technologies (think Azure, AWS and others) offering increasingly similar capabilities through commoditised ‘as a service’ solutions. So, deciding which platform to use is no longer about finding the one that can do what you need - usually all of them can.

Instead, you need to focus on your existing technology ecosystem, and look at the potential points of integration with the data platform, like extracting data from key operational systems.

Once you have this big picture view of the data platform and its role, you can then start deciding on the approach to take at each integration point, asking questions like:

  • How can we make best use of the technology we already have and so minimise spend on anything new? How can we make sure everything will fit together – legacy and new? Should we integrate with, replace or modernise components in the existing ecosystem?

  • If we have multiple sources for the same data, why is this? Can we combine or remove any sources? Can we get rid of any operational silos?

  • How can we shorten the path to value? What initial objectives should we focus on, and what slice of the data platform do we need to implement for data to be able to help achieve them?

  • Are we set up to develop our technology platform in an agile, iterative way, ensuring regular delivery of value and ROI and making sure we don’t spend a lot of time and money building the wrong thing?

Focusing on integration instead of capability in this way helps make sure that the key technology decisions that you make are more likely to be the right ones (in other words, that you are picking the right tools for the job).

How to start: moving from decision to action.

Once you’ve decided on the technologies you’re going to use, how do you turn those decisions into reality?

Historically you would probably have spun up a team of software or platform engineers to implement the technology platform, then let your data engineers loose on the platform to start moving data through it from source to target. Each team would have their own distinct jobs to do, with project management oversight making sure that everything came together successfully.

This siloed approach to developing technology platforms and data capabilities sounds reasonable on paper, but in practice it has several drawbacks. Each team only feels responsible for its own part of the whole. The relationship between the teams is that of customer and supplier, which naturally leads to a non-collaborative way of working. Even worse, it’s all too easy for one team to blame the other when things inevitably don’t work out right the first time. Siloing teams in this way turns out to be one of the biggest reasons why the technology aspects of data transformation fail to deliver the value they should.

The trend now is to have a single team to develop technology platforms and data capabilities comprising the complete set of roles needed to successfully implement the technologies and then push data through them to deliver information, insight and value. These teams typically include software engineers, data engineers, data analysts and scientists, and business analysts.

Having all these different disciplines working together towards a common goal, with each person understanding how their role fits into the bigger picture, solves the drawbacks of the siloed approach. Everyone understands why they are doing what they are doing, everyone feels a responsibility for the success of the whole solution, and a collaborative way of working becomes the natural approach.

How to avoid a governance nightmare along the way: encouraging data citizenship.

If you get the technology platform right, then you open up a world of data capability that enables everyone in your organisation to more easily use data and insights to achieve their goals and objectives. Low code, no code, self-serve and data democratisation all play their part in removing the historical barriers to data use.

But there is a cost. More data capability leads to more data, and more data requires more data governance. As organisations improve their technology platforms, they move from having data they can’t govern (for example, information in a locally produced spreadsheet shared through email) to data they can govern (for example, information in reports and dashboards shared online). Ultimately this is beneficial, of course, as the end state is a set of governed information and insight resulting in more confident decisions based on trusted data.

Reaching this end state for data governance is challenging though – there is no magic bullet to take you there, and technology is only part of the answer. Technology can help you identify what you need to govern (for example, by generating a list of dashboards and who has access to them), but it can’t (yet!) tell you how to govern it (for example, which dashboards duplicate information, or whether the people who can access them are the right people).

To fill in this gap, you need to look at process and, more importantly, people.

The first step is establishing a data governance framework, comprising policies and procedures together with a team who can help ensure that these are followed. This takes care of the process part of the equation, but to get the people part right starts with recognising that human nature is to take the path of least resistance when it comes to data governance, and that enforcing policies and procedures is probably the least effective way of establishing the governance that’s needed.

That’s where the idea of data citizenship comes in. Show people where they fit into the organisation’s data landscape, how they rely on others upstream to provide them with the right data of the right quality, and how what they do with data in turn affects others downstream. In other words, show people what it means to be a good data citizen, how it benefits them and why it’s important to others. This completes the data governance equation, and helps ensure success by working with human nature instead of against it (encouraging rather than enforcing).

Bringing everyone along for the ride.

Finally, having the right technology platform able to serve up the right data to the right people in the right way is still only part of the story. To achieve value, people need to be actively engaged with, and using, the technology and the insight it provides.

Remember that not everyone will be excited by the in-depth details of how everything works. As we said at the outset, technology is just a tool, and too much focus on the technology aspects of your data platform risks building apathy towards it, which is far from ideal.

Instead, focus on how the technology benefits people - by making sure they know how to use it, and what to use it for. Once people see how data and insight helps them achieve objectives and goals, they’ll buy in to the platform and ultimately become champions and advocates for it. Word of mouth anecdotal evidence will start to augment official communication and messaging, leading to a virtuous circle that guarantees that the technology platform delivers the intended value from the data it provides.

So, do you have the right technology platform to succeed?

To close, let’s revisit the question we started with: does your company have the right technology platform for a successful digital and data transformation?

If you’ve followed the above recommendations then you’ll have the right technology implemented in the right way. The data in the platform will be governed effectively, and the technology will be actively and consistently used by people in your organisation to provide them with tangible value.

And the answer will be a resounding ‘yes’!

Jon Stafford

Principal Data Consultant

Ascent

Jon helps businesses drive more value from their data by ensuring they use the right information, in the right way, for the right reasons.

With over 16 years’ experience in data strategy and business intelligence, Jon has worked with organisations of all sizes to better align their data approach and capabilities with business goals and objectives.

Jon believes that data and technology should always makes peoples’ lives easier, and that this philosophy should be at the heart of every organisation’s data vision.

genericSection

[Think tank] Let’s get started - Get In Touch

Let’s get started.

We help customers build game-changing products, deliver pivotal data and software projects and build strong internal teams. Got a challenge in mind? We’re ready when you are.

Get In Touch

False