Continuous improvement requires continuous intelligence

Business leaders must take the initiative to leverage their data using new technologies and approaches to adapt and succeed in the digital world.

The digital age has presented businesses with a significant challenge: adaptation. Organizations can only hope to survive in this new era if they are able to adapt to the new reality of doing business.

For the past few years, adaptation efforts have fallen under the umbrella of digital transformation. It is now widely understood that organizations must engage the groundswell of digital data and refine it into a byproduct that can inform decisions or instantaneous actions. However, because digital data flows continuously, the data engagement model should also be continuous, leveraging advances in machine learning, AI, IoT, and analytics. This sort of continuity will catalyze organizations to adapt and thrive in the new digital reality.

Continuous intelligence

Because IT has historically focused on batch processing, the concept of continuous processing is fairly new to most organizations. Continuous intelligence waits for nothing. Not data collection periods, not resource availability, not processing time. It is the non-stop generation of insight and actions based on operational data stores as well as streams of data and events generated in the moment. It is the ability to harness an ever-changing environment where the data is constantly flowing and the insights and actions are perishable.

According to Gartner, success can only be achieved in a world that is constantly changing by implementing a continuous approach. Gartner suggests that continuous intelligence is at the heart of fast-paced digital business and process optimization. However, continuous intelligence is not only about IT architectures. Successful implementation requires a change in managerial approach as well.

New leadership approach

Conway’s Law gave us the insight that system designs reflect the communication structures of the organizations that design them. Because designing continuous intelligence requires new architectures, it is critical that the organizations designing them reflect the architectural intent.

Most organization structures today assume they are performing in a batch-processing world. One team works to complete a task before handing it off to the next team, there is no continuity of visibility or activity. Initiating a continuous intelligence effort with the limitations inherent in a batch processing management model will produce feet but no wings.

To fully implement the continuous intelligence approach, business leaders need to adapt to agile management methodologies. Just as the DevOps world engages continuous integration across teams, so must the larger IT organization engage in a more active and constant way. The rate of engagement is necessarily radically higher, that is the only way for the broader team to understand what’s going on in the organization. This approach will facilitate initial success and be the foundation for staying ahead in an era of new, dynamic technologies and continuous change.

The need for speed

One of the fundamental changes to the IT stack required for continuous intelligence is a new data processing layer designed to perform at extremely low levels of latency. Regardless of whether the data already exists in operational data stores or arrives in event-based streams, the concept of continuity is at odds with latency.

Our traditional systems of record do not have this design point, nor should we expect them to. They will continue to do their job well while a new, complementary data processing layer is added.

Innovations in IoT, machine learning, and AI assume both constancy and immediacy. Business value has become inextricably linked with real-time action. New applications require speed and scalability in the underlying data processing to produce responses as well as to 'feed the beast' to inform models. The money is in the microseconds, whether the data is at rest or in motion.

Digitization has permanently changed the business landscape. Continuous intelligence is achievable. Business leaders must take the initiative to leverage their data through new technologies and approaches to adapt and succeed in the digital world.

Author: Kelly Herrell

Source: TDWI

Measuring data maturity: How data-driven is your business?

With the number of data-driven businesses in decline and the number of legacy businesses being disrupted on the up, we wanted to create a deeper understanding of what it actually means to be data-driven and what it would take for businesses to improve their data maturity.

The outcome of this journey of discovery is a data maturity curve that benchmarks entire businesses and provides a clear roadmap for where organisations need to advance across seven dimensions to drive their competitive advantage.

Data-driven businesses in decline

The number of data-driven businesses has declined in the last two years, according to a NewVantagePartners study. Only 31% of businesses say they are data-driven and they claim that 95% of challenges that stop them from adopting data-led practices stem from cultural challenges, such as agility and organisational alignment.

In most businesses, data is still largely managed and governed by finance and IT. In a survey assessing which departments in business manage data for decision making, only 24% of marketing and 26% of sales teams were directly involved while finance, IT or business intelligence (or a combination of the three) were always involved. This is a clear indicator that data is still siloed, not valued as an asset, and that a data-driven culture is not embedded throughout the organisation.

A tale of two companies

Let me give you two examples:

Company A has recognised the tremendous asset that is data, given that they collect over 1 million customer data points every week. The senior management team have conducted interviews with middle managers in the business which has made it apparent that there’s a distinct lack of analytical skills to derive meaningful insight from data. In addition, the business is missing the right data infrastructure to make data available more broadly. Company A decides to build a big data platform, operated by a data centre that’s available to all teams. What the exec team failed to realise is that they have just built a new data silo. Adding new technology and capability does not automatically mean that new value will be unlocked, unless there is a shift in the behaviours of the workforce. People need to be upskilled and processes need to change so the expertise bought in starts to be shared and creates a ripple effect.

Company B has been obsessing over customers and how they can serve them better since day 1. They have a multitude of insights platforms, all available to everyone in the business. They recognise data should be helping them make decisions every day. The senior management team recognise the need for agility and have embraced new ways of working. They drive all middle leaders to run new experiments weekly. Company B has built their operating model and processes so they can work as agile, cross-functional teams and make small, iterative changes every week. Data and continuous learning through data are baked into their processes. It’s daily routine and muscle memory. The decision on technology infrastructure investments is made by company B to support data-driven operations, not to force them.

The AI advantage

The crucial two differences between company A and company B are in the wealth of customer data company A possesses and the data culture company B has been able to build. Now imagine company A embarks on a journey to build that same data-driven muscle memory as company B.

Katie King, author of AI in Marketing, pointed out at a recent gathering of digital marketing leaders that large corporations, those we so often view as struggling against young, agile, tech businesses, are sitting on a pot of gold. And that’s the wealth of data they have collected over dozens of years. If these businesses were able to capitalise on data, their competitive advantage would be significant.

Data readiness and maturity

Much has been written about data readiness, and it boils down to two components: 

• The tools to generate accurate and clean data
• Building habits and behaviours that drive continuous learning

Measuring and evaluating a business’ habits, behaviours, and tools and benchmarking these against best in class is what we call data maturity.

You can’t improve what you can’t measure. As we saw in the example of company A, many businesses try to improve their ability to capitalise on data without fully understanding their current strengths and weaknesses in this area. How a business operates and how effectively and quickly it is able to make everyday decisions that affect performance consists of a combination of its processes, the skills available, and how well it can tap into those skills. Or in other words, for us to measure data maturity, we have to measure an entire ecosystem. This is a holistic exercise, meaning everyone’s skills, perceptions and working practices matter: from the exec level all the way to where the rubber meets the road.

The 7 dimensions 

Colin Smith and Jonathan Brech (both partners at CambridgeData) started to methodically record the challenges their customers were facing along with the questions and barriers that would come up repeatedly in internal workshops, no matter the size of business, how advanced their tech was or how data-driven they felt they already were. Initially, they had over 15 dimensions but found there to be overlap, which have been simplified into 7 dimensions: 

Culture: How strongly embedded are the processes and how advanced is the organisational ability and agility to learn from insights?

Mindset / Discipline: How disciplined are teams to follow set processes and how advanced is their mindset in believing how much of a difference small iterations are able to make?  

Leadership: Is there shared understanding amongst all levels of leadership about how a data-driven business should operate and is the business able to answer most business-critical questions?

Technology: Is the right technology in place making data and insights available across the organisation, while maintaining a single source of truth?

Collaboration: How is collaboration across different departments structured and to what degree can each function’s strengths and abilities be harnessed at pace?

Organisational structure: 
To what degree does the organisational structure enable cross-functional collaboration and fast decision making? 

Agile ways of working: Are test-and-learn methodologies used just within the tech function or are other departments running experiments at high frequency as well to achieve small improvements continuously?

It’s important to notice that these dimensions are all related and do not work in isolation. Each dimension displays tools, habits, and behaviours through which the data maturity is measured.

For example:

• You may believe that change happens iteratively with incremental gains (habit).
• You achieve this by querying known and accessible data sources (behaviour).
• You have a dashboard that pulls in the relevant traffic for you to query (tools).

You’ll realise that a change in habit, behaviour, or tool triggers a different outcome and conveys a different meaning in how quickly and accurately you’re able to leverage data.

Author: Jonathan Brech

Source: Oracle