Data Architecture concerns the holistic approach to data form as well as function. Now more than ever, businesses require clarity of purpose and contextual awareness of the information which supports decision making and competitive advantage.
Information is abundant. We are drowning in it. The skills of selectivity, relevance ranking and artful exclusion are essential to finding a meaningful direction that can enrich competitive advantage and serve other business goals. Consider by way of analogy, the medical industry. A diagnostician may be a clinician, a surgeon, or neither – yet the value of taking a global view of all information available, before proceeding on a treatment plan, is broadly understood. The same need for oversight exists for businesses managing multiple sources and high volumes of data. Corporates can accelerate their market positioning with the unique function embodied by a Data Architect.
Where engineers and scientists execute, build, map and construct – the architect composes and selects the minimal variables, inputs, materials and operations necessary to inform the outcome desired of the values-driven effort. So why only now, do we see companies begin to take data architecture seriously?
Business has a need for such a function in 2019 as information volume, complexity and interdependence for optimal outcomes, continues apace. We need to make sense of our resources before they overwhelm us. If we wait and become overwhelmed, businesses often default into a reactive stance, responding to signals and noise as they present themselves. That exposes a company to miss the quiet, but all important signals, or risk over focusing on the noise.
These businesses pay the price of not proactively designing a data environment that gives them control of the music.
Big Data borrows the term ‘architect’ from the design and construction industries, and with good reason. Where a building might be a composite of the bricks, mortar, windows and wiring you have at hand, it might also be the outcome of a use-purpose you envisage to meet the goals and aspirations of the inhabitants who will use, live and work in the space. Yes, you could proceed to build without much planning. You might use the same critical materials from your inventory with or without use-design foresight. But you will arrange those materials and use them more selectively with a functional outcome in mind from the get-go. You might also learn that some materials do not add any value, just weight! You will also notice that some materials are missing to achieve optimal use of the space, and can acquire them long before you would have noticed their absence in the traditional approach.
Several successful companies have flourished by parsing high volumes of business data to hone in on sales opportunities teased out through smart inference and the application of machine learning to targeted industrial data sets. Chorus.ai is one such company, driving sales growth for its customers by revealing insight to call centre audio data that reflects nuanced conversational patterns and anomalies, that in turn maps to sales rep behaviours. Competitive advantage soon follows for those who have drawn lessons from their own Big Data silo, which would otherwise remain untapped through generalised human analysis of those telephonic interactions.
In finance, where data has always been central to issues of compliance and regulatory standards, there remains a long way to go in the development of systems that bring the full value of hierarchical contextuality to life for so many businesses in the sector. Most of us our familiar with the need to preserve financial records for many years, for example, which leads to many companies defaulting to archival processes that silo data in ways that preference security over dynamic accessibility. This can rob a company of information that might otherwise offer a nuanced advantage to their business through the application of AI to previous trends and behaviours to help shape future decisions and reveal signals that lead to opportunity.
At VATBox for example, where tax regulation and corporate compliance inform the process of automated VAT & GST reclaims; we’ve begun pushing the boundaries of what we believe data can do when it is viewed, cross-referenced and selected meaningfully and minimally towards a values-based goal. For example, the capacity to explore travel trends over multiple periods and destinations can lead to powerful insights that allow our customers to better plan and negotiate their expense budgets, while also developing early warning flags for credit card fraud or other anomaly-based alerts to underpin their regulatory and tax compliance.
More than anything, the relational values afforded a company by a lead Data Architect, allow for non-linear outcomes that can give a company counter-intuitive direction in their decision making. As layers of data build upon one another, the relationships and the granularity of these data sets are as important as the sequence and timing of other operational processes in building a roadmap for success. The layers of data reveal meaning, which then motivates and animate our collective efforts. They become our narrative and reason for being as a living, breathing company.
Data well designed becomes our story worth telling.
Data Architecture leads this process, in order that data scientists, engineers and analysts can have ‘just enough’ material to hand, in a sequence that relates to relevant business goals in the short, medium and long term. In fact, tomorrow’s narrative weaves itself from the optimisation and intuitive meaning that surfaces from implementing this business philosophy, today.
Joe Hyams, MBA is the Manager of Behavioral Data Analytics at VATBox