Metrics and KPIs for Information Groups – Atlan

Examples of Quantification from the Atlan Group

Survey outcomes from Vendr’s SaaS Tendencies Report (Q3 2023) affirm what many information groups have already skilled: It’s getting loads tougher to spend money on new applied sciences and approaches. Purchases are down 37% year-over-year, and scrutiny over the associated fee and advantage of investments imply that greater than half the time information groups spend contemplating the precise know-how is spent on negotiation, alone.

With growing scrutiny on the price of present know-how and groups, and better requirements for investing in new applied sciences and assets, information groups are sometimes requested to speak the worth of their packages and proposed initiatives. The place technical advantages have been as soon as persuasive sufficient to shut gaps and inefficiencies in course of and know-how, information groups should deal with extra planes of worth, and start to quantify the operational advantages, and enterprise worth, of their work.

Information leaders all through the Atlan Group have already begun this significant change, transferring past funding arguments that embody modernity, scalability, velocity, and consumer expertise. Within the under examples, you’ll see meticulous measurement of the progress of their packages, the operational advantages yielded by information groups and customers, and the enterprise worth their information helped to unlock.

Discovering, Measuring, and Speaking Worth

The frequent thread of all worth measurement is baseline evaluation. Whether or not information groups are beginning an enrichment program, automating a key workflow, or delivering new capabilities or information to their customers, taking a second to grasp the “earlier than state” makes it potential to measure your impression as your work progresses, then completes.

Within the Atlan Group, we see three ranges of worth and related Metrics and KPIs that information groups can and must be measuring:

  1. Program Worth – Holding information groups and stakeholders to account on the tempo, completeness, and high quality of their work.
  2. Operational Worth – Understanding the operational advantages and quantified efficiencies of latest packages, processes, and applied sciences.
  3. Enterprise Worth – Shifting past the “information as a service” mannequin, and into certainly one of partnership with the enterprise, understanding the impression of information and analytics outputs on enterprise initiatives.

1: Program Worth

The primary stage of measurement for information groups is that of their packages. From understanding the satisfaction of their finish customers to monitoring stakeholder contributions, there are quite a few “low-hanging fruit” metrics that must be recorded and communicated to stakeholders and leaders.

Outlined by progress in opposition to your group’s objectives, and suggestions out of your finish customers, information groups ought to pay cautious consideration to the relative enhancements they make by way of new applied sciences and processes, persistently communicate with and survey information customers, and set measurable objectives for his or her stakeholders to decide to.

Monitoring Stakeholder Contribution

Swapfiets, the world’s first bicycle subscription service, launched their Information Governance program by providing a self-service, fashionable information catalog on Atlan. Key to their success is a daily collaborating and reporting cadence, with six cross-functional Information Governance Committees composed of information analysts, information homeowners, and information stewards throughout their enterprise domains.

In that month-to-month assembly, we deal with three governance subjects, that are possession, documentation, and high quality. For documentation and possession, Atlan is admittedly helpful. It’s additionally helpful for high quality to seek out out the place within the information lifecycle high quality points come up. Throughout the assembly, we write out motion factors and decide to duties like verifying extra phrases.”

Lisa Smits, Information & Analytics Crew Supervisor

Swapfiets now maintains 125 outlined metrics throughout these six enterprise domains, and with month-to-month committee conferences, has created a mechanism for accountability and attribution, and a platform for usually reporting their progress towards enrichment objectives.

Quantifying Expertise & Satisfaction

Utilizing experiential or NPS surveys, or a proxy metric like Slack messages, tickets, or escalations, are all probably efficient methods to quantify consumer satisfaction. Essential to speaking a transparent before-and-after is accumulating a baseline measurement previous to releasing new tasks or experiences. 

For instance, measuring the variety of information questions a Information Engineering group receives previous to the discharge of a glossary, then three months into its adoption, paints a transparent image of fixing consumer behaviors that may be simply communicated with stakeholders. 

Chargebee, a market-leading resolution for recurring income administration, makes use of Atlan to create reusable information merchandise, and to fulfill a rising inside urge for food for information. Earlier than the introduction of self-service, Chargebee’s Information Engineering group was liable for responding to requests for information. Additionally liable for delivering information to Chargebee’s prospects, their Information Engineers usually needed to de-prioritize inside requests.

This was an enormous problem. Inner information requests have been pushed to the again of the queues as prospects have been at all times a precedence. This meant that we weren’t assembly SLAs and there was unhappiness amongst our stakeholders and plenty of escalation, which led to unhappiness throughout the group as effectively.”

Lloyd Lamington, Enterprise Options Supervisor, Chargebee

Previous to implementing Atlan, Lloyd and his group fastidiously scoped their “earlier than state”, logging the variety of information requests obtained per quarter, what number of of them have been repetitive or requested for uncooked information, and the variety of escalations when SLAs have been missed on account of de-prioritization.

With a baseline established, Chargebee’s information group have been capable of quantify vital enhancements after launching Atlan to assist self-service, and that zero escalation requests have been obtained for the reason that rollout.

2: Operational Worth

Past monitoring consumer satisfaction and stakeholder contribution, vital alternative exists for information groups to save lots of price, and drive efficiencies in the best way that they, and their end-users, function.

Essential to correct measurement, and a compelling narrative to speak to management and stakeholders, is establishing a baseline measurement, or “earlier than state” to check enhancements in course of and know-how in opposition to. Instruments for establishing these baseline measurements could embody:

  1. Monetary Evaluation – Preserving a radical file of the price of consumer licenses, compute, and information storage.
  2. Effort Scoping – Figuring out the accountable group members, and the trouble, price, and time required of competing approaches or applied sciences.
  3. Information Crew Surveys – Figuring out how a lot time is spent on servicing requests, repetitive duties, and diagnosing breakages.
  4. Information Client Surveys – Figuring out how a lot time is spent on information discovery, or how lengthy their requests take from submission to closure.
  5. Analyzing Techniques – Calculating common mission completion instances or the amount of tickets in instruments like Jira, or measuring the common variety of Slack messages and the time their underlying points take to finish.

With a baseline established, any enhancements to groups, processes, and know-how can and must be measured in opposition to it, then communicated to stakeholders and management.

Measuring Value Financial savings

Acquainted to many information groups are potential price saving alternatives, equivalent to optimizing information warehouse belongings or deprecating unused dashboards, eradicating unutilized information pipelines, or optimizing costly queries.

Mistertemp, a pacesetter in recruitment and non permanent staffing based mostly in France, adopted Atlan to enhance the navigability and usefulness of their newly carried out fashionable information stack.

Previous to implementing Atlan, Mistertemp’s coverage of connecting with supply information, then figuring out what was helpful after the actual fact, was resulting in vital technical debt, and lowered the navigability of their information property.

All these fast choices created loads of belongings in Snowflake that mainly with no enterprise use have been by no means actually touched or by no means actually documented or by no means actually linked to our BI software or another software. So they simply stayed there being synchronized, costing us cash.”

David Milosevic, Head of Information and Analytics, Mistertemp

Utilizing automated lineage and recognition metrics, David and his group decided that of their 30,000 information belongings in Snowflake, simply one-third have been used within the previous 12 months. These belongings have been then deprecated, considerably bettering the navigability of their information property, and decreasing pointless price and energy to keep up them.

Operational Worth for Information Groups

Measuring Time-to-value for Technical Implementations

There are myriad choices for information groups implementing new applied sciences and tasks, from hand-coding, to open-source instruments, to an array of SaaS options. However key to quantifying the worth of those efforts is cautious evaluation of the trouble concerned in every of those options.

Within the case of changing or modernizing present applied sciences, information groups ought to take care to stock the technical and experiential shortcomings of their legacy know-how, and punctiliously scope the trouble expended on their earlier method, drawing a transparent comparability between new and outdated.

Porto, a Brazil-based insurance coverage and banking chief with 13 million shoppers, sought to exchange a legacy information catalog to maximise the ROI of their information stack, and enhance information literacy throughout their group.

We used to have a knowledge catalog. We had Informatica EDC. The motivation to search for a unique resolution got here from some difficulties in implementing sure options like lineage, a enterprise glossary. It was form of irritating, and it got here to a degree the place we felt just like the know-how was form of blocking us from reaching the place we needed to go.”

Danrlei Alves, Senior Information Governance Analyst, Porto

Selecting Atlan to exchange their legacy catalog, Porto’s group rapidly built-in key information sources, then used customized metadata fields as a migration goal for his or her present documentation, metadata, and relationships between tables, columns, schemas, and databases.

In simply six weeks, Porto’s information group launched their new catalog, with content material at parity with their legacy resolution. However by drawing on expertise with their earlier resolution, and maintaining a selected stock of the enhancements they desired, Porto’s group have been in a position to attract a stark comparability between the 2 options, reaching in six weeks what as soon as took them greater than 2 years, speaking vital enhancements in time-to-value.

Within the case of one-time tasks, information groups should spend time estimating the trouble concerned in every proposed method, inclusive of the variety of group members to allocate, time spent for every of those group members throughout the period of the mission, and the estimated completion date.

Tide, a UK-based digital financial institution with almost 500,000 small enterprise prospects, sought to enhance their compliance with GDPR’s Proper to Erasure. Guaranteeing all private information was deleted upon request first meant aligning on what constituted private information, then figuring out the place that information lived and the way it moved by way of their structure. With a complicated information property, the handbook effort concerned was intensive.

Individuals must go into the databases and attempt to translate my record of private information components. There have been 31 components to seek out in our databases, and we’ve got greater than 100 schemas, every with between 10 to twenty tables. So it could be loads of work to determine it. If we have been very diligent and did it for each schema, then it could in all probability be half a day for every schema. So half a day, 100 instances.”

Michal Szymanski, Information Governance Supervisor, Tide

Utilizing rules-based bulk automation in Atlan, the Tide group was capable of determine, tag, and safe this private information in mere hours, a powerful feat in and of itself. However by first scoping the baseline handbook effort wanted to finish the method, Tide’s information group have been capable of report almost 50 days of effort saved.

Measuring Information Service Effort

For a lot of information practitioners, a continuing back-and-forth answering easy questions from information customers is a big drag on productiveness. Although this downside is simple to grasp, information groups ought to take care to measure the impression these requests have on their productiveness. 

By surveying group members to grasp the period of time they spend on these questions, and by together with a measurable metric like Slack messages obtained or tickets processed, information groups can create a transparent baseline of effort, in opposition to which they will quantify productiveness positive factors.

Octane, a speciality lender providing financing for powersports automobiles, makes use of Atlan to doc their information belongings, then make them obtainable by way of self-service. Previous to introducing Atlan, questions on Octane’s information have been answered in a shared Slack channel, the place their Information Engineering group was liable for responding. 

Aware of the potential effectivity advantages of introducing self-service with Atlan, Octane’s Information Product Supervisor, Alex Bendix, analyzed the period of time their group spent on these questions.

Every engineer was answering questions 10 to twenty% of their time. And then you definitely multiply that over a sure variety of workers throughout the group, that’s a whole lot of hours monthly that you simply’re dropping by way of productiveness.”

Alex Bendix, Information Product Supervisor

By meticulously quantifying the period of time their group spent answering questions on information, and measuring the variety of Slack messages the group was receiving after implementing Atlan, Alex and his group have been capable of report that they have been now saving 200 hours of Information Engineering effort monthly.

Measuring Efficiencies by Eliminating Repetitive Duties

Much like measuring the trouble expended answering questions, information groups also needs to take care to measure the trouble they expend on recurring obligations, setting a baseline of effort to measure productiveness positive factors from options equivalent to automation.

Past their documented success accelerating implementation time, Porto’s Information Governance group has additionally quantified recurring financial savings from automating beforehand handbook duties.

Danrlei and his group began by figuring out repetitive work, together with assigning homeowners, documenting belongings, and securing private information, then created a framework to make sure that solely the info belongings that demanded their private consideration have been surfaced to the Information Governance group.

If we take into account every little thing we’re doing now with Atlan in comparison with earlier than we had Atlan, we’re saving 40% in effectivity, by way of time and costly operational duties for every little thing associated to governance. This can be a 40% discount of 5 individuals’s time. We’re utilizing the time financial savings to deal with optimizing our processes and upleveling the kind of work we’re doing.”

Danrlei Alves, Senior Information Governance Analyst, Porto

With these automations operating in opposition to a knowledge property of over 1 million belongings, Danrlei and his group have been rapidly capable of determine the lowered quantity of governance duties they have been now liable for, then report this vital enchancment.

Measuring Time-to-resolution

Lastly, productiveness positive factors from accelerating “fireplace drill” duties like Root Trigger Evaluation, or tough investigatory duties like Impression Evaluation, additionally profit from a transparent accounting of effectivity positive factors. Fortunately, with these workstreams usually managed on instruments like Jira, measuring reductions in time-to-resolution could be so simple as analyzing effort and time spent on these duties earlier than and after enhancements.

Takealot, a South African eCommerce and Retail chief, makes use of Atlan to enhance technical understanding of their information property, and drive enterprise self-service. Previous to Atlan, every time Takealot’s information group have been knowledgeable of a bug, they might conduct conventional dash planning, figuring out the time wanted to research and resolve the problem.

By analyzing these decision workstreams, their group decided they have been spending half of their time investigating the issue and figuring out the placement of a breakage, setting a baseline metric to enhance in opposition to. Then, by using automated column-level lineage on Atlan, the Takealot group might extra rapidly examine the foundation trigger of information points, slicing their problem decision time in half.

As a substitute of trawling by way of all of the code, you’ll be able to rapidly observe lineage backwards and test it at each level to see what’s occurring. Earlier than, it might take per week or two weeks relying on how tough a bug was to handle, with 50% of that point being investigating what the issue was and the place it’s damaged earlier than truly making use of the repair and getting it into manufacturing. I’d say we’ve in all probability halved that point.

Group BI Supervisor, Takealot

Operational Worth for Information Shoppers

Measuring Friction in Information Discovery

Whether or not points with navigability, strict entry controls, or restricted context, there are myriad causes information customers have a tough time discovering and understanding the info obtainable to them. With a purpose to perceive the place to make enhancements, and to measure the good thing about these enhancements, information groups ought to analysis the trouble their end-users expend on these actions.

Nasdaq, liable for their eponymous inventory trade in New York Metropolis, and an array of know-how and monetary service merchandise, makes use of Atlan to evangelize their information technique and enhance their governance follow.

Key to their implementation course of was time spent surveying Nasdaq’s information customers, driving higher understanding of how the info group might enhance their technique of discovering and making use of information.

Practically 100 customers responded to the survey, with 75% of respondents reporting spending time attempting to grasp information. Respondents who spent six hours or extra hours pers week on information, spent two of these hours attempting to grasp the context round what they already had entry to.

Take into consideration that! A 3rd of their time each week is spent simply attempting to grasp what’s there. Think about if we might deliver a product in that helps scale back that effort and actually allows them to get proper to the guts of the issue — to drive information merchandise from insights into the enterprise. And that’s what we’re attempting to get to.”

Michael Weiss, Senior Product Supervisor, Nasdaq

With this survey information in-hand, and a mechanism for bettering the invention expertise for his or her information customers, Michael and his group have a “North Star” metric to measure their success, probably representing a whole lot of 1000’s of {dollars} in financial savings in information client effort, alone.

Measuring Time to Perception

For information groups which have a service-driven relationship with their information customers and prospects, baseline measurement of the time processes take to finish, like constructing dashboards or finishing queries, makes it easy to speak the impact of course of enhancements and new applied sciences.

Infillion, a Digital Media Platform serving the Fortune 50 and extra, makes use of Atlan to sew collectively a posh information property, and supply data-driven insights to their prospects

An important a part of their Enterprise Intelligence group’s obligations are supporting account groups, whose prospects count on frequent, correct reporting on advertising marketing campaign efficiency. Beforehand processed manually, these requests would take as many as three weeks to finish, with one week of ready for Information Engineering bandwidth, and one other two to supply a report.

Utilizing Atlan Scheduled Queries, Daniel’s group now spends half-hour or much less responding to requests for information, eliminating the necessity for scarce Information Engineering assets, and reporting a 3-week time financial savings for his or her enterprise companions and prospects on every request.

It’s allowed us to completely automate reporting with none engineering assets and turn out to be self-sufficient, and it’s positively eliminated an enormous bottleneck in our course of. A 3-week ready interval is simply not scalable. So this positively reduce that all the way down to a half an hour, possibly much less, to arrange.

Daniel Chon, Director of Enterprise Intelligence, Infillion

Measuring Effort

Lastly, on a extra refined stage, an in depth stock of the duties information customers undertake, mixed with an evaluation of the trouble expended and common personnel price for every of those duties, is a labor-intensive, however highly effective software for measuring the precise impression of a knowledge group’s efforts.

Zip, a digital monetary service firm and buy-now-pay-later pioneer, makes use of Atlan to assist their information customers rapidly, reliably, and simply discover, perceive, and use the capabilities of their fashionable information stack.

Leroy Kahn, Zip’s Information Administration Lead, developed a mannequin to estimate the enterprise worth yielded from every exercise his colleagues would carry out on Atlan, and a course of to trace the amount of those actions as a part of their enterprise case for a contemporary information catalog.

By way of consumer surveys, and the alternatives to enhance that they inventoried throughout duties involving discovering, sharing, understanding, and utilizing Zip’s information, Leroy decided a median period of time saved for every exercise or occasion sort for Atlan customers. Based mostly on common wage, every minute of time saved was price $1 (AUD) in effectivity positive factors, which might then be multiplied by the point saved per occasion in Atlan.

By analyzing utilization metrics, Leroy and his group have been capable of account for every exercise carried out on Atlan throughout a 30-day interval, evaluating it in opposition to the time and greenback financial savings, projecting $390,000 AUD in effectivity positive factors.

3: Enterprise Worth

Whereas measuring the efficiency of information group packages, and punctiliously recording the worth of operational enhancements is vital, there are a restricted variety of alternatives to make these enhancements. With a purpose to proceed speaking the worth of their work, information groups should try to grasp what information is used for, and to what impact. This necessitates transferring past a mannequin the place requests are serviced, and towards a mannequin of partnership with enterprise customers.

Whereas information groups may need grown accustomed to constructing new capabilities or servicing requests which can be then picked up by an analyst or data-savvy enterprise consumer, transferring past this paradigm means understanding the aim of those requests, and following up on their impression.

Information groups can start placing this into follow by recording a enterprise aim every time a request is made. From a easy stock of which groups or colleagues made these requests, to extra complicated information of tasks like decreasing buyer churn or figuring out cross-sell or up-sell alternatives, information groups can start evaluating their work in opposition to achievement of enterprise goals, and following up for particular anecdotes of what impression these insights made.

A quick-growing pet care firm is among the many finest examples of enterprise worth measurement within the Atlan Group. After migrating to a brand new telecommunications system, their name middle group was underperforming, resulting in diminished buyer engagement, and lack of income.

Whereas information was obtainable from their new system, its format was distinctive to telecommunications methods, dubbed a Name Element File (CDR), making it tough for his or her Information Analysts to research the issue and suggest an answer.

I’ve labored in conditions like this earlier than, and if we didn’t have Atlan it could have taken six months. We’d be answering questions like ‘What do you imply by occasion code?’ or ‘What’s the distinction between extension quantity and regroup quantity?’.”

Information Governance Lead

Utilizing Atlan, a member of their group with Telecommunications expertise labored to narrate Snowflake belongings with CDR information, and translated, then documented what every of the info factors meant. Now capable of perceive the context round this information in a self-service method, their Information Analysts rapidly carried out their evaluation in PowerBI, and recognized three core issues for under-performance.

Taking an Operational Worth lens to this achievement, their information group might report that they have been capable of assist this evaluation in 30 days with Atlan, quite than 6 months. However by sustaining a detailed relationship with their counterparts within the enterprise, and by maintaining knowledgeable about how information evaluation drove a decision, their information group can now report that they supported a ten% enhance in bookings by way of their name middle, a big enhance in income, and a mission-critical achievement.

Whereas fastidious reporting on Program Worth and Operational Worth are essential, information groups who construct robust relationships with their enterprise counterparts, and have the curiosity and persistence to grasp how their work is used downstream, can tie their work to a very powerful issues their organizations are dealing with.

Key Takeaways

Throughout Program, Operational, and Enterprise worth, the very best examples of quantification within the Atlan Group have quite a few key commonalities.

  1. Curiosity about, and dedication to, the worth of their groups, packages, processes, and know-how.
  2. Constant assortment of quantitative and qualitative information previous to funding or modifications to processes or know-how.
  3. A mechanism for capturing modifications in habits, sentiment, or effort, persistently measured in opposition to a baseline and communicated to stakeholders and leaders.

Picture by Patrick Perkins on Unsplash

Related Articles


Please enter your comment!
Please enter your name here

Latest Articles