Three metrics help to assess the value of an OBI-Object: quantitative, qualitative, and the ‘So-What’ question. These are supplemented by the overall ‘game-rules’.

AGOV’s efforts to reduce its fleet of OBI reports involved six phases. This is the third: Measuring Mount OBI.

1. Understanding Mount OBI
   • Death of Mount OBI [Optional blog, history and context of OBI]
   • Tagging [Optional blog, inventorying an Organization’s Reporting Fleet]
2. Prepare the Environment and Data [TECHNOLOGY]
3. Measuring Mount OBI: How Will the Objects be Evaluated? [PROCESS, this blog]
4. Initiate Communications and Change Management [PEOPLE]
5. Conduct Increasingly Focused Passes with sign off [PROCESS]
6. Integrate into an overarching Data Governance Framework [GOVERNANCE]
7. Note: the above blogs can be downloaded as a single PDF: AGOV Tackles Mount OBI.

Establishing trust is critical before asking business owners to apply more detailed metrics:

• Building Trust Before Climbing Mount OBI
• The Game Rules
• Quantitative Measurements
• But Is It Any Good – Qualifying
• The ‘So What’ Measure

Building Trust Before Climbing Mount OBI

As discussed in the first blog, organizations are hoarders at heart – and for good reason. Poor consultation leaves a long lasting and bitter taste in the mouth of users. The only effective cleanse to this taste is a good scrubbing of trust. Trust is built upon the following ‘Game Rules’:

1. AGOV-Clients ‘Own’ OBI: Although this project was run by the IT Department, the OBI-Objects are fundamentally business assets. This means that no decision would be made without consultation.
2. Ownership Has Responsibilities: The flip side to ‘owning’ and object is to be accountable for it: providing the respective quantitative and qualitative measures discussed below.
3. IT Owns, Business Contributes to the Process: The overall OBI review process is owned by the IT Department as it bears the cost of maintaining thousands of OBI-Objects. Nevertheless, the process to review and potentially delete OBI Objects is defined in consultation with business representatives.
4. Senior Support, Communicate Until it Hurts: The final rule is to start with senior level support. AGOV did this by having executives nominate business representatives (call OBI-Champions, discussed more in the next blog). The project laid out the review process, walked through it with clear communications to the organization and executives.

With the ground rules defined, it is time to answer the questions of ‘How’ to climb Mount OBI and ‘How’ to mark the trail as we go. In our metaphorical climbing kit, we have three types of rules or measures: Quantitative, Qualitative, and the So-What question.

Quantitative Measurements

These metrics help to quantify Mount OBI. In most cases they are obvious and straightforward to define with a word of caution. There may be an inclination on the part of business to want all the upside of accessing reports with none of the downsides of accountability. IT should be the Department or ‘Owner’ of last resort.

Reports may also end up being passed back and forth between potential departments. AGOV had a few of these reports and tracking each one of these discussions through a QDIRF Project Log helped.

1. Department: Which Area ‘owns’ this Dashboard? This is typically the area where the Dashboard Owner works. This value is populated first to narrow down ‘Owner’ or afterwards.
2. Owner: What position and/or person is accountable for this OBI-Object.
   a. Person: a real-live staff member, ideally a middle manager or more senior position.
   b. Role: the position, department, contact details, etc. the position of the above person.
   c. Definition: The Owner is accountable for the OBI Object (Dashboard, Page, Report/Analysis).
   • This person approves all changes to the Object. In doing so, they must take into consideration the overall organizational risks and benefits of doing so.
   • While the OBI Owner is often the Data Steward as well, they need not be.
   • However, in their capacity as an OBI-Owner, they are a Data Custodian.
   • These roles are ideally defined by an organization in its Data Governance Framework or use industry standards [1].
3. Consumers: While an owner is singular, consumers are numerous and categorized as follows:
   • a. Responsible: While they don’t own the report (there can only be one owner), this group of individuals or position rely on it.
     • They will be consulted about changes and have a veto on them as well. If there is no owner, one of the responsible may be nominated to take on this role.
     • Information collected should include the person, department(s), contact details, etc.
     • Seven or fewer is the sweet spot for the number of individuals responsible.
     • Remember, with great responsibility comes great power (to veto changes).
   • b. Consulted: This group should be asked before about changes to an OBI-Report. Minor changes don’t usually require asking but de-commissioning or significant changes requires more consultation.
   • c. Informed: (Optional), consumers who use this report; if de-commissioned or changed, they may be informed of such changes by those accountable or responsible.
4. Short Description: a 255-character limited summary of the report’s business purpose.
   • This is the elevator pitch and is captured for all levels of the OBI-Report.
   • This description is also the ‘canary in the coal mine’ when it comes to the value of the object.
   • The more difficult it is to explain why a report is needed, the less value it probably adds.
5. Long Description (Optional): Additional details can be provided in this field but only AFTER the short description has been filled in. Consider including in this section a few sentences on what happens if this report ceases to exist.
6. IT Owner: Parallel to the business owner, who is the IT owner.
   • If the IT shop has adopted a BRM model, then it may be one of these.
   • The IT owner should be who the business Owner or Consumer calls for changes or problems.
   • There is a sweet spot between the CIO and the help desk analyst for this role; senior enough to make things happen but close enough to the business to have the time and context.
7. Data Steward: Although listed as an attribute to collect, data governance questions were not collected by AGOV. This was not a function of their importance but instead the organizational maturity in implementing a usable governance framework. Definitely an attribute to revisit in future passes!

But Is It Any Good – Qualifying

The above measures are quantitative; they deal with the facts of an OBI-Report. This measure asks, is the report meeting the expectations of the consumer? A five-point scale is used:

• -1 Negative Value: the report is/can be wrong, untimely, or reduces the certainty on the part of a user. The user would be better off not having this report.
• 0 Neutral Value: the report neither contributes nor diminishes certainty on the part of the user. The user would neither gain nor lose if this report ceased to exist.
• 1 Marginal Value: this report adds some value to the user who often must correlate and confirm the results with other sources. There would be loss if the report ceased to exist to the user, but the information can be obtained through other means.
• 2 Good Value: this report adds value although it could be improved. The user seldom must use other sources to trust or augment the report.
• 3 Excellent Value: this report is a gold standard is (nearly) always trusted without external confirmation. No significant improvements are required.
• Explanation: notes on why the above value has been selected.

For AGOV, only individuals nominated as the owner were allowed to provide a score. Ideally, they did this after consulting with other users.

The ‘So What’ Measure

What is next for an OBI-Report. Actions range from de-commissioning to preserving it. For this project, a ‘Were Not Sure’ value was permitted. This was a reprieve from de-commissioning – for now.

Only the Department representative or the OBI-Owner could assign the ‘So What’ status. This was typically done after reviewing usage reports (prior blog) and consulting within their area.

• -1-Delete. This OBI-Object can be immediately removed.
• 00-Pending: This OBI-Object is yet to be evaluated; this is the default value and can only be assigned by the IT Department.
• 01-Reprieve. A temporary reprieve while more information is gathered.
• 02-Preserve. This OBI-Report will remain on the respective page/dashboard until at least the next review cycle.
• 03-Remediate. Some OBI structures required remediation. Although out of scope for the review project, this was a handy reminder for both business and IT.
• Comment/Explanation. Required if the Object for all values but ‘Preserve’. However, if an Object was rated providing poor value but was recommended for preservation, an explanation was required.

Other Metrics

The above metrics focused on values provided by the business in respect to an OBI-Object (Dashboard, Page, Object). Other metrics were discussed in the prior blog, Preparing to Climb Mount OBI, such as its location or usage. One key metric not discussed was the cost of ownership for the OBI-Object.

For most OBI-Objects, after their development, their cost is free. For a few Dashboards, they are constantly under review and updating. Typically, this is not related to the OBI-Objects but instead the underlying data (e.g. poor underlying data architecture). The cost of ownership, IT and business time spent on an object may be a useful measure.

Using the Metrics

AGOV was able to complete the above metrics. Because AGOV had not systematically reviewed its OBI-Fleet prior to this, there were several blank fields such as descriptions, etc. For the initial pass, AGOV accepted a less complete review. Ideally, metrics would be reviewed yearly as part of the ERP planning process. Further discussion on this in subsequent blogs.

Results, Notes and References

• The use of negative values was something I considered in the blog: Thinking Positive About a Negative Score.

This blog series is about a public sector organization ‘AGOV’; a fictional mash up of recent experience, past knowledge, and research. I like to blog to remember, and a good memory device is to tell a story. A story always works better with sympathetic characters, such as AGOV.