It’s all about Governance

My first job was building KPIs. Over 10 years later, I’m back to building KPIs. Looking back at how it was then, or digging even further as to how it was done when I was still a child, we’ll notice not much has changed. The principles are still the same: Put stuff in a database, create a model, plug a report on top of it and complain because its too slow.

What has changed, is the tech. We’ve seen all sorts of solutions, by more editors than I can remember. The tech has gotten faster, smarter and enabled new usage. I’ve played with two of them: Business Objects and SQL Server. Actually, three, as I also played with MS Access… and I should probably talk about it in another post as there is a lot to say about what it is meant for vs how it is usually used. Another day.

Business Objects seemed visionary back in the day in a sense that it could deliver real self-BI. Building the universe was a way to give business users a window to the data in a way they could exploit it. No SQL, no modeling, no tuning, just grabbing objects from the universe and building reports from them. At least, that is the theory. And that theory is the same I then found with PowerPivot when users could hit cubes or DBs and build their own reports. It’s now even more extreme with tools like PowerBI where everything is done in one app.

So, the tech allows us to do awesome stuff. But even in the early days, how were BO universe created? By people modeling the data, spending time to think about how it was going to be used, planning and documenting everything? Probably. But not where I worked, not where any of my friends worked. Actually, probably not. Objects were copies of underlying tables. I even saw universes with multiple instances of the same object. How is that going to turn out when its handed to users who expect things to be clean, efficient?
And what about the reports? Built by IT? Why bother with another level of complexity that is the BO Universe, generating SQL that usually can be perfected? Why bother with PowerBI when putting more work into making SSRS good-looking would have been cheaper for everybody? Clearly, the message was: End users need to be building their own reports. Business Analysts needs to do their thing, transforming complex business concepts into exploitable reports.

And this is where it all started to go downhill. Let’s liberate BI! Let’s liberate the data! Awesome! Let’s take a perfect world with perfect BO universes, perfect SSAS cubes. Data is clean, organizeed, documented and available. But what about the reports themselves? Where do they go? On a portal? Call it Infoview, Powerpivot on Sharepoint, PowerBi Report Server. Where isn’t it a mess? And I know I haven’t seen it all. Reports scheduled with nobody reading them. Multiple versions of the same one. I’ve seen portals that look like my graphic designer friend’s desktop when he builds logos: v1, v1_final, v1_fortheclient, v1_thisistheone, v1.1 … does v1.1 go after or before v1_thisistheone ?

So, we have more tools than I’m sure anybody can remember. We need to manage data access and organization. We need to organise how reports are built and accessed. And killed when obsolete. And it comes down to one notion I used to hate because it was all about writing documents I thought nobody would read: Governance. But I was right, nobody was going to read the documents I wrote. Because they weren’t meant to be read. They were meant for me to think aloud, to take the time to reflect on how to organise stuff. And hadn’t I done it, the proof of concept I did in 2010 on PowerPivot in Sharepoint that was a brand new concept at the time would have failed. And today I see people come up with great concepts, implementing what is the state of the art, but not considering their environment and the fact that it is going to break because one thing hasn’t been factored in: users. Yeah, those guys!

And looking back at it all, governance really is key. Tech can be migrated. Layer by layer, progressively. Its just tech and finding people wanting to play around with a tech migration is never an issue. But defining rules, setting habits for users, that usually sticks much faster that you would expect and last for much longer that you would hope. And it being for BI or Data, its all the same. You need to plan for how you will build your platform, give access to it and manage it over time. Imagine telling users they can do whatever they want and then realizing the impacts (unlike what a lot of cloud vendors say, compute isn’t cheap) and telling them you need put in restrictions once they’ve started to love your new tool. Ever took a toy away from a kid at Christmas? Well, the results will be similar. And users can be much more vocal than children in their own way.

So how to go about it? Well… plan. Plan. PLAN. You could hire consultants charging your monthly salary for a day’s work. But just thinking about what you can reasonably deliver, what costs you can afford and then imagining how your users will use your service is usually enough for a start. Here are the questions I ask myself:

• What business scope do we want to address?
• Where is the data? (if its not accessible, you can stop here. We sell magic but it doesn’t exist, remember?)
• Who owns it?
• Who knows its lifecycle?
• Who will be using it? How often? (no, they don’t need real-time, this is BI, not monitoring!)
• Who will build the reports? Who will own them?

You’ll probably come to the conclusion that your data is owned by a lot of people with no clear golden source, split over multiple applications and nobody has a clear vision of the lifecycle. Its O-K. You just need to work around that.

At this point, I hope you are convinced that you will need data governance to manage your sources. You’ll need report governance so that not everybody can publish anything anywhere. Let’s keep it civil people. And there are resources out there to help. I wish I had found them when I started.

I’m currently in a very Microsoft-friendly environment and working on SQL Server and PowerBI, I came across two contributions Microsoft made to the community that could really kick-start any organisation trying to implement BI governance:

The Power BI Adoption Framework

It could be applied to any other tool and it’ll make you ask yourself the right questions. Here is a look at all the interrogations it will raise.

Power BI Governance and deployment approaches

If you are deploying Power BI, read the 300 page document provided. It will answer almost everything and will prevent you from making the mistakes we ignorants all made. Its a mix of documentation, architecture, suggestions and as a bonus, links to tools that can boost the experience.

Conclusion:

We all probably agree that governance is essential but it is usually the first thing that goes out the window when we lack time or vision. Or both. It shouldn’t. Its the hardest thing to correct once you screw it up. So please, if you read my entire post, don’t mess up your BI governance anymore.

I’m aware I’ve only raised questions here. I’ll be giving my answers in the next blog post.