Whether you're dealing with a data warehouse or data lake architecture, data governance is about balancing opposing forces. Hortonworks' Gnau keyed in on a similar point. It's about building auditability and access control into the data platform to make sure data discovery and analytics are transparent, whether you're a business manager looking at financial data sets or a data scientist working with raw upstream data.' 'From a data governance standpoint, you can have different stages of the data that exist within a system that can be snapshotted so you can return at any point in time in the pipeline. What's the data you have, who has access to it, and how are you managing the lineage of that data over time?' said Norris. 'When you start scaling the variety and velocity of the Big Data we're dealing with, you've got to have data governance but it's in this broader context. MapR's Norris explained why businesses need to look at data governance from a higher level and focus on the larger data pipeline at play. It's not an easy balance to strike, particularly when the reality of where and how data is housed and processed is constantly in flux. What Exactly Is Data Governance and Why Do We Need It?Governance means making sure enterprise data is authorized, organized, and permissioned in a database with as few errors as possible, while maintaining both privacy and security. They discussed how to tackle the complex challenge of ensuring data governance within the complex data architectures and organizational hierarchies of a large enterprise. Scott Gnau, Chief Technology Officer at Hortonworks, and Jack Norris, Senior Vice President of Data and Applications at MapR, each explained what data governance means to their organizations. I talked to Hortonworks and MapR, two of the biggest enterprise Hadoop vendors in the market. But, at its core, data governance is about data trust and accountability, married with comprehensive data security best practices.

The processes by which you ensure governance within an enterprise differ depending upon who you talk to. It starts with simply understanding what Big Data is, what it isn't (hint: not a crystal ball), and how to manage data storage, organization, permissions, and security within your enterprise data architecture. Though, before you can employ BI tools or run predictive analytics on a data set, there are a host of factors to square away.

Pcmag best data visualization tools