All good data starts with business metadata. This business metadata is the information we need to build a dataset. There is someone in the business who approved the collection and processing of data in the first place. He/she also provides requirements and descriptions on what is needed. The challenge is that this information is often not managed good enough throughout time which leads business metadata quality to decrease. And thereby decrease good data. And that affects your AI solutions, BI reporting and integration with platforms.
When we know what business stakeholders want, we can design and implement this into physical form through technical metadata. We can now build the solution or buy it of the shelf and map it to the business metadata.
Now that we know what data we need, what it means and have a place to store and process data; we can start doing business. Doing business will generate operational metadata. Operational metadata is very valuable in monitoring our data processes. We get insights in what data is processed, how often, the speed and frequency. This is great input in analysing the performance of our IT landscape and see where improvements can be made. Further we monitor the access to systems and data. When we take it a step further we can even start analysing patterns and possibly spot odd behaviour as signals of threats to our data.
Social Media metadata
Finally we take the social metadata as an inspiration. This is where the value of your data becomes even more tangible. Value is determined by the benefit the user experiences. The way that he uses the data is then an indicator of value. Thus if we start measuring what data is used often by many users, this data must be important and valuable. Invest in improving the quality of thatdata to improve the value created. Behaviour is also a good indicator to measure. How much time is spent on content and which content is skipped quickly. Apparently that content doesn’t match up with what the user is looking for.
All metadata required to correctly control the data like retention, purpose, classifications and responsibilities.
– Data ownership & responsibilities
– Data retention
– Data sensitivity classifications
– Purpose limitations
All metadata that helps understand and use and find the data.
– Business terms, data descriptions, definitions and business tags
– Data quality and descriptions of (incidental) events to the data
– Business data models & bus. lineage
All metadata that allows for tracking authorisations on data.
– Metadata versioning & creation
– Access requests, approval & permissions
All metadata that relates to the structure of the data itself required to properly process it.
– Data types
– Data Models
– Design lineage
All metadata that is required for assurance of the storage & integrity of the data.
– Data storage characteristics
– Technical environment
All metadata that is necessary for exchanging data like API’s and Topics.
– Configurations & system names
– Data scheduling
All metadata generated and captured in execution of data processes.
– Data process statistics (record counts, start & end times, error logs, functions applied)
– Runtime lineage & ETL/ actions on data
All metadata that keeps track of the data processing performance & reliability.
– Data processing runtime, performance & exceptions
– storage usage
Controlling (logging) metadata
All metadata required for security monitoring & proof of operational compliance.
– Data access & frequency, audit logs
– Irregular access patterns
All metadata generated by users of data to
– User provided content
– User tags (groups)
– Ratings & reviews
All metadata that can be derived from observation to
– Time content viewed
– Number of users/ views/ likes/ shares