Master Data Management Blind Spots - Mar 07

Source: Blog: William McKnight

Forming and interacting with Master Data Management teams is interesting because the people tend to come from one of two worlds.  While each world is legitimate, the best team will comprise the best of both worlds, if not in experience, at least in perspective. 

These worlds I'm talking about are the post-operationally-minded information management professionals and the operational-systems-minded professionals.

On the one hand, MDM does comprise quite a bit of traditional information management - the data modeling, the data quality programs, the data profiling, the data integration - all familiar to builders of data warehousing. 

On the other hand, MDM is operational and will interact in real-time, not batch, with other operational systems.  MDM will utilize governed workflows to bring together its data and will need to be keenly aware of operational issues and operational data needs.  Changing the data so suit analytics is not always best.

While the perspectives are not mutually exclusive, the skill gap to what is needed can be astounding if teams are formed, and therefore opinions of MDM formed, exclusively from one camp or the other.  Make sure your MDM program achieves the right balance of skills and focus.  Take inventory.  This can be done through consulting and education, as well as conscious effort towards balance and achievement in both operational and post-operational issues.

Packaged Data Warehouses: Benefits and misconceptions - Feb 02

Source: Blog: William McKnight

Santa brought many a package this year - a packaged data warehouse.  There are a host of these offerings now available which represent the consultant dream of repackaging custom work to more customers.  Of course, the popular ones come from the large vendors who own the systems which originate the data that is challenging to warehouse.  For many, these approaches yields the following practical benefits:

• Enterprise Data Model using relatively acceptable best-practices and standards
• Views for the Business Model metadata
• ETL framework containing mappings/routines to load and refresh the data warehouse
• 100s of reports and dashboard content
• ETL routines have built-in Change Data Capture logic
• Open Interfaces to  other data sources
• Industry standard Metadata for Common Business Functions (i.e. AP, AR, GL, Sales Forecasting, Pipeline, etc.)
• Faster Time to some value
• Small team to deploy initially
• New features available via upgrades

It IS the fastest route to SOME value.  Is it the right value for you?  Is the time savings worth the extra cost?  Will it pay off in the long run?  It's not something to get into lightly.  Unfortunately, packaged data warehousing tends to come with many misconceptions that make it seem more attractive.  These are the most common:

1.      It's a total plug-and-play

2. Data modeling is not important
3. The client does not have to staff data warehouse expertise
4. The business does not have to be involved
5. Data quality is not important
6. Your individual enhancements will merge seamlessly with vendor enhancements in their releases
7. The vendor will continue to upgrade its solution in real-time with the corresponding source system
8. There are no other costs, i.e., Hardware costs, Employee costs, Consulting costs

Be sure to take a complete and multi-faceted view of this important decision.

Comparison of Enterprise Data Platforms - Jan 25

Source: Blog: William McKnight

I'm preparing my content for Enterprise Data World 2010, where I will speak on March 17.  My topic is "Comparison of Enterprise Data Platforms".  It's mostly based on my popular service helping get end clients into the right platforms for them.  I'm covering the DBMS market (where we've come, where we're going), data warehouse appliances, columnar data storage, open source, on-demand and virtualization.  In 1 hour! 

I'll do my best, but one thing that struck me as interesting as I reviewed was that the future of data management will have many opportunities as well as confusion in it.  Unlike about 5 years ago, the enterprise data warehouse was not only hitting stride, it was considered a "holy grail" and an end-game.  Many data management resources went into building data warehouses (with single platforms, strictly row-based structures, on standard hardware and in-house.)

This is not so true any more.  We've re-entered chaos.  A columnar orientation is working its way into enterprise architectures and popular database offerings (Microsoft, Oracle).  You need to know how it works and when to use it.  Appliances and on-demand services are pushing function out of the enterprise.  You need to know how to manage the changing roles and responsibilities.  With departmental budgets buying software-as-a-service business intelligence, low-end appliances, and open source BI tools, shops need to have integration capabilities now more than ever.  And, something I've been saying for a while, master data management is leading the charge of business intelligence back into the operational world.

The economies and performance abilities of data management are changing and the benefits need to go to your bottom line.  Most unhelpful is the terminology wars surrounding all of this.

And finally what to start doing about the MapReduce approach, and syndicated data, and the data cloud.... 

Sorry, no answers here, but sometimes it's helpful to identify the questions.

SQL Server 2008 R2 - 12/28/09

Source: Blog: William McKnight

There are several new features that will be in SQL Server 2008 R2, which is due to be GA in May.  As someone who implements on multiple platforms, I'm constantly comparing platform capabilities and consequently have a hard time getting too excited about releases, but R2 is giving me some reason to be excited.

Like many of SQL Server's R2 releases, it builds on its corresponding R1.  SQL Server 2008 been commercially available since mid 2008.  From a data warehousing perspective, SQL Server has long been a choice for data marts, regardless of the data warehouse platform.  It has also been a data warehouse platform for the midmarket and occasionally a Fortune 100 company.  Some of the scalability concerns that have limited SQL Server's reach may be being answered in R2 with the added support for 256 processors.  This is quite a move up from 64 processors.  Also improving scalability will be the acquired DataAllegro technology, rebranded Parallel Data Warehouse, Microsoft's data warehouse appliance.

What I am most interested in and excited about is Master Data Services, Microsoft's entry into the crowding master data management market.  Microsoft is the first to make it a part of the DBMS package.  They are clearly targeting the Microsoft shops that are having master data issues.  I've had the CTP for some time (this is a worked over form of the product from Stratature, which Microsoft acquired in 2007) and have been able to exercise it and even see it implemented at a client.  While its capabilities are limited compared to its more mature competition, it has a lot of potential. Microsoft will be putting strong development effort behind Master Data Services.  Even today, it can certainly, with some effort, play the technical role in a MDM program.

And then there's Gemini or PowerPivot for Excel. Gemini is the new generation of Analysis Services.  Those who chagrin at the notion that Microsoft Excel is the #1 business intelligence tool (it is) will have a lot more concern now.  Bye-bye ProClarity interface.  We must embrace Excel.  I'm increasingly crafting procedures for IT's role with Excel and this need will only increase as Gemini will cause an even more fluid spreadsheet environment in shops.  Data security strategies are imperative. 

Gemini will also be a collaborative environment.  Everyone in a workgroup can cooperate in managing Excel.  Excel is certainly already mission critical and R2 will create even more possibilities to depend on Excel.  The Gemini server is SharePoint, another element gaining traction in the SQL Server family.

I am also completely impressed with the addition of the columnar, in-memory storage option for this downloaded data, called VeriPaq.  Some data just belongs in columnar, though most would not want to put all their data in this structure.  It's great for data where a lot of columnar functions will be done to it, as well as generally for those queries that don't return a lot of columns.  It's also great for compressing data.  I have a seminar on columnar and expect to be helping more clients effectively tier their data to this format now that SQL Server will be utilizing the option.

So, are you ready?  Are you on SQL Server 2008?  Are you ready to upgrade to Office 2010 to take advantage of Gemini?  SQL Server 2008 R2 will be one of the big BI stories of 2010.

Evasive Pervasive Business Intelligence - 12/01/09

Source: Blog: William McKnight

If a little bit of something is good, more must be better.  This is true of some things - exercise, community service, patience, etc.  Well, it's true to a degree.  What about business intelligence?  Almost all businesses have the proverbial business intelligence user community now.  Though sometimes fragmented and informal, the IT organizations supporting these communities are planning to expand their reach and their community.  In most cases, this is completely warranted.  Most organizations are not at the point of diminishing returns with their user community rollout.

Remember the first Inmon definition of data warehousing? "a subject-oriented, integrated, non-volatile, time-variant collection of data, organized to support management needs."  It's a solid definition that has stood the test of time.  I know it's for data warehousing, not business intelligence, but you can see the mentality of the management user there and no mention of the rest of the organization.  Of course, BI has all kinds of users - people and systems - now.  And organizations are better off for it.

However, the so-called Pervasive BI movement goes beyond any of this.  It's "BI for everyone."  Unlike my thoughts on self-service BI, I am less optimistic about pervasive BI in the short term.  I'm all for smarter organizations, but not many workforces are structured such that everyone, or even most, employees have impressive key business decisions to make that incorporate the need for interaction with data well beyond their immediate environment.  I can think of a few exceptions - financial research firms, pharmaceutical research, etc. 

Probably the key thing to find palatable ground here is to define user.  Everyone in an organization can be the beneficiary of BI in the organization.  However, the tiered nature of the user community will remain true for the next decade.  Some will:

1.       Mine the data

2.       Interact with the data

3.       Consume and interact with reports

4.       Consume reports for decision making

5.       Make tactical decisions for others based on information seen in a report

6.       Make small adjustments to their team's workdays based on information seen in a report

7.       Make tactical decisions based on information seen in a report

8.       Make small adjustments to their workday based on information seen in a report

At some point in this hierarchy, the need for actual BI tools stop.  So, if you have a project that will entail a large number of new BI users - and they're going to productively benefit the business through the knowledge they gain from being a user - that is great.  If you are displaying corporate KPIs throughout the organization, I can see that.  If the culture that arises from this sharing, and possibly decomposing the KPIs to multiple levels, is empowering everyone to do their best job, that too is great.  But that's not BI for everyone.

Let your organization benefit from BI.  However, a project to blanket the business with BI tools in an untargeted fashion because it is thought that [pervasive BI is good, it means everyone needs BI and BI means tools] is not the best use of resources.