Machine Learning Beyond the Stars

Ryan Mcfarland
Principal, Chicago Office

The early days in machine learning (ML) are analogous to a sailor’s experience in the 14th century.  After generations of human history navigating celestially by the sun and stars, one day someone shows up with a compass. Can you imagine the range of emotions most sailors experienced upon seeing this innovation?  Distrust in the effectiveness of the new tool and a fear it could lead to fateful outcomes.  Intrigue in the power and inner workings of the new tool.  Hope that the next cloudy night might not mean a longer journey home.  Conflict with others who might not share their perspective on the compass moving forward.

A captain on a ship that just set his first course via compass had to maintain the confidence of his crew.  That captain had to prove the effectiveness of this new approach.  Stories of other ship’s navigational success helped, but nothing solidifies support more than personal experience.  That first course needed to be right in order to avoid giving the skeptics in the crew ammunition to revert to the proven approach of the sky.  Or worse, mutiny and disaster.

ML poses very similar challenges for companies seeking to innovate.  At Pariveda we’ve found there are any number of barriers for organizations to achieve their goals.  For some people, using technology to predict the future is inherently scary.  What happens if the prediction is wrong…or right?  What can or should we do about it?  Is it even possible?  Should we try and if so, how do we get started?  What should we look at first?  While all of these questions have merit and demand answers, the one question that’s rarely asked is most important.  How can our organization do this together?

The Modern Data Enterprise

Pariveda has partnered with many of our clients on their first journey into ML.  Those initial efforts explored use cases such as sales forecasting, threat detection and image classification.  The Modern Data Enterprise realizes sustained business-value outcomes when the organization effectively leverages new ML technology in the context of business activities that they are willing and able to change.  This recognition drove the creation of the Modern Data Enterprise Maturity Model shown below:

Organizations may reside at different levels of maturity for different pillars.  For example, they may be more mature on the Platform pillar and have a Data Lake used for some experimentation while the Value and Governance pillars lag in the Reactive and Uncontrolled level respectively.  In this example the holistic organization is considered at an Uncontrolled level, or the level of maturity associated with their weakest pillar.  Machine Learning and other analytic efforts will yield unpredictable and inconsistent results in this organization regardless of their advanced Platform maturity due to inconsistent data quality, lack of data contracts and other governance-oriented limitations.  The core precept of this maturity model is that the organization needs to advance within all three pillars to see sustained results.

Driving Organizational Improvement through the Modern Data Enterprise

Many of our clients have difficulty realizing the benefits of a Modern Data Enterprise due to disconnects among business, IT and the lack of data governance and stewardship.  Organizations may lack a clear vision for ML, something that must be defined in order to drive any impactful outcomes.  They may have engaged on building out their data lake with all their data, rather than the subset of data needed for a business-driven purpose.  Or they may have relegated management and governance of data to IT rather than identifying business experts better equipped to steward data oriented towards business goals.  Some organizations may have urgently prioritized time-to-market solutions that cut corners on the technical platform, failed to surface success indicators based on experimentation results and bypassed governance effectively sabotaging maturity across the board.   

These represent some of the common mistakes we see across the Value, Platform and Governance pillars of the Modern Data Enterprise.   Pariveda’s approach ensures that business and IT stakeholders work together with a focus on end-to-end processes and cooperation in order to realize business value.  We do so by having awareness of how one pillar impacts another and tactics that rapidly mature weaker areas.  By prioritizing holistic maturity versus celebrating achievements in a single area we ensure the delivery of tangible value versus perceived progress.   Ultimately, we are setting a course that many people need to follow in order to arrive to a planned event at a specific destination.

Celestial navigation was replaced with the compass.  The compass was largely replaced with GPS.  Decades of operational reporting, manual analysis and decision-making processes are being replaced by ML solutions.  Soon, the ML solutions we are utilizing today will be replaced by more sophisticated AI solutions.  The Modern Data Enterprise recognizes the key challenge now is not what solutions we should attempt first or even how we should get started.  The key challenge the Modern Data Enterprise approach addresses is how we do this together.