Can We Predict the Future?

Mike Strange
Office Managing Vice President, Los Angeles Office

Well, maybe.  Machine Learning is powerful, and can be used as a predictive tool in select cases.  But many misunderstandings exist, and lessons to be learned before you start down this the seemingly magical Machine Learning path.

As with any new tool or technique, expectations of the value and precision run amok, unless explained. We have implemented ML a number of times, using several different tools, often with promising results, but always complicated by misunderstandings and preconceived notions that challenge attempts to foster consensus.

Machine Learning (ML) tools and techniques have many possible uses, including detecting patterns that will likely repeat themselves. This is a powerful concept, grounded in statistical methods that reinforce its relevance, and numerous practical commercial uses exist.  Recent releases of cloud-based tools make ML even more approachable.

Let’s explore a few misconceptions that we have noted in the last year or so.

1. ML is magic that few understand.

On the surface, many will imagine that ML is a magical utility, using some modern form of pixie dust to crunch huge volumes of data using methods that only Ph.D. scientists could understand.  In the end, they can actually predict future events. Not true. To be fair, we (the IT community) have not done everything we can to dispel the myth, perhaps clinging to the romantic notion that we will finally find a way to over-please our business leaders.  Instead, if we decide to invest in ML, we need to explain the concepts, input, limitations and expected value. As with many elements of IT, our success will be measured, in part, by the expectations we set.

2. ML is accurate, first time, every time.

This is another flavor of misaligned expectations. ML programs are often iterative- growing over time through consistent research and incremental improvement. We have been part of several proof-of-concept projects where the definition of success was the accuracy of pinpoint predictions. As we all explore ML, we should reinforce that long-term value will come from an organizational commitment to continuous learning – much like successful analytics programs.  The payback will be huge once all the executives align their thinking - that ML starts with research and grows through iteration.

3. ML is an algorithm.

Many imagine that ML is a transaction processing utility, an algorithmic version of a huge spreadsheet, into which facts are entered, and direct answers are provided. This leads to the incorrect conclusion that individual events can be “pumped into” an ML system and specific recommendations for improvement are provided. In fact, ML is often most accurate, and valuable in a business setting, when data is analyzed in aggregate rather than in specific. For example, predicting whether a particular call into you call center will result in cancelation is difficult. But predicting which marketing campaign will be effective for a select demographic can be very powerful. As you launch your ML program, look to define the use cases in your organization that are most practical, and be clear about those that are not a good fit.

4. Get everything organized before starting. 

This is a classic tenet of waterfall-style development – that deep analysis and careful documentation is needed before investing in technology.  In this case, bad idea.  As mentioned above, ML is best done iteratively.  But, on the other hand, the quality (and structure) of the data matters greatly.  Surely, “garbage in garbage out” applies here, particularly with such a magical black-box like ML?  True, but you can get started in parallel.  Let us take an example: imagine that you are trying to correlate product placement with sales uplift.  Seems like a good use-case, with lots of available data.  But, in this case, it would be best to have a full year of data, for multiple products, for multiple locations, to allow for seasonality. And the sales data and store locations must be accurate.  You could work for six months to get all the input organized and cleansed, or you could get started, improving quality and breadth in parallel. Take a subset of data, manually cleansed if you have to, and attempt to correlate factors.  Demonstrate the potential.  Small insights will result, and you can direct the next iteration.  Virtually every company would love to have highly accurate data in their data warehouse, but few do. Think “minimally viable product.” A simple POC (properly explained) can go far to build confidence and spark imagination on possible applications.

ML is a powerful set of concepts and methods, and the tools are becoming productive and practical. We recommend experimentation, research, and the all-important concise communication that will help set expectations. Value and use cases are everywhere and have the potential to be highly differentiating. Start small and give it a try.

What misconceptions have you uncovered in Machine Learning?

Mike Strange

About the Author

Mike Strange Office Managing Vice President, Los Angeles Office
Mr. Mike Strange is a passionate technology leader, with 20+ years of IT architecture and IT leadership experience. He has served in numerous IT leadership roles, including CTO and Senior Vice President. He is passionate about bringing clarity to the design and development of modern architectures and systems, and speaks at conferences regularly regarding IT trends. Mike is Vice President in our Los Angeles office.

More Perspectives

Perspective
by Bruce Ballengee
Teaching Roots Run Deep
Perspective
by Kent Norman and Kevin Moormann
I’ll be Home in Five Story-points
Perspective
By Russell Clarkson
Is your Ecosystem a pipeline or a platform?
Perspective
by Dimitrios Vilaetis
Business Capabilities: a Spotlight for Strategic Decision Making
Perspective
by Russell Clarkson
Mark Your Exits
Perspective
by Bruce Ballengee
Leadership and Company Culture
Perspective
by Kerry Stover
I Know You Believe You Understood What You Think I Said
Perspective
by Brian Duperrouzel
How to Handle Management Kidnapping
Perspective
By Russell Clarkson
What Chuck Yeager Taught Me About Disruption
Perspective
by Brian Duperrouzel
Lessons on Project and Team Management
Perspective
by Bruce Ballengee
Consulting that Focuses on the Individual
Perspective
By Bruce Ballengee
Developing the Individual
Perspective
by Lori Dipprey
Why Performance Reviews are Here to Stay at Pariveda
Perspective
By Nathan Hanks
What it Means to be "in the People Business"
Perspective
By Bruce Ballengee
Pariveda Fin-Centric Design
Perspective
By Bruce Ballengee
Unleashing the Power of Humility
Perspective
By Mike Strange
4 Reasons to Leverage the Power of Small Teams
Perspective
By David Watson
The Benefits of Working in Teams
Perspective
by Sean McCall
Forget Coffee- Energize Your Work Morning
Perspective
By Allison Esenkova
Wearing Heels to the Top
Perspective
by David Watson
Work-Life Balance
Perspective
by Allison Esenkova
Listen to the Still Small Voice
Perspective
By Kerry Stover
Must Read Books from the COO
Perspective
By Russell Clarkson
Why You Should take a Massive Open Online Course
Perspective
By Scott Hajer
The Importance of Making Space to Think
Perspective
by Sean McCall
4 Ways Sports Can Benefit Careers
Perspective
By Russell Clarkson
Stand Up for Good Presentations
Perspective
by Derrick Bowen
How Much of your Project Value is At Risk Due to Cognitive Bias
Perspective
by Susan Paul
Capabilities as Building Blocks
Perspective
By Susan Paul
When Corporate Culture Undermines IT Success
Perspective
by Kent Norman
Limit, to Win It
Perspective
By Brian Duperrouzel
The Case for Onshore Technology Teams
Perspective
By Tim Hurst
Unlocking Marketing ROI Analytics
Perspective
by Brian Duperrouzel
Hippos and Cows are Stopping Your Machine Learning Projects
Perspective
By Jack Warner
Building Smart Deployment Pipeline
Perspective
By Tom Cunningham
The Sound of the Future
Perspective
By Scott Hajer
You Need Only One Kind of Recruiting Technology
Point of View
by Ansley Galjour
Continuous Brand Engagement
Perspective
By Marc Helberg
3 Ways You Can Begin to Take Patient Experience More Seriously
Perspective
By Ryan Gross
What Does It Really Mean to Operationalize a Predictive Model
Perspective
by Sophie Zugnoni
Did You Catch Machine Learning Fever?
Perspective
By Collins DeLoach
What does Cloud Transformation mean it IT?
Perspective
By Mike Strange
Untangling and Understanding DevOps
Perspective
by Clayton Rothschild
Blockchain in an Enterprise Setting
Perspective
by Mike Strange
DevOps a Practical Framework for Technology Process and Organizational Change
Perspective
by Julio Santos - Finity Member
Context as Currency
Perspective
by Oussama Housini
Why DevOps?
Perspective
by Dave Winterhalter
Data in the Dugout
Perspective
by Brian Duperrouzell
3 Tips to get the Blockchain Ball Rolling
Perspective
by Victor Diaz
6 Things to Consider when Choosing a Data Visualization Library
Perspective
By Julio Santos and Jon Landers
How Customer Context and Smarter Algorithms will Power the Next Level of Experiences and Engagement
Perspective
by Mike Strange
Can We Predict the Future?
Perspective
By Brian Duperrouzel
No Estimates, No Approval, No Dice
Perspective
By Marc Helberg
How AI Will Affect Your Patient’s Experience
Perspective
By Sean Beard and Brian Orrell
Life After Mobile
Perspective
by Phillip Manwaring
Let Serverless Solve the Tech Problems You Don't Have
Perspective
by Mike Strange
Bigger is Not Necessarily Better
Perspective
By Marc Helberg
Patient Experience – Taking the Next Step Through Machine Learning
Load More