Heykuki News

TopNewBestAskShowJobs
TopNewBestAskShowJobs
Ask HN: How many % of Machine Learning projects “fail”?
16 points
satishgupta
5 years ago
There are all these reports/surveys claiming ~80% of ML projects fail:

- Jan 2019: https://blogs.gartner.com/andrew_white/2019/01/03/our-top-data-and-analytics-predicts-for-2019/ Gartner predicted that through 2020, 80% of AI projects will remain alchemy, run by wizards and through 2022, only 20% of analytic insights will deliver business outcomes.

- May 2019: https://content.alegion.com/dimensional-researchs-survey Dimensional Research - Alegion Survey reported 78% of AI or ML projects stall at some stage before deployment, and 81% admit the process of training AI with data is more difficult than they expected.

- July 2019: https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/ VentureBeat reported 87% of data science projects never make it into production.

There used to be similar claims in 1990s about failure of Software Development projects:

- 1994: https://www.standishgroup.com/sample_research_files/chaos_report_1994.pdf The Standish Group’s CHAOS Report in 1994 claimed that only 16% of the software projects succeeded.

That claim was doubted: https://ieeexplore.ieee.org/document/1438340

Overtime, CHAOS reports became nuanced: https://www.standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf

For building ML-based software applications/product-features, my questions are:

- How "failure" is defined for ML projects, when to call a project failure?

- Is failure rate really 80%?

5 comments