- Jan 2019: https://blogs.gartner.com/andrew_white/2019/01/03/our-top-data-and-analytics-predicts-for-2019/ Gartner predicted that through 2020, 80% of AI projects will remain alchemy, run by wizards and through 2022, only 20% of analytic insights will deliver business outcomes.
- May 2019: https://content.alegion.com/dimensional-researchs-survey Dimensional Research - Alegion Survey reported 78% of AI or ML projects stall at some stage before deployment, and 81% admit the process of training AI with data is more difficult than they expected.
- July 2019: https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/ VentureBeat reported 87% of data science projects never make it into production.
There used to be similar claims in 1990s about failure of Software Development projects:
- 1994: https://www.standishgroup.com/sample_research_files/chaos_report_1994.pdf The Standish Group’s CHAOS Report in 1994 claimed that only 16% of the software projects succeeded.
That claim was doubted: https://ieeexplore.ieee.org/document/1438340
Overtime, CHAOS reports became nuanced: https://www.standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf
For building ML-based software applications/product-features, my questions are:
- How "failure" is defined for ML projects, when to call a project failure?
- Is failure rate really 80%?