Show HN: Claude and ChatGPT can't do basic operations?

2 points

2 years ago

Fascinating experience in trying to get Claude and ChatGPT to do basic data analysis. I pasted a list of 290 locations into each tool and asked each one to tell me the percentage each location represented in the data set as well as a count for each location.

Both of them were wildly wrong for different reasons.

Claude completely miscounted the number of instances for one location. https://imgur.com/a/ShaORko

And then it kind of tried to gaslight me about it. https://imgur.com/a/ehbQ9hu

(EDIT) ChatGPT incorrectly counted instances and then reported that one location it said had 189 instances (actually 91) represented 99.47% of the dataset. Other locations had also incorrect percentages and the total percentage was far over 100%. This seems to be because despite identifying the instances individually, it was calculating using a total of 190 instances. https://imgur.com/a/yNcAHZB

Presumably this has something to do with the data format, but it was simply a copy-paste from a single column in a Google Sheet, which strikes me as a pretty standard format for data. I also pasted the same data into a different Google Sheet and it behaved normally.