In a discussion on the forums of Scott’s Bass Lessons, a thread was recently started where someone confessed to using ChatGPT to try and help with some harmony analysis homework and the disappointing results it had achieved. I chipped in early to note how “AI” systems have a tendency to move freely from well-attested facts to making up nonsense with no warning. Following the ongoing discussion, it struck me that the AI systems I have looked at all seem to be missing two vital factors – sources and workings.

Why don’t they provide copious footnotes? When they have drawn from a particular source, they could give a reference as you would expect in an academic paper. If they have synthesised several sources, they could list all of them and, where they have made deductions, that could also be made clear. That would deal with all the concerns about plagiarism and, if you were asking a system to inform you about a topic you aren’t an expert in, you would still have grounds to assess how much to trust it.

I suspect one of the reasons this practice doesn’t seem to be commonplace (I’ve not seen it on any of the ‘knowledge’ systems I’ve looked at) is that it would reveal how much is still smoke and mirrors. The Wizard of Oz turned out to be unimpressive when Dorothy looked behind the curtain and that might why the creators of AI systems don’t want to reveal the workings behind the answers given. However, I think it would improve our ability to both assess and eventually to trust the results.

I put this out there as a free suggestion both for those working on AI and machine learning in academic environments (where referencing sources ought to be second nature) and the systems themselves. Prove yourselves to not only be able to generate content but to do so with integrity and rigour.