A site devoted mostly to everything related to Information Technology under the sun - among other things.

Monday, September 8, 2025

"ChatGPT is bullshit" etc.

I came across this paper:  "ChatGPT is bullshit" by Michael Townsen Hicks, James Humphries and Joe Slater recently.  

I think that the authors are quite right in pointing out and emphasizing the limitation of LLMs, their tendency to generate new text whose referents are not True, i.e. do not exist.  I experienced that multiple time with earlier versions of ChatGPT in which it supplied books on the history of ancient Assyria that did not exist; even though the authors existed - to give an example.

(Some, who have used Perplexity, state that they found it to satisfy most of their needs, including pretty sophisticated scientific ones with a lot more integrity than ChatGPT.)

Perhaps, as some claim, this whole arena is out of control; especially with Gen Z being headed for even more psychological issues...Please see: https://futurism.com/ai-dot-companion-controversy

Nevertheless, the LLM-based tools, in my opinion, are very very useful...To wit:

  • Please look at this item that I created, interacting with ChatGPT, after watching the TV Series - "Wolfe Hall" - Crimson Reason: Using ChatGPT: Who Poisoned Cardinal Wolsey?
  • I used ChatGPT and research produced by another person to create an article in Wikipedia: please see https://en.m.wikipedia.org/wiki/Qian_family_lineage
  • ChatGPT could diagnose me, based on my full symptoms, with a cracked tooth, which was confirmed when my tooth actually broke while chewing!
  • A Professor of Mechanics told me that ChatGPT "knows" more than him of the Theory of Elasticity; basically, the entire corpus of that field is available to ChatGPT.
  • Staff in General Motors have been using it to create skeleton presentations...in some instances. ChatGPT could easily eliminate an entire role/job.
So where is this going?

The LLMs are taking us back to Real Scientific Discovery: i.e. they now make it possible to discriminate between real scientists - those who can generate interesting questions and hypothesis - and all these scientific hacks that are consuming resources and are occupying chairs and lecterns.

Mind you, ChatGPT cannot produce new knowledge.  If you ask it to compute the electric field of a charged conducting torus, it cannot do it.

But for basic reference work it is fine.

I think, just like the advent of the Ethernet, HTTP, and the Internet and their adoption by the public and private enterprises, LLMs (and likely the Artificial Neural Nets, and Ontology Systems) are going to be adopted.  In a manner similar to the adoption of the Web technologies, we shall see first the construction of the hardware IT Infrastructure to support AI technologies, followed by several decades of software work to "rewire", reorganize, and completely re-do the workflows of enterprises.

In the scientific arena, I think MMLs are here to stay. Just like Mathematica/Maple and other computer algebra systems, they are tools of the 21st century.

The impact on Liberal Arts education will be tremendous; gone will be the days of reading 4 books, 2 papers, and 3 exams to get an A.  Hopefully, this will cause the Liberal Arts to become, once again, the education of those who care about ideas and not a meal-ticket.

Already, in Qum, where many religious universities are located, there have been conferences on AI, its advantages and its threats, even the mullahs are thinking ahead to an LLM system that has Arabic and Persian, and Turkish texts of any scrap of text on Islam, Quran, the Traditions of the Prophet & Imamas, and Islamic Law available for query... 

The adoption of this plethora of AI technologies is enabling a lot of creative work which would otherwise have been very difficult, very expensive, or impossible to realize; e.g. Redneck Star Trek – Beam Me Up, Bubba | AI Country Star Trek Parody - YouTube

Just like the current Smartphones, with their built-in sensors and video displays, and apps such as phyphox, which I first saw - called a Tricorder  - in Star Trek episodes, I saw ChatGPT also in an episode of the initial series.

In time, I expect the retrofitting of Ontologies into LLMs to reduce their "hallucinations" - but not to totally eliminate them; in a manner similar to the ubiquitous bugs in software/hardware system (yes, even in construction field, there are defects that have to be corrected with steel and mortar after the fact, when the structure has been put to use...)

About Me

My photo
I had been a senior software developer working for HP and GM. I am interested in intelligent and scientific computing. I am passionate about computers as enablers for human imagination. The contents of this site are not in any way, shape, or form endorsed, approved, or otherwise authorized by HP, its subsidiaries, or its officers and shareholders.

Blog Archive