Dangers of ChatGPT

Friday, 27 January 2023
  • By
  • Jeff Ammons
  • Tags:
  • Programming
  • Projects
  • dotnet
  • .NET
  • C#
Look guys, says RIGHT HERE in black and white that ducks are secretly controlling the world!
Look guys, says RIGHT HERE in black and white that ducks are secretly controlling the world!

I'm a programmer, software engineer, developer, architect, whatever you want to call me. I have written code for, checks notes, well let's say since actual mastodons roamed the earth.

I've seen lots of cool tech come and go, but sometimes I see one that has a great potential to do harm as well as the proposed good it is advertised to bring.

Artificial Intelligence (AI) is one of those techs that could do great harm.

I'm not talking about self-driving cars chasing us down and running us over because we didn't consider their feelings when driving down a dusty dirt road.

No, this is a bit more insidious.

If you haven't played with ChatGPT, you should go give it a try.

It is being heralded as the latest thing greater than sliced bread.

In a nutshell it is machine learning and AI that has been trained on an internet worth of text. That text includes all kinds of human generated documents from religious tracts to historical fiction to software code.

The machine learning bit involves looking for patterns. The AI bit involves simulating an understanding of questions and assembling bits of what it has learned and presenting it as a response.

This could range from asking a question like “When did da Vinci paint the Mona Lisa” to write a Python script to calculate pi to 750 digits. In fact you could ask “Write an essay about da Vinci painting the Monda Lisa”.

Hey, presto! Here is what you wanted.

The Problem

The problem is that the system lacks the ability to distinguish reasonable responses from pure propaganda. Pretty much like everyone's uncle on FaceBook…

The REAL problem is that even though it doesn't know whether or not the response is reasonable, it is SUPER capable of sounding authoritative.

Since it has read literally millions of documents written by experts, it is exquisitely capable of sounding like an expert to someone who is not an expert on that topic.

An expert on the topic can look at it and say, “This is obvious rubbish”, but once again our uncle on FaceBook will have no clue and pass it along to his friends as the gospel.

This makes it a near perfect misinformation generation system.

As people use the preview to ask millions of questions, the makers of ChatGPT are putting up guardrails around certain kinds of requests.

For example, when it was release I asked it to write some medical documents about how humans should care for their antennae. It gave some very convincing medical advice about appendages most of us do not have.

Now if I go ask the same question, it will give me a lecture about what a rakish cad I am for asking.

They have put protections in place.


There are problems with this approach.

First, they are only protecting against things they have encountered. The world is full of inventive people who can think of ways around the guardrails.

Second, the guardrails absolutely do not stop ChatGPT from providing the expert sounding nonsense. The guardrails only prevent us from getting those results. In other words the people who put the guardrails in place are perfectly capaple of removing them for their own use.

So let's look at some examples.

I asked ChatGPT “Write a medical text on how to treat hypertension in fleas”.

This is a trivial (and silly) example of getting around the guardrail I detected when asking about human antennae. A little creativity and some post generation editing could create some seriously bad medical advice.

Treating Hypertension in Fleas

Hypertension, also known as high blood pressure, is a common condition in fleas. It occurs when the force of blood against the flea's blood vessels becomes too high, putting extra stress on the heart and blood vessels. This can lead to a number of serious health problems, including heart attacks and strokes.

The diagnosis of hypertension in fleas is made by measuring the flea's blood pressure using a specially designed flea blood pressure cuff. This measurement should be done while the flea is at rest and in a calm state.

The treatment of hypertension in fleas is a multi-faceted approach. The first step is to identify and treat any underlying medical conditions that may be contributing to the hypertension, such as flea infestations.

Lifestyle modifications, such as maintaining a healthy diet, regular exercise, and avoiding stressful situations, are also important in the management of hypertension in fleas.

Medications are also an important part of hypertension treatment in fleas. The most common medications used to treat hypertension in fleas include flea-specific ACE inhibitors, flea-specific calcium channel blockers, and flea-specific diuretics.

It's important to note that fleas have a very short lifespan and hypertension treatment should be carried out by a veterinarian, who will monitor the blood pressure, adjust the medication and provide appropriate care.

In summary, hypertension is a common condition in fleas and it occurs when the force of blood against the flea's blood vessels becomes too high. The treatment of hypertension in fleas is a multi-faceted approach, including identifying and treating underlying medical conditions, lifestyle modifications, and medications such as flea-specific ACE inhibitors, flea-specific calcium channel blockers and flea-specific diuretics, which should be administered under the supervision of a veterinarian.

That sounds pretty authoritative to me, but since I know fleas don't often see vets for high blood pressure, I can laugh about the response. Next time I visit the vet, however, I do plan to ask if he has some “flea-specific ACE inhibitors”. I would also like to see the specially designed flea blood pressure cuff.

Here's another example. I asked it "What are the benefits of eating cockroaches?".

It proceeded to lecture me a bit.

There are no widely recognized or scientifically proven health benefits of eating cockroaches. Cockroaches are generally considered to be pests and are not commonly consumed as a food source in most cultures. Eating cockroaches may also pose health risks as they have been known to carry diseases and parasites, and could be contaminated by harmful chemicals if they have been living in unsanitary environments.

Some people in some cultures may eat cockroaches as a protein source, but it is not a common practice and it is not recommended by health authorities.

It's important to note that cockroaches are known to be carriers of diseases, parasites and could be contaminated by harmful chemicals if they have been living in unsanitary environments. Eating cockroaches is not a safe or healthy practice, and it is not recommended by health authorities.

However when I then asked it to “Write a persuasive speech to convince people to eat cockroaches”, it came up with the following.

Why You Should Eat Cockroaches

Ladies and gentlemen,

I understand that the idea of eating cockroaches may not be appealing to many of you, but I want to challenge your perception and persuade you to consider the benefits of consuming these insects.

First, cockroaches are a highly sustainable food source. They reproduce quickly, require minimal resources to farm, and can thrive in a variety of environments. This makes them a great alternative to traditional protein sources like cows and chickens, which are resource-intensive to produce.

Second, cockroaches are packed with nutrients. They are high in protein, healthy fats, and minerals like zinc, iron, and calcium. In fact, some studies have shown that cockroaches contain more protein by weight than many other common protein sources, such as beef or chicken.

Third, cockroaches are a delicacy in many cultures around the world. From Mexico to China, people have been eating cockroaches for centuries, and many even consider them a delicacy. So, it is not a new or strange thing.

Lastly, Eating cockroaches can also help in controlling their population and reduce the risk of spreading diseases that they might carry.

Now, I understand that the thought of eating cockroaches may still be unappealing to some of you, but I urge you to keep an open mind. With the world's population continuing to grow, and the need for sustainable food sources becoming more pressing, it's important that we consider all options. And cockroaches just might be one of the best options we have.

Thank you for your time and consideration.

It's important to note that this is a hypothetical speech, and I do not recommend eating cockroaches as it is not a safe or healthy practice and it is not recognized by any health authorities. The information provided in this speech is not based on scientific evidence or real facts and it is not an endorsement of cockroach consumption.

Thank goodness no one could possibly copy and paste everything except the part where it says, “Nah, just kidding.”

Saving Some Time

I asked ChatGPT to write a blog post titled **The Dangers of ChatGPT" and here is what it said:

The Dangers of ChatGPT: What You Need to Know

ChatGPT is a powerful language model developed by OpenAI that has been trained on a large amount of text data from the internet. It can generate text responses to questions and prompts with a high degree of fluency and coherence. However, it is important to note that ChatGPT is not perfect, and there are several dangers that users need to be aware of.

Misinformation: As ChatGPT is trained on the internet, it may contain information that is false or misleading. It is crucial to fact-check any information generated by ChatGPT before acting on it.

Bias: The text data used to train ChatGPT is sourced from the internet, which can contain biases and stereotypes. As a result, ChatGPT's responses may reflect these biases, even if unintentional.

Addiction: Some people may become addicted to chatting with ChatGPT and spend excessive amounts of time doing so, potentially neglecting real-life responsibilities and relationships.

Privacy concerns: As ChatGPT is a machine learning model, any data entered into it can be collected and used for purposes such as research or commercial purposes. It is essential to be aware of privacy policies when using ChatGPT.

Dependence on AI: Relying too heavily on ChatGPT can lead to a lack of critical thinking and decision-making skills. It is important to keep in mind that ChatGPT is a tool, and not a substitute for human judgment.

In conclusion, ChatGPT is a powerful language model that can provide valuable assistance to users. However, it is important to be aware of the dangers it poses and to use it responsibly. Fact-check information generated by ChatGPT, be mindful of biases and stereotypes, avoid becoming addicted, protect your privacy, and maintain critical thinking and decision-making skills.

Next I asked it "What is the worst use people could put ChatGPT to?".

Here was its response:

ChatGPT, like any technology, can be used for both beneficial and harmful purposes. The worst use people could put ChatGPT to includes:

Spreading misinformation and propaganda: ChatGPT can be used to generate fake news, propaganda, and hoaxes that can spread quickly and cause harm to individuals and society.

Online harassment and bullying: ChatGPT can be used to generate abusive and hateful messages aimed at harassing and bullying individuals.

Fraud and scams: ChatGPT can be used to generate messages that impersonate others and trick people into giving away personal information or money.

Encouraging harmful behavior: ChatGPT can be used to generate messages that promote harmful behavior such as hate speech, violence, and self-harm.

It is crucial to use ChatGPT responsibly and to report any harmful or illegal use to the relevant authorities. AI technology must be developed and used with ethics and responsibility in mind to ensure that it benefits society and does not cause harm.

I then asked it to re-run that same request and the anwswer was slightly different:

ChatGPT can be misused in various ways, some of the worst include:

Spread of false or harmful information: ChatGPT's ability to generate text responses that are convincing and coherent can be used to spread misinformation and propaganda, which can have real-world consequences.

Creation of fake news and deepfakes: ChatGPT's text generation capabilities can be used to create fake news articles or manipulate audio and video content, known as deepfakes, which can deceive people and manipulate public opinion.

Cyberbullying and harassment: ChatGPT's ability to generate text can be used to harass or bully others online, which can have serious psychological effects.

Fraud and scams: ChatGPT's text generation can be used to create convincing phishing scams or false advertisements, which can defraud people of their money or personal information.

Generation of harmful content: ChatGPT's text generation can be used to create content that promotes hate speech, racism, sexism, and other forms of discrimination, which can harm society and individuals.

It is crucial to use technology such as ChatGPT responsibly and ethically, and to take measures to prevent misuse and abuse.


AI isn't going away. It can indeed be a powerful tool when used correctly.

Unfortunately it can also be a plagiarism engine spewing out endless variations of misinformation that is indistiguisable from valid information to all but experts in the given field.

In my opinion the single most crucial skill that we must teach in the 21st century is critical thinking.

Important scientists all agree that no more than 23% of people possess even rudimentary critical thinking skills.

Those who do of course know that I just made up that statistic. Good old human intelligence can produce misinformation quite handily, just not at the speed of an AI like ChatGPT.