Ethics, history, and LLMs

I have just attended the first lecture of CS 211 this morning. CS 211 is an ethics course specifically designed for CS students at UIUC and is a required core course for CS majors. I didn't expect the in-class atmosphere to be a bit intense as the lecturer, Prof. Ryan Cunningham, discussed ethical and unethical professional conduct during the Holocaust in WWII.

The first impression I had of this class came from a few fellow students. They mentioned that Ryan is a great professor, and that most of the course content discusses philosophical ideologies and their relevance to our fields. I had high expectations for this class, partly because I wondered what would be covered in this semester-long course and what I would be writing about. Additionally, I have friends studying AI ethics in the field of sociology who also reminded me of the slogan "tech for good."

During the lecture, Ryan discussed several events and drew an analogy between the prologue of the Holocaust and software development cycles. These events included Aktion T4 as the alpha release and Dehomag (I was unaware that IBM Germany, known as Dehomag, was involved in the Holocaust, building census tabulating machines that would later count innocent Jewish lives and appear on Eichmann's list) as the beta release. Eventually, the Wannsee Conference served as the full release. While I personally find this analogy inadequate, and I don't believe Ryan does either, it does provoke serious contemplation regarding the ethics of computer science for freshmen. I strongly believe that studying ethics is much more essential than other technical courses.

The scientists and engineers who contributed to the Holocaust lacked the moral compass and ethics to prevent their involvement in those dreadful events. Instead, they competed with one another to devise so called "solutions" like mobile gas chambers. It is our responsibility as a generation and the next to uphold strong ethics in our professional work and prevent atrocities like the Holocaust from occurring in various engineering or research fields.

Ethics in LLM?

It hasn't been long since transformer models evolved to the point where they have become the most emerging and powerful tools capable of performing tasks almost as well as humans, despite some debates on their accuracy and hallucinations. Numerous open-source models have been released to rival OpenAI's GPT-4, such as Mistral's torrent models and Meta's Ilama. Open source projects are great, and I strongly believe that democratizing LLMs for educational/learning purposes is crucial. It's not reliable to store all your data, let OpenAI train it on their end, and the strong dependency on their service availability. Many libraries, such as OpenLLM and LMStudio, have also been developed to enable local inference on PCs or Macs. These libraries have been evolving over the past year but are still not quite up to par. This discrepancy can be attributed to the lack of sufficient data and training time in comparison to what OpenAI has achieved.

However, I can't help but think about the potential consequences if these models fall into the wrong hands as they continue to advance, as many people would. You wouldn't want terrorists to exploit them and launch attacks on innocent people, leveraging the knowledge contained within these models without even needing to learn about certain subjects. Though currently, expert tuning is required for better performance, making such scenarios highly unlikely for now. But what if we reach a point where this technology becomes more accessible? Do we have ways to deterministically prevent such misuse? It is important to consider that a "Terminator" scenario may not happen until humans themselves contribute to their own extinction.

So, why aren't there enough mechanisms in place to prevent LLMs from responding to unethical questions? In the current state, we have heard about LLM injections and poisoning, along with several terrible chat examples that pose racial and gender adversaries, as well as real-world safety concerns. Whether these vulnerabilities are due to business reasons, the notion that engineering work requires iteration and exposure for improvement, or other factors, we are not discussing a minor invention here. This invention, LLM, has the potential to cause significant harm to society if it hasn't already done so in recent times. This is what concerns me the most, apart from the major disruptive projects associated with LLM or other generative AI technologies that I have been closely following.

While I am not an expert in the field of deep learning, I am actively keeping up with the latest work to become one, by taking various courses, conducting research, and engaging with people both inside and outside of UIUC. Last semester, I took CS 445 and gained an understanding of the technical aspects and intrigue behind generating fake photography. Meandhilw, I also learned that detecting the authenticity of such generated content is becoming increasingly difficult and, in certain scenarios, has already become a challenge.

The road ahead is long.

Efficiency v. safety, which is more important? Would you sacrifice the ethics of constantly improving LLM without considering whether it could provide unethical answers, simply for the benefit of finishing homework or work faster? Or would you prioritize safety aspects, whether from a mathematical or engineering perspective, even if it means slower performance or offering fewer choices?

Personally, I would choose safety. I don't mind if the response time is faster for a better user experience. As long as the LLM provides ethical, accurate and unbiased answers through iterations, it remains a valuable and helpful tool. User experience should only be considered once the product has undergone thorough engineering processes.

What would be your choice?

Reference

1. Aktion T4, Wannsee Conference, SS Race and Settlement Main Office are all notable events or organizations related to the Holocaust.
2. Dehomag, the machine used by the Nazis to create a census in preparation for the Holocaust, is further described in detail in The Nazi Party: IBM & “Death’s Calculator”. The Höfle Telegram refers to a message intercepted by the British Enigma machine that "contains detailed statistics on the 1942 killings of Jews in extermination camps."
3. René Carmille was a French military officer who sabotaged the Nazis' census in order to protect Jewish people in France and save them from extermination camps. The White Rose was a group of university students and a professor who campaigned against the Nazi regime during the Holocaust.
4. OWASP (Open Worldwide Application Security Project) on Prompt Injections and Training Data Poisoning, and more.
5. LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked.
6. AI poisoning could turn open models into destructive “sleeper agents,” says Anthropic, on ArsTechnica.
7. LLM Vulnerabilities: Understanding and Safeguarding Against Malicious Prompt Engineering Techniques, on HackerNoon.