ChatGPT
Last week OpenAI released ChatGPT, the latest installment of the company’s Large Language Model. This model, GPT3, uses billions of parameters and almost the entire written internet as its testing data. ChatGPT is a conversational version of the model - you’re able to ask follow up questions, elaborate on the discussion, and ask for technical examples like code snippets.
Since its release, nerds such as myself have been toying with the new technology and collectively losing our minds because of how cool it is. Machine learning and AI has existed for decades, but this one is different. It makes me feel FOMO despite the fact that it’s currently happening.
It’s not perfect, don’t get me wrong. Someone on Reddit mentioned - disparagingly - that it has the intelligence of a 10-year-old. I just hope we can all agree how crazy it is that they built an AI with human-level intelligence. And even more insane: the humanlike chatbot knows almost everything on the internet and can understand what you want and relay that information to you immediately.
It can eliminate hours of research and sort through millions of data points in seconds. It can write compelling stories, debug your code, and if you're a Twitter employee, generate a report of the work you completed this week. My favorite example might be this dude who had the bot create a Harry Potter themed multiple choice game from scratch.
Because the next version will inevitably be much, much better, it’s important to point out the significance of this technology and its potential impact.
What's the big deal?
If you don’t understand why it’s awesome, I recommend trying it out. Ask the AI about anything as long as it's not specific to 2022 (the training data ends in 2021). Ask it about the details of something you're knowledgeable about. Ask it to write a song or a screenplay. It’s as if someone memorized Wikipedia, Stack Overflow and every book ever and was available to provide you with a friendly explanation of whatever you want.
I got a similar feeling while trying out Jasper.ai. It’s another language model AI that will write for you - from blog posts to Instagram captions. I’ve spent hours trying Jasper but only used it once for two sentences in my encryption vs hashing blog post (try to spot it). I had a lot of fun playing with it, but it wasn’t very helpful for what I want to write.
The problem is that it doesn't sound like me, and it doesn’t have the technical knowledge for most of what I write about. The result is that it will repeat the phrases that I use to give it context, or it will sound like a generic blog that cares more about SEO than being interesting.
Input to Jasper.ai: give me an intro paragraph for a blog post about the difference between encryption, hashing, and encoding, where I'll provide examples of each
Output:
Data security is more important than ever in today's world, and there are a number of different ways to protect your data.
One of the most common ways to do this is through encryption, hashing, and encoding. But what's the difference between them?
Here's a look at the differences between them and some examples of each.
It knows what to write about, but it’s boring. It’s probably great for writing marketing material and getting your article to appear on Google, but I don’t think I’ll ever write an entire blog with it. Despite being able to provide a ‘tone of voice’ for customization, it wasn’t very witty.
But it was still magic.
And OpenAI’s ChatGPT is far more magical due its deep technical knowledge and the ability to form coherent opinions that require nuanced thinking, which I’ve never seen from a chatbot before.
Nerds rejoice
Perhaps the most interesting part about the release of ChatGPT is who is talking about it. It’s not the typical media who is letting people know about it - it’s the people who have the best understanding of how the technology works. Normally you’d think that game-changing technology like this would spread to every part of the internet.
I would’ve guessed that the mainstream media would either (1) let people know that it’s fun to play with, or (2) tell everyone to be worried because robots are coming for your job. If you search for info from the Wall Street Journal and New York Times, however, you won’t find anything (as of 12/5/22):
But if you ask the nerds - who very rarely pontificate about these sorts of things - they’re freaking out. This is especially true for software engineers. This new model is so powerful because although we don’t want to admit it, software engineering is predominantly using other people’s code for your own use case. And because OpenAI’s model is based on all the publicly available code on the internet, it shrinks the ‘Google > StackOverflow > copy/paste > lookup error > verify it’s correct’ flow into one quick sentence, with follow ups to get more precise answers.
The reason why technical people are so interested in ChatGPT is because it doesn’t just copy/paste code from a particular website - it creates never-before-seen, documented code that works most of the time. Sure there are plenty of flaws, but no more than you get from humans. And it’s fast af.
Here it is solving a coding interview problem from Leetcode:
It doesn’t just give one solution, it gives two: the “brute force” solution that a human would consider first, then the more efficient solution using a Hash Map, along with explanations as to why it made the decisions it did. Pretty wild.
What’s next?
After the curiosity and eyebrow raising, the thought you get after using ChatGPT is how can I make this work for me? More specifically: how can I run a business where the bot does 90% of the work for me?
What comes to mind is research, where hours can be reduced to minutes if you know what you’re looking for. Customer support and copywriting seem like low-hanging fruit as well.
For now it can't replace your job - in its current form it can’t do high-level tasks and context switching, like sending an email, getting a response and then acting on it. However, it can 100% do the most time-consuming tasks of software engineers, copywriters and customer support agents. And it’s inevitable that there will be applications built on top of it that can handle the context-switching for you, which will most likely lead to a reduction in headcount.
I believe the next step, which I’ll concede has been happening for a while, is finding sets of data that can be used to train the models. Importantly, the most valuable sets of data are those that nobody else has. On the All In podcast, Chamath Palihapitiya described this concept as MAAS (model as a service).
This is the idea that companies with proprietary data can have tremendous leverage for building models. He gives the example of a medical company that screens for cancer and has a large collection of images of tumors. If they have the best data set across genders and/or ethnic backgrounds and they keep it proprietary, they could drive tremendous value by making decisions that are impossible for a human (or any other company) to make.
We’re entering a bubble, folks
Because ChatGPT can take millions of data points and summarize it in a natural way, it can function beautifully as a search engine. Whereas with Google you search something and sift through the results, with ChatGPT you don’t need to search through anything because the result it gives you is the information you’re looking for.
It can also behave as a personal assistant - this guy told the chatbot about his habits and body measurements, and the AI was able to formulate an entire weight loss plan, complete with exercises and meal plans.
So Google search could be disrupted, but betting that AI is going to disrupt Google is a really dumb statement if you know anything about Google. Nevertheless, GPT's ability to form coherent thoughts about complex topics is something that could certainly chip away at Google’s monopoly in the short term.
Despite my hype, if history is any indication then we're probably heading into a ChatGPT bubble. It’s awesome tech, but it’s somewhere around 95% correct, which could cause significant problems if it actually replaced engineers in its current state.
According to the Hype Cycle of technology, ChatGPT probably lands somewhere in the “Peak of Inflated Expecations” portion of the graph. In the next few months and years there will be plenty of excitement. Companies will form and millions of dollars will be invested as people try to ride the wave. Then the hype will bottom out - similar to the hype surrounding self-driving cars - and as soon as you ask “what happened with that GPT thing?” you’ll realize that your company has been using it for the past few years.