Perspective Newsletter # 3
I curate good content from the internet and send it out as a digest every fortnight with my summary and inferences. This fortnight's topic is Generative AI.
ChatGPT, GenerativeAI, LLMs, OpenAI are some of the buzzwords one gets to hear everywhere nowadays and if you browse LinkedIn’s feed by mistake, it seems the world has already crossed singularity and any company or professional not using AI is doomed. Just to be clear, I am not a pessimist, I believe that LLMs are powerful and will change a lot in the world but I am definitely a pessimist in the short run.
Currently we are in the peak of GenerativeAI Hype. I have seen a similar hype around Deep Learning in 2015-16 (time when we started our AI startup, which closed in 2020). Everyone thought self driving cars, chat bots, computer vision would bring in paradigm change. They did, but the impact was way less than expected. It is generally true as well, we humans are either too optimistic or too pessimistic (be it Stock market, Real estate or personal relationships).
Let me take you back to 2015, Deep learning(DL) has just made its mark on the world and everyone was trying to find applications where DL could bring disruption. GenerativeAI, where we were able to create 28*28 low quality useless images, was not even considered a serious contender. The front runners were self-driving cars, Chat bots, sentiment analysis, speech to text, AI assistants (Siri, Alexa and Google Assistant), automated diagnosis. Deepmind broke the internet when their AI became the best in Go. Now, at the time of writing this i.e. 2023, AI assistants are used to play music or ask for weather or set reminders, Self-driving cars are struggling to say the least and Chat bots generally end up handing over conversations to human agents and are no less irritating as they were in 2016 and let’s not even discuss the automation of diagnosis claims. So if you see, all the promises were hyped and success came from an underdog - GenerativeAI.
It is difficult to predict future even in a small domain and even more difficult is to predict the impact of something so new. All great inventions took decades to actually bring an impact in the world.
And the good thing is, Sam Altman, CEO of OpenAI, knows this. He himself said this here and this is the reason he is traveling around the world and marketing the shit out of this hype. He knows that this is once in a lifetime opportunity and he should do whatever he can to leverage the hype and cover as much mental space as possible. The impact is, every CEO would be asking his/her executives, what is our AI/GPT strategy. This in turn will bring in business to OpenAI.
Slowly, people will realize the limitations of LLMs. That is when the reality hits. I myself have seen ChatGPT give wrong answers to simple API integration questions and the reason is simple, it is a generative model and not a fact checker. A very simple example is here - it created non-existent cases and citations.
What I think will happen is LLMs will become infrastructure building blocks similar to virtual machines or containers or cloud in general. Since each model is costly to train, requires highly skilled people and huge amount of data, very few companies will actually be doing this. Instead, they will open up their platforms and make it easy for developers to fine tune models and make production ready applications. This is the reason we are not seeing investments increasing in AI space - investors are not clear on where the moats are. A good piece on the same and how the ecosystem might unfold is written by a16z here.
I find this piece by Benedict Evans - On Disruption particularly interesting in seeing the impact of disrupting changes and how they can play out.
A few places where I see LLMs having a big impact are - 1. Chat bots fine tuned on the required context, 2. Content creation ( especially text and images - I created the logo for this newsletter using Midjourney) and 3. Education (take home assignments become a joke). Slowly, they will creep into each and everything just like Cloud or Docker.
Now lets cover some of the remaining worth reading posts I have come across in the last few months around GenerativeAI -
ChatGPT and the Imagenet moment
If I ask for ‘the chest burster scheme in Alien as directed by Wes Anderson’ and get a 92% accurate output, no-one will complain that Sigourney Weaver had a different hair style. But if I ask for some JavaScript, or a contract, I might get a ‘98% accurate’ result that looks a lot like the JavaScript I asked for, but the 2% error might break the whole thing. To put this another way, some kinds of request don’t really have wrong answers, some can be roughly right, and some can only be precisely right or wrong, and cannot be ‘98% correct’.
Read full text here
We all are trying to figure out the best prompts for getting something from LLMs. This official guide makes it a little easier. (It made me think about Google Search Guide that I went through a long time back to understand the best way of searching. Times are changing)
Read full text here
How AI could save (not destroy) education
A good short video on possibility GenerativeAI brings in education. Made me wonder that education might not be the same in the next 20 years.
Watch here
Ending with a quote:
The biggest generator of long term results is learning to do things when you don't feel like doing them. Discipline is more reliable than motivation.
We hear from people who are excited, we hear from people who are concerned, we hear from people who feel both those emotions at once. And honestly, that's how we feel. Above all, it feels like we're entering an historic period right now where we as a world are going to define a technology that will be so important for our society going forward. And I believe that we can manage this for good.