Slow and Small AI: Somehow we have to put back the jin in the bottle
In Islamic mythology, jin are supernatural other worldly beings. They were once under the control of king Solomon, and legend has it that they never stopped working until creates of the earth ate his staff which caused the now dead king to topple from his thrown.
At that moment, Pandora's box was opened, so to speak. The jin became free*. Yes yes what a labored metaphor. But here we are, in the post-genAI area with large language models. But fuck them. I don't want LARGE models. I want small, boutique models that do one thing very well.
I work with a ton of dirty data filled to the brim with quirky errors, most fields are free text, what's a phone number? nah let's have 3, oh you like country code too? This person doesn't have a last name, unique identifier per patient? haha. No no merging data based on multiple fields is is too hard for our senior data analyst. Changing formats, changing database fields. Yeah you name it. Now you'd think after reading this I would want an LLM to clean up my data. You would be wrong!
Ignoring the fact that I'd blow out the context window, or that legally I can not use external llm providers or host one locally or whatever else comes to mind. What I really want is a tiny language model that does just one cleaning step well. Take the example above of phone numbers, sometimes for a variety of reasons the field has multiple phone numbers, someone else's number or none. All possibilities are completely valid, an example would be: a child is admitted, of course the child has no phone number so you enter the mothers number. A child will have two parents (or more carers who need to be contacted). The list goes on.
I want a tiny LM to understand the above situations, take just one cell and decide what to do. One phone number with a country code? Remove the +, extact the country code, extract the phone number, check the number of digits is either landline or mobile. Multiple? Same as before, but for each subsequent number append to the "other_ph" field. The same for people's names, date of birth, unique id cleanup (because our system so shit and old it can not handle any non-alphanumeric characters).
Small tiny LM that does one thing. Predictable, simple[1] to use and lightweight to run.
What about slow AI? Rory Sutherland has some very interesting things to say:
we’ve sometimes allowed the urgent to drown out the important. The short-term consideration drowns out the long-term consideration.
and
There’s an extraordinary case of this bias toward time-saving, that faster must be better.
The point translates over nicely, there's a large (heh) bias towards LARGE language models. But what about small ones? Bespoke models llfor specific domains (like Meta's ESM protein language model)? When ChatGPT was first in vogue there was a cambrian explosion of fun use cases like the bot which would roast you hard.
And to be honest, all the stupid shit people are using LLMs for? Where is the cool stuff! GenAI allows us to create bizarre things, don't venture on 4chan those people are wack but at least their degen use of AI is interesting unlike the god damn 60billion dollar stock options for fucking curser - more than Illumina, ONT and PacBio combined!
Lazy and crazy. Rich people: do something interesting with all that money and LLM power.
- Not really. I've exaggerated here a little, nothing is said of the aftermath of Solomon's death, if all jin were enslaved or just the naughty ones.
[1] dataset |> lm_clean_ph(col=PHONE, extra="combine", preprompt="concat extra phone numbers using ;")