Some Fun Projects
Stuff I’m excited about
These are a handful of toy projects that I’m working on in the next 12 months. This is a brand new website, so I’ll gradually post more content about them as I get around to it. Most of these mini projects use some form of AI because I’m curious to learn more about AI methods and tools.
A universal lexicon
What useful, fun, or interesting words exist in other languages but not in English?
Since the dawn of the Internet, people have published lists of words like this.
We think in language, and so the limits of our vocabulary create boundaries on our ideas. I don’t have a strong desire to learn entire other languages right now, but I’m enthusiastic to expand the universe of words I can pull from any language.
One approach for finding useful words is to get a bunch of bilingual (English+) speakers together and to ask them what is missing in English. But this is difficult to do comprehensively without a large time commitment by others.
Now, LLMs have produced a beautiful solution to this problem through text embeddings. We can simply locate words and phrases in other languages that are “distant” from English words in certain relevant dimensions and forms. The specifics of making it work are tricky but very doable.
Once we have relevant lists, I think there are a bunch of other minor challenges:
- Provide definitions and grammar (for an English speaker) and examples of words in context to make this practically useful
- Organize the words in a thesaurus-like book
- As with all of these projects, open it up for others to contribute!
AI debates
I recently hosted a live and in-person (for the human audience) spoken debate between OpenAI’s GPT-4 Turbo and Google’s Gemini Pro on the topic: “Standardized testing should be abolished.”
OpenAI was randomly assigned the affirmative position and Gemini was assigned the negative position. The debate had the following format.
- Opening Statements (2 minutes each)
- Arguments (3 minutes)
- Rebuttal (3 minutes)
- Conclusion (2 minutes)
After the debate, the audience and the AIs judged the winner.
These debates have a variety of fun technical problems: (1) How do we improve the responses AIs provide? For instance, they can search Internet references or use RAGs to retrieve and reference useful material? (2) How do we turn AI responses into spoken audio? (text-to-speech) (3) How do we visualize the debate audio?
My current implementation of these is extremely rough - just enough to run a debate that people truly enjoyed.
Trends in aesthetics
We love things that are slightly different in style from products we already know. This influences our preferences in every art form: clothing, music, food, software, etc.
The corollary is also true. We dislike things that are too different. Rock music would not have been welcome in the 1700s.
So if we want to create something that people will love - aesthetically - we usually have three options:
Incremental Change: We can slowly push the limits of aesthetic preferences in new directions.
Reversion: We can bring back a familiar style that is no longer current. This leads to well-known loops in aesthetics that often swing backward by 20 to 50 years.
Combination: We can take aesthetics that people already appreciate in one sphere and apply them to another. Styles in clothing might be extended to rugs, upholstery, etc.
The fascinating thing about incremental trends in aesthetics is that you might be able to see where they are going without being able to create products that jump ahead to wherever they will be. People still have to gradually adjust their preferences.
The frequency of cuts in movies will get faster or slower. Songs will get louder or quieter. Foods in some cuisines might get spicier. But audiences today are not ready for the preference that will follow 10 years from now. (These are not new observations, of course!)
As a challenge for neural nets, I’d like to look at trends in color palettes and patterns in commercial clothing. I’ve been capturing images of clothes from hundreds of design clothing companies’ archived websites - and I’ll be curious to see what the patterns have been in the past, and see how well I can predict them in the future.
This would be possible to do with color and shape, but to keep is simple, I’ll start with color.
General purpose AI chess
Since IBM Deep Blue in the 1980s (and, of course, earlier), chess has been a fun proving ground for computer problem solving.
I’ve been testing the ability of OpenAI’s GPT 4.0 to play chess because it isn’t something that naturally follows from the “provide the best next text” emphasis that was used to train these generalist AI systems. They surely pick up chess data during training, but playing chess well is still a specialized problem, requiring memory, tactics, and strategy that we think of as a distinct area of expertise.
The other great thing about chess is that we have strong chess engines like Stockfish that test the limits of AI systems and handily beat them.
GPT 4.0 was pretty impressive against Stockfish (~1900 rating). Over time, I’ll track the improvements of AI chess, pit different AIs against each other, and explore their ability to reason about their own chess moves.