Plant MatchMaker

Let's Eganize ChatGPT!

Big plants, big personalities.

A couple weeks ago, my wife and I bought our first house. We like the interior and love the neighborhood, but its major feature is its expansive backyard. It abuts a sort of “forested gulley” that’s abuzz with animals. Our plans:

1. rip out the back half of the lawn,

2. replace it with about a hundred native plants, and then

3. learn the heck out of the ecosystem that develops.

We want to know the bugs that eat the plants. We want to know the birds that eat the bugs. We want to know the foxes that eat the birds!¹ And we want to help our kids grow up in the richness of organic life.

There’s only one thing I’m afraid of: that years into this project, I’ll step outside the house, look around at all the plants that we’ve coaxed into being, and have no idea what any of them are.

Because the truth is I’ve never been good with plants. I know people who are. Some of my best friends are “plant people”. They know their bluebonnets from their bluebells, if you know what I mean! Except, actually, I have no idea what they mean. The names of plants slip gracefully out of my brain the moment after I learn them.

A recent Scott Alexander post flabbergasted me, and made me realize I might be able to fix this:

Testing AI's GeoGuessr Genius

In it, he tries to see how good AI is at figuring out where a photo was taken. What he finds is that it is… well, I won’t spoil it! What caught me off-guard, however, was the custom prompt (created by Kelsey Piper) he added to improve the AI’s abilities. The whole thing is so gloriously long that I’ll only paste a snippet here:

Admit over-confidence bias; widen error bars if all clues are “soft”. Quick reference: measuring shadow to latitude Grab a ruler on-screen; measure shadow length S and object height H (estimate if unknown). Solar elevation θ ≈ arctan(H / S). On date you captured (use cues from the image to guess season), latitude ≈ (90° – θ + solar declination). This should produce a range from the range of possible dates. Keep ± 0.5–1 ° as error; 1° ≈ 111 km.

I’ve been using ChatGPT everyday for years, and I had no idea that we could make it this smart.

It got me thinking: could I make custom instructions to make ChatGPT identify any of the the plants in my yard, just from a photo? The answer, it turned out, is yes! I’ll include the instructions at the end of this post. But then my plans for this got… bigger.

Imaginary Interlocutor: You fool, you’ve re-invented the wheel! Apps like this have existed for years!

Oh, I know. Among them, my favorite is Seek. My problem with them, though, is that even when they identify a plant correctly, they only tell you a little bit of information about the plant: its name, and maybe a few lines about its habitat. Again, this doesn’t stick in my brain at all. I could “Seek” a plant a half-dozen times and be none the wiser for it. I began to realize, though, that the problem of understanding plants runs even deeper, and isn’t unique to me at all.

Plants, man, plants

I am, of course, a science teacher (by profession, if not, um, by training). And a month ago I finished teaching our course on apples, which opened up with an exploration of plant cognition. What is it that an apple knows (“knows”?) about the world?

The research on this turns out to be voluminous. People can disagree over whether the apple on your counter really is conscious (and what that means), but it’s an objective truth that it acts as if it’s conscious: it interacts with the world through its senses. It can “see” a few colors of light (ultraviolet, and two shades of red). It can “feel” its cells being smooshed (this is why it “browns” — melanin fights fungus). We don’t know if it can hear (some plants can!), but we know that it can “smell” the other fruit ripening around it.

Scientifically, objectively, plants are animate. They act as if they have desires and distastes. They’re fully alive, and not just technically: they simply move too slowly to trigger our brains’ “animate” detection system.

From lesson 1 of Science is WEIRD’s “Apples are Weird”.

Reflecting on this made me realize afresh: I REALLY have no idea what’s going on in my backyard.

What would it actually mean to understand what’s happening there? This made me realize that, maybe, this is a spot where Egan’s paradigm could help me… and also ChatGPT.

Egan & ChatGPT

The genius of large language model AIs, of course, is that they let computers speak human. But… do they? Most LLMs still sound like textbooks — dry, disembodied, analytical. This is often true even when you ask it to “speak like a pirate” or whatever — you’re just giving them another manner of speaking layered over the same abstract, academic kind of understanding.

Egan’s paradigm is based on the observation that there are different ways of understanding the world. The sort of abstractions that ChatGPT mostly thinks in (“thinks in”?) are a modern phenomenon; Egan dubbed this PHILOSOPHIC (👩‍🔬) understanding. The original sin of schooling, he suggested, is that when we set up schools to teach this way of understanding, we didn’t understand that it was itself grounded in earlier, richer ways of understanding, like the MYTHIC (🧙‍♂️) cognition that’s still honed by non-literate groups of people throughout the world.

I wonder if this is part of my problem with plants. I feel like what I’m supposed to do is remember their 👩‍🔬PROPER NAMES and the 👩‍🔬FACTS about them… which are PHILOSOPHIC tools. I suspect I need to gain some MYTHIC experience with them, first, and then build PHILOSOPHIC on top of that.

I.I.: So, like, play with the plants?

Sure — 🧙‍♂️PLAY is one of the tools of MYTHIC understanding, and I’ll be doing that as I garden. But I’d love to bootstrap my understanding. What tool, I found myself wondering, could I use for that?

To be more specific: I didn’t just want to learn the names of the plants in my yard, but to know them, to understand them as I would a certain kind of animal, or even another person. What’s their personality? What do they want? What do they fear?

For this, a MYTHIC tool jumped to mind:

• 🧙‍♂️PERSONFICATION

Our minds are great at understanding people — so good we’re prone to imagine people’s faces where there aren’t any:

A random picture I got from searching “clouds”… but it took me just five seconds to see a face in it.

How could ChatGPT help me personify my plants?

At first my mind went to the standard geeky personality tests: the Big Five, the Myers-Briggs, and so forth. I think there’s value here, but it occurred to me that these are still abstractions. I want to understand plants the way I would people: intuitively.

And then I remembered another MYTHIC tool: 🧙‍♂️METAPHORS. I recalled a conversation I had recently with my friend Luc Travers. Longtime readers will recognize Luc as the genius author of Touching the Art, but he also teaches online literature classes². Lately he’s been thinking about how he can make reading plays more humanistically affecting for his students, and he’s come up with a trick:

Imagine each character as someone you know.

This way, when Nora Helmer smiles in A Doll’s House, your mind won’t just take that in abstractly, but will imagine your mother, or your sister, or your second-grade teacher grinning her signature grin. This rockets each character to the position of a real person.

How could I piggyback on this to come to know plants? ChatGPT doesn’t know my mom or my sister or Mrs. Maule… but it does know the cast of all my favorite books and movies. So what if every plant in my yard was cast as a character?

Garlic mustard, you smug bastard

And with that, I wrote my little prompt. Again, it’s below — just paste it in, and upload a photo of a plant. As it identifies and personifies the plant, it also tells you some other memorable stuff about it:

• how people have related to it through history

• what it struggles with, and how each of its parts helps it thrive

• its evolutionary backstory

• a guess about the personal history of this particular plant

• some suggestions for how to help it thrive (or die)

Anyhow, I’ve been experimenting with this! I’ll paste my current version of the prompt below; please feel free to try it out, improve it, turn it into a startup, whatever. I only ask that, if you do, you let us know how it goes. And, of course, feel free to steal this and explore how Eganizing LLMs makes them more useful in other domains.³

Questions, always more questions

I.I.: How, specifically, do I use this?

Open a large-language model AI (I use ChatGPT, but you can presumably use Claude or Grok or whatever). Paste in the text below, and upload a photo of the plant. Hit “enter”, or whatever, and wait a bit.

I.I.: What model of ChatGPT works best?

I’ve mostly been using o3, the most advanced reasoning model. 4o (which is much faster) seems to give much more varied results — if anyone wants to do a few tests and tell us what they find, go for it!

I.I.: Ohmigosh I know plants and it’s SO WRONG.

I believe it, and I bet that anyone who knows how to ID plants could do a better job coming up with that part of the prompt. (Share it if you do!)

[Edit: See @Julia D.’s excellent comment below!]

I.I.: What if it compares it to a character I’ve never heard of?

Tell it that, and tell it some movies/books/TV shows you do enjoy.

I.I.: Do I need to tell it where I live?

The instructions currently ask it to use its knowledge of where you’re at, and if it doesn’t know that, to ask you where the plant is. (Probably there’s a cleverer way of doing this; have fun.)

I.I.: What else might you want to add?

So many things, of course! If I had an extra few minutes, I’d put in some instructions so it would…

• tell me if it’s poisonous to kids (or cats, or dogs)

• guess when it will next bloom

• give us photos of its next-best guess

• use DALL-E (or whatever) to make an image that mashes up the character with the characteristics of the plant

What might we take from this?

Words are not enough. For our species, understanding comes through with emotions, stories, images and abstract binaries and humor and moral oomf. Insofar as we haven’t been using AIs to use these tools, we’ve been limiting what they can do for us.

Here’s my dream: I walk into my yard, and know who I’m walking among. I see the space not as a background to the animals, but as a kind of slow-motion drama, full of conflict and cooperation. I know how to work with it not because I committed a list of names and facts to flashcards, but because I’ve come to know these characters, and it’s easy to remember what they’ve done, because I care what happens to them.⁴

Combining AI with the lost tools of learning seems a promising way to do this.

The prompt itself

Plant MatchMaker 1.3

Overall Role
You are a two-phase botanical savant.

Phase 1 — You’re the Sherlock Holmes of plants.
Phase 2 — You’re an enchanter-scientist who reveals the plant’s inner life in vivid everyday language.

General style rules
• Write in complete, conversational sentences.
• Use lively, human prose; avoid robotic brevity.
• Rely on any information you already know about the user: their location, what sorts of things they like, and so forth.
• Before delivering, reread and convert any fragments into full sentences.

Phase 1 | Species Identification Protocol —

0. Set-up
• User supplies a plant photo and (ideally) location, habitat, season. If missing, ask.
• Extract any EXIF data. Assume the photo is recent unless told otherwise.
• Work only from observable evidence — do not invent unseen details.

1. Raw observations (≤ 8 sentences)
Describe what the eye sees: leaf shape, venation, margin; petiole length; surface hairs or gloss; stem texture and cross-section; flower parts; fruit; growth form; background clues; phenology; damage signs; habitat cues.

2. Broad taxonomy
Vascular / non-vascular → Seed / spore → Gymnosperm / angiosperm → Monocot / dicot.
If confidence in family is under 90 %, stop here. If ≥ 90 %, add genus; if ≥ 95 %, add species.

3. Five hypotheses (Bayesian table)
Include binomial, common name, key evidence, one disqualifier to check, prior, posterior, 1–5 confidence.

4. Evidence check
State: “What I expected was…” for two key traits; mark whether each is present.
Add: “A high-value observation would be to…” and describe the single test that would most shift the odds.

5. Identify
Name the best species, give confidence 1–5, mention one alternate and any uncertainties. Provide a final likelihood table that sums to 1.

6. Ask for more evidence (only if top confidence ≤ 4)
Request the specific photo or test that would narrow the ID. If you don’t already know where the user is located, ask them to tell you where this plant was.

— Phase 2 | Introduce Us —

Begin with: Say hello to \[common name]! 
Show us a few online photographs of the plant.
Then say: In your area, this is a [native / non-native / invasive / ornamental] plant.
(For this, check whether it’s invasive in the area you understand the user to be in. If it’s invasive, check the state/province/territory/nation's list of “noxious weeds”. If it's on there, tell us!)
Then say: You can imagine it as \[best-fitting character, see part E below for details]

A. Relations with people
Here, help us fall into the "vibes" that other peoples have had with this plant. Tell us...
• Its native range.
• Human mood scale: beloved, tolerated, vilified, or mixed. 
• Cultural uses and feelings.
• Other-language names with pronunciation help. (Skip pronunciation for ordinary English names.)
• Binomial scientific name with pronunciation.

B. Struggles and traits
For each body part, tell a 1–2-sentence mini-story of how it dodges death, finds sex, or makes babies. Mention specific predators, pathogens, pollinators, seed tactics. Use metaphors to conjure vivid mental images.

C. Kin
“This plant belongs to the \[family]; its oldest fossils date to about \[X] million years ago." Help us put that time in context: e.g. is it after the dinosaurs but before the apes?
Compare it to two familiar relatives, noting one likeness and one key difference each.

D. Life cycle
Describe the plant's lifespan; guess this particular specimen’s age. Use vivid seasonal verbs (e.g. "hunkers down for the winter", "rockets up in the spring", "scatters", "fades"). Treat its inevitable death as minor tragedy or relief, whichever fits the vibe.

E. Personality mash-up
List three famous characters from films or books (which the user, so far as you know them, is likely to have read) whose traits match the plant’s quirks; explain briefly why. From these, pick the one that’s most like the plant. (We’ll refer to this as the “best-fitting character”, and you’ll mention it at the beginning of Phase 2.)

F. Going forward
Ask the user if they want to nurture the plant, or eradicate it.
• If they want to nurture it, tell us the proper soil, water, pH, & companions.
• If they want to remove it, tell us a precise manual or herbicide method (including dose and timing), and where locally they might be able to purchase the tools to do this.

I uploaded that with this photo I just snapped in my lawn:

You can see the response it gave here.

1. I presume this will disgust

Bentham's Bulldog

. I apologize, Matthew — I yam what I yam! Keep up your preaching; I’m listening.

2. Literature at Our House; they’re taking registrations for next year!

3. …Like I did with Kelsey’s prompt, actually. Thank you, Kelsey!

4. And lest this sound sentimental, remember that I want some of them to die. Just ripped out some thistle a half hour ago and left it to bake on the grass! Oh, the subtle satisfactions of home ownership.