I tried it on a photo of what I'm positive is either chalk maple or the very closely related southern sugar maple. The photo was excellent for ID: seeds, leaf backsides (a distinguishing factor between those two), etc. I told it when and where the photo was taken.
Claude thought it was 85% likely red maple, a definite mistake; 10% likely northern sugar maple, slightly more plausible; and only a 5% chance of another kind of maple.
iNaturalist, my go-to plant ID app, is overall shockingly good: better than I am, and I'm the best plant ID person I know in real life. It thinks my photo is southern sugar maple, with a second guess of chalk maple; I personally think those two should be reversed.
I don't think I can come up with a better prompt than yours. I think Claude's knowledge base is just lacking. I've even asked Claude how to help me distinguish between those two species before. It tells me something and then will switch the characteristics later, of course apologizing for its mistake.
Your prompt is great for most people, though so is Seek or the full iNaturalist app. I just gave Claude a moderately tricky problem, though I was surprised it was as wrong as it was. Perhaps ChatGPT is better for this.
Ha! Thanks for calling attention to this, and I'm updating on how accurate LLMs are on this — though I'll say that I used ChatGPT rather than Claude, so it's still an open question as to whether one is better than the other. (The difference between those, however, presumably is dwarfed by the difference in our respective abilities to CHECK them — with you far, far in the lead.) In retrospect, it should be more surprising to me that an LLM (trained on text) is even as good as it is in identifying plants.
And I will emphasize that, at least insofar as my knowledge of plants has made me able to check this, I AM impressed. That it can tell that something is a maple or blackberry, for example, is now more surprising than before.
In any case, the thing I'm more hopeful about is an LLM's ability to humanize my understanding of the plants in my yard — so maybe the best bet (at present levels of technology) would be to "Seek" my plants, and then to copy that list into the LLM. Any thoughts on that?
Yep, that's what I'd do. Seek/iNaturalist for ID and then LLM for elaboration. I do like your overall mission for this project.
I agree LLMs seem like they'd be inherently limited in plant ID. There are a lot of ID characteristics that I rely on that I haven't ever heard described in words, much less text. For example, the relative degree to which a leaf is glossy or transparent, how thick the edges are, how much the surface bumps up between veins, the pattern of veins, how it smells when you crush the leaves, how the leaves flutter in the wind, the branching pattern, the bark texture, etc. Some of those have some words to describe them, but not very detailed ones. It's like trying to describe a face. There's a joke in plant communities about trying to explain your ID of a plant by summarizing the gestalt as "because of the way it is." I don't know what kind of AI iNaturalist uses, but it seems to be much closer to what I do.
Thanks! That makes me reflect on how surprising it is that current LLMs get *anything* about our world correct, given that their only model of the world is through our language. (Max Bennett, in his excellent A Brief History of Intelligence, comments that this is evidence of just how good language is at modeling the world.) I've updated the post to point to your comment.
I am preemptively mourning how LLMs will result in the loss of unwritten, embodied knowledge and the devaluation of already illegible and marginalized experiences.
Maybe see if there's a way to encourage that kind of connection in your prompt?
And to be honest, the output from ChatGPT still seems pretty dry and academic to me. It's trying to follow your instructions and be creative, but somehow that can't make up for its lack of a soul.
You might get better results from searching for connections in children's books. My three-year-old's favorite book at the moment is Owl Moon, which does a superb job of capturing the wonder of going owling.
What an excellent idea! This seems like it would go really well with A Walking Curriculum.
I like how you frame mythic understanding (or even romantic) as a useful precursor to philosophic. I think I sometimes forget that not everyone implicitly sees everything in the world as wonderful and philosophic detail can feel empty without employing some of the older tools to humanize the topic.
I stumbled across a book of Native American myths while waiting for a friend in a library, and they sure personified plants as well as animals. Can't remember the details but at one point in the Creation, all plants were asked to go to sleep for a week. The trees that managed this keep their leaves all year, those that woke up early drop them in winter and have to regrow them the following year. Because there were so many different cultures with their own stories, you even get different takes on the same plant.
You could probably get something out of the Greek myths too - Narcissus comes to mind.
I was also impressed by how trees spread across the British Isles since the end of the last ice age, based on a book by Oliver Rackham who was kind of the greatest academic authority on the matter. If you speed up time a lot and draw maps, you can see different species colonising the "Wild West" (expect there's no natives to fight), forming their own nations, and occasionally starting a war and taking territory and pushing the loser into more marginal territory. For example, Beech is the pioneer and homesteader, and to this day the first tree to appear if you leave some farmland empty and there's any of its pollen not too far away. But it declines as soon as more "civilised" cultures take over.
Ohmigosh I know plants and it’s SO WRONG.
I tried it on a photo of what I'm positive is either chalk maple or the very closely related southern sugar maple. The photo was excellent for ID: seeds, leaf backsides (a distinguishing factor between those two), etc. I told it when and where the photo was taken.
Claude thought it was 85% likely red maple, a definite mistake; 10% likely northern sugar maple, slightly more plausible; and only a 5% chance of another kind of maple.
iNaturalist, my go-to plant ID app, is overall shockingly good: better than I am, and I'm the best plant ID person I know in real life. It thinks my photo is southern sugar maple, with a second guess of chalk maple; I personally think those two should be reversed.
I don't think I can come up with a better prompt than yours. I think Claude's knowledge base is just lacking. I've even asked Claude how to help me distinguish between those two species before. It tells me something and then will switch the characteristics later, of course apologizing for its mistake.
Your prompt is great for most people, though so is Seek or the full iNaturalist app. I just gave Claude a moderately tricky problem, though I was surprised it was as wrong as it was. Perhaps ChatGPT is better for this.
Ha! Thanks for calling attention to this, and I'm updating on how accurate LLMs are on this — though I'll say that I used ChatGPT rather than Claude, so it's still an open question as to whether one is better than the other. (The difference between those, however, presumably is dwarfed by the difference in our respective abilities to CHECK them — with you far, far in the lead.) In retrospect, it should be more surprising to me that an LLM (trained on text) is even as good as it is in identifying plants.
And I will emphasize that, at least insofar as my knowledge of plants has made me able to check this, I AM impressed. That it can tell that something is a maple or blackberry, for example, is now more surprising than before.
In any case, the thing I'm more hopeful about is an LLM's ability to humanize my understanding of the plants in my yard — so maybe the best bet (at present levels of technology) would be to "Seek" my plants, and then to copy that list into the LLM. Any thoughts on that?
Yep, that's what I'd do. Seek/iNaturalist for ID and then LLM for elaboration. I do like your overall mission for this project.
I agree LLMs seem like they'd be inherently limited in plant ID. There are a lot of ID characteristics that I rely on that I haven't ever heard described in words, much less text. For example, the relative degree to which a leaf is glossy or transparent, how thick the edges are, how much the surface bumps up between veins, the pattern of veins, how it smells when you crush the leaves, how the leaves flutter in the wind, the branching pattern, the bark texture, etc. Some of those have some words to describe them, but not very detailed ones. It's like trying to describe a face. There's a joke in plant communities about trying to explain your ID of a plant by summarizing the gestalt as "because of the way it is." I don't know what kind of AI iNaturalist uses, but it seems to be much closer to what I do.
Thanks! That makes me reflect on how surprising it is that current LLMs get *anything* about our world correct, given that their only model of the world is through our language. (Max Bennett, in his excellent A Brief History of Intelligence, comments that this is evidence of just how good language is at modeling the world.) I've updated the post to point to your comment.
I am preemptively mourning how LLMs will result in the loss of unwritten, embodied knowledge and the devaluation of already illegible and marginalized experiences.
I was curious what stories I could find about purple coneflowers already, and it seems like ChatGPT missed something that's actually FIRE: https://www.nytimes.com/2020/01/27/science/fire-coneflowers-echinacea-pollination.html
Maybe see if there's a way to encourage that kind of connection in your prompt?
And to be honest, the output from ChatGPT still seems pretty dry and academic to me. It's trying to follow your instructions and be creative, but somehow that can't make up for its lack of a soul.
You might get better results from searching for connections in children's books. My three-year-old's favorite book at the moment is Owl Moon, which does a superb job of capturing the wonder of going owling.
It's harder to find books like that about plants than about animals, but we like The Lost Words (https://a.co/d/gGXdUbA) and A Seed Is Sleepy (https://a.co/d/1JtMrRE).
Being able to identify flowers before they bloom like this is quite impressive and useful. Very cool!
What an excellent idea! This seems like it would go really well with A Walking Curriculum.
I like how you frame mythic understanding (or even romantic) as a useful precursor to philosophic. I think I sometimes forget that not everyone implicitly sees everything in the world as wonderful and philosophic detail can feel empty without employing some of the older tools to humanize the topic.
I stumbled across a book of Native American myths while waiting for a friend in a library, and they sure personified plants as well as animals. Can't remember the details but at one point in the Creation, all plants were asked to go to sleep for a week. The trees that managed this keep their leaves all year, those that woke up early drop them in winter and have to regrow them the following year. Because there were so many different cultures with their own stories, you even get different takes on the same plant.
You could probably get something out of the Greek myths too - Narcissus comes to mind.
I was also impressed by how trees spread across the British Isles since the end of the last ice age, based on a book by Oliver Rackham who was kind of the greatest academic authority on the matter. If you speed up time a lot and draw maps, you can see different species colonising the "Wild West" (expect there's no natives to fight), forming their own nations, and occasionally starting a war and taking territory and pushing the loser into more marginal territory. For example, Beech is the pioneer and homesteader, and to this day the first tree to appear if you leave some farmland empty and there's any of its pollen not too far away. But it declines as soon as more "civilised" cultures take over.
This is brilliant. I did this something like this once with geopolitics:
https://radicalcentrism.org/2025/01/25/autonomy-anonymous-how-thailand-bhutan-and-nepal-stayed-single/
but I never considered automating it as way to understand species in an ecosystem. I wonder what else I could apply it to…