Wildlife Discovery App

Overview

My wife and I have started taking a walk every day. It’s a way to get some sunshine, unwind after work, and get a little exercise while we catch up on each other’s days. It’s become my favorite part of the day, steep inclines aside.

We see a lot along the way. Mostly birds, some lizards, and the occasional skunk or coyote, though only after dark. My best friend and I have a running joke, “what kind of dog is that?”, and somewhere along the way it turned into a question I kept asking about everything else: what kind of ___ is that?

I wanted something that would answer it. I also wanted it to be fun, so the app is a collection too. Every animal you identify gets saved, and over time you can look back through the photos you’ve taken. It doesn’t have a name yet, so for now it’s the Wildlife Discovery App.

Here’s how it works. You open the app and it’s already a camera. You photograph an animal, the app tells you what it is, and if you agree, the sighting goes into a collection of everything you’ve found. I think of it as a field guide you fill in yourself.

It’s meant for someone who’s curious about what they just saw, not for experts. Two choices shape most of the rest. Photos have to come from the in-app camera, with no importing from your library, because I’d rather the app nudge people to go outside and look. And it starts with a curated list of 40 Southern California species instead of trying to know every animal alive. A small list I can test properly beats a huge one I can’t.

I’m learning a lot building it: a React Native app, a Python backend, PostgreSQL, and how to evaluate an AI system without fooling myself, which turns out to be a surprising amount of the work.

The project starts with one question: can the identification be trusted? Everything below comes from chasing that.

What it does

The first version is one small loop:

1. Launch straight into the camera.
2. Take a photo.
3. The app returns a species, a confidence label (Confident, Likely, or Not sure), and a couple of alternates.
4. You confirm it, pick an alternate, or say none of these. Nothing is saved without you.
5. The sighting is saved and the species appears in your collection.

A collection screen, species details, and a map of where you’ve seen things come after that loop works.

Technology and architecture

Testing the identification first

An app like this lives on one promise: when it names an animal, you can believe it, and when it doesn’t know, it says so. A wrong answer delivered with confidence is worse than “Not sure.”

So I built a test. It has 763 photos: 632 of the 40 species, and 131 of things that aren’t on the list. That second group matters as much as the first. It includes close relatives, animals from other parts of the country (I used North Carolina wildlife on purpose), pets, and objects. A photo from somewhere the app doesn’t cover should end in “Not sure,” not a forced guess.

The photos come from iNaturalist, using only permissively licensed ones, checked one at a time. I split them by photographer rather than by photo, so one person’s pictures never end up on both sides of the test. And the test photos were uploaded after the models’ training data was gathered, which makes it unlikely they’d seen them (it doesn’t rule it out). A small Python tool runs each approach over the same photos and reports the same numbers every time.

Why 40 species, and not more or fewer

I considered keeping the 40, dropping the species that get confused most, or adding more. I kept the 40.

Size wasn’t the problem. The right species was in the top three guesses for every listed photo, and the top guess was right 98.9% of the time. The seven misses were three pairs of lookalikes: crows and ravens, two kinds of hawk, two kinds of egret. Dropping those species would make the numbers look better and the app worse, since they’re exactly the common animals people will photograph. Adding species has its own catch: every new one brings new lookalikes, and nothing I’ve measured covers more than 40.

The real difficulty is at the edge of the list. A rabbit that isn’t on it looks a lot like the rabbit that is. That’s where the next problem comes from.

Teaching it to say “none of these”

The first version of “Not sure” didn’t work. The model’s confidence percentages always add up to 100% across the 40 species, so an animal that isn’t on the list still looks like a confident match for its closest relative. A raw similarity score, which measures whether a photo looks like any listed species at all, fixed most of it. It caught 88.5% of the animals that don’t belong, up from 82.4%, without sending a single listed photo to “Not sure.”

The full story, including the dead ends, is in Teaching an AI to say “none of the above”.

One AI or two

I tested five approaches, but the interesting comparison is two of them.

Option C asks a general-purpose AI model what’s in the photo, and that’s it. It’s simple and runs on the smallest server. Option E is a hybrid. A specialized vision model that runs on my own machine makes the call, and the general model only gets involved when the photo is ambiguous, to check the answer.

	C: general AI alone	E: hybrid
Correct species in the top three	98.7%	100%
Photos it’s Confident about	39.7%	82.3%
Accuracy on those Confident photos	96.7%	97.6%
Listed species ending in “Not sure”	49.4%	1.9%
Not-on-list photos ending in “Not sure”	89.3%	93.1%
Server it needs	the smallest	at least 2 GB of memory

C isn’t bad. Its answers are usually in the top three. The trouble is that its confidence scores bunch together, mostly between 80 and 97, so any cutoff for “Not sure” throws away a lot of right answers. About half of the photos of listed animals ended up there.

I also tried letting the general model choose among the specialized model’s top five guesses, and it made things worse: it overruled 38 right answers and fixed one. So in the hybrid, the specialized model always names the animal, and the general model only gets a veto on the close calls. The longer story is in Giving my smartest AI a veto, not a vote.

The cost of E is the last row of the table. The specialized model has to live inside the server, which means a bigger machine and a bigger monthly bill. So the plan is to build for E and develop locally, where hosting is free, and settle the hosting question before other people try the app. If a 2 GB host doesn’t work out, the app falls back to C. That’s a configuration switch, not a rebuild, so the choice stays open.

The rest of the stack

The stack is React Native with TypeScript for the app, mostly because I already know React and can test on my own iPhone, and a single FastAPI service in Python for the backend, since Python is something I want to continue learning. PostgreSQL stores the data.

The backend design is deliberately boring, though it’s still a proposal and not a final decision. It’s one service instead of several, and scans are answered in a single request instead of a background job queue. A queue would add a broker, a worker, and a new way to fail, all to serve a handful of testers. The result is also safe to retry, so a flaky connection can’t make the same photo get identified twice.

Screenshots

I’ll add screenshots once I’ve created the app UI.

Development history

1. Nov 4, 2026 Chose the hybrid design for now, with the general model alone as the fallback. Kept the species list at 40. Wrote both decisions up as records, along with what I learned about “none of these.”
2. Nov 3, 2026 Ran the specialized models (including a bigger one), a setup where the general model picks among their top guesses, and the hybrid, all against the same test photos.
3. Nov 2, 2026 Froze the test set and scored the first approach, the general AI model on its own.
4. Oct 29, 2026 Approved the 40-species list. Two species got swapped out because I couldn’t find enough good photos of them (Bobcat and Blainville’s Horned Lizard, replaced by Vermilion Flycatcher and California Thrasher).
5. Oct 25, 2026 Wrote the plan: milestones, a long list of things I’m not building yet, and the rule that the testing comes before the app.

Lessons learned

• Measure what a score actually separates before you build on it. The percentages looked like confidence, and they weren’t. Looking at saved results instead of building something new moved “Not sure” from 82% to 88.5%.
• Different models are good at different jobs. The specialized model is the better namer. The general model is the better skeptic. Asking either to do the other’s job made things worse.
• Test the edges, not just the middle. Almost all the interesting failures were close relatives of listed species. A test with only listed animals would have looked great and told me very little.

Status and what’s next

Status: in development. The decision about how identification works is made. Next up is the backend, then a bare-bones app on my iPhone, then camera to identification end to end.

Things I’m still unsure about:

• Hosting. The hybrid needs a bigger server, and I haven’t picked one.
• Lookalikes. Nine of the 131 not-on-list photos still get through as Confident, all close relatives of listed species, like a different kind of rabbit. I have ideas for catching them, but I haven’t confirmed that any of them work.
• Seasons and real photos. The test photos were taken between July and October, and iNaturalist photos tend to be better than what people take on a hike. Winter and spring, and my own photos, are still untested.

Still in progress. I update this page as I make more discoveries.