154 lines
12 KiB
Markdown
154 lines
12 KiB
Markdown
---
|
|
title: "Computer Vision Edge Cases: Why Plant Models Agree on Species But Disagree on Health"
|
|
description: "An in-depth data science and machine learning essay exploring feature extraction overlap, chlorosis vs necrosis, and hybrid multi-input models in computer vision."
|
|
tags: ["ai", "computervision", "python", "datascience"]
|
|
canonical_url: "https://greenlenspro.com/why-are-my-plant-leaves-yellow"
|
|
cover_image: "https://greenlenspro.com/images/blog/computer-vision-edge-cases.jpg"
|
|
---
|
|
|
|
# Computer Vision Edge Cases: Why Plant Models Agree on Species But Disagree on Health
|
|
|
|
In modern computer vision, image classification accuracy for botanical species identification (`pflanze erkennen`) has largely reached production maturity. With datasets like PlantNet or iNaturalist fine-tuned on ResNet or Vision Transformer (ViT) backbones, top-1 accuracy for identifying plant species routinely exceeds 95%.
|
|
|
|
However, when developers attempt to apply the exact same neural network architectures to **plant health diagnosis** (`braune blätter an pflanzen` / chlorosis / pest damage), model performance frequently collapses.
|
|
|
|
Why can a Vision Transformer instantly identify a *Ficus benjamina* leaf, yet completely fail to distinguish whether its yellowing is caused by overwatering, root rot, low light (`zimmerpflanzen mit wenig licht`), or nitrogen deficiency?
|
|
|
|
In this technical article, we'll analyze the root causes of model divergence in plant pathology. We'll explore the mathematical problem of **overlapping visual feature spaces**, examine why a single deterministic label is the wrong output shape for this problem, and walk through how a production system — GreenLens, specifically — handles this ambiguity by prompting a multimodal LLM to reason like a differential diagnosis rather than a hard classifier, backed by a two-tier model strategy for the cases that are genuinely hard to call.
|
|
|
|
---
|
|
|
|
## 1. The Overlapping Feature Problem in Plant Pathology
|
|
|
|
In taxonomic classification, species possess distinct, stable visual boundaries—such as leaf serration, venation geometry, or petal arrangement.
|
|
|
|
In contrast, plant stress responses are bottlenecked by plant physiology. Plants have a limited repertoire of visual stress signals. Chlorosis (loss of chlorophyll leading to yellow leaves) looks visually near-identical whether triggered by:
|
|
|
|
1. **Overwatering:** Oxygen-starved roots cannot uptake nutrients.
|
|
2. **Underwatering:** Dehydration causes lower leaves to drop.
|
|
3. **Low Ambient Light (`zimmerpflanzen mit wenig licht`):** Plant re-absorbs mobile nitrogen from old leaves to fuel new top growth.
|
|
4. **Nitrogen Deficiency:** Chlorophyll synthesis halts in lower foliage.
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
SubGraph1[Root Cause A: Overwatering] --> Feature[Symptom: Interveinal Chlorosis / Yellow Leaf]
|
|
SubGraph2[Root Cause B: Underwatering] --> Feature
|
|
SubGraph3[Root Cause C: Low Light Exposure] --> Feature
|
|
SubGraph4[Root Cause D: Nitrogen Starvation] --> Feature
|
|
|
|
Feature --> Model{Standard CNN Model}
|
|
Model -->|High Ambiguity| ConfusedPrediction[Incorrect Diagnosis / False Confidence]
|
|
```
|
|
|
|
Because four fundamentally different care conditions produce **overlapping RGB feature vectors**, a pure image-only model faces a mathematically ill-posed problem.
|
|
|
|
---
|
|
|
|
## 2. Analyzing Spatial & Color Feature Breakdown
|
|
|
|
Let's break down the visual features of chlorosis and tip burn (`braune spitzen pflanze`) in different color spaces:
|
|
|
|
### RGB vs. HSV Color Space Analysis
|
|
|
|
In standard RGB space, yellowing leaves display elevated Red and Green channels ($R \approx 200, G \approx 200, B \approx 50$). However, RGB channels are tightly coupled to ambient light intensity and shadow variations.
|
|
|
|
Converting image tensors to **HSV (Hue, Saturation, Value)** space isolates the true chlorophyll decay rate:
|
|
|
|
$$H = \arctan2(\sqrt{3} \cdot (G - B), 2R - G - B)$$
|
|
|
|
- **Healthy Foliage:** Hue angle $H \in [80^\circ, 140^\circ]$ (Deep Emerald Green).
|
|
- **Chlorosis / Fading:** Hue angle $H \in [45^\circ, 75^\circ]$ (Pale Yellow/Lime).
|
|
- **Necrosis / Crispy Edges (`braune blätter`):** Hue angle $H \in [10^\circ, 35^\circ]$ (Brown/Amber).
|
|
|
|
While HSV transformations help segment *where* the damage occurs, they still can't tell us *why* it occurred — and this is exactly the point at which a lot of plant-ID tooling either gives up (returns a single guess with false confidence) or, on paper, reaches for sensor fusion (soil moisture probes, light meters) that most consumer apps simply don't have access to. A phone camera has no idea how often the plant was watered last week.
|
|
|
|
---
|
|
|
|
## 3. The Actual Fix: Prompting for a Differential Diagnosis, Not a Label
|
|
|
|
GreenLens doesn't have soil moisture telemetry, and it doesn't train a fusion network. What it does have is a photo, and a general-purpose multimodal model (OpenAI's `gpt-5-mini` by default, `gpt-5` for a higher-accuracy review pass, both behind an `OPENAI_HEALTH_MODEL_CHAIN` fallback to `gpt-4.1-mini`) that already has broad world knowledge about plant care baked into its pretraining. The engineering problem isn't "how do we fuse more input modalities" — it's "how do we get the model to *admit* ambiguity instead of confidently picking a wrong single cause."
|
|
|
|
Concretely, the `/v1/health-check` prompt is designed around the overlapping-feature problem described above:
|
|
|
|
1. **It asks for multiple plausible causes with independent confidence scores**, not a single winning label. If interveinal yellowing could be overwatering, underwatering, low light, or nitrogen deficiency, a well-designed prompt gets the model to say so explicitly — "likely overwatering (based on the described leaf softness), but low light is also plausible" — rather than forcing a false single answer the way a Softmax output head would.
|
|
2. **It asks the model to point at the visual evidence that discriminates between causes.** A model that's actually reasoning (rather than pattern-matching a single label) can call out things like leaf turgor, spot pattern, distribution across the plant (all leaves vs. just older ones), which map onto real diagnostic heuristics botanists use — nitrogen deficiency shows up in older/lower leaves first because the plant is mobilizing nitrogen from them; overwatering tends to be more uniform.
|
|
3. **It leans on the model's own knowledge of species-typical vulnerabilities** instead of a hand-maintained lookup table, since that knowledge generalizes across far more species than any small team could realistically curate rules for.
|
|
4. **Genuinely ambiguous cases get escalated to the review-tier model.** When the fast-tier pass comes back with low confidence or multiple competing causes close in probability, that's a signal worth spending the extra cost of a `gpt-5` pass on — the same two-tier strategy described in the companion piece on GreenLens's diagnostic architecture.
|
|
|
|
Here's a simplified version of what that prompt-and-parse flow looks like server-side (the real implementation lives in `server/lib/openai.js`, but the shape is representative):
|
|
|
|
```javascript
|
|
// server/lib/openai.js (simplified)
|
|
const HEALTH_CHECK_SYSTEM_PROMPT = `
|
|
You are a plant health diagnostic assistant. Given a photo of a plant,
|
|
identify visible symptoms and return STRICT JSON matching this shape:
|
|
|
|
{
|
|
"species": string,
|
|
"possibleCauses": [
|
|
{ "cause": string, "confidence": number, "evidence": string }
|
|
],
|
|
"recommendedActions": string[]
|
|
}
|
|
|
|
Do not collapse ambiguous symptoms into a single cause. If multiple
|
|
causes are plausible from the image alone, list them ranked by
|
|
confidence and explain what visual evidence supports each one.
|
|
`;
|
|
|
|
async function runHealthCheck(imageBase64, { tier = 'standard' } = {}) {
|
|
const modelChain = tier === 'pro'
|
|
? [process.env.OPENAI_HEALTH_REVIEW_MODEL || 'gpt-5']
|
|
: (process.env.OPENAI_HEALTH_MODEL_CHAIN || 'gpt-5-mini,gpt-4.1-mini').split(',');
|
|
|
|
let lastError;
|
|
for (const model of modelChain) {
|
|
try {
|
|
const response = await callOpenAI({
|
|
model,
|
|
systemPrompt: HEALTH_CHECK_SYSTEM_PROMPT,
|
|
image: imageBase64,
|
|
});
|
|
const parsed = parseAndValidateHealthJson(response);
|
|
|
|
// Ambiguous fast-tier result — escalate to the review model
|
|
const topTwo = parsed.possibleCauses.slice(0, 2);
|
|
const isAmbiguous = topTwo.length === 2 &&
|
|
Math.abs(topTwo[0].confidence - topTwo[1].confidence) < 0.15;
|
|
|
|
if (isAmbiguous && model !== 'gpt-5') {
|
|
return runHealthCheck(imageBase64, { tier: 'pro' });
|
|
}
|
|
return parsed;
|
|
} catch (err) {
|
|
lastError = err; // fall through to the next model in the chain
|
|
}
|
|
}
|
|
throw lastError;
|
|
}
|
|
```
|
|
|
|
Nothing here is trained. There's no loss function, no gradient descent, no held-out validation split. The "model" in the traditional ML sense is entirely OpenAI's — what GreenLens owns is the prompt design, the JSON schema, the retry/fallback logic, and the escalation heuristic that decides when a case is worth a second, more expensive pass.
|
|
|
|
---
|
|
|
|
## 4. Why This Beats Forcing a Single Label — and Where It Still Breaks
|
|
|
|
The differential-diagnosis framing is a genuine improvement over a hard classifier for this problem, but it's worth being honest about where it still struggles:
|
|
|
|
- **It's only as good as the photo.** No amount of prompt engineering recovers information that isn't in the image — a single leaf photographed without context (no visible stem, no soil, no sense of scale) gives the model less to work with than a wider shot, and the model's confidence scores should (and generally do) reflect that.
|
|
- **It can't see root rot.** The single most common cause of chlorosis and wilting in houseplants is invisible from a leaf photo until the plant is already in serious trouble. This is a fundamental limitation of any vision-only system, not something a bigger model fixes.
|
|
- **User-supplied context helps more than better prompting.** A one-line answer to "how often do you water this?" disambiguates overwatering vs. underwatering far more reliably than any amount of visual reasoning about leaf color — which is why the most useful next step for this kind of system is usually collecting a little bit of user context, not chasing marginal gains on image analysis alone.
|
|
- **Confidence isn't free lunch.** Asking a model to express uncertainty is better than false certainty, but it also means the product has to be designed to *show* that uncertainty usefully — a ranked list of three possible causes is only helpful if the UI and the copy make clear that's not a definitive diagnosis.
|
|
|
|
---
|
|
|
|
## Summary & Key Insights
|
|
|
|
1. **Plant symptoms genuinely overlap.** Visual signals like yellowing leaves or brown tips can stem from multiple, opposing care mistakes, and no image classifier — trained or prompted — can fully resolve that ambiguity from pixels alone.
|
|
2. **A single deterministic label is the wrong output shape for this problem.** Prompting a multimodal model to return ranked, confidence-scored possible causes is a better fit than forcing a Softmax-style single answer.
|
|
3. **You don't need sensor fusion to make progress — you need honest uncertainty.** GreenLens doesn't have soil-moisture telemetry; it gets more mileage from asking the model to reason explicitly about competing causes and from escalating ambiguous cases to a higher-accuracy review pass.
|
|
4. **The real lever for disambiguation is user-supplied context**, not a bigger model. A single follow-up question about watering habits often resolves what a photo alone cannot.
|
|
|
|
To explore how AI diagnostics isolate root causes for yellowing leaves and plant stress, visit the [GreenLens Pro Symptom Guide](https://greenlenspro.com/why-are-my-plant-leaves-yellow).
|