Negative Space Is a Label

A car mask can pass review and still teach the model to keep the wrong pixels.

The outline looks clean. The bumper is inside. The wheels are inside. Then the trained network holds onto the dark patch under the tires, because the label treated that patch as part of the vehicle's visual neighborhood. Training stays quiet. Production gets loud the first time a listing photo drags a strip of the old lot onto a new backdrop.

AutoLensAI turns dealer photography into listing-ready vehicle media. This installment follows the earlier pieces on segmentation and image provenance, then narrows to one question: how do I teach a matting model that the shadow touching a tire is evidence against foreground rather than a faint version of it?

1. The failure arrives without an error message

Vehicle matting estimates which pixels belong to the vehicle, at finer boundary resolution than segmentation gives. Tires, rocker panels, glossy showroom floors, and the halo under a lowered front lip are where a pretty binary mask does its damage.

Two cases cause most of it. A cast shadow can touch rubber and still sit outside the object. A reflection can match paint color exactly and still belong to the floor. Both look like they belong to the car in a thumbnail. Neither belongs to it in geometry.

A binary target has no vocabulary for that distinction. Every pixel is in or out, so the annotator's only lever is where to put the line. Push the line outward and shadow becomes vehicle. Pull it inward and the wheel arch loses its edge. Neither answer says the thing that matters, which is that some exterior pixels are ordinary background and some are adversarial background sitting one pixel from the object.

The model learns the difference anyway. It learns it wrong, because nothing in the supervision ever separated the two.

2. Three states, not two

The supervision contract uses three:

The commit note that started this was blunt: "Hard negatives by construction: the render alpha separates a car from its own shadow." The renderer already knows which pixels came from vehicle geometry and which came from scene lighting, because it computed both. Flattening those two facts into a single soft matte discards the most useful contrast in the frame, then asks a network to rediscover it from RGB.

Hard negative mining normally means sweeping a dataset for examples the model gets wrong, scoring them, and resampling. Here there is nothing to sweep. The difficulty is a byproduct of how the image was made, so the negatives arrive addressed instead of found.

3. What counts as negative

Shadow and reflection are the obvious members. The interesting ones are the other cars.

Dealer photography happens on lots. A frame usually holds more than one vehicle, and a segmentation model asked for vehicles returns all of them, correctly. Concept segmentation is not wrong when it hands back four masks. It answered the question it was asked.

AutoLensAI settles that with a small arbitration layer before matting runs. Candidate masks are ranked by a weighted combination of normalized mask area, monocular depth proximity, and image centrality. The winner is the hero. That ranking encodes the rule a photographer follows without ever stating it: the subject is the large, near, centered one.

The arbitration has a second job. Once the hero is chosen, every losing candidate becomes known exterior. Not unknown. Not skipped. A sedan parked two spaces over is a vehicle-shaped, vehicle-colored, vehicle-textured region that is definitively not this vehicle, which makes it some of the strongest negative evidence available. A dataset that marks only shadow as negative leaves that signal on the floor.

The negative region therefore comes from three sources with three derivations:

The unknown zone is the only place the pipeline admits it does not know.

4. Encoding, and why a label has to name its parent

The exporter writes one label image plus the identity of the artifact that produced it. Pixel values carry the training meaning directly:

Generation is mechanical:

1. Threshold rendered vehicle alpha into the positive region.
2. Dilate the alpha boundary by a small radius and mark that band unknown.
3. Rasterize separated shadow and reflection pixels as negative, except where vehicle alpha already owns the pixel.
4. Mark rejected hero candidates and confirmed exterior as negative.
5. Store the result with the matte identifier, the source render, and the review decision.

Step five came from a second commit note: "The label tool could not have told you which matte it labelled, and now it cannot avoid saying."

That reads like housekeeping and is not. A reviewer approving a boundary is approving a specific artifact from a specific render at a specific moment. Store the approval against a filename and the next re-render silently inherits a judgment made about different pixels. The decision binds to the matte, or it is a guess wearing a timestamp.

5. The ignore band is a contract, not a hedge

Training reads the three values literally. Foreground loss applies where the label is 255. Background loss applies where it is 0, including every hard negative beside tires and lower panels. Pixels at 128 leave both the numerator and the denominator, in loss and in evaluation.

valid = (label != IGNORE)
target = (label == VEHICLE).float()

per_pixel = criterion(pred, target)          # no reduction
loss = (per_pixel * valid).sum() / valid.sum().clamp(min=1)

Unknown is absence with a specific meaning. It is not a weak vote for background, and treating it as one is the exact bug this design exists to prevent.

The cost is friction, and it lands on every stage that touches the file.

A binary mask previews anywhere, compresses well, and drops into any loader. A three-zone target asks for more. The exporter, the review tool, the manifest, the dataset reader, the metric code, and the visualization layer all have to hold the same definition of 128. Six places, one number.

Two failures show what happens when one of them forgets.

A preview tool that remaps 128 to black shows the reviewer a boundary that does not exist in the file. Approval then gets recorded against an image nobody looked at.

An evaluator that counts ignored pixels as correct background improves every number while the supervision underneath it gets worse. Consider what sits in that zone. Mostly edge pixels, and edges are mostly hard. Folding it into background buys a free win on the dashboard and pays for it in the product.

That failure is the worse of the two, because it looks like progress.

6. One region, two jobs

CarSegNet's refinement stage, NeuralSegJet, runs that geometry backwards.

A learned alpha refiner may modify pixels only inside an explicit uncertainty band. Outside it, the semantic prior is frozen. Inside it, a learned gate mixes the prior with the neural decoder's output. The refiner never holds authority over the whole frame.

At training time that region withholds judgment, because membership could not be proven there. Later it does the opposite job. Those pixels become the only ones a refiner is permitted to rewrite. One region, two roles, both descending from a single admission of uncertainty.

The constraint buys a specific safety property. An unconstrained refiner asked to sharpen a wheel arch is equally free to decide the vehicle has a different silhouette, since nothing stops it from moving mass far from the boundary. Freezing the prior outside the trust region lets the refiner repair an edge and forbids it from inventing topology. Mirror stalks and glass edges get their fine detail back without any freedom to grow a fender.

The trade is real. A trust region caps how much a strong refiner can fix, so a prior wrong by more than that width cannot be rescued downstream. That failure has to be corrected upstream in segmentation or hero selection instead. I accept it, because upstream errors are visible and diagnosable while a refiner rewriting object shape is neither.

7. Proving the contract survives the pipeline, without a GPU

A three-state target is only as good as the weakest stage that handles it, which makes this a testing problem more than a modeling one.

CarSegNet runs end to end on CPU with the segmenter, depth model, and background provider stubbed out. Everything else stays real: orchestration, compositing, encoding, and quality measurement.

The metrics computed in that mode are the production ones. IoU, boundary IoU, alpha error, and temporal stability across frames.

Separating model capability from pipeline correctness is what makes the ignore band testable at all. Does 128 survive the loader? Does the evaluator drop those pixels from both terms? Does compositing respect the negative region? None of those depend on weights.

They are ordinary software questions with ordinary answers, available before a GPU is warm and before any checkpoint exists.

Boundary IoU earns its seat here. Overall IoU is dominated by body pixels, so a mishandled band barely moves it.

The boundary variant measures the strip where the three states disagree. That is exactly where a folded ignore band does its damage, which makes it the number that fails loudly when someone gets this wrong. Break it on purpose in a stubbed run and the number should sink, while the overall figure stays nearly flat. If both hold steady, the test is watching the wrong thing.

8. Neighboring contracts stay separate

This discipline turns up in parts of AutoLensAI that have nothing to do with matting.

The field-resolution contract in docs/quality/04-integration-field-resolution.md separates an absent measurement from a nearby value that could pass for it. glassVehicleShare and glassCoverage are computed against different denominators, so neither percentage substitutes for the other however close they read. Matting labels obey the same rule. Positive, hard negative, and unknown answer three different questions, and they stay distinct through measurement or the distinction was decorative.

Presentation draws a different boundary, and blending it with training truth is its own hazard. docs/pitch/align_pairs.py registers before and after gallery images so a wipe slider compares framing honestly. It fits a global similarity transform against the painted-body mask using uniform scale, translation, and a bounded rotation with abs(theta) <= 0.06. It refuses shear, local warping, and large viewpoint change, because those reshape the thing being compared instead of aligning it.

That script exists so a viewer can trust a slider. It holds no opinion about what the model should learn and it never writes a label. Keeping the two jobs in separate files is not tidiness, it is the reason a demo cannot quietly become a dataset.

9. What review looks like now

The reviewer's question changed, and that is the practical result of everything above.

The old question was whether the silhouette looks clean. The new one is whether every object-adjacent pixel carries the right role: keep, reject, or skip.

The second question is slower and harder to eyeball, and it is answerable. A reviewer can point at the shadow under a rocker panel and say that should be rejected, not skipped. That sentence has a precise meaning in the file, a measurable effect on the loss, and one artifact it attaches to.

The payoff arrives during failures. When a model preserves a tire shadow, nothing about the diagnosis is guesswork. The renderer either separated the shadow pass or it did not. The exporter either rasterized it as negative or it did not. The label either carries 0 in that region or it carries 128. The loader either honored those values or flattened them. Four checks, each answerable yes or no, ending at the stage where negative space turned back into foreground.

Negative space is not the absence of a label. It is the part of the label doing the teaching.