Technical deep-dives into AI engineering, full-stack architecture, and lessons learned.
13-Part Series
Every post in this series is a decision I made that no model would have made on its own. Not because the model is bad , because the model doesn't know what it doesn't know.
13 of 13 parts published
Read the series
CommandCanvas lets people and supported agents work on the same canvas objects. Its shared commit path checks membership and revisions, preserves input provenance, and separates live previews from durable changes.
Daniel Anthony Romitelli Jr. · September 5, 2026
I built an AI message review console around a blunt rule: generation is allowed to suggest text, never to decide whether text is allowed to exist. The spine is a fail-closed pipeline where policy can stop a draft before any model call, hard rules can hold it after generation, and score-based release only runs after those gates pass.
Daniel Anthony Romitelli Jr. · August 26, 2026
I built a compliance audit sampling tool around a finite-population hypergeometric bound, then treated the confidence claim as something that could break in production. The interesting part is the test harness: exact enumeration, Monte Carlo replay through the same draw path, a tightness ceiling, and a shrink-by-one guard that makes a fake passing test fail.
Daniel Anthony Romitelli Jr. · August 26, 2026
Scenematic's think-frame rehearsal costs $0.002 against a $0.50 render, so an embedding-distance gate decides when the rehearsal's opinion is worthless, and a 100-contract baseline priced every way that gate was wrong.
Daniel Anthony Romitelli Jr. · August 23, 2026
Part two of Validation Geometry. Seven live assets, every one miscalibrated, and the direction differs by asset: ETH over-confident at the tail, six others under-confident in the middle. The hardened tree params that fixed one failure mode manufactured the other.
Daniel Anthony Romitelli Jr. · August 23, 2026
A rewrite anchored in ZCL active experimentation: hypotheses, randomized assignment during context provisioning, outcome joins, uncertainty-budgeted exploration, and minimum sample brakes.
Daniel Anthony Romitelli Jr. · August 23, 2026
Part 2 of the CarSegNet series narrows from training labels to runtime authority: the alpha model can repair uncertain vehicle edges, while the rest of the matte stays owned by the prior.
Daniel Anthony Romitelli Jr. · August 18, 2026
I traced a privacy value across a federated-learning pipeline and found that it was computed, logged, dropped, and then replaced with 0.0 at aggregation. The audit exposes a deeper systems error: independent training, accounting, model export, and zero-knowledge components cannot authorize a release until their evidence is bound to the same bytes in one atomic manifest.
Daniel Anthony Romitelli Jr. · August 16, 2026
A rewrite focused on the engineering pattern in Holographic, Law-Bound Memory: retrieval should return governed context with admissibility metadata and provenance, rather than loose nearest-neighbor text.
Daniel Anthony Romitelli Jr. · August 15, 2026
A production agent bridge dropped a date constraint while turning a model tool call into a real web request, then returned a plausible count over the wrong candidate set.
Daniel Anthony Romitelli Jr. · August 15, 2026
Most teams rank first, then ask a language model to explain the result. I did the opposite. The ranking arithmetic emits its own evidence, so every reason on a candidate card is one the scorer computed. That also makes the explanation testable, which a generated one never is.
Daniel Anthony Romitelli Jr. · August 12, 2026
A clean vehicle matte can still teach the wrong boundary. How render alpha, separated shadow and reflection passes, rejected hero candidates, and an explicit ignore band turn object-adjacent pixels into supervision, and why that same uncertainty band later decides what a learned refiner is allowed to touch.
Daniel Anthony Romitelli Jr. · August 11, 2026
Pramaana predicts which price barrier an asset reaches first within 24 hours, then turns those probabilities into long and short expected-return scores after trading cost. The model is trained on the same event structure that determines how a bracketed position actually closes.
Daniel Anthony Romitelli Jr. · August 10, 2026
How Neuroloq replaces static lesson completion with time-decayed mastery, stability scoring, and shared learner context across browser and phone tutoring.
Daniel Anthony Romitelli Jr. · August 7, 2026
A deploy can finish clean while the platform is dead. How I sequenced more than 56 Azure resources by treating creation order and readiness as different jobs: references and depends_on for the graph, retries and validation gates for the settle.
Daniel Anthony Romitelli Jr. · August 7, 2026
When I designed Hyperlogarithmic Benchmark Catch-Up Ratio, I treated correlation against existing measures as a validation gate. My dashboard and my paper disagree about the key coefficient -- 0.12 linear versus around 0.5 rank against Sharpe Ratio -- so this reconciles them and keeps the measured one, which makes HBCR a related but distinct axis rather than an orthogonal one.
Daniel Anthony Romitelli Jr. · August 4, 2026
A clean cutout still fails when the object's lighting belongs to the old scene. This rewrite focuses on bounded low-frequency relighting: change derived shading, preserve factual detail, and verify the edit.
Daniel Anthony Romitelli Jr. · August 4, 2026
A voice-control pipeline for a desktop IDE works better when speech is parsed as intent before it can change the workspace.
Daniel Anthony Romitelli Jr. · August 3, 2026
A PostgreSQL table for generated images works better when prompt, model, seed, edit lineage, metadata, and branding outputs live beside the file URL.
Daniel Anthony Romitelli Jr. · July 31, 2026
A clinic messaging thread needs durable states, clear ownership, and channel policy before any chat interface can be trusted.
Daniel Anthony Romitelli Jr. · July 31, 2026
I built an Ops Intelligence Agent alongside a recruitment platform Operations Dashboard, to turn a noisy real-time event stream into a small number of Microsoft Teams notifications worth reading. The interesting part isn’t “sending a webhook.” It’s adjudication: normalizing heterogeneous telemetry, scoring it fast, collapsing duplicates, and dispatching defensively so the first alert lands quickly without setting off an alert storm.
Daniel Anthony Romitelli Jr. · July 28, 2026
How I designed the video generation pipeline’s content-addressable lookup so retries reuse the same artifact while different model routes, seeds, and inputs get separate results.
Daniel Anthony Romitelli Jr. · July 28, 2026
A weighted average is the easy part. The hard part is what a review stage does when one judge returns truncated JSON, a half-filled object, or nothing at all. Here's how I made failure explicit with a sentinel review, kept partial streams observable for debugging, and how aggregation is meant to keep malformed output from poisoning the score.
Daniel Anthony Romitelli Jr. · July 27, 2026
My scene compiler kept returning a "top 5" that was one idea wearing five outfits. The fix sits in the post-scoring selection layer rather than in generation: a diversification pass that runs after scoring (and after any routing or gating decision), penalizes candidates that repeat what's already been picked, then re-ranks before final selection.
Daniel Anthony Romitelli Jr. · July 27, 2026
My portfolio repo references a RAG stack for code retrieval: OpenAI embeddings stored in pgvector with HNSW indexing, a cross-encoder rerank pass via BGE, and metadata-filtered search on top. The honest version: the components exist and I can show some of the glue code around retrieval, deduping, and formatting, but what I can point to doesn't include the actual SQL query shapes, the BGE batching code, or the latency-budget enforcement logic. So this post documents exactly what's implemented and names the parts I refuse to guess about.
Daniel Anthony Romitelli Jr. · July 27, 2026
A speech-to-text desktop app whose console entry point refuses to start until `pyaudio`, `keyboard`, and `openai` all import. A walk through the startup gate in `app/yapper.py`, the import order behind it, and the settings path that `app/core/config.py` pins to the app tree instead of the current working directory.
Daniel Anthony Romitelli Jr. · July 27, 2026
Generating n8n automations from screen analysis: the structure lives in typed Python objects, and JSON is only what comes out at the end.
Daniel Anthony Romitelli Jr. · July 27, 2026
A LightGBM capacity-control baseline for the ICAIF 2026 minute-ceiling paper that rebuilds its labels from parquet, and why label source, stride, purge, and temporal split geometry matter as much as the classifier in time-series ML.
Daniel Anthony Romitelli Jr. · May 31, 2026
Model failure is a time series, so I gave it its own model. CTF watches the primary predictor’s recent uncertainty telemetry, labels correctness without lookahead, holds training and serving to the same feature contract, and hands the execution gate a probability that the next forecast is worth using.
Daniel Anthony Romitelli Jr. · May 28, 2026
One blended embedding per candidate record kept surfacing people whose experience looked vaguely relevant while their skills were wrong, or the reverse. So I split each record into four semantic views and generate the four embeddings in parallel, cached under a stable Redis key. The job layer validates content length and vector dimension before it retries transient failures or routes terminal ones to the DLQ. The record stays one record. It just stops pretending it means one thing.
Daniel Anthony Romitelli Jr. · May 28, 2026
I split embeddings by chunk because the retrieval layer needed something finer than a whole-document vector could give me. That one choice runs through the whole pipeline: how text gets chunked, how each slice gets embedded, and the file-path lookup that can pull exact spans back when I need them.
Daniel Anthony Romitelli Jr. · May 27, 2026
Neuroloq decides whether an attachment is document-like before it decides which OCR mode reads it, and that routing call is the product.
Daniel Anthony Romitelli Jr. · May 27, 2026
Vector search handed back a plausible chunk from the wrong repository, so I made the vector, the scope filter, and the candidate count travel together through one `search_embeddings` RPC. The database applies the metadata predicate inside SQL, and the same contract drives the pgvector HNSW index on the embedding column.
Daniel Anthony Romitelli Jr. · April 16, 2026
I rebuilt the orchestration in the workflow analyzer SaaS around a narrow AgentGroupChat chain: analyzer, mapper, generator, validator, with state rehydration before the run and persistence after it. It can reject bad structure locally, retry from the right step, and resume from prior history instead of starting blind.
Daniel Anthony Romitelli Jr. · April 16, 2026
Before the generator writes a paragraph, a three-lane query fan-out, file-path-aware dedupe, a sufficiency threshold, and pinned excerpts decide whether the retrieval covered enough of the repository to deserve a draft. The most useful thing this pipeline does is turn down topics whose evidence set is too thin.
Daniel Anthony Romitelli Jr. · April 14, 2026
I built a reward calibrator that sits above the scoring signals and searches for better weights instead of hand-tuning them by feel. It normalizes each signal, combines them into a composite score, and evaluates candidate weight sets against benchmark data, so the system can measure itself against its own baseline.
Daniel Anthony Romitelli Jr. · April 13, 2026
Depth segmentation that keeps working with the lights off. The pipeline thresholds a LiDAR depth map, groups connected regions, and tests whether they behave like real planes, so it still returns labeled walls, windows, doors and trim when the RGB frame has nothing left to offer.
Daniel Anthony Romitelli Jr. · March 29, 2026
The voice path for a recruiting platform, rebuilt as a deterministic streaming pipeline: ACS media frames reach the backend as JSON metadata plus binary PCM, the server holds audio until `session.updated`, resamples 16kHz to 24kHz with `resample_poly`, and streams partial transcripts plus coaching back through SignalR without blocking the live call.
Daniel Anthony Romitelli Jr. · March 29, 2026
In this financial dashboard, freshness is a rule the data layer enforces rather than a guess the UI makes. The interesting part is the TTL split: quotes, intraday history, news, and search each age at their own pace, and the cache decides whether a read is still trustworthy enough to render or should fall through to upstream data.
Daniel Anthony Romitelli Jr. · March 29, 2026
Inside `lib/scene-compiler/text-detector.ts`: how I normalize fal.ai and Florence-2 OCR output, classify subtitle and watermark regions by where they sit, and turn text contamination into a normalized router score the scene pipeline can act on.
Daniel Anthony Romitelli Jr. · March 29, 2026
I added a pre-generation pass that lets my video pipeline explore cheap sketches before it commits to a full-quality keyframe. A small cohort of “think frames” gets scored by the same reward mixer I use elsewhere, the one that weighs several things at once, and the best path is picked by group-relative rank instead of an absolute cutoff.
Daniel Anthony Romitelli Jr. · March 26, 2026
The web app never has to guess what a segment, bounding box, or measurement means. Geometry gets normalized before it crosses the boundary, and only the version the viewer can trust gets persisted.
Daniel Anthony Romitelli Jr. · March 24, 2026
A multilingual patient check-in kiosk I built for a specialty medical practice after watching the front desk come apart under real pressure. The interesting part isn't the iPad UI. The work sits in the queue engine, the real-time synchronization, and a fallback-heavy notification flow that keeps the room moving when patients, staff, and Wi-Fi are all imperfect at the same time.
Daniel Anthony Romitelli Jr. · March 23, 2026
Part 1 laid out the shape of my company research chain: Firecrawl first, Bing second, and a tracer so the whole thing stays legible when the web lies. Part 2 is about the decision boundary itself, the exact "needs improvement" gate that determines when I augment Firecrawl with Bing, and why I made that gate intentionally blunt.
Daniel Anthony Romitelli Jr. · March 13, 2026
I built a step-by-step extraction tracer so I can answer one question under pressure: what exactly happened inside this run? Here is the node schema, where the hooks sit in the extraction pipeline, how retries and DLQ boundaries show up in the trace, and how I replay a trace locally to reproduce and fix failures, without guessing.
Daniel Anthony Romitelli Jr. · March 12, 2026
A wall’s “roughness” spilled into a roof the first time I ran RGB‑X on a full-frame photo, and it made the whole result feel fake. No new model fixed it. The fix was an engineering decision: run diffusion-based PBR decomposition per architectural region, cropped from SAM3 masks with padding and U‑Net-friendly sizing, plus a procedural fallback so the viewer never goes dark when the GPU pipeline isn’t available.
Daniel Anthony Romitelli Jr. · March 11, 2026
I replaced a ~3,500ms linear voice pipeline with a parallel three-phase orchestrator that targets <600ms P95 by treating "agents" like a compilation pipeline: Phase 1 produces typed intermediate results, Phase 2 consumes them as precomputed search inputs, and Phase 3 formats the response. The trick isn't asyncio.gather. It's typed contracts plus a fallback cascade, so every phase keeps moving even when a component fails.
Daniel Anthony Romitelli Jr. · March 11, 2026
My email extraction pipeline kept burning money re-deriving fields it already had. The bug wasn't a missing cache, it was caching without a validity model. What replaced it: a confidence-gated cache that decides reuse vs partial rebuild using conformal prediction, plus a reasoning budget allocator that spends compute per span only when quality is genuinely below target.
Daniel Anthony Romitelli Jr. · March 11, 2026
Long-form visual consistency looks like a memory problem, so I assumed it needed a growing story-so-far context hauled through every generation step. Scenematic works the other way around: propagate what must persist, measure what actually persisted, then correct what didn't, locally, with a periodic re-anchor that keeps small errors from compounding across a 20-scene run.
Daniel Anthony Romitelli Jr. · March 11, 2026
In my voice-first operations product, I stopped treating “search” as retrieval and started treating it as compilation: speech → intent → entities → an executable query plan. The query parser came after an LLM-first attempt produced latency spikes and inconsistent structure. Here is the intent contract, the rule engine (compiled regex + token maps), the entity extractors (locations, titles, numeric limits), the caching strategy, ambiguity detection, and the benchmark I used to check latency at scale.
Daniel Anthony Romitelli Jr. · March 5, 2026
An embedding pipeline for our recruitment platform that represents each record as four typed vectors instead of one pooled blob: profile, experience, skills, and general. Most of the engineering lives around the model call rather than in it: cache key design that includes model IDs, idempotent storage of multi-vector blobs, retry logic with backoff, and a DLQ path that keeps bad records from stalling reindex/backfill.
Daniel Anthony Romitelli Jr. · March 5, 2026
CLIP-only ranking chose candidates that looked right in embedding space and still broke continuity, so I replaced it with a multi-signal reward mixer. It scores each candidate across independent reward signals, normalizes them group-relatively (GRPO-style), then composes a final score, skipping null signals and re-normalizing the weights over the heads that remain. Treating "unknown" as a first-class state instead of faking certainty makes the ranking harder to game and easier to debug.
Daniel Anthony Romitelli Jr. · March 4, 2026
I rebuilt my voice agent's orchestration around a stubborn rule: don't spend an LLM call on a problem a regex can solve. What forced the change was a real latency incident caused by model-first routing, where obvious intents still waited on classification and p95 crossed the point where a voice turn starts to feel broken. The fix was a pattern-first RouterAgent with an LLM fallback reserved for genuine ambiguity, an orchestrator that coordinates instead of improvising, and a voice integration layer that enforces formatting, timeouts, and stable contracts. This post walks through the architecture, the post-mortem, the measurement methodology behind the latency numbers, and a runnable reference implementation of the router, the orchestrator, and the voice processor.
Daniel Anthony Romitelli Jr. · March 4, 2026
A company research pipeline that treats enrichment like an investigation: start with the best source, log every step, and hand off gracefully to a second one when the web lies or stops answering. The idea underneath is separating "fetching" from "deciding," so Firecrawl, the Bing fallback, and the tracer each do their jobs without smearing uncertainty across the output.
Daniel Anthony Romitelli Jr. · March 3, 2026
Our Azure Maps integration is cache-first because the first real failures were wasted calls rather than bad results. The LangGraph enrichment flow picked up address-vs-POI routing and a hard short-circuit when Firecrawl had already produced city/state, and that pushed the whole thing toward treating geocoding as a budgeted, deduplicated service: stable cache keys (geohash plus query normalization), TTLs that reflect how volatile a location answer is, and quota smoothing with backoff so bursts don't turn into 429 storms. The same discipline runs through the advisor-enrichment worker (credits_used, max_credits, feature flags), where cost and reliability are explicit outputs of the pipeline.
Daniel Anthony Romitelli Jr. · March 3, 2026
Uniform frame sampling made my screen-workflow analyzer expensive and blind to short UI bursts at the same time. I replaced it with an adaptive sampler that scores cheap visual change, segments the timeline, and allocates a fixed keyframe budget with guardrails. Here's the failure that forced the rewrite, the production integration seam it sits in (Cloud Run to webhook to Next.js), and complete runnable code for scoring, segmentation, allocation, and frame extraction, plus proper HMAC verification on the webhook receiver.
Daniel Anthony Romitelli Jr. · March 2, 2026
My GPU server streams SAM3 masks frame by frame. When the masks flickered and the labels churned, I blamed "model instability." The real culprit was how I handled session reuse, mixed-resolution frames, and reprojecting outputs back to each frame's original size. Here's the inference session lifecycle, the streaming output shape assumptions (batch size 1), and the debug hooks I added, like /segment/debug-model, to make flicker diagnosable instead of mystical.
Daniel Anthony Romitelli Jr. · February 28, 2026
A pipeline for rebuilding deal timelines out of messy webhook and API audit trails: normalize heterogeneous events into one shape, split them into deal-centric sessions, compress those into canonical state paths, then extract a transition graph whose edges carry both counts and observed durations. Missing and out-of-order events get handled without pretending the data is perfect.
Daniel Anthony Romitelli Jr. · February 28, 2026
2h 16m · 13 chapters
The full series narrated as a 2h 16m audiobook. Listen on your preferred platform.