Articles · Benchmark
One row, several buildings.
Schedules of values bundle campuses, apartment complexes and portables into single rows. We measured how well a decision model finds them, against plain rules, on real public schedules.
92.4%
Bundled rows found on real public schedules
Plain rules: 73.4%
1.8%
False alarms on single-building rows there
Plain rules: 32.0%
98.7%
Found among hand-written hard cases
with 0.0% false alarms
A schedule of values is meant to list one building a row. Real ones often do not. The Lebanon County housing authority in Pennsylvania lists “3BR 23 2-story duplexes (46 total units)” on one line with one value. A Louisiana school district lists “PORTABLE BUILDING (CONFIRM NUMBER)(7)”. A Florida county lists “Parks - Eagle Lake (6 Shelters @$100K ea.)”. Each is one row and several buildings. The model flagged the first at 0.84 and the third at 0.95; the second, as we will see, it missed.
Why one row matters
A catastrophe model reads each row as one building standing at one point. A bundled row breaks that in four ways:
- Place. All the value lands on one coordinate. A campus spread over a hillside, a coastline or a floodplain edge is modeled as if every building stood where the geocoder put the address.
- Construction. One row carries one construction class, so wood-frame portables beside a masonry school are modeled as masonry, or the other way round.
- Height and age. One story count and one year built stand in for buildings that differ in both.
- Size. Damage curves are written per building. Twenty-three duplexes read as one very large building are not the same risk as twenty-three small ones.
The fix is simple and manual: split the row, one row per building. The hard part is finding the rows to split in a schedule of thousands. That is what we tested.
What we tested
The model is Jev, TypeSafe’s decision model, reached through Vercel’s AI Gateway. It does not write text. We gave it each row and one fixed statement, “this row describes more than one separate building (for example several buildings, a campus, or a main building plus separate outbuildings); units, floors or wings inside one building do not count”, and it answered with the probability that the statement holds. We flag a row at 0.5 or above.
- Real public schedules. Rows, verbatim, from property schedules that US public entities published: housing authorities in Pennsylvania, school districts in Louisiana and Texas, a Florida county and city, the State of Mississippi, Chicago’s airports, California’s courts and New Mexico’s risk pool. 102 of them bundle several buildings and 955 describe one. The labels are ours, and we set aside 58 rows we could not decide.
- Hand-written hard cases. 149 bundled rows and 141 single ones, written to be difficult on both sides: ranges, “x 6”, lists, campuses, a house with a detached garage, and rows in Spanish, Portuguese, French, German, Italian, Japanese, Chinese, Korean and Dutch; against them, “24-unit apartment building”, “hospital with 3 wings”, “Building 3 of 7”, “Fire Station #3”, address ranges and typos.
- Single buildings from the National Structure Inventory, 300 US structures written out as schedule rows, to count false alarms on ordinary rows.
The comparison is a set of plain rules we wrote before asking the model anything: plural building words, a count before a building noun, a range of building numbers, “x N”, and words such as “campus” and “total”.
Found, and false alarms, at the gate of 0.5
Real public schedules, total lines left out · 79 bundled, 955 single
Real public schedules, every row · 102 bundled, 955 single
Hand-written hard cases · 149 bundled, 141 single
Single buildings from the National Structure Inventory · 300 rows
- Jev
- Plain rules
Where it earns its keep
On real schedules the model found more of the bundled rows than the rules did: 92.4% against 73.4% once total lines are left out. The bigger difference is everything else. Real schedules are full of words the rules fire on: a column called “Bldg #”, a school named for a campus, “Restrooms 4” as a building’s name. The rules raised a false alarm on 32.0% of single-building rows, so most of what they flagged was wrong (precision 19.7%). The model raised one on 1.8%, and 82.3% of what it flagged was a bundled row.
The hand-written cases show where the rules break:
| Case | Rows | Jev right | Plain rules right |
|---|---|---|---|
| A count of buildings (“14 buildings”, “# Bldgs: 18”) | 20 | 100.0% | 90.0% |
| A range (“Bldgs 1-4”, “Hangars 7 thru 12”) | 12 | 100.0% | 75.0% |
| A multiplier (“Duplex x 6”, “(x2)”) | 6 | 100.0% | 100.0% |
| A list of buildings (“admin bldg, bus garage and shop”) | 10 | 100.0% | 20.0% |
| A campus or site with no count | 8 | 100.0% | 62.5% |
| A detached garage, workshop or portable | 6 | 100.0% | 0.0% |
| Another language | 16 | 93.8% | 6.2% |
| Typos (“Buidlings 1-3, appartments”) | 6 | 100.0% | 50.0% |
| A small detached shed or kiosk | 6 | 83.3% | 0.0% |
| A total or blanket line (“Grand total, 37 buildings”) | 6 | 100.0% | 100.0% |
| Case | Rows | Jev right | Plain rules right |
|---|---|---|---|
| Many units, floors or wings (“Hospital with 3 wings”) | 16 | 100.0% | 93.8% |
| One building of a set (“Building 3 of 7”) | 10 | 100.0% | 90.0% |
| A plural or a number in the name (“Fire Station #3”) | 12 | 100.0% | 83.3% |
| Numbers that count something else (“4 loading docks”) | 8 | 100.0% | 75.0% |
| An address or unit range (“1204-1218 Oak St”) | 4 | 100.0% | 100.0% |
| Dimensions (“Pole barn 40 x 60”) | 4 | 100.0% | 50.0% |
| Another language | 16 | 100.0% | 100.0% |
Right means flagged for a bundled row and left alone for a single one. Data: by_case.csv.
What it misses
- Total lines. We labeled lines such as “Total for Pearl River Valley Water Supply District”, “SITE: 21 Garrison Bight Marina; Site Totals” and “Total 100% Blanket Buildings and Contents” as several buildings, because they sum several. The model flagged 26.1% of these 23 lines; the rules, which look for the word “total”, flagged 73.9%. Hand-written totals that said “all buildings” were easy; real ones mostly name only whose total it is. Take total lines out with a rule before anything else: they are sums to remove, not rows to split. Leave them out and the model found 92.4% of the bundled rows on real schedules.
- A short signal in a long row. Rows sent as twenty columns of “Header: value” can bury the one word that matters. “Occupancy Description: PORTABLE BUILDINGS”, halfway through a row of roof and cladding fields, came back at 0.24; “PORTABLE BUILDING (CONFIRM NUMBER)(7)” in the same kind of row at 0.20.
- A counting word it did not read. “厂房2栋” (two factory buildings) came back at 0.13, although it read “倉庫 3棟” (0.95) and “아파트 3개동” (0.89) correctly.
- Where it raises a false alarm, the row usually looks like several: units named by range inside a student village (“CERVANTES VILLAGE UNITS B2-B4”, 0.62), “Midway Pkg. Structure Office & Plaza” (0.58), “Former MECA Cafeteria & Old Graham Gyms” on one row with one building number (0.58). We labeled those one building from the rest of the schedule; a person reading the flag would check them too.
The gate
Single buildings come back low and tightly grouped: of the 1,396 single-building rows scored, 1,109 came back below 0.1. Bundled rows come back high but more spread out: of 251, 137 came back at 0.9 or above. That is why the gate sits low. We chose it on the hand-written rows only, before asking about any public row: at 0.5 it found 98.7% of the bundled hand-written rows with 0.0% false alarms, and every higher gate only lost bundled rows. The real schedules then checked it:
| Gate | Found, real | False alarms, real | Found, hand-written | False alarms, hand-written |
|---|---|---|---|---|
| 0.3 | 91.2% | 3.5% | 99.3% | 6.4% |
| 0.4 | 85.3% | 2.3% | 98.7% | 3.5% |
| 0.5 | 77.5% | 1.8% | 98.7% | 0.0% |
| 0.7 | 57.8% | 0.3% | 96.6% | 0.0% |
| 0.9 | 25.5% | 0.0% | 74.5% | 0.0% |
Real: every row of the public schedules, total lines included. Data: gate_tradeoff.csv.
On the real schedules a lower gate would have found more bundled rows and raised more false alarms, roughly one more of each for every step down. Every answer carries the probability, so a team that would rather read more rows can set its own line. The answers are close to repeatable but not exactly: asked a second time, 98.3% of 60 rows, most of them chosen near the gate, stayed on the same side of it; the median change was 0.01 and the largest 0.08. A row just either side of 0.5 is worth a look.
How many buildings, as a hint
In the same request we asked how many buildings each row describes, in five buckets: one, two to five, six to twenty, more than twenty, or several with the number not given. It named the right bucket for 89.2% of the 203 bundled rows that state a count. We show it beside a flag as a hint for splitting, never as a count, and it never flags a row on its own.
Three wordings
The words of the statement matter. We tried three, all on the hand-written rows, before settling on one.
| Wording | Separation | Found | False alarms |
|---|---|---|---|
| Short, with what does not count (chosen) | 1.00 | 98.7% | 0.0% |
| Long, listing the patterns | 0.99 | 98.7% | 0.7% |
| Framed as splitting the row | 1.00 | 92.6% | 0.0% |
Separation is the chance that a random bundled row gets a higher probability than a random single one (1.00 is perfect). Data: wording.csv.
The third wording, which asked whether splitting the row would give two or more rows, separated the rows as well but pushed its probabilities down, so a gate of 0.5 missed more. The short one, first written for the coding benchmark two days earlier, held up best.
Speed and cost
The median reply took 0.27 seconds and the 95th percentile 0.41 seconds. Each row cost about 531 input tokens, with the count question included, which at the gateway’s list price is $0.02 per 1,000 rows. The whole benchmark, every wording and every repeat, cost $0.05.
What we would tell a colleague
- Remove total and blanket lines with a rule first. Then run the check.
- Treat a flag as a prompt to read the row, not as a split. The count bucket tells you roughly how many rows to expect.
- Send table rows with their column names, but think about which columns you send: fewer, relevant columns make the one word that matters easier to see.
- Test it on your own schedules. Ours were US public entities, and their templates are not yours.
How to read these numbers. The labels are ours, and a row we could not decide was left out rather than guessed. Some sources are more repetitive than others: most bundled rows from the Louisiana school district read “PORTABLE BUILDINGS (N)”, and 82.9% of its 41 bundled rows were found. The model sits behind an alias that can change without notice, so these are figures for September 27, 2026. Every figure on this page is copied from the benchmark’s chart data, published here with the file and cell each came from.