Audit Log

This document records data-quality issues discovered during pre-launch and post-launch audits, what was done about each one, and what remains open. Entries are dated and listed newest-first.

The purpose is twofold: an internal record so issues don't get lost between work sessions, and a transparent account for external readers (environmental attorneys, watchdog groups, journalists) who want to understand how the underlying data is verified.

Permit Minder's defensibility standard: every number on the platform should be something an environmental attorney would be comfortable forwarding to opposing counsel. This log is part of how that standard is maintained.

2026-08-21 — Historical copies of the exceedance data moved out of the live database

Permit Minder retains twenty-three historical copies of the exceedance data, taken between 2026-05-04 and 2026-07-09 before specific changes to how records are stored or corrected. Together they hold 898,840 rows. They have been moved out of the live database and are kept separately. All of them are retained; nothing was discarded.

What this changes, and what it does not. Earlier entries below describe these copies as retained rollback anchors and, for the 2026-05-04 copy, as legal history. That remains true. What changed is only where the copies are kept, which is an operational detail rather than a statement about the data.

Why. The copies occupied more than half of the live database. Keeping them alongside the served records consumed the same capacity those records depend on, and a second copy stored beside the original is not a safeguard in any useful sense.

How it was verified, in this order.Each copy was exported with its full column structure. Each export was then read back and its row count compared against the original: 23 of 23 matched exactly. One copy — the 109,862-row 2026-05-04 record, the one retained as legal history — was then fully rebuilt from its export and confirmed complete at 109,862 rows across all 23 columns, because a stored file that can be listed is not the same as one that can be restored. Only then was anything removed from the live database, and each removal was preceded by a fresh check that the row count still matched what had been exported.

Effect on published data: none. The live exceedance records were not touched. The headline counts, the served records, and every figure on the site are unchanged: 136,157 exceedances across 7,356 permits at the time of this entry. These copies were never a source for anything published; they are the record of what the data looked like before specific changes, kept so that any published figure can be checked against its own history.

2026-08-05 — Pennsylvania is collected from two sources again, and the headline counts one of them

Pennsylvania is collected from the Pennsylvania Department of Environmental Protection's eDMR system, the state-primary source, and from the EPA's ICIS-NPDES bulk discharge-monitoring feed. Every other state currently published is collected from the federal feed alone. Pennsylvania's headline counts are drawn from the state-primary records; the federal feed carries the same underlying reports, which is why the two are not summed, and it also carries records that have no state counterpart, which is why a search can return more permits than the headline count. Collection runs weekly, with a monthly pass covering annual reporting periods.

What changed, and what this supersedes. Collection from eDMR stopped, and for the period since, the EPA's federal feed has been Pennsylvania's sole source — which is what the 2026-07-14 entry below records, accurately for the period it describes. It is superseded on that one question as of this entry: Pennsylvania is no longer collected from the federal feed alone. The reason collection stopped — that eDMR's reporting endpoint returned a monitoring period only when the query window fully contained it, so semi-annual and annual periods were invisible to a monthly query — has been measured rather than assumed, and the limitation it described does not exist at the statewide level. A single statewide render returns every permit's rows for the window, annual periods included.

The remaining limitation is temporal, and it is named here rather than left to be discovered. An annual monitoring period is collected on the next monthly pass, so it can lag by up to about 31 days. That is shorter than the 54-day rotation the earlier path required. Each collection window is wider than the interval between its own runs, so a skipped run cannot leave a permanent gap.

A correction to the earlier entry's wording. That entry says the collection path was "retired on 2026-07-12."That overstates what happened: the connector was never deleted, it was left unwired, and the date came from a code comment that was itself stale. What is accurate — and what that entry's substance rests on — is that collection from eDMR stopped, and the federal feed was Pennsylvania's sole source for the period. The entry stands as written; this paragraph is the correction.

Why this is a new entry rather than an edit. The 2026-07-14 entry was accurate when written, and this log's value depends on it being possible to see what was said and when. It stands.

2026-07-14 — 49 readings were compared against their permit limit in the wrong direction, and have been withdrawn

A permit limit has a direction. Most are ceilings: the reading must stay below the limit. Some are floors: the reading must stay aboveit — a treatment plant required to remove at least 89.25% of a pollutant, for example.

Direction is not something we are free to infer. It comes from a field the regulator publishes alongside the reading, called the statistical basis. Our rule, written down before this happened, is that if the source does not establish the direction, we do not publish a comparison.

The rule was not enforced in the code. Where the statistical basis was missing, the software did not stop — it treated the row as a ceiling. That is not a neutral assumption. On a floor limit it reverses the answer:

A facility required to remove at least 89.25% of a pollutant reported removing 9.09%. Compared as a ceiling, the software asked “is 9.09 greater than 89.25?”, found that it was not, and recorded the reading as within its limit.

49 readings were affected, across 4 Pennsylvania permits, for monitoring periods in 2021. All 49 are minimum percent-removal parameters (CBOD20, CBOD5, BOD5, and Total Suspended Solids). All 49 were reported below their limit and displayed as though they were not.

The error ran in one direction: it understated. It did not create an exceedance against any facility; it concealed readings that appear to fall short of a minimum.

Why the 49 are being withdrawn rather than corrected. We went back to the source. Pennsylvania DEP publishes the statistical basis field emptyfor these rows — we confirmed this against the original data files as we received them. There is nothing to correct them from. The parameter names say “Minimum,” and the federal secondary-treatment standard sets minimum removal rates, but neither of those is the regulator’s statement about this reading. Asserting a direction on that basis would mean publishing a comparison the source never supported — which is the same mistake in the opposite direction.

So we assert nothing. The 49 readings are being withdrawn from the site as this entry is published. They remain in our records, unpublished, so that this account can be checked.

What has changed.The software now stops on any reading whose direction the source did not establish, and holds it instead of guessing. The database no longer supplies a placeholder value when the basis is missing; a reading that arrives without one is now rejected outright rather than quietly labeled “UNKNOWN.” A standing check runs against the live database and fails if any published reading carries that placeholder again.

What remains open.These 49 readings may represent real shortfalls against a minimum treatment requirement. We cannot say so on this data. Whether EPA’s national ICIS-NPDES dataset carries these same permit-periods with the basis populated is being checked; if it does, the readings can be restored with a direction the regulator actually stated. Until then they stay withdrawn.

2026-07-14 — Two earlier entries described a collection schedule that no longer runs

The entries dated 2026-06-13 and 2026-06-23 describe how Pennsylvania’s long-period records were collected. They say, in the present tense, that a per-permit five-year pass “is now scheduled to run daily” across 54 shards, “completing a full rotation in about 54 days,” and that a newly-closed semi-annual or annual period may therefore lag “up to about one 54-day rotation.”

That is no longer how Pennsylvania is collected, and it has not been since 2026-07-12.

Those passes existed to work around a limitation of the PA DEP eDMR system: its reporting endpoint returned a monitoring period only when the query window fully contained it, so semi-annual and annual periods were invisible to a monthly query and needed a separate per-permit sweep. That collection path was retired on 2026-07-12. Pennsylvania is now collected from the EPA’s ICIS-NPDES bulk discharge-monitoring feed — the same federal feed used for every other state currently published — on a weekly cadence.

The federal feed publishes every monitoring period in the same file, whatever its length. So the sweep is not merely stopped — the problem it solved does not arise in the current path, and the 54-day rotation lag described in those entries no longer applies.

Why this correction is recorded rather than edited into the older entries. Those entries were accurate when they were written, and this log’s value depends on it being possible to see what was said and when. Rewriting them would make the record tidier and less true. They stand, each now carrying a pointer to this entry; this entry supersedes them on the question of how Pennsylvania is collected today.

The earlier entries remain correct on their own subject — the alert-suppression guarantees they describe (historical records recovered by a backfill are never emailed) are unchanged and still enforced.

A related sync problem, recorded because it is the more useful finding. This page and the internal log it mirrors had drifted apart. The 2026-05-19 entry’s scheduling paragraph was corrected in the internal log on 2026-06-16, when the long-period passes were put on a recurring schedule — and this page’s copy was not touched. So from 2026-06-16 until this entry — 28 days — this page told readers that recurring scheduling of the long-period passes “remains an open item” and that periods were “not yet refreshed automatically,” while the internal record said the opposite. That is a statement about how completely we collect, and it was wrong in the direction of understating our coverage.

Because that edit was made in place, the two files now hold different text for the same paragraph: the internal log carries the June rewrite, this page carries the 2026-05-19 original. Neither is rewritten again here — each was accurate when it was written — and both now carry a pointer to this entry. The two surfaces are required to move together, and this is the record of the one time they did not.

2026-07-13 — Pennsylvania restored from the federal feed; records briefly removed in error, and reinstated

This entry covers a sequence of changes made on one day, including a mistake and its correction. It is written in the order the events happened.

1. Pennsylvania was re-collected from the federal feed. After the withdrawal described in the entry below, Pennsylvania was collected from the EPA's ICIS-NPDES feed — the source every other state is collected from — which reports the value qualifier in its own column rather than as a symbol inside the value. 7,702 exceedance records were published.

2. A duplicate period was identified, and records were removed on a mistaken basis. The re-collection covered federal fiscal year 2025 (October 2024 through September 2025), a period the retired Pennsylvania DEP corpus already covered. On the understanding that the feed republished the same discharges, the DEP records for that period (8,010) were withdrawn.

That understanding was wrong, and the check that would have shown it had not been run. The two sources are not equivalent. For the same permit and period they use different parameter names, report values and limits in different units — in some cases differing by a factor of 1,000 (micrograms against milligrams per litre) — and assert different permit limits. Of the 8,010 DEP records withdrawn, only 134 had any counterpart in the feed. 7,876 exceedances were removed from the platform that the feed does not report.

3. The records were reinstated. All 8,010 were restored the same day. They were never deleted — a withdrawal marks records superseded and leaves them in the underlying table — so the reinstatement returned them exactly as they were.

4. The reinstatement was five records too wide, and was corrected. 8,005 records had been withdrawn in step 2, but 8,010 were reinstated. The five extra had been withdrawn earlier, for unrelated and correct reasons, and the reinstatement did not distinguish them: four pH measurements, which this platform withholds under 40 CFR 401.17 and does not report as exceedances at all; and one measurement of 11.0 against a limit of 45.0, which is not an exceedance and had been corrected away in June.

They were identified against a database backup taken the previous day and withdrawn again. They were served for approximately twenty minutes. The published total now reconciles exactly: 42,235 records before the re-collection, plus 7,702 from the feed, is the 49,937 served today.

What the platform now serves. Both sources are published for fiscal year 2025 and neither is treated as authoritative over the other, because which of them is correct is not yet established.

Exceedance records currently served, all states:

A known duplication. 134 discharges are currently reported twice — once from each source. Of those, 118 are the same measurement expressed in different units. The remaining 16 have not been reconciled. All 134 are left in place rather than removed, because removing them correctly requires normalizing units across the two sources first, and a removal performed without that step is what caused the error described above. This is disclosed rather than quietly corrected.

What remains open. Which source is authoritative for Pennsylvania — and in particular which permit limit is enforceable where the two disagree — is an open question. Until it is answered, both sources are shown.

2026-07-13 — Three qualifier codes the federal feed publishes were not recognized, and would have been recorded as measurements

What happened. Reviewing every value qualifier the EPA feed publishes — 9.4 million records across seven states — found seven distinct codes. Three of them carry meaning the platform was not reading.

Why it matters. “<=” is a censored reading: the laboratory is reporting a bound, not a measurement. Recording “<= 3” as a plain “3” asserts a number the laboratory did not report — the same defect described in the entry below, which withdrew 21,568 Pennsylvania records the same day. “E” is an estimate, and we do not know what it was estimated from.

Whether it affected published data. No. No Pennsylvania record carries any of these codes, and Pennsylvania is the only state currently published. The finding is a defect in what the platform would have published: about 4,753 records had the remaining states been loaded without this fix.

What was done.

• “<=” is now treated as a non-detect, and “>=” as a reading above the detection limit. Both are screened out of headline counts and alerts, as “<” and “>” already were.
• A distinction was drawn that the platform had not made. A qualifier bounds the true value; it does not state it. Against a maximum limit of 10, a reading of “> 10” proves an exceedance (the true value is strictly above 10), but “>= 10” does not (the true value may be10). Against a minimum limit of 2, “< 2” proves an exceedance but “<= 2” does not. The two forms differ in exactly one case — the reported number equal to the limit — and the platform now reports an exceedance only where the source proves one.
• “E” (estimated) values are withheld rather than guessed at. They are counted and reported on every collection run: “E” is routine on flow parameters, so the withheld volume is expected to be substantial, and a withheld record that nobody can see is indistinguishable from one that was never received.
• Every qualifier is displayed with its value on the permit page, the CSV export, and the PDF export.

What remains open. The meaning of “E” in this data set is not established, so those records are withheld rather than published. Establishing it is tracked separately.

2026-07-13 — Qualifiers were not preserved on a set of Pennsylvania records, and those readings were stored as measurements

What happened. On 2026-07-13, 21,568 Pennsylvania records were loaded into the platform. None of them carried a value qualifier. All were stored as though the reading had been measured directly.

That is not what the source says. Pennsylvania operators report a value that may carry a leading less-than or greater-than symbol — “< 3”, “> 2420” — and roughly a quarter of the rows in the source files do. The collection step read the number and discarded the symbol. A later step, finding no qualifier on the record, supplied “measured” in its place.

Why it matters. A reading the laboratory reported as belowa detection limit (“< 3”) was stored as a measurement atthat limit (3). A reading reported as above the range of the test (“> 2420”, the ceiling of the standard IDEXX Quanti-Tray method for fecal coliform) was stored as an exact measurement of 2420. A detection limit stored as a measurement can be compared against a permit limit and produce an exceedance the laboratory never reported.

This is the defect the 2026-05-08 entry below assumed could not reach per-row data. That entry states that the permit detail page, exports, and per-row data show the value the laboratory actually reported, with the qualifier preserved alongside the number. For these 21,568 records, that was not true.

The mislabel did not defeat our safeguards. It made the rows ineligible for their protection. This is the part worth understanding, because it is not what a “missing column” sounds like.

Permit Minder treats a non-detect deliberately and carefully. A non-detect whose substituted value would otherwise exceed the permit limit is classified as non-detect screening: kept out of headline exceedance counts, kept out of alert digests, and never characterized as a measured exceedance (see “Non-detect values” in the methodology). A non-detect below the limit is not an exceedance at all. Every one of those safeguards was working correctly throughout.

A reading relabelled “measured” does not trip them. It walks past them — because each one asks what kind of reading it is, and the row now gives the wrong answer. The screening classification, the headline exclusion, the alert filter: all functioning, all bypassed, because the reading misstated its own kind at the point where those decisions are made.

No alerts were sent. Alert delivery was suppressed for the entire load, so no subscriber was emailed about any of these rows. It is worth being precise about why: these rows were eligibleto alert, because the alert filter passes rows marked “measured” — and they had been marked “measured” in error. Suppression is what held, and it held only because it does not ask what kind of reading it is. The filter that does ask had already been given the wrong answer.

This is why the repair is a refusal rather than a better default. A missing value fails closed — it stops. A wrong label fails open, through every check that trusts the label.

A second, wider gap was found while repairing the first. Pennsylvania is now collected from the EPA’s national feed, which reports the qualifier in a dedicated column rather than as a symbol inside the value. That column was being read and classified correctly — and then discarded three times in a row, by three separate lists of fields to carry forward, before it reached the store. The classification was never missing. It was computed, correct, and dropped. Had a national load run, every state would have been published the same way.

What was done. A record that does not carry its own qualifier is now refused rather than stored with an assumed one; absence of a qualifier is not evidence of a clean numeric reading. All three lists now carry the qualifier through to the store, along with the value exactly as the source reported it — symbol and all — so a reader can check a published number against what the laboratory actually said.

The affected records have been withdrawn. On 2026-07-13 at 14:24 UTC, all 21,568 were superseded. They are no longer served: they do not appear on the site, in search results, or in the alert digest, each of which reads only non-superseded records. This was verified three ways — the write path's own report, a direct count of the table, and the alert query's own filter — which agree: 0 affected records still served, 21,568 withdrawn, and the number of records served falling from 63,803 to 42,235.

They were withdrawn, not deleted. Each remains in the table, marked as superseded, with its values intact. That is deliberate: this entry discloses that those records were published, and deleting them would destroy the evidence of the thing being disclosed. A reader who followed a link to one of them can still be shown what happened to it.

What remains open. Pennsylvania is being re-collected from the EPA feed, which carries the qualifier. The affected records could not be corrected in place, because the qualifier is not present in the stored intermediate data they were built from. Re-collecting was never going to withdraw them, which is why withdrawal came first: the re-collected records are attributed to the EPA feed while the affected records are attributed to the retired state scrape, so a re-collection would have sat alongside them rather than replacing them, and Pennsylvania would have appeared twice — once correct, once not, with nothing visible to tell a reader which was which. The re-collected records are expected to contain both non-detect readings (a reading below a detection limit is still below a minimum limit, so it is reported) and above-range readings; a rebuild showing none of either has not worked, because that is what the affected load looked like. No further state loads will run until the rebuild is verified.

How this was found. A test that checks the distribution of qualifier kinds against expected bounds. It found the readings after they were published, not before. The bounds were not adjusted to accommodate the new data.

2026-07-12 — An unreadable “less-than” reading no longer produces an exceedance

Scope. A correction to how the pipeline handles a malformed detection-limit reading. No rows in the live Pennsylvania data are known to be affected — a survey of the current records found no reported value beginning with “<” that could not be read. The change is preventive: it closes a path by which a value the source never supplied could have been reported as an exceedance.

What the issue was. A reported value beginning with “<” denotes a non-detect at a stated detection limit — “<0.05” means below 0.05 — and the pipeline records half that limit. When the text after the “<” could not be read as a number, the pipeline recorded zero instead.

Zero is not a neutral placeholder. Some permit limits are minimums rather than maximums: dissolved oxygen, for instance, must stay above a floor. Against such a limit a zero falls below the limit, so the record would have been reported as an exceedance, and the percentage would have been rendered as 100% below the limit. That figure would have been an artifact of the placeholder, not a measurement the source ever provided. This is the same category as the detection-limit handling addressed in the 2026-05-08 entry below (“Headline exceedance percentages now exclude non-detect readings”).

What was done. A value that cannot be read is now recorded as not comparable rather than as zero, and no exceedance determination is made, because there is nothing to compare against the limit. A reading that is not an exceedance is not carried in the exceedances table, so an unreadable reading now produces no entry there, in the same way any within-limit reading produces none. The underlying source record is unaffected and remains checkable at the state agency.

Well-formed non-detects are unchanged, and a genuine reading below a floor limit is still reported. The corrected behavior is covered by tests so it cannot change unnoticed.

2026-07-10 — Pennsylvania pH hold: 15 residual rows held after a pre-fix ingest

Scope. A same-day follow-through on the 2026-07-09 correction below. Between that correction and the moment the retrofitted Pennsylvania path reached production, an ingest ran on the earlier code and re-computed 15 Pennsylvania pH rows across 9 permits. They were held the same day. The active Pennsylvania exceedance count moved from 42,202 to 42,187. No other parameter or state was affected.

Why. These rows are the same category held under 40 CFR 401.17 (see the entry below): a monthly DMR pH value cannot establish an excursion-duration determination, so it is not a defensible exceedance. They existed only because the ingest that produced them predated the retrofit reaching production.

What was done. The 15 rows were held using the same mechanism the site already uses to exclude a superseded record from search and facility pages; no rows were deleted and the action is reversible. Verified afterward: no active Pennsylvania pH rows remain on the public search or facility pages, and pH no longer appears in the parameter list. The retrofit that prevents recurrence is now live in production, so later ingests run on the corrected path.

2026-07-09 — Pennsylvania pH exceedances retired under 40 CFR 401.17 (1,753 rows removed)

Scope. A data-quality correction. 1,753 Pennsylvania pH rows were removed from the exceedances dataset; every affected row is preserved in a backup table. The Pennsylvania exceedance count moved from 43,943 to 42,190. No other parameter or state was affected.

Why. pH is not a simple over/under-limit parameter. Under 40 CFR 401.17, brief pH excursions outside the permitted range are allowable for facilities with continuous monitoring, subject to duration limits — so a reported pH value outside the range is not automatically a reportable exceedance. A monthly DMR summary carries only the value, not the excursion duration, so it cannot establish whether a given out-of-range reading qualifies. Flagging these as exceedances asserted a determination the source data cannot support — which fails the standard this log exists to protect.

How this arose. The pH hold was already implemented on the newer multi-state classifiers, which hold pH and never flag it, but was never retrofitted onto the older Pennsylvania ingest path. Pennsylvania was the only path still computing pH exceedances.

What was done. The affected rows were backed up (reversible), then removed from the live dataset (verified: none remain for Pennsylvania). The Pennsylvania ingest path was retrofitted to hold pH, with unit tests added so a re-ingest cannot recreate pH exceedances.

2026-07-02 — Chain-of-custody record of state amendments (no change to published data)

Scope. A record-keeping change, not a data-quality finding. No permit data was added, removed, or modified. Production held 43,717 total records / 43,714 active before and after, unchanged.

Why. PA DEP amends eDMR records in place after initial publication (for example, when a facility submits a corrected DMR). Permit Minder already handled these amendments correctly and marked each affected record as corrected, but the prior value was overwritten — the record proved a correction happened, not what the value had been before it. From this date forward, every correction also appends a permanent entry recording the old and new reported value, permit limit, exceedance percentage, and monitoring-period bounds, together with the state's own rendering of the value at each collection, and the timestamp of the correction. The entry is written at the same moment as the correction itself, so the two cannot diverge.

Tamper-evidence. The record is append-only by enforcement rather than by convention: edits, deletions, and clearing are refused, and a record with correction history cannot be removed without the history being cleared explicitly first. This history is internal audit material and is not published.

Verification. Before the change was committed, the full amendment scenario was walked on a synthetic permit: original collections and unchanged re-collections write no history; each kind of correction writes exactly one history entry with the correct before and after values; and edits or deletions against the history are refused. A failure of any of those checks would have abandoned the change. Afterwards, direct inspection confirmed the history is live and empty, no synthetic records remained, and published data was untouched.

Alerting unchanged. Corrections to already-reported records still do not trigger new email alerts. Whether a correction of a given magnitude should notify subscribers is a separate product decision; this history is the record such a feature would draw on.

Status. Applied and verified on 2026-07-02. From this date forward, Permit Minder retains a tamper-evident record of every state amendment it observes, alongside the existing per-record source links back to PA DEP eDMR.

2026-06-23 — Source labelling hardened ahead of multi-state data (no change to published data)

Scope. Three backward-compatible hardening changes were applied ahead of any future multi-state load. This is a record-keeping entry, not a data-quality finding: no permit data was added, removed, or modified. A full-content fingerprint taken before and after was identical across all 43,174 records, and the composition counts were unchanged. The published data remained entirely Pennsylvania throughout.

Why. Multi-state machinery had been written but deliberately held inactive so the live data could never silently mislabel a record's source. Each change removes a way a future non-PA record could be stored without an explicit, correct source: an optional state filter was added to the search paths, with the default behavior byte-for-byte unchanged; an implicit source label was removed, so an unstated source is now genuinely unstated rather than being asserted as a particular one; and the single write path now refuses a record that carries federal-feed markers but no stated source, rather than defaulting it to Pennsylvania.

Verification. A record-exact snapshot of all 43,174 rows was captured beforehand and a named reversal prepared for each change. The content fingerprint was identical before and after; composition was unchanged; alert eligibility was unchanged; and the public search paths returned the expected first page and site statistics (2,510 permits / 43,171 exceedances). All three changes were independently confirmed live.

Status. Applied and verified; no reversal required. No collection was run, no non-PA data was loaded, and no alert, scheduling, storage, or site change was made.

2026-06-13 — Alert-eligibility safety: historical records suppressed so they are never emailed as current

Scope. A set of corrections ensuring that historical exceedance records recovered by a backfill are never emailed to subscribers as if newly published. No published permit data was changed: the only thing affected is whether a record is eligible for an email alert. All measurement values, monitoring periods, exceedance percentages, and per-record source links are unchanged.

Why this matters. The daily alert digest decides whether an exceedance is “new” to a subscriber. Records recovered by a backfill of older quarterly, semi-annual, and annual reporting periods carry a recent collection timestamp, so without a dedicated eligibility signal they could be sent as current events. Mailing an environmental attorney a years-old exceedance as “newly available” is a defensibility failure.

1. An explicit eligibility marker. Each record now carries a marker saying whether it is a current publication a subscriber may be alerted about, or a suppressed historical recovery that is never alert-eligible. The alert sender keys on that marker rather than on the collection timestamp, so a suppressed record is excluded regardless of how recently it was collected.

2. Historical-backfill suppression (2026-06-12). The May 2026 five-year historical load had left 39,185 older records alert-eligible. These were marked never-eligible in a single guarded operation, reducing the alert-eligible set from 41,125 to 1,940 records and preserving eligibility only on genuinely recent ones. A record-exact snapshot was retained as a reversal point.

3. First alert run after the change (2026-06-13, PASS). The first daily alert run after the suppression was verified read-only: it completed successfully (0 sent, 1 skipped, 0 errors); zero long-period records were alert-eligible; no historical record was poised to send; and the previously-noted record for permit PA0254967 (a February 2021 monitoring period) remained suppressed.

4. Residual cleanup (2026-06-13). A read-only review found 14 further records — old 2021 monitoring periods recovered by a per-permit sweep — that remained alert-eligible because the first suppression rule keyed on monitoring-period length (treating month-length periods as current) rather than period age. None had been emailed and none matched any active subscription. They were marked never-eligible, reducing the alert-eligible set from 1,940 to 1,926. A record-exact snapshot was retained.

5. Durable guard (2026-06-13). The two long-period collection paths now refuse to run unless an explicit alert-suppression mode is selected, and suppression is computed by monitoring-period age against a fixed freshness window, not by period length. This prevents the same class of record from being introduced as alert-eligible by any future long-period sweep. The recurring monthly collection is unchanged and remains alert-eligible by default.

Verified state (2026-06-13). Production held 1,249 active records whose monitoring period spans more than one calendar month (quarterly, semi-annual, and annual periods recovered by the long-period sweep); all 1,249 were alert-suppressed. The active alert-eligible set was 1,926 records, all recent monthly-period rows. Zero long-period records and zero old historical short-period records were alert-eligible. Production totals: 42,375 total / 42,374 active / 1 superseded.

Recurring long-period coverage. As of this entry, the monthly statewide pass ran weekly, the quarterly pass ran on a recurring schedule, and a per-permit five-year pass ran across 54 deterministic shards, completing a full rotation in about 54 days. Both long-period passes ran with mandatory period-aware alert suppression, so historical records they recovered were never emailed. The remaining limitation was timing, not absence: a newly-closed quarterly, semi-annual, or annual period could lag until the relevant sweep reached it — up to about one 54-day rotation. That lag is stated plainly rather than implied as continuous coverage. [Superseded 2026-07-14: the per-permit sweep and its 54-day rotation no longer run — Pennsylvania is collected from the EPA ICIS-NPDES feed. See the 2026-07-14 entry above.]

Method. Guarded, single-step updates affecting only alert eligibility, each preceded by a read-only identity check and a retained record-exact snapshot; read-only verification afterwards; and a durable guard against recurrence.

Status: RESOLVED for the historical alert-safety gap. Remaining caveat: timing lag until the relevant scheduled sweep reached a newly-closed long period. [Superseded 2026-07-14 — see the entry above.]

2026-05-30 — Launch data-correctness verification: four-permit source-to-published spot-check plus live search checks (no mismatch found)

Scope. A targeted, read-only spot-check comparing what Permit Minder publishes against PA DEP eDMR source data for four permits, plus an end-to-end recomputation of the exceedance calculation and three live search checks against the public site. This was a four-permit spot-check and a small set of live searches — not a full or exhaustive audit of the dataset. It does not establish that the dataset is complete, official, guaranteed, comprehensive, or presented in real time.

Method. Source records were pulled directly from PA DEP eDMR and classified through the same production calculation the site uses; published records and search counts were read through the same paths the live site reads, with the public site confirmed serving normally. Nothing was written.

Permits checked (one recent monitoring period each):

• PA0026671 — Philadelphia Water Dept, Southwest WPC (urban POTW), period ending 2026-02-28: 9 published records vs 9 source-classified exceedances, exact match.
• PA0008869 — Pixelle Specialty Solutions, Spring Grove Mill (industrial, three monitoring locations), period ending 2025-11-30: 3 vs 3, exact match.
• PA0012637 — Trainer Refinery (industrial), period ending 2025-11-30: 2 vs 2, exact match.
• PA0028673 — Gallitzin Borough WWTP (small/rural POTW), period ending 2026-03-31: 1 vs 1, exact match; plus the annual-period Total Phosphorus record (period 2024-10-01 to 2025-09-30) present and source-exact.

The comparison was two-directional: every source-classified exceedance was found in the published data with matching reported value, permit limit, units, and monitoring location, and the published data contained no extra records for those periods. Readings reported in two units for the same sample (lbs/day and mg/L), and readings at multiple monitoring locations, resolved correctly.

End-to-end calculation trace. Three calculation paths were traced from the source value to the published exceedance percentage: a maximum limit (Total Suspended Solids, 191,390 against 75,060 lbs/day → 154.98%), a minimum percent-removal limit (CBOD5 percent removal, 82.82 against 89.25 → 7.20%, with the minimum direction applied correctly), and a reading the laboratory reported as above its detection limit (CBOD5 reported as more than 42,750 against a 29,700 limit → 43.94%). Each recomputed value matched the published value.

Search checks. County = Allegheny returned 101 facilities, matching 101 distinct published permit numbers for that county; permit = PA0026671 returned 1; parameter = Total Suspended Solids returned 1,299 facilities — each internally consistent with the published data.

Source-retrieval note (no published gap observed). During retrieval, one unusually wide single request (spanning 2024-01-01 to 2026-04-30 for one permit) returned records only through period-end 2025-12-31, silently omitting the most recent periods, which do exist at the source. Re-requesting those periods in narrower spans, and re-requesting with the standard five-year span (2021-05-01 to 2026-04-30), each returned the full range including 2026 periods. This appears to be an inconsistent response from the source on that one wide request rather than a systematic limit. It was not observed to have created any gap in the published data— the recent and annual-period records for the checked permits are all present and source-accurate. It is recorded here as a retrieval-robustness item worth a future completeness re-probe, not a defect found in published data.

Outcome. No mismatches were found across the four permits, the three calculation traces, or the three search checks. It remains a spot-check rather than a guarantee about the entire dataset.

2026-05-22 — Second scheduled alert run clear; subscriber state preserved

Outcome. The daily alert run fired on schedule on 2026-05-22 and recorded 0 sent, 1 skipped, 0 errors — exactly the expected result. The single daily-frequency subscriber had no new matching exceedance records since the 2026-05-20 cutoff, because no new collection had run in between. Nothing was emailed, and nothing was in a paused or preview mode.

Subscriber state preserved. The last-alerted marker for every subscriber remained at the 2026-05-20 cutoff after the run — the same cutoff the 2026-05-21 run preserved. A run that matches nothing does not advance that marker.

Public pages. All seven checked public pages served normally, including the homepage, search, alerts, methodology, this audit log, pricing, and a permit detail page.

Status: CLEAR. The suppression set by the 2026-05-20 cutoff held across two consecutive scheduled runs. No data-quality defect is recorded by this entry.

2026-05-21 — Pre-launch verification: first scheduled alert run clear, public copy corrected, reversal points confirmed

First scheduled alert run. The daily alert run fired on 2026-05-21 at 12:00 UTC and recorded 0 sent, 1 skipped, 0 errors — not paused, not a preview. Only daily-frequency subscribers were considered on that day; the single daily subscriber had zero matching exceedance records since the 2026-05-20 cutoff, confirmed by direct count. Every subscriber's last-alerted marker remained at that cutoff after the run, which was set precisely to prevent a backlog of recovered historical records going out as one catch-up digest. All public pages served normally afterwards and the run logged no errors. Outcome: CLEAR.

Public copy corrected. A pre-deploy review found that three public pages still carried “pre-launch / paused / waitlist” wording that contradicted the live alert system and the recovered longer-than-monthly reporting periods. The wording was corrected: the homepage hero and alert prompts; the alerts page heading and its explanatory callout, which changed from “alert delivery is currently paused” to a description of how alerts work; and the methodology page's account of which reporting periods are collected and its data-freshness section, which no longer describes alerts as paused. Text only — no data, calculation, alert, or collection behavior was changed.

Public smoke test after deploy. All eight checked public pages served normally with no redirects. A scan of the served pages for the nine stale phrases (“waitlist”, “paused”, “will resume”, and the rest) returned zero hits across all eight. The alerts page renders in the browser, so its loaded scripts were scanned separately: all six current wordings were present and none of the six stale ones remained. The methodology page's current wording was confirmed in the served text, including its “Last Updated: 2026-05-21” line. A banned-term scan returned zero new occurrences. Export endpoints correctly refused to serve without a signed token — fail-closed. Outcome: CLEAR.

Baseline and reversal points. Published data at this point: 42,342 total / 42,341 active / 1 superseded. The most recent addition was the April 2026 monthly catch-up recorded in its own entry below. Records by qualifier: 39,588 measured / 1,491 reported above the laboratory detection limit / 1,262 non-detect. Records by reporting-period length: 41,092 of one month or less, 1,006 quarterly, 166 semi-annual, 77 annual, none longer. All 42,341 active records carried the Pennsylvania source. Thirteen retained snapshots were inventoried and every count matched a documented point in the operational record, including the current-state reversal point at 42,342 records, the point before the April catch-up at 42,200, the point before the long-period sweep at 40,393, and the original contaminated state from 2026-05-04 at 109,862 records, retained as legal history. Outcome: CLEAR.

Verification. Read-only inspection of the alert run record, subscriber timestamps, published totals, and every retained snapshot; deployment metadata confirmed against the deployed commit; public pages checked by unauthenticated request. Nothing was written.

Status. All four checks closed CLEAR. Public copy is consistent with the live alert system and with the recovered longer-than-monthly reporting periods. Reversal coverage is record-exact for every operational state change since 2026-05-04. No data-quality defect is recorded by this entry.

2026-05-20 — April 2026 monthly catch-up collection (142 records; explains the 42,200 → 42,342 change)

Context. On 2026-05-20, a monthly catch-up collection covering the April 2026 PA DEP reporting period was run out of its usual weekly cycle. This entry records it as a discrete event so that the earlier baseline (42,200 total / 42,199 active / 1 superseded, recorded in the 2026-05-19 entry below) and the baseline at the time alerts were switched on later the same day (42,342 / 42,341 / 1, recorded in the entry below) reconcile through a single named group rather than appearing as unexplained drift.

What was added. 142 records were added in a single batch at 12:42 UTC on 2026-05-20. All 142 carry the Pennsylvania source and a monitoring period of 2026-04-01 to 2026-04-30 — a single calendar month. The group spans 75 distinct permits. By qualifier: 129 measured, 4 reported above the laboratory detection limit, 9 non-detect — consistent with a normal statewide month. By statistical basis: Average Monthly 63, Instantaneous Maximum 22, Daily Maximum 18, Weekly Average 16, Instantaneous Minimum 9, Geometric Mean 8, Daily Minimum 4, Annual Average 1, Minimum 1. None was a correction to an earlier record and none was superseded. All were classified through the same calculation verified in the checks above; every spot-checked record recomputed its published exceedance percentage exactly. The monitor-and-report exclusion described in the 2026-05-19 entry below was already in effect, so no record without a numeric permit limit entered the published set.

Catch-up alert risk mitigated. The collection landed at 12:42 UTC, before the subscriber cutoff was set at 14:41 UTC and well before alerts were switched on at 16:55 UTC. That cutoff was set precisely to suppress this group — plus the broader window of recovered historical records — from going out as a single catch-up digest to the four eligible subscribers. The first scheduled alert run on 2026-05-21 recorded 0 sent, 1 skipped, 0 errors, confirming the suppression worked as designed.

Verification. Direct inspection confirmed 142 records, a single collection timestamp, a single monitoring period, a single source, and zero corrections, zero superseded, and zero monitor-and-report records. The arithmetic reconciles in both directions: 42,342 − 42,200 = 142 total, and 42,341 − 42,199 = 142 active. The snapshot taken before the catch-up is preserved at exactly 42,200 records and remains a strict subset of the published data; the snapshot taken after is preserved at exactly 42,342, a record-exact match to the live data at that time.

Reversal. Had a defect later been found in this group, it was uniquely identifiable three independent ways — by the range of identifiers it occupies, by its single collection timestamp, and by its April 2026 monitoring period — and removable back to the 42,200-record snapshot, verified against it.

Status. Closed. The group is in the published set and the baseline reconciles record-exactly in both directions. No data-quality defect is recorded by this entry.

2026-05-20 — Alert email delivery switched on

Context. Email digest delivery had been held since 2026-05-08 while seven dependencies were closed: the subscriber cutoff that prevents a catch-up digest; a preview mode that exercises the full path without sending; an explicit on/off control that fails closed; rotation of the credential that authorizes the scheduled run, verified by preview; an audit confirming no misclassified records could reach a digest; email deliverability with SPF, DKIM, and DMARC all passing and inbox delivery verified; and the activation step itself. All seven are now closed.

What changed. Three coordinated changes enabled the first live alert cycle: a daily schedule was added for the alert digest at 12:00 UTC; the credential authorizing that scheduled run was rotated to a fresh value, held only in memory, never written to disk and never re-read after use; and the on/off control was set to enabled, followed by a redeploy so the change took effect on fresh instances. The preview mode and the fail-closed control remain in place as defenses against an accidental send.

Verification. An end-to-end preview run was executed at 16:21 UTC on 2026-05-20 using the rotated credential. It reported one eligible subscriber, zero would-send, one skipped, zero errors, and was recorded as a successful preview. It exercised the full eligibility, matching, and digest-building path without sending any email and without advancing any subscriber's last-alerted marker. No subscriber record was changed. No email was sent during any part of the setup. Public pages served normally after the final redeploy.

Subscriber state at activation. Four eligible subscribers, all reset earlier the same day to a common cutoff of 2026-05-20 14:41 UTC. That reset suppressed a catch-up digest that would otherwise have shipped roughly 1,591-record digests to two Lancaster-county subscribers, covering the window of recovered historical records. One unverified, inactive subscriber exists and is correctly excluded. The cutoff guarantees the first live cycle includes only records collected after activation.

First live cycle. 2026-05-21 at 12:00 UTC. Only daily-frequency subscribers are considered on a non-Monday, and the single daily subscriber (following permit PA0254967) had zero matching exceedances since the cutoff, so the first run was expected to send nothing. Weekly subscribers were first eligible on 2026-05-25.

Reversal. Fastest path: set the on/off control back to disabled and redeploy. Slower path: remove the daily schedule so the alert path is not invoked at all. Last resort: rotate the credential, which causes the next scheduled run to be refused. Subscriber state, exceedance data, and the cutoff are independent of any reversal step and require no separate action.

Status. The alert hold was liftedas of 16:55 UTC on 2026-05-20. All seven dependencies closed. Published data unchanged from the pre-activation state (42,342 total / 42,341 active / 1 superseded). No data-quality defect is recorded by this entry — it documents an operational state change.

2026-05-19 — Longer-than-monthly reporting periods recovered, and a monitor-and-report collision corrected

Discovered during: Manual verification on 2026-05-10 of permit PA0243949 (Landis Block & Concrete Company, Telford Plant) against PA DEP eDMR. The 2024-10-01 to 2024-12-31 Annual Average Total Suspended Solids reading (99.0 mg/L against a permit limit of 50.0 mg/L) was visible at the source but absent from the published data. A sweep confirmed that all 38,967 active records at that time carried monitoring periods of 31 days or fewer; not a single longer period was present. (That is the 2026-05-10 discovery-state figure; after the recovery sweep the published set contained 1,249 long-period records — see the 2026-06-13 entry above.) A second, narrower defect was found on 2026-05-19 while spot-checking PA0244708 (Kinder Morgan Fairless Hills Facility): one record was in a superseded state carrying no numeric permit limit, occupying the identity that should have belonged to an Annual Average Total Dissolved Solids exceedance for calendar year 2021 (21,186.5 lbs/day against a 5,000 lbs/day permit limit).

Issue: The PA DEP eDMR source returns a reading only when the whole monitoring period falls inside the requested date range. Permit Minder's backfill asked month by month, which mechanically dropped every quarterly, semi-annual, and annual reporting period across the entire five-year span. A seven-window probe against PA0243949 was consistent only with that whole-period-containment behavior; three alternative explanations were each ruled out by at least one window.

Separately, the identity Permit Minder uses to recognize the same reading across collections — permit, parameter, monitoring period, outfall, statistical basis, units, and monitoring location — does not include whether the row carries a numeric permit limit. PA DEP eDMR returns both “monitor and report” readings (no permit limit, a measurement only) and limited readings (a numeric permit limit) under that same identity, and the original collection path kept both. When the canonical full-calendar-year reading for PA0244708 was later collected, the two collided, leaving the monitor-and-report row superseded and the real exceedance absent from the published set.

Affected scope: All PA DEP eDMR data with monitoring periods longer than 31 days — quarterly, semi-annual, and annual reporting periods — across the full five-year span. Permit detail pages for affected permits omitted those readings, and search aggregates inherited the omission. EPA ECHO data was not affected. The monitor-and-report collision was confined to a single record for PA0244708. A full sweep afterwards confirmed no monitor-and-report record remains in the published set and no similar collision exists anywhere else.

Disposition (recovery sweep): Three passes were run against PA DEP eDMR to recover the missing readings: statewide calendar-month queries, statewide calendar-quarter queries, and per-permit five-year queries against the combined universe of the EPA ICIS-NPDES Pennsylvania permit registry, every permit already published, and any additional permit list supplied. The five-year pass has to be per-permit because the source does not serve statewide windows longer than about three months. All three passes write through the same idempotent path, so an already-collected reading is a no-op.

Scheduling (as of this 2026-05-19 entry). Of these passes, only the calendar-month statewide pass ran on a recurring schedule. The quarterly and per-permit five-year passes were one-time recovery sweeps. Recurring scheduling of the long-period passes remained an open item, so newly-closed quarterly, semi-annual, and annual periods were not yet refreshed automatically. [Superseded twice. The long-period passes WERE put on a recurring schedule in June 2026, closing this open item within a month. That schedule has since been retired in turn: as of 2026-07-12 Pennsylvania is collected from the EPA ICIS-NPDES bulk feed, which publishes every monitoring period whatever its length, so no separate long-period sweep is needed. See the 2026-07-14 entry above. This paragraph is left as written because it was accurate on 2026-05-19.]

The full statewide five-year sweep was approved on 2026-05-11, after the permit-identifier correction recorded in the entry below, and ran against a universe of 37,431 permits (37,336 from the federal permit registry combined with 2,419 permits already published, deduplicated). It started on 2026-05-11 and completed all 37,431 permits on 2026-05-19. It was interrupted once mid-run and resumed without loss of state. Final counts: 1,807 records added, 36,245 already present and unchanged, 4,066,080 readings examined and not exceedances, 3 updated, 3 superseded, 2 restored, no unrecognized outcomes and no errors, across 4,104,140 readings submitted in total. The arithmetic balances exactly: the outcomes sum to 4,104,140.

Disposition (monitor-and-report collision): The collection path was changed to drop monitor-and-report readings before they reach the write path. The reasoning: “monitor and report” readings carry no permit limit and cannot represent an exceedance under any rule; allowing them through under an identity that does not distinguish them creates a collision with the real reading whenever both exist for the same monitoring period. Dropping them at the boundary preserves the real reading's identity end to end.

The single affected published record was corrected under a careful reversal protocol. A full copy of the record was captured first, with an explicit path to restore it under its original identifier, and a check confirmed nothing else referenced it. The superseded monitor-and-report record was then removed and the canonical Annual Average reading collected again through the corrected path. Exactly one record was added. It was captured as well, and reflects the corrected full-calendar-year monitoring period (2021-01-01 to 2021-12-31, rather than 2021-12-01 to 2021-12-31), a maximum limit of 5,000.0 lbs/day where the original carried none, and the resulting exceedance percentage of 323.73% against the same reported value of 21,186.5 lbs/day. A fresh PA DEP source check confirmed an exact match on every substantive field.

Verification: After the recovery sweep and before the collision fix, the published total moved from 40,393 to 42,200 against the snapshot taken beforehand (40,393 records), and the 1,807 newly-present records matched the sweep's own count exactly. Duplicate identities across the whole published set: zero. Split permit identifiers of the kind corrected in the entry below: zero. Missing qualifiers or missing monitoring-period bounds on new records: zero. Records outside the queried universe: zero. The hypothesis was confirmed on published data: recovered records surfaced in both the semi-annual and annual period ranges — exactly the categories the sweep was designed to recover.

After the collision fix: the superseded record was gone; the corrected record was present and published; exactly one record occupied that identity; duplicate identities across the whole set were zero; monitor-and-report records in the published set were zero; records with no numeric permit limit in the published set were zero; both shape-consistency checks passed; and the snapshot remained a strict subset of the published data at 40,393 of 40,393 records, so it stayed a valid reversal point. The arithmetic reconciles: 42,200 − 40,393 = 1,807 total, and 42,199 − 40,392 = 1,807 active. No reversal was required, and both copies of the corrected record are retained.

Status: Both the long-period recovery and the monitor-and-report collision are closed against published data. Baseline at entry time: 42,200 total / 42,199 active / 1 superseded. Alert delivery had not yet been switched on at this point.

[Superseded 2026-05-20: alert delivery was switched on the next day and has run daily since. This paragraph describes the 2026-05-19 point-in-time state only — see the 2026-06-13 and 2026-05-20 entries above for current alert posture.]

2026-05-11 — Leading zeros stripped from permit numbers on per-permit collection

Discovered during: A limited pre-launch trial of per-permit collection on 2026-05-11. A request was issued against PA DEP eDMR using the canonical zero-padded permit number 0104402; the response came back carrying 104402 — the source silently strips leading zeros from numeric permit numbers in permit-filtered responses. The response value was stored as written. The result was 160 new published records, of which 136 (across 21 distinct facilities) were written under the stripped form, splitting one facility's identity across two permit numbers. A parallel trial on the staging copy introduced the same defect across 18 facilities.

Issue: Permit Minder recognizes the same reading across collections by an identity that includes the permit number as text, so 0104402 and 104402 are treated as two different facilities. The standard checks all passed, because none of them detects leading-zero drift. A source spot-check exposed it: querying PA DEP eDMR with the stored stripped value returned nothing, meaning the permit number the site displayed would not lead a reader back to the source data the site cites. Email alert subscriptions are keyed on the permit number, so a subscriber following the zero-padded identifier would not have received alerts for new exceedances filed under the stripped form for the same facility.

Affected scope: Per-permit collection only. The statewide monthly and quarterly paths do not filter by permit and do not trigger the stripping; they were unaffected. Permits with a PA prefix (for example PA0243949, PAG058397) are unaffected because the source does not strip the prefix. The defect was confined to digit-only permit numbers with leading zeros, such as 0104402, 0102406, and 0214403.

Disposition (recovery): Published data was rolled back on 2026-05-11 by removing the 160 trial records, identified by the window in which they were collected and verified against the snapshot taken beforehand. Published data was restored to 40,364 total / 40,363 active. The staging copy was rolled back the same way, removing 195 records across two windows (165 from the first trial and 30 from a follow-up four-permit probe), restoring it to 39,166 total / 39,165 active.

Disposition (fix): The per-permit collection path now reasserts the permit number it asked for onto the response before anything is classified or stored, so the canonical zero-padded identifier is preserved end to end. The change is narrowly scoped; the statewide paths are untouched. Three tests were added — leading-zero preservation, PA-prefix preservation, and no writes on an empty response — and the full test suite passed.

Verification (staging). The fixed path was re-run on 2026-05-11 against four leading-zero permits (0104402, 0214403, 0998419, 0999405). It added 16 records; every one carried the canonical zero-padded permit number. The 41 records it recognized as already present matched the existing canonical counts for those four permits exactly (5 + 10 + 14 + 12 = 41), demonstrating that repeated runs converge. No stripped variants appeared, and a sweep of the whole staging copy for any padded/stripped pair returned nothing. A source check against PA DEP eDMR for two of the new records (0104402 Fecal Coliform Daily Maximum, first quarter 2022; 0214403 Total Nitrogen Daily Maximum, fourth quarter 2025) confirmed the reported value, permit limit, units, and monitoring location all match the source exactly, with the permit number in its canonical form. The staging additions were then rolled back so staging matched its 39,166-record baseline.

Verification (renewed trial on published data). After the fix was merged on 2026-05-11, a fresh 40,364-record snapshot was taken and the trial was re-run against a 100-permit universe. It completed all 100 permits in about 31 minutes: 29 records added, 902 already present and unchanged, 52,393 readings examined and not exceedances, nothing updated, superseded, or restored, and no unrecognized outcomes. Verification against the snapshot: total moved 40,364 → 40,393, active moved 40,363 → 40,392, superseded held at 1. All 29 added records were published and carried the Pennsylvania source. Both required checks held — zero records outside the queried universe, and zero padded/stripped pairs anywhere in the published data. Duplicate identities: zero. Missing qualifiers or monitoring-period bounds on new records: zero. A cross-check between the staging and published trials found no divergence in permit set or aggregates. A fresh PA DEP source check on one added record (permit 4600421, Fecal Coliform, monitoring period 2021-02-01 to 2021-02-28, outfall 001, Final Effluent, Geometric Mean, 2,800.0 CFU/100 ml against a permit limit of 200.0) returned an exact source match on every substantive field. The one representational difference — the source returned outfall “1” where Permit Minder stores the canonical three-digit “001” — is a deterministic normalization, not a value drift. The earlier defective trial that had added 160 records (136 of them stripped, across 21 facilities) had been rolled back beforehand; the renewed trial reintroduced none.

New checks now required before any per-permit collection run:

• Every newly added record's permit number must be a member of the permit universe that was queried.
• No two published records may carry the zero-padded form and the leading-zero-stripped form of the same facility's permit number.

Status: Fix verified end to end on both the staging copy and published data, with a fresh PA DEP source spot-check. Baseline after the renewed trial: 40,393 total / 40,392 active / 1 superseded. The 40,364-record snapshot remains a strict subset of the published data and a valid reversal point.

2026-05-08 — Headline exceedance percentages now exclude non-detect readings

Discovered during: Pre-launch defensibility review of the search-results card and the homepage statistics. Permit PA0253308 was the canonical example: laboratories report polychlorinated biphenyls (PCBs) using non-detect notation such as “less than 250 ng/L” when the substance is below the analytical detection limit. Against a permit limit of 0.064 ng/L, the arithmetic produced an apparent exceedance of 195,212.5%. That number was a property of detection-limit math (the lab can only state the substance was somewhere below 250 ng/L), not a property of the facility's discharge.

Issue: The two figures that drive headline numbers — the highest exceedance percentage on the search-results card, and the count of readings over 100% on the homepage — were computed across every reading regardless of how the underlying value was reported. Readings where the lab reported “less than the detection limit” were pooled with readings where the lab reported an actual concentration. Per-reading exceedance values on the permit detail page were not affected — the issue was confined to the headline figures on the search and homepage surfaces.

Affected scope:The search-results card's highest-exceedance column, the homepage count of readings over 100%, and the minimum-percentage search filter. The permit detail page, CSV and PDF exports, and per-reading data were not affected — those surfaces show the value the laboratory actually reported, with its qualifier preserved alongside the number.

Alert delivery was not affected, because the alert path reads the underlying records directly and does not depend on the search and homepage calculations that changed. The alert digest was already non-detect-aware through two independent layers: non-detect readings are excluded from the digest entirely at the point the records are selected, and a second filter inside the digest builder ensures a per-facility headline percentage cannot be driven by a non-detect reading even if a future caller passed unfiltered records in. The first of those is a stronger guarantee than the change described here, because non-detect readings are removed from the digest altogether rather than merely excluded from a maximum.

Disposition: Each reading now carries a classification recording whether its reported value is a measurement, a non-detect (a “less than” report), a reading above the detection limit (a “greater than” report at the detection-limit floor), or something else. On 2026-05-08 the four read paths behind the search card, the search count, the permit detail page, and the site statistics were changed so the headline figures exclude non-detect readings. Readings above the detection limit are retained in the headline figures, because a “greater than” report carries a known floor — the actual concentration is at least the displayed value — so including it is the conservative choice.

These were read-side changes only. Nothing was rewritten, deleted, or superseded. The same per-reading data Permit Minder always stored remains intact; only the calculation that produces the headline display number changed. Underlying counts, exports, and the per-reading exceedance figures on the permit detail page are unchanged.

Verification. After the change, the headline figure for the canonical pathological case PA0253308 changed from 195,212.5% to 597.33% — the highest measured exceedance for that permit, with non-detect readings no longer driving it. The site-wide count of readings over 100% changed from 14,740 to 14,255, the difference being 485 readings whose percentage was produced by non-detect arithmetic and which are now excluded from that count.

Why this matters for legal defensibility. A non-detect reading establishes only that the substance was below the laboratory's detection limit. Treating that detection-limit ceiling as if it were a measured value produces percentages that cannot be defended as the facility's actual discharge. Excluding non-detect readings from headline figures is the conservative reading: the platform now reports the highest measured exceedance, plus where applicable the highest known floor (above-detection-limit reports), and surfaces non-detect readings separately as per-reading data on the permit detail page so a reader can examine them with the qualifier intact.

Status: Shipped and live in production as of 2026-05-08. A future enhancement may surface per-permit counts of measured, non-detect, and above-detection-limit readings on the search-results card so readers can see, at a glance, how many readings of each kind contributed to a permit's record; the underlying data is already available and that work is tracked separately.

2026-05-07 — Non-detect screening separation + cross-surface streak removal

During pre-launch defensibility review we identified three independent issues affecting how Permit Minder presents laboratory non-detect samples and a removed feature. Fixes were implemented and pending production deployment at the time of this entry.

Non-detect samples ranked alongside measured exceedances. When a laboratory cannot detect a substance at the stated detection limit, the operator reports a value such as “less than 250 ng/L”. For percent-over-limit calculations, Permit Minder applied the conventional half-detection-limit substitution. For some parameters and permits — most prominently PCBs at permit PA0253308, with a permit limit of 0.064 ng/L and a reported “less than 250 ng/L” — that substitution mathematically exceeds the permit limit and produced an exceedance percentage of 195,212.5%. The actual concentration is unknown and may well have been at or below the permit limit. Approximately 1,160 published readings carry this pattern.

Disposition. Measured exceedances are now separated from non-detect screening readings. The permit detail page renders non-detect readings in a distinct "Non-detect Screening" section. The PDF export does the same. The CSV export adds a "Value Type" column. Email digests exclude non-detect screening readings entirely. The search results page no longer ranks them alongside measured exceedances. The methodology page documents the substitution and the separation.

Cross-surface streak inconsistency. An earlier design decision removed rolled-up streak summaries from the permit detail page, but several downstream surfaces had not been updated to match. The PDF export had a streak column, the methodology page described streaks as a feature, the search results page displayed streak filters and badges, and the alerts subscription form collected a streak threshold that was no longer used. All of this has been removed; the reasoning is preserved so the feature's intentional absence survives future work. Permit Minder does not display, compute, or describe consecutive-exceedance streaks on any shipped surface.

Methodology coverage figure corrected. The methodology page previously stated approximately 23% of permits had no receiving-stream value. The current figure is approximately 6%, after recent backfills. The page was replaced with a new draft that includes a field-by-field "Source of every field" table, a non-detect values section, and the corrected coverage figure.

Status. Implementation complete with all tests green. Data changes were authored and pending application at the time of this entry, and the site remained under maintenance posture until verification completed.

2026-05-04 — Data quality cleanup; corrected coverage count

During pre-launch verification we identified approximately 47,700 published readings that should not have been there: about 24,800 were measurements that complied with permit limits (for example, a pH reading well within the allowed range) that had been incorrectly retained as if they were exceedances; about 21,900 were readings where the permit limit or measured value was missing entirely, with nothing to compare; and about 1,000 came from a previously-known column-shift defect that had recurred through an intermediate file.

Coverage count correction. Because some of the bad readings had garbage permit identifiers, our total permits count was inflated by 987. The corrected count is 2,322 permits, down from a previously-displayed 3,309. This is a correction of an over-count, not a reduction in what we cover — we have always tracked the same set of real PA NPDES permits.

What this means for prior exports. If you previously exported data or generated reports for specific permits, the underlying exceedance counts are unchanged; only the permit-detail row counts (which included compliant measurements alongside exceedances) and the site-wide total permits figure were affected.

Verification. We compared our data against EPA ECHO source records for three sample permits (2,988 source readings) and confirmed the contamination was compliant measurements being incorrectly retained, not real exceedances being missed. Exceedance counts, alert emails, and search results were already filtered to actual exceedances and were not affected.

Root cause. A maintenance step preserved cleaned-up readings in an intermediate file, which subsequent scheduled loads then re-inserted. Both the step and the loader have been corrected.

Resolution (verified). The cleanup was executed and the corrected collection path has run on every scheduled collection since. Read-only verification confirmed the contamination is gone: zero misclassified EPA ECHO readings and zero compliant readings stored as exceedances remain in the published set. The legacy direct write paths were removed, so there is now a single, filtered path by which anything can be published. The original contaminated state is retained as a snapshot for legal history. This item is resolved.

2026-05-01 — Permit detail “Periods Within Permit Limits” card removed (label/denominator mismatch)

Discovered during: Pre-launch surface audit, while reviewing the permit detail page rendering for permit PA0001406 (which displayed “0 of 24 records — 0.0% Periods Within Permit Limits” alongside “Total Exceedances: 24”).

Issue: A summary card on every permit detail page reported a percentage labeled “Periods Within Permit Limits”. The label suggests a compliance rate across the permittee's full monitoring history. What the card actually computed was the share of the readings Permit Minder had collected and displayed for that permit that were not exceedances. The two are not the same. There were three structural gaps between the label and the number: the denominator depended on what we had collected, not on the permit's full monitoring history; readings with both the permit limit and the measured value missing (the 4,687 placeholder readings tracked separately) were silently filtered out before the calculation; and for 439 permits (14.3% of the 3,079 then published) the card always showed 0.0% because every compliant signal we have for those permits sits in that placeholder set. A reader of an affected permit page would reasonably infer “this facility never complied” when the actual statement was “every compliant reading we have for this permit was filtered out before this card rendered.”

Affected scope: Permit detail page only. Methodology, audit log, search results, homepage stats, alerts, and exports were not affected.

Disposition: Card removed pre-launch, along with the calculation that fed only that card. The page still shows the latest monitoring period in our data for the permit — a defensible statement, since it makes no claim about compliance.

Status: Removed pre-launch. A defensible compliance-rate display would require data-model work to (a) define what counts as a monitoring period for a given permit, (b) collect non-exceedance periods completely or document the coverage gap, and (c) decide how to present coverage gaps to attorneys and journalists. Open question post-launch.

2026-05-01 — Search results card: highest percentage and most common parameter computed independently

Discovered during: Scoping for direction-explicit display copy on the permit detail and search surfaces.

Issue:The search results card shows two figures per permit that a reader naturally associates with each other but which are computed separately. The highest exceedance percentage is the maximum across every displayed reading for the permit, while the parameter shown beside it is the permit's most frequently occurring parameter. The two do not necessarily come from the same reading. A permit whose most common parameter is Total Suspended Solids (a maximum limit) but whose single highest-percentage reading is a Dissolved Oxygen minimum would show the parameter from the first calculation beside the percentage from the second — visually implying a relationship that does not exist in the underlying data. The defect was invisible while the display copy assumed every limit was a maximum, and surfaced when the direction-explicit rewrite needed to know the statistical basis of the reading that produced the percentage, and found that it was not available on that surface at all.

Affected scope:Search results card and the search-results CSV export. The permit detail page renders per-reading data with full context — parameter, statistical basis, qualifier, permit limit, and reported value all present for each reading — and is unaffected.

Disposition: Permit detail surfaces shipped direction-explicit copy using the per-reading classification. Search card wording remains the previous maximum-limit-only “+X% over reported limit” phrasing until a post-launch follow-up makes the direction of the highest reading available on that surface. The methodology page discloses that direction-aware detection applies and notes the historical recovery work in flight, which covers the gap in the meantime.

Status: Open. Pending post-launch work.

2026-05-01 — Same logical exceedance published more than once (cross-source and within a single source)

Discovered during: Verification work, while cross-checking the one published reading with an unrecognized statistical basis against the rendered preview.

Issue:Published readings carry no uniqueness requirement on their natural identity — permit, parameter, monitoring-period end, permit limit, and reported value. The same logical exceedance can therefore appear more than once: when a reading arrives from PA DEP and again from EPA ECHO with the same number but a different statistical basis (PA DEP sometimes records the basis as unknown where ECHO has the proper code), or when ECHO collection writes the same reading twice with slightly different field state (a qualifier present in one and absent in the other, or a monitoring-period end populated in one and missing in the other).

Platform-wide totals across displayed readings: 155 distinct natural identities appear in more than one reading; 334 readings in total belong to those duplicated identities; about 1.1% of all displayed readings are attributable to duplication. Cross-source duplication accounts for 1 of the 155; the remaining 154 are duplications within a single source, mostly within ECHO.

Worst-affected permits by extra readings: PA0254967 (38 extra readings), PA0053091 (21), PA0001937 (20), PA0215520 (10), PA0008281 (7).

Reader-visible symptom on the permit detail page: the same parameter and period appear in the exceedance table multiple times, with the same numbers but different Limit Type values (a “—” row from PA DEP plus an “Instantaneous Maximum” row from ECHO, for example).

Affected scope: Permit detail page (visible duplicate rows for affected permits). Search-card figures that count readings inherit small over-counts on those permits. Methodology and audit-log surfaces are unaffected.

Disposition:De-duplication is post-launch. Two approaches are under consideration: enforcing uniqueness on the natural identity with a tie-break preferring the source with the more specific statistical basis, or collapsing duplicates at display time. Either requires deciding which reading is canonical — likely PA DEP for permit-level detail and ECHO for the statistical basis where PA DEP records it as unknown. Until then, readers on the affected permits will see duplicate rows on the permit detail page.

Status: Open. Pending post-launch work.

2026-04-30 — ECHO collection: minimum-limit parameter override never took effect

Discovered during: Fixture-verification work (regression tests for direction-aware exceedance detection).

Issue:The collection path has an override that classifies certain parameters (Dissolved Oxygen, % Removal, Stream Flow Minimum) as minimum limits regardless of the statistical basis, because those parameters are always minimums by EPA convention. The override was silently inactive on ECHO data: it looked for a parameter field under one name, while raw ECHO data carries it under a different name until later in the pipeline. So for ECHO readings where the parameter was one of those three, the statistical basis was empty, and the qualifier was not a “greater than or equal”, the override did not fire and the reading was misclassified as a maximum limit. Real minimum-limit exceedances in that slice were silently dropped.

Affected scope: ECHO data only. PA DEP data was not affected (its collection has always been direction-aware and uses the later field names). Slice size: zero readings visibly miscategorized in the published data at the time (every parameter matching the override already had a recognized statistical basis or an explicit qualifier). The upper bound is unknown without re-pulling raw ECHO data, since dropped readings were never stored.

Disposition: Fixed. The override now accepts the raw ECHO field name as a fallback. This is a pure widening — where the original field is present, behavior is identical to before; where only the raw name is present, the override now fires. Verified against fixtures: a previously-dropped Dissolved Oxygen reading with no statistical basis is now correctly flagged, and a compliant one is still correctly not flagged (no over-flagging). Regression tests assert the post-fix behavior. Historical recovery of dropped readings is part of the ECHO re-pull.

Status: Fix shipped. Historical recovery pending the ECHO re-pull.

2026-04-30 — Homepage statistics included placeholder readings

Discovered during: Audit of readings whose limit direction could not be determined.

Issue: The homepage statistics did not filter to actual exceedances, so they counted 4,687 PA DEP readings with no permit limit and no reported value (placeholder readings from a separate data-quality issue). Every other surface — search, permit detail, alerts, exports — already filtered those out. The homepage was the only inconsistent surface. Pre-fix counts were inflated: total exceedances by 45.9%, total permits by 24.6%, parameters tracked by 28.3%.

Affected scope: Homepage display only. No alerts, exports, or permit-detail pages were affected.

Disposition: The homepage statistics now filter to actual exceedances. Verified post-change counts match expectation: total exceedances 30,022, total permits 2,322, parameters tracked 129.

Status: Shipped. The 4,687 placeholder readings themselves are a separate data-quality issue tracked in the post-launch backlog.

Pre-2026-04-30 — Historical ECHO collection was direction-blind

Discovered during: Pre-launch defensibility audit.

Issue: Historical ECHO collection compared the reported value against the permit limit uniformly, without considering whether the limit was a maximum or a minimum. For minimum limits (Dissolved Oxygen, the lower bound of a pH band, % Removal, Stream Flow Minimum, and any limit carrying a minimum-type statistical basis), this meant every compliant reading was incorrectly flagged as an exceedance, and every reading that should have been flagged as a minimum-limit exceedance was incorrectly recorded as compliant. Roughly 660 ECHO-only permits were affected.

Affected scope: ECHO data only. PA DEP data was not affected. The current collection path is direction-aware; only historical collection was affected.

Disposition: Three cleanup passes removed the incorrect exceedance flags. The opposite error — minimum-limit exceedances that were never stored because the direction-blind comparison rejected them — could not be recovered from the existing data; those readings must be re-pulled from ECHO source data. Estimated recovery: about 17,936 historical minimum-limit exceedances. The re-pull is scheduled post-launch, with a methodology-disclosure fallback if it does not pass sample validation.

Status: Incorrect flags cleaned. Recovery of the missed readings pending the ECHO re-pull.

How this document is maintained

Entries are added when a data-quality issue is discovered, regardless of whether it's user-visible. Each entry covers: what was found, what was affected, what was done, and current status. Entries are not deleted when issues close — closure is recorded in the Status field.

Future audit findings follow the same template: Discovered during / Issue / Affected scope / Disposition / Status.