AptifoldContact

Statistics / The Reading Room

How can a 95% filter
find nothing?

You need melting temperatures to choose a material for a hot part. Try a filter that flags research paragraphs and see what reaches your desk.

Teaching example with an invented collection and known answers.

The researcher's desk

2,000 paragraphs · 100 contain the measurement

The example flags 90 of 100 useful paragraphs and 5% of the rest.

Set a reading budget
Decisions correct
95%
Flagged with measurements
0

1,900 correct skips count toward the score.

Nothing flagged to read. All 100 useful paragraphs were skipped.

Useful, flaggedUseful, missedFalse alarmCorrect skip

One tile = one paragraph. Select a count below to find its tiles; exact counts carry the detail on small screens.

All four outcomes shown.

What reached the reader?

The score counts skips. The desk stays empty.

Skipping the 1,900 other paragraphs counts as success. Missing all 100 useful ones still leaves 95% of the decisions correct. The researcher gets none of the measurements.

Go further: compare two ranked searches on the same labelled paragraphs ↗. Inspect shared misses and what a reading budget reaches.

Look inside the percentage

The score answers
a different question.

“How often was the tool right?” counts useful finds and correct skips together. The researcher's job also needs two other questions: How much useful material did it keep? And how much of the reading pile is useful?

Try the useful filter. It keeps 90 useful paragraphs, but 95 other paragraphs also enter the pile. Its 94.75% accuracy is lower than the empty-desk score of 95%. The researcher finally has measurements to check.

Useful for this task
“The sample melted at 72 °C.”

It contains the measurement being collected.

Other text for this task
“We weighed the sample before the test.”

It may matter elsewhere; it doesn't supply this temperature.

Authored examples, not quotations from a paper. Finding a temperature does not establish its reliability or suitability for a material.

From a published study / Swain & Cole, 2016

100% precision.
For which job?

To build a materials database, a researcher needs each measurement tied to the right substance. Swain and Cole tested ChemDataExtractor on 50 hand-checked chemistry articles. Table 6 scores values and complete property records separately.

Choose what counts as success

Property value: 100% precision, 95.9% recall.

Reported precision100%

Correct retrieved values / all retrieved values

Reported recall95.9%

Correct retrieved values / reference values

100% is the reported precision for values. Recall is 95.9%, so this is not complete extraction. A correctly read value still needs the right subject and context to make a usable record.

What would a usable record need?
Subject
Sample AContext for this unit
Property
Melting pointContext for this unit
Value
72Included in this unit
Unit
°CContext for this unit

Authored illustration, not an extracted study row. Full records can include more context.

A right value attached to the wrong substance is still a wrong record.

Published, rounded results from a 2016 evaluation. This extracts records; the desk above filters paragraphs. These scores do not change its hypothetical rates, reading budget or saved setup.

Inspect the source table and what the study excluded
Two reported rows from Table 6 · not new calculations
Scored unitPrecisionRecallF-score
Property value100%95.9%97.9%
Full property record93.5%89.6%91.5%

F-score combines precision and recall using their harmonic mean. These are the authors' rounded results.

The evaluation included abstracts, body text, tables and figure captions. It excluded information found only in scheme/figure images or separate supplements. Scores cover multiple property types, not melting points alone. These table rows do not supply exact confusion counts or a reading order.

Swain & Cole (2016) · journal DOI ↗ · Authors' accepted manuscript ↗. See the evaluation on manuscript p.21 and Table 6 on p.24. Scientific credit stays with the authors; no extractor runs here.

Same chosen rates, another collection

A small false-alarm rate
can fill a large pile.

Keep the filter's rates fixed. Keep all 100 useful paragraphs. Change only how much other text surrounds them.

Each bar divides its own flagged pile; bar widths do not compare total collection sizes. This controlled arithmetic does not predict a real tool's rates after a collection changes.

Now add a deadline

How much can you find
before you stop reading?

A budget of 100 lets you check the first 100 flagged paragraphs, or the whole pile if it’s smaller. When the order is unknown, the result shows the fewest and most useful paragraphs you could find.

Nothing flagged to read. All 100 useful paragraphs were skipped.

Try the example filter first

0 useful paragraphs found by checking.

Each check uses one place in the budget.

Try two possible orders

The two extreme orders illustrate what could happen. The calculation assumes you recognize each useful paragraph you check.

What the totals establish

0

useful paragraphs found by checking

There are no flags to inspect. A bigger reading budget alone cannot recover a paragraph the tool left out.

Inspect the live calculations and exact reading bounds

All counts below come from the selected constructed collection. Percentages are rounded for display; exported fractions retain the calculation.

What a reading budget can establish

Let T be useful flags, F be false alarms, and B the reading budget. You inspect I = min(B, T + F) flags. The fewest useful paragraphs you could find is max(0, I − F); the most is min(I, T).

Putting false alarms first attains the lower count. Putting useful flags first attains the upper count. These are exact limits over possible orders, not a confidence interval. Multiplying precision by the budget does not tell you what this unknown order contains.

Useful paragraphs left unchecked range from 100 minus the upper count to 100 minus the lower count. This includes useful paragraphs the tool never flagged. A real review can also misread text, take unequal time, or need source verification; none of those effects is modelled here.

Keep the comparison

Take the workings
with the score.

Save the constructed desk, share its settings, or export both collections and their reading bounds. The calculations and setup files stay local until you choose to share them.

Scientific source, model limits and credits

The native Atlas classification kernel performs exact counts for two constructed collections: 100 target paragraphs with either 1,900 or 100 other paragraphs. Whole-percentage rates produce integer counts. This is a teaching instrument, not a trained classifier, literature prevalence estimate, attainable threshold curve or benchmark. It does not assess whether an extracted measurement is correct. Tiles are grouped outcomes, not source documents or a ranked list.

Metric definitions and the class-imbalance warning were checked against Google's Machine Learning Crash Course ↗, 3 November 2026. The reading-budget limits are direct counting bounds, independently checked by enumerating small queues. The desk counts and example paragraphs are constructed. The published scores are separately attributed; no article text, figures or full paper are bundled.

Aptifold’s AI agent researched the sources, implemented the models, designed the visuals, wrote the code and ran the tests. Aptifold’s founder set the goals and reviewed the results. Software: MIT License. Linked scientific and educational sources retain their own authorship and terms. No institutional endorsement is implied.

APTIFOLD / AGENTIC AI

What would you like to work through?

Aptifold develops its own agentic AI to investigate questions, test ideas and build working software. This experience is one example of that work.

Explore Aptifold Discuss a question

Company links open in a new tab, so you can keep this experiment open.