A search index is only as trustworthy as the pages it lets in. That matters even more when the reader is an AI system.
A person can open a doubtful result, notice that it is thin or irrelevant, and leave. An AI system grounding an answer has less room for that kind of judgment once retrieval has already happened. If the index returns spam, copied pages, stale material, or sources that only look relevant, the answer inherits that weakness.
IsonSearch Curator is the quality layer we are building to reduce that failure mode. Its job is to make sure the pages entering IsonSearch are useful enough to base answers on.
What a clean index keeps out
The open web is not a clean library. Useful Indonesian sources sit next to spam, tracker pages, scraper traps, copied articles, empty pages, and content built mainly to rank.
Left alone, that material does more than waste storage. It changes what a search system sees as normal. Low-value pages can crowd out primary sources, duplicate the same claims across many URLs, or make a weak source look more important than it is. For AI retrieval, that is a real problem: a noisy index produces noisy evidence.
Curator is built to keep that material out where possible, and to push it down when removal is not the right action. The goal is not to collect the largest possible number of pages. The goal is an index where credible Indonesian sources are easier to find first.
Judging quality the web does not label
The web rarely labels quality in a way a machine can simply read. Some signals are measurable. Others require judgment.
Curator combines both. Clear cases are handled by rules. Harder cases are reviewed with help from an AI model, which assesses whether a source is credible, thin, copied, misleading, or simply not useful for an Indonesian reader. The model also produces a short rationale, so the system is not acting on an unexplained score.
Two principles guide this judgment.
First, keep when in doubt. Removing a useful source by mistake is usually more costly than tolerating a weak one. The bar for exclusion should be higher than the bar for caution.
Second, compare like with like. A regional news site, a local government page, or a small institutional website should be judged against its peers, not against the presentation standards of a national outlet. A clean index should not erase useful local sources just because they look informal.
Automation that can be corrected
Curating a web index is continuous work. It cannot be done page by page by hand, so routine decisions have to run automatically. But autonomy is not the first goal. Correctability is.
Curator is designed around that constraint. Routine and reversible actions can run without waiting for a person: adjusting crawl priority, clearing obvious junk, handling routine failures, and demoting pages that clearly should not surface. Each action is recorded. Each action can be undone.
Before acting, the system has to preserve the evidence behind a decision and check for signals that argue against it. A confident result with weak support is not a reason to move faster. It is a reason to wait.
Anything that cannot be safely reversed stays with a person. We want automation that earns trust because it can be inspected and corrected, not because it sounds certain.
Where it stands
IsonSearch Curator is in active development as part of IsonSearch, our search engine for the Indonesian web optimized for AI retrieval. The search engine itself is introduced in a separate note, Introducing IsonSearch.
Organizations interested in Indonesian-specialist, AI-ready search can reach us at research@epithre.com.