Improving the Information Behind AI-Assisted Assessments: Lessons from Vulnerability Assessment

When assessment AI returns “no information,” does that simply mean that more data needs to be registered?

Through our work providing AskDona Batch Assessment, we have come to see that information gaps have several different causes—and that each calls for a different response.

This article outlines a practical approach: rather than completing a large-scale data cleanup before using assessment AI, start with the information already available, run the assessment, and use the results to identify what actually needs to be improved.

About this article
This article shares practical observations from GFLOPS as the provider of AskDona Batch Assessment. It does not describe any specific customer, organization, system, external service, assessment result, or operating environment. The examples combine and generalize common issues in a way that does not identify any particular organization.

What is Batch Assessment?

Assessment, audit, review, and checklist-based work often involves hundreds of items. For each one, someone must locate relevant policies, design documents, specifications, inventories, contracts, and operational records, then compare them with the applicable evaluation criteria.

AskDona Batch Assessment supports this process by having an AI agent locate relevant source material for each item and organize a first-pass assessment, rationale, supporting quotations, source references, and missing information.

Organizations can use their existing Excel assessment sheets and draw on documents and evidence stored in AskDona’s RAG knowledge base. A human reviewer then checks the AI-organized result, adds or corrects information where necessary, and makes the final decision.

Batch Assessment is not limited to one type of evaluation. It can be applied to work such as:

system risk assessments;
information security assessments;
supplier and vendor assessments;
internal control and audit reviews;
quality audits and specification-conformance reviews;
contract and application reviews; and
customer questionnaires and other structured checklists.

These activities share a common structure: they require people to review multiple sources, compare them with evaluation criteria, and produce an evidence-based first-pass judgment.

Using system risk assessment as a concrete example

This article uses the review of security controls in a system risk assessment as a practical example for examining why information gaps arise and how they can point to targeted data improvements.

Here, “vulnerability assessment” is not limited to automated technical scanning. It is used more broadly to include work that reviews design documents, policies, specifications, architecture materials, configuration records, inventories, logs, and test results in order to identify weaknesses, verify control implementation, and assess a system against defined criteria.

We use this example not because system risk assessment is uniquely difficult, but because it brings together many issues that also appear in other assessment work:

a single question may contain several conditions that must be checked separately;
terms such as “appropriate,” “publicly trusted,” “equivalent,” or “managed” may not have one self-evident boundary;
the subject, scope, or assessment date may be omitted from the wording;
a stated policy must be distinguished from evidence that a control was actually implemented or operated;
different forms of evidence—policies, design documents, configuration records, inventories, and logs—must be interpreted according to their role;
internally implemented functions must be distinguished from capabilities provided by external services or shared platforms;
missing information, non-compliance, and out-of-scope conditions must be treated separately; and
several findings may need to be aggregated into one final result.

Because language interpretation, scope definition, evidence review, technical understanding, and organization-specific judgment all intersect, this is a useful example for examining what an AI agent needs, where it may apply the wrong premise, and what remains unresolved as missing information.

The same structure also appears in supplier assessments, quality reviews, contract checks, customer questionnaires, and legal or standards-conformance reviews—anywhere documents and evidence must be turned into a formal judgment.

How does assessment AI produce a first-pass result?

In Batch Assessment, the process begins by importing the relevant items from an existing assessment sheet.

For each item, the system then follows the configured rating definitions and execution prompt, searches the RAG knowledge base for relevant documents and evidence, and compares the retrieved information with the assessment requirement. The first-pass result is based on facts that can be confirmed from the available sources.

Read the assessment item
        ↓
Break down the required conditions
        ↓
Search RAG for relevant documents and evidence
        ↓
Compare the requirement with the retrieved information
        ↓
Organize the proposed result, rationale, quotations,
sources, and missing information
        ↓
A human reviews the evidence and makes the final decision

Batch Assessment allows each assessment to define its own rating labels, number of rating options, and decision criteria. The options are not limited to labels such as “compliant,” “non-compliant,” “out of scope,” or “indeterminate.” Depending on the workflow, an organization may define options such as “no information,” “insufficient evidence,” “additional review required,” or “conditionally compliant.”

The organization can also specify what must be confirmed for each result, what type of evidence is required, how multiple conditions should be aggregated, and how insufficient information should be handled. The overall execution prompt can be configured separately to define the order of review, distinguish policies from implementation evidence, verify scope and assessment date, and prohibit unsupported inferences.

When the necessary statements are found and the configured conditions can be confirmed, the AI can prepare a first-pass assessment supported by evidence.

When the information required for a decision cannot be confirmed, the result may instead be “no information,” “insufficient evidence,” or “indeterminate,” depending on the organization’s definitions.

An AI agent not confirming information does not mean the information does not exist

The critical distinction is that an AI agent’s inability to confirm information is not the same as the information not existing within the organization.

AskDona is designed to retrieve relevant information from registered RAG data and generate accurate, evidence-grounded answers. Even strong retrieval, however, cannot create evidence that lies outside the sources selected for a particular assessment, nor can it invent the missing context needed to establish that a document applies to the assessment target.

Strictly speaking, the AI can conclude only that:

Within the specified RAG database, mandatory reference documents, reference scope, and retrieval conditions, it could not confirm information supporting the assessment item.

Several different situations may sit behind that result:

The required information is registered in RAG, but it is not included in the database, mandatory references, or filter conditions selected for this assessment.
The document does not clearly identify its subject, version, environment, or scope, so it cannot be treated as evidence for the assessment target.
The document exists elsewhere in the organization but has not been registered in RAG.
The control is performed, but no execution record has been created or retained.
The control itself has not yet been implemented.
The item does not apply, so the corresponding evidence does not need to exist.
The relevant source should be an external official document rather than an internal record.
The context connecting external information to internal use or implementation is missing.
The evaluation definition itself does not specify what must be confirmed.
Target-specific architecture or change history exists only in the knowledge of the responsible staff.

These may all appear as “the required information could not be confirmed,” but the appropriate improvement is different in each case.

“No information” is not the end of an assessment. It is the starting point for identifying where the gap lies and who needs to address it.

Four areas for improving missing information

When information cannot be confirmed, it is useful to avoid assuming a single cause. Dividing the issue into four areas makes the next action clearer.

Improvement area	What is happening	Primary improvement
1. Assessment-design gaps	It is unclear what must be confirmed to reach a decision	Define terminology, applicability, evidence requirements, and decision rules
2. Retrieval and registration gaps	The information exists, but it is not usable in the current assessment	Review the reference scope, mandatory references, registration status, document structure, and target mapping
3. Evidence and control gaps	The record does not exist, or the control itself has not been implemented	Improve evidence retention or the actual control and operating process
4. Source and responsibility-boundary gaps	It is unclear whether internal or external information is required, or who owns the control	Clarify target context, external dependencies, applicability, and responsibility boundaries

1. Assessment-design gaps

The organization may not yet have defined what must be confirmed in order to reach a decision.

Key terms or their boundaries are ambiguous.
Applicability conditions are unclear.
The evidence required for a positive result is not defined.
Aggregation rules for multiple conditions are missing.
The result to use when information is insufficient has not been defined.

Adding more documents will not stabilize the assessment in this situation. The first things to review are the rating definitions, the wording of the assessment item, and the execution prompt.

For example, the organization may need to specify whether a statement that “a certificate is used” is sufficient to conclude that a publicly trusted certificate is used, or whether a policy statement alone is enough—or whether configuration records and evidence of operation are also required.

2. Retrieval and registration gaps

The required information exists, but it is not available to the assessment in a usable form.

This does not simply mean that the RAG engine has weak retrieval. To make full use of AskDona’s retrieval capabilities, the organization must also configure which information is passed into the assessment, from which scope, and in what unit of context.

Possible causes include:

The selected RAG database is not the appropriate one for the assessment.
Documents that should be reviewed for every item have not been configured as mandatory references.
Filters for target, department, or category do not match the actual data.
Formal names, aliases, and former names are not mapped, so the system cannot establish that two references concern the same target.
Only an outdated document or a document for a different environment has been registered.
The document exists internally but has not been registered in RAG.
List-based information is split too finely, preventing the assessment from retrieving the related premises together.

When each CSV row is registered as one information block, it can be effective to keep the key conditions needed at the start of an assessment in the same row—for example, whether System A is internally developed, whether it handles personal data, its criticality, external connectivity, and who manages its identities.

System ID,System Name,Delivery Model,Personal Data,Criticality,External Connectivity,Identity Management Owner,Assessment Date
SYS-A,System A,Internally developed with selected external services,Yes,A,Yes,Shared identity platform,2026-07-01

One-to-many information—such as multiple locations, external services, or communication paths—can be managed in separate lists. Each row should still contain the system ID and system name so that the row remains understandable even when retrieved on its own.

3. Evidence and control gaps

In some cases, the document that would need to be registered does not exist. This still requires further separation.

The control is performed, but no evidence is retained

A periodic review or approval process may be operating, but the organization may not retain completion records or approval history.

The necessary improvement is not another RAG upload. The underlying process must generate and retain evidence when the work is performed.

The control itself is not implemented

The required configuration, approval, review, monitoring, or other activity has not been carried out, so no evidence exists.

This is not primarily a data problem. The organization must improve the actual control or operating process.

The item does not apply

The assessment target does not meet the applicability conditions, so the corresponding evidence is not required.

The absence of evidence should not itself be used as proof that the item is out of scope. The organization should retain the target-context information that supports the out-of-scope conclusion.

4. Source and responsibility-boundary gaps

When external services, products, standards, or industry platforms are involved, the necessary information may not be contained entirely in internal documents.

The issue is not simply whether more external information should be added. The organization must clarify what should be confirmed from external official sources, what should be confirmed from internal design, configuration, and operational evidence, and who is responsible for the control.

Questions include:

Is an official external requirement or product capability relevant to the item?
Can internal records confirm which function, version, and configuration the assessment target actually uses?
Is the responsibility boundary between the provider and the organization clear?
Can the organization confirm that the external requirement or capability applies to the assessment target?
Is the control performed by the organization, the provider, or both?

External official documentation can establish what a service requires or provides. It cannot, by itself, prove that the function is used, configured, and operated for the internal assessment target. Conversely, internal documents may not establish the authoritative requirements of an external standard or product.

When the source and responsibility model cannot be connected, that missing context should itself be reported as an information gap.

Dividing missing information into these four areas makes it possible to determine whether the real improvement lies in the evaluation specification, the reference scope, the evidence process, or the control itself—before simply adding more data.

Run the assessment first. The information that needs improvement will become clearer.

Assessment AI creates a practical chicken-and-egg problem:

Without well-organized information, assessments are difficult to stabilize
                            ↕
Without running assessments, it is difficult to know what information to improve

Starting with a company-wide information-restructuring program is not always the most effective way to resolve this problem.

Without concrete assessment results, discussions about what to organize, at what level of detail, and across which scope can remain abstract—and the proposed cleanup can expand without a clear priority.

A more practical approach is to begin with the assessment sheets and documents already in use. The results can then show:

which information was used correctly;
where the assessment applied an incorrect premise;
which evidence was insufficient for a decision;
which items required follow-up with a responsible person; and
which gaps recur across multiple assessment targets.

The organization can then prioritize information that has a material effect on decisions and is repeatedly needed across multiple targets.

When a gap is unique to one target, the corresponding target-specific document or evidence can be improved. When the same gap recurs across several targets, it may be appropriate to elevate the improvement into a shared attribute, standard template, controlled vocabulary, or evidence-retention rule.

Data improvement is not a preparation task that must be completed before assessment AI can be used. It is an ongoing process of running assessments, identifying what prevents a reliable decision, and narrowing the improvement to what is actually needed.

Divide the roles of AI and people

The cause of missing information cannot always be established from RAG results alone.

AI is well suited to:

breaking an assessment item into the conditions that must be checked;
locating relevant documents and evidence;
extracting supporting passages;
separating confirmed facts from missing information;
identifying contradictions or scope differences across documents;
researching official external specifications or standards;
drafting follow-up questions for responsible staff; and
preparing a first-pass assessment based on predefined criteria.

People still need to determine:

whether the relevant document exists elsewhere in the organization;
whether external information can legitimately be applied to the internal assessment target;
whether the external/internal responsibility boundary is appropriate;
whether the available evidence is sufficient for the assessment;
whether a control is operating without retained records;
whether the control itself is absent;
whether an exception or compensating control should be accepted;
whether a risk should be accepted; and
which final result the organization is prepared to confirm.

The goal is not for a person to restart the investigation from scratch every time the AI returns “no information.” Instead, the AI should make the reviewed scope and the specific missing facts visible, so that the person can determine the appropriate improvement path.

Turning information gaps into improvements with Batch Assessment

For each assessment, Batch Assessment can define the RAG database to use, documents that must always be reviewed, and reference scopes based on metadata or other conditions.

The organization can also freely define the names, number, and meanings of the rating options, while configuring the overall execution prompt to reflect the specific assessment workflow.

Based on those settings, the AI organizes a first-pass result, rationale, quotations, source references, and missing information for each item. A person reviews not only the result but also the evidence, adds or corrects information where necessary, and makes the final decision.

The objective is not for AI to fill missing information through speculation.

It is to make the following traceable:

what was confirmed;
what could not be confirmed;
which sources were reviewed;
which premises were missing; and
where human judgment became necessary.

When the assessment result is used not merely to score the AI as right or wrong, but to determine whether the organization should improve assessment design, retrieval and registration, evidence and controls, or source and responsibility boundaries, assessment work and information improvement can reinforce each other over time.

Conclusion

Assessment AI does not require perfectly organized data from the outset.

What matters first is the ability to run an assessment with the existing checklist and documents, then trace which information was used, where the assessment applied the wrong premise, what it could not confirm, and which evidence was insufficient.

Those results can then guide targeted improvements to the evaluation specification, reference scope, registered data, evidence-retention practices, or the underlying controls. When the same issue recurs across multiple assessment targets, it can be elevated into a shared input field, document template, evidence-retention rule, or other organization-wide improvement.

Rather than beginning with a large-scale information-cleanup program, we believe a more sustainable approach is to start with the gaps revealed through actual assessments and address the highest-value improvements at a manageable pace.

Learn more about AskDona Batch Assessment