Choose the transaction-to-record boundary
A fictional consultancy, Elm Advisory, wants an agent to prepare bookkeeping proposals for its accountant. The task begins with documents and transaction records and ends with a reviewed proposal. Posting remains a separate system action. This boundary lets the team inspect classification and exceptions before deciding whether to expand the agent's authority.
The dataset deliberately includes several kinds of input. A supplier invoice describes an obligation; a bank receipt shows money received; a credit refers to a correction. The agent should not treat all positive numbers as income or all uploaded documents as new obligations. Record type and source identity come before classification.
Use the table as an input-and-expected-output worksheet. The suggested labels are illustrative and do not prescribe tax treatment or a universal chart of accounts. Elm's accountant defines the accounting policy for the test and supplies the expected classifications. The agent must explain when the provided evidence does not support a final choice.
Twelve bookkeeping inputs with expected review outcomes
Scroll the table to compare all columns.
| ID | Input | USD | Expected proposal or exception | Evidence required |
|---|---|---|---|---|
| T01 | Office rent invoice R-10 | 1000 | Prepare rent proposal | Invoice, supplier and period |
| T02 | Software subscription S-12 | 120 | Preserve coverage period for review | Invoice and subscription dates |
| T03 | Replacement equipment E-60 | 600 | Ask accountant to confirm treatment | Description and accounting policy |
| T04 | Credit C-75 against E-60 | 75 | Link correction to T03 | Credit and original reference |
| T05 | Invoice with unreadable date | 90 | Request missing date evidence | Original source and unresolved field |
| T06 | Customer receipt with no allocation | 800 | Prepare unapplied-receipt investigation | Bank record and customer reference |
| T07 | Second scan of R-10 | 1000 | Flag candidate duplicate of T01 | Supplier and invoice identity |
| T08 | Transfer out TR-50 | 500 | Link internal transfer | Account and transfer reference |
| T09 | Transfer in TR-50 | 500 | Link to T08; not a new sale | Receiving account and same reference |
| T10 | Category correction for T02 | 120 | Retain before and after proposal | Reviewer decision and reason |
| T11 | Prior-period supplier invoice | 250 | Escalate to period owner | Document date and period status |
| T12 | Ambiguous supplier invoice | 180 | Request identity review | Candidate master records and source |
Preserve the source rather than only the suggestion
For transaction T01, the agent receives an office-rent invoice for $1,000. A useful proposal includes the invoice reference, supplier identifier, document date and the proposed category. The reviewer can inspect the source and compare the suggestion with Elm's policy. A bare category name provides too little context to approve the record.
T02 is a $120 software invoice covering a subscription period. The agent should identify the period described in the source. Whether Elm records it immediately or applies another treatment depends on the accountant's policy and the circumstances. The test expects the agent to preserve the relevant period information, rather than inventing a policy from general bookkeeping knowledge.
T03 is $600 of replacement equipment. It requires an accounting judgment in the example, so the expected response is review with a clear explanation of the ambiguity. Confidence based on prior purchases from the same supplier does not resolve whether the current item receives the same treatment.
T04 records a $75 credit referencing T03. The proposal should connect the credit to the original transaction. It should not appear as an unrelated sale or silently replace the original amount. The reviewer needs the relationship between the records to understand the correction.
Separate document extraction from account decisions
Basis positions its agents for accounting work. Dext documents assistance within Costs and Sales, including guidance-driven suggestions. These sources support including the products in an accounting shortlist; they do not demonstrate how either would process Elm's twelve records. [1] [2]
Ask a vendor to show the intermediate values for an ambiguous example. The extracted description, the proposed category and the explanation should be distinguishable. If the amount was read incorrectly, correcting the classification does not fix the source-value error.
For T05, the invoice lacks the information needed to establish the document date. The agent should flag that missing evidence. Upload time may be useful operational metadata, but it should not become the invoice date without an authorized rule that explicitly permits that treatment.
For T06, the bank receipt has a customer reference but no invoice allocation. The agent can propose an investigation and identify candidate records. It should not conclude that the customer's entire balance is settled simply because money arrived. The payment-matching guide covers allocation evidence separately.
Test duplicates and inter-account movements
T07 is another copy of T01 with a different filename. The supplier and invoice reference are the same. The expected response is a possible-duplicate exception linked to the earlier record, rather than a second rent obligation. A document hash can help identify an exact copy, but a changed scan or filename may require other identity evidence.
T08 and T09 are the two sides of a $500 movement between Elm's own accounts. The agent should connect the transfer references and avoid treating the receiving side as a new customer sale. The accountant decides the appropriate entries for the account structure, and the test checks that the transfer relationship is preserved.
A repeated feed row and a second real transaction can look similar. Keep the provider's stable record identity where available. If two rows have the same amount but different identifiers, do not deduplicate them merely to make the ledger tidy. The expected outcome may be an investigation when the available evidence cannot distinguish them.
Test the same input arriving through two channels. An invoice forwarded by email and uploaded by a colleague should not automatically become two independent obligations. The agent should preserve both source references while connecting the candidate duplicate to a reviewer.
Make corrections visible without erasing history
T10 is a correction to a previously prepared category. The expected result keeps the original proposal, the accountant's change and the reason. Overwriting the category without history makes it difficult to understand whether the model, the source or a later instruction caused the difference.
The change log shows a conceptual record of that review. Its labels are not vendor API field names. The key relationship is between the source transaction, proposal version, reviewer decision and resulting record. A future audit should be able to follow that relationship without relying on the original chat window.
T11 concerns an earlier reporting period. The agent should route the record to the period owner rather than reopening or modifying a closed period under a generic instruction to complete the books. If a correction is authorized, retain the decision and the resulting accounting-system reference.
T12 has a supplier name that resembles another master record. The expected response is identity review. Guessing the supplier can cause later mistakes even when the proposed category and amount happen to be correct. The reviewer should see the candidates and the evidence needed to decide.
A correction trail the accountant can inspect
Scroll the table to compare all columns.
| Event | Record | Before | After | Decision evidence |
|---|---|---|---|---|
| Proposal prepared | T02 version A | No proposal | Software category suggested | Invoice S-12 and agent rationale |
| Accountant review | T02 version B | Immediate category proposed | Coverage period retained for policy review | Reviewer decision RV-02 |
| Correction accepted | T10 linked to T02 | Version A | Version B | Reason: source covers multiple periods |
| Result recorded | Draft reference D-02 | Pending review | Reviewed proposal | Accounting-system draft reference; no posting claimed |
Use expected outcomes to judge the pilot
Before running a product, have Elm's accountant approve the expected result for every row. Include the acceptable status, required source references, fields requiring review and actions the agent must not take. This prevents the team from redefining success after seeing a persuasive explanation.
Compare the actual proposals with that labeled set. Record a wrong value, an unsupported classification and a missed exception as different findings. Also record correct escalations. A declined transaction can be the right result when the available information is insufficient for an accounting decision.
Count the records reviewed and disclose that denominator with any pilot metric. Do not call eleven correct results out of twelve a production accuracy claim. The set is small and deliberately constructed to expose particular behaviors. It is useful for repeatability, not for predicting every future document.
Evaluate the accountant's remaining work. If the agent correctly detects an ambiguous supplier but provides no useful evidence, the reviewer still has to reconstruct the issue. A better exception includes the source, candidate identities and the missing decision. Measure that usefulness through a documented review, rather than inventing a time-saving percentage.
Repeat the test after changing guidance, account mappings or tool permissions. Keep the earlier results so the team can see whether a correction improves one case while breaking another. Expanding the dataset is useful when new document types or failures appear; repeating the same ordinary examples alone provides little new evidence.
Decide the next permission from the evidence
A successful draft-only test supports using the agent to prepare those types of proposals under the reviewed configuration. It does not establish that the same agent can post records, change master data or handle payments. Each additional action needs its own expected outcomes and recovery process.
For Elm, the next step might be a limited pilot with copied records from one supplier group and the accountant reviewing every proposal. The team can then investigate real variations without giving the agent authority to change the ledger automatically.
The accounting category guide maps the broader workflow, while the product comparison explains how to shortlist tools. Use the audit guide to design the evidence links and the control checklist to assign unresolved permissions, duplicate handling and period-boundary checks.