AI Software / Test results
AITest results
113 automated tests. 113 passed. Published, with the method.
Finance software should be tested like a control. Every AppIQ AI engine is checked against data sets where the right answer is known, so each match can be marked right or wrong. Latest run: 8 November 2026.
What the tests show
0 wrong automatic matches
Across 40 random bank data sets of 300 lines each, the matching engine made no wrong automatic match, and matched 100.00% of the ledger lines that were on the bank statement.
1,189 → 2,755
On 3,311 bank and ledger pairs worded differently, the rules alone matched 1,189. With the on-device language model, 2,755, with no wrong High or Medium match.
Under 5 seconds
5,000 card lines against 6,000 expense claims, or 5,000 x 5,000 intercompany lines, matched twice (rules only, then rules + AI) and scored, inside the 5-second test limit.
Results by product
| Area | Tests passed | What is tested |
|---|---|---|
| Bank matching engine | 7 of 7 | 40 random data sets of 300 lines: 0 wrong automatic matches and 100.00% of ledger lines that are on the bank statement matched. Batch deposits, same-amount-same-week pairs and ambiguous combinations (suggested, never matched). Reconciliation proof balances to zero. |
| Language model (bank) | 8 of 8 | On 3,311 bank and ledger pairs worded differently: 1,189 matched by the rules alone, 2,755 with the rules and the on-device AI, with no wrong High or Medium match. The AI-added count equals the difference between the two runs. |
| Supplier statements | 1 of 1 | Five test sets: missing invoices, unposted credit notes, price differences, duplicate postings, payments in transit and supplier charges. |
| Cash flow | 1 of 1 | Payee joining, recurring streams, categories and trend totals that add up. |
| Customer payments | 1 of 1 | Receipts applied to open invoices, short and over payments, unplaced receipts with a probable customer. |
| T&E card matching | 14 of 14 | Five matching rules and seven flags each fire on a crafted case; 5,000 card lines against 6,000 claims matched twice (rules only, rules + AI) and scored in under 5 seconds. |
| Intercompany | 20 of 20 | Six rules, FX differences, grouped settlements, in-transit items. On messy random ledgers no line is used twice and the reconciliation always ties; 5,000 x 5,000 lines in under 5 seconds. |
| Journal entry testing | 11 of 11 | Benford first-digit tests: MAD and chi-square match hand-computed values; Nigrini conformity bands; clean data fires no test. |
| Balance sheet rec review | 12 of 12 | Ageing buckets exact by date, item sums exact to the cent, planted problem recs found; 5,000 recs and 100,000 items in under 5 seconds. |
| Vendor master audit | 9 of 9 | Duplicate and lookalike vendors, vendor-to-employee checks, bank-detail changes, planted cases ranked at the top. |
| Trial balance mapping | 10 of 10 | Mapping proposals with confidence, existing-mapping checks, unbalanced amounts exact to the cent. |
| Anomaly model (all products) | 1 of 1 | The Isolation Forest puts every planted unusual line at the top with true reasons; 20,000 rows scored in about half a second on our test machine. |
| File reading | 10 of 10 | CSV, Excel, OFX/QFX, CAMT.053 and MT940 bank formats read without outside libraries. |
| Licence keys | 5 of 5 | Signed keys are checked on your computer; edited, refunded and expired keys are refused. |
| Time-limited keys | 3 of 3 | Personal mailboxes and incomplete requests refused; a key is issued only after approval, works for the requested product for a fixed period, and can be cancelled; tampered approval links fail. |
Speeds were measured on our test machine; yours will vary with the computer and browser.
How the tests work
Test generators build bank statements, ledgers, card files and journals with a recorded right answer for every line, including deliberate traps: duplicates, same-amount lines in the same week, split payments, wording changes.
Each match the engine makes is compared with the recorded answer. A single wrong automatic match fails the test.
The AI's contribution is measured by running the same files with and without the model. The difference is the AI's count, never a typed-in figure.
Synthetic data can't capture every real-world file. Match rates on your data will differ; the engines' behaviour, sending doubtful pairs to To check rather than guessing, does not.
Questions
What data are the tests run on?
Synthetic data made by test generators, where the right answer for every line is known in advance, plus small crafted cases for each rule. No customer data is used. That is what lets each test check every match for correctness.
Will I get the same results on my files?
Not necessarily. Real files differ in wording, volume and quality, so match rates on your data will differ. What the tests show is how the engines behave: they don't make wrong automatic matches on the test sets, doubtful pairs go to To check, and the reconciliations tie to the cent.
How often are the tests run?
Every time an engine changes, before a new version is published. This page shows the date of the latest run.
What does 'no wrong High or Medium match' mean?
Every match carries a confidence. On the test sets, no match marked High or Medium paired the wrong lines. Lower-confidence pairs are not matched automatically; they go to a To check list for you to decide.