Lorikeet launched Dreamweaver on November 4, 2026, describing AI account managers that work toward customer goals. The announcement says a holdout group measures impact. For a team evaluating the product, that comparison deserves a place in the pilot brief before anyone counts messages sent.
Separate activity from the intended result
Lorikeet says Dreamweaver can send a message, create a staff task or take no action, with checks on contact hours, limits and consent. Its product page also describes a holdout group. These are supplier descriptions, not results independently tested for this article.
A useful pilot question might be whether an eligible new customer completes a clearly defined setup step within an agreed period. The count of messages sent answers a different question: how much communication occurred. More messages do not, by themselves, establish that more customers completed the step.
Write the comparison before the pilot
For a proposed evaluation, define the eligible population, outcome, observation period and group assignment in advance. Document what the holdout receives instead of the new intervention. Keep required customer service available and make the intended difference between groups explicit. Have the responsible analyst review assignment and measurement before drawing causal conclusions.
Ask the supplier how the comparison is implemented in the actual account: how customers enter each group, what ordinary communications continue, and where group sizes and outcomes can be inspected. This article does not establish Dreamweaver’s allocation algorithm, default holdout size or reporting interface.
Read an entirely fictional result
Imagine two invented groups of 100 eligible customers each. In the comparison group, 20 complete the setup step; in the intervention group, 26 do. Those are completion rates of 20% and 26%. The difference is 6 percentage points. Relative to the 20% comparison rate, the increase is 30%: (26% − 20%) ÷ 20%.
These numbers are a teaching example, not Lorikeet customer data or a reconstruction of a supplier case study. The arithmetic alone does not establish statistical reliability or causation. A credible interpretation also needs the actual group design, sample sizes, timing and uncertainty.
Keep a compact evidence sheet
- Outcome: name the customer action being measured.
- Comparison: state what differs between the intervention and holdout groups.
- Denominator: retain eligible customer counts as well as completion counts.
- Timing: use the same observation window for both groups.
- Interpretation: distinguish observed rates, calculated differences and conclusions that remain unproven.
Use that sheet to make a supplier demonstration concrete. A well-labeled comparison can be informative even when the result is inconclusive. No Lorikeet account, customer contact, holdout assignment or experiment was run for this article.