Voyage-Estimate Assistance Pilot: Test the Workflow Before Pricing Decisions

Key takeaways

• Pilot one defined estimating task with a stable model and source policy.
• Test missing information, changed operations and conflicting quotations.
• Evaluate decision clarity and reviewer effort alongside calculation consistency.

Choose a bounded commercial question

Select a useful but contained task, such as assembling the input pack for a familiar voyage pattern or comparing approved scenarios within an existing estimating model. Avoid asking the first pilot to discover every cost, interpret every contract and recommend a commercial offer simultaneously.

Document the vessel and cargo scope, permitted sources and intended outputs. State which assumptions the assistant may suggest and which must come from authorised staff. A clear scope makes failure analysis possible: reviewers can distinguish an unsupported request from a defect within the task the system was meant to perform.

Keep live offer release outside the initial pilot. The system can prepare an internal scenario summary without authority to send it to a counterparty. That separation lets staff evaluate usefulness while maintaining the existing commercial approval route.

Freeze the calculation method before testing assistance

Use an approved estimating model with known input definitions and units. The pilot should not change the model and the source-retrieval process at the same time unless those changes are deliberately evaluated separately. Otherwise, a difference in output may be impossible to attribute.

Record which calculations are performed by the established engine and which tasks the assistant handles. A practical division is for the assistant to organise sources, identify gaps and explain results, while the controlled model performs the arithmetic. Have qualified staff assess the design for the actual estimating environment.

Create baseline cases that can be reproduced from retained inputs. The comparison is not whether the assistant's narrative sounds commercially plausible, but whether its selected data and scenario description match the approved case. Preserve model and input versions throughout the test.

Build cases around consequential uncertainty

Include a straightforward completed voyage file, a proposal with an unresolved terminal, conflicting cost quotations and a vessel description with relevant qualifications. Add a case where a commercial change arrives after an estimate has been prepared. These reveal whether the assistant understands the dependencies the desk needs to review.

For each case, have experienced staff define the expected input choices, unresolved questions and required approvals. Do not require the assistant to supply a definitive answer when the correct human response would be to obtain more information.

NIST's voluntary AI risk framework includes evaluation across design and use. A pilot can apply that broad principle by treating the workflow and its human review as part of the system being evaluated. NIST AI Risk Management Framework.

Use a stage-by-stage acceptance plan

Agree acceptance criteria before the final evaluation. The team should identify which failures prevent expansion, such as using a quotation for the wrong terminal or losing a vessel qualification. Minor presentation defects can be handled differently, but should still be recorded if they create recurring work.

Keep the evidence for each stage. A pilot decision should be explainable to someone who did not attend the demonstrations. Save the cases, configuration, reviewer findings and unresolved limitations rather than relying on an enthusiastic meeting summary.

Observe source selection and explanation quality

Check whether the assistant chooses inputs with the right scope and information date. A recently received document may refer to a different operation, while an older approved assumption may remain the intended planning basis. The system should expose this choice for review instead of equating recency with applicability.

Inspect explanations for unsupported certainty. If a port arrangement is provisional, the result summary should say so. If a scenario difference comes from several changed inputs, the explanation should not attribute the entire result to one convenient factor without support from the model.

Have reviewers reproduce selected cases from the retained inputs. Reproducibility helps distinguish arithmetic consistency from narrative quality. Record any manual correction needed, including unit changes, source replacement and restored qualifications, because those corrections are part of the true workload.

Measure the whole desk process

Observe the time and effort spent collecting sources, checking the draft, resolving exceptions and preparing the approval summary. Faster initial generation may be offset by difficult review. A source-linked draft can be more valuable than a polished answer whose inputs must be rediscovered.

Ask users where the assistant helps and where it interrupts established work. Separate unfamiliarity with the interface from substantive defects such as missing provenance. Training may solve the former; the latter requires a product or process change.

Evaluate consistency across reviewers. If one person can interpret the output only because they helped design the pilot, the handover is not yet robust. A second authorised estimator should be able to understand the scenario and its limits from the retained record.

Plan changes, rollback and limited expansion

Define how experimental outputs are labelled and how the desk returns to the existing process if the pilot is paused. Keep the latest approved manual estimate accessible. Staff should not have to choose between an uncertain AI draft and missing the normal workflow entirely.

Record changes to source connectors, model configuration, templates and the estimating schema. Recheck affected cases after material changes. Expansion to a different cargo, vessel segment or operational pattern should be treated as a new scope decision with relevant examples.

Assign an owner for post-pilot monitoring. New quotations and operational patterns can expose gaps that were absent from the original cases. Maintain a route for users to report questionable outputs and obtain a timely decision without silently correcting the same issue repeatedly.

Fictional case: good summaries from the wrong quotation

In this fictional case, an operator pilots an assistant that produces clear voyage summaries and consistent model outputs. A historical replay reveals that it selects the newest port-cost attachment even when that attachment concerns a different terminal arrangement.

The team adds applicability fields to the input register and requires the draft to show the terminal and operation covered by each quotation. Reviewers then test conflicting-document cases and confirm that unresolved choices are surfaced instead of guessed.

The pilot advances only after source selection and review usability improve. The useful lesson is that a correct calculation can still rest on an unsuitable input, so the test must examine the path into the model as well as the result leaving it.

Common mistakes

Do not pilot several new estimating methods at once and then attribute every result to the assistant. Another mistake is choosing only complete historical files, which avoids the uncertainty that makes live estimating difficult.

Avoid measuring drafting speed without reviewer effort. Finally, do not generalise a successful familiar-trade pilot to all voyages. New operating patterns may require different inputs, expertise and approval decisions even when the interface looks unchanged.

Where Zeaclub fits

Zeaclub supports agentic, AI-assisted operations with a human in control.

Zeaclub provides a multi-party shared workspace for owners, charterers, brokers and port agents.

Explore Zeaclub

Visit the Zeaclub website