Tonnage Email Parsing Pilot: A Bounded Test Before Desk Adoption

Key takeaways

• Define message scope and excluded actions before collecting examples.
• Test difficult formats and corrections alongside ordinary circulars.
• Judge review effort and consequential errors together before expanding access.

Choose one practical parsing job

Begin with a specific task, such as preparing draft dry-cargo positions from a designated shared mailbox. State which message types, languages and attachment formats belong in scope. Keep recaps, cargo enquiries and unrelated correspondence outside the initial job unless they are deliberately included for routing tests.

Describe the output fields and their intended use. A draft opening list needs different treatment from a historical market archive. Decide how uncertain dates, geographical ranges and conditional availability will appear, and preserve the source expression where normalisation could remove meaning.

Define the pilot's action boundary. A useful first stage can read permitted messages and generate isolated draft records while the normal desk process continues. It need not edit the production position list or send anything externally. This lets the team compare outputs without confusing experimental data with approved commercial information.

Assemble examples that reflect the desk's difficulty

Select messages across relevant senders and formats, including plain text, tables, attachments and forwarded chains where permitted. Include revised circulars, repeated vessel names, missing dates and messages that do not contain a new position. Easy examples alone provide little evidence about the work that consumes broker attention.

Have experienced staff prepare reference records using a written annotation guide. The guide should explain how to treat quoted history, uncertain opening ranges and duplicate updates. When reviewers disagree, resolve the interpretation or label the example genuinely ambiguous instead of forcing a false answer.

Separate examples used to improve the parser from examples reserved for evaluation. Preserve the reserved set so repeated tuning does not gradually teach the system the answers being used to judge it. Keep access to real commercial correspondence aligned with the organisation's information rules.

Run the pilot through distinct stages

Start with an offline replay of approved examples. Check whether the parser associates each field with the correct vessel and retains material qualifications. Use the findings to improve the extraction instructions, data schema or review interface, recording each change.

Next run in shadow mode alongside the desk. Compare draft outputs with the records staff actually prepare, including the time spent resolving exceptions. Staff should know which view is experimental so a plausible pilot record is not inadvertently used in a commercial message.

Only consider a controlled live drafting stage after the earlier checks support it. Keep publication and external communication under the established approval process. The pilot is testing a workflow, not granting a general right for a model to act across the desk's systems.

Use a pilot acceptance checklist

1. Confirm that the parser reads only the approved mailbox sources and file types.
2. Check vessel-to-field association on ordinary lists and forwarded chains.
3. Verify that uncertain dates, ranges and conditions remain visible.
4. Test corrections, withdrawals and repeated circulars without creating misleading duplicates.
5. Confirm that reviewers can inspect the source and reject a proposed update.
6. Demonstrate that experimental records cannot silently publish or send externally.
7. Compare total review work with the existing process and document remaining failure patterns.

Set acceptance criteria before examining the final evaluation results. Criteria should reflect the desk's tolerance for different errors. A misspelled nonessential note and a wrong opening attached to the wrong vessel should not be treated as equivalent defects.

NIST's AI risk framework presents evaluation as part of managing AI trustworthiness throughout use. It is a voluntary framework, not a maritime product certification. Use that lifecycle perspective to keep the pilot connected to continuing review. NIST AI Risk Management Framework.

Measure the work around the model

Track more than whether a field was populated. Record wrong associations, omitted positions, invented values, lost qualifications and stale information presented as current. Distinguish parser failures from source messages that a human also cannot resolve without clarification.

Measure reviewer effort through observed work rather than a model's claimed time saving. A draft that takes little time to generate may still be expensive to check if source links are missing. Capture repeated searches, corrections and reopened records as part of the pilot experience.

Look at who bears the work. Automation may reduce a broker's copying while increasing an analyst's reconciliation burden. That can still be useful, but the adoption decision should reflect the whole process and the skills needed to operate it sustainably.

Plan rollback and ongoing sampling

Before live drafting, define how the team will disable the parser and identify its recent outputs. Retain the ordinary manual workflow as a tested fallback. Staff should know where to find the last approved list and how to avoid mixing it with unreviewed suggestions during an interruption.

Keep a change log for parsing rules, model configuration and field definitions. After a material change, replay the reserved examples and review affected live cases. A pilot result belongs to the configuration and scope that were tested, not every future version bearing the same product name.

Continue sampling after adoption, particularly when a new sender or format becomes common. Unexpected patterns should enter an exception set for review. Expanding coverage is a fresh operational decision, supported by evidence, rather than an automatic consequence of the original pilot passing.

Fictional case: strong extraction, poor duplicate handling

In this fictional case, a small chartering desk pilots a parser on broker circulars. The field extraction looks accurate, but shadow use shows repeated updates appearing as separate open positions. Reviewers spend more effort deciding which entry is current than they previously spent copying the original list.

The team keeps the pilot in draft mode and adds a source-aware update comparison. It also gives reviewers a clear option to supersede an earlier entry without deleting the history. Reserved examples are rerun, including circulars where similar vessel names refer to different ships.

The adoption decision then considers both extraction quality and the reduced reconciliation burden. The lesson is specific: a parser can read fields well while still failing the desk's actual position-maintenance task.

Common mistakes

Do not select only neatly formatted messages supplied by the implementation team. Include the untidy correspondence the desk must process. Another mistake is changing acceptance criteria after seeing results, which makes the decision difficult to defend.

Avoid measuring generation speed while ignoring review and correction work. Finally, do not expand mailbox access or publication powers merely because a narrow pilot performed well. New sources and actions require their own assessment and operational owner.

Where Zeaclub fits

Zeaclub supports agentic, AI-assisted operations with a human in control.

Zeaclub provides a multi-party shared workspace for owners, charterers, brokers and port agents.

Explore Zeaclub

Visit the Zeaclub website