Agenthon 2026 / Submission format

Submission format

What You Submit

Prepare a container image, describe it in submission.json, and package the descriptor with your team-verification proof. This guide explains that workflow and the validated format. See your logged-in agenthon.net account for submission availability.

The two parts of a submission

A submission has two parts: a container image that implements the one command for your track, and a small descriptor file, submission.json, that names the image by digest. You upload a ZIP containing submission.json and the toolkit-generated team-claim.json; the organizers pull the image separately. The ZIP contains neither container layers nor your Team Key. Do not put registry passwords or other credentials in it.

The image

Build a Linux/amd64 image with its dependencies and permitted artifacts already included. Each track has one stable command verb. Your image must either expose it as an executable on PATH, or accept it as the leading positional argument of its entrypoint.

Pin the image by digest. A floating tag is rejected before your image is pulled, and the digest also lets organizers rerun the exact bytes during the joint Final + Verification phase. For the public image route, check that the image is anonymously pullable: a successful pull using your own registry login does not establish access for the evaluator.

All four tracks run without general internet access. Supported Coding, Forecasting and Explainability submissions can call the provided House Nemotron model through a restricted connection. Simulation has no network access. Downloads and external API calls are unavailable during evaluation.

Public image contents can be downloaded by others. If your image must remain confidential, contact the organizers privately before submitting and obtain confirmation of the handoff and the exact usable image reference. Selecting organizer_mirror alone does not arrange access. The Privacy Notice and Licensing Policy govern submission handling.

Prepare and upload

1. Follow the installation instructions in the shared toolkit repository and update your track's public starter package. Use the released versions specified there, then run your track's local checks.
2. Start with your track's Development descriptor and set its image, track, phase, category and actual model disclosures. Use the supported mode and the model metadata published in your track's README. Do not invent a training cutoff or copy the fictional models in the conformance examples below.

1. Choose one CodaBench account for your team across all tracks. Get your Team Number and Team Key from your agenthon.net account, then derive the descriptor's team ID: qfbench2 submission alias --team-number YOUR_TEAM_NUMBER Enter your Team Key at the hidden prompt. Put the resulting team ID into submission.json, replacing the example team ID. The numeric Team Number and the derived team ID are different.
2. Package the descriptor after local checks: qfbench2 submission pack --descriptor submission.json --team-number YOUR_TEAM_NUMBER --out submission.zip The toolkit prompts for the Team Key, seals the descriptor and creates the proof. Repack after changing the descriptor or image digest. Keep the Team Key and the packaged verification proof private. A matching email alone does not verify a team.
3. When submissions are available, use the competition links in your logged-in agenthon.net account. Request entry on CodaBench, accept the terms, follow any approval instructions shown, and upload submission.zip under My Submissions → Development. The first successfully verified submission links your designated account to the team; use it across all tracks.

When Development submissions open, the daily upload limit per team is 1 for Track 1 (Coding) and 5 for each of Tracks 2–4. The total limit is 20 uploads per team per track. Held or cancelled uploads still count, even without a score. Local checking and packing do not consume a submission attempt.

Development runs through 12 October 2026. The joint Final + Verification phase runs from 13 to 25 October 2026. Registration and Development close on 12 October at 23:59 Anywhere on Earth (AoE, UTC−12). Final + Verification closes on 25 October at 23:59 AoE. Each team makes one final submission per entered track; organizers handle verification within that same phase without a second participant upload.

The descriptor

Every field is required, the descriptor is validated in full before your image is pulled, and unknown fields are rejected rather than ignored. The table is generated from the canonical schema. Nested fields are shown with dotted names; requirements on models[] apply to each model entry. An empty model array is permitted by the format and does not grant additional model or data permissions.

category is required on every track. Track 3 uses simulator. Use the supported Development categories in the guide above. The complete schema also lists other category values; format acceptance alone does not mean a serving mode is available. An absent or unrecognised category is an error, never a default.

models is a required array listing every model your submission uses, with its name, version, training cutoff, access of api or local, and revision. A genuinely model-free submission declares models: []; omitting the field is invalid. Follow the published disclosure guidance when a training cutoff is unpublished; do not substitute the model's release date. license is a required SPDX identifier.

image_access declares how the organizers obtain your image: public for a public registry, organizer_mirror if it is mirrored into an organizer-operated registry instead. The confidential route requires prior organizer confirmation; declaring it does not configure credentials or arrange a mirror.

descriptor_digest covers every other field. Use the toolkit to repack after any edit, rather than calculating or editing this digest by hand.

Conformance examples for the descriptor formats

These canonical fixtures demonstrate the validated format. Their image references, team IDs and model names are examples, not runnable submission images or approved model choices. Use your track's current starter descriptor for an actual submission. The final and verification examples preserve the canonical schema formats. They do not create separate calendar phases or a second participant submission. Final submission and organizer verification both belong to the joint Final + Verification phase; these examples do not indicate that it is open.

What a failure is reported as

These are the organizers' published failure codes, rendered from the registry itself rather than paraphrased. Participant failures remain in the evaluation denominator under the track's failure rules; missing or failed outputs are not silently dropped to improve a score. An organizer or infrastructure fault is held for review rather than recorded as a participant's zero. An absent score does not automatically mean zero.

Help and results

Follow processing status on CodaBench. Validated Development results appear on the public leaderboards on agenthon.net, visible to everyone. Private evaluation tasks and reference answers remain confidential. Participant-facing status does not include all internal diagnostics or private evaluation logs.

If an upload is held or appears stuck, contact the organizers with the submission ID and visible status before using another attempt. See the FAQ for team verification, limits and image privacy. Never include your Team Key, credentials or verification proof in a public GitHub issue.