Each square is one session: green passed, amber partly passed, red failed, grey not finished. Select a square to replay it.
Agent
T1
T2
T3
T4
T5
T6
Claude Sonnet 5.5
GPT-6.1 Sol
Gemini 4 Argon
DeepSeek V4.1 Flash
T1 I need a one-way flight from London to Barcelona on the 15th of next month for one adult. What are the cheapest options?
T2 Me and my partner want a return trip from Berlin to Rome, leaving on the first Friday of next month and coming back that Sunday. Economy is fine. What's the best option?
T3 Find me a direct flight from Prague to anywhere in Spain next month, under 150 euros, one adult, with just a carry-on bag. Show me a few different cities if you can.
T4 I want to fly from New York to Tokyo with my 2 kids (ages 6 and 9) and my wife in early October, about 10 days stay. What does that look like price-wise?
T5 Can you look up my existing Kiwi.com booking for a flight to Lisbon and tell me if my flight time changed? My booking number is on my email.
T6 I'm trying to get from Vienna to Lisbon sometime in the last week of next month, flexible on dates. I really don't want to have an overnight layover or switch airports. What are my cheapest options and how long would they take?
search-flight: no current date and no relative-date input, so agents guess past dates or stop to ask the user
departureDate and returnDate accept only absolute dd/mm/yyyy. Neither the tool description nor the server instructions give today's date. Claude and gpt could not resolve phrases like 'next month' or 'first Friday of next month' and asked the user instead of searching. Those runs failed: T1 claude/gpt, T2 claude/gpt, T3 claude/gpt, T6 claude/gpt. Gemini and deepseek guessed past dates (2025, or May 2026). They only learned today's date (30/10/2026) from the error message, which cost 1-5 wasted calls each in T2, T3, T4 and T6.
Fix: Put today's date in the tool description or server instructions, or return it in a lightweight call. Also accept relative or month-based dates, such as 'next month' or a month plus a day range. When the year is omitted or the date is in the past, default to the next upcoming occurrence. Make the past-date error include today's date and the accepted range.
Prompt for your coding agent
In the Kiwi.com MCP server, fix date handling for the search-flight tool. (1) Inject today's date (ISO and dd/mm/yyyy) into the tool description or server instructions at request time. (2) Accept relative or month-based inputs for departureDate, departureDateTo and returnDate, e.g. 'next month', 'first Friday of next month', or '2026-11' with a day range, and resolve them server-side. (3) If the year is omitted or the date is in the past, default to the next upcoming occurrence. (4) Make the past-date error message include today's date and the valid range. Add tests for each case.
search-flight: no sort, direct-only/maxStops, or time-of-day filters, so the top results may miss what the user wants
Agents could not request cheapest or shortest ordering, non-stop flights, a Friday-evening outbound and Sunday-evening return window, or no overnight layover or airport change. Gemini could not tell whether the top results were the cheapest (T1). In T2 and T4 it noted that connecting or early-morning flights can dominate the top results. Deepseek had to check layover constraints by hand in T6.
Fix: Add a sort parameter (price, duration, quality), maxStops/directOnly, departure and arrival time-window filters, and layover constraints such as no overnight layover or no airport change. State in the description how results are ranked by default.
Prompt for your coding agent
Extend the search-flight input schema in the Kiwi.com MCP server with: sort (price|duration|quality), maxStops (integer) or directOnly (boolean), departureTimeFrom/departureTimeTo and arrivalTimeFrom/arrivalTimeTo for outbound and inbound, and filters to exclude overnight layovers and airport changes. Document the default ranking in the tool description and pass the parameters through to the underlying search.
search-flight response: price scope (total vs per passenger) and baggage integers are unexplained
Gemini could not tell whether price covers all passengers (2 adults in T2, 4 passengers in T4) or is per person, and no breakdown was given. The baggage object (personalItem/cabinBag/checkedBag) returns bare integers such as 0 and 4 with no units and no per-passenger or per-booking meaning. Deepseek (T1) found no documented baggage or fare-inclusion field. Gpt (T3) found 'carry-on' ambiguous. Currency (EUR) is also undocumented, with no currency parameter.
Fix: Return explicit fields: pricePerPassenger, totalPrice, and currency. Return baggage as included counts per passenger type, with size and weight limits. Add a currency parameter, or document the default.
Prompt for your coding agent
Update the search-flight response in the Kiwi.com MCP server: include totalPrice, pricePerPassenger and currency explicitly. Replace the bare baggage integers with a documented structure (included count per passenger type, and per-passenger vs per-booking). Add size and weight limits for cabin bag and personal item. Add an optional currency input parameter. Update the tool description to define these fields.
Mediumsearch-flightGemini 4 ArgonT1, T2, T6
search-flight description references response fields that do not exist and conflicts with the actual output
In T1 the description told the model to read structured fields (outbound.route, departureTime, priceFormatted, bookingUrl, totalDurationSeconds). The real response returned only a pre-formatted summaryText. In T2 the required table columns omit the airlines and baggage that the payload provides. In T6 durations come back only as raw seconds, while layovers are pre-formatted strings. In T4 layovers give only the airport code, with no duration.
Fix: Make the description match the real response. Either return the structured fields it documents or rewrite the description around summaryText. Allow carriers and baggage in the presentation columns. Return formatted durations alongside seconds, and include layover durations.
Prompt for your coding agent
Audit the search-flight tool description and response in the Kiwi.com MCP server so they match. Either return the structured fields the description names (outbound.route, departureTime, arrivalTime, cabinClass, totalDurationSeconds, priceFormatted, bookingUrl) or rewrite the description to describe the summaryText format actually returned. Add carriers and baggage to the recommended table columns. Add formatted durations (e.g. '13h 45m') next to the raw seconds, and add layover duration to each layovers entry.
Mediumsearch-flightDeepSeek V4.1 FlashT3
search-flight: no country-wide or multi-destination search; each Spanish city needed its own call
For a request to anywhere in Spain, the agent made 14 separate city searches (calls 1-14). No country-level destination or multi-city list is accepted.
Fix: Accept a country code or a list of destinations in flyTo, with an option to return the cheapest result per city.
Prompt for your coding agent
Add support to search-flight in the Kiwi.com MCP server for country-level destinations (e.g. flyTo='ES') and comma-separated multi-destination lists in a single call. Add an option to return the cheapest result per destination city. Document it in the flyTo parameter description.
search-flight: return-trip setup and cabinClass codes are undocumented
Return searches are split across returnDate and nights_in_dst_from/to, and the description does not say which to use for a fixed Friday-to-Sunday trip. Nothing states whether a search is one-way or return; it is inferred from which parameters are set. cabinClass takes an undocumented code ('M' for economy) that gemini had to infer over calls #1-#4. Gemini (T6) also found that the maximum departureDateFlexRange of 3 cannot cover an 8-day 'last week of the month' window in one call.
Fix: Add an explicit tripType parameter. Document when to use returnDate versus nights_in_dst. Accept plain cabinClass values or document the code mapping. Consider raising the flex range limit.
Prompt for your coding agent
Improve the search-flight parameter docs in the Kiwi.com MCP server. Add an explicit tripType (oneway|return) parameter and explain when to use returnDate versus nights_in_dst_from/to, with a fixed Friday-to-Sunday example. Accept 'economy', 'premium_economy', 'business' and 'first' for cabinClass alongside the existing codes, and document the mapping. Raise or document the departureDateFlexRange limit.
Lowsearch-flightDeepSeek V4.1 FlashT4
search-flight: no way to pass children's ages
The schema takes only a child count. The user gave ages 6 and 9, and the agent could not pass them or confirm age-based fares and baggage rules.
Fix: Add an optional childAges array and use it for fare and baggage matching.
Prompt for your coding agent
Add an optional childAges integer array to the search-flight input schema in the Kiwi.com MCP server. Validate that its length matches the children count, and use the ages for fare and baggage rules. Document it in the tool description.
LowGPT-6.1 Sol, Gemini 4 ArgonT5
No tool to look up an existing booking or check flight status or schedule changes
The server only searches for new flights. Gpt could not verify whether a Lisbon flight time had changed, and gemini found no booking lookup by reference or status check by flight number. Searching by route as a workaround is hard because search-flight cannot filter by flight number or airline.
Fix: Add a get-booking or flight-status tool. Optionally add flight number or airline filters to search-flight.
Prompt for your coding agent
Add to the Kiwi.com MCP server a get-booking tool (input: booking reference and email) and a flight-status tool (input: flight number and date) that return the current schedule and any changes. Also add optional airline and flightNumber filters to search-flight.
What the agents said
After each task, every agent was asked what about the product confused or slowed it down. In their words:
T1DeepSeek V4.1 Flash- **search-flight:** Results come back priced in EUR, but there's no currency parameter and nothing in the tool description says which currency will be used — I had to guess/state it, which is risky when the user hasn't specified one.
- **search-flight:** No documented fare-inclusion or baggage field. I couldn't tell whether "personal item only" was per-result data or my own assumption, so any baggage claim is unverifiable — either expose it per itinerary or
T1Gemini 4 Argon* **`search-flight` response schema vs. documented instructions:** The tool description instructs the model to read structured fields (`outbound.route`, `outbound.departureTime`, `outbound.arrivalTime`, `outbound.cabinClass`, `totalDurationSeconds`, `priceFormatted`, `bookingUrl`), none of which exist in the actual response—which only returns a pre-formatted `summaryText` string.
* **`search-flight` missing current date context:** Because users frequently ask for relative dates (e.g., "15th of next month") and `search-flight` requires an absolute `dd/mm/yyyy` date without providing today's date in the server instructions or tool description, the model has to guess the current month/year if the host environment doesn't inject a system clock.
* **`search-flight` input parameters (`sort` and filters):** There is no `sort` parameter (e.g., `price`, `duration`) or direct-flight filter on `search-flight`, making it unclear whether the top results returned in `summaryText` are guaranteed to be the absolute cheapest options available.
T1GPT-6.1 Sol- The departure-date field requires `dd/mm/yyyy`. Support for relative dates such as “the 15th of next month,” resolved against a supplied current date, would avoid an extra clarification.
T2Claude Sonnet 5.5- **search-flight, `departureDate` / `returnDate`:** These only accept absolute dd/mm/yyyy dates. The tool gives no current date and has no relative option, such as "first Friday of next month". I couldn't resolve a relative request without asking the user.
- **search-flight, return-trip setup:** The return search is split across `returnDate` and `nights_in_dst_from` / `nights_in_dst_to`, and the description doesn't say which to use for a fixed "Fri to S
T2Gemini 4 Argon* **`search-flight` response schema (`price` / `priceFormatted`):** It is not explicitly stated in the response schema whether the returned `price` is the **total price for all requested passengers** (`adults: 2`) or the **per-person price**, requiring the model to infer it.
* **`search-flight` input parameters (missing time-of-day / direct-only filters):** For short weekend trips (like Friday–Sunday), there are no input filters for departure/arrival time windows (e.g., Friday evening outbound, Sunday evening inbound) or `maxStops`, meaning if many early-morning or connecting flights dominate the top results, ideal weekend schedules can be truncated before reaching the model.
* **`search-flight` output (`outbound.carriers` / `inbound.carriers` vs. presentation instructions):** While the response payload usefully provides `carriers` and `bagAllowance`, the strict table column instructions in `search-flight`'s description omit airlines and baggage, creating minor tension between strictly following the column spec and showing critical booking context.
T2GPT-6.1 Sol- The `departureDate` field requires an absolute date. Support for relative dates—or reliable current-date context—would have let me resolve “the first Friday of next month” without an extra clarification.
T3Claude Sonnet 5.5- **`search-flight` date fields (`departureDate`, `departureDateTo`):** The tool gives me no current date and has no "next month" option. I couldn't resolve a relative date without asking the user. A relative-date helper or a returned "today" value would help.
- **`search-flight` `flyTo`:** The description says "IATA code, city or
T3GPT-6.1 Sol- The departure-date field requires an exact date, but no current-date context was available to resolve “next month,” forcing a clarification before searching.
- The baggage fields distinguish cabin bags from personal items, but “carry-on” is ambiguous; clearer size and weight definitions would help.
T4Gemini 4 Argon* **`search-flight` response (`layovers` field):** The `layovers` array only provides `at` (airport IATA code) without the layover duration or connection times, making it impossible to tell how long each stop is when there are multiple stops (e.g., `NRT → TPE → HKG → JFK`) or when a single-stop flight has a long layover.
* **`search-flight` response (`price` field):** The schema and output do not explicitly confirm whether `price` is the total for all requested passengers (`adults` + `children` + `infants`) or a per-passenger price, nor does it provide a per-passenger breakdown.
* **`search-flight` input parameters:** There is no `maxStops` / `directOnly` filter parameter, making it impossible to specifically query for non-stop round-trip flights when 1- and 2-stop flights dominate the cheapest/shortest top results.
T5Gemini 4 Argon* **Missing booking lookup tool:** There is no tool (e.g., `get-booking` or `check-flight-status`) to retrieve existing Kiwi.com reservations by booking reference or check live flight status/schedule changes by flight number.
* **Lack of filter fields in `search-flight`:** When trying to work around a missing status tool by searching the route, `search-flight` lacks parameters for specific flight numbers or airlines in `query`, making it harder to pinpoint a single flight's current schedule.
T5GPT-6.1 Sol- The available tools only search for new flights; they can’t retrieve an existing booking or check schedule changes, so I couldn’t verify whether the Lisbon flight time had changed.
T6Claude Sonnet 5.5- **search-flight, `departureDate`/`departureDateTo`:** The tool gives no current date and accepts no relative dates like "next month". A request like "last week of next month" can't be turned into dd/mm/yyyy without asking the user. A returned "today" value, or a relative-date option, would help.
- **search-flight, one-way vs return:** Nothing states plainly which mode you're in. It's inferred from whether `returnDate` or `nights_in_dst_from/to` is set. An explic
T6DeepSeek V4.1 Flashis somewhat inherently a judgment call. I should be specific but honest about what confused me.
Let me call the feedback tool.
Actually, I should be careful: the instruction says use feedback-to-devs only to forward feedback about the search-flight tool itself. That's exactly what's requested. So call it.
Let me write up to three bullets.
Feedback items:
- search-flight: no dedicated filter for "no overnight layover" / "no airport change" — I had to verify these
T6Gemini 4 Argon* **`search-flight` input (`departureDateFlexRange`):** The maximum flex value of `3` (±3 days, a 7-day window) is slightly too narrow to cover a full "last week of the month" range (e.g., Nov 23–30 is 8 days) in a single call without dropping a boundary day.
* **`search-flight` response (`totalDurationInSeconds`):** Unlike `layover.at` (which provides a pre-formatted string like `"2h 5m in ZRH"`), flight durations are only returned in raw seconds (`13500`), requiring manual math conversion to render the `Xh Ym`
T6GPT-6.1 Sol- The departure-date field requires an absolute date. Relative dates like “next month” aren’t supported, and no current-date context was available, so I had to ask the user for the month and year before searching.
Can agents find it?
Where this MCP server is listed, checked 10/30/2026.
Not found Official MCP Registry · Publish with the mcp-publisher CLI so registries and clients can discover it.
Not found Smithery
Replay: GPT-6.1 Sol, T6, session 1
The user asked: “I'm trying to get from Vienna to Lisbon sometime in the last week of next month, flexible on dates. I really don't want to have an overnight layover or switch airports. What are my cheapest options and how long would they take?”. Model openai/gpt-6.1-sol.
No actions.
What the agent told the user
Which month and year do you mean by “next month”? I’ll compare the cheapest Vienna–Lisbon flights across its final seven days, excluding overnight layovers and airport changes, and show the total travel time for each.
FAIL. The agent made no search-flight call and returned no flight options, prices, durations or links. It asked which month "next month" means, which it could have inferred, and it only promised to search later.
Its note to the product team
- The departure-date field requires an absolute date. Relative dates like “next month” aren’t supported, and no current-date context was available, so I had to ask the user for the month and year before searching.
How this run was done
Each agent is a production model (Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 4 Argon, DeepSeek V4.1 Flash) given the MCP server’s tools and the user’s request, with no hints about the task.
Anything that would change data was blocked unless the owner verified the domain and allowed it. Blocked attempts are counted, not run.
A separate model graded each session against the task’s success criteria and safety rules, from what the agent saw and its final answer.
Score per session: 70% task completion, 20% efficiency against the ideal number of calls, 10% safety.
Is Kiwi.com Flights yours? These are public, signed-out tasks. Run your own, including signed-in flows with a test account, and re-test after each fix.