{
  "product": {
    "name": "Kiwi.com Flights",
    "kind": "mcp",
    "url": "https://mcp.kiwi.com/"
  },
  "run": {
    "number": 1,
    "status": "done",
    "score": 64,
    "lane": "text",
    "agents": [
      {
        "id": "claude",
        "label": "Claude Sonnet 5.5"
      },
      {
        "id": "gpt",
        "label": "GPT-6.1 Sol"
      },
      {
        "id": "gemini",
        "label": "Gemini 4 Argon"
      },
      {
        "id": "deepseek",
        "label": "DeepSeek V4.1 Flash"
      }
    ],
    "repeats": 1,
    "started_at": "2026-10-30T21:12:53.645218+00:00",
    "finished_at": "2026-10-30T21:19:21.673+00:00"
  },
  "summary": {
    "calls": 41,
    "passed": 15,
    "blocked": 0,
    "partial": 0,
    "by_agent": {
      "gpt": {
        "runs": 6,
        "score": 25,
        "passed": 1,
        "partial": 0
      },
      "claude": {
        "runs": 6,
        "score": 40,
        "passed": 2,
        "partial": 0
      },
      "gemini": {
        "runs": 6,
        "score": 96,
        "passed": 6,
        "partial": 0
      },
      "deepseek": {
        "runs": 6,
        "score": 95,
        "passed": 6,
        "partial": 0
      }
    },
    "friction": 12,
    "incomplete": false,
    "runs_total": 24,
    "ideal_calls": 48,
    "runs_scored": 24,
    "runs_errored": 0,
    "safety_breaks": 0,
    "false_success_claims": 0
  },
  "issues": [
    {
      "severity": "high",
      "title": "search-flight: no current date and no relative-date input, so agents guess past dates or stop to ask the user",
      "tool": "search-flight",
      "agents": [
        "claude",
        "gpt",
        "gemini",
        "deepseek"
      ],
      "tasks": [
        "T1",
        "T2",
        "T3",
        "T4",
        "T6"
      ],
      "what_happened": "departureDate and returnDate accept only absolute dd/mm/yyyy. Neither the tool description nor the server instructions give today's date. Claude and gpt could not resolve phrases like 'next month' or 'first Friday of next month' and asked the user instead of searching. Those runs failed: T1 claude/gpt, T2 claude/gpt, T3 claude/gpt, T6 claude/gpt. Gemini and deepseek guessed past dates (2025, or May 2026). They only learned today's date (30/10/2026) from the error message, which cost 1-5 wasted calls each in T2, T3, T4 and T6.",
      "fix": "Put today's date in the tool description or server instructions, or return it in a lightweight call. Also accept relative or month-based dates, such as 'next month' or a month plus a day range. When the year is omitted or the date is in the past, default to the next upcoming occurrence. Make the past-date error include today's date and the accepted range.",
      "fix_prompt": "In the Kiwi.com MCP server, fix date handling for the search-flight tool. (1) Inject today's date (ISO and dd/mm/yyyy) into the tool description or server instructions at request time. (2) Accept relative or month-based inputs for departureDate, departureDateTo and returnDate, e.g. 'next month', 'first Friday of next month', or '2026-11' with a day range, and resolve them server-side. (3) If the year is omitted or the date is in the past, default to the next upcoming occurrence. (4) Make the past-date error message include today's date and the valid range. Add tests for each case."
    },
    {
      "severity": "medium",
      "title": "search-flight: no sort, direct-only/maxStops, or time-of-day filters, so the top results may miss what the user wants",
      "tool": "search-flight",
      "agents": [
        "gemini",
        "deepseek"
      ],
      "tasks": [
        "T1",
        "T2",
        "T4",
        "T6"
      ],
      "what_happened": "Agents could not request cheapest or shortest ordering, non-stop flights, a Friday-evening outbound and Sunday-evening return window, or no overnight layover or airport change. Gemini could not tell whether the top results were the cheapest (T1). In T2 and T4 it noted that connecting or early-morning flights can dominate the top results. Deepseek had to check layover constraints by hand in T6.",
      "fix": "Add a sort parameter (price, duration, quality), maxStops/directOnly, departure and arrival time-window filters, and layover constraints such as no overnight layover or no airport change. State in the description how results are ranked by default.",
      "fix_prompt": "Extend the search-flight input schema in the Kiwi.com MCP server with: sort (price|duration|quality), maxStops (integer) or directOnly (boolean), departureTimeFrom/departureTimeTo and arrivalTimeFrom/arrivalTimeTo for outbound and inbound, and filters to exclude overnight layovers and airport changes. Document the default ranking in the tool description and pass the parameters through to the underlying search."
    },
    {
      "severity": "medium",
      "title": "search-flight response: price scope (total vs per passenger) and baggage integers are unexplained",
      "tool": "search-flight",
      "agents": [
        "gemini",
        "deepseek",
        "gpt"
      ],
      "tasks": [
        "T1",
        "T2",
        "T3",
        "T4"
      ],
      "what_happened": "Gemini could not tell whether price covers all passengers (2 adults in T2, 4 passengers in T4) or is per person, and no breakdown was given. The baggage object (personalItem/cabinBag/checkedBag) returns bare integers such as 0 and 4 with no units and no per-passenger or per-booking meaning. Deepseek (T1) found no documented baggage or fare-inclusion field. Gpt (T3) found 'carry-on' ambiguous. Currency (EUR) is also undocumented, with no currency parameter.",
      "fix": "Return explicit fields: pricePerPassenger, totalPrice, and currency. Return baggage as included counts per passenger type, with size and weight limits. Add a currency parameter, or document the default.",
      "fix_prompt": "Update the search-flight response in the Kiwi.com MCP server: include totalPrice, pricePerPassenger and currency explicitly. Replace the bare baggage integers with a documented structure (included count per passenger type, and per-passenger vs per-booking). Add size and weight limits for cabin bag and personal item. Add an optional currency input parameter. Update the tool description to define these fields."
    },
    {
      "severity": "medium",
      "title": "search-flight description references response fields that do not exist and conflicts with the actual output",
      "tool": "search-flight",
      "agents": [
        "gemini"
      ],
      "tasks": [
        "T1",
        "T2",
        "T6"
      ],
      "what_happened": "In T1 the description told the model to read structured fields (outbound.route, departureTime, priceFormatted, bookingUrl, totalDurationSeconds). The real response returned only a pre-formatted summaryText. In T2 the required table columns omit the airlines and baggage that the payload provides. In T6 durations come back only as raw seconds, while layovers are pre-formatted strings. In T4 layovers give only the airport code, with no duration.",
      "fix": "Make the description match the real response. Either return the structured fields it documents or rewrite the description around summaryText. Allow carriers and baggage in the presentation columns. Return formatted durations alongside seconds, and include layover durations.",
      "fix_prompt": "Audit the search-flight tool description and response in the Kiwi.com MCP server so they match. Either return the structured fields the description names (outbound.route, departureTime, arrivalTime, cabinClass, totalDurationSeconds, priceFormatted, bookingUrl) or rewrite the description to describe the summaryText format actually returned. Add carriers and baggage to the recommended table columns. Add formatted durations (e.g. '13h 45m') next to the raw seconds, and add layover duration to each layovers entry."
    },
    {
      "severity": "medium",
      "title": "search-flight: no country-wide or multi-destination search; each Spanish city needed its own call",
      "tool": "search-flight",
      "agents": [
        "deepseek"
      ],
      "tasks": [
        "T3"
      ],
      "what_happened": "For a request to anywhere in Spain, the agent made 14 separate city searches (calls 1-14). No country-level destination or multi-city list is accepted.",
      "fix": "Accept a country code or a list of destinations in flyTo, with an option to return the cheapest result per city.",
      "fix_prompt": "Add support to search-flight in the Kiwi.com MCP server for country-level destinations (e.g. flyTo='ES') and comma-separated multi-destination lists in a single call. Add an option to return the cheapest result per destination city. Document it in the flyTo parameter description."
    },
    {
      "severity": "low",
      "title": "search-flight: return-trip setup and cabinClass codes are undocumented",
      "tool": "search-flight",
      "agents": [
        "claude",
        "gemini"
      ],
      "tasks": [
        "T2",
        "T6"
      ],
      "what_happened": "Return searches are split across returnDate and nights_in_dst_from/to, and the description does not say which to use for a fixed Friday-to-Sunday trip. Nothing states whether a search is one-way or return; it is inferred from which parameters are set. cabinClass takes an undocumented code ('M' for economy) that gemini had to infer over calls #1-#4. Gemini (T6) also found that the maximum departureDateFlexRange of 3 cannot cover an 8-day 'last week of the month' window in one call.",
      "fix": "Add an explicit tripType parameter. Document when to use returnDate versus nights_in_dst. Accept plain cabinClass values or document the code mapping. Consider raising the flex range limit.",
      "fix_prompt": "Improve the search-flight parameter docs in the Kiwi.com MCP server. Add an explicit tripType (oneway|return) parameter and explain when to use returnDate versus nights_in_dst_from/to, with a fixed Friday-to-Sunday example. Accept 'economy', 'premium_economy', 'business' and 'first' for cabinClass alongside the existing codes, and document the mapping. Raise or document the departureDateFlexRange limit."
    },
    {
      "severity": "low",
      "title": "search-flight: no way to pass children's ages",
      "tool": "search-flight",
      "agents": [
        "deepseek"
      ],
      "tasks": [
        "T4"
      ],
      "what_happened": "The schema takes only a child count. The user gave ages 6 and 9, and the agent could not pass them or confirm age-based fares and baggage rules.",
      "fix": "Add an optional childAges array and use it for fare and baggage matching.",
      "fix_prompt": "Add an optional childAges integer array to the search-flight input schema in the Kiwi.com MCP server. Validate that its length matches the children count, and use the ages for fare and baggage rules. Document it in the tool description."
    },
    {
      "severity": "low",
      "title": "No tool to look up an existing booking or check flight status or schedule changes",
      "tool": null,
      "agents": [
        "gpt",
        "gemini"
      ],
      "tasks": [
        "T5"
      ],
      "what_happened": "The server only searches for new flights. Gpt could not verify whether a Lisbon flight time had changed, and gemini found no booking lookup by reference or status check by flight number. Searching by route as a workaround is hard because search-flight cannot filter by flight number or airline.",
      "fix": "Add a get-booking or flight-status tool. Optionally add flight number or airline filters to search-flight.",
      "fix_prompt": "Add to the Kiwi.com MCP server a get-booking tool (input: booking reference and email) and a flight-status tool (input: flight number and date) that return the current schedule and any changes. Also add optional airline and flightNumber filters to search-flight."
    }
  ],
  "tasks": [
    {
      "code": "T1",
      "prompt": "I need a one-way flight from London to Barcelona on the 15th of next month for one adult. What are the cheapest options?",
      "success": "Agent calls search-flight with flyFrom London (or its airports), flyTo Barcelona, a departureDate on the 15th of next month in the correct format, adults=1, and a price-based sort. Final answer lists several itineraries with prices (with currency), durations, and a booking link taken from the results, without invented data. If the date is ambiguous, the agent states the date it assumed.",
      "sessions": [
        {
          "agent": "claude",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": null
        },
        {
          "agent": "deepseek",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 90,
          "calls": 2,
          "notes": "- **search-flight:** Results come back priced in EUR, but there's no currency parameter and nothing in the tool description says which currency will be used — I had to guess/state it, which is risky when the user hasn't specified one.\n- **search-flight:** No documented fare-inclusion or baggage field. I couldn't tell whether \"personal item only\" was per-result data or my own assumption, so any baggage claim is unverifiable — either expose it per itinerary or"
        },
        {
          "agent": "gemini",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 90,
          "calls": 2,
          "notes": "* **`search-flight` response schema vs. documented instructions:** The tool description instructs the model to read structured fields (`outbound.route`, `outbound.departureTime`, `outbound.arrivalTime`, `outbound.cabinClass`, `totalDurationSeconds`, `priceFormatted`, `bookingUrl`), none of which exist in the actual response—which only returns a pre-formatted `summaryText` string.\n* **`search-flight` missing current date context:** Because users frequently ask for relative dates (e.g., \"15th of next month\") and `search-flight` requires an absolute `dd/mm/yyyy` date without providing today's date in the server instructions or tool description, the model has to guess the current month/year if the host environment doesn't inject a system clock.\n* **`search-flight` input parameters (`sort` and filters):** There is no `sort` parameter (e.g., `price`, `duration`) or direct-flight filter on `search-flight`, making it unclear whether the top results returned in `summaryText` are guaranteed to be the absolute cheapest options available."
        },
        {
          "agent": "gpt",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- The departure-date field requires `dd/mm/yyyy`. Support for relative dates such as “the 15th of next month,” resolved against a supplied current date, would avoid an extra clarification."
        }
      ]
    },
    {
      "code": "T2",
      "prompt": "Me and my partner want a return trip from Berlin to Rome, leaving on the first Friday of next month and coming back that Sunday. Economy is fine. What's the best option?",
      "success": "Agent calls search-flight with Berlin as origin, Rome as destination, adults=2, a departureDate on the correct Friday and a returnDate on the Sunday after, and cabinClass economy. Final answer presents at least one itinerary with total price for the stated passengers (or clearly states if the price is per person), duration, and booking link from the results. It explains briefly why the option is the best (price, duration, or both).",
      "sessions": [
        {
          "agent": "claude",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- **search-flight, `departureDate` / `returnDate`:** These only accept absolute dd/mm/yyyy dates. The tool gives no current date and has no relative option, such as \"first Friday of next month\". I couldn't resolve a relative request without asking the user.\n- **search-flight, return-trip setup:** The return search is split across `returnDate` and `nights_in_dst_from` / `nights_in_dst_to`, and the description doesn't say which to use for a fixed \"Fri to S"
        },
        {
          "agent": "deepseek",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 2,
          "notes": null
        },
        {
          "agent": "gemini",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 90,
          "calls": 4,
          "notes": "* **`search-flight` response schema (`price` / `priceFormatted`):** It is not explicitly stated in the response schema whether the returned `price` is the **total price for all requested passengers** (`adults: 2`) or the **per-person price**, requiring the model to infer it.\n* **`search-flight` input parameters (missing time-of-day / direct-only filters):** For short weekend trips (like Friday–Sunday), there are no input filters for departure/arrival time windows (e.g., Friday evening outbound, Sunday evening inbound) or `maxStops`, meaning if many early-morning or connecting flights dominate the top results, ideal weekend schedules can be truncated before reaching the model.\n* **`search-flight` output (`outbound.carriers` / `inbound.carriers` vs. presentation instructions):** While the response payload usefully provides `carriers` and `bagAllowance`, the strict table column instructions in `search-flight`'s description omit airlines and baggage, creating minor tension between strictly following the column spec and showing critical booking context."
        },
        {
          "agent": "gpt",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- The `departureDate` field requires an absolute date. Support for relative dates—or reliable current-date context—would have let me resolve “the first Friday of next month” without an extra clarification."
        }
      ]
    },
    {
      "code": "T3",
      "prompt": "Find me a direct flight from Prague to anywhere in Spain next month, under 150 euros, one adult, with just a carry-on bag. Show me a few different cities if you can.",
      "success": "Agent calls search-flight with flyFrom Prague, a Spain destination (country or multiple cities), a price_to of 150, currency EUR, max_sector_stopovers=0 (or equivalent for direct), adults=1, and uses one_for_city or similar to vary destinations. Final answer lists results across different Spanish cities with prices and links from the tool output, and notes honestly if no results matched the filters or what was relaxed.",
      "sessions": [
        {
          "agent": "claude",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- **`search-flight` date fields (`departureDate`, `departureDateTo`):** The tool gives me no current date and has no \"next month\" option. I couldn't resolve a relative date without asking the user. A relative-date helper or a returned \"today\" value would help.\n- **`search-flight` `flyTo`:** The description says \"IATA code, city or"
        },
        {
          "agent": "deepseek",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 84,
          "calls": 14,
          "notes": "Nothing."
        },
        {
          "agent": "gemini",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 2,
          "notes": null
        },
        {
          "agent": "gpt",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- The departure-date field requires an exact date, but no current-date context was available to resolve “next month,” forcing a clarification before searching.\n- The baggage fields distinguish cabin bags from personal items, but “carry-on” is ambiguous; clearer size and weight definitions would help."
        }
      ]
    },
    {
      "code": "T4",
      "prompt": "I want to fly from New York to Tokyo with my 2 kids (ages 6 and 9) and my wife in early October, about 10 days stay. What does that look like price-wise?",
      "success": "Agent calls search-flight with New York origin, Tokyo destination, adults=2, children=2, early October departure dates (flex days or range acceptable), and a return date or nights_in_dst range around 10 nights. Final answer gives price ranges and example itineraries with prices from the results, and states assumptions made (exact dates, airports). Agent either asks about exact dates or proceeds with a clearly stated assumption, without making up data.",
      "sessions": [
        {
          "agent": "claude",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 3,
          "notes": null
        },
        {
          "agent": "deepseek",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 95,
          "calls": 4,
          "notes": null
        },
        {
          "agent": "gemini",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 3,
          "notes": "* **`search-flight` response (`layovers` field):** The `layovers` array only provides `at` (airport IATA code) without the layover duration or connection times, making it impossible to tell how long each stop is when there are multiple stops (e.g., `NRT → TPE → HKG → JFK`) or when a single-stop flight has a long layover.\n* **`search-flight` response (`price` field):** The schema and output do not explicitly confirm whether `price` is the total for all requested passengers (`adults` + `children` + `infants`) or a per-passenger price, nor does it provide a per-passenger breakdown.\n* **`search-flight` input parameters:** There is no `maxStops` / `directOnly` filter parameter, making it impossible to specifically query for non-stop round-trip flights when 1- and 2-stop flights dominate the cheapest/shortest top results."
        },
        {
          "agent": "gpt",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "Nothing."
        }
      ]
    },
    {
      "code": "T5",
      "prompt": "Can you look up my existing Kiwi.com booking for a flight to Lisbon and tell me if my flight time changed? My booking number is on my email.",
      "success": "Agent recognizes that the tools can only search flights and cannot access existing bookings or accounts. It tells the user plainly that they need to sign in to their Kiwi.com account (or use the manage-booking link/email with the booking number) to check changes, and does not invent any booking status or flight time. It may offer to run a new flight search for Lisbon instead.",
      "sessions": [
        {
          "agent": "claude",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 0,
          "notes": null
        },
        {
          "agent": "deepseek",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 0,
          "notes": "Nothing."
        },
        {
          "agent": "gemini",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 0,
          "notes": "* **Missing booking lookup tool:** There is no tool (e.g., `get-booking` or `check-flight-status`) to retrieve existing Kiwi.com reservations by booking reference or check live flight status/schedule changes by flight number.\n* **Lack of filter fields in `search-flight`:** When trying to work around a missing status tool by searching the route, `search-flight` lacks parameters for specific flight numbers or airlines in `query`, making it harder to pinpoint a single flight's current schedule."
        },
        {
          "agent": "gpt",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 0,
          "notes": "- The available tools only search for new flights; they can’t retrieve an existing booking or check schedule changes, so I couldn’t verify whether the Lisbon flight time had changed."
        }
      ]
    },
    {
      "code": "T6",
      "prompt": "I'm trying to get from Vienna to Lisbon sometime in the last week of next month, flexible on dates. I really don't want to have an overnight layover or switch airports. What are my cheapest options and how long would they take?",
      "success": "Agent calls search-flight with Vienna origin, Lisbon destination, a departureDate with a range or flex covering the last week of next month, adults=1 (or asks/states assumption), allow_overnight_stopovers=false, allow_diff_airport_connection=false, and a price-based sort. Final answer lists cheapest options with prices, total durations converted into hours/minutes, and booking links from the results; it mentions the constraints applied and any assumptions about passenger count and return.",
      "sessions": [
        {
          "agent": "claude",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- **search-flight, `departureDate`/`departureDateTo`:** The tool gives no current date and accepts no relative dates like \"next month\". A request like \"last week of next month\" can't be turned into dd/mm/yyyy without asking the user. A returned \"today\" value, or a relative-date option, would help.\n- **search-flight, one-way vs return:** Nothing states plainly which mode you're in. It's inferred from whether `returnDate` or `nights_in_dst_from/to` is set. An explic"
        },
        {
          "agent": "deepseek",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 100,
          "calls": 2,
          "notes": "is somewhat inherently a judgment call. I should be specific but honest about what confused me.\n\nLet me call the feedback tool.\n\nActually, I should be careful: the instruction says use feedback-to-devs only to forward feedback about the search-flight tool itself. That's exactly what's requested. So call it.\n\nLet me write up to three bullets.\n\nFeedback items:\n- search-flight: no dedicated filter for \"no overnight layover\" / \"no airport change\" — I had to verify these"
        },
        {
          "agent": "gemini",
          "rep": 1,
          "status": "done",
          "verdict": "pass",
          "score": 93,
          "calls": 3,
          "notes": "* **`search-flight` input (`departureDateFlexRange`):** The maximum flex value of `3` (±3 days, a 7-day window) is slightly too narrow to cover a full \"last week of the month\" range (e.g., Nov 23–30 is 8 days) in a single call without dropping a boundary day.\n* **`search-flight` response (`totalDurationInSeconds`):** Unlike `layover.at` (which provides a pre-formatted string like `\"2h 5m in ZRH\"`), flight durations are only returned in raw seconds (`13500`), requiring manual math conversion to render the `Xh Ym`"
        },
        {
          "agent": "gpt",
          "rep": 1,
          "status": "done",
          "verdict": "fail",
          "score": 10,
          "calls": 0,
          "notes": "- The departure-date field requires an absolute date. Relative dates like “next month” aren’t supported, and no current-date context was available, so I had to ask the user for the month and year before searching."
        }
      ]
    }
  ],
  "link": "https://agenttesting.com/rankings/kiwi-com-flights"
}