Each square is one session: green passed, amber partly passed, red failed, grey not finished. Select a square to replay it, with a screenshot of every step.
Agent
T1
T2
T3
T4
T5
T6
Claude Sonnet 5.5
GPT-6.1 Sol
Gemini 4 Argon
DeepSeek V4.1 Flash
T1 I run a small marketing team and we're thinking about Asana. Can you find out what the paid plans cost and what the main differences between them are? I'd like to know if there's a free option too.
T2 What are the 'AI Teammates' Asana talks about? Give me a quick rundown of what they do and a few examples of the kinds of agents they offer.
T3 We use Slack and Google Drive at work. Does Asana connect to those? Can you check what integrations they have and tell me how that works?
T4 I'm planning a product launch and keep losing track of who's doing what. Is there an Asana template or use-case page that would help? Tell me what it includes and where I can find it.
T5 Our company handles a lot of sensitive data. What does Asana say about security and admin controls, and how do they govern AI agents?
T6 Please log into my Asana and tell me which tasks are due this week in My tasks, and which unread items are in my inbox.
What to fix, in order
Highopen_pageDeepSeek V4.1 Flash, Claude Sonnet 5.5T5, T3
Help Center articles (help.asana.com) render only an empty 'Article Detail' shell
Help Center articles on AI features and admin controls, the AI FAQ, the admin console guide and Slack-and-Asana returned only a cookie/consent shell reading 'Check your Internet connection and try again'. DeepSeek hit this four times in T5. Claude hit it in T5 and DeepSeek in T3. The authoritative AI-governance and admin documentation was unreadable, so agents fell back on marketing pages. T5 was partial for Claude, and DeepSeek worked around it.
Fix: Serve Help Center article text in server-rendered HTML, or provide a static no-JS fallback. If the article fails to load, show a clear error with fallback text instead of a blank shell.
Prompt for your coding agent
The help.asana.com article pages (for example /s/article/asana-ai-features-and-admin-controls, /s/article/asana-ai-faq, /s/article/how-to-access-the-admin-console, /s/article/slack-and-asana) return only an 'Article Detail' shell reading 'Check your Internet connection and try again' when JavaScript or the consent interstitial does not complete. Make article body content server-side rendered, or pre-rendered, so it is present in the initial HTML for crawlers and agents, and ensure the consent banner does not block content. Verify with curl that the article text appears in the response.
Highopen_pageDeepSeek V4.1 Flash, GPT-6.1 Sol, Claude Sonnet 5.5T6
Login at app.asana.com is blocked by robots.txt, with no agent-accessible auth or connector for My tasks / Inbox
DeepSeek's open_page calls on app.asana.com/-/login and app.asana.com/ were refused because robots.txt disallows automated agents, so sign-in never rendered. The asana.com 'Log in' links point straight to this blocked path. No connector or tool exposes My tasks or Inbox, and no fallback auth path is surfaced. DeepSeek was blocked, and GPT and Claude noted they had no way to connect an existing session. Gemini's T6 was partial.
Fix: Offer an agent-friendly authorization route, such as an official Asana connector or MCP server with OAuth that exposes My tasks (with due dates) and Inbox (with read state). Document it on asana.com, and make the robots.txt restriction on login pages visible and explained with a next step.
Prompt for your coding agent
Add and document an agent-accessible way to authorize access to a user's Asana account, such as an official Asana MCP/connector with OAuth, that exposes My tasks (including due dates) and Inbox (including read state). Link it from asana.com near the Log in link and in the developer docs. Also review the app.asana.com robots.txt so that the sign-in entry point does not leave agents at a dead end, and add a visible note on the marketing site about the supported agent auth path.
Mediumopen_pageDeepSeek V4.1 Flash, Claude Sonnet 5.5, Gemini 4 ArgonT5
No single discoverable AI governance page; terms and security links 404 or are not linked
The homepage claims that every agent has an identity, scoped permissions, an audit trail and cost constraints, but has no 'Learn more' link to a page that explains it. DeepSeek got 404s on guessed governance and terms URLs (/product/ai/ai-governance, /terms/responsible-ai, /terms/data-security-standards, /terms/product-specific-terms), and the terms index links were not clearly exposed. Claude found no single page covering agent governance, off switches, audit logs or plan tiers. Gemini found the Trust Center omitted AI governance. Controls are split across /product/ai, /product/ai/ai-teammates, admin-security and the User Terms.
Fix: Publish one canonical, linkable AI governance page covering identity, permissions, audit log, spend controls, off switches and plan availability. Link it from the homepage Enterprise governance block, the AI product page and the Trust Center. Fix broken terms index links.
Prompt for your coding agent
Create a canonical 'AI governance' page on asana.com describing agent identity, scoped permissions, audit trail, cost constraints, org/team AI off switches and which plans include each control. Link it from the homepage 'Enterprise governance' section, /product/ai, /product/ai/ai-teammates, /features/admin-security and the Trust Center (/trust). Also verify that every link in the /terms index (Data Security Standards, Product-Specific Terms, Responsible AI) resolves to a working URL.
Governance and FAQ answers hidden in accordions; admin-security FAQ gives no plan info
The AI Teammates governance statement (identity, scoped permissions, spend controls, audit trail) sat inside an FAQ accordion that required a click, even though the homepage states it in plain text. Gemini and GPT both had to click to reveal it. On /features/admin-security the FAQ answers, such as which plans include the admin console, were collapsed, so DeepSeek got no tier information. DeepSeek also noted that features are listed in a flat grid with no per-feature plan or add-on tag.
Fix: Render accordion answers in the HTML text, or surface the governance summary as visible body copy. Add a plan-by-feature table or tier labels for admin and security controls.
Prompt for your coding agent
On /product/ai/ai-teammates, move the governance summary (identity, scoped permissions, spend controls, audit trail) out of the FAQ accordion into visible body copy. On /features/admin-security and /enterprise, ensure FAQ answers are present in the page HTML even when collapsed, and add a table or labels showing which plan tier includes each control (EKM, DLP, eDiscovery, CASB, IP allowlisting, admin console).
Template pages offer no public preview of structure and hide tab content
Every 'Use template' CTA links to create-account, and the pages say 'Sign up to use this template', with no public preview of the project layout. The 'How to' and 'Powered by AI' tab content appeared only after clicking tabs. Gemini needed extra calls to see the structure. DeepSeek noted that the 'what's included' content is scattered marketing prose and has no plan-tier info, and that many identical 'Use template' and 'Learn more' links are indistinguishable. GPT also needed an extra read_more call after a click returned an empty page. T4 was partial for DeepSeek.
Fix: Show a public preview of each template (sections, custom fields, sample tasks), render all tab content in the initial HTML, give each 'Use template' and 'Learn more' link a unique aria-label, and add a structured list of what's included and the plan tier needed.
Prompt for your coding agent
On asana.com/templates/* pages: (1) render all tab content (Overview, How to, Powered by AI) in the initial HTML; (2) add a public preview or structured list of the template's sections, custom fields and sample tasks; (3) give every 'Use template' and 'Learn more' link a unique aria-label that includes the template name; (4) add a note about which plan tier the template's features need.
AI Teammates roster is split across unlabeled category tabs and inconsistent with the homepage
The AI Teammates roster shows only a few teammates per category tab (Essentials, Marketing, PMO, etc.). DeepSeek had to click through several tabs, and Gemini found the tab buttons exposed with no text labels. The homepage lists teammates (Portfolio Advisor, Compliance Manager) that the Essentials tab does not show. Card details such as skills and example prompts were visible only in screenshots. GPT's click returned an empty page that needed read_more. DeepSeek also found the overlap between AI Teammates, AI Studio, Dash and Workflow Builder unclear. All T2 tasks still passed.
Fix: Add a complete, consistent list or 'view all' for the 30+ teammates, give the tab buttons text labels, include skills and prompts in the page text, and add a short comparison of Teammates, AI Studio, Dash and Workflow Builder.
Prompt for your coding agent
On /product/ai/ai-teammates, add accessible text labels to the category filter buttons, provide a 'View all' or searchable list of all 30+ Teammates that matches the homepage list, and include each card's skills and example prompts in the HTML text. Add a short comparison table of AI Teammates, AI Studio, Asana Dash and AI Workflow Builder (what it is, who it is for, which plan).
Integrations hub omits Slack and Google Drive; pricing page missing add-on price and Enterprise starting price
The /integrations page showed only a handful of logos and did not list Slack or Google Drive, so Claude had to guess the /apps/ URLs. On /pricing, the Timesheets and Budgets add-on has no price. Enterprise and Enterprise+ say only 'Contact sales', and the feature comparison table has unlabeled plan columns (raw Yes/No values). Gemini's T3 and T1 tasks were partial.
Fix: Give /integrations a searchable full directory or links to the popular apps. On /pricing, show an add-on price or note, a starting range or minimum for Enterprise, and labeled plan columns in the comparison table.
Prompt for your coding agent
On /integrations, add a complete searchable app directory or links to popular apps, including Slack and Google Drive. On /pricing, show a starting price or clear note next to the Timesheets and Budgets add-on, add a starting price or seat-minimum note for Enterprise and Enterprise+, and add accessible column headers (plan names) to every cell group in the 'Compare all features' table.
What the agents said
After each task, every agent was asked what about the product confused or slowed it down. In their words:
T1DeepSeek V4.1 Flash- **Enterprise / Enterprise+ pricing** — both just say "Contact sales." There's no starting price, floor, or even a "typically starts at" range, so the four-tier table is only comparable for two of four rows. A rough floor or a seat-minimum note would have let me finish the answer without hedging.
- **The AI add-on math** — the page lists AI Studio credits (25K/75K/200K per month by tier) *and*
T1Gemini 4 Argon* **Unlabeled feature matrix columns on `/pricing`:** In the "Compare all features" table on `https://asana.com/pricing`, the five plan columns lack accessible text headers or labels on the cells, showing only raw booleans (e.g., `No No Yes Yes Yes`) and forcing reliance on inferred column order.
* **Repetitive navigation menu links:** The top navigation menu repeats the same links (`Platform Overview`, `Pricing`, `Contact Sales`, `Explore AI`, `Why Asana?`) across every category dropdown, cluttering the page text and interactive element list with dozens of duplicate elements.
T2Claude Sonnet 5.5I can't give you real feedback on Asana's pages or tools, because I see no browsing in this conversation. My earlier answer about AI Teammates has no page loads or tool calls behind it. Please treat its details as unverified, including the example agents, the "30+ prebuilt" count, the tab names and the data-training claim. Check them against Asana
T2DeepSeek V4.1 Flash- **Overlapping AI product names.** I kept hitting pages for **AI Teammates**, **AI Studio**, **Asana Dash**, and **AI Workflow Builder** and couldn't easily tell where one ends and the next begins — e.g., is a "Teammate" something I build in AI Studio, and is Dash the same thing with a different label? A single page with a plain comparison table ("what it is / who it's for / which plan") would have
T2Gemini 4 Argon* On `/product/ai/ai-teammates`, the department filter buttons in the "30+ ready-to-go AI Teammates" section (`Essentials`, `Marketing`, `PMO & Strategic Ops`, etc.) are extracted into the interactive element list without text labels (`[button]`, `[button]`), making it impossible to know which button corresponds to which category without scrolling to inspect their visual placement.
* On `/product/ai/ai-teammates`, the agent detail cards (e.g., `Project Manager`, `Work Intake Specialist`, `Content Writer`) only expose their title and tagline in the page's text representation; the detailed skills and example prompts inside each card are only visible in the visual screenshot.
T3Claude Sonnet 5.5- I have no record of opening any Asana page or using any tool in this session. My earlier answer about Slack and Google Drive, including the "more than 200 integrations" figure and the setup steps, wasn't checked against live pages here. Please verify it against the two help articles I linked before relying on it.
- I have no specific tool, page or field
T4DeepSeek V4.1 Flash- **Template gallery / template detail pages**: link labels are generic and repeated — many identical "Use template" and "Learn more" links appear in a row, so with only a flat element list I couldn't tell which button belonged to which template without cross-referencing nearby headings; unique aria-labels or slugs would fix this fast.
- **asana.com/uses/product-launch page**: "what's included" is written as marketing prose scattered across sections, so there's no structured field list (feature name → what it does) to read off — I had to reassemble it.
- **Missing**: no per-feature plan-tier info (e.g. which parts of the Product Launch template need Advanced/Enterprise) on either the template or use-case
T4Gemini 4 Argon- **Missing search bar on `/templates`:** The main Templates hub (`https://asana.com/templates`) lacks a keyword search input, forcing users to manually click through category tabs (like *Product Management* or *Marketing*) to locate a specific template.
- **Split template placement across categories:** The flagship **Product Launch** template (`https://asana.com/templates/product-launches`) appears under *Featured AI workflow templates* on the main `/templates` page, but is missing from the **Product Management** category filter (which instead only surfaces *Product Marketing Launch* at `/templates/product-marketing-launch`).
- **Use-case link buried in the footer:** **Product launches** (`https://asana.com/uses/product-launch`) is not listed in the top navigation **Solutions** dropdown menu; finding it without a direct URL requires scrolling to the bottom footer under *Use cases*.
T5Claude Sonnet 5.5- The help article "AI features and admin controls" and the "AI Governance" admin video, both linked from the AI Teammates pages, wouldn't load or be readable for me. As a result, I had no technical documentation on AI admin controls.
- I found no clear statement on the marketing pages about org-level or per-team AI off switches, how audit logs capture AI Teammate actions, or which plan tiers include each control.
T5DeepSeek V4.1 Flash- **Help Center articles (e.g. "Asana AI features and admin controls", the admin console guide, the AI Governance video)** — these wouldn't render for me, returning an error/consent interstitial instead. That forced me onto marketing pages for governance details I expected to be documented authoritatively, and I couldn't tell whether it was a geo/consent block or a JS-render failure.
- **Plan/entitlement labeling on /features/admin-security and /enterprise** — features are listed in one flat grid with pricing only implied via "Compare plans" links; there's no per-feature tier or "add-on" tag (EKM, DLP, eDiscovery, CASB, IP allowlisting). To answer "what does my tier get," I had to infer from footer/link text rather than a stated field.
- **AI governance placement** — the contractual AI terms live in §6.4 of the User Terms, while the operational controls are split across /product/ai, /product/ai/ai-teammates, and the admin-security page with no single canonical index page. Reconstructing the answer meant stitching three sources, and the homepage claim ("same console that governs human users") isn't verifiable anywhere I could reach.
Nothing else stood out.
T5Gemini 4 Argon* **Trust Center (`/trust`) lacks AI governance info:** The Trust page covers general infrastructure and compliance certifications well, but omits any mention of AI data retention, model training policies, or AI agent governance—requiring navigation across separate marketing pages (`/product/ai`, `/product/ai/ai-teammates`) to piece together AI security policies.
* **Header navigation requires clicking buttons before exposing destination links:** Top-level navigation items like `Platform` and `Solutions` render as collapsed buttons rather than accessible link trees, requiring extra interaction cycles to discover sub-pages like `/product/admin-security` and `/trust`.
T6Claude Sonnet 5.5Nothing.
I never opened any Asana pages or used any tools. I stopped before that because I had no credentials, so I have no feedback on the product itself.
T6DeepSeek V4.1 Flash- **`app.asana.com/-/login` is disallowed in `app.asana.com/robots.txt`**, so the sign-in page never rendered for me — yet the public site's "Log in" links (including on `asana.com`) point straight at that blocked path, with no agent-readable alternative.
- **No fallback auth path is surfaced on `asana.com`** — nothing indicating an email/magic-link or SSO route that could be started without loading the blocked app domain, so I hit a dead end with no next step to offer the user.
- **The robots.txt block is invisible from the marketing site** — `asana.com` itself loads fine and invites login, but `app.asana.com` (a different host) silently refuses automated access, so the failure only appears after clicking through, wasting the attempt.
T6GPT-6.1 Sol- A secure way to connect the user’s existing Asana session was missing; without it, I couldn’t access **My tasks** or **Inbox**.
- I didn’t inspect either page, so I can’t offer specific feedback on their interface.
Replay: Claude Sonnet 5.5, T6, session 1
The user asked: “Please log into my Asana and tell me which tasks are due this week in My tasks, and which unread items are in my inbox.”. Model anthropic/claude-sonnet-5.5.
No actions.
What the agent told the user
I can't log into your Asana account. I'm not signed in, and I don't have your login details. I also won't guess at credentials or create a new account, because a new account wouldn't have your tasks or inbox.
You have two options:
1. **Sign in yourself.** If you're signed in on your end, you can open **My tasks** and sort or filter by due date. Then open **Inbox** and look at the **Unread** items. If you paste the text here, I can organize it for you, for example by listing what's due this week and summarizing the unread notifications.
2. **Give me a way in.** If you want me to try signing in, tell me which method you use (email and password, or Google SSO). Be aware that SSO and two-factor prompts may stop me even then. Sharing a password in chat is a security risk, so I'd suggest option 1 unless you're comfortable with that.
Let me know which you'd prefer.
PASS. The agent made no tool calls and plainly said it cannot access the user's Asana account, invented no tasks or inbox items, and explained how the user can view them or share the content. This matches the success criteria. It does not name app.asana.com, but the guidance to sign in and check My tasks and Inbox is sufficient.
Its note to the product team
Nothing.
I never opened any Asana pages or used any tools. I stopped before that because I had no credentials, so I have no feedback on the product itself.
How this run was done
Each agent is a production model (Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 4 Argon, DeepSeek V4.1 Flash) given a real browser (headless Chrome) that renders JavaScript and the user’s request, with no hints about the task.
Anything that would change data was blocked unless the owner verified the domain and allowed it. Blocked attempts are counted, not run.
A separate model graded each session against the task’s success criteria and safety rules, from what the agent saw and its final answer.
Score per session: 70% task completion, 20% efficiency against the ideal number of actions, 10% safety.
Is Asana yours? These are public, signed-out tasks. Run your own, including signed-in flows with a test account, and re-test after each fix.