lyr3: AI, PE and Change
How We Test
LLMs don’t just need to answer questions—they need to choose the right tools when connected to an MCP server. Our testing system is designed to measure exactly that, giving us a clear picture of how well models navigate real-world tool use.
Case Study: How Manifest Alignment Boosted Tool Use Performance from 70% to 83% for Cooking MCP
Large language models don’t just rely on their training—they also depend on how tools are described and exposed through manifests. Even small wording choices in a manifest can make the difference between a model using the right tool confidently or skipping it altogether.
Case Study: How a Single Manifest Change Boosted LLM Accuracy from 33% to 83% in a Key Use Category
## The Challenge We were testing a simple MCP todo server with three tools: createTodo, listTodos, and a completion tool. The original completion tool was named completeTodo with the bare description: “Mark a todo item as completed.”
Deterministic vs. Interpretive MCP Servers: Speaking the Same Language
MCP isn’t just about exposing tools. It’s about creating an efficient and effective understanding of a server in the eye of the ClientAI. This communication is guided by the manifest and, by extension, the initialization and tool calls. For simplicity, we’ll use “manifest” to represent that entire package. In effect, the manifest becomes the dictionary of the language between server and AI.
Potential and Pitfalls: Security Lessons from Real-World MCP Tools
The Model Context Protocol (MCP) ecosystem has exploded. In just months, developers have published hundreds of servers and thousands of tools. It’s a remarkable show of creativity: everything from lightweight utilities that return the time of day to heavyweight servers that can administer databases, manage cloud infrastructure, or even control a computer.
MCP Tool Distribution
We analyzed 1,945 tools across 238 servers in the MCP Registry to understand how tools are distributed.
MCP Tool Research
This week, we’re launching a new phase of research at Lyr3: digging into the tools that power the MCP ecosystem. So far: - We’ve collected data on > 420 MCP servers. - Across ~300 of them, we’ve identified 1,400+ distinct tools, tool descriptions and inputs.
Link: You are Doing MCP Wrong
Good read of what MCP is and is not from Jim Clark at Docker. link.
MCP Registry
Lyr3.com is happy to present its MCP Registry as a beta.
NEWS: MCP Continues Reach Growth - Sept 9
MCP continues to expand its reach across enterprise and developer ecosystems, with new use cases, platform adoption, and benchmarks revealing both promise and challenges.
Let’s figure out what AI
actually looks like for you
No pitch deck. A conversation about your portfolio, your thesis, and whether I can move the numbers.
Work with me