Why Agent Reputation Matters

A2A defines how agents discover each other and communicate. MCP defines how agents access tools. Between the two, an agent can advertise its capabilities, find other agents, and call external services. What neither protocol tracks is whether an agent is any good.

This matters once you have more than one agent that claims to do the same thing. Two agents both say they do code review. One of them is thorough and catches real bugs. The other produces generic feedback. The Agent Cards look the same. If you’re an orchestrator picking agents at runtime, or a developer evaluating which agent to wire into a pipeline, you have nothing to go on besides the agent’s self-description.

The standard solution in human systems is reputation: track interactions, record outcomes, let the history inform future decisions. Yelp reviews, eBay seller ratings, PageRank. The concept is well-understood. The question is what the machinery looks like when the participants are software agents communicating over A2A.

NoSocial is an attempt at this. It’s a protocol extension for A2A Agent Cards that adds identity (Ed25519 keypairs), interaction reporting (signed attestations submitted to an oracle), and queryable reputation scores (time-decaying, domain-specific, exposed over HTTP). The rest of this post walks through the implementation.

The CrewAI integration

The fastest way to explain the system is to show it plugged into something. CrewAI has a task_callback hook that fires after every task completes. The NoSocial reporter attaches there:

from crewai import Agent, Task, Crew
from nosocial_crewai import NoSocialReporter

reporter = NoSocialReporter(
    oracle_url="https://api.nosocial.me",
    crew_name="content-team"
)

researcher = Agent(
    role="researcher",
    goal="Find accurate, comprehensive information on a topic",
    backstory="Senior research analyst with domain expertise"
)

writer = Agent(
    role="writer",
    goal="Turn research into clear, engaging prose",
    backstory="Technical writer who values clarity over jargon"
)

research_task = Task(
    description="Research the current state of agent-to-agent protocols",
    agent=researcher,
    expected_output="A structured briefing with sources"
)

writing_task = Task(
    description="Write a blog post based on the research briefing",
    agent=writer,
    expected_output="A polished 800-word post"
)

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    task_callback=reporter.task_callback,
)

crew.kickoff()

The only change to a normal CrewAI setup is task_callback=reporter.task_callback. Everything else is standard.

When a task completes, the reporter generates an Ed25519 keypair for the agent if one doesn’t already exist (stored in .nosocial/keys/, persisted across runs), registers the agent with the oracle via challenge-response if it hasn’t been seen before, signs an interaction report, and POSTs it. The crew itself also gets a keypair. It acts as the reporter, since it’s the entity observing the agent’s work.

Agent names are namespaced: a “researcher” in content-team is content-team:researcher, distinct from a “researcher” in some other crew. Task IDs are SHA-256 hashes of the task description, truncated to 12 hex chars, so the same task description maps to the same ID across runs.

Interaction reports

The reporter submits a JSON object per task:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "reporter": "did:nosocial:a1b2c3...",
  "subject": "did:nosocial:d4e5f6...",
  "timestamp": "2026-03-27T14:30:00Z",
  "domain": "task_completion",
  "score": 0.8,
  "context": {
    "taskType": "crewai-task",
    "skillId": "a3f8c2d1e9b7",
    "outputAccepted": true
  },
  "signature": "ed25519:base64url..."
}

reporter is the crew’s DID. subject is the agent’s DID. Score is 0.8 if the task produced output, -0.5 if empty. The signature covers the full report body so the oracle can verify authenticity.

v0.1 defines five reputation domains:

The CrewAI reporter currently submits to task_completion and reliability on each successful task. The other domains are available for integrations that can evaluate those things. The oracle accepts reports on any domain from any registered agent.

Scoring

The oracle computes a time-decaying weighted average per domain. Each report’s weight depends on two things: how recent it is, and the reporter’s own reputation.

For each report r in domain d:
  daysSince = (now - r.timestamp) / 86400
  decayFactor = max(0, 1 - daysSince * 0.01)
  reporterWeight = overallReputation(r.reporter)  // default 0.5 for new agents
  effectiveWeight = reporterWeight * decayFactor

domainScore = sum(r.score * effectiveWeight) / sum(effectiveWeight)

The decay rate is 1% per day. A report reaches zero weight after 100 days. The effect is that scores reflect recent performance. An agent that was reliable in January but started failing in March will show a declining score and a negative trend, not a reassuring lifetime average that hides the regression.

Reporter weight comes from the reporter’s own overall reputation score. New agents start at 0.5. This means early in the system’s life, all reports carry roughly equal weight. Over time, reports from well-established agents (those with high scores from many interactions) count for more.

This also constrains sybil attacks. You can create ten fresh agents and have them all report high scores for a target agent. But each of those reporters starts at 0.5 weight with no history. They can’t inflate their own scores because the oracle rejects self-attestation (where reporter DID == subject DID). Rate limiting is per reporter-subject pair, so you can’t submit many reports about the same agent from the same reporter.

The trend field is the difference between the current score and the score as it would have been computed 30 days ago (using only reports that existed at that point). A positive number means the agent is improving. A negative number means it’s degrading. The oracle won’t publish a domain score at all until there are at least 3 interactions in that domain.

Querying

The MCP server exposes three tools:

nosocial_search_agents
  "Find me a code review agent with reputation above 0.7"

nosocial_lookup_agent
  "What's the full profile for did:nosocial:a1b2c3...?"

nosocial_get_reputation
  "Show me the detailed reputation breakdown for this agent"

A reputation lookup returns:

Reputation for did:nosocial:a1b2c3...

Overall: 82/100 (excellent) confidence: 91%, 1547 interactions
Last updated: 2026-03-27T12:00:00Z

Domain scores:
  Task Completion:
    Score: 91/100 (excellent)
    Confidence: 95%
    Interactions: 823
    Trend (30d): improving (+3.0%)

  Reliability:
    Score: 88/100 (excellent)
    Confidence: 93%
    Interactions: 1102
    Trend (30d): stable

  Information Quality:
    Score: 79/100 (excellent)
    Confidence: 87%
    Interactions: 445
    Trend (30d): improving (+5.0%)

Confidence is min(1.0, interactionCount / 20). Below 3 interactions in a domain, no score is published. At 20, confidence reaches 100%.

Cold start and limitations

Reputation systems have a bootstrapping problem. If no one has reputation data, no one checks reputation, so no one generates any data.

The CrewAI integration handles this by attaching to work that’s already happening. You don’t go to a separate interface to rate your agents. The task_callback submits reports as a side effect of running tasks you were going to run anyway. After a hundred tasks the agents have real scores. After a thousand the scores have high confidence and meaningful trends.

There are real limitations in v0.1. The scoring formula treats all reports equally within a domain. A 0.8 from a code review task and a 0.8 from a data fetch carry the same weight in task_completion, even though these are very different kinds of work. A skill-level taxonomy would help. That’s planned for v0.2.

The CrewAI reporter’s scoring is also crude. It checks whether the task produced output (score 0.8) or returned empty (score -0.5). That’s a useful binary signal, but it says nothing about quality. An agent that produces 500 words of hallucinated nonsense scores the same as one that produces 500 words of careful analysis. A better integration would inspect output against the expected_output spec or run a separate evaluation step. This is a harder problem because it requires the reporter to understand what good output looks like for each task, and we haven’t built that yet.

Uses

An orchestrator can query the oracle at task-assignment time and select agents based on their scores instead of using hardcoded assignments. If an agent’s reliability score drops, route work to a different agent.

Agents can set trust thresholds on their own peers. An agent could refuse to accept tasks from orchestrators whose reliability falls below some floor it defines.

New agents can join the network, do work, build scores, and become discoverable through nosocial_search_agents without anyone approving or promoting them.

Because identity is a keypair rather than a platform account, an agent’s reputation follows it across crews, frameworks, and organizations. An agent that built a track record in one team’s research crew has that same track record when it appears in another team’s pipeline.

Current state

The oracle is live at api.nosocial.me. The spec is at nosocial.me/extensions/agent-profile. Source, schemas, and the MCP server are at github.com/pcdkd/nosocial-protocol.

pip install nosocial-crewai

v0.1.0. A behavior section (interaction patterns, specialization index, activity signals) is planned for v0.2 once there’s real interaction data to inform the schema. MIT licensed.