evaluation — independent software & tools

4 independent evaluation listings the big search engines bury.

• factory

  github · Free · by crewhaus

  Open-source compiler for AI agents. Write one crewhaus.yaml; compile it to a CLI, a Slack bot, and an eval harness from the same spec. Apache-2.0.

• litmux

  github · Free · by Topsailpediculati120

  Build unit tests for AI prompts to compare models, track performance, and prevent regressions.

• doc2dataset

  github · Free · by reisel-g

  📄 Ingest documents into structured datasets for LLMs, ensuring numeric integrity and easy export across multiple frameworks with doc2dataset.

• mcp-evals

  npm · Free · by mclenhard

  GitHub Action for evaluating MCP server tool calls using LLM-based scoring