WoT-Lab

WoT-Lab is a benchmark and framework for evaluating service-consuming agents. Every service is a virtual W3C Web of Things Thing, exposed over the standard WoT HTTP protocol using node-wot. Actions in a Thing Description can carry a declarative effect specification, so the description an agent consumes is the same behaviour that gets executed. Things, environments and tasks are defined as JSON data rather than code.

The benchmark comprises six environments: e-commerce, supply chain, social media, smart home, a smart office derived from the IBM Dublin Building 3 Brick model, and a factory shopfloor based on the MOSAIK model. Together they hold 76 tasks across seven difficulty levels. A Python harness scores agent runs by goal success and plan efficiency, and a React dashboard lets you inspect Things and replay runs.

Links: Source code · Paper (BenchSOC 2026, to appear)

License: AGPL-3.0.