NeuroCluster Labs · Datasets & benchmarks

Industrial intelligence should be measured, not claimed.

European Industrial Intelligence Dataset & Benchmark

Research programme·In development

Generic AI benchmarks measure language, code and general knowledge. They do not tell a grid operator, a plant manager or a regulator whether a model understands the system it is asked to reason about.

NeuroCluster Labs is designing an open benchmark programme for industrial AI: shared tasks, curated datasets and transparent evaluation for reasoning over assets, networks, time, imagery and operational authority.

Not a leaderboard for its own sake. A shared measure of whether industrial AI is ready for operational use.

Explore the benchmark Contribute a dataset or task

Isometric line drawing of an evaluation lab: miniature grid and factory models on test benches, measurement instruments and a server room, with blue data lines between them.

Benchmarks before claims

General benchmarks do not measure operational capability.

A model can score well on public leaderboards and still misread a network model, confuse two transformers or recommend an action it has no authority to take.

What generic benchmarks measure

Language fluency
General knowledge
Code and mathematics
Single-turn questions
Text-only inputs

What industrial operation requires

Network topology and connectivity
Measurements and events over time
Imagery tied to the correct physical asset
Physical and operational constraints
Evidence behind every conclusion
Staying within assigned authority

If a capability is not measured, it should not be claimed.

The benchmark makes industrial capability inspectable.

Evaluation tracks

Six tracks. Each isolates one industrial capability.

Together they cover what industrial AI must demonstrate before it can be trusted in operations. Select a track to see what it measures.

TRACK 01

Industrial reasoning

Technical reasoning over procedures, standards, engineering documents and operational situations.

Measures: Multi-step technical reasoning grounded in domain documents and system context.
Example task: Given a maintenance procedure and an equipment state, identify which steps apply and which preconditions are not yet met.
Scoring: Correctness
Reasoning validity
Source citation
Data: Procedures
Standards
Engineering documents
Incident reports

The tracks evaluate the research directions of European Industrial Intelligence and Grid Foundation Models.

Anatomy of a task

Every task says what the model sees, may do and must prove.

Inputs, the question, allowed tools, authority, expected output and the scoring rule are fixed before any model is run.

Task · Topology & graph

Illustrative

Question

Which substations lose supply if line L-12 and breaker B-4 are opened?

Inputs: Network model extract
Switching state
Asset register excerpt
Allowed tools: Topology query
Power-flow solver (read-only)
Authority: Read only. No actions.
Expected output: The affected substations, the path that isolates each one, and the evidence for every claim.
Scoring: Match to solver reference
Path validity
Every claim cited

Task network

Illustrative task. Identifiers and network are fictional.

Data tiers

Open where possible. Controlled where necessary.

Industrial data is sensitive. The benchmark is designed in tiers, so operational data never has to leave the organisation that owns it.

Open

Anyone

Public datasets, open network models and synthetic environments, published with tasks and reference answers.

Public grid models
Open weather data
Synthetic scenarios

Controlled

Research partners, under agreement

Anonymised or licensed datasets held in controlled European environments and evaluated under agreement.

Anonymised telemetry
Licensed imagery
Partner task sets

Private

The data owner only

Operational data that stays inside the owner's environment. The model comes to the data; only results leave.

Live operational data
Confidential asset records
Internal procedures

Federated evaluation for the private tier

Model under test

Data owner's environment

Private data

Evaluation harness

Data stays

Only results leave

Scores and evidence summary

Only results leave

Method

From raw data to a result you can trust.

Six stages, each with its own checks. Select a stage to see what it involves.

01 — SOURCE

Collect data from open sources, partners and synthetic environments, with provenance recorded.

Open datasets
Partner data
Synthetic generation
Provenance

Every published result states

Model and version
Track and task set
Tools and context allowed
Compute and deployment setting
Scores with uncertainty
Known failure modes

Principles

Designed to be trusted.

Operationally grounded

Tasks come from real industrial work and are written or reviewed by domain experts.

Transparent

Methods, scoring rules and task formats are published. Hidden test sets stay hidden; how they are used does not.

Contamination-resistant

Held-out and rotating test sets, so results reflect capability rather than memorisation.

Authority-aware

Safety and permission compliance are scored alongside accuracy, not as an afterthought.

Sovereign by design

Evaluation can run inside European and customer-controlled environments.

Model-neutral

Open to any model, open or proprietary. Our own models are evaluated under the same rules.

Who it is for

One shared measure, many uses.

Industrial operators

Compare models on your kind of problems before you deploy them.

Model developers

Measure and improve industrial capability against shared, expert-built tasks.

Universities & research institutes

Work with curated industrial tasks and a common evaluation method.

AI Factories & compute providers

Offer reproducible industrial evaluation as part of sovereign AI infrastructure.

Regulators & public bodies

Ground discussions about industrial AI in inspectable evidence.

Your task, in the benchmark

Have a problem that models should be able to solve? Propose it as a benchmark task.

Propose a task

Roadmap

How the programme is built.

Phases, not dates. Each phase opens once the previous one has been validated.

01
Task design
Define tracks, task formats and scoring with domain experts.
In development
02
First open release
Open tasks on public and synthetic data, with a reference harness.
Planned
03
Controlled tracks
Partner datasets evaluated in controlled European environments.
Planned
04
Federated evaluation
Private operational tasks evaluated where the data lives.
Planned
05
Continuous benchmark
Rotating test sets and regular public reporting.
Planned

Help define how industrial AI is measured.

We are looking for organisations that can contribute tasks, datasets, domain expertise, evaluation environments or compute.

Contribute to the benchmark Explore NeuroCluster Labs

Contribute a task.
Contribute a dataset.
Contribute domain expertise.
Host an evaluation environment.