GeoHarness

View on GitHub

A minimal ReAct agent evaluating geospatial tasks — currently focused on solar siting.

Commercial AI products built on graph-based architectures — large sets of tools, planners, prescribed traversals — trade flexibility for reliability. As those graphs grow, they become harder to extend and increasingly constrain what newer models can do.

This project tests the alternative: a single ReAct loop (act → observe → reflect) with just two primitive geospatial tools, asking whether a simple agent can reason correctly about complex data without heavy scaffolding.

The eval below runs the agent against 45 hand-labelled locations spanning three verdicts — GOOD, MARGINAL, BAD — using only NASA POWER climate data and OpenTopography terrain data. Web search is excluded so the agent must reason from tool data alone, not training knowledge.

Eval results