SokoData — build story
Open market-price intelligence for African food markets: one API over 22 scattered data sources, live at sokodata.onrender.com.
The question
What does maize cost today — in Harare, in Lusaka, in any of the hundreds of African markets WFP publishes? A trader, a farmer, or an NGO watching food security can't get a straight answer, because the data exists but it's scattered.
Market prices live in WFP datasets as raw CSVs. National economic series live at central banks and statistics offices. Climate at meteorological services. Each is a different format, a different license, a different level of messiness — and no unified API, no provenance, no product for end users.
SokoData is the commons that turns those scattered sources into a clean, documented, queryable product.
What I built
One ETL pipeline per dataset, one catalog, one FastAPI service. The core is African market-price intelligence: hundreds of WFP markets across the continent. Wrapped around it is a deep Zimbabwe data commons — 22 datasets spanning economy, climate, demographics, agriculture and more. As of today it serves:
Datasets
Markets, economy, climate, demographics, agriculture, health, education, energy, water, transport, mining, governance, trade, labour, environment, poverty, ICT, finance, tourism, aid, gender, geospatial.
Markets, with coordinates
A directory queryable by location, with 31 commodities and per-market coverage counts.
Live surfaces
REST API with interactive docs, a browsable dashboard on GitHub Pages, and a versioned catalog endpoint for discovery.
Decisions worth defending
Why an API and not a report?
A report answers one question once. An API answers every question forever — and forces me to keep the data honest, because someone will actually query it.
How do you survive messy currency history?
Every price carries USD and local values with provenance per record. No silently rebased series. If a number changed meaning when a currency was redenominated, the API says so.
Why scraping alongside official APIs?
World Bank and WFP have APIs; RBZ, ZERA and ZIMSTAT mostly don't. Open data means meeting sources where they are — with licensing respected per source (HDX CC BY-IGO, WDI, Open-Meteo, NASA POWER).
What it looks like
$ curl sokodata.onrender.com/v1/insights/movers?window_days=90
Fish (kapenta) at Marula: $6.10 → $10.53/kg (+72%)
Oil (vegetable) at Gokwe: $1.70 → $2.74/L (+61%)One call answers what used to require downloading four CSVs and cleaning them yourself. That's the whole pitch — for any market WFP publishes, not just one country's.
What it taught me
Data engineering is analysis at scale. The cleaning instincts from my BI work — type inference, duplicate handling, provenance — turned out to be exactly what an ETL pipeline needs.
Honest analytics survive reality. Currency redenominations and source quirks break naive price series. Designing for that made the product trustworthy.
Ship the boring infrastructure. A catalog endpoint nobody applauds is what makes 22 datasets discoverable.
Tech stack: Python, FastAPI, pandas, Render, WFP/HDX, World Bank WDI, Open-Meteo, NASA POWER — plus respectful scraping where no API exists.