At Laterite, we consistently seek to bridge the gap between data and actionable insights. We transform complex datasets into clear, intuitive, and timely information. A common challenge during large-scale fieldwork is building a live monitoring dashboard that allows real-time progress tracking and effective communication with stakeholders.
For our recent work with the Rainforest Alliance on the Thriving Landscapes project in San Martín, Peru, we moved beyond traditional, static Excel reports. Instead, we developed a live monitoring dashboard using Python. This tool provides a secure, intuitive, and constantly updated window into our fieldwork. As a result, it enables better management and more transparent communication.
Below, we share the journey of how we built this dashboard, layering different data sources to create a comprehensive monitoring and visualization platform.
The challenge: Moving beyond automated spreadsheets
The first step in automation is connecting a data collection platform like SurveyCTO directly to a live spreadsheet in Google Sheets. This workflow solves many issues around manual data entry and version control. However, it has limitations:
- Scalability with Complex Data: Google Sheets can face performance issues when handling large datasets or merging different file types, such as survey and geospatial data.
- Limited Customization and Interactivity: Built-in charting and filtering options make it hard to create highly customized, multi-layered interactions. For example, zooming a map to a selected farm from a dropdown menu is difficult.
- Integrating Geospatial Data: Spreadsheets are not designed to visualize and interact with geographic data formats like shapefiles. A more powerful backend is needed.
- Need for On-the-Fly Computation: Spreadsheet formulas cannot match the power of a Python backend for real-time complex data cleaning, processing, and analysis.
To overcome these challenges, we needed a live monitoring dashboard that could scale with the project.
The Goal: Creating a real-time project monitoring tool
The core objective of this project was to establish a comprehensive baseline of agricultural producers in San Martín region for long-term monitoring of the project’s impact. This baseline needed to integrate three distinct data types:
- Socioeconomic Data: Collected through direct farmer surveys (Tab 1).
- Farm Boundaries: Geographic polygons of each farm, essential for tracking land use at the farm level (Tab 3).
- Land Cover Classification: A detailed map of land use (forest, crops, pasture), created using GPS ground-truth points and a spatial classification model (Tab 2).
By combining these layers into a live monitoring dashboard, we created a tool that supports long-term impact monitoring.
The Solution: From spreadsheets to interactive dashboards
To meet the demands of this complex baseline project, we chose to build our dashboard using Streamlit, an open-source Python library. Streamlit lets us to rapidly build and deploy interactive live dashboards directly from Python scripts.
This approach created an interactive window into our rich dataset. Furthermore, it enabled the team to explore relationships between farmers, their land, and the environment in real-time. We combined Streamlit with a suite of powerful data science libraries:
- Pandas for data manipulation
- Plotly for dynamic charts
- GeoPandas with Folium for geospatial visualization
This stack allowed us to develop robust, interactive tools without a dedicated web development team.
Layering the data in a live monitoring dashboard
Layer 1: Monitoring fieldwork progress
The first tab serves as the project’s command center. It provides an at-a-glance overview of survey completion against the target. We load survey data into a Pandas DataFrame, then using Streamlit’s widgets (st.sidebar, st.radio, st.selectbox) to filter data dynamically. All metrics and charts update instantly.
How this can be useful for other projects: This pattern works for any project that involves monitoring progress. For example, it could track daily patient consultations in public health, monitor school visits in education studies, or visualize survey completion across regions. The sidebar acts as a universal control panel adaptable to any categorical or time-series data.
Layer 2: Understanding land use
Next, we added a secondary dataset containing GPS points that classify local land use. For a project aiming to monitor changes in land cover, these points are invaluable. Using them, we trained a Random Forest machine learning model, which generated a detailed land cover map of the entire project area. This baseline makes it possible to measure deforestation or reforestation over time.
The “Land Use Analysis” tab shows the distribution of land use types. Technically, this requires a simple value_counts() operation on the relevant column in a Pandas DataFrame. Nevertheless, it demonstrates a powerful concept: one dashboard can serve as a hub for many data sources.
How this can be useful for other projects: This “contextual data” tab works for quick exploratory analysis or data validation. In an agricultural survey, it could plot reported crop types. In infrastructure projects, it could show road quality. Project managers can quickly spot anomalies or patterns without opening separate files.
Note: The six land-use categories used follow the Greenhouse Gas (GHG) Protocol framework.
Layer 3: Visualizing farm boundaries with geospatial data
Lastly, the most visual feature is the “Farm Polygons Map” tab. Here, we used geospatial data to provide a picture of the project’s geographic footprint. We used GeoPandas to read a shapefile (packaged as a .zip archive) and overlay farm polygons onto a high-resolution satellite basemap with Folium.
To improve usability, we added two interactive features:
- Search and Zoom: A dropdown menu lets users select a Farm ID. The map zooms and centers on the selected polygon, creating a seamless experience.
- Enhanced Styling: We used dynamic styling functions to set high-contrast colors and highlight polygons on hover.
How this can be useful for other projects: Interactive maps have wide applications. For impact evaluations, they can show treatment and control areas. For logistics, they can map clusters against roads to optimize routes. For reporting, they can produce thematic maps, such as color-coded polygons by crop yield or income.
Note: The polygon shown is a randomly generated example for illustration.
Access the code: build your own dashboard
We believe in sharing tools that benefit the research community. The complete Python script for this dashboard is available for adaptation. It can serve as a template for organizations that want to build similar real-time monitoring tools.
Why live monitoring dashboards matter
This project shows how open-source Python tools can transform monitoring. By moving from static spreadsheets to live dashboards, we created a tool that is more efficient for our team and more engaging for partners at the Rainforest Alliance.
Layering survey, contextual, and geospatial data into a live monitoring dashboard advances how we manage and communicate research. The dashboard turns data into a conversation, enabling deeper exploration and shared understanding of project progress.
This blog was written by Ricardo Vargas, a Research Associate from Laterite’s Peru office.