Jacobs’ Dream: Improving Disaster Risk Management Using Visual Clues in the Age of AI and Machine Learning

In late 1981, the legendary urban design expert Allan B. Jacobs was walking along a road in Tangshan, China, visiting a new housing development when he noticed improvised, hand-made grates covering many windows and porches. He commented to his Chinese colleague that in the United States, such grates would indicate to visitors that the residents considered the neighborhood unsafe. When his colleague confirmed the same was true in China, Jacobs realized that most professions use these types of simple visual clues as a way to understand a neighborhood and wondered why urban planners did not follow suit.

This is how Jacobs opens his seminal book, Looking at Cities, dedicated to the art of urban observation and detection of urban clues.¹ Four decades later, Jacobs’ framework remains relevant, even as the tools we use to study and visualize cities have transformed.We no longer require a notebook and comfortable shoes to understand an urban environment. Today, from a continent away, we can see not just whether a neighborhood has grates on its windows, but also whether building foundations are cracking, roads are paved, and sidewalks navigable for wheelchairs. Quite simply, with a computer and internet connection, we can bring Jacobs’ approach to almost any neighborhood anywhere in the world.

Recent technological advancements enable us to efficiently review images, identify urban clues, and reveal spatial patterns across an entire neighborhood or city. By applying machine learning algorithms to high-resolution imagery taken above the city (aerial imagery) and at ground level (street view imagery), we can make comprehensive, multi-view urban observations from the sky and street. The resulting georeferenced datasets and accompanying visualizations hold enormous potential to help create resilient, healthy communities. That includes unprecedented opportunities for spatial planning, disaster and climate risk management and reduction.

Using this data, planners and engineers can analyze built components to gather clues about blocks, neighborhoods, and cities through access to a wide-ranging overview of specific urban characteristics, such as each unit’s:

• Size (area, height, and volume of building)
• Use (residential, commercial, critical infrastructure, or mixed)
• Masonry (unreinforced, reinforced, or unknown)
• Vintage (for example, pre-1940, 1941-1974, 1975-1999, and 2000-present. These are locally determined based on field surveys conducted by structural engineers.)
• Roof condition (good, fair, poor, under construction or vacant)
• Roof material (concrete, metal, mixed, tile, other)
• Wall condition (good, fair, poor)
• Wall material (typically: brick or concrete block, plaster, mix/unclear/other; can include: wood – polished, wood - crude/plank, adobe, corrugated metal, stone with mud/ashlar with lime or cement, container/trailer, plant material)
• Total condition (composite estimation based on roof and wall conditions)

Taken together, these unprecedented rapid, high-resolution screenings across multiple square kilometers enable cheaper, more efficient, and more targeted pre- and post-disaster planning. While collecting a house-by-house census can take months or require thousands of workers, high-resolution images can now be captured by a team of four within a week, often using a single car, a drone, and a few cameras.

As outlined in a newly published paper supported by Global Facility for Disaster Reduction and Recovery’s (GFDRR’s) Global Program for Resilient Housing, this approach enables planners to identify specific buildings at scale that need improvements or strengthening. It holds promise for use as a proxy for social vulnerability, a crucial aspect of disaster risk management. Related work can inform region-wide traffic and infrastructure management decisions and scan for buildings’ structural vulnerabilities in earthquake-prone areas. Future developments might pinpoint roofs that could generate solar power or identify emergency routes and shelters.

Where and how might this be applied?

This approach is suitable for almost any environment. Local officials can prioritize neighborhoods and cities with limited, recent data describing their built environment, especially locations with natural hazards or other risks. Areas of interest typically span 15 to 25 square kilometers, though this approach has been implemented in locations up to 80 square kilometers, making it suitable for small island states.

An important step is to gather the street view imagery. While this is easiest to capture from a car mounted with many cameras (e.g., Google Street View), in informal areas or places with narrow pathways, capturing these images can require attaching a 360° camera (e.g., GoPro Fusion) to a backpack mounted on a motorcycle or carried on foot.

Once the street view data has been collected, it can be uploaded to Mapillary, a crowdsourced platform, which blurs faces and license plates before the imagery is publicly available. As street view coverage is not yet ubiquitous, this provides open-source access to recent street view imagery and circumvents licensing requirements while still protecting personal privacy.

Next, as many contemporary solutions leverage artificial intelligence (AI), machine learning (ML), or deep learning (DL), it is crucial to follow ethical guidelines and mitigate potential biases in training data for a successful project. And, of course, local experts and other users can then review the datasets and ensure that the predictions and classifications made by the algorithms are accurate. This approach has been effective in various countries, including Colombia, Guatemala, Indonesia, Mexico, Paraguay, Peru, Saint Lucia, and Sint Maarten.

Quite notably, in these projects, the urban imagery and machine learning results are navigable in a browser interface. This approach makes detailed information accessible to local planners and officials with an internet connection and login credentials to the open-source geospatial portal.² This means that users of diverse expertise can verify the machine learning predictions with the imagery and conduct simple analyses to visualize patterns in the built environment. The portal can export machine learning predictions and, given their granularity, can be combined with other data for further hazard and risk analysis.

Altogether, using classic observational principles and machine learning algorithms to evaluate buildings, neighborhoods, and urban areas, we can now make educated and efficient inferences about vast, complex built environments. Specific, granular information and visualizations are available to save and improve lives, protect assets, and shield economies from increasing disaster risks.

In short, in 2024, using machine learning and imagery collected from the sky and street, we can finally make Jacobs’ vision from 1981 a reality: widespread, simple visual analysis that helps inform urban development and, ultimately, improves people’s lives.

¹Jacobs A.B. (1985) Looking at Cities. Cambridge: Harvard University Press. 
²The code for the open-source portal: https://github.com/GPRH/housing_portal