Police API Stop and Search lookup with Streamlit
Police API Stop and Search lookup with Streamlit
Following on from the quick API demo I built for the Vehicle Enquiry Service API, I wanted to revisit some work I’d previously done that used the UK Police Data API along with metabase and other technologies to help interrogate public policing data.
Using the streamlit library to provide an interface for querying UK Police stop and search data.
As with the previous code that queried an API the application needs a mechanism to query the API, process the returned data and display the returning information. Further the application should also take inputs from a user, that specify Forces and date ranges to be looked up.
The Streamlit library is an incredible resource for building and rapidly sharing data-based applications. Streamlit will provide the layout, text inputs, buttons, and seamless integrations to popular libraries.
The Pandas library provides ready to use data structures and data analysis tools. The Requests library is the de facto standard for making HTTP requests in Python, so the application will use this to interact with the UK Police Data API. Altair and pydeck are libraries that provide engaging ways of presenting the data returned from any query.
All together the application uses Python, with Streamlit, Pandas, Requests, Datetime, Altair and pydeck and the UK Police Data API.
Using Anaconda I can create a Python environment that contains only the libraries that the application needs, and avoids creating issues with my local copies of python used by the OS for my device. With anaconda available I can either use Anaconda Navigator to import the required libraries to the virtual python environment or I can open the virtual environment from the command line and interact directly using pip. For example, to install streamlit, I could run the following from the virtual environment command line.
pip install streamlit
With the libraries available the python script that streamlit will call can be created and we can launch streamlit from the virtual environment command line.
streamlit run app.py
The Python Script
Rather than go through line by line what the script is doing in this post I’ve commented the code and also made the project available on github for review.
import streamlit as st
import pandas as pd
from pandas import json_normalize
import requests
import datetime
import altair as alt
import pydeck as pdk
def police_api():
def police_query():
st.info("Change the selection below to set different parameters for the API query.")
# By building a df object from the returned json values we can use these in a selectbox, capturing the st.session_state key allows us to pass the value back to the API query string
# On value change updates the state and recalls the function
st.selectbox(
'Select the Police Force',
(force_json_df["id"].unique()), key="force", on_change=police_query)
st.session_state.force
# We can do something similar with the date, but we need to transform the datetime returned value to YYYY-MM format, later rather than calling the key we're going to call the variable that the key updates, because then we can get it in the correct format
# on value change updates the state recalls the function
date = st.date_input("Select a date", datetime.date(2020, 1, 1), key="qdate", on_change=police_query)
date = date.strftime("%Y-%m")
st.write(date)
# This is better, passing the force and date time values into the API query string for stop and search
url = (f'https://data.police.uk/api/stops-force?force={st.session_state.force}&date={date}')
# error check write the API query string to the page
st.markdown(f'<h1 style="color:#6ac94f;font-size:12px;">The Police API Query String URL = {url}</h1>', unsafe_allow_html=True)
# execute the API call and build the Stop and search df added simple error handling to catch the error if the API call fails
try:
res = requests.get(url)
j = res.json()
police = json_normalize(j)
# rebuild a smaller dataframe with just the plotable columns to pass to st.pydeck_chart
# some error checking to only pull in numeric values and to drop na values in lat column,
police["lat"] = police["location.latitude"]
police['lat'] = pd.to_numeric(police['lat'])
police["lon"] = police["location.longitude"]
police['lon'] = pd.to_numeric(police['lon'])
df = pd.DataFrame(police, columns=['lat', 'lon'])
df.dropna(subset=['lat'], inplace=True)
# build the st.pydeck_chart configuration and display the map
st.markdown(f'<h1 style="font-size:18px;">Stop and Search Data Map for {st.session_state.force, date}</h1>', unsafe_allow_html=True)
st.pydeck_chart(pdk.Deck(
map_style='mapbox://styles/mapbox/light-v9',
initial_view_state=pdk.ViewState(
latitude=51.50,
longitude=-0.1276,
zoom=7,
pitch=40,
),
layers=[
pdk.Layer(
'HexagonLayer',
data=df,
get_position='[lon, lat]',
radius=200,
elevation_scale=4,
elevation_range=[0, 1000],
pickable=True,
extruded=True,
),
pdk.Layer(
'ScatterplotLayer',
data=df,
get_position='[lon, lat]',
get_color='[200, 30, 0, 160]',
get_radius=200,
),
],
))
# show the full df from the API call
st.markdown(f'<h1 style="font-size:18px;">Stop and Search Data Table for {st.session_state.force, date}</h1>', unsafe_allow_html=True)
st.dataframe(police)
# download function for df
def convert_df(police):
return police.to_csv().encode('utf-8')
csv = convert_df(police)
st.download_button(
label="Download Output as CSV",
data=csv,
file_name=(f'StopandSearch_{st.session_state.force}_{date}.csv'),
mime='text/csv',
)
# build and show and altair chart
st.markdown(f'<h1 style="font-size:18px;">Stop and Search Data Chart, age_range and object_of_search {st.session_state.force, date}</h1>', unsafe_allow_html=True)
chart = alt.Chart(police).mark_area().encode(
alt.X('age_range:N'),
alt.Y('count(kind):Q'),
color='object_of_search:N'
).properties(
width=500,
height=500
)
st.altair_chart(chart, use_container_width=True)
# Error handling for the API call, error appears if nothing returned by JSON stream
except:
st.error("Error, please check the API query string - no data returned for the selected Police Force and date")
st.info("Change the selection below to set different parameters for the API query.")
# To pass a force into the stop and search API we need to capture the force IDs from the police API.
force = requests.get("https://data.police.uk/api/forces")
force_json = force.json()
force_json_df = json_normalize(force_json)
# By building a df object from the returned json values we can use these in a selectbox, capturing the st.session_state key allows us to pass the value back to the API query string
# On value change updates the state and calls the function above
st.selectbox(
'Select the Police Force',
(force_json_df["id"].unique()), key="force", on_change=police_query)
st.session_state.force
# We can do something similar with the date, but we need to transform the datetime returned value to YYYY-MM format, later rather than calling the key we're going to call the variable that the key updates, because then we can get it in the correct format
# on value change updates the state calls the function above
date = st.date_input("Select a date", datetime.date(2020, 1, 1), key="qdate", on_change=police_query)
date = date.strftime("%Y-%m")
st.write(date)
# first API call, implementation without any variables, for the first select and date data inputs
url = "https://data.police.uk/api/stops-force?force=avon-and-somerset&date=2020-01"
# error check write the API query string to the page
st.markdown(f'<h1 style="color:#6ac94f;font-size:12px;">The Police API Query String URL = {url}</h1>', unsafe_allow_html=True)
# execute the API call and build the Stop and search df added simple error handling to catch the error if the API call fails
try:
res = requests.get(url)
j = res.json()
police = json_normalize(j)
# rebuild a smaller dataframe with just the plotable columns to pass to st.pydeck_chart
# some error checking to only pull in numeric values and to drop na values in lat column,
police["lat"] = police["location.latitude"]
police['lat'] = pd.to_numeric(police['lat'])
police["lon"] = police["location.longitude"]
police['lon'] = pd.to_numeric(police['lon'])
df = pd.DataFrame(police, columns=['lat', 'lon'])
df.dropna(subset=['lat'], inplace=True)
st.markdown(f'<h1 style="font-size:18px;">Stop and Search Data Map for {st.session_state.force, date}</h1>', unsafe_allow_html=True)
# build the st.pydeck_chart configuration and display the map
st.pydeck_chart(pdk.Deck(
map_style='mapbox://styles/mapbox/light-v9',
initial_view_state=pdk.ViewState(
latitude=51.50,
longitude=-0.1276,
zoom=7,
pitch=40,
),
layers=[
pdk.Layer(
'HexagonLayer',
data=df,
get_position='[lon, lat]',
radius=200,
elevation_scale=4,
elevation_range=[0, 1000],
pickable=True,
extruded=True,
),
pdk.Layer(
'ScatterplotLayer',
data=df,
get_position='[lon, lat]',
get_color='[200, 30, 0, 160]',
get_radius=200,
),
],
))
# show the df from the API call
st.markdown(f'<h1 style="font-size:18px;">Stop and Search Data Table for {st.session_state.force, date}</h1>', unsafe_allow_html=True)
st.dataframe(police)
# download function for the df
def convert_df(police):
return police.to_csv().encode('utf-8')
csv = convert_df(police)
st.download_button(
label="Download Output as CSV",
data=csv,
file_name=(f'StopandSearch_{st.session_state.force}_{date}.csv'),
mime='text/csv',
)
# build and show and altair chart
st.markdown(f'<h1 style="font-size:18px;">Stop and Search Data Chart, age_range and object_of_search {st.session_state.force, date}</h1>', unsafe_allow_html=True)
chart = alt.Chart(police).mark_area().encode(
alt.X('age_range:N'),
alt.Y('count(kind):Q'),
color='object_of_search:N'
).properties(
width=500,
height=500
)
st.altair_chart(chart, use_container_width=True)
# Error handling for the API call, error appears if nothing returned by JSON stream
except:
st.error("Error, please check the API query string - no data returned for the selected Police Force and date")
st.set_page_config(layout="wide", page_title="Police Data API Explorer")
st.title("Police Data API Explorer")
st.info("This is a Streamlit app that allows you to explore the Police Data API. Click the button in the side bar to start with Stop and Search data")
if st.button("Click to explore the Police Stop and Search Data API"):
police_api()
Packaging the application
I covered how to package a streamlit application in a previous post, the process for packaging this is identical.
Video of running MVP
I appreciate looking at the code is a little dry, so here is a video (hosted on github) that showcases what the code above does.
The video might not load on mobile, and can be accessed from this link instead
Summary
The goal was to take what I learned through building a web based application to retrieve vehicle information from the UK DVLA Vehicle Enquiry Service API and apply that to the UK Police Data API.
A repository with the code required to build this is available on github. The police data is open, so there isn’t even a requirement for a key to access the data.
No support offered or liability accepted, use is entirely at your own risk.
Hopefully this is interesting or useful to someone else!
Simon