Title: | Aid Querying 'nomis' and 'Office for National Statistics Open Geography' APIs |
Version: | 1.1.0 |
Description: | Facilitates extraction of geospatial data from the 'Office for National Statistics Open Geography' and 'nomis' Application Programming Interfaces (APIs). Simplifies process of querying 'nomis' datasets https://www.nomisweb.co.uk/ and extracting desired datasets in dataframe format. Extracts area shapefiles at chosen resolution from 'Office for National Statistics Open Geography' https://geoportal.statistics.gov.uk/. |
License: | MIT + file LICENSE |
Encoding: | UTF-8 |
RoxygenNote: | 7.3.2 |
URL: | https://defra-data-science-centre-of-excellence.github.io/sgapi/, https://github.com/Defra-Data-Science-Centre-of-Excellence/sgapi |
BugReports: | https://github.com/Defra-Data-Science-Centre-of-Excellence/sgApi/issues |
Imports: | magrittr, sf, dplyr, httr, readr, xml2, methods |
Suggests: | testthat (≥ 3.0.0) |
Config/testthat/edition: | 3 |
Depends: | R (≥ 3.6) |
LazyData: | true |
NeedsCompilation: | no |
Packaged: | 2025-05-08 12:28:10 UTC; michaelstocks |
Author: | Michael Stocks [aut, cre], Andrew Christy [aut], James Macpherson [aut], Hadyn Jenkins [aut], Crown Copyright (Department for Environment Food and Rural Affairs) [cph] |
Maintainer: | Michael Stocks <sgapi.orhub@gmail.com> |
Repository: | CRAN |
Date/Publication: | 2025-05-08 12:50:02 UTC |
Escape Function in Case of Errors
Description
Evaluate condition and return error message if condition not satisfied.
Usage
assert_function(condition, msg)
Arguments
condition |
a logical expresssion (e.g. is.null(test_variable)) |
msg |
the error message returned if the condition is not met. |
Value
No return value, called for side effects.
Examples
assert_function(1==2,"Incorrect inequality")
List of available ONS boundaries
Description
A list of the available boundary layers that the Office for National Statistics (ONS) has for ArcGIS queries
Usage
availableBoundaries
Format
A list of with 3348 rows and 1 variable:
- boundary
name of boundary table within 'ONS Open geography'
Source
availableBoundaries.rda
Build constituency section of get_boundary_areaname query string
Description
Create correctly formatted 'where' part of url query string used by get_boundary_areaname to fetch contituency boundaries
Usage
build_constituency_query_string(constituency_list, ...)
Arguments
constituency_list |
Vector of constituency names |
... |
Additional strings to be added to the query string |
Value
A string formatted for 'where' portion of url for querying Open Geography portal
Examples
build_constituency_query_string("Westminster", "LAD22NM", "3D")
"%20(%20'LAD22NM%20%3D%20'Westminster')%20"
build_constituency_query_string(c("Westminster", "Tower Hamlets"), "LAD22NM", "3D")
"%20(%20'LAD22NM%20%3D%20'Westminster'%20OR%20LAD22NM%20%3D%20'Tower%20Hamlets')%20"
Build URL Query String
Description
Takes named arguments (...) and creates a URL query string starting with 'prefix' with each named pair of arguments separated with 'sep' and each item within a value that is a vector separated by 'value_sep'.
Each value part of the named arguments must be coercible to character. For example, list(key1 = "value1", key2 = 42, key3 = c(1, 2, 4)) would be acceptable as it can all be coerced to characters
Usage
build_url_query_string(prefix = "?", sep = "&", value_sep = ",", ...)
Arguments
prefix |
Prefix to returned URL. Defaults to "?". |
sep |
. String separating used to separate in the URL output each named argument in (...). Defaults to "&". |
value_sep |
String separating each item in value if the value part of a named argument is a vector. Defaults to ",". |
... |
Any number of named argument pairs where the value must be coercible to character |
Value
A string containing the named arguments parsed into the URL query string format
Examples
build_url_query_string()
""
build_url_query_string(field1 = "option1")
"?field1=option1"
build_url_query_string(field1 = "option1", field2 = 42, field3 = c(1, 2, 3))
"?field1=option1&field2=42&field3=1,2,3"
build_url_query_string(prefix = "/query?", sep = "|", value_sep = ":",
field1 = 1, field2 = c(7, 8, 9))
"/query?field1=1|field2=7:8:9"
Available Boundary Scales for 'nomis' Table
Description
Retrieve available spatial scales for a given 'nomis' table id. This is useful as each table only has data at a specific set of scales, e.g. many census tables are available at MSOA and LSOA resolutions but not at Regional level.
Usage
get_available_scales(id, base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
id |
A valid 'nomis' table id given as a string. |
base_url |
Nomis API base url |
Value
A tidy dataframe listing the geographical scales available for the 'nomis' table selected.
Examples
get_available_scales(id="NM_1003_1")
Get Boundaries Using Geospatial Filter
Description
Retrieve boundaries from the Office for National Statistics (ONS) 'ONS Geography Portal' given a valid boundary name and layer name. If the submitted geometry is outwith the ONS Boundary, e.g. the geometry is in France, the function will return an empty shape file.
Usage
get_boundaries(boundary, geometry_filter = NULL,
base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services")
Arguments
boundary |
A valid ONS boundary name given as a string. |
geometry_filter |
geospatial shape or point (using latitude and longitude). Currently limited to a rectangular box or dropped pin. |
base_url |
Open Geography Portal base url |
Value
An sf object for all constituencies in the geospatial area submitted through the geometry_filter, at the chosen ONS Boundary.
Examples
## Not run:
get_boundaries(boundary="MSOA_Dec_2011_Boundaries_Generalised_Clipped_BGC_EW_V3_2022",
geometry_filter="-1.282825,52.354169,0.206626,52.7106")
## End(Not run)
Get Boundaries Using Area Names
Description
Extract a geojson shapefile of the chosen areas at the user-selected resolution.
Usage
get_boundaries_areaname(boundary, col_name_var, chosen_constituency_list,
base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services")
Arguments
boundary |
The resolution of constituencies, e.g. Census Output Areas or Westminster Constituencies. Available boundaries can be found here: https://geoportal.statistics.gov.uk/ |
col_name_var |
The name of the datafield where the constituency name is held, e.g. PCON22NM for 2022 Parliamentary Constituencies. |
chosen_constituency_list |
List of chosen constituencies. |
base_url |
Open geography portal base url |
Value
An sf object of the constituencies submitted to the function. If there are no constituencies, return is NULL.
Examples
get_boundaries_areaname(boundary="Local_Authority_Districts_December_2022_UK_BGC_V2",
col_name_var="LAD22NM",chosen_constituency_list=c("Westminster","Tower Hamlets","County Durham"))
Keyword Search of 'nomis'
Description
Function to return dataframe of all 'nomis' tables, and their corresponding codes and descriptions, which have the chosen keyword.
Usage
get_keyword_table_id(
usr_keyword,
base_url = "https://www.nomisweb.co.uk/api/v01"
)
Arguments
usr_keyword |
Keyword to search 'nomis' tables for, e.g. "religion", "employment", "housing" |
base_url |
nomis url to query |
Value
A tidy dataframe of all nomis tables and their ids, which contain the chosen keyword.
Examples
get_keyword_table_id(usr_keyword="passports")
Get ONS data
Description
Retrieve data from the Office of National Statistics (ONS) via the Nomis API for a given dataset.
Usage
get_ons_table(
id,
base_url = "https://www.nomisweb.co.uk/api/v01",
csv_parser_fnc = utils::read.csv,
...
)
Arguments
id |
a table ID recognised by 'nomis' (e.g. "NM_1_1") |
base_url |
Nomis base url to query. Default: "https:www.nomisweb.co.uk/api/v01" |
csv_parser_fnc |
Function for parsing the csv file from Nomis API url. Default: read.csv |
... |
Additional parameters used to filter and aggregate the data retrieved from ONS. These are passed to build_url_query_string to build everything in the url after "?". See "https://www.nomisweb.co.uk/api/v01/help for information on the options available |
Value
A dataframe containing the data downloaded from ONS' Nomis API
Examples
# Pull all data from the "Jobseeker's Allowance" dataset
## Not run: get_ons_table("nm_1_1")
# Filter "Jobseeker's Allowance" dataset and select columns to output ('select')
get_ons_table("nm_1_1", geography = "TYPE480", time = "latest", measures = 20100, item = 1,
select = c("geography_name", "sex_name", "obs_value"))
# Aggregate statistics using 'rows' and 'cols'
get_ons_table("nm_1_1",
geography = "TYPE480", time = "latest", measures = 20100, item = 1,
select = c("geography_name", "sex_name", "obs_value"),
rows = c("geography_name"), cols = c("sex_name"))
Table Overview
Description
Retrieve dataset overview for a valid 'nomis' table id. Returned object includes description of the dataset, last update date, contact for the data. It also extracts all of the available instances of the available table dimensions, which can then be used to filter the dataset 'nomis' table.
Usage
get_overview(id, base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
id |
A valid 'nomis' id. |
base_url |
Base nomis url to query |
Value
An object with overview information of chosen data set. Object has the structure of the extracted JSON object.
Examples
get_overview(id="NM_1_1")
Extract 'nomis' Table Data for Chosen Dimension
Description
Retrieve dataset for a valid 'nomis' table id and dimension. This extracts all of the available instances of the chosen dimension, which can then be used to filter the 'nomis' table.
Usage
get_structure(id, dim, base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
id |
A valid 'nomis' id. |
dim |
The name of the g which dimension is queried. |
base_url |
Base nomis url to query |
Value
An object with JSON structure for the chosen dimension of the selected 'nomis' table.
Examples
get_structure(id="NM_187_1",dim="industry")
Get 'nomis' Table
Description
Retrieve a 'nomis' table, containing the columns of interest, using a given table ID. Option to apply filters to the query, and to circumnavigate the limit on number of rows returned using your 'nomis' uid.
Usage
get_table(
id,
options,
selection = NULL,
uid = NULL,
base_url = "https://www.nomisweb.co.uk/api/v01"
)
Arguments
id |
a table ID recognised by 'nomis' (e.g. "NM_1_1") |
options |
a list of paramaters to pass to the API query. |
selection |
a vector of column names to return. NULL returns all. Defaults to NULL. |
uid |
Unique 'nomis' identifier to enable larger 'nomis' queries - https://www.nomisweb.co.uk/. Defaults to NULL. |
base_url |
Nomis base url to query |
Value
A tidy dataframe of selected 'nomis' table with the selected parameters and user filters applied.
Examples
get_table(id="NM_1_1", options = list("geography" = "TYPE480", "time" = "latest"))
get_table(id="NM_1002_1", options = list("geography" = "TYPE265", "time" = "latest"),
selection = "GEOGRAPHY_NAME,C_AGE_NAME,OBS_VALUE",uid=NULL)
Table Dimensions
Description
Extract dimensions available for a given 'nomis' table ID. e.g. the dimensions of the table 'RM011 - Country of birth by age ' are age, country and geography; this function will return all of the available age, country and geography filters available on the table.
Usage
get_table_dimensions(id, base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
id |
A table ID recognised by 'nomis' (e.g "NM_1_1"). |
base_url |
Nomis API base url |
Value
A tidy dataframe of the dimensions, and available filtering values, of your chosen 'nomis' table.
Examples
get_table_dimensions(id="NM_1240_1")
Get 'nomis 'Table IDs
Description
Extract unique table ids for 'nomis' tables containing given name in their title, these unique table ids can be used to rapidly query census data in other functions e.g. 'get_overview("NM_102_1")'
Usage
get_table_id(name, base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
name |
A string to search for within 'nomis' table titles. |
base_url |
Nomis API base url |
Value
A dataframe of 'nomis' table codes and names, as strings, for all 'nomis' tables containing the selected 'name' in their title.
Examples
get_table_id(name="employment")
Key 'nomis' Table Information
Description
Retrieve summary information about a given 'nomis' dataset. This is useful as it provides the description of the dataset and any caveats. It also returns information about the current status of the data, and when it was last updated.
Usage
get_table_info_brief(id, base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
id |
A valid 'nomis' table id given as a string, e.g. NM_46_1. |
base_url |
Base nomis url to query |
Value
A json file containing the DatasetInfo, DatasetMetadata, Dimensions (variables), Dataset Contact, Units from the target 'nomis' table.
Examples
get_table_info_brief(id="NM_1_1")
Lookup Between Boundary Scales
Description
Extract a lookup table between two boundary scales from 'ONS Open Geography' portal, e.g. get a lookup between Regions and Parliamentary constituencies.
Usage
get_table_link_lookup(
lookup_table,
col_name_1,
col_name_2,
col_name_3,
col_name_4,
base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services"
)
Arguments
lookup_table |
A valid ONS lookup table |
col_name_1 |
Field in ONS table containing the constituency code of the smaller scale resolution. |
col_name_2 |
Field in ONS table containing the constituency code of the larger scale resolution. |
col_name_3 |
Field in ONS table containing the constituency name of the smaller scale resolution. |
col_name_4 |
Field in ONS table containing the constituency name of the larger scale resolution. |
base_url |
Open geography portal base url |
Value
A tidy dataframe, providing a lookup between two chosen boundary resolutions.
Examples
## Not run:
get_table_link_lookup(lookup_table="LAD22_CTY22_EN_LU",col_name_1="LAD22CD",
col_name_2="CTY22CD",col_name_3="LAD22NM",col_name_4="CTY22NM")
## End(Not run)
List Available Boundaries
Description
Retrieve all available ArcGIS boundary layers from the 'ONS Open Geography Portal'.
Usage
list_boundaries(
base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services"
)
Arguments
base_url |
Open geography portal base url |
Value
A vector of available boundary layers on 'ONS Open Geography'.
List 'nomis' Data Sources
Description
Return a list of the data sources available on 'nomis'.
Returns a list including the name, id and description of each data source available on 'nomis'. More information can be found here: https://www.nomisweb.co.uk/api/v01/help
Usage
list_data_sources(base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
base_url |
Base nomis url to query |
Value
A tidy dataframe of all available data sources accessible through the 'nomis' API system.
Examples
list_data_sources()
Available 'nomis' Tables
Description
List all available datasets on 'nomis'. User can then use the items on this list to query 'nomis' data.
Usage
list_tables(base_url = "https://www.nomisweb.co.uk/api/v01")
Arguments
base_url |
Url of the API from which the available tables are listed. |
Value
A tidy dataframe containing the name and ID of each table available on 'nomis'.
Lookup to match Office for National Statistics (ONS) and 'nomis' boundary names
Description
Lookup table providing ONS and 'nomis' references for a given ONS boundary layer. The variables are as follows:
Usage
lookup
Format
A data frame with 30 rows and 4 variables:
- resolution
boundary layer, written in interpretable manner
- ons
corresponding boundary layer in the format that the 'ONS Open Geography' API will interpret
- nomis
corresponding boundary layer in the format that the 'nomis' API will interpret
- Names_and_Codes
lookup file containing the constituency names and codes of the boundary layer
Source
lookup.rda
#' List of tables on 'nomis'
Description
A dataset containing the available tables, codes and sources of data available on 'nomis'. The variables are as follows:
Usage
nomisTables
Format
A data frame with 1605 rows and 3 variables:
- name
name of dataset
- id
code corresponing to dataset
- sourceName
source of dataset
Source
nomisTables.rda
#' List of tables available at each ONS resolution
Description
A dataset containing the available tables on 'nomis', and the boundary layers at which this data is held, e.g. demographic information may only be held in the Census output areas and not NHS regions. The variables are as follows:
Usage
scalesForEachDataset
Format
A data frame with 22564 rows and 3 variables:
- name
name of boundary layer
- value
type of data
- table
nomis code for the table
Source
scalesForEachDataset.rda