Title: Aid Querying 'nomis' and 'Office for National Statistics Open Geography' APIs
Version: 1.1.0
Description: Facilitates extraction of geospatial data from the 'Office for National Statistics Open Geography' and 'nomis' Application Programming Interfaces (APIs). Simplifies process of querying 'nomis' datasets https://www.nomisweb.co.uk/ and extracting desired datasets in dataframe format. Extracts area shapefiles at chosen resolution from 'Office for National Statistics Open Geography' https://geoportal.statistics.gov.uk/.
License: MIT + file LICENSE
Encoding: UTF-8
RoxygenNote: 7.3.2
URL: https://defra-data-science-centre-of-excellence.github.io/sgapi/, https://github.com/Defra-Data-Science-Centre-of-Excellence/sgapi
BugReports: https://github.com/Defra-Data-Science-Centre-of-Excellence/sgApi/issues
Imports: magrittr, sf, dplyr, httr, readr, xml2, methods
Suggests: testthat (≥ 3.0.0)
Config/testthat/edition: 3
Depends: R (≥ 3.6)
LazyData: true
NeedsCompilation: no
Packaged: 2025-05-08 12:28:10 UTC; michaelstocks
Author: Michael Stocks [aut, cre], Andrew Christy [aut], James Macpherson [aut], Hadyn Jenkins [aut], Crown Copyright (Department for Environment Food and Rural Affairs) [cph]
Maintainer: Michael Stocks <sgapi.orhub@gmail.com>
Repository: CRAN
Date/Publication: 2025-05-08 12:50:02 UTC

Escape Function in Case of Errors

Description

Evaluate condition and return error message if condition not satisfied.

Usage

assert_function(condition, msg)

Arguments

condition

a logical expresssion (e.g. is.null(test_variable))

msg

the error message returned if the condition is not met.

Value

No return value, called for side effects.

Examples

assert_function(1==2,"Incorrect inequality")


List of available ONS boundaries

Description

A list of the available boundary layers that the Office for National Statistics (ONS) has for ArcGIS queries

Usage

availableBoundaries

Format

A list of with 3348 rows and 1 variable:

boundary

name of boundary table within 'ONS Open geography'

Source

availableBoundaries.rda


Build constituency section of get_boundary_areaname query string

Description

Create correctly formatted 'where' part of url query string used by get_boundary_areaname to fetch contituency boundaries

Usage

build_constituency_query_string(constituency_list, ...)

Arguments

constituency_list

Vector of constituency names

...

Additional strings to be added to the query string

Value

A string formatted for 'where' portion of url for querying Open Geography portal

Examples

build_constituency_query_string("Westminster", "LAD22NM", "3D")
"%20(%20'LAD22NM%20%3D%20'Westminster')%20"

build_constituency_query_string(c("Westminster", "Tower Hamlets"), "LAD22NM", "3D")
"%20(%20'LAD22NM%20%3D%20'Westminster'%20OR%20LAD22NM%20%3D%20'Tower%20Hamlets')%20"


Build URL Query String

Description

Takes named arguments (...) and creates a URL query string starting with 'prefix' with each named pair of arguments separated with 'sep' and each item within a value that is a vector separated by 'value_sep'.

Each value part of the named arguments must be coercible to character. For example, list(key1 = "value1", key2 = 42, key3 = c(1, 2, 4)) would be acceptable as it can all be coerced to characters

Usage

build_url_query_string(prefix = "?", sep = "&", value_sep = ",", ...)

Arguments

prefix

Prefix to returned URL. Defaults to "?".

sep

. String separating used to separate in the URL output each named argument in (...). Defaults to "&".

value_sep

String separating each item in value if the value part of a named argument is a vector. Defaults to ",".

...

Any number of named argument pairs where the value must be coercible to character

Value

A string containing the named arguments parsed into the URL query string format

Examples

build_url_query_string()
""
build_url_query_string(field1 = "option1")
"?field1=option1"
build_url_query_string(field1 = "option1", field2 = 42, field3 = c(1, 2, 3))
"?field1=option1&field2=42&field3=1,2,3"
build_url_query_string(prefix = "/query?", sep = "|", value_sep = ":", 
                       field1 = 1, field2 = c(7, 8, 9))
"/query?field1=1|field2=7:8:9"


Available Boundary Scales for 'nomis' Table

Description

Retrieve available spatial scales for a given 'nomis' table id. This is useful as each table only has data at a specific set of scales, e.g. many census tables are available at MSOA and LSOA resolutions but not at Regional level.

Usage

get_available_scales(id, base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

id

A valid 'nomis' table id given as a string.

base_url

Nomis API base url

Value

A tidy dataframe listing the geographical scales available for the 'nomis' table selected.

Examples

get_available_scales(id="NM_1003_1")


Get Boundaries Using Geospatial Filter

Description

Retrieve boundaries from the Office for National Statistics (ONS) 'ONS Geography Portal' given a valid boundary name and layer name. If the submitted geometry is outwith the ONS Boundary, e.g. the geometry is in France, the function will return an empty shape file.

Usage

get_boundaries(boundary, geometry_filter = NULL, 
base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services")

Arguments

boundary

A valid ONS boundary name given as a string.

geometry_filter

geospatial shape or point (using latitude and longitude). Currently limited to a rectangular box or dropped pin.

base_url

Open Geography Portal base url

Value

An sf object for all constituencies in the geospatial area submitted through the geometry_filter, at the chosen ONS Boundary.

Examples

## Not run: 
get_boundaries(boundary="MSOA_Dec_2011_Boundaries_Generalised_Clipped_BGC_EW_V3_2022",
geometry_filter="-1.282825,52.354169,0.206626,52.7106")

## End(Not run)


Get Boundaries Using Area Names

Description

Extract a geojson shapefile of the chosen areas at the user-selected resolution.

Usage

get_boundaries_areaname(boundary, col_name_var, chosen_constituency_list, 
base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services")

Arguments

boundary

The resolution of constituencies, e.g. Census Output Areas or Westminster Constituencies. Available boundaries can be found here: https://geoportal.statistics.gov.uk/

col_name_var

The name of the datafield where the constituency name is held, e.g. PCON22NM for 2022 Parliamentary Constituencies.

chosen_constituency_list

List of chosen constituencies.

base_url

Open geography portal base url

Value

An sf object of the constituencies submitted to the function. If there are no constituencies, return is NULL.

Examples

get_boundaries_areaname(boundary="Local_Authority_Districts_December_2022_UK_BGC_V2",
col_name_var="LAD22NM",chosen_constituency_list=c("Westminster","Tower Hamlets","County Durham"))


Keyword Search of 'nomis'

Description

Function to return dataframe of all 'nomis' tables, and their corresponding codes and descriptions, which have the chosen keyword.

Usage

get_keyword_table_id(
  usr_keyword,
  base_url = "https://www.nomisweb.co.uk/api/v01"
)

Arguments

usr_keyword

Keyword to search 'nomis' tables for, e.g. "religion", "employment", "housing"

base_url

nomis url to query

Value

A tidy dataframe of all nomis tables and their ids, which contain the chosen keyword.

Examples

get_keyword_table_id(usr_keyword="passports")

Get ONS data

Description

Retrieve data from the Office of National Statistics (ONS) via the Nomis API for a given dataset.

Usage

get_ons_table(
  id,
  base_url = "https://www.nomisweb.co.uk/api/v01",
  csv_parser_fnc = utils::read.csv,
  ...
)

Arguments

id

a table ID recognised by 'nomis' (e.g. "NM_1_1")

base_url

Nomis base url to query. Default: "https:www.nomisweb.co.uk/api/v01"

csv_parser_fnc

Function for parsing the csv file from Nomis API url. Default: read.csv

...

Additional parameters used to filter and aggregate the data retrieved from ONS. These are passed to build_url_query_string to build everything in the url after "?". See "https://www.nomisweb.co.uk/api/v01/help for information on the options available

Value

A dataframe containing the data downloaded from ONS' Nomis API

Examples

# Pull all data from the "Jobseeker's Allowance" dataset
## Not run: get_ons_table("nm_1_1")
# Filter "Jobseeker's Allowance" dataset and select columns to output ('select')
get_ons_table("nm_1_1", geography = "TYPE480", time = "latest", measures = 20100, item = 1, 
              select = c("geography_name", "sex_name", "obs_value"))

# Aggregate statistics using 'rows' and 'cols'
get_ons_table("nm_1_1", 
              geography = "TYPE480", time = "latest", measures = 20100, item = 1, 
              select = c("geography_name", "sex_name", "obs_value"),
              rows = c("geography_name"), cols = c("sex_name"))


Table Overview

Description

Retrieve dataset overview for a valid 'nomis' table id. Returned object includes description of the dataset, last update date, contact for the data. It also extracts all of the available instances of the available table dimensions, which can then be used to filter the dataset 'nomis' table.

Usage

get_overview(id, base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

id

A valid 'nomis' id.

base_url

Base nomis url to query

Value

An object with overview information of chosen data set. Object has the structure of the extracted JSON object.

Examples

get_overview(id="NM_1_1")


Extract 'nomis' Table Data for Chosen Dimension

Description

Retrieve dataset for a valid 'nomis' table id and dimension. This extracts all of the available instances of the chosen dimension, which can then be used to filter the 'nomis' table.

Usage

get_structure(id, dim, base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

id

A valid 'nomis' id.

dim

The name of the g which dimension is queried.

base_url

Base nomis url to query

Value

An object with JSON structure for the chosen dimension of the selected 'nomis' table.

Examples

get_structure(id="NM_187_1",dim="industry")

Get 'nomis' Table

Description

Retrieve a 'nomis' table, containing the columns of interest, using a given table ID. Option to apply filters to the query, and to circumnavigate the limit on number of rows returned using your 'nomis' uid.

Usage

get_table(
  id,
  options,
  selection = NULL,
  uid = NULL,
  base_url = "https://www.nomisweb.co.uk/api/v01"
)

Arguments

id

a table ID recognised by 'nomis' (e.g. "NM_1_1")

options

a list of paramaters to pass to the API query.

selection

a vector of column names to return. NULL returns all. Defaults to NULL.

uid

Unique 'nomis' identifier to enable larger 'nomis' queries - https://www.nomisweb.co.uk/. Defaults to NULL.

base_url

Nomis base url to query

Value

A tidy dataframe of selected 'nomis' table with the selected parameters and user filters applied.

Examples

get_table(id="NM_1_1", options = list("geography" = "TYPE480", "time" = "latest"))
get_table(id="NM_1002_1", options = list("geography" = "TYPE265", "time" = "latest"),
selection = "GEOGRAPHY_NAME,C_AGE_NAME,OBS_VALUE",uid=NULL)


Table Dimensions

Description

Extract dimensions available for a given 'nomis' table ID. e.g. the dimensions of the table 'RM011 - Country of birth by age ' are age, country and geography; this function will return all of the available age, country and geography filters available on the table.

Usage

get_table_dimensions(id, base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

id

A table ID recognised by 'nomis' (e.g "NM_1_1").

base_url

Nomis API base url

Value

A tidy dataframe of the dimensions, and available filtering values, of your chosen 'nomis' table.

Examples

get_table_dimensions(id="NM_1240_1")


Get 'nomis 'Table IDs

Description

Extract unique table ids for 'nomis' tables containing given name in their title, these unique table ids can be used to rapidly query census data in other functions e.g. 'get_overview("NM_102_1")'

Usage

get_table_id(name, base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

name

A string to search for within 'nomis' table titles.

base_url

Nomis API base url

Value

A dataframe of 'nomis' table codes and names, as strings, for all 'nomis' tables containing the selected 'name' in their title.

Examples

get_table_id(name="employment")


Key 'nomis' Table Information

Description

Retrieve summary information about a given 'nomis' dataset. This is useful as it provides the description of the dataset and any caveats. It also returns information about the current status of the data, and when it was last updated.

Usage

get_table_info_brief(id, base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

id

A valid 'nomis' table id given as a string, e.g. NM_46_1.

base_url

Base nomis url to query

Value

A json file containing the DatasetInfo, DatasetMetadata, Dimensions (variables), Dataset Contact, Units from the target 'nomis' table.

Examples

get_table_info_brief(id="NM_1_1")


Description

Extract a lookup table between two boundary scales from 'ONS Open Geography' portal, e.g. get a lookup between Regions and Parliamentary constituencies.

Usage

get_table_link_lookup(
  lookup_table,
  col_name_1,
  col_name_2,
  col_name_3,
  col_name_4,
  base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services"
)

Arguments

lookup_table

A valid ONS lookup table

col_name_1

Field in ONS table containing the constituency code of the smaller scale resolution.

col_name_2

Field in ONS table containing the constituency code of the larger scale resolution.

col_name_3

Field in ONS table containing the constituency name of the smaller scale resolution.

col_name_4

Field in ONS table containing the constituency name of the larger scale resolution.

base_url

Open geography portal base url

Value

A tidy dataframe, providing a lookup between two chosen boundary resolutions.

Examples

## Not run: 
get_table_link_lookup(lookup_table="LAD22_CTY22_EN_LU",col_name_1="LAD22CD",
col_name_2="CTY22CD",col_name_3="LAD22NM",col_name_4="CTY22NM")

## End(Not run)


List Available Boundaries

Description

Retrieve all available ArcGIS boundary layers from the 'ONS Open Geography Portal'.

Usage

list_boundaries(
  base_url = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services"
)

Arguments

base_url

Open geography portal base url

Value

A vector of available boundary layers on 'ONS Open Geography'.


List 'nomis' Data Sources

Description

Return a list of the data sources available on 'nomis'.

Returns a list including the name, id and description of each data source available on 'nomis'. More information can be found here: https://www.nomisweb.co.uk/api/v01/help

Usage

list_data_sources(base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

base_url

Base nomis url to query

Value

A tidy dataframe of all available data sources accessible through the 'nomis' API system.

Examples

list_data_sources()


Available 'nomis' Tables

Description

List all available datasets on 'nomis'. User can then use the items on this list to query 'nomis' data.

Usage

list_tables(base_url = "https://www.nomisweb.co.uk/api/v01")

Arguments

base_url

Url of the API from which the available tables are listed.

Value

A tidy dataframe containing the name and ID of each table available on 'nomis'.


Lookup to match Office for National Statistics (ONS) and 'nomis' boundary names

Description

Lookup table providing ONS and 'nomis' references for a given ONS boundary layer. The variables are as follows:

Usage

lookup

Format

A data frame with 30 rows and 4 variables:

resolution

boundary layer, written in interpretable manner

ons

corresponding boundary layer in the format that the 'ONS Open Geography' API will interpret

nomis

corresponding boundary layer in the format that the 'nomis' API will interpret

Names_and_Codes

lookup file containing the constituency names and codes of the boundary layer

Source

lookup.rda


#' List of tables on 'nomis'

Description

A dataset containing the available tables, codes and sources of data available on 'nomis'. The variables are as follows:

Usage

nomisTables

Format

A data frame with 1605 rows and 3 variables:

name

name of dataset

id

code corresponing to dataset

sourceName

source of dataset

Source

nomisTables.rda


#' List of tables available at each ONS resolution

Description

A dataset containing the available tables on 'nomis', and the boundary layers at which this data is held, e.g. demographic information may only be held in the Census output areas and not NHS regions. The variables are as follows:

Usage

scalesForEachDataset

Format

A data frame with 22564 rows and 3 variables:

name

name of boundary layer

value

type of data

table

nomis code for the table

Source

scalesForEachDataset.rda