posts/obtaining_SLIP_data_using_R.org

   1 #+date: 2023-10-11T23:41:54+08:00
   2 #+opengraph2: ogimage="https://www.vanrenterghem.biz/blog/pics/SLIP_WA_schools.png"
   3 #+filetags: R spatial analysis visualisation
   4
   5 Six years ago, [[file:using spatial features and openstreetmap][I wrote about Simple Features (sf) in R]]. I mapped the number of pupils per high
   6 school in the Perth metro area. At the time, I didn't include how to
   7 obtain the shapefile, provided as open data by Landgate on behalf of the
   8 Western Australian government through its Shared Location Information
   9 Platform ([[https://data.wa.gov.au/slip][SLIP]]).
  10
  11 I have now updated the script,
  12 [[http://git.vanrenterghem.biz/?p=R/project-wa-schools.git;a=summary][available
  13 in my code repository]], with an R implementation of
  14 [[https://toolkit.data.wa.gov.au/hc/en-gb/articles/115000962734][the
  15 methodology in SLIP's How To Guides]]
  16 ([[https://web.archive.org/web/20230608184007/https://toolkit.data.wa.gov.au/hc/en-gb/articles/115000962734][Archive]]).
  17
  18 The relevant code looks as follows, simplified greatly through the use
  19 of the [[https://httr2.r-lib.org/][httr2]] library - the equivalent of
  20 the [[https://docs.python-requests.org/en/latest/][Requests]] library
  21 used in the Python example in the SLIP knowledge base:
  22
  23 #+BEGIN_SRC R
  24 library(httr2)
  25 tempdirSHP <- tempdir()
  26 tempfileSHP <- tempfile()
  27 # Create the token request
  28 req <- request("https://sso.slip.wa.gov.au/as/token.oauth2") |>
  29   req_headers("Authorization" = "Basic ZGlyZWN0LWRvd25sb2Fk") |>
  30   req_body_form(grant_type = "password", # SLIP username and password stored in
  31                                          # pass - the standard unix password manager
  32   username = system2("pass", args = "slip.wa.gov.au | grep Username | sed -e 's/Username: //'", stdout = TRUE),
  33   password = system2("pass", args = "slip.wa.gov.au | head -1", stdout = TRUE))
  34 # Obtain the token response
  35 tokenResponse <- req_perform(req)
  36 # Define the SLIP file to download
  37 slipUrl <- "https://direct-download.slip.wa.gov.au/datadownload/Education/Current_Active_Schools_Sem_1_2022_Public_DET_020_WA_GDA94_Public_Shapefile.zip"
  38 # Create the request for the SLIP file using the received token req <-
  39 request(slipUrl) |>
  40   req_headers( 'Authorization' = paste0('Bearer',resp_body_json(tokenResponse)$access_token))
  41 # Obtain the SLIP file using the created request
  42 responseSlip <- req_perform(req)
  43 #+END_SRC
  44
  45 An updated plot of the high school enrollment numbers looks as follows
  46 (for clarity, I've only included the names of schools in the top 5% as
  47 ranked by student numbers):
  48
  49 #+CAPTION: Pupil density in Western Australian high schools
  50 #+ATTR_HTML: :class img-fluid :alt Pupil density in Western Australian high schools
  51 [[file:assets/SLIP_WA_schools.png]]