r - API Query for loop

Question

I'm trying to pull some data from an API throw it all into a single data frame. I'm trying to put a variable into the URL I'm pulling from and then loop it to pull data from 54 keys. Here's what I have so far with notes.

library("jsonlite")
library("httr")
library("lubridate")
options(stringsAsFactors = FALSE)
url <- "http://api.kuroganehammer.com"

### This gets me a list of 58 observations, I want to use this list to 
### pull data for each using an API

raw.characters <- GET(url = url, path = "api/characters")
## Convert the results from unicode to a JSON
text.raw.characters <- rawToChar(raw.characters$content)
## Convert the JSON into an R object. Check the class of the object after 
## it's retrieved and reformat appropriately
characters <- fromJSON(text.raw.characters)
class(characters)

## This pulls data for an individual character. I want to get one of 
## these for all 58 characters by looping this and replacing the 1 in the
## URL path for every number through 58.
raw.bayonetta <- GET(url = url, path = "api/characters/1/detailedmoves")
text.raw.bayonetta <- rawToChar(raw.bayonetta$content)
bayonetta <- fromJSON(text.raw.bayonetta)

## This is the function I tried to create, but I get a lexical error when 
## I call it, and I have no idea how to loop it.
move.pull <- function(x) {
  char.x <- x
  raw.x <- GET(url = url, path = cat("api/characters/",char.x,"/detailedmoves", sep = ""))
  text.raw.x <- rawToChar(raw.x$content)
  char.moves.x <- fromJSON(text.raw.x)
  char.moves.x$id <- x
  return(char.moves.x)
}

Answer 1

The first part of this:

library(jsonlite)
library(httr)
library(lubridate)
library(tidyverse)

base_url <- "http://api.kuroganehammer.com"

res <- GET(url = base_url, path = "api/characters")
content(res, as="text", encoding="UTF-8") %>% 
  fromJSON(flatten=TRUE) %>% 
  as_tibble() -> chars

Gets you a data frame of the characters.

This:

pb <- progress_estimated(length(chars$id))
map_df(chars$id, ~{

  pb$tick()$print()

  Sys.sleep(sample(seq(0.5, 2.5, 0.5), 1)) # be kind to the free API

  res <- GET(url = base_url, path = sprintf("api/characters/%s/detailedmoves", .x))

  content(res, as="text", encoding="UTF-8") %>% 
    fromJSON(flatten=TRUE) %>% 
    as_tibble() 

}, .id = "id") -> moves

Gets you a data frame of all the "moves" and adds the "id" for the character. You get a progress bar for free, too.

You can then either left_join() as needed or group & nest the moves data into a separate list-nest column. If you want that to begin with you can use map() vs map_df().

Leave in the time pause code. It's a free API and you should likely increase the pause times to avoid DoS'ing their site.