Handling Pagination in Custom API Response

Hi all!

I’ve set up a custom connection to the Genesys PureCloud API and I’m looking for ideas on how to deal with pagination in regards to the response data. Pagination details are required in the API request body (pictured below).


However, there’s nothing in the response data that indicates how many pages your request yielded, what page you’re currently pulling or if you’ve reached the last page in the response. Additionally, the response is never blank or null. For example, if I were to make a request that only yields two pages of data, but I specify in my request body that I want to pull the seventh page of the response, the API would return the last page in the response (in this case the second page) without indicating that the page you’ve requested doesn’t actually exist. As a result, I haven’t been able to utilize Rivery’s built in pagination functionality.

My initial idea was to use a Rivery variable to indicate the page number in the request body. The variable value would increase by one each time the action river with the API request body is called and the action river that contains the request would be looped over using a condition step in my logic river. The tricky piece is figuring out how to configure a dynamic stopping point for the condition loop. The number of response pages that need to be loaded into the target (Snowflake in this case) varies day by day so I can’t think of any way I can hard code this process.

3 Likes

Hi Gwen,

Agreed, this is a tricky one!

I do have one idea to detect the first page that returns no results. Snowflake offers a TABLES view that contains a field called ROW_COUNT. Thus, we can create a loop that pulls a page from the API, and then checks if the ROW_COUNT has changed in the table.

So in addition to your initial idea, it would be another step in the loop that gets the difference in ROW_COUNT from the previous iteration and if this is unchanged, the loop will end.

Something like this:

Let me know if this is helpful!

5 Likes