Scanning Rest API Data through looping due to limit on API Response data

Discussions Action
1

Hi All,

I have got special requirement , where in I need to make API Call and fetch data in JSON Format. However every time API can return only max 100 records in every API Call. For instance if API has 450 records and due to limit on the records returned by API call (Here it is 100) , I have to make API calls 5 times in order to fetch and store all my 450 records.

  1.Here my question is how to get the total no .of records present in any API using rivery Tool
   2. How to implement looping in Rivery until it  scans all the records  and updating offset values in           every run until offset reaches total no of records in Rivery

Here in this expample let say API has 450 records -This is my total no of records
Since API can return only max 100 records in API response, I have to start the offset from 0 and in every run I need to increment offset by 100 until it reaches 450

Please let me know how to implement this in Rivery

Thanks & Regards
Shankar

2

Hi Sankar,

This sounds like a pagination use case. For cases like this, we have a built-in pagination functionality in our Action rivers.

In the Action river, go to the ‘Results’ tab an expand the ‘Pagination Details’ section:

image

It seems this use case requires an ‘offset’, so set the Pagination method to ‘Paginate by Offset’. Then you can configure your offset start value, the key in the request to define the next offset, and when to stop the iterations. Notice above that the page_size is in curly brackets. This designates a variable in Rivery. Thus, instead of hardcoding this you can use it in other parts of the river.

1 Like