Create Connection Flow for OAuth2 using the Rivery API

Knowledge Tutorial
1

Introduction & Summary

The recent introduction of new endpoints to the Rivery API has opened up a new host of programmatic workflows developers can take advantage of. In this community article, we explore the programmatic creation of connections using OAuth2.0 in a rivery account and the use cases that this flow enables.

In this workflow, using the curl command line tool we make a series of calls to retrieve an auth_uri link for a data source that can be opened in a browser, authorized by the user (via OAuth2.0) and it’s credentials saved in a connection in the Rivery account UI. In summary the workflow is as followed:

  1. Create a Rivery API token in your Rivery account.
  2. Get list of all connection types (GET Connection Types List)
  3. Make a new credentials request (OAuth2 New Credentials Request)
  4. Send the auth_uri link to the user for authentication.
  5. Wait for the user/client to authorize the source.
  6. Once authorized, retrieve the credentials that the user inputted (OAuth2 GET Credentials)
  7. Provide the credentials obtained in the response into a PUT request (Create New Connection)
  8. Update connection (PUT Update Connection by ID) - send in whole connection body, with changed parameter

Implementation

1. Create a new API token in your Rivery account.

Under the “API Tokens” tab in the Rivery console, click “Add Token”. Give it a name and assign the scopes as shown below:

1036Ă—732 44.4 KB

Keep the access_token in a safe place once the token is created. You will NOT be able to view this access token again after the screen.

2. Get the list of all available connection types using GET Connection Types List.

This will provide you with a list of datasource_id’s that you can use in the next step.

3. Execute the OAuth2 New Credentials Request.

Ensure that you provide the correct string datasource_id for the source type that you wish to add. In making this and all subsequent API calls, make sure you use the access token created in step(1). In the subsequent example calls and responses, we are using the datasource_id for the google sheets connection (gspreadsheet) to create the connection. Note that the auth_uri link retrieved from the response to this call will only be valid for the duration of the validity_duration parameter specified in the call.

1318Ă—998 176 KB

List of example datasource_id’s.

Example call:

curl -X POST \
-d '{"datasource_id": "<datasource_id>","validity_duration": 1300}' \
-H "Content-Type: application/json"  \
-H "Authorization: Bearer <access_token>"\
https://console.rivery.io/api/oauth2

Example Response (200):

{

"pull_id": "eyJhbGciOiJIUzI1NiIsImV4cCI6MTYwOTE5MjA2NCwiaWF0IjoxNjA5MTkxNzY0fQ.eyJwdWxsX2lkIjoiNWZlYTUxNTRmNTY4M......",
"auth_uri": `"https://accounts.google.com/o/oauth2/v2/auth?prompt=consent&client_id=<CLIENT ID>&access_type=offline&state=_wbqCDbGUKj9t0BD6YIp2ZsB_70pT9ipXSs1B0x1oT0._wbqCDbGUKj9t0BD6YIp2ZsB_70pT9ipXSs1B0x1oT0&redirect_uri=<redirectUrl>&response_type=code&include_granted_scopes=true&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fspreadsheets.readonly+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&display=popup"`

}

4. Send the auth_uri to the user/client that you wish to create the connection for.

Get the auth_uri and pull_id from the response to step (3). Let the client open the auth_uri in an external browser.

5. Let the client complete the OAuth2.0 authorization process to the desired source connection that you are adding.

Let the user complete this (possibly over a zoom call). The authorization is successful once the client sees a JSON with the key value “Connected = True” in their browser.

6. Request the OAuth2 GET Credentials endpoint.

Use the pull_id and the auth_uri retrieved in step (3).

1342Ă—820 73.6 KB

Example call:

curl -d '{"pull_id": "eyJhbGciOiJIUzI1NiIsImV4cCI6MTYwOTE5MzQ5NSwiaWF0IjoxNjA5MTkyMTk1fQ.eyJwdWxsX2lkIjoiNWZlYTUzMDNmNTY4MmM3ZWJlZ...", "auth_uri": ""auth_uri": `"https://accounts.google.com/o/oauth2/v2/auth?prompt=consent&client_id=<CLIENT ID>&access_type=offline&state=_wbqCDbGUKj9t0BD6YIp2ZsB_70pT9ipXSs1B0x1oT0._wbqCDbGUKj9t0BD6YIp2ZsB_70pT9ipXSs1B0x1oT0&redirect_uri=<redirectUrl>&response_type=code&include_granted_scopes=true&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fspreadsheets.readonly+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&display=popup"}' -H "Content-Type: application/json" -X POST https://console.rivery.io/api/oauth2/credentials -H "Authorization: Bearer <access_token>"

Example response (200):

{

"application_id": null, 
"access_token": "<accessToken>", "status_code": 200, "expires_in": 3599, "token_type": "Bearer", "client_id": "<client Id>", "scope": "https://www.googleapis.com/auth/drive https://www.googleapis.com/auth/spreadsheets.readonly", "message": "Connected to Google", "refresh_token": "<Refresh Token>"

}

7. Make a Create New Connection request.

Replace the “credentials” key in the payload with the full response retrieved in step (6). In this step you can also set the name of the connection in the connection_name parameter as it will appear in the “Connections” tab in the Rivery UI.

868Ă—1132 92.5 KB

Example call:

curl -X PUT \
-d '{"connection_type": "gspreadsheet","connection_name": "google sheets Connection (OAuth2 Connection)","credentials": {
"application_id": null, "access_token": "<access token>...",
 "status_code": 200, 
"expires_in": 3599, 
"token_type": "Bearer", 
"client_id": "<client id>", 
"scope": "https://www.googleapis.com/auth/drive https://www.googleapis.com/auth/spreadsheets.readonly", 
"message": "Connected to Google", "refresh_token": "<refresh token>"}}' \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <access_token>" \
  https://console.rivery.io/api/connections

Example response (200):

{
"cross_id": "XXXXXXXXXXXXXXXX", 
"status_code": 200, "message": 
"connection saved", 
"_id": "XXXXXXXXXXXXXXXX", 
"connection_name": "google sheets Connection (OAuth2 Connection)"
}

If the response comes back as 200, the connection has been created in your Rivery account and you can find it under the “Connections” tab.

Use-Cases

This workflow helps enable several user cases that could be beneficial to your organization.

  1. Enabling a single Rivery Account to manage connections for multiple clients that are using the same sources.
  2. Creating a partition between the client and the Rivery platform to divide business operations, and data pipeline RND.
  3. Allowing for a Rivery account to serve as a service provider for ELT development and testing across a range of environments, connections, and users.
2 Likes