Categories
Me

Introducing amazonwebshark.com

It gives me great pleasure to introduce amazonwebshark.com! A project that I started in October 2021 is now seeing the light of day and I thought I’d take the time to go through some of the decisions behind it and what my ambitions for the site are.

What Is amazonwebshark For?

At time of writing I’ve been a Data Engineer for a few months and there are several elements of the role that I’d like to get more familiar with – for example my Python skills need sharpening and I’ve not used Git properly yet. By writing about my experiences I can check and confirm my understanding of new topics, give myself points of reference for future projects and exam revision, evidence my development where necessary and help myself out in the moments when my imposter syndrome sees an opportunity to strike.

How Did You Set amazonwebshark Up?

The domain was purchased using Amazon Route 53. This keeps my setup and overall billing somewhat simplified and also means I can try out Route 53’s integrations with other AWS services.

My hosting is via Bluehost. For around £2.66 a month they’re sorting out my server, database, CDN and SSL – that price was a Black Friday special and goes up after a year but so far I’ve been very impressed with their communication and customer service so I’ll see how it goes.

Why Didn’t You Use AWS For Hosting?

Besides the bargain price, Bluehost brings with it a level of convenience. I could have used my own EC2/RDS setup. Or Lightsail. Or even a static S3 site. But my main focus was to get the ball rolling and get something online. I’ve wanted to start a blog for some time, but have run into problems like knowledge gaps, time pressures and running out of enthusiasm. Bluehost offered a quick and simple process that sorted out everything I needed, letting me get the actual blog started.

This is not to say I’m unwilling to roll my own in the future, of course. Stay tuned.

Why Start A Blog At All When LinkedIn And Medium Are Around?

I’m not keen on the Medium model. Many times I’ve seen an article that I fancy reading, only to be met with forced login requests or paywalls. Both of which are deeply frustrating. And LinkedIn is great for articles, but this way I have all my content ringfenced together and under my control. Social media platforms change constantly and there is no guarantee that features, formats and content availability will stay the same or continue to exist from month to month. This site will hopefully let me sidestep any unpleasant surprises of that nature.

What’s The Plan For The Next Few Months?

I’m currently studying towards the AWS Certified Developer – Associate certification, and want to try using my work with the Strava API as the basis for a Lambda function and a CI/CD pipeline to get experience with some AWS services I’ve yet to use. Plus I have a Trello board full of ideas and an unused Rasberry Pi, so plenty to get on with!

Why amazonwebshark?

The name for the site comes from the original Amazon Web Shark – Terabyte:

amazonwebshark Terabyte

He hails from Lanzarote and his favourite topics are FinTech, Dy-Nom-oDB and S3 (Shark Storage Service).

Thanks for reading ~~^~~

Categories
Developing & Application Integration

Authenticating Strava API Calls Using OAuth 2.0 And Visual Studio Code

Picking up from last time, I continued on with the Strava API Getting Started page and got myself very confused over how things were supposed to work. The Strava API requires authentication via OAuth 2.0, and in fairness Strava includes a graph that shows how the process works which unfortunately went right over my head. By chance I found a YouTube video produced by InterSystems Learning Services that gave me a good entry-level understanding of the process, boiling down to this slide:

oauth 2.0 workflow

As it turned out I was misunderstanding the purpose of the access token Strava was giving me on the My API Application page. That access token wasn’t provided by OAuth and so couldn’t be used to get any data – instead I needed to use Strava’s OAuth API to request different credentials and define a scope for the access needed.

To that end, Strava provides this link on their Getting Started page:

http://www.strava.com/oauth/authorize?client_id=[REPLACE_WITH_YOUR_CLIENT_ID]&response_type=code&redirect_uri=http://localhost/exchange_token&approval_prompt=force&scope=read

Which was used in conjunction with my app’s ClientID to get an authorization code in the form of:

http://localhost/exchange_token?state=&code=MYAUTHCODE123456789&scope=read

Next, a cURL request must be made to exchange the authorization code and scope for a refresh token, access token, and access token expiration date. The page suggests https://www.strava.com/oauth/token which turns out to be wrong – the URL used in the Authentication documentation for this is https://www.strava.com/api/v3/oauth/token. At this point I was having varying success with Postman, so instead installed REST Client on Visual Studio Code on the advice of Vu Long Tran’s blog which made things much simpler.  

Sending a request of:

curl -X POST https://www.strava.com/api/v3/oauth/token 

  -d client_id=ReplaceWithClientID \

  -d client_secret=ReplaceWithClientSecret \

  -d code=ReplaceWithCode \

  -d grant_type=authorization_code\

Produced a response containing my Strava profile data along with new tokens and details of their expiry:

"token_type": "Bearer"

"expires_at": 1640810729,

"expires_in": 4002,

"refresh_token": REFRESHTOKEN123456789

"access_token": "ACCESSTOKEN123456789"

Awesome! Now to put these to work…

Part of my confusion coming into this was that I had been given a Python script in my travels that I was expecting to work with the access token from the My API Application screen, which of course with the benefit of hindsight was never going to be successful. The Python script is as follows:

import requests

activities_url = "https://www.strava.com/api/v3/athlete/activities"

header = {'Authorization': 'Bearer ' + "access_token"}
param = {'per_page': 200, 'page': 1}

my_dataset = requests.get(activities_url, headers=header, params=param).json()

print(my_dataset)

Time to test it out with the new access token! Straight into a problem:

{'message': 'Authorization Error', 'errors': [{'resource': 'Athlete', 'field': 'access_token', 'code': 'invalid'}]}

The Strava API and SDK Reference page was quick to indicate the source of the problem:

strava api get activity

The previous requests to Strava’s OAuth API were with scope=read. I changed this to scope=activity:read_all and received a new authorization code with the updated scope. This authorization code was then exchanged for a new access token which, when pasted into the Python script, outputted so much JSON that in the interests of sanity I’ve screenshotted a small section for illustration:

python json strava api response

To be fair it was what I asked for. And it all makes a bit more sense to me now!

Future plans for this involve making use of the refresh tokens to automate access token generation (assuming I’ve understood that part correctly – stay tuned!) and getting some working code into a Lambda function so I can start turning some cogs in AWS.

Thanks for reading ~~^~~

Categories
Developing & Application Integration

Sending GET Requests To The Strava API With Postman: Getting Started

Today I went out for my first run post-Christmas. It was about as much fun as you’d expect but that’s not the point.

strava run summary

When I finish a run, my watch uploads the data to Garmin Connect, which in turn syncs the data with FetchEveryone (for their great analytics) and Strava (for their API). The Strava API is generally more accessible than Garmin’s and I already use it for integrations with Google Calendar, so it dawned on me that my Strava account has years’ worth of data that I can tap into for various personal projects. Then I had to change course to avoid three horses coming down the centre of the trail.

To get the ball rolling post-run I logged onto my Strava account and accessed my profile, only to find no mention of the API anywhere. Some Googling established that the URL needed updating from https://www.strava.com/settings/profile to https://www.strava.com/settings/api to show the My API Application options, which helpfully includes a link to the Strava Developer documentation:

strava api create app

I completed the fields using the recommendations in the documentation and in response Strava provided a set of API credentials:

strava api application credentials

The next step was to make a cURL request against the Strava API for my profile data, for which the Strava docs suggested Postman – a platform for building and using APIs. I made an account there, created a GET request for https://www.strava.com/api/v3/athlete and set an Authorization key-value pair using my Strava Access Token:

postman console

I then received the below response in JSON:

    "id": 18701823,

    "username": null,

    "resource_state": 2,

    "firstname": "Damien",

    "lastname": "Jones",

    "bio": "",

    "city": "[REDACTED]",

    "state": "England",

    "country": "United Kingdom",

    "sex": "M",

    "premium": false,

    "summit": false,

    "created_at": "2016-12-02T00:37:23Z",

    "updated_at": "2021-12-27T00:05:26Z",

    "badge_type_id": 0,

    "weight": 0.0,

    "profile_medium": "[REDACTED]",

    "profile": "[REDACTED]",

    "friend": null,

    "follower": null

}

Success! 

That’s as far as I’m going to take this today but in the coming days I want to take this forward and try out some of the Python examples in the Strava docs.

Thanks for reading! ~~^~~