Author: Damien Jones

Categories
Internet Of Things & Robotics

Building A Raspberry Pi Zero Lean Green Machine

In this post I will set up my new Raspberry Pi Zero, wire up a moisture sensor and use Python to convert the sensor data into email alerts.

Table of Contents

Introduction

I recently wrote about setting up my Raspberry Pi 4. This isn’t the only Pi I have in the house though, as I put a Raspberry Pi Zero on my Christmas wishlist at about the same time and it turns out Santa was taking notes. Glad I got it before the current supply issues!

This post will flow differently from the last one as the Raspberry Pi Zero didn’t come as a kit. This time my board came in a plastic sleeve inside an unbranded white box. It came with GPIO pins attached (so I don’t have to learn how to solder properly yet!), but it didn’t come with a MicroSD card containing the Raspberry Pi OS. So let’s start there!

Installing Raspberry Pi OS 11 On A MicroSD Card

Raspberry Pis have no hard drives. Instead, they rely on memory cards for their storage. These memory cards can run a wide range of operating systems, the main limitations being the processor’s power and the operating system’s security.

For example, Windows 10 could be loaded onto a memory card but a Raspberry Pi Zero would have no chance of running it well. On the other hand, a Raspberry Pi might be able to run Windows 95 but it’d be a security nightmare.

Most use cases for a Raspberry Pi lend themselves to Raspberry Pi OS (previously called Raspbian) – a Debian-based operating system that is optimized for most Raspberry Pi hardware. Here, I’ll be installing it using the Raspberry Pi Imager.

Raspberry Pi Imager: Main Options

In my last post I mentioned that I’d need to take a look at the Raspberry Pi Imager. This software installs Raspberry Pi OS on MicroSD cards and replaces NOOBS. Versions are available for Windows, Mac and Ubuntu.

Firstly I need to choose an Operating System. There are a lot of choices here! There are different versions of the Raspberry Pi OS depending on whether I want Buster or Bullseye, and whether I want a desktop environment or not. Alternatively, I can choose to install a non-Raspberry Pi OS such as Ubuntu or Manjaro Linux.

Here the recommended setup is fine.

Next I need to choose my storage. I got a good deal on a multipack of Sandisk 64GB MicroSD cards, and will use one of those instead of overwriting the MicroSD card that came with my Labists Raspberry Pi 4 4GB Complete Starter Kit.

Raspberry Pi Imager: Advanced Options

After selecting the main options I can then access some advanced options to further customise my Raspberry Pi OS installation.

These Advanced Options are as follows:

Set Hostname

The hostname of a Raspberry PI enables it to be addressed by a name as well as an IP address. It is how a Raspberry Pi identifies itself to other systems on a local network. By default, the hostname is set to raspberrypi, but as I have more than one Raspberry Pi I will change this to avoid confusion.

Arise, mako!

Enable SSH

This option is essentially the same as the one in the Raspberry Pi Configuration settings. Enabling this now will save me a job later on.

Set Username And Password

This is a recent addition. The default username and password for any new Pi are pi and raspberry respectively, but a default password carries obvious security problems if unchanged.

Knowing a username isn’t as risky, but leaving the default in place makes life easier for a potential hacker so changing it is in my best interests. And no – it’s not mako.

Configure Wireless LAN

This is a technical way of saying ‘Set Up Wifi’.

Set Locale Settings

This selects the timezone and keyboard layout. Both were already set to GB, so this might have used my laptop’s settings. No changes needed here.

Writing To The MicroSD Card

Finally I started writing to the card. Seven minutes later I had a card which, upon insertion into my Raspberry Pi Zero and after a couple of minutes finding its feet, gave me a working installation of the latest version of Raspberry Pi OS 11 to play with!

Setting Up The Moisture Sensor

Time for my Raspberry Pi Zero to get its first job! This comes in the form of the ModMyPi Soil Moisture Sensor Kit.

The moisture sensor board features both analogue and digital outputs. The digital output gives a On or Off signal when the soil moisture content is above a certain value. The value can be set or calibrated using the adjustable onboard potentiometer.

Let’s get building!

Wiring Up The Sensor

The sensor kit comes with a jumper wire for connecting the sensor spade to the comparator board. It doesn’t come with any wires to connect the board to the Raspberry Pi though! Some hastily ordered jumper wires later and I was back in business.

As a first-timer potentially talking to other first-timers, I will say that the process of connecting the jumper cables to the various pins is an anxious experience.

All my senses were telling me that the pins looked delicate, so I used minimal pressure with the jumper wires and wondered why they kept coming off. It turns out these pins were designed to cope with some abuse after all, so being heavy-handed is encouraged. Just give the wires a good push!

It is also important to connect the wires to the correct GPIO pins on the Raspberry Pi. I used the Pi4J Project’s GPIO diagram to make sure the correct wires were connected to the correct pins.

Checking The Python Script

ModMyPi offer a Python script for this sensor on their Github repo. Let’s run through it and see what it does.

Importing Libraries

The script uses three libraries, all of which are included by default with Raspberry Pi OS 11:

  • RPi.GPIO: for controlling the GPIO pins on the Raspberry Pi.
  • smtplib: sends emails via SMTP.
  • time: for a sleep function that is part of the script.

Sending Emails

A sendEmail function uses the smtplib library and a set of variables to send emails when the function is called. The script prints "Successfully sent email" for successes and "Error: unable to send email" for failures.

Monitoring GPIO Pin

A callback function uses the RPi.GPIO library and a pair of variables to monitor the GPIO pin that the sensor is connected to. When a change is registered, one of two emails is sent via the sendEmail function depending on whether the sensor’s output is On or Off.

To keep the script running, the time library is used to add a slight delay to an infinite loop. This stops all of the CPU being used by the script, which would leave none for Raspberry Pi OS.

Testing The Sensor

To check that the sensor readings were coming through correctly, I needed a way to make the sensor wet or dry quickly. Necessity is the mother of invention, so I came up with this:

A sliced up milk bottle with a small hole in the lid and wires coming out of it. What a time to be alive.

My first attempts showed that the sensor was working (LED off and LED on) but the emails were failing:

Python 3.9.2 (/usr/bin/python3)
>>> %Run moisture.py
LED off
Error: unable to send email
LED on
Error: unable to send email

Troubleshooting: Statements Must Be Separated By Newlines Or Semicolons

An example of one of the uses of the print function in the script is:

print "LED off"

Visual Studio Code flags this as a problem, stating that Statements must be separated by newlines or semicolons.

This is down to differences between Python 2 and Python 3. Those differences are beyond the scope of my post, but the problem itself is easy to fix. As print is considered a function in Python 3, it requires parentheses to work correctly:

print ("LED off")

Troubleshooting: Blocked SMTP Port

The original script sets the smtp_port to 25. This wouldn’t be a problem if my Raspberry Pi Zero was sending the emails. However, here I’m using Google’s Gmail SMTP server to send emails instead.

TCP port 25 is frequently blocked by Internet Service Providers, including Google, as an anti-spam technique. ISPs prefer port 587 as it is more advanced and supports secure communication via Transport Layer Security (TLS).

TLS enables secure and trustworthy communication. This security requires some additional information to work properly though…

Troubleshooting: Missing SMTP Methods

This is the section of the sample script that handles sending emails:

smtpObj = smtplib.SMTP(smtp_host, smtp_port)
smtpObj.login(smtp_username, smtp_password) 
smtpObj.sendmail(smtp_sender, smtp_receivers, smtp_message)
  • The first line sends a request to smtp.gmail.com on TCP port 25.
  • The second line provides a user name and password for smtp.gmail.com.
  • Lastly, strings are given for the email sender, the recipients and what the email says.

In its current form this request will be rejected, as Gmail blocks TCP port 25. Initially, I just changed the port from 25 to 587 and ran the script again. This still didn’t work so I continued my research.

Having consulted Stack Overflow and Python’s smtplib documentation I realised that the script needed some additional methods. The sendEmail function needed two extra lines:

smtpObj = smtplib.SMTP(smtp_host, smtp_port)
smtpObj.ehlo()
smtpObj.starttls()
smtpObj.login(smtp_username, smtp_password)
smtpObj.sendmail(smtp_sender, smtp_receivers, smtp_message)

With the move to TCP port 587 and TLS, these new methods are needed to correctly introduce the script (and by extension the Raspberry Pi) to the Gmail SMTP server.

SMTP.ehlo opens communication between the script and Gmail. The script identifies itself via the Raspberry Pi’s fully qualified domain name, giving Gmail a way to identify the source of the request.

SMTP.starttls then asks Gmail if it supports TLS encryption. Gmail replies that it does, and all SMTP commands that follow are encrypted.

That’ll work now, right? Right?!

Troubleshooting: Insufficient Authentication

Even after these changes I was still getting problems. A Stack Abuse article suggested enabling the Less Secure App Access setting of my Gmail account. It turns out that Google and I have something in common – neither of us is keen on plain-text passwords flying around the Internet.

Google had to find some sort of middle ground and came up with this setting:

It is disabled by default and can’t be enabled on accounts with active MFA. Google are actively in the process of removing this setting, and will no longer support it from the end of May 2022. But for now this should be enough to get the emails flowing.

Retesting The Sensor

I ran the script again and got the feedback I was hoping for:

Python 3.9.2 (/usr/bin/python3)
>>> %Run moisture.py
LED off
Successfully sent email
LED on
Successfully sent email

And a string of emails in my inbox:

Success!

Next Steps

Although this approach does work, it isn’t ideal for several reasons and will stop working completely when Google pull the plug on the Less Secure App Access setting. There are a number of changes I want to make.

Use AWS For Emails Instead Of Python

Sending emails via Python caused the bulk of the problems here. The approach this script uses is not secure and will soon be unsupported by the third party it relies on.

I could set up the Raspberry Pi in such a way that it could send emails itself, but ideally I want the Raspberry Pi to be doing as little work as possible with as few credentials as possible.

Enter AWS. I’ve already used SNS a number of times for emails, and the various AWS IoT services offer several options for communication with my device. This would let me decouple the email functionality from the sensor functionality.

In addition, AWS can handle the security side of things. Instead of my Raspberry Pi having root credentials for a Google account, it can have an AWS IoT certificate that will only allow specific actions.

Disable Google Less Secure App Access Setting

If AWS are handling the emails then I don’t need to use the smtplib library anymore. Which means I don’t need to use the Gmail SMTP. Which means I can disable the Less Secure App Access setting!

Google is sending me security warnings about this on almost a weekly basis at the moment. They want this OFF. So I want this off too.

Control Email Frequency

As the earlier screenshot showed, I got hammered with emails by the script. I would prefer to get emails based on a series of events or periods of time instead of a blow-by-blow account. CloudWatch is an obvious candidate here, and AWS IoT Events may also be a contender.

Monitor Device Health

Finally, there’s the question of device health. Any number of problems could occur with the current setup. The sensor could stop working (soil is acidic after all). There could be a power cut. Wolfie could knock everything over. In all of these situations, nothing happens. The readings just stop.

With AWS I can monitor the flow of data. I can look for periods with no data, and set alarms based on those periods. I might have to adjust how the Python script reads the outputs from the sensor, but at least I’ll know that the sensor is working and that my plant is lush and green instead of brown and crisp.

Summary

In this post I have set up my new Raspberry Pi Zero, wired up a moisture sensor and used Python to convert the sensor data into email alerts. I have modernised the Python script and removed syntax errors, and have identified areas where I can improve the security, reliability and operational excellence of the overall solution.

If this post has been useful, please feel free to follow me on the following platforms for future updates:

Thanks for reading ~~^~~

Categories
Internet Of Things & Robotics

Getting Started With My Raspberry Pi 4 And AWS IoT

In this post I unbox and configure my new Raspberry Pi 4, and then register it with my AWS account as an AWS IoT device.

Table of Contents

Introduction

After earning my AWS Certified Developer – Associate certification last month, my attention turned to the Raspberry Pi my partner got me as a birthday present. I’ve had it for a while and done nothing with it because of a lack of time and ideas. I promised myself that I’d open it up after finishing my exam, so let’s go!

What’s In The Box?

My birthday gift came in the form of the Labists Raspberry Pi 4 4GB Complete Starter Kit. Having seen the price, I must have been good that year!

The set includes:

  • Raspberry Pi 4 Model B 4GB RAM with 1.5GHz 64-bit Quad-core CPU
  • 32GB Class 10 MicroSD Card Preloaded with NOOBS
  • Premium Black Case (High Gloss) for Pi 4B
  • Mini Silent Fan
  • Two Micro HDMI to HDMI Cables

Labists have a great video for assembling the Raspberry Pi. Fiddling with exposed circuitry is anxiety-inducing for a heavy-handed data professional like myself, so the video was very welcome!

The steps basically boil down to:

  • Attach Heat Sinks To Pi
  • Screw Fan To Case
  • Screw Pi To Case
  • Connect Fan Pins To Pi
  • Close Case

My Raspberry Pi is now out of the box and fully assembled, so let’s get some advice on how it works.

Getting To Know My Pi With FutureLearn

FutureLearn is a global learning platform with a mission to transform access to education by offering online courses from the world’s leading universities and brands. They offer a range of all-online, on-demand courses and offer free and paid content.

The Educators

The Getting Started with Your Raspberry Pi course is one of a number of free courses by the Raspberry Pi Foundation. The Foundation is a UK charity seeking to increase the availability of computing and digital making skills by providing low-cost, high-performance single-board computers, highly available training and free software.

The Course

The course is split into three weeks, although the lessons can be completed at the pace of the user. The first week of the course “Setting Up Your Raspberry Pi” introduces the facilitation team, walks through the hardware and software and gives a basic introduction to Raspberry Pi OS.

Week Two “Using Your Raspberry Pi” offers insight into what the Raspberry Pi can do. This includes the compute resources, the ability to connect peripherals and the built-in software such as the visual programming language Scratch and the introductory Python editor Thonny.

Finally, Week Three “Taking More Control Of Your Raspberry Pi” goes full SysAdmin and introduces security measures, the command line and remote access. Instructions are given on how to control the Pi via VNC Viewer and SSH, and commands like mkdir, cp and mv are covered.

Most significantly, the APT Package Manager is introduced along with commands including:

  • sudo apt update
  • apt list --upgradable
  • sudo apt autoclean.

A beginners course that introduces the ideas of keeping devices updated, tidy and secure is a welcome sight as it encourages good user behaviour early on and ultimately prolongs the life of the Raspberry Pi.

My Raspberry Pi is now accessible, updated and ready to take on jobs, so let’s give it something to do!

Connecting My Pi To AWS

AWS offer several IoT services that are summarised as Device Software, Control Services and Analytics. To simplify the process of connecting a new IoT device, AWS has added a wizard to the Build A Solution widget on the newest version of the AWS Management Console:

This loads the AWS IoT wizard used by AWS IoT Core, consisting of a three-step process:

A word of advice – check the region the wizard is running in! I mainly use eu-west-1 but the IoT wizard changed this to us-west-2 and would have created my resources in the wrong place!

Before starting, AWS need to know which operating system my IoT device uses and which SDK I want to use. I tell AWS that my Raspberry Pi is running Linux and that I intend to use the Python SDK, and in response AWS offers some advice before starting the wizard:

Some prerequisites to consider: the device should have Python and Git installed and a TCP connection to the public internet on port 8883.

This has already been taken care of so let’s continue.

AWS IoT Configuration

Step 1 involves creating an IoT Thing with a matching Thing Record. A Thing Record is how AWS represents and records a physical device in the cloud, and is used for recording properties of the IoT Thing including certificates, jobs and the ARN.

I name my Raspberry Pi dj-raspberrypi4-labists. AWS then attach a Device Shadow to the Thing Record. These make a device’s state available to apps and other services. whether the device is connected to AWS IoT or not. For example, my Pi’s state could be Online or Offline.

In Step 2 AWS confirm that a new thing was created. A new AWS IoT Core policy is also created to enable sending and receiving messages. AWS IoT Core policies are basically IAM for AWS IoT devices. They control access to operations including:

AWS also supply a downloadable connection kit. This contains certificates and keys for authentication and an SSH script for device configuration and message processing. This is provided as a ZIP archive, which I put on my Raspberry Pi in a new folder specifically for AWS objects.

Device Configuration

Finally, the wizard gives a list of commands to send to the IoT device to test the AWS connection. The first command unzips the connection kit:

unzip connect_device_package.zip

The second command adds execution permissions to the start.sh script in the connection kit:

chmod +x start.sh

I’m never keen on running unfamiliar code off the Internet without knowing what it does first, so I did some searching – it turns out that chmod +x makes a file executable.

Now start.sh is runnable, it can be executed using the command ./start.sh. This is a short script that performs the following actions:

The result is an infinite stream of Hello Worlds:

Finally, AWS give a summary of the steps completed:

Cost Analysis

AWS IoT Core hasn’t cost me any money so far. This might be because I’m only running test loads on it currently, but looking at the new lines on my bill it’s going to be a while before I start making AWS any money here:

Next Steps

Having set up my Raspberry Pi, I have found some upgrades that I need to take care of:

Operating System Upgrade

Firstly, my Raspberry Pi’s operating system has an update available. It is currently running Rasbian 10, known as Buster:

In November 2021 Raspberry Pi released Bullseye. This is a major upgrade so the recommended process is to download a new image, reinstall any applications, and move data across from the current image. This makes sense to do while there isn’t much data on my Pi.

This leads me on to…

Raspberry Pi Imager

A common task with a Raspberry Pi is installing an operating system onto an SD card. In 2013 Raspberry Pi released NOOBS, or New Out Of the Box Software to give it its full name. Someone at Raspberry Pi HQ clearly has a sense of humour.

NOOBS was designed to simplify the process of setting up a new Pi for first time users, and the Labists kit included an SD card with NOOBS preinstalled. However Raspberry Pi no longer support it, and now recommend the Raspberry Pi Imager for installing Raspberry Pi OS instead.

So plenty to be getting on with!

Summary

In this post I’ve unboxed and configured my Raspberry Pi and linked it to my AWS account as an IoT Thing. I’ve described the basic concepts of AWS IoT Core and have identified some important upgrades that my Pi needs before I consider using it for anything serious.

If this post has been useful, please feel free to follow me on the following platforms for future updates:

Thanks for reading ~~^~~

Categories
Developing & Application Integration

Earning My AWS Developer Associate Cert By Fighting A Cow

In this post I talk about my recent experience with the AWS Certified Developer – Associate certification, discuss why and how I studied for the exam and explain why part of the process was like an early 90s puzzle game.

Table of Contents

Introduction

On 25 March 2022 I earned the AWS Certified Developer – Associate certification. This is my fourth AWS certification and I now hold all the associate AWS certifications. People wanting to know more are welcome to view my Credly badges.

Motivation For Earning The AWS Developer Associate

Firstly I’ll explain why I took the exam. I like to use certifications as evidence of my current knowledge and skillset, and as mechanisms to introduce me to new topics that I wouldn’t otherwise have interacted with.

There’s a gap of around 18 months between my last AWS certification and this one. There are a few reasons for that:

  • I wanted to give the knowledge from the Solutions Architect and SysOps Administrator certifications time to bed in.
  • I wanted to use my new skills for the AWS data migration project at work.
  • My role at the time didn’t involve many of the services covered in the Developer Associate exam.

After the AWS migration was completed and I became a Data Engineer, I felt that the time was right for the Developer Associate. My new role brought with it new responsibilities, and the AWS migration made new tooling available to the business. I incorporated the Developer Associate into the upskilling for my new role over a four month period.

The benefits of the various sections and modules of the Developer Associate can be split across:

  • Projects the Data Engineering team is currently working on.
  • Future projects the Data Engineering team is likely to receive.
  • Projects I can undertake in my own time to augment my skillset.

Current Work Projects

  • Our ETLs are built using Python on AWS Lambda. The various components of Lambda were a big part of the exam and helped me out when writing new ETLs and modernising legacy components.
  • Git repos are a big part of the Data Engineering workstream. I am a relative newcomer to Git, and the sections on CodeCommit helped me better understand the fundamentals.
  • Build tests and deployments are managed by the Data Engineering CICD pipelines. The CodeBuild, CodeDeploy and CodePipeline sections have shown me what these pipelines are capable of and how they function.
  • Some Data Engineering pipelines use Docker. The ECS and Fargate sections helped me understand containers conceptually and the benefits they offer.

Future Work Projects

  • Sections about CloudWatch and SNS will be useful for setting up new monitoring and alerting as the Data Engineering team’s use of AWS services increases.
  • The DynamoDB module will be helpful when new data sources are introduced that either don’t need a relational database or are prone to schema changes.
  • Sections about Kinesis will help me design streams for real-time data processing and analytics.

Future Personal Projects

  • The CloudFormation and SAM modules will help me build and deploy applications in my AWS account for developing my Python knowledge.
  • Sections on Cognito will help me secure these applications against unauthorized and malicious activity.
  • The API Gateway module will let me define how my applications can be interacted with and how incoming requests should be handled.
  • Sections on KMS will help me secure my data and resources when releasing homemade applications.

Resources For The AWS Developer Associate

Because AWS certifications are very popular, there are many resources to choose from. I used the following resources for my AWS Developer Associate preparation.

Stéphane Maarek Udemy Course

I’ve been a fan of Stéphane Maarek for some time, having used his courses for all of my AWS associate exams. His Ultimate AWS Certified Developer Associate is exceptional, with 32 hours of well presented and informative videos covering all exam topics. In addition, his code and slides are also included.

Stéphane is big on passing on real-world skills as opposed to just teaching enough to pass exams, and his dedication to keeping his content updated is clearly visible in the course.

À votre santé Stéphane!

Tutorials Dojo Learning Portal

Tutorials Dojo, headed by Jon Bonso, is a site with plentiful resources for AWS, Microsoft Azure and Google Cloud. Their practice exams are known for being hard but fair and are comparable to the AWS exams. All questions include detailed explanations of both the correct and incorrect answers. These practice exams were an essential part of my preparation.

Their Certified Developer Associate practise exam package offers a number of learning choices:

  • Want to mimic the exam? Timed Mode poses 65 questions against the clock.
  • Prefer immediate feedback? Review Mode shows answers and explanations after every question.
  • Practising a weak area? Section-Based Mode limits questions to specific topics.

Tutorials Dojo also offers a variety of Cheat Sheets and Study Guides. These are free, comprehensive and regularly updated.

AWS Documentation & FAQs

AWS documentation is the origin of most questions in the exam and Stéphane and Jon both reference it in their content. I refer to it in situations where a topic isn’t making sense, or if a topic is a regular stumbling block in the practice exams.

For example, I didn’t understand API Gateway integration types until I read the API Gateway Developer Guide page. I am a visual learner, but sometimes there’s no substitute for reading the instruction manual! The KMS FAQs cleared up a few problem areas for me as well.

AWS also have their own learning services, including the AWS Skill Builder. While I didn’t use it here, some of my AWS certifications will expire in 2023 so I’ll definitely be looking to test out Skill Builder then.

Anki

Anki is a free and open-source flashcard program. It has a great user guide that includes an explanation of how it aids learning. I find Anki works best for short pieces of information that I want regular exposure to via their mobile app.

For example, one of my Anki cards was:

CodeCommit: Migrate Git = CLONE Git; PUSH Git
PULL = NULL

This was explaining the process of migrating a Git repo to CodeCommit. PULL = NULL was a way for me to remember that pulling objects from the Git repo was incorrect in that scenario.

If an Anki card goes over two lines I use pen and paper for it instead. Previous experience has taught me that I can visualise small notes better on Anki and large notes better on paper.

Blogging

My best exam performance is with the AWS services I am most familiar with. Towards the end of my exam preparation, I wanted to fill some knowledge gaps by getting my hands dirty!

My posts about creating security alerts and enhanced S3 notifications let me get to grips with CloudTrail, CloudWatch, EventBridge and SNS. These all made an appearance in my exam so this was time well spent!

I also ran through an AWS guide about Building A Serverless Web Application to get some quick experience using API Gateway, CodeCommit and Cognito. This has given me some ideas for future blog projects, so stay tuned!

Approach To Studying The AWS Developer Associate

This section goes into detail about how I approached my studies. I didn’t realise it at the time but, on review, the whole process is basically a long ETL. With sword fighting.

Extract

I started by watching Stéphane’s course in its entirety, ‘extracting’ notes as I went. Since Stéphane provided his slides and since I already knew some topics from previous experience, the notes were mostly on topics that I either didn’t know or was out of practice with.

Transform

Having finished Stéphane’s course, I started the Tutorials Dojo practice exams. The aim here is to ‘transform’ my knowledge from notes and slides to answers to exam questions.

I have a spreadsheet template in Google Sheets for this process:

As I work through a practice exam, I record how I feel about my answers:

I can choose from:

  • Confident: I’m totally confident with my answer
  • 5050: I’m torn between two answers but have eliminated some
  • Guess: I have no idea what the answer is

When I get the results of the practice exam, I add the outcomes:

The Gut Feel and Outcome columns then populate tables elsewhere on the spreadsheet:

I use these tables for planning my next moves:

  • The top table quantifies overall confidence, and can answer questions like “Is my confidence improving between practise exams?”, “How often am I having to guess answers?” and “How confident am I about taking the real exam?”
  • I can get the middle table from Tutorials Dojo, but have it on the sheet for convenience.
  • The bottom table shows me an analysis of Gut Feel and Outcome. This shows me how many of my correct answers were down to knowledge, and in addition how many were down to luck.

I then update the Question column of the spreadsheet depending on the results in the bottom table:

  • I assume that anything listed as Confident and Correct is well known. Nothing is changed.
  • All 5050s and Correct Guesses are coloured orange. Here some knowledge is apparent, but more revision is needed.
  • All Incorrect Guesses are coloured red, because there are clear knowledge gaps here.
  • Anything listed as Confident and Incorrect is also coloured red. These are the biggest red flags of all, as here knowledge has either been misread or misunderstood.

Load

As the knowledge gaps and development areas become clear, I began to ‘load’ the topics that still didn’t make sense or were proving hard to remember.

Based on the Tutorials Dojo practise exam outcomes, I made a second set of notes that were more concise than the first. So where the first set was mostly “Things I Don’t Know” the second set was mostly “Things I Can’t Remember”.

As you might imagine, this uses a fair amount of paper. I recycle this afterwards because I’m an environmentally-conscious shark.

Insult Sword Fighting

I’ve come to know part of the ‘load’ as Insult Sword Fighting. Some people will know exactly what I’m talking about here, while others will quite rightly need some explanation.

Insult Sword Fighting is part of the 1990 point and click adventure game The Secret of Monkey Island. In this section of the game, the player wins fights by knowing the correct responses to an opponent’s insults.

For example, during a fight the opponent might say:

“You fight like a dairy farmer.”

To which the player’s response should be:

“How appropriate. You fight like a cow!”

The player starts out with two insult-response pairs, and learns more during subsequent fights.

The aim of the section is to learn enough to defeat the Sword Master. However, her insults are different to the ones the player has previously seen. For the final challenge, the player must match their existing knowledge to the new insults.

So if the Sword Master says:

“I will milk every drop of blood from your body!”

The player should pick up on the word “milk” and respond with:

“How appropriate. You fight like a cow!”

OK But What Does This Have To Do With The Exam?

So let me explain. The first time with a practice exam is like the player’s first Insult Sword Fight. Most responses are unknown or unfamiliar, so things usually don’t go well.

The player gets better at Insult Sword Fighting by challenging new opponents. This time the player will know some responses, but will also encounter new insults to learn.

In the same way, the subsequent practice exams will pose some questions that are similar to those in the previous exam. Of course there will also be entirely new questions that need further investigation.

The player will decide they are ready to face the Sword Master when they are able to win the majority of their Insult Sword Fights because they know the logic behind the correct responses.

Like the insults, the logic behind the practice exam questions can also be learned. Knowing the logic well enough to regularly answer these questions correctly is a good indicator that the real exam is a good idea.

The Sword Master’s insults are different to the ones the player has trained with. To win, the player must look for key words and phrases in the new insults and match them to their existing responses during battle.

The real exam will use unfamiliar questions. However the key words and phrases in the questions will match the knowledge built up during the practice exams, revealing the logic to arrive at the correct answers!

For those wondering how I made these images, I direct you to this awesome tool.

Next Steps

Now that the Developer Associate exam is over, I have a number of ideas for blog posts and projects to try out:

  • Building an ETL for my own data
  • Creating an API to query that data
  • Deploying the solution using Git and CICD

Plus I have a bookmarks folder and Trello board full of ideas to consider. So plenty to keep me busy!

If this post has been useful, please feel free to follow me on the following platforms for future updates:

Thanks for reading ~~^~~