Category: Me

Categories
Me

Video Thrilled The Dataflow Shark

VideoThrilled CoverWithText800600

In this post, I debut both the amazonwebshark YouTube channel and my first demonstration video and shark shorts.

Table of Contents

Introduction

So, this is amazonwebshark’s fiftieth post! It’s also my first post as a consultant! I joined the Steamhaus team this month and am looking forward to what the future brings!

In my last YearCompass review, I committed to building my personal brand. So far 2024 has seen this shark presenting at AWS Summit London and AWS User Group Liverpool, and making video content seems like the logical next step.

Before this, the amazonwebshark YouTube channel held playlists for videos I’ve referenced in previous posts. And as of July 2024 I’ve joined the video content creator ranks with some humble contributions of my very own!

So what’s this post about? Firstly, I’ll examine my motivation for making video content. Then I’ll link my current uploads, and finally I’ll set initial expectations for the channel.

Why?

So why am I doing this? This section explains my motivation for recording videos and what I hope to achieve.

Practice & Improve Speaking

Generally, doing something more often reveals improvements, efficiencies and optimisations. And I want to improve my speaking. So I need to do more of it!

I got some great advice from Laurie Kirk a while ago on this topic. She has a habit of filming herself daily and reviewing the footage for improvements. By her own admission, this has improved her confidence and quality.

Besides this, speaking practice draws parallels with training runs. Becoming an optimal runner involves various types of training. Want to build endurance? Run slower over distance. Want to improve speed? Focus on faster, shorter bursts.

And it’s the same with speaking. Want to practise lightning talks? Make short videos. Want to improve sessions? Film a demo. My work so far made me more confident at AWS UG Liverpool earlier this month, so I hope to see more improvements in the coming months by practising different types of speaking.

Audience Diversification

In addition to improving my current abilities, I want to upskill my ability to communicate with people outside my field of expertise.

It’s well known that technical people love speaking with technical people. Getting into the weeds about operating systems, functions, architectures and paradigms regularly see hours fly by at meetups.

This is, however, inherently limiting to those less knowledgeable in those areas. I was originally going to label this as ‘non-tech’, but on reflection this extends in all directions. For example, I won’t understand an architectural discussion that turns to von Neumann architectures. I then have equal potential to confuse if I start talking about Lakehouse architectures. This can even happen with people sharing a speciality: I had no idea what instantiate meant when I first heard it from another Data Engineer.

Amongst the best sessions and videos I’ve seen are ones where topics are made accessible and inclusive to a diverse audience with ranging skills and experience. Some viewers will have years of experience in the field and are looking for the latest insights. Others will be hearing about the topic for the very first time. Appealing to both ends of the spectrum is the ideal scenario.

This is the skill level I’m aiming towards. Creating sessions and videos that appeal to a diverse range of viewers will make me a more inclusive and effective communicator. And it’s not just about audience diversification…

Content Diversification

Next, producing videos will let me make different kinds of content.

Applying the Diátaxis framework, my blog posts lean more towards Tutorial than the others. This is intentional, as I’ve always preferred practice over theory and like sharing cool stuff that enables people.

That’s not to say I don’t get curious about the other Diátaxis ‘needs’ of How-To Guide, Explanation and Reference. While past exploration of these with text hasn’t worked out, videos offer new opportunities here such as:

  • Trying out new AWS services and features.
  • Running through concepts and architectures.
  • Exploring unfamiliar and less common settings and parameters.

In short, content ideas that lend themselves better to video than text. And speaking of ideas…

Failing Fast

Videos may save a post idea that has promise but isn’t working out.

I am never stuck for blog post ideas. There’s always something to write about – from services and architectures to current events. I probably have more potential post topics than I can ever write.

This isn’t to say that every post I start is completed though. Some ideas begin well but start to unravel. They might meander, lack cohesion or simply become uninteresting. And while I’m getting better at seeing the early signs of this, occasionally some still slip through.

I’m a big believer in avoiding the sunken cost fallacy. So in those situations, I admit defeat and defer to the Cult of Done Manifesto’s fifth principle:

“If you wait more than a week to get an idea done, abandon it.”

The Cult of Done Manifesto – Bre Pettis

There are several interpretations of this principle. NoBoilerplate‘s advice is that:

“Ideas in your brain are like a pipe full of random stuff. Some of it will be good; some not so good. If you’re not feeling it, don’t try to make a bad idea better – try the next idea.”

The Cult of Done: How To Get *Started* – No Boilerplate

I agree. But it’s still disheartening sometimes to delete something that still feels like it has legs – just not for the body you’re trying to stitch them to. More recently, I found Jason Fladlien‘s interpretation which has a different take:

“The longer you go not getting something done the more baggage you create around getting it done. “Abandoning” an idea simply means throwing this version of it in the trash. You can start it fresh later.”

The Cult of Done – The Drive-Contentment Connection

This applies very well to situations where an idea loses traction as a blog post but is still worth pursuing. Instead of deleting everything, post material could be repurposed into video material.

Additionally, I’m likely to have session abstracts that either don’t work out or aren’t accepted. If I consider the idea to be sound, videos are ideal solutions to these situations too.

Chasing Internet Stardom

Yeah ok not really.

Current Uploads

So what have I produced so far on the video and shark short front?

Firstly, I’ve filmed a pair of data-themed YouTube shorts. The first examines one of the functions of an AWS Glue Crawler:

The other considers one of the differences between Parquet files and CSV files:

Future shorts will be uploaded to YouTube, Instagram and TikTok. I’ll see how this goes over the coming months.

I’ve also uploaded an extended demo for my Building And Automating Serverless Auto-Scaling Data Pipelines In AWS session:

The demo I use in this session begins with some existing AWS resources. This keeps me within the session’s time limit, but at the cost of an incomplete picture of what the Step Function workflow is doing. This extended version starts with a blank workflow and shows the Glue and Athena setup behind the scenes.

I’m not holding these up as works of art! They are rough around the edges, and I’m sure I’ll improve over time. In the meantime, a different Cult of Done Manifesto principle applies:

“Pretending you know what you’re doing is almost the same as knowing what you are doing, so just accept that you know what you’re doing even if you don’t and do it.”

The Cult of Done Manifesto – Bre Pettis

Expectations

So what are my expectations for the video and shark shorts?

This is all very much early days. There are no grand plans or ambitions, and I don’t have an upload schedule planned. I already have lots going on personally and professionally and don’t want to burn myself out. Much like this blog, it’s something for me to experiment and upskill with.

That said, I recently bought some streaming gear and a posh microphone in the Prime Day sales. So let’s see where this goes!

Summary

In this post, I debuted both the amazonwebshark YouTube channel and my first demonstration video and shark shorts.

As I said, there’s no grand vision for any of this and I’m totally winging it. It’s a bit of fun and I’m interested to see where it goes. In the meantime, the button below has links for contact, socials, projects and sessions:

SharkLinkButton 1

Thanks for reading ~~^~~

Categories
Me

Attitudes Towards Certification

SharkHoodyStudy800600

In this post, I examine my attitudes toward certification and how I use my certifications after earning them.

Table of Contents

Introduction

In October 2023, I renewed my AWS SysOps Administrator Associate certification. I was going to write about how I did it, but I used similar resources to those in my Developer Associate post from March 2022.

So I’m writing a different post instead. Many people examine how they earn cloud certifications, but few explore the reasons why. Here, I’ll share my attitudes and motivations toward earning and renewing a cloud certification, and how I apply my new knowledge after the fact.

Self Validation

For me, the biggest certification benefit is the confidence it gives me that I know what I’m talking about.

I’ve mentioned previously that I have anxiety and imposter syndrome issues. One way I approach that is to keep my technical knowledge current and relevant. This goes beyond the cloud. I have DataCamp and A Cloud Guru subscriptions for honing other areas, and various newsletters and feeds for news and opinions.

Certifications let me distil my knowledge into the recognised and understood forms of certification badges. These badges in turn give me the piece of mind that I can validate and evidence my cloud knowledge.

This doesn’t just benefit me…

Knowledge Validation

Having active cloud certifications gives my employer confidence that my cloud knowledge is accurate and reliable.

My employer uses many AWS services across various teams. My AWS certifications have given me T-shaped knowledge that helps me contribute to the success of both Data Engineering and the other tech teams. This knowledge lets me:

Troubleshoot Problems

Lambda function throwing unexpected exceptions? Slow running Athena query? Unreachable EC2 instance? I’ve solved all these problems using certification knowledge applied to real-world use.

For the record, the Lambda’s IAM role was missing actions, the S3 objects needed different partitioning and the instance was missing a security group.

Collaborate Across Teams

Whether preparing for failovers, replacing legacy packages with cloud processes or building APIs, I can work with my non-data colleagues by understanding the AWS services they use and what their outputs tell me.

For example, I don’t directly use AWS services like Systems Manager Patch Manager and API Gateway in my role. But I understand what those services are, what their outputs mean and how they relate to the services I do directly use.

Architect Agile Solutions

When faced with a new requirement or unexpected bug, I can call on my certification knowledge for optimal and appropriate solutions. Should those solutions turn out to be ineffective or undesirable, I can pivot or re-architect accordingly.

For example, when recently repartitioning an S3 bucket I approached the task using Python and boto3. During testing, it became clear this approach would take days to complete.

So I used my AWS CLI SysOps knowledge and refactored the Python code to script out the S3 CLI operations for each object. Then I completed the entire repartition in about two hours using Bash.

Same task. Same result. Different solutions.

Wider View

Studying for and passing certifications exposes me to cloud services I’ve never used, or don’t use often.

AWS constantly changes. Most weeks see new services, features or improvements. Keeping abreast of over two hundred services is difficult, and useful info will inevitably slip through the cracks.

Cloud certifications give me a wider view of the cloud landscape. While I’m primarily a data professional, knowing about services outside that scope improves my diversity and value. It also helps me manage my own cloud accounts. While I’m not responsible for my employer’s security or networking, I am responsible for mine!

Some recent useful discoveries from my SysOps renewal include:

EventBridge Pipes

From Amazon EventBridge’s Product page:

Amazon EventBridge Pipes helps create point-to-point integrations between event producers and consumers with optional transform, filter and enrich steps. EventBridge Pipes reduces the amount of integration code needed to write and maintain when building event-driven applications.

https://aws.amazon.com/eventbridge/pipes/

For me, EventBridge Pipes feels like ETL for events. It sits between event sources and event targets, removing unneeded data and transforming what’s left. As event-driven architectures become increasingly common, EventBridge Pipes have great efficiency and cost-saving potential.

IAM NotAction

I thought this was a misprint until I checked the docs! Where Action matches the action(s) that will be allowed or denied, NotAction matches everything except the specified list of actions.

So, where this policy allows the deletion of any S3 bucket in an account:

"Effect": "Allow",
"Action": "s3:DeleteBucket",
"Resource": "arn:aws:s3:::*"

This policy allows all S3 actions except deleting an S3 bucket in an account:

"Effect": "Allow",
"NotAction": "s3:DeleteBucket",
"Resource": "arn:aws:s3:::*"

S3 Access Points

From Amazon S3’s Product page:

Amazon S3 Access Points simplify data access for any AWS service or customer application that stores data in S3. With S3 Access Points, customers can create unique access control policies for each access point to easily control access to shared datasets.

Customers with shared datasets including data lakes, media archives, and user-generated content can easily scale access for hundreds of applications by creating individualized access points with names and permissions customized for each application.

https://aws.amazon.com/s3/features/access-points/

S3 Access Points look like they can take lots of pain out of bucket policies and IAM config. This would be a big help with sharing datasets, controlling object access and setting environment variables.

Summary

In this post, I examine my attitudes toward certification and how I use my certifications after earning them.

Certifications aren’t for everyone, and that’s fine. As regards my attitudes toward certification, they’re great at improving my confidence, expanding my horizons and making me a better, more complete engineer.

AWS have a range of skill builders, workshops and whitepapers to help people on their certification journey. Check them out! If this post has been useful, the button below has links for contact, socials, projects and sessions:

SharkLinkButton 1

Thanks for reading ~~^~~

Categories
Me

YearCompass 2023-2024

YearCompass 20232024 800600

In this post, I use the free YearCompass booklet to reflect on 2023 and to plan some professional goals for 2024.

Table of Contents

Introduction

Towards the end of last year, I used YearCompass for the first time because I wanted to commit to some 2023 goals. YearCompass is a proven and long-lived framework with over 18k Facebook Likes and availability in 52 languages, so it made sense to try it out.

It went very well! So much so that I have used YearCompass again to choose my 2024 professional goals. The first half of this post covers 2023; the second half 2024.

Firstly, let’s examine YearCompass itself.

YearCompass

From the YearCompass site:

YearCompass is a free booklet that helps you reflect on the year and plan the next one. With a set of carefully selected questions and exercises, YearCompass helps you uncover your own patterns and design the ideal year for yourself.

YearCompass.com

YearCompass started as a reflection tool for a small group of friends and was made publicly available in 2012. It is available as an A4 and A5 PDF, with options to fill out the booklet both digitally and by hand. YearCompass is currently available in 52 languages.

YearCompass positions itself as an alternative to New Year’s Resolutions. Each PDF has two sections. The first half examines the previous year and the second half considers the next one.

Each section consists of a series of prompts and questions. These guide the user through the reflection process and help them identify their priorities and plan for the future.

Some of the questions are:

  • What are you most proud of?
  • Who are the three people who influenced you the most?
  • What does the year ahead of you look like?

While prompts include:

  • List your three biggest challenges from last year.
  • This year, I will be bravest when…
  • I want to achieve these three things the most.

There are no hard and fast rules for completing YearCompass. The book suggests a break between sections, although some prefer to do the whole thing in one sitting. Personally, I don’t complete every section as by a certain point I have what I need.

This year, I had my 2022 and 2023 YearCompass PDFs open side by side. It made sense for 2022’s compass to inform 2023’s, and it gave me an idea of which goal-setting approaches worked best.

2023 Retrospective

In this section, I look back at my 2023 goals and see how things went with them.

Confidence Building & Anxiety Management

This goal was focused on self-belief. I wanted to bolster my confidence and improve my technical skillset.

2022 saw my first tech conference at AWS Summit London. While London is an intense experience for a socially anxious shark, it successfully expanded my comfort zone by putting me around unfamiliar faces with similar interests.

2023’s summit was easier on the senses, and I had chats with suppliers and AWS Solution Architects about topics including data lineage, lakehousing and event orchestration.

PXL 20230607 1106476362

I also attended some events closer to home: May’s DTX Manchester and October’s Data Relay Manchester.

PXL 20230517 142206077

Separately, I used my DataCamp subscription to improve my Data Engineering skills. Their Data Engineer track has several great Python courses and helped me benchmark my T-SQL skills. Their Object-Oriented Programming in Python and Writing Functions in Python courses also helped me plug some work-related gaps.

Helpfully, DataCamp has a new My Year In Data feature that summarises the 17 courses I completed this year:

DataCamp HoursLearning
DataCamp XPDayStreak

Finally, I also recertified my AWS SysOps Administrator certification in October using the now-traditional duo of Stephane Maarek and Tutorials Dojo. This certification validates my experience in deploying, managing, and operating workloads on AWS and puts me in a good position for my 2024 goals.

AWSSysOpsBadge

Collaborating & Communicating

This goal was focused on finding my voice and improving my work quality. I wanted to strengthen my contributions and increase my value.

A big part of this was making sure that I understood the languages and terms being used around me. My ultimate aim was to use and apply these terms correctly and appropriately. 2023 was the year I became familiar with data and programming terms including:

I also signed up for some Data Engineering-focused feeds to improve my industry knowledge. Examples include the Data Engineering Weekly and Seattle Data Guy newsletters, and the Advancing Analytics and nullQueries YouTube channels.

Collaboration-wise, I also hosted my first T-SQL Tuesday this year. It was great to work with Steve Jones, and I get to include myself on a pretty illustrious list of industry professionals now!

Finally, I also made it to my first User Group meeting! While I only made it to one event this year, I overcame a lot of personal anxiety there and look forward to exploring my local user groups more in 2024.

Knowledge Sharing & Presenting

This goal was focused on creating value. I wanted to improve my presentation skills, and find real-world applications for the knowledge gained from my posts and certifications.

It’s time to discuss New Stars of Data!

Bicuit & Terrabyte on a New Stars Of Data T-shirt

Upon viewing the event’s Call For Speakers, I saw a great chance to work on this goal. I’d already started writing a VSCode Data Wrangler post at the time (which ultimately became the New Stars Of Data Retrospective post) and quickly realised the post would lend itself very well to the requested abstract.

When creating the session, the combination of a sport I enjoy, data and code I’m familiar with and an impressive VSCode extension resulted in a smooth journey from storyboarding to delivery. The session was a pleasure to create and deliver, and was exactly the presenting introduction I was after!

I also wrote a blog series while creating the session, both as something to look back on and to potentially help future speakers.

2024 Goals

In this section, I use YearCompass to decide on my 2024 professional goals. For each goal, I’ll write a user story and then explain my reasoning.

Build Technology Projects

As a cloud enthusiast, I want to complete valuable project builds so that I can develop and validate my knowledge and skills, and have subject matter for future session abstracts.

It’s fair to say that 2023 has been a year of learning, with sources including:

All of which have given me ideas for things I can build! Some completely new things. Some things that have been gaining steam for a while. Other things that recent innovations have put within reach.

My first YearCompass 2024 goal is to start building them! As well as testing my skills and validating my knowledge, some of these projects would probably lend themselves to a session abstract or two!

Additionally, I’m considering studying towards an AWS Professional certification in 2025. So if I decide to go ahead with that, building a bunch of stuff would be well worth the effort and investment!

Finally, although I’ve gotten better at finishing projects since starting amazonwebshark there’s always room for improvement. This No Boilerplate video about The Cult Of Done Manifesto really resonated with me upon first watch, and I’ll be benchmarking my 2024 projects against it.

Build My Personal Brand

As an IT professional I want to build my personal brand so that I improve my soft skills and define my public image.

Continuing the build theme, my second YearCompass 2024 goal is focused on my soft skills and visibility.

I’ve spoken about confidence and anxiety previously. I will always be working on this, but it isn’t something I want to hide behind. As my contributions to this site and the wider community grow, I need to consider how those contributions influence the projected image of my personality, skills, experience, and behaviour.

Furthermore, in an age where AI tools and models are getting increasingly adept at a range of tasks, practising and demonstrating my soft skills is perhaps more important than ever. As technology becomes increasingly democratised, it is no longer enough to focus on technical skills alone.

I’ve already begun to establish my personal brand via amazonwebshark and social media. With my 2024 goals likely to put me in front of more fresh faces for the first time, now is definitely the time to make my personal brand a primary focus.

Build A Second Brain

As just a normal man I want to build a second brain so that I can organise my resources and work more efficiently.

For my final YearCompass 2024 goal, I want to take steps to solve a long-standing problem.

I have lots of stuff. Books, files, hyperlinks, videos…STUFF. Useful stuff, but unorganised and unstructured stuff. I also have lots of ideas. Ideas for efficiency and growth. Ideas for reliability and resilience. And I have various ways of capturing these ideas depending on where I happen to be. Even my car has a notepad.

Finally, and perhaps most importantly, I have several partially-enacted systems for handling all of this. Some systems turned out to be unfit for purpose. Some were overwhelmed, while others became unwieldy.

Recently, I’ve made efforts to organise and define everything. I’m already finding success with this, and with the recent discovery of the Second Brain and CODE methodologies I now have a framework to utilise. A well-built second brain will help organise my backlog, assist my day-to-day and support my future goals.

Summary

In this post, I used the free YearCompass booklet to reflect on 2023 and to plan some professional goals for 2024.

Having finished this post, I’m happy with my 2024 goals and am looking forward to seeing where the year takes me! I’ll post updates here and via my social, project and session links, which are available via the button below:

SharkLinkButton 1

Thanks for reading ~~^~~