Categories
Me

Emotive Amiga Code

In this post, I respond to July 2023’s T-SQL Tuesday #164 Invitation and talk about emotive code on the Amiga.

tsql tuesday

Table of Contents

Introduction

This month’s T-SQL Tuesday comes from Erik Darling. To paraphrase his invitation:

Think back to the last time you saw code that made you feel a thing. Hopefully a positive thing. Think along the lines of: surprise, awe, inspiration, excitement. Or maybe it was just code that sort of sunk its teeth into you and made you want to learn a whole lot more.

Now, while Erik’s invitation mentions the last time, I want to talk about the first time. This is an ideal opportunity to expand on something I wrote on my About page about AMOS tutorials. But before I start explaining why I consider some mid-nineties Amiga code to be emotive, I should explain a few things.

Key Topics

This section introduces some of the key topics I talk about in this post.

Amiga

The Amiga home computer was first released by Commodore in 1985, and was one of the first computers with custom graphics and sound hardware.

Amiga500 system
An Amiga 500

The Amiga was designed to compete with the Apple Macintosh and the IBM PC. It had a custom chipset and a multitasking operating system. These made the Amiga appeal to a wide range of audiences including:

The Amiga also had expansion capabilities, with various third-party add-ons available like the Video Toaster production tool. This allowed the Amiga to be used for visual effects in TV shows like Babylon 5 and SeaQuest DSV, and films like Apollo 13 and Titanic.

The Amiga Wikipedia page has information on history, chipsets, trivia and more.

Amiga Format

Amiga Format was a monthly UK computer magazine published from 1989 to 2000.

logo amiga format

Broadly, there were three types of Amiga magazines:

  • Game-focused magazines like Amiga Power (Yayyy – Ed).
  • Serious magazines like Amiga Shopper.
  • Magazines with some of both, like Amiga Format.

Amiga Format was about 50% games to 50% serious. This expanded to their coverdisks, with a fairly even split of utilities and game demos across each month’s disks.

My first Amiga Format was issue 30, purchased alongside the family Amiga 500+ Cartoon Classics bundle:

AmigaFormat iss30

And I still have them both! Young me chucked the magazine around a fair bit, so years later I bought a copy from eBay in better condition. I’ll get some photos on Instagram when I get a chance.

The Amiga Format Wikipedia page has some additional information including some regular features.

AMOS

AMOS (or to give its full name AMOS BASIC) is an Amiga dialect of the BASIC game programming language.

AMOSBox

First published in 1990 by Europress Software, AMOS was well received and sold over 40,000 copies worldwide.

Interaction comes via the AMOS Editor: a text editor with the AMOS language built into it. This allows users to type, edit, save and run AMOS programs from within the editor, without the need for a separate runtime tool or compiler.

amos

The AMOS Wikipedia page goes into more technical detail, and this PowerPrograms page has a user’s AMOS perspective.

In addition, AMOS Professional has a GitHub repo, and I would be remiss if I didn’t include Dan Wood’s excellent AMOS retrospective video:

From AMOS To YAML

In this section, I talk about the Amiga code that ended up being so emotive and indirectly set my future career in motion.

Scene Setting

It’s Christmas 1992, and I’m a young shark pup still in single digits. I hit the newsagent’s magazine shelves with pocket money in hand and buy that month’s Amiga Format:

AmigaFormat iss42

Yes – it does say January 1993 on the cover. Before writing this post I never had a good explanation for this, so I reached out to the Amiga Addict Discord for some clarity.

It turns out that, during this period, Amiga Format produced issues every four weeks instead of monthly. Former Amiga Format editor Marcus Dyson speaks about this at around 38:30 in Maximum Power Up podcast episode 139.

But anyway.

Throughout 1992, Amiga Format’s coverdisks featured a series of fully-functional software packages known as The Amiga Format Collection. This collection included:

  • Fractal landscape generator Vista (AF33)
  • HAM paint and animation package Spectracolor Jr (AF35)
  • Machine code assembler Devpac 2 (AF39)

Adorning Amiga Format 42’s cover was AMOS:

af42 1993 01 d1

I’d read about AMOS before. Amiga Format issue 35 had a demo of Easy AMOS, and even the games-centric Amiga Power mentioned AMOS in its Public Domain section. And now, suddenly every Amiga Format buyer had an unrestricted copy of it.

Amiga Format partnered this with a Learning AMOS magazine section. Basic principles, syntax and screenshots were introduced across nine pages, with the tenth page having a tutorial for a scrolling text demo. This was to be the first part of a monthly Mastering AMOS series.

What A Pong

Now, young me didn’t pay much attention to this part of issue 42. It was Christmas after all, and I had demos of Lemmings 2, Fire & Ice and Bill’s Tomato Game to keep me busy!

Enter Amiga Format issue 43. This month’s AMOS tutorial was Solo Pong – a simple version of Pong with the player controlling both bats. This was deliberate – the focus here was on learning AMOS as opposed to learning coding.

Young me was intrigued! The scrolling demo didn’t really appeal, but this was a game! The complete script was only half a page long, and the tutorial explained each line. So I gave it a shot, and within an afternoon I had my very own Pong clone!

2023 07 07 solopong

The code itself made no sense to me – I was copying words from a magazine onto a screen and hoping for the best. But I found the process itself captivating. The idea that all the software I’d run on my Amiga was made by words on a screen – and words that I could type myself at that – was a revelation!

On an unrelated note, Amiga Format 43 included the last of the Amiga Format Collection series – the database package Prodata. So not only did issue 43 introduce me to coding – it also foreshadowed my eventual tech career!

Mastering AMOS Legacy

Amiga Format continued the Mastering AMOS series until issue 51. They went on to include AMOS Professional with issue 67, and ran an Ultimate AMOS tutorial series from issue 68 to issue 73.

Damien Junior didn’t do much more with the Mastering AMOS series after Amiga Format 43. I vaguely remember having a go at the Amiga Format 44 tutorial, which added some bells and whistles to the existing Pong code. But after that, it started going over my head and I lost interest. Although to be fair, I was a child!

The coding curiosity remained. As the 1990s went by I found myself trying out Amiga Format’s DevPac 2 and Blitz Basic tutorials, and by the end of the 1990s I was learning Excel formulae and VBScript at school. During the 2000s I started using HTML, CSS and PHP, and began using T-SQL, Powershell and DAX at work in the 2010s.

Which brings me to the present day, where I find myself learning Python and experimenting with reStructuredText (for Read The Docs) and YAML (for CloudFormation). And I thought, after thirty years, why not revisit the Amiga Format tutorial and see how I feel about AMOS now?

Thirty Years Later

Like many self-respecting Amiga fans I have WinUAE installed on my laptop, and an ADF of the AMOS coverdisk isn’t hard to find. So I fired everything up! And, after thirty minutes of typing, squinting and error troubleshooting, witness the result of my mighty endeavour – SHARK PONG:

2023 07 07 sharkpong

I clearly wasn’t squinting hard enough though. In this script, AMOS uses the ball’s X coordinate to determine if a player has lost. Where the tutorial’s LOSE conditions are:

If X>=320 Then LOSE : Goto RESTART
If X<=-20 Then LOSE : Goto RESTART

I missed the minus in the second IF statement and wrote:

If X<=20 Then LOSE : Goto RESTART

Thus creating a version of Shark Pong that Left Paddle can never win, as the losing X coordinate is in front of the paddle:

SharkPongFail

But then again, it wouldn’t be an early 90s tutorial follow-along without an unintended and slightly hilarious bug, right? And the command to stop running the code? Control-C. That broke my brain briefly.

But yeah. Revisiting the same code that captivated me 30 years ago was a blast! The original tutorial code is in my GitHub for those curious to see it, both as an unformatted text file and in glorious BlitzBasic!

Further Reading

Summary

In this post, I responded to July 2023’s T-SQL Tuesday #164 Invitation and talked about emotive code on the Amiga.

I had a lot of fun writing this! A phrase like ’emotive Amiga code’ might not sound fascinating on paper, but it was interesting to revisit that tutorial after so long. I even finally answered a question that’s lingered in my head for a few decades.

Thanks to Erik for this month’s topic! My previous T-SQL Tuesday posts are here. If this post has been useful, please feel free to follow me on the following platforms for future updates:

Thanks for reading ~~^~~

Categories
Developing & Application Integration

Stress-Free AWS Invoice PDF Administration With Power Automate

In this post, I create a stress-free AWS invoice PDF administration workflow with Microsoft Power Automate.

Table of Contents

Introduction

Each month, AWS sends me VAT invoices for all of my AWS accounts. Their filenames aren’t descriptive, with names like EUINGB21-2079861, EUINGB22-3276756 and EUINGB23-2216483. As a result, I rename them to make the contents clearer.

Now while I have the best intentions in the world, life happens and the PDFs usually end up in a to-do folder. As a result, I now have a folder like this:

2023 05 08 LakeFolder

I don’t want to sort through that. No one wants to sort through that. There must be another way! After successfully automating my application management with Winget, I’m keen to see what options I have here.

Firstly, let’s examine what currently happens with my new AWS invoice PDFs.

Manual Solution

This section examines my manual AWS invoice PDF workflow.

Here are several sample AWS invoices for reference. Although they’re from 2018, the 2023 bills are mostly the same. Let’s focus on a document header:

2023 05 08 AWSSampleInvoice

I copy the account number and VAT invoice date from a bill and combine them with some extra strings to produce a filename schema of:

AWS-{VAT Invoice Date}-{Account Number}-VATInvoice

In this example, that would produce:

AWS-2018-08-03-292122068034-VATInvoice

There are several pain points here:

  • The PDF must be opened to view the data.
  • The invoice date needs converting to ISO 8601 standard.
  • An open file can’t be renamed, so the new filename must be created in Notepad and then copy/pasted over.
  • Everything is manual and needs my full attention.

Wouldn’t it be nice if all this could be automated?

Microsoft Power Automate

This section examines Microsoft Power Automate and how it works.

What Is Power Automate?

Microsoft Power Automate is an automation tool. It uses triggers and actions to create workflows that complete manual, repetitive, and time-consuming tasks without human involvement. Power Automate is part of the Microsoft Power Platform.

Benefits of Power Automate include:

  • A no-code GUI with drag-and-drop functionality.
  • An extensive library of pre-defined templates.
  • Onboard testing and deployment features.
  • A free plan for work or school accounts.
  • Integration with the other Power Platform applications, Office 365 and other Microsoft and third-party services.

Power Automate Versions

Power Automate has two versions:

  • Power Automate Desktop focuses on desktop flows like file & folder manipulation, user input and script triggering. I’ll be using Power Automate Desktop in this post.

It’s worth pointing out that there are several visual differences between the two versions. This confused me when researching this post, as most screenshots I found were from the cloud service!

Power Automate Flows

Central to Power Automate is the concept of Flows. A Power Automate flow is a sequence of actions that are triggered by an event, performing a series of operations to achieve the desired results.

Flows are created through a visual interface that defines triggers, actions, and conditions using a drag-and-drop approach. Code can be written, but is optional. Completed flows can then be saved, published and run as needed.

Creating A Power Automate Flow

In this section, I create my AWS invoice Power Automate flow.

Getting All Files In The Folder

Firstly, I capture the AWS invoice PDFs using the Get Files In Folder action:

2023 05 11 PAGetFilesInFolder

Here, I specify the folder containing the invoices. I tell Power Automate to only get PDF files, to stop any other file types causing errors.

Power Automate stores a list of these files in a %Files% variable. I want to work on each file separately, so I pass the %Files% variable to a For Each action:

2023 05 11 PAForEach

This creates a loop that lets Power Automate capture each file in the %Files% variable. Each time the For Each action triggers, Power Automate stores a file from %Files% in a new %CurrentItem% variable.

Each subsequent action within the loop will then be applied to the %CurrentItem% file. At the end of the loop, Power Automate captures the next file from %Files% and overwrites the %CurrentItem% variable with it. This continues until all %Files% files are processed and the loop ends.

Getting The PDF Text

Secondly, I need to get the text from each %CurrentItem% PDF. There’s a great Extract Text From PDF action that can do this:

2023 05 11 PAExtractTextPDF

The PDF text is stored in an %ExtractedPDFText% variable that is going to be very useful for the rest of this flow!

So far, the flow looks like this:

2023 05 11 PAFlowFolderAndFiles

With the PDF text captured, the rest of the automation can begin!

Getting The Account Number

Next, let’s examine the PDF text more closely. This is part of %ExtractedPDFText% for an AWS invoice:

VAT Invoice
Email or talk to us about your AWS account or bill, visit console.aws.amazon.com/support
More information regarding your service charges is available by accessing your Billing Management Console
Account number:
[REDACTED]
Address:
[REDACTED]
Invoice Summary
VAT Invoice Number: EUINGB22-3617620
VAT Invoice Date: September 2, 2022
TOTAL AMOUNT GBP 0.34

On AWS invoices, the account number is always between ‘Account number:’ and ‘Address:’. Knowing this, I can use Crop Text to get the text between these strings:

2023 05 11 PACropTextAWSAccNo

This action produces two new variables:

  • %CroppedText%
  • %IsFlagFound%

Usually, these names would be fine. This isn’t the only text I’m going to be cropping though, so I change these to %AWSAccNoCroppedText% and %AWSAccNoIsFlagFound% respectively.

Before continuing, I’d like to check that %AWSAccNoCroppedText% contains the expected value. A good way to test this is to use the Display Message action:

2023 05 11 PAFlowCropAcctNo

(The account number is obfuscated here, but it was correct!)

Getting The Invoice Date

Now to get the invoice date! This is similar to getting the account number, using Crop Text to capture the date between ‘VAT Invoice Date:’ and ‘TOTAL AMOUNT’:

2023 05 11 PAAWSBillCropText

This produces variables %CroppedText% and %IsFlagFound% again, which I then change to %AWSBillDateCroppedText% and %AWSBillDateIsFlagFound%.

This isn’t enough this time though. Currently, the %AWSBillDateCroppedText% value is a date formatted as August 3, 2018. This won’t produce the filename I want, as I need the date to be in the ISO format of 2018-08-03.

To do this, I use the Convert Text To Datetime action. This converts a text representation of a date value to a datetime value:

2023 05 11 PAAWSBillTextoDatetime

I supply %AWSBillDateCroppedText% to convert. A datetime value of 03/08/2018 00:00:00 is returned, which I then assign to a %AWSBillDateTextAsDateTime% variable.

This is still unacceptable though! The date format doesn’t match the requirement and the slashes and colons are not legal filename characters.

To get the required date format, I use the Convert Datetime to Text action to convert 03/08/2018 00:00:00 to the custom format yyyy-MM-dd, in which:

  • yyyy is the year as a four-digit number.
  • MM is the month as a two-digit number, from 01 through 12.
  • dd is the day of the month as a two-digit number, from 01 through 31.
  • - is…a hyphen.
2023 05 11 AWSBillDateDatetimeToText

There is a table describing the custom date and time format specifiers in the Microsoft documentation.

This action produces a %AWSBillDateFormattedDateTime% variable with a value of 2018-08-03. Success!

Creating The New Filename

Now I can start thinking about the filename! To recap, I currently have:

  • A %AWSAccNoCroppedText% variable with a value of (for example) 1234123412341234.
  • A %AWSBillDateFormattedDateTime% variable with a value of 2018-08-03.

To create the filename, I need to combine my variables with some additional strings. For this, I use the Set Variable action to create a %AWSFilename% variable with the following value:

AWS-%AWSBillDateFormattedDateTime%-%AWSAccNoCroppedText%-VATInvoice

This breaks down to:

  • AWS- string
  • The %AWSBillDateFormattedDateTime% variable
  • A hyphen: -
  • The %AWSAccNoCroppedText% variable
  • -VATInvoice string

This should produce a string like AWS-2018-08-03-123456781234-VATInvoice. However, this was %AWSFilename%‘s initial output:

AWS-2018-08-03-
123456781234
-VATInvoice

This is unacceptable as a filename, but why is it happening?

It turns out that the Crop Text action capturing the account number is also capturing the line breaks on either side of it. I remove these with the Trim Text action:

2023 05 13 PATrimText

This creates a new %AWSAccNoTrimmedAndCroppedText% variable. Replacing %AWSAccNoCroppedText% with this new variable when setting the %AWSFilename% variable produces the required output:

AWS-2018-08-03-123456781234-VATInvoice

The %AWSFilename% variable can then be used by the Rename File(s) action to rename the file assigned to %CurrentItem% by the earlier For Each action:

2023 05 14 PARenameFiles

The updated file is assigned to a %AWSRenamedFiles% variable, which I use in the final part of the loop.

Moving The File

I usually move renamed invoices to a different path manually. I might as well add this process to my Power Automate flow since it will save me even more time!

Here, I move the file assigned to the %AWSRenamedFiles% variable to my destination folder using the Move File(s) action:

2023 05 14 PAMoveFIles

This is the last action in the loop. Or at least it was, until I reviewed the flow and identified a few improvements.

Enhancements

Originally, this AWS invoice Power Automate flow was just for AWS invoices. However, with some additional Power Automate actions the flow could easily extend to other kinds of invoices. This would allow one flow to administrate multiple invoice types instead of each type needing its own flow.

To begin, I limit the scope of the flow’s actions by adding an If action inside the For Each loop. It will now only apply the AWS invoice flow actions if %ExtractedPDFText% contains the string ‘AWS account or bill’:

2023 05 11 PAIf

Non-AWS invoices will not meet this condition, and will not be mislabeled by the flow or cause the flow to fail.

So what will happen to non-AWS invoices then? Currently, they’ll just be left in the source folder. Ideally, something should happen to them too, so let’s add a dead letter queue to the flow.

I add two new actions to the very end of the flow, positioned after the For Each loop has processed each file in the source folder:

2023 05 14 PAFlowDLQ

Both actions have already been used in this flow, but this time their focus has changed. By the time Power Automate reaches these actions, all files in the source folder will have gone through the For Each loop.

At this point, one of two sequences has happened:

  • The file did meet the AWS invoice condition, is renamed and then moved to the destination folder.
  • The file didn’t meet the AWS invoice condition and is still in the source folder.

So the only files now in the source folder are not AWS invoices.

The Get File In Folder action will get these files and store them in a %DLQFiles% variable. This is then passed to the Move File(s) action, which moves the unprocessed files to a PowerAutomateDLQ folder.

The end result is that:

  • The source folder is empty.
  • AWS invoices are renamed and then moved to the destination folder.
  • Any other files are moved to the PowerAutomateDLQ folder.

So, does it all work?

Testing

This section tests my AWS invoice Power Automate flow.

I put three files into the source folder and then ran the Power Automate flow:

  • A DALL-E PNG.
  • An AWS invoice PDF.
  • A utility bill PDF.
2023 05 14 TestFiles

The AWS invoice is renamed and then moved to the destination folder:

2023 05 13 TestAfter

The remaining files are unaltered and moved to the DLQ folder.

The flow also works far faster than my manual workflow. The old process – from opening the PDF to moving the renamed folder – took about 2 minutes each on average. Conversely, the flow processed 12 files in 2 seconds – this would have taken me 24 minutes!

Summary

In this post, I created a stress-free AWS invoice PDF administration workflow with Microsoft Power Automate.

I’m very impressed by Power Automate! It can certainly save me a ton of time with tasks I currently do, and with other actions like the ability to run Powershell and Python scripts, execute SQL statements and emulate the terminal, I feel like I haven’t even scratched the surface yet!

If this post has been useful, please feel free to follow me on the following platforms for future updates:

Thanks for reading ~~^~~

Categories
Data & Analytics

WordPress MySQL Database Tables Deep Dive

In this post, I do a deep dive into some of the amazonwebshark WordPress MySQL database tables following the journey of a recent post.

Table of Contents

Introduction

In January I used Python and Matplotlib to create some visualisations using the WordPress amazonwebshark MySQL database.

Since then I’ve been doing a lot with Power BI at work, so I’ve created a Power BI connection to the amazonwebshark database to reacquaint myself with some features and experiment with a familiar dataset.

I talked about doing a views analysis in January’s post. While some of the 2022 data is missing, I can still accurately analyse 2023 data. I plan to measure:

  • Total views for each post.
  • Total views for each category.

I’ll use this post to examine some of the MySQL tables, and link back to it in future analysis posts.

Let’s begin with a brief WordPress database overview.

WordPress Database 101

In this section, I take a high-level view of a typical WordPress database and identify the tables I’ll need.

There’s plenty of great documentation online about typical WordPress installations. I’m particularly keen on The Ultimate Developer’s Guide to the WordPress Database by DeliciousBrains, which includes an in-depth tour of the various tables.

As for table relationships, this WordPress ERD shows object names, primary keys and relationship types:

I’ll be concentrating on these WordPress tables:

And the wp_statistics_pages table used by WPStatistics.

I’ll examine each table in the context of a recent post: DBeaver OpenAI ChatGPT Integration.

wp_posts

In this section of my WordPress database deep dive, I examine the most important WordPress database table: wp_posts.

Table Purpose

WordPress uses wp_posts to manage a site’s content. Each row in the table is an event relating to a piece of content, like a post, page or attachment. Examples of these events in the context of a blog post are:

  • Creating A New Draft: A new row is created with a post_status of draft. This row is the parent of all future activity for the blog post.
  • Updating A Draft: A new row is created with details of the update. The new row’s post_parent is set to the initial post’s ID.
  • Publishing A Draft: The initial row’s post_status is changed to publish, and the post_date is changed to the publication date. WordPress finds revisions to the post by filtering rows with a post_parent matching the initial row’s ID.

Post Journey

Let’s start by finding DBeaver OpenAI ChatGPT Integration‘s parent row, which is its earliest record. The following query finds rows where the post_title is DBeaver OpenAI ChatGPT Integration, then orders by ID and returns the first result.

SELECT 
  id, 
  post_date, 
  post_title, 
  post_status, 
  post_name, 
  post_parent, 
  post_type 
FROM 
  `wp_posts` 
WHERE 
  post_title = 'DBeaver OpenAI ChatGPT Integration' 
ORDER BY 
  id 
LIMIT 
  1

Note that I order by ID, not post_date. The publication process changes the parent post’s post_date, so I must use ID to find the earliest post.

This record is returned:

Name Value
ID 1902
post_date 2023-02-19 20:28:22
post_title DBeaver OpenAI ChatGPT Integration
post_status publish
post_name dbeaver-openai-chatgpt-integration
post_parent 0
post_type post

So the DBeaver OpenAI ChatGPT Integration parent row is ID 1902. I can use this to count the number of changes to this post by searching for wp_posts rows with a post_parent of 1902:

SELECT 
  COUNT(*) 
FROM 
  `wp_posts`
WHERE 
  post_parent = 1902

81 rows are returned:

Name    |Value|
--------+-----+
COUNT(*)|81   |

Now let’s examine these results more closely.

In the following query, I get all rows relating to DBeaver OpenAI ChatGPT Integration and then group the results by:

  • Date the post was made (using the MySQL DATE function to remove the time values for more meaningful aggregation).
  • Status of the post.
  • Post’s parent post.
  • Type of post.

I also count the rows that match each group and order the results by ID to preserve the event order:

SELECT 
  COUNT(*) AS ID_count, 
  DATE(post_date) AS post_date, 
  post_status, 
  post_parent, 
  post_type 
FROM 
  `wp_posts`
WHERE 
  ID = 1902 
  OR post_parent = 1902 
GROUP BY 
  DATE(post_date), 
  post_status, 
  post_parent, 
  post_type 
ORDER BY 
  ID

The query results are below. A couple of things to note:

  • The first two columns show what happens when a post is published. Row 1 is ID 1902 as it has no post_parent, and it has a post_status of publish and a post_date of 2023-02-19.
  • Row 2 is the first revision of ID 1902, and it has a post_status of inherit and a post_date of 2023-02-15. This is why I order by ID instead of post_date – ordering by post_date would show the revisions before the parent post in the results.
  • There are various post_type valves – revisions are text updates and attachments are image updates.
ID_count post_date post_status post_parent post_type
1 2023-02-19 publish 0 post
1 2023-02-15 inherit 1902 revision
19 2023-02-16 inherit 1902 revision
7 2023-02-16 inherit 1902 attachment
24 2023-02-17 inherit 1902 revision
1 2023-02-17 inherit 1902 attachment
7 2023-02-18 inherit 1902 revision
21 2023-02-19 inherit 1902 revision
1 2023-02-26 inherit 1902 revision

Spotlighting some of these results for context:

  • On 2023-02-16 there were 19 text revisions and 7 images attached. I save a lot!
  • On 2023-02-19 there were 21 text revisions and then the post was published.
  • There was a further text revision on 2023-02-26 in response to a DBeaver software update.

That’s enough about wp_posts for now. Next, let’s start examining how WordPress groups content.

wp_term_relationships

In this section, I examine the first of the WordPress taxonomy tables: wp_term_relationships.

Table Purpose

wp_term_relationships stores information about the relationship between posts and their associated taxonomy terms (More on taxonomies in the next section). WordPress uses it as a bridge table between wp_posts and the various taxonomy tables.

Post Journey

In this query, I join wp_term_relationships to wp_posts on object_id (this is ID in wp_posts), then find the rows where either wp_posts.id or wp_posts.post_parent is 1902:

SELECT 
  yjp.ID, 
  DATE(yjp.post_date) AS post_date, 
  yjp.post_type, 
  yjp.post_status,
  yjtr.object_id, 
  yjtr.term_taxonomy_id 
FROM 
  `wp_posts` AS yjp 
  INNER JOIN `wp_term_relationships` AS yjtr 
    ON yjtr.object_id = yjp.ID 
WHERE 
  yjp.ID = 1902 
  OR yjp.post_parent = 1902

wp_term_relationships only contains published posts, so the only rows returned concern the parent ID 1902:

ID post_date post_type post_status object_id term_taxonomy_id
1902 2023-02-19 post publish 1902 2
1902 2023-02-19 post publish 1902 69
1902 2023-02-19 post publish 1902 71
1902 2023-02-19 post publish 1902 74
1902 2023-02-19 post publish 1902 76
1902 2023-02-19 post publish 1902 77

The query returned six distinct wp_term_relationships.term_taxonomy_id values. My next step is to establish what these IDs relate to.

wp_term_taxonomy

In this section, I examine the table that groups term_taxonomy_id values into taxonomy types: wp_term_taxonomy.

Table Purpose

WordPress uses the wp_term_taxonomy table to store the taxonomy data for terms. Taxonomies in WordPress are used to group posts and custom post types together. Examples of WordPress taxonomies are category, post_tag and nav_menu.

Post Journey

In this query, I add a new join to the previous query, joining wp_term_taxonomy to wp_term_relationships on term_taxonomy_id. Some of the wp_posts columns have been removed from the query to save space.

SELECT 
  yjp.ID,  
  yjtr.term_taxonomy_id, 
  yjtt.taxonomy
FROM 
  `wp_posts` AS yjp 
  INNER JOIN `wp_term_relationships` AS yjtr 
    ON yjtr.object_id = yjp.ID 
  INNER JOIN `wp_term_taxonomy` AS yjtt 
    ON yjtr.term_taxonomy_id = yjtt.term_taxonomy_id 
WHERE 
  yjp.ID = 1902 
  OR yjp.post_parent = 1902

These results give some content to the previous results. I can now see that wp_posts.id 1902 has one category and five tags.

ID term_taxonomy_id taxonomy
1902 2 category
1902 69 post_tag
1902 71 post_tag
1902 74 post_tag
1902 76 post_tag
1902 77 post_tag

To get the names of the categories and tags, I must bring one more table into play…

wp_terms

In this section of my WordPress database deep dive, I examine the table that holds the names and details of the taxonomy terms used on amazonwebshark: wp_terms.

Table Purpose

The wp_terms table stores all of the terms that are used across all taxonomies on a WordPress site. Each row represents a single term, and the columns in the table contain information about that term, including name and ID.

Post Journey

In this query, I add another join to the previous query, joining wp_terms to wp_term_taxonomy on term_id.

SELECT 
  yjp.ID, 
  yjtr.term_taxonomy_id, 
  yjtt.taxonomy,
  yjt.name 
FROM 
  `wp_posts` AS yjp 
  INNER JOIN `wp_term_relationships` AS yjtr 
    ON yjtr.object_id = yjp.ID 
  INNER JOIN `wp_term_taxonomy` AS yjtt 
    ON yjtr.term_taxonomy_id = yjtt.term_taxonomy_id 
  INNER JOIN `wp_terms` AS yjt 
    ON yjtt.term_id = yjt.term_id 
WHERE 
  yjp.ID = 1902 
  OR yjp.post_parent = 1902

The results now identify the category and each of the five tags by name:

ID term_taxonomy_id taxonomy name
1902 2 category AI &amp; Machine Learning
1902 69 post_tag WordPress
1902 71 post_tag DBeaver
1902 74 post_tag MySQL
1902 76 post_tag OpenAI
1902 77 post_tag ChatGPT

This is a perfect match for the post’s taxonomy in the WordPress portal:

2023 03 10 WordPressPanelChatGPT

So that’s the categories. What about the views?

wp_statistics_pages

In this final section, I examine the WPStatistics table that holds view counts: wp_statistics_pages.

Table Purpose

WPStatistics uses wp_statistics_pages to store data about page views. Each row shows a URI’s total views on the date specified.

WPStatistics documentation isn’t as in-depth as WordPress, so here are the table’s DDL and column descriptions:

CREATE TABLE `1yJ_statistics_pages` (
  `page_id` bigint(20) NOT NULL AUTO_INCREMENT,
  `uri` varchar(190) NOT NULL,
  `type` varchar(180) NOT NULL,
  `date` date NOT NULL,
  `count` int(11) NOT NULL,
  `id` int(11) NOT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `date_2` (`date`,`uri`),
  KEY `url` (`uri`),
  KEY `date` (`date`),
  KEY `id` (`id`),
  KEY `uri` (`uri`,`count`,`id`)
)
Table NameDescription
page_idPrimary key. Unique identifier for the table.
uriUniform Resource Identifier used to access a page.
typeuri type: home / page / post
dateDate the uri was viewed
counturi total views on the specified date
iduri ID in wp_posts.ID

Post Journey

As wp_statistics_pages.id is the same as wp_posts.id, I can use id 1902 in a query knowing it will still refer to DBeaver OpenAI ChatGPT Integration.

For example, this query counts the number of rows in wp_statistics_pages relating to id 1902:

SELECT 
  COUNT(*) 
FROM 
  `wp_statistics_pages` 
WHERE 
  id = 1902
COUNT(*)|
--------+
      14|

I can also calculate how many visits DBeaver OpenAI ChatGPT Integration has received by using SUM on all wp_statistics_pages.count values for id 1902:

SELECT 
  SUM(yjsp.count) 
FROM 
  `wp_statistics_pages` AS yjsp
WHERE 
  yjsp.id = 1902
SUM(count)|
----------+
        40|

So the page currently has 40 views. I can see how these views are made up by selecting and ordering by wp_statistics_pages.date:

SELECT 
  yjsp.date, 
  yjsp.count 
FROM 
  `wp_statistics_pages` AS yjsp
WHERE 
  yjsp.id = 1902 
ORDER BY 
  yjsp.date

date count
2023-02-19 1
2023-02-20 5
2023-02-21 1
2023-02-22 4
2023-03-07 6
2023-03-08 3
2023-03-09 2
2023-03-10 1

I can also join wp_posts to wp_statistics_pages on their id columns, bridging the gap between the WPStatistics table and the standard WordPress tables:

SELECT 
  yjsp.date, 
  yjsp.count, 
  yjp.post_title 
FROM 
  `wp_statistics_pages` AS yjsp 
  INNER JOIN `wp_posts` AS yjp 
    ON yjsp.id = yjp.id 
WHERE 
  yjsp.id = 1902 
ORDER BY 
  yjsp.date
date count post_title
2023-02-19 1 DBeaver OpenAI ChatGPT Integration
2023-02-20 5 DBeaver OpenAI ChatGPT Integration
2023-02-21 1 DBeaver OpenAI ChatGPT Integration
2023-02-22 4 DBeaver OpenAI ChatGPT Integration
2023-03-07 6 DBeaver OpenAI ChatGPT Integration
2023-03-08 3 DBeaver OpenAI ChatGPT Integration
2023-03-09 2 DBeaver OpenAI ChatGPT Integration
2023-03-10 1 DBeaver OpenAI ChatGPT Integration

Summary

In this post, I did a deep dive into some of the amazonwebshark WordPress MySQL database tables following the journey of a recent post.

I’ve used this post to present the journey a typical post goes through in the WordPress database. Future posts will use this knowledge and the WordPress database as a data source for various dashboards, scripting and processes. Watch this space!

If this post has been useful, please feel free to follow me on the following platforms for future updates:

Thanks for reading ~~^~~