The Azure cloud platform is more than 200 products and cloud services designed to help you bring new solutions to life—to solve today’s challenges and create the future.
Earlier this month, I hosted March 2023’s T-SQL Tuesday with an invitation concerning the ongoing Microsoft and OpenAI partnership:
What is on your wishlist for the partnership between Microsoft and OpenAI?
What follows is some commentary on, and links to, each of the responses.
Chad Callihan
Chad’spost considers potential PowerShell-OpenAI functionality, which would write scripts in response to user prompts. PowerShell is a mainstay of many data professionals, enabling modules like dbatools, Pester and the AWS, Azure and GCPSDKs. An AI with access to the PowerShell Gallery would be very helpful.
Chad also points out some security concerns linked with ChatGPT use, which are good advice in general:
Chris’spost considers an AI model for file ingestion. Data pipelines frequently rely on source data with specific types and layouts. Unfortunately, source data can change between ingestion times.
At best, this breaks pipelines and causes problems and downtime for data teams. At worst, incorrect data is ingested causing potential business and customer detriment.
A no-code AI model would save hours of work if it considered previous source data and could make decisions like “This column is formatted as VARCHAR, but yesterday it was DATETIME2 and has hyphens in the right place, so I’ll CAST it as DATETIME2 today and raise a warning in the log.”
Rob Farley
Rob’spost was partially written by ChatGPT! Rob takes a pragmatic approach to AI’s progress and draws a Clippy analogy. I really want to see this AI family tree now.
ChatGPT suggests that it can help with Excel formulae, SQL Server optimization and PowerPoint visuals. It also wants to democratize technology interaction and remove traditional barriers to entry.
Steve Jones
Steve’spost imagines the next generation of AI personal assistant. One that can:
Learn from the user and correct common errors in all applications.
Suggest code optimizations in a variety of IDEs and languages.
Learn the user’s schedule and create automated calendar events and reminders.
Recognise repeat tasks and create related automation.
Summary
In this post, I wrote a round-up of the community responses to my March 2023 T-SQL Tuesday #160 invitation: Microsoft OpenAI Wishlist.
Thanks to everyone who contributed to my first T-SQL Tuesday invitation. It was great to read your responses! Anyone interested in hosting future events should contact Steve Jones.
If this post has been useful, please feel free to follow me on the following platforms for future updates:
Artificial Intelligence has been a big deal in recent months. One of the main drivers of this has been OpenAI, whose DALL-E 2 and ChatGPT services have seen extraordinary public interest and participation.
Microsoft has been one of OpenAI’s most prominent supporters. In July 2019 Microsoft invested $1 billion in OpenAI and became their exclusive cloud provider.
However, Power BI is reliant on user skill levels. Like all data visualization tools, Power BI can create bad dashboards in the wrong hands. Like, really bad. Dashboards can suffer from several problems that make them useless at best and misleading at worst.
"Improve my annual sales dashboard"
>> I have changed the pie chart showing 12 team members to a bar chart, as this will improve the visualization's legibility.
Azure IAC AI Assistant
IAC (Infrastructure As Code) has revolutionized the public cloud industry, bringing with it benefits like:
Automated, faster deployments.
Repeatable and consistent deployments.
Self-documenting infrastructure.
But IAC also presents challenges:
IAC scripts rely on the skills of the engineer writing them.
It’s not easy to incorporate existing infrastructure.
ChatGPT could resolve many of these problems, turning infrastructure creation into a conversation. It could, for example:
Create infrastructure based on non-technical requests:
"Make me what I need to start a blog."
>> I have created a LAMP stack on a virtual machine in your default region. Your access details are here:
Username: Username
Password: Password
Learn current infrastructure usage patterns and create optimisations for busy and quiet periods.
Spot potential conflicts and step in to prevent data loss or downtime.
Make existing infrastructure faster, cheaper or more performant without the need for manual refactoring.
Resolve problems like high latency, failing connections and unexpected cost increases:
"Why is my web app generating errors?"
>> One of your virtual machines does not allow connection requests from CIDR range 10.01.10.01/28. Do you want me to fix this?
"Yes please."
>> I have now amended virtual machine MYAPP001's Network Security Group to accept incomming connection requests from CIDR range 10.01.10.01/28.
Summary
In this post, I hosted March 2023’s T-SQL Tuesday with an invitation concerning the ongoing Microsoft and OpenAI partnership. I look forwards to reading everyone’s responses!
If this post has been useful, please feel free to follow me on the following platforms for future updates:
Firstly, I’ll talk about my motivation for studying for the Microsoft AI-900. Then I’ll talk about the resources I used and how they fitted into my learning plan.
Motivation
In this section, I’ll talk about my reasons for studying for the Microsoft AI-900.
Increased Effectiveness
A common Data Engineering task is extracting data. This usually involves structured data, which have well-defined data models that help to organise and map the data available.
Sources of structured data include:
CSV data extracts.
Excel spreadsheets.
SQL database tables.
Increasingly, insights are being sought from unstructured data. This is harder to extract, as unstructured data aren’t arranged according to preset data models or schemas.
Examples of unstructured data sources include:
Inbound correspondence.
Recorded calls.
Social media activity.
Historically, extracting unstructured data needed special equipment, complex software and dedicated personnel. In recent years, public cloud providers have produced Artificial Intelligence and Machine Learning services aimed at quickly and easily extracting unstructured data.
Knowing that these tools exist and understanding their use cases will help me create future data pipelines and ETL processes for unstructured data sources. This will add value to the data and will make me a more effective Data Engineer.
Ransom’s article made me realise that I’ve been developing T-shaped skills for a while. I’ve then applied these skills back to my Data Engineering role. For example:
My studying for the AI-900 is a continuation of this. This isn’t me saying “I want to be a Machine Learning Engineer now!” This is me seeing a topic, being interested in it and examining how it could be useful for my personal and professional interests.
With multi-cloud fluency, decisions can be made based on using the best services for the job as opposed to choosing services based on vendor or familiarity alone.
Conversely, data stored in Azure (Azure SQL Database in the video) can be accessed by other Azure services with a single click. As a multi-cloud fluent Data Engineer in this scenario, I now have options where previously there was only one choice.
Improved multi-cloud fluency means I can use AWS for some jobs and Azure for others, in the same way that I use Windows for some jobs and Linux for others. It’s about having the knowledge and skills to choose the best tools for the job.
Resources
In this section, I’ll talk about the resources I used to study for the Microsoft AI-900.
Having watched John’s SC-900 video I knew I was in good hands. John has a talent for simple, straightforward discussions of important topics. His AI-900 video was the first resource I used when starting to study, and the last resource I used before taking the exam.
Exceptional work as usual John!
Microsoft Learn
Microsoft Learn was my main study resource for the AI-900. It has a lot going for it! The content is up to date, the structure makes it easy to dip in and out and the knowledge checks and XP system keep the momentum up.
To start, I attended one of Microsoft’s Virtual Training Days. The courses are free, and their AI Fundaments course currently provides a free certification voucher when finished. Microsoft Product Manager Loraine Lawrence presented the course and it was a great introduction to the various Azure AI services.
Complimenting this, Microsoft Learn has a free learning path with six modules tailed for the AI-900 exam. These modules are well-organised and communicate important knowledge without being too complex.
The modules include supporting labs for learning reinforcement. The labs are well documented and use the Azure Portal, Azure Cloud Shell and Git to build skills and real experience.
I didn’t end up using the labs due to time constraints, but someone else had me covered on that front…
I’ve used some of Andrew’s AWS resources before and found this to be of his usual high standard. The video is four hours long, with dozens of small lectures that are time-stamped in the video description. This made it easy to replay sections during my studies.
Andrew also includes two hours of him using Azure services like Computer Vision, Form Recognizer and QnAMaker. This partnered with the Microsoft Learn material very well and helped me understand and visualise topics I wasn’t 100% on.
Summary
In this post, I talked about my recent experience with the Microsoft AI-900 certification and the resources I used to study for it. I can definitely use the skills I’ve picked up moving forwards, and the certification is some great self-validation!
If this post has been useful, please feel free to follow me on the following platforms for future updates:
