Yay!
Wrote my first blog post.
In my work as a data engineer I sometimes use Azure Data Factory as an ETL tool, but I love playing around with tech to learn how it works. Except Azure will smack you with a huge bill if you're not careful.
Solution: self host the engine! It's called Integration Runtime.
Here is a simple tutorial on:
- Setting up a VM that runs Integration Runtime on an Unraid OS Server
- Run a MS SQL Server docker on the unraid
- Set up the pipelines to run using the self hosted IR and SQL server
In the process of getting this blog post live, I had to play around with setting up a webpage using HTML, CSS and Javascript. It was fun building because I had to use CHATGPT to help me write javascript code to convert markdown to HTML.
Open for any feedback and good tips and tricks to make my blog better. It should work somewhat on both desktop and mobile.
Connect with me on LinkedIn also if you know what ADF is. Link to LI is on the bottom of the webpage.
#azure #data #dataengineer
